You already have metrics, they’re buried in your BI tool

Up until recently, the goal of most people's initial modern data stack was to get data ingested with ETL then centralized and models in a data warehouse and finally to attach a BI tool where it would be transformed into charts. The visualization was usually a huge unlock for people because for the first time they could see their data in an understandable way. As someone who has made a career out of business intelligence for over 10 years, I'm a huge advocate for the importance of data visualization and recognize that it will always be a valuable piece of the data ecosystem. However, the data landscape has evolved and implementing BI as the final step is not enough. Teams need to go further and define the metrics explicitly in a semantic layer using the new wave of metric stores.

For helpful context, there has only been one main player for data warehouse modeling: Looker. The company's main innovation is its data modeling language LookML. This language provides a semantic layer for the data warehouse and allows users to create dynamic queries with strong data governance. However, data warehouse semantic modeling has been proprietary and devoted only to Looker … until recently.

The outstanding success of Looker has been largely built on the back of its realization that a huge amount of value comes from developing a semantic layer that models the raw data into meaningful business concepts. The main reason to adopt Looker as the BI tool has been to get access to LookML which has helped thousands of teams provide a better self-serve experience. However, Looker's has kept the power of LookML closed off to other tools, making it only available for developing charts and dashboards in Looker.

In recent years the semantic layer became available in open ecosystems like dbt by dbt Labs and Metrics Flow by Transform. With semantic layers now being openly available, advanced data teams need to adopt a better approach of supporting their company’s data consumers. From our discussions with 50+ innovative data professionals across many industries, we are seeing teams take the time to figure out what metrics they need to put into their semantic modeling using the metric layer so that their company can rely on centralized metric definitions.

To help you start the process of defining your metrics, we've compiled a list of some of the best practices we've seen so far. One of the first steps is to identify all the places that your teams are implicitly defining metrics and to bring the metrics into the source of truth through documented definitions that everyone aligns on. As you're getting started, one important note to keep in mind is that although they are probably the biggest offender, BI tools are not the only culprit that have caused metric definition siloing. Any tool or team outside of the centralized dated team can be a source of metric definition silos. Business teams could be creating metrics in spreadsheets, data science could have metric calculations in Python notebooks, and pretty much any SaaS tool that gets to deploy could have a transformation layer that has data calculations that people rely on. The due diligence process may take as little as an hour or sometimes weeks but the payoff will be significantly increased velocity for your company and dramatically more value out of your data.

What does centralizing definitions into a metric store do for your company?

There are both data and business improvements that come from explicitly defining and aligning on metrics as a company.

From the data side, your definition can be maintained in one centralized place and propagated to all of the downstream tools to use. As the world moves on to data consumption in more than just BI, maintaining definitions in each tool will become exponentially harder. Centralized, open metric definitions allow them to maintain that deliver more ROI out of data without much incremental cost. Look at emergence of Reverse ETL, ML, data apps and business observability tools as ways to get more from metrics. Data teams can provide more value from the existing infrastructure and headcount which is a huge win for them and for their organizations.

On the business side, by explicitly defining and agreeing on metrics, teams are committing to the specific numbers that they are going to own and drive. These same teams can also show how their work contributes to the overall company objectives if the metrics are created in a metrics framework. This framework establishes relationships between metrics to identify the inputs they control to produce the short-term outputs to achieve the long-term business outcomes. Defining metrics creates clear accountability at all levels of an organization and helps companies achieve business results reliably.

🔎 Metrics frameworks can be extremely helpful to make sure all the important requirements are gathered during a metrics implementation. Find more here!

Finally, not only is it easier to maintain, the time and cost of implementing new solutions goes down significantly. Any team will be able to easily connect data tools without having to worry about inputting metric definitions and maintaining them in a silo. At Push.AI, we believe that teams should be able to get our product started in minutes, therefore, we prefer to onboard teams by importing from an existing metrics store. However we know that the metrics layer is new for many companies and, if a team doesn’t have a metrics store yet, we will help them identify and build their metric store through advice and the Metrics Builder in our product. They can then export the metric definitions created in the dbt yaml format so that they can reuse those metrics elsewhere.

Defining metrics drives clear accountability at all levels of an organization and helps companies achieve business results reliably.

So, what’s the best way to find existing definitions and implement metrics?

Conclusion: how to get started with metrics!

To get started with your metrics journey, look to the existing places that data is being consumed and identify the definitions that are already built into those tools. Is the number of tickets completed being counted to calculate Support Tickets Completed? The amount of revenue being summed to calculate Annual Recurring Revenue? Or an expression of the sum of customers divided by this sum of leads to calculate the Sales Funnel Conversion Rate? Doing an audit of existing metric definitions built within tools will allow your company to document the metrics that are relevant to each team. Then, adding those metric definitions to the metric layer will dramatically increase their impact.

You’ll probably find quite a few surprises as you go through the process and you can document the ways teams might be using data incorrectly. Many teams will need help managing the transition from the wild west of “everyone build your own metrics” to a centralized definition layer. From personal experience, the biggest source of headaches can be spreadsheets where teams have manipulated data and hidden calculation logic within many layers of cell calculations. Documenting this logic and showing that it is translated accurately to the metric definition in the semantic layer will be critical to getting buy-in from those teams to use the source of truth.

If you’ve been at your company for a few years, you probably can do a huge amount of the initial due diligence by taking a few hours and writing down every metric you can think of onto a diagram. It could be one of the most clarifying experiences you’ve had working in data and the resulting diagram can be an invaluable asset on your company’s journey to being data driven. To make the process easier, we recommend adding a few  pieces of information to each metric as you do your definition due diligence:

• Document common dimensions that are used to analyze a given metric. This will streamline future investigations by giving people recommendations on the most important associated data that people should be slicing over. As investigations occur and new dimensions are found to be critical to analyze a given metric, they should be added back to the metric definition. As a reference, this is the basis for tools that have created a semantic layer dynamically based on user interactions like ThoughtSpot.
• Add metadata so that you can organize metrics into actionable parts for your business. The right metadata will allow teams to better navigate your metric framework and will drive significantly more value from data for an organization. We recommend at least having a department and team so that it's clear who is accountable for the metric. There is a full list of metadata that we recommend and use for our Metrics Builder in our documentation.

Wrapping all this up into a clear metric framework is critical and its an easy task to put them together in a diagram. You can see recommended ways of organizing and visualizing metrics and the semantic layer in our Metrics Frameworks post.

‍