How to make sure KPIs are not 'proxies' to what we want to achieve but directly measure it?

KPI’s are measured and recorded to track performance and business decisions are rendered with the use of this supporting data. The intent behind this movement is to provide managers and leaders with more information and drive better decision making.  While there is a need for Proxies in some cases, it is very important to choose these with utmost care and deliberation.

1. Having clear goals to achieve will provide a great way to avoid choosing the false "proxies".

2. Choose KPIs that should be measuring the actual objective itself and any other key actions or outcomes that will lead to that objective.

3. Ensure the sanity of Data used so that when relying on data to make decisions, then what is being utilized needs to be accurate.

Every KPI exercise should start by asking, "What are the direct business outcomes we hope to move the needle on with this intiative?"

So often I see KPIs focused on something like pure adoption metrics. Ok great, we want people to adopt this feature but WHY do we want them to adopt it?

• Do we think it will help reduce churn?

• Do we think it will drive conversion?

• Do we see a correlation between customers that adopt this feature and stay past the 30 day mark?

• Is this feature a key part of our Aha moment?

I find that a good measure of if a KPI is more than just a proxy is if it cuts across at least two areas of measurement. For example, looking not only at the customers that adopted that feature, but also their overall net retention. The more data you can can layer on to a KPI to help understand the business outcome it's driving, the stronger measurement it will be.

When thinking about KPIs, it starts with understanding your customers and the value you expect to create with your product:

1. Think, first, in terms of outcomes

   • Outcomes = What does success look like for your users and your business?

   • A good way to imagine user outcomes is to ask "What would change in our users' lives if this is successful?"

   • This lays the foundation for working backwards towards useful metrics/KPIs

2. When you think you've found a good metric, ask yourself "So what?"

   • Our daily active users increased -> "So What?"

   • More people are logging in -> "So what?"

   • If you can't find a good answer to "So What" that connects to the actual value for your users or business, then your KPI is a proxy for something else, or its just a bad KPI

3. Pair metrics

   • KPIs don't exist in a vacuum. Sometimes it takes a combination of metrics / KPIs to get the full picture

   • For instance "Time in app" alone could be a misleading metric. Someone could be spending a lot more time if they are confused or stuck. But if you pair it with another metric, you have a better triangulation on the real outcome.

     • For instance, for a task management app, you could combine "Time in app" + "Tasks completed on time" to understand if the time being spent in the app is helping improve the desired customer outcome.

4. When possible, take the time to validate your KPIs with qualitative feedback

   • A good PM talks to customers: Validate your KPIs with user interviews/real user feedback

   • If your metrics show success but your qualitative feedback suggests a different story, then it might be time to revisit your KPIs

For sure, whenever possible you should be tracking KPIs that indicate user / business value. That said, it is not always so straightforward to track a direct outcome, so if you do find yourself needing a "proxy" metric, follow some of the steps above to at least confirm that it is a good proxy.