How do you approach setting crisp KPIs and targets for Engine features and linking them to your topline metrics? Do you directly try to assess visual quality and performance budget?

To build up the best Objectives and Key Results, I usually ask the Product teams to follow a yearly schedule of strategic planning (aligned on fiscal year, ideally also aligned on performance review cycles). This is aligned on the basic scientific methodology.

Q1 define business hypotheses (observation, question, hypothesis)

This exercise starts with refreshing our predictions about the long term state of the market based on ongoing trends. We then look at how to fund in the medium term our path to successfully get to the long term target. We do this by going over the categories of customer acquisition, retention, share of wallet, market shares, customer costs, customer success, customer satisfaction, brand value, and express targeted business hypotheses. This could be revenue, user behaviours, competition numbers, specific required features for the engine to serve evolving needs…

Q1-Q2 validate/invalidate business hypotheses (experimentation, data collection, analysis)

We spend the next phase of the fiscal year diving deeper in market research and data collection/analysis to try to validate or invalidate our business hypotheses. As we increase our confidence in identifying precise problems worth solving, we put together business cases to pitch strategic investment adjustments to senior leadership.

Q3 define strategic initiatives (conclusions and communications)

Going into this phase we start gathering support across senior leadership for the various business cases we built, find sponsors, and pitch those in the context of defining strategic initiatives for the next execution cycle. Those business case contain a clear set of success criteria in the form of metrics that meaningfully illustrate achieving the business outcomes of the business case. (See literature on outcomes vs outputs).

Q4 adjust organizations for execution

Once strategic planning is complete and senior leadership is aligned on those initiatives and their investment priorities, they usually design changes to the organization to set us up for successful execution. The Product Management team would at that point draft OKRs and start reviewing those with the updated leadership team. Those OKRs are most of the time Objectives representing the desired goal for the business outcomes described in the business case, and the Key Results are chosen for their ability to be realistic to collect and meaningfully represent progress.

Rinse and repeat (iterations)

At this point the organization is setup to execute with clear priorities and goals and Product Management can restart the cycle!

See question: “What are good OKRs for product management?” for guidance on shaping good OKRs.