At what point do you talk about success metrics with your development team?

At the beginning of the project. Before we start implementation, I make sure to define: 

• What problem are we trying to solve and for whom?
• Why is this problem important to solve?
• What is the business impact? 
• How does this project ladder up to the broader product/company strategy?
• How will we measure success? 

Our team believes that success metrics have the highest likelihood of driving focus and execution quality when they are tied to our strategy and something we consistently track and explore. Our teams are well versed in our strategy and the related metrics we are trying to move. We, therefore, set our success metrics early in our opportunity assessment phase. We maintain flexibility and will adjust as necessary if new information surfaces throughout our product development lifecycle that suggests we should select different metrics.

We bring engineering team leads into the process early. As soon as we have the opportunity assessment to 50% we are sharing it with the engineering leads whose teams would be most likely to be involved in the execution phase. The more context engineers have on what problem we are trying to solve for the customer, why they have that problem, how a solution could benefit the financial or usage outcomes and more, the more engineers are positioned to deliver high-quality solutions that exceed the customer’s expectations, and ask critical questions in the process to make sure we are building the right thing and solving the right problem.

The engineering leaders then decide when they want to bring their team in. Sometimes they like to bring them in early so we can do a POC or tests on an experience, while some prefer to wait until we have the work prioritized to avoid distracting their team and reducing velocity.

In either case, being transparent about the impact this work could have, and showing engineering team members videos of customers sharing their unmet needs or product feedback calls can be extremely helpful. The more of this content engineers are exposed to, the more likely you are to have a high-quality outcome.

Metrics work best when they are continuously embedded into the process. We use OKRs, and cascade them down from top level corporate goals into product line metrics, product team metrics and sprint metrics. Use them in a regular cadence to assess progress towards high level strategic objectives. We continually refer to them, in quarterly business reviews, sprint retrospectives, grooming and planning.

As soon as we start talking about the idea. PMs need to have a hypothesis about what a good outcome looks lie in the beginning.

For example we were talking about some changes to our API recently and I asked if there was a way we could measure how many accounts change thier use of our API to include a particualr type of call and it turns out we have a bunsh of monitoring already in place!

Success metrics should be front and center of every conversation from the get go. Success metric is what I would refer to as ‘impact number’ or key results in an OKR. When defining the "why" behind every opportunity, a PM should identify the impact first. This impact helps not only drive prioritization but should also be used by development teams to understand the impact that their work has on the business and users.

So my recommendation is that there is not a good time to talk about these metrics but every conversation should start with these metrics.

Success metrics should be discussed early with engineers to ensure alignment with the product or feature's goals. By setting these metrics early, it helps prioritize what features to build and define how success will be measured. While it's fine for the metrics to change as new learnings come up, it's important to involve engineers early and often. Many times engineers will also help pull the success metrics data so they need to understand why it's important.

At the very beginning! It's so important to align on the why for any initiative. Why now? Why is it important? And a big part of that why is what the world will look like when you're ultimately very successful. To bring that to light, I usually define 1) a success metric -- the one thing we're going to focus on moving and 2) the health metrics -- what we need to watch to make sure that we aren't tanking another key metric with our project. 

The more invested you can get the folks you work with in what success looks like and why you're doing the project, the more buy-in you'll have. Never too early to talk about what the world looks like when you knock it out of the park!