Question Page

How do you identify which experiments to prioritize first?

Willie Tran
Willie Tran
Dropbox Group Product Manager, DocSend Growth | Formerly Mailchimp, CalendlySeptember 20

I mentally use the ICE (Impact, Confidence, Ease) framework when prioritizing experiments. Impact = How many people does this affect? Confidence = How sure are you that this will work? Ease = How easy is it to implement?

Impact and Ease are pretty simple to calculate. However, I find that most people don't know how to accurately assess "Confidence." This is likely because your ideation strategy consists mainly of throwing things at the wall and seeing what sticks. Or just working off of best practices. This strategy works well for the first six months to a year of an experimentation strategy, but falls short after you've picked up all the low hanging fruit. 

Instead, you want to create a higher level process that's focused on generating high level user problem statements. You create these problem statements by doing a lot of user research around why users are not doing the thing you want them to do (e.g. Activate). Once you do your set of interviews seeking to answer a few questions, you should have some good idea as to why people are Activating (in this case). Turn those into well written problem statements (As a [user type], I want to [do a thing], but [reason why I can't]). 

Then for each problem statement, do an ideation session with your team where you and your xfn partners come together and come up with ideas on how to solve this problem. Then vote on it. You'll see a lot of ideas have common themes which will give your designer a lot of material to come up with the best way to address said problem. 

TL;DR - Do a lot of user research to understand the problem. If you understand the problem really well, then it's very easy to come up with solutions you have a lot of confidence in.

2218 Views
Nicolas Liatti
Nicolas Liatti
Adobe Senior Director of Product Management, 3D CategoryDecember 6

Product management is all about reducing risks and blurriness on the value and viability of the product.

So I would say start from the user, move backwards and chose the area where you have the biggest unknown. Make an experiment there so it helps reduce the risks.

426 Views
Top Product Management Mentors
Paresh Vakhariya
Paresh Vakhariya
Atlassian Director of Product Management (Confluence)
Sheila Hara
Sheila Hara
Barracuda Sr. Director, Product Management
Farheen Noorie
Farheen Noorie
Grammarly Monetization Lead, Product
Anton Kravchenko
Anton Kravchenko
Carta Sr. Director of Product Management
Natalia Baryshnikova
Natalia Baryshnikova
Atlassian Head of Product, Enterprise Strategy and Planning
Nicolas Liatti
Nicolas Liatti
Adobe Senior Director of Product Management, 3D Category
Julian Dunn
Julian Dunn
Chainguard Senior Director of Product Management
Rupali Jain
Rupali Jain
Optimizely Chief Product Officer
Trisha Price
Trisha Price
Pendo Chief Product Officer
Kara Gillis
Kara Gillis
Splunk Sr. Director of Product Management, Observability