Product Experimentation

A CPO I worked for once told me that qualitative data is still data. (And this person had previously worked for Amazon, a notoriously quantitative data-driven company!)Ri...

I remind my CEO that experiments aren’t about being right, they’re about reducing risk. If a test disproves our idea early, that’s a win — because we didn’t waste six mon...

I focus on building it by making learning safer and more valuable than being “right.” That means people are rewarded for running good bets and sharing honest results, not...

I love this subject! Start from a mindset shift: experiments are not about proving ideas right, they’re about reducing the most critical unknowns, fast and safely. Be hon...

Good question! In the last few years, a big wave of new tools has transformed how we can approach product experimentation. Today, you can find great solutions that cover ...

I believe the era of "growth hacking" (one-off tricks to spike numbers) is dead (or dying). The current trend is Systematization. Teams are moving toward building "perman...

I don't like the idea that an experiment is successful just because it "lifted conversions" by 5% today or during some weeks. Long-term impact should be measured using re...

I’ve seen two models work, and both can be effective depending on company stage and culture.In some orgs, growth and core product are separate teams. Growth focuses on ac...

I am firmly in the camp of shipping features quickly as the highest priority! (Well -- shipping the highest incremental customer value possible as expressed through featu...

I’ve had the privilege of running experiments across very different industries, and each one has taught me something new.One of the most successful was what’s now Endicia...

Less important than the format of communications is the content of communicating results to stakeholders. The following advice applies no matter whether the experiment is...

For me, prioritization is about clarity on business impact, not just curiosity.Strategic Fit – I expect my team to propose experiments that tie back to our biggest produc...

There is no one-size-fits-all process for experimentation; I tend to think of experimentation as a mindset rather than a rigid, dogmatic process that needs to be designed...

Bear in mind that I am answering questions about product experimentation as a PM on B2B SaaS products who is not chiefly concerned with "growth product". In my world, som...