Sign Up / Sign In

User Insights · Product Experimentation

How do you measure the success of your experiments when you're at an early stage company and there may not be enough quantitative data?

Julian Dunn

Chainguard Senior Director of Product Management • 10mo

A CPO I worked for once told me that qualitative data is still data. (And this person had previously worked for Amazon, a notoriously quantitative data-driven company!)Ri...

383 Views

How do you help your CEO view experiment failures as progress?

Sheila Hara

Barracuda Sr. Director, Product Management • 10mo

I remind my CEO that experiments aren’t about being right, they’re about reducing risk. If a test disproves our idea early, that’s a win — because we didn’t waste six mon...

388 Views

How do you build a culture of experimentation and failure within your team?

Bruno Gobbis

Nuvemshop Director, Product Growth | Formerly Superhuman, RD Station, IBM, Bosch • 7mo

I focus on building it by making learning safer and more valuable than being “right.” That means people are rewarded for running good bets and sharing honest results, not...

411 Views

How do you deal with uncertainty and ambiguity when running experiments?

Bruno Gobbis

Nuvemshop Director, Product Growth | Formerly Superhuman, RD Station, IBM, Bosch • 7mo

I love this subject! Start from a mindset shift: experiments are not about proving ideas right, they’re about reducing the most critical unknowns, fast and safely. Be hon...

404 Views

What are some of the new tools and technologies that you're using for product experimentation?

Bruno Gobbis

Nuvemshop Director, Product Growth | Formerly Superhuman, RD Station, IBM, Bosch • 7mo

Good question! In the last few years, a big wave of new tools has transformed how we can approach product experimentation. Today, you can find great solutions that cover ...

391 Views

What are some of the trends you're seeing in product experimentation?

Bruno Gobbis

Nuvemshop Director, Product Growth | Formerly Superhuman, RD Station, IBM, Bosch • 7mo

I believe the era of "growth hacking" (one-off tricks to spike numbers) is dead (or dying). The current trend is Systematization. Teams are moving toward building "perman...

426 Views

How do you measure the long-term impact of your experiments?

Bruno Gobbis

Nuvemshop Director, Product Growth | Formerly Superhuman, RD Station, IBM, Bosch • 7mo

I don't like the idea that an experiment is successful just because it "lifted conversions" by 5% today or during some weeks. Long-term impact should be measured using re...

1,045 Views

How does your product growth team and core product team collaborate on product roadmap?

Sheila Hara

Barracuda Sr. Director, Product Management • 10mo

I’ve seen two models work, and both can be effective depending on company stage and culture.In some orgs, growth and core product are separate teams. Growth focuses on ac...

409 Views

How do you balance the need to experiment with the need to ship features quickly?

Julian Dunn

Chainguard Senior Director of Product Management • 10mo

I am firmly in the camp of shipping features quickly as the highest priority! (Well -- shipping the highest incremental customer value possible as expressed through featu...

420 Views

What are some of the most successful product experiments you've run, and how did you decide to experiment on that?

Sheila Hara

Barracuda Sr. Director, Product Management • 10mo

I’ve had the privilege of running experiments across very different industries, and each one has taught me something new.One of the most successful was what’s now Endicia...

429 Views

How do you communicate the results of your experiments to stakeholders?

Julian Dunn

Chainguard Senior Director of Product Management • 10mo

Less important than the format of communications is the content of communicating results to stakeholders. The following advice applies no matter whether the experiment is...

381 Views

How do you prioritize which experiments to run?

Sheila Hara

Barracuda Sr. Director, Product Management • 10mo

For me, prioritization is about clarity on business impact, not just curiosity.Strategic Fit – I expect my team to propose experiments that tie back to our biggest produc...

404 Views

What is your process for designing and running experiments?

Julian Dunn

Chainguard Senior Director of Product Management • 10mo

There is no one-size-fits-all process for experimentation; I tend to think of experimentation as a mindset rather than a rigid, dogmatic process that needs to be designed...

433 Views

What are some of the biggest lessons you learned on running product experiments that are not intuitive?

Julian Dunn

Chainguard Senior Director of Product Management • 10mo

Bear in mind that I am answering questions about product experimentation as a PM on B2B SaaS products who is not chiefly concerned with "growth product". In my world, som...

486 Views