Sharebird
Sign Up / Sign In

How do you ensure your experiments are not impacted by other improvements done by other teams?

Answer
2 Answers
Growth Product Management
  1. Deepti Pradeep
    Deepti Pradeep

    Adobe Senior Director of Product Management & Growth (Creative Cloud) • 1y

    Governance / air-traffic control of AB tests is essential. Running parallel AB tests on the same KPI and same audience will not guarantee clean results. If improvements done by teams are direct to production, timed before an AB test, it may not be an issue as both control and variant receive the changes thereby retaining the validity of the results. Most testing platforms come with some inbuilt conflict resolution, but I highly recommend bringing teams together in targeted forums to not only ens ...Read More

    896 Views
  2. Bruno Gobbis
    Bruno Gobbis

    Nuvemshop Director, Product Growth | Formerly Superhuman, RD Station, IBM, Bosch • Jun 4

    This is one of the trickiest parts of running a growth program inside a company — and I'll be honest, you can never fully isolate experiments. But you can get close. Centralized experiment registry: Every experiment going live anywhere in the product gets logged in one place — owner, hypothesis, audience, dates, expected metric impact. At Superhuman we used a shared review process (slower, high quality/control); at Nuvemshop we use a more structured tool and a centralized place to updates (faste ...Read More

    382 Views

Related Ask Me Anything Sessions

Bruno Gobbis

Nuvemshop Director, Product Growth, Bruno Gobbis on Growth Product Management

June 4 @ 10:00AM PT

View Answers
Ruchi Aggarwal

Former BILL Director, Product Management - Payments, Ruchi Aggarwal on Growth Product Management

March 17 @ 10:00AM PT

View Answers
Deepti Pradeep

Adobe Director of Product Management, Growth, Deepti Pradeep on Growth Product Management

February 19, 2025 @ 10:00AM PT

View Answers
See All Upcoming AMA's

Top Product Management Mentors