How do you ensure your experiments are not impacted by other improvements done by other teams?
1 Answer
Adobe Director of Product Management, Growth • February 19
Governance / air-traffic control of AB tests is essential. Running parallel AB tests on the same KPI and same audience will not guarantee clean results. If improvements done by teams are direct to production, timed before an AB test, it may not be an issue as both control and variant receive the changes thereby retaining the validity of the results. Most testing platforms come with some inbuilt conflict resolution, but I highly recommend bringing teams together in targeted forums to not only ensure there is no test pollution but also, and most importantly, align learnings.
260 Views
Related Ask Me Anything Sessions
Adobe Director of Product Management, Growth, Deepti Pradeep on Growth Product Management
Adobe Senior Director of Product Management, 3D Category, Nicolas Liatti on Growth Product Management
ezCater Director of Product Management, Ojus Padstonon Growth Product Management
Top Product Management Mentors
Ashka Vakil
strongDM Sr. Director, Product Management
Deepak Mukunthu
Salesforce Senior Director of Product, Agentforce AI Platform
Clara Lee
PayPal VP, Product
Deepti Pradeep
Adobe Director of Product Management, Growth
Abhijit Limaye
Matterport Sr. Director of Product Management
Rodrigo Davies
Asana Director of Product Management, AI
Tom Alterman
Notable Head of Product
Preethy Vaidyanathan
Matterport VP of Product
Farheen Noorie
Grammarly Monetization Lead, Product
Jacqueline Porter
GitLab Director of Product Management
Related Questions
When transitioning your growth team into an integrated product organization, how did you determine which priorities would drive the most impact early on? What were the biggest challenges you faced in navigating that shift?What roles do you hire for when building a growth team?What KPI's do you share with core product managers? How do you divide responsibilities for these KPI's?How do you approach working with cross-functional teams to ensure work is done on the highest impact initiatives?How is the role of product growth manager different from other product managers?How do you interact with marketing as a growth product manager? How do you split/share KPI’s?