AMA: Dropbox Group Product Manager, DocSend Growth, Willie Tran on Growth Product Management and Experimentation

The number one thing first time growth PMs get wrong is only focusing on wins and not involving their team. When creating a Growth squad, your number one enemy isn't not making the number bigger, it's xfn attrition. Designers and engineers who are new to Growth have historically been rewarded for releasing new features. When joining a Growth team, you have to be able to prepare them to launch something and then delete it. This is very new behavior and goes against what they are used to doing.  S ...Read More

By far the best material you could read to learn about PLG has nothing to do with PLG. PLG is based off of the Scientific Method you learned in grade school. Go back and reread the fundamentals of how to go through the Scientific Method. Lessons around experiment design/methodology are highly valuable and should not be glossed over. There are other parts though that are more focused around experiment logistics and team dynamics that the Scientific Method won't cover. But in my opinion, a good bo ...Read More

Ideally, you design experiments in a way that seek to answer a question regardless of the result. For example, "do users not adopt this feature because of discoverability or usability?" is a great question to be answered with an experiment. If the exeriment execution puts that feature front and center of the main page when a user logs in, and the experiment is flat, then you could be pretty sure it's a usability issue. If it's stat sig positive, then you can be sure it's a discoverability issue. ...Read More

I mentally use the ICE (Impact, Confidence, Ease) framework when prioritizing experiments. Impact = How many people does this affect? Confidence = How sure are you that this will work? Ease = How easy is it to implement? Impact and Ease are pretty simple to calculate. However, I find that most people don't know how to accurately assess "Confidence." This is likely because your ideation strategy consists mainly of throwing things at the wall and seeing what sticks. Or just working off of best pra ...Read More

If you're trying to measure the results of an experiment, assuming they're under a binomial distribution, you just need the numerator and denominator (people exposed to the experiment) for the control and treatment. Then just throw it into your favorite chi square significance calculator (I'm a big fan of the Evan Miller calculator). In regards to other product tools, it's the same as usual. You can use Amplitude to look into user behavior. StatSig or Optimizely or all the other ones for running ...Read More

I've wrestled with this over the course of ten years now. I used to think Growth PMs and Core PMs should just be one in the same and every Core PM should be able to do Growth work. However, I don't really believe that anymore. From my experience, there's a place in the business to do optimizations (Growth) and a place to do exploration (Core) work.  Core PMs generally have a stronger skillset around taking big bets which require a lot of coordination and patience. Growth PMs require high analyti ...Read More

The areas I index the most when hiring someone: Partial - Do they fall in love with their own ideas? If so, then I don't believe they will be a strong Growth PM. Given so many experiments fail, you want someone who is focused on answering questions about the user and the science of it all. Analytical - They don't have to be a statistician, but they need to understand how to read the data and extrapolate user behavior from it Collaborative - This is probably the most important part. A person shou ...Read More

Personally, I've always used pirate metrics. Acquisition - Getting the user to sign up Activation - Getting the signed up user to realize the value of the product Revenue - Getting the user to pay for the product Retention - Getting the user to stick around Referral - Getting the user to refer someone else to sign up These are solid high level metrics to measure the overall effect of a team. However, when running a specific experiment, these metrics aren't always the best to choose when measurin ...Read More