AMA: Google Product Marketing Lead, Vishal Naik on Market Research

Whether to do user research in-house or hire a vendor is a decision that depends on three key factors: budget, time, and quality. Engaging a research vendor is the more expensive option. It's kind of a you get what you pay for scenario. If you engage a vendor and brief them well, you'll probably get better results simply because its the research vendors field of expertise. If you have a limited budget or need results immediately, then doing the research yourself may be the best option. However, it is important to keep in mind that your research may be limited to desk research or scrappier qual or quant research. A vendor might have a larger playbook of studies. As a marketer, I can conduct research, but some questions are beyond the scope of a PMM. Some complex methodologies such as MaxDiff and Conjoint Analysis have come up as means to uncover the insights my team has needed, and those are best left to vendors. It's important to be aware that research can be biased if not done correctly. The order of questions or how they are phrased may lead to confirmation bias. In my experience, a research vendor or dedicated UXR resource tends to think about this inherently vs as a PMM I need to put in extra effort to do so. For me, it comes down to what problem I'm trying to get an answer to and whats the overall best solution for the business based on the time and budget we have, and then based on type of research needed.

A product source or truth deck that points to your personas, messaging and positioning is a sound format for this. We use this as a model for hero products. Another approach that we're using for a new product that hasn't yet launched but we have user research, personas, and positioning around is a product onboarding deck. Same concept as the source of truth or "bible", but being used as the intro deliverable for all of those joining the project as the first touchpoint they should read. An outline to consider for the source of truth deck could be: * Marketing goals * Key marketing moments to plan around * Market context * User personas * Messaging and positioning * message framework/narrative * style/tone guide * Product overview * key features * mocks/screens * sales strategy (if b2b) * call scripts * key questions to ask * playbook to navigate from stakeholder to decision maker * branding * use cases/case studies

My personal view on this is that your personas are effective if youre influencing the conversation around who should be using/buying your product. Some questions you could ask yourself about your personas, based on the teams you may work with: * Marketing: is the way the product is highlighted/showcased in various marketing channels geared at the right audience? * Product: are the features being created/put into roadmap enabling the right type of usage? * Sales: is your win rate improving because you're asking the right questions to the right people? * Data/Business Intelligence: does usage data validate your persona hypotheses? is your team sharing out KPIs that map to your highest value user profiles? While a persona doc or deck might not directly answer all of those questions, if indirectly you're seeing influence across how the entire org things about the specific users of your product, then I'd say your personas are effective and to keep doing what you're doing.

I see generative AI helping marketers understand what options they have for research so that they can ask agencies better questions around what methodologies to use. Generative AI can also help you with desk research as it can point you in the direction of the studies and analyst reports that cater to your nuanced needs. I personally wouldn't use generative AI to replace research agencies because the chance of a hallucination isn't worth it to me when my goal out of research is to remove potential doubt in my strategy and give clear direction.

Honestly I don't. I tend to view frameworks as templates to help guide a business plan. In my experience, no two business questions have been met by the same research plan. If you had a recurring business problem around each product launch, then you might want to seek specific frameworks for the nuances you're looking for--such as message testing vs feature prioritization vs pricing strategy, etc.

There are specific roles that are research based, but usually tangential to PMM (aka might sit in the PMM team, might sit elsewhere)--such as competitive intel or research and insights. But speaking frankly, the volume of PMM roles that focus on GTM likely outweigh--by some order of magnitude--the volume of PMM roles that focus on research.

In my opinion, its a valuable proxy for interest and demand. I've seen, and used, Google Trends data to measure interest. But would note that its probably best to use search volume to compare interest and demand. For example compare vs another point in time or compare vs competition.

The challenge if your company doesnt always do market research is that you have to prove that there is a gap in your foundational knowledge. Which is always tricky because you're effectively saying that the company has been doing something inefficiently (and potentially "wrong") in the past. A way to get around this (which admittedly has its own hurdles) is to pilot a research program from your own cost center and then prove the value of it to XFN stakeholders (and once youve got it budgeted, get their support and input into the study so they are more excited about the output) and then in the future use that pilot study as a reference point to showcase what impact market research can have.

Based on your budget, ranging from zero to small, I'd consider the following options: Zero Budget: I'd do a combination of desk research on the industry and listening in on prospecting calls from SDRs and seeing how those prospects in different industries resonate with your messaging/describe their problems. Limited Budget: I'd try something scrappy like building landing pages on my site and then creating highly targeted SEM campaigns with a fixed budget so that I could see what segments converted and showed a signal that can be used as a proxy for interest. Small Budget: I'd try something like the targeting features within a platform like Survey Monkey or try to self recruit for a small focus group.

The right agency. Yes it costs money so that does limit whether or not you can leverage an agency, but an agency can be leveraged like an outside consultant giving you the insights you need based on your business strategy. At my last role, regardless of the business challenge that faced us, the first call I made was to my preferred research agency. In the companies where I've worked where that budget ask would have not been approved, I've had some good experience with Survey Monkey. While it still does cost money to use, its less than an outsourced agency. I tend to be pro-Analyst as well, so I've liked Gartner insights. In terms of free desk research, I like to read what consultants like McKinsey publish on a topic.

I'd start with my preferred search engine and read up on anything i can find, and look for industry reports or stats that paint my product in the right light. As for competitive intelligence, I'd also start with a Google search, but also look for review forums (think like a G2 type site) and look for how customers are talking about a product. If my company has any clients who have switched from a competitor, then those are great customers to target to learn from. It's probably also beneficial to do win/loss analysis and deal post mortems to learn more about the competition you're running into on a daily basis.

I'll break this up into two parts: Industry trends: I tend to look to public information here. Press pickup, analyst reports, what's going viral online (for a consumer product). If I were to turn to research for this, I'd suggest a qualitative study where I wrap this into a line of questions around another topic. As in, it's probably not meaty enough to do a study just to understand industry trends, but if you're covering another topic you could wrap this in. Customer Behavior: I worked with a research vendor on a customer behavior study last year and the process we followed was a qual, quant, qual. * For the initial qual, we did a series of in-depth interviews (you could also do a diary study) to set a baseline for how users were engaging with the space we were trying to explore. So specifically not our product, but the ecosystem in which our product sits in. * Then we used that to shape a quant where we could fine tune use cases to test and broader business impact around the behaviors we were studying. * Then we used the quant to do another qual that was more specific to our product, but anchoring on that original qual's baseline and the learnings from the quant to validate some of our hypotheses. You dont necessarily need to be that broad, and MVP on this would just be the front-end qual (IDI or Diary Study being good places to start) .

I think the minimum viable approach to MVP feedback would Beta programs or limited release priority user programs where the user is under NDA and can provide product feedback that you can iterate off of. I tend to like to do some research in advance to find target use cases and areas that users might want out of the product so that you can build a better MVP, and then study dogfood/beta/trusted tester program usage, but that's budget and time dependent.

At my current company, we have three main ways to conduct product research: in-house with our UXR team, with an agency synthesized by our Insights team, or with an agency fully managed by PMM. We choose the right path based on resources, budget, and research goals. Here are a few tips for conducting product research: * Start by defining your research goals. What question are you trying to answer? What will you do with the information? * Sync with your team to see who has the resources and bandwidth to help. This may include your Insights team, UXR team, and Marketing team. * If you're working with an agency, be clear and concise in your briefing. Share context, your ideal state, and the nuances you're looking to learn. * Carefully craft your research stimuli. Make sure the items that respondents see are diverse and clear enough to guide them down the right path, while avoiding unintended bias. * Review and interpret the results yourself. You know your business better than any agency does. * Be involved in the analysis of the information, even if it's in-house. Ask market questions and let your brainstorm shape the readout. Keep in mind research is like any other data point, people will take away different interpretations of what the information is saying, so be sure to spend your own time understanding the study so that you can steer conversations towards the ideal outcome for your business.