AMA: Unity Vice President Product Marketing, Jon Rooney on Market Research

For Primary Research on a tight budget, surveys via any number of free online tools are super common and can provide at least directional insights, though you may have to clear legal/privacy issues and you'll definitely have to be thoughtful about how you structure your survey (i.e. a small number of the most strategic questions) for maximum impact. Customer and prospect interviews provide higher fidelity insights since your subjects will be more engaged and you can blend exploratory with directed research. Whether it's via Beta programs, Customer Advisory Boards (CABs), Insider or MVP programs, be sure you have some structured mechanism to regularly reach out to customers to conduct interviews so you don't spend a ton of time recruiting subjects. If your company has a Customer Success team, become their best friend. Even better than interviews alone if you're doing any type of use case, Job-to-be-done (JTBD) or product research is observing a customer or prospect practitioner actually doing the things your product presumes to improve, streamline or re-imagine. Try to conduct this type of research regularly not only for primary research purposes, but also to develop empathy and understanding for your customers. If hearing what customers say in an interview is more valuable than what they put down on a survey, than observing what they actually do is that much more valuable than what they may say. For more on this idea, check out the Lead User Research work Eric von Hippel did at MIT. For Secondary Research, if you're looking for user or buyer research (as opposed to 3rd party TAM/SAM/SOM or other market sizing research that can be expensive), great places to glean insights for little or no cost are online user forums (Reddit, Stack Overflow, as well as industry-specific forums and persona specific forums). These forums can skew toward vocal minorities but have plenty of nuggets of wisdom buried in there. Also be sure to frequent 3rd party product review sites (G2, TrustRadius) to see what customers have to say (good and bad) about your products as well as how your offerings are positioned and compared against competitors'. If your company has an enterprise GTM motion that involves responding to formal Requests for Proposals (RFPs), read them methodically - they're gold! RFPs are jam-packed with valuable insights about what problems customers identify as having and what they're looking for in a solution. And as a PMM, RFPs are also a great way to refine messaging because they include the language your customers use to talk about what they're looking to buy. So before you get too clever with branded buzzwords and vague abstractions, pour through a bunch of RFPs.

If you have budget, definitely go with a specialty firm whose expertise is paneling the right mix of enterprise decision makers who may or may not be current customers and offer representation across geographies, firm size and other key designations. In most cases, interviewees associated with these firms are compensated and the specialty firm will manage everything from payments to anonymity (if required) to ensuring employees of public companies don't disclose insider information. If budgets are tight, it's best to work with decision makers at your existing customers through programs like Customer Advisory Boards (CABs) that will manage things like NDAs and operate with oversight so the account owners can rest assured that nothing wacky is happening with key contacts at their customers. If you're at a start-up and you don't have many customers much less a CAB, try to work within the "friends and family" network of executives and decision makers that your investors and board brings to the table. Those networks can be strong and discreet, offering "Friend-DAs" that can be water tight. In either case, compensation is very likely not necessary nor possibly even allowed.

User Experience research is an important double-click into overall market opportunity analysis, going deeper into a detailed examination of how a given persona/role completes certain tasks or solves certain issues around use cases that have been identified as being most important. Basically, once you've identified (or at least hypothesized) "we want to help a certain group of people do certain things faster/easier/cheaper/better," UX research uncovers and lays out exactly how those folks do things in their current workflows, dissecting the pros, cons, bottlenecks and limitations so that both the product teams and product marketing teams have a complete "before" picture of the use case you're going after. As your product or solution is designed and prototype, UX research (often conducted by a dedicated UX or Product Design team that partners with PMM) can work closely with existing or prospective customers in a tight feedback cycle to iterate until the complete product experience is sufficiently better to bring to market broadly. PMMs should be plugged into this tight feedback cycle to make sure messaging is anchored in real customer sentiment.

Having skilled, experience market researchers in-house (whether in PMM or not) certainly makes it easy to take on more projects yourself. No matter how good outside firms are, they'll never be as deep as your team is on company strategy, product portfolio or brand identity. So, all things being equal, it's generally best to bring in a consultant or outside firm for research projects where objectivity and neutrality are particularly helpful. Exploratory brand, messaging, use case and pricing interviews that are blind or semi-blind can be particularly effective when run by an outside firm. An outside firm can also synthesize findings with less bias than internal teams. Another strong reason to go with an outside firm is if your study involves recruiting an interview panel, particularly of senior decision makers, beyond your customer base. Firms and consultants with networks strong enough to fill these types of panels can be expensive but are worth it if your study is structured properly.

If there's only an ad-hoc process in place, put in something programmatic that's simple and straightforward so customers and partner teams can understand and follow. If your company has a Customer Success team responsible for activation, usage, renewals, etc, I'd start there with periodic customer surveys/interviews (quarterly would be great) that gather both quantitative and qualitative feedback consistently so you can track changes over time. Partner with Product Management, Sales Engineers, Customer Success and other teams to design the surveys/interviews. Don't be overly reliant on comments in either Net Promoter Score (NPS) surveys or Customer Satisfaction (CSAT) surveys - you need to go deeper and be more consistent. The context and sentiment alluded to in these surveys might be well understood by the Customer Success team based on regular touch points, formal QBRs, support tickets, etc but if the information is floating all over the place in different formats (including people's heads), it's nearly impossible to manage meaningful change with it. Put it in a consistent format, graph changes over time with key metrics and make sure the customer data includes industry, size, geo - any meaningful demarcation by which you can create cohorts to better understand your customer base. If you don't have a Customer Success team but you have a SaaS offering (which seems unlikely), deliver the surveys 1:many via in-product communications mechanisms like Intercom. If that's not an option, you'll have to go after it the hard way with online surveys and recruiting interview panels which will take longer and may produce lower fidelity signals so it's even more important to be consistent to show directional changes over time.

The rise of UX research as a practice over the past 10+ years has unfortunately created it's share of organizational overlap, friction and feudal rivalries (leading to wasteful, demoralizing outcomes like different teams at the same company having completely different persona profiles that don't cohere at all) but it doesn't have to be that way. Just as, if defined poorly, PMM can seem to blur into roles like PM or Growth/Demand Gen, the same happens all the time with UX teams. The most natural role delineation is the core positioning statement and value prop ("we solve x problems for y people in z awesome ways") live with PMM while the detailed analysis and classifications of said x, y and z live with the UX team and drive their research. For example, the Y people for your business could be DevOps engineers or Insurance Claims Adjusters that the UX research further flushes out, defining their how they work and what tools they use while working. The output of that research is super useful for PM prioritizing backlog as well as PMM refining messaging but it doesn't supersede either function. So messaging research (validation, refinement) should be a clear PMM responsibility that complements, rather than overlaps with what the UX team is driving. So mapping out who does what and where one team begins and the other ends up front will save you a ton of time and heartache once work gets moving. If the UX team starts staffing up with copywriters or messaging folks, however, you've really got a problem. Stamp out fuzziness with clarity, clarity is a PMM's best friend.