How do you measure the effectiveness of the messaging behind a product? What would you do if you weren't able to A/B (example - if the product's audience were a handful of large Enterprise customers)?

When looking at the effectiveness of the messaging behind a product, I have always tried to focus on what are we trying to achieve with the messaging? How does the messaging tie back to overall business objectives? 

Answering that question can then help guide you to how to measure the effectiveness of the messaging. 

If the product messaging is focused on expanding the scope of the product or the market you're targeting, then you'll want to look at things like ASP and if there is an expansion of titles/personas involved in sales opportunities. 

If the product messaging is centered around creating a new category of products, then you'll want to track higher level awareness metrics such as message resonance (how many times the message is repeated in media/the market), share of voice, analyst/media mentions of the key message, competitor pick up of the message you're trying advocate. 

If you're trying to streamline the product positioning to increase adoption by the sales team and the company overall then you'll want to track the adoption of new sales decks, collateral and other assets. We've started to use Highspot as a sales enablement platform and will be closely tracking the usage of content we push out through the platform by reps and how the content influences deals. I would also track the usage of content and messaging in Gong calls to see how the message is being utilized in live customer interactions. 

If you're looking for product messaging to improve conversion then you can track that through the performance of landing pages, ads, SDR outbound cadences, etc. 

It's much easier to think about the purpose of what you're trying to accomplish with your messaging and then orient your measurement around that vs. trying to measure the effectiveness overall. 

A few other things to track in terms of messaging performance (or overall impact of pmm activities):

• Deal cycle times
• Average selling price (mentioned above) 
• Competitive win rates
• Close rates (funnel performance overall)
• Sales rep performance

I would also always be speaking to reps and other stakeholders across the company to get feedback as well as industry analysts, partners, etc.. to get qualitative feedback on your positioning. These conversations always bring up something you've never thought of that can improve your positioning. 

Great question and not sure I have a very scientific answer here! I'd say we use a combination of things to validate effectiveness of messaging: feedback from customers (gathered via surveys, live conversations etc.), sales team (particularly on what is resonating/not resonating with customers), analysts of third-party firms (E.g. Gartner/IDC etc.) and independent research (either with our own market research team or a vendor). If the product's audience were a handful of large Enterprises, this process might be a easier in the sense that you can get direct feedback from the customers/account team on messaging effectiveness. Very often tho - customers likely won't focus as much on the messaging as they will on the product/solution itself and what it can/cannot do. One additional idea: consider a customer advisory board with those customers and involve them more in developing the product roadmap/design etc.

If the product's audience includes a handful of large Enterprise customers, I'd ask for approval from Sales and Customer Success to reach out to the customers and run the messaging by them on a video or phone call.

"Hey, Big Customer. We value your opinion, and would love just 15 minutes to run by you some words we're using to explain this new product. We thought you could give us your honest opinion — we won't be offended! — on what's working and what's not working with our messaging."

One of our solutions marketers — with the help of our customer marketing manager — just did this at Calendly, and the results were invaluable.

Who doesn't love giving their honest opinions to people who say they won't be offended? If they don't love it, try offering an incentive (gift card, discount, free trial, etc.).

When I'm working in other environments where Sales and CS aren't as comfortable with employees talking to customers about non-Sales/CS motion stuff, we in Marketing volunteer to help the Sales and CS team close deals by playing Sales Engineer. We did this at Clearbit for a new product line that we understood very well in Marketing, and not only did Sales and CS get help with closing deals, but we were able to test our messaging via a slide deck used in the beginning of the calls. After every call, I tweaked something in the deck — or I realized I needed more than one deck depending on who I was talking with.

You measure it through message testing. An effective message resonates with the target buyer, and that's what you want to check: how it lands on the people you're trying to influence.

Yes, you can wait for the full go-to-market cycle and eventually get some feedback from the market, or measure conversion rates - but only relying on this makes your feedback loops extremely slow and inefficient.

Message testing is a form of qualitative research, hence you don't need large sample sizes. You need to get your page/pitch (whatever the medium) in front of the target buyers and ask them research questions. Doing it on a 1:1 basis is very high quality, but slow and expensive. Doing it via surveys is cheap, fast, and can be high quality if the selected panel is excellent. 

There are 4 heuristics you need to measure:

1) Clarity: do they get it? After reading everything, what's still unclear? What questions do they have? A confused mind doesn't buy.
2) Relevance: does it align with their priorities and challenges at hand? If the message is not about what's important to them now, it won't work.
3) Value: do they want it? Effective message increases user motivation to take action. Don't just describe things and hope for intrinsic motivation. Make a case of how your thing adds the value they seek.
3) Differentiation: why choose you? Is it clear why go with your offer instead of competing solutions? Don't talk about yourself as if you're the only one doing what you're doing.

For each heuristic, do a likert-scale (1...5) as well as qualitative, open-ended questions. Ideally, you do this section by section, so you know exactly where the problems are, and you can fix them.

What specifically is unclear?
How exactly it does or does not align with your priorities?
What makes you want a demo with them (or not)?
How does this seem different or better than other options out there?