What has been the most useful mechanism you have used to test messaging and determine the best message?

Great question. There are a few primary mechanisms we've used to test messaging:

• Research Reports / Messaging Surveys - These work pretty well, especially when you're entering a brand-new market and need to test messaging for and audience that isn't in your database already. 
• In-App - I mentioned this in another answer, but if you can directly control, or work with your product team to edit in-app messaging easily then it can be a great way to test various messaging. Within HubSpot, depending on the product and tier you are on there are a few limited places where upgrade modals will appear. We'll test messaging in some of those spaces especially if we need key feedback or metrics from an existing audience like marketers.
• Product Pages - This is probably the most straightforward, and if you have access to a tool that makes A/B or multivariate testing easy then it should be pretty seamless to pull-off. Assuming your product pages get a fair amount of traffic it's possible to quickly test ideas this way as well.

Overall, I like to use the three above in different circumstances and if my team is working on a big important launch -- then potential multiple options above.

• a/b testing emails and landing pages
• using an agency to survey audiences for large brand campaings
• direct user interviews 
• interviewing your frontline teams 
• competitive research

Sometimes you don't have time to do all of the above and you have to go with your gut and then evolve it over time as you get to know your product, users, and market better. 

Running focus groups (with a diverse collection of existing customers and another with prospects) has been one of the most useful methods that I've leveraged to test messaging. Presenting a few different narratives and messaging options directly to your target audiences, and listening to them share what resonates and doesn't resonate (and why) can provide concrete insight in what message(s) would compel them to take serious interest in your offering.

Message testing should take place during strategy development, and again when going to market. You can leverage qualitative methods to gather open ended feedback on messaging directions, and quantitative methods to make statistically significant decisions between your options. I often start with a qualitative approach, conducting interviews or focus groups to understand what sort of messaging appeals to my target audience. What I’m assessing at this stage is clarity, value, relevance, appeal, differentiation, timeliness, and urgency. I’ll use those findings to write messaging that simultaneously leads with emotion and is built on logic. Once I’ve narrowed down the options, I’ll run a survey or experiment to confirm the messaging hypothesis at scale. You can also take the opposite approach, depending on the resources you have at your disposal, starting with a survey or poll to customers asking about their experience with your messaging and then follow up with more in-depth interviews, focus groups and user tests to get at the why behind some of the trends that emerged in the survey results. Other useful quantitative testing methods for when you go to market include on-site behavior analysis and A/B testing across channels, including web, email, and paid, which can be continually optimized for the most performant variant. Don’t forget to revisit your messaging framework and update it according to the learnings you collected in-market.