What questions do you ask users when trying to improve user onboarding from a product marketing perspective?

I truly believe the onboarding experience is the make or break moment for your product, especially for a self-serve user base. You could have the most incredible product in the world, but if it’s a pain to get set up, you’ll lose your customer.

To improve the onboarding experience, I’d focus both on the folks who did not succeed with using your product and have gone inactive as well as those that did successfully get up and running with your product. What you want to understand is what makes one person succeed and the other fail. Is it a difference in motivation? Skill level? Use case? Once you’ve got your group of users, here are some questions I’d think about asking:

• What were you looking to use to accomplish? For a horizontal product like Airtable, people’s use cases vary wildly. Having this intel helps you spot common problems across user groups 
• How important is a tool like to your current workflow? This helps you get at people’s motivations and intent. If this is something they need in their role or something they’re just trying on for size. Another great question to ask here is ‘What happens if you don’t get a tool like set up?’ so you can gauge impact. 
• Where did you run into problems or roadblocks in getting set up with ? In this question, I find it helpful to guide users through the steps of your onboarding process as they likely won’t remember themselves. This gives you insight into whether there’s a common drop off point in onboarding.
• What help or resources would you have liked to see as part of your onboarding process? Rather than trying to use instincts to make updates, keep it simple and just ask folks for what would have been helpful to them. Some of the best suggestions I've ever gotten have come from this question. 
• (For inactive users) What are you using now to manage your process? If they are not successfully using your tool, it will be helpful to get insight into what they're using instead. Did they go to a competitor? If so, can you look at their onboarding process and get inspiration? Did they go back to not using a tool at all for this? At Airtable, people often continue using spreadsheets, so we focus on how to make the transition from spreadsheets to Airtable easier knowing that’s a sticking point for folks. You might get some similar insight to help people move from whatever their manual process is to your product.
• (For successfully onboarded users) How long did it take for you to start seeing value out of the product? Sometimes products are complicated and it’s going to be difficult to get up and running. If you can get insight into how long it takes for someone to see value, you can understand how to help coach other users to that promised land and can potentially find opportunities to speed up that time to value
  
  Another suggestion I’d make is to focus on more than just surveys. Having a few conversations with users can be much more valuable than hundreds of survey responses. Sometimes we over-index on data and quantitative insights when really what we need are a few qualitative pearls of wisdom. Get folks into user research sessions and have them sign up for and get set up with the product while they’re on a video call with you. Have them talk through their thoughts, questions, and experience as they do it and I promise you’ll get a ton of insight into what it’s like to get up and running with your product.

Oh, I love this question! I’ve been tasked to overhaul many onboarding programs and customer insights are integral to my approach. I believe the best insights are validated through multiple sources so here are a few methods I’d recommend using to inform your strategy:

1. Customer Interviews: Talk to your customers! Make sure you’re not just talking to the successful ones either. While they might be harder to get in touch with (and you may need to offer some $$), interviewing customers who churned early in their lifecycle can open your eyes to challenges or missed expectations—which then become opportunities for improvement.
2. Support Tickets: In particular, looking at support tickets that customers file in their early lifecycle can help you proactively hedge off any painful experiences by providing help or training content in onboarding.
3. NPS & Satisfaction Survey Data: Take a look at your promoters and understand why they’re happy while also understanding how you could have improved the experience for your detractors. I would look at this for all users and also narrow in on the early lifecycle to see if there are things you can address in onboarding that would put someone on a successful path with your product.
4. Product Data: Working with your Data Science team, you can uncover key actions that correlate to higher success with your product. If you can identify those key actions, then the onboarding path’s job is to lead users to taking those actions.

With a bit of a strategic lens and synthesizing from these sources of insight, you’ll want to answer some of these questions (which you can also ask a customer directly):

• What expectations did they have when they discovered your product
• What were they trying to accomplish when they first signed up for your product
• What challenges did they run into
• What surprised them
• Were they able to accomplish what they set after? Why or why not?

Ultimately, we want our customers to find success—so understanding what success means to them and figuring out how your product helped (or eek! hindered) their success is what you’re after. Once you understand that, you can curate an onboarding flow to support them.

Optimizing onboarding becomes very important in a self-serve model because you don’t have the benefit of a sales team to guide prospects through sign-up and onboarding funnels.

It’s great that you already have a list of new users who dropped off because you can start collecting insights by looking at your own data. I recommend segmenting this list by prospects who didn’t complete and did complete the funnel. Start by understanding where in the funnel prospects are dropping off and how you can remove friction from the experience - a better UX, more in-product education, follow up emails, an incentive? You should also implement a quick poll asking prospects why they haven’t finished signing up yet and potential answers can cover topics like a need for more info, pricing, general confusion, or technical issues.

For prospects who actually completed the funnel, take a look at what they did after they opened an account. Did they explore the product, set up any workflows, review any specific help center entries? Also review traffic sources and see if there are any correlations between where users came from and what they did in the product. Is it possible they didn’t return because they had different expectations than what your product actually does? You should also send this group a survey as well to better understand the lack of engagement. Your questions could focus on how users found out about your product, what incentivized them to sign up, how often they plan on using it and if there is anything you could improve for this group.

When I think about improving onboarding, I think about the initial motivations and expectations users are bringing to my product and the specific pain points during onboarding. Here are questions you can ask:

• Initial motivations and expectations

  • What initially motivated you to sign up for our product?

  • What problem were you trying to solve/what were you trying to do?

  • How much time did you dedicate to solving this problem with our product?

  • What expectations did you have before using our product?

  • What resources did you explore before using our product?

  • Were there areas of the product or use cases you expected to see in the onboarding?

  • Have you used a product like ours in the past?

• Specific pain points

  • Where did you get stuck?

    • At what point before that did things become unclear or frustrating?

    • At what point before that did you feel unsure about what to do next?

  • What information or resources would have been helpful to have during the onboarding?

  • How would you redesign the onboarding flow to better suit your needs?

  • How much do you want to be guided through the process versus be able to explore the product independently?

This sounds like the classic "leaky bucket" problem: your bucket is leaking (users), and you're not sure why, so it's hard to figure out what to do about it. Solving this requires a combination of quantitative and qualitative insights.

As a starting point, make sure you understand exactly where and when in the customer journey you are losing engagement with users. This might be really granular: if people are signing up for your service and then never come back, there might be an issue with your post-purchase email communication. If people sign up and you can get them to log back in, but they don't go any further, the answer might be somewhere in the user experience. Having great data and being really specific about this can help you narrow down what is happening and where the journey is falling apart, so you can come up with the right strategy and the best channel for re-engaging users.

Of course, you also need to know "why" users aren't continuing, and that's where asking questions comes in. These can be helpful:

• Why did you stop using the platform?

• How well does the product meet your needs?

• What are your expectations for the product / what are you hoping to accomplish?

• How would you rate the onboarding experience?

• Are there any specific features you found lacking or missing?

• How easy is it to find help?

• Is there anything you would improve?

(Self-promotional warning:) if you're stuck, SurveyMonkey has a "Build with AI" feature that will craft a survey for you based on your prompt. Try asking it, "Why aren't customers returning to our product after signing up?" and send that survey to your customers.

Gather a mix of qualitative and quantitative responses so that you understand what you need to address, and capture some benchmarks so you'll know if you've fixed it.

Whether the answer is better onboarding, product improvements, or something else, this type of insight and feedback will get you pointed in the right direction. Good luck!

I would approach this in two ways:

1. Identify where the problem lies -- use quant data to understand where drop-off is occuring
2. Identify why the problem exists -- understand user motivation and where the friction is in their flow that's preventing them from succeeding.

For the latter, you can ask questions along the lines of:

- Why did you sign-up for this product? What was the immediate trigger and the underlying pain?
- What were you hoping to accomplish during your first session?
- What part of the product were you most excited about playing with?
- How much time and investment were you willing to make in this first session?

This will help you understand the motivations, and then you can tailor your user onboarding towards that. If you can get users to an "aha moment " then they will have enough energy to continue to invest time and energy into your product. 

Think of onboarding as a path towards value. Product marketing, growth, and product should all work together to make the time to value as short as possible. The end goal is always to get the user "activated" (also known as the aha moment). So the first question your product team should be able to answer is --> how do we define activation in the platform? Once that north star is identified (and validated), all work you do on onboarding will be guiding that user towards activation.

If we use AirBnB as an example, and assume that activation is defined as booking a stay, then you can trigger an email nurture sequence if someone created an account and browsed listings but never booked. The end goal is to get them to book, so all of your CTAs will be focused on that.

When talking with customers who have gone through your existing onboarding but have not returned, I would ask them the following:

1. When you first heard about [product], what did you assume the value would be to you?

2. Did you receive this value when you created an account with us?

3. Ask them to screen share and point out what the most valuable feature is. If they are unable to point to anything, direct them to click on the activation feature but then fall silent again as they go through the experience themself.

These questions will teach you a few things:

1. Is there a disconnect between how we're messaging the product (perceived value) and the value they actually receive in the product (experienced value)?

2. Does the existing onboarding flow drop the user off at the activation feature? If not, how can we implement in-app tours or pop ups to help them navigate there?

3. When presented with the activation feature, where are the gaps in knowledge? Sometimes if a feature is flexible and not prescription, the user may need help imagining how to apply that feature for their use case/persona.

Once you have clarity on the above, it should become clear which levers to pull (email, in-app messaging, tours, etc), when they should be utilized/triggered, and what messaging and educational resources should be presented along that journey.

I do want to heavily stress that without activation defined and validated with testing, all work you do on onboarding improvement will be a timely trial and error process - so, it's really important to start with that! Best of luck!!

Important question: Are your "users" also your "buyers" or are the two separate? If they are not, then there are a few things you will want to do before you ever begin soliciting and analyzing feedback:

1. Reassess your messaging; was it geared towards users, or just buyers? The two will often have different roles, and more importantly, respond to different incentives. Did you capture that for the users the first time around and really demonstrate to them the benefits of adoption?

2. Consult with a UX expert, either internally or externally. If it's more than a messaging issue, someone with this background will be better able to understand where the product is falling short of expectations, or being misunderstood/misused. 

3. Have conversations with your buyers, to understand the relationship between them, the users and the product itself. Presumably, you understand your buyer very, very well. So see what they have to say about the situation. Remember, usage is in their interest as well, and they should be equally interested in solving this problem (to realize the benefits they were sold, and not look stupid).