AMA: Atlassian Director of Product Management (Confluence), Paresh Vakhariya on User Interviews

I was a UX designer and researcher before I became a PM. Here are some best practices I will certainly recommend. Ideally, it would be great to invite engineering and design partners to customer interviews but it is not always possible. * Storytelling and narrative are the big part of any customer interview findings * Using appropriate slides, wiki pages, etc. truly helps stakeholders grasp complex topics. Condense findings for quick and easy consumption. * Key customer quotes and videos is another way to get them develop the same level of empathy. Include user quotes to humanize findings and provide authenticity. * Ideally, all findings in priority order will help stakeholders. * Bringing in metrics impact (how big) of the findings will provide an additional dimension. * Conducting workshops on the findings and determining solutions and next steps would make it even more effective. * Clear concise findings in an easy-to-retrieve portal (e.g. Jira or Confluence) will make it easy to find anytime stakeholders need it.

Here are some common pitfalls to avoid: * Ask Open-Ended and Neutral Questions: Avoid leading or biased questions that may skew the responses. * Focus on Understanding Problems, Not Solutions: Prioritize identifying root problems rather than jumping to potential fixes. Solutions can vary; a deep understanding of the problem ensures the right one emerges. * Observe Non-Verbal Cues: Pay attention to body language, tone, and hesitation to gauge emotions and detect unspoken concerns. * Avoid Fatiguing Participants: Keep interviews short and focused (30–45 minutes). * Probe Deeply for Insights: Avoid surface-level understanding. Use follow-up questions like: * Evaluate Whether Interviews Are the Best Method * For some objectives, other methods like: * Surveys (to gather broad trends). * Participatory design (to involve users in co-creating solutions). * Usability testing (to validate designs or workflows) may be more effective. * Recruit a Diverse Range of Participants: Ensure representation across different demographics, experience levels, and user types. * Create a Tailored Discussion Guide: Develop a structured guide specific to your research goals and audience. Adapt questions based on customer size, industry, and other relevant criteria. * Address Remote Interview Challenges: * Zoom or other virtual tools can introduce unique challenges: * Set clear protocols for observers (e.g., being silent and off-camera). * Communicate and ensure adherence to data privacy policies. * Ensure Data Privacy and Compliance * Have participants sign NDAs to protect both parties. * Anonymize data and restrict its use to internal purposes. * Avoid recording sensitive information unless necessary and agreed upon.

Here are some tips: * A simple yet comprehensive User Research plan that aligns all interviewers with the project’s objectives. * Provide context about the product, target audience, and desired objectives. * Collaborate on a discussion guide: * Include a list of key open-ended questions * Conduct a pilot interview where all interviewers can listen in on the interview and make any adjustments to the discussion guide * Document use of the tools (Zoom etc.) so that all interviewers are using the same tools and process to gather feedback * Daily stand-ups during the research study can also help align on feedback. Adjust the guide or approach based on team feedback. * Video Recording ensures accuracy and allows for unbiased review. * Use tools to document and categorize findings (e.g. Confluence) * All interviewers should collaborate on the research report and present findings together to stakeholders.

Here are some ways you can include the findings on to the roadmap: 1. Organize Findings into Themes * Group findings into categories such as: * User pain points: Challenges or frustrations users face. * Desires: Features or improvements users want. * Behaviors: Patterns observed during the research. 2. Translate Findings into Problems and Solutions * Articulate findings as clear problem statements. * Propose potential solutions that address user needs. 3. Categorize by Time Horizon * Classify initiatives based on feasibility and urgency: * Near-term: Quick fixes or small improvements. * Medium-term: Moderately complex changes requiring planning. * Long-term: Strategic initiatives requiring substantial investment. 4. Evaluate Alignment with Vision and Strategy * Assess whether findings support the product’s overall vision and strategic goals. * Filter out ideas that don’t align, or document them for future consideration. 5. Prioritize Using Frameworks * Apply prioritization methods like: * RICE: (Reach, Impact, Confidence, Effort). * MoSCoW: (Must have, Should have, Could have, Won’t have). * Focus on solutions with the highest value and feasibility. 6. Engage Design and Engineering Teams * Collaborate with design and engineering to: * Estimate effort and technical feasibility. * Understand constraints or dependencies. * Adjust prioritization based on scope and resources. 7. Add High-Impact Ideas afterward to the Roadmap * Integrate prioritized initiatives into the roadmap. * Clearly map out objectives for each time horizon (e.g., sprint, quarter). 8. Validate Solutions Through Usability Testing * Before full implementation, prototype solutions. * Conduct usability testing to confirm they effectively address user needs. 9. Communicate the Roadmap * Share the updated roadmap with: * Leadership: Highlight how findings align with strategic goals. * Stakeholders: Ensure transparency and alignment across teams. * Users: (Optional) Share how their feedback influenced the product.

Here's some thoughts on how to strike the balance between qual and quant: * Qualitative Data: * Provides in-depth insights into why users behave a certain way. * Useful for uncovering motivations, pain points, and unmet needs. * Quantitative Data: * Offers measurable trends and patterns in what users are doing. * Useful for validating findings and understanding scope or scale. Here are some thoughts: * Have a clear research objective and plan (identify the questions and the type of data you need) * Start with some qual data to ensure you have the right problem and solutions. e.g. Interviews * Use quantitative data to confirm whether the issues are widespread or isolated. e.g. Surveys * Use experimentation (via tools like Statsig) to test and learn with real users * Cross-check insights from multiple sources to form an opinion: * Interviews, surveys, experimentation, usability testing, etc.

Here’s a list of tools I have used in my current and past roles: * Participant recruiting: UserTesting.com * Conducting interviews: Zoom, Usertesting.com, UserZoom * Surveys: Qualtrics * Note taking: Confluence, Notion etc. * Data analysis: Dovetail or Spreadsheets * Reporting: Confluence, Figma etc. * Experimentation: Statsig Some other tools include Miro, Mural, Hotjar etc. that can be considered on a case by case basis for various types of research.