While both traditional (non-AI) PMs and AI PMs require strong customer understanding, product sense, execution, and stakeholder management skills, AI PMs need additional expertise that makes some people better suited for the role: 1. Technical Knowledge – AI PMs don’t need to code, but they must understand the ML lifecycle, model performance metrics, drift, explainability, data pipelines, and experimentation. They should be able to collaborate effectively with data scientists and ask the right q ...Read More
Shruti Tiwari
Principal Product Management, AI/ML
Content
With product vision, we need to carefully tread the line between making it too generic/vague or too technical. Other things to consider are - it should clarify what impact it creates, what unique value it offers and what is the long term goal while also making it inspiring. If target audience or problem area is niche, then it should also specify the target audience or the specific problem. An example for an unclear product vision is “ We want to make customer service better for companies”. Utili ...Read More
My one piece of advice would be to be outcome-driven. A lot of them don’t make it to deployment or don’t deliver as expected. To ensure that efforts turn into outcomes, be thorough about the user needs. Don’t focus only on the model performance but also make sure that the user interface is smooth. Learning to work well with data scientists and business stakeholders, and proactively solving the potential roadblocks are some ways to succeed as a Product Manager.
Yes, we incorporate user feedback to improve the model and reduce drift over time. There are several ways to gather feedback: 1. Direct user feedback through thumbs-up and thumbs-down ratings. 2. User override rate – measuring how often users override AI-generated suggestions. 3. Tracking drop-off points where users disengage from the AI feature. 4. User support tickets – identifying recurring issues raised by users. To incorporate feedback, we analyze failure points to determine the root cause. ...Read More
Based on my experience, some common mistakes AI PMs make include focusing too much on technology instead of user needs and business impact, ignoring data quality and feasibility early on, and deploying models without a feedback loop. My advice for new AI PMs: 1. Take a user-first approach, with AI as an enabler, not the driver. Work backward from how users will interact with the product and then develop the model that best fits their needs. 2. Thoroughly analyze data quality and availability and ...Read More
I would think about Gen AI integration into any application based on its use case. There are two major purposes that Gen AI serves - Content generation and natural language understanding. We can further break it down into multiple use cases- Content generation a. By modality like text, audio, video, image, code, 3D, multimedia b. By domain like healthcare, pharma, retail etc. c. By application like conversational chatbots, summarization, editing, generation of lists, reminders, emails, social me ...Read More
I would divide the hard skills into 4 categories - AI foundational knowledge at conceptual level including understanding of AI/ML model lifecycle, different algorithms like regression, decision trees, neural network and their use cases, model evaluation metrics like accuracy, precision, recall etc. Latest evolutions in architecture and models like knowing which new open source models can be explored for specific use case, new architectures to be tried etc. Comfort with Data analysis - I found ha ...Read More
Exact metrics would vary by product and application, but I would think about metrics in three categories-
Model performance metrics like accuracy, precision, recall, AUC-ROC, latency etc. Key metric will be based on the application like minimizing false negatives in email filtration app to avoid missing any important emails
Product adoption/engagement metrics like adoption rate, user override rate, CTR etc.
Business impact metrics like cost savings, revenue impact, CSAT impact, time savings etc.
There are already Product managers focused towards data science teams in many companies and I am one of them. Generally, I see three different types of Product Management roles emerging with AI - Customer facing Product managers- working with customers to gather requirements and performing functions of traditional PMs Technical Product managers (B2B or B2C) - Working with data science teams to build, validate and iterate the Product Technical Product managers (developer focused)- Building produc ...Read More
Well, the good news is as the first PM in the team, you can introduce some new ways of doing things. The bad news is that its a journey and it'll take some time to change the mindset. You'll need to establish trust with the team and show some wins before the team can buy into the change. The first thing I would do is to invite engineers to listen to user interviews or support calls. That'll make customer real for them and once they see the impact of what they build, they'll start thinking differ ...Read More