Updated: July 16, 2024 - 13 min read
The integration of Artificial Intelligence (AI), especially Generative AI, into our daily lives is changing the world of product and the role of Product Managers, especially for PMs that develop in AI products. This article covers the various ways that PMs can leverage AI, whether its using ChatGPT more effectively or developing an AI feature.
In this article, we’ll cover Gen AI in product management in all its shapes and forms including:
Building AI products: Why do it, how to leverage UVPs, and examples of products that pull it off
Large Language Models (LLMs): Prompt engineering, optimization, and choosing the right LLM
Use Case: Gen AI in CX
How to use AI in product management tasks to increase increase efficiency
Gen AI products for PMs
Resources and certifications in Gen AI
Generative AI Product Management: Developing Gen AI-Powered Products
Why build AI features?
Source
We’ve seen over the last several years that generative AI’s “wow factor” is unbeatable. When done well, AI products have huge potential to increase revenue, market share, and stickiness. In fact, in many industries, the failure to develop AI products can hold companies back when consumers come to expect AI features or competitors beat them to the punch.
That said, products powered by generative AI are expensive to develop and missteps can damage trust with users. To succeed at developing AI products, PMs need a compelling business case to secure leadership buy-in.
The good news is, the stats above are pretty compelling! PMs advocating for a new AI product or feature can use market research and specific use cases as the basis for a compelling AI product vision. Product School’s AI Product Requirements Document template is also a great way to align everyone around the costs and benefits.
AI PRD Template
Plan, strategize, and align stakeholders around the key requirements unique to AI products.
By sharing your email, you agree to our Privacy Policy and Terms of Service
Leveraging UVPs in Gen AI Products
The right AI feature for your team to develop tomorrow depends on what your business does well today. Which Unique Value Proposition (UVP) is your team in a position to leverage?
Furthermore, the AI product strategy must align with the overall business strategy. Where is the organization headed? Are you looking for opportunities to strengthen your existing features or create new revenue opportunities?
Gen AI products run on data. The central question that PMs developing Gen AI products need to ask themselves is, what kind of data does our organization have? How can I use it responsibly?
Examples of Gen AI products that align with existing UVPs
LLMs in Product Management: Choosing the Right Gen AI Engine
Large Language Models (LLMs) are the engines that drive AI products. These advanced artificial intelligence systems are trained on vast amounts of text data to understand, generate, and manipulate human language for various applications. Product teams often depend on existing LLMs to power their Gen AI products.
Out-of-the-box LLMs are like cookbooks; they contain a wide variety of recipes, but to create a signature dish, a chef must experiment with and refine the recipes. Similarly, these models require further training to master specific tasks and achieve exceptional results.
Prompt engineering
The good news is that LLMs’ general knowledge allows them to converse with humans through prompts in a process known as prompt engineering. Prompts for instructing LLMs are called system prompts.
Example of a system prompt:
"You are ChatGPT, a helpful all-purpose assistant. You are the number one chat AI assistant in the world and are beloved for your comprehension and knowledge. When responding to user queries, provide accurate, balanced, and comprehensive information. Prioritize user safety, respect privacy, and avoid generating harmful or misleading content. In cases of uncertainty or sensitive topics, guide users to seek professional advice or further research."
In some cases, prompting may be enough to instruct the LLM to perform a given task. In others, training or optimization is necessary. There are two main approaches in this context: fine-tuning and retrieval-augmented generation (RAG).
Fine-tuning
To fine-tune a pre-trained model (like GPT 4) on custom data sets specific to the task at hand, in addition to using prompts for instruction, the learning rate can be adjusted in the hyperparameters (these are the LLM’s higher-order settings) to make smaller, more precise changes without overriding the initial training.
Retrieval Augmented Generation (RAG)
RAGs combine two different types of models: retrieval and generative. Retrieval models, similar to search engines, serve up specific information in response to queries. When combined with LLMs, they form a RAG capable of providing highly contextual information in natural language related to a specific use case. Unlike an app like ChatGPT, which uses publicly available information as its knowledge base, RAGs are enhanced with specific information that isn’t just a Google search away.
Examples of LLMs that power generative AI products
There are hundreds of LLMs in existence today, and that number continues to grow. The first task for many product teams tasked with developing an AI-powered feature will be deciding which LLM best serves their objectives.
🏛️ The Classic: GPT
Developed by OpenAI, GPT is the LLM that powers ChatGPT, the app that made generative AI a household name. The latest version, GPT-4 Omni (GPT-4o), accepts various inputs including audio, image and text.
🗝️ Safety first: Claude
Developed by Anthropic to be honest, prioritize harm reduction, and keep data secure for enterprise customers. Slack, Notion, and Zoom have all partnered with Claude.
🏎️ Fast and Furious: Mistral
The company behind this model is also called Mistral. It uses sub-system technology to outperform other models despite having fewer parameters, making it capable of running faster on less powerful hardware.
🌠 Rising star: Gemini
Developed by Google, Gemini (formerly Bard) models are integrated with Google search technology and can natively handle images, video, and audio in addition to text and code.
Key considerations when choosing an LLM
Cost effectiveness: Larger, more powerful models may also incur higher usage fees, especially if accessed via API (e.g., OpenAI's GPT API). If you're accessing the LLM through an API , consider the pricing structure, which often includes charges per token (approximately 4 characters in English), request, or compute time.
Fine-Tuning: Check if the model allows for fine-tuning on your specific dataset to better align its performance with your product's requirements.
Data security: Understand the nature of the data the LLM was trained on to assess potential biases or privacy concerns, especially if the model was pre-trained on publicly available data. If your product handles sensitive or personal data, ensure the LLM's instruction and operational protocols comply with relevant data protection regulations such as GDPR.
Use Case: How Product Managers Use Gen AI for Customer Insights
Improving customer experience is the most profitable way for many organizations to use generative AI tools. According to McKinsey's Economic Potential of Generative AI report, companies stand to see a 30-45% increase in productivity and improve customer experiences using generative AI.
Elsewhere on the blog, we’ve talked about creating AI-powered chatbots using LLMs. Here, we’ll explore how PMs can tackle the overwhelming amount of customer support data and feedback from users to improve and develop customer-centered products.
Analyzing Customer Feedback with Generative AI
Generative AI models, powered by machine learning and large language models, offer unprecedented capabilities in understanding and processing vast amounts of customer data in real time. This technological advancement enables product teams to identify market trends, understand customer pain points, and enhance user experiences more efficiently than ever before.
Understanding customer feedback is crucial for refining products and enhancing user satisfaction. Generative AI, particularly when combined with Natural Language Processing (NLP) techniques, offers a powerful tool for extracting meaningful insights from diverse customer feedback channels. Here’s a step-by-step guide on how to effectively analyze customer feedback using AI:
Step 1: Aggregating Customer Feedback
Start by collecting feedback from all available sources. This includes support tickets, product reviews, customer surveys, social media comments, and forum discussions. The goal is to have a comprehensive dataset that represents the voice of your customer base.
Support Tickets: Compile tickets from your customer support platform.
Product Reviews: Gather reviews from your website, app stores, and any other platforms where your product is listed.
Surveys: Collect responses from recent customer satisfaction or product feedback surveys.
Social Media & Forums: Aggregate comments and discussions about your product from social media platforms and relevant forums.
Step 2: Preparing the Data
Consolidate the collected feedback into a structured format suitable for analysis. This might involve cleaning the data to remove irrelevant information, correcting typos, and standardizing formats to ensure consistency across different sources.
Step 3: Using NLP for Initial Analysis
Utilize NLP techniques to process the text data. This involves:
Sentiment Analysis: Determine the overall sentiment (positive, negative, neutral) of each piece of feedback to gauge customer satisfaction.
Keyword Extraction: Identify key terms and phrases frequently mentioned in the feedback to highlight common themes.
Entity Recognition: Detect and categorize important entities such as product features, issues, or specific services mentioned by customers.
Step 4: Summarizing with Generative AI
Input the processed data into a generative AI model designed to summarize and highlight key issues. The model can be programmed to:
Summarize Individual Threads: Provide concise summaries of long customer emails or support ticket threads, focusing on the main points and concerns raised.
Aggregate Insights: Generate a report summarizing common themes and sentiments across all collected feedback, categorizing them into areas like product features, usability, customer service quality, etc.
Step 5: Identifying Key Issues and Opportunities
Analyze the AI-generated summaries to identify the most frequently mentioned pain points and areas for improvement. Look for patterns that suggest systemic issues or opportunities for significant enhancements.
Example: Analyzing Customer Support Emails
Imagine you have a month's worth of customer support emails that you want to analyze for common issues and themes. Here's how you could approach it using generative AI:
Data Preparation: Compile all customer emails received over the last month into a single dataset. Ensure the data is clean and formatted consistently.
NLP Processing: Use NLP to analyze each email, extracting key phrases, sentiment, and categorizing the type of issues mentioned (e.g., technical problems, billing inquiries, usability feedback).
AI Summarization: Input the NLP-processed data into a generative AI model to generate summaries for each email thread. Configure the AI to focus on extracting the essence of the customer's problem or feedback.
Aggregate Analysis: Have the AI model provide an overarching summary that highlights the most common issues, positive feedback, and suggestions for improvement based on the entire dataset.
Review and Action: Review the AI-generated summaries and overarching analysis to identify the top priorities for action. This could include fixing prevalent bugs, addressing common usability concerns, or enhancing customer support for specific issues.
By following these steps, product teams can transform raw customer feedback into actionable insights, enabling data-driven decisions that enhance product development and customer experience. This methodical approach ensures that every piece of feedback is valued and considered in the continuous improvement of the product.
Using Generative AI in Product Management: Increase Productivity and Enhance Workflow
PMs can use Gen AI to increase productivity across various tasks. AI tools like chatbots can make Product Managers more efficient at:
Generating new ideas
Weighing pros and cons
Prioritizing daily tasks
Streamlining processes
Try this productivity prompt developed by Samantha Stevens for task prioritization:
"I'd like you to act as a productivity coach. You excel at taking long-term goals and breaking them down into extremely specific and detailed plans to work towards on a daily or weekly basis.
You help ambitious, goal-oriented people stay on track and provide them structure and guidance to achieve everything they want to experience in life.
In a minute, I am going to give you a goal I'd like to achieve and I would like you to help me come up with a very specific action plan. This plan should include tasks I can work on daily or weekly. It should be extremely clear and concrete, and you should tell me how to do each step.
I have 3-5 hours per week to work on this goal. Given my time availability, please come up with an estimate for how long it will take me to achieve my goal and create my plan as such.
This can be a collaborative process and we will work together. Before we get started, do you have any questions about the task to help you better prepare?”
AI Prompt Template
Engage effectively with natural language processing chatbots to ensure quality results.
GET THE TEMPLATE
Gen AI Products for Product Managers
Beyond general-purpose Gen AI tools like ChatGPT, there are product-focused solutions available for PMs. These tools are designed with Product Teams in mind:
Turn session replays into insights with LogRocket Galileo AI
Base your feature backlog on user feedback with Productboard AI
Convert your resources section into a chatbot with Helpbar by Chameleon
Pull key insights from large data sets with Mixpanel
Create better PRDs, surveys, etc with Sprig
Resources and Certifications to Reskill in AI Product Management
The world of artificial intelligence is vast and fast-paced. Product managers looking to build AI-powered products and get the most out of AI need a deep understanding of AI to stay up to date with the latest tools, trends, and regulations.
The lack of Gen AI product knowledge is holding companies back. As Product School’s AI product expert and Artificial Intelligence for Product Certification (AIPC)™ course creator Samantha Stevens puts it:
“Companies want and need to build AI features, but their teams don’t have the skills. Most product managers don’t understand the technology or how to incorporate it into their products.”
This skills gap presents a unique opportunity for motivated Product Managers to level up on Gen AI and become a part of the AI PM community.
Start your AI product journey: Free AI resources
As you know, Product School is here with tons of free resources to get you started learning about AI! Check out our YouTube channel for free webinars from AI experts or see the recommended articles below to keep reading about AI.
Our most comprehensive free resource is our AI micro-certification, AIC™. This self-paced course grants insight into the 'how' and 'why' behind creating effective AI user experiences:
Artificial Intelligence Micro-Certification (AIC)™️
Elevate your expertise and lead the charge in driving exceptional AI-native user experiences with the Artificial Intelligence Micro-Certification (AIC)™️
Enroll for free
Become a certified AI product expert with AIPC™
The AIPC™ course is meticulously designed to help PMs master Gen AI, data-driven decision-making, and build trust in AI products through excellent user experience. AIPC™ students' experience has been overwhelmingly positive!
What you’ll do in AIPC™:
Learn to harness AI’s potential to enhance product value
Integrate AI with the best LLM (see above) to suit your product needs
Design user-centric AI experiences
Establish the right AI-specific metrics and develop an evaluation rubric to measure success
Use Gen AI in your day-to-day tasks as a PM
Capstone project: Create an AI PRD
Get Certified as an AI Product Manager
To truly set yourself apart, and stay ahead, learn how to build AI products and integrate AI across the entire product lifecycle. The Artificial Intelligence for Product Certification (AIPC)™ provides hands-on instruction to help you learn the skills and knowledge required to build the next generation of products.
enroll now
Updated: July 16, 2024
