Updated: September 11, 2024- 16 min read
Data can be intimidating. It’s a complicated process to get it right. As Jake Canaan, VP of Global Sales Engineering at Quantum Metric puts it:
“If you think data is not hard, you’re probably underthinking it.”
The first step toward mastering data in product management is understanding metrics. Metrics are the building blocks that create data-driven cultures.
Not every product can be measured in the same way, and not every metric is relevant to every product. Some products are meant to be used every day, like a fitness tracker. For other products, like a dating app, the day you stop using them might be a great day for the user.
Why is data important for Product Managers?
Before diving into the metrics, here’s a quick reminder of the importance of data for product managers.
In short, data enables you to understand the needs and behaviors of your customers, identify opportunities for growth, troubleshoot potential issues, and predict trends. With proper data analysis, you can transform abstract numbers into real-world strategies, leading to improved product design, better user experiences, and, ultimately, increased business success. Deciding which metrics will be used to measure success is an essential part of the product management process.
Of course, nobody becomes a data whiz overnight, and there are plenty of tools to help you understand and fully leverage data, including Google Analytics, Amplitude, and Mixpanel. These tools form an integral part of a Product Manager’s toolbox and can help you to collect, track, analyze, and interpret a wealth of data about product usage, user behavior, market trends, and customer feedback.
Ultimately, data-driven product management leads to better product decisions – something every good product manager strives for.
Measuring Success: KPIs vs Metrics vs OKRs
Metrics vs KPIs
In product management, Key Performance Indicators (KPIs) and metrics are often used interchangeably, but they serve distinct purposes. KPIs are key indicators of performance over time, whereas metrics are measurements of specific activities or processes. While every KPI is a metric, not every metric is a KPI.
For example, take an eCommerce business that tracks website traffic. Website traffic is a metric that measures a specific activity. In this case, how many people are visiting the website.
On the other hand, a KPI for the same business could be 'conversion rate'. This KPI measures the percentage of website visitors who complete a desired action (like making a purchase).
Unlike the 'website traffic' metric, the 'conversion rate' KPI is more strategic. It indicates how well the eCommerce business is performing in terms of its primary goal: converting visitors into customers.
KPIs vs OKRs
OKRs consist of an Objective, which is a clearly defined goal, and Key Results, which are measurable outcomes used to track the achievement of that goal. OKRs are used to set and communicate ambitious goals and results at various levels of an organization. They aim to create alignment and engagement around measurable goals.
An OKR for improving a product might look like:
Objective: Enhance user engagement in our mobile app.
Key Result 1: Increase the average session length from 2 minutes to 5 minutes.
Key Result 2: Improve the daily active users (DAU) by 20%.
Key Result 3: Launch two new features aimed at increasing user interaction within the app.
KPIs are often ongoing with no set end date, used to monitor continuous performance, while OKRs are typically set for a specific timeframe (often quarterly), after which new OKRs are formulated. OKRs often encourage setting stretch goals to push the team beyond its current capabilities, whereas KPIs tend to focus on maintaining or incrementally improving performance.
Product OKR Template
Use this Product OKR template to set and track your OKRs (Objectives and Key Results). Align your team’s daily tasks with product and company strategy!
get free template
Reaching for the Stars: North Star Metrics
This key metric aligns with the company's vision and reflects the value the product delivers to customers. Your North Star is your product’s most important metric. For instance, Netflix's North Star Metric might be session duration, indicating user engagement and aligning with its goal of becoming the premier entertainment distribution service.
Acquisition Metrics for Product Managers
1. Cost Per Acquisition (CPA)
Cost Per Acquisition, or CPA, is simply the cost of acquiring a new potential customer. Note that we’ve specified it’s a new potential customer and not a paying customer. This could take the form of webinar registrations, account activations, or other types of leads. (If you want to calculate the cost of acquiring paying customers, you need to use customer acquisition cost (CAC), but we’ll get to that!
Tracking CPA helps product managers to better allocate their marketing budget and strategies.
For example:
If one month you spend $1000 on marketing that leads to 500 potential customers signing up for a free trial of your accounting software, your CPA is $2 per lead. If the next month you spend $1500 and get 520 leads, your CPA is $2.88. This indicates that your marketing efforts have been less successful in the second month. Sure, you got more leads than the previous month, but for a much higher cost.
2. Customer Acquisition Cost (CAC):
CAC shows the total cost of gaining a new customer, including marketing, sales, and other related costs. To work out your CAC, divide the cost of your marketing efforts by the number of customers you acquire.
For example: Let’s say you ran a Facebook ad, and spent $2000 on the design, building the landing page it leads to, and ad fees. If that ad gained you 200 customers, that’s a CAC of $10.
CAC is a great metric to keep track of alongside your CLTV (see the Revenue section below). Your customers may be expensive to acquire, but if the CLTV is high enough, it’s worth it. However, if you find that your average CLTV is pretty low (let’s say you’re operating on a freemium model), a high acquisition cost could put you out of business!
An ideal CLTV/CAC ratio is 3:1, meaning that the revenue generated by a customer should be three times more than the cost of acquiring them.
3. Bounce Rate
Bounce rate is the percentage of users who leave your platform after viewing only one page. For instance, if 40 out of 100 users drop off after visiting one page, the bounce rate is 40 percent. A high bounce rate could indicate potential issues with user experience, site design, or content relevance.
Conversion Metrics in Product
4. Conversion Rate (CVR)
Conversion Rate is the percentage of users who take a desired action. The 'desired action' can be anything from making a purchase to signing up for a newsletter, to downloading a whitepaper, depending on the company's objectives. By monitoring CVR, you can identify bottlenecks in your conversion funnel and improve user experience.
For example:
Imagine you’re a product manager at an eCommerce platform. In August, your website attracts 50,000 visitors who add products to the shopping cart. However, only 20,000 complete their purchases, i.e. a CVR of 40%. This gets you thinking. Is the account sign-up process too complicated? Is there a lack of payment options? Is there a bug?
You make some changes and see that in September only 40,000 visitors added products to their shopping basket, but 30,000 completed their purchases. Your CVR is now 75%, or 35% higher than the previous month.
5. Time to First Value (TTFV):
The time it takes for a new user to realize the value of the product after their first interaction. This metric is crucial for understanding initial user engagement and satisfaction. When processes are too complicated, TTFV increases, leading to drop-off.
For example, in their whitepaper on digital onboarding, Visa reports:
“When it comes to digital onboarding, the average amount of time after which customers abandon their application is 14 minutes and 20 seconds. Any longer than this, and 55 percent of customers leave the process.”
6. Conversion Funnel Drop-off Rate:
This metric analyzes at which stage in the conversion funnel (from awareness to purchase) potential customers are lost. This helps in identifying and rectifying bottlenecks in the user journey.
Engagement Metrics for Product Managers:
7. Daily Active Users (DAU) and Monthly Active Users (MAU):
These metrics provide valuable insights into how often consumers use your product in a given day or month. These metrics help identify the level of engagement and adoption of your product.
If your product is a fitness tracker app or anything that is designed to be used daily, then this is a very relevant metric. In contrast, DAU is less valuable for a vacation rental platform like that isn’t intended to be used daily.
The ratio of DAU to MAU calculates the stickiness of a product, showing how often users return to the product within a month. It's a measure of your product's ability to engage users over time. A stickiness of 20% is considered good, indicating that users engage with the product at least once every five days on average. Anything above that is excellent and shows a high level of user engagement.
8. Session Duration
Session duration is an average measure of the time users spend on your platform during a single session. Longer sessions typically indicate higher engagement levels.
A session begins when a user opens an application and ends when they disengage, which can be defined in different ways: closing the app, of course, but also opening a separate window or otherwise going inactive.
9. Session Frequency
This measures how often users return to the app or product within a given timeframe. It helps in understanding the habit-forming potential of the product.
10. Feature Adoption Rate
Measures the percentage of users who start using a new feature within a certain time frame after its release. This helps in understanding how well new features are being accepted by the user base.
The monthly feature adoption rate = feature MAU / monthly logins.
Users who pay yearly or monthly for a service are choosing a product again and again. They have to keep finding value in the product, and the best way to ensure that is to keep releasing better and better features. Product managers spend a lot of time and effort working out which features to prioritize and how to release them in the perfect order at the perfect time.
But if no one knows the feature is there? Or, even worse, they know but they don’t care? This is vital information as feature adoption is a strong predictor for retention.
Retention Metrics
11. Customer Retention or Churn rate
New customers every day looks great on the growth charts, but if those customers are dropping off after only a few days (or in the fickle world of apps, a few seconds!) then you’ve essentially got a leaky bucket instead of a product. There’s no point filling the bucket with new users unless you can keep them. What you want is a high Customer Retention Rate, where more people stay than go.
As Pratik Shah, Growth Product Manager at AirBnB said,
“Companies that haven’t understood retention, and stepped on the gas too fast with their acquisition, have then lost all of their users very quickly. Without users, your product is nothing.”
For example:
Imagine you’re a PM at a subscription-based streaming service. You’re interested in how many customers you retain after a 6 month period. You gain 3,000 new customers between January and June, bringing your total number of customers to 15,000. To work out the CRR you subtract the number of new customers (3,000) from the total number of customers (15,000). You then divide that number (12,000) by the number of customers at the start of January (13,000) and multiply by 100 to get a percentage. This gives you a CRR of 92.31%.
The fact that you manage to retain over 90% of your customers after 6 months is an indicator that your customer retention strategies are working well. If your CRR goes down, it’s a sign you need to iterate and focus more on this area.
The definition of a “good” churn rate depends on the product. For example, dating apps like Badoo and Tinder are sometimes uninstalled because they’ve served their purpose and the users are now in relationships. Not all products are designed to be used for the rest of their user’s lives!
12. Net Promotor Score (NPS)
Definition: Net Promoter Score, or NPS, is a metric used to gauge customer loyalty and satisfaction. NPS was developed by Fred Reichheld, Bain & Company and Satmetrix as a simple yet effective tool for understanding customer sentiment.
The NPS is determined by asking customers a single question: "On a scale of 0-10, how likely are you to recommend our company/product/service to a friend or colleague?" Based on the responses to this question, customers are categorized as Promoters (9-10), Passives (7-8), or Detractors (0-6), and the NPS is calculated.
For example:
You’re a product manager in the healthcare industry and you recently launched a new app that collects and analyzes data about user’s menstrual cycles. You’d like to understand if your product has been well received.
To do this you send your users a push notification with the question "On a scale of 0-10, how likely are you to recommend our app to a friend or colleague?”. 1000 users respond. 100 give a score between 0-6, 200 give a score of 7-8 and 700 give a score between 9-10. This means that 10% of your users are detractors, 20% are passives and 70% are promoters. To calculate your NPS score you subtract the percentage of detractors (10) from the percentage of promoters (70), giving you an NPS score of 60. Success! Most of your users would recommend your product.
13. Customer satisfaction score (CSAT)
CSAT is a simple metric and can be used to measure how happy users are with individual processes and features. While NPS is more often used to measure customers' happiness with the entire user journey, CSAT allows you to be more specific.
For example:
You can ask users to score your onboarding experience out of ten once they’ve finished a tutorial. A one-tap survey of with a score of 1-5 or 1-10 is a common feedback collection method within customer service.
14. Customer Effort Score (CES):
Measures the ease of interaction with the product or service, based on the premise that customers are more loyal to products that are easier to use.
The most common way to collect data for CES is through customer surveys.
Revenue Metrics for Product Management
15. Lifetime Value (LTV) or Customer Lifetime Value (CLV)
Lifetime Value (LTV), also called Customer Lifetime Value (CLV), represents the predicted net profit from the entire relationship with a customer.
LTV helps you to put a value on your users. For example, if you run a snack box subscription service that the average customer pays $30 per month for and the average customer lifetime is 2 years, you can predict that the LTV per customer is $720.
You can calculate the LTV for each segment of your users and tweak your product strategy accordingly. If the average B2B snack box customer spends $40 per month for an average of 5 years, you’ll find the LTV is $2,400, meaning you would be justified in spending a bigger proportion of your marketing budget attracting B2B customers.
LTV provides significant insights into customer behavior and the overall health of the business and a product's long-term profitability.
16. Monthly Recurring Revenue (MRR) and Annual Recurring Revenue (ARR)
This metric is a measure of the predictable revenue a business can expect to receive each month or year. Recurring revenue indicates that the customer has committed to pay a certain amount for a given period as opposed to a one-time purchase.
MRR is an important indicator of financial health for monthly subscription-based businesses like Amazon Prime, Spotify, or Netflix. ARR offers a wider scope allowing for the inevitable ups and downs of revenue due to external factors such as holidays and seasons.
17. Average Revenue Per User (ARPU):
The average revenue generated per user can be segmented by cohort, product line, or subscription type. This metric is important for understanding the revenue impact of user acquisition and retention strategies.
For example, you can calculate the CAC (Customer Acquisition Cost) Payback Period in months by dividing the CAC / Monthly ARPU. This will tell you how many months, on average, a user has to be a paying customer in order to break even on the investment it took to convert them into a paying customer in the first place.
Product Feedback and Optimization Metrics
18. Feature Request Tracking
The number and nature of feature requests can provide direct insights into user needs and desired improvements, helping prioritize product development efforts.
While it might seem like more requests is a bad thing, it doesn’t have to be. It can also be an indication of an engaged user base that believes in the product and is committed to growing along with it.
19. User Feedback Volume
The quantity of feedback received through various channels (support tickets, social media, forums, etc.).
There are various channels through which customers can submit feedback. If these channels are never viewed as a whole, then you’ll miss the forest for the trees. This metric can help gauge overall user engagement and the urgency of addressing certain issues or requests.
20. Customer Sentiment Score:
This score aggregates positive, neutral, and negative sentiments to provide an overall metric indicating customer satisfaction, loyalty, and overall sentiment toward the product or brand. Higher scores reflect more positive customer sentiments, while lower scores indicate areas that may require attention or improvement.
The most expedient way to measure customer sentiment is with artificial intelligence. Natural Language Processing (NLP) allows AI tools to analyze the overall sentiment of vast quantities of customer data. This includes not only internal surveys but also comments on social media and online review platforms.
21. Escalation Rate
The rate at which customer issues are escalated to higher levels of support or management. A high escalation rate might indicate underlying problems with product complexity, bugs, or customer support effectiveness. This metric could be particularly relevant when a company implements a new customer support process, such as an AI-powered chatbot.
Managing your product data
Managing your data can seem like a daunting task if you don’t come from a data science background. There are a few keys to proper data management that will set you up for success:
Centralize your data. If you have your data split up across several different sources, keeping track of metrics is a challenge. When different teams/team members are accessing different data locations, you risk miscommunication.
Share your data. Shared data is more powerful. Empower team members to make good, data-driven decisions by giving them access to the metrics they need.
Visualize your data. Using tools with data visualization features turns complicated numbers into simple visuals. This makes them easy to understand for all team members, not just those who are well-versed in data.
Updated: September 11, 2024
