Ask Me Anything: Product Discovery

February may be the shortest month of the year, but you sure filled it with questions! In this month's #AskMeAnything sessions, these Product Experts answered audience questions:

• Zsofia Kerekes, Product Leader at Meta

• Saurav Sharma, Senior Product Manager at Amazon

• Prasad MK, Senior Product Leader at Amazon Web Services

• Ulaize Hernandez, Product Lead at Shopify

Read on to hear their experience with the Product Discovery process, top skills for PMs, and AI in Product Management. To navigate the questions, click through the Table of Contents above 👆

What is your Product Discovery process? How do you know when it’s time to leave the problem space and start building?

Zsofia Kerekes

I actually worked for years as a Customer Development consultant for startups so discovery is close to my heart.

1. One general rule of thumb I advised was not doing a single line of code before doing enough problem interviews w users. Usually after 5-6 you start seeing patterns

2. Form a hypothesis around the biggest pain points and start digging into existing data points then move to smoke tests then more high fidelity tests like A/B testing. Pivot at each point as needed.

Itamar Gilad and Theresa Torres are some experts I advise following for techniques

Prasad MK

Product Discovery process starts with customer problem statement definition, followed by hypothesis creation & validation. This will help define problem-fitment. Post this, market-fitment is done to define the business potential. With that, we move into the next phase of defining the Minimum Lovable Product and roadmap to build it.

Saurav Sharma

Product Discovery is the stage when you are doing research about your Product idea to see if it is a good fit for the market. It involves  both Primary research (talking to target customers) and Secondary Research (Surveys, User testing,Competitive benchmarking etc.) to see if there is qualitative & quantitative evidence that customers will want your Product for their Problems. Once you achieve that, I think you are in a good spot to start your Building Process.

Ulaize Hernandez

Product Discovery differs a lot between discovery for strategy and discovery for a more narrow problem area. I will focus on a more narrow problem area but happy to share strategy discovery as well.

My Product Discovery process heavily relies on research and/or data.

First, I will dig into the data. What do we know about the usage of the product? Sometimes, a lot will become clear only from the data. For example, I once found out a feature was only discovered 250 days after the customer onboarded into the product. This insight was enough to know we should prioritize the problem. It was also easy to know how we could improve it.  We "simply" had to move the discovery of the feature to a flow that was intuitive but it was earlier in the customer onboarding journey.

However, sometimes data is not enough. For example, when looking into inventory management for Shopify, it is really difficult to know when a product inventory level does not match reality (it is out of stock in reality but it is not in the system). For this you need research. I like interviewing at least 10 customers that meet a specific set of criteria. I always draft a script but leave it quite open and carry out a "jobs to be done" type of interview. These types of research plans have a very long shelf life. Not only they help with the discovery at hand, but they bring huge learnings that can be carried into the future. Once you interview a good amount of customers in this systematic manner, trends become clear and the problem is easy to define.

There are obviously many other inputs and tools. Talk to support and sales. These teams are your best friends!

You need to remember that this is not a template. Each problem space requires different tools and as a PM, you need to learn what tools work in which scenarios.

Any tips to convince the stakeholders to invest more in the discovery process?

Zsofia Kerekes

Any past examples of how much dev and operational money was lost on features no one used in the end can be an angle.

Cheap tests first (e.g. smoke tests) and impact they bring on metrics like conversion rate can convince them to invest more as well.

Can you give a real example of a validation of the hypothesis in the discovery step?

Prasad MK

Hypothesis validation primarily happens to validate the following :

• Who is the customer?

• What is the customer problem or opportunity?

• What is the most important customer benefit?

• How do you know what customers need or want?

• What does the customer experience look like?

To do this, we may not need a tech resource. A PO combined with a UI/UX designer (optional) should be able to validate the hypothesis through customer interviews. 

How do you improve retention?

Zsofia Kerekes

Wow this is a tricky question! I’d first break it down: in which user segment is this a problem?

Afterwards I’d understand using user data to form and validate hypotheses what the reasons for churn may be. You can then map out different ideas e.g using an Opportunity Map technique, prioritize which may be lowest effort and biggest impact.

Practical tests I could think of is AB testing ‘nudges’, educational materials, promotional offers, depending on those pain points, your product customers and your budget…

How do you decide when and where to incorporate AI for task automation and ML for personalized products?

Zsofia Kerekes

I’d think of criteria such as:

1. Whether the decision making or process always unfolds in a repetitive way

2. Is task repeated or one off?

3. Compare costs of engineering and maintenance with manual operations
   

What is the difference in approach when building a 0->1 product vs when scaling a B2B SaaS from 100 paying clients to 1000 paying clients?

Saurav Sharma

That's a great question. I call the building 0->1 Product an experimental approach of trying different concepts before you land on to the one where you want to focus using the Product Discovery process I described above. But scaling B2B SaaS from 100 to 1000 paying clients is all making sure you have your Product expansion motion right with the Sales & marketing teams assuming you have the right product to serve your Enterprise customers. Plus also ensuring that you are solving big gaps in your Product which hinder you adoption along the way for Enterprises.

What are the top 3 soft skills you look for when hiring PMs?

Saurav Sharma

That's a great question. 3 key soft skills for PM's:

1. Being able to Empathize with Customers

2. Good Communication

3. Negotiation skills. Being able to negotiate well requires you to be a good listener too to understand other person's POV. 
   

How long do you like to spend in a problem space or team before moving on in your career to the next challenge and growing your breadth of experience. Any advice to share?

Ulaize Hernandez

I am not good at this but I see most people taking too long to leave.

Product is a difficult space. It takes a long time to know a product area, figure it out, build a strategy and ship it. Some products can take years to come to fruition. Ideally you will stay until that happens but it is not that simple.

There is not a perfect formula. But asking yourself the following questions can give you a good sense of wether you should change or not:

• Will moving teams/product harm your career? How about staying? You want to prioritize having impact.

• Are you able to succeed with the team you have? If not, is that going to change? A PM is only as good as the team they have. You can do a lot to improve this but it is not all under your control.

• How do you like your manager and the people you work with? PMs are good at adapting to different people but you should also have fun! Who you work with is important to enjoy what you do.

• Are you aligned with the product principles of your leaders? Building a product within an organization that does not align with your values and principles will be hard.

There are probably more questions to this but this should be a good start. And remember, don't be afraid to talk to other teams and network.  You have little to lose.

What are some of the top 2-3 differences that you have observed recurringly between new PMs and experienced PMs?

Ulaize Hernandez

• Frustration. Younger PMs get frustrated a lot easier, mostly with other people. Once you are experienced, you have been down that road before. You know what to do and probably have a clear sense of how things will unfold. This frustration usually stems from wanting things to go well and do the right thing so it is not a bad trait. I have mentored junior PMs that have been in these situations. Empathy goes a long way. As a Senior PM you can say: this is normal, I have felt the same frustration, let's focus that energy on next steps. This mix of empathy, acknowledgment and pragmatism has worked well in my experience.

• Jumping to conclusions. A more junior PM will hear 1 or 2 insights and jump to conclusions. More Senior PMs can see the gaps in knowledge more clearly and are able to question those insights with a different perspective.

Any final tips?

Zsofia Kerekes

I recommend you save some time to read and learn a lot if you have the chance, find courses and events like that of Product School or sign up to expert newsletters. From more general to more specific skill building as you progress.

Saurav Sharma

I would say being a Product Manager is a way of thinking that you need to inculcate in your life, and its not just on the job. So try to get that thinking going in which ever field you are and then try for ways to move to Product Management.

Prasad MK

I would recommend our Product Managers to be (a) customer obsessed, (b) think long term, (c) be stubborn on the vision, but be flexible on the implementation. Blend of these characteristics will enable us launch differentiated products and make them sustainable for years to come.

Ulaize Hernandez

Keep being passionate, relentless and humble. These traits will go a long way to build amazing products and have fun while doing so.