Measuring Product Success with Pinterest Head of Growth

This week, Product School hosted Seth Weisfeld, Head of Growth Product at Pinterest for a special #AskMeAnything session. He answered questions on how to properly measure success, and the secrets behind great Growth Product Managers.

Seth Weisfeld Head of Growth Product Pinterest

Meet Seth

Seth has 15+ years of experience leading technical, product, and marketing strategies to drive consumer acquisition and engagement. Currently, he is the Head of Growth Product at Pinterest where he is responsible for leading a team of PM’s focused on all aspects of User Growth at Pinterest – SEO, authentication, activation, performance, marketing, emails and notifications, audience growth, and tooling.

Prior to this, Seth was a Product Manager of CRM and User Acquisition at Brigade, responsible for the consumer experience and in-house developed notifications systems. Moreover, Seth was the Co-Founder of AdRevolution, a pioneer in personalized online advertising driven by the powerful algorithms of its Natural Intelligence Marketing Engine. Seth managed teams across advertising operations, account management, sales, and product management.

How do you go about setting strategy and aligning teams to take action, as well as monitoring adjusting? Use OKRs, quarterly meetings, etc..?

There are a variety of tactics we employ but the main process is a half yearly company wide OKR planning. We set company OKRs, then org level OKRs, team level, etc. We strive for all team level OKRs to have clear measurable outcomes.

On growth that outcome is often an increase in usage or engagement but it can also be a number like X number of experiments shipped or Y number of new topics launched if the work doesn’t directly align to metric outcomes. We then have a monthly company wide checkin cycle where all teams update their OKR status. All teams have their own check-in meetings. Once a quarter leadership checks in at the org level.

Can you describe how being a “growth” product manager is different from other kinds of product management?

This is a vibrant debate and there are many perspectives but IMHO it comes down to how a team’s roadmap is set. Growth product is driven by a bottoms up ideation where each team member is expected to create ideas and determine the best work they can do this sprint, that work should be within a set of a strategic guardrails and a long term roadmap to bound what is done.

While a traditional product team the long term roadmap is broken down into individual sprints that are then executed against in a set order. Traditional is less chaotic but you don’t get the big wins from just trying some crazy ideas.

green leafed seedlings on black plastic pots growth

How do you measure ROI/success of products where benefits are largely intangibles?

Ah – this is always a challenging issue. One we deal with every week at Pinterest. One thing we do is to simply allocate a set amount (say 20%) of engineering time to this sort of work. The intagible is much like the Quality that Pirsig talkings about ZAMM – you know when you see it, you can feel it. You have to inspire people that that the feeling is real, it’s something to care about, users will care about…even if we can’t measure it right now. So key in on the agreement that Quality matters and agree to a set time that quality can be worked on.

The other thing I’ve found here is that the measurement is often just a timing issue, many things that used to be immeasurable at Pinterest we can now measure… either because technology has gotten better or our sample sizes has increased.

You might also be interested in: These Are The Metrics Great Product Managers Track

What have you found to be the best way to align the various ongoing growth efforts/departments in order to have a holistic view of the entire growth engine?

The best thing is if you can establish a clear northstar metric. One that everyone can agree. At Pinterest we use a metric called 12 week WAU where we extrapolate the impact of an experiment over the 12 weeks after it is launched.

This helps to normalize the impact of a say a notifications experiment that may have a huge week 1 impact but little ongoing impact while a new user experiment will have impact every week. Ultimately you get to that northstar metric by stepping back and talking about everyone’s goals and finding what is tangibly driving each team.

By designing a metric that meets each teams needs it lifts a huge burden on decision making. You stop fighting over territory and start focusing on what drives that one most important metric.

Do you use AI / Machine Learning in your growth strategy?

Absolutely, ML is critical to all components of our growth strategies. This is partially unique to Pinterest and partially the state of the industry. At Pinterest our product excels as a source of personalized inspiration (driven by ML) and our goal on the Growth team is to extend that product value into the surfaces we own.

So emails & notifications are primarily ML recommendations, SEO is largely driven by ML connecting & highlighting the most interesting ideas (and of course the Google ML is the intermediary between us and the user anyways), performance marketing relies on ML for bidding, targeting, and now even creative development, our onboarding is personalized based on what you were doing before you signed up and the interests you state to us. We would be far less successful without ML.

You might also be interested in: AI and ML with Microsoft PM

photo of girl laying left hand on white digital robot

Can you share your vision and strategies for growth this year? Anything different that you want the team to pivot from? Anything you want to keep doing that’s working?

This year is a unique year for all growth teams. COVID introduced massive external variables that dramatically changed user behavior. I expect post-COVID to create the same dramatic changes but in different ways. Instead of everyone being locked in to their homes we’ll all want to be out in public, doing things, away from our phones and screens.

All growth teams will need to figure out how to adapt to that changing behavior, particularly without just sending a ton more emails or plastering advertising everywhere.

My team and I are thinking about how will emphasize Pinterest’s role as a tool during the next phase of life we’ll be undertaking. How do we help all the people who are deep scrolling Pinterest because they are bored today form a habit to keep Pins in their life once they are on the go. We have some fun ideas but I can’t share them!

Company wide we’ll be launching a new story pins format that will emphasize creators and a sense of community. I think small community is going to be crucial in our new world so I’m really excited about that core product feature

How important do you see outside – in learning for the development of your product team? 

I think it can be really important. We encourage all of our teams to talk to teams facing similar issues. When I joined Pinterest to lead the emails & notifs teams we got together a symposium of folks from Linkedin, eBay, Facebook, Twitter, etc to talk about how we were all approaching the email/notifs problem set.

The learnings were invaluable for everyone. I highly recommend that PMs find their peer set and learn from people doing similar jobs outside their company. It can either be as I described above or just attending/participating in communities / trade organization but I either way I think it’s really important.

Check out: So You Want to Work in Silicon Valley?

What type of questions are asked during the PM interview / What are the best ways to prep for a PM interview/role at Pinterest?

macbook pro displaying  pinterestpage

Every company is different in how they interview PMs but I think the most important rule is to truly be yourself. Bring your honest answers. I remember a decade plus ago I had a horrible interview at Facebook for a PM role, worst interview I’ve ever been a part of…and I was one the one being interviewed.

The biggest problem is I wasn’t being my honest self. I wasn’t admitting where I needed help or what I did (or didn’t know). Broadly you can expect that the interview process will encompass a role fit interview, a product sense, a working with eng, working with design, and analytical skills interview.

What qualities and practices do you suggest younger PMs look for in their managers/leadership?

Do they share what their job is and how they do it. There are a lot of important coaching & mentoring skills but if you want to advance your career, you need to know how the next level is getting things done.

What do you think is the difference between a Good Product Manager and a Great Product Manager?

A great PM can see where the human beings are in the equation and builds for them. Sometimes this is easy if you working on a direct consumer facing feature with no stakeholders. With more complicated problems you have to be able to identify every part that relies on a human and make sure they are happy and getting what they need.

For example in SEO a lot of people get caught up in thinking about what the GoogleBot is doing. In reality the GoogleBot is just doing whatever makes humans happy. See the human on the other side and build what will make them happy not just the GoogleBot.

Can you please what are some of the metrics to look out when you’re launching your SaaS product to public?

Well I’ve never launched a saas product per se but most online business have the same metrics. Total Addressable Market, Signups Per Day, (1-3) month activation rate, long term retention rate. If people are paying you then add in the maths around paid account conversion rate, churn rate, etc.

You might also be interested in: Basic Metrics for Each Organization in your SaaS Company

If you had to do a pie chart of what your mind is usually focused on as a Head of Growth Product (even though I realise each day/week is different), what would it look like?

Fun question. 25% – how is my team doing, 15% – how are the metrics, 10% – what does my direct leadership team need to know, 5% – what does corporate leadership team need to know, 10% – what can we do to make decisions faster, 10% – how are my cross functional partners doing, 10% – what experiments are teams running, 14% – what is the team’s long term vision & are we sharing properly across the Growth org, and 1% – OMG why can’t I ever clear out my inbox!!!!

Is there unique training you’ve acquired in Big Tech that you think would likely be stronger than going straight to serving in product manager roles at startups?

I started in small startups and for me and my style I wouldn’t change it. I had to do ALLL the jobs and work crazy hours but I learned a ton. There are plenty of other paths like getting an MBA, doing consulting for a few years, and then hitting up big tech. I think it’s just about knowing yourself and what will help you learn best.

Looking for a path to product? Check out our certifications!

How did data science / analytics apply to growth product management and does that application differ from the traditional PM role?

Pinterest data analytics

In today’s world it is critical that growth product managers have the ability to query data and answer basic questions and understand day to day trends with their product, they should also be able to work effectively with a full time data scientist / analyst to understand larger trends with their product/user base.

I’m have been a PM for 2 years at young startups and now aspiring for Growth roles – what are the key skills one needs to learn to navigate path to a Growth PM role?

Bringing inventiveness & curiosity to the role along with the skills to self serve those skills. You should be able to come up with novel ideas to solve user growth problems and have the skills to query data, work with user research, understand the market, etc to satisfy your curiosity of what the user problems are and how to solve them.

Join us next week for another #AskMeAnything Session for more insights from Product Managers around the world!

The Product economy

Enjoyed the article? You may like this too: