Updated: April 12, 2023 - 4 min read
In this talk, Ruben Lozano illustrates what Machine Learning is, how it differs from Artificial Intelligence and how it is related to statistics. He also explains that data is the foundation on which the machine learning algorithms work, and predict future outcomes.
Further, he narrates that it is not advisable or practical to use ML in every scenario, and also on how to collaborate with data scientists to evaluate the training model, and use certain methods to clean and make sense of the data.
Ruben Lozano is a Product M at Google Cloud, overseeing customer experience for all the online experiences before customers begin using the cloud. He also worked at Amazon for three years before joining Google, managing a team of senior product managers.
Machine Learning for Product Managers
What is Machine Learning (ML)?
Unlike in classical programming, in Machine Learning, you don’t have rules beforehand, but you do have a lot of data to make rules.
ML and Statistics
To ML or Not to ML, that is the question!
Implementing ML begins with data. Once enough relevant data is obtained, it is used for predictions, and these predictions are in turn, used for better customer experience, which has the potential to bring more traffic. This traffic again leads to more data. Hence, a cycle is formed.
ML can be implemented when your problem…
Handles very complex logic
Adapts in real-time
Requires specialized personalization
Has existing examples of actual answers
Sample ML Problems
Ranking: Helping users find the most relevant thing
Example: Ranking algorithm within Amazon search
Recommendation: Giving users the thing they may be most interested in
Example: Recommendations from Netflix
Classification: Figuring out what kind of thing something is
Example: Product classification for Amazon catalog
Predicting a numerical value of a thing
Example: Predicting sales for specific Amazon products
Clustering: Putting similar things together
Example: Related news from Google search
Anomaly: Finding uncommon things
Example: Fruit freshness
Ruben then shows us a chart depicting that ML can be implemented when your data…
ML should not be implemented when your problem…
Can be solved by simple rules
Does not adapt to new data
Requires full interpretability
Requires 100% accuracy
And ML should not be implemented when your data…
What do you need for ML?
Tools and Systems
Formulate the problem
Great ML problems cannot be productionized due to high implementation costs or inability to be tested in practice.
Preprocessing the data
How do you evaluate the model?
How to best partner with scientists?
Treat your ML project as a partnership:
“A PM from an ML project I worked on basically threw the requirements over the fence to me and was mostly unavailable. To meet timelines, I kept moving forward. Unfortunately, the deliverable at the end of the three-month project, though aligned with initial business requirements, was not what the PM wanted and didn’t meet the need. The model never made it into production and we really didn’t gain any learnings.”
Have a clear problem, hypothesis, and success metric
“PMs who come prepared with a clear, preferably data-driven, problem and hypothesis will have a much more productive discussion with me than otherwise. The problem definition need not be perfect, but I do want to understand what’s been tried, why it isn’t working and what we’re aiming for.”
Be willing to make tradeoffs
Time vs Quality
White Box vs Black Box
False Positives vs False Negatives
Go vs No-Go Metrics
Help get data and explain it
Scientists are not Software Engineers
ML creates tech debt
Be considerate of scientist time and momentum
Ruben concludes by saying that there’s always room for experimentation, especially in big companies. Only with more experimentation can the Product Managers go deeper and make the most use of machine learning.
Updated: April 12, 2023