Artificial Intelligence (AI) and Machine Learning (ML) products are unique. They hold enormous power and are by definition constantly changing.
Due to the level of sophistication involved, the development process for AI products is distinct from traditional products. In this presentation, Ria Sankar, Director of Program Management at Microsoft, introduces the best practices for developing AI products with insight, integrity, and consistency.
Making Benevolent AI: Ria Sankar
Ria Sankar is a founding member of the AI for Good Research Lab at Microsoft. Her specialties include Analytics, Product Management, Business Management, Business and Customer Intelligence, Lean Customer Development, Strategic Partnerships, Digital Marketing, Pre-sales, and Strategic Sourcing.
With 11 years of experience at Microsoft, she’s a real authority on building AI products and building them well.
Making AI Work
First, let’s take a look at the differences and similarities between a typical product lifecycle, and a typical AI lifecycle:
What are the best practices for AI?
Now let’s dive deeper into those three stages within the context of building AI products. As we go through, you’ll notice some similarities to the classic product development formula, as well as some unique considerations that will help you understand why building AI products is a distinctive challenge.
1. Find your niche: Niche is the intersection of Customer, Business and Data. Articulate AI value in terms of:
- Growth drivers
- Brand value/Industry status
- Risk reduction
- Customer delight
2. Customer understanding & jobs theory – seeing tasks from a customer vs. product context. See the tasks your product performs in their life, this is sometimes known as “Jobs To Be Done (JTBD).”
3. Business consequences: Building trust – will this feature help you gain or lose the trust of your users? To help figure this out, Ria suggests using FATE principles:
- Reliability & Safety
- Privacy & Security
4. Data Understanding: Preventing bias
- Comprehensive test cases (represent the real world)
- Data stratification
- Diverse workforce (avoid tech bro AI)
- Unconscious bias – review model outputs for correlations to race and gender
5. Metric understanding: Problem & output definition
- Data quality
- Primary vs. secondary goals
- Product vs. feature
- Standard vs. derived metrics
1. Data Preparation – good data is: correct, current, consistent and consolidated.
2. Designing with ethics – know the context in which your product will be used, and make the human users the heroes. The product should feel like it empowers people to live better, more productive lives. It shouldn’t make them feel threatened or replaced.
The AI tools and services we create must assist humanity and augment our capabilities.Harry Shum, Executive Vice President, AI and Research
3. Model Selection – make this critical decision based on consideration both of your business, and of the environment and context in which your product will be used. How is value measured? Will it be supplied online, or through a one-off batch?
- Model deployment – When deploying your model, Ria advises to be conscious of scale, check outliers and bias, visualize the output
- Ongoing learning – Your software will continually be learning and improving itself. There are many examples of this, one that Ria explores in the video is the Uber Michelangelo platform
Balancing interactions between humans and AI
Ria moves on to quickly recap CRISP-DM methodology and how to apply it.
- Scope your problem
- Build the business case for ML/AI
- Select your ML model
- Balance model performance and accuracy
- Ensure model relevance to changing business needs
- Human powered vs. machine AI
Ria concludes by suggesting that being a Product Manager, one should assess which of the AI categories, a specific customer would be suited for. She gave the examples of 2 restaurants, the fine dining restaurant at the Space Needle, Seattle, and McDonald’s.
She explains that if these are one’s customers, then it would be better to deploy the AI-first model for the Seattle restaurant, and pre-AI model for McDonald’s, as the former would have a bigger budget.