What if I told you there is a real-life courtroom where Snoop Dogg, or Lion, or whatever animal he goes by these days, sits on the bench as an AI judge? It is called Courtroom Chaos, a real-time AI party game where players argue absurd cases in front of the Lion Judge, and it holds some of the sharpest lessons in AI product design.
The game comes from the team of Penny Szeto, Head of Gen AI at Amazon Games, who presented those lessons live at ProductCon New York. Her core argument is that customer-facing AI fails not because of the model, but because of the product system designed around it.
The four decisions that determine whether a consumer AI experience feels magical or broken are: identity (who the AI is), context (what it knows), judgment (how it draws boundaries), and interaction (how it responds in real time).
Szeto's opening line on stage is the whole problem in two sentences. "When customer-facing AI works, it feels magical. It feels personal, and it feels alive. But when it doesn't work, it's really disappointing, and it breaks trust."
This piece is for the people responsible for that trust: AI builders, product designers, and product leadership shipping consumer AI experiences, from games and companions to coaches and support agents.
Key Takeaways
Customer-facing AI is judged on coherence, responsiveness, and trust, not just accuracy or time saved.
The four levers of consumer AI experience design are identity, context, judgment, and interaction.
The ELIZA effect, first documented in 1966, means customers bond with your AI whether you design for that relationship or not.
The model is becoming table stakes. The product system around it is where differentiation lives.
AI behavior design, not feature optimization, is the new job of the consumer AI product manager.
What Is Customer-Facing AI?
Customer-facing AI is any AI experience your customers interact with or feel directly, from recommendations and support agents to voice companions and AI-native products. It sits in front of real people in real time, which means every response shapes how they perceive your product and your brand.
The opposite category is internal AI: coding assistants, agentic architecture, and AI tooling that help employees move faster. Both run on similar models, but the stakes are completely different. Internal AI failures waste time while customer-facing failures happen in front of the people who pay you.
The category has also changed shape. For years, customer-facing AI meant invisible systems ranking and personalizing in the background. Today, customers increasingly talk to the AI itself, in natural language, in real time, across apps, devices, support lines, and games.
Why Customer-Facing AI Is Different From Enterprise AI
Consumer AI experience design differs from enterprise and productivity AI because customers judge the whole behavior, not just the output. As AI moves from invisible infrastructure to being the experience itself, the bar shifts from "does it work" to "does it feel coherent and trustworthy."
Szeto maps it as a spectrum:
Where AI sits | Examples | What customers judge |
Invisible AI | Spotify playlists, Prime Video rows, AI summaries | Whether the output works |
Guiding AI | Copilots, support agents, shopping assistants | Whether it completes the task correctly |
AI as the experience | Companions, coaches, voice agents, AI-native games like Courtroom Chaos | Whether it feels coherent, responsive, and trustworthy |
Most production AI sits in the middle today, but the right side, where AI agents meet consumers directly, is growing fastest. In Szeto's words: "Customers are judging whether it feels coherent, if it's responsive, if it feels trustworthy."
The stakes are measurable. Qualtrics' 2026 Consumer Experience Trends Report, covering more than 20,000 consumers in 14 countries, found one in five people who used AI for support got no benefit at all, and only 29% trust organizations to use AI responsibly.
The ELIZA Effect: Why Customers Bond With AI Either Way
The ELIZA effect is the human tendency to give AI credit for intelligence, empathy, and intent, even though we know it's not real. It was first documented in 1966, when MIT computer scientist Joseph Weizenbaum built ELIZA, a simple pattern-matching program that reflected people's statements as questions.
Primitive as it was, people told it their deepest secrets, and Weizenbaum's own secretary asked him to leave the room so she could talk to ELIZA in private. Sixty years later, the instinct is unchanged, but the exposure is not: half of the world, basically, is now using AI.
The product lesson is that this bond forms whether or not you design it. So design it.
"The goal as product managers isn't to try to make AI feel human," Szeto says. "It's to design behavior intentionally, so the experience feels engaging, trustworthy, and appropriate for the moment."
The 4 Levers of Customer-Facing AI Product Design
Most AI product work concentrates on the model: which one to choose, how strong the reasoning is, whether to fine-tune. Szeto argues those decisions no longer differentiate.
"These decisions still matter, but increasingly they're becoming table stakes. What really determines the product experience is the product system that you design around the model."
That product system has four levers.
Lever | The question it answers | What failure looks like |
Identity | Who is the AI, and how does it show up? | A flat default chatbot, or drift into sycophancy |
Context | What does it know, remember, and have access to? | A judge forgetting case details; recommending a closed restaurant |
Judgment | How does it reason and draw limits? | Microsoft Sydney; a trade that blows a client's risk profile |
Interaction | How does it respond to humans in real time? | Jokes with no room to land; instant replies that read as dismissive |
1. Identity lever: Who is your AI?
Identity defines what role the AI plays and how it shows up, from voice and humor to visual presence. It is a character spec, and it needs to be written deliberately, especially around celebrity or licensed IP.
Snoop Dogg made the call himself: "I really want to lean into my Olympics, Martha Stewart's best friend, Uncle Snoop persona rather than the drop it like it's hot persona." Skip this lever, and you get flat, default AI that sounds like every other chatbot, or identity drift. Just think about ChatGPT's infamous sycophancy phase, where the AI rewards whatever behavior customers feed back to it, and try not to build that.
2. Context lever: What does your AI know?
Context is what the AI knows, remembers, and has access to, mixing customer data with institutional knowledge. This is where memory, personalization, and RAG live. RAG, or retrieval-augmented generation, lets a model pull in external data at answer time.
Early versions of Courtroom Chaos would forget case details mid-verdict, and playtesters were right to be annoyed: "I should have won that argument." Szeto's non-game example is just as sharp: a hotel AI recommending a restaurant that turns out to be closed for a holiday has failed on context, not intelligence.
3. Judgment lever: How does your AI draw boundaries?
Judgment is how the system reasons, prioritizes, and draws its limits, which is where values, guardrails, and policy live. Teams enforce it through red teaming, meaning deliberately attacking your own product to find failures before customers do.
Courtroom Chaos shipped as a teen-rated game with a long blocklist, and the celebrity added his own. "Even Snoop had this big list of things he wanted to block. Funnily, one of them was don't have too many marijuana references."
The stakes scale with the domain. A trading assistant should refuse a move that blows up a client's risk profile, which is where human-in-the-loop escalation earns its keep. The cautionary tale is Microsoft Sydney, the 2023 chatbot that became infamous for gaslighting and threatening users who challenged it.
4. Interaction lever: How does your AI respond in real time?
Interaction is how the AI responds to humans moment to moment: tone calibration, pacing, latency, and emotional response. The goal is not to feel human but to work well with humans.
Szeto's team hit a latency paradox: they made Snoop respond so fast that "we weren't allowing enough time for jokes to land." The same physics applies in serious products, because in a mental health app, "an immediate response can actually come off as dismissive, like you're not actually listening."
Her live demo made it personal, putting her on trial for stealing from the company fridge before an AI judge modeled on her dog Yogi. When she stayed defiant, Yogi matched her aggression and called her "you absolute prawn." When she expressed remorse, he slowed down, softened, and called her "my boy."
As Szeto put it: "Same model, but depending on the product decisions you make, you can change the interaction."
How to Apply Customer-Facing AI to Your Product
Applying customer-facing AI well comes down to four steps.
Step | Ask your team | You are ready when |
1. Define the role | Where does this sit on the spectrum, and what is its one job? | A signed behavioral spec exists, including what the AI never says or does |
2. Engineer context | What must it know, remember, and forget, and how fresh is each source? | Every factual answer is grounded in a source, and memory rules are documented and visible to customers |
3. Build judgment | What is our worst realistic day, and does the system survive it? | Red-team scenarios pass, evals run on every change, and human escalation works end to end |
4. Prototype and pilot | Have real customers used a working version yet? | Latency budgets, tone rules, and fallbacks are tested, and trust metrics are live before full launch |
Here is how I would run each step with a product team, and what I would ask about each one as a product leader.
Step 1: Define the role and write the behavioral spec
Before anyone opens a model playground, decide where on the spectrum your AI sits and what single job it does. An AI that guides a task and an AI that is the experience carry different risks, different latency budgets, and different success metrics. This choice shapes everything downstream.
Then write a behavioral spec: a short document defining the AI's role, voice, values, and, most importantly, its negative space, meaning the things it never says and never does. Treat it like a PRD for behavior, and get brand, legal, and any IP owners to sign it before a single prompt is written.
Pay attention to specificity, because "friendly and helpful" is not a spec. The thing to avoid is the everything-assistant, since an AI with ten jobs has no identity, no clear boundaries, and no way to be evaluated.
Step 2: Engineer the context layer
Context is an architecture decision. Identify what the AI genuinely needs to know for its one job, from customer history and policies to operational data like opening hours. Then, define a freshness requirement for each source.
Design memory deliberately. Decide what persists across sessions, what expires, and what the customer can see and delete. This is where personalization and privacy collide. Ground factual answers in retrieval rather than the model's general knowledge, and build an explicit "I don't know, but here is what I can do" path.
Pay attention to the creepiness threshold. Personalization should feel like good service, so lean on what customers knowingly shared. Avoid unbounded context, because feeding the model everything degrades quality, raises cost, and multiplies the ways a stale or sensitive detail leaks into a response.
Step 3: Build judgment into the system, then prove it with evals
Guardrails that live only in the prompt are suggestions. Real judgment is layered: policy checks before and after the model, hard blocks for non-negotiables, and escalation paths that hand risky moments to a human.
Start from your domain's worst realistic day. List the scenarios that would end up in the press or in court, write the required behavior for each one, and red flag against that list continuously rather than once before launch.
Then prove the behavior holds with evals: golden datasets, adversarial prompts, and regression suites that run every time you change a prompt, a model, or a data source. This discipline runs deep enough that we built the AI Evals certification around it, because eval design is fast becoming a core PM skill to master in 2026.
AI Evals Certification
Learn to build trusted AI products. Design eval suites, integrate CI/CD gates, monitor drift and bias, and lead responsible AI adoption.
Enroll now
The failure mode to avoid here is over-refusal. A product that blocks everything is safe the way a parked car is safe, and customers will treat it accordingly.
Step 4: Prototype the interaction, pilot small, and instrument trust
Interaction quality only reveals itself in use, so get a working version in front of real customers fast. Yes, even an ugly one.
If engineering capacity is the blocker, this is exactly the gap vibe coding closes, and our Vibe Coding certification teaches product people to build these test versions themselves.
Set latency budgets per moment rather than one global target, because a joke, a verdict, and a refund confirmation each have a different “right speed”. Write tone rules for the hard moments: when to match the customer's energy, when to stabilize it, and what the fallback says when the model fails mid-conversation.
Instrument trust. Track resolution, repeat usage, escalation rates, and mid-conversation abandonment, and read real transcripts weekly, because dashboards hide the exact failures that break trust. Avoid the silent full launch. Rather, pilot with a small cohort, publish what the AI can and cannot do, and expand only when the transcripts stop surprising you.
The PM Implication: AI Behavior Design Is the New Job
AI behavior design means shaping how an AI product behaves across identity, context, judgment, and interaction. It is fast becoming the defining skill of the consumer-facing AI product manager, because the model is shared and the behavior is not.
Szeto is clear this is systems work, not clever prompting. "It's not simple prompt engineering. It's orchestration, evals, fallback design, and a ton of iteration." Evals, short for evaluations, are the structured tests that measure whether AI behavior meets your bar, and we covered how PMs run them in our guide to AI evals.
Her closing framing is the one I would put on the wall. "Models are chaotic because of their outputs. Humans introduce chaos because of their inputs. Our job is to design the systems that turn that chaos into interactions that customers can trust, that they enjoy, and that they keep coming back to."
If that is where your career is heading, Product School's AI Product Management certification teaches exactly this: working with models, designing guardrails and evals, and building AI products customers trust.
Become an AI-Native Product Manager
Adopt an AI-first mindset: design AI-native UX, build agents, define modern PRDs, and ship trustworthy AI-powered products.
Enroll now
Updated: October 15, 2026
