From Prototype to Production: The Evals That Get You There

Most AI features get stuck in prototype. They work on curated examples, they fall apart in production, and no one has a reliable way to know if the next release made things better or worse. In this hands-on session, Shruti Anand (AI Product Lead at Google) walks through the eval infrastructure that separates AI features that survive production from the ones that don't. She'll build a real eval setup live, covering the three outcome dimensions every AI Product Leader eventually confronts: cost, quality, and security. You'll leave with a template for evaluating your own AI features, plus a clearer picture of how evals evolve as your product moves from first prototype to live production traffic.