Beyond Technical Evals: How to Evaluate AI for Real-World Success

People using an AI bot or agent don't judge it just on accuracy. They also judge the relationship it creates: do they trust it, does it understand them, and do they like its personality? Traditional AI evals don't measure those human feelings.

This session is a walkthrough. Rachel Walsh and Mike Mace from UserTesting start with a real problem, then show how they evaluate the human side of an AI experience: the signals they look for, what those signals surfaced, and what changed as a result. They work through concrete examples, with tips and tricks you can put to use right away.

Mary Gribbons, Global Head of UXR at Indeed, joins to share how her team evaluates AI in the real world and how customer feedback shapes the decisions they make.

You'll leave with a practical framework for measuring the human-AI relationship, identifying improvements, and communicating that information in a compelling way to executives and stakeholders.

What you'll walk away with:

1. A framework for reading the human side of AI. The concrete signals, trust, control, understanding, and outcome, that reveal whether people actually buy into an AI experience, not just whether the model performed correctly.
2. Tips you can apply to your own evaluation process this week. Rachel and Mike work through real examples of how UserTesting surfaces these signals, plus the moves that turned what they found into product changes.
3. A way to make the case to leadership. Language for connecting AI performance to the human-AI relationship, so the results land with executives and stakeholders instead of stopping as a research footnote.