Evals, in plain English.

“It looked right in the demo.” That's the sentence that gets an AI feature greenlit — and, weeks later, the one everyone wishes they'd checked harder. An eval is the unglamorous fix: a standing test that scores an AI feature's answers against real examples every time something changes, so “good” becomes a number you can point to — not a memory of the one run that went well.

So what actually is an eval?

Strip away the research-paper feel and an eval is just two things: a set of real examples with a known-good answer, and a way to grade new answers against them automatically. Run it once before the feature ships, then run it again every time the prompt, the model, or the underlying data changes — because any one of those three can quietly make things worse without anyone noticing until a client does.

How does an eval actually work?

What's actually real?

• •Good success criteria are specific and measurable, not just summarized as “good performance.” Anthropic and OpenAI's own guidance turns a vague goal into a number: instead of saying “we want safe outputs,” they explicitly state “we want fewer than 0.1% of outputs flagged for toxicity across 10,000 trials¹” — a target you can actually check, not just hope for.
• •Design the test set to look like the real job/use case — weird/edge cases included. The same guidance explicitly states that eval questions should mirror the real-world task distribution¹, edge cases and all — sarcasm, ambiguous inputs, the messy request that never made it into the demo.
• •Automate the grading, and favor/weight the volume of automated-graded examples over a handful of hand-graded ones. Grading can be exact match, run as code, or done by another model acting as judge — and the guidance's own preference is blunt: more questions with slightly lower signal automated grading is better than fewer questions with high-quality human hand-graded evals¹.
• •This isn't a one-lab idea. OpenAI open-sourced its own framework on the same premise — an eval is a standing artifact you keep, not a one-off checklist. Its own description: a framework for evaluating large language models (LLMs) or systems built using LLMs², built around a registry you grow over time as new cases turn up worth checking.

The “so what” — for anyone serving tech clients

• •When a client says “it looked great in the demo,” ask the follow-up kindly: “what does it score on a real test set?” That's the difference between proving a feature is feasible and proving it's ready — the demo-to-production gap we broke down in Ground Truth Nº 7³ — worth asking before it ships, not after it breaks.
• •Owning the eval is the billable, ongoing part. Building the feature is a project; maintaining the test set and re-running it every time the model or prompt changes is a service — the “we're still watching this” work that keeps an agency in the room after launch.