AI models don’t break like code, but they can drift, hallucinate, or mislead — which is why teams are turning to evals. The debate over whether every team needs them signals that we’re still learning how to measure quality in systems that learn on their own.
What Evals Are
For someone not familiar with evals, here’s a quick overview.
Evals are structured tests that measure how well a model performs on real-world tasks. Unlike conventional QA, which checks if software functions correctly, evals assess whether a model behaves as intended — accurate, relevant, and safe.
A support chatbot might pass QA because it sends a response, but fail an eval if that response is misleading or off-tone. QA validates functionality. Evals validate intelligence.
The Practical Loop
Most teams use a hybrid loop. They define success metrics such as factual accuracy, tone alignment, or safety. Automated scripts run large batches of prompts to score outputs. Human reviewers step in where nuance matters: clarity, reasoning, empathy. Findings are compared across model versions to detect regressions or improvements.
Tools like OpenAI Evals or Anthropic’s console help scale this process, but the principle is simple: Evals turn subjective feedback into repeatable testing.
Insights from Hamel Husain’s Workflow
Hamel Husain’s post Your AI Product Needs Evals offers one of the clearest practical frameworks I’ve found. He breaks evaluation into a workflow grounded in visibility, annotation, and iteration.
A trace is a record of everything that happened with user prompts, model responses, and tool calls. In his Rechat example, traces captured each decision step using LangSmith. The goal is transparency: understanding not just what the model answered, but how it got there.
Once you have traces, you label them. Hamel notes that annotation should remove friction — reviewers see context, pipeline versions, and relevant data. Start simple with good/bad labels, then cluster issues into categories. He says his teams spend most of their time here, often 60–80%, because this is where insights surface.
LLMs can help scale annotation, but shouldn’t replace human judgment. After a few dozen manual labels, you can use a model to suggest groupings, but every cluster still needs human review. The aim is acceleration, not automation.
Hamel describes three levels of evals:
- Unit tests: Fast, low-cost checks like format or constraint validation.
- Model and human evals: Reviewing traces for quality and reasoning.
- A/B testing: Comparing versions with real users to observe behavior changes.
Run Level 1 constantly, Level 2 regularly, Level 3 for major releases.
For multi-step or agentic systems, log every stage and analyze where failures occur. A simple failure matrix, last successful step vs. first failed step, reveals which transitions cause most errors. It’s basic but effective for debugging.
Why It Matters
I am digging deeper here, but from what I can see, this workflow makes evals operational, not theoretical. Traces show where breakdowns happen. Annotations turn those breakdowns into patterns. Layered testing turns those patterns into measurable progress. It’s how AI products move from intuition to reliability.
Over time, I expect eval dashboards to sit alongside analytics dashboards, one tracking engagement and the other trust.