TL;DR: Keep many inexpensive checks around the model, then evaluate behavior and complete tasks on chosen datasets. Repeated runs reveal instability; fresh cases reveal blind spots. A high average pass rate cannot compensate for a critical authorization or safety failure.
The layers answer different questions
A test pyramid is a planning aid, not a prescribed ratio. “Most checks should be cheap” does not mean “run fewer safety checks.” The right mix follows the failures your product can cause.
| Layer | Example for an unread-message summary | Evidence and release consequence |
|---|---|---|
| Deterministic contracts | Refuse a message outside the viewer's permissions | Any reproduced violation blocks that path |
| Integration and recovery | Source API times out after returning one of three pages | Partial result stays visibly incomplete; no silent claim of coverage |
| Model behavior | Preserve a deadline and its owner | Grade against labelled facts; report counts and failure types |
| End-to-end task | Reader finds an urgent request without opening every thread | Compare the whole experience with the existing inbox |
| Production monitoring | Edits, missed actions and source-version changes | Investigate sampled cases; roll back affected behavior when needed |
A code assertion can grade a model-generated field. A human can grade an end-to-end software test. The useful distinction is the question being checked and how repeatability is established, not “tests are code, evals are humans.”
A release sequence you can inspect
Follow the flow, one stage at a time.
Illustrated sequence; animation speed does not represent processing time.Read diagram description
Define task and severity, test deterministic contracts, test integration failures, curate task samples, repeat behavior trials, check complete outcomes, enforce release gates and monitor limited production exposure. New failures feed the next test cycle.
Separate representative samples from diagnostic cases such as known attacks or rare failures. Both belong in the plan, but deliberately oversampling attacks does not estimate their production frequency. Anthropic's agent evaluation guidance, reviewed in October 2026, distinguishes tasks, repeated trials, graders and the execution environment. Use those distinctions to make a failure reproducible without treating the model's prose as the ground truth of what happened.
Evaluate non-determinism without inflating the sample
Suppose a fictional summary evaluation has 50 distinct threads and three trials per thread. The run produces 150 outputs, of which 138 pass. The output pass rate is 92%, but those 150 outputs are not 150 independently sampled user situations.
Keep the thread ID attached to every trial. Report how many threads passed all three attempts, which ones fluctuate and which fail consistently. If 40 threads pass three times, eight pass twice and two pass once, the counts total 138 passes. The all-three-pass share is 40/50 = 80%. These measures answer different questions; neither establishes behavior on unseen thread types.
Do not compute an ordinary independent-binomial confidence interval over all 150 outputs while ignoring the shared threads. Use a task-level analysis appropriate to the study design, and sample new tasks to test generalization. Fix model and prompt versions, tool fixtures and graders when comparing a candidate. Retain paired results when both systems see the same tasks.
Severity can veto the average
If one of the failures exposes a private message, stop that path and repair the permission boundary. If it omits a low-priority greeting, the consequence is different. Define those categories before looking at the release score.
Zero critical failures in a small set is useful evidence but cannot prove zero production risk. Add deterministic enforcement where possible, adversarial cases, exposure limits and a rollback owner. Do not average a tenant-isolation violation together with a harmless wording preference.
Human feedback is not automatically a label
A thumbs-down might mean a wrong fact, an unhelpful format or an accurate answer the user disliked. Ask for an optional reason, attach the permitted trace context and route high-consequence cases to review. Respect consent, data minimization and retention rules when capturing examples.
Reviewers need criteria, examples and a way to disagree. Only accepted labels enter the regression set or training pool. Keep a separate fresh evaluation set so a product does not improve only on the complaints the team has already memorized.
Start with the no-evals case. Then rehearse a model migration and a working evaluation in the Foundations course.