Premium membership
Continue this concept with Premium
Premium includes all published courses, graded answers, concepts and PDF guides. See the price and access terms on the pricing page.
View Premium accessone payment · six months of site access
Advanced
Trajectory Evals: Grading AI Agents on the Path as Well as the Outcome
Agents take many steps, call tools and change real systems, so grading only the final answer misses the most expensive failures. Trajectory evals check tool-argument validity, authorisation, step order, policy compliance, wasted actions, cost and time, alongside outcome checks run against a sandboxed end state.
CONCEPT / EVALS
RELATED CONCEPTS
PRACTISE THIS CONCEPT
Evals & Launch DecisionsHow would you evaluate an agent that books business travel for employees?→Live Prototyping & AI-Native WorkflowIn a one-hour prototyping round you're asked to build 'an agent that schedules meetings for a team'. How do you approach it?→Agentic ProductsDesign a research agent that produces a cited market brief for analysts in about ten minutes.→AI Product SenseHow would you design an AI-first spreadsheet?→Live Prototyping & AI-Native WorkflowIn a live prototyping round, you get an AI coding tool and 40 minutes to build something. How do you approach it?→Evals & Launch DecisionsYour support agent completes 82% of tasks in testing. Is it ready to launch?→