hardNewNotionPerplexityIntercom

Support tickets say the AI got worse this month, but every eval still passes. What do you do?

Try it in the practice workspaceDraft your answer before comparing it with the rubric.

Green dashboards and angry users at the same time. The strong answer treats the complaints as evidence about the eval, not noise to explain away.

Updated Nov 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.

Follow-up roundsAnswer first, then revise for each changed constraint

Round 2. The new Hindi eval cases pass at 91%, close to the English slice, but Hindi complaints stay at 25 per 1,000.
Round 3. Leadership asks for an alert that would have caught this two weeks earlier.

Green dashboards and angry users at the same time. The strong answer treats the complaints as evidence about the eval, not noise to explain away.

This reference answer is in Premium. Free signup opens other answers.

9 answers per topic instead of 5, and your progress kept · no cardor unlock all 420 remaining answers

UP NEXT ON YOUR JOURNEY

Next in this trackEval Plan for an Ambient Medical Scribe Next in this trackAn Eval Suite Too Slow to Run Next in this trackWho Owns Evals on an AI Team

NOTES & DISCUSSION · 0

No comments yet. How would you have answered?