← Evals & Launch Decisions / 15
expertNewMicrosoftGoogleAmazon

Design the evaluation plan for an ambient AI scribe that drafts clinical notes from doctor-patient conversations.

Try it in the practice workspaceDraft your answer before comparing it with the rubric.

A principal-level question because every simple answer is dangerous. The doctor signs the note, so the tempting conclusion is that errors get caught. The strongest candidates show why that assumption fails.

Updated Nov 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.

Follow-up roundsAnswer first, then revise for each changed constraint
  1. Round 2. Shadow mode found 2 critical errors in 1,200 notes. Both were missed allergies, and both were among the 300 noisy emergency-department notes.
  2. Round 3. The reader study is in: at their normal pace, clinicians caught 9 of 20 seeded critical errors.

A principal-level question because every simple answer is dangerous. The doctor signs the note, so the tempting conclusion is that errors get caught. The strongest candidates show why that assumption fails.

This reference answer is in Premium. Free signup opens other answers.

9 answers per topic instead of 5, and your progress kept · no cardor unlock all 420 remaining answers
UP NEXT ON YOUR JOURNEY
NOTES & DISCUSSION · 0

No comments yet. How would you have answered?