Design task evidence, privacy controls and a separate diagnostic path without pretending that minimal telemetry proves semantic correctness.
Updated Oct 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.
A fictional voice assistant creates timers and private reminders. The product promises not to collect raw conversations in routine telemetry.
| Opted-in cohort | 1,000 eligible users; 5,000 task attempts. |
|---|---|
| Action records | 4,300 confirmed executions; 200 linked and deduplicated immediate reversals. |
| Unknowns | Intent correctness, non-participant outcomes and semantic failure details are not fully observable. |
Your task. Design useful measurement and a separate diagnostic path while preserving the collection limit.
Design task evidence, privacy controls and a separate diagnostic path without pretending that minimal telemetry proves semantic correctness.
This reference answer is in Premium. Free signup opens other answers.
9 answers per topic instead of 5, and your progress kept · no cardor unlock all 420 remaining answersNo comments yet. How would you have answered?