Choose data that can change a measured failure. Price accepted examples, reserve evaluation capacity and explain why a cheaper label may teach the wrong behavior.
Updated Oct 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.
You own a task assistant that updates customer records in a sandbox. A review of 200 failed tasks assigns each failure to one primary cause. The prices and acceptance rates below are fictional planning inputs.
| Available budget | $30,000 in total |
|---|---|
| Failure review | 90 incorrect action sequences; 60 false completion claims; 30 unsuitable responses despite correct task results; 20 tool-contract or authorization defects. |
| Demonstration quote | $20 per collected example; 80% passed the pilot acceptance review. |
| Preference quote | $3 per collected comparison; 90% passed the pilot acceptance review. |
| Execution-trace quote | $5 per collected trace; 60% passed the pilot acceptance review. |
| Budget scope | Collection, acceptance review, model adaptation and independent evaluation must all fit. Other costs have not yet been quoted. |
Your task. Recommend the first investment, calculate usable-data costs and explain how you will decide whether to spend the remaining budget. State any additional assumptions.
Choose data that can change a measured failure. Price accepted examples, reserve evaluation capacity and explain why a cheaper label may teach the wrong behavior.
This reference answer is in Premium. Free signup opens other answers.
9 answers per topic instead of 5, and your progress kept · no cardor unlock all 420 remaining answersNo comments yet. How would you have answered?