Choose an operating threshold using prevalence, error costs and review capacity. A good ranking score can still produce an unusable queue.
Updated Oct 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.
All figures are fictional. You own a fraud alert system that sends transactions to trained reviewers; full automatic blocking is also being proposed.
| Daily population | 10,000 transactions: 100 confirmed frauds and 9,900 legitimate transactions. |
|---|---|
| Candidate A | ROC-AUC 0.96; selected threshold detects 80% of frauds and flags 2% of legitimate transactions. |
| Candidate B threshold | 60 true positives and 40 false positives on the same population. |
| Review capacity | 200 cases per day. |
| Illustrative error costs | $100 per missed fraud; $5 per unnecessary review. Other operational and customer costs need investigation. |
| Calibration sample | Among 200 representative cases scored at about 80% fraud probability, 100 were fraud. |
Your task. Choose an operating policy and rollout scope. Calculate the confusion matrix, precision and workload; explain which evidence would change your decision.
Choose an operating threshold using prevalence, error costs and review capacity. A good ranking score can still produce an unusable queue.
This reference answer is in Premium. Free signup opens other answers.
9 answers per topic instead of 5, and your progress kept · no cardor unlock all 420 remaining answersNo comments yet. How would you have answered?