A document assistant gets 98% of fields right but mixes line items across pages. Would you launch it?
Make your call first. Use the published answer and rubric to examine your reasoning when you are ready.
Case briefRead the inputs, then make your call
A purchasing team uploads multi-page supplier quotes. The assistant prepares purchase orders, but reviewers have found values attached to the wrong line items. All figures below are illustrative.
| Evaluation sample | 100 documents, each with 20 required fields. |
|---|---|
| Field result | 1,960 of 2,000 field values match their labels. The metric does not establish correct row associations. |
| Known critical errors | 40 distinct documents each contain one incorrect critical field; the other 60 have all required field values correct. |
| Incoming volume | 1,000 documents each day. |
| Review estimate | Two minutes per draft. Exception-handling time needs verification. |
| Staffed capacity | 12 reviewer-hours each day. |
Your task. Choose a launch scope and review experience. Define the unit of success, calculate operating capacity and state what evidence would justify more autonomy.
0 words