PRACTICE WORKSPACEhardPremium answer

A competitor's agent runs on the same model as ours but completes 71% of tasks to our 48%. Where do you invest?

Make your call first. Use the published answer and rubric to examine your reasoning when you are ready.

Case briefFacts and constraints for your answer

All figures are illustrative. On your 200-task internal set, your agent passes 96 tasks (48%) and a competitor's agent passes 142 (71%). Both run the same public model at default settings.

Scenario inputs
Your failures104 failed runs, each labelled by the first thing that went wrong.
Failure labelsLost earlier context 38; declared done without verifying 30; tool errors with no retry 21; beyond the model 15.
Current cost$0.40 per task.

Your task. Decide where to invest, how you would test changes, and how you would judge whether an improvement is worth its cost.

Elapsed time00:00
Think it through
0 words