Original practice question. The scoring rubric is our teaching guide. How we research and score answers
TL;DR: Cost per token measures the price of an input, not the cost of the outcome users pay for. The product number is cost per successful task: cost per call, times calls per task, adjusted for retries, divided by the success rate. A cheaper model that needs more calls or fails more often can cost more per task.
How to approach it
Define both metrics in one sentence each, then show with a small calculation how they can point in opposite directions. Finish with what you would put on the dashboard instead.
A strong answer
Token prices matter, but they are an input. Users and finance care about a finished job: a ticket resolved, a document summarised correctly, a pull request merged. Between the token price and that outcome sit three multipliers:
- How many tokens each call uses. Long system prompts, conversation history and retrieved documents usually dwarf the user's actual message: a 40-word question can travel with several thousand tokens of instructions and context.
- How many calls each task needs. An agent may plan, call tools, check its work and retry.
- How often the task succeeds. A failed attempt still costs money, and the task then needs a retry, a human, or both.
So the metric I would report is:
cost per successful task = cost per call × calls per task × (1 + retry rate) / success rate
A quick comparison shows why it matters. Say a small model costs a twentieth of a large one per token, but on a hard extraction task it needs three attempts with a checking step and succeeds 70% of the time, while the large model needs one call and succeeds 95% of the time.
| Small model | Large model | |
|---|---|---|
| Cost per call | $0.001 | $0.02 |
| Calls per task (with checks and retries) | 6 | 1.1 |
| Success rate | 70% | 95% |
| Cost per successful task | about $0.009 | about $0.023 |
Here the small model still wins on cost, but by less than three times, not twenty. Add a human who fixes each failure at $0.50 and the picture flips: the small model's effective cost is about $0.16 per task and the large model's about $0.047. The token price pointed the wrong way.
Read diagram description
Cost per successful task in the worked example. Counting model cost only, the small model costs about 0.009 dollars and the large model about 0.023 dollars, so the small model wins by less than three times rather than the twenty times its token price suggests. Adding a human who fixes each failure at 0.50 dollars, the small model costs about 0.16 dollars and the large model about 0.047 dollars, so the order flips.
Two more reasons cost per token misleads. Tokenizers differ between models, so the same text can be a different number of tokens; Anthropic's recent models, for instance, produce around 30% more tokens for the same text than earlier ones, as of November 2026. And output is priced well above input on most rate cards (five times on Anthropic's and on OpenAI's GPT-6 tiers as of November 2026), so a model that writes longer answers costs more at the same listed price.
Research on cost-aware AI points the same way. Prices across providers "can differ by two orders of magnitude", and a cascade that tries a cheap model first and escalates only when needed matched the best single model on its benchmarks at up to 98% lower cost (FrugalGPT, 2023). A 2024 study of agent benchmarks argued that leaderboards should optimise cost and accuracy jointly, because a top score bought with many expensive calls is a different product from the same score bought cheaply (Kapoor et al.). Both are measurements per task, not per token.
On the dashboard I would put cost per successful task by feature, next to success rate, so nobody can cut one without seeing the other.
What interviewers probe next
"So should we never pick the cheapest model?" Pick it when it holds success rate on that task. Routing easy requests to a small model and hard ones to a large one is often the best of both.
"Where do token counts come from in practice?" Mostly input: system prompts, history and retrieval. Trimming context is usually the first cost lever. For agents, the number of calls dominates: Anthropic reported that token usage alone explained 80% of the performance variance in one of its research-agent evaluations (company-reported, June 2025), which is why agent cost must be read next to agent quality.
How it's scored
Weak: Says cost per token is fine because it is what the provider bills.
Hire: Defines cost per successful task and explains calls, retries and success rate.
Strong Hire: Works a comparison where the cheaper-per-token model is more expensive per task once failures and human fixes are counted, and mentions tokenizer and output-price effects.
Common mistakes
Comparing models on rate cards alone.
Forgetting that failed attempts cost money.
Leaving human correction out of the task cost.