TL;DR: C_task = C_call × calls per task × (1 + retry rate) ÷ success rate. Token prices feed the first term; product design decides the other three. A model with a lower token price can cost more per successful task, and a prompt change that raises success rate can save more than any price negotiation.
Why price per token is the wrong headline
Model providers bill per token, so cost reports tend to start there. But nobody buys tokens. Users want a ticket resolved, a contract reviewed, a meeting summarised correctly. Between the token price and that outcome sit several multipliers, and in most AI products those multipliers vary more than the token price does.
Consider two ways to answer the same question. One uses a large model in a single call with a short prompt. The other uses a small model, retrieves ten documents into a long prompt, calls a tool, checks its own answer and retries a third of the time. The second can cost more per finished answer even if its model is a twentieth of the price per token.
The formula
Cost per call. Input tokens times input price plus output tokens times output price, plus any per-call tool or retrieval cost.
C_call = tokens_in × price_in + tokens_out × price_out + tool_cost
Input tokens are usually much larger than the user's message, because each call carries system instructions, conversation history and retrieved context. Output tokens are fewer but priced higher: five times input on Anthropic's rate card as of October 2026.
Calls per task. How many model calls one attempt takes. A simple Q&A feature makes one. An agent may plan, call several tools, reflect and finalise, making eight or more.
Retries. Attempts that fail a check and rerun. A 20% retry rate multiplies cost by 1.2.
Success rate. Share of attempts that produce an acceptable result. Every failed attempt is paid for, so dividing by success rate spreads the cost of failures across the successes.
C_task = C_call × calls × (1 + retry) ÷ success
A worked example
A document-extraction feature pulls fields from invoices.
| Input | Value |
|---|---|
| Input tokens per call | 3,500 (instructions, schema, the invoice text) |
| Output tokens per call | 250 |
| Price, Claude Sonnet 5.5, October 2026 | $2 input, $10 output per million |
| Calls per attempt | 2 (extract, then validate) |
| Retry rate | 15% |
| Success rate (all fields correct) | 88% |
- C_call = 3,500 × $2/M + 250 × $10/M = $0.007 + $0.0025 = $0.0095
- Per attempt = $0.0095 × 2 × 1.15 ≈ $0.0219
- C_task = $0.0219 ÷ 0.88 ≈ $0.0248
At 400,000 invoices a month, about $9,900. Now compare the levers:
- Cut input tokens by a third (shorter instructions, schema cached): C_task falls to about $0.019. Saves roughly $2,400 a month.
- Raise success rate from 88% to 95% with better instructions: C_task falls to about $0.023. Saves about $700 a month in inference, and far more in manual corrections.
The second lever looks small in inference terms. Once the cost of a person fixing each failed invoice is added, it is the bigger one. That is the subject of effective cost with human fallback.
Where the number misleads
Success needs a definition. "Success" might be all fields correct, the user accepting the output, or no complaint within a week. Each gives a different number. Use the definition closest to the outcome the business cares about, and keep it fixed when comparing options.
Averages hide the tail. Agent tasks often have a long tail: most finish in four calls, some take forty. Report the distribution, or at least the 95th percentile, alongside the mean. The tail is where loops and runaway costs live.
It omits non-inference costs. Retrieval infrastructure, vector storage, logging, evaluation runs and human review all belong in the full unit cost. Inference is often the smaller part for products with human fallback.
Tokenizers change. Moving to a new model can change token counts for the same text. Anthropic's newer models, for example, produce about 30% more tokens for the same text than earlier generations, as of October 2026. Measure cost per task on real traffic after a model change, not from rate cards.
How teams use it
The most useful habit is putting cost per successful task on the same dashboard as success rate, per feature. That makes trade-offs visible. A cheaper model that drops success rate shows up as a smaller saving than the rate card promised, or no saving at all. A prompt that adds 500 tokens but lifts success rate by ten points may lower the cost per task.
It also changes model-selection conversations. Instead of "model B is 40% cheaper," the comparison becomes "model B costs $0.018 per successful extraction against $0.025, at a success rate two points lower." That is a decision someone can make.
In the interview
Any estimation or unit-economics question ("what will this cost to run?", "should we switch to a cheaper model?") rewards building cost per successful task out loud. Strong candidates layer it: per call, per attempt, per success, per month, and then name which term they would attack first and why. Candidates who stop at a monthly token bill usually score Hire at best; those who divide by success rate and then add human fallback cost score Strong Hire. The AI cost calculator runs this formula with current prices.
Next: effective cost with human fallback · inference cost levers
Price source: Anthropic rate card, checked 22 October 2026.