A team picks the smaller model for its support assistant because the price per request is a twentieth of the alternative. Three months later the support budget is higher than before the assistant launched. Nobody made an arithmetic error. They divided by the wrong thing.
It makes a good interview trap, and it is worth working through with real prices, because the effect is larger than intuition suggests.
The prices
As of October 2026, Anthropic lists Claude Haiku 5.5 at $0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100,000 tokens, and Claude Sonnet 5.5 at $2 and $10.
Take a support task that reads 6,000 tokens (instructions, the customer's history, a few policy passages) and writes 500. On Haiku, that is $0.0006 for input and $0.00025 for output, about $0.00085 per attempt. On Sonnet it is $0.012 plus $0.005, about $0.017. The small model is twenty times cheaper per request.
If the analysis stops there, the decision makes itself.
Add what failure costs
Now suppose, for illustration, that the small model resolves 70% of these tasks correctly and the larger one 90%. These are assumptions, not benchmarks; the real numbers come from your own golden set. Suppose too that a failed task goes to a human agent, and a human-handled ticket costs $2.50 once salary and overhead are spread across volume.
The effective cost of one task is the attempt plus the expected cost of the failure:
| Model cost per attempt | Failure rate | Human cost per failure | Effective cost per task | |
|---|---|---|---|---|
| Small model | $0.00085 | 30% | $2.50 | about $0.75 |
| Mid-tier model | $0.017 | 10% | $2.50 | about $0.27 |
The model that is twenty times cheaper per request is nearly three times more expensive per task. At a million tasks a month, that is roughly $750,000 against $267,000.
The model bill is almost irrelevant here. In both rows, human fallback is more than 90% of the effective cost. That is the sentence worth saying in an interview, because it moves the conversation from price per token to the lever that actually matters: success rate on the tasks you route to the model.
Where the cheap model wins anyway
The arithmetic flips when failure is cheap. If a failed attempt is retried automatically by a stronger model instead of going to a person, the failure cost is the second model call, a few cents rather than a few dollars, and the small model can win comfortably. It also wins when the small model's success rate is close to the large one's, which is true for many narrow tasks: classifying a message, extracting a field, choosing a template.
So the useful question is never "which model is cheaper?" It is "what does a failure cost on this task, and how often does each model fail it?"
Routing: getting both
Routing can get the best of both. Suppose 60% of tickets are routine (order status, simple policy questions) and the small model resolves 95% of those, while the remaining 40% are harder and the mid-tier model resolves 85% of them. Per task:
- Routine share: $0.00085 plus 5% of $2.50, about $0.126.
- Hard share: $0.017 plus 15% of $2.50, about $0.392.
- Blended: 0.6 × $0.126 + 0.4 × $0.392, about $0.23.
Read diagram description
Effective cost per support task split into model cost and human fallback. The small model costs about 0.75 dollars per task despite being twenty times cheaper per request, because 30 percent of tasks fall to a human at 2.50 dollars. The mid-tier model costs about 0.27 dollars. Routing routine tickets to the small model and hard ones to the mid-tier model costs about 0.23 dollars. In every case the model bill is a sliver of the total.
That beats either model alone. It also depends on three things a PM should name before claiming the saving. The router must sort tickets accurately, because a hard ticket sent to the small model fails at a much higher rate than 5%. The success rates must be measured per task type on real traffic, not assumed. And the routine share has to stay routine as the product grows, which it often does not.
Published research supports the direction, with caveats. The RouteLLM work from LMSYS in 2024 reported cost reductions of over 85% on MT Bench while keeping 95% of GPT-4's quality, against a baseline of sending everything to GPT-4. Your own traffic will not look like a benchmark, so treat that as evidence that routing can work rather than as a number to promise.
What to say in the room
When an interviewer asks you to cut the cost of an AI feature, the strong opening is to ask what a failure costs and who absorbs it. Then rank the levers by how much of the effective cost they touch. Raising success on the tasks that fall back to people usually comes first. Routing comes next. Prompt caching and batch pricing help the model bill, which as the table shows may be the smallest line.
Prompt caching deserves its place on the list all the same. As of October 2026, a cache read on Sonnet 5.5 costs $0.10 per million tokens against $2 for fresh input, so a long, stable instruction block that is reused across requests becomes close to free. The Batch API takes 50% off input and output for work that can wait. Both are worth doing. Neither fixes a model that fails a third of the time.
The general rule fits in one line: divide by successful tasks, include what failures cost, and choose the model after that, not before.
Price source: Anthropic rate card, checked 22 October 2026.
Questions about this topic
The model cost of one attempt plus the expected cost of handling the attempts that fail, usually by a person. It is the number that tells you whether a cheaper model actually saves money.