model routing

The token bill drops by two-thirds and failures rise by one point. Price the failures before you call it a saving, and check what routing does to your cache.

Candidates jump to 'switch to a cheaper model'. The interviewer wants to see you find out why cost per user rose before choosing a lever, then pull the levers in the order that protects quality.

Routing is one of the biggest cost levers in AI products, and one of the easiest to get subtly wrong. The interviewer wants the savings arithmetic and the misrouting risk in the same answer.

A principal-level question because the cost of serving a user is the same in Mumbai and Minneapolis while the price you can charge is not. The free tier is an acquisition budget, and the paid tier needs a model mix by market.

Routing as a platform feature saves customers money and moves quality decisions onto you. Make it opt-in, show which model answered, bound the quality loss and never change the model set under production traffic.