inference cost

Reasoning tokens are billed as output and add latency. Find the lowest effort that holds quality for each feature, and spend high effort where it measurably helps.

An estimation question that tests commercial judgment. The inference bill is where the arithmetic starts; the number that decides the answer is somewhere else.

Candidates jump to 'switch to a cheaper model'. The interviewer wants to see you find out why cost per user rose before choosing a lever, then pull the levers in the order that protects quality.

Caching is the cheapest large saving in most AI products, and a single line of personalisation at the top of the prompt can turn it into a 22% cost increase. The arithmetic is short; the layout insight is what scores.

Batch pricing halves the bill on eligible work, and the real product question is what you can precompute before anyone asks for it.

Cheaper tokens bring longer prompts, more steps and more tasks, and some prices rise. Forecast spend as price times tokens per task times tasks times users, in bands, with caps per feature.