capacity

Choose an operating threshold using prevalence, error costs and review capacity. A good ranking score can still produce an unusable queue.

Turn a registered-user count into active usage, model calls and billable tokens. Compare an uncached baseline with a stated caching scenario without inventing a vendor quote.

Build a monthly task estimate from a fictional shop market. Cross-check it through technicians, then explain which assumption you would investigate first.

Rate limits protect capacity and fairness, and they shape the developer experience as much as any feature on the API. The interviewer wants limits by dimension, tiers that grow with trust, and clear behaviour when limits hit.

Model latency depends on input length, output length, load and the model itself, so a naive guarantee is either meaningless or impossible. The interviewer wants a guarantee that is defined precisely, measurable and backed by capacity.

Two labs moved from capacity commitments to a pay-per-token fast mode. Decide what speed means, how you stop the fast lane being swamped and who still needs reserved capacity.