routing

There is no single best model, and the choice will change within a year. Principal answers design a selection process and an architecture that make the next choice cheap.

A small architectural component with outsized strategic value. The interviewer wants to hear what it does day to day and why it matters when a provider changes prices or goes down.

A full-stack AI system design with scale, money, languages and angry customers. The interviewer wants tiers: cheap triage for everything, an agent where it pays, and humans where it matters, with the cost per day worked out.

At a billion requests a day, a fraction of a cent per call is millions of dollars a month and every millisecond is capacity. Principal answers start from the arithmetic and reshape the feature, the models and the serving to fit.

The cost of the model is rarely the biggest number in this decision. The cost of a wrong answer is. One calculation makes the choice obvious in some products and wrong in others.