small models

A platform strategy question with hardware, privacy, cost and capability all in play. Principal-level answers design a split by task, with a privacy story users can understand and a plan for the devices that cannot run local models.

Distillation is how teams get large-model quality at small-model cost on a specific job. The interviewer wants to know when that trade is worth the engineering, and what can go wrong.