streaming

Streaming is the cheapest latency improvement in AI products. The interviewer wants the reason, the trade-offs, and the cases where showing partial output is a mistake.

Streaming text, unlabelled buttons and moving focus break screen readers in ways sighted testing never shows. Fix announcements, structure and controls, then test with disabled users.

Output limits, reasoning that eats the budget and models trained on shorter answers all cut reports short. Generate by section, check why each response stopped and count failures as cost.

The candidate who says 'use a faster model' in the first sentence has already lost points. Decompose the budget, find the biggest bar, and check whether two seconds is the right target at all.