← AI System Design / 21
expertNewMetaGoogleMicrosoft

An AI reply-suggestion feature in our messaging app will get a billion requests a day. How do you make cost and latency work?

Try it in the practice workspaceDraft your answer before comparing it with the rubric.

At a billion requests a day, a fraction of a cent per call is millions of dollars a month and every millisecond is capacity. Principal answers start from the arithmetic and reshape the feature, the models and the serving to fit.

Updated Oct 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.

At a billion requests a day, a fraction of a cent per call is millions of dollars a month and every millisecond is capacity. Principal answers start from the arithmetic and reshape the feature, the models and the serving to fit.

This reference answer is in Premium. Free signup opens other answers.

9 answers per topic instead of 5, and your progress kept · no cardor unlock all 420 remaining answers
UP NEXT ON YOUR JOURNEY
NOTES & DISCUSSION · 0

No comments yet. How would you have answered?