Microsoft AI System Design interview questions

A RAG pipeline is the obvious first sketch. A strong answer starts with permissions instead, because one leaked salary document ends the product.

A small architectural component with outsized strategic value. The interviewer wants to hear what it does day to day and why it matters when a provider changes prices or goes down.

A pipeline design question with an automation trap. Writing summaries is the easy part; writing into the system of record that forecasts revenue is where the design needs care.

Latency-dominated system design. A suggestion that arrives after the developer has typed the next characters is worthless, so every design choice is about milliseconds, context and cost per keystroke.

Without traces, every quality complaint is a guess. The interviewer wants the fields that make debugging, evaluation and cost control possible, and an awareness of what should not be stored.

Search plus generation at consumer scale. The design has to deliver fresh, cited answers in a couple of seconds, resist manipulated pages, and keep cost per query viable.

A principal-level platform question. Each team building its own agent stack means dozens of security models, cost blind spots and incompatible tools. The platform has to make the safe path the easy path without slowing teams down.

At a billion requests a day, a fraction of a cent per call is millions of dollars a month and every millisecond is capacity. Principal answers start from the arithmetic and reshape the feature, the models and the serving to fit.

Guardrails are often described as a box in a diagram. The interviewer wants specific checks, where each runs, what it costs in latency, and what happens when one fires.

A lawyer will not trust a summary. Design clause extraction, playbook comparison and cited deviations, with a measured miss rate on the clauses that carry liability.

Define what unread means, preserve edits and permissions, and make every action item traceable. The design must tell users when the summary covers only part of their inbox.

A model wins on connected tasks but leaves field staff without help during outages. Compare all scheduled work, then defend local data, expiring permissions and synchronization.

Self-reported minutes saved, a randomised trial and one deployer's own records disagree by an order of magnitude. Decide what the scribe may do, how patients are told and what you will measure.

The token bill drops by two-thirds and failures rise by one point. Price the failures before you call it a saving, and check what routing does to your cache.

Two hundred tool calls at 99.9% each fail somewhere 18% of the time. Decide what is checkpointed, what must be idempotent and what happens to a run in flight when you deploy.

Tokens cost under a dollar per run; reviewer time costs thirty times that. Choose the task classes, the permissions and the gates that decide what a human ever sees.

Accept rate can rise 28% while accepts stay flat. Define the reward, correct for what users never saw and decide whose data you may train on.

A full-stack AI system design with scale, money, languages and angry customers. The interviewer wants tiers: cheap triage for everything, an agent where it pays, and humans where it matters, with the cost per day worked out.

Every AI product depends on services it does not control. The interviewer wants a degradation plan by feature, written and tested before the outage starts.

A short question with a large payoff. A PM who knows the one rule of prompt caching can cut a feature's input bill dramatically without touching quality.

A deadline you did not choose and a change that touches every output. The interviewer wants a plan that treats API compatibility and behavioural compatibility as different problems.

A design question with a constraint dropped halfway. The interviewer wants to see residency flow through every component: model calls, logs, caches, embeddings, evals and support access.

A security design question that decides whether enterprise customers will sign. The answer has to make cross-tenant leakage impossible by construction, and then prove it with tests.