← Metrics & Unit Economics / 24
expertNewMicrosoftAmazonGoogle

An AI feature is mid-rollout when outcome telemetry fails. What decisions can you still make?

Try it in the practice workspaceDraft your answer before comparing it with the rubric.

Rank surviving signals by what they actually prove, decide when to pause exposure and reconcile missing events without inventing outcomes.

Updated Oct 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.

Case briefRead the inputs, then make your call

An AI document assistant is mid-rollout when its task-completion event stops arriving. The case figures are fictional.

Scenario inputs
Rollout10% of workspaces have access; the next step would double exposure.
Last hour12,000 accepted requests; 11,700 model invocations in billing; 9,000 completed jobs in a durable store.
Fallback audit18 failures in a random sample of 200 eligible completed jobs. Abandoned and unfinished jobs were not sampled.

Your task. Decide what can keep operating, rank the evidence and define recovery and resumption conditions.

Rank surviving signals by what they actually prove, decide when to pause exposure and reconcile missing events without inventing outcomes.

This reference answer is in Premium. Free signup opens other answers.

9 answers per topic instead of 5, and your progress kept · no cardor unlock all 420 remaining answers
UP NEXT ON YOUR JOURNEY
NOTES & DISCUSSION · 0

No comments yet. How would you have answered?