A diagnosis question with five plausible causes. Strong candidates rank them by likelihood and name the cheapest check for each instead of reciting all five.
free accounts read more answers per topic · no card
← Evals & Launch Decisions / 08
hardNewGoogleMetaLinkedIn
The new model scores 6 points higher on our offline eval, but the A/B test shows no change in user metrics. What happened?
Try it in the practice workspaceDraft your answer before comparing it with the rubric.A diagnosis question with five plausible causes. Strong candidates rank them by likelihood and name the cheapest check for each instead of reciting all five.
Updated Nov 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.
Follow-up roundsAnswer first, then revise for each changed constraint
- Round 2.
Product analytics says long, formal messages will be 45% of drafts next quarter, up from 20%.
- Round 3.
Routing needs a classifier call on every request, which adds 400 ms at p95. Past tests on this feature lost about 1% of acceptance, relative, for each 100 ms.
UP NEXT ON YOUR JOURNEY
NOTES & DISCUSSION · 0
No comments yet. How would you have answered?