← Evals & Launch Decisions / 36
hardNewOpenAIGoogle DeepMindPerplexity

Model A answers everything, right 24% of the time. Model B declines half and is wrong far less. Which do we ship?

Try it in the practice workspaceDraft your answer before comparing it with the rubric.

Accuracy alone rewards guessing. Set a penalty for wrong answers from what errors cost in your product, score both models under it, and the choice usually flips.

Updated Nov 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.

Accuracy alone rewards guessing. Set a penalty for wrong answers from what errors cost in your product, score both models under it, and the choice usually flips.

This reference answer is in Premium. Free signup opens other answers.

9 answers per topic instead of 5, and your progress kept · no cardor unlock all 420 remaining answers
UP NEXT ON YOUR JOURNEY
NOTES & DISCUSSION · 0

No comments yet. How would you have answered?