A new model wins 64% of preference tests but expert users prefer the old one. Would you replace it?

A global preference win can conceal a regression for the people doing the hardest work. Reweight the evidence, inspect the behavior change and choose a release policy.

Updated Oct 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.