28The new model wins the A/B test on thumbs-up rate, but reviewers say it agrees with users too easily. Do you ship it?▼hardNewOpenAIAnthropicGoogle DeepMind◆ premiumUsers rate flattering answers higher, so the launch metric points the wrong way. Build the measurement that catches it and decide what blocks a launch.Practise this question →Read answer →
24An insurer wants a computer-use agent to enter claims into a legacy web system with no API. How do you decide it is ready?▼hardNewAnthropicOpenAIMicrosoft◆ premiumA smooth demo on ten claims proves little. Build a task-level evaluation on a test copy, price wrong entries against staff time, and launch with review before submission.Practise this question →Read answer →