21You now own AI product at a company where 30 teams each evaluate their AI features differently. What eval strategy do you set?▼expertNewSalesforceServiceNowJPMorgan Chase◆ premiumA portfolio question. Standardise too much and teams stop evaluating what matters to their users; too little and nobody can compare quality, swap models or answer an auditor.Practise this question →Read answer →
33Every AI team put its eval pass rate in its OKRs. Six months later every score is up and nothing users see has changed. What went wrong?▼hardNewMetaAnthropicOpenAI◆ premiumWhen a measure becomes a target, teams tune to it. Take eval scores out of targets, keep a sealed holdout teams cannot tune against, separate capability from regression evals, and tie OKRs to user outcomes.Practise this question →Read answer →