The new model scores 6 points higher on our offline eval, but the A/B test shows no change in user metrics. What happened?

Make your call first. Use the published answer and rubric to examine your reasoning when you are ready.