contamination

Saturated and contaminated benchmarks separate models less than the noise in your own traffic. Use public scores to shortlist and a private, refreshed eval to decide.

Benchmarks die from leakage, label errors and saturation. Hold back a private split, audit labels with experts, score abstention, watch the public-private gap and publish when you will retire it.