Engineering generated 5,000 eval cases with an LLM in an afternoon. Can they replace the 300 cases we pulled from real user traffic?

Synthetic cases add coverage cheaply and inherit the generator's blind spots. Use them for named slices and adversarial variants, keep real traffic as the launch gate, and measure the gap between the two.

Updated Nov 2026 · Original practice questions with worked answers and teaching rubrics. Our editorial policy explains the sources, review process and limits.