preference data

Choose a training objective from observed failures. Separate preference labels, demonstrated answers and adapter methods from factual grounding.

Choose data that can change a measured failure. Price accepted examples, reserve evaluation capacity and explain why a cheaper label may teach the wrong behavior.

Budget for the whole labeling job, distinguish disagreement from poor work and test whether the resulting data improves the model.