Should we use an LLM or a tabular model to predict customer churn?
Make your call first. Use the published answer and rubric to examine your reasoning when you are ready.
Case briefRead the inputs, then make your call
Fictional churn planning exercise. A time-held-out set has mature 30-day churn labels; all features were available at prediction time. Each model selects the same outreach population size.
| Accounts | 10,000 |
|---|---|
| Observed churners | 800 |
| Outreach capacity | 500 |
| Usage rule | 150 true churners in the 500 selected accounts. |
| Tabular model | 220 true churners in the 500 selected accounts. |
| LLM on serialized records | 190 true churners in the 500 selected accounts. |
| Unknown | The effect of outreach on retention has not been measured. |
Your task. Choose a model and an intervention pilot. Calculate precision, recall and F1 at the supplied capacity, identify leakage risks and explain what would change your decision.
0 words