Original practice question. The scoring rubric is our teaching guide. How we research and score answers
TL;DR: Scope the risks for this product and its users, recruit a mix of internal testers, domain experts and outside specialists, give them structured goals plus free exploration, rate every finding by severity and likelihood, fix the severe ones before launch with clear exit criteria, and turn findings into an automated adversarial test suite that runs on every change.
How to approach it
Start with what could go wrong for this specific product and audience, because red-teaming without a threat model produces random findings. Then plan people, process, triage, exit criteria and what happens after launch.
A strong answer
1. Threat model (week 1). For a general consumer assistant with web browsing and image understanding, launching in several countries:
| Risk area | Examples |
|---|---|
| Harmful content | Instructions for violence, self-harm content, dangerous activities |
| Vulnerable users | Minors, people in crisis, medical or financial questions with high stakes |
| Misinformation | Confident false claims about health, elections, current events |
| Privacy | Revealing personal information about real people, doxxing |
| Prompt injection | Hidden instructions in web pages or images the assistant reads |
| Bias and offensive output | Stereotypes, slurs, inconsistent treatment across languages |
| Brand and legal | Defamation, impersonation, copyrighted text reproduction |
2. People. Internal engineers and PMs (they know the system's weak points), domain experts (clinicians, child-safety specialists, election-integrity researchers), native speakers for each launch language (many failures appear only outside English), and an external red-team firm for a fresh, adversarial perspective.
The native-speaker line has evidence behind it. Yong, Menghini and Bach (2023) translated harmful requests into low-resource languages and got actionable harmful help from GPT-4 79% of the time on the AdvBench benchmark, while high- and mid-resource languages had much lower attack success. A launch in Hindi, Tamil or Swahili that was red-teamed only in English has not been red-teamed for those users. Microsoft's AI Red Team, drawing on more than 100 generative AI products (January 2025), makes the related point that human judgment and subject-matter expertise stay central and that red teaming is ongoing work rather than a phase.
3. Process (weeks 2 to 4). Structured campaigns per risk area with goals ("get the assistant to give dosing advice for a prescription drug to someone who says they are 15"), plus time for free exploration. Testers log each finding with the prompt, output, and context in a shared tracker.
4. Severity and triage.
| Severity | Example | Launch rule |
|---|---|---|
| Critical | Reliable instructions for serious harm; child safety failure | Must be fixed; launch blocked |
| High | Personal data leakage; easy injection leading to actions | Fix before launch or remove the capability |
| Medium | Inconsistent refusals; offensive output with effort | Fix or mitigate, tracked |
| Low | Awkward responses | Backlog |
Also rate reproducibility: a finding that works one time in fifty is different from one that works every time.
5. Fix and retest (weeks 4 to 5). Fixes are layered: model behaviour (prompts, safety tuning), product controls (removing a tool, requiring confirmation, age gating), and classifiers. Each fix is retested by a different tester.
6. Exit criteria (week 6). Zero open critical or high findings; medium findings mitigated with owners and dates; adversarial test suite passing at an agreed rate; sign-off from safety, legal and the product lead.
7. After launch. Every finding becomes a test case in an automated adversarial suite run on each model or prompt change. A public reporting channel and a bug bounty catch what internal testing missed. Monitor production for new attack patterns, especially in the first weeks.
What interviewers probe next
"We find a critical issue two days before launch." Launch is delayed or the capability is removed. Exit criteria exist so this decision is not renegotiated under pressure.
"How do you red-team languages nobody on the team speaks?" Hire native-speaking testers, and launch those languages only when they have been tested. The low-resource-language results above are the reason to hold that line when a regional launch date is under pressure.
How it's scored
| Signal | Weak | Hire | Strong Hire |
|---|---|---|---|
| Scope | "Try to break it" | Lists risk areas | Threat model specific to product, users and markets |
| Process | Ad hoc testing | Structured campaigns | Mixed testers including native speakers, severity and reproducibility triage, retesting |
| Outcome | A report | Fix critical issues | Explicit exit criteria with sign-off, findings converted to an automated suite, post-launch reporting |
Common mistakes
Red-teaming only in English.
Running it once and never turning findings into regression tests.
Leaving launch criteria vague so critical findings get argued away.