Seniority is not the problem
Experienced PMs fail AI PM interviews for reasons that have little to do with seniority. Their instincts are sound; a few of them point the wrong way when the product is probabilistic and every request costs money. These are the patterns the Weak lines in our scoring rubrics describe most often, each with the answer that replaces it.
Treating quality as a feeling
"The responses looked great in testing" is not evidence. Replace it with a number on a defined set: "On 200 real tickets, 86% passed our rules. Billing questions were weakest at 71%, so they stay with agents for now."
Averaging away the failures
An overall pass rate hides segments. A feature at 92% overall can sit at 60% for Hindi queries or for the longest documents. Ask which segment is worst, and whether it should block the launch, before the interviewer does.
Ignoring cost per request
Classic PMs rarely think about the marginal cost of a feature, because it is close to zero. In AI products, heavy users can cost more than they pay. Bring one cost calculation into every product and strategy answer, even a rough one on a whiteboard.
Promising capability instead of outcomes
"The agent will handle all refund requests" is a promise about a model you do not control. "Refund handling time halves, with the agent resolving simple cases and drafting replies for the rest" is a commitment you can keep even when the model falls short.
Adding a human at the end
"And a human reviews it" sounds safe and is usually a design nobody has costed. Say which cases go to a person, how they are detected, roughly how many there will be, and what the reviewer sees when one arrives.
Trusting the prompt for safety
A system prompt that says never to send money without approval is not a control. Permissions belong in code outside the model, because injected instructions can override what the prompt says. Candidates who raise this without being asked stand out.
Measuring the AI instead of the job
Messages sent to an assistant, summaries generated and suggestion acceptance rates all climb when a feature is used, whether or not it helps. Tie the primary metric to the job users came for: tickets resolved without a repeat contact, code that survives review, tasks completed.
Hedging every answer
AI questions are uncertain, and some candidates respond by never committing. The decision is what gets graded. State your default, the assumption underneath it, and the evidence that would make you change it.
Sources and further reading
Questions about this topic
Judgment backed by numbers: a quality bar, a cost estimate and a plan for failure, each tied to the user's problem.