monitoring

Aliases move, providers change infrastructure and retired models get re-routed. Pin versions in production, watch continuously and treat every model change as a release.

Green dashboards and angry users at the same time. The strong answer treats the complaints as evidence about the eval, not noise to explain away.

The most expensive AI failures are the confident wrong answers that users accept. This question tests whether you can find their footprints in behaviour data.

A fundamentals question for platform and model PM roles. The interviewer wants the stages in order and, at each, the decision a PM is responsible for.