GleanProduct Manager, Agents; Product Manager, AI Quality interview preparation

Citations are the main trust device in AI answers, and published audits of shipped search engines find many that do not support the sentence they sit on. The interviewer wants citations at the right granularity, checkable in seconds, and honest about what they support.

Trust that outruns accuracy is as dangerous as distrust. The interviewer wants design that keeps experts engaged where it matters, without making the tool tedious.

The strategy question every AI application company faces. 'Better prompts' is not an answer. The interviewer wants a ladder of defensibility and a plan to climb it.

Platform risk made real. The answer is neither panic nor dismissal: work out which customers are actually at risk, double down on what the provider cannot easily do, and reduce dependence where it matters.

A pricing question about where the value of falling model costs should go. The answer depends on margin targets, competition and whether price or capability wins customers.

A positioning question where horizontal products face the model providers head-on and vertical products face smaller markets. The interviewer wants a framework and a choice.

Recommend an approach to a CEO who needs a product decision. Explain what retrieval and fine-tuning each change, then connect that difference to freshness, reliability and cost.

Retrieval quality decides most RAG products, and embeddings are where it starts. The interviewer wants the intuition, the failure modes, and the fixes a PM would push for.

A larger context window changes what fits in a request. Decide whether to use it by checking corpus size, freshness, permissions and cost per successful query, including cache misses.

Retrieval failures look like model failures to users. This question tests whether you can locate the failure precisely and choose fixes in the order that pays, rather than swapping models.

Many enterprise RAG failures happen before retrieval: the system never read the document correctly. The interviewer wants you to find the parsing problem and fix it at the right layer.

Explain what the model sees on each call, why more context can make answers worse and costlier, and which parts of the context recipe a PM owns.

A RAG pipeline is the obvious first sketch. A strong answer starts with permissions instead, because one leaked salary document ends the product.

A short question with a large payoff. A PM who knows the one rule of prompt caching can cut a feature's input bill dramatically without touching quality.

A security design question that decides whether enterprise customers will sign. The answer has to make cross-tenant leakage impossible by construction, and then prove it with tests.

A cost lever that can silently serve wrong answers and leak one user's data to another. The interviewer wants the failure modes and a design that keeps the savings where they are safe.

Define what unread means, preserve edits and permissions, and make every action item traceable. The design must tell users when the summary covers only part of their inbox.

A runaway agent is a cost problem, a reliability problem and a product problem at once. The answer is part diagnosis, part guardrails, and part design for what happens when a budget runs out.

A range is not a pricing input. Work out cost per accepted task by effort level and cost per account by cohort, compare flat, allowance and usage plans on the same customers, then say what would change your choice.

Every connected tool adds a description the model must read on every call. Diagnose tool confusion from traces, then cut, rename and load tools by task.

Slack allowed permission-checked search and banned bulk export, indexing and training by outside apps. Decide what access serves customers, what protects them and what keeps your own AI features competitive.

A RAG answer can fail in the retriever or in the generator, and one accuracy score cannot say which. The interviewer wants two evals, a production loop and a launch bar tied to what a wrong answer costs.

Leaderboards move every few weeks and every team feels the pull. The interviewer is checking whether you have a repeatable switching process or just enthusiasm.

Caching is the cheapest large saving in most AI products, and a single line of personalisation at the top of the prompt can turn it into a 22% cost increase. The arithmetic is short; the layout insight is what scores.

Every connected server is code you did not write, holding credentials you issued, describing tools your agent will trust. Write the approval checklist and the runtime controls.

Embeddings are infrastructure that customers store and rebuild around. The interviewer wants retrieval quality on real tasks, multilingual coverage, cost, and above all a plan for version changes, because switching embeddings means re-indexing everything.

RAG is the most common live-build topic. Strong candidates choose the simplest approach that works for 30 documents, make citations real, and test retrieval and answers separately.

Decide which support tasks an agent can safely own. An 82% pass rate needs a denominator, a breakdown of failures and a comparison with today's service.

The central AI UX question. The interviewer wants patterns that make errors cheap to spot and cheap to fix, sized to what an error costs.

A shared vocabulary for one of the biggest decisions in AI product design: how much the AI does on its own. The interviewer wants the levels, and the rule for choosing among them.

Accuracy and trust are different things. The feature saves no time if people double-check everything, and the fix starts with finding out why experts are cautious.

Users let an assistant act when they can see what it did and take it back. The interviewer wants an activity log, real undo, and clear handling of actions that cannot be undone.

A deceptively simple design question. Numbers feel transparent, but most users cannot act on them, and the numbers are often less meaningful than they look.

Users either expect too much from AI features and churn after the first failure, or too little and never try the useful parts. Onboarding sets the mental model that decides which happens.

Approval screens fail in a predictable way: people click 'approve all'. The interviewer wants a design that focuses attention on the changes that matter and makes careful review fast.