OpenAI LLM & RAG Fundamentals interview questions

Recommend an approach to a CEO who needs a product decision. Explain what retrieval and fine-tuning each change, then connect that difference to freshness, reliability and cost.

The first vocabulary word of AI product management. Tokens are the unit of cost, speed and memory, and a PM who can't reason in them can't price, budget or design an AI feature.

Fine-tuning is sometimes the best decision a team makes and often a costly detour. The interviewer wants the test that tells them apart, with numbers.

A vocabulary check on one of the most important model developments of recent years. The good answer explains the trade (accuracy for time and money) and gives a rule for when it pays.

A puzzle with a precise answer. Price per token is only comparable if a token means the same thing, and across model generations it often does not.

A quality problem with a technical cause and a product solution. The strong answer explains why long contexts degrade and designs conversation management rather than hoping a bigger window fixes it.

Explain what the model sees on each call, why more context can make answers worse and costlier, and which parts of the context recipe a PM owns.

Models that take images, audio and documents open new inputs and new failure modes. Explain what they do, what they cost, and the product decisions that change.

Turn confidence scores into a defensible approval policy. Work through selective accuracy, uncertainty, correlated errors and review capacity before deciding which invoice fields can bypass a human.

Saturated and contaminated benchmarks separate models less than the noise in your own traffic. Use public scores to shortlist and a private, refreshed eval to decide.

Shorter and binary embeddings cut storage by orders of magnitude and lose a few points of retrieval. Decide with recall on your own queries and a rescoring step.

Reasoning tokens are billed as output and add latency. Find the lowest effort that holds quality for each feature, and spend high effort where it measurably helps.

Constraining output format helps parsing and can hurt thinking. Decide where the reasoning happens, then test the structure on your own eval.

The model only knows what it was trained on. Internal policy belongs in your own dated retrieval, public news in search, and only the questions that need it should pay for search.

Output limits, reasoning that eats the budget and models trained on shorter answers all cut reports short. Generate by section, check why each response stopped and count failures as cost.

Hundreds of examples in a cached prompt can match fine-tuning and let you change behaviour in minutes. Price both, then decide by volume, latency and how often the examples change.

Language models predict tokens rather than calculate. Let the model extract numbers and let code do the maths, then check the result against the printed total.

Aliases move, providers change infrastructure and retired models get re-routed. Pin versions in production, watch continuously and treat every model change as a release.

A fundamentals question that comes up when a stakeholder notices the assistant answered differently yesterday. The good answer explains sampling in one minute and then turns it into product decisions.

Retrieval quality decides most RAG products, and embeddings are where it starts. The interviewer wants the intuition, the failure modes, and the fixes a PM would push for.

A larger context window changes what fits in a request. Decide whether to use it by checking corpus size, freshness, permissions and cost per successful query, including cache misses.

Everyone knows models make things up. The interviewer wants to hear why, which kinds of hallucination exist, and a layered set of product defences, without promising to eliminate it.

A fluency question about the request path. The interviewer does not need the maths; they want to know you can connect each stage to latency, cost and design decisions.

Retrieval failures look like model failures to users. This question tests whether you can locate the failure precisely and choose fixes in the order that pays, rather than swapping models.