AI Metrics & Unit Economics Interview Questions

An estimation question that tests commercial judgment. The inference bill is where the arithmetic starts; the number that decides the answer is somewhere else.

A vocabulary question with a sharp edge. The cheapest model per token is regularly the most expensive per finished job.

Usage counts are the trap. The answer that scores connects quality, behaviour, cost and risk to one business outcome, and says what evidence would change the primary metric.

The cheap option per conversation is the expensive one per customer, until you read how the vendor defines 'resolved'. Two calculations, and the second one is where the offer is decided.

Candidates jump to 'switch to a cheaper model'. The interviewer wants to see you find out why cost per user rose before choosing a lever, then pull the levers in the order that protects quality.

A vocabulary question about speed. The right metric depends on whether the user is watching text stream in or waiting for a finished result.

More AI-generated code can mean more code to review and maintain. Choose a metric that moves when developers deliver useful, reliable work.

A margin question with a constraint added halfway through. The average user is not the problem; the distribution is, and a flat price hides it until the model gets more expensive.

Survey says everyone saves five hours a week. Finance does not believe it, and they are right not to. The answer is to measure a few workflows properly instead of the whole company vaguely.

AI metrics have unusually soft edges: what counts as an AI product, a resolution or an active user can widen without anyone announcing it. This question checks whether you audit a good number as hard as a bad one.

The candidate who says 'use a faster model' in the first sentence has already lost points. Decompose the budget, find the biggest bar, and check whether two seconds is the right target at all.

Routing is one of the biggest cost levers in AI products, and one of the easiest to get subtly wrong. The interviewer wants the savings arithmetic and the misrouting risk in the same answer.

AI features are unusually good at producing engagement that is actually friction. The question tests whether you can tell the two apart in the data.

The most expensive AI failures are the confident wrong answers that users accept. This question tests whether you can find their footprints in behaviour data.

Caching is the cheapest large saving in most AI products, and a single line of personalisation at the top of the prompt can turn it into a 22% cost increase. The arithmetic is short; the layout insight is what scores.

A principal-level question because the cost of serving a user is the same in Mumbai and Minneapolis while the price you can charge is not. The free tier is an acquisition budget, and the paid tier needs a model mix by market.

A portfolio and stakeholder question. Across-the-board cuts are the easy answer and the wrong one. The strong answer finds where spend creates value, cuts waste first, and gives the CFO a number they can hold you to.

Metric trees for AI agents need two layers classic product trees do not have: quality and risk. This question checks whether you include them and connect them to the business outcome.

A fundamentals question. The AI twist is that the fastest leading indicators come from what users do with the output in the first few seconds.

The cost of the model is rarely the biggest number in this decision. The cost of a wrong answer is. One calculation makes the choice obvious in some products and wrong in others.

When the metric is the invoice, every edge case becomes a billing dispute. Principal-level answers design a definition customers can audit and a price that survives the worst month.

Batch pricing halves the bill on eligible work, and the real product question is what you can precompute before anyone asks for it.

One person's AI output becomes another person's input. Choose the experiment unit, account for shared work and decide what a large user count can actually prove.

Rank surviving signals by what they actually prove, decide when to pause exposure and reconcile missing events without inventing outcomes.

Design task evidence, privacy controls and a separate diagnostic path without pretending that minimal telemetry proves semantic correctness.

Separate saved meeting time from lost participation, and test outcomes at the team level when the feature changes shared behavior.

Messages sent and time spent reward an assistant that wastes users' time. Choose a metric tied to tasks users come back to finish, and pair it with quality and cost.

First AI features have unknown quality and cost. Write OKRs that reward learning and a measured launch, and keep out key results the team cannot honestly commit to.

Turn a registered-user count into active usage, model calls and billable tokens. Compare an uncached baseline with a stated caching scenario without inventing a vendor quote.

Two forecasts make errors in opposite directions. Calculate the operational consequences before deciding which evaluation metric should control the release.

Build a monthly task estimate from a fictional shop market. Cross-check it through technicians, then explain which assumption you would investigate first.

A model release and a campaign happened together. Reconstruct the success rate, explain the mix shift and decide what evidence should drive the next change.

In one randomised trial, developers believed AI made them 20% faster while it made them 19% slower. Run staggered access on real work, measure time to merge and defects, and renew team by team.

The landmark study of AI in a call centre found the same pattern: big gains for new agents, small speed gains and small quality losses for the best. Report by skill tier and give experts a lighter mode.

Cheaper tokens bring longer prompts, more steps and more tasks, and some prices rise. Forecast spend as price times tokens per task times tasks times users, in bands, with caps per feature.

A follow-up can mean the answer led somewhere useful or that it failed. Split refinements from extensions, make successful sessions the primary metric, and keep clicks and abandonment as guardrails.

A low attach rate can mean few accounts buy, or buyers don't use it, or the denominator is wrong. Measure penetration inside buying accounts and active use of paid seats before changing the packaging.

Industry data shows AI apps earn more per payer and churn faster. Compute lifetime value net of serving cost, find which jobs are one-off, and either build a recurring loop or price for the one-off job.

Generating more code moves the bottleneck to review, testing and release. Switch the scorecard to delivery metrics and put the next budget into review and testing capacity.

Shadow-bill everyone first, roll out to new customers before existing ones, protect annual plans and default overage to zero. Set stop rules on bill-shock tickets, refunds and downgrades before you start.