19We launched AI summaries for long documents three months ago. Only 4% of eligible users have used them. What do you do?▼mediumNewNotionAtlassianGoogle◆ premiumLow adoption has four usual causes: people don't see it, don't need it, try it and don't trust it, or find it in the wrong place. Each has a cheap test, and the interviewer wants you to run them before redesigning.Practise this question →Read answer →
10Our RAG assistant keeps retrieving the wrong documents. How would you diagnose and fix it?▼hardNewGleanAtlassianNotion◆ premiumRetrieval failures look like model failures to users. This question tests whether you can locate the failure precisely and choose fixes in the order that pays, rather than swapping models.Practise this question →Read answer →
23A model passes offline tests but fails in production. Is this overfitting, training-serving skew or drift?▼hardNewGoogleAmazonMicrosoft◆ premiumUse feature parity, time-aware holdouts and delayed outcomes to decide what needs fixing before requesting another training run.Practise this question →Read answer →
08The new model scores 6 points higher on our offline eval, but the A/B test shows no change in user metrics. What happened?▼hardNewGoogleMetaLinkedIn○ sign inA diagnosis question with five plausible causes. Strong candidates rank them by likelihood and name the cheapest check for each instead of reciting all five.Practise this question →Read answer →
13After launching an AI chat assistant in our app, session length went up 25% but 60-day retention fell. What's going on?▼hardNewDuolingoSpotifyMeta◆ premiumAI features are unusually good at producing engagement that is actually friction. The question tests whether you can tell the two apart in the data.Practise this question →Read answer →