The prompt
"Design an AI feature for a rental housing platform that helps tenants report maintenance problems." The platform connects tenants with landlords and property managers in several countries.
The answer below follows six steps. Scoring notes after some of them explain what an interviewer is listening for, using the Weak, Hire and Strong Hire language of our rubrics.
Step 1: choose the user and the job
Two users matter, and a strong answer picks one to lead with. Tenants want the problem fixed with little effort. Property managers want reports they can act on without phoning to find out what "the sink is weird" means. The manager's pain costs more, since a vague report means a wasted visit or a second trip with the right part. Lead with the manager's job, delivered through a better tenant experience.
Scoring note: naming two users and choosing one with a reason is a Hire signal. Designing for everyone at once is among the most common Weak signals in product sense.
Step 2: decide where AI helps and where it must not
Scoring note: this split is where many candidates win or lose the conversation. Putting a model in sole charge of emergency detection is marked Weak, because it shows no feel for the cost of a wrong answer.
| Part of the flow | Model or rule | Why |
|---|---|---|
| Understanding a free-text description and photos | Model | Varied, messy input, and a misread can be caught by a follow-up question |
| Asking for missing details | Model | The right question depends on the problem; a fixed form misses most of them |
| Spotting emergencies: gas smell, flooding, no heat in winter | Rule first, model as a second net | A miss is dangerous, so keywords and a mandatory safety question run whatever the model thinks |
| Choosing a contractor and approving spend | Rules and the manager | Money and contracts; the model may suggest, never commit |
Step 3: design the experience, wrong cases included
The tenant describes the problem in their own words and language, adds a photo, and answers at most three follow-up questions the model chooses. The model then writes a structured report: category, location in the unit, urgency, likely parts and the tenant's availability. The tenant sees that summary and confirms or corrects it before it is sent. The confirmation is the main protection against a misread, and it costs the tenant about ten seconds.
When the model is unsure of the category, it says so and files the report as needing review instead of guessing. Anything the emergency rule catches shows a safety instruction and a phone number at once, before any AI step runs.
Step 4: set the quality bar
Build a golden set from a few hundred past reports where the eventual repair is recorded, so the true category and urgency are known. A proposed bar: category correct on 90% overall with no language below 80%; zero missed emergencies on the emergency subset, counting the rule layer; and fewer than one report in ten needing a clarifying phone call, measured in a pilot.
Scoring note: numbers, a segment floor and a zero-tolerance class put an answer in Strong Hire territory, provided the candidate can say where each number comes from.
Step 5: check the economics
Suppose a report takes three short model turns plus image understanding, around 10,000 input tokens and 1,000 output tokens in total. At Claude Sonnet 5.5 list prices as of October 2026 ($2 and $10 per million), that is $0.02 for input and $0.01 for output: about three cents a report.
Set against one wasted contractor visit, the feature pays for itself if it prevents a wasted trip in a small fraction of reports. Cost is not the risk in this design. The emergency path is, and the answer should say so.
Step 6: metrics
Primary metric: share of repairs completed in one contractor visit, against a holdout group or the period before launch. Guardrails: missed emergencies (target zero, every case reviewed), the share of tenants who finish the reporting flow, and manager edits per report.
Do not count reports created with AI. That measures usage, not whether repairs got easier.
What the interviewer adds next
Expect constraints midway. A landlord wants the AI to reject requests the lease makes the tenant's responsibility: the model may flag a likely lease question, a person decides, and the tenant sees the reason and a way to appeal. A market where most tenants send voice notes: transcribe first, run the same pipeline, and evaluate that language separately. Finance wants the cost halved: route simple text-only reports to a smaller model after checking it against the same golden set.
Sources and further reading
Questions about this topic
User and job; where AI helps and where it must not; the experience including failure cases; the quality bar; the economics; then metrics. Spend the most time on the second and fourth steps, since that is where AI rounds differ from classic product sense.