coding agents

Two well-run teams published opposite defaults. Decide by repository size, how fast code changes, latency and whether customers will let you store embeddings of their code.

Tokens cost under a dollar per run; reviewer time costs thirty times that. Choose the task classes, the permissions and the gates that decide what a human ever sees.

Agent output is cheap; human review is not. This question tests whether you measure the agent by what it costs the team to accept its work, not by how much work it produces.

Models rarely ask unless told to, and research shows asking at the right moment raises success substantially. Ask when the answer changes the outcome and the action is costly, batch questions up front, and measure both kinds of error.

A textbook confused-deputy attack on a powerful agent. The interviewer wants the mechanism, the full set of controls (sandbox, network, secrets, approvals), and a response plan for the disclosure.

A coding agent builds the wrong thing quickly from a vague brief. The answer that scores shows what a PM adds to a PRD before handing it to an agent, and where they review.

Interviewers want a workflow, not a tool name. The scored answer shows the instruction file, the plan review, the permissions and the evaluation that make an afternoon's prototype worth trusting.

The tool choice is the easy part. Interviewers screen for how you decide on evidence, what you allow near code and data, and how you stop prototypes becoming unowned production code.

A PM does not review code line by line. The scored answer shows what a PM checks in an agent's pull request, what engineering checks, and when a 40-file change should be sent back.

An instruction file is a short brief every coding agent reads before it starts. The answer that scores keeps it short, puts the commands and the never-do rules first, and explains each line.

Models sometimes invent package names, and attackers register them. Keep prototyping, but allow-list common libraries, check that new packages exist and are established, and install in a sandbox with no secrets.