DECISION TOOL · FREE TO USE
Agent reliability calculator
Ten steps at 95% each finish together only 60% of the time. See what retries can and cannot fix.
P(task) = p^kA retry only happens if a step's failure is noticed: a schema check, a failed tool call, a validator. Confident wrong steps are never retried.
Solid: with your retries. Dashed: p to the power k, no retries.
Cutting the task from 10 to 5 steps takes success to 77.4%. Making steps deterministic (a lookup, a validated API call) usually beats buying a better model.
FROM THE RESULT TO THE DECISION
How to reason about it
Reliability compounds against agents. If each of ten steps succeeds 95% of the time and the steps are independent, the whole task succeeds about 60% of the time. At 99% per step it is about 90%.
Retries help, but only with failures the system can detect: a tool error, a schema violation, a failed check. A confident wrong step is never retried, and every later step builds on it.
Before buying a better model, cut steps and make the ones you keep deterministic or verifiable. Fewer, checkable steps usually beat a smarter planner.
Questions about this tool
Rarely. Failures cluster: a bad retrieval early makes later steps likelier to fail. The independent model is a lower bound on how badly long chains behave, and a useful way to argue for shorter ones.