Agent reliability math: why success collapses over long task chains
Post257 words
Agent tasks are serial systems, so success is s^n: at 99% per step, 20 steps give 0.82 and 100 steps give 0.37. Real errors are correlated, so outcomes are often worse than the formula.
Key claims
At 99% per step, 20 steps = 0.82 success
At 99% per step, 100 steps = 0.37 success
Typical web workflows hit 20-50 micro-steps
Agent reliability math: why success rate collapses over long task chains
Most “agent” tasks are serial systems: the job succeeds only if every required step succeeds (tool call, API request, extraction, decision, write-back). In reliability engineering, a pure series system has overall reliability equal to the product of step reliabilities.
If each step succeeds with probability s, and the task needs n dependent steps:
➤Task success = s^n
➤Task failure = 1−s^n
That’s the whole problem: even small per-step error rates become large over long chains.
If an agent is 99% correct per step (s = 0.99):
20 steps = 0.82 success (≈18% fail)
100 steps = 0.37 success (≈63% fail)
Patronus AI uses the same compounding logic in its risk modeling examples (e.g., 1−(0.995)^20 for per-step error 0.5% across 20 steps).
Take a typical workflow like “collect info from a website - extract fields - validate - fill a form - submit - confirm.” It’s easy to hit 20-50 micro-steps (navigation actions + parsing + tool calls + retries). Even with strong single-step accuracy, the end-to-end probability drops fast because one wrong step can poison the rest (wrong field, wrong page, wrong assumption).
This is also why long-horizon agent research explicitly calls out that a low error rate that looks good on short benchmarks becomes “inevitable failure” when you require thousands of correct steps in a row.
The formula assumes steps are independent. In practice, agents often do worse because errors are correlated: one bad extraction leads to wrong tool inputs, which creates more failures downstream.