Resolve rate
The percentage of benchmark issues a system fixes such that all hidden tests pass; the headline metric for SWE-bench-style benchmarks.
Resolve rate is binary per task — the patch either makes the fail-to-pass tests pass without breaking pass-to-pass tests, or it does not. It ignores code quality, style and whether the fix is the one a maintainer would merge. Confidence intervals matter: SWE-bench Pro reports ±3 points at the top, so models within a few points are statistically tied.