Benchmarks · no Elo
SubagentBench
Best model for each sub-agent task (explore, webfetch, compaction), on real prompts from our todos.
scoring: judge scores · todo-bench/
Explore sub-agent
r7 · 2026-10-08 · 2 runs each10 real code-search questions our agents asked (the ones where models differed most), 2 runs each.
ModelScoreHalluc. / runTimeCost (list price, per run)
Claude Haiku 5.56.651.7751s$0.011
GPT-6 Sol6.221.932m 37s$0.35
Claude Opus 5.56.132.732m 20s$0.71
GPT-6 Luna5.602.081m 35s$0.014
Claude Haiku 4.52.775.001m 54s$0.22
Webfetch extraction
r2 · 2026-10-08177 real url + question pairs from our todos; every model reads the same page snapshot.
ModelScoreHalluc. / runTimeCost (list price, all 177)
GPT-6 Sol9.080.066s$3.67
Claude Haiku 5.58.570.455s$0.55
Claude Opus 5.58.530.538s$12.91
GPT-6 Luna8.410.215s$0.18
Claude Haiku 4.56.671.044s$2.15
Compaction summary
r4 · 2026-10-0820 real long conversations. The question: can a fresh agent continue from the summary alone?
ModelScoreHalluc. / runTimeCost (list price)
GPT-6 Sol8.950.004m 27s—
Claude Sonnet 5.58.530.7052s—
GPT-6 Luna8.150.721m 46s—
Claude Haiku 5.56.882.3037s—
Claude Haiku 4.54.105.0343s—
Scores 1–10 from two blinded judges (Claude Opus 5.5 and GPT-6 Sol), averaged; on close calls they can disagree (explore, webfetch). Time is the median per call. Cost = tokens × OpenRouter list price (uncached input for webfetch; explore counts its real in-loop cache reads). Scores compare within one round only.