Coding benchmarks
DeepSWE
Illustrative snapshot from the public DeepSWE leaderboard (mini-swe-agent). Scores and costs are not live OpenCodex metering.
| Model | Effort | Score | Cost / task | Score / $ | Use case |
|---|---|---|---|---|---|
| gpt-5.6-sol | max | 73% ±3 | $8.39 | 8.7 | Frontier, Planner |
| claude-fable-5 | max | 70% ±4 | $21.63 | 3.2 | Planner, Frontier |
| gpt-5.6-terra | max | 70% ±3 | $4.95 | 14.1 | Workhorse, Frontier |
| gpt-5.6-luna | max | 67% ±4 | $3.03 | 22.1 | Workhorse, Cheap subagent |
| gpt-5.5 | xhigh | 67% ±6 | $7.23 | 9.3 | Frontier, Workhorse |
| claude-opus-4.8 | max | 59% ±2 | $13.22 | 4.5 | Frontier, Planner |
| claude-sonnet-5 | max | 54% ±4 | $26.4 | 2.0 | Planner |
| grok-4.5 | high | 54% ±2 | $2.42 | 22.3 | Workhorse, Cheap subagent |
| muse-spark-1.1Best value | xhigh | 53% ±3 | $2.36 | 22.5 | Workhorse, Cheap subagent |
| gpt-5.4 | xhigh | 52% ±2 | $5.65 | 9.2 | Workhorse |
| glm-5.2 | max | 44% ±2 | $3.92 | 11.2 | Workhorse, Cheap subagent |
| gemini-3.5-flash | medium | 37% ±2 | $7.34 | 5.0 | Fast |
| kimi-k2.7-code | — | 31% ±1 | $2.82 | 11.0 | Cheap subagent, Workhorse |
| claude-sonnet-4.6 | high | 30% ±4 | $5.52 | 5.4 | Workhorse |
| gemini-3.1-pro | high | 12% ±2 | $9.48 | 1.3 | Frontier |
AA Coding Agent
Artificial Analysis Coding Agent Index snapshot (post GPT-5.6 GA, 2026-07-09). See provenance.url. Not live OpenCodex metering.
| Model | Effort | Score | Cost / task | Score / $ | Use case |
|---|---|---|---|---|---|
| composer-2.5Best value | std | 62 | $0.07 | 885.7 | Workhorse, Cheap subagent, Fast |
| composer-2.5 | fast | 62 | $0.44 | 140.9 | Workhorse, Fast |
| claude-opus-4.7 | max | 66 | $4.1 | 16.1 | Frontier, Planner |
| gpt-5.5 | xhigh | 65 | $4.82 | 13.5 | Frontier, Workhorse |
| gpt-5.6-luna | max | 75 | $1.55 | 48.4 | Workhorse, Cheap subagent |
| gpt-5.6-terra | max | 77 | $2.4 | 32.1 | Workhorse |
| gpt-5.6-sol | max | 80 | $3.9 | 20.5 | Frontier, Planner |
| grok-4.5 | high | 76 | $2.54 | 29.9 | Workhorse, Cheap subagent |
| claude-sonnet-4.6 | high | 58 | $2.8 | 20.7 | Workhorse |
| claude-fable-5 | high | 64 | $9.5 | 6.7 | Planner, Frontier |
FrontierCode
Illustrative snapshot from cognition.com/frontiercode (FrontierCode 1.1 Main, Jul 2026). Score = mergeability rubric; cost = mean USD per rollout. Best vs all reasoning modes match Cognition’s leaderboard toggle. Not live OpenCodex metering.
| Model | Effort | Score | Cost / task | Score / $ | Use case |
|---|---|---|---|---|---|
| claude-fable-5 | xhigh | 53.5% | $13.09 | 4.1 | Frontier, Workhorse, Planner |
| claude-fable-5 | high | 52.7% | $9.48 | 5.6 | Frontier, Workhorse, Planner |
| claude-fable-5 | max | 51.6% | $19.07 | 2.7 | Frontier, Workhorse, Planner |
| claude-fable-5 | medium | 49.8% | $7.15 | 7.0 | Frontier, Workhorse, Planner |
| claude-fable-5 | low | 48% | $4.92 | 9.8 | Frontier, Workhorse, Planner |
| gpt-5.6-sol | max | 47.5% | $6.3 | 7.5 | Frontier, Workhorse, Planner |
| gpt-5.6-sol | xhigh | 46.8% | $5 | 9.4 | Frontier, Workhorse, Planner |
| claude-opus-4.8 | max | 46.5% | $9.62 | 4.8 | Frontier, Workhorse, Planner |
| claude-opus-4.8 | xhigh | 45.5% | $6.78 | 6.7 | Frontier, Workhorse, Planner |
| gpt-5.6-sol | high | 45.1% | $4.1 | 11.0 | Frontier, Workhorse, Planner |
| gpt-5.5 | xhigh | 43% | $4.03 | 10.7 | Frontier, Workhorse |
| claude-sonnet-5 | xhigh | 42.7% | $9.06 | 4.7 | Frontier, Workhorse |
| grok-4.5 | high | 42.4% | $1.3 | 32.6 | Workhorse, Cheap subagent, Fast |
| claude-sonnet-5 | max | 42.4% | $15.4 | 2.8 | Frontier, Workhorse |
| swe-1.7 | — | 42.3% | $1.97 | 21.5 | Workhorse, Cheap subagent |
| grok-4.5 | medium | 41.9% | $1.2 | 34.9 | Workhorse, Cheap subagent, Fast |
| gpt-5.6-terra | max | 41.3% | $2.3 | 18.0 | Workhorse |
| claude-opus-4.8 | high | 41% | $4.28 | 9.6 | Frontier, Workhorse, Planner |
| gpt-5.5 | high | 40.6% | $3.12 | 13.0 | Frontier, Workhorse |
| claude-opus-4.8 | medium | 40.5% | $3.7 | 10.9 | Frontier, Workhorse, Planner |
| gpt-5.6-sol | medium | 39.9% | $3.1 | 12.9 | Frontier, Workhorse, Planner |
| gpt-5.6-luna | max | 39.8% | $1.8 | 22.1 | Workhorse, Cheap subagent |
| claude-sonnet-5 | high | 39.4% | $5.49 | 7.2 | Frontier, Workhorse |
| gpt-5.6-luna | xhigh | 38.9% | $1.5 | 25.9 | Workhorse, Cheap subagent |
| gpt-5.6-terra | xhigh | 38.7% | $1.6 | 24.2 | Workhorse, Cheap subagent |
| claude-opus-4.7 | max | 38.5% | $9.09 | 4.2 | Workhorse |
| grok-4.5 | low | 37.9% | $0.8 | 47.4 | Workhorse, Cheap subagent, Fast |
| gpt-5.6-terra | high | 36.9% | $1.4 | 26.4 | Workhorse, Cheap subagent, Fast |
| gpt-5.5 | medium | 36.6% | $2.35 | 15.6 | Frontier, Workhorse |
| gpt-5.6-luna | high | 35.9% | $1.1 | 32.6 | Workhorse, Cheap subagent, Fast |
| gpt-5.6-sol | low | 35.4% | $2.1 | 16.9 | Frontier, Workhorse, Planner |
| claude-sonnet-5 | medium | 35.2% | $3.43 | 10.3 | Frontier, Workhorse |
| claude-opus-4.7 | high | 35.2% | $5.03 | 7.0 | Workhorse |
| claude-opus-4.8 | low | 35.1% | $2.68 | 13.1 | Frontier, Workhorse, Planner |
| claude-opus-4.7 | xhigh | 34.9% | $6.87 | 5.1 | Workhorse |
| gpt-5.6-terra | medium | 33.9% | $0.9 | 37.7 | Workhorse, Cheap subagent, Fast |
| gpt-5.5 | low | 30.6% | $1.65 | 18.5 | Frontier, Workhorse, Cheap subagent |
| kimi-k2.7 | — | 30.1% | $3.01 | 10.0 | — |
| claude-opus-4.7 | medium | 28.9% | $3.22 | 9.0 | Workhorse |
| claude-sonnet-5 | low | 28.7% | $2.15 | 13.3 | Frontier, Workhorse |
| claude-opus-4.7 | low | 27.6% | $2.47 | 11.2 | Workhorse |
| gpt-5.6-luna | medium | 25.7% | $0.6 | 42.8 | Workhorse, Cheap subagent, Fast |
| composer-2.5 | — | 25.6% | $3.09 | 8.3 | — |
| glm-5.2 | — | 24.5% | $2.47 | 9.9 | — |
| gpt-5.6-terra | low | 24.1% | $0.5 | 48.2 | Workhorse, Cheap subagent, Fast |
| deepseek-v4-pro | — | 17.6% | $1.6 | 11.0 | Cheap subagent |
| gpt-5.6-lunaBest value | low | 15.4% | $0.2 | 77.0 | Workhorse, Cheap subagent, Fast |
| minimax-m3 | — | 14.7% | $0.7 | 21.0 | Cheap subagent, Fast |
| inkling | 0.99 | 14% | $3.6 | 3.9 | — |
| qwen-3.7-plus | — | 10.2% | $0.2 | 51.0 | Cheap subagent, Fast |
FrontierSWE
Illustrative snapshot from frontierswe.com (Mean@5 dominance). X-axis uses blended public API $/MTok as a relative cost proxy — not measured $/task. Not live OpenCodex metering.
Costs on this board are estimates or API-price blends — score/$ ranking and Best value are disabled until every row has a measured cost-per-task from the source.
| Model | Effort | Score | Cost / task | Use case |
|---|---|---|---|---|
| claude-fable-5 | claude-code | 89% | $30 | Planner, Frontier |
| grok-4.5 | grok-cli | 78% | $4 | Workhorse, Frontier |
| claude-opus-4.8 | claude-code | 73% | $15 | Frontier, Planner |
| glm-5.2 | claude-code | 72% | $1.98 | Workhorse, Cheap subagent |
| gpt-5.5 | codex | 70% | $17.5 | Frontier, Workhorse |
| claude-opus-4.7 | claude-code | 61% | $15 | Frontier |
| claude-opus-4.6 | claude-code | 53% | $15 | Workhorse |
| gpt-5.4 | codex | 51% | $8.75 | Workhorse |
| composer-2.5 | cursor-cli | 38% | $1 | Workhorse, Cheap subagent, Fast |
| gemini-3.1-pro | gemini-cli | 37% | $8.75 | Frontier |
| glm-5.1 | claude-code | 29% | $2.9 | Workhorse |
| deepseek-v4-pro | claude-code | 27% | $2.4 | Cheap subagent, Workhorse |
| kimi-k2.6 | kimi-cli | 25% | $2.13 | Cheap subagent |
| kimi-k2.5 | kimi-cli | 25% | $2.13 | Cheap subagent |
| qwen3.6-plus | qwen-code | 21% | $1.75 | Cheap subagent |
Program Bench
Snapshot from programbench.com extended leaderboard (mini-SWE-agent, 200 tasks). Score = almost-resolved (≥95% tests). Costs = published average API $/task. Not live OpenCodex metering.
| Model | Effort | Score | Cost / task | Score / $ | Use case |
|---|---|---|---|---|---|
| gpt-5.5Best value | xhigh | 13.5% | $8.85 | 1.5 | Frontier, Planner |
| gpt-5.5 | high | 5% | $3.65 | 1.4 | Frontier, Workhorse |
| claude-opus-4.7 | xhigh | 4.5% | $10.96 | 0.4 | Planner, Frontier |
| claude-opus-4.7 | — | 3% | $3.81 | 0.8 | Frontier |
| claude-opus-4.6 | — | 2.5% | $11.38 | 0.2 | Workhorse |
| gpt-5.5 | — | 1.5% | $1.21 | 1.2 | Workhorse |
| claude-sonnet-4.6 | — | 1% | $26.73 | 0.0 | Workhorse |
| gpt-5.4 | — | 0% | $0.33 | 0.0 | Fast, Cheap subagent |
| gemini-3.1-pro | — | 0% | $1.51 | 0.0 | Frontier |
| gemini-3-flash | — | 0% | $0.3 | 0.0 | Fast, Cheap subagent |
| claude-haiku-4.5 | — | 0% | $0.8 | 0.0 | Fast, Cheap subagent |
| gpt-5.4-mini | — | 0% | $0.04 | 0.0 | Fast, Cheap subagent |
| gpt-5-mini | — | 0% | $0.03 | 0.0 | Fast, Cheap subagent |
SWE Marathon
Illustrative snapshot from swe-marathon.org (pass@1 on 20 ultra-long-horizon tasks). Cost axis is an estimated relative run cost (long trajectories); not published $/task. Not live OpenCodex metering.
Costs on this board are estimates or API-price blends — score/$ ranking and Best value are disabled until every row has a measured cost-per-task from the source.
| Model | Effort | Score | Cost / task | Use case |
|---|---|---|---|---|
| grok-4.5 | grok-build | 29% | $8 | Frontier, Planner |
| claude-opus-4.8 | claude-code | 26% | $28 | Frontier, Planner |
| claude-fable-5 | claude-code | 24% | $45 | Planner, Frontier |
| claude-opus-4.7 | claude-code | 16% | $22 | Frontier |
| glm-5.2 | claude-code | 13% | $6 | Workhorse, Cheap subagent |
| gpt-5.5 | codex | 12% | $18 | Frontier, Workhorse |
| claude-opus-4.7 | terminus-2 | 11% | $20 | Workhorse |
| gemini-3.5-flash | gemini-cli | 7% | $5 | Fast |
| gpt-5.5 | terminus-2 | 6% | $16 | Workhorse |
| gemini-3.1-pro | terminus-2 | 4% | $12 | Frontier |
| deepseek-v4-pro | terminus-2 | 4% | $7 | Cheap subagent |
| gemini-3.1-pro | gemini-cli | 2% | $12 | Frontier |
| glm-5.1 | terminus-2 | 1% | $5 | Workhorse |
| minimax-m2.7 | terminus-2 | 0% | $3 | Cheap subagent |
| kimi-k2.6 | kimi-cli | 0% | $4 | Cheap subagent |