Across 21 comparable benchmarks, GPT-5.1 has better reported results on 20 and worse on 1. 2 further shared results are listed below, each with the reason it is not counted.
Moderate evidence21 of 23 shared results countedHow we compare
Price basis: Standard rates. The price section below shows the applicable periods.
| Fact | Haiku 4.5 | GPT-5.1 |
|---|---|---|
| Context window | 200K tokens | 400K tokens |
| Input / 1M | $1.00 | $1.25 |
| Output / 1M | $5.00 | $10.00 |
| Blended (4:1) | $1.15 | $2.19 |
| Status | Available | Deprecated |
| Released | October 15, 2025 | November 12, 2025 |
Blended is 4:1 input to output with 90% cache hits at listed rates — set your own mix.
Mean of 15 shared benchmarks grouped by capability on a 0–100 scale. How averages are built
GPT-5.1 ahead on 5 of 5 counted
GPT-5.1 ahead on 1 of 1 counted
GPT-5.1 ahead on 2 of 3 counted
GPT-5.1 ahead on 3 of 3 counted
GPT-5.1 ahead on 1 of 1 counted
GPT-5.1 ahead on 2 of 2 counted
GPT-5.1 ahead on 2 of 2 counted
No shared comparable results for Research & search, Long context, Cybersecurity.
Published scores, with matching modes where available. Gaps show Haiku 4.5 relative to GPT-5.1. Reading the results
| Benchmark | Haiku 4.5 | GPT-5.1 | Gap |
|---|---|---|---|
| LiveCodeBenchAlgorithmic Coding · Pass @K | 62.0Extended · no tools | 86.8≠Thinking High · no tools | 0.71×#69 vs #19 of 113 |
| SWE-Bench Pro - PublicRepository Engineering · Accuracy | 39.5Extended · tools | 50.8≠Thinking High · no tools | −11.3 pts#63 vs #48 of 66 |
| SWE-bench VerifiedRepository Engineering · Accuracy | 73.3Thinking · tools · 128K | 76.3≠Thinking High · no tools | −3.0 pts#47 vs #32 of 108 |
| Terminal Bench HardAgentic Development · Accuracy | 27.3Normal · tools | 45.5≠Thinking High · tools | −18.2 pts#74 vs #20 of 157 |
| Vibe Code Bench v1.1Code Generation & Editing · Pass rate (%) | 11.4Thinking · tools | 24.6≠Thinking High · tools | −13.2 pts#102 vs #82 of 121 |
| ARC-AGI-1Abstract Generalization · Score (%) | 47.7Extended · no tools | 72.8≠Thinking High · no tools | −25.2 pts#67 vs #51 of 87 |
| ARC-AGI-2Abstract Generalization · Score (% solved); cost per task (USD)Not counted · Speed and cost telemetry is reported, not scored | 4.5Extended · no tools | 17.6≠Thinking High · no tools | 0.26×#70 vs #55 of 80 |
| Vending-Bench 2Logical Planning · Money Balance (USD)Not counted · Speed and cost telemetry is reported, not scored | 459Unspecified Reasoning · tools | 1473Unspecified Reasoning · tools | 0.31×#58 vs #51 of 64 |
| AIME2025Mathematics · Accuracy | 96.3Thinking · tools · 128K | 94.2≠Thinking High · tools | +2.1 pts#18 vs #26 of 86 |
| FrontierMathMathematics · Accuracy | 4.1Normal · no tools | 26.7≠Thinking High · tools | −22.6 pts#32 vs #12 of 48 |
| FrontierMath - Tier 4Mathematics · Accuracy | 2.1Thinking · no tools · 32K | 12.5≠Thinking High · no tools | −10.4 pts#31 vs #13 of 50 |
| GPQA DiamondScientific Reasoning · Accuracy | 73.3Extended · no tools | 88.1≠Thinking · no tools | −14.8 pts#126 vs #65 of 200 |
| MedCodeClinical Workflows · Accuracy (%) | 32.7Thinking · tools | 52.7≠Thinking High · tools | −20.0 pts#81 vs #13 of 97 |
| MedScribeClinical Workflows · Rubric pass rate (%) | 85.2Thinking · tools | 88.1≠Thinking High · tools | −2.9 pts#30 vs #14 of 99 |
| MCP-AtlasTool Orchestration · Pass rate / claim coverage | 40.2Normal · tools | 50.1≠Thinking High · tools | −9.9 pts#55 vs #53 of 55 |
| MMMUVisual Understanding · Accuracy | 73.2Thinking · no tools · 128K | 85.4≠Thinking High · no tools | −12.2 pts#27 vs #1 of 62 |
| MMMU-ProVisual Understanding · Accuracy | 58.6Thinking · no tools | 79.0Thinking · no tools | −20.4 pts#109 vs #38 of 136 |
| SAGEService Workflows · Accuracy (%) | 31.8Thinking · tools | 43.2≠Thinking High · tools | −11.4 pts#71 vs #51 of 82 |
| τ³-BankingService Workflows · Score | 9.3Thinking · tools | 15.9≠Thinking High · tools | 0.58×#95 vs #69 of 120 |
| HLEKnowledge Exams · Accuracy | 9.7Extended · no tools | 42.7≠Thinking High · tools + web | −33.0 pts#129 vs #74 of 151 |
| MMLU-ProKnowledge Exams · Accuracy | 80.0Normal · no tools | 87.0≠Thinking High · no tools | −7.0 pts#62 vs #16 of 153 |
| ECICapability Indices · ECI score (capability index, higher is better) | 142Reported best | 150Reported best | 0.95×#90 vs #51 of 168 |
| LiveBenchCross-capability Suites · Accuracy | 61.3Thinking · no tools · 64K | 72.0≠Thinking High · no tools | −10.7 pts#82 vs #50 of 106 |
Algorithmic Coding · Pass @K
Repository Engineering · Accuracy
Repository Engineering · Accuracy
Agentic Development · Accuracy
Code Generation & Editing · Pass rate (%)
Abstract Generalization · Score (%)
Abstract Generalization · Score (% solved); cost per task (USD)
Not counted · Speed and cost telemetry is reported, not scored
Logical Planning · Money Balance (USD)
Not counted · Speed and cost telemetry is reported, not scored
Mathematics · Accuracy
Mathematics · Accuracy
Mathematics · Accuracy
Scientific Reasoning · Accuracy
Clinical Workflows · Accuracy (%)
Clinical Workflows · Rubric pass rate (%)
Tool Orchestration · Pass rate / claim coverage
Service Workflows · Accuracy (%)
Service Workflows · Score
A dash means no result is available.
| Benchmark | Haiku 4.5 | GPT-5.1 |
|---|---|---|
| CritPt | — | 4.9 |
| IMO-ProofBench Advanced | — | 7.1 |
| MathArena Apex | — | 1.0 |
| SciCode | 42.2 | — |
| Code Migration | 7.5 | — |
| GSO | — | 13.7 |
| IC SWE-Lancer (Diamond) | — | 69.7 |
| SWE-Bench Pro V2 Hard | 25.5 | — |
| Terminal Bench 2.0 | — | 47.6 |
| Terminal-Bench 1.0 | 41.8 | — |
| WeirdML v2 | — | 60.8 |
| Claw Bench | 89.4 | — |
| OSWorld 2.1 | 15.7 | — |
| Pinch Bench | 82.0 | — |
| PinchBench v2 | 67.7 | — |
| ExploitBench | 13.7 | — |
| AA Intelligence Index v5.0.1 | 7.4 | — |
| LiveBench 2.0 | 46.0 | — |
| Text Arena (Coding) | — | 1387 |
| Vals Index | 22.9 | — |
| IF Bench | 54.3 | — |
| Context Arena | 36.3 | — |
| GDP.pdf | 3.8 | — |
| VPCT | — | 58.7 |
| AA-AnalystAgent | 15.0 | — |
| AutomationBench | 0.46 | — |
| EMB (Excel Modeling) | 22.7 | — |
| Finance Agent v2 | 31.0 | — |
| Harvey Lab-AA | 61.0 | — |
| Harvey's Legal Agent Benchmark | 0.83 | — |
| Legal Research Bench | 10.6 | — |
| Public Benefits Bench v1.1 | 54.3 | — |
| τ²-Bench | 33.0 | — |
| BALROG | 31.2 | — |
| SimpleBench | — | 53.2 |
| BrowseComp | — | 50.8 |
| GAIA | 56.4 | — |
Time-of-day providers use peak-hour rates; fixed-price providers use standard rates. Off-peak discounts remain in the published details.
Drag the handle above the bar for cache hits, the one below it for the input : output mix · 5% steps
Blended price = effective input rate at 90% cache hits × 80% + output rate × 20%
Haiku 4.5 costs47%lessthan GPT-5.1 per million tokens.Using standard rates at this token mix.
Haiku 4.5: Standard pricing. Anthropic official pricing (platform.claude.com/docs/en/about-claude/pricing) verified 2026-11-13
GPT-5.1: Standard pricing. OpenAI official pricing (developers.openai.com/api/docs/pricing) verified 2026-11-13; standard tier; Deprecated; shutdown 2027-04-01 (replacement: gpt-6-sol)
| Price / 1M tokens | Haiku 4.5 | GPT-5.1 | Ratio |
|---|---|---|---|
| Input | $1.00 | $1.25 | 0.80× |
| Output | $5.00 | $10.00 | 0.50× |
| Cached input | $0.10 | $0.125 | 0.80× |
| Cache write | — | — | — |
| Blended | $1.15 | $2.19 | 0.53× |
Estimates use the selected token mix. Cache writes, tool fees and long-context surcharges are excluded.
Reported cost per task, paired with the score from the same run. Settings and sources may differ; costs do not affect the verdict.
Haiku 4.5 relative to GPT-5.1
No matching recorded settings for a cost summary.
Summary: 0 of 1 shared benchmarks have matching recorded settings and sources. The average is a geometric mean of 0 positive-cost ratios, one equal weight per benchmark. This benchmark set can include related tests and indexes; it is not a workload cost estimate.
Settings or sources differ or are not recorded; interpret the comparison with care.
Haiku 4.5
GPT-5.1
| Benchmark | Haiku 4.5 | GPT-5.1 |
|---|---|---|
| AA Intelligence Index v5.0.1 | Not reported | |
| ARC-AGI-2 | Not reported | |
| AutomationBench | $0.07 / task Average · Pass Rate: 0.46 Reported best (effort unspecified) · zapier-private-heldout | Not reported |
| LiveBench 2.0 | $0.934 / task Average · Overall score (equal-weight mean of 7 suite scores, 0-100): 46.0 Max (task-specific tools) | Not reported |
| MedCode | Not reported | |
| MedScribe | Not reported | |
| Public Benefits Bench v1.1 | $0.2245 / task Average · Criterion pass rate (%): 54.3 Unspecified Reasoning (With Tools) | Not reported |
| SAGE | Not reported | |
| Vibe Code Bench v1.1 | Not reported |
| Attribute | Haiku 4.5 | GPT-5.1 |
|---|---|---|
| Availability | Available | Deprecated |
| Provider | Anthropic | OpenAI |
| Model type | Multimodal model | Reasoning model |
| First release | October 15, 2025 | November 12, 2025 |
| Context window | 200K tokens | 400K tokens |
| Max output | 64K tokens | 128K tokens |
| Architecture | Undisclosed | Undisclosed |
| Documented capabilities | Reasoning, Multilingual | Reasoning, Multilingual |
| Input modalities | Text, Image | Text, Image |
| Output modalities | Text | Text |
| Weights | Closed / API only | Closed / API only |
| Commercial use | Proprietary | Proprietary |
These are reported-score comparisons, not controlled head-to-head tests.
What people found comparing these models, or using either one. Comments also appear on the models' own pages.
Used Haiku 4.5 or GPT-5.1? Sign in with Google to share what worked, what did not, and which API or setup you tested.
No community comments on either model yet.
Alternatives with the most benchmark overlap for this matchup.