Published benchmark scores

Published benchmark scores for the models we route, GPQA Diamond, SWE-bench, HLE and more, each linked to its source. Reported by makers, not measured by us.

What each model’s maker published for the benchmarks below, one score per row, each linked to the page it comes from. We don’t run these tests or change the numbers. A maker’s own score is not an independent measurement: effort, tools and test harnesses differ between makers, so compare with care.

GPQA Diamond

Graduate-level biology, physics and chemistry questions written to resist web search; the hardest 198. GPQA Diamond

ModelScoreSettingSource
google/gemini-3.1-pro94.3%no tools, thinking highMaker, 2026-02-19
Anthropic: Claude Opus 4.794.2%avg of 10 trialsMaker, 2026-04-16
Anthropic: Claude Opus 4.893.6%avg of 25 trialsMaker, 2026-05-28
MoonshotAI: Kimi K393.5%maxMaker, 2026-11-03
Qwen: Qwen3.7 Max92.4%—Maker, 2026-11-03
Z.ai: GLM 5.291.2%—Maker, 2026-11-03
MoonshotAI: Kimi K2.690.5%thinkingMaker, 2026-11-03
Qwen: Qwen3.7 Plus90.3%—Maker, 2026-11-03
DeepSeek: DeepSeek V4 Pro 042390.1%think maxMaker, 2026-11-03
Anthropic: Claude Sonnet 4.689.9%avg of 10 trialsMaker, 2026-02-17
Qwen: Qwen3.8 27B89.2%—Maker, 2026-11-03
DeepSeek: DeepSeek V4 Flash 042388.1%think maxMaker, 2026-11-03
Qwen: Qwen3.6 27B87.8%—Maker, 2026-11-03
Google: Gemini 3.1 Flash Lite86.9%no tools, thinking highMaker, 2026-03-03
Qwen: Qwen3.5-122B-A10B86.6%—Maker, 2026-11-03
Z.ai: GLM 5.186.2%—Maker, 2026-11-03
Qwen: Qwen3.6 35B A3B86%—Maker, 2026-11-03
Qwen: Qwen3.5-27B85.5%—Maker, 2026-11-03
Google: Gemma 4 31B84.3%—Maker, 2026-07-30
Qwen: Qwen3.5-35B-A3B84.2%—Maker, 2026-11-03
Google: Gemma 4 26B A4B82.3%—Maker, 2026-07-30
Qwen: Qwen3.5-9B81.7%—Maker, 2026-11-03
openai/gpt-oss-120b80.9%with tools, reasoning highMaker, 2025-08-05
openai/gpt-oss-120b80.1%no tools, reasoning highMaker, 2025-08-05
Kimi K2 Instruct75.1%Avg@8Maker, 2026-11-03
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP869.8%0-shotMaker, 2025-04-05
DeepSeek R1 Distill Qwen3 8B 052861.1%—Maker, 2026-11-03
google/gemma-4-E4B-it58.6%—Maker, 2026-07-30
meta-llama/Llama-4-Scout-17B-16E-Instruct57.2%0-shotMaker, 2025-04-05
meta-llama/Meta-Llama-3.3-70B-Instruct50.5%CoT, 0-shotMaker, 2024-12-06
mistralai/Mistral-Small-3.2-24B-Instruct-250646.13%5-shot CoTMaker, 2026-11-03

Humanity’s Last Exam

Expert-written questions at the frontier of human knowledge, across many subjects. Humanity’s Last Exam

ModelScoreSettingSource
Anthropic: Claude Fable 5.165%with toolsMaker, 2026-09-01
Claude Opus 564.7%with toolsMaker, 2026-07-24
Z.ai: GLM 5.362.5%with toolsMaker, 2026-11-03
Anthropic: Claude Fable 5.160.9%no toolsMaker, 2026-09-01
DeepSeek: DeepSeek V4 Pro 081360%with toolsMaker, 2026-11-03
Anthropic: Claude Opus 4.857.9%with toolsMaker, 2026-05-28
Anthropic: Claude Sonnet 557.4%with toolsMaker, 2026-06-30
Claude Opus 556.3%no toolsMaker, 2026-07-24
MoonshotAI: Kimi K356%max; with toolsMaker, 2026-11-03
Anthropic: Claude Opus 4.754.7%with toolsMaker, 2026-04-16
Z.ai: GLM 5.254.7%text-only subset; with toolsMaker, 2026-11-03
MoonshotAI: Kimi K2.654%thinking; with toolsMaker, 2026-11-03
Qwen: Qwen3.7 Max53.5%with toolsMaker, 2026-11-03
Z.ai: GLM 5.152.3%with toolsMaker, 2026-11-03
DeepSeek: DeepSeek V4 Flash 073151.5%with toolsMaker, 2026-11-03
google/gemini-3.1-pro51.4%search (blocklist) + code, thinking highMaker, 2026-02-19
Anthropic: Claude Opus 4.849.8%no toolsMaker, 2026-05-28
Anthropic: Claude Sonnet 4.649%with toolsMaker, 2026-02-17
Qwen: Qwen3.5-27B48.5%with toolsMaker, 2026-11-03
DeepSeek: DeepSeek V4 Pro 042348.2%think max; with toolsMaker, 2026-11-03
Qwen: Qwen3.5-122B-A10B47.5%with toolsMaker, 2026-11-03
Qwen: Qwen3.5-35B-A3B47.4%with toolsMaker, 2026-11-03
Anthropic: Claude Opus 4.746.9%no toolsMaker, 2026-04-16
DeepSeek: DeepSeek V4 Flash 042345.1%think max; with toolsMaker, 2026-11-03
google/gemini-3.1-pro44.4%no tools, thinking highMaker, 2026-02-19
MoonshotAI: Kimi K343.5%max; no toolsMaker, 2026-11-03
Anthropic: Claude Sonnet 543.2%no toolsMaker, 2026-06-30
DeepSeek: DeepSeek V4 Pro 081342.7%no toolsMaker, 2026-11-03
Qwen: Qwen3.7 Max41.4%no toolsMaker, 2026-11-03
Z.ai: GLM 5.240.5%text-only subset; no toolsMaker, 2026-11-03
Google: Gemini 3.5 Flash40.2%—Maker, 2026-05-19
DeepSeek: DeepSeek V4 Flash 073137.8%no toolsMaker, 2026-11-03
DeepSeek: DeepSeek V4 Pro 042337.7%think max; no toolsMaker, 2026-11-03
DeepSeek: DeepSeek V4 Flash 042334.8%think max; no toolsMaker, 2026-11-03
MoonshotAI: Kimi K2.634.7%thinking; no toolsMaker, 2026-11-03
Qwen: Qwen3.7 Plus34.7%judged by GPT-4oMaker, 2026-11-03
Anthropic: Claude Sonnet 4.633.2%no toolsMaker, 2026-02-17
Z.ai: GLM 5.131%no toolsMaker, 2026-11-03
Qwen: Qwen3.8 27B30.8%judged by GPT-4oMaker, 2026-11-03
Google: Gemma 4 31B26.5%with searchMaker, 2026-07-30
Qwen: Qwen3.5-122B-A10B25.3%no tools (w/ CoT)Maker, 2026-11-03
Qwen: Qwen3.5-27B24.3%no tools (w/ CoT)Maker, 2026-11-03
Qwen: Qwen3.6 27B24%—Maker, 2026-11-03
Qwen: Qwen3.5-35B-A3B22.4%no tools (w/ CoT)Maker, 2026-11-03
Qwen: Qwen3.6 35B A3B21.4%—Maker, 2026-11-03
Google: Gemma 4 31B19.5%no toolsMaker, 2026-07-30
openai/gpt-oss-120b19%with tools, reasoning highMaker, 2025-08-05
Google: Gemma 4 26B A4B17.2%with searchMaker, 2026-07-30
Google: Gemini 3.1 Flash Lite16%no tools, thinking highMaker, 2026-03-03
openai/gpt-oss-120b14.9%no tools, reasoning highMaker, 2025-08-05
Google: Gemma 4 26B A4B8.7%no toolsMaker, 2026-07-30
Kimi K2 Instruct4.7%text onlyMaker, 2026-11-03

ARC-AGI-2

Visual puzzles that test learning a new rule from a few examples. ARC-AGI-2

ModelScoreSettingSource
Claude Opus 590.42%max effort, ARC Prize verified, semi-privateMaker, 2026-07-24
Anthropic: Claude Fable 5.190%max effort, ARC Prize verified, semi-privateMaker, 2026-09-01
google/gemini-3.1-pro77.1%ARC Prize Verified, thinking highMaker, 2026-02-19
Anthropic: Claude Opus 4.775.83%max effortMaker, 2026-04-16
Anthropic: Claude Opus 4.872.1%max effortMaker, 2026-07-24
Google: Gemini 3.5 Flash72.1%—Maker, 2026-05-19
Anthropic: Claude Sonnet 4.660.42%high effort, 120k thinking tokensMaker, 2026-02-17
Anthropic: Claude Sonnet 4.658.3%max effort, VerifiedMaker, 2026-02-17

MMLU-Pro

Multiple-choice questions across 14 subjects, with ten options each. MMLU-Pro

ModelScoreSettingSource
DeepSeek: DeepSeek V4 Pro 042387.5%think maxMaker, 2026-11-03
Qwen: Qwen3.5-122B-A10B86.7%—Maker, 2026-11-03
DeepSeek: DeepSeek V4 Flash 042386.2%think maxMaker, 2026-11-03
Qwen: Qwen3.6 27B86.2%—Maker, 2026-11-03
Qwen: Qwen3.5-27B86.1%—Maker, 2026-11-03
Qwen: Qwen3.5-35B-A3B85.3%—Maker, 2026-11-03
Google: Gemma 4 31B85.2%—Maker, 2026-07-30
Qwen: Qwen3.6 35B A3B85.2%—Maker, 2026-11-03
Google: Gemma 4 26B A4B82.6%—Maker, 2026-07-30
Qwen: Qwen3.5-9B82.5%—Maker, 2026-11-03
Kimi K2 Instruct81.1%EMMaker, 2026-11-03
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP880.5%0-shotMaker, 2025-04-05
meta-llama/Llama-4-Scout-17B-16E-Instruct74.3%0-shotMaker, 2025-04-05
google/gemma-4-E4B-it69.4%—Maker, 2026-07-30
mistralai/Mistral-Small-3.2-24B-Instruct-250669.06%5-shot CoTMaker, 2026-11-03
meta-llama/Meta-Llama-3.3-70B-Instruct68.9%CoT, 5-shotMaker, 2024-12-06

AIME 2025

The 2025 American Invitational Mathematics Examination problems. AIME 2025

ModelScoreSettingSource
openai/gpt-oss-120b97.9%with tools, reasoning highMaker, 2025-08-05
Anthropic: Claude Sonnet 4.695.6%no tools, max effortMaker, 2026-02-17
openai/gpt-oss-120b92.5%no tools, reasoning highMaker, 2025-08-05
DeepSeek R1 Distill Qwen3 8B 052876.3%—Maker, 2026-11-03
Kimi K2 Instruct49.5%Avg@64Maker, 2026-11-03

AIME 2026

The 2026 American Invitational Mathematics Examination problems. AIME 2026

ModelScoreSettingSource
Z.ai: GLM 5.299.2%—Maker, 2026-11-03
MoonshotAI: Kimi K2.696.4%thinkingMaker, 2026-11-03
Z.ai: GLM 5.195.3%—Maker, 2026-11-03
Qwen: Qwen3.6 27B94.1%full AIME 2026 I & IIMaker, 2026-11-03
Qwen: Qwen3.6 35B A3B92.7%full AIME 2026 I & IIMaker, 2026-11-03
Google: Gemma 4 31B89.2%no toolsMaker, 2026-07-30
Google: Gemma 4 26B A4B88.3%no toolsMaker, 2026-07-30
google/gemma-4-E4B-it42.5%no toolsMaker, 2026-07-30

SWE-bench Verified

Real GitHub issues in Python projects, fixed so the project’s tests pass; a human-checked set of 500. SWE-bench Verified

ModelScoreSettingSource
Claude Opus 596%avg of 5 trials, max effortMaker, 2026-07-24
Anthropic: Claude Fable 595%max effort, avg of 5 trialsMaker, 2026-06-09
Anthropic: Claude Opus 4.888.6%max effort, avg of 5 trialsMaker, 2026-05-28
Anthropic: Claude Opus 4.787.6%max effort, avg of 5 trialsMaker, 2026-04-16
Anthropic: Claude Sonnet 585.2%avg of 5 trials, max effortMaker, 2026-06-30
DeepSeek: DeepSeek V4 Pro 042380.6%think maxMaker, 2026-11-03
google/gemini-3.1-pro80.6%single attempt, thinking highMaker, 2026-02-19
MoonshotAI: Kimi K2.680.2%thinkingMaker, 2026-11-03
Anthropic: Claude Sonnet 4.679.6%avg of 10 trials, max effortMaker, 2026-02-17
DeepSeek: DeepSeek V4 Flash 042379%think maxMaker, 2026-11-03
Mistral: Mistral Medium 3.577.6%max reasoningMaker, 2026-11-03
Qwen: Qwen3.6 27B77.2%internal scaffold (bash + file-edit)Maker, 2026-11-03
Qwen: Qwen3.6 35B A3B73.4%internal scaffold (bash + file-edit)Maker, 2026-11-03
Qwen: Qwen3.5-27B72.4%—Maker, 2026-11-03
Qwen: Qwen3.5-122B-A10B72%—Maker, 2026-11-03
Qwen: Qwen3.5-35B-A3B69.2%—Maker, 2026-11-03
Kimi K2 Instruct65.8%agentic; single attemptMaker, 2026-11-03
openai/gpt-oss-120b62.4%reasoning highMaker, 2025-08-05
Kimi K2 Instruct51.8%agentless; single patch w/o testMaker, 2026-11-03

SWE-Bench Pro

Harder, longer software-engineering tasks across several languages. SWE-Bench Pro

ModelScoreSettingSource
Anthropic: Claude Fable 5.181.2%avg of 5 trials, max effortMaker, 2026-09-01
Anthropic: Claude Fable 580%max effort, avg of 5 trialsMaker, 2026-06-09
Claude Opus 579.2%avg of 5 trials, max effortMaker, 2026-07-24
Anthropic: Claude Opus 4.869.2%max effort, avg of 5 trialsMaker, 2026-05-28
SpaceXAI: Grok 4.564.7%highMaker, 2026-07-20
Anthropic: Claude Opus 4.764.3%max effort, avg of 5 trialsMaker, 2026-04-16
Anthropic: Claude Sonnet 563.2%avg of 5 trials, max effortMaker, 2026-06-30
Z.ai: GLM 5.262.1%OpenHandsMaker, 2026-11-03
Qwen: Qwen3.8 27B61.7%Claude Code harness; refined task setMaker, 2026-11-03
Qwen: Qwen3.7 Max60.6%Claude Code harness; refined task setMaker, 2026-11-03
Google: Gemini 3.6 Flash58.7%public setMaker, 2026-07-21
MoonshotAI: Kimi K2.658.6%thinkingMaker, 2026-11-03
Z.ai: GLM 5.158.4%—Maker, 2026-11-03
Qwen: Qwen3.7 Plus57.6%Claude Code harness; refined task setMaker, 2026-11-03
MiniMax: MiniMax M2.756.22%—Maker, 2026-11-03
DeepSeek: DeepSeek V4 Pro 042355.4%think maxMaker, 2026-11-03
Google: Gemini 3.5 Flash55.1%public set, single attemptMaker, 2026-05-19
google/gemini-3.1-pro54.2%public set, single attempt, thinking highMaker, 2026-02-19
Google: Gemini 3.5 Flash Lite54.2%public setMaker, 2026-07-21
Qwen: Qwen3.6 27B53.5%internal scaffold; refined task setMaker, 2026-11-03
DeepSeek: DeepSeek V4 Flash 042352.6%think maxMaker, 2026-11-03
Qwen: Qwen3.6 35B A3B49.5%internal scaffold; refined task setMaker, 2026-11-03
Google: Gemini 3.1 Flash Lite38.3%public setMaker, 2026-07-21

LiveCodeBench

Competitive-programming problems collected after a model’s training cut-off. LiveCodeBench

ModelScoreSettingSource
DeepSeek: DeepSeek V4 Pro 042393.5%think maxMaker, 2026-11-03
DeepSeek: DeepSeek V4 Flash 042391.6%think maxMaker, 2026-11-03
Qwen: Qwen3.8 27B90.3%v6Maker, 2026-11-03
MoonshotAI: Kimi K2.689.6%thinking; v6Maker, 2026-11-03
Qwen: Qwen3.7 Plus89.6%v6Maker, 2026-11-03
Qwen: Qwen3.6 27B83.9%v6Maker, 2026-11-03
Qwen: Qwen3.5-27B80.7%v6Maker, 2026-11-03
Qwen: Qwen3.6 35B A3B80.4%v6Maker, 2026-11-03
Google: Gemma 4 31B80%v6Maker, 2026-07-30
Qwen: Qwen3.5-122B-A10B78.9%v6Maker, 2026-11-03
Google: Gemma 4 26B A4B77.1%v6Maker, 2026-07-30
Qwen: Qwen3.5-35B-A3B74.6%v6Maker, 2026-11-03
Google: Gemini 3.1 Flash Lite72%1/1/2025-5/1/2025 window, thinking highMaker, 2026-03-03
Qwen: Qwen3.5-9B65.6%v6Maker, 2026-11-03
DeepSeek R1 Distill Qwen3 8B 052860.5%2408-2505Maker, 2026-11-03
Kimi K2 Instruct53.7%v6 (Aug 24 - May 25); pass@1Maker, 2026-11-03
google/gemma-4-E4B-it52%v6Maker, 2026-07-30
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP843.4%pass@1, 10/01/2024-02/01/2025Maker, 2025-04-05
meta-llama/Meta-Llama-3.3-70B-Instruct33.3%pass@1, 10/01/2024-02/01/2025Maker, 2025-04-05
meta-llama/Llama-4-Scout-17B-16E-Instruct32.8%pass@1, 10/01/2024-02/01/2025Maker, 2025-04-05

Terminal-Bench 2.0

Tasks completed by working in a real command-line terminal. Terminal-Bench 2.0

ModelScoreSettingSource
Anthropic: Claude Opus 4.769.4%Terminus-2 harness, thinking disabledMaker, 2026-04-16
Z.ai: GLM 5.169%Claude Code harnessMaker, 2026-11-03
google/gemini-3.1-pro68.5%Terminus-2 harness, thinking highMaker, 2026-02-19
DeepSeek: DeepSeek V4 Pro 042367.9%think maxMaker, 2026-11-03
MoonshotAI: Kimi K2.666.7%thinking; Terminus-2Maker, 2026-11-03
Z.ai: GLM 5.163.5%Terminus-2Maker, 2026-11-03
Qwen: Qwen3.6 27B59.3%Terminus-2, avg of 5 runsMaker, 2026-11-03
Anthropic: Claude Sonnet 4.659.1%max effort, no thinkingMaker, 2026-02-17
MiniMax: MiniMax M2.757%—Maker, 2026-11-03
DeepSeek: DeepSeek V4 Flash 042356.9%think maxMaker, 2026-11-03
Qwen: Qwen3.6 35B A3B51.5%Terminus-2, avg of 5 runsMaker, 2026-11-03
Qwen: Qwen3.5-122B-A10B49.4%—Maker, 2026-11-03
Qwen: Qwen3.5-27B41.6%—Maker, 2026-11-03
Qwen: Qwen3.5-35B-A3B40.5%—Maker, 2026-11-03

τ²-bench

Customer-service conversations where an agent uses tools under a company’s policy, scored per domain. τ²-bench

ModelScoreSettingSource
google/gemini-3.1-pro99.3%telecom, thinking highMaker, 2026-02-19
Anthropic: Claude Sonnet 4.697.9%telecomMaker, 2026-02-17
Anthropic: Claude Sonnet 4.691.7%retailMaker, 2026-02-17
google/gemini-3.1-pro90.8%retail, thinking highMaker, 2026-02-19
Kimi K2 Instruct70.6%retail; Avg@4Maker, 2026-11-03
Kimi K2 Instruct65.8%telecom; Avg@4Maker, 2026-11-03
Kimi K2 Instruct56.5%airline; Avg@4Maker, 2026-11-03

BrowseComp

Hard-to-find facts that take many web searches to locate. BrowseComp

ModelScoreSettingSource
MoonshotAI: Kimi K391.2%max; context compaction at 300KMaker, 2026-11-03
Claude Opus 590.8%with tools, max effortMaker, 2026-07-24
MoonshotAI: Kimi K390.4%max; full 1M context, no context managementMaker, 2026-11-03
Anthropic: Claude Opus 4.888.5%multi-agentMaker, 2026-05-28
Anthropic: Claude Sonnet 586.6%multi agentMaker, 2026-06-30
google/gemini-3.1-pro85.9%search + python + browse, thinking highMaker, 2026-02-19
Anthropic: Claude Sonnet 584.7%single agentMaker, 2026-06-30
Anthropic: Claude Opus 4.884.3%single-agentMaker, 2026-05-28
DeepSeek: DeepSeek V4 Pro 042383.4%think maxMaker, 2026-11-03
MoonshotAI: Kimi K2.683.2%thinkingMaker, 2026-11-03
Anthropic: Claude Sonnet 4.682.07%multi-agentMaker, 2026-02-17
Anthropic: Claude Opus 4.779.3%—Maker, 2026-04-16
Z.ai: GLM 5.179.3%with context managementMaker, 2026-11-03
Anthropic: Claude Sonnet 4.674.01%single-agent, 10M token budgetMaker, 2026-02-17
DeepSeek: DeepSeek V4 Flash 042373.2%think maxMaker, 2026-11-03
Z.ai: GLM 5.168%—Maker, 2026-11-03
Qwen: Qwen3.5-122B-A10B63.8%context-folding 256kMaker, 2026-11-03
Qwen: Qwen3.5-27B61%context-folding 256kMaker, 2026-11-03
Qwen: Qwen3.5-35B-A3B61%context-folding 256kMaker, 2026-11-03

OSWorld-Verified

Tasks on a real computer desktop, through its screen, mouse and keyboard. OSWorld-Verified

ModelScoreSettingSource
Anthropic: Claude Fable 585%max effortMaker, 2026-06-09
MoonshotAI: Kimi K384.8%maxMaker, 2026-11-03
Qwen: Qwen3.8 27B84.3%—Maker, 2026-11-03
Anthropic: Claude Opus 4.883.4%first-attempt, avg of 5 runsMaker, 2026-05-28
Google: Gemini 3.6 Flash83%—Maker, 2026-07-21
Anthropic: Claude Sonnet 581.2%first-attempt, avg of 5 runsMaker, 2026-06-30
Google: Gemini 3.5 Flash78.4%—Maker, 2026-05-19
Anthropic: Claude Opus 4.778%first-attempt, avg of 5 runsMaker, 2026-04-16
google/gemini-3.1-pro76.2%—Maker, 2026-05-19
Google: Gemini 3.5 Flash Lite74%—Maker, 2026-07-21
Qwen: Qwen3.7 Plus73.3%—Maker, 2026-11-03
MoonshotAI: Kimi K2.673.1%thinkingMaker, 2026-11-03
Anthropic: Claude Sonnet 4.672.5%first-attemptMaker, 2026-02-17
Qwen: Qwen3.6 27B63.9%—Maker, 2026-11-03
Qwen: Qwen3.5-122B-A10B58%—Maker, 2026-11-03
Qwen: Qwen3.5-27B56.2%—Maker, 2026-11-03
Qwen: Qwen3.5-35B-A3B54.5%—Maker, 2026-11-03
Google: Gemini 3.1 Flash Lite54.3%—Maker, 2026-07-21
Qwen: Qwen3.5-9B41.8%—Maker, 2026-11-03

MMMU

College-level questions that need reading images, charts and diagrams. MMMU

ModelScoreSettingSource
Qwen: Qwen3.5-122B-A10B83.9%—Maker, 2026-11-03
Qwen: Qwen3.6 27B82.9%—Maker, 2026-11-03
Qwen: Qwen3.5-27B82.3%—Maker, 2026-11-03
Qwen: Qwen3.6 35B A3B81.7%—Maker, 2026-11-03
Qwen: Qwen3.5-35B-A3B81.4%—Maker, 2026-11-03
Qwen: Qwen3.5-9B78.4%—Maker, 2026-11-03
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP873.4%0-shotMaker, 2025-04-05
meta-llama/Llama-4-Scout-17B-16E-Instruct69.4%0-shotMaker, 2025-04-05
mistralai/Mistral-Small-3.2-24B-Instruct-250662.5%—Maker, 2026-11-03

Scores are percentages; higher is better. Only models we can route are listed. A model with no published score has no row, and a missing score is not a zero. A setting shows how the maker ran the test, such as with or without tools.

References

Every score on this page was read from one of these 39 pages, published by the model’s maker. Each benchmark’s own page is listed beside them.

Where the scores come from

  1. ai.google.dev/gemma/docs/core/model_card_4: Scores for Google: Gemma 4 26B A4B, Google: Gemma 4 31B, google/gemma-4-E4B-it; as of 2026-07-30
  2. anthropic.com/claude-fable-5-1-mythos-5-1-system-card: Scores for Anthropic: Claude Fable 5.1; as of 2026-09-01
  3. anthropic.com/claude-fable-5-mythos-5-system-card: Scores for Anthropic: Claude Fable 5; as of 2026-06-09
  4. anthropic.com/claude-opus-4-7-system-card: Scores for Anthropic: Claude Opus 4.7; as of 2026-04-16
  5. anthropic.com/claude-opus-4-8-system-card: Scores for Anthropic: Claude Opus 4.8; as of 2026-05-28
  6. anthropic.com/claude-opus-5-system-card: Scores for Anthropic: Claude Opus 4.8, Claude Opus 5; as of 2026-07-24
  7. anthropic.com/claude-sonnet-4-6-system-card: Scores for Anthropic: Claude Sonnet 4.6; as of 2026-02-17
  8. anthropic.com/claude-sonnet-5-system-card: Scores for Anthropic: Claude Sonnet 5; as of 2026-06-30
  9. cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/oai_gpt-oss_model_card.pdf: Scores for openai/gpt-oss-120b; as of 2025-08-05
  10. deepmind.google/models/model-cards/gemini-3-1-flash-lite: Scores for Google: Gemini 3.1 Flash Lite; as of 2026-03-03
  11. deepmind.google/models/model-cards/gemini-3-1-pro: Scores for google/gemini-3.1-pro; as of 2026-02-19
  12. deepmind.google/models/model-cards/gemini-3-5-flash: Scores for google/gemini-3.1-pro, Google: Gemini 3.5 Flash; as of 2026-05-19
  13. deepmind.google/models/model-cards/gemini-3-5-flash-lite: Scores for Google: Gemini 3.1 Flash Lite, Google: Gemini 3.5 Flash Lite; as of 2026-07-21
  14. deepmind.google/models/model-cards/gemini-3-6-flash: Scores for Google: Gemini 3.6 Flash; as of 2026-07-21
  15. huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B: Scores for DeepSeek R1 Distill Qwen3 8B 0528; as of 2026-11-03
  16. huggingface.co/deepseek-ai/DeepSeek-V4-Flash: Scores for DeepSeek: DeepSeek V4 Flash 0423; as of 2026-11-03
  17. huggingface.co/deepseek-ai/DeepSeek-V4-Pro: Scores for DeepSeek: DeepSeek V4 Pro 0423; as of 2026-11-03
  18. huggingface.co/deepseek-ai/DeepSeek-V4-Pro-0813: Scores for DeepSeek: DeepSeek V4 Flash 0731, DeepSeek: DeepSeek V4 Pro 0813; as of 2026-11-03
  19. huggingface.co/meta-llama/Llama-3.3-70B-Instruct: Scores for meta-llama/Meta-Llama-3.3-70B-Instruct; as of 2024-12-06
  20. huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct: Scores for meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8, meta-llama/Meta-Llama-3.3-70B-Instruct; as of 2025-04-05
  21. huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct: Scores for meta-llama/Llama-4-Scout-17B-16E-Instruct; as of 2025-04-05
  22. huggingface.co/MiniMaxAI/MiniMax-M2.7: Scores for MiniMax: MiniMax M2.7; as of 2026-11-03
  23. huggingface.co/mistralai/Mistral-Medium-3.5-128B: Scores for Mistral: Mistral Medium 3.5; as of 2026-11-03
  24. huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506: Scores for mistralai/Mistral-Small-3.2-24B-Instruct-2506; as of 2026-11-03
  25. huggingface.co/moonshotai/Kimi-K2-Instruct: Scores for Kimi K2 Instruct; as of 2026-11-03
  26. huggingface.co/moonshotai/Kimi-K2.6: Scores for MoonshotAI: Kimi K2.6; as of 2026-11-03
  27. huggingface.co/moonshotai/Kimi-K3: Scores for MoonshotAI: Kimi K3; as of 2026-11-03
  28. huggingface.co/Qwen/Qwen3.5-122B-A10B: Scores for Qwen: Qwen3.5-122B-A10B; as of 2026-11-03
  29. huggingface.co/Qwen/Qwen3.5-27B: Scores for Qwen: Qwen3.5-27B; as of 2026-11-03
  30. huggingface.co/Qwen/Qwen3.5-35B-A3B: Scores for Qwen: Qwen3.5-35B-A3B; as of 2026-11-03
  31. huggingface.co/Qwen/Qwen3.5-9B: Scores for Qwen: Qwen3.5-9B; as of 2026-11-03
  32. huggingface.co/Qwen/Qwen3.6-27B: Scores for Qwen: Qwen3.6 27B; as of 2026-11-03
  33. huggingface.co/Qwen/Qwen3.6-35B-A3B: Scores for Qwen: Qwen3.6 35B A3B; as of 2026-11-03
  34. huggingface.co/Qwen/Qwen3.8-2.4T-A95B: Scores for Qwen: Qwen3.7 Max; as of 2026-11-03
  35. huggingface.co/Qwen/Qwen3.8-27B: Scores for Qwen: Qwen3.6 27B, Qwen: Qwen3.7 Plus, Qwen: Qwen3.8 27B; as of 2026-11-03
  36. huggingface.co/zai-org/GLM-5.1: Scores for Z.ai: GLM 5.1; as of 2026-11-03
  37. huggingface.co/zai-org/GLM-5.2: Scores for Z.ai: GLM 5.2; as of 2026-11-03
  38. huggingface.co/zai-org/GLM-5.3: Scores for Z.ai: GLM 5.3; as of 2026-11-03
  39. media.x.ai/v1/website/4p5-5184fdf9.pdf: Scores for SpaceXAI: Grok 4.5; as of 2026-07-20

The benchmarks