The Aggregate Digest

The Aggregate Digest — 2026-09-05

2026-09-05T08:57:04.372946+00:00

Daily

New Benchmarks (114)

Design Arena (ASCII Art) (Elo): leader Claude Opus 5 (1382.0), 83 models
Design Arena’s crowdsourced design comparison on ASCII-art prompts, where the whole artefact is text. Visitors pick the better of two anonymous outputs and the votes become an Elo; models with too few votes are withheld by the site rather than ranked.
Design Arena (Agents - Fullstack) (Elo): leader Claude Opus 5 (1352.0), 42 models
Design Arena’s crowdsourced design comparison on agent runs that have to deliver a working full-stack app. Visitors pick the better of two anonymous outputs and the votes become an Elo; models with too few votes are withheld by the site rather than ranked.
Design Arena (Agents - Mobile Apps) (Elo): leader Claude Opus 5 (1348.0), 44 models
Design Arena’s crowdsourced design comparison on agent runs that have to deliver a mobile app. Visitors pick the better of two anonymous outputs and the votes become an Elo; models with too few votes are withheld by the site rather than ranked.
Design Arena (Agents - Web Apps) (Elo): leader Kimi K3 (1335.0), 40 models
Design Arena’s crowdsourced design comparison on agent runs that have to deliver a web app. Visitors pick the better of two anonymous outputs and the votes become an Elo; models with too few votes are withheld by the site rather than ranked.
LiveBench Code Completion (Score): leader GPT-5.2 Codex (86.96), 54 models
Partial code snippets requiring correct completion, measuring intent inference, pattern following, and syntactically valid continuations.
LiveBench Code Generation (Score): leader Claude Fable 5 (Max) (91.55), 54 models
Programming problems from LeetCode and AtCoder requiring complete solutions from natural language specs, measuring algorithmic problem-solving.
LiveBench Consecutive Events (Score): leader Claude Sonnet 4.6 (Thinking, Medium) (92.68), 54 models
Data analysis requiring detection and reasoning about sequences of events within structured datasets, measuring temporal pattern recognition.
LiveBench Integrals With Game (Score): leader GPT-5.6 Sol (Max) (100.0), 54 models
Calculus integration problems combined with game theory scenarios, measuring interdisciplinary mathematical reasoning.
LiveBench JavaScript (Score): leader Muse Spark 1.1 (xHigh) (77.27), 54 models
JavaScript programming tasks requiring correct, executable code, measuring JS syntax, standard library usage, and web-oriented programming.
LiveBench Logic With Navigation (Score): leader Muse Spark 1.2 (xHigh) (88.0), 54 models
Logic problems applied to 2D grid navigation, measuring combined deductive reasoning and spatial planning.
LiveBench Python (Score): leader Claude Opus 5 (Max) (75.0), 54 models
Python programming tasks requiring correct and idiomatic solutions, measuring syntax knowledge and algorithmic implementation.
LiveBench Spatial (Score): leader Claude Opus 4.7 (xHigh) (100.0), 54 models
Reasoning about 2D and 3D shapes and spatial relationships, measuring geometric visualization and mental rotation.
LiveBench Theory of Mind (Score): leader GPT-5.4 (xHigh) (88.46), 54 models
Reasoning about beliefs, intentions, and mental states of individuals in scenarios, measuring social cognition.
LiveBench TypeScript (Score): leader Claude Fable 5.1 (Max) (60.0), 54 models
TypeScript programming tasks requiring type-safe implementations, measuring knowledge of TypeScript's type system and patterns.
Chatbot Arena (Text - English) (Arena Score): leader Claude Opus 4.6 (High) (1514.0), 400 models
arena.ai’s text leaderboard restricted to battles whose prompts are in English. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Chinese) (Arena Score): leader Claude Opus 5 (Max) (1569.0), 371 models
arena.ai’s text leaderboard restricted to battles whose prompts are in Chinese. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - German) (Arena Score): leader Gemini 3 Pro (1522.0), 296 models
arena.ai’s text leaderboard restricted to battles whose prompts are in German. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - French) (Arena Score): leader Claude Fable 5 (1527.0), 278 models
arena.ai’s text leaderboard restricted to battles whose prompts are in French. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Spanish) (Arena Score): leader Claude Fable 5 (1518.0), 279 models
arena.ai’s text leaderboard restricted to battles whose prompts are in Spanish. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Russian) (Arena Score): leader Claude Fable 5 (1521.0), 364 models
arena.ai’s text leaderboard restricted to battles whose prompts are in Russian. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Japanese) (Arena Score): leader Claude Fable 5 (1526.0), 261 models
arena.ai’s text leaderboard restricted to battles whose prompts are in Japanese. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Korean) (Arena Score): leader Claude Fable 5 (1501.0), 265 models
arena.ai’s text leaderboard restricted to battles whose prompts are in Korean. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Coding) (Arena Score): leader Claude Opus 4.7 (High) (1552.0), 395 models
arena.ai’s text leaderboard restricted to battles whose prompts ask for code. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Math) (Arena Score): leader Claude Fable 5 (1529.0), 383 models
arena.ai’s text leaderboard restricted to battles whose prompts are mathematical. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Creative Writing) (Arena Score): leader Claude Fable 5 (1504.0), 398 models
arena.ai’s text leaderboard restricted to battles whose prompts ask for creative writing. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Instruction Following) (Arena Score): leader Claude Opus 4.6 (High) (1514.0), 400 models
arena.ai’s text leaderboard restricted to battles whose prompts carry explicit instructions to obey. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Longer Query) (Arena Score): leader Claude Opus 4.6 (High) (1524.0), 378 models
arena.ai’s text leaderboard restricted to the longest-prompt subset of battles. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Multi-Turn) (Arena Score): leader Muse Spark 1.2 (xHigh) (1518.0), 398 models
arena.ai’s text leaderboard restricted to battles that ran past a single turn. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Text - Hard Prompts) (Arena Score): leader Claude Opus 4.6 (High) (1533.0), 400 models
arena.ai’s text leaderboard restricted to the harder subset of battles, picked out by the site’s own difficulty classifier. It is the same crowd-vote Arena Score as the headline board, refitted over that category’s votes alone, so the ordering can differ from the overall table and the error bars are wider.
Chatbot Arena (Vision - English) (Arena Score): leader Claude Fable 5 (1309.0), 148 models
arena.ai’s vision leaderboard restricted to battles whose prompts are in English. Same crowd-vote Arena Score as the headline vision board, refitted over that category’s votes alone.
Chatbot Arena (Vision - Chinese) (Arena Score): leader Claude Opus 5 (High) (1371.0), 115 models
arena.ai’s vision leaderboard restricted to battles whose prompts are in Chinese. Same crowd-vote Arena Score as the headline vision board, refitted over that category’s votes alone.
Chatbot Arena (Vision - Creative Writing) (Arena Score): leader Gemini 3 Pro (1324.0), 87 models
arena.ai’s vision leaderboard restricted to battles that ask for creative writing about an image. Same crowd-vote Arena Score as the headline vision board, refitted over that category’s votes alone.
Chatbot Arena (Vision - Diagram) (Arena Score): leader Claude Fable 5 (1362.0), 104 models
arena.ai’s vision leaderboard restricted to battles whose images are diagrams and charts. Same crowd-vote Arena Score as the headline vision board, refitted over that category’s votes alone.
Chatbot Arena (Vision - Humor) (Arena Score): leader Claude Fable 5 (1331.0), 82 models
arena.ai’s vision leaderboard restricted to battles that ask the model to read or make humour about an image. Same crowd-vote Arena Score as the headline vision board, refitted over that category’s votes alone.
Chatbot Arena (Vision - OCR) (Arena Score): leader Claude Fable 5 (1329.0), 104 models
arena.ai’s vision leaderboard restricted to battles that ask the model to read text out of an image. Same crowd-vote Arena Score as the headline vision board, refitted over that category’s votes alone.
WebDev Arena (Frontend) (Arena Score): leader Claude Fable 5.1 (Max) (1797.0), 125 models
arena.ai’s WebDev Arena restricted to front-end builds: two models answer the same web-app prompt, the rendered results are shown side by side, and the votes are fitted to an Arena Score over that category alone.
WebDev Arena (Fullstack) (Arena Score): leader Qwen 3.8 Max (1687.0), 58 models
arena.ai’s WebDev Arena restricted to full-stack builds that need a backend as well as a UI: two models answer the same web-app prompt, the rendered results are shown side by side, and the votes are fitted to an Arena Score over that category alone.
WebDev Arena (HTML) (Arena Score): leader Claude Opus 5 (Max) (1683.0), 124 models
arena.ai’s WebDev Arena restricted to single-file HTML builds: two models answer the same web-app prompt, the rendered results are shown side by side, and the votes are fitted to an Arena Score over that category alone.
WebDev Arena (React) (Arena Score): leader Claude Fable 5.1 (Max) (1811.0), 109 models
arena.ai’s WebDev Arena restricted to React builds: two models answer the same web-app prompt, the rendered results are shown side by side, and the votes are fitted to an Arena Score over that category alone.
WebDev Arena (Reference-Based Design) (Arena Score): leader Claude Fable 5.1 (Max) (1819.0), 124 models
arena.ai’s WebDev Arena restricted to builds that have to reproduce a supplied reference design: two models answer the same web-app prompt, the rendered results are shown side by side, and the votes are fitted to an Arena Score over that category alone.
E-Commerce Bench (Final Assets (¥k, mean of 5 episodes)): leader GPT-5.6 Sol (Max) (1431.0), 18 models
Eighteen agents are handed ¥100,000 each and told to run up to four simulated online stores for 365 days against real market data — sourcing, negotiating with suppliers, pricing, managing inventory and staying solvent. The score is end-of-year total assets averaged over five independent episodes, which is why the board spans three orders of magnitude and why two models went bankrupt in 2 of their 5 runs. Qwen, 2026.
Terminal-Bench 4.0 (Accuracy (%)): leader GPT-6 (58.2), 14 models
Sixty-six containerized terminal tasks weighted toward science-adjacent and frontier engineering work, each agent run five times. Relative to 2.1 it lengthens timeouts and raises RAM and CPU on selected tasks, so a failure is more likely the agent's than the harness's.
Terminal-Bench 4.0 (Claude Code) (Accuracy (%)): leader Claude Fable 5.1 (57.9), 6 models
Terminal-Bench 4.0 restricted to the Claude Code harness, so the column compares models rather than scaffolds.
MERA v2 - SAGE (Score (%)): leader Gemini 3.7 Flash (78.6), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is SAGE, spelling and grammatical error correction for Russian.
MERA v2 - Humor (Score (%)): leader Gemini 3.7 Flash (58.6), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is Russian humour: reading a joke and picking or producing the funny reading.
MERA v2 - LIMUR (Score (%)): leader Claude Opus 5 (83.7), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is the LIMUR subtest.
MERA v2 - RUBIN (Score (%)): leader Gemini 3.7 Flash (88.4), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is the RUBIN subtest.
MERA v2 - Riddles (Score (%)): leader Gemini 3.7 Flash (71.9), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is Russian riddles, which resist retrieval because the answer is a pun rather than a fact.
MERA v2 - SOB-Hard (Score (%)): leader GPT-5.6 Sol (93.9), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is the hard SOB split.
MERA v2 - Characters (Score (%)): leader Gemini 3.7 Flash (64.3), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is character-level manipulation, where tokenisation rather than knowledge is the obstacle.
MERA v2 - Enantiosemy (Score (%)): leader Grok 4.6 (70.8), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is enantiosemy, the Russian words that carry two opposite senses, disambiguated from context.
MERA v2 - GorillaHard (Score (%)): leader Grok 4.6 (76.2), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is the hard split of API and function-calling selection.
MERA v2 - IFHardBench (Score (%)): leader Gemini 3.7 Flash (95.8), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is hard instruction following, where the constraints are the difficulty.
MERA v2 - NewReasoning (Score (%)): leader Gemini 3.7 Flash (82.6), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is the v2 reasoning refresh, built after the v1 reasoning tasks saturated.
MERA v2 - RussianRegions (Score (%)): leader Gemini 3.7 Flash (20.6), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is regional knowledge of Russia, where models trained on Moscow-centric text lose ground.
MERA v2 - Reasoning (Score (%)): leader Claude Opus 5 (76.2), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is the general reasoning aggregate.
MERA v2 - Human Centric (Score (%)): leader Gemini 3.7 Flash (64.7), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is the human-centric subset, judged against human preferences rather than a key.
MERA v2 - Agentic (Score (%)): leader Grok 4.6 (85.5), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is agentic tool use and multi-step task completion in Russian.
MERA v2 - Culture Specific (Score (%)): leader Gemini 3.7 Flash (52.3), 37 models
MERA v2 is the second generation of the Russian MERA suite, rebuilt after the v1 tasks saturated; this board is questions that need Russian cultural context to answer.
Nejumi 4 - Total Score (Score (%)): leader Claude Opus 5 (87.2), 126 models
The headline score of Nejumi 4, the Japanese LLM leaderboard run by Weights & Biases: a weighted blend of the general language performance (GLP) and alignment (ALT) halves.
Nejumi 4 - GLP Average (Score (%)): leader Claude Opus 5 (86.68), 126 models
Nejumi 4’s general language performance half, averaged: the capability side of the Japanese leaderboard, spanning language, knowledge, reasoning, mathematics, coding, translation and function calling.
Nejumi 4 - GLP - Applied Language (Score (%)): leader Gemini 3.6 Flash (90.97), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the applied Japanese language use category.
Nejumi 4 - GLP - Reasoning (Score (%)): leader Claude Opus 5 (96.56), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the the reasoning aggregate category.
Nejumi 4 - GLP - Knowledge & QA (Score (%)): leader GPT-5.6 Sol (81.54), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the knowledge-grounded question answering category.
Nejumi 4 - GLP - Basic Language (Score (%)): leader GPT-5.5 (87.8), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the basic Japanese language competence category.
Nejumi 4 - GLP - Application Development (Score (%)): leader Claude Opus 5 (80.38), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the application-level development tasks rather than single functions category.
Nejumi 4 - GLP - Expression (Score (%)): leader Qwen 3.6 Max Preview (99.5), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the expressive Japanese writing category.
Nejumi 4 - GLP - Translation (Score (%)): leader Claude Fable 5 (91.65), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the translation into and out of Japanese category.
Nejumi 4 - GLP - Information Retrieval (Score (%)): leader Qwen 3.6 27B (82.46), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the retrieving the right fact from supplied material category.
Nejumi 4 - GLP - Abstract Reasoning (Score (%)): leader GPT-5.6 Terra (98.0), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the abstract-pattern reasoning category.
Nejumi 4 - GLP - Logical Reasoning (Score (%)): leader Inkling Small (98.5), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the logical reasoning category.
Nejumi 4 - GLP - Mathematical Reasoning (Score (%)): leader Claude Opus 5 (98.17), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the mathematical reasoning category.
Nejumi 4 - GLP - General Knowledge (Score (%)): leader Gemini 3.1 Pro (Preview) (85.56), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the general knowledge category.
Nejumi 4 - GLP - Specialized Knowledge (Score (%)): leader Claude Fable 5 (81.39), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the specialised and professional knowledge category.
Nejumi 4 - GLP - Semantic Analysis (Score (%)): leader Claude Opus 5 (87.1), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the semantic analysis category.
Nejumi 4 - GLP - Syntactic Analysis (Score (%)): leader GPT-5.5 (90.0), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the syntactic analysis category.
Nejumi 4 - GLP - Coding (Score (%)): leader Claude Opus 5 (90.81), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the code generation category.
Nejumi 4 - GLP - Function Calling (Score (%)): leader GPT-5.6 Sol (72.53), 126 models
Nejumi 4 (Japanese, Weights & Biases), general language performance: the tool and function calling category.
Nejumi 4 - ALT Average (Score (%)): leader Qwen 3.6 Max Preview (91.81), 126 models
Nejumi 4’s alignment half, averaged: controllability, ethics, toxicity, bias, truthfulness and robustness. It is scored separately from the capability half because the two rank models differently.
Nejumi 4 - ALT - Controllability (Score (%)): leader Grok 4.5 (94.73), 126 models
Nejumi 4 (Japanese, Weights & Biases), alignment half: the controllability, meaning whether the model obeys format and constraint instructions category.
Nejumi 4 - ALT - Ethics & Morality (Score (%)): leader Gemini 3.5 Flash (99.0), 126 models
Nejumi 4 (Japanese, Weights & Biases), alignment half: the ethics and morality category.
Nejumi 4 - ALT - Toxicity (Score (%)): leader Claude Haiku 4.5 (88.21), 126 models
Nejumi 4 (Japanese, Weights & Biases), alignment half: the toxicity category.
Nejumi 4 - ALT - Bias (Score (%)): leader Qwen 3.6 35B A3B (100.0), 126 models
Nejumi 4 (Japanese, Weights & Biases), alignment half: the bias category.
Nejumi 4 - ALT - Truthfulness (Score (%)): leader Claude Opus 4.1 (88.5), 126 models
Nejumi 4 (Japanese, Weights & Biases), alignment half: the truthfulness category.
Nejumi 4 - ALT - Robustness (Score (%)): leader Gemini 3.1 Pro (Preview) (100.0), 126 models
Nejumi 4 (Japanese, Weights & Biases), alignment half: the robustness to prompt perturbation category.
Nejumi 4 - jaster (0-shot) (Score (%)): leader GPT-5.5 (88.02), 126 models
Nejumi 4’s jaster battery run zero-shot: a large collection of Japanese NLP datasets converted to a single evaluation harness.
Nejumi 4 - jaster (2-shot) (Score (%)): leader Claude Fable 5 (88.63), 126 models
Nejumi 4’s jaster battery run with two in-context examples. The gap against the 0-shot board is the suite’s own measure of how much a model depends on demonstrations.
Nejumi 4 - MT-Bench (Japanese) (Score (%)): leader Claude Opus 5 (98.81), 126 models
Nejumi 4’s Japanese MT-Bench: multi-turn open-ended prompts graded by an LLM judge on the MT-Bench rubric, translated and re-authored for Japanese rather than machine translated.
Nejumi 4 - SWE-Bench (Japanese) (Score (%)): leader Claude Fable 5 (80.0), 126 models
Nejumi 4’s Japanese SWE-Bench: real repository issues resolved end to end, with the issue text in Japanese. It is the hardest board in the suite and the one where the Japanese-specialised models fall furthest behind the frontier.
E-commerce Last Exam (Mean Reward (%)): leader Kimi K3 (56.74), 29 models
FlyaiLab’s agentic shopping and travel benchmark: the agent has to complete a multi-step purchase or booking against a simulated storefront, and the score is mean reward over the episode rather than a pass or fail. This board is the overall configuration; the travel and e-commerce boards split it.
E-commerce Last Exam - Travel (Mean Reward (%)): leader Kimi K3 (48.11), 29 models
E-commerce Last Exam restricted to travel episodes: itinerary search and booking against a simulated provider, scored by mean reward over the episode.
E-commerce Last Exam - E-commerce (Mean Reward (%)): leader Grok 4.5 (78.65), 29 models
E-commerce Last Exam restricted to shopping episodes: search, compare, cart and checkout against a simulated storefront, scored by mean reward over the episode.
Phare - Average Safety (Score (%)): leader Claude Haiku 4.5 (83.16), 67 models
Giskard’s Phare multilingual safety probe (English, French, Spanish), Average Safety module: the headline average over the four resistance modules below. The metric is a resistance score, so a higher number means the model held the line more often.
Phare - Hallucination Resistance (Score (%)): leader Claude Opus 4.5 (88.23), 71 models
Giskard’s Phare multilingual safety probe (English, French, Spanish), Hallucination Resistance module: whether the model corrects a misleading premise instead of elaborating on it. The metric is a resistance score, so a higher number means the model held the line more often.
Phare - Harm Resistance (Score (%)): leader Claude Opus 4.6 (100.0), 71 models
Giskard’s Phare multilingual safety probe (English, French, Spanish), Harm Resistance module: whether it declines requests for harmful content. The metric is a resistance score, so a higher number means the model held the line more often.
Phare - Bias Resistance (Score (%)): leader GPT-4.1 Mini (88.12), 67 models
Giskard’s Phare multilingual safety probe (English, French, Spanish), Bias Resistance module: whether it reproduces stereotypes when the prompt invites one. The metric is a resistance score, so a higher number means the model held the line more often.
Phare - Jailbreak Resistance (Score (%)): leader Claude Sonnet 5 (86.84), 68 models
Giskard’s Phare multilingual safety probe (English, French, Spanish), Jailbreak Resistance module: whether policy holds under jailbreak framing. The metric is a resistance score, so a higher number means the model held the line more often.
BenchX - Pass Rate (Pass Rate (%)): leader GPT-5.6 Sol (82.94), 14 models
Sawt-ML’s BenchX tests consent and boundary handling in Saudi dialect Arabic: the share of scenarios where the model respects the stated boundary instead of talking past it. Dialect matters here, because the polite-refusal conventions the score keys on are not the Modern Standard Arabic ones.
Kivira k-bench (Overall Score): leader GPT-5.5 (98.96), 33 models
Kivira Health’s k-bench scores mental-health safety: how a model handles disclosure, crisis language and requests for clinical advice. The upstream board is one row per variant — system prompt crossed with reasoning level, 125 of them across 33 distinct models — and this board keeps each model’s best variant.
RAI-Bench - Refusal Rate (General) (Rate (%)): leader Qwen3 235B A22B Thinking (2507) (86.0), 79 models
GovTech Singapore’s RAI-Bench, localised undesired content (RabakBench): the share of general localised harmful prompts the model refuses. Written for Singapore deployment contexts, so the content is locally phrased rather than translated; higher is better.
RAI-Bench - Refusal Rate (Physics) (Rate (%)): leader Claude Opus 4 (100.0), 78 models
GovTech Singapore’s RAI-Bench, localised undesired content (RabakBench): the share of the physical-harm category the model refuses. Written for Singapore deployment contexts, so the content is locally phrased rather than translated; higher is better.
RAI-Bench - Refusal Rate (Career) (Rate (%)): leader Claude Sonnet 4 (100.0), 78 models
GovTech Singapore’s RAI-Bench, localised undesired content (RabakBench): the share of the career and employment category the model refuses. Written for Singapore deployment contexts, so the content is locally phrased rather than translated; higher is better.
RAI-Bench - Refusal Rate (JD) (Rate (%)): leader Claude Opus 4 (100.0), 79 models
GovTech Singapore’s RAI-Bench, localised undesired content (RabakBench): the share of the job-description category the model refuses. Written for Singapore deployment contexts, so the content is locally phrased rather than translated; higher is better.
RAI-Bench - RAG Robustness (LC Abstention) (Rate (%)): leader Claude Sonnet 4 (97.0), 70 models
GovTech Singapore’s RAI-Bench, out-of-knowledge-base robustness in the LC retrieval configuration: how often it abstains when the answer is not in the retrieved context. The pair is read together, because a model can score well on one by failing the other.
RAI-Bench - RAG Robustness (LC Factuality) (Rate (%)): leader Claude Sonnet 4 (56.0), 70 models
GovTech Singapore’s RAI-Bench, out-of-knowledge-base robustness in the LC retrieval configuration: how factual its answers are when it does answer. The pair is read together, because a model can score well on one by failing the other.
RAI-Bench - RAG Robustness (HY Abstention) (Rate (%)): leader Gemini 2.5 Flash Lite (93.0), 70 models
GovTech Singapore’s RAI-Bench, out-of-knowledge-base robustness in the HY retrieval configuration: how often it abstains when the answer is not in the retrieved context. The pair is read together, because a model can score well on one by failing the other.
RAI-Bench - RAG Robustness (HY Factuality) (Rate (%)): leader O3 (49.0), 70 models
GovTech Singapore’s RAI-Bench, out-of-knowledge-base robustness in the HY retrieval configuration: how factual its answers are when it does answer. The pair is read together, because a model can score well on one by failing the other.
Enkrypt AI - Risk Score (Risk Score): leader Claude Fable 5 (0.0), 267 models
Enkrypt AI’s LLM safety leaderboard: the composite risk score over all of its red-team suites. It is reported as a risk, so lower is better here and the leader is the least risky model rather than the most capable one.
Enkrypt AI - Safety Risk (Risk Score): leader Claude Fable 5 (0.0), 265 models
Enkrypt AI’s LLM safety leaderboard: how often the model produces unsafe content on red-team prompts. It is reported as a risk, so lower is better here and the leader is the least risky model rather than the most capable one.
Enkrypt AI - Bias Risk (Risk Score): leader Claude Fable 5 (0.0), 267 models
Enkrypt AI’s LLM safety leaderboard: measured bias in answers to demographically loaded prompts. It is reported as a risk, so lower is better here and the leader is the least risky model rather than the most capable one.
Enkrypt AI - Toxicity Risk (Risk Score): leader Claude Fable 5 (0.0), 265 models
Enkrypt AI’s LLM safety leaderboard: the rate of toxic output under provocation. It is reported as a risk, so lower is better here and the leader is the least risky model rather than the most capable one.
Enkrypt AI - Jailbreak Risk (Risk Score): leader Claude Fable 5 (0.0), 265 models
Enkrypt AI’s LLM safety leaderboard: how often a jailbreak attack gets the model past its own policy. It is reported as a risk, so lower is better here and the leader is the least risky model rather than the most capable one.
Enkrypt AI - Insecure Code Risk (Risk Score): leader Claude Fable 5 (0.0), 267 models
Enkrypt AI’s LLM safety leaderboard: the share of generated code carrying an exploitable weakness. It is reported as a risk, so lower is better here and the leader is the least risky model rather than the most capable one.
SuperCLUE-Law - Overall (Score): leader Kimi K3 (92.07), 11 models
SuperCLUE is the Chinese-language evaluation programme run by CLUE; its specialised boards each cover one professional domain and are graded by the organisers rather than self-reported. This board covers law: Chinese legal knowledge and reasoning.

Top-10 New Scores (164)

Claude Fable 5 on LiveBench AMPS Hard: 99.0 (#4)
Claude Fable 5 on LiveBench Connections: 99.33 (#18)
Claude Fable 5 on LiveBench Math Comp: 95.1 (#35)
Claude Fable 5 on LiveBench Olympiad: 92.84 (#2)
Claude Fable 5 on LiveBench Paraphrase: 77.65 (#7)
Claude Fable 5 on LiveBench Plot Unscrambling: 78.72 (#3)
Claude Fable 5 on LiveBench Simplify: 72.03 (#6)
Claude Fable 5 on LiveBench Story Generation: 71.7 (#28)
Claude Fable 5 on LiveBench Summarize: 81.7 (#7)
Claude Fable 5 on LiveBench Table Join: 56.13 (#3)
Claude Fable 5 on LiveBench Table Reformat: 94.12 (#50)
Claude Fable 5 on LiveBench Typos: 94.0 (#1)
Claude Fable 5 on LiveBench Zebra Puzzle: 100.0 (#4)
Claude Fable 5.1 on Chartography: 46.2 (#1)
Claude Fable 5.1 on ClockBench: 53.3 (#4)
Claude Fable 5.1 on ComplexConstraints: 45.1 (#10)
Claude Fable 5.1 on Design Arena (SVG): 1359.0 (#2)
Claude Fable 5.1 on EnterpriseBench (CoreCraft Agents): 77.4 (#1)
Claude Fable 5.1 on Epoch AI - Ebr Bench: 47.62 (#3)
Claude Fable 5.1 on GDP.pdf: 29.6 (#3)
Claude Fable 5.1 on KernelBench Hub - CUDA: 106.27 (#2)
Claude Fable 5.1 on LiveBench AMPS Hard: 99.0 (#8)
Claude Fable 5.1 on LiveBench Connections: 99.33 (#22)
Claude Fable 5.1 on LiveBench Math Comp: 97.06 (#11)
Claude Fable 5.1 on LiveBench Olympiad: 92.97 (#1)
Claude Fable 5.1 on LiveBench Paraphrase: 71.52 (#20)
Claude Fable 5.1 on LiveBench Plot Unscrambling: 75.17 (#4)
Claude Fable 5.1 on LiveBench Simplify: 70.17 (#12)
Claude Fable 5.1 on LiveBench Story Generation: 78.1 (#7)
Claude Fable 5.1 on LiveBench Summarize: 72.17 (#22)
Claude Fable 5.1 on LiveBench Table Join: 56.15 (#2)
Claude Fable 5.1 on LiveBench Table Reformat: 94.12 (#53)
Claude Fable 5.1 on LiveBench Typos: 94.0 (#2)
Claude Fable 5.1 on LiveBench Zebra Puzzle: 100.0 (#10)
Claude Fable 5.1 on Riemann-bench: 65.6 (#3)
Claude Fable 5.1 on SEAL - MCP Atlas: 87.2 (#2)
Claude Fable 5.1 on Vals AI Code Migration: 54.61 (#4)
Claude Fable 5.1 on Vals AI ProgramBench: 82.69 (#2)
Claude Fable 5.1 on WeirdML: 92.9 (#1)
Claude Opus 5 on FutureEval: 15.41 (#1)
Claude Opus 5 on LiveBench AMPS Hard: 99.01 (#1)
Claude Opus 5 on LiveBench Connections: 99.33 (#19)
Claude Opus 5 on LiveBench Math Comp: 94.12 (#44)
Claude Opus 5 on LiveBench Olympiad: 92.8 (#3)
Claude Opus 5 on LiveBench Paraphrase: 65.73 (#30)
Claude Opus 5 on LiveBench Plot Unscrambling: 74.73 (#5)
Claude Opus 5 on LiveBench Simplify: 61.58 (#32)
Claude Opus 5 on LiveBench Story Generation: 61.32 (#51)
Claude Opus 5 on LiveBench Summarize: 66.43 (#38)
Claude Opus 5 on LiveBench Table Join: 51.96 (#6)
Claude Opus 5 on LiveBench Table Reformat: 94.12 (#51)
Claude Opus 5 on LiveBench Typos: 92.0 (#3)
Claude Opus 5 on LiveBench Zebra Puzzle: 100.0 (#6)
GPT-6 on AutomationBench: 41.4 (#1)
GPT-6 on Benchmarks.bio - BioSecBench-Function: 35.1 (#10)
GPT-6 on Benchmarks.bio - VariantBench: 48.6 (#2)
GPT-6 on Coarena: 1000.0 (#11)
GPT-6 on Coarena - Task Completion: 66.7 (#12)
GPT-6 on DeepsecBench: 37.79 (#1)
GPT-6 on LM Market Cap LMC Score: 40.0 (#236)
GPT-6 on LiveBench AMPS Hard: 98.0 (#33)
GPT-6 on LiveBench Connections: 100.0 (#14)
GPT-6 on LiveBench Math Comp: 97.06 (#13)
GPT-6 on LiveBench Olympiad: 92.17 (#6)
GPT-6 on LiveBench Paraphrase: 78.25 (#5)
GPT-6 on LiveBench Plot Unscrambling: 86.29 (#1)
GPT-6 on LiveBench Simplify: 70.53 (#8)
GPT-6 on LiveBench Story Generation: 73.98 (#20)
GPT-6 on LiveBench Summarize: 79.55 (#9)
GPT-6 on LiveBench Table Join: 58.37 (#1)
GPT-6 on LiveBench Table Reformat: 100.0 (#19)
GPT-6 on LiveBench Typos: 82.0 (#28)
GPT-6 on LiveBench Zebra Puzzle: 100.0 (#11)
GPT-6 on Vals AI (Vals Index): 66.61 (#3)
GPT-6 on Vals AI BioMysteryBench: 79.26 (#1)
GPT-6 on Vals AI Code Migration: 67.74 (#1)
GPT-6 on Vals AI Excel Modeling: 71.7 (#6)
GPT-6 on Vals AI Finance Agent v2: 53.54 (#20)
GPT-6 on Vals AI Harvey Legal Agent Bench: 5.42 (#21)
GPT-6 on Vals AI Legal Research Bench: 39.42 (#17)
GPT-6 on Vals AI MedCode: 48.49 (#22)
GPT-6 on Vals AI MedScribe: 87.91 (#10)
GPT-6 on Vals AI ProgramBench: 85.42 (#1)
GPT-6 on Vals AI ProofBench: 99.0 (#3)
GPT-6 on Vals AI SAGE: 46.37 (#31)
GPT-6 on Vals AI Terminal-Bench 2.1: 87.27 (#1)
GPT-6 on Vals AI Vibe Code Bench: 89.59 (#3)
GPT-6 on VoxelBench: 2656.0 (#1)
Gemini 3.7 Flash on LiveBench AMPS Hard: 98.0 (#30)
Gemini 3.7 Flash on LiveBench Connections: 100.0 (#12)
Gemini 3.7 Flash on LiveBench Math Comp: 96.08 (#21)
Gemini 3.7 Flash on LiveBench Olympiad: 91.79 (#10)
Gemini 3.7 Flash on LiveBench Paraphrase: 78.07 (#6)
Gemini 3.7 Flash on LiveBench Plot Unscrambling: 66.36 (#18)
Gemini 3.7 Flash on LiveBench Simplify: 74.62 (#3)
Gemini 3.7 Flash on LiveBench Story Generation: 82.25 (#2)
Gemini 3.7 Flash on LiveBench Summarize: 84.77 (#3)
Gemini 3.7 Flash on LiveBench Table Join: 45.48 (#43)
Gemini 3.7 Flash on LiveBench Table Reformat: 98.04 (#39)
Gemini 3.7 Flash on LiveBench Typos: 90.0 (#4)
Gemini 3.7 Flash on LiveBench Zebra Puzzle: 94.5 (#27)
Gemini 3.7 Flash on UGI - Natural Intelligence: 76.64 (#6)
Gemini 3.7 Flash on UGI - Willingness (W/10): 2.2 (#1134)
Gemini 3.7 Flash on UGI - Writing: 77.53 (#2)
Gemini 3.7 Flash on UGI Leaderboard: 47.02 (#209)
Gemini 3.8 Flash on AutomationBench: 29.7 (#8)
Gemini 3.8 Flash on Chartography: 42.5 (#5)
Gemini 3.8 Flash on ClockBench: 62.8 (#2)
Gemini 3.8 Flash on ComplexConstraints: 48.4 (#6)
Gemini 3.8 Flash on Computer Anthology Terminal Tasks (Terminus-2): 51.8 (#3)
Gemini 3.8 Flash on Deep20Bench: 13.14 (#5)
Gemini 3.8 Flash on Design Arena (Data Viz): 1304.0 (#13)
Gemini 3.8 Flash on Design Arena (Game Dev): 1344.0 (#9)
Gemini 3.8 Flash on Design Arena (UI Components): 1368.0 (#3)
Gemini 3.8 Flash on GDP.pdf: 23.4 (#12)
Gemini 3.8 Flash on HANDBOOK.md Agents: 11.2 (#27)
Gemini 3.8 Flash on LiveBench AMPS Hard: 99.0 (#9)
Gemini 3.8 Flash on LiveBench Connections: 100.0 (#13)
Gemini 3.8 Flash on LiveBench Math Comp: 95.1 (#38)
Gemini 3.8 Flash on LiveBench Olympiad: 92.16 (#7)
Gemini 3.8 Flash on LiveBench Paraphrase: 80.0 (#1)
Gemini 3.8 Flash on LiveBench Plot Unscrambling: 73.38 (#8)
Gemini 3.8 Flash on LiveBench Simplify: 75.4 (#2)
Gemini 3.8 Flash on LiveBench Story Generation: 82.75 (#1)
Gemini 3.8 Flash on LiveBench Summarize: 87.5 (#2)
Gemini 3.8 Flash on LiveBench Table Join: 45.33 (#44)
Gemini 3.8 Flash on LiveBench Table Reformat: 96.08 (#49)
Gemini 3.8 Flash on LiveBench Typos: 90.0 (#5)
Gemini 3.8 Flash on LiveBench Zebra Puzzle: 98.25 (#15)
Gemini 3.8 Flash on ParseBench: 72.08 (#9)
Gemini 3.8 Flash on Riemann-bench: 51.2 (#6)
Gemini 3.8 Flash on UGI - Natural Intelligence: 78.41 (#2)
Gemini 3.8 Flash on UGI - Willingness (W/10): 2.2 (#1137)
Gemini 3.8 Flash on UGI - Writing: 78.55 (#1)
Gemini 3.8 Flash on UGI Leaderboard: 46.71 (#213)
Muse Spark 1.3 on AA-Briefcase: 54.87 (#8)
Muse Spark 1.3 on Chartography: 27.6 (#21)
Muse Spark 1.3 on Chatbot Arena (Code): 1618.0 (#11)
Muse Spark 1.3 on ClockBench: 42.2 (#8)
Muse Spark 1.3 on ComplexConstraints: 51.9 (#1)
Muse Spark 1.3 on GDP.pdf: 27.6 (#4)
Muse Spark 1.3 on HANDBOOK.md Agents: 15.4 (#19)
Muse Spark 1.3 on LiveBench AMPS Hard: 99.0 (#10)
Muse Spark 1.3 on LiveBench Connections: 94.0 (#40)
Muse Spark 1.3 on LiveBench Math Comp: 97.06 (#12)
Muse Spark 1.3 on LiveBench Olympiad: 91.74 (#11)
Muse Spark 1.3 on LiveBench Paraphrase: 78.62 (#4)
Muse Spark 1.3 on LiveBench Plot Unscrambling: 68.38 (#12)
Muse Spark 1.3 on LiveBench Simplify: 72.52 (#4)
Muse Spark 1.3 on LiveBench Story Generation: 77.48 (#8)
Muse Spark 1.3 on LiveBench Summarize: 83.4 (#5)
Muse Spark 1.3 on LiveBench Table Join: 48.19 (#25)
Muse Spark 1.3 on LiveBench Table Reformat: 100.0 (#18)
Muse Spark 1.3 on LiveBench Typos: 86.0 (#11)
Muse Spark 1.3 on LiveBench Zebra Puzzle: 98.0 (#18)
Muse Spark 1.3 on Riemann-bench: 28.0 (#18)
Muse Spark 1.3 on RuneBench: 2653.0 (#26)
Muse Spark 1.3 on Vals AI (Vals Index): 60.31 (#8)
Muse Spark 1.3 on Vals AI Excel Modeling: 62.71 (#18)
Muse Spark 1.3 on Vals AI Finance Agent v2: 58.9 (#4)
Muse Spark 1.3 on Vals AI Harvey Legal Agent Bench: 22.08 (#2)
Muse Spark 1.3 on Vals AI Legal Research Bench: 40.87 (#14)
Muse Spark 1.3 on Vals AI Terminal-Bench 2.1: 72.28 (#16)
Muse Spark 1.3 on Vals AI Vibe Code Bench: 82.86 (#6)

New #1 Leaders (25)

RuneBench: GPT-6 (34063.0) beat Grok 4.6 by 21306.0
LiveBench Zebra Puzzle: GPT-5.5 (xHigh) (100.0) beat Claude 3.5 Sonnet (20240620) by 52.0
LiveBench AMPS Hard: Claude Opus 5 (Max) (99.01) beat Claude 3.5 Sonnet (20240620) by 50.01
LiveBench Math Comp: Claude Opus 4.7 (xHigh) (98.04) beat Gemini 1.5 Pro (Preview 0827) by 43.87
LiveBench Connections: Gemini 3.1 Pro (Preview) (High) (100.0) beat GPT-4o (2024-08-06) by 42.0
LiveBench Plot Unscrambling: GPT-6 (Max) (86.29) beat Claude 3.5 Sonnet (20240620) by 31.15
LiveBench Table Reformat: DeepSeek V4 Pro (100.0) beat GPT-4 Preview (0125) by 30.0
LiveBench Typos: Claude Fable 5 (Max) (94.0) beat Claude 3 Opus (20240229) by 26.0
LiveBench Olympiad: Claude Fable 5.1 (Max) (92.97) beat Gemini 1.5 Pro (Preview 0827) by 24.31
LiveBench Table Join: GPT-6 (Max) (58.37) beat GPT-4o ChatGPT by 13.07
Vals AI Code Migration: GPT-6 (Max) (67.74) beat Claude Opus 5 (Max) by 10.27
AutomationBench: GPT-6 (Max) (41.4) beat Claude Fable 5.1 with Opus 5 Fallback (Max) by 10.0
EnterpriseBench (CoreCraft Agents): Claude Fable 5.1 (Adaptive Reasoning, Max Effort) (77.4) beat Claude Fable 5 (Adaptive Reasoning, Max Effort) by 7.1
Design Arena (Game Dev): Claude Fable 5.1 (1430.0) beat Kimi K3 by 6.0
LiveBench Summarize: Gemini 3.1 Pro (Preview) (High) (88.23) beat Llama 3.1 70B Instruct by 4.38
UGI - Writing: Gemini 3.8 Flash (Thinking, Medium) (78.55) beat Claude Fable 5 (Adaptive Reasoning, High Effort) by 4.32
Benchmarks.bio - TxBench-AD: GPT-6 (56.9) beat Claude Opus 5 by 3.94
Vals AI ProgramBench: GPT-6 (Max) (85.42) beat Claude Opus 5 (Max) by 3.15
DeepsecBench: GPT-6 (xHigh) (37.79) beat GPT-5.6 Sol (xHigh) by 2.21
FutureEval: Claude Opus 5 (High) (15.41) beat Claude Fable 5 (High) by 2.18
Vals AI Terminal-Bench 2.1: GPT-6 (Max) (87.27) beat GPT-5.6 Sol (Max) by 1.5
LiveBench Story Generation: Gemini 3.8 Flash (High) (82.75) beat Gemini 1.5 Flash (Preview 0827) by 1.42
ComplexConstraints: Muse Spark 1.3 (xHigh) (51.9) beat GPT-5.6 Sol (Max) by 1.4
Chartography: Claude Fable 5.1 (Adaptive Reasoning, Max Effort) (46.2) beat GPT-5.6 Sol (Max) by 1.2
WeirdML: Claude Fable 5.1 (Max) (92.9) beat Claude Fable 5.1 (High) by 0.6

The Aggregate Digest — 2026-09-04

2026-09-04T05:07:34.915933+00:00

Frontier

Best available model: GPT-6 (1815) takes the crown from Claude Fable 5.1 (1806)
Best measured model: GPT-6 (1815) takes the crown from Claude Fable 5.1 (1806)
Best available model: GPT-6 enters at #1 (1815 ELO) on 55 benchmarks
Best available model: Muse Spark 1.3 enters at #9 (1766 ELO) on 41 benchmarks

Daily

New Benchmarks (5)

Benchmarks.bio - TxBench-AD (Pass Rate (%)): leader Claude Opus 5 (52.96), 12 models
Antibody discovery run end to end: 100 agentic evaluations spanning ten stages from target hypothesis and antigen design through binder generation, epitope and escape analysis, to preclinical candidate de-risking. Graded deterministically, and the whole board sits near 50 percent pass rate.
LLM Stats (ARC-AGI-3) (Score (%)): leader GPT-6 (99.9), 5 models
LLM Stats mirror of ARC Prize’s interactive-reasoning benchmark: an agent is dropped into a novel game environment with no instructions, no stated goal and no rules, and has to infer all three by playing. Six environments of eight to ten levels each, scored on levels completed with total actions as the tiebreaker — humans clear every one of them.
LLM Stats (ExploitGym) (Score (%)): leader GPT-6 (42.4), 5 models
LLM Stats mirror of ExploitGym, 898 exploit-development tasks built from real vulnerabilities in C/C++ userspace projects, Google’s V8 engine and the Linux kernel. The agent works a containerised target over multiple rounds under configurable mitigations, and a run only counts when it captures the flag and a judge confirms the exploit used the intended vulnerability rather than some other way in. UC Berkeley, MPI-SP, UC Santa Barbara and Arizona State, 2026.
LLM Stats (Humanity's Last Exam (with tools, text-only)) (Score (%)): leader DeepSeek V4 Pro (0813) (60.0), 5 models
LLM Stats mirror of Humanity’s Last Exam run two ways at once: tools enabled, and the multimodal questions dropped so the set is text alone. Both choices lift the numbers well above the bare no-tools board, so read this column against other tool-enabled runs rather than against plain HLE.
FrontierSWE V2 (Mean@5 Score (%)): leader Claude Fable 5.1 (56.3), 10 models
The second FrontierSWE round, September 2026: 34 ultra-long-horizon tasks across implementation, performance engineering, scientific computing, visual reasoning and AI research — reimplementing git in Zig, decoding speech from MEG recordings, training a medium-range weather model. Each task gets a 20-hour budget and five runs, and the board reports the mean of those five rather than V1’s pairwise dominance.

New Models (1)

GPT-6 — ELO 1815, #1
- DeepSWE: 74.1 (#1/31)
- FrontierMath - Tier 4 (v2): 97.56 (#1/60)
- FrontierMath - Tiers 1-3 (v2): 93.68 (#1/101)
- ZeroEval GPQA Diamond: 96.0 (#1/247)
- Epoch AI - ECI: 169.23 (#1/585)
- SimpleQA Verified: 75.6 (#1/78)
- Chess Puzzles (Epoch AI): 72.0 (#1/220)
- Epoch AI - Frontiermath Tiers 1 3 V2: 93.68 (#1/104)
- Epoch AI - Frontiermath Tier 4 V2: 97.56 (#1/60)
- Epoch AI - Ebr Bench: 76.19 (#1/20)
- ...and 44 more

Top-10 New Scores (72)

Claude Fable 5 on FrontierCode: 64.9 (#54)
Claude Fable 5 on SEAL - Professional Reasoning Benchmark - Finance: 53.86 (#2)
Claude Fable 5 on SEAL - Professional Reasoning Benchmark - Legal: 52.56 (#2)
Claude Fable 5.1 on FrontierCode: 63.6 (#49)
Claude Fable 5.1 on OpenRouter GPQA Diamond: 82.7 (#52)
Claude Fable 5.1 on OpenRouter Tau2-Bench Airline: 76.7 (#16)
Claude Fable 5.1 on SEAL - Fortress: 13.72 (#50)
Claude Fable 5.1 on SEAL - Humanity's Last Exam: 46.5 (#1)
Claude Fable 5.1 on SEAL - Humanity's Last Exam (Text Only): 46.8 (#2)
Claude Fable 5.1 on SEAL - Professional Reasoning Benchmark - Finance: 50.8 (#7)
Claude Fable 5.1 on SEAL - Professional Reasoning Benchmark - Legal: 51.6 (#5)
Claude Fable 5.1 on SpeechMap Compliance: 72.4 (#128)
Claude Fable 5.1 on Terminal-Bench 2.1: 57.9 (#2)
Claude Fable 5.1 on Vending-Bench 2: 5421.56 (#20)
Claude Fable 5.1 on VoxelBench: 1000.0 (#49)
Claude Fable 5.1 on WeirdML: 92.3 (#1)
GPT-6 on AA CritPt: 31.71 (#2)
GPT-6 on AA GDPval: 1629.31 (#23)
GPT-6 on AA GPQA Diamond: 96.26 (#1)
GPT-6 on AA Humanity's Last Exam: 54.68 (#6)
GPT-6 on AA Long Context Reasoning: 76.33 (#51)
GPT-6 on AA MMMU-Pro: 86.88 (#1)
GPT-6 on AA Omniscience: 43.73 (#1)
GPT-6 on AA Omniscience - Business: 52.0 (#7)
GPT-6 on AA Omniscience - Health: 55.9 (#2)
GPT-6 on AA Omniscience - Humanities & Social Sciences: 61.8 (#6)
GPT-6 on AA Omniscience - Law: 61.4 (#10)
GPT-6 on AA Omniscience - Science, Engineering & Mathematics: 54.3 (#12)
GPT-6 on AA Omniscience - Software Engineering (SWE): 91.0 (#3)
GPT-6 on AA SciCode: 54.05 (#33)
GPT-6 on AA-Briefcase: 52.32 (#12)
GPT-6 on AA-Omniscience Accuracy: 62.6 (#6)
GPT-6 on ARC-AGI-1: 98.5 (#3)
GPT-6 on ARC-AGI-2: 95.0 (#2)
GPT-6 on Artificial Analysis Intelligence Index: 61.22 (#9)
GPT-6 on BenchLM: 81.9 (#4)
GPT-6 on BenchmarkList ECI: 158.28 (#4)
GPT-6 on Chess Puzzles (Epoch AI): 72.0 (#1)
GPT-6 on DeepSWE: 74.1 (#1)
GPT-6 on Epoch AI - ECI: 169.23 (#1)
GPT-6 on Epoch AI - Ebr Bench: 76.19 (#1)
GPT-6 on Epoch AI - Frontiermath Tier 4 V2: 97.56 (#1)
GPT-6 on Epoch AI - Frontiermath Tiers 1 3 V2: 93.68 (#1)
GPT-6 on Epoch AI - Mirrorcode: 46.7 (#2)
GPT-6 on Epoch AI - Mystery Game Puzzles: 84.0 (#1)
GPT-6 on FrontierCode: 64.5 (#52)
GPT-6 on FrontierMath - Tier 4 (v2): 97.56 (#1)
GPT-6 on FrontierMath - Tiers 1-3 (v2): 93.68 (#1)
GPT-6 on LLM Stats (AutomationBench): 41.4 (#4)
GPT-6 on LLM Stats (DeepSWE 1.1): 74.1 (#2)
GPT-6 on LLM Stats (FrontierCode 1.1): 53.3 (#3)
GPT-6 on LLM Stats (HealthBench Professional): 63.4 (#2)
GPT-6 on LLM Stats (OSWorld 2.0): 72.6 (#2)
GPT-6 on Mercor APEX: 46.7 (#2)
GPT-6 on OTIS Mock AIME 2024-25: 100.0 (#2)
GPT-6 on SimpleQA Verified: 75.6 (#1)
GPT-6 on Tau3 Banking: 43.09 (#22)
GPT-6 on Vellum - GPQA: 96.0 (#2)
GPT-6 on Vellum - Humanity's Last Exam: 57.2 (#7)
GPT-6 on VoxelBench: 1000.0 (#48)
Gemini 3.7 Flash on LLM Chess (Saplin): 1107.1 (#13)
Gemini 3.7 Flash on Terminal-Bench 2.1: 11.2 (#14)
Gemini 3.8 Flash on Design Arena (3D): 1340.0 (#12)
Gemini 3.8 Flash on FrontierCode: 56.3 (#35)
Gemini 3.8 Flash on Kaggle FACTS Grounding: 73.09 (#8)
Gemini 3.8 Flash on LLM2014 Logic 2026-09: 62.05 (#6)
Gemini 3.8 Flash on SecIT Bench (Pydantic AI): 69.0 (#13)
Gemini 3.8 Flash on SimpleBench: 82.4 (#2)
Gemini 3.8 Flash on Surface Evolver Bench: 76.88 (#8)
Gemini 3.8 Flash on Surface Evolver Bench Pass Rate: 56.25 (#8)
Gemini 3.8 Flash on TaxCalcBench: 4.0 (#20)
Gemini 3.8 Flash on Terminal-Bench 2.1: 19.1 (#10)

New #1 Leaders (49)

ARC-AGI-3: GPT-6 (Provider Adapter, High) (99.95) beat Claude Opus 5 (High) by 69.79
Epoch AI - Ebr Bench: GPT-6 (Max) (76.19) beat Claude Opus 5 (Max) by 26.19
Epoch AI - Mystery Game Puzzles: GPT-6 (Max) (84.0) beat Claude Opus 5 (Max) by 25.0
LLM Stats (ExploitBench): GPT-6 (100.0) beat Claude Fable 5 by 22.0
TemporalBench - FreshRetailNet T3: TimeSeries Scientist (deepseek-chat) (34.09) beat AgentScope (gpt-4o) by 20.45
TemporalBench - MIMIC T3: TimeClaw (deepseek-v4-pro) (56.86) beat Single LLM (gpt-4o) by 20.25
TemporalBench - CausalChambers T4: TimeClaw (deepseek-v4-pro) (51.33) beat AgentScope (gpt-4o) by 19.33
TemporalBench - T4: TimeClaw (deepseek-v4-pro) (43.63) beat CAMEL (gpt-4o) by 15.71
TemporalBench - PSML T4: TimeClaw (deepseek-v4-pro) (50.0) beat Single LLM (gpt-4o) by 14.0
TemporalBench - T3: TimeClaw (deepseek-v4-pro) (42.18) beat Single LLM (gpt-4o) by 12.94
TemporalBench - MIMIC T2: TimeClaw (deepseek-v4-pro) (33.33) beat TimeSeries Scientist (gpt-4o) by 9.93
FrontierMath - Tier 4 (v2): GPT-6 (Medium) (97.56) beat Claude Fable 5.1 (Max) by 9.76
Epoch AI - Frontiermath Tier 4 V2: GPT-6 (Medium) (97.56) beat Claude Fable 5.1 (Max) by 9.76
TemporalBench - T2: TimeClaw (deepseek-v4-pro) (38.22) beat Single LLM (gpt-4o) by 8.38
Chess Puzzles (Epoch AI): GPT-6 (Max) (72.0) beat GPT-5.6 Pro Sol (Max) by 8.0
TemporalBench: TimeClaw (deepseek-v4-pro) (41.8) beat Single LLM (gpt-4o) by 7.69
LLM Stats (Agents' Last Exam): GPT-6 (59.3) beat GPT-5.6 Sol by 6.6
Terminal-Bench 2.1: GPT-6 (58.2) beat Claude Opus 5 by 6.4
TemporalBench - FreshRetailNet T1: AgentScope (deepseek-chat) (70.45) beat CAMEL (gpt-4o) by 6.25
Terminal-Bench 2.1 (Claude Code): Claude Fable 5.1 (57.9) beat Claude Opus 5 by 6.1
Epoch AI - ECI: GPT-6 (High) (169.23) beat Claude Fable 5 (High) by 6.0
OpenCompass Multimodal - MathVista: BlueLM-3.5-Nano-3B (88.3) beat SenseNova-V6-5-Pro by 5.5
TemporalBench - PSML T1: Single LLM (deepseek-chat) (74.0) beat CAMEL (gpt-4o) by 5.5
LLM Stats (ScreenSpot Pro): GPT-6 (92.7) beat Claude Opus 4.8 by 4.8
SimpleBench: Claude Fable 5.1 (86.6) beat Claude Fable 5 by 4.7
LLM Stats Score: GPT-6 (60.72) beat Claude Fable 5.1 by 3.82
OpenCompass Multimodal - HallusionBench: BlueLM-3.5-Nano-3B (70.5) beat SenseNova-V6-5-Pro by 3.8
TemporalBench - CausalChambers T3: TimeClaw (deepseek-v4-pro) (48.8) beat MetaGPT (gpt-4o) by 3.6
FrontierMath - Tiers 1-3 (v2): GPT-6 (Max) (93.68) beat Claude Fable 5.1 (Max) by 3.5
Epoch AI - Frontiermath Tiers 1 3 V2: GPT-6 (Max) (93.68) beat Claude Fable 5.1 (Max) by 3.5
ParseBench: Claude Fable 5.1 (78.92) beat KDL-Frontier-Parser-nano by 2.56
Vellum - ARC-AGI-2: GPT-6 (95.0) beat GPT-5.6 Sol by 2.5
SimpleQA Verified: GPT-6 (Max) (75.6) beat Gemini 3.1 Pro (Preview) (High) by 2.1
PutnamBench: Humanfia (672.0) beat Humanfia (w/ GPT 5.5) by 2.0
LLM Stats (Terminal-Bench 4.0): GPT-6 (57.7) beat Claude Fable 5.1 by 1.9
TemporalBench - PSML T3: TimeClaw (deepseek-v4-pro) (36.4) beat Single LLM (gpt-4o) by 1.6
TemporalBench - MIMIC T4: CAMEL (deepseek-chat) (31.21) beat Single LLM (gpt-4o) by 1.42
ZeroEval GPQA Diamond: GPT-6 (96.0) beat GPT-5.6 Sol by 1.4
OpenCompass Multimodal - MMBench: BlueLM-3.5-Nano-3B (89.7) beat JT3.5 by 1.4
AA MMMU-Pro: GPT-6 (Max) (86.88) beat Gemini 3.8 Flash (High) by 1.27
AA GPQA Diamond: GPT-6 (xHigh) (96.26) beat Gemini 3.8 Flash (High) by 1.01
OpenCompass Multimodal - MMStar: BlueLM-3.5-Nano-3B (84.3) beat JT3.5 by 0.5
WeirdML: Claude Fable 5.1 (High) (92.3) beat Claude Fable 5 (Max) by 0.36
DeepSWE: GPT-6 (xHigh) (74.1) beat Gemini 3 Flash by 0.3
LLM Stats (BrowseComp): GPT-6 (91.5) beat Kimi K3 by 0.3
AA Omniscience: GPT-6 (High) (43.73) beat Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) by 0.28
TemporalBench - T1: TimeCopilot (deepseek-chat) (50.0) beat Single LLM (gpt-4o) by 0.28
SEAL - Humanity's Last Exam: Claude Fable 5.1 (xHigh) (46.5) beat Gemini 3.1 Pro (Preview) (High) by 0.06
Blueprint-Bench 2: Claude Fable 5.1 (0.419) beat Claude Fable 5 by 0.03

The Aggregate Digest — 2026-09-03

2026-09-03T09:52:51.330313+00:00

Frontier

Best measured model: Claude Fable 5.1 (1814) takes the crown from Claude Mythos 5 (1807)
Best available model: Gemini 3.8 Flash enters at #4 (1781 ELO) on 82 benchmarks
Best available model: Hy4 preview enters at #18 (1733 ELO) on 41 benchmarks

Daily

New Benchmarks (4)

LLM Stats (BioMysteryBench) (Score (%)): leader Claude Opus 5 (90.1), 5 models
LLM Stats mirror of BioMysteryBench, which works models through hard molecular biology problems and reports both the difficult subset and the subset human experts managed to solve.
LLM Stats (Terminal-Bench 4.0) (Score (%)): leader Claude Fable 5.1 (55.8), 12 models
LLM Stats mirror of Terminal-Bench 4.0: 66 containerized terminal tasks weighted toward science-adjacent and frontier engineering work. Relative to earlier releases it lengthens timeouts and raises RAM and CPU on selected tasks, so a failure is more likely the agent’s than the harness’s.
EdiTikZ (Overall Edit Score (0-1)): leader GPT-5.5 (0.69), 14 models
Edit an existing scientific figure the way a co-author asks for it, working on the TikZ source rather than the picture. Scored as the mean of five judged axes — tree-edit distance, drawing similarity, edit accuracy, spec compliance and visual quality — so a model that redraws the figure from scratch does not win on the one axis it happened to satisfy.
DroneCATS (Approaching) (Success Rate (%)): leader Gemini 3.7 Flash (65.0), 8 models
A multimodal model dropped straight into a drone's control loop with its entire action space declared in the prompt, told to fly to a described target. This is the approach task's success rate, and the ceiling is 65 percent — the frontier can see where to go far more reliably than it can get there.

New Models (2)

Gemini 3.8 Flash — ELO 1781, #3
- AA GPQA Diamond: 95.25 (#1/604)
- AA MMMU-Pro: 85.61 (#1/252)
- LLM Stats (LVBench): 87.1 (#1/28)
- LLM Stats (Terminal-Bench 2.1): 89.4 (#1/35)
- Vals AI Finance Agent v2: 61.44 (#1/55)
- BioMysteryBench Human-Difficult: 56.5 (#1/14)
- LVBench: 87.8 (#1/43)
- Finance Agent v2: 61.4 (#1/42)
- LLM Stats (DeepSWE 1.1): 73.7 (#2/30)
- LLM Stats (GDP.pdf): 35.0 (#2/7)
- ...and 75 more
Muse Spark 1.3 — ELO 1774
- Tau3 Banking: 52.37 (#1/196)
- LLM Stats (AutomationBench): 49.4 (#1/17)
- LLM Stats (DeepSWE 1.1): 75.4 (#1/30)
- LLM Stats (Job Bench): 64.9 (#1/8)
- LLM Stats (MRCR v2 (8-needle)): 98.5 (#1/24)
- LLM Stats (Terminal-Bench 2.1): 88.8 (#2/35)
- LLM Stats (OSWorld 2.0): 66.9 (#3/10)
- LLM Stats (GDPval-AA): 1754.0 (#4/9)
- AA GPQA Diamond: 94.14 (#5/604)
- LLM Stats (DeepSearchQA): 89.4 (#5/10)
- ...and 20 more

Top-10 New Scores (144)

Claude Fable 5 on NL2Repo: 70.2 (#2)
Claude Fable 5 on SWE Atlas: 39.0 (#22)
Claude Fable 5.1 on AA-Briefcase: 55.35 (#8)
Claude Fable 5.1 on AA-LCR: 80.0 (#8)
Claude Fable 5.1 on AI Chess Leaderboard (Continuation): 1357.0 (#21)
Claude Fable 5.1 on Agents on Rails: 92.1 (#2)
Claude Fable 5.1 on Arena AI Code: 1765.37 (#1)
Claude Fable 5.1 on Chatbot Arena (Code): 1765.0 (#1)
Claude Fable 5.1 on Chatbot Arena (Text): 1504.0 (#3)
Claude Fable 5.1 on Computer Anthology Terminal Tasks (Claude Code): 55.2 (#1)
Claude Fable 5.1 on Computer Anthology Terminal Tasks (Terminus-2): 62.4 (#1)
Claude Fable 5.1 on Creative Writing (Lechmazur): 4.1 (#1)
Claude Fable 5.1 on CritPt: 29.7 (#211)
Claude Fable 5.1 on Deep20Bench: 12.11 (#2)
Claude Fable 5.1 on Design Arena (3D): 1409.0 (#3)
Claude Fable 5.1 on Design Arena (UI Components): 1377.0 (#2)
Claude Fable 5.1 on Design Arena (Website): 1330.0 (#4)
Claude Fable 5.1 on Epoch AI - Apex Agents: 44.4 (#3)
Claude Fable 5.1 on Epoch AI - Critpt: 31.14 (#2)
Claude Fable 5.1 on Epoch AI - Cursorbench: 73.4 (#1)
Claude Fable 5.1 on Epoch AI - ECI: 162.96 (#8)
Claude Fable 5.1 on Epoch AI - Proofbench: 100.0 (#1)
Claude Fable 5.1 on Epoch AI - Scicode: 62.04 (#1)
Claude Fable 5.1 on KernelBench Hub - Mega: 15.84 (#4)
Claude Fable 5.1 on LMArena WebDev Arena: 1765.37 (#1)
Claude Fable 5.1 on MineBench: 2040.0 (#3)
Claude Fable 5.1 on TaxCalcBench: 38.0 (#4)
Claude Fable 5.1 on WebDev Arena: 1765.37 (#1)
Claude Opus 5 on AA-Briefcase: 53.84 (#10)
Claude Opus 5 on BioMysteryBench Human-Difficult: 49.4 (#3)
Claude Opus 5 on Finance Agent v2: 58.6 (#3)
Claude Opus 5 on LVBench: 75.4 (#9)
Claude Opus 5 on SWE Atlas: 63.2 (#3)
Gemini 3.7 Flash on Agents on Rails: 71.4 (#14)
Gemini 3.7 Flash on Epoch AI - Critpt: 14.29 (#48)
Gemini 3.7 Flash on Epoch AI - Scicode: 57.87 (#8)
Gemini 3.7 Flash on Finance Agent v2: 59.0 (#2)
Gemini 3.8 Flash on AA CritPt: 18.29 (#37)
Gemini 3.8 Flash on AA GDPval: 1545.45 (#35)
Gemini 3.8 Flash on AA GPQA Diamond: 95.25 (#1)
Gemini 3.8 Flash on AA Humanity's Last Exam: 47.82 (#15)
Gemini 3.8 Flash on AA Long Context Reasoning: 82.0 (#3)
Gemini 3.8 Flash on AA MMMU-Pro: 85.61 (#1)
Gemini 3.8 Flash on AA Omniscience: 29.55 (#13)
Gemini 3.8 Flash on AA Omniscience - Business: 45.7 (#23)
Gemini 3.8 Flash on AA Omniscience - Health: 44.6 (#33)
Gemini 3.8 Flash on AA Omniscience - Humanities & Social Sciences: 54.1 (#22)
Gemini 3.8 Flash on AA Omniscience - Law: 60.1 (#12)
Gemini 3.8 Flash on AA Omniscience - Science, Engineering & Mathematics: 53.3 (#17)
Gemini 3.8 Flash on AA Omniscience - Software Engineering (SWE): 69.8 (#41)
Gemini 3.8 Flash on AA SciCode: 54.4 (#30)
Gemini 3.8 Flash on AA-Briefcase: 42.12 (#17)
Gemini 3.8 Flash on AA-Omniscience Accuracy: 54.6 (#24)
Gemini 3.8 Flash on AI Chess Leaderboard (Continuation): 1071.0 (#44)
Gemini 3.8 Flash on AI Chess Leaderboard (Reasoning): 1547.0 (#19)
Gemini 3.8 Flash on Agent Arena: 5.94 (#15)
Gemini 3.8 Flash on Agent Arena - Bash Recovery: 4.46 (#30)
Gemini 3.8 Flash on Agent Arena - Confirmed Success: 11.12 (#9)
Gemini 3.8 Flash on Agent Arena - Praise vs Complaint: 14.78 (#7)
Gemini 3.8 Flash on Agent Arena - Steerability: -1.43 (#36)
Gemini 3.8 Flash on Agent Arena - Tool Hallucination: -0.78 (#28)
Gemini 3.8 Flash on Arena AI Code: 1567.33 (#19)
Gemini 3.8 Flash on Artificial Analysis Intelligence Index: 58.68 (#18)
Gemini 3.8 Flash on BenchLM: 75.4 (#11)
Gemini 3.8 Flash on BenchmarkList ECI: 148.9 (#11)
Gemini 3.8 Flash on BioMysteryBench Human-Solvable: 88.8 (#2)
Gemini 3.8 Flash on Chatbot Arena (Code): 1567.0 (#19)
Gemini 3.8 Flash on Chatbot Arena (Text): 1494.0 (#8)
Gemini 3.8 Flash on Coarena: 1029.7 (#7)
Gemini 3.8 Flash on Coarena - Task Completion: 76.9 (#5)
Gemini 3.8 Flash on Design Arena (Website): 1311.0 (#14)
Gemini 3.8 Flash on Epoch AI - Critpt: 18.29 (#34)
Gemini 3.8 Flash on Epoch AI - Cursorbench: 69.2 (#9)
Gemini 3.8 Flash on Epoch AI - Proofbench: 48.0 (#15)
Gemini 3.8 Flash on Epoch AI - Scicode: 54.4 (#27)
Gemini 3.8 Flash on KernelBench Hub - CUDA: 36.37 (#4)
Gemini 3.8 Flash on KernelBench Hub - Mega: 2.74 (#6)
Gemini 3.8 Flash on LLM Stats (CharXiv-R): 86.2 (#16)
Gemini 3.8 Flash on LLM Stats (DeepSWE 1.1): 73.7 (#2)
Gemini 3.8 Flash on LLM Stats (GDP.pdf): 35.0 (#2)
Gemini 3.8 Flash on LLM Stats (GDPval-AA): 1545.0 (#7)
Gemini 3.8 Flash on LLM Stats (LVBench): 87.1 (#1)
Gemini 3.8 Flash on LLM Stats (OSWorld 2.0): 59.0 (#5)
Gemini 3.8 Flash on LLM Stats Score: 51.79 (#11)
Gemini 3.8 Flash on LM Market Cap LMC Score: 40.0 (#237)
Gemini 3.8 Flash on LMArena Text Arena: 1493.74 (#8)
Gemini 3.8 Flash on LMArena WebDev Arena: 1567.33 (#21)
Gemini 3.8 Flash on MineBench: 1893.0 (#8)
Gemini 3.8 Flash on RuneBench: 9333.0 (#4)
Gemini 3.8 Flash on SvelteBench: 100.0 (#11)
Gemini 3.8 Flash on Tau3 Banking: 45.77 (#13)
Gemini 3.8 Flash on Vals AI (Vals Index): 62.25 (#5)
Gemini 3.8 Flash on Vals AI BioMysteryBench: 62.22 (#7)
Gemini 3.8 Flash on Vals AI Code Migration: 36.55 (#17)
Gemini 3.8 Flash on Vals AI CyberBench: 43.75 (#21)
Gemini 3.8 Flash on Vals AI Excel Modeling: 72.2 (#5)
Gemini 3.8 Flash on Vals AI Finance Agent v2: 61.44 (#1)
Gemini 3.8 Flash on Vals AI GPQA: 94.44 (#4)
Gemini 3.8 Flash on Vals AI Harvey Legal Agent Bench: 10.0 (#9)
Gemini 3.8 Flash on Vals AI Legal Research Bench: 38.94 (#16)
Gemini 3.8 Flash on Vals AI LegalBench: 86.99 (#6)
Gemini 3.8 Flash on Vals AI LiveCodeBench: 89.48 (#3)
Gemini 3.8 Flash on Vals AI MMLU-Pro: 90.22 (#5)
Gemini 3.8 Flash on Vals AI MMMU: 89.08 (#4)
Gemini 3.8 Flash on Vals AI MedCode: 48.13 (#23)
Gemini 3.8 Flash on Vals AI MedScribe: 84.5 (#24)
Gemini 3.8 Flash on Vals AI MortgageTax: 65.34 (#44)
Gemini 3.8 Flash on Vals AI ProgramBench: 71.9 (#7)
Gemini 3.8 Flash on Vals AI ProofBench: 48.0 (#17)
Gemini 3.8 Flash on Vals AI Public Benefits Bench: 65.29 (#12)
Gemini 3.8 Flash on Vals AI SAGE: 35.06 (#57)
Gemini 3.8 Flash on Vals AI SWE-bench Verified: 80.0 (#24)
Gemini 3.8 Flash on Vals AI SkillsBench: 57.98 (#11)
Gemini 3.8 Flash on Vals AI TaxEval v2: 74.45 (#36)
Gemini 3.8 Flash on Vals AI Terminal-Bench 2.1: 81.27 (#4)
Gemini 3.8 Flash on Vals AI Vibe Code Bench: 78.65 (#10)
Gemini 3.8 Flash on VoxelBench: 1816.0 (#12)
Gemini 3.8 Flash on WebDev Arena: 1567.33 (#16)
Muse Spark 1.3 on AA CritPt: 26.0 (#18)
Muse Spark 1.3 on AA GDPval: 1753.92 (#8)
Muse Spark 1.3 on AA GPQA Diamond: 94.14 (#5)
Muse Spark 1.3 on AA Humanity's Last Exam: 49.07 (#11)
Muse Spark 1.3 on AA Long Context Reasoning: 79.33 (#14)
Muse Spark 1.3 on AA MMMU-Pro: 82.02 (#17)
Muse Spark 1.3 on AA Omniscience: 24.93 (#25)
Muse Spark 1.3 on AA Omniscience - Business: 38.3 (#50)
Muse Spark 1.3 on AA Omniscience - Health: 37.4 (#76)
Muse Spark 1.3 on AA Omniscience - Humanities & Social Sciences: 39.9 (#74)
Muse Spark 1.3 on AA Omniscience - Law: 38.3 (#58)
Muse Spark 1.3 on AA Omniscience - Science, Engineering & Mathematics: 46.9 (#49)
Muse Spark 1.3 on AA Omniscience - Software Engineering (SWE): 62.2 (#72)
Muse Spark 1.3 on AA SciCode: 58.56 (#6)
Muse Spark 1.3 on AA-Omniscience Accuracy: 43.83 (#62)
Muse Spark 1.3 on Artificial Analysis Intelligence Index: 62.09 (#6)
Muse Spark 1.3 on BenchmarkList ECI: 147.74 (#16)
Muse Spark 1.3 on Coarena: 1000.0 (#11)
Muse Spark 1.3 on Coarena - Task Completion: 0.0 (#18)
Muse Spark 1.3 on LLM Stats (DeepSearchQA): 89.4 (#5)
Muse Spark 1.3 on LLM Stats (GDPval-AA): 1754.0 (#4)
Muse Spark 1.3 on LLM Stats (OSWorld 2.0): 66.9 (#3)
Muse Spark 1.3 on LLM Stats (Terminal-Bench 2.1): 88.8 (#2)
Muse Spark 1.3 on LM Market Cap LMC Score: 80.8 (#80)
Muse Spark 1.3 on MineBench: 1832.0 (#14)
Muse Spark 1.3 on Tau3 Banking: 52.37 (#1)

New #1 Leaders (32)

Chatbot Arena (Code): Claude Fable 5.1 (Max) (1765.0) beat Claude Opus 5 (Max) by 77.0
WebDev Arena: Claude Fable 5.1 (Max) (1765.37) beat Claude Opus 5 (Max) by 74.73
Arena AI Code: Claude Fable 5.1 (Max) (1765.37) beat Claude Opus 5 (Max) by 74.73
LMArena WebDev Arena: Claude Fable 5.1 (Max) (1765.37) beat Claude Opus 5 (Max) by 74.73
Design Arena (Data Viz): Claude Fable 5.1 (1382.0) beat Kimi K3 by 12.0
LLM2014 Logic 2026-09: Claude Fable 5.1 (xhigh ~Estimated) (81.33) beat GPT-5.6 Sol (xHigh) by 11.3
BioMysteryBench Human-Difficult: Gemini 3.8 Flash (56.5) beat GPT-5.6 Terra by 7.1
SEAL - SWE Atlas - Test Writing: Claude Fable 5.1 (Claude Code) xHigh (67.04) beat Claude Opus 5 (Claude Code) xHigh by 4.82
LLM Stats (Job Bench): Muse Spark 1.3 (64.9) beat Hy4 preview by 3.2
Finance Agent v2: Gemini 3.8 Flash (61.4) beat Claude Opus 5 (Max) by 2.77
Epoch AI - Cursorbench: Claude Fable 5.1 (Max) (73.4) beat Grok 4.6 (xHigh) by 2.6
BioMysteryBench Human-Solvable: Claude Opus 5 (90.1) beat Claude Sonnet 5 by 2.6
NL2Repo: Claude Opus 5 (72.3) beat Claude Opus 4.8 by 2.6
Epoch AI - Apex Agents: Claude Fable 5.1 (Unknown) (47.4) beat Claude Fable 5 by 2.4
LLM Stats (DeepSWE 1.1): Muse Spark 1.3 (75.4) beat GPT-5.6 Sol by 2.4
LVBench: Gemini 3.8 Flash (87.8) beat Gemini 3.7 Flash by 2.4
SWE Atlas: Qwen3.8 Max 0902 (66.3) beat Hy4 preview claude-code by 2.3
SEAL - SWE Atlas - Refactoring: Claude Fable 5.1 (Claude Code) xHigh (56.67) beat Claude Fable 5 (Claude Code) xHigh by 1.91
Epoch AI - Scicode: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (62.04) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 1.85
LLM Stats (CyberGym): Gemini 3.8 Flash Cyber (86.2) beat GLM-5.3 by 1.7
LLM Stats (MRCR v2 (8-needle)): Muse Spark 1.3 (98.5) beat Gemini 3.7 Flash by 1.5
Computer Anthology Terminal Tasks (Claude Code): Claude Fable 5.1 (High) (55.2) beat Claude Fable 5 (High) by 1.2
Thinkingbox: Claude Opus 5 (66.5) beat GPT-5.4 by 1.14
Tau3 Banking: Muse Spark 1.3 (Max) (52.37) beat Qwen 3.8 Max by 1.03
Epoch AI - Proofbench: Claude Fable 5.1 (Max) (100.0) beat Claude Opus 5 (Max) by 1.0
Vals AI Finance Agent v2: Gemini 3.8 Flash (High) (61.44) beat Muse Spark 1.2 (xHigh) by 0.84
LLM Stats (AutomationBench): Muse Spark 1.3 (49.4) beat GLM-5.3 Flash by 0.6
LLM Stats (Terminal-Bench 2.1): Gemini 3.8 Flash (89.4) beat GPT-5.6 Sol by 0.6
Computer Anthology Terminal Tasks (Terminus-2): Claude Fable 5.1 (High) (62.4) beat Claude Opus 5 (High) by 0.4
AA GPQA Diamond: Gemini 3.8 Flash (High) (95.25) beat Grok 4.6 (High) by 0.3
Gert Labs Rankings: Claude Fable 5.1 (74.46) beat Claude Opus 5 by 0.28
AA MMMU-Pro: Gemini 3.8 Flash (High) (85.61) beat Gemini 3.7 Flash (High) by 0.12

The Aggregate Digest — 2026-09-02

2026-09-02T04:57:11.198104+00:00

Frontier

Best available model: Claude Fable 5.1 (1809) takes the crown from Claude Opus 5 (1786)
Best available model: Claude Fable 5.1 enters at #1 (1809 ELO) on 66 benchmarks

Daily

New Benchmarks (4)

Epoch AI - Frontiermath Tiers 1 3 V2 (Score): leader Claude Fable 5.1 (Max) (90.18), 103 models
The three lower tiers of Epoch's FrontierMath v2, spanning advanced undergraduate mathematics up through early-career research problems. Every question is original and vetted by working mathematicians, and this is the wide end of the board — a hundred-odd models, and the frontier now clears 90.
Epoch AI - Frontiermath Tier 4 V2 (Score): leader Claude Fable 5.1 (Max) (87.8), 54 models
Tier 4 of Epoch's FrontierMath v2: the research-level problems a working mathematician would need hours to solve. Held separately from tiers 1–3 because it is the part that was supposed to stay hard, and the top of the board has moved from single digits to the high eighties in under a year.
MERA v2 (Total score): leader Gemini 3.7 Flash (0.68), 37 models
The 2.0 text suite that replaced MERA v1 as the default board at mera.a-ai.ru: twelve newly written Russian tasks (SAGE, Humor, LIMUR, RUBIN, Riddles, SOB-Hard, Characters, Enantiosemy and four more) scored 0-1 as a total over the set. Answers stay private and submissions are run by the organisers, so the task mix differs from v1 and the two scores are not comparable.
GDP.pdf (Strict Pass Rate (%)): leader GPT-5.6 Sol (Max) (30.7), 34 models
Surge AI multimodal document-reasoning benchmark over native professional PDF pages containing visually structured layouts, tables, charts, and forms across finance, healthcare, legal, engineering, insurance, and related domains.

New Models (1)

Claude Fable 5.1 — ELO 1809, #1
- CursorBench 3.1: 73.4 (#1/50)
- FrontierMath - Tier 4 (v2): 87.8 (#1/54)
- FrontierMath - Tiers 1-3 (v2): 90.18 (#1/100)
- OTIS Mock AIME 2024-25: 100.0 (#1/287)
- Artificial Analysis Intelligence Index: 65.65 (#1/618)
- AA Humanity's Last Exam: 59.13 (#1/590)
- AA SciCode: 62.04 (#1/590)
- AA Omniscience: 43.45 (#1/502)
- AA GDPval: 1853.05 (#1/226)
- AA-Briefcase: 61.52 (#1/31)
- ...and 54 more

Top-10 New Scores (68)

Claude Fable 5 on MILU: 92.9 (#4)
Claude Fable 5.1 on AA CritPt: 31.14 (#2)
Claude Fable 5.1 on AA GDPval: 1853.05 (#1)
Claude Fable 5.1 on AA GPQA Diamond: 93.74 (#5)
Claude Fable 5.1 on AA Humanity's Last Exam: 59.13 (#1)
Claude Fable 5.1 on AA Long Context Reasoning: 80.0 (#7)
Claude Fable 5.1 on AA Omniscience: 43.45 (#1)
Claude Fable 5.1 on AA Omniscience - Business: 57.3 (#2)
Claude Fable 5.1 on AA Omniscience - Health: 57.4 (#1)
Claude Fable 5.1 on AA Omniscience - Humanities & Social Sciences: 66.1 (#1)
Claude Fable 5.1 on AA Omniscience - Law: 68.6 (#1)
Claude Fable 5.1 on AA Omniscience - Science, Engineering & Mathematics: 62.0 (#1)
Claude Fable 5.1 on AA Omniscience - Software Engineering (SWE): 92.4 (#1)
Claude Fable 5.1 on AA SciCode: 62.04 (#1)
Claude Fable 5.1 on AA-Briefcase: 61.52 (#1)
Claude Fable 5.1 on AA-Omniscience Accuracy: 67.23 (#1)
Claude Fable 5.1 on AI Chess Leaderboard (Reasoning): 1428.0 (#28)
Claude Fable 5.1 on ARC-AGI-1: 97.5 (#9)
Claude Fable 5.1 on ARC-AGI-2: 90.0 (#5)
Claude Fable 5.1 on Artificial Analysis Intelligence Index: 65.65 (#1)
Claude Fable 5.1 on BenchLM: 82.7 (#1)
Claude Fable 5.1 on BenchmarkList ECI: 157.38 (#2)
Claude Fable 5.1 on Chess Puzzles (Epoch AI): 47.0 (#11)
Claude Fable 5.1 on Coarena: 1002.8 (#9)
Claude Fable 5.1 on Coarena - Task Completion: 29.4 (#16)
Claude Fable 5.1 on CursorBench 3.1: 73.4 (#1)
Claude Fable 5.1 on Epoch AI - Mystery Game Puzzles: 58.0 (#2)
Claude Fable 5.1 on FrontierMath - Tier 4 (v2): 87.8 (#1)
Claude Fable 5.1 on FrontierMath - Tiers 1-3 (v2): 90.18 (#1)
Claude Fable 5.1 on Harvey Legal Agent Benchmark: 19.09 (#2)
Claude Fable 5.1 on LLM Stats (AutomationBench): 31.4 (#5)
Claude Fable 5.1 on LLM Stats Score: 54.8 (#5)
Claude Fable 5.1 on LM Market Cap LMC Score: 97.1 (#1)
Claude Fable 5.1 on MILU: 93.0 (#2)
Claude Fable 5.1 on OTIS Mock AIME 2024-25: 100.0 (#1)
Claude Fable 5.1 on RuneBench: 9813.0 (#3)
Claude Fable 5.1 on Senior SWE-Bench: 34.7 (#1)
Claude Fable 5.1 on SimpleQA Verified: 70.8 (#2)
Claude Fable 5.1 on Tau3 Banking: 47.22 (#7)
Claude Fable 5.1 on Toolathlon: 77.8 (#6)
Claude Fable 5.1 on Vals AI (Vals Index): 67.87 (#1)
Claude Fable 5.1 on Vals AI Excel Modeling: 76.67 (#1)
Claude Fable 5.1 on Vals AI Finance Agent v2: 58.88 (#3)
Claude Fable 5.1 on Vals AI GPQA: 93.43 (#8)
Claude Fable 5.1 on Vals AI Harvey Legal Agent Bench: 6.67 (#18)
Claude Fable 5.1 on Vals AI Legal Research Bench: 55.29 (#2)
Claude Fable 5.1 on Vals AI LegalBench: 88.51 (#2)
Claude Fable 5.1 on Vals AI LiveCodeBench: 90.52 (#1)
Claude Fable 5.1 on Vals AI MMLU-Pro: 92.38 (#1)
Claude Fable 5.1 on Vals AI MMMU: 90.64 (#1)
Claude Fable 5.1 on Vals AI MedCode: 53.51 (#7)
Claude Fable 5.1 on Vals AI MedScribe: 91.29 (#1)
Claude Fable 5.1 on Vals AI MortgageTax: 70.79 (#2)
Claude Fable 5.1 on Vals AI ProofBench: 100.0 (#1)
Claude Fable 5.1 on Vals AI Public Benefits Bench: 74.9 (#2)
Claude Fable 5.1 on Vals AI ReverseEngBench: 22.9 (#2)
Claude Fable 5.1 on Vals AI SAGE: 48.53 (#25)
Claude Fable 5.1 on Vals AI SkillsBench: 61.55 (#5)
Claude Fable 5.1 on Vals AI TaxEval v2: 75.96 (#9)
Claude Fable 5.1 on Vals AI Terminal-Bench 2.1: 85.02 (#2)
Claude Fable 5.1 on Vals AI Vibe Code Bench: 90.26 (#2)
Claude Mythos 5 on Toolathlon: 79.3 (#4)
Claude Mythos Preview on BenchmarkList ECI: 155.94 (#3)
Claude Opus 5 on AA-Omniscience Net Score: 49.0 (#4)
Claude Opus 5 on MILU: 92.1 (#7)
Claude Opus 5 on Toolathlon: 80.6 (#2)
GPT-5.6 Sol on AutomationBench: 26.3 (#5)
GPT-5.6 Sol on ProteinGym Hard: 35.5 (#9)

New #1 Leaders (25)

LLM Stats (GDPval-AA): Claude Fable 5.1 (1853.0) beat GLM-5.3 Flash by 80.0
AA GDPval: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (1853.05) beat Claude Opus 5 (Adaptive Reasoning, Max Effort) by 29.11
LLM Stats (OSWorld 2.0): Claude Fable 5.1 (77.9) beat Claude Opus 5 by 7.3
AA Omniscience - Health: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (57.4) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 3.68
AA Humanity's Last Exam: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (59.13) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 3.66
AA-Briefcase: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (61.52) beat Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) by 3.54
Vals AI Excel Modeling: Claude Fable 5.1 (Max) (76.67) beat Claude Fable 5 (Max) by 3.0
CursorBench 3.1: Claude Fable 5.1 (Max) (73.4) beat Grok 4.6 (xHigh) by 2.6
Artificial Analysis Intelligence Index: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (65.65) beat Claude Opus 5 (Adaptive Reasoning, Max Effort) by 2.6
AA Omniscience - Law: Claude Fable 5.1 (Adaptive Reasoning, Xhigh Effort, Default Fallback) (68.6) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 2.4
Mercor APEX: Claude Fable 5.1 (47.4) beat Claude Fable 5 by 2.4
AA-Omniscience Accuracy: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (67.23) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 1.88
AA SciCode: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (62.04) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 1.85
AA Omniscience - Humanities & Social Sciences: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (66.1) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 1.8
AA Omniscience - Software Engineering (SWE): Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (92.4) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 1.3
FrontierMath - Tiers 1-3 (v2): Claude Fable 5.1 (Max) (90.18) beat GPT-5.6 Sol (Max) by 1.06
AutomationBench: Claude Fable 5.1 with Opus 5 Fallback (Max) (31.4) beat Gemini 3.7 Flash (High) by 1.0
Vals AI MMLU-Pro: Claude Fable 5.1 (Max) (92.38) beat Claude Opus 5 (Max) by 0.79
Vals AI MMMU: Claude Fable 5.1 (Max) (90.64) beat Claude Opus 5 (Max) by 0.76
Vals AI LiveCodeBench: Claude Fable 5.1 (Max) (90.52) beat Claude Fable 5 (Max) by 0.74
AA Omniscience - Science, Engineering & Mathematics: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (62.0) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 0.72
Vals AI (Vals Index): Claude Fable 5.1 (Max) (67.87) beat Claude Opus 5 (Max) by 0.66
Vals AI MedScribe: Claude Fable 5.1 (Max) (91.29) beat Claude Opus 5 (Max) by 0.31
Vellum - Humanity's Last Exam: Claude Fable 5.1 (65.0) beat Claude Opus 5 by 0.3
AA Omniscience: Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback) (43.45) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 0.15

The Aggregate Digest — 2026-09-01

2026-09-01T05:45:14.477929+00:00

Daily

New Benchmarks (1)

LLM2014 Logic 2026-09 (Median Score): leader GPT-5.6 Sol (xHigh) (70.03), 43 models
September 2026 monthly snapshot of the LLM2014 project public leaderboard, scoring models on logic and reasoning problems from its continuously updated test set.

Top-10 New Scores (3)

Claude Mythos 5 on ExploitGym: 247.0 (#2)
Claude Opus 5 on ExploitGym: 191.0 (#3)
Gemini 3.7 Flash on Vals AI ReverseEngBench: 4.58 (#3)

The Aggregate Digest — 2026-08-31

2026-08-31T09:08:07.449360+00:00

Daily

New Benchmarks (3)

LLM Stats (SWE-Marathon) (Score (%)): leader GLM-5.3 (42.5), 5 models
llm-stats.com’s mirror of SWE-Marathon, the ultra-long-horizon software-engineering benchmark whose tasks run to building compilers, optimizing kernels and standing up production-grade services. It measures whether an agent holds quality across an extremely long trajectory; the roster here is self-reported and much smaller than the first-party board.
LLM Stats (Vision2Web) (Score (%)): leader Qwen 3.8 Max (69.0), 5 models
llm-stats.com’s mirror of Vision2Web, which asks multimodal models to turn a visual design or screenshot into a working web page and scores the end-to-end design-to-code result. Scores are self-reported and the roster is currently almost entirely one vendor’s models, so it reads as a vendor-reported slice rather than a broad comparison.
BenchmarkList ECI (Capability Index (ECI)): leader Claude Mythos 5 (162.93), 506 models
benchmarklist.com's site-wide capability index, the primary ranking score of its model directory. Despite the “ECI” label it is benchmarklist's own fit rather than Epoch's index republished — it correlates at about r=0.74 with Epoch's ECI on their shared models, with per-model gaps up to ~41 points, and scores far more models. Like LLM Stats Score and BenchGecko Score it is one aggregator's fused view, not an independent measurement.

New #1 Leaders (3)

LLM Stats (Job Bench): Hy4 preview (61.7) beat Qwen3.8-Flash-Next by 6.0
LLM Stats (WideSearch): Hy4 preview (83.9) beat Qwen 3.8 Max by 2.0
LLM Stats (CritPT): Hy4 preview (16.9) beat GLM-5.2 by 0.2

The Aggregate Digest — 2026-08-30

2026-08-30T10:45:53.611904+00:00

Daily

Top-10 New Scores (15)

Claude Opus 5 on Appwrite Arena (With Skills): 97.4 (#4)
Claude Opus 5 on Appwrite Arena (Without Skills): 97.4 (#3)
Claude Opus 5 on LLM2014 Logic 2026-07: 71.88 (#4)
Claude Opus 5 on LLM2014 Logic 2026-08: 64.74 (#4)
Claude Opus 5 on React Native Evals: 88.11 (#1)
Claude Opus 5 on Terminal-Bench 2.1: 51.8 (#1)
Claude Opus 5 on Terminal-Bench 2.1 (Claude Code): 51.8 (#1)
GPT-5.6 Sol on Appwrite Arena (With Skills): 96.9 (#8)
GPT-5.6 Sol on Appwrite Arena (Without Skills): 95.5 (#5)
GPT-5.6 Sol on Epoch AI - Mystery Game Puzzles: 33.0 (#17)
GPT-5.6 Sol on Terminal-Bench 2.1: 37.3 (#4)
Gemini 3.7 Flash on Appwrite Arena (With Skills): 96.0 (#13)
Gemini 3.7 Flash on Appwrite Arena (Without Skills): 93.6 (#10)
Gemini 3.7 Flash on Featherbench: 93.0 (#10)
Gemini 3.7 Flash on Featherbench - Rubric Quality: 8.8 (#8)

New #1 Leaders (3)

SWE-Milestone: GLM-5.3 (58.75) beat Claude Opus 4.8 (Max, 1M) by 6.91
Featherbench: GLM-5.3 (100.0) beat Gemini 3.6 Flash by 4.0
Appwrite Arena (With Skills): Muse Spark 1.2 (97.8) beat GPT-5.5 by 0.1

Weekly

New Benchmarks (16)

WorkSurface-Bench (Aggregate Score (0-1)): leader Gemini 3.1 Pro (Preview) (0.7649), 4 models
Enterprise-agent benchmark that splits a workspace into three knowledge surfaces — retrievable documents, structured tables and a dependency graph — and asks whether an agent routes each of 1,151 atomic questions to the right one, acquires the evidence and answers correctly. The published score is the all-tools ReAct setting, where every surface is exposed and the model has to choose; the aggregate weights answer 0.35, evidence 0.30, routing 0.25 and efficiency 0.10.
Pre-Flight (Accuracy (%)): leader GPT-5.5 (82.7), 42 models
Aviation operational knowledge: regulations, procedures and the documentation airline operations actually run on. Built because general benchmarks say nothing about whether a model reasons safely inside a domain where being confidently wrong grounds aircraft.
PredAct-Bench (Overall (%)): leader Gemini 3 Flash (63.4), 12 models
Tool-augmented dialogue where the tools lie. Most task-oriented benchmarks assume perfect tool output; this one injects controlled noise and scores whether the model still reaches the right multi-step decision in education, health and finance settings.
SaliTrap (Overall (%)): leader Claude Opus 4.7 (62.5), 12 models
Salience bias in everyday commonsense: models have learned to weight explicit stated conditions so heavily that an obvious unstated one gets ignored. The traps are ordinary situations where a human would never need the condition spelled out.
ContractScrub (F1 (0-1)): leader Gemini 3.1 Pro (Preview) (0.66), 10 models
Contract scrubbing: the final review pass over a transactional agreement, hunting errors and internal inconsistencies. Scored by F1 because both missing a real defect and inventing one are failures a lawyer would notice.
EpiBench (Accuracy (%)): leader Grok 4.20 (33.6), 9 models
Epitope understanding for antibody drug discovery: where an antibody binds an antigen, which determines functional blockade and escape resistance downstream. Scores sit in the twenties and thirties, so this is a long way from saturated.
CARE-Bench (Overall (%)): leader Llama 4 Scout (50.4), 10 models
Patient-facing triage: given a symptom question asked before any clinician contact, what should the user do next. Source-grounded and sequential, because the safety-relevant output is the recommended action rather than the explanation.
OrchBench (Score (0-1)): leader GPT-5.5 (0.81), 9 models
Multi-agent orchestration plans judged in isolation through deterministic simulation, so plan quality is separated from the worker models that would execute it. End-to-end evaluation conflates the two and cannot say which one failed.
Patient-Context Ambiguity (Accuracy (%)): leader Claude Opus 4.8 (91.8), 5 models
Short underspecified health queries that are linguistically clear but support several plausible answers depending on patient context. Scores whether the model resolves the ambiguity rather than confidently picking one reading.
Skill Coverage (Task Success Rate (%)): leader GPT-5.5 (53.26), 5 models
Task success as a function of which skills an agent has available, built to test whether more skills actually help. The paper's answer is that they often do not.
ORCA-bench (Root-Cause Accuracy (%)): leader Claude Sonnet 4.6 (30.9), 5 models
Root-cause analysis of incidents: given the symptoms and the telemetry, name what actually broke. Accuracy sits between 15 and 31 percent, so the gap to a competent on-call engineer is still wide.
OpenUI Generative UI Benchmark (Structural Validity (%)): leader Grok 4.6 (99.5), 30 models
OpenUI Generative UI Benchmark (Thesys) — generated user interfaces scored by the shipped OpenUI parser: a screen counts as valid only if it parses, has a root, resolves every reference, invents no components, uses valid props and is not truncated. Four generations per brief across 46 briefs; headline is structural validity percent.
Benchmarks.bio - BioSecBench-Function (Pass Rate (%)): leader Claude Opus 5 (50.3), 11 models
BioSecBench-Function (benchmarks.bio) — genomic function prediction under biosecurity framing: agentic tasks that ask a model to reason from sequence to function across 111 evaluations, scored as pass rate over gradeable trials with refusals reported separately.
LLM Stats (CoWorkBench) (Score (%)): leader Qwen 3.8 Max (74.8), 5 models
LLM Stats mirror of CoWorkBench, Qwen's cowork benchmark for long-horizon office and productivity tasks; the published roster is small and Qwen-heavy.
LLM Stats (GDPval-AA) (ELO): leader GLM-5.3 Flash (1773.0), 5 models
GDP validation benchmark testing economic data retrieval and reasoning, measuring accuracy on GDP-related factual questions.
LLM Stats (Job Bench) (Score (%)): leader Qwen3.8-Flash-Next (55.7), 5 models
LLM Stats mirror of Job Bench, scoring AI agents on realistic professional tasks that require multi-step planning.

New Models (201)

Claude Opus 5 — ELO 1791, #1
- React Native Evals: 88.11 (#1/25)
- OpenCompass LLM - Knowledge: 94.1 (#1/15)
- OpenCompass LLM - Reasoning: 68.4 (#1/15)
- OpenCompass LLM - Math: 77.3 (#1/15)
- OpenCompass Knowledge - Social Science: 94.6 (#1/15)
- OpenCompass Knowledge - Science: 97.5 (#1/15)
- OpenCompass Reasoning - Academic: 56.7 (#1/15)
- OpenCompass Math - Competition: 74.7 (#1/15)
- Terminal-Bench 2.1: 51.8 (#1/10)
- Terminal-Bench 2.1 (Claude Code): 51.8 (#1/5)
- ...and 26 more
Gemini 3.7 Flash — ELO 1782, #3
- OpenCompass Knowledge - Engineering: 95.8 (#1/15)
- OpenCompass Knowledge - Common Sense: 92.9 (#1/15)
- OpenCompass Math - College: 85.6 (#1/15)
- HieroglyphBench: 55.0 (#1/23)
- OpenCompass LLM - Knowledge: 94.0 (#2/15)
- OpenCompass LLM - Math: 76.8 (#2/15)
- AI for Education Visual Maths - Geometry: 90.77 (#2/76)
- OpenCompass Knowledge - Humanities: 94.2 (#3/15)
- BoundaryBench (Unrestricted): 79.8 (#3/14)
- AI for Education Visual Maths: 89.45 (#3/76)
- ...and 27 more
Claude Fable 5 — ELO 1777, #4
- PostTrainBench: 41.79 (#1/12)
- AI for Education Visual Maths - Number and Operations: 86.49 (#2/76)
- AI for Education Visual Maths - Statistics and Probability: 71.43 (#3/76)
- AI for Education Visual Maths: 86.92 (#4/76)
- AI for Education Visual Maths - Geometry: 83.85 (#5/76)
- AI for Education Visual Maths - Measurement: 97.3 (#5/76)
- OSWorld-Verified: 85.0 (#5/132)
- MCP Atlas: 83.3 (#9/51)
- AI for Education Visual Maths - Algebra: 92.31 (#16/76)
GPT-5.6 Sol — ELO 1776, #5
- Chatbot Arena (Search): 1257.0 (#1/34)
- AI for Education Visual Maths: 91.14 (#1/76)
- AI for Education Visual Maths - Geometry: 90.77 (#1/76)
- AI for Education Visual Maths - Number and Operations: 89.19 (#1/76)
- WildClawBench: 67.2 (#1/66)
- OpenCompass LLM - Code: 96.3 (#2/15)
- OpenCompass Reasoning - Academic: 55.8 (#2/15)
- OpenCompass Code - Comprehensive: 96.3 (#2/15)
- AI for Education Visual Maths - Algebra: 96.15 (#2/76)
- AI for Education Visual Maths - Measurement: 97.3 (#2/76)
- ...and 20 more
Kimi K3 — ELO 1756, #7
- AA-LCR: 82.6667 (#2/210)
- NatureBench: 14.44 (#4/16)
- OpenCompass Knowledge - Social Science: 90.4 (#5/15)
- OpenCompass Math - Competition: 68.9 (#5/15)
- CAIS Vision Capabilities Index: 63.2 (#5/38)
- PostTrainBench: 31.96 (#6/12)
- OpenCompass LLM - Math: 74.1 (#6/15)
- OpenCompass Knowledge - Science: 92.5 (#6/15)
- OpenCompass Reasoning - Academic: 51.7 (#6/15)
- Appwrite Arena (Without Skills): 95.0 (#6/24)
- ...and 18 more
Grok 4.6 — ELO 1754, #8
- Benchmarks.bio - BioSecBench-Refusal: 64.2 (#1/18)
- OpenCompass Knowledge - Humanities: 95.8 (#1/15)
- OpenCompass Knowledge - Social Science: 92.5 (#2/15)
- OpenCompass Math - College: 83.8 (#2/15)
- Appwrite Arena (With Skills): 97.8 (#2/24)
- OpenCompass LLM - Knowledge: 93.8 (#3/15)
- OpenCompass Knowledge - Science: 95.8 (#3/15)
- OpenCompass Knowledge - Engineering: 94.2 (#4/15)
- OpenCompass Knowledge - Common Sense: 90.8 (#5/15)
- Agent Arena - Confirmed Success: 13.17 (#6/54)
- ...and 25 more
Muse Spark 1.2 — ELO 1754, #9
- Appwrite Arena (With Skills): 97.8 (#1/24)
- AA-LCR: 83.3333 (#1/210)
- EQ-Bench Longform Writing: 81.5 (#6/130)
- Appwrite Arena (Without Skills): 94.8 (#7/24)
- SimpleQA Verified: 60.3 (#9/66)
- Epoch AI - Proofbench: 43.0 (#13/24)
- CritPt: 17.7 (#21/284)
- DuelLab Overall: 52.8 (#23/121)
- AIIQ Composite IQ: 120.0 (#23/130)
- WebDev Arena: 1533.68 (#27/116)
- ...and 2 more
Qwen 3.8 Max — ELO 1750, #10
- OpenCompass Math - Competition: 70.8 (#2/15)
- OpenCompass LLM - Code: 95.5 (#3/15)
- OpenCompass Code - Comprehensive: 95.5 (#3/15)
- OpenCompass LLM - Reasoning: 65.2 (#4/15)
- OpenCompass LLM - Math: 75.0 (#4/15)
- OpenCompass Reasoning - Common: 78.1 (#4/15)
- OpenCompass Reasoning - Academic: 52.3 (#4/15)
- NatureBench: 11.11 (#6/16)
- OpenCompass Math - College: 79.2 (#7/15)
- OpenCompass Knowledge - Social Science: 87.9 (#8/15)
- ...and 9 more
GPT-5.5 — ELO 1748, #11
- OpenCompass LLM - Code: 96.5 (#1/15)
- OpenCompass Code - Comprehensive: 96.5 (#1/15)
- OpenCompass LLM - Reasoning: 66.9 (#2/15)
- OpenCompass Reasoning - Common: 78.6 (#3/15)
- OpenCompass Reasoning - Academic: 55.2 (#3/15)
- GeneBench: 25.0 (#3/16)
- OpenCompass Math - Competition: 68.9 (#4/15)
- OpenCompass Knowledge - Social Science: 90.0 (#6/15)
- SWE Atlas: 45.43 (#6/29)
- SimpleQA Verified: 63.0 (#8/66)
- ...and 13 more
Gemini 3.6 Flash — ELO 1747, #13
- AI for Education Visual Maths - Measurement: 97.3 (#1/76)
- Crosby micro1 RedlineBench: 51.0 (#4/11)
- AI for Education Visual Maths - Statistics and Probability: 71.43 (#6/76)
- AI for Education Visual Maths - Number and Operations: 70.27 (#10/76)
- AI for Education Visual Maths: 78.48 (#12/76)
- AA-LCR: 79.0 (#12/210)
- AI for Education Visual Maths - Geometry: 73.85 (#15/76)
- AIIQ Composite IQ: 123.0 (#20/130)
- AI for Education Visual Maths - Algebra: 88.46 (#23/76)
Qwen3.8-Flash-Next — ELO 1747, #14
- LLM Stats (RealWorldQA): 88.5 (#1/30)
- RealWorldQA: 88.5 (#1/48)
- LLM Stats (AndroidWorld): 84.5 (#2/6)
- LLM Stats (ERQA): 72.3 (#2/25)
- LLM Stats (MathVision): 95.7 (#2/34)
- LLM Stats (Agents' Last Exam): 51.2 (#3/13)
- LLM Stats (CharXiv-R): 90.6 (#4/53)
- LLM Stats (Toolathlon): 73.5 (#4/39)
- LLM Stats (LVBench): 76.6 (#5/26)
- LVBench: 76.6 (#5/40)
- ...and 26 more
Gemini 3.1 Pro (Preview) — ELO 1737, #15
- SimpleQA Verified: 73.5 (#1/66)
- Epoch AI - Enigma Eval: 36.78 (#3/46)
- DeepResearchBench: 47.8 (#22/41)
- ForecastBench: 64.0 (#115/300)
Qwen 3.7 Max — ELO 1736, #16
- LLM GPU Kernel Generation: 97.1 (#3/6)
- StructureClaw (Automatic Workflow): 89.3 (#4/10)
GPT-5.6 Terra — ELO 1732, #17
- Crosby micro1 RedlineBench: 49.3 (#6/11)
- Appwrite Arena (Without Skills): 93.5 (#11/24)
- AA-Briefcase: 34.14 (#15/26)
- Appwrite Arena (With Skills): 95.2 (#19/24)
- APEX-Agents-AA: 0.39 (#31/56)
- Epoch AI - Mystery Game Puzzles: 19.0 (#47/110)
GLM-5.3 — ELO 1732, #18
- SWE-Milestone: 58.75 (#1/27)
- Humanity's Last Exam (Self-Reported, With Tools): 62.5 (#1/27)
- Featherbench: 100.0 (#1/17)
- SecIT Bench (Pydantic AI): 81.44 (#1/16)
- KernelBench Hub - Hard: 63.19 (#2/9)
- OpenCompass Knowledge - Engineering: 95.4 (#2/15)
- ProphetArena: 0.9541 (#3/20)
- Terminal-Bench 2.1: 41.8 (#3/10)
- Terminal-Bench 2.1 (Claude Code): 41.8 (#3/5)
- Featherbench - Rubric Quality: 9.3 (#3/17)
- ...and 42 more
Claude Opus 4.8 — ELO 1729, #19
- OpenCompass Reasoning - Common: 80.6 (#1/15)
- PostTrainBench: 33.84 (#4/12)
- LLM2014 Logic 2026-05: 68.32 (#5/43)
- LLM2014 Logic 2026-06: 68.32 (#5/43)
- OpenCompass LLM - Reasoning: 64.1 (#6/15)
- OpenCompass Math - Competition: 68.3 (#8/15)
- OpenCompass LLM - Math: 71.5 (#9/15)
- OpenCompass Knowledge - Humanities: 90.4 (#10/15)
- OpenCompass Knowledge - Social Science: 87.5 (#10/15)
- OpenCompass Knowledge - Science: 89.6 (#10/15)
- ...and 15 more
Gemini 3.5 Flash — ELO 1729, #20
- AA-LCR: 81.0 (#5/210)
- AI for Education Visual Maths - Measurement: 94.59 (#8/76)
- MCP Atlas: 83.6 (#8/51)
- AI for Education Visual Maths - Number and Operations: 70.27 (#11/76)
- AI for Education Visual Maths - Geometry: 74.62 (#14/76)
- AI for Education Visual Maths: 75.95 (#17/76)
- AI for Education Visual Maths - Statistics and Probability: 42.86 (#18/76)
- CritPt: 10.9 (#35/284)
- AI for Education Visual Maths - Algebra: 73.08 (#40/76)
DeepSeek V4 Pro (0813) — ELO 1728, #21
- AA-Briefcase: 43.52 (#9/26)
- CritPt: 18.0 (#19/284)
- AA-LCR: 75.3333 (#36/210)
- Bullshit Benchmark: 32.7 (#65/188)
- Epoch AI - ECI: 155.18 (#93/507)
- LM Market Cap LMC Score: 40.0 (#228/403)
Grok 4.5 — ELO 1726, #22
- Chatbot Arena (Search): 1213.0 (#8/34)
- PostTrainBench: 23.45 (#10/12)
- AI for Education Visual Maths - Geometry: 75.38 (#12/76)
- FrontierCode: 56.6 (#13/47)
- AI for Education Visual Maths: 77.64 (#14/76)
- AI for Education Visual Maths - Measurement: 94.59 (#14/76)
- AI for Education Visual Maths - Statistics and Probability: 42.86 (#17/76)
- AI for Education Visual Maths - Algebra: 92.31 (#19/76)
- AI for Education Visual Maths - Number and Operations: 64.86 (#20/76)
Seed 2.1 Pro — ELO 1726, #23
- LLM2014 Logic 2026-06: 60.15 (#8/43)
- LLM2014 Logic 2026-07: 49.35 (#13/45)
- LLM2014 Logic 2026-08: 45.78 (#18/46)
GLM-5.3 Flash — ELO 1725, #24
- LLM Stats (AutomationBench): 48.8 (#1/14)
- LLM Stats (MVBench): 77.8 (#1/18)
- LLM Stats (Toolathlon): 78.4 (#1/39)
- LLM Stats (Artificial Analysis): 57.0 (#2/8)
- Toolathlon: 78.4 (#2/41)
- AA GDPval: 1764.66 (#3/219)
- Humanity's Last Exam (Self-Reported, With Tools): 55.3 (#3/27)
- KernelBench Hub - Mega: 13.64 (#4/5)
- LLM Stats (NL2Repo): 56.3 (#4/20)
- Chatbot Arena (Code): 1634.0 (#5/121)
- ...and 58 more
Gemini 3 Pro — ELO 1720, #25
- Arena AI Code: 1437.8 (#50/113)
Gemini 3 Pro (Preview) — ELO 1716, #27
- DeepResearchBench: 46.3 (#28/41)
- Epoch AI - ECI: 153.06 (#134/507)
Qwen3.8 2.4T A95B — ELO 1714, #28
- AI for Education SEND: 86.24 (#12/235)
- CritPt: 20.0 (#16/284)
- AI for Education Pedagogy - Maths: 91.27 (#17/243)
- OpenRouter Tau2-Bench Airline: 76.7 (#19/118)
- AIIQ Composite IQ: 119.0 (#26/130)
- AI for Education Pedagogy: 87.99 (#33/243)
- AI for Education Pedagogy - Science: 91.8 (#33/243)
- OpenRouter GPQA Diamond: 84.3 (#34/119)
- AA-LCR: 75.3333 (#34/210)
- AI for Education Pedagogy - Primary: 92.02 (#36/243)
- ...and 3 more
Claude Opus 4.7 — ELO 1712, #29
- PostTrainBench: 28.56 (#8/12)
- MCP Atlas: 79.1 (#19/51)
- CritPt: 5.1 (#61/284)
GPT-5.4 — ELO 1709, #30
- SWE Atlas: 40.8 (#11/29)
- PostTrainBench: 19.0 (#12/12)
- OSWorld-Verified: 75.0 (#25/132)
- Epoch AI - Mystery Game Puzzles: 28.0 (#26/110)
- APEX-Agents-AA: 0.33 (#35/56)
- CyberGym: 79.0 (#36/54)
- CritPt: 7.4 (#48/284)
- Arena AI Code: 1390.56 (#69/113)
GPT-5.6 Luna — ELO 1708, #31
- Crosby micro1 RedlineBench: 55.5 (#3/11)
- Appwrite Arena (Without Skills): 92.9 (#12/24)
- AA-Briefcase: 37.98 (#13/26)
- CritPt: 20.6 (#15/284)
- Appwrite Arena (With Skills): 95.2 (#18/24)
- Guesswork 2026-08: 1.0249 (#22/33)
- APEX-Agents-AA: 0.36 (#33/56)
- FrontierMath - Tiers 1-3 (v2): 41.4 (#44/92)
- Epoch AI - Mystery Game Puzzles: 17.0 (#56/110)
GPT-5.3 Codex — ELO 1706, #32
- Arena AI Code: 1408.38 (#59/113)
Claude Sonnet 5 — ELO 1705, #33
- LLM2014 Logic 2026-07: 43.14 (#16/45)
- DuelLab Overall: 54.3 (#19/121)
- LLM2014 Logic 2026-08: 39.57 (#21/46)
- Epoch AI - ECI: 155.87 (#92/507)
GPT-5.2 Codex — ELO 1705, #34
- Arena AI Code: 1338.28 (#87/113)
Qwen 3.8 27B — ELO 1704, #35
- WebDev Arena: 1594.78 (#9/116)
- Agents on Rails: 76.2 (#10/16)
- LMArena WebDev Arena: 1594.78 (#10/30)
- Agent Arena - Confirmed Success: 8.05 (#11/54)
- LVBench: 72.4 (#12/40)
- Bullshit Benchmark: 76.4 (#13/188)
- AI for Education Pedagogy - Science: 93.44 (#13/243)
- Toolathlon: 67.1 (#15/41)
- Epoch AI - Proofbench: 16.0 (#21/24)
- AA-LCR: 77.3333 (#23/210)
- ...and 32 more
Kimi K2.7 Code — ELO 1703, #36
- WildClawBench: 46.9 (#26/66)
Claude Opus 4.6 — ELO 1702, #38
- LLM2014 Logic 2026-02: 78.02 (#1/46)
- VitaBench 2.0: 50.3 (#1/35)
- LLM2014 Logic 2026-03: 77.23 (#2/42)
- LLM2014 Logic 2026-04: 77.23 (#3/42)
- LLM2014 Logic 2026-05: 76.48 (#3/43)
- DeepResearchBench: 51.4 (#8/41)
- LLM2014 Logic 2026-06: 40.16 (#17/43)
- LVBench: 63.0 (#18/40)
- RealWorldQA: 73.9 (#20/48)
- CritPt: 2.8 (#85/284)
Gemini 3 Flash — ELO 1702, #39
- DeepResearchBench: 49.0 (#15/41)
- OmniDocBench 1.5: 90.37 (#20/73)
- Arena AI Code: 1437.85 (#49/113)
Seed 2.1 Turbo — ELO 1702, #41
- WildClawBench: 62.8 (#3/66)
- MCP Atlas: 80.3 (#15/51)
- Toolathlon: 49.1 (#36/41)
Gemini 3 Flash (Preview) — ELO 1701, #42
- SimpleQA Verified: 66.8 (#5/66)
- DeepResearchBench: 49.8 (#11/41)
- CritPt: 1.4 (#106/284)
GLM-5.2 — ELO 1697, #44
- SWE Atlas: 48.12 (#4/29)
- PostTrainBench: 31.7 (#7/12)
- MCP Atlas: 77.8 (#21/51)
- CritPt: 16.7 (#25/284)
- FrontierMath - Tiers 1-3 (v2): 54.74 (#33/92)
- Epoch AI - Mystery Game Puzzles: 15.0 (#67/110)
- Chess Puzzles (Epoch AI): 14.0 (#106/182)
- OTIS Mock AIME 2024-25: 75.56 (#113/256)
Qwen 3.7 Plus — ELO 1697, #45
- RealWorldQA: 86.9 (#2/48)
- IMO-AnswerBench: 86.0 (#7/15)
- Toolathlon: 50.6 (#29/41)
- FrontierMath - Tiers 1-3 (v2): 34.39 (#49/92)
- Epoch AI - Mystery Game Puzzles: 17.0 (#53/110)
Hy3 — ELO 1694, #46
- LLM2014 Logic 2026-07: 44.6 (#14/45)
- NL2Repo: 45.6 (#15/37)
- MCP Atlas: 79.1 (#18/51)
- LLM2014 Logic 2026-08: 39.51 (#22/46)
Claude Opus 4.5 — ELO 1693, #47
- LLM2014 Code 2025-11 - C#: 9.92 (#2/26)
- LLM2014 Code 2025-11 - C++: 9.92 (#2/26)
- LLM2014 Code 2025-11 - Golang: 9.1 (#2/26)
- LLM2014 Code 2025-11 - Python: 9.33 (#2/26)
- LLM2014 Code 2025-11: 90.75 (#3/26)
- LLM2014 Code 2025-11 - Java: 8.58 (#4/26)
- LLM2014 Code 2025-11 - TypeScript: 7.6 (#6/26)
- LLM2014 Logic 2025-11: 37.68 (#27/53)
- LLM2014 Logic 2026-01: 33.08 (#29/45)
- LLM2014 Logic 2025-12: 33.08 (#33/51)
- ...and 2 more
DeepSeek V4 Flash (0731) — ELO 1693, #48
- Toolathlon: 70.3 (#14/41)
- Vals AI ProgramBench: 57.29 (#20/42)
- Design Arena (SVG): 1219.0 (#40/112)
- AA-LCR: 74.3333 (#43/210)
- Epoch AI - ECI: 154.35 (#117/507)
- LM Market Cap LMC Score: 40.0 (#238/403)
GPT-5 — ELO 1690, #50
- VitaBench 2.0: 44.1 (#8/35)
- RealWorldQA: 82.75 (#8/48)
- CyberGym: 60.2 (#26/54)
- HMMT 2025: 88.33 (#27/67)
- AA-LCR: 76.3333 (#31/210)
- NL2Repo: 21.7 (#34/37)
- FrontierMath - Tiers 1-3 (v2): 37.19 (#46/92)
- Arena AI Code: 1419.14 (#57/113)
- Epoch AI - Mystery Game Puzzles: 16.0 (#60/110)
- CritPt: 1.1 (#118/284)
Claude Opus 4.5 (20251101) — ELO 1690, #51
- DeepResearchBench: 54.8 (#3/41)
GPT-5.2 — ELO 1688, #53
- HMMT 2025: 100.0 (#1/67)
- Epoch AI - Mystery Game Puzzles: 22.0 (#39/110)
- CritPt: 7.9 (#47/284)
- Arena AI Code: 1417.81 (#58/113)
DeepSeek V4 Pro — ELO 1687, #54
- Featherbench: 96.0 (#2/17)
- IMO-AnswerBench: 89.8 (#5/15)
- VitaBench 2.0: 45.6 (#6/35)
- MathArena Apex: 38.3 (#8/48)
- Featherbench - Rubric Quality: 8.7 (#11/17)
- SWE Atlas: 27.15 (#23/29)
- CritPt: 12.9 (#30/284)
Kimi K2.6 — ELO 1686, #55
- IMO-AnswerBench: 91.1 (#1/15)
- OpenCompass LLM - Math: 75.7 (#3/15)
- OpenCompass Math - College: 82.7 (#3/15)
- OpenCompass Reasoning - Common: 78.1 (#5/15)
- OpenCompass Knowledge - Humanities: 92.5 (#6/15)
- OpenCompass Knowledge - Engineering: 90.8 (#7/15)
- OpenCompass Knowledge - Common Sense: 89.6 (#7/15)
- OpenCompass Math - Competition: 68.6 (#7/15)
- OpenCompass LLM - Knowledge: 89.8 (#8/15)
- OpenCompass LLM - Reasoning: 62.9 (#8/15)
- ...and 8 more
Inkling — ELO 1685, #56
- MCP Atlas: 76.0 (#25/51)
- Arena AI Code: 1407.6 (#60/113)
GLM-5.1 — ELO 1683, #57
- VitaBench 2.0: 42.0 (#11/35)
- CritPt: 4.6 (#64/284)
Seed 2.0 Pro — ELO 1683, #59
- LLM2014 Logic 2026-02: 69.55 (#4/46)
- LLM2014 Logic 2026-03: 68.36 (#4/42)
- LLM2014 Logic 2026-04: 66.02 (#6/42)
- LLM2014 Logic 2026-05: 57.4 (#8/43)
- LLM2014 Logic 2026-06: 55.84 (#10/43)
- VitaBench 2.0: 42.8 (#10/35)
Claude Sonnet 4.6 — ELO 1678, #61
- DeepResearchBench: 50.4 (#9/41)
- OSWorld-Verified: 78.5 (#19/132)
- LLM2014 Logic 2026-02: 30.17 (#28/46)
- LLM2014 Logic 2026-03: 29.38 (#29/42)
- LLM2014 Logic 2026-04: 26.52 (#33/42)
- LLM2014 Logic 2026-05: 19.89 (#34/43)
- LLM2014 Logic 2026-06: 16.77 (#34/43)
- SimpleQA Verified: 35.5 (#37/66)
- CritPt: 0.9 (#136/284)
Qwen 3.6 Plus — ELO 1678, #62
- AI for Education Visual Maths - Geometry: 80.77 (#7/76)
- AI for Education Visual Maths - Algebra: 96.15 (#8/76)
- AI for Education Visual Maths: 80.59 (#9/76)
- AI for Education Visual Maths - Measurement: 94.59 (#13/76)
- AI for Education Visual Maths - Statistics and Probability: 42.86 (#16/76)
- AI for Education Visual Maths - Number and Operations: 62.16 (#24/76)
- FrontierMath - Tiers 1-3 (v2): 32.28 (#54/92)
Qwen 3.5 397B A17B — ELO 1678, #63
- Epoch AI - Mystery Game Puzzles: 18.0 (#50/110)
- FrontierMath - Tiers 1-3 (v2): 29.47 (#56/92)
- Arena AI Code: 1399.23 (#64/113)
- CritPt: 0.9 (#128/284)
O3 — ELO 1676, #64
- LLM2014 Logic 2025-04: 85.85 (#1/29)
- LLM2014 Logic 2025-05: 85.14 (#1/32)
- LLM2014 Logic 2025-06: 83.95 (#1/38)
- LLM2014 Logic 2025-08: 71.31 (#4/45)
- FrontierMath - Tiers 1-3 (v2): 33.33 (#52/92)
- OSWorld-Verified: 23.0 (#119/132)
MiMo-V2.5-Pro — ELO 1676, #65
- CritPt: 1.1 (#115/284)
GPT-5.1 Codex — ELO 1676, #66
- Arena AI Code: 1336.28 (#88/113)
GPT-5.3 — ELO 1674, #67
- SWE Atlas: 32.6 (#19/29)
Grok 4.20 0309 — ELO 1673, #69
- ForecastBench: 64.1 (#110/300)
DeepSeek V4 Flash — ELO 1670, #70
- WebDev Arena: 1431.07 (#63/116)
- CritPt: 3.4 (#74/284)
Gemini 3.5 Flash Lite — ELO 1670, #71
- AA-LCR: 74.6667 (#41/210)
- AIIQ Composite IQ: 108.0 (#61/130)
- CritPt: 0.0 (#283/284)
GPT-5.1 — ELO 1669, #72
- Epoch AI - Mystery Game Puzzles: 19.0 (#48/110)
- Arena AI Code: 1391.01 (#68/113)
- CritPt: 0.0 (#212/284)
MiniMax-M3 — ELO 1668, #73
- OpenCompass Knowledge - Science: 90.8 (#8/15)
- OpenCompass Knowledge - Engineering: 88.3 (#9/15)
- OpenCompass LLM - Knowledge: 86.6 (#13/15)
- OpenCompass Knowledge - Humanities: 87.1 (#14/15)
- OpenCompass Knowledge - Common Sense: 84.2 (#14/15)
- OpenCompass Reasoning - Common: 69.9 (#14/15)
- OpenCompass Reasoning - Academic: 38.1 (#14/15)
- OpenCompass Math - College: 74.3 (#14/15)
- OpenCompass LLM - Reasoning: 54.0 (#15/15)
- OpenCompass LLM - Math: 63.0 (#15/15)
- ...and 5 more
O3 (2025-04-16) — ELO 1668, #74
- Epoch AI - Mystery Game Puzzles: 29.0 (#23/110)
Inkling Small — ELO 1667, #75
- Arena AI Code: 1401.56 (#62/113)
- AIIQ Composite IQ: 106.0 (#63/130)
- WebDev Arena: 1401.56 (#68/116)
- AA-LCR: 69.3333 (#89/210)
Muse Glimmer 30B — ELO 1667, #76
- AA-LCR: 80.0 (#7/210)
- Arena AI Code: 1359.0 (#79/113)
- CritPt: 2.6 (#89/284)
- AIIQ Composite IQ: 92.0 (#111/130)
Claude Sonnet 4.5 — ELO 1665, #77
- LLM2014 Code 2025-11 - Java: 9.22 (#2/26)
- LLM2014 Code 2025-11 - TypeScript: 8.92 (#3/26)
- LLM2014 Code 2025-11: 81.67 (#4/26)
- LLM2014 Logic 2025-10: 69.78 (#4/50)
- LLM2014 Code 2025-11 - Golang: 7.85 (#5/26)
- LLM2014 Code 2025-11 - Python: 8.88 (#5/26)
- LLM2014 Code 2025-11 - C++: 7.17 (#6/26)
- LLM2014 Logic 2025-11: 66.7 (#6/53)
- LLM2014 Code 2025-11 - C#: 7.72 (#7/26)
- LLM2014 Logic 2026-01: 60.83 (#9/45)
- ...and 11 more
GPT-5.4 Mini — ELO 1665, #78
- FrontierCode: 43.01 (#31/47)
- APEX-Agents-AA: 0.28 (#38/56)
- FrontierMath - Tiers 1-3 (v2): 24.56 (#60/92)
- Arena AI Code: 1397.08 (#66/113)
- CritPt: 2.9 (#83/284)
- Epoch AI - Mystery Game Puzzles: 11.0 (#84/110)
Grok 4.3 — ELO 1665, #79
- AI for Education Visual Maths - Geometry: 45.38 (#42/76)
- AI for Education Visual Maths - Measurement: 75.68 (#42/76)
- AI for Education Visual Maths: 49.79 (#45/76)
- AI for Education Visual Maths - Algebra: 57.69 (#49/76)
- AI for Education Visual Maths - Number and Operations: 40.54 (#53/76)
- CritPt: 4.9 (#62/284)
- AI for Education Visual Maths - Statistics and Probability: 14.29 (#65/76)
- Arena AI Code: 1356.73 (#82/113)
Gemini 2.5 Pro — ELO 1664, #80
- LLM2014 Code 2025-11 - Golang: 8.87 (#3/26)
- LLM2014 Code 2025-11: 74.36 (#6/26)
- LLM2014 Code 2025-11 - C#: 7.77 (#6/26)
- LLM2014 Code 2025-11 - C++: 7.15 (#8/26)
- LLM2014 Code 2025-11 - Python: 7.9 (#8/26)
- LLM2014 Code 2025-11 - Java: 6.35 (#12/26)
- LLM2014 Code 2025-11 - TypeScript: 6.58 (#16/26)
- VitaBench 2.0: 33.1 (#24/35)
- Arena AI Code: 1225.69 (#106/113)
Kimi K2.5 — ELO 1664, #81
- HMMT 2025: 93.33 (#16/67)
- CyberGym: 41.3 (#20/54)
- SimpleQA Verified: 34.3 (#39/66)
- OSWorld-Verified: 63.3 (#44/132)
- APEX-Agents-AA: 0.12 (#51/56)
- Arena AI Code: 1435.92 (#52/113)
- CritPt: 0.6 (#150/284)
Grok 4.20 Beta (0309) — ELO 1664, #82
- Arena AI Code: 1373.56 (#74/113)
Qwen 3.6 27B — ELO 1661, #86
- FrontierMath - Tiers 1-3 (v2): 35.09 (#48/92)
- Epoch AI - Mystery Game Puzzles: 7.0 (#96/110)
- CritPt: 0.9 (#127/284)
DeepSeek V3.2 Speciale — ELO 1660, #87
- HMMT 2025: 97.5 (#6/67)
GLM-5 — ELO 1657, #89
- Arena AI Code: 1435.46 (#53/113)
- CritPt: 0.0 (#226/284)
MiMo-V2-Pro — ELO 1657, #90
- Arena AI Code: 1433.51 (#55/113)
Claude Opus 4.1 (20250805) — ELO 1656, #92
- Arena AI Code: 1388.83 (#70/113)
GLM-5V Turbo — ELO 1653, #96
- Arena AI Code: 1399.83 (#63/113)
Gemini 3.1 Flash Lite — ELO 1652, #97
- DeepResearchBench: 37.3 (#37/41)
- FrontierMath - Tiers 1-3 (v2): 21.4 (#64/92)
Gemini 2.5 Flash (Preview 05-20) — ELO 1647, #99
- LLM2014 Logic 2025-05: 55.28 (#9/32)
GPT-5 Mini — ELO 1645, #100
- RealWorldQA: 80.26 (#11/48)
- HMMT 2025: 89.17 (#25/67)
- FrontierMath - Tiers 1-3 (v2): 18.25 (#72/92)
- Epoch AI - Mystery Game Puzzles: 10.0 (#87/110)
- CritPt: 0.0 (#216/284)
Claude Opus 4 — ELO 1645, #101
- LLM2014 Logic 2025-05: 60.12 (#8/32)
- LLM2014 Logic 2025-06: 53.48 (#11/38)
- LLM2014 Logic 2025-08: 43.68 (#18/45)
- LLM2014 Logic 2025-09: 40.23 (#19/46)
- LLM2014 Logic 2025-10: 40.23 (#22/50)
- LLM2014 Logic 2025-11: 36.88 (#29/53)
Qwen 3.5 122B A10B — ELO 1644, #102
- Arena AI Code: 1357.68 (#80/113)
- CritPt: 0.6 (#145/284)
Gemini 3.1 Flash Lite (Preview) — ELO 1643, #104
- MCP Atlas: 57.1 (#46/51)
- Arena AI Code: 1254.07 (#99/113)
Qwen 3 Max — ELO 1643, #105
- VitaBench 2.0: 28.4 (#29/35)
Step 3.5 Flash — ELO 1642, #106
- AA-LCR: 48.0 (#183/210)
Grok 4 — ELO 1641, #107
- LLM2014 Logic 2025-08: 78.22 (#3/45)
- LLM2014 Logic 2025-09: 74.79 (#3/46)
- LLM2014 Logic 2025-10: 74.97 (#3/50)
- LLM2014 Code 2025-07: 55.12 (#5/19)
- LLM2014 Logic 2025-11: 69.52 (#5/53)
- LLM2014 Code 2025-09 - Java: 6.22 (#6/20)
- LLM2014 Code 2025-09: 37.44 (#8/19)
- LLM2014 Code 2025-09 - C#: 6.8 (#10/20)
- LLM2014 Code 2025-09 - Python: 5.76 (#11/20)
- LLM2014 Code 2025-09 - Golang: 4.06 (#13/20)
- ...and 2 more
GLM-4.7 — ELO 1641, #108
- Arena AI Code: 1434.29 (#54/113)
- CritPt: 0.0 (#228/284)
LongCat 2.0 — ELO 1641, #109
- WritingBench: 83.8 (#2/55)
- IMO-AnswerBench: 81.8 (#12/15)
- AIIQ Composite IQ: 98.0 (#91/130)
MiMo-V2.5 — ELO 1640, #110
- Arena AI Code: 1437.8 (#51/113)
Hy3-preview — ELO 1639, #112
- LLM2014 Logic 2026-04: 56.63 (#11/42)
- LLM2014 Logic 2026-05: 47.01 (#12/43)
- LLM2014 Logic 2026-06: 46.12 (#14/43)
- Arena AI Code: 1356.33 (#83/113)
- CritPt: 0.3 (#179/284)
Kimi K2 — ELO 1638, #113
- NL2Repo: 22.7 (#32/37)
Claude Sonnet 4 — ELO 1638, #114
- LLM2014 Code 2025-07: 66.98 (#1/19)
- LLM2014 Code 2025-09: 61.01 (#2/19)
- LLM2014 Code 2025-09 - TypeScript: 8.89 (#2/20)
- LLM2014 Code 2025-09 - C#: 8.31 (#3/20)
- LLM2014 Code 2025-09 - Java: 7.13 (#3/20)
- LLM2014 Code 2025-09 - Golang: 6.11 (#4/20)
- LLM2014 Vision 2025-07: 41.21 (#4/16)
- LLM2014 Logic 2025-05: 68.22 (#5/32)
- LLM2014 Code 2025-09 - Python: 6.94 (#6/20)
- LLM2014 Logic 2025-06: 62.96 (#8/38)
- ...and 3 more
Nemotron 3 Ultra — ELO 1637, #116
- IMO-AnswerBench: 88.6 (#6/15)
- AI for Education Pedagogy - Maths: 88.89 (#39/243)
- Epoch AI - Mystery Game Puzzles: 20.0 (#44/110)
- AI for Education SEND: 80.73 (#58/235)
- AI for Education Pedagogy - Secondary: 86.01 (#62/243)
- AI for Education Pedagogy: 86.21 (#64/243)
- OTIS Mock AIME 2024-25: 86.67 (#71/256)
- AI for Education Pedagogy - Primary: 88.26 (#78/243)
- AI for Education Pedagogy - Science: 87.43 (#81/243)
- AI for Education Pedagogy - Social studies: 80.91 (#103/243)
- ...and 2 more
Qwen 3.5 Flash — ELO 1637, #117
- Epoch AI - Mystery Game Puzzles: 20.0 (#43/110)
- FrontierMath - Tiers 1-3 (v2): 18.25 (#71/92)
- Arena AI Code: 1238.12 (#104/113)
Qwen 3.5 27B — ELO 1636, #118
- Roboflow Vision Evals - Visual Understanding: 71.64 (#16/77)
- Arena AI Code: 1357.46 (#81/113)
- CritPt: 0.3 (#163/284)
MiniMax-M2.7 — ELO 1636, #119
- VitaBench 2.0: 34.5 (#22/35)
- Arena AI Code: 1398.14 (#65/113)
Grok 4 Fast — ELO 1636, #120
- Arena AI Code: 1161.87 (#112/113)
- CritPt: 0.0 (#196/284)
DeepSeek V3.2 — ELO 1633, #126
- HMMT 2025: 92.5 (#19/67)
- Arena AI Code: 1360.31 (#78/113)
- CritPt: 0.9 (#135/284)
Claude 3.7 Sonnet (20250219) — ELO 1633, #127
- OSWorld-Verified: 35.8 (#90/132)
O4 Mini (2025-04-16) — ELO 1633, #128
- Epoch AI - Mystery Game Puzzles: 5.0 (#102/110)
Grok 4.1 Fast — ELO 1630, #129
- Arena AI Code: 1239.97 (#102/113)
- CritPt: 0.0 (#195/284)
MiniMax-M2.5 — ELO 1630, #130
- Arena AI Code: 1383.9 (#73/113)
Qwen 3.6 35B A3B — ELO 1629, #131
- AI for Education Visual Maths - Statistics and Probability: 71.43 (#4/76)
- AI for Education Visual Maths - Algebra: 92.31 (#12/76)
- AI for Education Visual Maths - Number and Operations: 67.57 (#15/76)
- AI for Education Visual Maths - Measurement: 91.89 (#19/76)
- AI for Education Visual Maths: 72.15 (#20/76)
- AI for Education Visual Maths - Geometry: 63.85 (#21/76)
- FrontierMath - Tiers 1-3 (v2): 17.54 (#74/92)
DeepSeek V3.2 Exp — ELO 1626, #135
- Arena AI Code: 1271.77 (#96/113)
- CritPt: 1.4 (#110/284)
O4 Mini — ELO 1626, #136
- LLM2014 Code 2025-09 - Python: 9.17 (#1/20)
- LLM2014 Code 2025-07: 66.75 (#2/19)
- LLM2014 Code 2025-09 - C#: 8.65 (#2/20)
- LLM2014 Code 2025-09 - Golang: 7.35 (#2/20)
- LLM2014 Code 2025-09 - Java: 7.89 (#2/20)
- LLM2014 Logic 2025-04: 80.87 (#2/29)
- LLM2014 Logic 2025-05: 80.16 (#2/32)
- LLM2014 Logic 2025-06: 78.97 (#2/38)
- LLM2014 Vision 2025-07: 53.4 (#2/16)
- LLM2014 Code 2025-09 - C++: 6.04 (#3/20)
- ...and 4 more
GLM-4.6 — ELO 1626, #137
- VitaBench 2.0: 34.2 (#23/35)
- Arena AI Code: 1340.18 (#86/113)
- CritPt: 0.0 (#231/284)
Qwen 3 235B A22B 2507 — ELO 1626, #138
- WritingBench: 82.34 (#3/55)
- Epoch AI - Mystery Game Puzzles: 9.0 (#89/110)
O1 — ELO 1624, #139
- ComplexFuncBench: 47.6 (#9/20)
- FrontierMath - Tiers 1-3 (v2): 14.74 (#78/92)
DeepSeek V3.1 Terminus — ELO 1623, #140
- CritPt: 0.0 (#266/284)
Gemini 2.5 Flash — ELO 1621, #143
- LLM2014 Code 2025-09 - Java: 6.97 (#4/20)
- LLM2014 Code 2025-09 - C#: 7.98 (#5/20)
- LLM2014 Logic 2025-04: 63.69 (#7/29)
- LLM2014 Code 2025-07: 45.04 (#9/19)
- LLM2014 Code 2025-09: 36.34 (#11/19)
- LLM2014 Code 2025-09 - Python: 5.28 (#16/20)
- LLM2014 Code 2025-09 - Golang: 2.88 (#17/20)
- LLM2014 Code 2025-09 - TypeScript: 5.15 (#17/20)
- LLM2014 Logic 2025-05: 42.85 (#17/32)
- LLM2014 Code 2025-09 - C++: 2.43 (#19/20)
- ...and 2 more
GPT-4.1 — ELO 1620, #145
- CyberGym: 7.23 (#8/54)
- FrontierMath - Tiers 1-3 (v2): 5.96 (#85/92)
MiniMax-M2.1 — ELO 1620, #146
- Arena AI Code: 1386.99 (#71/113)
DeepSeek R1 0528 — ELO 1618, #150
- VitaBench 2.0: 39.6 (#15/35)
Qwen 3 VL 235B A22B Instruct — ELO 1618, #151
- OmniDocBench 1.5: 89.15 (#28/73)
Claude Sonnet 4 (20250514) — ELO 1618, #152
- OSWorld-Verified: 43.9 (#77/132)
Gemma 4 31B — ELO 1617, #154
- LM Market Cap LMC Score: 80.5 (#71/403)
- Arena AI Code: 1363.56 (#76/113)
- CritPt: 0.0 (#238/284)
Qwen 3 235B A22B 2507 Instruct — ELO 1617, #155
- WritingBench: 80.26 (#8/55)
GLM-4.5 — ELO 1616, #156
- VitaBench 2.0: 30.7 (#27/35)
DeepSeek V3.1 — ELO 1615, #157
- HMMT 2025: 85.83 (#28/67)
- CritPt: 0.0 (#267/284)
Grok Code Fast 1 — ELO 1612, #162
- Arena AI Code: 1164.51 (#111/113)
Ling-3.0-flash — ELO 1612, #164
- ArtifactsBench: 77.0 (#1/8)
- Multi-IF: 87.7 (#14/15)
- MCP Atlas: 65.5 (#39/51)
- AA-LCR: 67.0 (#105/210)
- AIIQ Composite IQ: 92.0 (#112/130)
Claude Haiku 4.5 — ELO 1609, #167
- LLM2014 Code 2025-11 - Golang: 5.03 (#15/26)
- LLM2014 Code 2025-11 - C++: 5.43 (#17/26)
- LLM2014 Code 2025-11 - C#: 5.45 (#19/26)
- LLM2014 Code 2025-11: 51.44 (#22/26)
- LLM2014 Code 2025-11 - Java: 4.5 (#22/26)
- LLM2014 Code 2025-11 - TypeScript: 5.58 (#22/26)
- LLM2014 Code 2025-11 - Python: 4.87 (#23/26)
- AI for Education Visual Maths - Measurement: 81.08 (#33/76)
- AI for Education Visual Maths - Statistics and Probability: 28.57 (#33/76)
- LLM2014 Logic 2026-01: 16.95 (#41/45)
- ...and 8 more
Qwen 3.5 35B A3B — ELO 1608, #169
- Arena AI Code: 1249.75 (#100/113)
- CritPt: 0.6 (#144/284)
GPT-5.1 Codex Mini — ELO 1608, #171
- Arena AI Code: 1243.74 (#101/113)
KAT-Coder-Pro V1 — ELO 1605, #174
- Arena AI Code: 1255.43 (#98/113)
Claude 3.5 Sonnet — ELO 1602, #175
- Video-MME: 60.0 (#31/51)
MiMo-V2-Flash — ELO 1602, #176
- Arena AI Code: 1330.17 (#89/113)
GPT-5.4 Nano — ELO 1601, #177
- APEX-Agents-AA: 0.25 (#41/56)
- CritPt: 5.1 (#59/284)
- FrontierMath - Tiers 1-3 (v2): 20.35 (#65/92)
- Epoch AI - Mystery Game Puzzles: 9.0 (#91/110)
- AA GDPval: 738.5 (#143/219)
Mercury 2 — ELO 1600, #180
- Arena AI Code: 1166.17 (#110/113)
Qwen 3 Next 80B A3B — ELO 1597, #182
- LLM2014 Logic 2025-09: 43.59 (#13/46)
O3 Mini — ELO 1597, #183
- ComplexFuncBench: 17.6 (#14/20)
- FrontierMath - Tiers 1-3 (v2): 10.53 (#80/92)
Nemotron 3 Super — ELO 1596, #188
- AA-LCR: 60.3333 (#139/210)
MiniMax-M2 — ELO 1595, #189
- LLM2014 Code 2025-11 - Python: 7.57 (#10/26)
- LLM2014 Code 2025-11 - C++: 5.82 (#11/26)
- LLM2014 Code 2025-11 - C#: 5.95 (#17/26)
- LLM2014 Code 2025-11 - Golang: 3.2 (#20/26)
- LLM2014 Code 2025-11 - Java: 4.7 (#21/26)
- LLM2014 Code 2025-11 - TypeScript: 4.9 (#25/26)
- Arena AI Code: 1297.0 (#94/113)
Mistral Medium 3.5 — ELO 1595, #190
- Arena AI Code: 1265.04 (#97/113)
- CritPt: 0.0 (#282/284)
O3 Mini (2025-01-31) — ELO 1591, #195
- Epoch AI - Mystery Game Puzzles: 7.0 (#95/110)
GPT-4o — ELO 1590, #196
- Video-MME: 71.9 (#7/51)
Devstral 2 — ELO 1590, #200
- Arena AI Code: 1193.51 (#108/113)
Qwen 3 Coder 480B A35B Instruct — ELO 1590, #201
- Arena AI Code: 1273.25 (#95/113)
Gemma 4 12B — ELO 1590, #202
- AI for Education Pedagogy - Technology: 85.85 (#29/243)
- AI for Education SEND: 77.06 (#99/235)
- AI for Education Pedagogy - Primary: 85.45 (#102/243)
- AI for Education Pedagogy - Secondary: 81.29 (#102/243)
- AI for Education Pedagogy: 81.65 (#105/243)
- AI for Education Pedagogy - Maths: 80.16 (#108/243)
- AI for Education Pedagogy - Science: 82.51 (#109/243)
- AI for Education Pedagogy - Social studies: 78.18 (#116/243)
GPT-OSS-120B — ELO 1589, #203
- Epoch AI - Mystery Game Puzzles: 2.0 (#109/110)
Nemotron 3.5 Lightning — ELO 1588, #208
- AI for Education Pedagogy - Social studies: 81.82 (#95/243)
- AIIQ Composite IQ: 95.0 (#103/130)
- AI for Education Pedagogy - Science: 81.97 (#113/243)
- AI for Education SEND: 74.31 (#114/235)
- AI for Education Pedagogy: 78.75 (#119/243)
- AI for Education Pedagogy - Secondary: 77.99 (#119/243)
- AI for Education Pedagogy - Primary: 83.1 (#121/243)
- AI for Education Pedagogy - Maths: 74.6 (#140/243)
- AI for Education Pedagogy - Technology: 75.47 (#158/243)
- AA-LCR: 55.3333 (#159/210)
- ...and 1 more
GPT-4.1 Mini — ELO 1587, #210
- LLM2014 Code 2025-07: 50.87 (#6/19)
- LLM2014 Logic 2025-04: 54.55 (#10/29)
- LLM2014 Logic 2025-05: 53.52 (#10/32)
- LLM2014 Logic 2025-06: 48.53 (#14/38)
- RealWorldQA: 78.69 (#15/48)
- LLM2014 Logic 2025-08: 33.51 (#26/45)
- FrontierMath - Tiers 1-3 (v2): 6.67 (#84/92)
- Epoch AI - Mystery Game Puzzles: 7.0 (#98/110)
GLM-4.6V — ELO 1585, #215
- CritPt: 0.0 (#230/284)
GPT-4o (2024-11-20) — ELO 1580, #223
- ComplexFuncBench: 66.5 (#1/20)
GPT-4o (2024-08-06) — ELO 1579, #227
- ComplexFuncBench: 60.5 (#5/20)
- FrontierMath - Tiers 1-3 (v2): 0.35 (#91/92)
Gemma 4 26B A4B — ELO 1578, #232
- BIG-Bench Extra Hard: 64.8 (#2/17)
- MedXpertQA: 58.1 (#10/26)
- AI for Education Visual Maths - Measurement: 83.78 (#32/76)
- AI for Education Visual Maths - Number and Operations: 51.35 (#33/76)
- AI for Education Visual Maths: 58.65 (#35/76)
- AI for Education Visual Maths - Geometry: 52.31 (#35/76)
- AI for Education Visual Maths - Algebra: 76.92 (#37/76)
- AI for Education Visual Maths - Statistics and Probability: 14.29 (#50/76)
- Arena AI Code: 1361.65 (#77/113)
- CritPt: 0.0 (#240/284)
Qwen 3.5 9B — ELO 1577, #236
- MedXpertQA: 49.9 (#11/26)
- LVBench: 70.0 (#14/40)
- Chess Puzzles (Epoch AI): 12.0 (#117/182)
- OTIS Mock AIME 2024-25: 61.67 (#146/256)
- CritPt: 0.3 (#162/284)
GPT-5 Nano — ELO 1573, #243
- HMMT 2025: 74.17 (#39/67)
- FrontierMath - Tiers 1-3 (v2): 5.96 (#86/92)
- Epoch AI - Mystery Game Puzzles: 9.0 (#88/110)
- CritPt: 0.0 (#214/284)
Mistral Large 3 — ELO 1572, #245
- Arena AI Code: 1230.14 (#105/113)
Gemini 2.5 Flash Lite — ELO 1569, #247
- CritPt: 0.0 (#253/284)
Claude Haiku 4.5 (20251001) — ELO 1568, #249
- DeepResearchBench: 45.5 (#29/41)
- SimpleQA Verified: 13.2 (#62/66)
- Arena AI Code: 1327.41 (#90/113)
Mistral Small 4 — ELO 1568, #253
- CritPt: 0.3 (#171/284)
Gemini 1.5 Pro — ELO 1562, #262
- Video-MME: 75.0 (#2/51)
- PhysicsFinals: 38.4 (#4/33)
- Natural2Code: 42.3 (#7/24)
Qwen 3 32B — ELO 1553, #282
- MemPoison - Clean Task: 94.92 (#5/10)
- MemPoison - Poisoned Task: 2.18 (#9/10)
GLM-4.7 Flash — ELO 1552, #284
- CritPt: 0.0 (#227/284)
GLM-4.5V — ELO 1552, #286
- CritPt: 0.0 (#233/284)
Qwen 2.5 14B — ELO 1543, #309
- MemPoison - Poisoned Task: 2.64 (#6/10)
- MemPoison - Clean Task: 94.28 (#7/10)
Qwen 2.5 VL 72B Instruct — ELO 1540, #316
- OmniDocBench 1.5: 87.02 (#40/73)
QwQ-32B — ELO 1537, #327
- Chess Puzzles (Epoch AI): 5.0 (#149/182)
- OTIS Mock AIME 2024-25: 59.17 (#150/256)
Qwen 3.5 4B — ELO 1536, #328
- MedXpertQA: 42.9 (#14/26)
- LVBench: 66.4 (#16/40)
GPT-4 (0613) — ELO 1536, #330
- Epoch AI - Mystery Game Puzzles: 12.0 (#78/110)
Devstral Medium — ELO 1535, #332
- Arena AI Code: 1080.11 (#113/113)
GPT-OSS-20B — ELO 1532, #341
- CritPt: 1.4 (#101/284)
- OTIS Mock AIME 2024-25: 65.28 (#139/256)
- Chess Puzzles (Epoch AI): 4.0 (#156/182)
- Epoch AI - ECI: 136.81 (#363/507)
Gemini 1.5 Flash — ELO 1521, #359
- Video-MME: 70.3 (#11/51)
EXAONE 4.0 32B — ELO 1516, #364
- CritPt: 0.0 (#257/284)
Gemma 3 27B — ELO 1514, #367
- BIG-Bench Extra Hard: 4.9 (#12/17)
Gemma 3 12B (IT) — ELO 1513, #370
- Chess Puzzles (Epoch AI): 0.0 (#175/182)
- OTIS Mock AIME 2024-25: 16.67 (#207/256)
GPT-4o Mini — ELO 1511, #376
- ComplexFuncBench: 38.6 (#12/20)
- Video-MME: 64.8 (#22/51)
- FrontierMath - Tiers 1-3 (v2): 0.7 (#89/92)
GPT-4o Mini (2024-07-18) — ELO 1511, #377
- Epoch AI - Mystery Game Puzzles: 12.0 (#82/110)
Phi-4 — ELO 1508, #383
- Chess Puzzles (Epoch AI): 1.0 (#170/182)
Granite 4.1 8B — ELO 1508, #385
- Arena AI Code: 1191.25 (#109/113)
Qwen 3 8B — ELO 1496, #401
- MemPoison - Clean Task: 95.17 (#3/10)
- MemPoison - Poisoned Task: 2.76 (#5/10)
- Chess Puzzles (Epoch AI): 5.0 (#150/182)
- OTIS Mock AIME 2024-25: 56.11 (#156/256)
Gemma 3 12B — ELO 1489, #416
- BIG-Bench Extra Hard: 4.5 (#13/17)
Qwen 3.5 2B — ELO 1466, #452
- MedXpertQA: 26.9 (#19/26)
- LVBench: 57.1 (#19/40)
- HMMT 2025: 22.9 (#60/67)
Gemma 4 E4B — ELO 1463, #465
- CritPt: 0.3 (#184/284)
Gemma 3 4B (IT) — ELO 1457, #476
- Chess Puzzles (Epoch AI): 0.0 (#173/182)
- OTIS Mock AIME 2024-25: 7.5 (#219/256)
Llama 3.1 8B Instruct — ELO 1454, #481
- Chess Puzzles (Epoch AI): 0.0 (#177/182)
Llama 3.1 8B — ELO 1439, #512
- MemPoison - Poisoned Task: 3.29 (#2/10)
- MemPoison - Clean Task: 91.21 (#10/10)
GPT-3.5 Turbo (0125) — ELO 1428, #534
- Epoch AI - Mystery Game Puzzles: 3.0 (#108/110)
Gemma 4 E2B — ELO 1425, #536
- CritPt: 0.0 (#237/284)
Gemma 3 4B — ELO 1408, #563
- BIG-Bench Extra Hard: 3.4 (#16/17)
Qwen 3.5 0.8B — ELO 1398, #576
- LVBench: 45.1 (#23/40)
- MedXpertQA: 17.1 (#26/26)
Gemma 3 1B (IT) — ELO 1370, #609
- Chess Puzzles (Epoch AI): 0.0 (#174/182)
- OTIS Mock AIME 2024-25: 1.11 (#249/256)
Arcee Trinity Large — ELO 1605
- Arena AI Code: 1238.37 (#103/113)
Claude Mythos Preview — ELO 1779
- OSWorld-Verified: 85.4 (#3/132)
Doubao-Seed-1.6 — ELO 1628
- VitaBench 2.0: 32.6 (#26/35)
GLM-4 32B — ELO 1566
- MemPoison - Clean Task: 94.83 (#6/10)
- MemPoison - Poisoned Task: 2.64 (#7/10)
GPT-5.1 ChatGPT — ELO 1647
- AI Chess Leaderboard (Continuation): 1363.0 (#19/261)
- AI Chess Leaderboard (Reasoning): 837.0 (#98/320)
GPT-5.6 Pro Sol — ELO 1780
- Epoch AI - ECI: 161.06 (#26/507)
Grok 4.1 — ELO 1650
- Arena AI Code: 1209.85 (#107/113)
Kimi K2 Turbo — ELO 1654
- Arena AI Code: 1322.87 (#92/113)
LFM2.5-2.6B — ELO 1512
- BenchLM: 43.0 (#181/200)
Laguna M.1 — ELO 1570
- Arena AI Code: 1347.31 (#84/113)
Laguna XS.2 — ELO 1548
- Arena AI Code: 1301.97 (#93/113)
Nova 2 Lite — ELO 1550
- ParseBench: 40.43 (#50/56)
Qwen 3.7 Flash — ELO 1642
- Epoch AI - Mystery Game Puzzles: 15.0 (#65/110)
- FrontierMath - Tiers 1-3 (v2): 19.3 (#67/92)
- AIIQ Composite IQ: 92.0 (#110/130)
Seed 1.8 — ELO 1640
- LLM2014 Logic 2025-12: 63.06 (#9/51)
Solar Pro 3 — ELO 1552
- AIIQ Composite IQ: 85.0 (#121/130)
Solar Pro 4 — ELO 1649
- MCP Atlas: 61.4 (#42/51)
- AA-LCR: 70.6667 (#72/210)
- AIIQ Composite IQ: 100.0 (#85/130)
Spark X2 — ELO 1562
- LLM2014 Logic 2026-02: 28.21 (#31/46)
- LLM2014 Logic 2026-03: 25.04 (#34/42)
- LLM2014 Logic 2026-04: 20.55 (#38/42)
- LLM2014 Logic 2026-05: 14.04 (#40/43)
- LLM2014 Logic 2026-06: 11.14 (#40/43)
- LLM2014 Logic 2026-07: 6.21 (#44/45)
deepseek-llm-67B-chat — ELO 1490
- Chess Puzzles (Epoch AI): 0.0 (#176/182)
- OTIS Mock AIME 2024-25: 0.83 (#252/256)
uitars-1.5-7B — ELO 1467
- MobileGym-Bench: 13.8 (#8/10)

Top-10 New Scores (107)

Claude Fable 5 on AI for Education Visual Maths: 86.92 (#4)
Claude Fable 5 on AI for Education Visual Maths - Algebra: 92.31 (#16)
Claude Fable 5 on AI for Education Visual Maths - Geometry: 83.85 (#5)
Claude Fable 5 on AI for Education Visual Maths - Measurement: 97.3 (#5)
Claude Fable 5 on AI for Education Visual Maths - Number and Operations: 86.49 (#2)
Claude Fable 5 on AI for Education Visual Maths - Statistics and Probability: 71.43 (#3)
Claude Fable 5 on MCP Atlas: 83.3 (#9)
Claude Fable 5 on OSWorld-Verified: 85.0 (#5)
Claude Fable 5 on PostTrainBench: 41.79 (#1)
Claude Mythos Preview on OSWorld-Verified: 85.4 (#3)
Claude Opus 5 on AA-LCR: 78.6667 (#15)
Claude Opus 5 on AI for Education Visual Maths: 80.59 (#10)
Claude Opus 5 on AI for Education Visual Maths - Algebra: 96.15 (#3)
Claude Opus 5 on AI for Education Visual Maths - Geometry: 75.38 (#13)
Claude Opus 5 on AI for Education Visual Maths - Measurement: 94.59 (#12)
Claude Opus 5 on AI for Education Visual Maths - Number and Operations: 75.68 (#6)
Claude Opus 5 on AI for Education Visual Maths - Statistics and Probability: 71.43 (#2)
Claude Opus 5 on APEX v1: 68.6 (#2)
Claude Opus 5 on Appwrite Arena (With Skills): 97.4 (#4)
Claude Opus 5 on Appwrite Arena (Without Skills): 97.4 (#3)
Claude Opus 5 on CAIS Risk Index: 44.0 (#8)
Claude Opus 5 on CAIS Vision Capabilities Index: 63.9 (#3)
Claude Opus 5 on CritPt: 29.1 (#5)
Claude Opus 5 on LLM2014 Logic 2026-07: 71.88 (#4)
Claude Opus 5 on LLM2014 Logic 2026-08: 64.74 (#4)
Claude Opus 5 on MCP Atlas: 85.8 (#3)
Claude Opus 5 on OpenCompass Code - Comprehensive: 93.3 (#4)
Claude Opus 5 on OpenCompass Knowledge - Common Sense: 92.1 (#3)
Claude Opus 5 on OpenCompass Knowledge - Engineering: 93.3 (#5)
Claude Opus 5 on OpenCompass Knowledge - Humanities: 92.9 (#4)
Claude Opus 5 on OpenCompass Knowledge - Science: 97.5 (#1)
Claude Opus 5 on OpenCompass Knowledge - Social Science: 94.6 (#1)
Claude Opus 5 on OpenCompass LLM - Code: 93.3 (#4)
Claude Opus 5 on OpenCompass LLM - Knowledge: 94.1 (#1)
Claude Opus 5 on OpenCompass LLM - Math: 77.3 (#1)
Claude Opus 5 on OpenCompass LLM - Reasoning: 68.4 (#1)
Claude Opus 5 on OpenCompass Math - College: 79.9 (#5)
Claude Opus 5 on OpenCompass Math - Competition: 74.7 (#1)
Claude Opus 5 on OpenCompass Reasoning - Academic: 56.7 (#1)
Claude Opus 5 on OpenCompass Reasoning - Common: 80.1 (#2)
Claude Opus 5 on PostTrainBench: 35.04 (#3)
Claude Opus 5 on React Native Evals: 88.11 (#1)
Claude Opus 5 on Terminal-Bench 2.1: 51.8 (#1)
Claude Opus 5 on Terminal-Bench 2.1 (Claude Code): 51.8 (#1)
Claude Opus 5 on Vals AI ProgramBench: 82.27 (#1)
GPT-5.6 Pro Sol on Epoch AI - ECI: 161.06 (#26)
GPT-5.6 Sol on AI for Education Visual Maths - Algebra: 96.15 (#2)
GPT-5.6 Sol on AI for Education Visual Maths - Measurement: 97.3 (#2)
GPT-5.6 Sol on AI for Education Visual Maths - Statistics and Probability: 57.14 (#10)
GPT-5.6 Sol on Appwrite Arena (With Skills): 96.9 (#8)
GPT-5.6 Sol on Appwrite Arena (Without Skills): 95.5 (#5)
GPT-5.6 Sol on Chatbot Arena (Search): 1257.0 (#1)
GPT-5.6 Sol on Crosby micro1 RedlineBench: 56.2 (#2)
GPT-5.6 Sol on Epoch AI - Mystery Game Puzzles: 33.0 (#17)
GPT-5.6 Sol on FrontierCode: 60.6 (#9)
GPT-5.6 Sol on MCP Atlas: 81.8 (#13)
GPT-5.6 Sol on OpenCompass Code - Comprehensive: 96.3 (#2)
GPT-5.6 Sol on OpenCompass Knowledge - Common Sense: 86.3 (#11)
GPT-5.6 Sol on OpenCompass Knowledge - Engineering: 84.2 (#12)
GPT-5.6 Sol on OpenCompass Knowledge - Humanities: 90.8 (#9)
GPT-5.6 Sol on OpenCompass Knowledge - Science: 89.2 (#11)
GPT-5.6 Sol on OpenCompass Knowledge - Social Science: 92.1 (#3)
GPT-5.6 Sol on OpenCompass LLM - Code: 96.3 (#2)
GPT-5.6 Sol on OpenCompass LLM - Knowledge: 88.5 (#9)
GPT-5.6 Sol on OpenCompass LLM - Math: 71.4 (#10)
GPT-5.6 Sol on OpenCompass LLM - Reasoning: 66.8 (#3)
GPT-5.6 Sol on OpenCompass Math - College: 74.3 (#13)
GPT-5.6 Sol on OpenCompass Math - Competition: 68.6 (#6)
GPT-5.6 Sol on OpenCompass Reasoning - Academic: 55.8 (#2)
GPT-5.6 Sol on OpenCompass Reasoning - Common: 77.8 (#6)
GPT-5.6 Sol on SWE Atlas: 46.0 (#5)
GPT-5.6 Sol on Terminal-Bench 2.1: 37.3 (#4)
Gemini 3.7 Flash on AA-LCR: 81.0 (#4)
Gemini 3.7 Flash on AI for Education Visual Maths: 89.45 (#3)
Gemini 3.7 Flash on AI for Education Visual Maths - Algebra: 92.31 (#10)
Gemini 3.7 Flash on AI for Education Visual Maths - Geometry: 90.77 (#2)
Gemini 3.7 Flash on AI for Education Visual Maths - Measurement: 97.3 (#6)
Gemini 3.7 Flash on AI for Education Visual Maths - Number and Operations: 81.08 (#5)
Gemini 3.7 Flash on AI for Education Visual Maths - Statistics and Probability: 57.14 (#11)
Gemini 3.7 Flash on AIIQ Composite IQ: 122.0 (#22)
Gemini 3.7 Flash on Appwrite Arena (With Skills): 96.0 (#13)
Gemini 3.7 Flash on Appwrite Arena (Without Skills): 93.6 (#10)
Gemini 3.7 Flash on BoundaryBench (NIST High): 66.7 (#4)
Gemini 3.7 Flash on BoundaryBench (Unrestricted): 79.8 (#3)
Gemini 3.7 Flash on Bullshit Benchmark: 23.6 (#99)
Gemini 3.7 Flash on CAIS Risk Index: 60.7 (#33)
Gemini 3.7 Flash on CAIS Vision Capabilities Index: 63.7 (#4)
Gemini 3.7 Flash on CritPt: 14.3 (#27)
Gemini 3.7 Flash on Epoch AI - Rli: 5.0 (#5)
Gemini 3.7 Flash on Featherbench: 93.0 (#10)
Gemini 3.7 Flash on Featherbench - Rubric Quality: 8.8 (#8)
Gemini 3.7 Flash on Guesswork 2026-08: 0.9969 (#27)
Gemini 3.7 Flash on OpenCompass Code - Comprehensive: 89.3 (#7)
Gemini 3.7 Flash on OpenCompass Knowledge - Common Sense: 92.9 (#1)
Gemini 3.7 Flash on OpenCompass Knowledge - Engineering: 95.8 (#1)
Gemini 3.7 Flash on OpenCompass Knowledge - Humanities: 94.2 (#3)
Gemini 3.7 Flash on OpenCompass Knowledge - Science: 95.0 (#4)
Gemini 3.7 Flash on OpenCompass Knowledge - Social Science: 92.1 (#4)
Gemini 3.7 Flash on OpenCompass LLM - Code: 89.3 (#7)
Gemini 3.7 Flash on OpenCompass LLM - Knowledge: 94.0 (#2)
Gemini 3.7 Flash on OpenCompass LLM - Math: 76.8 (#2)
Gemini 3.7 Flash on OpenCompass LLM - Reasoning: 62.1 (#9)
Gemini 3.7 Flash on OpenCompass Math - Competition: 67.9 (#9)
Gemini 3.7 Flash on OpenCompass Reasoning - Academic: 47.4 (#11)
Gemini 3.7 Flash on OpenCompass Reasoning - Common: 76.8 (#8)
Gemini 3.7 Flash on SEAL - Remote Labor Index (RLI): 5.0 (#4)
Gemini 3.7 Flash on TaxCalcBench: 12.0 (#14)

New #1 Leaders (51)

ACEBench: GPT-4o (2024-11-20) (89.6) beat Hammer2.1-3B-local by 82.6
Design Arena (Slides): playyy (1316.0) beat Claude 3 Opus by 73.0
GDPval-MM: GPT-5.2 (70.9) beat GPT-4o by 58.6
OpenCompass LLM - Code: GPT-5.5 (High) (96.5) beat Claude Opus 4.7 (High) by 32.8
OpenCompass Code - Comprehensive: GPT-5.5 (High) (96.5) beat Claude Opus 4.7 (High) by 32.8
Lean AI Formalization Leaderboard: Axiom Prover (Axiom Math) (205.0) beat Humanifa + GPT 5.6 sol by 22.0
SWE Atlas: Hy4 preview claude-code (64.0) beat GPT-5.5 by 18.6
PostTrainBench: Claude Fable 5 (Max) (41.79) beat Claude Opus 4.6 by 18.59
Benchmarks.bio - BioSecBench-Refusal: Grok 4.6 (64.2) beat Gemini 3.5 Flash by 12.7
Multi-IF: Llama 3.1 8B (69.0) beat GPT-4.1 Nano by 11.8
SWE-Milestone: GLM-5.3 (58.75) beat Claude Opus 4.8 (Max, 1M) by 6.91
Humanity's Last Exam (Self-Reported, With Tools): GLM-5.3 (62.5) beat Ornith-1.5-397B by 6.4
Crosby micro1 RedlineBench: Claude Opus 5 (56.9) beat GPT-5.5 by 6.4
FEV-Bench - WAPE: TimesFM-3 (85.23) beat Toto-2.0-2.5B by 6.23
AI for Education Visual Maths - Number and Operations: GPT-5.6 Sol (89.19) beat GPT-5.5 by 5.41
WildClawBench: GPT-5.6 Sol (67.2) beat Claude Opus 4.7 by 5.0
OpenCompass Reasoning - Academic: Claude Opus 5 (High) (56.7) beat GPT-5.4 (High) by 4.7
FEV-Bench: TimesFM-3 (85.93) beat Chronos-2 by 4.61
FEV-Bench - WQL: TimesFM-3 (86.37) beat Chronos-2 by 4.58
ArtifactsBench: Ling-3.0-flash (77.0) beat GPT-5 by 4.45
FEV-Bench - MASE: TimesFM-3 (84.97) beat Chronos-2 by 4.11
Chatbot Arena (Search): GPT-5.6 Sol (xHigh) (1257.0) beat Claude Opus 4.6 by 4.0
Featherbench: GLM-5.3 (100.0) beat Gemini 3.6 Flash by 4.0
FEV-Bench - SQL: TimesFM-3 (87.17) beat Chronos-2 by 3.27
OpenCompass LLM - Reasoning: Claude Opus 5 (High) (68.4) beat Doubao-Seed-2-0-Pro-260215 (High) by 3.2
Chatbot Arena (Text-to-Video): Gemini 1.1 Flash (1515.0) beat gemini-omni-flash by 3.0
Chatbot Arena (Video Edit): wan3.0 (1414.0) beat dreamina-seedance-2.5-720p by 3.0
LLM Stats (Toolathlon): GLM-5.3 Flash (78.4) beat Muse Spark 1.1 by 2.8
Open Universal Arabic ASR Leaderboard: Audar-ASR-V1-Turbo (23.17) beat cohere-transcribe-arabic-07-2026 by 2.7
DuelLab Overall: GPT-5.6 Sol (xHigh) (87.3) beat Claude Fable 5 (xHigh) by 2.7
OpenCompass Math - Competition: Claude Opus 5 (High) (74.7) beat Kimi K2.6 by 2.6
MMEB Leaderboard: WeMM-Embedding-9B (59.55) beat Ovis-Omni-Embedding-v0.5-3B by 2.6
Open Universal Arabic ASR Leaderboard - CER: Audar-ASR-V1-Turbo (9.23) beat cohere-transcribe-arabic-07-2026 by 2.57
HieroglyphBench: Gemini 3.7 Flash (55.0) beat Gemini 3.5 Flash by 2.5
AI for Education Visual Maths - Geometry: GPT-5.6 Sol (90.77) beat GPT-5.5 by 2.31
OpenCompass Math - College: Gemini 3.7 Flash (85.6) beat Doubao-Seed-2-0-Pro-260215 (High) by 1.8
AndroidWorld: Gemini 3.7 Flash, Gemini Robotics ER 2 (99.1) beat AGI-0 by 1.7
RealWorldQA: Qwen3.8-Flash-Next (88.5) beat Qwen 3.7 Plus by 1.6
AI for Education Visual Maths: GPT-5.6 Sol (91.14) beat GPT-5.5 by 1.27
LLM Stats (MVBench): GLM-5.3 Flash (77.8) beat Qwen 3.5 122B A10B by 1.2
MMEB Image: WeMM-Embedding-9B (80.99) beat QQMM-embed-v4 by 1.18
Agents' Last Exam: Claude Opus 5 (31.6) beat GPT-5.6 Sol by 1.0
Vals AI ProofBench: alephprover (100.0) beat Claude Opus 5 by 1.0
AIIQ Composite IQ: fable-5 (137.0) beat GPT-5.6 Sol by 1.0
OpenCompass Knowledge - Humanities: Grok 4.6 (High) (95.8) beat Doubao-Seed-2-0-Pro-260215 (High) by 0.8
SecIT Bench (Pydantic AI): GLM-5.3 (81.44) beat Grok 4.6 by 0.79
LLM Stats (AutomationBench): GLM-5.3 Flash (48.8) beat GLM-5.3 by 0.6
Agent Arena - Bash Recovery: Claude Opus 5 (Max) (15.25) beat GPT-5.5 (xHigh) by 0.58
LLM Stats (RealWorldQA): Qwen3.8-Flash-Next (88.5) beat Qwen 3.8 Max by 0.5
Agent Arena - Praise vs Complaint: Claude Opus 4.8 (High) (22.6) beat GPT-5.6 Sol (xHigh) by 0.4
Appwrite Arena (With Skills): Muse Spark 1.2 (97.8) beat GPT-5.5 by 0.1

The Aggregate Digest — 2026-08-29

2026-08-29T11:40:29.518756+00:00

Daily

New Benchmarks (1)

Chatbot Arena (Text) (Elo): leader Claude Fable 5 (1507.0), 395 models
Crowdsourced Elo-rated leaderboard where users blindly compare LLM responses in pairwise battles across diverse real-world text prompts.

Top-10 New Scores (69)

Claude Fable 5 on MCP Atlas: 83.3 (#10)
Claude Fable 5 on OSWorld-Verified: 85.96 (#2)
Claude Mythos 5 on ArxivMath: 78.52 (#3)
Claude Mythos Preview on ArxivMath: 68.7 (#7)
Claude Opus 5 on ArxivMath: 91.33 (#1)
Claude Opus 5 on MCP Atlas: 85.8 (#3)
Claude Opus 5 on OpenCompass Code - Comprehensive: 93.3 (#4)
Claude Opus 5 on OpenCompass Knowledge - Common Sense: 92.1 (#3)
Claude Opus 5 on OpenCompass Knowledge - Engineering: 93.3 (#5)
Claude Opus 5 on OpenCompass Knowledge - Humanities: 92.9 (#4)
Claude Opus 5 on OpenCompass Knowledge - Science: 97.5 (#1)
Claude Opus 5 on OpenCompass Knowledge - Social Science: 94.6 (#1)
Claude Opus 5 on OpenCompass LLM - Code: 93.3 (#4)
Claude Opus 5 on OpenCompass LLM - Knowledge: 94.1 (#1)
Claude Opus 5 on OpenCompass LLM - Math: 77.3 (#1)
Claude Opus 5 on OpenCompass LLM - Reasoning: 68.4 (#1)
Claude Opus 5 on OpenCompass Math - College: 79.9 (#5)
Claude Opus 5 on OpenCompass Math - Competition: 74.7 (#1)
Claude Opus 5 on OpenCompass Reasoning - Academic: 56.7 (#1)
Claude Opus 5 on OpenCompass Reasoning - Common: 80.1 (#2)
GPT-5.6 Sol on Chatbot Arena (Search): 1257.0 (#1)
GPT-5.6 Sol on FrontierCode: 60.6 (#9)
GPT-5.6 Sol on MCP Atlas: 81.8 (#14)
GPT-5.6 Sol on OpenCompass Code - Comprehensive: 96.3 (#2)
GPT-5.6 Sol on OpenCompass Knowledge - Common Sense: 86.3 (#11)
GPT-5.6 Sol on OpenCompass Knowledge - Engineering: 84.2 (#12)
GPT-5.6 Sol on OpenCompass Knowledge - Humanities: 90.8 (#9)
GPT-5.6 Sol on OpenCompass Knowledge - Science: 89.2 (#11)
GPT-5.6 Sol on OpenCompass Knowledge - Social Science: 92.1 (#3)
GPT-5.6 Sol on OpenCompass LLM - Code: 96.3 (#2)
GPT-5.6 Sol on OpenCompass LLM - Knowledge: 88.5 (#9)
GPT-5.6 Sol on OpenCompass LLM - Math: 71.4 (#10)
GPT-5.6 Sol on OpenCompass LLM - Reasoning: 66.8 (#3)
GPT-5.6 Sol on OpenCompass Math - College: 74.3 (#13)
GPT-5.6 Sol on OpenCompass Math - Competition: 68.6 (#6)
GPT-5.6 Sol on OpenCompass Reasoning - Academic: 55.8 (#2)
GPT-5.6 Sol on OpenCompass Reasoning - Common: 77.8 (#6)
GPT-5.6 Sol on SWE Atlas: 46.0 (#5)
GPT-5.6 Sol on WildClawBench: 67.2 (#1)
Gemini 3.7 Flash on OpenCompass Code - Comprehensive: 89.3 (#7)
Gemini 3.7 Flash on OpenCompass Knowledge - Common Sense: 92.9 (#1)
Gemini 3.7 Flash on OpenCompass Knowledge - Engineering: 95.8 (#1)
Gemini 3.7 Flash on OpenCompass Knowledge - Humanities: 94.2 (#3)
Gemini 3.7 Flash on OpenCompass Knowledge - Science: 95.0 (#4)
Gemini 3.7 Flash on OpenCompass Knowledge - Social Science: 92.1 (#4)
Gemini 3.7 Flash on OpenCompass LLM - Code: 89.3 (#7)
Gemini 3.7 Flash on OpenCompass LLM - Knowledge: 94.0 (#2)
Gemini 3.7 Flash on OpenCompass LLM - Math: 76.8 (#2)
Gemini 3.7 Flash on OpenCompass LLM - Reasoning: 62.1 (#9)
Gemini 3.7 Flash on OpenCompass Math - Competition: 67.9 (#9)
Gemini 3.7 Flash on OpenCompass Reasoning - Academic: 47.4 (#11)
Gemini 3.7 Flash on OpenCompass Reasoning - Common: 76.8 (#8)
Kimi K3 on APEX-Agents-AA: 0.41 (#30)
Kimi K3 on LM Market Cap LMC Score: 40.0 (#248)
Kimi K3 on OpenCompass Code - Comprehensive: 88.0 (#8)
Kimi K3 on OpenCompass Knowledge - Common Sense: 87.9 (#8)
Kimi K3 on OpenCompass Knowledge - Engineering: 88.3 (#8)
Kimi K3 on OpenCompass Knowledge - Humanities: 92.1 (#7)
Kimi K3 on OpenCompass Knowledge - Science: 92.5 (#6)
Kimi K3 on OpenCompass Knowledge - Social Science: 90.4 (#5)
Kimi K3 on OpenCompass LLM - Code: 88.0 (#8)
Kimi K3 on OpenCompass LLM - Knowledge: 90.3 (#7)
Kimi K3 on OpenCompass LLM - Math: 74.1 (#6)
Kimi K3 on OpenCompass LLM - Reasoning: 63.9 (#7)
Kimi K3 on OpenCompass Math - College: 79.2 (#9)
Kimi K3 on OpenCompass Math - Competition: 68.9 (#5)
Kimi K3 on OpenCompass Reasoning - Academic: 51.7 (#6)
Kimi K3 on OpenCompass Reasoning - Common: 76.0 (#9)
Kimi K3 on WildClawBench: 54.5 (#12)

New #1 Leaders (13)

OpenCompass LLM - Code: GPT-5.5 (High) (96.5) beat Claude Opus 4.7 (High) by 32.8
OpenCompass Code - Comprehensive: GPT-5.5 (High) (96.5) beat Claude Opus 4.7 (High) by 32.8
Lean AI Formalization Leaderboard: Axiom Prover (Axiom Math) (205.0) beat Humanifa + GPT 5.6 sol by 22.0
SWE Atlas: Hy4 preview claude-code (64.0) beat GPT-5.5 by 18.6
Design Arena (Video): Gemini 1.1 Flash (1377.0) beat Gemini 2.0 Flash by 9.0
Humanity's Last Exam (Self-Reported, With Tools): GLM-5.3 (62.5) beat Ornith-1.5-397B by 6.4
OpenCompass Reasoning - Academic: Claude Opus 5 (High) (56.7) beat GPT-5.4 (High) by 4.7
Chatbot Arena (Search): GPT-5.6 Sol (xHigh) (1257.0) beat Claude Opus 4.6 by 4.0
OpenCompass LLM - Reasoning: Claude Opus 5 (High) (68.4) beat Doubao-Seed-2-0-Pro-260215 (High) by 3.2
OpenCompass Math - Competition: Claude Opus 5 (High) (74.7) beat Kimi K2.6 by 2.6
OpenCompass Math - College: Gemini 3.7 Flash (85.6) beat Doubao-Seed-2-0-Pro-260215 (High) by 1.8
AndroidWorld: Gemini 3.7 Flash, Gemini Robotics ER 2 (99.1) beat AGI-0 by 1.7
OpenCompass Knowledge - Humanities: Grok 4.6 (High) (95.8) beat Doubao-Seed-2-0-Pro-260215 (High) by 0.8

The Aggregate Digest — 2026-08-26

2026-08-26T18:35:49.913992+00:00

Daily

New Benchmarks (2)

SWE Refactor Bench (Score (out of 100)): leader Claude Opus 5 (47.0), 8 models
A whole-repository stack migration held open for a hundred-odd steps: change the framework the codebase is built on and leave it working. Every model is run at each of its harness's effort tiers, and the best any of them manages is 47 out of 100 — the behavioural test suite mostly passes, the migration mostly does not.
EarthVerse (Core Score (%)): leader Claude Fable 5 (84.97), 18 models
Earth-system and natural-hazard analysis with real observational data: reconstruct what happened from sources that disagree on scale, timing and modality. Scored on the answer and the trajectory that produced it, so a right number reached by a wrong route does not count.

Top-10 New Scores (3)

GPT-5.6 Pro Sol on Epoch AI - ECI: 161.08 (#26)
Gemini 3.7 Flash on Bullshit Benchmark: 23.6 (#99)
Gemini 3.7 Flash on Epoch AI - Rli: 5.0 (#5)

New #1 Leaders (5)

Design Arena (Slides): playyy (1304.0) beat Claude 3 Opus by 61.0
ArtifactsBench: Ling-3.0-flash (77.0) beat GPT-5 by 4.45
HieroglyphBench: Gemini 3.7 Flash (55.0) beat Gemini 3.5 Flash by 2.5
BEAR-Bench: Qwen 3.5 397B A17B (75.4) beat Gemini 3.1 Pro (Preview) by 0.3
Longitudinal MRI Progression Reasoning: InternVL3.5-Inst (35.15) beat Gemini 3 Pro by 0.05

The Aggregate Digest — 2026-08-25

2026-08-25T06:17:16.148062+00:00

Daily

New Benchmarks (3)

DGEval - IMDG Code MCQ (Accuracy (%)): leader Gemini 3.1 Pro (Preview) (95.8), 10 models
The closed-book multiple-choice half of a dangerous-goods shipping certification, asked exactly as human candidates get it. Certified practitioners score 83.8 percent, which is the number every model on the board is measured against — and most of them are below it.
DGEval - Regulatory Recall (Accuracy (%)): leader Gemini 3.1 Pro (Preview) (36.8), 10 models
The section of the same maritime dangerous-goods exam that nothing saturates: name the IMDG Code provision a scenario falls under, from memory. Scores run from 0 to 37 percent, so it separates the frontier where the multiple-choice half no longer does.
VIALS (Direct VLM Accuracy (%)): leader GPT-5.6 Sol (26.5), 7 models
Gel blots, microscopy, plasmid maps, flow cytometry plots — the visual artifacts a working life scientist reads before deciding what to do next. Scored as direct vision-language accuracy with no tools, where the best model manages roughly a quarter of the questions.

Top-10 New Scores (25)

Claude Fable 5 on AI for Education Visual Maths: 86.92 (#4)
Claude Fable 5 on AI for Education Visual Maths - Algebra: 92.31 (#16)
Claude Fable 5 on AI for Education Visual Maths - Geometry: 83.85 (#5)
Claude Fable 5 on AI for Education Visual Maths - Measurement: 97.3 (#5)
Claude Fable 5 on AI for Education Visual Maths - Number and Operations: 86.49 (#2)
Claude Fable 5 on AI for Education Visual Maths - Statistics and Probability: 71.43 (#3)
Claude Opus 5 on AI for Education Visual Maths: 80.59 (#10)
Claude Opus 5 on AI for Education Visual Maths - Algebra: 96.15 (#3)
Claude Opus 5 on AI for Education Visual Maths - Geometry: 75.38 (#13)
Claude Opus 5 on AI for Education Visual Maths - Measurement: 94.59 (#12)
Claude Opus 5 on AI for Education Visual Maths - Number and Operations: 75.68 (#6)
Claude Opus 5 on AI for Education Visual Maths - Statistics and Probability: 71.43 (#2)
Claude Opus 5 on Roboflow Playground Open Prompt: 1200.0 (#22)
GPT-5.6 Sol on AI for Education Visual Maths - Algebra: 96.15 (#2)
GPT-5.6 Sol on AI for Education Visual Maths - Measurement: 97.3 (#2)
GPT-5.6 Sol on AI for Education Visual Maths - Statistics and Probability: 57.14 (#10)
Gemini 3.7 Flash on AI for Education Visual Maths: 89.45 (#3)
Gemini 3.7 Flash on AI for Education Visual Maths - Algebra: 92.31 (#10)
Gemini 3.7 Flash on AI for Education Visual Maths - Geometry: 90.77 (#2)
Gemini 3.7 Flash on AI for Education Visual Maths - Measurement: 97.3 (#6)
Gemini 3.7 Flash on AI for Education Visual Maths - Number and Operations: 81.08 (#5)
Gemini 3.7 Flash on AI for Education Visual Maths - Statistics and Probability: 57.14 (#11)
Gemini 3.7 Flash on ProphetArena: 0.96 (#2)
Gemini 3.7 Flash on SEAL - Remote Labor Index (RLI): 5.0 (#4)
Gemini 3.7 Flash on TaxCalcBench: 12.0 (#14)

New #1 Leaders (5)

Benchmarks.bio - BioSecBench-Refusal: Grok 4.6 (64.2) beat Gemini 3.5 Flash by 12.7
AI for Education Visual Maths - Number and Operations: GPT-5.6 Sol (89.19) beat GPT-5.5 by 5.41
AI for Education Visual Maths - Geometry: GPT-5.6 Sol (90.77) beat GPT-5.5 by 2.31
AI for Education Visual Maths: GPT-5.6 Sol (91.14) beat GPT-5.5 by 1.27
SecIT Bench (Pydantic AI): GLM-5.3 (81.44) beat Grok 4.6 by 0.79

The Aggregate Digest — 2026-08-24

2026-08-24T10:31:51.987898+00:00

Daily

New Benchmarks (6)

BenchGecko Score (Composite Score): leader GPT-5.5 Pro (99.9), 20 models
BenchGecko's own composite index over the public boards it tracks, mirrored here the way LLM Stats Score and BenchmarkList ECI are. Coverage per model is uneven — between 3 and 14 boards feed each score — so it reads as one aggregator's view rather than an independent measurement.
SAST False-Positive Triage (Accuracy (%)): leader Kimi K3 (92.0), 15 models
A static analyser fires 142 findings over a codebase and 11 of them are real. The model has to keep the genuine vulnerabilities and dismiss the noise, so the accuracy column punishes both misses and credulity. Published as a blog table, which is why the roster is small and current.
StartupBench (Score (%)): leader Kimi K3 (73.67), 9 models
Ninety-seven market-research tasks drawn from real startup work across medicine, finance, legal, business, STEM and education, scored per domain and overall. Tasks are open-ended and judged item-wise on process logic rather than a single final answer, which is what separates it from the GAIA-style agent boards.
Agentic Commerce World (Overall (%)): leader GPT-5.6 Sol (85.6), 9 models
Vibe commerce: state a buying or selling goal in natural language and delegate the transaction to an agent. Runs in an auditable, verifiable environment because commerce failures cost money rather than a retry.
VibeLifeBench (Average of 3 runs (%)): leader Claude Opus 5 (32.5), 7 models
Personal-assistant tasks that run for weeks in a world that changes underneath them, scored on proactivity and persistence. Deliberately unlike the short self-contained requests in a static environment that most agent evaluations use.
BusinessCaseBench (Score (%)): leader Claude Sonnet 4.6 (88.4), 7 models
Case-grounded professional knowledge work across the business disciplines - the judgement calls that benchmarks built on recall, narrow QA, maths and coding do not reach.

Top-10 New Scores (2)

Gemini 3.7 Flash on BoundaryBench (NIST High): 66.7 (#4)
Gemini 3.7 Flash on BoundaryBench (Unrestricted): 79.8 (#3)

The Aggregate Digest — 2026-08-23

2026-08-23T07:31:51.012044+00:00

Daily

Top-10 New Scores (6)

Claude Opus 5 on Agents' Last Exam: 27.0 (#7)
Gemini 3.7 Flash on Arena AI Code: 1587.0 (#10)
Gemini 3.7 Flash on LMArena Text Arena: 1489.87 (#9)
Gemini 3.7 Flash on LMArena WebDev Arena: 1587.86 (#9)
Kimi K3 on Benchmarks.bio - BioSecBench-Refusal: 8.6 (#16)
Kimi K3 on Benchmarks.bio - BioSecBench-Surveillance: 48.3 (#12)

New #1 Leaders (2)

MobileWorld - GUI Only: Qwen-UI-Agent (82.1) beat Kimi K3 by 7.7
WebVoyager: browser-control (Fable 5) (99.19) beat Alumnium by 0.69

Weekly

New Benchmarks (1)

AtmosCoder-Bench (Accuracy (%, mean of 3 runs)): leader GPT-5.5 (97.6), 16 models
AtmosCoder-Bench grades quantitative atmospheric science by execution: each of 436 textbook-grounded problems is answered by a Python solve() the model writes, and the graded number is whatever that function computes, with the declared unit reconciled against ground truth. Three runs per model; accuracy (%) is the mean. Reasoning and thinking-off runs of the same checkpoint are listed separately, as upstream measures them.

New Models (123)

Claude Opus 5 — ELO 1792, #1
- Vals AI MedCode: 63.57 (#1/86)
- Vals AI MedScribe: 90.98 (#1/87)
- Vals AI MMMU: 89.88 (#1/90)
- LiveMathematicianBench: 45.1 (#3/19)
- Vals AI LegalBench: 86.97 (#5/139)
- Agents' Last Exam: 27.0 (#7/25)
- SpacetimeDB LLM Benchmark (TypeScript): 96.4 (#9/14)
- SpacetimeDB LLM Benchmark (Rust): 68.3 (#9/14)
- SpacetimeDB LLM Benchmark (C#): 66.7 (#12/14)
- Epoch AI - Cursorbench: 64.3 (#16/67)
- ...and 1 more
GPT-5.5 Pro — ELO 1786, #2
- OpenAI GPT-5.5 Launch - FrontierMath Tier 1-3: 52.4 (#1/6)
- OpenAI GPT-5.5 Launch - FrontierMath Tier 4: 39.6 (#1/6)
- AA CritPt: 30.57 (#2/490)
Claude Fable 5 — ELO 1783, #3
- FutureEval: 13.23 (#1/43)
- SpacetimeDB LLM Benchmark (C#): 98.9 (#1/14)
- Epoch AI - Enigma Eval: 39.28 (#1/46)
- ArxivMath: 78.6 (#2/11)
- FrontierCode: 46.3 (#3/22)
- CyberGym: 83.8 (#3/23)
- APEX v1: 66.0 (#4/5)
- APEX v1 Investment Banking: 66.6 (#4/39)
- APEX v1 Consulting: 66.6 (#5/39)
- SpacetimeDB LLM Benchmark (Rust): 68.3 (#8/14)
- ...and 4 more
GPT-5.6 Sol — ELO 1777, #4
- SpacetimeDB LLM Benchmark (TypeScript): 99.5 (#1/14)
- Design Arena (UI Components): 1387.0 (#1/152)
- Design Arena (Game Dev): 1414.0 (#2/157)
- Design Arena (3D): 1438.0 (#2/145)
- Epoch AI - Enigma Eval: 37.12 (#2/46)
- CyberGym: 83.6 (#4/23)
- Design Arena (Data Viz): 1338.0 (#5/153)
- Toolathlon: 74.9 (#6/36)
- SpacetimeDB LLM Benchmark (Rust): 82.9 (#7/14)
- SpacetimeDB LLM Benchmark (C#): 92.2 (#9/14)
- ...and 4 more
Gemini 3.7 Flash — ELO 1777, #5
- ProphetArena: 0.9687 (#1/19)
- Icelandic LLM Leaderboard - Average: 88.71 (#1/94)
- Icelandic LLM - WinoGrande-IS: 96.32 (#1/94)
- Icelandic LLM - ARC-Challenge-IS: 95.22 (#1/94)
- Kaggle FACTS (Google): 71.27 (#1/33)
- Kaggle FACTS Multimodal: 49.83 (#1/33)
- AI for Education Visual Reasoning: 89.9 (#1/67)
- AI for Education Visual Reasoning - match (figure): 87.4 (#1/67)
- AI for Education Visual Reasoning - match (process): 88.9 (#1/67)
- AI for Education Visual Reasoning - pattern completion (linear): 95.7 (#1/67)
- ...and 60 more
Kimi K3 — ELO 1761, #7
- APEX v1: 68.4 (#2/5)
- APEX v1 Medicine (MD): 64.3 (#2/39)
- Toolathlon: 76.5 (#2/36)
- APEX v1 Consulting: 69.7 (#3/39)
- NL2Repo: 58.0 (#3/25)
- AI for Education Visual Reasoning - odd one out: 79.2 (#4/67)
- APEX v1 Investment Banking: 66.4 (#5/39)
- AI for Education Visual Reasoning - reasoning by analogy: 83.7 (#5/67)
- APEX v1 Big Law: 73.2 (#7/39)
- AI for Education Visual Reasoning - match (process): 75.0 (#7/67)
- ...and 11 more
Grok 4.6 — ELO 1757, #8
- AI for Education Pedagogy - Science: 96.72 (#1/235)
- AI for Education Pedagogy - Secondary: 91.19 (#1/235)
- Epoch AI - Cursorbench: 70.8 (#3/67)
- AI for Education Pedagogy: 91.55 (#3/235)
- EnterpriseBench (CoreCraft Agents): 65.6 (#3/3)
- Epoch AI - Vending Bench 2: 9047.03 (#4/58)
- AA GDPval: 1765.76 (#4/210)
- KernelBench Hub - Hard: 62.17 (#5/13)
- Epoch AI - Apex Agents: 41.2 (#5/61)
- Epoch AI - Blueprint Bench 2: 0.332 (#5/23)
- ...and 33 more
Qwen 3.8 Max — ELO 1755, #9
- WebDev Arena: 1666.72 (#4/111)
- Agent Arena - Confirmed Success: 11.62 (#6/51)
- Epoch AI - Proofbench: 58.0 (#9/22)
- CyberGym: 78.5 (#11/23)
- Agent Arena: 6.2 (#15/51)
- Agent Arena - Praise vs Complaint: 6.81 (#17/51)
- Agent Arena - Steerability: 4.0 (#19/51)
- Agent Arena - Bash Recovery: 8.39 (#23/51)
- Agent Arena - Tool Hallucination: -0.18 (#39/51)
GPT-5.5 — ELO 1749, #10
- OpenAI GPT-5.5 Launch - FrontierMath Tier 1-3: 51.7 (#2/6)
- OpenAI GPT-5.5 Launch - FrontierMath Tier 4: 35.4 (#3/6)
- SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#4/14)
- SpacetimeDB LLM Benchmark (Rust): 96.6 (#4/14)
- SpacetimeDB LLM Benchmark (C#): 96.6 (#5/14)
- ArxivMath: 72.2 (#5/11)
- Epoch AI - Apex Agents: 38.5 (#10/61)
- ParseBench: 64.39 (#17/56)
Muse Spark 1.2 — ELO 1749, #11
- Agent Arena - Bash Recovery: 11.4 (#10/51)
- Agent Arena - Confirmed Success: 6.02 (#14/51)
- Agent Arena - Tool Hallucination: -1.13 (#17/51)
- Creative Writing (Lechmazur): 0.6 (#20/47)
- Agent Arena: 2.06 (#25/51)
- Agent Arena - Steerability: -2.51 (#32/51)
- Agent Arena - Praise vs Complaint: -5.72 (#35/51)
Gemini 3.6 Flash — ELO 1747, #12
- APEX v1 Investment Banking: 69.2 (#2/39)
- AI for Education Visual Reasoning - pattern completion (2d): 83.6 (#2/67)
- AI for Education Visual Reasoning - pattern completion (linear): 92.3 (#2/67)
- LVBench: 84.2 (#2/53)
- AI for Education Visual Reasoning: 85.7 (#3/67)
- AI for Education Visual Reasoning - match (figure): 84.7 (#3/67)
- AI for Education Visual Reasoning - reasoning by analogy: 87.9 (#3/67)
- AI for Education Visual Reasoning - match (process): 77.8 (#4/67)
- BioMysteryBench Human-Difficult: 41.2 (#4/11)
- APEX v1: 66.0 (#5/5)
- ...and 12 more
Muse Spark 1.1 — ELO 1744, #13
- Roboflow Playground Open Prompt: 1188.0 (#43/54)
GLM-5.3 — ELO 1744, #14
- CyberGym: 84.5 (#1/23)
- KernelBench Hub - Mega: 21.39 (#2/9)
- Creative Writing (Lechmazur): 3.5 (#2/47)
- Tau3 Banking: 50.31 (#3/176)
- AA GDPval: 1769.0 (#3/210)
- NL2Repo: 58.0 (#4/25)
- DeepSWE: 69.0 (#6/28)
- Agents on Rails: 79.4 (#6/12)
- Vals AI SWE-bench Verified: 95.4 (#6/86)
- Chatbot Arena (Code): 1599.0 (#8/118)
- ...and 33 more
Gemini 3.1 Pro (Preview) — ELO 1739, #16
- ArxivMath: 64.79 (#8/11)
- GAIA: 55.48 (#591/3523)
Claude Opus 4.8 — ELO 1735, #17
- SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#2/14)
- APEX v1 Medicine (MD): 64.0 (#3/39)
- SpacetimeDB LLM Benchmark (C#): 96.6 (#4/14)
- SpacetimeDB LLM Benchmark (Rust): 94.4 (#5/14)
- Epoch AI - Enigma Eval: 23.51 (#5/46)
- ArxivMath: 71.0 (#6/11)
- Agents on Rails: 79.4 (#7/12)
- APEX v1 Consulting: 62.6 (#9/39)
- Arena AI Document: 1475.0 (#11/38)
- WebDev Arena: 1564.26 (#13/111)
- ...and 6 more
GPT-5.6 Terra — ELO 1735, #18
- APEX v1: 69.5 (#1/5)
- APEX v1 Consulting: 76.2 (#1/39)
- APEX v1 Investment Banking: 71.8 (#1/39)
- BioMysteryBench Human-Difficult: 49.4 (#1/11)
- MRCR-v2 128k: 93.5 (#2/20)
- LVBench: 78.9 (#3/53)
- Vals AI ProgramBench: 72.34 (#4/41)
- Vals AI Code Migration: 47.8 (#4/51)
- BioMysteryBench Human-Solvable: 83.8 (#4/11)
- Vals AI SWE-bench Verified: 95.4 (#5/86)
- ...and 18 more
Gemini 3.5 Flash — ELO 1734, #19
- LiveMathematicianBench: 49.5 (#1/19)
- APEX v1 Investment Banking: 67.4 (#3/39)
- Epoch AI - Enigma Eval: 25.41 (#3/46)
- APEX v1 Medicine (MD): 61.7 (#11/39)
- SpacetimeDB LLM Benchmark (C#): 50.6 (#14/14)
- SpacetimeDB LLM Benchmark (TypeScript): 44.9 (#14/14)
- SpacetimeDB LLM Benchmark (Rust): 49.4 (#14/14)
- FutureEval: 10.01 (#17/43)
- APEX v1 Big Law: 66.2 (#21/39)
- APEX v1 Consulting: 53.6 (#25/39)
- ...and 1 more
DeepSeek V4 Pro (0813) — ELO 1734, #20
- ProphetArena: 0.9661 (#2/19)
- Epoch AI - Mystery Game Puzzles: 43.0 (#6/54)
- Riemann-bench: 38.4 (#9/30)
- OTIS Mock AIME 2024-25: 98.61 (#10/239)
- WebDev Arena: 1584.45 (#10/111)
- Chess Puzzles (Epoch AI): 47.0 (#11/162)
- Epoch AI - Proofbench: 50.0 (#12/22)
- SimpleQA Verified: 55.47 (#18/78)
- FrontierMath - Tiers 1-3 (v2): 64.56 (#20/49)
- FrontierMath - Tier 4 (v2): 26.83 (#23/51)
- ...and 3 more
Grok 4.5 — ELO 1729, #21
- Roboflow Playground Object Detection: 1200.0 (#30/48)
- Roboflow Playground Overall: 1200.0 (#40/72)
Qwen3.8 2.4T A95B — ELO 1725, #23
- WeirdML: 75.24 (#21/155)
- Arabic Broad Leaderboard: 8.862 (#23/109)
GPT-5.4 — ELO 1713, #27
- OpenAI GPT-5.4 Launch - FrontierMath Tier 1-3: 47.6 (#2/3)
- LiveMathematicianBench: 46.6 (#2/19)
- OpenAI GPT-5.4 Launch - FrontierMath Tier 4: 27.1 (#3/4)
- OpenAI GPT-5.5 Launch - FrontierMath Tier 1-3: 47.6 (#4/6)
- OpenAI GPT-5.5 Launch - FrontierMath Tier 4: 27.1 (#4/6)
- Epoch AI - Enigma Eval: 15.96 (#9/46)
- ParseBench: 62.23 (#23/56)
Claude Opus 4.7 — ELO 1711, #28
- LMArena Text Arena: 1502.01 (#3/24)
- OpenAI GPT-5.5 Launch - FrontierMath Tier 1-3: 43.8 (#5/6)
- OpenAI GPT-5.5 Launch - FrontierMath Tier 4: 22.9 (#5/6)
- Arena AI Document: 1497.0 (#6/38)
- scBench: 55.3 (#6/17)
- APEX v1 Medicine (MD): 63.3 (#7/39)
- APEX v1 Big Law: 70.9 (#10/39)
- APEX v1 Consulting: 61.0 (#15/39)
- WebDev Arena: 1557.31 (#15/111)
- LMArena WebDev Arena: 1557.29 (#15/25)
- ...and 3 more
Claude Sonnet 5 — ELO 1711, #29
- BioMysteryBench Human-Solvable: 87.5 (#1/11)
- APEX v1 Medicine (MD): 63.9 (#5/39)
- BioMysteryBench Human-Difficult: 34.1 (#6/11)
- FutureEval: 11.34 (#10/43)
- LVBench: 68.5 (#16/53)
- AA GDPval: 1502.39 (#33/210)
- APEX v1 Big Law: 54.4 (#34/39)
- Epoch AI - Cursorbench: 56.9 (#38/67)
- APEX v1 Consulting: 41.1 (#39/39)
- APEX v1 Investment Banking: 39.1 (#39/39)
GPT-5.6 Luna — ELO 1709, #30
- CritPt: 20.6 (#12/213)
- Epoch AI - Cursorbench: 61.1 (#26/67)
- ParseBench: 56.32 (#30/56)
Qwen 3.8 27B — ELO 1706, #33
- BRIDGE Medical Leaderboard - Few-Shot: 55.67 (#1/109)
- BRIDGE Medical Leaderboard: 48.31 (#2/109)
- BRIDGE Medical Leaderboard - Zero-Shot: 46.61 (#2/109)
- RealWorldQA: 85.9 (#2/14)
- BRIDGE Medical Leaderboard - CoT: 44.92 (#3/109)
- OpenRouter Tau2-Bench Airline: 80.7 (#3/114)
- OSWorld-Verified: 84.3 (#3/29)
- Tau3 Banking: 48.04 (#5/176)
- Vals AI Harvey Legal Agent Bench: 11.25 (#5/52)
- Vals AI SAGE: 52.4 (#6/77)
- ...and 53 more
GPT-5 Pro — ELO 1705, #35
- Epoch AI - Enigma Eval: 18.75 (#7/46)
Claude Opus 4.6 — ELO 1703, #38
- LMArena Text Arena: 1504.49 (#2/24)
- Arena AI Document: 1506.0 (#3/38)
- NL2Repo: 47.6 (#10/25)
- RealWorldQA: 73.9 (#11/14)
- LMArena WebDev Arena: 1545.76 (#17/25)
- OSWorld-Verified: 72.7 (#17/29)
- Epoch AI - Enigma Eval: 7.6 (#18/46)
- Arena AI Code: 1546.0 (#18/48)
- WebDev Arena: 1544.94 (#21/111)
- ComplexConstraints: 36.3 (#24/57)
- ...and 1 more
Gemini 3 Flash — ELO 1703, #39
- DystopiaBench: 77.0 (#40/42)
GLM-5.2 — ELO 1703, #40
- APEX v1 Investment Banking: 62.6 (#10/39)
- APEX v1 Consulting: 62.1 (#12/39)
- CyberGym: 77.2 (#12/23)
- FutureEval: 9.29 (#24/43)
- APEX v1 Big Law: 64.9 (#25/39)
- APEX v1 Medicine (MD): 56.3 (#27/39)
- LM Market Cap LMC Score: 82.7 (#76/428)
Kimi K2.7 Code — ELO 1703, #41
- APEX v1 Consulting: 61.7 (#13/39)
- APEX v1 Investment Banking: 59.4 (#18/39)
- APEX v1 Big Law: 65.1 (#24/39)
- APEX v1 Medicine (MD): 54.2 (#30/39)
- Epoch AI - Cursorbench: 49.7 (#55/67)
Grok Build 0.1 — ELO 1702, #42
- SpacetimeDB LLM Benchmark (Rust): 97.8 (#2/14)
- SpacetimeDB LLM Benchmark (C#): 95.5 (#6/14)
- SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#6/14)
Qwen 3.7 Plus — ELO 1698, #43
- OmniDocBench 1.5: 91.4 (#9/67)
- CharXiv-R: 85.9 (#12/61)
- OSWorld-Verified: 73.3 (#15/29)
- NL2Repo: 41.1 (#18/25)
- Design Arena (SVG): 1260.0 (#21/110)
- Roboflow Playground Object Detection: 1200.0 (#29/48)
- Roboflow Playground Overall: 1200.0 (#39/72)
- Epoch AI - ECI: 148.26 (#183/491)
DeepSeek V4 Flash (0731) — ELO 1697, #44
- BenCzechMark: 81.36 (#3/74)
- KernelBench Hub - Hard: 48.6 (#9/13)
- Epoch AI - Proofbench: 56.0 (#10/22)
- WebDev Arena: 1576.54 (#11/111)
- Epoch AI - ECI: 154.43 (#106/491)
- ARC-AGI-2: 2.08 (#154/214)
- ARC-AGI-1: 11.83 (#191/212)
Claude Opus 4.5 — ELO 1693, #45
- NarrativeWorldBench: 80.0 (#3/10)
GPT-5 — ELO 1692, #46
- LiveMathematicianBench: 32.2 (#7/19)
- Epoch AI - Enigma Eval: 10.47 (#14/46)
- MMMU Benchmark: 74.4 (#24/189)
- Epoch AI - Apex Agents: 18.3 (#37/61)
GPT-5.2 — ELO 1691, #48
- OpenAI GPT-5.4 Launch - FrontierMath Tier 1-3: 40.7 (#3/3)
- OpenAI GPT-5.4 Launch - FrontierMath Tier 4: 18.8 (#4/4)
- LiveMathematicianBench: 26.3 (#15/19)
- Epoch AI - Enigma Eval: 10.39 (#15/46)
- Epoch AI - Apex Agents: 23.0 (#31/61)
Kimi K2.6 — ELO 1691, #49
- Inverse Turing Bench: 57.45 (#12/29)
- Finance Agent v1.1: 57.06 (#14/53)
DeepSeek V4 Pro — ELO 1690, #51
- LiveMathematicianBench: 36.9 (#5/19)
- HANDBOOK.md Agents: 26.5 (#6/45)
- ComplexConstraints: 42.0 (#12/57)
- APEX v1 Investment Banking: 60.9 (#13/39)
- APEX v1 Consulting: 58.3 (#21/39)
- APEX v1 Big Law: 64.6 (#27/39)
- APEX v1 Medicine (MD): 54.8 (#28/39)
Claude Opus 4.5 (20251101) — ELO 1688, #53
- Epoch AI - Enigma Eval: 11.91 (#11/46)
- Arena AI Code: 1494.0 (#35/48)
Inkling — ELO 1685, #54
- APEX v1 Medicine (MD): 60.4 (#16/39)
- APEX v1 Big Law: 67.9 (#18/39)
- Riemann-bench: 15.2 (#19/30)
- APEX v1 Investment Banking: 55.6 (#21/39)
- APEX v1 Consulting: 48.4 (#33/39)
- HANDBOOK.md Agents: 2.3 (#42/45)
- Chartography: 10.0 (#44/45)
GLM-5.1 — ELO 1683, #57
- Finance Agent v1.1: 57.66 (#12/53)
Qwen 3.5 397B A17B — ELO 1681, #59
- ABRA: 70.0 (#1/10)
- Onyx Open LLM Leaderboard: 87.8 (#2/19)
- DystopiaBench: 58.9 (#16/42)
Claude Sonnet 4.6 — ELO 1677, #61
- SpacetimeDB LLM Benchmark (C#): 98.9 (#2/14)
- SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#3/14)
- SpacetimeDB LLM Benchmark (Rust): 96.6 (#3/14)
- scBench: 50.4 (#10/17)
- ComplexConstraints: 34.0 (#34/57)
O3 — ELO 1676, #62
- Epoch AI - Enigma Eval: 13.09 (#10/46)
MiniMax-M3 — ELO 1675, #65
- KernelBench Hub - Mega: 2.63 (#8/9)
- APEX v1 Medicine (MD): 61.3 (#14/39)
- APEX v1 Consulting: 59.8 (#19/39)
- APEX v1 Big Law: 64.9 (#26/39)
- APEX v1 Investment Banking: 53.1 (#28/39)
GPT-5.3 — ELO 1673, #67
- AI Chess Leaderboard (Continuation): 948.0 (#52/259)
- AI Chess Leaderboard (Reasoning): 1023.0 (#61/318)
Gemini 3.5 Flash Lite — ELO 1671, #68
- Epoch AI - ECI: 145.11 (#238/491)
Claude Sonnet 4.5 — ELO 1669, #69
- Epoch AI - Enigma Eval: 6.0 (#23/46)
GPT-5.1 — ELO 1668, #70
- Epoch AI - Enigma Eval: 11.23 (#13/46)
- Epoch AI - Apex Agents: 17.5 (#39/61)
GPT-5.4 Mini — ELO 1668, #71
- SpacetimeDB LLM Benchmark (Rust): 92.1 (#6/14)
- SpacetimeDB LLM Benchmark (TypeScript): 96.6 (#8/14)
- SpacetimeDB LLM Benchmark (C#): 88.8 (#11/14)
DeepSeek V4 Flash — ELO 1668, #72
- APEX v1 Consulting: 69.8 (#2/39)
- APEX v1 Investment Banking: 66.1 (#6/39)
- ProphetArena: 0.9503 (#9/19)
- APEX v1 Big Law: 65.7 (#23/39)
- APEX v1 Medicine (MD): 53.5 (#31/39)
Grok 4.3 — ELO 1667, #75
- SpacetimeDB LLM Benchmark (Rust): 97.8 (#1/14)
- SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#5/14)
- SpacetimeDB LLM Benchmark (C#): 93.3 (#7/14)
O3 (2025-04-16) — ELO 1667, #76
- LiveMathematicianBench: 27.0 (#13/19)
Inkling Small — ELO 1667, #77
- Agent Arena - Bash Recovery: 10.73 (#15/51)
- Agent Arena - Tool Hallucination: -0.36 (#36/51)
- Agent Arena: -6.82 (#40/51)
- Agent Arena - Praise vs Complaint: -14.84 (#46/51)
- Agent Arena - Steerability: -12.81 (#47/51)
- Agent Arena - Confirmed Success: -17.56 (#51/51)
- Chatbot Arena (Code): 1402.0 (#64/118)
- Chatbot Arena (Vision): 1206.0 (#70/147)
- Chatbot Arena (Text): 1405.0 (#136/394)
- Epoch AI - ECI: 150.17 (#140/491)
Muse Glimmer 30B — ELO 1667, #78
- Riemann-bench: 13.6 (#22/30)
- Chartography: 17.9 (#27/45)
- Roboflow Playground Object Detection: 1189.0 (#35/48)
- Gert Labs Rankings: 43.91 (#37/100)
- HANDBOOK.md Agents: 3.5 (#39/45)
- ComplexConstraints: 23.9 (#41/57)
- Roboflow Playground Overall: 1189.0 (#53/72)
Gemini 2.5 Pro — ELO 1666, #79
- LiveMathematicianBench: 32.1 (#8/19)
Gemini 2.5 Pro (Preview 03-25) — ELO 1661, #84
- Epoch AI - Enigma Eval: 4.14 (#28/46)
Qwen 3.6 27B — ELO 1657, #89
- RealWorldQA: 84.1 (#4/14)
- BenCzechMark: 70.57 (#16/74)
- OmniDocBench 1.5: 89.4 (#23/67)
- NL2Repo: 36.2 (#23/25)
- OSWorld-Verified: 63.9 (#23/29)
- CharXiv-R: 78.4 (#36/61)
Claude Opus 4.1 — ELO 1657, #90
- Artificial Analysis Intelligence Index: 28.84 (#183/603)
Claude Opus 4.1 (20250805) — ELO 1657, #91
- Epoch AI - Enigma Eval: 7.18 (#19/46)
Gemini 2.5 Pro (Preview 05-06) — ELO 1651, #95
- Epoch AI - Enigma Eval: 2.36 (#34/46)
Gemini 3.1 Flash Lite — ELO 1650, #97
- DystopiaBench: 72.9 (#35/42)
Claude Opus 4 — ELO 1649, #98
- Epoch AI - Enigma Eval: 5.57 (#25/46)
GPT-5 Mini — ELO 1648, #99
- Epoch AI - Enigma Eval: 8.19 (#17/46)
Gemini 3.1 Flash Lite (Preview) — ELO 1645, #102
- Epoch AI - Enigma Eval: 3.04 (#32/46)
Gemini 2.5 Flash (Preview 05-20) — ELO 1645, #103
- Epoch AI - Enigma Eval: 2.7 (#33/46)
O1 (2024-12-17) — ELO 1641, #107
- Epoch AI - Enigma Eval: 5.65 (#24/46)
Kimi K2 — ELO 1640, #108
- APEX v1 Consulting: 50.8 (#29/39)
- APEX v1 Big Law: 59.8 (#31/39)
- APEX v1 Investment Banking: 49.1 (#31/39)
- APEX v1 Medicine (MD): 50.0 (#32/39)
Claude Sonnet 4 — ELO 1640, #109
- Epoch AI - Enigma Eval: 3.12 (#31/46)
Nemotron 3 Ultra — ELO 1640, #111
- APEX v1 Investment Banking: 54.9 (#24/39)
- APEX v1 Big Law: 64.0 (#29/39)
- APEX v1 Medicine (MD): 54.8 (#29/39)
- APEX v1 Consulting: 50.8 (#30/39)
Qwen 3.5 27B — ELO 1638, #116
- HealthAdminBench: 13.3 (#9/11)
MiniMax-M2.7 — ELO 1638, #117
- APEX v1 Big Law: 52.5 (#36/39)
- APEX v1 Consulting: 41.9 (#38/39)
- APEX v1 Medicine (MD): 45.7 (#38/39)
- APEX v1 Investment Banking: 43.0 (#38/39)
DeepSeek V3.2 — ELO 1635, #121
- LiveMathematicianBench: 30.8 (#9/19)
- APEX v1 Investment Banking: 55.0 (#23/39)
- HMMT 2025: 90.2 (#31/82)
- APEX v1 Big Law: 55.7 (#32/39)
- APEX v1 Consulting: 49.8 (#32/39)
- APEX v1 Medicine (MD): 45.7 (#37/39)
- Vals AI LegalBench: 76.08 (#104/139)
Claude 3.7 Sonnet (20250219) — ELO 1634, #124
- Epoch AI - Enigma Eval: 2.26 (#35/46)
DeepSeek V3.2 Exp — ELO 1629, #131
- CritPt: 1.4 (#83/213)
DeepSeek V3.1 Terminus — ELO 1627, #134
- Tau3 Banking: 21.03 (#69/176)
O4 Mini — ELO 1625, #136
- Epoch AI - Enigma Eval: 9.21 (#16/46)
GPT-4.1 — ELO 1622, #142
- LiveMathematicianBench: 30.5 (#11/19)
- Epoch AI - Enigma Eval: 2.17 (#36/46)
Gemma 4 31B — ELO 1617, #152
- APEX v1 Investment Banking: 49.1 (#30/39)
- APEX v1 Consulting: 47.6 (#34/39)
- APEX v1 Big Law: 52.9 (#35/39)
- APEX v1 Medicine (MD): 42.8 (#39/39)
GPT-4.5 (Preview) — ELO 1609, #171
- Epoch AI - Enigma Eval: 3.18 (#30/46)
- Artificial Analysis Intelligence Index: 13.61 (#338/603)
GPT-5.4 Nano — ELO 1602, #175
- Roboflow Playground Captioning: 1200.0 (#24/49)
Claude 3.5 Sonnet — ELO 1601, #177
- Epoch AI - Enigma Eval: 0.91 (#39/46)
GPT-4o — ELO 1592, #197
- Epoch AI - Enigma Eval: 0.8 (#42/46)
GPT-OSS-120B — ELO 1590, #202
- APEX v1 Medicine (MD): 47.2 (#35/39)
- APEX v1 Investment Banking: 43.8 (#36/39)
- APEX v1 Big Law: 52.0 (#37/39)
- APEX v1 Consulting: 42.7 (#37/39)
Gemini 2.0 Pro (Preview 02-05) — ELO 1588, #206
- Epoch AI - Enigma Eval: 0.69 (#43/46)
GPT-4.1 Mini — ELO 1587, #207
- LiveMathematicianBench: 22.9 (#18/19)
Gemma 4 12B — ELO 1587, #209
- CritPt: 0.0 (#177/213)
Mistral Large 3 — ELO 1586, #210
- LiveMathematicianBench: 33.0 (#6/19)
GLM-4.6V — ELO 1586, #211
- CritPt: 0.0 (#171/213)
Llama 4 Maverick — ELO 1583, #216
- LiveMathematicianBench: 29.7 (#12/19)
- Epoch AI - Enigma Eval: 0.58 (#45/46)
Gemini 2.0 Flash — ELO 1583, #217
- Epoch AI - Enigma Eval: 1.1 (#38/46)
DeepSeek R1 — ELO 1580, #227
- LiveMathematicianBench: 30.6 (#10/19)
- IneqMath: 5.0 (#23/42)
Gemma 4 26B A4B — ELO 1580, #228
- CritPt: 0.0 (#176/213)
Gemini 2.0 Flash (001) — ELO 1579, #230
- Epoch AI - Enigma Eval: 0.63 (#44/46)
GPT-5 Nano — ELO 1577, #238
- CritPt: 0.0 (#165/213)
Gemini 2.5 Flash Lite — ELO 1569, #249
- CritPt: 0.0 (#188/213)
Mistral Small 4 — ELO 1566, #255
- CritPt: 0.3 (#128/213)
GLM-4.5V — ELO 1550, #291
- CritPt: 0.0 (#172/213)
Llama 3.3 70B Instruct — ELO 1543, #307
- LiveMathematicianBench: 27.0 (#14/19)
Llama 3.2 90B Vision Instruct — ELO 1542, #309
- Epoch AI - Enigma Eval: 0.38 (#46/46)
DeepSeek V2.5 — ELO 1536, #328
- Artificial Analysis Intelligence Index: 6.55 (#478/603)
- AA GPQA Diamond: 42.32 (#482/584)
GPT-OSS-20B — ELO 1533, #337
- Epoch AI - ECI: 136.65 (#345/491)
Claude 3 Opus — ELO 1525, #355
- Epoch AI - Enigma Eval: 0.82 (#41/46)
EXAONE 4.0 32B — ELO 1515, #369
- CritPt: 0.0 (#190/213)
LFM2 24B A2B — ELO 1451, #489
- BenCzechMark: 3.72 (#73/74)
Gemma 4 E2B — ELO 1433, #516
- CritPt: 0.0 (#175/213)
LLaMA-65B — ELO 1417, #546
- Artificial Analysis Intelligence Index: 1.71 (#584/603)
DeepSeek Coder V2 — ELO 1538
- Artificial Analysis Intelligence Index: 4.74 (#511/603)
DeepSeek V2 Chat — ELO 1539
- Artificial Analysis Intelligence Index: 3.27 (#543/603)
DeepSeek V4 Pro Preview — ELO 1683
- MineBench: 1394.44 (#28/61)
- HANDBOOK.md Agents: 9.2 (#29/45)
- ComplexConstraints: 28.0 (#40/57)
DeepSeek-V4-Flash-Vision-Exp — ELO 1752
- LLM Stats (ZEROBench): 35.0 (#2/10)
- LLM Stats (NL2Repo): 57.7 (#3/18)
- LLM Stats (DeepSWE): 59.3 (#6/12)
- LLM Stats (AutomationBench): 25.7 (#7/13)
- LLM Stats (Agents' Last Exam): 27.3 (#8/11)
- LLM Stats (Terminal-Bench 2.1): 83.9 (#10/29)
- LLM Stats Score: 48.67 (#15/342)
GPT-5.1 Instant — ELO 1600
- Epoch AI - Enigma Eval: 1.94 (#37/46)
GPT-5.2 ChatGPT — ELO 1692
- AI Chess Leaderboard (Continuation): 842.0 (#64/259)
- AI Chess Leaderboard (Reasoning): 835.0 (#96/318)
GPT-5.5 Instant — ELO 1666
- Epoch AI - ECI: 143.49 (#263/491)
Gemini 3 Deep Think — ELO 1761
- AA CritPt: 25.71 (#14/490)
LFM2.5-2.6B — ELO 1515
- AA Omniscience: -10.95 (#128/487)
- Tau3 Banking: 7.22 (#134/176)
- AA GDPval: 251.28 (#185/210)
- AA Humanity's Last Exam: 6.21 (#334/575)
- Artificial Analysis Intelligence Index: 10.99 (#380/603)
- AA GPQA Diamond: 55.76 (#403/584)
- AA Long Context Reasoning: 5.33 (#433/508)
- AA Omniscience - Science, Engineering & Mathematics: 12.2 (#437/486)
- AA CritPt: 0.0 (#466/490)
- AA Omniscience - Business: 3.8 (#479/487)
- ...and 6 more
Laguna M.1 — ELO 1568
- Vals AI MedScribe: 65.91 (#81/87)
Ministral 8B — ELO 1448
- LM Market Cap LMC Score: 37.4 (#395/428)
O1 Pro — ELO 1625
- Epoch AI - Enigma Eval: 6.14 (#22/46)
- Artificial Analysis Intelligence Index: 19.12 (#270/603)
Pixtral Large — ELO 1539
- Epoch AI - Enigma Eval: 0.84 (#40/46)
Qwen 3.5 — ELO 1629
- APEX v1 Consulting: 50.5 (#31/39)
- APEX v1 Big Law: 54.6 (#33/39)
- APEX v1 Medicine (MD): 48.2 (#33/39)
- APEX v1 Investment Banking: 47.8 (#34/39)
Qwen 3.6 Max — ELO 1715
- DystopiaBench: 71.1 (#31/42)
Qwen 3.7 Flash — ELO 1638
- Roboflow Playground Open Prompt: 1200.0 (#20/54)

Top-10 New Scores (126)

Claude Fable 5 on APEX v1: 66.0 (#4)
Claude Fable 5 on APEX v1 Big Law: 69.3 (#16)
Claude Fable 5 on APEX v1 Consulting: 66.6 (#5)
Claude Fable 5 on APEX v1 Investment Banking: 66.6 (#4)
Claude Fable 5 on APEX v1 Medicine (MD): 61.4 (#13)
Claude Fable 5 on ArxivMath: 78.6 (#2)
Claude Fable 5 on CyberGym: 83.8 (#3)
Claude Fable 5 on Epoch AI - Cursorbench: 65.2 (#11)
Claude Fable 5 on Epoch AI - Enigma Eval: 39.28 (#1)
Claude Fable 5 on FrontierCode: 46.3 (#3)
Claude Fable 5 on FutureEval: 13.23 (#1)
Claude Fable 5 on SpacetimeDB LLM Benchmark (C#): 98.9 (#1)
Claude Fable 5 on SpacetimeDB LLM Benchmark (Rust): 68.3 (#8)
Claude Fable 5 on SpacetimeDB LLM Benchmark (TypeScript): 95.3 (#10)
Claude Opus 5 on Agents' Last Exam: 27.0 (#7)
Claude Opus 5 on Epoch AI - Cursorbench: 64.3 (#16)
Claude Opus 5 on LiveMathematicianBench: 45.1 (#3)
Claude Opus 5 on SpacetimeDB LLM Benchmark (C#): 66.7 (#12)
Claude Opus 5 on SpacetimeDB LLM Benchmark (Rust): 68.3 (#9)
Claude Opus 5 on SpacetimeDB LLM Benchmark (TypeScript): 96.4 (#9)
Claude Opus 5 on Vals AI LegalBench: 86.97 (#5)
Claude Opus 5 on Vals AI MMMU: 89.88 (#1)
Claude Opus 5 on Vals AI MedCode: 63.57 (#1)
Claude Opus 5 on Vals AI MedScribe: 90.98 (#1)
Claude Opus 5 on Vals AI SAGE: 49.43 (#19)
GPT-5.4 Pro on AA CritPt: 30.0 (#4)
GPT-5.4 Pro on Epoch AI - Enigma Eval: 23.82 (#4)
GPT-5.4 Pro on OpenAI GPT-5.4 Launch - FrontierMath Tier 1-3: 50.0 (#1)
GPT-5.4 Pro on OpenAI GPT-5.5 Launch - FrontierMath Tier 1-3: 50.0 (#3)
GPT-5.5 Pro on AA CritPt: 30.57 (#2)
GPT-5.5 Pro on OpenAI GPT-5.5 Launch - FrontierMath Tier 1-3: 52.4 (#1)
GPT-5.5 Pro on OpenAI GPT-5.5 Launch - FrontierMath Tier 4: 39.6 (#1)
GPT-5.6 Sol on CyberGym: 83.6 (#4)
GPT-5.6 Sol on Design Arena (3D): 1438.0 (#2)
GPT-5.6 Sol on Design Arena (Data Viz): 1338.0 (#5)
GPT-5.6 Sol on Design Arena (Game Dev): 1414.0 (#2)
GPT-5.6 Sol on Design Arena (UI Components): 1387.0 (#1)
GPT-5.6 Sol on Epoch AI - Apex Agents: 37.7 (#12)
GPT-5.6 Sol on Epoch AI - Cursorbench: 64.5 (#14)
GPT-5.6 Sol on Epoch AI - Enigma Eval: 37.12 (#2)
GPT-5.6 Sol on ParseBench: 62.12 (#25)
GPT-5.6 Sol on Roboflow Playground Open Prompt: 1200.0 (#29)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (C#): 92.2 (#9)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (Rust): 82.9 (#7)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (TypeScript): 99.5 (#1)
GPT-5.6 Sol on Toolathlon: 74.9 (#6)
Gemini 3.7 Flash on AI for Education Pedagogy: 90.77 (#7)
Gemini 3.7 Flash on AI for Education Pedagogy - Maths: 92.06 (#14)
Gemini 3.7 Flash on AI for Education Pedagogy - Primary: 94.84 (#7)
Gemini 3.7 Flash on AI for Education Pedagogy - Science: 92.9 (#16)
Gemini 3.7 Flash on AI for Education Pedagogy - Secondary: 89.62 (#8)
Gemini 3.7 Flash on AI for Education Pedagogy - Social studies: 87.27 (#24)
Gemini 3.7 Flash on AI for Education Pedagogy - Technology: 86.79 (#22)
Gemini 3.7 Flash on AI for Education SEND: 87.16 (#8)
Gemini 3.7 Flash on AI for Education Visual Reasoning - match (process): 88.9 (#1)
Gemini 3.7 Flash on AI for Education Visual Reasoning - odd one out: 84.4 (#2)
Gemini 3.7 Flash on AI for Education Visual Reasoning - pattern completion (2d): 79.5 (#5)
Gemini 3.7 Flash on ARC-AGI-1: 95.5 (#15)
Gemini 3.7 Flash on ARC-AGI-2: 84.58 (#13)
Gemini 3.7 Flash on Agents on Rails: 71.4 (#10)
Gemini 3.7 Flash on Arena AI Code: 1587.0 (#10)
Gemini 3.7 Flash on BioMysteryBench Human-Difficult: 43.5 (#3)
Gemini 3.7 Flash on BioMysteryBench Human-Solvable: 87.1 (#2)
Gemini 3.7 Flash on CharXiv-R: 84.5 (#18)
Gemini 3.7 Flash on Chartography: 43.0 (#2)
Gemini 3.7 Flash on ComplexConstraints: 42.4 (#9)
Gemini 3.7 Flash on Deep20Bench: 14.03 (#5)
Gemini 3.7 Flash on Design Arena (3D): 1358.0 (#8)
Gemini 3.7 Flash on Design Arena (Data Viz): 1353.0 (#4)
Gemini 3.7 Flash on Design Arena (UI Components): 1319.0 (#14)
Gemini 3.7 Flash on Epoch AI - Cursorbench: 61.6 (#24)
Gemini 3.7 Flash on Epoch AI - ECI: 156.85 (#49)
Gemini 3.7 Flash on Epoch AI - Proofbench: 58.0 (#8)
Gemini 3.7 Flash on FrontierCode: 43.6 (#4)
Gemini 3.7 Flash on Gert Labs Rankings: 65.05 (#6)
Gemini 3.7 Flash on HANDBOOK.md Agents: 11.9 (#21)
Gemini 3.7 Flash on Icelandic LLM - ARC-Challenge-IS: 95.22 (#1)
Gemini 3.7 Flash on Icelandic LLM - Belebele-IS: 94.56 (#6)
Gemini 3.7 Flash on Icelandic LLM - GED: 82.0 (#4)
Gemini 3.7 Flash on Icelandic LLM - Inflection: 97.33 (#4)
Gemini 3.7 Flash on Icelandic LLM - WikiQA-IS: 66.82 (#3)
Gemini 3.7 Flash on InfoOps Bench: 48.5 (#18)
Gemini 3.7 Flash on Kaggle FACTS Parametric: 78.72 (#2)
Gemini 3.7 Flash on LMArena Text Arena: 1489.87 (#9)
Gemini 3.7 Flash on LMArena WebDev Arena: 1587.86 (#9)
Gemini 3.7 Flash on MMOU: 82.5 (#2)
Gemini 3.7 Flash on Riemann-bench: 39.2 (#7)
Gemini 3.7 Flash on Roboflow Playground OCR: 1186.0 (#45)
Gemini 3.7 Flash on Roboflow Playground Object Detection: 1218.0 (#19)
Gemini 3.7 Flash on Roboflow Playground Overall: 1202.0 (#35)
Gemini 3.7 Flash on Vals AI Code Migration: 34.8 (#17)
Gemini 3.7 Flash on Vals AI GPQA: 93.94 (#4)
Gemini 3.7 Flash on Vals AI LegalBench: 87.26 (#3)
Gemini 3.7 Flash on Vals AI LiveCodeBench: 88.65 (#3)
Gemini 3.7 Flash on Vals AI MMLU-Pro: 90.12 (#4)
Gemini 3.7 Flash on Vals AI MMMU: 88.96 (#3)
Gemini 3.7 Flash on Vals AI MedCode: 53.39 (#7)
Gemini 3.7 Flash on Vals AI MedScribe: 83.94 (#25)
Gemini 3.7 Flash on Vals AI MortgageTax: 66.65 (#32)
Gemini 3.7 Flash on Vals AI ProgramBench: 68.66 (#9)
Gemini 3.7 Flash on Vals AI SAGE: 49.23 (#21)
Gemini 3.7 Flash on Vals AI SWE-bench Verified: 80.8 (#22)
Gemini 3.7 Flash on Vals AI SkillsBench: 65.89 (#2)
Gemini 3.7 Flash on Vals AI TaxEval v2: 74.73 (#29)
Gemini 3.7 Flash on WebDev Arena: 1587.48 (#9)
Kimi K3 on AI for Education Visual Reasoning: 76.0 (#10)
Kimi K3 on AI for Education Visual Reasoning - match (figure): 63.9 (#13)
Kimi K3 on AI for Education Visual Reasoning - match (process): 75.0 (#7)
Kimi K3 on AI for Education Visual Reasoning - odd one out: 79.2 (#4)
Kimi K3 on AI for Education Visual Reasoning - pattern completion (2d): 71.2 (#9)
Kimi K3 on AI for Education Visual Reasoning - pattern completion (linear): 82.1 (#11)
Kimi K3 on AI for Education Visual Reasoning - reasoning by analogy: 83.7 (#5)
Kimi K3 on APEX v1: 68.4 (#2)
Kimi K3 on APEX v1 Big Law: 73.2 (#7)
Kimi K3 on APEX v1 Consulting: 69.7 (#3)
Kimi K3 on APEX v1 Investment Banking: 66.4 (#5)
Kimi K3 on APEX v1 Medicine (MD): 64.3 (#2)
Kimi K3 on Benchmarks.bio - BioSecBench-Refusal: 8.6 (#16)
Kimi K3 on Benchmarks.bio - BioSecBench-Surveillance: 48.3 (#12)
Kimi K3 on CyberGym: 80.0 (#8)
Kimi K3 on Epoch AI - Cursorbench: 60.8 (#28)
Kimi K3 on NL2Repo: 58.0 (#3)
Kimi K3 on SpacetimeDB LLM Benchmark (C#): 93.3 (#8)
Kimi K3 on SpacetimeDB LLM Benchmark (Rust): 65.1 (#12)
Kimi K3 on SpacetimeDB LLM Benchmark (TypeScript): 93.3 (#11)
Kimi K3 on Toolathlon: 76.5 (#2)

New #1 Leaders (38)

Multi-SWE-Bench (c): CodeArts Agent + CodeArts-GLM-5.1 (54.33) beat RepoRepair + Claude-4-Sonnet by 41.72
Lean AI Formalization Leaderboard: Humanifa + GPT 5.6 sol (183.0) beat Seed Prover (ByteDance) by 28.0
Multi-SWE-Bench (c++): CodeArts Agent + CodeArts-GLM-5.1 (47.29) beat InfCode + GPT5(Sep) by 21.71
Chatbot Arena (Video Edit): dreamina-seedance-2.5-720p (1411.0) beat minimax-h3 by 21.0
KernelBench Hub - Hard: Qwen 3.8 Max (69.26) beat Claude Fable 5 by 14.05
SpreadsheetBench v2: arito (45.46) beat Claude Opus 4.6 by 10.57
SWE-PolyBench Verified (TypeScript): HMigBot (53.0) beat Atlassian Rovo Dev (2025-12-08) by 10.0
APEX v1 Consulting: GPT-5.6 Terra (76.2) beat GPT-5.2 Codex by 9.3
MobileWorld - GUI Only: Qwen-UI-Agent (82.1) beat Kimi K3 by 7.7
LVBench: Gemini 3.7 Flash (85.4) beat Seed 2.1 Pro by 7.4
SWE-PolyBench Verified (Python): HMigBot (61.95) beat Atlassian Rovo Dev (2025-12-08) by 7.08
APEX v1 Investment Banking: GPT-5.6 Terra (71.8) beat GPT-5.3 Codex by 6.8
AI for Education Visual Reasoning - reasoning by analogy: Gemini 3.7 Flash (94.4) beat Gemini 3.5 Flash by 5.6
MRCR-v2 128k: Gemini 3.7 Flash (97.0) beat Gemini 3.6 Flash by 5.2
EnterpriseRAG Bench - Completeness: metor.com (86.22) beat Troml by 4.38
MMEB Leaderboard: Ovis-Omni-Embedding-v0.5-3B (56.95) beat Tianmu-Emb-Uni by 4.12
Kaggle FACTS (Google): Gemini 3.7 Flash (71.27) beat Gemini 3.1 Pro (Preview) by 4.02
AI for Education Visual Reasoning: Gemini 3.7 Flash (89.9) beat Gemini 3.5 Flash by 3.9
BioMysteryBench Human-Solvable: Claude Sonnet 5 (87.5) beat Claude Mythos 5 by 3.6
EnterpriseRAG Bench: metor.com (80.34) beat Troml by 3.55
AI for Education Visual Reasoning - pattern completion (linear): Gemini 3.7 Flash (95.7) beat GPT-5.6 Sol by 3.4
BioMysteryBench Human-Difficult: GPT-5.6 Terra (49.4) beat Claude Mythos 5 by 3.3
Kaggle FACTS Multimodal: Gemini 3.7 Flash (49.83) beat Gemini 3.6 Flash by 2.62
APEX v1: GPT-5.6 Terra (Max) (69.5) beat GPT-5.4 (High) by 2.3
AI for Education Visual Reasoning - match (figure): Gemini 3.7 Flash (87.4) beat Gemini 3.5 Flash by 2.2
Design Arena (UI Components): GPT-5.6 Sol (xHigh) (1387.0) beat Kimi K3 by 2.0
AI for Education Pedagogy - Science: Grok 4.6 (96.72) beat Qwen 3.5 Plus by 1.64
Humanity's Last Exam (Self-Reported, With Tools): Ornith-1.5-397B (56.1) beat GLM-5.2 by 1.4
Epoch AI - ECI: Claude Fable 5 (High) (162.49) beat GPT-5.6 Sol (Medium) by 0.84
CyberGym: GLM-5.3 (84.5) beat Claude Mythos 5 by 0.7
WebVoyager: browser-control (Fable 5) (99.19) beat Alumnium by 0.69
KernelBench Hub - Mega: Claude Fable 5 (24.61) beat Claude Opus 5 by 0.32
SpreadsheetBench: JT AlphaData (98.5) beat Qingqiu Agent by 0.25
Icelandic LLM - WinoGrande-IS: Gemini 3.7 Flash (96.32) beat Gemini 3.1 Pro (Preview) by 0.18
Icelandic LLM Leaderboard - Average: Gemini 3.7 Flash (88.71) beat Gemini 3.1 Pro (Preview) by 0.17
BRIDGE Medical Leaderboard - Few-Shot: Qwen 3.8 27B (Non-reasoning) (55.67) beat Gemini 1.5 Pro (002) by 0.16
AI for Education Pedagogy - Secondary: Grok 4.6 (91.19) beat GPT-5.5 by 0.15
ProphetArena: Gemini 3.7 Flash (0.9687) beat Gemini 3.6 Flash by 0.01

The Aggregate Digest — 2026-08-22

2026-08-22T06:30:27.535616+00:00

Daily

New Models (1)

DeepSeek-V4-Flash-Vision-Exp — ELO 1754
- LLM Stats (ZEROBench): 35.0 (#2/10)
- LLM Stats (NL2Repo): 57.7 (#3/18)
- LLM Stats (DeepSWE): 59.3 (#6/12)
- LLM Stats (AutomationBench): 25.7 (#7/13)
- LLM2014 Logic 2026-08: 58.1 (#8/44)
- LLM Stats (Agents' Last Exam): 27.3 (#8/11)
- LLM Stats (Terminal-Bench 2.1): 83.9 (#10/29)
- LLM Stats Score: 48.67 (#15/342)

Top-10 New Scores (16)

Claude Fable 5 on ArxivMath: 78.6 (#2)
Claude Fable 5 on CyberGym: 83.8 (#3)
Claude Fable 5 on FrontierCode: 46.3 (#3)
Claude Opus 5 on Benchmarks.bio - BioSecBench-Refusal: 29.4 (#12)
Claude Opus 5 on Vals AI MedCode: 63.57 (#1)
Claude Opus 5 on Vals AI MedScribe: 90.98 (#1)
GPT-5.6 Sol on CyberGym: 83.6 (#4)
GPT-5.6 Sol on Toolathlon: 74.9 (#5)
Gemini 3.7 Flash on BioMysteryBench Human-Difficult: 43.5 (#3)
Gemini 3.7 Flash on BioMysteryBench Human-Solvable: 87.1 (#2)
Gemini 3.7 Flash on CharXiv-R: 84.5 (#18)
Gemini 3.7 Flash on FrontierCode: 43.6 (#4)
Gemini 3.7 Flash on Roboflow Playground OCR: 1186.0 (#45)
Kimi K3 on CyberGym: 80.0 (#8)
Kimi K3 on NL2Repo: 58.0 (#3)
Kimi K3 on Toolathlon: 76.5 (#2)

New #1 Leaders (5)

LVBench: Gemini 3.7 Flash (85.4) beat Seed 2.1 Pro by 7.4
MRCR-v2 128k: Gemini 3.7 Flash (97.0) beat Gemini 3.6 Flash by 5.2
BioMysteryBench Human-Solvable: Claude Sonnet 5 (87.5) beat Claude Mythos 5 by 3.6
BioMysteryBench Human-Difficult: GPT-5.6 Terra (49.4) beat Claude Mythos 5 by 3.3
CyberGym: GLM-5.3 (84.5) beat Claude Mythos 5 by 0.7

The Aggregate Digest — 2026-08-21

2026-08-21T11:50:31.255079+00:00

Frontier

Best available model: Claude Opus 5 (1792) takes the crown from Gemini 3.7 Flash (1784)

Daily

Top-10 New Scores (32)

Claude Opus 5 on AA GDPval: 1844.67 (#1)
Claude Opus 5 on AA Omniscience - Business: 53.0 (#2)
Claude Opus 5 on AA Omniscience - Health: 52.42 (#2)
Claude Opus 5 on AA Omniscience - Humanities & Social Sciences: 58.6 (#2)
Claude Opus 5 on AA Omniscience - Law: 58.4 (#8)
Claude Opus 5 on AA Omniscience - Science, Engineering & Mathematics: 57.52 (#2)
Claude Opus 5 on AA Omniscience - Software Engineering (SWE): 86.0 (#3)
Claude Opus 5 on AA-Briefcase: 57.98 (#1)
Claude Opus 5 on AA-Omniscience Accuracy: 60.87 (#2)
GPT-5.6 Sol on AA GDPval: 1722.96 (#8)
GPT-5.6 Sol on AA Omniscience - Business: 49.6 (#5)
GPT-5.6 Sol on AA Omniscience - Health: 52.4 (#3)
GPT-5.6 Sol on AA Omniscience - Humanities & Social Sciences: 57.8 (#3)
GPT-5.6 Sol on AA Omniscience - Law: 55.9 (#12)
GPT-5.6 Sol on AA Omniscience - Science, Engineering & Mathematics: 55.7 (#5)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE): 85.5 (#6)
GPT-5.6 Sol on AA-Briefcase: 41.85 (#8)
GPT-5.6 Sol on AA-Omniscience Accuracy: 59.4 (#4)
GPT-5.6 Sol on Roboflow Playground Open Prompt: 1200.0 (#29)
Gemini 3.7 Flash on AA GDPval: 1531.55 (#24)
Gemini 3.7 Flash on AA Omniscience - Business: 45.6 (#18)
Gemini 3.7 Flash on AA Omniscience - Health: 44.7 (#27)
Gemini 3.7 Flash on AA Omniscience - Humanities & Social Sciences: 55.2 (#11)
Gemini 3.7 Flash on AA Omniscience - Law: 62.1 (#3)
Gemini 3.7 Flash on AA Omniscience - Science, Engineering & Mathematics: 52.0 (#19)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE): 72.3 (#26)
Gemini 3.7 Flash on AA-Briefcase: 36.36 (#11)
Gemini 3.7 Flash on AA-Omniscience Accuracy: 55.32 (#16)
Gemini 3.7 Flash on ARC-AGI-1: 95.5 (#15)
Gemini 3.7 Flash on ARC-AGI-2: 84.58 (#13)
Gemini 3.7 Flash on Roboflow Playground Object Detection: 1218.0 (#19)
Gemini 3.7 Flash on Roboflow Playground Overall: 1218.0 (#16)

New #1 Leaders (10)

AA GDPval: Claude Opus 5 (Adaptive Reasoning, Max Effort) (1844.67) beat Qwen3.8 2.4T A95B by 124.28
AA Omniscience - Software Engineering (SWE): Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (91.1) beat Kimi K3 (Max) by 18.9
AA Omniscience - Business: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (57.4) beat Gemini 3 Pro (Preview) (High) by 10.5
AA-Omniscience Accuracy: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (65.35) beat Gemini 3 Pro (Preview) (High) by 9.6
AA Omniscience - Science, Engineering & Mathematics: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (61.28) beat Gemini 3 Pro (Preview) (High) by 8.98
AA Omniscience - Humanities & Social Sciences: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (64.3) beat Gemini 3 Pro (Preview) (High) by 8.5
AA-Briefcase: Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) (57.98) beat Kimi K3 (Max) by 7.01
AA Omniscience - Health: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (53.72) beat Gemini 3 Pro (Preview) (High) by 6.42
MMEB Leaderboard: Ovis-Omni-Embedding-v0.5-3B (57.25) beat Tianmu-Emb-Uni by 4.42
AA Omniscience - Law: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (66.2) beat Gemini 3 Pro (Preview) (High) by 1.1

The Aggregate Digest — 2026-08-20

2026-08-20T05:51:23.006145+00:00

Frontier

Best available model: GLM-5.3 enters at #11 (1753 ELO) on 43 benchmarks

Daily

New Benchmarks (3)

SecIT Bench (Pydantic AI) (Accuracy (%)): leader Grok 4.6 (80.65), 14 models
Cribl's benchmark for IT and security operations agents, run under a plain Pydantic AI harness. Thirty telemetry scenarios across intrusion and breach, covert egress, service errors and performance degradation are each rolled out three times; an incomplete answer scores zero, and the reported figure is the mean scenario accuracy.
SecIT Bench (Claude Code) (Accuracy (%)): leader Claude Opus 5 (78.64), 3 models
The SecIT Bench scenario set run inside the Claude Code agent product rather than a plain model harness. Scored identically to the Pydantic AI board, and kept separate because the scaffold is part of what is being measured: the same model moves by up to three points between harnesses.
SecIT Bench (Codex) (Accuracy (%)): leader GPT-5.6 Terra (71.88), 3 models
The SecIT Bench scenario set run inside the Codex agent product rather than a plain model harness. Scored identically to the Pydantic AI board, and kept separate because the scaffold is part of what is being measured: GPT-5.6 Terra scores 71.9 here against 69.4 under the plain harness.

Top-10 New Scores (21)

Claude Fable 5 on APEX v1: 66.0 (#4)
Claude Fable 5 on APEX v1 Big Law: 69.3 (#16)
Claude Fable 5 on APEX v1 Consulting: 66.6 (#5)
Claude Fable 5 on APEX v1 Investment Banking: 66.6 (#4)
Claude Fable 5 on APEX v1 Medicine (MD): 61.4 (#13)
Claude Fable 5 on FutureEval: 13.23 (#1)
Claude Opus 5 on LiveMathematicianBench: 45.1 (#3)
Claude Opus 5 on Vals AI LegalBench: 86.97 (#5)
Claude Opus 5 on Vals AI MMMU: 89.88 (#1)
Claude Opus 5 on Vals AI SAGE: 49.43 (#19)
Gemini 3.7 Flash on Icelandic LLM - ARC-Challenge-IS: 95.22 (#1)
Gemini 3.7 Flash on Icelandic LLM - Belebele-IS: 94.56 (#6)
Gemini 3.7 Flash on Icelandic LLM - GED: 82.0 (#4)
Gemini 3.7 Flash on Icelandic LLM - Inflection: 97.33 (#4)
Gemini 3.7 Flash on Icelandic LLM - WikiQA-IS: 66.82 (#3)
Gemini 3.7 Flash on InfoOps Bench: 48.5 (#18)
Kimi K3 on APEX v1: 68.4 (#2)
Kimi K3 on APEX v1 Big Law: 73.2 (#7)
Kimi K3 on APEX v1 Consulting: 69.7 (#3)
Kimi K3 on APEX v1 Investment Banking: 66.4 (#5)
Kimi K3 on APEX v1 Medicine (MD): 64.3 (#2)

New #1 Leaders (9)

Multi-SWE-Bench (c): CodeArts Agent + CodeArts-GLM-5.1 (54.33) beat RepoRepair + Claude-4-Sonnet by 41.72
Multi-SWE-Bench (c++): CodeArts Agent + CodeArts-GLM-5.1 (47.29) beat InfCode + GPT5(Sep) by 21.71
APEX v1 Consulting: GPT-5.6 Terra (76.2) beat GPT-5.2 Codex by 9.3
APEX v1 Investment Banking: GPT-5.6 Terra (71.8) beat GPT-5.3 Codex by 6.8
APEX v1: GPT-5.6 Terra (Max) (69.5) beat GPT-5.4 (High) by 2.3
Humanity's Last Exam (Self-Reported, With Tools): Ornith-1.5-397B (56.1) beat GLM-5.2 by 1.4
Icelandic LLM - WinoGrande-IS: Gemini 3.7 Flash (96.32) beat Gemini 3.1 Pro (Preview) by 0.18
Icelandic LLM Leaderboard - Average: Gemini 3.7 Flash (88.71) beat Gemini 3.1 Pro (Preview) by 0.17
BRIDGE Medical Leaderboard - Few-Shot: Qwen3.8-27B-Non-Thinking (55.67) beat Gemini 1.5 Pro (002) by 0.16

The Aggregate Digest — 2026-08-19

2026-08-19T08:54:18.919667+00:00

Daily

New Benchmarks (4)

BenchBench-Protocol (Normalized Rubric Score (%)): leader Claude Opus 5 (59.2), 9 models
Benchling's wet-lab protocol-modification benchmark: 149 expert-reviewed tasks derived from the differences between a published protocol and the version a scientist actually ran, across 96 source protocols in nine domains of wet-lab biology. Responses are graded against weighted rubrics recovered from the real modification, so the score measures whether a model reasons about prior choices and downstream steps rather than reciting protocol text.
Guesswork 2026-08 (MAE (z-score units)): leader Llama 3.1 8B Instruct (1.9018), 31 models
Guesswork standings for 2026-08: each contestant predicts newly scraped (model, benchmark) scores from day-before information only, scored as mean absolute error in z-score units — lower is better. The Aggregate's own row is excluded so the site's predictor cannot feed itself a score.
Guesswork 2026-07 (MAE (z-score units)): leader Gemma 3 4B (IT) (6.881), 37 models
Guesswork standings for 2026-07: each contestant predicts newly scraped (model, benchmark) scores from day-before information only, scored as mean absolute error in z-score units — lower is better. The Aggregate's own row is excluded so the site's predictor cannot feed itself a score.
Guesswork 2026-06 (MAE (z-score units)): leader Llama 3.1 8B Instruct (2.7281), 19 models
Guesswork standings for 2026-06: each contestant predicts newly scraped (model, benchmark) scores from day-before information only, scored as mean absolute error in z-score units — lower is better. June was the immature-matrix month, when several LLMs beat the aggregate.

Top-10 New Scores (33)

Claude Fable 5 on Epoch AI - Cursorbench: 65.2 (#11)
Claude Fable 5 on SpacetimeDB LLM Benchmark (C#): 98.9 (#1)
Claude Fable 5 on SpacetimeDB LLM Benchmark (Rust): 68.3 (#8)
Claude Fable 5 on SpacetimeDB LLM Benchmark (TypeScript): 95.3 (#10)
Claude Opus 5 on Epoch AI - Cursorbench: 64.3 (#16)
Claude Opus 5 on SpacetimeDB LLM Benchmark (C#): 66.7 (#12)
Claude Opus 5 on SpacetimeDB LLM Benchmark (Rust): 68.3 (#9)
Claude Opus 5 on SpacetimeDB LLM Benchmark (TypeScript): 96.4 (#9)
GLM-5.3 on AA CritPt: 19.14 (#27)
GLM-5.3 on AA GPQA Diamond: 91.72 (#30)
GLM-5.3 on AA Humanity's Last Exam: 42.26 (#28)
GLM-5.3 on AA Long Context Reasoning: 76.33 (#40)
GLM-5.3 on AA Omniscience: 14.3 (#36)
GLM-5.3 on AA SciCode: 56.48 (#10)
GLM-5.3 on Artificial Analysis Intelligence Index: 59.51 (#8)
GLM-5.3 on LM Market Cap LMC Score: 82.7 (#75)
GLM-5.3 on Tau3 Banking: 50.31 (#3)
GPT-5.5 Pro on AA CritPt: 30.57 (#2)
GPT-5.6 Sol on Epoch AI - Apex Agents: 37.7 (#12)
GPT-5.6 Sol on Epoch AI - Cursorbench: 64.5 (#14)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (C#): 92.2 (#9)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (Rust): 82.9 (#7)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (TypeScript): 99.5 (#1)
Gemini 3 Deep Think on AA CritPt: 25.71 (#14)
Gemini 3.7 Flash on Chartography: 43.0 (#2)
Gemini 3.7 Flash on ComplexConstraints: 42.4 (#9)
Gemini 3.7 Flash on Epoch AI - Cursorbench: 61.6 (#24)
Gemini 3.7 Flash on Epoch AI - ECI: 156.36 (#55)
Gemini 3.7 Flash on Gert Labs Rankings: 65.17 (#6)
Gemini 3.7 Flash on HANDBOOK.md Agents: 11.9 (#21)
Gemini 3.7 Flash on MMOU: 82.5 (#2)
Gemini 3.7 Flash on Riemann-bench: 39.2 (#7)
Gemini 3.7 Flash on WebDev Arena: 1587.48 (#9)

New #1 Leaders (6)

SWE-PolyBench Verified (TypeScript): HMigBot (53.0) beat Atlassian Rovo Dev (2025-12-08) by 10.0
SWE-PolyBench Verified (Python): HMigBot (61.95) beat Atlassian Rovo Dev (2025-12-08) by 7.08
EnterpriseRAG Bench - Completeness: metor.com (86.22) beat Troml by 4.38
Kaggle FACTS (Google): Gemini 3.7 Flash (71.27) beat Gemini 3.1 Pro (Preview) by 4.02
EnterpriseRAG Bench: metor.com (80.34) beat Troml by 3.55
Agent Arena - Confirmed Success: Inkling Small (18.91) beat Claude Opus 5 (Max) by 0.95

The Aggregate Digest — 2026-08-18

2026-08-18T05:14:30.331602+00:00

Daily

New Benchmarks (6)

Z.ai GLM-5.3 Launch - Terminal Bench 3.0 (Accuracy (%)): leader GPT-5.6 Sol (34.6), 6 models
Terminal-Bench 3.0 terminal-agent tasks as reported in Z.ai's GLM-5.3 launch table, using Claude Code with maximum reasoning effort, isolated task containers, official verifiers, and avg@3 over three rollouts; Z.ai-run frontier comparison.
Z.ai GLM-5.3 Launch - SWE-Marathon v1.1 (Pass@1 (%)): leader Claude Opus 4.8 (48.8), 6 models
SWE-Marathon v1.1 ultra-long-horizon software-engineering tasks as reported in Z.ai's GLM-5.3 launch table; Z.ai-run frontier comparison using Claude Code at maximum effort.
Z.ai GLM-5.3 Launch - ExploitGym (2h) (Successful Intended Exploits (#)): leader GPT-5.6 Sol (216.0), 7 models
ExploitGym real-world exploit-generation tasks completed under a two-hour throughput-normalized budget, as reported in Z.ai's GLM-5.3 launch table; Z.ai-run frontier comparison.
Z.ai GLM-5.3 Launch - ExploitGym (6h) (Successful Intended Exploits (#)): leader GPT-5.6 Sol (293.0), 7 models
ExploitGym real-world exploit-generation tasks completed under a six-hour throughput-normalized budget, as reported in Z.ai's GLM-5.3 launch table; Z.ai-run frontier comparison.
Z.ai GLM-5.3 Launch - HLE with Tools (Accuracy (%)): leader GPT-5.6 Sol (64.5), 8 models
Humanity's Last Exam with tool use as reported in Z.ai's GLM-5.3 launch table; Z.ai-run comparison across eight recent frontier models.
Z.ai GLM-5.3 Launch - GDPval-AA v2 (ELO): leader GLM-5.3 (1769.0), 8 models
Artificial Analysis GDPval-AA v2 evaluation of economically valuable professional knowledge work, reported as Elo in Z.ai's GLM-5.3 launch table; Artificial Analysis runs compared by Z.ai.

Top-10 New Scores (8)

Claude Fable 5 on Epoch AI - Enigma Eval: 39.28 (#1)
GLM-5.3 on Agents on Rails: 79.4 (#6)
GPT-5.4 Pro on Epoch AI - Enigma Eval: 23.82 (#4)
GPT-5.6 Sol on Epoch AI - Enigma Eval: 37.12 (#2)
Gemini 3.7 Flash on Agents on Rails: 71.4 (#10)
Gemini 3.7 Flash on Deep20Bench: 14.03 (#5)
Gemini 3.7 Flash on Design Arena (Data Viz): 1358.0 (#4)
Gemini 3.7 Flash on Kaggle FACTS Parametric: 78.72 (#2)

New #1 Leaders (11)

Chatbot Arena (Video Edit): dreamina-seedance-2.5-720p (1411.0) beat minimax-h3 by 21.0
SpreadsheetBench v2: arito (45.46) beat Claude Opus 4.6 by 10.57
AI for Education Visual Reasoning - reasoning by analogy: Gemini-3.7 Flash (94.4) beat Gemini 3.5 Flash by 5.6
AI for Education Visual Reasoning: Gemini-3.7 Flash (89.9) beat Gemini 3.5 Flash by 3.9
AI for Education Visual Reasoning - pattern completion (linear): Gemini-3.7 Flash (95.7) beat GPT-5.6 Sol by 3.4
Kaggle FACTS Multimodal: Gemini 3.7 Flash (49.83) beat Gemini 3.6 Flash by 2.62
AI for Education Visual Reasoning - match (figure): Gemini-3.7 Flash (87.4) beat Gemini 3.5 Flash by 2.2
AI for Education Pedagogy - Science: Grok 4.6 (96.72) beat Qwen 3.5 Plus by 1.64
SpreadsheetBench: JT AlphaData (98.5) beat Qingqiu Agent by 0.25
AI for Education Pedagogy - Secondary: Grok 4.6 (91.19) beat GPT-5.5 by 0.15
ProphetArena: Gemini 3.7 Flash (0.9799) beat Gemini 3.6 Flash by 0.02

The Aggregate Digest — 2026-08-17

2026-08-17T18:32:32.040913+00:00

Daily

New Benchmarks (1)

LLM2014 Logic 2026-08 (Median Score): leader GPT-5.5 (xHigh) (73.89), 40 models
August 2026 monthly snapshot of the LLM2014 project public leaderboard, scoring models on logic and reasoning problems from its continuously updated test set.

Top-10 New Scores (23)

GPT-5.5 Pro on OpenAI GPT-5.5 Launch - FrontierMath Tier 1-3: 52.4 (#1)
GPT-5.5 Pro on OpenAI GPT-5.5 Launch - FrontierMath Tier 4: 39.6 (#1)
GPT-5.6 Sol on Design Arena (3D): 1426.0 (#2)
GPT-5.6 Sol on Design Arena (Data Viz): 1331.0 (#6)
GPT-5.6 Sol on Design Arena (Game Dev): 1440.0 (#1)
GPT-5.6 Sol on Design Arena (UI Components): 1371.0 (#2)
GPT-5.6 Sol on ParseBench: 62.12 (#25)
Gemini 3.7 Flash on Design Arena (3D): 1372.0 (#5)
Gemini 3.7 Flash on Design Arena (UI Components): 1314.0 (#13)
Gemini 3.7 Flash on Vals AI Code Migration: 34.8 (#17)
Gemini 3.7 Flash on Vals AI GPQA: 93.94 (#4)
Gemini 3.7 Flash on Vals AI LegalBench: 87.26 (#3)
Gemini 3.7 Flash on Vals AI LiveCodeBench: 88.65 (#3)
Gemini 3.7 Flash on Vals AI MMLU-Pro: 90.12 (#4)
Gemini 3.7 Flash on Vals AI MMMU: 88.96 (#3)
Gemini 3.7 Flash on Vals AI MedCode: 53.39 (#7)
Gemini 3.7 Flash on Vals AI MedScribe: 83.94 (#25)
Gemini 3.7 Flash on Vals AI MortgageTax: 66.65 (#32)
Gemini 3.7 Flash on Vals AI ProgramBench: 68.66 (#8)
Gemini 3.7 Flash on Vals AI SAGE: 49.23 (#20)
Gemini 3.7 Flash on Vals AI SWE-bench Verified: 80.8 (#19)
Gemini 3.7 Flash on Vals AI SkillsBench: 65.89 (#2)
Gemini 3.7 Flash on Vals AI TaxEval v2: 74.73 (#29)

New #1 Leaders (1)

Design Arena (Game Dev): GPT-5.6 Sol (xHigh) (1440.0) beat Kimi K3 by 8.0

The Aggregate Digest — 2026-08-16

2026-08-16T05:35:40.031129+00:00

Daily

Top-10 New Scores (17)

Claude Opus 5 on Agent Security League - Functional Correctness: 73.7 (#9)
GLM-5.3 on DeepsecBench: 22.0 (#6)
GPT-5.6 Sol on BabyVision: 88.9 (#3)
GPT-5.6 Sol on CADGenBench: 0.5319 (#2)
GPT-5.6 Sol on ProphetArena: 0.9506 (#4)
Gemini 3.7 Flash on Arabic Broad Leaderboard: 9.204 (#4)
Gemini 3.7 Flash on Chess Puzzles (Epoch AI): 47.0 (#11)
Gemini 3.7 Flash on Design Arena (Game Dev): 1310.0 (#16)
Gemini 3.7 Flash on Design Arena (Website): 1348.0 (#2)
Gemini 3.7 Flash on Epoch AI - Mystery Game Puzzles: 37.0 (#7)
Gemini 3.7 Flash on FrontierMath - Tier 4 (v2): 36.59 (#16)
Gemini 3.7 Flash on FrontierMath - Tiers 1-3 (v2): 71.58 (#14)
Gemini 3.7 Flash on Multi-turn Debate (Lechmazur): 1476.3 (#27)
Gemini 3.7 Flash on OTIS Mock AIME 2024-25: 97.22 (#19)
Gemini 3.7 Flash on ProgramBench: 0.0 (#9)
Gemini 3.7 Flash on ProgramBench Almost: 5.5 (#6)
Gemini 3.7 Flash on SimpleQA Verified: 71.2 (#4)

New #1 Leaders (4)

Coarena - Task Completion: Qwen 3.8 Max (100.0) beat GPT-5.6 Luna by 15.7
BabyVision: Qwen3.8-Max (with tools) (91.3) beat GPT-5.5 by 7.7
Agent Security League - Security Correctness: Claude Opus 5 (32.4) beat Claude Fable 5 by 3.4
CADGenBench: build123d-mcp-v0381-claude-opus-5-xhigh-full-r6 (0.6391) beat GPT-5.6 Sol (xHigh) by 0.11

Weekly

New Models (284)

Claude Opus 5 — ELO 1782, #1
- Computer Anthology Terminal Tasks (Terminus-2): 62.0 (#1/22)
- ProgramBench: 4.5 (#1/21)
- ProgramBench Almost: 37.0 (#1/21)
- Android Bench: 91.8 (#1/32)
- Design Arena (SVG): 1361.0 (#1/107)
- CHI-Bench: 54.7 (#1/26)
- NatureBench: 23.33 (#1/13)
- Creative Writing (Lechmazur): 4.2 (#1/44)
- GameCraft-Bench: 68.44 (#1/14)
- Vals AI (Vals Index): 67.21 (#1/46)
- ...and 25 more
Gemini 3.7 Flash — ELO 1782, #2
- AA MMMU-Pro: 85.49 (#1/240)
- AutomationBench: 30.4 (#1/10)
- LLM Stats (LVBench): 85.4 (#1/25)
- LLM Stats (MRCR v2 (8-needle)): 97.0 (#1/23)
- Coarena: 1039.4 (#2/16)
- Design Arena (Website): 1348.0 (#2/169)
- AA GPQA Diamond: 94.55 (#2/575)
- RuneBench: 10203.0 (#2/46)
- AI Chess Leaderboard (Reasoning): 1854.0 (#2/313)
- LLM Stats (Artificial Analysis): 56.0 (#2/7)
- ...and 82 more
GPT-5.5 Pro — ELO 1779, #3
- PM-LLM-Benchmark: 37.7 (#9/171)
Claude Fable 5 — ELO 1777, #4
- Computer Anthology Terminal Tasks (Claude Code): 54.0 (#1/4)
- FrontierCode: 63.6 (#1/22)
- APEX-Agents-AA: 59.2 (#1/28)
- Vals Index: 75.14 (#1/40)
- Vals Multimodal Index: 74.15 (#1/29)
- ArxivMath: 87.5 (#2/17)
- Computer Anthology Terminal Tasks (Terminus-2): 50.8 (#3/22)
- MedCode: 56.07 (#3/75)
- MedScribe: 88.52 (#3/74)
- ProofBench: 77.0 (#3/54)
- ...and 4 more
GPT-5.6 Sol — ELO 1772, #5
- Computer Anthology Terminal Tasks (Codex CLI): 58.0 (#1/4)
- LLM Chess (Saplin): 1549.7 (#1/158)
- GDP.pdf: 30.7 (#1/24)
- AIIQ Composite IQ: 136.0 (#1/115)
- Computer Anthology Terminal Tasks (Terminus-2): 51.6 (#2/22)
- CADGenBench: 0.5319 (#2/23)
- ProgramBench: 1.0 (#2/21)
- OpenRouter HLE (Search): 71.1 (#2/2)
- OpenRouter BrowseComp (Search): 82.4 (#2/5)
- OpenRouter DeepSearchQA (Search): 75.0 (#2/4)
- ...and 24 more
Kimi K3 — ELO 1756, #7
- Design Arena (Game Dev): 1432.0 (#1/155)
- DeepSearchQA: 95.0 (#1/15)
- Arena AI Code: 1674.0 (#2/48)
- LMArena WebDev Arena: 1674.48 (#2/25)
- Vals Index: 74.7 (#3/40)
- Vals Multimodal Index: 73.42 (#3/29)
- Android Bench: 90.2 (#4/32)
- Design Arena (SVG): 1339.0 (#4/107)
- BabyVision: 85.7 (#4/29)
- MCP Atlas: 84.2 (#4/39)
- ...and 22 more
Qwen 3.8 Max — ELO 1751, #8
- Coarena - Task Completion: 100.0 (#1/16)
- SpatialGen-Bench (Text Answering): 81.74 (#1/27)
- LLM Stats (WideSearch): 81.9 (#1/10)
- LLM Stats (Agents' Last Exam): 52.4 (#2/10)
- LLM Stats (MRCR v2 (8-needle)): 92.9 (#2/23)
- Vals AI IOI: 73.0 (#3/62)
- Arena AI Code: 1671.0 (#3/48)
- LMArena WebDev Arena: 1671.13 (#3/25)
- Vals AI GPQA: 93.69 (#4/132)
- AI for Education SEND: 88.07 (#4/225)
- ...and 43 more
Muse Spark 1.2 — ELO 1751, #9
- LMArena Text Arena: 1498.64 (#4/24)
- AI for Education Pedagogy - Maths: 94.44 (#5/233)
- AI for Education SEND: 88.07 (#5/225)
- FoodTruckBench: 41904.0 (#7/10)
- SimpleBench: 74.5 (#8/93)
- Vals AI IOI: 49.5 (#9/62)
- Coarena: 1001.7 (#10/16)
- RuneBench: 5677.0 (#10/46)
- Vals AI (Vals Index): 57.05 (#11/46)
- LLM Stats (DeepSWE 1.1): 59.3 (#11/25)
- ...and 22 more
Gemini 3.6 Flash — ELO 1748, #10
- ProgramBench: 0.5 (#5/21)
- Finance Agent v2: 56.3 (#5/40)
- Chatbot Arena (Vision): 1295.0 (#7/145)
- MedCode: 53.15 (#8/75)
- ProgramBench Almost: 4.0 (#9/21)
- Android Bench: 75.6 (#10/32)
- Vals Multimodal Index: 62.8 (#14/29)
- LMArena Text Arena: 1483.64 (#15/24)
- Vals Index: 62.43 (#15/40)
- Chatbot Arena (Text): 1484.0 (#16/391)
- ...and 11 more
GPT-5.5 — ELO 1747, #11
- HardMTBench: 91.4 (#1/22)
- PhysicianBench: 46.3 (#1/12)
- SWE Atlas: 45.4 (#1/15)
- ALE-Bench: 1942.97 (#1/89)
- LiveBench: 81.28 (#1/48)
- ClawProBench: 67.9 (#1/57)
- Computer Anthology Terminal Tasks (Codex CLI): 51.6 (#2/4)
- MathArena Apex: 80.21 (#2/50)
- ATLAS: 67.77 (#3/23)
- GeneBench: 25.0 (#3/16)
- ...and 22 more
Muse Spark 1.1 — ELO 1746, #12
- MCP Atlas: 88.1 (#1/39)
- MedScribe: 88.89 (#2/74)
- Finance Agent v2: 57.21 (#3/40)
- CharXiv-R: 88.4 (#5/58)
- OSWorld-Verified: 80.8 (#6/27)
- Vals Index: 68.41 (#8/40)
- DeepSearchQA: 84.9 (#10/15)
- LMArena Text Arena: 1488.67 (#10/24)
- Vals Multimodal Index: 66.74 (#10/29)
- Arena AI Document: 1472.0 (#12/38)
- ...and 3 more
Grok 4.6 — ELO 1746, #13
- CursorBench 3.1: 70.8 (#1/45)
- AA GPQA Diamond: 94.95 (#1/575)
- RuneBench: 12757.0 (#1/46)
- LLM Stats (APEX-Agents): 57.5 (#1/8)
- Benchmarks.bio - BioSecBench-Surveillance: 56.3 (#2/17)
- Benchmarks.bio - VariantBench: 44.63 (#2/14)
- Tau3 Banking: 50.72 (#2/166)
- SealedBench: 70.0 (#2/30)
- FrontierCode: 61.3 (#2/22)
- APEX-Agents-AA: 57.5 (#2/28)
- ...and 92 more
Gemini 3.1 Pro (Preview) — ELO 1738, #14
- AudioMC: 66.8 (#1/33)
- MMAU: 82.5 (#1/31)
- Computer Anthology Terminal Tasks (Gemini CLI): 19.6 (#2/2)
- MedCode: 59.06 (#2/75)
- MathArena Apex: 60.94 (#4/50)
- MirrorCode: 8.9 (#6/6)
- Finance Agent v1.1: 59.72 (#9/53)
- PhysicianBench: 6.0 (#11/12)
- OSWorld-Verified: 76.2 (#12/27)
- CaseLaw v2: 64.84 (#12/54)
- ...and 15 more
Qwen 3.7 Max — ELO 1735, #15
- IOI: 46.75 (#13/64)
- Finance Agent v2: 47.78 (#18/40)
- Vals Index: 57.49 (#18/40)
- Computer Anthology Terminal Tasks (Terminus-2): 9.8 (#19/22)
- ProofBench: 26.0 (#22/54)
- AIIQ Composite IQ: 118.0 (#26/115)
- Android Bench: 54.2 (#27/32)
- Vals AI Code Migration: 13.07 (#33/48)
- MedScribe: 79.4 (#35/74)
- MedCode: 38.75 (#44/75)
Gemini 3.5 Flash — ELO 1734, #16
- Computer Anthology Terminal Tasks (Gemini CLI): 24.4 (#1/2)
- Finance Agent v2: 57.86 (#2/40)
- MedCode: 55.83 (#5/75)
- ProgramBench Almost: 3.0 (#10/21)
- OSWorld-Verified: 78.4 (#10/27)
- ProgramBench: 0.0 (#11/21)
- Harvey Legal Agent Benchmark: 0.8 (#11/12)
- MathArena Apex: 32.29 (#11/50)
- Computer Anthology Terminal Tasks (Terminus-2): 22.8 (#12/22)
- Vals Multimodal Index: 62.93 (#13/29)
- ...and 8 more
Grok 4.5 — ELO 1733, #17
- FrontierCode: 56.6 (#4/22)
- APEX-Agents-AA: 47.1 (#4/28)
- MedScribe: 86.88 (#7/74)
- Vals Index: 65.3 (#11/40)
- Vals Multimodal Index: 63.42 (#12/29)
- Vals AI ProofBench: 31.0 (#16/24)
- AIIQ Composite IQ: 124.0 (#16/115)
- Finance Agent v2: 48.35 (#16/40)
- DeepsecBench: 15.58 (#17/35)
- ProofBench: 30.0 (#19/54)
- ...and 2 more
Claude Opus 4.8 — ELO 1731, #18
- Computer Anthology Terminal Tasks (Claude Code): 38.0 (#3/4)
- Agent Arena - Praise vs Complaint: 22.52 (#3/48)
- DeepSearchQA: 93.1 (#5/15)
- LiveBench: 77.55 (#5/48)
- Vals Index: 70.36 (#5/40)
- Vals Multimodal Index: 70.89 (#5/29)
- MCP Atlas: 83.6 (#7/39)
- ArxivMath: 65.0 (#7/17)
- MedCode: 53.22 (#7/75)
- ProofBench: 69.0 (#7/54)
- ...and 16 more
GPT-5.6 Terra — ELO 1728, #20
- ProofBench: 71.0 (#4/54)
- AutomationBench: 21.0 (#6/10)
- AIIQ Composite IQ: 132.0 (#6/115)
- Android Bench: 86.8 (#7/32)
- LLM Chess (Saplin): 1293.0 (#7/158)
- VoxelBench: 1936.0 (#8/44)
- Arena AI Document: 1479.0 (#10/38)
- IOI: 65.25 (#10/64)
- Finance Agent v2: 52.42 (#11/40)
- Vals Multimodal Index: 65.07 (#11/29)
- ...and 5 more
DeepSeek V4 Pro (0813) — ELO 1724, #21
- LLM Stats (NL2Repo): 61.5 (#1/17)
- LLM Stats (AutomationBench): 31.8 (#2/12)
- LLM Stats (CyberGym): 83.3 (#2/13)
- LLM Stats (Toolathlon): 74.1 (#2/37)
- Vals AI SWE-bench Verified: 96.4 (#2/82)
- NL2Repo: 61.5 (#2/21)
- CyberGym: 83.3 (#2/16)
- LLM Stats (Terminal-Bench 2.1): 87.9 (#4/28)
- LLM Stats (DeepSWE): 62.7 (#5/11)
- Vals AI Vibe Code Bench: 82.3 (#5/84)
- ...and 63 more
Gemini 3 Pro (Preview) — ELO 1718, #23
- OpenEval - GPQA CoT: 80.27 (#1/74)
- OpenEval - MMLU-Pro CoT: 90.3 (#1/77)
- MATH 500: 96.4 (#1/60)
- Vals AI MMLU-Pro: 90.1 (#4/132)
- OpenEval - Omni-MATH: 55.55 (#5/74)
- HMMT 2025: 97.5 (#5/82)
- MGSM: 93.93 (#7/74)
- OpenEval - IFEval Strict: 87.65 (#8/77)
- MedCode: 52.2 (#10/75)
- MathArena Apex: 23.44 (#15/50)
- ...and 10 more
Gemini 3 Pro — ELO 1718, #24
- FINAL Bench Metacognitive: 77.08 (#2/9)
- MageBench Season 1: 1722.0 (#3/34)
- MolViBench: 36.0 (#4/15)
- IDE-Bench: 55.0 (#8/15)
- LMArena Text Arena: 1485.45 (#14/24)
GPT-5.4 — ELO 1714, #25
- Sandboxed Coding Agents are Competitive Omni-m: 75.0 (#1/16)
- PlanningBench: 63.17 (#1/18)
- Vision2Code: 4.12 (#1/13)
- APEX-Agents: 52.7 (#2/45)
- LiveBench: 80.91 (#2/48)
- LLM Arena RU: 1187.0 (#3/104)
- Computer Anthology Terminal Tasks (Codex CLI): 38.0 (#3/4)
- FAM-Bench: 82.4 (#4/25)
- ATLAS: 63.23 (#4/23)
- PrivacySIM: 38.77 (#4/7)
- ...and 25 more
Claude Opus 4.7 — ELO 1714, #26
- Finance Agent v1.1: 64.37 (#1/53)
- MirrorCode: 31.1 (#2/6)
- Chatbot Arena (Vision): 1301.0 (#3/145)
- Chatbot Arena (Text): 1502.0 (#3/391)
- KnotBench: 51.65 (#3/4)
- PhysicianBench: 29.3 (#3/12)
- CaseLaw v2: 68.38 (#5/54)
- Chatbot Arena (Document): 1497.0 (#6/39)
- Arena AI Document: 1497.0 (#6/38)
- LiveBench: 77.1 (#6/48)
- ...and 22 more
GLM-5.2 — ELO 1709, #27
- Arena AI Code: 1588.0 (#7/48)
- LMArena WebDev Arena: 1588.2 (#7/25)
- GameCraft-Bench: 39.12 (#9/14)
- MCP Atlas: 82.6 (#9/39)
- Computer Anthology Terminal Tasks (Terminus-2): 31.4 (#10/22)
- ProphetArena: 0.9418 (#10/17)
- ArxivMath: 56.67 (#10/17)
- Vals Index: 65.02 (#13/40)
- Vals AI Excel Modeling: 61.53 (#15/46)
- Finance Agent v2: 49.7 (#15/40)
- ...and 6 more
Claude Sonnet 5 — ELO 1709, #28
- Computer Anthology Terminal Tasks (Claude Code): 25.6 (#4/4)
- Vals Index: 68.61 (#7/40)
- Vals Multimodal Index: 68.83 (#7/29)
- ProofBench: 66.0 (#8/54)
- Finance Agent v2: 53.91 (#9/40)
- Blueprint-Bench 2: 0.249 (#13/23)
- Vals AI MedCode: 47.54 (#22/84)
- LLM Chess (Saplin): 897.0 (#23/158)
- MedScribe: 76.05 (#49/74)
GPT-5.6 Luna — ELO 1707, #29
- OpenRouter DeepSearchQA (Search): 73.0 (#3/4)
- OpenRouter BrowseComp (Search): 74.0 (#4/5)
- Android Bench: 87.6 (#5/32)
- Finance Agent v2: 55.04 (#6/40)
- Vals Index: 69.88 (#6/40)
- Vals Multimodal Index: 69.06 (#6/29)
- IOI: 72.92 (#7/64)
- AIIQ Composite IQ: 129.0 (#9/115)
- VoxelBench: 1871.0 (#10/44)
- FoodTruckBench: 32932.0 (#10/10)
- ...and 8 more
Seed 2.1 Turbo — ELO 1707, #30
- SvelteBench: 88.9 (#70/169)
- LM Market Cap LMC Score: 40.0 (#241/419)
Claude Opus 4.6 — ELO 1706, #31
- VitaBench 2.0: 50.3 (#1/21)
- MageBench Season 1: 1747.0 (#1/34)
- Chatbot Arena (Text): 1505.0 (#2/391)
- PhysicianBench: 31.7 (#2/12)
- Chatbot Arena (Document): 1506.0 (#3/39)
- Arena AI Document: 1506.0 (#3/38)
- SWE-bench Verified: 75.6 (#4/47)
- Chatbot Arena (Vision): 1300.0 (#4/145)
- LMArena Text Arena: 1497.59 (#5/24)
- FINAL Bench Metacognitive: 76.17 (#5/9)
- ...and 16 more
Qwen 3.6 Max Preview — ELO 1705, #32
- Finance Agent v1.1: 52.78 (#24/53)
- ClawProBench: 57.4 (#26/57)
- CaseLaw v2: 47.91 (#51/54)
GPT-5.2 Codex — ELO 1705, #33
- APEX v1 Consulting: 66.9 (#1/18)
- APEX v1: 65.3 (#4/5)
- APEX v1 Investment Banking: 61.8 (#5/18)
- Vals AI LiveCodeBench: 87.99 (#6/137)
- SWE-bench Verified: 72.8 (#7/47)
- APEX v1 Big Law: 73.0 (#8/18)
- ALE-Bench: 1299.9 (#8/89)
- APEX v1 Medicine (MD): 59.7 (#9/18)
Gemini 3 Flash (Preview) — ELO 1704, #34
- MedCode: 55.92 (#4/75)
- MGSM: 93.31 (#10/74)
- IOI: 39.08 (#15/64)
- Vals Multimodal Index: 52.19 (#22/29)
- Finance Agent v2: 42.55 (#26/40)
- Vals Index: 49.55 (#28/40)
- Vals AI Legal Research Bench: 18.27 (#30/49)
- Epoch AI - Proofbench: 15.0 (#32/52)
- CaseLaw v2: 55.84 (#33/54)
- Finance Agent v1.1: 47.6 (#33/53)
- ...and 5 more
GPT-5.3 Codex — ELO 1703, #35
- APEX v1 Investment Banking: 65.0 (#1/18)
- SmellBench: 44.4 (#2/10)
- ALE-Bench: 1655.22 (#2/89)
- APEX v1 Consulting: 65.5 (#4/18)
- MageBench Season 1: 1717.0 (#4/34)
- APEX v1 Big Law: 69.8 (#12/18)
- APEX v1 Medicine (MD): 58.1 (#14/18)
- IOI: 43.83 (#14/64)
- AIIQ Composite IQ: 119.0 (#23/115)
- LiveBench: 71.97 (#31/48)
Kimi K2.7 Code — ELO 1702, #37
- Computer Anthology Terminal Tasks (Terminus-2): 15.4 (#15/22)
Gemini 3 Flash — ELO 1701, #38
- SWE-bench Verified: 75.8 (#2/47)
- OCR-Robust: 79.05 (#3/18)
- HMMT 2025: 97.5 (#7/82)
- MageBench Season 1: 1622.0 (#11/34)
- MathArena Apex: 15.62 (#18/50)
- InfoOps Bench: 20.1 (#33/37)
- AIIQ Composite IQ: 116.0 (#33/115)
- OmniDocBench 1.5: 0.12 (#65/65)
Qwen 3.7 Plus — ELO 1698, #40
- Vals Multimodal Index: 53.89 (#20/29)
- Vals Index: 52.33 (#24/40)
- Android Bench: 57.7 (#25/32)
- AIIQ Composite IQ: 116.0 (#29/115)
- Finance Agent v2: 38.22 (#29/40)
- Vals AI Legal Research Bench: 16.35 (#31/49)
- Vals AI Code Migration: 12.86 (#35/48)
DeepSeek V4 Flash (0731) — ELO 1694, #41
- OpenRouter BrowseComp (Search): 77.0 (#3/5)
- OpenRouter DeepSearchQA (Search): 72.0 (#4/4)
- GameCraft-Bench: 40.61 (#7/14)
- Vals AI ProofBench: 56.0 (#11/24)
- Vals AI Code Migration: 38.63 (#12/48)
- Epoch AI - Mystery Game Puzzles: 34.0 (#14/53)
- RuneBench: 2816.0 (#20/46)
- SvelteBench: 95.1 (#33/169)
- EQ-Bench Longform Writing: 61.2 (#51/126)
GPT-5 — ELO 1691, #42
- APEX v1 Big Law: 76.6 (#1/18)
- LingOly-TOO: 46.7 (#1/16)
- MCP-Universe: 43.72 (#2/28)
- KnotBench: 43.0 (#4/4)
- APEX v1 Medicine (MD): 62.4 (#5/18)
- VitaBench 2.0: 44.1 (#6/21)
- VTB: 16.96 (#6/20)
- APEX v1 Consulting: 60.0 (#8/18)
- APEX v1 Investment Banking: 59.1 (#9/18)
- MageBench Season 1: 1536.0 (#29/34)
- ...and 3 more
Claude Opus 4.5 — ELO 1691, #43
- SWE-bench Verified: 76.8 (#1/47)
- IDE-Bench: 83.75 (#3/15)
Nex N2 Pro — ELO 1691, #44
- AIIQ Composite IQ: 109.0 (#46/115)
GPT-5.2 — ELO 1690, #46
- MolDeTox: 15.59 (#1/18)
- IDE-Bench: 85.0 (#2/15)
- MageBench Season 1: 1737.0 (#2/34)
- FINAL Bench Metacognitive: 76.5 (#3/9)
- APEX v1 Investment Banking: 62.3 (#4/18)
- APEX-Agents: 48.4 (#4/45)
- APEX v1 Big Law: 73.4 (#6/18)
- APEX v1 Consulting: 61.4 (#6/18)
- APEX v1 Medicine (MD): 61.9 (#6/18)
- MGSM: 94.0 (#6/74)
- ...and 16 more
Kimi K2.6 — ELO 1690, #47
- VitaBench 2.0: 48.1 (#2/21)
- PhysicianBench: 17.0 (#7/12)
- Finance Agent v1.1: 57.06 (#14/53)
- ClawProBench: 59.31 (#17/57)
- Vals Multimodal Index: 56.43 (#17/29)
- Vals Index: 55.17 (#20/40)
- CaseLaw v2: 61.2 (#22/54)
- Finance Agent v2: 44.9 (#22/40)
- Inverse Turing Bench: 57.45 (#26/40)
- Mercor APEX: 18.9 (#29/53)
- ...and 3 more
Hy3 — ELO 1689, #48
- Computer Anthology Terminal Tasks (Terminus-2): 18.0 (#13/22)
- AIIQ Composite IQ: 103.0 (#65/115)
Claude Opus 4.5 (20251101) — ELO 1688, #49
- MGSM: 95.2 (#1/74)
- SWE-bench Verified: 74.4 (#5/47)
- Epoch AI - Proofbench: 36.0 (#15/52)
- MultiNRC: 41.23 (#17/43)
- TutorBench: 49.82 (#18/27)
- CaseLaw v2: 62.59 (#18/54)
- IOI: 23.58 (#21/64)
- EnigmaEval: 4.65 (#22/43)
- MedCode: 45.17 (#24/75)
- MedScribe: 83.25 (#24/74)
- ...and 3 more
GLM-5.1 — ELO 1686, #50
- VitaBench 2.0: 45.0 (#5/21)
- ArxivMath: 52.5 (#12/17)
- Finance Agent v1.1: 57.66 (#12/53)
- MathArena Apex: 11.5 (#21/50)
- Vals Index: 52.45 (#22/40)
- Finance Agent v2: 44.79 (#23/40)
- ProofBench: 22.22 (#24/54)
- MedCode: 41.6 (#32/75)
- CaseLaw v2: 51.55 (#48/54)
- MedScribe: 72.27 (#59/74)
Inkling — ELO 1686, #51
- AudioMC: 56.6 (#2/33)
- MMAU: 77.2 (#2/31)
- MedScribe: 85.41 (#13/74)
- Coarena: 998.6 (#14/16)
- Coarena - Task Completion: 40.0 (#14/16)
- Finance Agent v2: 46.6 (#20/40)
- MCP Atlas: 74.1 (#21/39)
- Vals Index: 49.28 (#29/40)
- DeepsecBench: 6.32 (#30/35)
- CharXiv-R: 78.1 (#33/58)
- ...and 8 more
DeepSeek V4 Pro — ELO 1684, #52
- MathArena Apex: 90.2 (#1/50)
- VitaBench 2.0: 47.2 (#4/21)
- Finance Agent v1.1: 60.39 (#5/53)
- DeepSWE: 62.8 (#12/27)
- NYT Connections Extended: 88.4 (#17/104)
- Creative Writing (Lechmazur): 0.8 (#17/44)
- IOI: 35.83 (#17/64)
- Computer Anthology Terminal Tasks (Terminus-2): 10.4 (#18/22)
- Vals Index: 55.62 (#19/40)
- Finance Agent v2: 44.08 (#24/40)
- ...and 6 more
Seed 2.0 Pro — ELO 1684, #53
- VitaBench 2.0: 47.4 (#3/21)
- Creative Writing (Lechmazur): -1.5 (#33/44)
Qwen 3.5 Plus — ELO 1683, #54
- ClawProBench: 64.19 (#3/57)
- Finance Agent v1.1: 54.48 (#19/53)
- CaseLaw v2: 59.7 (#26/54)
GPT-5 Codex — ELO 1682, #55
- APEX v1 Big Law: 73.5 (#5/18)
- APEX v1 Consulting: 60.3 (#7/18)
- APEX v1 Investment Banking: 59.8 (#7/18)
- APEX v1 Medicine (MD): 57.6 (#16/18)
- Lean AI Formalization Leaderboard: 1.0 (#30/56)
- IOI: 9.75 (#38/64)
GPT-5 (2025-08-07) — ELO 1681, #56
- MATH 500: 96.0 (#3/60)
- CaseLaw v2: 66.45 (#6/54)
- MedCode: 49.63 (#14/75)
- MGSM: 92.84 (#14/74)
- SWE-bench Verified: 65.0 (#21/47)
- MedScribe: 83.65 (#22/74)
- IOI: 20.0 (#26/64)
- Finance Agent v1.1: 52.15 (#27/53)
- ProofBench: 18.0 (#29/54)
Qwen 3.5 397B A17B — ELO 1680, #57
- HarmActionsEval: 23.4 (#1/10)
- ClawProBench: 64.18 (#4/57)
Qwen 3.6 Plus — ELO 1679, #58
- PhysicianBench: 13.7 (#9/12)
- ClawProBench: 60.2 (#14/57)
- Finance Agent v1.1: 54.63 (#17/53)
- Vals Multimodal Index: 51.52 (#23/29)
- Finance Agent v2: 40.85 (#28/40)
- Vals Index: 48.89 (#30/40)
- Vals AI Harvey Legal Agent Bench: 1.25 (#31/50)
- Vals AI Excel Modeling: 32.87 (#35/46)
- Vals AI Legal Research Bench: 14.9 (#37/49)
- MedScribe: 76.96 (#44/74)
- ...and 3 more
MiMo-V2.5-Pro — ELO 1679, #59
- ClawProBench: 63.3 (#5/57)
- PhysicianBench: 16.7 (#8/12)
- MedScribe: 83.73 (#21/74)
- Android Bench: 60.8 (#23/32)
- Vals AI Harvey Legal Agent Bench: 2.08 (#23/50)
- ProofBench: 24.0 (#23/54)
- Finance Agent v2: 41.5 (#27/40)
- Vals Index: 50.74 (#27/40)
- Vals AI Legal Research Bench: 15.87 (#32/49)
- MedCode: 32.48 (#62/75)
MiniMax-M3 — ELO 1678, #60
- Android Bench: 63.6 (#20/32)
- Chess Puzzles (Epoch AI): 14.0 (#102/161)
- OTIS Mock AIME 2024-25: 71.11 (#116/238)
Claude Sonnet 4.6 — ELO 1677, #61
- Finance Agent v1.1: 63.33 (#2/53)
- HarmActionsEval: 2.84 (#4/10)
- PhysicianBench: 23.0 (#5/12)
- ProphetArena: 0.9437 (#8/17)
- GDP.pdf: 18.0 (#10/24)
- ClawProBench: 60.5 (#11/57)
- CaseLaw v2: 63.99 (#14/54)
- Finance Agent v2: 51.03 (#14/40)
- ProofBench: 45.0 (#14/54)
- Vals Multimodal Index: 60.57 (#15/29)
- ...and 2 more
O3 — ELO 1675, #63
- VitaBench 2.0: 40.3 (#8/21)
- MageBench Season 1: 1609.0 (#13/34)
- HMMT 2025: 77.5 (#45/82)
GPT-5.1 Codex — ELO 1675, #64
- APEX v1 Big Law: 73.2 (#7/18)
- APEX v1 Investment Banking: 58.7 (#10/18)
- APEX v1 Consulting: 57.6 (#11/18)
- APEX v1 Medicine (MD): 58.1 (#13/18)
- SWE-bench Verified: 66.0 (#19/47)
- Vals AI LiveCodeBench: 85.55 (#28/137)
- Design Arena (UI Components): 1265.0 (#38/148)
- IOI: 3.67 (#54/64)
GPT-5.3 — ELO 1673, #65
- HarmActionsEval: 12.77 (#2/10)
Inkling Small — ELO 1673, #66
- LLM Stats (AIME 2026): 95.5 (#4/21)
- LLM Stats (Artificial Analysis): 40.0 (#7/7)
- LLM Stats (MCP Atlas): 79.6 (#7/33)
- LLM Stats (Toolathlon): 54.4 (#12/37)
- OpenRouter GPQA Diamond: 87.2 (#21/114)
- LLM Stats (Terminal-Bench 2.1): 64.7 (#22/28)
- LLM Stats (BrowseComp): 77.4 (#23/62)
- ZeroEval GPQA Diamond: 89.5 (#26/239)
- LLM Stats (CharXiv-R): 77.4 (#30/51)
- FrontierMath - Tier 4 (v2): 17.07 (#32/50)
- ...and 17 more
Claude Sonnet 4.5 — ELO 1670, #67
- IDE-Bench: 87.5 (#1/15)
- OpenEval - MMLU-Pro CoT: 86.9 (#3/77)
- MGSM: 94.33 (#4/74)
- OpenEval - GPQA CoT: 68.61 (#6/74)
- OpenEval - Omni-MATH: 55.3 (#6/74)
- VitaBench 2.0: 41.7 (#7/21)
- SWE-bench Verified: 71.4 (#11/47)
- OR-Space: 53.0 (#11/19)
- VTB: 5.6 (#11/20)
- DentalGPT Dental Eval Suite: 51.7 (#12/17)
- ...and 18 more
Grok 4.3 — ELO 1670, #68
- CaseLaw v2: 79.31 (#1/54)
- Finance Agent v1.1: 53.81 (#20/53)
- Vals Multimodal Index: 43.29 (#25/29)
- Chess Bench LLM: 956.0 (#29/110)
- Finance Agent v2: 37.73 (#31/40)
- IOI: 15.33 (#31/64)
- Vals Index: 46.48 (#32/40)
- Vals AI LiveCodeBench: 84.49 (#36/137)
- ProofBench: 11.0 (#38/54)
- Vals AI MMLU-Pro: 85.84 (#48/132)
- ...and 2 more
Gemini 3.5 Flash Lite — ELO 1669, #69
- Agent Arena - Tool Hallucination: 0.28 (#3/48)
- Agent Arena - Confirmed Success: 12.73 (#5/48)
- Agent Arena - Steerability: 10.28 (#7/48)
- Agent Arena - Bash Recovery: 15.08 (#7/48)
- Agent Arena: 10.38 (#10/48)
- Agent Arena - Praise vs Complaint: 13.51 (#15/48)
- Finance Agent v2: 47.44 (#19/40)
- IOI: 26.17 (#20/64)
- Vals Index: 51.0 (#26/40)
- MedCode: 43.49 (#27/75)
- ...and 8 more
Gemini 2.5 Pro — ELO 1668, #70
- LingOly-TOO: 42.3539 (#4/16)
- USAMO25: 24.4 (#5/13)
- MedCode: 50.59 (#12/75)
- VitaBench 2.0: 33.1 (#15/21)
- CaseLaw v2: 63.88 (#15/54)
- MageBench Season 1: 1540.0 (#28/34)
- IOI: 17.08 (#29/64)
- APEX-Agents: 17.0 (#37/45)
- Finance Agent v1.1: 41.59 (#42/53)
- HMMT 2025: 82.5 (#43/82)
- ...and 2 more
GPT-5.1 — ELO 1668, #71
- CaseLaw v2: 73.42 (#2/54)
- MedScribe: 88.09 (#4/74)
- MedCode: 52.73 (#9/75)
- Mirror, Mirror on the Wall: 52.0 (#10/44)
- MGSM: 92.98 (#12/74)
- Inverse Turing Bench: 67.68 (#15/40)
- Finance Agent v1.1: 55.31 (#15/53)
- HMMT 2025: 93.33 (#22/82)
- IOI: 21.5 (#23/64)
- MathArena Apex: 1.04 (#39/50)
- ...and 1 more
GPT-5.4 Mini — ELO 1668, #72
- Computer Anthology Terminal Tasks (Codex CLI): 21.8 (#4/4)
- Vision2Code: 3.85 (#4/13)
- HarmActionsEval: 0.71 (#7/10)
- APEX-Agents: 37.5 (#17/45)
- Vals Multimodal Index: 54.22 (#19/29)
- Computer Anthology Terminal Tasks (Terminus-2): 4.0 (#21/22)
- Finance Agent v2: 45.36 (#21/40)
- Finance Agent v1.1: 53.41 (#22/53)
- Vals Index: 52.42 (#23/40)
- ProofBench: 21.0 (#25/54)
- ...and 8 more
Grok 4.20 0309 — ELO 1668, #73
- IOI: 30.17 (#18/64)
- Finance Agent v1.1: 52.3 (#26/53)
- Vals Multimodal Index: 39.06 (#28/29)
- ProofBench: 14.0 (#35/54)
- Vals Index: 39.5 (#36/40)
- Finance Agent v2: 28.49 (#37/40)
- CaseLaw v2: 54.45 (#38/54)
- Vals AI Legal Research Bench: 13.94 (#39/49)
- CritPt: 6.0 (#41/213)
- Vals AI Excel Modeling: 11.74 (#44/46)
- ...and 4 more
O3 (2025-04-16) — ELO 1667, #75
- MATH 500: 94.6 (#8/60)
- MedCode: 47.29 (#21/75)
- MGSM: 91.75 (#26/74)
- TextClass Benchmark: 1625.36 (#30/108)
- MedScribe: 76.65 (#45/74)
DeepSeek V4 Flash — ELO 1666, #76
- IMO-AnswerBench: 91.1 (#2/23)
- Arena AI Code: 1585.0 (#8/48)
- LMArena WebDev Arena: 1584.86 (#8/25)
- AutomationBench: 18.1 (#10/10)
- DeepsecBench: 16.54 (#12/35)
- AI for Education SEND: 83.03 (#25/225)
- ALE-Bench: 324.98 (#79/89)
Muse Glimmer 30B — ELO 1665, #77
- CPTU Bench: 4.42 (#1/102)
- AA-LCR: 80.0 (#1/217)
- LLM Stats (ScreenSpot Pro): 75.4 (#6/25)
- LLM Stats (SkillsBench): 44.3 (#7/8)
- ScreenSpot-Pro: 75.4 (#7/35)
- LLM Stats (AIME 2026): 94.7 (#8/21)
- WildClawBench: 47.6 (#8/21)
- Coarena: 1002.6 (#9/16)
- LLM Stats (DeepSearchQA): 74.6 (#9/9)
- Coarena - Task Completion: 50.0 (#12/16)
- ...and 30 more
Kimi K2.5 — ELO 1663, #79
- FINAL Bench Metacognitive: 78.54 (#1/9)
- MageBench Season 1: 1652.0 (#8/34)
- SWE-bench Verified: 70.8 (#12/47)
- ClawProBench: 58.49 (#21/57)
- MathArena Apex: 8.85 (#25/50)
- CaseLaw v2: 58.73 (#28/54)
- IOI: 17.67 (#28/64)
- Finance Agent v1.1: 50.62 (#30/53)
- Vals AI Legal Research Bench: 15.87 (#33/49)
- Vals AI Excel Modeling: 28.47 (#38/46)
- ...and 7 more
Grok 4.20 — ELO 1663, #80
- PhysicianBench: 5.3 (#12/12)
- ClawProBench: 43.04 (#50/57)
GLM-5 — ELO 1662, #81
- FINAL Bench Metacognitive: 76.38 (#4/9)
- HMMT 2025: 97.5 (#6/82)
- SWE-bench Verified: 72.8 (#8/47)
- APEX v1 Investment Banking: 52.9 (#14/18)
- APEX v1 Consulting: 45.3 (#17/18)
- APEX v1 Medicine (MD): 47.8 (#17/18)
- APEX v1 Big Law: 50.0 (#18/18)
- MathArena Apex: 10.94 (#22/50)
- IOI: 22.0 (#22/64)
- Finance Agent v1.1: 53.18 (#23/53)
- ...and 2 more
Gemini 2.5 Pro (Preview 03-25) — ELO 1660, #82
- OpenEval - GPQA CoT: 74.89 (#2/74)
- OpenEval - MMLU-Pro CoT: 86.3 (#4/77)
- MATH 500: 95.2 (#5/60)
- OpenEval - Omni-MATH: 41.6 (#15/74)
- OpenEval - IFEval Strict: 83.98 (#20/77)
- TextClass Benchmark: 1517.98 (#61/108)
MiMo-V2-Pro — ELO 1659, #83
- ClawProBench: 57.92 (#22/57)
Qwen 3.6 27B — ELO 1658, #84
- CPTU Bench: 4.41 (#2/102)
- ClawProBench: 58.74 (#20/57)
- Android Bench: 45.1 (#29/32)
- CaseLaw v2: 53.16 (#41/54)
- AIIQ Composite IQ: 111.0 (#42/115)
- ProofBench: 8.0 (#44/54)
Gemini 3.1 Flash Lite — ELO 1657, #85
- HarmActionsEval: 0.71 (#6/10)
- MageBench Season 1: 1578.0 (#23/34)
- AIIQ Composite IQ: 101.0 (#75/115)
GPT-5 Mini (2025-08-07) — ELO 1657, #86
- SWE-bench Verified: 59.8 (#26/47)
Claude Opus 4.1 — ELO 1657, #87
- LingOly-TOO: 45.8 (#2/16)
- Inverse Turing Bench: 60.14 (#22/40)
DeepSeek V3.2 Speciale — ELO 1657, #88
- HMMT 2025: 99.2 (#3/82)
Claude Opus 4.1 (20250805) — ELO 1657, #89
- MGSM: 94.44 (#3/74)
- MATH 500: 95.4 (#4/60)
- VTB: 4.71 (#14/20)
- EnigmaEval: 4.81 (#21/43)
- MultiNRC: 29.67 (#22/43)
- MedCode: 47.23 (#22/75)
- Visual-Language Understanding: 45.25 (#29/58)
- IOI: 12.52 (#35/64)
- MedScribe: 73.9 (#53/74)
Step 3.7 Flash — ELO 1656, #90
- ArxivMath: 46.67 (#14/17)
- AIIQ Composite IQ: 101.0 (#71/115)
Solar Pro 4 — ELO 1650, #93
- Agent Arena - Steerability: 13.32 (#3/48)
- LLM Stats (AIME 2026): 95.3 (#5/21)
- Agent Arena - Tool Hallucination: 0.51 (#7/48)
- Agent Arena - Praise vs Complaint: 15.67 (#9/48)
- Agent Arena: 10.1 (#11/48)
- Agent Arena - Bash Recovery: 13.56 (#12/48)
- Agent Arena - Confirmed Success: 8.44 (#17/48)
- LLM Stats (Terminal-Bench 2.1): 57.0 (#24/28)
- ZeroEval GPQA Diamond: 89.0 (#29/239)
- AA GDPval: 1275.76 (#48/198)
- ...and 32 more
GPT-5 Mini — ELO 1649, #94
- Agentick: 30.9 (#3/11)
- CaseLaw v2: 68.49 (#4/54)
- MATH 500: 94.8 (#6/60)
- MGSM: 92.58 (#15/74)
- Finance Agent v1.1: 51.93 (#28/53)
- SWE-bench Verified: 56.2 (#29/47)
- MageBench Season 1: 1516.0 (#30/34)
- MedCode: 43.05 (#30/75)
- MedScribe: 80.58 (#31/74)
- IOI: 6.75 (#42/64)
- ...and 2 more
Qwen 3.5 122B A10B — ELO 1649, #95
- CPTU Bench: 4.36 (#3/102)
Claude Opus 4 — ELO 1648, #98
- MGSM: 93.78 (#8/74)
- MATH 500: 90.4 (#25/60)
- HMMT 2025: 60.0 (#60/82)
- AA-LCR: 33.7 (#190/217)
Gemma 4 31B (IT) — ELO 1647, #99
- CPTU Bench: 4.32 (#5/102)
- Finance Agent v1.1: 50.79 (#29/53)
- Android Bench: 37.1 (#31/32)
- ClawProBench: 51.59 (#39/57)
- CaseLaw v2: 52.63 (#44/54)
- ForecastBench: 61.4 (#166/268)
MiMo-V2.5 — ELO 1646, #101
- ClawProBench: 60.39 (#13/57)
- Vals Multimodal Index: 52.77 (#21/29)
- Vals AI Excel Modeling: 55.09 (#24/46)
- Vals AI Harvey Legal Agent Bench: 1.67 (#25/50)
- Vals Index: 51.57 (#25/40)
- ProofBench: 16.0 (#31/54)
- Finance Agent v2: 36.73 (#33/40)
- Vals AI Legal Research Bench: 9.13 (#43/49)
- AIIQ Composite IQ: 109.0 (#48/115)
- MedScribe: 72.15 (#60/74)
- ...and 1 more
GLM-4.7 — ELO 1645, #102
- MageBench Season 1: 1675.0 (#6/34)
- APEX v1 Consulting: 55.7 (#12/18)
- APEX v1 Investment Banking: 53.9 (#12/18)
- APEX v1 Big Law: 51.6 (#17/18)
- APEX v1 Medicine (MD): 46.7 (#18/18)
- CaseLaw v2: 54.88 (#37/54)
- Finance Agent v1.1: 45.98 (#38/53)
- IOI: 7.58 (#41/64)
- ProofBench: 6.0 (#45/54)
- MGSM: 88.18 (#53/74)
- ...and 2 more
Qwen 3 Max — ELO 1645, #103
- VitaBench 2.0: 28.4 (#17/21)
- MageBench Season 1: 1594.0 (#18/34)
- MGSM: 91.82 (#25/74)
- IOI: 15.67 (#30/64)
- Finance Agent v1.1: 44.3 (#41/53)
- CaseLaw v2: 47.48 (#52/54)
Grok 4 — ELO 1644, #104
- MATH 500: 96.2 (#2/60)
- CaseLaw v2: 65.81 (#9/54)
- HMMT 2025: 95.0 (#13/82)
- IOI: 26.17 (#19/64)
- Finance Agent v1.1: 53.51 (#21/53)
- MGSM: 90.91 (#32/74)
- MathArena Apex: 2.08 (#33/50)
- MageBench Season 1: 1459.0 (#34/34)
- MedScribe: 78.15 (#38/74)
- MedCode: 38.08 (#47/75)
Gemini 3.1 Flash Lite (Preview) — ELO 1644, #105
- MedCode: 47.6 (#20/75)
- Vals Multimodal Index: 41.35 (#27/29)
- CaseLaw v2: 54.98 (#35/54)
- Finance Agent v1.1: 46.12 (#35/53)
- Finance Agent v2: 29.99 (#36/40)
- Vals AI Harvey Legal Agent Bench: 0.0 (#37/50)
- Vals Index: 36.2 (#38/40)
- Vals AI MMLU-Pro: 86.24 (#41/132)
- Vals AI Excel Modeling: 8.63 (#46/46)
- Vals AI Legal Research Bench: 3.37 (#46/49)
- ...and 2 more
Kimi K2 — ELO 1642, #107
- CaseLaw v2: 65.7 (#10/54)
- MATH 500: 94.2 (#10/60)
- HMMT 2025: 93.33 (#23/82)
- MGSM: 90.95 (#31/74)
- Finance Agent v1.1: 36.65 (#44/53)
- ClawProBench: 47.83 (#46/57)
- Mercor APEX: 4.1 (#48/53)
- MathArena Apex: 0.0 (#50/50)
- IOI: 1.25 (#61/64)
Claude Sonnet 4 — ELO 1641, #108
- WildAgtEval: 63.6 (#2/10)
- MGSM: 93.02 (#11/74)
- MATH 500: 90.32 (#26/60)
- IOI: 6.5 (#44/64)
- MedCode: 33.94 (#56/75)
- MedScribe: 72.41 (#58/74)
- CritPt: 0.3 (#146/213)
- AA-LCR: 44.3 (#162/217)
Nemotron 3 Ultra — ELO 1641, #109
- Finance Agent v2: 37.67 (#32/40)
- Vals Index: 43.99 (#33/40)
- Vals AI Harvey Legal Agent Bench: 0.42 (#35/50)
- Vals AI Legal Research Bench: 15.38 (#36/49)
- Vals AI Excel Modeling: 31.73 (#37/46)
- Mercor APEX: 11.5 (#40/53)
- Vals AI Code Migration: 4.91 (#45/48)
- MedCode: 38.62 (#45/75)
- ProofBench: 2.0 (#52/54)
- AIIQ Composite IQ: 107.0 (#53/115)
O1 (2024-12-17) — ELO 1641, #110
- TextClass Benchmark: 1768.81 (#4/108)
- MATH 500: 90.4 (#24/60)
- MGSM: 89.31 (#47/74)
LongCat 2.0 — ELO 1641, #111
- SvelteBench: 88.9 (#73/169)
Claude Opus 4 (20250514) — ELO 1640, #112
- OpenEval - MMLU-Pro CoT: 87.5 (#2/77)
- OpenEval - Omni-MATH: 61.6 (#2/74)
- OpenEval - IFEval Strict: 91.77 (#3/77)
- OpenEval - GPQA CoT: 70.85 (#4/74)
- HELM Safety: 96.7675 (#8/59)
- TutorBench: 45.46 (#25/27)
Qwen 3.5 27B — ELO 1639, #114
- MathArena Apex: 2.08 (#34/50)
Qwen 3 Max (Preview) — ELO 1639, #115
- MGSM: 92.15 (#23/74)
- IOI: 7.75 (#39/64)
Step 3.5 Flash — ELO 1637, #117
- HMMT 2025: 98.33 (#4/82)
- MathArena Apex: 13.54 (#20/50)
- ClawProBench: 41.75 (#53/57)
Grok 4.1 Fast — ELO 1636, #118
- IDE-Bench: 35.0 (#9/15)
- MageBench Season 1: 1637.0 (#10/34)
- CaseLaw v2: 60.45 (#24/54)
- Finance Agent v1.1: 52.45 (#25/53)
- MathArena Apex: 5.21 (#29/50)
- HMMT 2025: 90.0 (#33/82)
- MedScribe: 78.73 (#36/74)
- IOI: 7.67 (#40/64)
- MGSM: 89.53 (#45/74)
- ProofBench: 4.0 (#48/54)
- ...and 1 more
O4 Mini (2025-04-16) — ELO 1636, #119
- MGSM: 93.42 (#9/74)
- MATH 500: 94.2 (#11/60)
- IOI: 4.83 (#48/64)
- TextClass Benchmark: 1538.33 (#55/108)
- MedCode: 33.79 (#57/75)
- MedScribe: 69.14 (#67/74)
DeepSeek V3.2 — ELO 1635, #120
- SWE-bench Verified: 70.0 (#14/47)
- ClawProBench: 60.13 (#15/57)
- Vals AI MMLU-Pro: 84.92 (#56/132)
- Vals AI LiveCodeBench: 80.69 (#62/137)
Claude 3.7 Sonnet (20250219) — ELO 1635, #121
- OpenEval - IFEval Strict: 94.0 (#1/77)
- OpenEval - MMLU-Pro CoT: 86.0 (#5/77)
- OpenEval - GPQA CoT: 66.0 (#10/74)
- MGSM: 92.98 (#13/74)
- OpenEval - Omni-MATH: 36.67 (#22/74)
- MATH 500: 91.6 (#22/60)
- TextClass Benchmark: 1500.76 (#65/108)
Ring-2.6-1T — ELO 1634, #123
- AIIQ Composite IQ: 101.0 (#72/115)
MiniMax-M2.5 — ELO 1632, #125
- SWE-bench Verified: 75.8 (#3/47)
Qwen 3.6 35B A3B — ELO 1632, #126
- ArxivMath: 37.5 (#16/17)
- ClawProBench: 56.94 (#29/57)
- Android Bench: 37.0 (#32/32)
- AIIQ Composite IQ: 100.0 (#77/115)
Grok 4 Fast — ELO 1632, #127
- CaseLaw v2: 65.7 (#11/54)
- MageBench Season 1: 1620.0 (#12/34)
- HMMT 2025: 91.67 (#28/82)
- MedScribe: 81.63 (#29/74)
- MathArena Apex: 5.21 (#30/50)
- MGSM: 90.87 (#34/74)
- IOI: 11.5 (#36/64)
- Finance Agent v1.1: 46.08 (#37/53)
- MedCode: 37.38 (#49/75)
GLM-4.6 — ELO 1631, #128
- VitaBench 2.0: 35.9 (#13/21)
- HMMT 2025: 93.33 (#24/82)
- SWE-bench Verified: 55.4 (#31/47)
- MGSM: 89.75 (#44/74)
- Finance Agent v1.1: 36.48 (#45/53)
- MathArena Apex: 0.52 (#47/50)
- IOI: 4.33 (#50/64)
KAT Coder Pro V2 — ELO 1631, #129
- ClawProBench: 54.74 (#34/57)
O1 — ELO 1629, #130
- HMMT 2025: 48.33 (#62/82)
DeepSeek V3.2 Exp — ELO 1628, #131
- CritPt: 1.4 (#83/213)
MiniMax-M2.7 — ELO 1628, #132
- VitaBench 2.0: 34.9 (#14/21)
- Vals AI Excel Modeling: 28.45 (#39/46)
- Vals AI Harvey Legal Agent Bench: 0.0 (#42/50)
- Vals AI Legal Research Bench: 10.58 (#42/49)
GPT-4.1 (2025-04-14) — ELO 1626, #134
- SWE-bench Verified: 39.6 (#38/47)
- TextClass Benchmark: 1520.39 (#60/108)
O4 Mini — ELO 1625, #136
- USAMO25: 19.05 (#6/13)
- VitaBench 2.0: 21.0 (#19/21)
- HMMT 2025: 83.33 (#41/82)
Qwen 3 235B A22B 2507 — ELO 1624, #139
- WritingBench: 88.3 (#1/67)
- MathArena Apex: 5.21 (#28/50)
- Creative Writing v3: 86.1 (#114/124)
MiMo-V2-Omni — ELO 1624, #140
- ClawProBench: 57.65 (#23/57)
Gemini 2.5 Flash — ELO 1622, #141
- VitaBench 2.0: 28.2 (#18/21)
- MageBench Season 1: 1572.0 (#24/34)
- MedScribe: 82.98 (#25/74)
- MedCode: 40.36 (#41/75)
- APEX-Agents: 6.4 (#43/45)
- IOI: 2.61 (#57/64)
- HMMT 2025: 64.17 (#58/82)
Claude Sonnet 4 (20250514) — ELO 1622, #142
- OpenEval - Omni-MATH: 60.2 (#3/74)
- OpenEval - GPQA CoT: 70.63 (#5/74)
- HELM Safety: 97.4369 (#5/59)
- OpenEval - MMLU-Pro CoT: 84.3 (#8/77)
- MATH 500: 93.8 (#16/60)
- OpenEval - IFEval Strict: 84.01 (#19/77)
- WritingBench: 74.07 (#35/67)
- MGSM: 90.87 (#36/74)
- IOI: 4.58 (#49/64)
- MedCode: 34.96 (#52/75)
- ...and 1 more
GLM-4.5 — ELO 1621, #143
- VitaBench 2.0: 36.9 (#12/21)
- SWE-bench Verified: 54.2 (#32/47)
- MGSM: 90.84 (#37/74)
- MathArena Apex: 1.04 (#42/50)
- HMMT 2025: 77.5 (#46/82)
- IOI: 2.92 (#56/64)
KAT-Coder-Pro V1 — ELO 1621, #145
- AIIQ Composite IQ: 101.0 (#73/115)
DeepSeek R1 0528 — ELO 1620, #146
- OpenEval - GPQA CoT: 66.59 (#8/74)
- VitaBench 2.0: 39.6 (#9/21)
- OpenEval - MMLU-Pro CoT: 79.3 (#12/77)
- OpenEval - Omni-MATH: 42.4 (#14/74)
- OpenEval - IFEval Strict: 78.37 (#42/77)
GPT-4.5 — ELO 1620, #147
- TextClass Benchmark: 1767.86 (#5/108)
- LingOly-TOO: 25.4502 (#10/16)
GPT-4.1 — ELO 1619, #149
- NL2Scratch: 99.1 (#1/11)
- Fin-RATE: 33.24 (#2/17)
- CaseLaw v2: 69.88 (#3/54)
- MATH 500: 87.2 (#33/60)
- MGSM: 87.67 (#58/74)
Qwen 3 235B A22B 2507 Instruct — ELO 1619, #151
- OpenEval - Omni-MATH: 71.83 (#1/74)
- OpenEval - GPQA CoT: 72.65 (#3/74)
- OpenEval - MMLU-Pro CoT: 84.4 (#7/77)
- WritingBench: 85.2 (#7/67)
- OpenEval - IFEval Strict: 83.52 (#23/77)
Kimi K2 0905 — ELO 1618, #153
- MageBench Season 1: 1558.0 (#26/34)
Gemini 2.5 Flash (Preview 04-17) — ELO 1618, #154
- OpenEval - IFEval Strict: 89.8 (#6/77)
- OpenEval - Omni-MATH: 38.45 (#20/74)
- MATH 500: 91.8 (#20/60)
- OpenEval - GPQA CoT: 39.01 (#32/74)
- OpenEval - MMLU-Pro CoT: 63.9 (#33/77)
O1 Preview — ELO 1616, #155
- LingOly-TOO: 32.2202 (#6/16)
- AA MMLU-Pro: 84.77 (#37/348)
- AA MATH-500: 92.4 (#61/195)
- AA AIME 2025: 79.67 (#69/270)
- AA GPQA Diamond: 76.46 (#214/575)
- Artificial Analysis Intelligence Index: 17.21 (#282/574)
Doubao Seed Code — ELO 1616, #156
- AIIQ Composite IQ: 97.0 (#85/115)
Claude 3.7 Sonnet — ELO 1615, #157
- LingOly-TOO: 42.8881 (#3/16)
- USAMO25: 3.65 (#10/13)
- HMMT 2025: 31.67 (#72/82)
Gemma 4 31B — ELO 1615, #158
- AIIQ Composite IQ: 101.0 (#74/115)
Qwen 3.5 35B A3B — ELO 1613, #161
- MathArena Apex: 4.17 (#31/50)
Grok 3 Mini Beta — ELO 1613, #162
- TextClass Benchmark: 1546.47 (#50/108)
Gemma 4 26B A4B (IT) — ELO 1612, #163
- CPTU Bench: 4.23 (#11/102)
Claude Haiku 4.5 — ELO 1611, #165
- IDE-Bench: 78.75 (#4/15)
- MageBench Season 1: 1637.0 (#9/34)
- HarmActionsEval: 0.0 (#10/10)
- SWE-bench Verified: 66.6 (#18/47)
- Inverse Turing Bench: 47.04 (#34/40)
- AIIQ Composite IQ: 98.0 (#81/115)
- PM-LLM-Benchmark: 27.7 (#126/171)
- AA-LCR: 43.7 (#166/217)
- CritPt: 0.0 (#206/213)
MiMo-V2-Flash — ELO 1610, #168
- MageBench Season 1: 1586.0 (#21/34)
- AIIQ Composite IQ: 105.0 (#57/115)
Claude 3.5 Sonnet (20241022) — ELO 1609, #170
- OpenEval - IFEval Strict: 85.55 (#14/77)
- OpenEval - MMLU-Pro CoT: 77.7 (#16/77)
- MGSM: 92.58 (#16/74)
- OpenEval - GPQA CoT: 56.5 (#17/74)
- OpenEval - Omni-MATH: 27.6 (#38/74)
- MATH 500: 72.4 (#50/60)
- TextClass Benchmark: 1384.79 (#90/108)
Claude 3.5 Sonnet (20240620) — ELO 1607, #173
- Aider Refactoring Benchmark: 64.0 (#4/14)
- Defects4J: 41.5 (#9/35)
- OpenEval - IFEval Strict: 85.98 (#12/77)
- OpenEval - MMLU-Pro CoT: 73.34 (#21/77)
Grok 3 — ELO 1607, #174
- USAMO25: 4.76 (#8/13)
- MATH 500: 89.8 (#27/60)
- MGSM: 91.35 (#29/74)
- TextClass Benchmark: 1545.77 (#51/108)
Grok Code Fast 1 — ELO 1605, #176
- IDE-Bench: 11.25 (#12/15)
- IOI: 4.33 (#51/64)
Qwen 3 Next 80B A3B — ELO 1603, #178
- OpenEval - Omni-MATH: 46.68 (#11/74)
- OpenEval - GPQA CoT: 63.0 (#12/74)
- OpenEval - MMLU-Pro CoT: 78.6 (#13/77)
- OpenEval - IFEval Strict: 80.96 (#35/77)
Mistral Medium 3.5 — ELO 1603, #179
- Vals AI ProofBench: 9.0 (#22/24)
- Vals Multimodal Index: 34.77 (#29/29)
- Finance Agent v2: 32.1 (#34/40)
- Vals AI Harvey Legal Agent Bench: 0.42 (#36/50)
- Finance Agent v1.1: 46.11 (#36/53)
- Vals Index: 37.81 (#37/40)
- ProofBench: 10.0 (#39/54)
- Vals AI Legal Research Bench: 9.13 (#44/49)
- Vals AI Code Migration: 5.13 (#44/48)
- Vals AI Excel Modeling: 11.19 (#45/46)
- ...and 4 more
Gemini 2.5 Flash Lite (Preview 09-2025) — ELO 1602, #180
- MGSM: 89.53 (#46/74)
- MedScribe: 75.82 (#50/74)
- MedCode: 34.19 (#54/75)
Ling-2.6-1T — ELO 1602, #181
- ClawProBench: 57.4 (#25/57)
- SvelteBench: 68.9 (#132/169)
Claude 3.5 Sonnet — ELO 1601, #182
- LingOly-TOO: 28.1014 (#8/16)
- ZEROBench-Sub: 20.71 (#26/46)
- HMMT 2025: 1.67 (#82/82)
GPT-5.4 Nano — ELO 1601, #183
- Vals Multimodal Index: 47.64 (#24/29)
- Vals AI Code Migration: 14.47 (#30/48)
- APEX-Agents: 25.5 (#30/45)
- Finance Agent v2: 38.22 (#30/40)
- Vals Index: 46.63 (#31/40)
- LiveBench: 71.31 (#32/48)
- Finance Agent v1.1: 47.8 (#32/53)
- IOI: 15.25 (#32/64)
- MedCode: 41.03 (#36/75)
- Vals AI Harvey Legal Agent Bench: 0.0 (#40/50)
- ...and 5 more
InternVL3-78B — ELO 1597, #186
- RealWorldQA: 78.0 (#7/12)
MiniMax-M2 — ELO 1597, #187
- SWE-bench Verified: 61.0 (#24/47)
Trinity Large — ELO 1597, #188
- CaseLaw v2: 57.88 (#29/54)
O3 Mini — ELO 1596, #189
- LingOly-TOO: 30.5909 (#7/16)
- USAMO25: 2.08 (#13/13)
- HMMT 2025: 67.5 (#55/82)
Llama 3.1 405B — ELO 1596, #190
- TextClass Benchmark: 1755.81 (#7/108)
Mistral Medium 3.1 — ELO 1595, #193
- MageBench Season 1: 1569.0 (#25/34)
Qwen 3 235B A22B — ELO 1595, #194
- MATH 500: 94.6 (#9/60)
- MageBench Season 1: 1594.0 (#17/34)
- MGSM: 92.47 (#17/74)
- HMMT 2025: 62.5 (#59/82)
- IOI: 0.0 (#64/64)
Mercury 2 — ELO 1595, #195
- AIIQ Composite IQ: 99.0 (#80/115)
Nemotron 3 Super — ELO 1593, #198
- AIIQ Composite IQ: 103.0 (#64/115)
Qwen3 Coder Next — ELO 1592, #199
- ClawProBench: 46.84 (#48/57)
GPT-4o — ELO 1591, #201
- ComplexFuncBench: 66.5 (#1/18)
- LingOly-TOO: 15.6334 (#12/16)
- HMMT 2025: 5.83 (#81/82)
Nemotron 3.5 Lightning — ELO 1591, #202
- LLM Stats (Terminal-Bench 2.1): 24.58 (#28/28)
- LLM Stats (BrowseComp): 36.97 (#56/62)
- OpenRouter GPQA Diamond: 68.2 (#84/114)
- ZeroEval GPQA Diamond: 75.44 (#107/239)
- SpeechMap Compliance: 75.0 (#114/367)
- AA GDPval: 823.59 (#117/198)
- Tau3 Banking: 8.87 (#118/166)
- AI Chess Leaderboard (Reasoning): 752.0 (#121/313)
- AI Chess Leaderboard (Continuation): 553.0 (#138/258)
- SvelteBench: 55.6 (#141/169)
- ...and 30 more
GPT-OSS-120B — ELO 1590, #203
- FINAL Bench Metacognitive: 73.33 (#7/9)
- MATH 500: 94.8 (#7/60)
- MGSM: 92.04 (#24/74)
- MageBench Season 1: 1516.0 (#31/34)
- HMMT 2025: 90.0 (#32/82)
- MathArena Apex: 1.04 (#44/50)
- Finance Agent v1.1: 21.54 (#47/53)
- CaseLaw v2: 48.77 (#50/54)
- AIIQ Composite IQ: 105.0 (#58/115)
Grok 3 Mini — ELO 1590, #204
- HMMT 2025: 74.17 (#50/82)
O3 Mini (2025-01-31) — ELO 1590, #205
- TextClass Benchmark: 1684.99 (#18/108)
- MATH 500: 91.8 (#19/60)
- MGSM: 91.35 (#28/74)
GPT-4.1 Mini — ELO 1588, #207
- MATH 500: 88.0 (#31/60)
- SWE-bench Verified: 23.9 (#42/47)
- TextClass Benchmark: 1547.62 (#49/108)
- MGSM: 87.78 (#57/74)
Mistral Large 3 — ELO 1588, #208
- AIIQ Composite IQ: 89.0 (#103/115)
GLM 4.5 Air — ELO 1587, #211
- HMMT 2025: 69.17 (#53/82)
DeepSeek V3 (0324) — ELO 1587, #212
- MGSM: 91.67 (#27/74)
- MATH 500: 88.6 (#30/60)
- IOI: 1.67 (#60/64)
INTELLECT-3 — ELO 1587, #213
- AIIQ Composite IQ: 94.0 (#95/115)
Gemma 4 12B — ELO 1586, #215
- AIIQ Composite IQ: 98.0 (#82/115)
Llama 4 Maverick — ELO 1584, #218
- IDE-Bench: 2.5 (#14/15)
- MageBench Season 1: 1590.0 (#19/34)
- TextClass Benchmark: 1473.88 (#73/108)
Gemini 2.0 Flash — ELO 1584, #219
- USAMO25: 4.17 (#9/13)
- TextClass Benchmark: 1701.95 (#15/108)
- SWE-bench Verified: 13.5 (#45/47)
- HMMT 2025: 35.83 (#67/82)
Qwen 2.5 72B Instruct — ELO 1583, #222
- OpenEval - IFEval Strict: 89.93 (#5/77)
- OpenEval - MMLU-Pro CoT: 69.31 (#26/77)
Gemini 2.0 Flash (001) — ELO 1583, #223
- OpenEval - Omni-MATH: 45.9 (#12/74)
- OpenEval - GPQA CoT: 55.61 (#18/74)
- OpenEval - IFEval Strict: 84.07 (#18/77)
- OpenEval - MMLU-Pro CoT: 73.7 (#20/77)
- MATH 500: 88.0 (#32/60)
- MGSM: 89.02 (#50/74)
Magistral Medium 1.2 — ELO 1583, #225
- IOI: 0.67 (#62/64)
- MGSM: 74.62 (#71/74)
- AIIQ Composite IQ: 94.0 (#93/115)
GPT-4o (2024-11-20) — ELO 1582, #228
- CaseLaw v2: 59.7 (#25/54)
- MGSM: 90.36 (#40/74)
- SWE-bench Verified: 21.6 (#43/47)
- MATH 500: 74.0 (#47/60)
Qwen 3.5 9B — ELO 1580, #234
- RealWorldQA: 80.3 (#5/12)
- MathArena Apex: 0.52 (#49/50)
Llama 3.1 405B Instruct — ELO 1580, #235
- MATH 500: 71.4 (#51/60)
DeepSeek R1 — ELO 1579, #240
- TextClass Benchmark: 1718.73 (#14/108)
- MATH 500: 92.2 (#18/60)
- MGSM: 92.25 (#20/74)
DeepSeek V3 — ELO 1579, #241
- TextClass Benchmark: 1732.54 (#11/108)
- OpenEval - Omni-MATH: 40.27 (#17/74)
- OpenEval - GPQA CoT: 53.81 (#19/74)
- MGSM: 92.15 (#21/74)
- OpenEval - MMLU-Pro CoT: 72.3 (#24/77)
- OpenEval - IFEval Strict: 83.24 (#25/77)
- MATH 500: 80.4 (#38/60)
GPT-4o (2024-08-06) — ELO 1576, #242
- Aider Refactoring Benchmark: 49.4 (#8/14)
- Defects4J: 34.1 (#15/35)
- MGSM: 90.69 (#38/74)
- MATH 500: 75.2 (#45/60)
- Finance Agent v1.1: 8.06 (#49/53)
GPT-5 Nano — ELO 1575, #245
- CommunityFact: 75.59 (#3/17)
- RealWorldQA: 71.8 (#11/12)
- MATH 500: 93.8 (#15/60)
- MageBench Season 1: 1499.0 (#33/34)
- ProofBench: 12.0 (#37/54)
- SWE-bench Verified: 34.8 (#39/47)
- CaseLaw v2: 52.63 (#43/54)
- MGSM: 89.31 (#48/74)
- MedScribe: 72.86 (#55/74)
- MedCode: 30.44 (#67/75)
- ...and 1 more
Gemini 1.5 Pro (002) — ELO 1573, #249
- OpenEval - MMLU-Pro CoT: 73.7 (#19/77)
- OpenEval - GPQA CoT: 53.36 (#20/74)
- OpenEval - IFEval Strict: 83.67 (#22/77)
- OpenEval - Omni-MATH: 36.4 (#23/74)
- MATH 500: 82.8 (#37/60)
- MGSM: 89.2 (#49/74)
Llama 4 Maverick Instruct — ELO 1573, #250
- MedCode: 36.51 (#51/75)
- MedScribe: 54.22 (#73/74)
GPT-4o (2024-05-13) — ELO 1572, #251
- TextClass Benchmark: 1825.22 (#1/108)
Claude Haiku 4.5 (20251001) — ELO 1571, #252
- OpenEval - Omni-MATH: 56.1 (#4/74)
- OpenEval - GPQA CoT: 60.54 (#15/74)
- MedScribe: 85.23 (#16/74)
- OpenEval - MMLU-Pro CoT: 77.7 (#17/77)
- MGSM: 92.15 (#22/74)
- Vals Multimodal Index: 42.88 (#26/29)
- CaseLaw v2: 56.48 (#30/54)
- Vals AI Harvey Legal Agent Bench: 0.83 (#32/50)
- Finance Agent v1.1: 46.93 (#34/53)
- Finance Agent v2: 31.01 (#35/40)
- ...and 6 more
Gemini 2.5 Flash Lite — ELO 1570, #255
- OpenEval - Omni-MATH: 47.97 (#10/74)
- OpenEval - IFEval Strict: 81.02 (#34/77)
- OpenEval - MMLU-Pro CoT: 53.7 (#43/77)
- OpenEval - GPQA CoT: 30.94 (#47/74)
- MedScribe: 72.83 (#56/74)
- MedCode: 27.11 (#71/75)
Mistral Small 4 — ELO 1568, #257
- ClawProBench: 45.26 (#49/57)
Devstral Small 2 — ELO 1566, #260
- SWE-bench Verified: 56.4 (#28/47)
Gemini 1.5 Pro — ELO 1564, #265
- PhysicsFinals: 63.9 (#1/34)
- TextClass Benchmark: 1782.7 (#2/108)
- Natural2Code: 85.4 (#2/31)
- LingOly-TOO: 20.4615 (#11/16)
Llama 3.1 70B Instruct — ELO 1561, #269
- MATH 500: 65.0 (#56/60)
Magistral Small 1.2 — ELO 1561, #271
- MGSM: 86.25 (#62/74)
Qwen 3 VL 30B A3B — ELO 1561, #272
- RealWorldQA: 77.4 (#8/12)
Mistral Large 2 (Nov) Instruct (2411) — ELO 1560, #276
- MATH 500: 74.4 (#46/60)
- MGSM: 87.24 (#60/74)
Qwen 3 30B A3B 2507 — ELO 1560, #278
- MathArena Apex: 0.52 (#48/50)
Command A+ — ELO 1560, #280
- AIIQ Composite IQ: 89.0 (#105/115)
Grok 2 (1212) — ELO 1558, #281
- TextClass Benchmark: 1758.36 (#6/108)
- MATH 500: 78.4 (#42/60)
- MGSM: 86.15 (#64/74)
Qwen 3 30B A3B — ELO 1556, #286
- HMMT 2025: 50.83 (#61/82)
GPT-4 Turbo — ELO 1553, #294
- TextClass Benchmark: 1781.47 (#3/108)
- Aider Refactoring Benchmark: 34.1 (#10/14)
jamba-large-1.7 — ELO 1550, #304
- Finance Agent v1.1: 0.37 (#52/53)
Llama 3.3 70B — ELO 1547, #311
- TextClass Benchmark: 1746.41 (#9/108)
Llama 3.3 70B Instruct — ELO 1546, #313
- LingOly-TOO: 8.2131 (#15/16)
- MGSM: 91.09 (#30/74)
- MATH 500: 73.4 (#48/60)
Nova Pro (v1) — ELO 1546, #314
- OpenEval - GPQA CoT: 44.62 (#26/74)
- OpenEval - MMLU-Pro CoT: 67.3 (#28/77)
- OpenEval - IFEval Strict: 81.55 (#33/77)
- OpenEval - Omni-MATH: 24.22 (#39/74)
Llama 4 Scout Instruct — ELO 1546, #315
- MATH 500: 79.2 (#40/60)
- MGSM: 87.96 (#55/74)
- MedCode: 23.31 (#74/75)
- MedScribe: 50.59 (#74/74)
GPT-4 Preview (0125) — ELO 1545, #319
- Aider Refactoring Benchmark: 33.7 (#11/14)
Yi 1.5 34B — ELO 1540, #332
- PhysicsFinals: 17.4 (#17/34)
- EHRBench: 58.94 (#18/31)
QwQ-32B — ELO 1538, #335
- USAMO25: 2.98 (#11/13)
- HMMT 2025: 47.5 (#63/82)
Qwen 3.5 4B — ELO 1536, #340
- RealWorldQA: 79.5 (#6/12)
GPT-OSS-20B — ELO 1535, #342
- MATH 500: 94.2 (#13/60)
- HMMT 2025: 76.67 (#47/82)
- MGSM: 89.02 (#51/74)
- CaseLaw v2: 43.84 (#54/54)
- AIIQ Composite IQ: 100.0 (#78/115)
Llama 4 Scout — ELO 1535, #343
- IDE-Bench: 2.5 (#13/15)
- TextClass Benchmark: 1500.45 (#66/108)
GPT-4 — ELO 1531, #351
- AA Omniscience - Business: 21.1 (#155/479)
- AA MATH-500: 56.8 (#159/195)
- AA Omniscience - Law: 16.5 (#164/479)
- AA Omniscience - Software Engineering (SWE) - Julia: 20.0 (#172/479)
- AA GDPval: 238.72 (#174/198)
- AA Omniscience - Humanities & Social Sciences: 23.6 (#180/479)
- AA Omniscience: -33.83 (#207/479)
- AA Omniscience - Health: 21.7 (#211/479)
- AA Omniscience - Software Engineering (SWE) - R: 14.0 (#216/479)
- AA-Omniscience Accuracy: 21.0 (#222/479)
- ...and 18 more
Falcon-H1R-7B — ELO 1530, #356
- HMMT 2025: 84.17 (#39/82)
Gemini 1.5 Flash (002) — ELO 1526, #360
- OpenEval - IFEval Strict: 83.12 (#26/77)
- OpenEval - GPQA CoT: 43.72 (#27/74)
- OpenEval - MMLU-Pro CoT: 67.8 (#27/77)
- OpenEval - Omni-MATH: 30.45 (#31/74)
- MATH 500: 78.8 (#41/60)
- MGSM: 86.58 (#61/74)
Mistral Large — ELO 1526, #363
- MageBench Season 1: 1501.0 (#32/34)
Gemini 1.5 Flash — ELO 1523, #370
- TextClass Benchmark: 1668.98 (#25/108)
Qwen 3 4B 2507 — ELO 1523, #371
- MathArena Apex: 2.08 (#36/50)
GPT-4 (0613) — ELO 1523, #372
- TextClass Benchmark: 1747.59 (#8/108)
Ling-2.6-flash — ELO 1523, #373
- ClawProBench: 27.04 (#57/57)
O1 Mini (2024-09-12) — ELO 1521, #375
- TextClass Benchmark: 1626.65 (#29/108)
Gemma 3 27B — ELO 1518, #378
- BIG-Bench Extra Hard: 19.3 (#8/20)
QwQ 32B-Preview — ELO 1514, #383
- HMMT 2025: 18.33 (#76/82)
GPT-4o Mini (2024-07-18) — ELO 1511, #387
- TextClass Benchmark: 1674.64 (#23/108)
- MATH 500: 72.6 (#49/60)
- MGSM: 86.18 (#63/74)
GPT-4o Mini — ELO 1508, #394
- NL2Scratch: 78.0 (#11/11)
- VitaBench 2.0: 6.7 (#21/21)
Phi-4 — ELO 1508, #395
- LingOly-TOO: 10.9966 (#14/16)
Granite 4.0 H Small — ELO 1504, #400
- OpenEval - IFEval Strict: 88.97 (#7/77)
- OpenEval - Omni-MATH: 29.57 (#34/74)
- OpenEval - GPQA CoT: 38.34 (#35/74)
- OpenEval - MMLU-Pro CoT: 56.9 (#39/77)
Nova Lite (v1) — ELO 1503, #403
- OpenEval - GPQA CoT: 39.69 (#31/74)
- OpenEval - MMLU-Pro CoT: 60.0 (#37/77)
- OpenEval - Omni-MATH: 23.3 (#40/74)
- OpenEval - IFEval Strict: 77.6 (#45/77)
aya-23-35B — ELO 1494, #418
- LingOly-TOO: 5.7082 (#16/16)
Claude 3.5 Haiku (20241022) — ELO 1493, #420
- OpenEval - MMLU-Pro CoT: 60.5 (#36/77)
- OpenEval - GPQA CoT: 36.32 (#39/74)
- OpenEval - IFEval Strict: 79.17 (#40/77)
- OpenEval - Omni-MATH: 22.4 (#42/74)
- MATH 500: 64.2 (#57/60)
- MGSM: 84.62 (#68/74)
- TextClass Benchmark: 1413.9 (#85/108)
Gemma 3 12B — ELO 1491, #426
- BIG-Bench Extra Hard: 16.3 (#9/20)
Yi 1.5 34B Chat — ELO 1481, #441
- INVESTORBENCH: 37.97 (#5/14)
GLM-4 9B Chat — ELO 1476, #446
- TextClass Benchmark: 1616.51 (#33/108)
GPT-4.1 Nano — ELO 1474, #449
- MATH 500: 80.2 (#39/60)
- TextClass Benchmark: 1533.06 (#58/108)
- MGSM: 69.27 (#73/74)
Gemma 4 E4B — ELO 1472, #451
- AIIQ Composite IQ: 93.0 (#99/115)
Yi-1.5-9B — ELO 1470, #457
- EHRBench: 45.51 (#27/31)
Qwen 3.5 2B — ELO 1469, #459
- ArxivMath: 10.62 (#17/17)
Ministral 3 3B — ELO 1469, #460
- HarmActionsEval: 2.13 (#5/10)
Command-R+ — ELO 1469, #461
- IDE-Bench: 0.0 (#15/15)
GPT-3.5 Turbo — ELO 1464, #470
- VitaBench 2.0: 14.0 (#20/21)
Mistral Small (2402) — ELO 1457, #482
- MATH 500: 70.6 (#53/60)
- MGSM: 83.96 (#70/74)
Granite 4.0 Micro — ELO 1455, #489
- OpenEval - IFEval Strict: 84.87 (#17/77)
- OpenEval - Omni-MATH: 20.93 (#45/74)
- OpenEval - GPQA CoT: 30.72 (#50/74)
- OpenEval - MMLU-Pro CoT: 39.5 (#53/77)
Llama 3.1 8B Instruct — ELO 1453, #494
- MATH 500: 44.4 (#59/60)
GPT-3.5 Turbo (0301) — ELO 1449, #501
- HELM: 76.03 (#13/66)
Yi-1.5-9B Chat — ELO 1444, #514
- INVESTORBENCH: 22.91 (#12/14)
OLMo-2-1124-7B-Instruct — ELO 1436, #525
- OpenEval - GPQA CoT: 29.6 (#52/74)
- OpenEval - IFEval Strict: 69.29 (#54/77)
- OpenEval - Omni-MATH: 11.63 (#56/74)
- OpenEval - MMLU-Pro CoT: 29.2 (#57/77)
Gemma 4 E2B — ELO 1434, #530
- AIIQ Composite IQ: 86.0 (#106/115)
GPT-3.5 Turbo (0125) — ELO 1424, #548
- TextClass Benchmark: 1560.51 (#44/108)
Gemma 3 4B — ELO 1409, #570
- BIG-Bench Extra Hard: 11.0 (#11/20)
Phi-4 Mini Instruct — ELO 1401, #578
- HarmActionsEval: 2.84 (#3/10)
Yi 6B (Base) — ELO 1335, #653
- BBH: 47.2 (#15/16)
Claude Mythos 5 — ELO 1809
- OSWorld-Verified: 85.0 (#1/27)
DeepSeek V4 Flash (0423) — ELO 1669
- LLM Stats (CodeForces): 93.87 (#3/17)
- LLM Stats (MathArena Apex): 72.1 (#3/8)
- LLM Stats (CSimpleQA): 73.2 (#7/8)
- LLM Stats (HMMT Feb 26): 91.9 (#7/12)
- LLM Stats (IMO-AnswerBench): 85.1 (#10/20)
- LLM Stats (MCP Atlas): 67.4 (#23/33)
- LLM Stats (Toolathlon): 43.5 (#27/37)
- LLM Stats (BrowseComp): 53.5 (#41/62)
- ZeroEval GPQA Diamond: 87.4 (#43/239)
- LLM Stats Score: 36.8 (#64/341)
DeepSeek V4 Pro Preview — ELO 1676
- GDPevo: 43.58 (#4/4)
- Creative Writing (Lechmazur): 0.0 (#22/44)
- Multi-turn Debate (Lechmazur): 1471.8 (#28/46)
- NYT Connections Extended: 59.9 (#53/104)
Doubao-Seed-1.6 — ELO 1635
- VitaBench 2.0: 37.3 (#10/21)
ERNIE 5.0 — ELO 1609
- AIIQ Composite IQ: 94.0 (#96/115)
GLM-5.3 — ELO 1778
- LLM Stats (AutomationBench): 48.2 (#1/12)
- LLM Stats (CyberGym): 84.5 (#1/13)
- LLM Stats (PostTrainBench): 39.8 (#1/6)
- FrontierSWE: 78.0 (#2/17)
- LLM Stats (NL2Repo): 58.0 (#2/17)
- LLM Stats (FrontierSWE): 78.1 (#3/16)
- LLM Stats (Terminal-Bench 2.1): 88.2 (#3/28)
- LLM Stats (Toolathlon): 73.0 (#4/37)
- Vending-Bench 2: 8163.61 (#6/60)
- LLM Stats (Program Bench): 19.0 (#6/6)
- ...and 5 more
GPT-4 Turbo (Preview) — ELO 1538
- Natural2Code: 51.5 (#11/31)
GPT-5.1 Codex Max — ELO 1686
- IDE-Bench: 73.75 (#5/15)
- ALE-Bench: 1228.25 (#12/89)
- IOI: 21.42 (#24/64)
- LiveBench: 72.39 (#29/48)
- Epoch AI - Proofbench: 9.0 (#40/52)
- ProofBench: 9.0 (#41/54)
GPT-5.5 Instant — ELO 1666
- Arena AI Document: 1402.0 (#36/38)
- FrontierMath - Tiers 1-3 (v2): 26.32 (#42/48)
- FrontierMath - Tier 4 (v2): 2.44 (#43/50)
- SimpleQA Verified: 38.0 (#45/77)
- Chess Puzzles (Epoch AI): 12.0 (#116/161)
- OTIS Mock AIME 2024-25: 68.06 (#128/238)
Gemini 2.0 Flash Lite (Preview 02-05) — ELO 1528
- OpenEval - Omni-MATH: 37.37 (#21/74)
- OpenEval - GPQA CoT: 50.0 (#23/74)
- OpenEval - MMLU-Pro CoT: 72.0 (#25/77)
- OpenEval - IFEval Strict: 82.44 (#29/77)
Gemini 2.5 Flash (Preview 09-2025) — ELO 1646
- MedScribe: 78.5 (#37/74)
- MedCode: 40.54 (#39/75)
- MGSM: 89.85 (#43/74)
Gemma 3n E2B Instructed LiteRT (Preview) — ELO 1392
- ECLeKTic: 2.5 (#12/16)
Gemma 3n E4B Instructed LiteRT Preview — ELO 1423
- ECLeKTic: 1.9 (#14/16)
Gemma 4 26B — ELO 1601
- AIIQ Composite IQ: 96.0 (#90/115)
Grok Beta — ELO 1543
- TextClass Benchmark: 1741.94 (#10/108)
Laguna M.1 — ELO 1574
- Finance Agent v2: 25.03 (#39/40)
- Vals Index: 33.33 (#39/40)
- ProofBench: 0.0 (#54/54)
- MedCode: 25.24 (#73/75)
- AIIQ Composite IQ: 84.0 (#108/115)
Laguna S 2.1 — ELO 1658
- LLM Stats (Terminal-Bench 2.1): 70.2 (#19/28)
- LLM Stats (Toolathlon): 49.7 (#19/37)
- LLM Stats (DeepSWE 1.1): 40.4 (#21/25)
- LLM Stats Score: 41.9 (#40/341)
Laguna XS.2 — ELO 1548
- Finance Agent v2: 15.6 (#40/40)
- Vals Index: 30.04 (#40/40)
- ProofBench: 0.0 (#53/54)
- MedCode: 20.7 (#75/75)
- AIIQ Composite IQ: 85.0 (#107/115)
LongCat-Flash-Chat — ELO 1574
- VitaBench 2.0: 29.8 (#16/21)
LongCat-Flash-Thinking-2601 — ELO 1636
- ClawProBench: 57.48 (#24/57)
Muse Spark — ELO 1731
- Finance Agent v1.1: 60.59 (#4/53)
- LMArena Text Arena: 1487.95 (#11/24)
- MedCode: 51.31 (#11/75)
- MedScribe: 85.9 (#11/74)
- CaseLaw v2: 63.13 (#17/54)
- ProofBench: 17.0 (#30/54)
O1 Preview (2024-09-12) — ELO 1584
- TextClass Benchmark: 1622.24 (#31/108)
O1 Pro — ELO 1624
- USAMO25: 2.83 (#12/13)
Palmyra Fin — ELO 1566
- OpenEval - GPQA CoT: 42.15 (#28/74)
- OpenEval - Omni-MATH: 29.47 (#35/74)
- OpenEval - MMLU-Pro CoT: 59.1 (#38/77)
- OpenEval - IFEval Strict: 79.3 (#39/77)
Palmyra Med — ELO 1482
- OpenEval - GPQA CoT: 36.77 (#37/74)
- OpenEval - IFEval Strict: 76.74 (#46/77)
- OpenEval - MMLU-Pro CoT: 41.1 (#48/77)
- HELM Safety: 85.6767 (#49/59)
- OpenEval - Omni-MATH: 15.57 (#51/74)
Palmyra X5 — ELO 1618
- OpenEval - GPQA CoT: 66.14 (#9/74)
- OpenEval - MMLU-Pro CoT: 80.4 (#11/77)
- OpenEval - Omni-MATH: 41.45 (#16/74)
- OpenEval - IFEval Strict: 82.29 (#30/77)
Palmyra-X-004 — ELO 1577
- OpenEval - IFEval Strict: 87.25 (#9/77)
- OpenEval - Omni-MATH: 31.97 (#28/74)
- OpenEval - MMLU-Pro CoT: 65.7 (#31/77)
Pixtral-12B-2409 — ELO 1408
- TextClass Benchmark: 1490.38 (#67/108)
Qwen 3 Max (2026-01-23) — ELO 1663
- ClawProBench: 55.76 (#33/57)
- IOI: 13.75 (#34/64)
- CaseLaw v2: 54.98 (#36/54)
- MedScribe: 72.71 (#57/74)
- MedCode: 31.37 (#66/75)
Qwen 3.5 — ELO 1706
- APEX-Agents: 27.7 (#29/45)
Qwen 3.6 Max — ELO 1714
- AIIQ Composite IQ: 114.0 (#36/115)
Qwen 3.7 Flash — ELO 1633
- AI for Education Pedagogy - Science: 91.26 (#33/233)
- AI for Education Pedagogy - Secondary: 84.75 (#65/233)
- AI for Education Pedagogy - Primary: 88.73 (#68/233)
- AI for Education Pedagogy - Maths: 84.92 (#69/233)
- AI for Education Pedagogy - Social studies: 83.64 (#69/233)
- AI for Education Pedagogy: 84.76 (#71/233)
- AI for Education SEND: 77.52 (#88/225)
- AI for Education Pedagogy - Technology: 81.13 (#89/233)
- SvelteBench: 82.5 (#113/169)
Qwen 3.8 27B — ELO 1717
- LLM Stats (BabyVision): 85.6 (#2/10)
- LLM Stats (MathVision): 94.6 (#2/33)
- LLM Stats (OSWorld-Verified): 84.3 (#3/24)
- LLM Stats (CharXiv-R): 90.2 (#4/51)
- LLM Stats (OmniDocBench 1.5): 91.1 (#4/18)
- LLM Stats (Agents' Last Exam): 42.9 (#5/10)
- LLM Stats (RealWorldQA): 85.9 (#5/29)
- LLM Stats (ERQA): 65.5 (#7/24)
- LLM Stats (NL2Repo): 42.3 (#11/17)
- LLM Stats (Terminal-Bench 2.1): 73.0 (#16/28)
- ...and 5 more
Qwen3.8 2.4T A95B — ELO 1726
- Tau3 Banking: 49.07 (#3/166)
- AA GPQA Diamond: 93.54 (#8/575)
- AA GDPval: 1720.39 (#8/198)
- Artificial Analysis Intelligence Index: 57.7 (#11/574)
- AA CritPt: 20.0 (#22/479)
- AA Humanity's Last Exam: 42.45 (#25/567)
- SvelteBench: 96.7 (#27/169)
- AA Omniscience - Software Engineering (SWE) - Rust: 82.0 (#30/479)
- AA SciCode: 51.62 (#39/567)
- AA Long Context Reasoning: 75.33 (#44/500)
- ...and 22 more
Step 3.5 Flash 2603 — ELO 1602
- ClawProBench: 42.59 (#52/57)
Yi Large — ELO 1519
- TextClass Benchmark: 1473.21 (#74/108)
Yi-1.5-9B-Chat-16K — ELO 1455
- Fin-Bias: 83.4 (#18/19)
Yi-Coder-9B-Chat — ELO 1391
- FullStackBench en: 47.13 (#20/31)

Top-10 New Scores (180)

Claude Fable 5 on APEX-Agents-AA: 59.2 (#1)
Claude Fable 5 on ArxivMath: 87.5 (#2)
Claude Fable 5 on Computer Anthology Terminal Tasks (Claude Code): 54.0 (#1)
Claude Fable 5 on Computer Anthology Terminal Tasks (Terminus-2): 50.8 (#3)
Claude Fable 5 on Finance Agent v2: 56.31 (#4)
Claude Fable 5 on FrontierCode: 63.6 (#1)
Claude Fable 5 on IOI: 72.25 (#8)
Claude Fable 5 on LiveBench: 78.57 (#4)
Claude Fable 5 on MedCode: 56.07 (#3)
Claude Fable 5 on MedScribe: 88.52 (#3)
Claude Fable 5 on ProofBench: 77.0 (#3)
Claude Fable 5 on Roboflow Playground Classification: 1200.0 (#13)
Claude Fable 5 on Vals Index: 75.14 (#1)
Claude Fable 5 on Vals Multimodal Index: 74.15 (#1)
Claude Mythos 5 on OSWorld-Verified: 85.0 (#1)
Claude Opus 5 on AIIQ Composite IQ: 134.0 (#2)
Claude Opus 5 on Agent Security League - Functional Correctness: 73.7 (#9)
Claude Opus 5 on Arena AI Code: 1692.0 (#1)
Claude Opus 5 on Arena AI Document: 1520.0 (#1)
Claude Opus 5 on Benchmarks.bio - BioSecBench-Refusal: 36.3 (#9)
Claude Opus 5 on Computer Anthology Terminal Tasks (Claude Code): 54.0 (#2)
Claude Opus 5 on Computer Anthology Terminal Tasks (Terminus-2): 62.0 (#1)
Claude Opus 5 on Creative Writing (Lechmazur): 4.2 (#1)
Claude Opus 5 on Design Arena (SVG): 1361.0 (#1)
Claude Opus 5 on Finance Agent v2: 58.63 (#1)
Claude Opus 5 on IOI: 91.67 (#5)
Claude Opus 5 on LLM Chess (Saplin): 1285.4 (#8)
Claude Opus 5 on LMArena Text Arena: 1493.6 (#7)
Claude Opus 5 on LMArena WebDev Arena: 1691.77 (#1)
Claude Opus 5 on OpenRouter BrowseComp (Search): 89.0 (#1)
Claude Opus 5 on OpenRouter DeepSearchQA (Search): 76.5 (#1)
Claude Opus 5 on OpenRouter HLE (Search): 77.4 (#1)
Claude Opus 5 on ProgramBench: 4.5 (#1)
Claude Opus 5 on ProgramBench Almost: 37.0 (#1)
Claude Opus 5 on TrackingAI IQ Test (Vision): 88.24 (#4)
Claude Opus 5 on Vals AI (Vals Index): 67.21 (#1)
Claude Opus 5 on Vals AI Code Migration: 57.47 (#1)
Claude Opus 5 on Vals AI Excel Modeling: 73.56 (#2)
Claude Opus 5 on Vals AI LiveCodeBench: 89.03 (#2)
Claude Opus 5 on Vals AI MMLU-Pro: 91.59 (#1)
Claude Opus 5 on Vals Index: 74.82 (#2)
Claude Opus 5 on Vals Multimodal Index: 73.9 (#2)
GLM-5.3 on DeepsecBench: 22.0 (#6)
GLM-5.3 on FrontierSWE: 78.0 (#2)
GLM-5.3 on LLM Stats (Agents' Last Exam): 28.5 (#7)
GLM-5.3 on LLM Stats (DeepSWE 1.1): 66.9 (#8)
GLM-5.3 on LLM Stats (FrontierSWE): 78.1 (#3)
GLM-5.3 on LLM Stats (NL2Repo): 58.0 (#2)
GLM-5.3 on LLM Stats (Program Bench): 19.0 (#6)
GLM-5.3 on LLM Stats (Terminal-Bench 2.1): 88.2 (#3)
GLM-5.3 on LLM Stats (Toolathlon): 73.0 (#4)
GLM-5.3 on LLM Stats Score: 54.66 (#6)
GLM-5.3 on Vending-Bench 2: 8163.61 (#6)
GLM-5.3 on VoxelBench: 1806.0 (#12)
GPT-5.4 Pro on MathArena Apex: 69.79 (#3)
GPT-5.5 Pro on PM-LLM-Benchmark: 37.7 (#9)
GPT-5.6 Sol on AIIQ Composite IQ: 136.0 (#1)
GPT-5.6 Sol on APEX-Agents: 39.9 (#12)
GPT-5.6 Sol on APEX-Agents-AA: 56.7 (#3)
GPT-5.6 Sol on Android Bench: 90.8 (#3)
GPT-5.6 Sol on Arena AI Code: 1623.0 (#6)
GPT-5.6 Sol on Arena AI Document: 1479.0 (#9)
GPT-5.6 Sol on BabyVision: 88.9 (#3)
GPT-5.6 Sol on CADGenBench: 0.5319 (#2)
GPT-5.6 Sol on CharXiv-R: 84.6 (#12)
GPT-5.6 Sol on Computer Anthology Terminal Tasks (Codex CLI): 58.0 (#1)
GPT-5.6 Sol on Computer Anthology Terminal Tasks (Terminus-2): 51.6 (#2)
GPT-5.6 Sol on Design Arena (Website): 1337.0 (#3)
GPT-5.6 Sol on Finance Agent v2: 53.76 (#10)
GPT-5.6 Sol on FrontierCode: 60.6 (#3)
GPT-5.6 Sol on GDP.pdf: 30.7 (#1)
GPT-5.6 Sol on IOI: 86.67 (#6)
GPT-5.6 Sol on LLM Chess (Saplin): 1549.7 (#1)
GPT-5.6 Sol on LMArena Text Arena: 1480.8 (#18)
GPT-5.6 Sol on LMArena WebDev Arena: 1622.87 (#6)
GPT-5.6 Sol on MCP Atlas: 83.6 (#6)
GPT-5.6 Sol on MedCode: 43.97 (#26)
GPT-5.6 Sol on MedScribe: 85.23 (#15)
GPT-5.6 Sol on OmniDocBench 1.5: 85.8 (#41)
GPT-5.6 Sol on OpenRouter BrowseComp (Search): 82.4 (#2)
GPT-5.6 Sol on OpenRouter DeepSearchQA (Search): 75.0 (#2)
GPT-5.6 Sol on OpenRouter HLE (Search): 71.1 (#2)
GPT-5.6 Sol on PLawBench: 72.3 (#2)
GPT-5.6 Sol on ProgramBench: 1.0 (#2)
GPT-5.6 Sol on ProgramBench Almost: 15.5 (#3)
GPT-5.6 Sol on ProofBench: 77.0 (#2)
GPT-5.6 Sol on ProphetArena: 0.9506 (#4)
GPT-5.6 Sol on Toolathlon: 74.9 (#6)
GPT-5.6 Sol on Vals Index: 73.12 (#4)
GPT-5.6 Sol on Vals Multimodal Index: 72.19 (#4)
Gemini 3.7 Flash on AA CritPt: 14.29 (#37)
Gemini 3.7 Flash on AA GDPval: 1525.3 (#24)
Gemini 3.7 Flash on AA GPQA Diamond: 94.55 (#2)
Gemini 3.7 Flash on AA Humanity's Last Exam: 47.87 (#8)
Gemini 3.7 Flash on AA Long Context Reasoning: 81.0 (#4)
Gemini 3.7 Flash on AA MMMU-Pro: 85.49 (#1)
Gemini 3.7 Flash on AA Omniscience: 26.48 (#13)
Gemini 3.7 Flash on AA Omniscience - Business: 45.6 (#18)
Gemini 3.7 Flash on AA Omniscience - Health: 44.7 (#27)
Gemini 3.7 Flash on AA Omniscience - Humanities & Social Sciences: 55.2 (#11)
Gemini 3.7 Flash on AA Omniscience - Law: 62.1 (#3)
Gemini 3.7 Flash on AA Omniscience - Science, Engineering & Mathematics: 52.0 (#18)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE): 72.3 (#26)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - C: 90.0 (#10)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Dart: 68.0 (#21)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Go: 74.0 (#25)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - HTML: 80.0 (#23)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Java: 57.0 (#24)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - JavaScript: 77.27 (#28)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Julia: 64.0 (#30)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Kotlin: 62.0 (#45)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - PHP: 78.0 (#25)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Python: 71.5 (#37)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - R: 60.0 (#35)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Rust: 82.0 (#26)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Swift: 80.0 (#17)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - TypeScript: 76.67 (#27)
Gemini 3.7 Flash on AA SciCode: 57.87 (#5)
Gemini 3.7 Flash on AA-Briefcase: 36.36 (#17)
Gemini 3.7 Flash on AA-Omniscience Accuracy: 55.32 (#16)
Gemini 3.7 Flash on AI Chess Leaderboard (Continuation): 1753.0 (#3)
Gemini 3.7 Flash on AI Chess Leaderboard (Reasoning): 1854.0 (#2)
Gemini 3.7 Flash on Agent Arena: 3.61 (#31)
Gemini 3.7 Flash on Agent Arena - Bash Recovery: 2.33 (#42)
Gemini 3.7 Flash on Agent Arena - Confirmed Success: 9.85 (#11)
Gemini 3.7 Flash on Agent Arena - Praise vs Complaint: 1.84 (#44)
Gemini 3.7 Flash on Agent Arena - Steerability: 2.83 (#38)
Gemini 3.7 Flash on Agent Arena - Tool Hallucination: 1.18 (#26)
Gemini 3.7 Flash on Arabic Broad Leaderboard: 9.204 (#4)
Gemini 3.7 Flash on Artificial Analysis Intelligence Index: 56.03 (#17)
Gemini 3.7 Flash on AutomationBench: 30.4 (#1)
Gemini 3.7 Flash on BenchLM: 61.4 (#50)
Gemini 3.7 Flash on Chatbot Arena (Code): 1587.0 (#8)
Gemini 3.7 Flash on Chatbot Arena (Text): 1490.0 (#9)
Gemini 3.7 Flash on Chess Puzzles (Epoch AI): 47.0 (#11)
Gemini 3.7 Flash on ClockBench: 48.3 (#4)
Gemini 3.7 Flash on Coarena: 1039.4 (#2)
Gemini 3.7 Flash on Coarena - Task Completion: 78.1 (#4)
Gemini 3.7 Flash on Computer Anthology Terminal Tasks (Terminus-2): 46.0 (#5)
Gemini 3.7 Flash on Creative Writing (Lechmazur): -0.4 (#25)
Gemini 3.7 Flash on CursorBench 3.1: 61.6 (#15)
Gemini 3.7 Flash on Design Arena (Game Dev): 1310.0 (#16)
Gemini 3.7 Flash on Design Arena (Website): 1348.0 (#2)
Gemini 3.7 Flash on Epoch AI - Mystery Game Puzzles: 37.0 (#7)
Gemini 3.7 Flash on FrontierMath - Tier 4 (v2): 36.59 (#16)
Gemini 3.7 Flash on FrontierMath - Tiers 1-3 (v2): 71.58 (#14)
Gemini 3.7 Flash on Kaggle FACTS Grounding: 75.63 (#5)
Gemini 3.7 Flash on LLM Stats (Agents' Last Exam): 26.3 (#8)
Gemini 3.7 Flash on LLM Stats (Artificial Analysis): 56.0 (#2)
Gemini 3.7 Flash on LLM Stats (AutomationBench): 30.4 (#4)
Gemini 3.7 Flash on LLM Stats (CharXiv-R): 88.7 (#7)
Gemini 3.7 Flash on LLM Stats (DeepSWE 1.1): 65.3 (#10)
Gemini 3.7 Flash on LLM Stats (FrontierCode 1.1): 43.6 (#5)
Gemini 3.7 Flash on LLM Stats (GDP.pdf): 34.0 (#2)
Gemini 3.7 Flash on LLM Stats (Terminal-Bench 2.1): 85.8 (#7)
Gemini 3.7 Flash on LLM Stats Score: 51.2 (#12)
Gemini 3.7 Flash on LM Market Cap LMC Score: 40.0 (#239)
Gemini 3.7 Flash on MineBench: 1788.74 (#9)
Gemini 3.7 Flash on Multi-turn Debate (Lechmazur): 1476.3 (#27)
Gemini 3.7 Flash on OTIS Mock AIME 2024-25: 97.22 (#19)
Gemini 3.7 Flash on OpenRouter GPQA Diamond: 92.3 (#6)
Gemini 3.7 Flash on OpenRouter Tau2-Bench Airline: 80.6 (#3)
Gemini 3.7 Flash on ProfBench: 57.5 (#8)
Gemini 3.7 Flash on ProgramBench: 0.0 (#9)
Gemini 3.7 Flash on ProgramBench Almost: 5.5 (#6)
Gemini 3.7 Flash on RuneBench: 10203.0 (#2)
Gemini 3.7 Flash on SealedBench: 56.0 (#17)
Gemini 3.7 Flash on SimpleQA Verified: 71.2 (#4)
Gemini 3.7 Flash on SpeechMap Compliance: 85.8 (#60)
Gemini 3.7 Flash on SvelteBench: 100.0 (#8)
Gemini 3.7 Flash on Tau3 Banking: 35.46 (#23)
Gemini 3.7 Flash on Vals AI (Vals Index): 59.31 (#7)
Gemini 3.7 Flash on Vals AI Excel Modeling: 71.33 (#4)
Gemini 3.7 Flash on Vals AI Finance Agent v2: 59.04 (#2)
Gemini 3.7 Flash on Vals AI Harvey Legal Agent Bench: 8.75 (#9)
Gemini 3.7 Flash on Vals AI Legal Research Bench: 34.62 (#18)
Gemini 3.7 Flash on Vals AI ProofBench: 58.0 (#9)
Gemini 3.7 Flash on Vals AI Terminal-Bench 2.1: 77.53 (#7)
Gemini 3.7 Flash on Vals AI Vibe Code Bench: 70.39 (#15)
Gemini 3.7 Flash on VoxelBench: 1893.0 (#9)

New #1 Leaders (61)

Sycophancy (Lechmazur): GPT-5.6 Terra (Thinking, High) (0.0) beat Claude Fable 5 (Thinking, Medium) by 980990.0
RuneBench: Grok 4.6 (12757.0) beat Grok 4.5 by 4041.0
The Compliance Gap: Mistral Small 24B (100.0) beat Llama 3.3 70B by 90.0
Design Arena (Slides): playyy (1303.0) beat Claude 3 Opus by 66.0
OpenEval - Omni-MATH: Qwen 3 235B A22B 2507 Instruct (71.83) beat Gemma 3 27B (IT) by 31.82
OpenEval - GPQA CoT: Gemini 3 Pro (Preview) (80.27) beat qwen-3-80B-instruct by 23.77
ProgramBench Almost: Claude Opus 5 (xHigh) (37.0) beat Claude Opus 4.8 (xHigh) by 20.5
LLM Stats (APEX-Agents): Grok 4.6 (57.5) beat Kimi K3 by 19.9
Coarena - Task Completion: Qwen 3.8 Max (100.0) beat GPT-5.6 Terra by 18.6
CHI-Bench: Claude Opus 5 (54.7) beat Claude Opus 4.8 by 17.4
LLM Stats (AutomationBench): GLM-5.3 (48.2) beat Kimi K3 by 17.4
BridgeBench Arena: Claude Opus 4.7 (1021.5) beat Claude Fable 5 by 16.2
ASCIIBench: Claude Opus 4.5 (1685.0) beat Gemini 3 Pro (Preview) by 16.0
OpenRouter BrowseComp (Search): Claude Opus 5 (High) (89.0) beat GPT-5.6 Sol by 14.0
Chatbot Arena (Video Edit): minimax-h3 (1390.0) beat dreamina-seedance-2.0-720p by 13.0
OpenEval - MMLU-Pro CoT: Gemini 3 Pro (Preview) (90.3) beat qwen-3-80B-instruct by 12.47
APEX-Agents-AA: Claude Fable 5 (Max) (59.2) beat Gemini 3.5 Flash (High) by 12.1
Chatbot Arena (Image-to-Video): minimax-h3 (1489.0) beat dreamina-seedance-2.0-720p by 11.0
FrontierCode: Claude Fable 5 (Max) (63.6) beat Claude Opus 5 by 10.2
Arena AI Document: Claude Opus 5 (High) (1520.0) beat Claude Opus 4.6 by 10.0
Arena AI Code: Claude Opus 5 (Max) (1692.0) beat Kimi K3 by 10.0
MathArena Apex: DeepSeek V4 Pro (Max) (90.2) beat GPT-5.5 (xHigh) by 9.99
LMArena WebDev Arena: Claude Opus 5 (Max) (1691.77) beat Kimi K3 by 9.76
NatureBench: Claude Opus 5 (23.33) beat Claude Opus 4.7 by 7.77
BabyVision: Qwen3.8-Max (with tools) (91.3) beat GPT-5.5 by 7.7
Android Bench: Claude Opus 5 (91.8) beat Claude Fable 5 by 7.3
Design Arena (Game Dev): Kimi K3 (1432.0) beat Claude Opus 5 by 6.0
LLM Stats (NL2Repo): DeepSeek V4 Pro (0813) (61.5) beat Qwen 3.8 Max by 5.6
LLM Stats (MRCR v2 (8-needle)): Gemini 3.7 Flash (97.0) beat GPT-5.6 Sol by 5.5
OpenRouter HLE (Search): Claude Opus 5 (High) (77.4) beat Claude Opus 5 by 4.7
MedCode: Claude Opus 5 (63.57) beat Gemini 3.1 Pro (Preview) (High) by 4.51
AA-LCR: Muse Glimmer 30B (80.0) beat GPT-5.2 Codex (xHigh) by 4.3
AutomationBench: Gemini 3.7 Flash (High) (30.4) beat Claude Opus 5 (Max) by 4.2
PhysicsFinals: Gemini 1.5 Pro (63.9) beat O3 Mini by 4.0
LLM Stats (LVBench): Gemini 3.7 Flash (85.4) beat Qwen 3.8 Max by 3.6
Agent Security League - Security Correctness: Claude Opus 5 (32.4) beat Claude Fable 5 by 3.4
OpenEval - IFEval Strict: Claude 3.7 Sonnet (20250219) (Thinking 10K) (94.0) beat qwen-3-80B-instruct by 2.75
GameCraft-Bench: Claude Opus 5 (68.44) beat Claude Fable 5 by 2.72
SEAL Showdown: GPT-4o Audio (Preview 2025-06-03) (1269.2) beat Gemini 3 Pro (Preview) by 2.7
LLM Stats (PostTrainBench): GLM-5.3 (39.8) beat MiniMax-M3 by 2.7
Agent Arena - Praise vs Complaint: Claude Fable 5 (High) (25.14) beat GPT-5.6 Sol (xHigh) by 2.44
MedScribe: Claude Opus 5 (90.98) beat Muse Spark 1.1 (xHigh) by 2.09
LLM Stats (CyberGym): GLM-5.3 (84.5) beat Gemini 3.5 Flash Cyber by 1.3
LLM Chess (Saplin): GPT-5.6 Sol (xHigh) (1549.7) beat GPT-5.6 Sol (High) by 1.2
LLM Stats (WideSearch): Qwen 3.8 Max (81.9) beat Kimi K2.6 by 1.1
Computer Anthology Terminal Tasks (Gemini CLI): Gemini 3.5 Flash (High) (24.4) beat Gemini 3.5 Flash by 1.0
WritingBench: Qwen 3 235B A22B 2507 (Thinking) (88.3) beat Qwen 3 Next 80B A3B Instruct by 1.0
ComplexFuncBench: GPT-4o (66.5) beat GPT-4.1 by 1.0
ProofBench: Claude Opus 5 (78.0) beat Claude Fable 5 (Max) by 1.0
Creative Writing (Lechmazur): Claude Opus 5 (xHigh) (4.2) beat Claude Fable 5 (High) by 0.9
AA GPQA Diamond: Grok 4.6 (High) (94.95) beat GPT-5.6 Sol (Max) by 0.81
AA MMMU-Pro: Gemini 3.7 Flash (High) (85.49) beat Claude Opus 5 (Adaptive Reasoning, Max Effort) by 0.75
Finance Agent v2: Claude Opus 5 (Max) (58.63) beat Gemini 3.5 Flash by 0.73
SpatialGen-Bench (Text Answering): Qwen 3.8 Max (81.74) beat Claude Fable 5 by 0.48
CursorBench 3.1: Grok 4.6 (xHigh) (70.8) beat Claude Fable 5 (Max) by 0.3
GDPevo: Claude Opus 4.8 (50.63) beat Claude Opus 4.6 by 0.23
OpenEval - CNN/DailyMail: command-xlarge-beta (25.84) beat Llama 2 70B by 0.22
ForecastBench: rice-demon (69.1) beat captain-jack by 0.2
Computer Anthology Terminal Tasks (Terminus-2): Claude Opus 5 (High) (62.0) beat Claude Opus 5 by 0.2
CADGenBench: build123d-mcp-v0381-claude-opus-5-xhigh-full-r6 (0.6391) beat GPT-5.6 Sol (xHigh) by 0.11
CPTU Bench: Muse-Glimmer-30B (Reasoning) (4.42) beat Qwen 3.5 27B (Thinking) by 0.08

The Aggregate Digest — 2026-08-15

2026-08-15T05:12:04.017807+00:00

Frontier

Best available model: Gemini 3.7 Flash (1782) takes the crown from GPT-5.5 Pro (1776)

Daily

New Benchmarks (10)

Agents on Rails (Successful Runs (%)): leader Claude Opus 5 (92.1), 8 models
The Rails Foundation's agentic coding benchmark: 21 atomic Rails tasks against the Writebook app — bug reports, security findings and feature requests written the way they would actually be filed — run three times per model through one frozen minimal harness at each provider's default reasoning effort, and graded by hidden behavioural test suites; the score is the share of runs whose tests pass.
OpenAI GPT-5.6 System Card - HealthBench Professional (Length-Adjusted Score (0-100)): leader GPT-5.6 Sol (60.5), 8 models
HealthBench professional subset for medically challenging, expert-oriented healthcare questions, on the length-adjusted scale; OpenAI's own runs as published in the GPT-5.6 system card.
OpenAI GPT-5.6 System Card - HealthBench (Length-Adjusted Score (0-100)): leader GPT-5 (57.7), 8 models
HealthBench health conversations graded against physician-written rubrics, on the length-adjusted scale the card headlines; OpenAI's own runs as published in the GPT-5.6 system card.
OpenAI GPT-5.6 System Card - HealthBench Hard (Length-Adjusted Score (0-100)): leader GPT-5 (34.7), 8 models
Hard subset of HealthBench, the conversations frontier models still fail, on the length-adjusted scale; OpenAI's own runs as published in the GPT-5.6 system card.
OpenAI GPT-5.6 System Card - HealthBench Consensus (Length-Adjusted Score (0-100)): leader GPT-5.4 (96.3), 8 models
HealthBench Consensus subset — the rubric criteria physicians agreed on — on the length-adjusted scale; OpenAI's own runs as published in the GPT-5.6 system card.
OpenAI GPT-5.6 System Card - AAV Capsid Packaging Prediction (Spearman Correlation): leader GPT-5.6 Sol (0.529), 2 models
Biosecurity capability slice from OpenAI's GPT-5.6 system card — predicting AAV capsid packaging fitness, scored as Spearman correlation against measured values; OpenAI-run frontier comparison.
OpenAI GPT-5.6 System Card - Hard Negative Protein Binding Prediction (pass@4 (%)): leader GPT-5.6 Sol (7.6), 4 models
Biosecurity capability slice from OpenAI's GPT-5.6 system card — hard-negative protein-binding prediction; OpenAI-run frontier comparison.
OpenAI GPT-5.6 System Card - DNA Sequence Design for TF Binding (pass@1 (%)): leader GPT-5.5 Pro (16.5), 4 models
Biosecurity capability slice from OpenAI's GPT-5.6 system card — designing DNA sequences for transcription-factor binding; OpenAI-run frontier comparison.
LLM Stats (AndroidWorld) (Score (%)): leader Qwen 3.8 Max (85.3), 5 models
AndroidWorld as aggregated by LLM Stats — autonomous-agent tasks on a real Android emulator covering app interaction, navigation, and multi-step task completion.
LLM Stats (ExploitBench) (Score (%)): leader Claude Fable 5 (78.0), 5 models
ExploitBench as aggregated by LLM Stats — offensive-security agent tasks built from patched browser-engine vulnerabilities, graded on how far a model gets toward a working exploit.

New Models (2)

GLM-5.3 — ELO 1772
- LLM Stats (AutomationBench): 48.2 (#1/12)
- LLM Stats (CyberGym): 84.5 (#1/13)
- LLM Stats (PostTrainBench): 39.8 (#1/6)
- FrontierSWE: 78.0 (#2/17)
- LLM Stats (NL2Repo): 58.0 (#2/17)
- LLM Stats (FrontierSWE): 78.1 (#3/16)
- LLM Stats (Terminal-Bench 2.1): 88.2 (#3/28)
- LLM Stats (Toolathlon): 73.0 (#4/37)
- Vending-Bench 2: 8163.61 (#6/60)
- LLM Stats (Program Bench): 19.0 (#6/6)
- ...and 3 more
Qwen 3.8 27B — ELO 1717
- LLM Stats (BabyVision): 85.6 (#2/10)
- LLM Stats (MathVision): 94.6 (#2/33)
- LLM Stats (OSWorld-Verified): 84.3 (#3/24)
- LLM Stats (CharXiv-R): 90.2 (#4/51)
- LLM Stats (OmniDocBench 1.5): 91.1 (#4/18)
- LLM Stats (Agents' Last Exam): 42.9 (#5/10)
- LLM Stats (RealWorldQA): 85.9 (#5/29)
- LLM Stats (ERQA): 65.5 (#7/24)
- LLM Stats (NL2Repo): 42.3 (#11/17)
- Humanity's Last Exam (Self-Reported, No Tools): 30.8 (#14/50)
- ...and 4 more

Top-10 New Scores (32)

Claude Opus 5 on Benchmarks.bio - BioSecBench-Refusal: 36.3 (#9)
Claude Opus 5 on Creative Writing (Lechmazur): 4.2 (#1)
GLM-5.3 on FrontierSWE: 78.0 (#2)
GLM-5.3 on LLM Stats (Agents' Last Exam): 28.5 (#7)
GLM-5.3 on LLM Stats (DeepSWE 1.1): 66.9 (#8)
GLM-5.3 on LLM Stats (FrontierSWE): 78.1 (#3)
GLM-5.3 on LLM Stats (NL2Repo): 58.0 (#2)
GLM-5.3 on LLM Stats (Program Bench): 19.0 (#6)
GLM-5.3 on LLM Stats (Terminal-Bench 2.1): 88.2 (#3)
GLM-5.3 on LLM Stats (Toolathlon): 73.0 (#4)
GLM-5.3 on Vending-Bench 2: 8163.61 (#6)
GLM-5.3 on VoxelBench: 1773.0 (#13)
Gemini 3.7 Flash on BenchLM: 61.4 (#50)
Gemini 3.7 Flash on Computer Anthology Terminal Tasks (Terminus-2): 46.0 (#5)
Gemini 3.7 Flash on Creative Writing (Lechmazur): -0.4 (#25)
Gemini 3.7 Flash on Kaggle FACTS Grounding: 75.63 (#5)
Gemini 3.7 Flash on OpenRouter GPQA Diamond: 92.3 (#7)
Gemini 3.7 Flash on OpenRouter Tau2-Bench Airline: 80.6 (#3)
Gemini 3.7 Flash on ProfBench: 57.5 (#8)
Gemini 3.7 Flash on SealedBench: 56.0 (#17)
Gemini 3.7 Flash on SvelteBench: 100.0 (#8)
Gemini 3.7 Flash on Vals AI (Vals Index): 59.31 (#7)
Gemini 3.7 Flash on Vals AI Excel Modeling: 71.33 (#4)
Gemini 3.7 Flash on Vals AI Finance Agent v2: 59.04 (#2)
Gemini 3.7 Flash on Vals AI Harvey Legal Agent Bench: 8.75 (#9)
Gemini 3.7 Flash on Vals AI Legal Research Bench: 34.62 (#18)
Gemini 3.7 Flash on Vals AI ProofBench: 58.0 (#9)
Gemini 3.7 Flash on Vals AI Terminal-Bench 2.1: 77.53 (#7)
Gemini 3.7 Flash on Vals AI Vibe Code Bench: 70.39 (#15)
Kimi K3 on CHI-Bench: 25.3 (#6)
Kimi K3 on Design Arena (SVG): 1340.0 (#4)
Kimi K3 on ProfBench: 57.9 (#7)

New #1 Leaders (6)

Design Arena (Game Dev): Kimi K3 (1436.0) beat Claude Opus 5 by 33.0
CHI-Bench: Claude Opus 5 (54.7) beat Claude Opus 4.8 by 17.4
LLM Stats (AutomationBench): GLM-5.3 (48.2) beat DeepSeek V4 Pro (0813) by 16.4
LLM Stats (PostTrainBench): GLM-5.3 (39.8) beat MiniMax-M3 by 2.7
LLM Stats (CyberGym): GLM-5.3 (84.5) beat DeepSeek V4 Pro (0813) by 1.2
Creative Writing (Lechmazur): Claude Opus 5 (xHigh) (4.2) beat Claude Fable 5 (High) by 0.9

The Aggregate Digest — 2026-08-14

2026-08-14T06:22:09.151361+00:00

Daily

New Benchmarks (1)

LLM Stats (OSWorld 2.0) (Score (%)): leader Claude Opus 5 (70.6), 5 models
llm-stats.com’s mirror of OSWorld 2.0, the computer-use agent benchmark on real desktop applications. Its roster is currently disjoint from the first-party OSWorld 2.0 board — newer models only — and its scale matches the partial-credit column rather than the binary one.

New Models (1)

Gemini 3.7 Flash — ELO 1780, #3
- AA MMMU-Pro: 85.49 (#1/240)
- AutomationBench: 30.4 (#1/10)
- LLM Stats (LVBench): 85.4 (#1/25)
- LLM Stats (MRCR v2 (8-needle)): 97.0 (#1/23)
- Coarena - Task Completion: 80.0 (#2/12)
- AA GPQA Diamond: 94.55 (#2/575)
- LLM Stats (Artificial Analysis): 56.0 (#2/7)
- LLM Stats (GDP.pdf): 34.0 (#2/6)
- RuneBench: 10203.0 (#2/46)
- AI Chess Leaderboard (Reasoning): 1855.0 (#2/313)
- ...and 55 more

Top-10 New Scores (73)

Claude Opus 5 on LLM Chess (Saplin): 1285.4 (#8)
Claude Opus 5 on ProgramBench: 4.5 (#1)
Claude Opus 5 on ProgramBench Almost: 37.0 (#1)
Claude Opus 5 on Vals AI (Vals Index): 67.21 (#1)
Claude Opus 5 on Vals AI Excel Modeling: 73.56 (#2)
GPT-5.6 Sol on Android Bench: 90.8 (#3)
GPT-5.6 Sol on ProgramBench: 0.5 (#6)
GPT-5.6 Sol on ProgramBench Almost: 2.5 (#11)
Gemini 3.7 Flash on AA CritPt: 14.29 (#37)
Gemini 3.7 Flash on AA GDPval: 1525.3 (#24)
Gemini 3.7 Flash on AA GPQA Diamond: 94.55 (#2)
Gemini 3.7 Flash on AA Humanity's Last Exam: 47.87 (#8)
Gemini 3.7 Flash on AA Long Context Reasoning: 81.0 (#4)
Gemini 3.7 Flash on AA MMMU-Pro: 85.49 (#1)
Gemini 3.7 Flash on AA Omniscience: 26.48 (#13)
Gemini 3.7 Flash on AA Omniscience - Business: 45.6 (#18)
Gemini 3.7 Flash on AA Omniscience - Health: 44.7 (#27)
Gemini 3.7 Flash on AA Omniscience - Humanities & Social Sciences: 55.2 (#11)
Gemini 3.7 Flash on AA Omniscience - Law: 62.1 (#3)
Gemini 3.7 Flash on AA Omniscience - Science, Engineering & Mathematics: 52.0 (#18)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE): 72.3 (#26)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - C: 90.0 (#10)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Dart: 68.0 (#21)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Go: 74.0 (#25)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - HTML: 80.0 (#22)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Java: 57.0 (#23)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - JavaScript: 77.27 (#28)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Julia: 64.0 (#30)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Kotlin: 62.0 (#45)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - PHP: 78.0 (#25)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Python: 71.5 (#36)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - R: 60.0 (#35)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Rust: 82.0 (#25)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - Swift: 80.0 (#17)
Gemini 3.7 Flash on AA Omniscience - Software Engineering (SWE) - TypeScript: 76.67 (#27)
Gemini 3.7 Flash on AA SciCode: 57.87 (#5)
Gemini 3.7 Flash on AA-Briefcase: 36.36 (#17)
Gemini 3.7 Flash on AA-Omniscience Accuracy: 55.32 (#16)
Gemini 3.7 Flash on AI Chess Leaderboard (Continuation): 1757.0 (#4)
Gemini 3.7 Flash on AI Chess Leaderboard (Reasoning): 1855.0 (#2)
Gemini 3.7 Flash on Agent Arena: 3.61 (#31)
Gemini 3.7 Flash on Agent Arena - Bash Recovery: 2.33 (#42)
Gemini 3.7 Flash on Agent Arena - Confirmed Success: 9.85 (#11)
Gemini 3.7 Flash on Agent Arena - Praise vs Complaint: 1.84 (#44)
Gemini 3.7 Flash on Agent Arena - Steerability: 2.83 (#38)
Gemini 3.7 Flash on Agent Arena - Tool Hallucination: 1.18 (#26)
Gemini 3.7 Flash on Artificial Analysis Intelligence Index: 56.03 (#17)
Gemini 3.7 Flash on AutomationBench: 30.4 (#1)
Gemini 3.7 Flash on Chatbot Arena (Code): 1588.0 (#9)
Gemini 3.7 Flash on Chatbot Arena (Text): 1490.0 (#9)
Gemini 3.7 Flash on ClockBench: 48.3 (#4)
Gemini 3.7 Flash on Coarena: 1002.3 (#8)
Gemini 3.7 Flash on Coarena - Task Completion: 80.0 (#2)
Gemini 3.7 Flash on Conceptual Reasoning Index: 64.56 (#11)
Gemini 3.7 Flash on Conceptual Reasoning Index - Argument Evaluation (LMCA): 50.38 (#14)
Gemini 3.7 Flash on Conceptual Reasoning Index - Consistency (ACCoRD): 76.98 (#13)
Gemini 3.7 Flash on Conceptual Reasoning Index - Decision Theory (DTBench): 94.67 (#4)
Gemini 3.7 Flash on LLM Stats (Agents' Last Exam): 26.3 (#6)
Gemini 3.7 Flash on LLM Stats (Artificial Analysis): 56.0 (#2)
Gemini 3.7 Flash on LLM Stats (AutomationBench): 30.4 (#3)
Gemini 3.7 Flash on LLM Stats (CharXiv-R): 88.7 (#6)
Gemini 3.7 Flash on LLM Stats (DeepSWE 1.1): 65.3 (#9)
Gemini 3.7 Flash on LLM Stats (FrontierCode 1.1): 43.6 (#5)
Gemini 3.7 Flash on LLM Stats (GDP.pdf): 34.0 (#2)
Gemini 3.7 Flash on LLM Stats (Terminal-Bench 2.1): 85.8 (#6)
Gemini 3.7 Flash on LM Market Cap LMC Score: 40.0 (#238)
Gemini 3.7 Flash on MineBench: 1786.21 (#8)
Gemini 3.7 Flash on RuneBench: 10203.0 (#2)
Gemini 3.7 Flash on SpeechMap Compliance: 85.8 (#60)
Gemini 3.7 Flash on Tau3 Banking: 35.46 (#23)
Gemini 3.7 Flash on VoxelBench: 1892.0 (#10)
Kimi K3 on Android Bench: 90.2 (#4)
Kimi K3 on Vals AI Excel Modeling: 66.4 (#7)

New #1 Leaders (17)

OpenEval - Omni-MATH: Qwen 3 235B A22B 2507 Instruct (71.83) beat Gemma 3 27B (IT) by 31.82
OpenEval - GPQA CoT: Gemini 3 Pro (Preview) (80.27) beat qwen-3-80B-instruct by 23.77
ProgramBench Almost: Claude Opus 5 (xHigh) (37.0) beat Claude Opus 4.8 (xHigh) by 20.5
Chatbot Arena (Video Edit): minimax-h3 (1390.0) beat dreamina-seedance-2.0-720p by 13.0
OpenEval - MMLU-Pro CoT: Gemini 3 Pro (Preview) (90.3) beat qwen-3-80B-instruct by 12.47
NatureBench: Claude Opus 5 (23.33) beat Claude Opus 4.7 by 7.77
Android Bench: Claude Opus 5 (91.8) beat Claude Fable 5 by 7.3
LLM Stats (NL2Repo): DeepSeek V4 Pro (0813) (61.5) beat Qwen 3.8 Max by 5.6
LLM Stats (MRCR v2 (8-needle)): Gemini 3.7 Flash (97.0) beat GPT-5.6 Sol by 5.5
LLM Stats (LVBench): Gemini 3.7 Flash (85.4) beat Qwen 3.8 Max by 3.6
AutomationBench: Gemini 3.7 Flash (High) (30.4) beat Claude Opus 5 (Max) by 3.5
OpenEval - IFEval Strict: Claude 3.7 Sonnet (20250219) (Thinking 10K) (94.0) beat qwen-3-80B-instruct by 2.75
LLM Stats (WideSearch): Qwen 3.8 Max (81.9) beat Kimi K2.6 by 1.1
LLM Stats (AutomationBench): DeepSeek V4 Pro (0813) (31.8) beat Kimi K3 by 1.0
AA MMMU-Pro: Gemini 3.7 Flash (High) (85.49) beat Claude Opus 5 (Adaptive Reasoning, Max Effort) by 0.75
OpenEval - CNN/DailyMail: command-xlarge-beta (25.84) beat Llama 2 70B by 0.22
LLM Stats (CyberGym): DeepSeek V4 Pro (0813) (83.3) beat Gemini 3.5 Flash Cyber by 0.1

The Aggregate Digest — 2026-08-13

2026-08-13T06:35:25.130271+00:00

Daily

New Benchmarks (4)

Conceptual Reasoning Index (Chance-Corrected Score (0-100)): leader Claude Opus 5 (73.57), 134 models
Redwood Research index for topics that cannot be graded against an answer key — AI alignment, collective action under transformative AI: a 60/20/20 blend of argument evaluation, belief consistency and decision theory, scaled so 0 is random guessing and 100 the maximum attainable.
Conceptual Reasoning Index - Argument Evaluation (LMCA) (Chance-Corrected Score (0-100)): leader Claude Opus 5 (63.3), 135 models
The 60% component of the CRI — models rate the strength of conceptual critiques of stated positions over 1,461 rated critiques, scored as the Pearson correlation with expert consensus ratings times 100; the estimated human ceiling is about 85, not 100.
Conceptual Reasoning Index - Consistency (ACCoRD) (Chance-Corrected Score (0-100)): leader Claude Fable 5 (84.95), 134 models
The consistency component of the CRI — models give probabilities for related claims, and 567 logical constraints such as P(A) = P(A|B)P(B) + P(A|¬B)P(¬B) measure how far the stated beliefs drift under reframing; 0-100, higher is more coherent.
Conceptual Reasoning Index - Decision Theory (DTBench) (Chance-Corrected Score (0-100)): leader Claude Fable 5 (97.33), 191 models
The decision-theory component of the CRI — 407 expert-verified Newcomb-like problems where the model reasons about interacting with near-copies of itself; accuracy chance-corrected against the 40% random baseline onto a 0-100 scale.

New Models (3)

Grok 4.6 — ELO 1745, #12
- RuneBench: 12757.0 (#1/44)
- LLM Stats (APEX-Agents): 57.5 (#1/8)
- Mercor APEX: 41.2 (#6/53)
- Vals AI Code Migration: 44.57 (#6/42)
- DeepSWE: 67.5 (#7/27)
- LLM Stats (DeepSWE 1.1): 65.9 (#8/21)
- LLM Stats Score: 45.3 (#21/333)
- LM Market Cap LMC Score: 88.8 (#44/416)
- BenchLM: 62.5 (#45/200)
- SpeechMap Compliance: 83.6 (#74/366)
DeepSeek V4 Pro (0813) — ELO 1724, #20
- NL2Repo: 61.5 (#2/21)
- CyberGym: 83.3 (#2/16)
- Toolathlon: 74.1 (#8/34)
- BenchLM: 60.9 (#48/200)
- LM Market Cap LMC Score: 86.7 (#62/416)
- SpeechMap Compliance: 58.3 (#211/366)
GLM-4.1V-9B — ELO 1446, #504
- Math-VR: 29.0 (#16/31)

Top-10 New Scores (21)

Claude Fable 5 on PLawBench: 70.2 (#3)
Claude Fable 5 on Roboflow Playground Classification: 1200.0 (#13)
Claude Fable 5 on Toolathlon: 77.9 (#2)
Claude Opus 5 on AIIQ Composite IQ: 134.0 (#2)
Claude Opus 5 on MedCode: 63.57 (#1)
Claude Opus 5 on MedScribe: 90.98 (#1)
Claude Opus 5 on ProofBench: 78.0 (#1)
Claude Opus 5 on Vals Index: 74.82 (#2)
Claude Opus 5 on Vals Multimodal Index: 73.9 (#2)
GPT-5.4 Pro on MathArena Apex: 69.79 (#3)
GPT-5.6 Sol on AIIQ Composite IQ: 136.0 (#1)
Kimi K3 on AA-LCR: 74.7 (#5)
Kimi K3 on AIIQ Composite IQ: 122.0 (#19)
Kimi K3 on AutomationBench: 22.7 (#4)
Kimi K3 on CritPt: 23.4 (#9)
Kimi K3 on Finance Agent v2: 54.36 (#7)
Kimi K3 on MedCode: 48.88 (#18)
Kimi K3 on MedScribe: 87.96 (#5)
Kimi K3 on ProofBench: 70.0 (#6)
Kimi K3 on Vals Index: 74.7 (#3)
Kimi K3 on Vals Multimodal Index: 73.42 (#3)

New #1 Leaders (10)

RuneBench: Grok 4.6 (12757.0) beat Grok 4.5 by 4041.0
LLM Stats (APEX-Agents): Grok 4.6 (57.5) beat Kimi K3 by 19.9
Design Arena (SVG): Claude Opus 5 (1381.0) beat GPT-5.6 Sol by 17.0
APEX-Agents-AA: Claude Fable 5 (Max) (59.2) beat Gemini 3.5 Flash (High) by 12.1
FrontierCode: Claude Fable 5 (Max) (63.6) beat Claude Opus 5 by 10.2
PLawBench: Qwen 3.8 Max (73.2) beat GPT-5.2 by 3.53
GameCraft-Bench: Claude Opus 5 (68.44) beat Claude Fable 5 by 2.72
LLM Chess (Saplin): GPT-5.6 Sol (xHigh) (1549.7) beat GPT-5.6 Sol (High) by 1.2
AA GPQA Diamond: Grok 4.6 (High) (94.95) beat GPT-5.6 Sol (Max) by 0.81
CursorBench 3.1: Grok 4.6 Extra High (70.8) beat Claude Fable 5 (Max) by 0.3

The Aggregate Digest — 2026-08-12

2026-08-12T05:29:49.784589+00:00

Daily

New Benchmarks (4)

Chartography (Score (%)): leader GPT-5.6 Sol (Max) (45.0), 40 models
Surge AI professional chart-understanding benchmark covering Kaplan-Meier curves, candlestick charts, contour maps, Sankey diagrams, Bode plots, and other specialist graphics, scoring visual perception, domain-aware interpretation, and multi-step graphical reasoning.
HANDBOOK.md Agents (Score (%)): leader Claude Fable 5 (Adaptive/Max) (36.2), 37 models
Surge AI long-context agentic instruction-following benchmark in which an agent must apply a 100-page company handbook inside a unique RL environment with internal tools and external MCP servers, across five enterprise domains.
ComplexConstraints (Score (%)): leader GPT-5.6 Sol (Max) (50.5), 49 models
Surge AI enterprise instruction-following benchmark whose constraints depend on one another, fire conditionally, and must be inferred from context rather than stated outright.
EnterpriseBench (CoreCraft Agents) (Score (%)): leader Claude Fable 5 (Adaptive/Max) (70.3), 3 models
Surge AI agentic benchmark set in CoreCraft, a large-scale simulated startup, measuring whether agents complete realistic enterprise tasks outside small self-contained environments.

New Models (10)

Claude Haiku 4.5 20251001 ThinkingAnthropic — ELO 1602
- MedScribe: 85.23 (#16/74)
- MGSM: 92.15 (#22/74)
- Vals Multimodal Index: 42.88 (#26/29)
- CaseLaw v2: 56.48 (#30/54)
- Finance Agent v1.1: 46.93 (#34/53)
- Finance Agent v2: 31.01 (#35/40)
- Vals Index: 40.9 (#35/40)
- IOI: 6.17 (#46/64)
- MedCode: 32.68 (#61/75)
Claude Sonnet 4.5 20250929 ThinkingAnthropic — ELO 1682
- MGSM: 94.33 (#4/74)
- Finance Agent v1.1: 54.5 (#18/53)
- CaseLaw v2: 62.16 (#19/54)
- MedScribe: 84.1 (#20/74)
- MedCode: 44.13 (#25/75)
- IOI: 18.33 (#27/64)
- ProofBench: 19.0 (#28/54)
DeepSeek V4 Promax — ELO 1707
- Finance Agent v1.1: 60.39 (#5/53)
- IOI: 35.83 (#17/64)
- Vals Index: 55.62 (#19/40)
- Finance Agent v2: 44.08 (#24/40)
- CaseLaw v2: 59.38 (#27/54)
- MedCode: 40.45 (#40/75)
- ProofBench: 10.0 (#40/54)
- MedScribe: 75.14 (#51/74)
GLGLM 4.7Zhipu AI — ELO 1605
- CaseLaw v2: 54.88 (#37/54)
- Finance Agent v1.1: 45.98 (#38/53)
- IOI: 7.58 (#41/64)
- ProofBench: 6.0 (#45/54)
- MGSM: 88.18 (#53/74)
- MedCode: 32.77 (#60/75)
- MedScribe: 68.63 (#68/74)
Gemini 2.5 ProGoogle — ELO 1665
- LingOly-TOO: 42.3539 (#4/16)
- USAMO25: 24.4 (#5/13)
- MedCode: 50.59 (#12/75)
- CaseLaw v2: 63.88 (#15/54)
- IOI: 17.08 (#29/64)
- Finance Agent v1.1: 41.59 (#42/53)
- HMMT 2025: 82.5 (#43/82)
- MathArena Apex: 0.52 (#45/50)
- MedScribe: 73.55 (#54/74)
Gemini 3 Flash PreviewhighGoogle — ELO 1689
- MedCode: 55.92 (#4/75)
- MGSM: 93.31 (#10/74)
- IOI: 39.08 (#15/64)
- Vals Multimodal Index: 52.19 (#22/29)
- Finance Agent v2: 42.55 (#26/40)
- Vals Index: 49.55 (#28/40)
- CaseLaw v2: 55.84 (#33/54)
- Finance Agent v1.1: 47.6 (#33/53)
- ProofBench: 15.0 (#34/54)
- MedScribe: 69.92 (#65/74)
Gemini 3 Pro PreviewhighGoogle — ELO 1708
- MATH 500: 96.4 (#1/60)
- MGSM: 93.93 (#7/74)
- MedCode: 52.2 (#10/75)
- Finance Agent v1.1: 55.15 (#16/53)
- IOI: 38.83 (#16/64)
- ProofBench: 20.0 (#26/54)
- CaseLaw v2: 53.05 (#42/54)
- MedScribe: 72.04 (#61/74)
KIkimi-k2.5 — ELO 1689
- Hack-Verifiable TextArena: 19.0 (#6/12)
- TutorBench: 54.56 (#6/27)
- MultiNRC: 35.17 (#20/43)
- EnigmaEval: 3.38 (#25/43)
- Visual-Language Understanding: 41.86 (#36/58)
- ALE-Bench: 821.65 (#36/89)
- AIIQ Composite IQ: 111.0 (#43/115)
KIkimi-k2.6 — ELO 1714
- Personalized Turn-Level User Conversation Sati: 4.74 (#1/7)
- RankJudge: 1713.0 (#4/21)
- MathConstraint: 35.9 (#8/12)
- From Knowing to Doing: -24.23 (#10/10)
- Arena AI Document: 1451.0 (#21/38)
- ALE-Bench: 1092.67 (#21/89)
- AIIQ Composite IQ: 119.0 (#22/115)
- Arena AI Code: 1509.0 (#27/48)
Muse Glimmer-30B — ELO 1663
- LLM Stats (AIME 2026): 94.7 (#5/18)
- LLM Stats (ScreenSpot Pro): 75.4 (#6/25)
- LLM Stats (SkillsBench): 44.3 (#7/8)
- LLM Stats (DeepSearchQA): 74.6 (#9/9)
- LLM Stats (MCP Atlas): 75.5 (#12/32)
- LLM Stats (OmniDocBench 1.5): 75.8 (#13/17)
- LLM Stats (OSWorld-Verified): 65.9 (#17/23)
- LLM Stats (Terminal-Bench 2.1): 51.7 (#19/19)
- LLM Stats (CharXiv-R): 78.8 (#22/48)
- LLM Stats Score: 35.64 (#65/326)

Top-10 New Scores (32)

Claude Fable 5 on ArxivMath: 87.5 (#2)
GPT-5.4 Pro on MathArena Apex: 69.79 (#3)
GPT-5.5 Pro on PM-LLM-Benchmark: 37.7 (#9)
GPT-5.6 on AIIQ Composite IQ: 136.0 (#1)
GPT-5.6 on AIIQ Composite IQ: 136.0 (#1)
GPT-5.6 on AIIQ Composite IQ: 136.0 (#1)
GPT-5.6 on APEX-Agents: 39.9 (#12)
GPT-5.6 on CharXiv-R: 84.6 (#12)
GPT-5.6 on Finance Agent v2: 55.04 (#6)
GPT-5.6 on Finance Agent v2: 55.04 (#6)
GPT-5.6 on Finance Agent v2: 55.04 (#6)
GPT-5.6 on IOI: 86.67 (#6)
GPT-5.6 on IOI: 86.67 (#6)
GPT-5.6 on IOI: 86.67 (#6)
GPT-5.6 on MCP Atlas: 83.6 (#6)
GPT-5.6 on MedCode: 43.97 (#26)
GPT-5.6 on MedCode: 43.97 (#26)
GPT-5.6 on MedCode: 43.97 (#26)
GPT-5.6 on MedScribe: 85.23 (#15)
GPT-5.6 on MedScribe: 85.23 (#15)
GPT-5.6 on MedScribe: 85.23 (#15)
GPT-5.6 on OmniDocBench 1.5: 85.8 (#41)
GPT-5.6 on ProofBench: 77.0 (#2)
GPT-5.6 on ProofBench: 77.0 (#2)
GPT-5.6 on ProofBench: 77.0 (#2)
GPT-5.6 on Toolathlon: 74.9 (#6)
GPT-5.6 on Vals Index: 73.12 (#4)
GPT-5.6 on Vals Index: 73.12 (#4)
GPT-5.6 on Vals Index: 73.12 (#4)
GPT-5.6 on Vals Multimodal Index: 72.19 (#4)
GPT-5.6 on Vals Multimodal Index: 72.19 (#4)
GPT-5.6 on Vals Multimodal Index: 72.19 (#4)

New #1 Leaders (10)

OpenRouter BrowseComp (Search): Claude Opus 5 · high (89.0) beat GPT-5.6 Sol by 14.0
Arena AI Code: Claude Opus 5 (Max) (1692.0) beat Kimi K3 by 10.0
LMArena WebDev Arena: Claude Opus 5 (Max) (1691.77) beat Kimi K3 by 9.76
OpenRouter HLE (Search): Claude Opus 5 · high (77.4) beat Claude Opus 5 by 4.7
AA-LCR: Muse Glimmer 30B (80.0) beat GPT-5.2 Codex (xHigh) by 4.3
Computer Anthology Terminal Tasks (Gemini CLI): Gemini 3.5 Flash (High) (24.4) beat Gemini 3.5 Flash by 1.0
ComplexFuncBench: GPT-4o (66.5) beat GPT-4.1 by 1.0
ProgramBench: GPT-5.6 Sol (xHigh) (1.0) beat GPT-5.5 (xHigh) by 0.5
Computer Anthology Terminal Tasks (Terminus-2): Claude Opus 5 (High) (62.0) beat Claude Opus 5 by 0.2
CPTU Bench: Muse-Glimmer-30B reasoning (API, FP8) (4.42) beat Qwen3.5-27B thinking (API) by 0.08

The Aggregate Digest — 2026-08-11

2026-08-11T05:10:50.566795+00:00

Daily

New Benchmarks (1)

Humanity's Last Exam (Self-Reported, No Tools) (Accuracy (%)): leader Kimi K3 (56.0), 44 models
PhD-level questions at the frontier of human expert knowledge. Lab-submitted scores from the official cais/hle dataset board on Hugging Face (model-card numbers, none independently verified), No Tools configuration; mostly open-weights models.

Top-10 New Scores (6)

Claude Fable 5 on GDP.pdf: 34.2 (#9)
Claude Fable 5 on Riemann-bench: 34.2 (#6)
Claude Opus 5 on GDP.pdf: 29.6 (#20)
Claude Opus 5 on Riemann-bench: 29.6 (#15)
Claude Opus 5 on TrackingAI IQ Test (Vision): 88.24 (#4)
GPT-5.6 Sol on Riemann-bench: 39.5 (#2)

New #1 Leaders (1)

Design Arena (Slides): playyy (1310.0) beat manus by 67.0

The Aggregate Digest — 2026-08-10

2026-08-10T05:23:34.260300+00:00

Daily

Top-10 New Scores (13)

Claude Opus 5 on AIIQ Composite IQ: 134.0 (#5)
Claude Opus 5 on Vals Index: 74.82 (#2)
Claude Opus 5 on Vals Multimodal Index: 73.9 (#2)
Kimi K3 on AIIQ Composite IQ: 122.0 (#26)
Kimi K3 on Finance Agent v2: 54.36 (#8)
Kimi K3 on LMArena Text Arena: 1485.33 (#11)
Kimi K3 on MedCode: 48.88 (#18)
Kimi K3 on MedScribe: 87.96 (#5)
Kimi K3 on ProofBench: 70.0 (#6)
Kimi K3 on Vals Index: 74.7 (#3)
Kimi K3 on Vals Multimodal Index: 73.42 (#3)
Muse Spark 1.2 on Roboflow Playground Object Detection: 1215.0 (#16)
Muse Spark 1.2 on Roboflow Playground Overall: 1215.0 (#16)

New #1 Leaders (6)

Arena AI Document: Claude Opus 5 (High) (1520.0) beat Claude Opus 4.6 by 10.0
MedCode: Claude Opus 5 (63.57) beat Gemini 3.1 Pro (Preview) (High) by 4.51
MedScribe: Claude Opus 5 (90.98) beat Muse Spark 1.1 (xHigh) by 2.09
ProofBench: Claude Opus 5 (78.0) beat Claude Fable 5 (Max) by 1.0
Finance Agent v2: Claude Opus 5 (Max) (58.63) beat Gemini 3.5 Flash by 0.73
SpatialGen-Bench (Text Answering): Qwen 3.8 Max (81.74) beat Claude Fable 5 by 0.48

The Aggregate Digest — 2026-08-09

2026-08-09T06:48:29.141755+00:00

Daily

New Benchmarks (4)

Coarena (Elo (Bradley-Terry)): leader Claude Fable 5 (1079.1), 13 models
Coarena - Task Completion (Completion Rate (%)): leader GPT-5.6 Terra (81.4), 13 models
Featherbench (Pass Rate (%)): leader Gemini 3.6 Flash (96.0), 13 models
Featherbench - Rubric Quality (Rubric Score (0-10)): leader Kimi K3 (9.5), 13 models

Weekly

New Benchmarks (4)

Coarena (Elo (Bradley-Terry)): leader Claude Fable 5 (1079.1), 13 models
Coarena - Task Completion (Completion Rate (%)): leader GPT-5.6 Terra (81.4), 13 models
Featherbench (Pass Rate (%)): leader Gemini 3.6 Flash (96.0), 13 models
Featherbench - Rubric Quality (Rubric Score (0-10)): leader Kimi K3 (9.5), 13 models

New Models (470)

GPT-5.5 Pro — ELO 1778, #1
- TaxBench: 29.27 (#1/16)
- GDPval-MM: 82.3 (#2/14)
- LM Market Cap LMC Score: 92.7 (#11/406)
- BLXBench: 44.5 (#14/25)
Claude Opus 5 — ELO 1776, #2
- FrontierCode: 53.4 (#1/21)
- Harvey Legal Agent Benchmark: 23.58 (#1/12)
- Toolathlon: 80.6 (#1/39)
- ArxivMath: 91.33 (#1/20)
- ProteinGym Hard: 47.7 (#1/7)
- InferenceBench: 8.9 (#1/29)
- Design Arena (Game Dev): 1426.0 (#1/150)
- MCP Atlas: 85.8 (#2/42)
- Tau3-Bench Banking_Knowledge: 48.7 (#2/28)
- INCLUDE: 89.8 (#4/16)
Claude Fable 5 — ELO 1774, #3
- CAIS Risk Index: 27.3 (#1/45)
- EQ-Bench: 2049.7 (#1/79)
- MirrorCode: 63.9 (#1/4)
- SpacetimeDB LLM Benchmark (C#): 98.9 (#1/13)
- Crosby micro1 RedlineBench: 47.3 (#2/5)
- Creative Writing v3: 2156.3 (#2/123)
- Arena AI Document: 1505.0 (#2/27)
- Arena AI Code: 1630.0 (#2/42)
- LMArena WebDev Arena: 1629.56 (#2/22)
- OSWorld-Verified: 85.0 (#2/30)
GPT-5.6 Sol — ELO 1766, #4
- Creative Writing v3: 2208.0 (#1/123)
- Graphwalks BFS 1M F1: 83.4 (#1/10)
- Graphwalks BFS 256k F1: 95.4 (#1/10)
- SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#1/13)
- PLawBench: 72.3 (#2/29)
- MirrorCode: 20.0 (#2/4)
- Korean LLM Eval - CLIcK: 95.5 (#2/8)
- Korean LLM Eval - HAE-RAE Bench: 94.64 (#2/8)
- Korean LLM Eval - KMMLU-HARD: 85.04 (#2/8)
- SpacetimeDB LLM Benchmark (Rust): 97.8 (#3/13)
GPT-5.4 Pro (xHigh) — ELO 1758, #5
- EnigmaEval: 23.82 (#1/39)
- BilliardPhys-Bench: 72.29 (#2/13)
- MathArena Apex: 69.79 (#2/51)
- TutorBench: 56.62 (#2/23)
- TaxBench: 27.03 (#2/16)
- MultiNRC: 62.27 (#3/39)
- Visual-Language Understanding: 53.89 (#3/54)
- CritPt: 30.0 (#3/342)
- ARC-AGI-2: 83.33 (#15/202)
- LM Market Cap LMC Score: 91.9 (#18/406)
Kimi K3 — ELO 1755, #6
- Arena AI Code: 1682.0 (#1/42)
- LMArena WebDev Arena: 1682.01 (#1/22)
- Epoch AI - Scicode: 58.68 (#3/128)
- GameCraft-Bench: 56.96 (#3/11)
- AA-LCR: 74.7 (#4/355)
- TrackingAI IQ Test: 93.75 (#4/39)
- TrackingAI IQ Test (Vision): 88.0 (#4/25)
- SpacetimeDB LLM Benchmark (C#): 93.3 (#9/13)
- CritPt: 23.4 (#10/342)
- SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#10/13)
Qwen 3.8 Max — ELO 1755, #7
- PLawBench: 73.2 (#1/29)
- Tau3 Banking: 51.34 (#1/188)
- Tau3-Bench Banking_Knowledge: 55.2 (#1/28)
- LLM Stats (ERQA): 77.8 (#1/23)
- LLM Stats (HealthBench): 60.2 (#1/9)
- LLM Stats (LongBench v2): 66.3 (#1/17)
- LLM Stats (LVBench): 81.8 (#1/24)
- LLM Stats (NL2Repo): 55.9 (#1/14)
- LLM Stats (OSWorld-Verified): 86.1 (#1/22)
- LLM Stats (RealWorldQA): 88.0 (#1/26)
Muse Spark 1.2 — ELO 1750, #8
- ErdosBench: 2.73 (#3/9)
- Design Arena (Data Viz): 1361.0 (#3/147)
- Design Arena (Game Dev): 1375.0 (#4/150)
- Design Arena (UI Components): 1368.0 (#4/145)
- Design Arena (Website): 1331.0 (#5/164)
- Design Arena (3D): 1364.0 (#5/140)
- Vals AI (Vals Index): 71.88 (#5/45)
- Vals AI Multimodal Index: 69.8 (#6/32)
- DeepSWE: 54.9 (#14/25)
- MineBench: 1505.11 (#21/58)
GPT-5.5 — ELO 1747, #9
- CombEval: 93.6 (#1/11)
- Crosby micro1 RedlineBench: 50.5 (#1/5)
- MUSE: 67.14 (#1/15)
- MathConstraint: 66.9 (#1/12)
- GDPval-MM: 84.9 (#1/14)
- AIIQ Composite IQ: 136.0 (#1/138)
- From Knowing to Doing: 61.26 (#2/10)
- DiscoverPhysics: 59.0 (#2/11)
- RankJudge: 1867.0 (#2/21)
- JuICE: 49.67 (#3/10)
Gemini 3.6 Flash — ELO 1743, #10
- MRCR-v2 128k: 91.8 (#1/18)
- ProphetArena: 0.9638 (#1/23)
- MMOU: 82.5 (#2/25)
- Icelandic LLM - WikiQA-IS: 62.58 (#6/93)
- OSWorld-Verified: 83.0 (#7/30)
- Icelandic LLM - WinoGrande-IS: 95.5 (#7/93)
- Icelandic LLM - Belebele-IS: 94.11 (#7/93)
- Icelandic LLM Leaderboard - Average: 86.68 (#8/93)
- Icelandic LLM - ARC-Challenge-IS: 94.88 (#8/93)
- Icelandic LLM - GED: 76.5 (#9/93)
Muse Spark 1.1 — ELO 1743, #11
- CAIS Risk Index: 34.2 (#2/45)
- Toolathlon: 75.6 (#6/39)
- Tau3-Bench Banking_Knowledge: 40.5 (#6/28)
- Epoch AI - Vending Bench 2: 6520.47 (#9/57)
- CAIS Vision Capabilities Index: 57.5 (#10/35)
- Arena AI Code: 1536.0 (#11/42)
- LMArena WebDev Arena: 1536.32 (#11/22)
- AIIQ Composite IQ: 116.0 (#36/138)
- Roboflow Playground Captioning: 1188.0 (#39/48)
Qwen 3.7 Max — ELO 1737, #12
- HMMT February 2026: 97.1 (#1/25)
- IMO-AnswerBench: 90.0 (#3/22)
- ITBench-AA: 42.5 (#3/23)
- MRCR-v2 128k: 90.4 (#3/18)
- ResearchClawBench: 18.7 (#4/20)
- MathArena Apex: 44.5 (#5/51)
- When Safe Skills Collide: 1.0 (#7/9)
- NL2Repo: 47.2 (#7/20)
- MLS-Bench Lite: 31.7 (#9/10)
- INCLUDE: 86.2 (#10/16)
Gemini 3.1 Pro (Preview) — ELO 1736, #13
- AGORA: 59.39 (#1/8)
- AgentCIBench: 98.3 (#1/15)
- MMGist: 66.8 (#1/27)
- PlanBench-XL: 77.06 (#1/10)
- Trip+: 73.31 (#1/18)
- LaViSA: 88.9 (#1/10)
- GRACE: 81.46 (#1/13)
- SpatialAct: 20.6 (#1/7)
- NICE: 78.1 (#1/6)
- IFMTBench: 89.08 (#1/15)
Claude Opus 4.8 — ELO 1731, #14
- Agentic Skills Evaluation Framework: 92.7 (#1/19)
- MyPCBench: 62.0 (#1/11)
- CAIS Text Capabilities Index: 53.8 (#2/40)
- EQ-Bench: 2029.8 (#2/79)
- CAIS Risk Index: 39.6 (#4/45)
- Crosby micro1 RedlineBench: 44.4 (#5/5)
- Creative Writing v3: 1943.4 (#5/123)
- Harvey Legal Agent Benchmark: 10.4 (#5/12)
- PLawBench: 69.6 (#5/29)
- ROSE: 24.3 (#6/10)
Gemini 3.5 Flash — ELO 1730, #15
- SearchGEO: 25.8 (#2/13)
- PlanBench-XL: 52.19 (#3/10)
- MMOU: 81.9 (#3/25)
- Crosby micro1 RedlineBench: 45.1 (#4/5)
- ZeroBench: 22.0 (#6/69)
- MRCR-v2 128k: 77.3 (#8/18)
- ResearchClawBench: 17.9 (#9/20)
- ArxivMath: 44.17 (#18/20)
- Arena AI Code: 1493.0 (#21/42)
- AIIQ Composite IQ: 122.0 (#23/138)
Grok 4.5 — ELO 1728, #16
- SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#2/13)
- SpacetimeDB LLM Benchmark (Rust): 97.8 (#2/13)
- Tau3-Bench Banking_Knowledge: 47.9 (#3/28)
- SpacetimeDB LLM Benchmark (C#): 96.6 (#6/13)
- Arena AI Code: 1550.0 (#7/42)
- LMArena WebDev Arena: 1550.2 (#7/22)
- CAIS Vision Capabilities Index: 60.8 (#7/35)
- MRCR-v2 128k: 81.4 (#7/18)
- CAIS Risk Index: 52.2 (#17/45)
- CharXiv-R: 81.6 (#22/60)
GPT-5.6 Terra — ELO 1727, #17
- SpacetimeDB LLM Benchmark (Rust): 100.0 (#1/13)
- Graphwalks BFS 256k F1: 94.9 (#2/10)
- SpacetimeDB LLM Benchmark (C#): 98.9 (#2/13)
- Graphwalks BFS 1M F1: 75.5 (#3/10)
- Korean LLM Eval - CLIcK: 92.24 (#3/8)
- Korean LLM Eval - KMMLU-HARD: 77.86 (#3/8)
- Korean LLM Eval - HAE-RAE Bench: 92.85 (#5/8)
- Rogo Big Finance Bench: 51.0 (#6/14)
- Creative Writing v3: 1927.6 (#7/123)
- SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#9/13)
Seed 2.1 Pro — ELO 1727, #18
- LVBench: 78.0 (#1/49)
- NL2Repo: 47.0 (#8/20)
- CharXiv-R: 86.4 (#8/60)
- CyberGym: 70.2 (#9/15)
- MathArena Apex: 31.3 (#12/51)
- Arena AI Code: 1528.0 (#12/42)
- LMArena WebDev Arena: 1528.27 (#12/22)
Gemini 3 Pro — ELO 1718, #19
- LiveK12Bench: 88.3 (#1/12)
- WildRoadBench: 42.1 (#1/25)
- BacktestBench: 67.41 (#1/23)
- PaliBench: 94.6 (#1/10)
- PrivacySIM: 40.39 (#1/6)
- SpatialBabel: 83.4 (#1/14)
- Omni-DeepSearch: 43.44 (#1/12)
- Soohak: 44.12 (#1/11)
- DiffCap-Bench: 83.1 (#1/16)
- ADBench: 83.0 (#1/10)
GPT-5.2 Pro — ELO 1717, #20
- HMMT 2025: 100.0 (#1/89)
- DEEPSYNTH: 8.7 (#2/11)
- BilliardPhys-Bench: 62.34 (#3/13)
- TaxBench: 17.77 (#7/16)
- LM Market Cap LMC Score: 90.5 (#27/406)
Gemini 3 Pro (Preview) — ELO 1716, #21
- LongWebBench: 7.22 (#1/13)
- Checkup2Action: 60.2 (#1/8)
- Magis-Bench: 6.97 (#1/23)
- SciPredict: 25.27 (#1/15)
- PlaceboBench: 73.91 (#1/7)
- ATC Safety-Oriented Language Understanding Eval: 68.84 (#2/11)
- Omi SOAP Note Safety Benchmark: 4.7 (#2/6)
- MechVQA: 77.28 (#3/17)
- EnigmaEval: 18.24 (#4/39)
- VTB: 26.85 (#4/17)
GPT-5.4 — ELO 1715, #22
- BehaviorBench: 31.4 (#1/20)
- Geo-Eval: 7.15 (#1/11)
- Vision2Code: 4.12 (#1/11)
- TriBench-Ko: 83.5 (#1/13)
- VTB: 29.17 (#1/17)
- DiagnosticIQ: 73.27 (#2/33)
- AIIQ Composite IQ: 134.0 (#2/138)
- SmellBench: 42.8 (#3/9)
- MirrorCode: 15.6 (#3/4)
- FAM-Bench: 82.4 (#4/21)
Claude Opus 4.7 — ELO 1711, #23
- DiscoverPhysics: 61.0 (#1/11)
- EvoCode-Bench: 54.0 (#1/13)
- RoadmapBench: 39.1 (#1/13)
- Are LLMs Ready to Assist Physicians? PhysAssis: 68.3 (#2/14)
- ResearchClawBench: 20.7 (#2/20)
- IntentGrasp: 57.31 (#2/20)
- Causal Sensitivity Score (CSS): 43.2 (#3/6)
- MUSE: 53.46 (#3/15)
- ProactBench: 51.9 (#3/16)
- Creative Writing v3: 2083.1 (#3/123)
Seed 2.1 Turbo — ELO 1710, #24
- LVBench: 76.8 (#3/49)
- MathArena Apex: 35.4 (#8/51)
- NL2Repo: 43.7 (#9/20)
- CyberGym: 67.0 (#11/15)
- CharXiv-R: 83.6 (#18/60)
Claude Sonnet 5 — ELO 1707, #25
- SpacetimeDB LLM Benchmark (C#): 97.8 (#5/13)
- SpacetimeDB LLM Benchmark (TypeScript): 98.9 (#6/13)
- SpacetimeDB LLM Benchmark (Rust): 96.6 (#7/13)
- InferenceBench: 6.43 (#9/29)
- Creative Writing v3: 1827.4 (#11/123)
- MRCR-v2 128k: 71.6 (#11/18)
- TrackingAI IQ Test (Mensa Norway): 80.0 (#17/38)
- Epoch AI - Mystery Game Puzzles: 16.0 (#37/47)
GPT-5 Pro — ELO 1707, #26
- MultiNRC: 65.2 (#1/39)
- EnigmaEval: 18.75 (#3/39)
- Visual-Language Understanding: 52.39 (#4/54)
- LiveBench: 71.29 (#29/43)
- LM Market Cap LMC Score: 88.7 (#51/406)
GLM-5.2 — ELO 1706, #27
- Spider 2.0-Lite: 76.23 (#1/37)
- IMO-AnswerBench: 91.0 (#2/22)
- Crosby micro1 RedlineBench: 45.7 (#3/5)
- CAIS Risk Index: 42.4 (#6/45)
- EQ-Bench: 1575.1 (#8/79)
- HMMT February 2026: 92.5 (#8/25)
- FrontierCode: 24.5 (#12/21)
- ArxivMath: 56.67 (#13/20)
- Creative Writing v3: 1745.8 (#15/123)
- HMMT 2025: 94.4 (#20/89)
GPT-5.6 Luna — ELO 1706, #28
- Graphwalks BFS 256k F1: 92.5 (#3/10)
- Korean LLM Eval - HAE-RAE Bench: 93.17 (#3/8)
- Korean LLM Eval - CLIcK: 91.53 (#5/8)
- Korean LLM Eval - KMMLU-HARD: 75.76 (#5/8)
- Creative Writing v3: 1932.2 (#6/123)
- Graphwalks BFS 1M F1: 56.8 (#7/10)
- MRCR-v2 128k: 74.8 (#9/18)
- Toolathlon: 74.6 (#11/39)
- CAIS Vision Capabilities Index: 55.5 (#12/35)
- Rogo Big Finance Bench: 36.0 (#12/14)
GPT-5.2 Codex — ELO 1706, #29
- MolViBench: 33.2 (#8/13)
- LiveBench: 74.33 (#19/43)
- Mercor APEX: 27.6 (#20/53)
Qwen 3.6 Max Preview — ELO 1705, #30
- Qwen-AgentWorld: 52.42 (#11/16)
- Arena AI Code: 1478.0 (#23/42)
- AA-LCR: 69.7 (#26/355)
- ClawProBench: 57.4 (#26/57)
- Finance Agent v1.1: 52.78 (#28/56)
- CaseLaw v2: 47.91 (#51/54)
- CritPt: 3.7 (#56/342)
Claude Opus 4.6 — ELO 1704, #31
- EComAgentBench: 57.1 (#1/7)
- MineExplorer: 41.08 (#1/18)
- VitaBench 2.0: 50.3 (#1/35)
- HIDBench: 60.2 (#1/9)
- STT-Arena: 35.39 (#1/23)
- TraceEval: 72.9 (#1/10)
- DiagnosticIQ: 73.59 (#1/33)
- Arena AI Document: 1510.0 (#1/27)
- JamSet/JamBench: 42.0 (#2/9)
- MedCTA: 31.32 (#2/18)
Gemini 3 Flash (Preview) — ELO 1704, #32
- CulMind: 50.7 (#1/14)
- ATC Safety-Oriented Language Understanding Eval: 69.22 (#1/11)
- JuICE: 52.14 (#2/10)
- GlobalDentBench: 61.59 (#2/12)
- CPCD-Bench: 4.46 (#2/14)
- CiteVQA: 45.4 (#2/20)
- Magis-Bench: 6.67 (#2/23)
- LongWebBench: 6.53 (#3/13)
- BilliardPhys-Bench: 58.06 (#4/13)
- PlaceboBench: 44.93 (#5/7)
GPT-5.3 Codex — ELO 1704, #33
- Time to REFLECT: 47.5 (#1/14)
- Verus-SpecBench: 57.8 (#2/6)
- AgentTrap: 55.0 (#6/8)
- Boiling the Frog: 23.1 (#7/9)
- SWE Atlas: 32.6 (#7/15)
- LiveSQLBench: 33.33 (#8/33)
- gwBenchmarks: 17.8 (#9/12)
- MolViBench: 32.7 (#9/13)
- OpenSkillEval: 3.76 (#10/10)
- Mercor APEX: 31.8 (#18/53)
Gemini 3 Flash — ELO 1701, #34
- OCR-Robust: 80.71 (#1/16)
- P3B3: 99.8 (#1/20)
- SearchGEO: 31.4 (#1/13)
- Scammer4U: 93.1 (#1/4)
- ERGeoBench: 8.83 (#1/9)
- A Matter of TASTE: 82.0 (#1/6)
- Plant, Persist, Trigger: 51.1 (#1/7)
- StreamProfileBench: 54.97 (#1/14)
- ChartFI: 3.49 (#1/5)
- Perception or Prejudice: 33.5 (#1/27)
Grok Build 0.1 — ELO 1700, #35
- CritPt: 9.1 (#30/342)
- LiveBench: 69.59 (#35/43)
- Epoch AI - Scicode: 50.23 (#35/128)
- Epoch AI - Critpt: 9.14 (#47/138)
- AA-LCR: 64.7 (#71/355)
Qwen 3.7 Plus — ELO 1699, #36
- LVBench: 76.2 (#4/49)
- IMO-AnswerBench: 86.0 (#7/22)
- OmniDocBench 1.5: 91.4 (#9/64)
- CharXiv-R: 85.9 (#9/60)
- NL2Repo: 41.1 (#13/20)
- Arena AI Document: 1442.0 (#16/27)
- Vals Multimodal Index: 53.89 (#16/26)
- OSWorld-Verified: 73.3 (#18/30)
- Vals Index: 52.33 (#20/35)
- Finance Agent v2: 38.22 (#28/39)
Kimi K2.7 Code — ELO 1698, #37
- FrontierCode: 30.1 (#9/21)
- InferenceBench: 6.27 (#10/29)
- Arena AI Code: 1474.0 (#25/42)
- CritPt: 10.0 (#27/342)
- AIIQ Composite IQ: 117.0 (#34/138)
- Epoch AI - Scicode: 47.45 (#49/128)
- AA-LCR: 66.3 (#54/355)
- LM Market Cap LMC Score: 54.1 (#222/406)
Gemini 2.5 Pro (Preview 06-05) — ELO 1694, #38
- Visual-Language Understanding: 54.63 (#2/54)
- Balrog: 43.3 (#2/16)
- CadEval: 64.0 (#2/10)
- TutorBench: 55.65 (#3/23)
- VTB: 11.75 (#7/17)
- MultiNRC: 45.12 (#13/39)
- EnigmaEval: 5.57 (#19/39)
- EQ-Bench: 1545.0 (#19/79)
- Creative Writing v3: 1391.7 (#58/123)
Nex N2 Pro — ELO 1693, #39
- CritPt: 8.6 (#34/342)
- AA-LCR: 67.7 (#42/355)
- AIIQ Composite IQ: 110.0 (#59/138)
GPT-5.2 — ELO 1692, #40
- TriggerBench: 90.88 (#1/14)
- EHRBench: 70.91 (#1/31)
- MolDeTox: 15.59 (#1/15)
- Omi SOAP Note Safety Benchmark: 4.72 (#1/6)
- Poker Agent: 1131.83 (#1/17)
- C3-Bench: 5.72 (#2/33)
- Polaris-Bench: 77.4 (#2/15)
- DiffCap-Bench: 75.1 (#2/16)
- IDE-Bench: 85.0 (#2/15)
- PlaceboBench: 63.24 (#2/7)
DeepSeek V4 Flash (0731) — ELO 1692, #41
- LLM Stats (NL2Repo): 54.2 (#2/14)
- LLM Stats (AutomationBench): 25.1 (#3/8)
- LLM Stats (Toolathlon): 70.3 (#3/31)
- Korean LLM Eval - HAE-RAE Bench: 92.98 (#4/8)
- LLM Stats (CyberGym): 76.7 (#5/11)
- LLM Stats (DeepSWE): 54.4 (#5/10)
- Korean LLM Eval - CLIcK: 90.78 (#6/8)
- Korean LLM Eval - KMMLU-HARD: 74.2 (#6/8)
- KernelBench Hub - CUDA: 14.53 (#7/8)
- LLM Stats (Terminal-Bench 2.1): 82.7 (#8/17)
GPT-5 — ELO 1691, #42
- GraphARC: 91.0 (#1/11)
- Fin-RATE: 43.37 (#1/16)
- Seneca-TRBench: 93.5 (#1/5)
- IneqMath: 47.0 (#1/42)
- AfroBench-Lite: 77.74 (#1/24)
- TW-LegalBench: 76.7 (#2/13)
- EHRBench: 69.06 (#3/31)
- Xent Games: 62.77 (#3/12)
- Causal Sensitivity Score (CSS): 42.7 (#4/6)
- TutorBench: 55.33 (#4/23)
Seed 2.0 Pro — ELO 1691, #43
- VisualNeedle: 36.8 (#3/9)
- Trip+: 64.74 (#7/18)
- BilliardPhys-Bench: 48.46 (#7/13)
- GlobalDentBench: 55.77 (#7/12)
- MedBench v5: 92.44 (#9/10)
- MineExplorer: 15.5 (#9/18)
- SGR-Bench: 29.88 (#10/11)
- ClawProBench: 61.07 (#10/57)
- RoadmapBench: 5.2 (#13/13)
- SuiChat-CN: 59.5 (#29/44)
Kimi K2.6 — ELO 1690, #44
- Personalized Turn-Level User Conversation Sati: 4.74 (#1/7)
- ContextEcho: 100.0 (#1/23)
- Greenland Sovereignty Game (implicitly as a st: 22.0 (#1/8)
- OJBench: 60.6 (#1/44)
- IOI: 585.0 (#1/58)
- MedBench v5: 96.49 (#2/10)
- Trip+: 67.5 (#3/18)
- EComAgentBench: 46.4 (#3/7)
- Cookie-Bench: 78.3 (#3/13)
- OpenComputer: 58.8 (#3/8)
Claude Opus 4.5 — ELO 1690, #45
- The Metanym Game: 7.0 (#1/12)
- MineExplorer: 27.31 (#3/18)
- Multi-domain Multi-modal Document Classificati: 61.59 (#3/15)
- Magis-Bench: 6.46 (#3/23)
- From Perception to Action: 15.6 (#3/16)
- Omi SOAP Note Safety Benchmark: 4.54 (#5/6)
- Soohak: 18.82 (#7/11)
- CAIS Risk Index: 42.7 (#7/45)
- VerdictBench: 59.1 (#8/17)
- CAIS Text Capabilities Index: 36.6 (#9/40)
DeepSeek V4 Pro — ELO 1686, #46
- GroupTravelBench: 10.47 (#1/9)
- CAM-Bench: 19.67 (#1/5)
- Beyond the All-in-One Agent: 62.0 (#1/12)
- When Safe Skills Collide: 9.0 (#2/9)
- HardMTBench: 90.91 (#2/20)
- LITMUS: 69.8 (#2/6)
- ClawProBench: 64.38 (#2/57)
- IOI: 580.1 (#2/58)
- Beyond Function Calling: 42.5 (#3/12)
- ReproRepo: 19.3 (#3/4)
Claude Opus 4.5 (20251101) — ELO 1686, #47
- SciPredict: 23.05 (#2/15)
- LongWebBench: 6.43 (#5/13)
- Arena AI Document: 1461.0 (#10/27)
- Poker Agent: 1033.38 (#11/17)
- Creative Writing v3: 1758.8 (#12/123)
- EQ-Bench: 1545.7 (#18/79)
- WritingBench: 79.78 (#20/66)
- Arena AI Code: 1467.0 (#26/42)
- Chess Puzzles (Epoch AI): 4.0 (#142/153)
GPT-5 Codex — ELO 1686, #48
- Hack-Verifiable TextArena: 23.3 (#3/12)
- Mercor APEX: 20.1 (#28/53)
- LM Market Cap LMC Score: 88.7 (#53/406)
GLM-5.1 — ELO 1685, #49
- TimeSage-MT: 40.5 (#1/6)
- POLAR-Bench: 94.34 (#1/22)
- Are Agents Ready to Teach? A Multi-Stage Bench: 63.8 (#1/11)
- Trip+: 68.96 (#2/18)
- Personalized Turn-Level User Conversation Sati: 4.71 (#2/7)
- SGR-Bench: 65.64 (#2/11)
- Hack-Verifiable TextArena: 25.9 (#2/12)
- Greenland Sovereignty Game (implicitly as a st: 19.0 (#2/8)
- AGORA: 50.0 (#3/8)
- Beyond Function Calling: 42.0 (#4/12)
Inkling — ELO 1684, #50
- AudioMC: 56.6 (#2/31)
- MMAU: 77.2 (#2/31)
- Tau3-Bench Banking_Knowledge: 25.0 (#19/28)
- MCP Atlas: 74.1 (#24/42)
- Vals AI Multimodal Index: 49.4 (#26/32)
- Toolathlon: 45.5 (#35/39)
- CharXiv-R: 78.1 (#35/60)
- Arena AI Code: 1418.0 (#40/42)
- Epoch AI - Proofbench: 2.0 (#50/52)
- Design Arena (UI Components): 1213.0 (#58/145)
Qwen 3.5 Plus — ELO 1684, #51
- StemBind: 42.2 (#1/24)
- TOBench: 41.0 (#1/18)
- FINESSE-Bench: 87.76 (#1/31)
- Can Agents Price a Reaction? Evaluating LLMs o: 34.6 (#2/12)
- CC-OCR V2: 73.03 (#2/15)
- Disentangling Language Roles in Multilingual L: 82.0 (#3/20)
- GroupTravelBench: 8.63 (#3/9)
- ChartFI: 3.19 (#3/5)
- EgoCoT-Bench: 70.68 (#3/20)
- StereoTales: 252.0 (#3/23)
Hy3 — ELO 1682, #52
- Arena AI Code: 1517.0 (#17/42)
- LMArena WebDev Arena: 1516.66 (#17/22)
- AA-LCR: 66.7 (#47/355)
- CritPt: 4.9 (#47/342)
- AIIQ Composite IQ: 99.0 (#91/138)
MiMo-V2.5-Pro — ELO 1681, #53
- PaperFit-Bench: 100.0 (#3/4)
- AutoLab: 45.0 (#4/11)
- PhysicianBench: 16.7 (#8/12)
- QuestBench: 22.73 (#9/13)
- RoadmapBench: 13.9 (#9/13)
- ORAgentBench: 22.43 (#10/14)
- GeneBench: 3.0 (#12/16)
- BLXBench: 44.0 (#15/25)
- MedScribe: 83.73 (#15/61)
- ProofBench: 24.0 (#20/49)
Qwen 3.6 Plus — ELO 1680, #54
- From Knowing to Doing: 85.29 (#1/10)
- CC-OCR V2: 75.77 (#1/15)
- TIR-Bench: 61.6 (#1/27)
- MMGist: 61.0 (#2/27)
- BIM-Edit: 47.63 (#2/7)
- WildRoadBench: 36.8 (#2/25)
- IndustryBench: 2.07 (#2/17)
- CUDABeaver: 23.0 (#2/6)
- CalBench: 53.0 (#3/7)
- ROSE: 50.3 (#4/10)
Qwen 3.5 397B A17B — ELO 1680, #55
- TAEF (Text Analytics Evaluation Framework): 79.5 (#1/11)
- GeoSym-Bench: 87.47 (#1/22)
- HarmActionsEval: 23.4 (#1/10)
- EHR-Complex: 62.2 (#2/14)
- ProactBench: 52.5 (#2/16)
- EnvSimBench: 42.3 (#2/8)
- Do Agents Know What They Can't Do? Evaluating: 58.2 (#3/9)
- DiscoverPhysics: 33.0 (#3/11)
- Can LLM Agents Respond to Disasters? Benchmark: 53.45 (#3/15)
- Polaris-Bench: 72.9 (#3/15)
Inkling Small — ELO 1680, #56
- AudioMC: 54.9 (#3/31)
- MMAU: 77.0 (#3/31)
- HMMT February 2026: 90.2 (#9/25)
- ProphetArena: 0.9417 (#9/23)
- Vals AI CyberBench: 68.75 (#12/22)
- MCP Atlas: 79.6 (#13/42)
- Vals AI SWE-bench Verified: 82.2 (#13/79)
- Vals AI TaxEval v2: 75.51 (#15/136)
- Vals AI LiveCodeBench: 85.93 (#20/134)
- Vals AI Harvey Legal Agent Bench: 1.67 (#20/31)
MiniMax-M3 — ELO 1679, #57
- MedScribe: 87.25 (#4/61)
- OmniDocBench 1.5: 91.6 (#8/64)
- FutureEval: 10.85 (#11/46)
- AA-LCR: 74.0 (#12/355)
- Vals Multimodal Index: 59.97 (#12/26)
- Vals Index: 58.94 (#13/35)
- Arena AI Document: 1435.0 (#17/27)
- Finance Agent v2: 48.27 (#18/39)
- MedCode: 46.29 (#19/63)
- LMArena WebDev Arena: 1492.53 (#21/22)
Claude Sonnet 4.6 — ELO 1677, #58
- Every Act Has Its Price: 1005.3 (#1/10)
- CodeClinic: 53.1 (#1/8)
- An Empirical Study of Proactive Coding Assista: 13.57 (#1/7)
- TimeSage-MT: 40.44 (#2/6)
- A Matter of TASTE: 72.0 (#2/6)
- When Context Flips, Safety Breaks: 43.8 (#2/12)
- OpenComputer: 64.4 (#2/8)
- CAM-Bench: 19.28 (#2/5)
- TraceEval: 64.5 (#2/10)
- LibEvoBench: 81.0 (#3/13)
GPT-5.3 — ELO 1677, #59
- HarmActionsEval: 12.77 (#2/10)
- Creative Writing v3: 1716.2 (#17/123)
- EQ-Bench: 1393.0 (#33/79)
GPT-5 (2025-08-07) — ELO 1677, #60
- HELM MedQA: 96.8191 (#1/13)
- Poker Agent: 1103.17 (#2/17)
GPT-5.1 Codex — ELO 1676, #61
- MonitoringBench: 71.1 (#3/13)
- Visual Aesthetic Benchmark (VAB): 50.2 (#12/23)
- Mercor APEX: 20.7 (#27/53)
- LiveBench: 69.31 (#36/43)
O3 — ELO 1675, #62
- DRInQ: 67.0 (#2/13)
- Forge: 91.5 (#2/21)
- CommonWhy: 17.2 (#3/7)
- EnactToM: 13.6 (#4/7)
- TRLawBench: 81.3 (#4/32)
- IneqMath: 37.0 (#5/42)
- From Perception to Action: 10.1 (#8/16)
- VitaBench 2.0: 40.3 (#14/35)
- CharXiv-R: 78.6 (#33/60)
- ZeroBench: 6.0 (#35/69)
GLM-5-Turbo — ELO 1674, #63
- ClawProBench: 61.92 (#8/57)
- ALE-Bench: 633.98 (#47/79)
- AA-LCR: 60.7 (#95/355)
- CritPt: 0.3 (#124/342)
GPT-5.4 Mini — ELO 1671, #64
- LLM-as-a-Reviewer: 99.7 (#1/12)
- TempGlitch: 52.4 (#1/10)
- NRT-Bench: 11.4 (#2/4)
- Causal Sensitivity Score (CSS): 43.9 (#2/6)
- Chartographer: 94.0 (#2/15)
- TriBench-Ko: 78.1 (#2/13)
- BehaviorBench: 29.0 (#3/20)
- Managing Procedural Memory in LLM Agents: 2.0 (#3/13)
- MedCTA: 28.31 (#3/18)
- Momento: 75.15 (#3/5)
Grok 4.3 — ELO 1670, #65
- CaseLaw v2: 79.31 (#1/54)
- AgentCIBench: 91.5 (#4/15)
- EnterpriseArena: 40.0 (#4/27)
- OmniMatBench: 29.1 (#5/13)
- TukaBench: 11.6 (#8/13)
- SearchGEO: 7.6 (#9/13)
- CommunityFact: 53.85 (#13/15)
- AIIQ Composite IQ: 125.0 (#15/138)
- scBench: 44.27 (#15/19)
- ResearchClawBench: 12.4 (#20/20)
Gemini 2.5 Pro — ELO 1668, #67
- From Hallucination to Grounding: 57.28 (#1/13)
- Active Evidence-Seeking and Diagnostic Reasoni: 49.4 (#1/15)
- Xent Games: 65.86 (#1/12)
- TRAP: 36.9 (#2/22)
- ERGeoBench: 6.4 (#2/9)
- K-FinHallu: 85.9 (#2/12)
- SuiChat-CN: 75.9 (#2/44)
- QUIET: 8.44 (#2/12)
- Emotional intelligence in large language model: 64.1 (#2/7)
- TRACER: 81.0 (#2/4)
Claude Sonnet 4.5 — ELO 1668, #68
- TW-LegalBench: 81.0 (#1/13)
- VeriTrip: 69.23 (#1/11)
- AssertLLM2: 28.32 (#1/6)
- PreScam: 79.36 (#1/5)
- LegalCiteBench: 77.8 (#1/21)
- MAVEN-Bench: 70.0 (#2/8)
- StressEval: 37.4 (#2/10)
- ThaiSafetyBench: 9.75 (#2/24)
- DRInQ: 65.0 (#3/13)
- HIDBench: 56.1 (#3/9)
GPT-5.1 — ELO 1668, #69
- FactoryBench: 20.6 (#1/4)
- CombEval: 74.1 (#3/11)
- ATC Safety-Oriented Language Understanding Eval: 67.75 (#3/11)
- Poker Agent: 1038.59 (#9/17)
- OR-Space: 53.0 (#10/18)
- Mirror, Mirror on the Wall: 52.0 (#13/43)
- scBench: 38.8 (#16/19)
- EQ-Bench: 1551.0 (#17/79)
- AIIQ Composite IQ: 120.0 (#24/138)
- Arena AI Document: 1401.0 (#27/27)
Gemini 3.5 Flash Lite — ELO 1668, #70
- Sycophancy (Lechmazur): 990990.0 (#15/24)
- Roboflow Playground Object Detection: 1198.0 (#22/42)
- ParseBench: 57.15 (#29/55)
- Roboflow Playground Overall: 1198.0 (#35/66)
- Epoch AI - Scicode: 40.86 (#79/128)
- LM Market Cap LMC Score: 76.9 (#115/406)
O3 (2025-04-16) — ELO 1665, #71
- HELM Safety: 98.1606 (#1/57)
- VTB: 13.74 (#6/17)
- WritingBench: 80.46 (#17/66)
- TextClass Benchmark: 1625.36 (#30/108)
Kimi K2.5 — ELO 1664, #72
- EHR-Complex: 62.3 (#1/14)
- PinTrace: 45.78 (#1/10)
- TuRTLe Module Completion (NotSoTiny): 31.57 (#1/15)
- MMVU: 80.4 (#1/40)
- FINAL Bench Metacognitive: 78.54 (#1/9)
- UXBench: 20.5 (#2/8)
- Dr. DocBench: 60.38 (#2/12)
- TuRTLe Code Completion (Icarus Verilog): 83.38 (#2/44)
- TuRTLe Code Completion (Verilator): 81.65 (#2/44)
- TuRTLe Spec-to-RTL (Icarus Verilog): 81.47 (#2/44)
Grok 4.20 — ELO 1664, #73
- MathConstraint: 58.1 (#2/12)
- ConsumerSimBench: 44.9 (#3/13)
- FrontierOR: 22.0 (#5/7)
- MedBench v5: 94.6 (#6/10)
- The Wrong Kind of Right: 25.3 (#7/26)
- XL-SafetyBench: 30.6 (#7/10)
- AutoLab: 35.0 (#8/11)
- CAIS Risk Index: 44.5 (#8/45)
- When Context Flips, Safety Breaks: 18.2 (#10/12)
- PhysicianBench: 5.3 (#12/12)
GLM-5 — ELO 1662, #75
- Are LLMs Ready to Assist Physicians? PhysAssis: 69.4 (#1/14)
- Ego2World: 183.0 (#1/6)
- From 0-Order Selection to 2-Order Judgment: 38.33 (#2/13)
- ActTraitBench: 1.79 (#3/15)
- Metacognitive Probe: 74.25 (#3/8)
- FINESSE-Bench: 87.27 (#4/31)
- An Empirical Study of Proactive Coding Assista: 7.77 (#4/7)
- Claw-Eval-Live: 61.9 (#4/13)
- FINAL Bench Metacognitive: 76.38 (#4/9)
- JamSet/JamBench: 37.0 (#5/9)
Gemini 2.5 Pro (Preview 03-25) — ELO 1661, #77
- CodeGolf Bench: 73.9 (#1/9)
- OJBench: 38.91 (#5/44)
- Defects4J: 41.4 (#9/33)
- BigCodeBench-Hard: 29.7 (#12/20)
- HELM Safety: 91.3978 (#26/57)
- EQ-Bench: 1393.2 (#32/79)
- Creative Writing v3: 1395.3 (#55/123)
- TextClass Benchmark: 1517.98 (#61/108)
MiMo-V2-Pro — ELO 1661, #78
- AttuneBench: 53.61 (#4/11)
- Cookie-Bench: 76.3 (#5/13)
- Claw-Eval-Live: 53.3 (#6/13)
- DeepSearchQA: 86.7 (#10/15)
- WildClawBench: 40.2 (#11/20)
- GeneBench: 1.6 (#16/16)
- ALE-Bench: 785.17 (#36/79)
- Arena AI Code: 1433.0 (#37/42)
- AIIQ Composite IQ: 105.0 (#71/138)
DeepSeek V3.2 Speciale — ELO 1659, #79
- HMMT 2025: 97.5 (#6/89)
- MathArena Apex: 9.38 (#26/51)
- Creative Writing v3: 1238.1 (#73/123)
GLM-5V Turbo — ELO 1659, #80
- SpatialAct: 3.5 (#3/7)
- ROSE: 33.8 (#5/10)
- MineExplorer: 19.93 (#5/18)
- CiteVQA: 14.9 (#7/20)
- Arena AI Document: 1417.0 (#23/27)
- Arena AI Code: 1400.0 (#42/42)
Gemini 3.1 Flash Lite — ELO 1658, #81
- LaViSA: 84.0 (#2/10)
- ChildAgentEval: 48.0 (#3/6)
- IntentGrasp: 50.99 (#4/20)
- STALE: 22.4 (#4/15)
- TAEF (Text Analytics Evaluation Framework): 70.3 (#5/11)
- LLM-as-a-Reviewer: 48.3 (#6/12)
- SpaceDG: 48.8 (#6/29)
- AGORA: 6.35 (#7/8)
- MolDeTox: 0.43 (#8/15)
- Are Agents Ready to Teach? A Multi-Stage Bench: 46.3 (#9/11)
DeepSeek V4 Flash — ELO 1658, #82
- PlanBench-XL: 63.08 (#2/10)
- FullHome: 63.0 (#2/8)
- Beyond the All-in-One Agent: 57.33 (#2/12)
- SmartHome-Bench: 85.16 (#3/14)
- When Safe Skills Collide: 9.0 (#3/9)
- Can Large Language Models Handle Discourse Par: 69.8 (#3/10)
- Do LLMs Build World Models From Text? A Multil: 26.4 (#3/14)
- SearchGEO: 20.1 (#4/13)
- AGORA: 40.06 (#5/8)
- HOLMES: 50.98 (#5/11)
Claude Opus 4.1 — ELO 1658, #83
- The Metanym Game: 6.05 (#4/12)
- MCP-Universe: 29.44 (#4/27)
- ZeroBench: 5.0 (#40/69)
- ALE-Bench: 674.77 (#42/79)
GPT-5 Mini (2025-08-07) — ELO 1657, #84
- HELM MedQA: 95.6262 (#2/13)
Claude Opus 4.1 (20250805) — ELO 1657, #85
- SciPredict: 22.22 (#4/15)
- Seneca-TRBench: 90.06 (#4/5)
- Xent Games: 58.65 (#6/12)
- WritingBench: 76.36 (#29/66)
- IOI: 12.52 (#33/58)
Nemotron 3 Ultra 550B A55B — ELO 1657, #86
- Vals Index: 43.99 (#27/35)
- Finance Agent v2: 37.67 (#31/39)
- MedCode: 38.62 (#36/63)
- ProofBench: 2.0 (#47/49)
Step 3.7 Flash — ELO 1656, #87
- ArxivMath: 46.67 (#17/20)
- AA-LCR: 63.7 (#77/355)
- AIIQ Composite IQ: 104.0 (#79/138)
- Epoch AI - Scicode: 40.05 (#85/128)
Qwen 3.6 27B — ELO 1655, #88
- CheXpercept: 92.2 (#1/14)
- The Age of Curiosity Meets the Age of AI: Benc: 4.98 (#1/13)
- RefSpatialBench: 70.0 (#1/7)
- Momento: 75.15 (#2/5)
- Done, But Not Sure: 37.8 (#2/20)
- RxEval: 71.7 (#4/17)
- CUDABeaver: 19.2 (#4/6)
- Trip+: 66.35 (#5/18)
- PerfCodeBench: 65.07 (#5/20)
- VideoMME w sub.: 87.7 (#5/65)
Qwen3-8B-Base — ELO 1653, #89
- IndoBias: 67.3 (#11/26)
Gemma 4 31B (IT) — ELO 1651, #90
- Momento: 78.26 (#1/5)
- TuRTLe Spec-to-RTL (Icarus Verilog): 81.51 (#1/44)
- TuRTLe Spec-to-RTL (Verilator): 79.83 (#1/44)
- TuRTLe Module Completion (NotSoTiny): 29.54 (#2/15)
- K12-Bench: 46.4 (#3/11)
- CUDABeaver: 21.6 (#3/6)
- TuRTLe Code Completion (Icarus Verilog): 82.57 (#3/44)
- TuRTLe Code Completion (Verilator): 80.51 (#3/44)
- TableVista: 54.3 (#4/29)
- GENSTRAT: 25.0 (#5/9)
Qwen 3.5 122B A10B — ELO 1650, #91
- ZEROBench-Sub: 36.2 (#1/45)
- VideoMME w/o sub.: 83.9 (#1/61)
- CheXpercept: 91.3 (#3/14)
- SceneFunRI: 12.63 (#3/17)
- RefSpatialBench: 69.3 (#3/7)
- MMVU: 74.7 (#3/40)
- FullStackBench en: 62.6 (#3/31)
- TimeSage-MT: 37.34 (#4/6)
- TAEF (Text Analytics Evaluation Framework): 72.2 (#4/11)
- TIR-Bench: 53.2 (#4/27)
GPT-5 Mini — ELO 1649, #92
- CulturaQA: 75.09 (#1/9)
- From Hallucination to Grounding: 56.75 (#2/13)
- On the Cultural Anachronism and Temporal Reaso: 51.2 (#2/11)
- BLUEX v2: 8.9 (#3/10)
- Scammer4U: 61.0 (#3/4)
- Seeing Isn't Knowing: 49.9 (#3/9)
- VeriTrip: 67.26 (#3/11)
- JMed48k: 87.1 (#3/21)
- Time to REFLECT: 43.5 (#3/14)
- A2RBench: 36.9 (#3/14)
Claude Opus 4 — ELO 1649, #93
- ComplexMCP: 41.84 (#5/17)
- MGSM: 93.78 (#6/67)
- MATH 500: 90.4 (#23/58)
- Creative Writing v3: 1609.7 (#28/123)
- EQ-Bench: 1399.8 (#31/79)
- ZeroBench: 4.0 (#43/69)
Gemini 2.5 Pro (Preview 05-06) — ELO 1649, #94
- HELM MedQA: 93.4394 (#4/13)
Gemini 2.5 Flash (Preview 09-2025) — ELO 1649, #95
- MedCode: 40.54 (#30/63)
- MGSM: 89.85 (#40/67)
Hy3-preview — ELO 1647, #96
- PlanningBench: 36.17 (#5/16)
- AgentTrap: 48.0 (#7/8)
- Trip+: 61.3 (#13/18)
Gemini 2.5 Flash (Preview 05-20) — ELO 1647, #97
- Visual-Language Understanding: 49.15 (#12/54)
- EnigmaEval: 2.7 (#26/39)
Gemini 3.1 Flash Lite (Preview) — ELO 1645, #98
- GENSTRAT: -27.0 (#7/9)
- Beyond the All-in-One Agent: 16.67 (#10/12)
- TutorBench: 51.5 (#14/23)
- Visual-Language Understanding: 46.93 (#19/54)
- MultiNRC: 25.02 (#24/39)
- EnigmaEval: 3.04 (#25/39)
GLM-4.7 — ELO 1645, #99
- StreamProfileBench: 44.21 (#3/14)
- ComplexMCP: 42.55 (#4/17)
- FINESSE-Bench: 86.39 (#5/31)
- ADBench: 81.0 (#5/10)
- PrepBench: 41.5 (#6/10)
- CUDABeaver: 12.7 (#6/6)
- From 0-Order Selection to 2-Order Judgment: 30.0 (#6/13)
- PaliBench: 92.0 (#7/10)
- IMO-AnswerBench: 82.0 (#12/22)
- SuiChat-CN: 63.4 (#25/44)
Qwen 3 Max — ELO 1645, #100
- Disentangling Language Roles in Multilingual L: 84.0 (#1/20)
- Every Act Has Its Price: 1004.4 (#2/10)
- BacktestBench: 61.84 (#2/23)
- CPCD-Bench: 4.23 (#4/14)
- ATC Safety-Oriented Language Understanding Eval: 52.34 (#4/11)
- Exploring Autonomous Agentic Data Engineering: 1.58 (#6/6)
- DisasterBench: 63.95 (#6/12)
- IDE-Bench: 65.0 (#6/15)
- ComplexMCP: 31.2 (#8/17)
- IndustryBench: 1.97 (#8/17)
Nemotron 3 Ultra — ELO 1645, #101
- IMO-AnswerBench: 92.3 (#1/22)
- IOI: 570.0 (#3/58)
- GDPval: 46.7 (#13/18)
- FutureEval: 6.56 (#30/46)
- Toolathlon: 34.3 (#38/39)
- MCP Atlas: 44.7 (#40/42)
- AIIQ Composite IQ: 104.0 (#76/138)
Grok 4 — ELO 1644, #102
- MCP-Universe: 33.33 (#2/27)
- MATH 500: 96.2 (#2/58)
- Xent Games: 63.22 (#2/12)
- USAMO25: 37.5 (#3/13)
- MAVEN-Bench: 55.0 (#4/8)
- GlobalDentBench: 57.77 (#5/12)
- Every Act Has Its Price: 993.3 (#6/10)
- CaseLaw v2: 65.81 (#9/54)
- VerdictBench: 54.36 (#10/17)
- TIR-Bench: 22.5 (#10/27)
Kimi K2 — ELO 1643, #103
- SuiChat-CN: 76.3 (#1/44)
- MLB: 77.29 (#1/10)
- AutoLogi: 89.5 (#1/10)
- MAVEN-Bench: 57.0 (#3/8)
- RealityTest: 55.2 (#4/17)
- Emotional intelligence in large language model: 60.6 (#5/7)
- ADBench: 79.0 (#7/10)
- AsyncTool: 24.44 (#8/19)
- Rogo Big Finance Bench: 45.0 (#8/14)
- ROK-FORTRESS: 7.3 (#9/14)
MiMo-V2.5 — ELO 1642, #104
- Omni-DeepSearch: 11.72 (#5/12)
- Vals Multimodal Index: 52.77 (#17/26)
- BLXBench: 15.7 (#19/25)
- Vals Index: 51.57 (#21/35)
- CharXiv-R: 81.0 (#27/60)
- ProofBench: 16.0 (#29/49)
- Finance Agent v2: 36.73 (#32/39)
- Arena AI Code: 1435.0 (#34/42)
- MedScribe: 72.15 (#50/61)
- MedCode: 31.89 (#54/63)
Claude Sonnet 4 — ELO 1641, #105
- PrivacyPeek: 86.46 (#1/10)
- Exploring Autonomous Agentic Data Engineering: 3.0 (#2/6)
- HIDBench: 56.6 (#2/9)
- When the Manual Lies: 100.0 (#3/8)
- MLB: 74.53 (#3/10)
- MCP-Universe: 29.44 (#5/27)
- AfroBench-Lite: 68.69 (#5/24)
- The Metanym Game: 5.3 (#6/12)
- ComplexMCP: 38.29 (#7/17)
- TRLawBench: 75.6 (#9/32)
Qwen 3 Max (Preview) — ELO 1641, #107
- Poker Agent: 994.51 (#15/17)
- MGSM: 92.15 (#19/67)
- CritPt: 0.9 (#99/342)
- AA-LCR: 39.7 (#162/355)
Qwen 3.5 27B — ELO 1640, #108
- TIR-Bench: 59.8 (#2/27)
- ZEROBench-Sub: 36.2 (#2/45)
- VideoMME w/o sub.: 82.8 (#2/61)
- OJBench: 40.1 (#3/44)
- IFMTBench: 81.06 (#4/15)
- OpenComputer: 32.3 (#4/8)
- MMVU: 73.3 (#4/40)
- PrivacySIM: 36.41 (#6/6)
- TeleResilienceBench: 17.6 (#6/8)
- FullStackBench en: 60.1 (#6/31)
O1 (2024-12-17) — ELO 1640, #110
- BigCodeBench-Hard: 32.4 (#2/20)
- TextClass Benchmark: 1768.81 (#4/108)
- HELM Safety: 97.58 (#4/57)
- EnigmaEval: 5.65 (#17/39)
- Visual-Language Understanding: 45.25 (#26/54)
Claude Opus 4 (20250514) — ELO 1639, #111
- Xent Games: 58.35 (#7/12)
- WritingBench: 74.45 (#33/66)
GPT-5 Chat — ELO 1639, #112
- OdysseyBench: 40.33 (#3/10)
- PreScam: 71.84 (#4/5)
- EQ-Bench: 1488.2 (#26/79)
- AA MMLU-Pro: 82.04 (#75/346)
LongCat 2.0 — ELO 1639, #113
- CritPt: 2.6 (#67/342)
- AA-LCR: 58.0 (#109/355)
DeepSeek V3.2 — ELO 1638, #114
- LITMUS: 71.51 (#1/6)
- RepoMirage: 39.8 (#2/8)
- StreamProfileBench: 44.63 (#2/14)
- PreScam: 75.8 (#2/5)
- Correction Suppression Benchmark: 84.0 (#2/8)
- GR-Ben: 55.4 (#2/22)
- LiveFMBench: 43.55 (#2/11)
- ABC-Bench: 50.1 (#2/11)
- PersonaArena: 3.9 (#3/18)
- TraceEval: 62.1 (#3/10)
Claude 3.7 Sonnet (20250219) — ELO 1638, #115
- BigCodeBench-Hard: 32.4 (#3/20)
- Defects4J: 47.8 (#3/33)
- MedHELM: 63.5714 (#3/9)
- MedGUIDE: 55.6 (#6/25)
- HELM MedQA: 85.6859 (#8/13)
- OPT-BENCH: 53.0 (#11/19)
- HELM Safety: 94.4914 (#16/57)
- MGSM: 92.4 (#17/67)
- EnigmaEval: 2.26 (#27/39)
- Visual-Language Understanding: 43.02 (#30/54)
Step 3.5 Flash — ELO 1638, #116
- From Knowing to Doing: 21.86 (#6/10)
- IMO-AnswerBench: 85.4 (#8/22)
- Can LLM Agents Respond to Disasters? Benchmark: 46.68 (#11/15)
- HMMT February 2026: 86.36 (#14/25)
O4 Mini (2025-04-16) — ELO 1636, #117
- HELM MedQA: 94.831 (#3/13)
- HELM Safety: 97.3247 (#5/57)
- VTB: 11.12 (#8/17)
- WritingBench: 72.9 (#38/66)
- TextClass Benchmark: 1538.33 (#55/108)
qwen3.5-flash — ELO 1635, #118
- StereoTales: 157.0 (#15/23)
- ALE-Bench: 265.93 (#71/79)
- Epoch AI - ECI: 143.94 (#234/463)
Grok 4.1 Fast — ELO 1634, #119
- Hack-Verifiable TextArena: 28.5 (#1/12)
- TAEF (Text Analytics Evaluation Framework): 77.3 (#3/11)
- Soohak: 30.88 (#3/11)
- Can LLM Agents Respond to Disasters? Benchmark: 52.1 (#5/15)
- QUIET: 8.11 (#6/12)
- Checkup2Action: 40.8 (#6/8)
- DentalGPT Dental Eval Suite: 56.7 (#6/16)
- SpaceDG: 35.6 (#23/29)
- APEX-Agents: 24.8 (#31/44)
- EQ-Bench: 1184.6 (#44/79)
GLM-4.6 — ELO 1634, #120
- A2RBench: 21.0 (#8/14)
- MCP-Universe: 25.97 (#11/27)
- SuiChat-CN: 69.1 (#17/44)
- PLawBench: 60.49 (#17/29)
- ClawProBench: 56.29 (#32/57)
- Mercor APEX: 4.0 (#49/53)
- Creative Writing v3: 1398.7 (#54/123)
- ALE-Bench: 340.82 (#67/79)
- Epoch AI - Critpt: 1.14 (#85/138)
MiniMax-M2.5 — ELO 1634, #121
- ActTraitBench: 2.17 (#1/15)
- PinTrace: 45.42 (#2/10)
- HOLMES: 54.77 (#3/11)
- Every Act Has Its Price: 994.7 (#4/10)
- GDPval-MM: 59.0 (#5/14)
- ContractBench: 62.6 (#7/20)
- An Empirical Study of Proactive Coding Assista: 2.77 (#7/7)
- StreamProfileBench: 37.9 (#8/14)
- ConsumerSimBench: 35.2 (#9/13)
- MUSE: 5.35 (#11/15)
Qwen 3.6 35B A3B — ELO 1634, #122
- HardMTBench: 90.68 (#3/20)
- ZEROBench-Sub: 34.4 (#3/45)
- MMGist: 55.2 (#4/27)
- SpatialAct: 1.6 (#4/7)
- VideoMME w/o sub.: 82.5 (#4/61)
- Momento: 67.7 (#5/5)
- RepoMirage: 24.2 (#5/8)
- RefSpatialBench: 64.3 (#5/7)
- AgentCIBench: 76.9 (#7/15)
- MedLayXPlain: 61.5 (#10/32)
Grok 4 Fast — ELO 1633, #123
- MCP-Universe: 27.27 (#7/27)
- HMMT 2025: 93.3 (#28/89)
O1 — ELO 1631, #124
- LinAlg-Bench: 100.0 (#1/10)
- MedGUIDE: 60.5 (#2/25)
- DiagnosticIQ: 65.41 (#8/33)
- OJBench: 26.45 (#13/44)
- IneqMath: 8.0 (#17/42)
- LM Market Cap LMC Score: 74.4 (#133/406)
Ring-2.6-1T — ELO 1631, #126
- AIIQ Composite IQ: 104.0 (#78/138)
DeepSeek V3.2 Exp — ELO 1629, #127
- EHR-Complex: 59.2 (#3/14)
- DVMap: 45.1 (#8/9)
- PlanningBench: 29.27 (#10/16)
- MCP-Universe: 19.91 (#19/27)
- HMMT 2025: 83.6 (#46/89)
DeepSeek V3.1 Terminus — ELO 1629, #128
- LiveFMBench: 38.48 (#3/11)
- AsyncTool: 28.93 (#5/19)
- LingOly-TOO: 42.2 (#5/16)
- MCP-Universe: 21.65 (#18/27)
- ALE-Bench: 745.17 (#38/79)
- Epoch AI - Critpt: 1.71 (#76/138)
KAT Coder Pro V2 — ELO 1629, #129
- AA-LCR: 66.0 (#57/355)
- CritPt: 0.0 (#223/342)
Qwen 3 VL 235B A22B — ELO 1627, #130
- Visual Aesthetic Benchmark (VAB): 44.9 (#19/23)
MiniMax-M2.7 — ELO 1626, #131
- When Lower Privileges Suffice: 43.4 (#3/11)
- DataClawBench: 41.3 (#3/8)
- SearchGEO: 17.4 (#5/13)
- TimeSage-MT: 36.44 (#5/6)
- From Knowing to Doing: 56.31 (#5/10)
- EnvSimBench: 41.8 (#5/8)
- Personalized Turn-Level User Conversation Sati: 4.26 (#7/7)
- RepoMirage: 14.8 (#7/8)
- Claw-Anything: 13.5 (#8/9)
- Can LLM Agents Respond to Disasters? Benchmark: 48.35 (#8/15)
MiniMax-M2.1 — ELO 1626, #132
- ContractBench: 60.6 (#8/20)
- MedScribe: 80.78 (#23/61)
- MATH 500: 89.0 (#27/58)
- CaseLaw v2: 55.84 (#33/54)
- ClawProBench: 48.08 (#45/57)
- MedCode: 34.08 (#46/63)
- ALE-Bench: 623.83 (#48/79)
- Finance Agent v1.1: 33.35 (#49/56)
- MGSM: 87.85 (#51/67)
- IOI: 2.33 (#53/58)
O4 Mini — ELO 1625, #133
- GraphARC: 87.0 (#2/11)
- TRLawBench: 84.4 (#2/32)
- FormInv: 80.0 (#3/9)
- Reward Hacking Benchmark (RHB): 8.4 (#3/13)
- MedGUIDE: 58.9 (#3/25)
- FinanceArena: 48.6 (#3/19)
- C4STYLI: 70.1 (#4/18)
- A2RBench: 33.7 (#4/14)
- MedFrameQA: 49.4 (#4/11)
- When Simulation Lies: 50.2 (#5/22)
DeepSeek R1 0528 — ELO 1625, #134
- USAMO25: 30.06 (#4/13)
- TuRTLe Code Completion (Icarus Verilog): 78.86 (#4/44)
- TuRTLe Code Completion (Verilator): 78.08 (#4/44)
- TuRTLe Spec-to-RTL (Icarus Verilog): 76.79 (#4/44)
- TuRTLe Spec-to-RTL (Verilator): 75.83 (#4/44)
- TuRTLe Module Completion (NotSoTiny): 20.73 (#5/15)
- OR-Space: 55.0 (#7/18)
- FinanceArena: 42.9 (#10/19)
- FINESSE-Bench: 82.66 (#13/31)
- IneqMath: 9.5 (#13/42)
GLM-4.5 — ELO 1625, #135
- MAVEN-Bench: 43.0 (#6/8)
- MCP-Universe: 24.68 (#12/27)
- MultiNRC: 17.44 (#34/39)
- EQ-Bench: 1276.3 (#38/79)
- Creative Writing v3: 1357.0 (#61/123)
- ALE-Bench: 344.82 (#66/79)
Gemini 2.5 Flash — ELO 1624, #136
- K-FinHallu: 86.4 (#1/12)
- OmniToM: 85.95 (#1/9)
- CommonWhy: 27.7 (#1/7)
- MedFrameQA: 54.75 (#1/11)
- SocialPersona: 32.9 (#2/6)
- TriViewBench: 79.55 (#2/19)
- Seeing Isn't Knowing: 53.2 (#2/9)
- K12-Bench: 48.3 (#2/11)
- A Matter of TASTE: 66.0 (#3/6)
- StakeBench: 17.5 (#3/15)
DeepSeek V3.1 — ELO 1624, #137
- Exploring Autonomous Agentic Data Engineering: 7.13 (#1/6)
- EQUITRIAGE: 36.4 (#3/5)
- EHR-Complex: 55.9 (#4/14)
- When Context Flips, Safety Breaks: 35.9 (#4/12)
- LegalCiteBench: 70.5 (#4/21)
- POLAR-Bench: 90.81 (#5/22)
- ActTraitBench: 0.75 (#6/15)
- Disentangling Language Roles in Multilingual L: 78.6 (#7/20)
- FinChain: 56.76 (#12/25)
- MCP-Universe: 22.08 (#16/27)
GPT-4.1 (2025-04-14) — ELO 1623, #139
- MedAgentGym: 70.15 (#1/29)
- HELM Safety: 96.2853 (#9/57)
- TextClass Benchmark: 1520.39 (#60/108)
Qwen3 Coder — ELO 1623, #141
- Do Coding Agents Understand Least-Privilege Au: 63.3 (#7/11)
- OrgForge-IT: 80.0 (#8/10)
- ALE-Bench: 461.45 (#60/79)
GPT-4o ChatGPT — ELO 1622, #142
- WritingBench: 73.22 (#37/66)
- AA-LCR: 53.0 (#127/355)
Claude Sonnet 4 (20250514) — ELO 1622, #143
- Xent Games: 48.45 (#9/12)
- WritingBench: 74.1 (#34/66)
GPT-4.1 — ELO 1621, #144
- NL2Scratch: 99.1 (#1/8)
- ComplexFuncBench: 65.5 (#1/18)
- EHRBench: 69.43 (#2/31)
- AIM-Bench (inventory manager): 20.71 (#3/5)
- BehaviorBench: 27.5 (#4/20)
- TRLawBench: 80.0 (#5/32)
- BigCodeBench-Hard: 31.8 (#5/20)
- Defects4J: 45.2 (#5/33)
- OPT-BENCH: 58.0 (#7/19)
- The Metanym Game: 4.44 (#8/12)
MiMo-V2-Omni — ELO 1621, #145
- Omni-DeepSearch: 9.69 (#6/12)
- Sandboxed Coding Agents are Competitive Omni-m: 25.8 (#11/12)
GPT-4.5 — ELO 1620, #146
- ComplexFuncBench: 63.0 (#3/18)
- TextClass Benchmark: 1767.86 (#5/108)
- HELM Safety: 96.4672 (#8/57)
- LingOly-TOO: 25.5 (#10/16)
- Multi-IF: 70.8 (#19/33)
- ZeroBench: 6.0 (#36/69)
- CharXiv-R: 55.4 (#53/60)
KAT-Coder-Pro V1 — ELO 1620, #147
- AIIQ Composite IQ: 95.0 (#103/138)
Gemini 2.5 Flash (Preview 04-17) — ELO 1619, #148
- CodeGolf Bench: 55.19 (#3/9)
- Visual-Language Understanding: 46.97 (#17/54)
- HELM Safety: 91.1812 (#27/57)
Kimi K2 0905 — ELO 1617, #149
- AutoLogi: 89.5 (#2/10)
- GR-Ben: 49.1 (#3/22)
- TuRTLe Module Completion (NotSoTiny): 19.0 (#6/15)
- LiveFMBench: 30.44 (#7/11)
- Xent Games: 42.89 (#10/12)
- OJBench: 27.1 (#12/44)
- MCP-Universe: 19.91 (#20/27)
- MageBench Season 1: 1558.0 (#26/35)
- ACEBench: 76.5 (#31/31)
- ALE-Bench: 267.13 (#70/79)
Claude 3.7 Sonnet — ELO 1616, #151
- Beyond Continuity: 94.5 (#2/10)
- TriViewBench: 68.62 (#3/19)
- VRBench: 68.15 (#3/29)
- MedFrameQA: 49.67 (#3/11)
- VeriTrip: 66.16 (#4/11)
- DiagnosticIQ: 70.61 (#4/33)
- WildAgtEval: 61.6 (#4/9)
- GeoCode Leaderboard: 70.35 (#4/27)
- MUSE: 17.92 (#5/15)
- FinChain: 57.89 (#5/25)
Gemma 4 31B — ELO 1615, #152
- TeleResilienceBench: 29.1 (#1/8)
- P3B3: 98.0 (#2/20)
- TempGlitch: 52.3 (#2/10)
- GRACE: 77.46 (#3/13)
- DisasterBench: 66.52 (#3/12)
- AgingBench: 7.4 (#3/8)
- POLAR-Bench: 91.2 (#3/22)
- CodeClinic: 43.4 (#4/8)
- HardMTBench: 90.62 (#5/20)
- IFMTBench: 79.61 (#5/15)
Qwen 3 235B A22B 2507 Instruct — ELO 1615, #153
- LiveFMBench: 15.83 (#10/11)
- WritingBench: 80.26 (#18/66)
- MCP-Universe: 18.18 (#25/27)
- Multi-IF: 77.5 (#28/33)
- AA MMLU-Pro: 82.79 (#64/346)
- Creative Writing v3: 87.5 (#112/123)
Qwen 3 VL 235B A22B Instruct — ELO 1615, #154
- CFMME: 63.31 (#2/16)
- Math-VR: 65.0 (#2/31)
- WritingBench: 85.5 (#4/66)
- OCRBench-V2 (en): 67.1 (#5/38)
- VideoMME w/o sub.: 79.2 (#6/61)
- LongWebBench: 5.85 (#7/13)
- HarmVideoBench: 79.7 (#9/21)
- OCRBench-V2 (zh): 61.8 (#9/37)
- MineExplorer: 10.58 (#13/18)
- DentalGPT Dental Eval Suite: 51.3 (#13/16)
Doubao Seed Code — ELO 1615, #156
- ClawProBench: 59.22 (#18/57)
- AIIQ Composite IQ: 95.0 (#100/138)
Grok 3 Mini Beta — ELO 1615, #157
- BigCodeBench-Hard: 31.1 (#7/20)
- HELM Safety: 90.2239 (#32/57)
- EQ-Bench: 1064.5 (#58/79)
Claude Haiku 4.5 — ELO 1614, #158
- When Safe Skills Collide: 9.0 (#1/9)
- LLM-as-a-Reviewer: 99.7 (#2/12)
- NRT-Bench: 8.7 (#4/4)
- JuICE: 40.38 (#4/10)
- DRInQ: 64.0 (#4/13)
- SearchGEO: 12.5 (#7/13)
- StakeBench: 13.8 (#7/15)
- CounterCount: 36.06 (#7/7)
- TOBench: 27.0 (#7/18)
- MedCTA: 23.08 (#8/18)
Qwen 3.5 35B A3B — ELO 1614, #159
- TIR-Bench: 55.5 (#3/27)
- VideoMME w/o sub.: 82.5 (#3/61)
- AgentCollabBench: 0.9 (#4/4)
- ZEROBench-Sub: 34.1 (#4/45)
- MMVU: 72.3 (#5/40)
- RefSpatialBench: 63.5 (#6/7)
- OJBench: 36.0 (#6/44)
- Managing Procedural Memory in LLM Agents: 2.0 (#7/13)
- Beyond Function Calling: 37.2 (#9/12)
- MedXpertQA: 61.4 (#9/26)
Gemma 4 26B A4B (IT) — ELO 1612, #160
- NRT-Bench: 12.1 (#1/4)
- StemBind: 33.0 (#12/24)
- MT-JailBench: 73.42 (#13/21)
- SeePhys Pro: 9.0 (#13/16)
- ArchSIBench: 52.0 (#15/28)
- TableVista: 42.5 (#17/29)
- ALE-Bench: 927.17 (#24/79)
- SpaceDG: 31.0 (#28/29)
- EQ-Bench: 1430.6 (#28/79)
- Creative Writing v3: 1286.9 (#66/123)
MiMo-V2-Flash — ELO 1612, #163
- Fin-RATE: 18.05 (#6/16)
- BacktestBench: 40.43 (#14/23)
- WildClawBench: 30.8 (#17/20)
- ALE-Bench: 737.95 (#39/79)
- HMMT 2025: 84.4 (#44/89)
- AIIQ Composite IQ: 101.0 (#87/138)
O1 Preview — ELO 1611, #164
- FullStackBench en: 65.62 (#1/31)
Ling-3.0-flash — ELO 1610, #165
- Tau3 Banking: 27.22 (#41/188)
- AA GDPval: 1107.63 (#75/184)
- Artificial Analysis Intelligence Index: 37.82 (#87/559)
- AA GPQA Diamond: 85.45 (#96/560)
- AA Humanity's Last Exam: 23.68 (#109/556)
- AA CritPt: 1.71 (#113/466)
- BenchLM: 49.9 (#122/200)
- AA Long Context Reasoning: 67.0 (#125/488)
- AA SciCode: 41.09 (#130/554)
- AA Omniscience: -17.88 (#134/465)
GPT-4.5 (Preview) — ELO 1609, #166
- Open FinLLM Leaderboard: 43.4 (#3/25)
- EnigmaEval: 3.18 (#23/39)
- Visual-Language Understanding: 42.11 (#31/54)
- WritingBench: 67.61 (#47/66)
- EQ-Bench: 1053.3 (#59/79)
- Creative Writing v3: 1230.6 (#74/123)
Claude 3.5 Sonnet (20241022) — ELO 1608, #168
- Aider Refactoring Benchmark: 92.1 (#1/10)
- MedHELM: 63.3929 (#4/9)
- Defects4J: 44.1 (#6/33)
- HELM MedQA: 86.4811 (#7/13)
- OPT-BENCH: 48.0 (#12/19)
- MGSM: 92.58 (#13/67)
- ACEBench: 0.76 (#20/31)
- OJBench: 10.4 (#27/44)
- Creative Writing v3: 1420.9 (#46/123)
- MATH 500: 72.4 (#48/58)
Grok Code Fast 1 — ELO 1608, #169
- MCP-Universe: 26.41 (#8/27)
- IDE-Bench: 11.25 (#12/15)
- IOI: 4.33 (#47/58)
- ALE-Bench: 587.73 (#54/79)
Qwen 3.5 Omni Plus — ELO 1608, #171
- Omni-DeepSearch: 13.91 (#4/12)
- FraudBench: 0.8 (#20/21)
Claude 3.5 Sonnet (20240620) — ELO 1607, #172
- Clembench Multimodal v1.6.5: 80.77 (#1/20)
- HELM Lite: 91.2171 (#2/77)
- HELM Safety: 97.6697 (#3/57)
Grok 3 — ELO 1607, #173
- TukaBench: 31.5 (#2/13)
- OPT-BENCH: 63.0 (#2/19)
- StoryAlign: 62.4 (#3/15)
- PersonaArena: 3.63 (#8/18)
- K-12EduBench: 63.91 (#16/23)
- MATH 500: 89.8 (#25/58)
- MGSM: 91.35 (#26/67)
- WritingBench: 71.72 (#43/66)
- EQ-Bench: 1149.8 (#48/79)
- Creative Writing v3: 1183.5 (#80/123)
Mistral Medium 3.5 — ELO 1604, #174
- Agent Arena - Steerability: 11.31 (#4/46)
- Agent Arena - Confirmed Success: 10.43 (#10/46)
- Agent Arena: 6.82 (#20/46)
- Agent Arena - Praise vs Complaint: 8.71 (#25/46)
- Agent Arena - Tool Hallucination: 3.53 (#44/46)
- Agent Arena - Bash Recovery: 0.13 (#46/46)
- Epoch AI - Scicode: 39.58 (#89/128)
- AA-LCR: 61.0 (#93/355)
- AIIQ Composite IQ: 96.0 (#97/138)
- CritPt: 0.0 (#268/342)
Ling-2.6-1T — ELO 1604, #175
- BLXBench: 75.3 (#8/25)
Claude 3.5 Sonnet — ELO 1601, #178
- Translation en to Set1 COMET22: 89.4 (#1/13)
- Translation Set1 to en COMET22: 89.1 (#1/13)
- AgentLeak: 55.2 (#1/5)
- RAGTruth: 86.1 (#1/11)
- Chain-of-Procedure: 47.5 (#2/6)
- Translation en to Set1 spBleu: 42.5 (#4/13)
- AutoLogi: 60.97 (#4/10)
- ComplexFuncBench: 61.0 (#4/18)
- FullStackBench en: 62.57 (#4/31)
- Translation Set1 to en spBleu: 43.5 (#5/13)
GPT-5.4 Nano — ELO 1600, #182
- Honeyval: 96.9 (#1/6)
- ABRA: 69.0 (#4/10)
- UXBench: 16.2 (#6/8)
- STALE: 9.8 (#8/15)
- CheXpercept: 76.9 (#10/14)
- MedCTA: 20.3 (#10/18)
- SearchGEO: 6.4 (#11/13)
- The Wrong Kind of Right: 21.5 (#12/26)
- Can LLM Agents Respond to Disasters? Benchmark: 38.14 (#12/15)
- Agentic Skills Evaluation Framework: 81.9 (#13/19)
Gemini 2.5 Flash Lite (Preview 09-2025) — ELO 1600, #183
- MedScribe: 75.82 (#40/61)
- MGSM: 89.53 (#42/67)
MiniMax-M2 — ELO 1599, #185
- ContractBench: 53.5 (#10/20)
- AA-LCR: 61.0 (#92/355)
- CritPt: 0.9 (#97/342)
O3 Mini — ELO 1598, #186
- GraphARC: 81.0 (#3/11)
- ComplexFuncBench: 17.6 (#12/18)
- IneqMath: 9.5 (#14/42)
- Multi-IF: 79.5 (#32/33)
- CAIS Text Capabilities Index: 7.8 (#37/40)
- CAIS Risk Index: 69.9 (#43/45)
- LM Market Cap LMC Score: 75.3 (#124/406)
Qwen 3 VL 32B — ELO 1596, #189
- AgroVG: 37.5 (#3/13)
- LaViSA: 81.9 (#4/10)
- SVFSearch: 90.8 (#4/75)
- From Hallucination to Grounding: 51.56 (#5/13)
- CardioLens: 51.94 (#6/24)
- LiveK12Bench: 82.7 (#7/12)
- EgoCoT-Bench: 67.09 (#8/20)
- DocScope: 51.8 (#11/23)
- TRAP: 15.8 (#15/22)
- Done, But Not Sure: 18.9 (#15/20)
Llama 3.1 405B — ELO 1595, #190
- BBH: 82.9 (#2/16)
- TW-LegalBench: 60.5 (#7/13)
- NarrativeWorldBench: 71.0 (#8/10)
- Balrog: 27.9 (#8/16)
- Multi-IF: 0.85 (#12/33)
- TuRTLe Code Completion (Icarus Verilog): 54.74 (#15/44)
- TuRTLe Code Completion (Verilator): 55.06 (#15/44)
- ComplexFuncBench: 4.0 (#16/18)
- TuRTLe Spec-to-RTL (Icarus Verilog): 53.22 (#17/44)
- MixRea: 12.4 (#19/21)
Mercury 2 — ELO 1595, #191
- ALE-Bench: 785.58 (#34/79)
- Epoch AI - Scicode: 38.66 (#94/128)
- AIIQ Composite IQ: 92.0 (#114/138)
Qwen 3 235B A22B — ELO 1594, #192
- ASPI: 41.9 (#2/10)
- FinVerBench: 65.7 (#3/15)
- ClinPivot: 65.74 (#3/6)
- Fin-RATE: 24.39 (#3/16)
- TW-LegalBench: 69.1 (#4/13)
- CodeGolf Bench: 48.24 (#4/9)
- Prosa: 80.1 (#4/16)
- BacktestBench: 55.01 (#5/23)
- EHR-Complex: 53.4 (#6/14)
- Ukrainian Legal Text Benchmark (EDRSR subset): 79.2 (#6/7)
Qwen 3 Next 80B A3B Instruct — ELO 1594, #193
- WritingBench: 87.3 (#1/66)
- ATLAS: 28.77 (#19/23)
- Multi-IF: 75.8 (#26/33)
- Creative Writing v3: 85.3 (#118/123)
Qwen3 Coder Next — ELO 1594, #194
- AgentTrap: 58.0 (#5/8)
- RepoMirage: 22.6 (#6/8)
- Agentic Skills Evaluation Framework: 70.8 (#17/19)
- AIIQ Composite IQ: 93.0 (#107/138)
- Epoch AI - Scicode: 32.29 (#112/128)
Mistral Medium 3.1 — ELO 1593, #195
- MageBench Season 1: 1569.0 (#25/35)
- Creative Writing v3: 1480.6 (#39/123)
- ALE-Bench: 210.18 (#76/79)
- Epoch AI - Scicode: 33.8 (#110/128)
- AA-LCR: 19.7 (#225/355)
- AA MMLU-Pro: 68.33 (#235/346)
- CritPt: 0.0 (#267/342)
Qwen 3 Coder 480B A35B Instruct — ELO 1593, #196
- TuRTLe Module Completion (NotSoTiny): 23.43 (#3/15)
- LiveFMBench: 30.37 (#8/11)
- MCP-Universe: 22.94 (#14/27)
- AA-LCR: 42.3 (#155/355)
- CritPt: 0.0 (#313/342)
GPT-4o — ELO 1592, #197
- DVMap: 48.5 (#2/9)
- AIM-Bench (inventory manager): 15.14 (#2/5)
- VRBench: 68.68 (#2/29)
- ECLeKTic: 38.8 (#2/14)
- PhiBench: 72.4 (#2/10)
- LLM Game Benchmark: 40.3 (#3/7)
- Beyond Continuity: 70.8 (#4/10)
- Aider Refactoring Benchmark: 62.9 (#4/10)
- TW-LegalBench: 66.2 (#5/13)
- TriBench-Ko: 71.5 (#5/13)
GPT-OSS-120B — ELO 1592, #198
- Do Agents Know What They Can't Do? Evaluating: 60.7 (#2/9)
- Time to REFLECT: 46.5 (#2/14)
- WildAgtEval: 62.5 (#3/9)
- Legal Hallucination Detection Benchmark (no ex: 34.4 (#4/6)
- ArtifactsBench: 57.69 (#4/7)
- DiagFlowBench: 76.2 (#5/10)
- MAVEN-Bench: 48.0 (#5/8)
- Decomposing and Measuring Evaluation Awareness: 1.3 (#5/9)
- SCICONVBENCH: 72.6 (#5/5)
- CodeClinic: 36.1 (#5/8)
Grok 3 Mini — ELO 1592, #199
- Balrog: 29.5 (#7/16)
Nemotron 3 Super — ELO 1591, #201
- EQUITRIAGE: 35.2 (#4/5)
- HMMT 2025: 94.73 (#18/89)
- Epoch AI - Critpt: 3.14 (#66/138)
- ALE-Bench: 213.9 (#75/79)
- AIIQ Composite IQ: 104.0 (#77/138)
Devstral 2 — ELO 1591, #202
- PrepBench: 33.3 (#8/10)
- CritPt: 0.0 (#159/342)
- AA-LCR: 30.0 (#189/355)
GLM 4.5 Air — ELO 1590, #203
- MCP-Universe: 19.48 (#21/27)
- MultiNRC: 10.43 (#37/39)
- AA MMLU-Pro: 81.47 (#87/346)
- AA-LCR: 43.7 (#152/355)
- CritPt: 0.0 (#182/342)
Mistral Large 3 — ELO 1590, #204
- ROK-FORTRESS: 28.4 (#1/14)
- XL-SafetyBench: 98.8 (#1/10)
- FactoryBench: 17.9 (#2/4)
- Ukrainian Legal Text Benchmark (EDRSR subset): 81.1 (#4/7)
- Greenland Sovereignty Game (implicitly as a st: 16.2 (#4/8)
- SWE-PRBench: 14.7 (#4/8)
- ABRA: 67.0 (#6/10)
- Qiskit QuantumKatas: 48.6 (#11/16)
- DentalGPT Dental Eval Suite: 48.6 (#14/16)
- StereoTales: 109.0 (#22/23)
DeepSeek V3 (0324) — ELO 1590, #205
- RefusalBench: 47.5 (#3/19)
- HalluWorld: 15.4 (#4/13)
- GeoCode Leaderboard: 70.25 (#5/27)
- CodeGolf Bench: 38.0 (#7/9)
- Defects4J: 43.0 (#8/33)
- LiveFMBench: 24.29 (#9/11)
- OJBench: 25.54 (#16/44)
- IneqMath: 7.0 (#18/42)
- Creative Writing v3: 1473.8 (#41/123)
- EQ-Bench: 1082.2 (#54/79)
O3 Mini (2025-01-31) — ELO 1590, #206
- BigCodeBench-Hard: 33.1 (#1/20)
- MedHELM: 64.1071 (#2/9)
- HELM MedQA: 92.0477 (#5/13)
- SciPredict: 19.84 (#7/15)
- HELM Safety: 96.1961 (#10/57)
- TextClass Benchmark: 1684.99 (#18/108)
GPT-4.1 Mini — ELO 1589, #207
- AgentCollabBench: 3.4 (#2/4)
- When2Speak: 49.9 (#2/6)
- StructEval: 75.64 (#2/12)
- EO-Gym: 64.0 (#3/11)
- Paraphrase-Induced Output-Mode Collapse: 26.2 (#4/5)
- MedAgentGym: 61.72 (#4/29)
- EHRBench: 66.79 (#5/31)
- LLM-as-a-Reviewer: 60.7 (#5/12)
- RecoAtlas: 24.4 (#5/12)
- BigCodeBench-Hard: 31.8 (#6/20)
Mistral Large 2 (Jul) — ELO 1587, #209
- RefusalBench: 87.2 (#2/19)
- RAGTruth: 85.9 (#2/11)
- Multi-IF: 0.81 (#7/33)
- ComplexFuncBench: 20.1 (#11/18)
- HELM Lite: 78.7704 (#11/77)
- Defects4J: 24.5 (#25/33)
- HealthBench Hard: 35.0 (#30/55)
- AA-LCR: 1.7 (#293/355)
GLM-4.6V — ELO 1587, #210
- MechVQA: 78.91 (#2/17)
- LithoBench: 94.5 (#3/13)
- LongWebBench: 5.96 (#6/13)
- ROSE: 20.7 (#9/10)
- OmniMatBench: 21.2 (#11/13)
- CULTURE-MT: 16.07 (#11/15)
- TOBench: 14.0 (#11/18)
- CheXpercept: 69.3 (#12/14)
- From Perception to Action: 7.3 (#14/16)
- Perception or Prejudice: 4.6 (#17/27)
INTELLECT-3 — ELO 1587, #211
- AIIQ Composite IQ: 88.0 (#120/138)
DeepSeek V3 Chat — ELO 1586, #213
- ChaosBench-Logic v2: 46.9 (#4/10)
- ATC Safety-Oriented Language Understanding Eval: 40.03 (#9/11)
- Open FinLLM Leaderboard: 29.49 (#9/25)
- Xent Games: 35.48 (#11/12)
- Natural2Code: 33.2 (#19/30)
- ACEBench: 0.79 (#23/31)
Llama 4 Maverick — ELO 1585, #214
- XL-SafetyBench: 92.0 (#2/10)
- TableVista: 54.7 (#3/29)
- DentalGPT Dental Eval Suite: 58.5 (#3/16)
- TukaBench: 21.9 (#4/13)
- CodeGolf Bench: 44.85 (#5/9)
- ClinPivot: 61.93 (#6/6)
- AssertLLM2: 12.69 (#6/6)
- FrontierOR: 13.0 (#7/7)
- HalluScore: 1.33 (#10/17)
- DiagnosticIQ: 63.29 (#11/33)
Gemini 2.0 Flash — ELO 1585, #215
- Natural2Code: 92.9 (#1/30)
- ERGeoBench: 5.77 (#3/9)
- RoboBench: 45.04 (#4/15)
- XDomainBench: 27.2 (#7/14)
- StructEval: 62.55 (#7/12)
- AfroBench-Lite: 66.43 (#7/24)
- OPT-BENCH: 53.0 (#10/19)
- LibEvoBench: 66.0 (#11/13)
- Time to REFLECT: 4.5 (#14/14)
- K-12EduBench: 64.05 (#15/23)
Qwen 2.5 72B Instruct — ELO 1583, #217
- INVESTORBENCH: 46.15 (#1/14)
- C4STYLI: 79.9 (#2/18)
- ThaiSafetyBench: 10.99 (#4/24)
- PhiBench: 64.6 (#4/10)
- Open FinLLM Leaderboard: 41.36 (#5/25)
- AutoLogi: 53.5 (#7/10)
- TuRTLe Module Completion (NotSoTiny): 14.7 (#8/15)
- RAGTruth: 81.9 (#9/11)
- HealthBench Hard: 49.0 (#11/55)
- FullStackBench en: 56.88 (#11/31)
DeepSeek R1 — ELO 1583, #218
- Enhancing Agent Safety Judgment: 94.53 (#1/6)
- LexGenius: 64.59 (#1/9)
- MedHELM: 66.25 (#1/9)
- CodeGolf Bench: 60.21 (#2/9)
- Plant, Persist, Trigger: 42.7 (#2/7)
- Disentangling Language Roles in Multilingual L: 82.2 (#2/20)
- OpenHuEval: 62.31 (#2/10)
- EduGuardBench: 75.0 (#2/14)
- Exploring Autonomous Agentic Data Engineering: 2.5 (#3/6)
- Decomposing and Measuring Evaluation Awareness: 5.6 (#3/9)
Gemma 4 12B — ELO 1583, #220
- AIIQ Composite IQ: 92.0 (#112/138)
Hermes 4 405B — ELO 1582, #221
- EQ-Bench: 1361.4 (#36/79)
- Creative Writing v3: 1416.8 (#47/123)
Qwen 3.5 9B — ELO 1581, #223
- Managing Procedural Memory in LLM Agents: 3.0 (#2/13)
- Honeyval: 63.1 (#4/6)
- OpenComputer: 7.8 (#7/8)
- TeleResilienceBench: 17.4 (#7/8)
- IFMTBench: 71.22 (#10/15)
- SeePhys Pro: 12.8 (#10/16)
- Multi-domain Multi-modal Document Classificati: 38.69 (#13/15)
- STT-Arena: 19.53 (#16/23)
- HMMT February 2026: 71.21 (#23/25)
- HMMT 2025: 83.2 (#48/89)
Gemini 2.0 Flash (001) — ELO 1581, #224
- Defects4J: 33.0 (#15/33)
- MATH 500: 88.0 (#29/58)
- Visual-Language Understanding: 39.85 (#35/54)
- EnigmaEval: 0.63 (#37/39)
- MGSM: 89.02 (#48/67)
- EQ-Bench: 910.9 (#66/79)
- Creative Writing v3: 1248.8 (#69/123)
- CritPt: 0.0 (#168/342)
- AA-LCR: 28.3 (#194/355)
GPT-4o (2024-11-20) — ELO 1581, #225
- Defects4J: 35.0 (#12/33)
- BigCodeBench-Hard: 27.7 (#20/20)
- CaseLaw v2: 59.7 (#26/54)
- LVBench: 48.9 (#27/49)
- ACEBench: 0.9 (#29/31)
- MGSM: 90.36 (#38/67)
- MATH 500: 74.0 (#45/58)
- CritPt: 0.0 (#191/342)
Magistral Medium 1.2 — ELO 1581, #226
- IOI: 0.67 (#57/58)
- MGSM: 74.62 (#64/67)
- Epoch AI - Scicode: 39.24 (#91/128)
- CritPt: 0.3 (#129/342)
- AA-LCR: 51.3 (#134/355)
North Mini Code — ELO 1581, #228
- AA-LCR: 32.3 (#181/355)
Qwen 2.5 72B — ELO 1580, #229
- NeedleBench: 81.02 (#1/12)
- StatABench: 60.6 (#3/9)
- The Wrong Kind of Right: 38.6 (#3/26)
- MIRAGE: 22.0 (#3/6)
- AIM-Bench (inventory manager): 21.28 (#4/5)
- BBH: 79.8 (#4/16)
- StressEval: 33.9 (#5/10)
- MUSE: 10.38 (#7/15)
- AttuneBench: 52.23 (#7/11)
- MixRea: 16.0 (#7/21)
Llama 3.1 405B Instruct — ELO 1580, #230
- AutoLogi: 59.42 (#5/10)
- RAGTruth: 82.9 (#5/11)
- ComplexMCP: 26.94 (#9/17)
- Defects4J: 28.9 (#19/33)
- EQ-Bench: 799.0 (#73/79)
- Creative Writing v3: 969.8 (#91/123)
- AA-LCR: 24.3 (#206/355)
- CritPt: 0.0 (#240/342)
DeepSeek V3 — ELO 1579, #232
- MIRAGE: 26.0 (#1/6)
- Diagnosing LLM Arbitration Behavior over Pre-e: 23.5 (#1/7)
- AIM-Bench (inventory manager): 14.72 (#1/5)
- K-12EduBench: 79.67 (#2/23)
- LinAlg-Bench: 99.5 (#3/10)
- OpenHuEval: 57.1 (#3/10)
- MedAgentBench: 62.67 (#3/12)
- StatABench: 57.4 (#4/9)
- AIPatient Arena: 4.75 (#4/5)
- EduGuardBench: 73.0 (#4/14)
GPT-5 Nano — ELO 1578, #236
- Seneca-TRBench: 92.9 (#2/5)
- CommunityFact: 75.59 (#3/15)
- Checkup2Action: 44.9 (#3/8)
- MultiEmo-Bench: 72.5 (#11/12)
- RealWorldQA: 71.8 (#11/12)
- Xent Games: 23.27 (#12/12)
- DRInQ: 45.0 (#13/13)
- ArchSIBench: 51.9 (#16/28)
- EHRBench: 57.8 (#19/31)
- CAIS Risk Index: 53.6 (#23/45)
Gemma 4 26B A4B — ELO 1576, #239
- BIG-Bench Extra Hard: 64.8 (#2/20)
- HardMTBench: 89.65 (#8/20)
- Qiskit QuantumKatas: 61.4 (#8/16)
- MedXpertQA: 58.1 (#11/26)
- Trip+: 61.36 (#12/18)
- MMGist: 45.6 (#13/27)
- RankJudge: 1444.0 (#13/21)
- ContractBench: 38.4 (#14/20)
- CiteVQA: 3.0 (#17/20)
- PerfCodeBench: 38.13 (#17/20)
GPT-4o (2024-08-06) — ELO 1576, #240
- Clembench Multimodal v1.6.5: 80.04 (#2/20)
- AutoLogi: 62.04 (#3/10)
- OPT-BENCH: 61.0 (#4/19)
- HumanEval+: 87.2 (#4/24)
- EvalPlus: 79.7 (#5/25)
- Generate README Eval: 33.13 (#6/14)
- MedAgentGym: 48.75 (#7/29)
- AfroBench-Lite: 65.8 (#8/24)
- The Metanym Game: 3.49 (#9/12)
- Forge: 62.1 (#9/21)
GPT-4o (2024-05-13) — ELO 1574, #241
- TextClass Benchmark: 1825.22 (#1/108)
- ToolSandbox: 73.0 (#1/13)
- HELM Lite: 95.9457 (#1/77)
- RepoQA: 90.6 (#3/33)
- RAGTruth: 84.3 (#3/11)
- Clembench Multimodal v1.6.5: 69.56 (#4/20)
- MedHELM: 56.9643 (#5/9)
- HELM MedQA: 87.674 (#6/13)
- HELM Safety: 94.5905 (#15/57)
- LVBench: 30.8 (#41/49)
Gemini 1.5 Pro (001) — ELO 1574, #242
- ToolSandbox: 60.4 (#6/13)
- MedHELM: 24.1071 (#9/9)
- HELM Lite: 78.1802 (#12/77)
- HELM MedQA: 76.9384 (#12/13)
- HELM Safety: 92.4464 (#25/57)
DiffusionGemma 26B-A4B — ELO 1574, #243
- BIG-Bench Extra Hard: 47.6 (#4/20)
- MedXpertQA: 49.0 (#13/26)
- OmniDocBench 1.5: 31.9 (#59/64)
- CritPt: 0.3 (#118/342)
- AA-LCR: 14.3 (#245/355)
Gemini 2.5 Flash Lite — ELO 1573, #244
- MCJudgeBench: 85.5 (#2/7)
- TRAP: 31.2 (#4/22)
- Truthful AI Advisors (Crawford-Sobel cheap-tal: 77.6 (#4/4)
- AIM-Bench (inventory manager): 30.38 (#5/5)
- Agentick: 18.7 (#6/15)
- DVMap: 45.3 (#7/9)
- Disentangling Language Roles in Multilingual L: 76.7 (#10/20)
- Omni-DeepSearch: 2.19 (#10/12)
- VerdictBench: 52.92 (#11/17)
- CAIS Vision Capabilities Index: 47.9 (#27/35)
Llama 4 Maverick Instruct — ELO 1573, #245
- MGSM: 92.44 (#16/67)
- Poker Agent: 890.5 (#17/17)
- MATH 500: 85.2 (#33/58)
- HealthBench Hard: 32.0 (#37/55)
- MedCode: 36.51 (#42/63)
- MedScribe: 54.22 (#60/61)
- EQ-Bench: 806.8 (#71/79)
- Creative Writing v3: 942.4 (#97/123)
Gemini 1.5 Pro (002) — ELO 1571, #247
- Translation Set1 to en spBleu: 45.6 (#1/13)
- Translation Set1 to en COMET22: 89.1 (#2/13)
- Translation en to Set1 COMET22: 89.1 (#3/13)
- Translation en to Set1 spBleu: 43.0 (#3/13)
- MBPP+: 74.6 (#5/25)
- EvalPlus: 76.95 (#11/25)
- Defects4J: 36.4 (#11/33)
- HumanEval+: 79.3 (#12/24)
- MATH 500: 82.8 (#35/58)
- MGSM: 89.2 (#46/67)
Claude Haiku 4.5 (20251001) — ELO 1569, #249
- Arena AI Document: 1422.0 (#21/27)
- WritingBench: 77.09 (#28/66)
O1 Mini — ELO 1568, #251
- FullStackBench en: 64.73 (#2/31)
- GraphARC: 58.0 (#4/11)
- K-12EduBench: 54.46 (#19/23)
Qwen 3 VL 32B Instruct — ELO 1566, #254
- FIND: 77.8 (#1/9)
- CulMind: 46.6 (#3/14)
- TriViewBench: 61.38 (#4/19)
- MechVQA: 76.5 (#4/17)
- OCRBench-V2 (en): 67.4 (#4/38)
- OCR-Robust: 72.82 (#5/16)
- WikiVQABench: 64.0 (#5/15)
- EO-Gym: 59.0 (#7/11)
- SpaceDG: 45.2 (#10/29)
- HarmVideoBench: 79.0 (#11/21)
Nova Premier — ELO 1566, #255
- VTB: 2.0 (#15/17)
- AA-LCR: 30.0 (#190/355)
- CritPt: 0.0 (#283/342)
Devstral Small 2 — ELO 1565, #258
- Epoch AI - Scicode: 28.82 (#113/128)
Llama 3.1 70B — ELO 1564, #259
- SAGE: 71.54 (#3/12)
- CLEAR: 3.1 (#3/15)
- The Wrong Kind of Right: 26.5 (#6/26)
- StressEval: 31.7 (#6/10)
- StreamProfileBench: 40.38 (#7/14)
- LegalCiteBench: 64.3 (#7/21)
- NeedleBench: 72.37 (#7/12)
- Multi-IF: 0.83 (#9/33)
- K-FinHallu: 70.0 (#10/12)
- MUSE: 5.5 (#10/15)
Gemini 2.0 Flash Lite — ELO 1563, #260
- BIG-Bench Extra Hard: 8.0 (#10/20)
- IneqMath: 1.5 (#38/42)
Gemini 1.5 Pro — ELO 1562, #261
- Multi-IF: 0.76 (#4/33)
- LLM Game Benchmark: 35.2 (#4/7)
- MedAgentBench: 62.0 (#4/12)
- StructEval: 71.75 (#5/12)
- PhysicsFinals: 38.4 (#5/34)
- AfroBench-Lite: 62.79 (#10/24)
- K-12EduBench: 67.88 (#12/23)
- Natural2Code: 42.3 (#13/30)
- ACEBench: 0.73 (#18/31)
- MedXpertQA: 26.16 (#21/26)
Qwen 2.5 32B — ELO 1562, #262
- ChaosBench-Logic v2: 47.8 (#3/10)
- EHRBench: 64.97 (#10/31)
- MixRea: 14.6 (#11/21)
Magistral Small 1.2 — ELO 1562, #263
- Epoch AI - Scicode: 35.19 (#108/128)
Mistral Small 4 — ELO 1562, #264
- BLXBench: 75.2 (#9/25)
- ContractBench: 42.4 (#13/20)
- StereoTales: 117.0 (#21/23)
- ALE-Bench: 497.63 (#59/79)
GPT-4 Preview (1106) — ELO 1562, #265
- Clembench Multimodal v1.6.5: 73.55 (#3/20)
- Aider Refactoring Benchmark: 50.6 (#5/10)
Llama 3.1 70B Instruct — ELO 1561, #266
- MathTutorBench: 55.22 (#3/8)
- INVESTORBENCH: 38.95 (#4/14)
- AutoLogi: 53.79 (#6/10)
- MedGUIDE: 54.3 (#7/25)
- AgentIF: 56.3 (#9/15)
- Geo-Eval: 4.12 (#11/11)
- FullStackBench en: 51.45 (#16/31)
- ThaiSafetyBench: 24.49 (#17/24)
- K-MetBench: 59.9 (#28/59)
- HealthBench Hard: 29.0 (#44/55)
Mistral Large 2 (Nov) Instruct (2411) — ELO 1560, #268
- TextClass Benchmark: 1720.36 (#13/108)
- Defects4J: 25.2 (#24/33)
- WritingBench: 54.44 (#55/66)
- EQ-Bench: 950.7 (#65/79)
- Creative Writing v3: 1090.1 (#87/123)
- CritPt: 0.0 (#264/342)
- AA-LCR: 5.3 (#282/355)
Command A — ELO 1559, #270
- Defects4J: 26.1 (#22/33)
Claude 3 Opus (20240229) — ELO 1559, #271
- RepoQA: 90.6 (#1/33)
- ToolSandbox: 69.2 (#2/13)
- Aider Refactoring Benchmark: 72.3 (#3/10)
- Clembench Multimodal v1.6.5: 68.16 (#5/20)
- HELM Safety: 96.745 (#7/57)
- HELM Lite: 71.2737 (#19/77)
Qwen 3 32B — ELO 1558, #273
- ChLogic: 99.07 (#2/7)
- StakeBench: 17.6 (#2/15)
- Active Evidence-Seeking and Diagnostic Reasoni: 48.3 (#2/15)
- ASPI: 18.7 (#3/10)
- RedSage-Bench: 85.4 (#3/17)
- AgentIF: 58.4 (#3/15)
- Finetuning with Scientific Data Increases Hall: 66.0 (#4/18)
- NRITYAM: 43.76 (#4/7)
- Diagnosing LLM Arbitration Behavior over Pre-e: -11.5 (#4/7)
- K-FinHallu: 77.5 (#4/12)
Grok 2 (1212) — ELO 1557, #276
- MATH 500: 78.4 (#40/58)
- MGSM: 86.15 (#58/67)
Llama 3 70B — ELO 1557, #277
- EHRBench: 63.35 (#11/31)
- MixRea: 14.6 (#12/21)
Qwen 3 VL 8B — ELO 1557, #278
- CounterCount: 47.74 (#2/7)
- From Hallucination to Grounding: 52.16 (#4/13)
- AgroVG: 32.7 (#4/13)
- LaViSA: 72.5 (#7/10)
- Beyond Binary: 75.23 (#7/14)
- Done, But Not Sure: 25.3 (#9/20)
- LiveK12Bench: 79.6 (#10/12)
- SVFSearch: 86.5 (#10/75)
- SpatialBabel: 64.0 (#10/14)
- EgoCoT-Bench: 65.42 (#11/20)
Qwen 3 30B A3B — ELO 1556, #279
- HalluWorld: 23.3 (#2/13)
- PrivacyPeek: 79.95 (#5/10)
- StakeBench: 14.0 (#6/15)
- Forge: 76.8 (#6/21)
- EComAgentBench: 19.5 (#7/7)
- Prosa: 73.9 (#8/16)
- OPT-BENCH: 54.0 (#9/19)
- PassBench: 13.9 (#10/11)
- LegalCiteBench: 62.8 (#10/21)
- PinTrace: 32.85 (#10/10)
GPT-4 Turbo — ELO 1556, #280
- TextClass Benchmark: 1781.47 (#3/108)
- HELM Lite: 89.8402 (#4/77)
- LLM Game Benchmark: 35.0 (#5/7)
- MedFrameQA: 46.69 (#6/11)
- HumanEval+: 86.6 (#6/24)
- ComplexFuncBench: 49.5 (#7/18)
- Aider Refactoring Benchmark: 34.1 (#8/10)
- RepoQA: 76.4 (#10/33)
- HELM Safety: 96.0619 (#11/57)
- BigCodeBench-Hard: 29.1 (#13/20)
Command A+ — ELO 1556, #282
- CharXiv-R: 52.7 (#55/60)
- Epoch AI - Scicode: 37.85 (#97/128)
- CritPt: 0.3 (#117/342)
- AIIQ Composite IQ: 88.0 (#119/138)
- AA-LCR: 46.0 (#147/355)
GLM-4.7 Flash — ELO 1555, #283
- K12-Bench: 31.7 (#7/11)
- VeriContest: 0.21 (#10/10)
- POLAR-Bench: 80.0 (#12/22)
- MUSE: 2.83 (#14/15)
- EQ-Bench: 1083.0 (#53/79)
- Creative Writing v3: 1185.4 (#78/123)
QVQ-72B-Preview — ELO 1555, #285
- MedFrameQA: 46.44 (#7/11)
nova-2-lite-v1 — ELO 1554, #287
- ALE-Bench: 236.25 (#73/79)
Solar Pro 3 — ELO 1553, #290
- Epoch AI - Scicode: 24.65 (#123/128)
Gemma 3 27B (IT) — ELO 1551, #291
- Do LLMs Build World Models From Text? A Multil: 28.7 (#2/14)
- HealthBench Hard: 59.0 (#2/55)
- TriggerBench: 79.0 (#4/14)
- FIND: 73.7 (#4/9)
- Clembench Multimodal v1.6.5: 61.39 (#6/20)
- Geo-Eval: 5.16 (#7/11)
- GR-Ben: 34.7 (#7/22)
- CodeGolf Bench: 20.92 (#8/9)
- SceneFunRI: 7.6 (#9/17)
- DentalGPT Dental Eval Suite: 51.5 (#12/16)
GLM-4.5V — ELO 1550, #295
- CulMind: 43.8 (#6/14)
- StemBind: 34.6 (#8/24)
- DentalGPT Dental Eval Suite: 54.2 (#9/16)
- Math-VR: 49.6 (#11/31)
- HarmVideoBench: 65.5 (#20/21)
GPT-4 Preview (0125) — ELO 1550, #296
- ToolSandbox: 64.3 (#4/13)
Qwen 3 30B A3B 2507 Instruct — ELO 1549, #297
- Fin-RATE: 17.63 (#7/16)
- JuICE: 33.74 (#8/10)
- SuiChat-CN: 63.8 (#24/44)
- K-MetBench: 64.7 (#25/59)
- AA-LCR: 22.7 (#213/355)
- CritPt: 0.0 (#307/342)
Nova Pro (v1) — ELO 1548, #300
- FinanceArena: 20.3 (#19/19)
Sarvam 105B — ELO 1547, #304
- HMMT 2025: 85.8 (#42/89)
Llama 3.3 70B — ELO 1546, #305
- DiagFlowBench: 85.0 (#1/10)
- Medical Information Response Audit (MIRA): 2.95 (#1/5)
- The Compliance Gap: 10.0 (#1/6)
- MIRAGE: 24.0 (#2/6)
- Truthful AI Advisors (Crawford-Sobel cheap-tal: 92.6 (#2/4)
- CommonWhy: 21.4 (#2/7)
- When Context Flips, Safety Breaks: 38.0 (#3/12)
- Ukrainian Legal Text Benchmark (EDRSR subset): 81.6 (#3/7)
- PreScam: 74.8 (#3/5)
- EHRBench: 67.28 (#4/31)
Llama 3.3 70B Instruct — ELO 1545, #307
- SECQUE: 65.0 (#2/7)
- DVMap: 46.4 (#4/9)
- PlanBench-XL: 18.96 (#5/10)
- PrivacyPeek: 79.61 (#6/10)
- PhiBench: 57.1 (#6/10)
- RAGTruth: 82.6 (#6/11)
- TraceEval: 44.3 (#7/10)
- CombEval: 36.6 (#8/11)
- TAEF (Text Analytics Evaluation Framework): 66.5 (#8/11)
- Fin-RATE: 16.76 (#8/16)
Mistral Small 3.2 — ELO 1545, #308
- EQUITRIAGE: 35.0 (#5/5)
- FINESSE-Bench: 57.51 (#28/31)
- SuiChat-CN: 58.1 (#33/44)
- EQ-Bench: 1071.6 (#57/79)
- Creative Writing v3: 1242.2 (#71/123)
- Epoch AI - Scicode: 26.39 (#118/128)
- AA-LCR: 17.3 (#233/355)
- AA MMLU-Pro: 68.05 (#237/346)
- CritPt: 0.0 (#270/342)
Qwen 3 VL 8B Instruct — ELO 1545, #309
- CulMind: 48.7 (#2/14)
- FIND: 74.1 (#2/9)
- SocialPersona: 32.2 (#4/6)
- TriViewBench: 60.32 (#6/19)
- WikiVQABench: 63.1 (#6/15)
- CFMME: 52.83 (#7/16)
- Checkup2Action: 33.1 (#7/8)
- Chartographer: 90.7 (#8/15)
- WildRoadBench: 21.3 (#9/25)
- OCRBench-V2 (en): 65.4 (#9/38)
Mistral Medium 3 — ELO 1544, #311
- Defects4J: 34.9 (#13/33)
- Visual-Language Understanding: 34.59 (#42/54)
- AA-LCR: 28.0 (#197/355)
- CritPt: 0.0 (#266/342)
Devstral Medium — ELO 1544, #313
- CritPt: 0.0 (#160/342)
- AA-LCR: 28.7 (#193/355)
- AA MMLU-Pro: 70.8 (#217/346)
Llama 4 Scout Instruct — ELO 1543, #314
- JuICE: 17.32 (#10/10)
- HealthBench Hard: 32.0 (#38/55)
- MATH 500: 79.2 (#38/58)
- MGSM: 87.96 (#50/67)
- WritingBench: 52.02 (#58/66)
- MedCode: 23.31 (#61/63)
- MedScribe: 50.59 (#61/61)
- EQ-Bench: 608.2 (#76/79)
- Creative Writing v3: 909.0 (#98/123)
DeepSeek R1 Distill Llama 70B — ELO 1543, #315
- CodeGolf Bench: 44.18 (#6/9)
- MedAgentGym: 50.07 (#6/29)
- AgentIF: 55.0 (#11/15)
- HealthBench Hard: 47.0 (#13/55)
- POLAR-Bench: 69.93 (#19/22)
- OJBench: 16.38 (#21/44)
- Defects4J: 22.1 (#29/33)
- HMMT 2025: 33.33 (#78/89)
- CritPt: 0.0 (#156/342)
- AA-LCR: 11.0 (#258/355)
Magistral Medium — ELO 1543, #316
- Defects4J: 32.1 (#16/33)
- FinanceArena: 31.8 (#17/19)
- CritPt: 0.3 (#128/342)
- AA-LCR: 0.0 (#295/355)
Hermes-4-14B — ELO 1542, #317
- TuRTLe Spec-to-RTL (Verilator): 44.04 (#26/44)
- TuRTLe Spec-to-RTL (Icarus Verilog): 42.77 (#27/44)
- TuRTLe Code Completion (Icarus Verilog): 27.96 (#41/44)
- TuRTLe Code Completion (Verilator): 28.99 (#41/44)
DeepSeek V2.5 — ELO 1542, #319
- FullStackBench en: 58.65 (#8/31)
- Defects4J: 26.6 (#21/33)
QwQ-32B — ELO 1541, #321
- StressEval: 34.6 (#3/10)
- When the Manual Lies: 100.0 (#7/8)
- Merge-Bench: 32.1 (#8/18)
- TuRTLe Spec-to-RTL (Verilator): 63.75 (#11/44)
- FinanceArena: 42.6 (#11/19)
- TuRTLe Spec-to-RTL (Icarus Verilog): 62.6 (#13/44)
- OJBench: 19.02 (#18/44)
- TuRTLe Code Completion (Icarus Verilog): 40.05 (#28/44)
- TuRTLe Code Completion (Verilator): 41.14 (#28/44)
- WritingBench: 72.4 (#41/66)
Llama 4 Scout — ELO 1540, #323
- PinTrace: 38.59 (#7/10)
- CodeGolf Bench: 16.65 (#9/9)
- TableVista: 49.4 (#9/29)
- Qiskit QuantumKatas: 34.3 (#15/16)
- VTB: 1.58 (#16/17)
- PerfCodeBench: 19.6 (#20/20)
- MedLayXPlain: 51.1 (#23/32)
- IneqMath: 1.5 (#37/42)
- ZeroBench: 3.0 (#46/69)
- TextClass Benchmark: 1500.45 (#66/108)
Nova Pro — ELO 1540, #324
- Translation en to Set1 spBleu: 43.4 (#1/13)
- VIBench: 12.2 (#2/10)
- Ukrainian Legal Text Benchmark (EDRSR subset): 82.7 (#2/7)
- Translation Set1 to en spBleu: 44.4 (#2/13)
- Translation en to Set1 COMET22: 89.1 (#4/13)
- Translation Set1 to en COMET22: 89.0 (#4/13)
- LVBench: 41.6 (#33/49)
- Visual-Language Understanding: 26.27 (#49/54)
- AA-LCR: 19.0 (#228/355)
- CritPt: 0.0 (#284/342)
Qwen 3 14B — ELO 1539, #325
- ChLogic: 99.13 (#1/7)
- AgingBench: 7.9 (#1/8)
- LegalCiteBench: 71.6 (#2/21)
- StakeBench: 17.3 (#4/15)
- TRACER: 72.0 (#4/4)
- Active Evidence-Seeking and Diagnostic Reasoni: 45.1 (#4/15)
- GraphARC: 37.0 (#5/11)
- K-FinHallu: 75.6 (#5/12)
- StreamProfileBench: 41.33 (#5/14)
- CULTURE-MT: 41.32 (#6/15)
Llama 3.2 90B Vision Instruct — ELO 1539, #326
- MineExplorer: 9.96 (#16/18)
Qwen 3.5 4B — ELO 1539, #327
- MCJudgeBench: 85.3 (#3/7)
- SceneFunRI: 10.99 (#6/17)
- CodeClinic: 21.0 (#7/8)
- Benchmarking Large Language Models for Graphem: 43.98 (#8/37)
- SVFSearch: 87.9 (#8/75)
- TeleResilienceBench: 16.9 (#8/8)
- SAGE: 60.5 (#9/12)
- StemBind: 33.8 (#10/24)
- Are LLMs Ready to Assist Physicians? PhysAssis: 48.7 (#14/14)
- MMGist: 42.0 (#16/27)
Llama 3.1 Nemotron 70B Instruct — ELO 1539, #328
- HealthBench Hard: 41.0 (#22/55)
- Defects4J: 22.8 (#27/33)
- CritPt: 0.0 (#243/342)
- AA-LCR: 7.0 (#271/355)
Qwen 2.5 VL 72B Instruct — ELO 1539, #329
- MedFrameQA: 42.65 (#10/11)
- VideoMME w/o sub.: 73.3 (#11/61)
- OCRBench-V2 (en): 61.5 (#19/38)
- K-MetBench: 67.1 (#22/59)
- ScreenSpot-Pro: 53.3 (#27/34)
- LVBench: 47.3 (#29/49)
Pixtral Large — ELO 1539, #330
- ZEROBench-Sub: 18.68 (#32/45)
- AA-LCR: 10.3 (#261/355)
Gemini 1.5 Flash (001) — ELO 1536, #334
- HELM Lite: 71.6184 (#18/77)
- HELM Safety: 92.7331 (#23/57)
GPT-OSS-20B — ELO 1535, #336
- CombEval: 59.4 (#5/11)
- Fin-RATE: 18.69 (#5/16)
- PDP-Bench: 57.23 (#7/7)
- Managing Procedural Memory in LLM Agents: 2.0 (#8/13)
- Decomposing and Measuring Evaluation Awareness: 0.2 (#9/9)
- Large Language Models Lack Temporal Awareness: 54.05 (#9/14)
- AgentFloor: 42.9 (#9/17)
- Qiskit QuantumKatas: 59.7 (#10/16)
- MATH 500: 94.2 (#10/58)
- StreamProfileBench: 34.68 (#11/14)
InternVL3-8B — ELO 1534, #339
- MedLayXPlain: 64.7 (#5/32)
- On the Cultural Anachronism and Temporal Reaso: 36.2 (#6/11)
- CulMind: 42.8 (#7/14)
- Seeing Isn't Knowing: 47.5 (#8/9)
- Chartographer: 84.7 (#12/15)
- SceneFunRI: 4.09 (#12/17)
- SpatialBabel: 54.1 (#14/14)
- WildRoadBench: 10.9 (#19/25)
- C3-Bench: 4.1 (#21/33)
- SpaceDG: 40.1 (#21/29)
Claude 3 Opus — ELO 1534, #341
- OSWorld: 83.39 (#4/66)
- MBPP+: 73.3 (#9/25)
- Natural2Code: 48.3 (#10/30)
- EvalPlus: 75.35 (#12/25)
- HumanEval+: 77.4 (#16/24)
- SVFSearch: 69.0 (#21/75)
Mistral Small 3.1 — ELO 1530, #343
- Polaris-Bench: 19.4 (#14/15)
- Defects4J: 22.5 (#28/33)
- TextClass Benchmark: 1484.82 (#70/108)
- EQ-Bench: 783.6 (#74/79)
- Creative Writing v3: 867.9 (#102/123)
- Epoch AI - Scicode: 26.5 (#117/128)
- AA-LCR: 19.7 (#226/355)
- CritPt: 0.0 (#269/342)
GPT-4 — ELO 1530, #344
- CyberSecEval: 19.87 (#2/7)
- Open FinLLM Leaderboard: 48.34 (#2/25)
- HumanEval+: 79.3 (#13/24)
Qwen 3 VL 30B A3B Instruct — ELO 1529, #345
- MechVQA: 64.47 (#9/17)
- WritingBench: 82.6 (#12/66)
- OCRBench-V2 (en): 63.2 (#14/38)
- HarmVideoBench: 76.3 (#15/21)
- Multi-IF: 66.1 (#16/33)
- LVBench: 62.5 (#19/49)
- OCRBench-V2 (zh): 57.8 (#24/37)
- K-MetBench: 62.2 (#26/59)
- CharXiv-R: 48.9 (#57/60)
- Creative Writing v3: 84.6 (#119/123)
Ling-2.6-flash — ELO 1529, #346
- AA-LCR: 25.0 (#204/355)
- CritPt: 0.0 (#235/342)
gemma-4-E4B-it — ELO 1528, #347
- SciPaths: 6.1 (#6/10)
- Chartographer: 86.0 (#11/15)
- MT-JailBench: 78.62 (#11/21)
- StemBind: 30.6 (#15/24)
- TableVista: 27.0 (#24/29)
- EnterpriseArena: 0.0 (#26/27)
Mistral Small 3 — ELO 1528, #348
- TRLawBench: 62.5 (#21/32)
- EQ-Bench: 801.7 (#72/79)
- Creative Writing v3: 833.2 (#107/123)
- AA-LCR: 0.0 (#324/355)
Mixtral 8x22B Instruct — ELO 1525, #351
- ENAMEL: 40.8 (#5/31)
Mistral Large — ELO 1525, #352
- LinAlg-Bench: 98.2 (#5/10)
- AgentLeak: 99.3 (#5/5)
- DiagnosticIQ: 63.15 (#13/33)
- HELM Lite: 36.8548 (#52/77)
Gemini 1.5 Flash (002) — ELO 1524, #353
- Translation Set1 to en COMET22: 88.8 (#5/13)
- Translation Set1 to en spBleu: 42.9 (#7/13)
- Translation en to Set1 COMET22: 88.5 (#8/13)
- Translation en to Set1 spBleu: 40.0 (#9/13)
- EvalPlus: 71.55 (#17/25)
- HumanEval+: 75.6 (#17/24)
- MBPP+: 67.5 (#23/25)
- MATH 500: 78.8 (#39/58)
- Visual-Language Understanding: 34.03 (#43/54)
- MGSM: 86.58 (#55/67)
Ministral 3 14B — ELO 1524, #354
- P3B3: 90.7 (#7/20)
- POLAR-Bench: 71.96 (#16/22)
- ContractBench: 28.3 (#16/20)
- AgentFloor: 18.8 (#16/17)
- BacktestBench: 26.98 (#17/23)
- Mirror, Mirror on the Wall: 33.0 (#27/43)
- OCRBench-V2 (zh): 44.0 (#35/37)
- OCRBench-V2 (en): 43.3 (#36/38)
- AA-LCR: 22.0 (#215/355)
Gemini 1.5 Flash — ELO 1524, #355
- PhysicsFinals: 57.4 (#2/34)
- Multi-IF: 0.72 (#2/33)
- Natural2Code: 79.8 (#4/30)
- LLM Game Benchmark: 32.5 (#6/7)
Claude 3.5 Haiku — ELO 1524, #356
- Fin-Bias: 97.1 (#6/19)
- ComplexFuncBench: 45.8 (#9/18)
- K-12EduBench: 44.98 (#21/23)
- DiagnosticIQ: 46.93 (#30/33)
- Epoch AI - Scicode: 27.43 (#116/128)
Mixtral 8x22B Instruct (v0.1) — ELO 1523, #358
- RepoQA: 67.8 (#12/33)
GPT-4 (0613) — ELO 1522, #362
- ENAMEL: 45.4 (#3/31)
- HELM Lite: 90.8908 (#3/77)
- TextClass Benchmark: 1747.59 (#8/108)
- Inverse Turing Bench: 48.29 (#19/24)
- Chess Puzzles (Epoch AI): 4.0 (#139/153)
Qwen3 Omni 30B A3B Instruct — ELO 1522, #363
- Omni-DeepSearch: 3.44 (#9/12)
- OCRBench v2: 61.3 (#10/29)
- OCRBench-V2 (zh): 60.0 (#15/37)
- AudioMC: 24.34 (#16/31)
- OCRBench-V2 (en): 61.3 (#20/38)
Qwen 2.5 Coder 32B Instruct — ELO 1521, #365
- EvalPlus: 82.1 (#3/25)
- MBPP+: 77.0 (#3/25)
- HumanEval+: 87.2 (#3/24)
- TraceEval: 46.6 (#6/10)
- TuRTLe Module Completion (NotSoTiny): 11.85 (#11/15)
- FullStackBench en: 56.88 (#12/31)
- BigCodeBench-Hard: 27.7 (#19/20)
- Defects4J: 27.1 (#20/33)
nemotron-3-nano-30B-a3B — ELO 1521, #366
- TW-LegalBench: 52.6 (#9/13)
O1 Mini (2024-09-12) — ELO 1521, #367
- EvalPlus: 83.9 (#2/25)
- MBPP+: 78.8 (#2/25)
- HumanEval+: 89.0 (#2/24)
- HELM Safety: 95.5323 (#12/57)
- TextClass Benchmark: 1626.65 (#29/108)
Llama 3 70B Instruct — ELO 1520, #368
- CyberSecEval: 29.27 (#3/7)
- AgentClinic: 19.0 (#6/7)
- RepoQA: 82.2 (#7/33)
- Open FinLLM Leaderboard: 31.23 (#7/25)
- C4STYLI: 62.4 (#12/18)
- MBPP+: 69.0 (#19/25)
- EvalPlus: 70.5 (#20/25)
- HealthBench Hard: 41.0 (#21/55)
- HumanEval+: 72.0 (#23/24)
- CritPt: 0.0 (#238/342)
EXAONE 4.0 32B — ELO 1520, #369
- K-FinHallu: 71.7 (#8/12)
- K-MetBench: 59.9 (#29/59)
aya-expanse-32B — ELO 1519, #370
- CommunityFact: 69.93 (#5/15)
Gemma 3 27B — ELO 1517, #371
- NeedleBench: 80.38 (#2/12)
- Natural2Code: 84.5 (#2/30)
- SAGE: 66.17 (#5/12)
- MixRea: 16.2 (#6/21)
- CounterCount: 36.96 (#6/7)
- MolDeTox: 1.55 (#6/15)
- ECLeKTic: 16.7 (#6/14)
- TukaBench: 15.6 (#7/13)
- CardioLens: 51.77 (#7/24)
- When Context Flips, Safety Breaks: 27.8 (#7/12)
QwQ 32B-Preview — ELO 1516, #374
- VRBench: 35.9 (#28/29)
Qwen 3 VL 4B — ELO 1513, #375
- TRAP: 40.4 (#1/22)
- LaViSA: 78.2 (#5/10)
- Done, But Not Sure: 25.6 (#8/20)
- SVFSearch: 84.6 (#12/75)
Ministral 3 8B — ELO 1512, #377
- AgentFloor: 55.2 (#3/17)
- POLAR-Bench: 76.12 (#14/22)
GPT-4 (0314) — ELO 1512, #378
- EQ-Bench: 577.6 (#77/79)
- Creative Writing v3: 875.4 (#101/123)
GPT-4o Mini — ELO 1511, #379
- MASLegalBench: 84.32 (#2/5)
- Finetuning with Scientific Data Increases Hall: 66.1 (#3/18)
- On Stable Long-Form Generation: 33.9 (#3/13)
- LaborBench: 67.0 (#3/5)
- DVMap: 46.3 (#5/9)
- PhiBench: 58.7 (#5/10)
- DiagFlowBench: 75.4 (#6/10)
- ATC Safety-Oriented Language Understanding Eval: 44.83 (#6/11)
- Beyond Continuity: 64.5 (#6/10)
- Translation en to Set1 COMET22: 88.7 (#6/13)
GPT-4o Mini (2024-07-18) — ELO 1511, #380
- Clembench Multimodal v1.6.5: 58.46 (#7/20)
- MedHELM: 39.2857 (#7/9)
- HumanEval+: 83.5 (#8/24)
- EvalPlus: 77.85 (#9/25)
- Generate README Eval: 32.16 (#11/14)
- MBPP+: 72.2 (#12/25)
- HELM MedQA: 74.9503 (#13/13)
- Balrog: 17.4 (#14/16)
- HELM Lite: 75.6818 (#14/77)
- ACEBench: 0.76 (#19/31)
Reka Flash 3 — ELO 1511, #381
- Creative Writing v3: 1244.5 (#70/123)
Jamba 1.7 Large — ELO 1511, #383
- CritPt: 0.0 (#212/342)
- AA-LCR: 17.3 (#232/355)
- AA MMLU-Pro: 57.65 (#276/346)
Gemma 3 12B (IT) — ELO 1510, #385
- FIND: 71.7 (#5/9)
- ALARB: 29.6 (#5/12)
- Human-Grounded Multimodal Benchmark with 900K: 54.5 (#8/11)
- TriBench-Ko: 62.9 (#8/13)
- GeoRC: 31.21 (#11/15)
- SceneFunRI: 4.09 (#13/17)
- GR-Ben: 25.2 (#14/22)
- ThaiSafetyBench: 20.4 (#15/24)
- FAM-Bench: 71.8 (#17/21)
- C4STYLI: 54.7 (#17/18)
Nova Micro — ELO 1510, #387
- Translation en to Set1 spBleu: 40.2 (#7/13)
- Translation Set1 to en COMET22: 88.7 (#8/13)
- Translation Set1 to en spBleu: 42.6 (#8/13)
- Translation en to Set1 COMET22: 88.5 (#9/13)
- AA-LCR: 9.7 (#263/355)
- CritPt: 0.0 (#282/342)
Phi-4 — ELO 1508, #388
- Diagnosing LLM Arbitration Behavior over Pre-e: -0.4 (#3/7)
- NRITYAM: 40.36 (#5/7)
- SECQUE: 56.0 (#5/7)
- BLUEX v2: 7.09 (#6/10)
- TriBench-Ko: 71.2 (#6/13)
- PhiBench: 56.2 (#7/10)
- Halluverse-M3: 70.38 (#8/14)
- PhysicsFinals: 29.1 (#10/34)
- BBH: 59.4 (#10/16)
- PersonaArena: 3.49 (#11/18)
Gemma 2 27B (IT) — ELO 1508, #389
- Fin-Bias: 97.5 (#5/19)
- Open FinLLM Leaderboard: 19.81 (#14/25)
- BIG-Bench Extra Hard: 4.0 (#17/20)
Granite 4.1 8B — ELO 1507, #391
- BLXBench: 64.6 (#12/25)
- Qiskit QuantumKatas: 32.3 (#16/16)
Qwen 1.5 72B Chat — ELO 1505, #394
- RepoQA: 67.0 (#13/33)
Qwen 2.5 VL 32B Instruct — ELO 1503, #396
- VideoMME w sub.: 77.9 (#16/65)
- VideoMME w/o sub.: 70.5 (#19/61)
- OCRBench-V2 (zh): 59.1 (#20/37)
- OCRBench-V2 (en): 57.2 (#26/38)
- LVBench: 49.0 (#26/49)
Qwen 3 8B — ELO 1499, #400
- Benchmarking Large Language Models for Graphem: 100.15 (#1/37)
- Finetuning with Scientific Data Increases Hall: 67.1 (#1/18)
- When Lower Privileges Suffice: 64.9 (#1/11)
- ProtStructQA: 68.05 (#2/4)
- Fin-Bias: 98.3 (#2/19)
- ChLogic: 98.4 (#3/7)
- MedCTA: 27.8 (#4/18)
- AgingBench: 6.2 (#4/8)
- CULTURE-MT: 49.6 (#4/15)
- Pseudo-Deliberation in Language Models: 44.2 (#4/4)
Magistral Small — ELO 1498, #403
- TuRTLe Spec-to-RTL (Icarus Verilog): 40.82 (#28/44)
- TuRTLe Spec-to-RTL (Verilator): 41.02 (#28/44)
- TuRTLe Code Completion (Icarus Verilog): 22.62 (#44/44)
- TuRTLe Code Completion (Verilator): 23.47 (#44/44)
- AA MMLU-Pro: 74.63 (#184/346)
- CritPt: 0.0 (#254/342)
- Epoch AI - ECI: 132.91 (#336/463)
Gemma 3 12B — ELO 1493, #409
- Natural2Code: 80.7 (#3/30)
- MixRea: 18.2 (#4/21)
- CounterCount: 40.22 (#5/7)
- NeedleBench: 75.31 (#5/12)
- P3B3: 92.9 (#6/20)
- Relevance as a Vulnerability: 2.46 (#6/8)
- The Age of Curiosity Meets the Age of AI: Benc: 4.83 (#6/13)
- SAGE: 64.5 (#7/12)
- ECLeKTic: 10.3 (#7/14)
- FullHome: 37.0 (#8/8)
DeepSeek R1 Distill Qwen 32B — ELO 1493, #410
- MedAgentGym: 47.05 (#9/29)
- POLAR-Bench: 81.19 (#10/22)
- AgentIF: 55.1 (#10/15)
- Active Evidence-Seeking and Diagnostic Reasoni: 33.9 (#11/15)
- HealthBench Hard: 44.0 (#15/55)
- SuiChat-CN: 68.9 (#18/44)
- OJBench: 17.83 (#20/44)
- PhysicsFinals: 6.8 (#27/34)
- Defects4J: 21.4 (#30/33)
- HMMT 2025: 33.33 (#77/89)
Claude 3.5 Haiku (20241022) — ELO 1493, #413
- MedGUIDE: 47.5 (#14/25)
- MATH 500: 64.2 (#55/58)
- WritingBench: 49.06 (#60/66)
- MGSM: 84.62 (#61/67)
- Creative Writing v3: 1178.6 (#82/123)
- TextClass Benchmark: 1413.9 (#85/108)
Gemma 2 27B — ELO 1492, #415
- MixRea: 14.3 (#14/21)
- AfroBench-Lite: 44.87 (#14/24)
- Benchmarking Large Language Models for Graphem: 24.56 (#16/37)
- PhysicsFinals: 18.3 (#16/34)
DeepSeek R1 0528 Qwen3 8B — ELO 1492, #416
- AA MMLU-Pro: 73.88 (#196/346)
gemma-4-E2B-it — ELO 1488, #420
- StemBind: 28.6 (#18/24)
Nova Lite — ELO 1488, #422
- Translation en to Set1 COMET22: 88.8 (#5/13)
- Translation en to Set1 spBleu: 41.5 (#5/13)
- Translation Set1 to en COMET22: 88.8 (#6/13)
- Translation Set1 to en spBleu: 43.1 (#6/13)
- LVBench: 40.4 (#36/49)
- Visual-Language Understanding: 25.5 (#51/54)
- AA-LCR: 17.7 (#231/355)
- CritPt: 0.0 (#281/342)
Sarvam 30B — ELO 1488, #424
- HMMT 2025: 73.3 (#59/89)
Qwen 1.5 32B Chat — ELO 1487, #425
- RepoQA: 33.8 (#25/33)
Qwen 2.5 7B — ELO 1484, #427
- Relevance as a Vulnerability: 3.45 (#1/8)
- LexGenius: 59.38 (#3/9)
- Benchmarking Large Language Models for Graphem: 79.39 (#4/37)
- Large Language Models Lack Temporal Awareness: 59.64 (#4/14)
- ERGeoBench: 2.24 (#6/9)
- MHGraphBench: 43.61 (#7/15)
- StatABench: 38.1 (#8/9)
- On Stable Long-Form Generation: 17.0 (#9/13)
- TW-LegalBench: 49.7 (#10/13)
- MixRea: 15.0 (#10/21)
Qwen 3 VL 4B Instruct — ELO 1481, #432
- FIND: 69.6 (#6/9)
- TriViewBench: 52.0 (#7/19)
- WikiVQABench: 60.2 (#7/15)
- EO-Gym: 49.0 (#9/11)
- MechVQA: 60.23 (#12/17)
- OCRBench-V2 (en): 63.7 (#12/38)
- WritingBench: 82.5 (#13/66)
- OCR-Robust: 49.79 (#16/16)
- SpaceDG: 39.7 (#22/29)
- LVBench: 56.2 (#22/49)
LFM2 24B A2B — ELO 1481, #433
- CritPt: 0.0 (#229/342)
- AA-LCR: 0.0 (#314/355)
InternVL3.5-8B — ELO 1480, #436
- AgroVG: 39.6 (#2/13)
- CardioLens: 52.51 (#5/24)
- From Hallucination to Grounding: 43.59 (#9/13)
- OCR-Robust: 57.26 (#10/16)
- MechVQA: 49.82 (#14/17)
- EgoCoT-Bench: 64.06 (#14/20)
- CC-OCR V2: 47.83 (#14/15)
- TriViewBench: 36.34 (#15/19)
- MedLayXPlain: 58.7 (#15/32)
- DiffCap-Bench: 51.9 (#16/16)
GPT-4.1 Nano — ELO 1477, #438
- EQUITRIAGE: 37.4 (#2/5)
- LexGenius: 53.43 (#5/9)
- GraphARC: 12.0 (#8/11)
- The Metanym Game: 3.22 (#10/12)
- ComplexFuncBench: 5.7 (#14/18)
- Multi-IF: 57.2 (#15/33)
- EHRBench: 60.48 (#16/31)
- BigCodeBench-Hard: 28.4 (#17/20)
- RefusalBench: 0.2 (#18/19)
- HELM Safety: 93.765 (#18/57)
GLM-4 9B Chat — ELO 1477, #439
- Fin-Bias: 98.1 (#3/19)
- NeedleBench: 66.51 (#11/12)
- AsyncTool: 6.88 (#15/19)
Hunyuan A13B-Instruct — ELO 1476, #440
- SuiChat-CN: 53.4 (#36/44)
Gemini 1.5 Flash-8B (001) — ELO 1476, #441
- Translation Set1 to en COMET22: 88.5 (#9/13)
- Translation Set1 to en spBleu: 41.4 (#10/13)
- Translation en to Set1 COMET22: 88.0 (#11/13)
- Translation en to Set1 spBleu: 38.2 (#11/13)
Claude 3 Sonnet (20240229) — ELO 1474, #443
- RepoQA: 87.4 (#5/33)
Gemma 4 E4B — ELO 1472, #445
- P3B3: 94.4 (#4/20)
- POLAR-Bench: 90.87 (#4/22)
- TeleResilienceBench: 23.7 (#5/8)
- IFMTBench: 72.56 (#9/15)
- IntentGrasp: 45.87 (#9/20)
- HardMTBench: 82.69 (#15/20)
- AIIQ Composite IQ: 85.0 (#125/138)
Ministral 3 3B — ELO 1469, #450
- AgentFloor: 42.7 (#10/17)
- POLAR-Bench: 65.25 (#21/22)
- AA-LCR: 11.7 (#253/355)
- AA MMLU-Pro: 52.35 (#290/346)
Qwen 3.5 2B — ELO 1468, #452
- Benchmarking Large Language Models for Graphem: 87.3 (#2/37)
- StemBind: 35.9 (#7/24)
- Agentick: 13.3 (#7/15)
- SAGE: 54.12 (#11/12)
- ArxivMath: 10.62 (#20/20)
- SVFSearch: 68.5 (#24/75)
- AIIQ Composite IQ: 79.0 (#135/138)
Mistral Nemo Instruct (2407) — ELO 1468, #453
- SECQUE: 46.0 (#7/7)
- Generate README Eval: 25.62 (#13/14)
- Creative Writing v3: 969.6 (#92/123)
Command-R+ — ELO 1468, #454
- IDE-Bench: 0.0 (#15/15)
Phi-3-medium-128k-instruct — ELO 1467, #457
- BBH: 81.4 (#3/16)
- RepoQA: 63.2 (#14/33)
Claude 3 Haiku — ELO 1466, #458
- Multi-IF: 0.73 (#3/33)
- Natural2Code: 36.2 (#17/30)
- MBPP+: 68.8 (#21/25)
- EvalPlus: 68.85 (#24/25)
- Creative Writing v3: 877.2 (#100/123)
GPT-3.5 Turbo — ELO 1465, #460
- CyberSecEval: 39.13 (#6/7)
- NarrativeWorldBench: 54.0 (#10/10)
- AutoLogi: 18.93 (#10/10)
- QUIET: 6.44 (#12/12)
- Natural2Code: 40.7 (#14/30)
- LM Market Cap LMC Score: 40.0 (#355/406)
Qwen 3 4B — ELO 1464, #463
- ProtStructQA: 70.33 (#1/4)
- CULTURE-MT: 60.98 (#3/15)
- MuPPET: 59.26 (#4/7)
- BehaviorBench: 26.9 (#5/20)
- Benchmarking Large Language Models for Graphem: 57.72 (#5/37)
- TIDE-Bench: 89.6 (#5/8)
- When2Speak: 20.6 (#6/6)
- Agentick: 8.5 (#9/15)
- PassBench: 10.8 (#11/11)
- When Seekers Are Hard to Help: 1.74 (#11/17)
c4ai-command-r-v01 — ELO 1464, #465
- ToolSandbox: 26.2 (#11/13)
Gemma 3n E4B — ELO 1463, #466
- CulturaQA: 69.1 (#7/9)
- TRLawBench: 50.0 (#30/32)
Molmo2-8B — ELO 1463, #467
- TableVista: 39.9 (#21/29)
- OCRBench-V2 (en): 44.7 (#34/38)
- OCRBench-V2 (zh): 21.1 (#37/37)
- AA-LCR: 0.0 (#297/355)
Claude 3 Haiku (20240307) — ELO 1462, #468
- ToolSandbox: 54.9 (#7/13)
- RepoQA: 81.8 (#8/33)
- HELM Safety: 87.7981 (#40/57)
- HELM Lite: 29.4206 (#63/77)
text-davinci-002 — ELO 1462, #470
- HELM: 90.5 (#3/65)
Qwen 2 7B Instruct — ELO 1461, #471
- IndoBias: 65.3 (#15/26)
Qwen 1.5 14B Chat — ELO 1461, #472
- RepoQA: 26.0 (#27/33)
Gemma 2 9B (IT) — ELO 1459, #474
- Fin-Bias: 97.0 (#7/19)
- Do LLMs Build World Models From Text? A Multil: 13.7 (#9/14)
- IndoBias: 67.3 (#12/26)
- Open FinLLM Leaderboard: 14.32 (#18/25)
- SuiChat-CN: 47.1 (#41/44)
Qwen 2.5 7B Instruct — ELO 1458, #475
- NL2Scratch: 97.9 (#4/8)
- Do LLMs Build World Models From Text? A Multil: 18.9 (#5/14)
- BLUEX v2: 6.93 (#7/10)
- Finetuning with Scientific Data Increases Hall: 65.2 (#7/18)
- ThaiSafetyBench: 14.43 (#7/24)
- AutoLogi: 27.28 (#8/10)
- StructEval: 59.03 (#9/12)
- MedGUIDE: 50.1 (#11/25)
- When Reviews Disagree: 28.54 (#13/13)
- Active Evidence-Seeking and Diagnostic Reasoni: 23.3 (#14/15)
Mixtral 8x7B Instruct (v0.1) — ELO 1458, #476
- RepoQA: 68.0 (#11/33)
- MedGUIDE: 49.1 (#12/25)
Gemma 2 9B — ELO 1455, #482
- StakeBench: 15.2 (#5/15)
- PhysicsFinals: 11.9 (#22/34)
Pixtral-12B — ELO 1454, #484
- Chartographer: 82.7 (#14/15)
- MMVU: 32.2 (#36/40)
Phi-3-small-8k-instruct — ELO 1454, #485
- BBH: 79.1 (#5/16)
Claude 2.1 — ELO 1454, #486
- Natural2Code: 34.4 (#18/30)
Llama 3.1 8B Instruct — ELO 1453, #487
- MathTutorBench: 47.0 (#4/8)
- EnterpriseArena: 40.0 (#5/27)
- SECQUE: 48.0 (#6/7)
- NL2Scratch: 96.5 (#7/8)
- Finetuning with Scientific Data Increases Hall: 65.0 (#8/18)
- PrivacyPeek: 72.76 (#8/10)
- StructEval: 61.77 (#8/12)
- BLUEX v2: 4.35 (#9/10)
- TAEF (Text Analytics Evaluation Framework): 65.5 (#9/11)
- AutoLogi: 25.53 (#9/10)
Llama 3.2 11B Instruct — ELO 1453, #488
- GeoRC: 25.86 (#12/15)
- CFMME: 20.2 (#16/16)
- MechVQA: 25.48 (#17/17)
- CritPt: 0.0 (#245/342)
- AA-LCR: 11.7 (#256/355)
Gemini 1.5 Flash-8B — ELO 1453, #489
- Natural2Code: 75.5 (#5/30)
Gemini 1.0 Pro — ELO 1448, #500
- EnactToM: 53.0 (#1/7)
Qwen 2.5 VL 7B Instruct — ELO 1448, #501
- LVBench: 45.3 (#30/49)
- VideoMME w sub.: 71.6 (#30/65)
- VideoMME w/o sub.: 65.1 (#31/61)
Gemma 3 4B (IT) — ELO 1446, #502
- IndoBias: 69.4 (#6/26)
- FIND: 61.5 (#7/9)
- ALARB: 24.5 (#10/12)
- SceneFunRI: 4.21 (#11/17)
- C4STYLI: 55.8 (#16/18)
- ThaiSafetyBench: 28.11 (#19/24)
- MedAgentGym: 21.57 (#19/29)
- EQ-Bench: 977.2 (#63/79)
- Creative Writing v3: 1113.6 (#85/123)
- CritPt: 0.0 (#175/342)
GPT-3.5 — ELO 1446, #504
- TukaBench: 38.1 (#1/13)
- PersonaArena: 3.46 (#12/18)
Mixtral 8x7B Instruct — ELO 1445, #506
- ENAMEL: 26.6 (#13/31)
- EnterpriseArena: 0.0 (#23/27)
- LiveSQLBench: 2.41 (#33/33)
Phi-3 Mini 4K Instruct — ELO 1443, #510
- BBH: 71.7 (#7/16)
DeepSeek R1 Distill Qwen 14B — ELO 1441, #512
- Active Evidence-Seeking and Diagnostic Reasoni: 23.3 (#13/15)
- OJBench: 14.17 (#25/44)
- HMMT 2025: 31.67 (#80/89)
Qwen 2.5 3B — ELO 1441, #513
- Relevance as a Vulnerability: 3.22 (#2/8)
- Forge: 13.8 (#20/21)
- EHRBench: 37.87 (#29/31)
Phi-4 Multimodal Instruct — ELO 1441, #514
- DentalGPT Dental Eval Suite: 52.0 (#11/16)
- AudioMC: 15.49 (#24/31)
- OCRBench v2: 38.1 (#29/29)
- OCRBench-V2 (zh): 37.3 (#36/37)
- OCRBench-V2 (en): 38.1 (#38/38)
Llama 3.1 8B — ELO 1439, #515
- When Context Flips, Safety Breaks: 77.6 (#1/12)
- Multi-IF: 0.69 (#1/33)
- When Lower Privileges Suffice: 55.9 (#2/11)
- MixRea: 20.3 (#2/21)
- CLEAR: 3.1 (#2/15)
- IndoBias: 71.4 (#3/26)
- The Wrong Kind of Right: 28.2 (#4/26)
- When2Speak: 33.7 (#4/6)
- TSCG: 78.4 (#4/13)
- Beyond Continuity: 69.9 (#5/10)
Gemma 3n E4B Instruct — ELO 1438, #516
- ECLeKTic: 19.0 (#5/14)
- CritPt: 0.0 (#177/342)
- AA MMLU-Pro: 48.83 (#298/346)
- AA-LCR: 0.0 (#308/355)
Claude 3 Sonnet — ELO 1437, #517
- Multi-IF: 0.78 (#5/33)
Jamba 1.7 Mini — ELO 1435, #521
- AA MMLU-Pro: 38.82 (#322/346)
GPT-3.5 Turbo (0613) — ELO 1434, #522
- HELM: 78.3 (#10/65)
- HELM Lite: 40.0283 (#47/77)
- HELM Safety: 85.2869 (#49/57)
- Creative Writing v3: 505.0 (#110/123)
GPT-3.5 Turbo (1106) — ELO 1434, #524
- BBH: 61.59 (#9/16)
- MBPP+: 69.7 (#15/25)
- EvalPlus: 70.2 (#21/25)
- Inverse Turing Bench: 39.86 (#24/24)
Gemma 4 E2B — ELO 1431, #528
- TeleResilienceBench: 24.5 (#4/8)
- IFMTBench: 58.69 (#13/15)
- POLAR-Bench: 79.74 (#13/22)
- IntentGrasp: 40.52 (#14/20)
- HardMTBench: 78.44 (#17/20)
- AIIQ Composite IQ: 80.0 (#134/138)
Qwen 2.5 3B Instruct — ELO 1431, #529
- When Simulation Lies: 39.0 (#17/22)
- OPT-BENCH: 15.0 (#19/19)
- PlantMarkerBench: 24.7 (#46/47)
Mistral-7B-v0.1 — ELO 1431, #530
- BBH: 56.1 (#12/16)
Mistral Nemo — ELO 1429, #532
- ECLeKTic: 7.1 (#9/14)
- TRLawBench: 59.4 (#24/32)
GPT-3.5 Turbo (0125) — ELO 1428, #534
- ToolSandbox: 65.6 (#3/13)
- Open FinLLM Leaderboard: 23.16 (#12/25)
- RepoQA: 60.4 (#17/33)
- TextClass Benchmark: 1560.51 (#44/108)
- Chess Puzzles (Epoch AI): 0.0 (#151/153)
- OTIS Mock AIME 2024-25: 2.22 (#220/230)
Llama 3 8B — ELO 1426, #538
- MixRea: 18.3 (#3/21)
- Diagnosing LLM Arbitration Behavior over Pre-e: -66.1 (#7/7)
- IntentGrasp: 28.36 (#19/20)
- EHRBench: 48.9 (#24/31)
- SuiChat-CN: 23.3 (#44/44)
SmolLM3-3B — ELO 1425, #540
- POLAR-Bench: 62.22 (#22/22)
Llama 3 8B Instruct — ELO 1423, #544
- Pseudo-Deliberation in Language Models: 47.7 (#2/4)
- CyberSecEval: 45.27 (#7/7)
- SciPaths: 4.1 (#9/10)
- K12-Bench: 7.2 (#10/11)
- When Reviews Disagree: 35.41 (#11/13)
- StructEval: 51.59 (#11/12)
- Active Evidence-Seeking and Diagnostic Reasoni: 24.8 (#12/15)
- EnterpriseArena: 20.0 (#13/27)
- Open FinLLM Leaderboard: 18.93 (#15/25)
- C4STYLI: 53.5 (#18/18)
Granite 4.0 H 1B — ELO 1422, #547
- AA MMLU-Pro: 27.74 (#336/346)
DeepSeek R1 Distill Llama 8B — ELO 1420, #548
- OJBench: 8.46 (#32/44)
- SuiChat-CN: 43.7 (#42/44)
Command R — ELO 1419, #549
- HELM Safety: 80.9403 (#54/57)
- HELM Lite: 33.365 (#56/77)
Qwen 3 1.7B — ELO 1417, #551
- When Multiple Scripts Matter: 29.73 (#2/6)
- CULTURE-MT: 83.53 (#2/15)
- ProtStructQA: 28.69 (#3/4)
- ActTraitBench: 0.19 (#15/15)
- ProactBench: 10.3 (#16/16)
- PersonaArena: 3.09 (#18/18)
- K-MetBench: 46.8 (#42/59)
Llama 2 70B Chat — ELO 1417, #552
- Finetuning with Scientific Data Increases Hall: 62.2 (#12/18)
Gemma 3 4B — ELO 1413, #556
- CounterCount: 44.57 (#4/7)
- Large Language Models Lack Temporal Awareness: 58.16 (#5/14)
- Natural2Code: 70.3 (#6/30)
- Benchmarking Large Language Models for Graphem: 56.69 (#7/37)
- SAGE: 60.79 (#8/12)
- LaViSA: 51.0 (#10/10)
- ECLeKTic: 4.6 (#10/14)
- The Age of Curiosity Meets the Age of AI: Benc: 4.4 (#11/13)
- NeedleBench: 64.42 (#12/12)
- BIG-Bench Extra Hard: 3.4 (#19/20)
Mistral 7B Instruct (v0.3) — ELO 1413, #557
- Do LLMs Build World Models From Text? A Multil: 1.0 (#12/14)
- RepoQA: 62.0 (#16/33)
Gemma 2 2B (IT) — ELO 1408, #563
- XDomainBench: 27.3 (#6/14)
OLMo 3 7B Instruct — ELO 1407, #565
- IndoBias: 51.0 (#25/26)
- CritPt: 0.0 (#289/342)
Phi-4 Mini Instruct — ELO 1405, #568
- Relevance as a Vulnerability: 1.67 (#8/8)
- HarmActionsEval: 0.0 (#9/10)
- StructEval: 56.97 (#10/12)
- Korean Canonical Legal Benchmark: 24.75 (#31/31)
- K-MetBench: 30.4 (#55/59)
- Epoch AI - Scicode: 10.76 (#128/128)
Llama 3.2 3B Instruct — ELO 1404, #570
- MathTutorBench: 46.89 (#5/8)
- PrivacyPeek: 54.48 (#10/10)
- TAEF (Text Analytics Evaluation Framework): 38.5 (#10/11)
- MedCTA: 11.29 (#12/18)
- MedGUIDE: 41.7 (#19/25)
- When Simulation Lies: 34.1 (#20/22)
- HealthBench Hard: 26.0 (#48/55)
- K-MetBench: 33.8 (#50/59)
- WritingBench: 38.14 (#65/66)
- Creative Writing v3: 709.4 (#109/123)
Qwen 3.5 0.8B — ELO 1403, #571
- Agentick: 9.4 (#8/15)
- StemBind: 30.2 (#16/24)
- SceneFunRI: 3.63 (#16/17)
- SVFSearch: 34.1 (#72/75)
Phi-3 Mini 128K Instruct — ELO 1403, #572
- RepoQA: 22.4 (#29/33)
Mistral 7B — ELO 1402, #574
- The Wrong Kind of Right: 26.8 (#5/26)
- Diagnosing LLM Arbitration Behavior over Pre-e: -19.0 (#5/7)
- MHGraphBench: 42.1 (#8/15)
- ChaosBench-Logic v2: 22.8 (#10/10)
- StakeBench: 4.5 (#14/15)
- LegalCiteBench: 58.5 (#15/21)
- J1-ENVS: 26.71 (#15/17)
- PhysicsFinals: 11.7 (#24/34)
- EHRBench: 38.23 (#28/31)
Qwen 1.5 7B Chat — ELO 1401, #576
- RepoQA: 2.8 (#32/33)
Mistral 7B Instruct — ELO 1400, #577
- PlantMarkerBench: 69.0 (#8/47)
- SuiChat-CN: 24.2 (#43/44)
gemma-3n-E2B-it Instruct — ELO 1399, #579
- ECLeKTic: 2.5 (#12/14)
- CritPt: 0.0 (#176/342)
- AA-LCR: 0.0 (#307/355)
- AA MMLU-Pro: 37.82 (#324/346)
LFM2 2.6B — ELO 1398, #580
- AA MMLU-Pro: 29.78 (#334/346)
gemma-7B — ELO 1398, #581
- NRITYAM: 35.32 (#7/7)
- BBH: 55.1 (#13/16)
Mistral 7B Instruct (v0.2) — ELO 1396, #583
- RepoQA: 47.4 (#20/33)
Qwen 2.5 1.5B — ELO 1392, #586
- On Stable Long-Form Generation: 19.6 (#7/13)
Llama 3.2 3B — ELO 1383, #598
- SAGE: 59.54 (#10/12)
- StakeBench: 9.2 (#11/15)
- XDomainBench: 26.8 (#11/14)
- Large Language Models Lack Temporal Awareness: 53.63 (#11/14)
- The Wrong Kind of Right: 18.2 (#14/26)
- PersonaArena: 3.29 (#16/18)
- LegalCiteBench: 43.1 (#20/21)
- EHRBench: 49.85 (#23/31)
- PlantMarkerBench: 44.9 (#41/47)
- IneqMath: 0.0 (#41/42)
Qwen 3 0.6B — ELO 1380, #601
- When Multiple Scripts Matter: 59.09 (#1/6)
- CULTURE-MT: 95.41 (#1/15)
- ProtStructQA: 21.15 (#4/4)
- ChLogic: 78.3 (#6/7)
- AgentFloor: 29.2 (#15/17)
Llama 2 13B — ELO 1380, #602
- VerdictBench: 50.42 (#12/17)
Mistral 7B Instruct (v0.1) — ELO 1365, #615
- Open FinLLM Leaderboard: 9.14 (#20/25)
- RepoQA: 11.0 (#30/33)
Granite 4.0 H 350M — ELO 1364, #619
- AA MMLU-Pro: 12.7 (#344/346)
Granite 4.0 350M — ELO 1361, #623
- AA MMLU-Pro: 12.36 (#345/346)
gemma-7B (IT) — ELO 1360, #624
- Fin-Bias: 97.6 (#4/19)
Gemma 3 1B (IT) — ELO 1357, #625
- CritPt: 0.0 (#172/342)
- AA-LCR: 0.0 (#305/355)
- AA MMLU-Pro: 13.47 (#343/346)
Phi-2 — ELO 1357, #626
- HELM Lite: 19.1392 (#70/77)
Llama 2 7B Chat — ELO 1356, #627
- Finetuning with Scientific Data Increases Hall: 61.4 (#13/18)
- IndoBias: 61.2 (#21/26)
Gemma 3 1B — ELO 1356, #628
- Natural2Code: 56.0 (#7/30)
- BIG-Bench Extra Hard: 7.2 (#11/20)
- ECLeKTic: 1.4 (#14/14)
Qwen 2.5 0.5B — ELO 1341, #641
- PlantMarkerBench: 66.7 (#23/47)
Llama 3.2 1B Instruct — ELO 1338, #643
- Open FinLLM Leaderboard: 8.95 (#21/25)
- MedGUIDE: 28.9 (#22/25)
- HealthBench Hard: 25.0 (#49/55)
- K-MetBench: 3.5 (#59/59)
- WritingBench: 28.74 (#66/66)
- EQ-Bench: 200.0 (#79/79)
- Creative Writing v3: 200.0 (#111/123)
Llama 2 7B — ELO 1338, #645
- BBH: 58.5 (#11/16)
- IndoBias: 63.3 (#18/26)
DeepSeek R1 Distill Qwen 1.5B — ELO 1333, #648
- OJBench: 2.59 (#41/44)
Llama 3.2 1B — ELO 1317, #658
- SAGE: 53.67 (#12/12)
- LegalCiteBench: 39.1 (#21/21)
- PlantMarkerBench: 66.7 (#22/47)
gpt-neox-20B — ELO 1308, #664
- HELM: 35.1 (#45/65)
gpt-j-6B — ELO 1274, #680
- HELM: 27.28 (#51/65)
Claude Mythos Preview — ELO 1772
- USAMO25: 97.6 (#1/13)
CodeLlama-34B-Instruct — ELO 1405
- CyberSecEval: 36.33 (#4/7)
DeepSeek Coder V2 — ELO 1548
- OJBench: 8.24 (#33/44)
DeepSeek Reasoner — ELO 1652
- Xent Games: 62.67 (#4/12)
- TRLawBench: 73.8 (#12/32)
Doubao-Seed-1.6 — ELO 1635
- StressEval: 28.1 (#8/10)
- VerdictBench: 56.26 (#9/17)
ERNIE 5.0 — ELO 1612
- AIIQ Composite IQ: 92.0 (#111/138)
GLM-4 32B — ELO 1566
- EHRBench: 66.12 (#7/31)
- Benchmarking Large Language Models for Graphem: 35.62 (#9/37)
- Active Evidence-Seeking and Diagnostic Reasoni: 35.7 (#9/15)
- AsyncTool: 15.17 (#13/19)
- Trip+: 54.43 (#17/18)
GLM-5 FP8 — ELO 1693
- TuRTLe Code Completion (Icarus Verilog): 83.98 (#1/44)
- TuRTLe Code Completion (Verilator): 82.23 (#1/44)
- TuRTLe Spec-to-RTL (Icarus Verilog): 79.46 (#3/44)
- TuRTLe Spec-to-RTL (Verilator): 78.17 (#3/44)
GPT-4 Turbo (Preview) — ELO 1540
- ENAMEL: 47.0 (#2/31)
- MBPP+: 73.3 (#8/25)
- EvalPlus: 77.5 (#10/25)
GPT-5.1 Codex Max — ELO 1686
- IDE-Bench: 73.75 (#5/15)
- SmellBench: 34.7 (#7/9)
GPT-5.1 Instant — ELO 1600
- TutorBench: 49.08 (#18/23)
- EnigmaEval: 1.94 (#30/39)
- MultiNRC: 18.29 (#32/39)
- Visual-Language Understanding: 34.87 (#41/54)
GPT-5.2 ChatGPT — ELO 1696
- VerdictBench: 63.44 (#5/17)
GPT-5.5 Instant — ELO 1669
- CharXiv-R: 81.6 (#23/60)
- OmniDocBench 1.5: 87.5 (#35/64)
- Epoch AI - Critpt: 0.0 (#136/138)
GPT-5.6 Pro Sol — ELO 1771
- LM Market Cap LMC Score: 89.0 (#41/406)
Gemini 1.5 Flash 8B (Preview 0827) — ELO 1487
- Generate README Eval: 32.12 (#12/14)
Gemini 2.0 Flash Lite (Preview 02-05) — ELO 1520
- HELM Safety: 90.7886 (#29/57)
Gemini 2.5 Flash (Preview) — ELO 1629
- Reward Hacking Benchmark (RHB): 0.8 (#9/13)
- EQ-Bench: 1164.5 (#47/79)
- Creative Writing v3: 1221.8 (#76/123)
Gemini 2.5 Pro (Preview) — ELO 1710
- Reward Hacking Benchmark (RHB): 4.6 (#6/13)
Gemini 3.1 Flash — ELO 1651
- HarmVideoBench: 81.3 (#3/21)
- Beyond Function Calling: 41.6 (#6/12)
- CheXpercept: 87.1 (#8/14)
- Chess Puzzles (Epoch AI): 24.0 (#58/153)
- OTIS Mock AIME 2024-25: 37.78 (#167/230)
Gemma 4 12B (IT) — ELO 1605
- EQ-Bench: 1361.2 (#37/79)
- Creative Writing v3: 1270.4 (#67/123)
Gemma 4 26B — ELO 1605
- TeleResilienceBench: 27.2 (#3/8)
- Polaris-Bench: 47.2 (#7/15)
- AgentCIBench: 67.5 (#8/15)
- DiagnosticIQ: 63.2 (#12/33)
- AIIQ Composite IQ: 95.0 (#101/138)
Grok 2 — ELO 1572
- K-12EduBench: 64.1 (#14/23)
Grok 4.1 — ELO 1644
- ResearchClawBench: 13.5 (#18/20)
- Creative Writing v3: 85.43 (#117/123)
Kimi Linear 48B A3B Instruct — ELO 1503
- ATLAS: 33.1 (#18/23)
Laguna M.1 — ELO 1572
- Vals Index: 35.27 (#33/35)
- ProofBench: 0.0 (#49/49)
- MedCode: 25.24 (#60/63)
- AIIQ Composite IQ: 83.0 (#128/138)
Laguna XS.2 — ELO 1546
- AIIQ Composite IQ: 83.0 (#129/138)
LongCat-Flash-Thinking-2601 — ELO 1640
- IMO-AnswerBench: 78.6 (#19/22)
MiniMax M1 40k — ELO 1577
- AA MMLU-Pro: 80.83 (#99/346)
Ministral 8B — ELO 1458
- ChLogic: 92.03 (#5/7)
- CommunityFact: 36.74 (#15/15)
- EHRBench: 56.48 (#22/31)
Mistral Medium — ELO 1479
- ContextEcho: 47.0 (#10/23)
- HELM Lite: 30.3197 (#62/77)
Mistral Small Creative — ELO 1568
- Creative Writing v3: 1371.1 (#60/123)
Muse Spark — ELO 1731
- MedXpertQA: 78.4 (#2/26)
- CharXiv-R: 86.4 (#7/60)
- Arena AI Document: 1444.0 (#14/27)
- DeepSearchQA: 74.8 (#14/15)
- OSWorld-Verified: 53.3 (#29/30)
- Epoch AI - Scicode: 51.5 (#29/128)
- AIIQ Composite IQ: 117.0 (#32/138)
- Toolathlon: 49.4 (#33/39)
NVIDIA Nemotron 3 Super — ELO 1595
- MathArena Apex: 7.81 (#29/51)
NVIDIA-Nemotron-3-Super-120B-A12B-BF16 — ELO 1590
- HMMT February 2026: 84.85 (#16/25)
O1 Preview (2024-09-12) — ELO 1586
- TextClass Benchmark: 1622.24 (#31/108)
O1 Pro — ELO 1629
- EnigmaEval: 6.14 (#15/39)
- Visual-Language Understanding: 47.32 (#16/54)
- ZEROBench-Sub: 22.4 (#22/45)
- LM Market Cap LMC Score: 73.6 (#143/406)
O3 Pro — ELO 1686
- IneqMath: 46.0 (#2/42)
- NarrativeWorldBench: 79.0 (#5/10)
- MCP Atlas: 44.5 (#41/42)
- LM Market Cap LMC Score: 86.7 (#64/406)
Phi-3-small-128k-instruct — ELO 1446
- RepoQA: 39.6 (#24/33)
Qwen 2.5 Coder 14B — ELO 1445
- OJBench: 6.3 (#34/44)
Qwen 2.5 Coder 32B — ELO 1530
- FormalRewardBench: 37.6 (#6/26)
- OJBench: 5.77 (#35/44)
- IneqMath: 1.5 (#36/42)
Qwen 2.5 Coder 7B — ELO 1444
- OJBench: 3.5 (#39/44)
Qwen 2.5 Max — ELO 1561
- RefusalBench: 29.8 (#4/19)
- BigCodeBench-Hard: 29.1 (#14/20)
- Balrog: 16.2 (#15/16)
Qwen 3 4B Instruct — ELO 1548
- SciPaths: 6.0 (#7/10)
- PrivacyPeek: 71.32 (#9/10)
Qwen 3 Coder Plus — ELO 1597
- AssertLLM2: 18.55 (#5/6)
- FrontierOR: 20.0 (#6/7)
- SpacetimeDB LLM Benchmark (TypeScript): 68.5 (#12/13)
- SpacetimeDB LLM Benchmark (Rust): 82.0 (#12/13)
- SpacetimeDB LLM Benchmark (C#): 76.4 (#12/13)
- ALE-Bench: 456.5 (#61/79)
Qwen 3 Max (2026-01-23) — ELO 1667
- IOI: 13.75 (#32/58)
- ClawProBench: 55.76 (#33/57)
- CaseLaw v2: 54.98 (#35/54)
- MedScribe: 72.71 (#47/61)
- MedCode: 31.37 (#56/63)
Qwen 3.6 Max — ELO 1715
- AgentCIBench: 97.4 (#2/15)
- GroupTravelBench: 9.66 (#2/9)
- PerfCodeBench: 66.32 (#2/20)
- FORTIS: 51.3 (#2/10)
- PDP-Bench: 71.31 (#3/7)
- LiveSQLBench: 33.79 (#6/33)
- AIIQ Composite IQ: 111.0 (#56/138)
Qwen Max — ELO 1551
- AsyncTool: 25.56 (#6/19)
- MixRea: 15.4 (#8/21)
- QUIET: 7.62 (#10/12)
- FullStackBench en: 55.16 (#15/31)
- SuiChat-CN: 68.6 (#20/44)
- ACEBench: 0.82 (#26/31)
- WritingBench: 66.78 (#49/66)
Qwen Plus — ELO 1613
- ATC Safety-Oriented Language Understanding Eval: 43.52 (#7/11)
Seed 1.8 — ELO 1647
- Are LLMs Ready to Assist Physicians? PhysAssis: 67.4 (#5/14)
- PlanningBench: 22.4 (#12/16)
Seed 2.0 Lite — ELO 1639
- Beyond Function Calling: 51.3 (#1/12)
- Boiling the Frog: 87.2 (#2/9)
- PerfCodeBench: 60.19 (#11/20)
- ClawProBench: 60.4 (#12/57)
codestral-2508 — ELO 1548
- ALE-Bench: 137.78 (#79/79)
deepseek-coder-6.7B-instruct — ELO 1340
- EvalPlus: 68.45 (#25/25)
- RepoQA: 10.6 (#31/33)
deepseek-llm-7B-chat — ELO 1412
- MedCTA: 11.0 (#13/18)
glm-4-9B — ELO 1493
- EHRBench: 59.62 (#17/31)
phi-4-14B — ELO 1503
- When Context Flips, Safety Breaks: 35.9 (#6/12)
- TSCG: 0.0 (#13/13)
- CardioLens: 36.4 (#24/24)
qwen3.7-flash — ELO 1647
- Gert Labs Rankings: 44.71 (#51/93)
- Chess Puzzles (Epoch AI): 23.0 (#61/153)
- OTIS Mock AIME 2024-25: 86.67 (#65/230)
- PM-LLM-Benchmark: 31.5 (#77/170)
- SpeechMap Compliance: 28.3 (#341/359)
uitars-1.5-7B — ELO 1469
- Beyond Binary: 56.78 (#12/14)

Top-10 New Scores (76)

Claude Fable 5 on Arena AI Code: 1630.0 (#2)
Claude Fable 5 on Arena AI Document: 1505.0 (#2)
Claude Fable 5 on CAIS Vision Capabilities Index: 62.1 (#4)
Claude Fable 5 on Creative Writing v3: 2156.3 (#2)
Claude Fable 5 on Crosby micro1 RedlineBench: 47.3 (#2)
Claude Fable 5 on EQ-Bench: 2049.7 (#1)
Claude Fable 5 on LMArena Text Arena: 1504.0 (#2)
Claude Fable 5 on LMArena WebDev Arena: 1629.56 (#2)
Claude Fable 5 on OSWorld: 85.96 (#2)
Claude Fable 5 on OSWorld-Verified: 85.0 (#2)
Claude Fable 5 on PLawBench: 70.2 (#3)
Claude Fable 5 on SpacetimeDB LLM Benchmark (C#): 98.9 (#1)
Claude Fable 5 on SpacetimeDB LLM Benchmark (Rust): 96.6 (#6)
Claude Fable 5 on SpacetimeDB LLM Benchmark (TypeScript): 98.9 (#5)
Claude Fable 5 on Tau3-Bench Banking_Knowledge: 39.7 (#9)
Claude Opus 5 on Chess Puzzles (Epoch AI): 33.0 (#34)
Claude Opus 5 on Epoch AI - Mystery Game Puzzles: 37.0 (#7)
Claude Opus 5 on INCLUDE: 89.8 (#4)
Claude Opus 5 on Lean AI Formalization Leaderboard: 4.0 (#16)
Claude Opus 5 on MCP Atlas: 85.8 (#2)
Claude Opus 5 on MILU: 92.1 (#4)
Claude Opus 5 on OTIS Mock AIME 2024-25: 97.78 (#12)
Claude Opus 5 on Roboflow Playground OCR: 1212.0 (#17)
Claude Opus 5 on SpacetimeDB LLM Benchmark (C#): 94.4 (#8)
Claude Opus 5 on SpacetimeDB LLM Benchmark (Rust): 96.6 (#4)
Claude Opus 5 on SpacetimeDB LLM Benchmark (TypeScript): 98.9 (#4)
Claude Opus 5 on Tau3-Bench Banking_Knowledge: 48.7 (#2)
GPT-5.4 Pro on ARC-AGI-1: 94.5 (#18)
GPT-5.4 Pro on ARC-AGI-2: 83.33 (#15)
GPT-5.4 Pro on BilliardPhys-Bench: 72.29 (#2)
GPT-5.4 Pro on CritPt: 30.0 (#3)
GPT-5.4 Pro on EnigmaEval: 23.82 (#1)
GPT-5.4 Pro on LM Market Cap LMC Score: 91.9 (#18)
GPT-5.4 Pro on MathArena Apex: 69.79 (#2)
GPT-5.4 Pro on MultiNRC: 62.27 (#3)
GPT-5.4 Pro on TaxBench: 27.03 (#2)
GPT-5.4 Pro on TutorBench: 56.62 (#2)
GPT-5.4 Pro on Visual-Language Understanding: 53.89 (#3)
GPT-5.5 Pro on BLXBench: 44.5 (#14)
GPT-5.5 Pro on GDPval-MM: 82.3 (#2)
GPT-5.5 Pro on LM Market Cap LMC Score: 92.7 (#11)
GPT-5.5 Pro on TaxBench: 29.27 (#1)
GPT-5.6 Pro Sol on LM Market Cap LMC Score: 89.0 (#41)
GPT-5.6 Sol on AIIQ Composite IQ: 131.0 (#6)
GPT-5.6 Sol on CAIS Risk Index: 50.8 (#16)
GPT-5.6 Sol on CAIS Vision Capabilities Index: 62.0 (#5)
GPT-5.6 Sol on Chess Puzzles (Epoch AI): 7.0 (#123)
GPT-5.6 Sol on Creative Writing v3: 2208.0 (#1)
GPT-5.6 Sol on Epoch AI - Critpt: 5.14 (#55)
GPT-5.6 Sol on Korean LLM Eval - CLIcK: 95.5 (#2)
GPT-5.6 Sol on Korean LLM Eval - HAE-RAE Bench: 94.64 (#2)
GPT-5.6 Sol on Korean LLM Eval - KMMLU-HARD: 85.04 (#2)
GPT-5.6 Sol on LM Market Cap LMC Score: 89.0 (#43)
GPT-5.6 Sol on Mercor APEX: 39.9 (#7)
GPT-5.6 Sol on MirrorCode: 20.0 (#2)
GPT-5.6 Sol on OTIS Mock AIME 2024-25: 68.89 (#116)
GPT-5.6 Sol on PLawBench: 72.3 (#2)
GPT-5.6 Sol on Rogo Big Finance Bench: 53.0 (#5)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (C#): 97.8 (#3)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (Rust): 97.8 (#3)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#1)
GPT-5.6 Sol on Tau3-Bench Banking_Knowledge: 46.9 (#4)
GPT-5.6 Sol on Toolathlon: 74.9 (#8)
Kimi K3 on AA-LCR: 74.7 (#4)
Kimi K3 on CritPt: 23.4 (#10)
Kimi K3 on Epoch AI - Scicode: 58.68 (#3)
Kimi K3 on Epoch AI - Vending Bench 2: 5165.04 (#19)
Kimi K3 on GameCraft-Bench: 56.96 (#3)
Kimi K3 on InferenceBench: 5.7 (#12)
Kimi K3 on SpacetimeDB LLM Benchmark (C#): 93.3 (#9)
Kimi K3 on SpacetimeDB LLM Benchmark (Rust): 92.1 (#10)
Kimi K3 on SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#10)
Kimi K3 on Sycophancy (Lechmazur): 990990.0 (#20)
Kimi K3 on Tau3-Bench Banking_Knowledge: 37.1 (#11)
Kimi K3 on TrackingAI IQ Test: 93.75 (#4)
Kimi K3 on TrackingAI IQ Test (Vision): 88.0 (#4)

New #1 Leaders (79)

LMArena WebDev Arena: Kimi K3 (1682.01) beat Claude Opus 4.7 by 114.16
Arena AI Code: Kimi K3 (1682.0) beat Claude Opus 4.7 by 112.0
USAMO25: Claude Mythos Preview (97.6) beat DeepSeek R1 0528 by 67.54
Natural2Code: Gemini 2.0 Flash (92.9) beat GPT-4 by 40.1
Design Arena (Game Dev): Claude Opus 5 (1426.0) beat Claude Fable 5 by 32.0
AutoLogi: Kimi K2 (89.5) beat GPT-4o (2024-08-06) by 27.46
TIR-Bench: Qwen 3.6 Plus (61.6) beat o4-mini (Tool Use) by 24.1
OJBench: Kimi K2.6 (60.6) beat Gemini 2.5 Pro (Preview 03-25) by 21.69
Toolathlon: Claude Opus 5 (80.6) beat Claude Fable 5 by 18.9
MMOU: Gemini 3.1 Pro (Preview) (82.67) beat Gemini 2.5 Pro by 18.47
Tau3 Banking: Qwen 3.8 Max (51.34) beat Kimi K3 (Low) by 17.32
GDPval-MM: GPT-5.5 (84.9) beat GPT-5.2 by 14.0
Design Arena (SVG): GPT-5.6 Sol (1362.0) beat Claude Fable 5 by 14.0
Korean LLM Eval - KMMLU-HARD: Claude Opus 5 (reasoning=medium) (88.4) beat GPT-5.2 (Medium) by 13.77
ArxivMath: Claude Opus 5 (91.33) beat Claude Fable 5 by 12.73
LMArena Text Arena: Claude Opus 4.6 (Thinking) (1512.42) beat Claude Opus 4.6 by 12.18
AudioMC: Gemini 3.1 Pro (Preview) (High) (66.8) beat Gemini 3 by 12.15
LLM Stats (SkillsBench): Qwen 3.8 Max (70.2) beat Qwen 3.7 Max by 11.0
Tau3-Bench Banking_Knowledge: Qwen 3.8 Max (55.2) beat GPT-5.5 by 8.8
MirrorCode: Claude Fable 5 (63.9) beat Claude Opus 4.7 by 7.9
AA Long Context Reasoning: Muse Spark 1.2 (xHigh) (83.33) beat GPT-5.2 Codex (xHigh) by 7.66
FrontierCode: Claude Opus 5 (53.4) beat Claude Fable 5 (xHigh) by 7.1
Design Arena (Data Viz): Claude Opus 5 (1389.0) beat Kimi K3 by 7.0
ZeroBench: GPT-5.6 Sol (Max) (30.0) beat Claude Fable 5 (Max) by 7.0
LLM Stats (NL2Repo): Qwen 3.8 Max (55.9) beat GLM-5.2 by 7.0
Harvey Legal Agent Benchmark: Claude Opus 5 (23.58) beat Claude Mythos 5 by 6.67
SEAL - SWE Atlas - Test Writing: Claude Opus 5 (Claude Code) xHigh (62.22) beat Claude Fable 5 (Claude Code) xHigh by 6.62
MMAU: Gemini 3.1 Pro (Preview) (High) (82.5) beat Audio-Thinker by 6.52
MMEB Leaderboard: Tianmu-Emb-Uni (52.83) beat e5-omni-7B by 6.3
SEAL - SWE Atlas - Codebase QnA: Claude Opus 5 (Claude Code) xHigh (63.17) beat Claude Opus 4.8 (Claude Code) xhigh by 5.91
LLM Stats (ERQA): Qwen 3.8 Max (77.8) beat Seed 2.1 Pro by 5.8
SEAL Showdown: Gemini 3 Pro (Preview) (1266.5) beat GPT-4o Audio (Preview 2025-06-03) by 5.7
CAIS Risk Index: Claude Fable 5 (27.3) beat Claude Opus 4.7 by 5.6
Vals AI Harvey Legal Agent Bench: Muse Spark 1.2 (xHigh) (25.42) beat Muse Spark 1.1 by 5.42
ZEROBench-Sub: Qwen 3.5 122B A10B (36.2) beat Seed1.5-VL (Thinking) by 5.4
Korean LLM Eval - CLIcK: Claude Opus 5 (reasoning=medium) (95.84) beat GPT-5.2 (Medium) by 4.92
MCP Atlas: Muse Spark 1.1 (xHigh) (88.1) beat Gemini 3.5 Flash by 4.5
ComplexFuncBench: GPT-4.1 (65.5) beat Claude 3.5 Sonnet by 4.5
MMVU: Kimi K2.5 (80.4) beat O1 by 4.3
Graphwalks BFS 256k F1: GPT-5.6 Sol (95.4) beat Claude Mythos 5 by 4.3
VideoMME w/o sub.: Qwen 3.5 122B A10B (83.9) beat Video-SALMONN 2 by 4.2
Korean LLM Eval - HAE-RAE Bench: Claude Opus 5 (reasoning=medium) (95.84) beat GPT-5.2 (Medium) by 4.03
Graphwalks BFS 1M F1: GPT-5.6 Sol (83.4) beat Claude Mythos 5 by 4.0
AA Omniscience - Software Engineering (SWE) - Julia: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (92.0) beat GPT-5.4 (Low) by 4.0
MyPCBench: Claude Opus 4.8 (62.0) beat Claude Opus 4.6 by 3.8
LLM Stats (LVBench): Qwen 3.8 Max (81.8) beat Seed 2.1 Pro by 3.8
PLawBench: Qwen 3.8 Max (73.2) beat GPT-5.2 by 3.53
WritingBench: Qwen 3 Next 80B A3B Instruct (87.3) beat GPT-5 by 3.43
When Lower Privileges Suffice: Qwen 3 8B (64.9) beat Qwen 3 8B by 3.1
LLM Stats (LongBench v2): Qwen 3.8 Max (66.3) beat Qwen 3.5 397B A17B by 3.1
AA Omniscience - Health: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (53.72) beat GPT-5.6 Sol (High) by 3.02
AA Omniscience - Software Engineering (SWE) - C: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (95.0) beat GPT-5.5 (High) by 3.0
TrackingAI IQ Test (Mensa Norway): GPT-5.6 Sol (Ultra) (100.0) beat GPT-5 Pro by 2.86
ProteinGym Hard: Claude Opus 5 (47.7) beat Claude Mythos 5 by 2.7
LLM Stats (VideoMME w sub.): Qwen 3.8 Max (90.4) beat Qwen 3.6 27B by 2.7
Spider 2.0-Lite: GLM-5.2 (76.23) beat ktx by 2.56
AA Omniscience - Software Engineering (SWE) - TypeScript: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (94.44) beat GPT-5.6 Sol (Max) by 2.22
LLM Stats (HealthBench): Qwen 3.8 Max (60.2) beat Kimi K2 (0905) (Thinking) by 2.2
Vals AI Finance Agent v2: Muse Spark 1.2 (xHigh) (60.6) beat Claude Opus 5 by 1.97
TrackingAI IQ Test: GPT-5.6 Sol (Ultra) (98.04) beat GPT-5 Pro by 1.96
AA Omniscience - Law: Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (66.2) beat Gemini 3 Pro (Preview) (High) by 1.9
PhiBench: Phi-4-reasoning-plus (Thinking) (74.2) beat GPT-4o by 1.8
SWE-rebench: Claude Fable 5 (High) (64.5) beat GPT-5.5 (xHigh) by 1.77
CritPt: GPT-5.6 Sol (Max) (32.3) beat GPT-5.5 Pro by 1.7
IMO-AnswerBench: Nemotron 3 Ultra (92.3) beat DeepSeek V4 Flash by 1.2
LLM Stats (OSWorld-Verified): Qwen 3.8 Max (86.1) beat Claude Fable 5 by 1.1
LLM Stats (RealWorldQA): Qwen 3.8 Max (88.0) beat Qwen 3.7 Plus by 1.1
Vals AI Time Horizon Index: Claude Opus 5 (Max) (13.67) beat GPT-5.6 Sol by 0.67
Vals AI TaxEval v2: Muse Spark 1.2 (xHigh) (80.38) beat Muse Spark 1.1 by 0.66
DeepSearchQA: Kimi K3 (Max) (95.0) beat Claude Mythos Preview by 0.6
LVBench: Seed 2.1 Pro (78.0) beat GPT-5.4 by 0.6
ARC-AGI-1: Claude Fable 5 (Max) (98.5) beat Human Panel by 0.5
MedScribe: Muse Spark 1.1 (xHigh) (88.89) beat Claude Fable 5 (Max) by 0.37
Deep20Bench: Claude Fable 5 (High) (12.06) beat Claude Opus 5 (High) by 0.28
ForecastBench: captain-jack (68.9) beat Cassi-2026-05-10 by 0.2
Agent Arena - Tool Hallucination: Gemini 3 Flash (0.01) beat Nemotron 3 Ultra by 0.18
InferenceBench: Claude Opus 5 (8.9) beat Claude Fable 5 (Low) by 0.16
MRCR-v2 128k: Gemini 3.6 Flash (91.8) beat Qwen 3.7 Plus by 0.1
ProphetArena: Gemini 3.6 Flash (0.9638) beat Gemini 3.1 Pro (Preview) by 0.01

The Aggregate Digest — 2026-08-08

2026-08-08T04:40:31.488512+00:00

Daily

New Benchmarks (7)

Korean LLM Eval - CLIcK (Accuracy (%, 0-shot)): leader Claude Opus 5 (reasoning=medium) (95.84), 8 models
daekeun-ml Korean proficiency suite on CLIcK (Cultural and Linguistic Intelligence in Korean): overall accuracy %, 0-shot, culture and language categories.
Korean LLM Eval - HAE-RAE Bench (Accuracy (%, 0-shot)): leader Claude Opus 5 (reasoning=medium) (95.84), 8 models
daekeun-ml Korean proficiency suite on HAE_RAE_BENCH 1.0: overall accuracy % across general knowledge, history, loan/rare words, reading comprehension and nomenclature, 0-shot.
Korean LLM Eval - KoBALT-700 (Accuracy (%, 0-shot)): leader Claude Opus 5 (reasoning=medium) (88.14), 8 models
Korean LLM Eval - KMMLU-HARD (Accuracy (%, 0-shot)): leader Claude Opus 5 (reasoning=medium) (88.4), 8 models
daekeun-ml Korean proficiency suite on the KMMLU-HARD subset: overall accuracy % on the hardest KMMLU questions, 0-shot.
Korean LLM Eval - KMMLU-Pro (Accuracy (%, 0-shot)): leader Claude Opus 5 (reasoning=medium) (95.64), 8 models
Korean LLM Eval - MuSR (Ko) (Accuracy (%, 0-shot)): leader Claude Opus 5 (reasoning=medium) (86.53), 8 models
MIST (Selective Trust) (Overall Accuracy (%)): leader Gemini 3.1 Pro (Preview) (94.2), 23 models

Top-10 New Scores (3)

Claude Fable 5 on OSWorld: 85.96 (#2)
GPT-5.4 Pro on ARC-AGI-1: 94.5 (#18)
GPT-5.4 Pro on ARC-AGI-2: 83.33 (#15)

New #1 Leaders (2)

Design Arena (Game Dev): Claude Opus 5 (1444.0) beat Claude Fable 5 by 48.0
ZeroBench: GPT-5.6 Sol (Max) (30.0) beat Claude Fable 5 (Max) by 7.0

The Aggregate Digest — 2026-08-07

2026-08-07T05:41:54.632016+00:00

Daily

New Benchmarks (4)

SciCode-Verified (Subproblem Resolve Rate (%)): leader Gemini 3.5 Flash (97.2), 4 models
SciCode regraded after its benchmark defects were removed: the same 64 research-level scientific-coding problems (287 subproblems), rerun on a cleaned dataset with a corrected grading layer. The authors' point is the size of the correction — the frontier moves from roughly 50–58% subproblem accuracy on the original grading to 84–97% here, so the original board understated scientific coding rather than measuring its limit.
BoundaryBench (Unrestricted) (Success Rate (%)): leader GPT-5.6 Sol (83.9), 13 models
BoundaryBench runs coding agents over Terminal-Bench 2.1's 89 tasks with no sandbox restrictions, three trials per configuration. This is the control arm: each model is scored under its native harness (Claude Code, Codex, Grok Build or Terminus 2) with full filesystem and network access.
BoundaryBench (NIST High) (Success Rate under NIST high policy (%)): leader Grok 4.5 (74.9), 13 models
BoundaryBench under the NIST-high least-privilege policy: the same Terminal-Bench 2.1 tasks and harnesses, but the agent runs inside a sandbox that enforces restrictive authorization. Measures how much capability survives real deployment constraints — every model loses ground, and the size of the drop is the benchmark\
CEO-Bench - Max Survival Days (Max survival days (self-reported)): leader Claude Opus 4.8 (500.0), 10 models
CEO-Bench longevity slice: how many simulated days an agent keeps the company solvent at its best run, rather than the cash it ends with. Reported by the benchmark authors.

New Models (1)

Nemotron 3 Ultra 550B A55B — ELO 1659, #85
- Vals Index: 43.99 (#27/35)
- Finance Agent v2: 37.67 (#31/39)
- MedCode: 38.62 (#36/63)
- ProofBench: 2.0 (#47/49)

Top-10 New Scores (60)

Claude Fable 5 on Arena AI Code: 1630.0 (#2)
Claude Fable 5 on Arena AI Document: 1505.0 (#2)
Claude Fable 5 on CAIS Vision Capabilities Index: 62.1 (#4)
Claude Fable 5 on Creative Writing v3: 2156.3 (#2)
Claude Fable 5 on Crosby micro1 RedlineBench: 47.3 (#2)
Claude Fable 5 on EQ-Bench: 2049.7 (#1)
Claude Fable 5 on LMArena Text Arena: 1504.0 (#2)
Claude Fable 5 on LMArena WebDev Arena: 1629.56 (#2)
Claude Fable 5 on OSWorld-Verified: 85.0 (#2)
Claude Fable 5 on PLawBench: 70.2 (#3)
Claude Opus 5 on Chess Puzzles (Epoch AI): 33.0 (#34)
Claude Opus 5 on Epoch AI - Mystery Game Puzzles: 37.0 (#7)
Claude Opus 5 on INCLUDE: 89.8 (#4)
Claude Opus 5 on MCP Atlas: 85.8 (#2)
Claude Opus 5 on MILU: 92.1 (#4)
Claude Opus 5 on OTIS Mock AIME 2024-25: 97.78 (#12)
Claude Opus 5 on Roboflow Playground OCR: 1212.0 (#17)
GPT-5.4 Pro on BilliardPhys-Bench: 72.29 (#2)
GPT-5.4 Pro on CritPt: 30.0 (#3)
GPT-5.4 Pro on EnigmaEval: 23.82 (#1)
GPT-5.4 Pro on LM Market Cap LMC Score: 91.9 (#18)
GPT-5.4 Pro on MathArena Apex: 69.79 (#2)
GPT-5.4 Pro on MultiNRC: 62.27 (#3)
GPT-5.4 Pro on TaxBench: 27.03 (#2)
GPT-5.4 Pro on TutorBench: 56.62 (#2)
GPT-5.4 Pro on Visual-Language Understanding: 53.89 (#3)
GPT-5.5 Pro on BLXBench: 44.5 (#14)
GPT-5.5 Pro on GDPval-MM: 82.3 (#2)
GPT-5.5 Pro on LM Market Cap LMC Score: 92.7 (#11)
GPT-5.5 Pro on TaxBench: 29.27 (#1)
GPT-5.6 Pro Sol on LM Market Cap LMC Score: 89.0 (#41)
GPT-5.6 Sol on AIIQ Composite IQ: 131.0 (#6)
GPT-5.6 Sol on CAIS Risk Index: 50.8 (#16)
GPT-5.6 Sol on CAIS Vision Capabilities Index: 62.0 (#5)
GPT-5.6 Sol on Chess Puzzles (Epoch AI): 7.0 (#115)
GPT-5.6 Sol on Creative Writing v3: 2208.0 (#1)
GPT-5.6 Sol on LM Market Cap LMC Score: 89.0 (#43)
GPT-5.6 Sol on OTIS Mock AIME 2024-25: 68.89 (#108)
GPT-5.6 Sol on PLawBench: 72.3 (#2)
GPT-5.6 Sol on Rogo Big Finance Bench: 53.0 (#5)
GPT-5.6 Sol on Toolathlon: 74.9 (#8)
Qwen 3.8 Max on MineBench: 1267.73 (#30)
Qwen 3.8 Max on NL2Repo: 55.9 (#2)
Qwen 3.8 Max on Toolathlon: 72.5 (#15)
muse-spark-1.2 on ErdosBench: 2.73 (#3)
muse-spark-1.2 on Vals AI Excel Modeling: 56.98 (#16)
muse-spark-1.2 on Vals AI Legal Research Bench: 43.75 (#5)
muse-spark-1.2 on Vals AI LegalBench: 85.26 (#16)
muse-spark-1.2 on Vals AI MMLU-Pro: 88.28 (#15)
muse-spark-1.2 on Vals AI MMMU: 86.13 (#18)
muse-spark-1.2 on Vals AI MedCode: 49.35 (#15)
muse-spark-1.2 on Vals AI MedScribe: 90.06 (#2)
muse-spark-1.2 on Vals AI MortgageTax: 65.42 (#41)
muse-spark-1.2 on Vals AI Multimodal Index: 69.8 (#6)
muse-spark-1.2 on Vals AI ProofBench: 57.0 (#9)
muse-spark-1.2 on Vals AI Public Benefits Bench: 68.47 (#3)
muse-spark-1.2 on Vals AI SAGE: 47.66 (#22)
muse-spark-1.2 on Vals AI SWE-bench Verified: 86.6 (#9)
muse-spark-1.2 on Vals AI SkillsBench: 53.04 (#11)
muse-spark-1.2 on Vals AI Vibe Code Bench: 79.1 (#7)

New #1 Leaders (36)

LMArena WebDev Arena: Kimi K3 (1682.01) beat Claude Opus 4.7 by 114.16
Arena AI Code: Kimi K3 (1682.0) beat Claude Opus 4.7 by 112.0
USAMO25: Claude Mythos Preview (97.6) beat DeepSeek R1 0528 by 67.54
Natural2Code: Gemini 2.0 Flash (92.9) beat GPT-4 by 40.1
AutoLogi: Kimi K2 (89.5) beat GPT-4o (2024-08-06) by 27.46
TIR-Bench: Qwen 3.6 Plus (61.6) beat o4-mini (Tool Use) by 24.1
OJBench: Kimi K2.6 (60.6) beat Gemini 2.5 Pro (Preview 03-25) by 21.69
Toolathlon: Claude Opus 5 (80.6) beat Claude Fable 5 by 18.9
GDPval-MM: GPT-5.5 (84.9) beat GPT-5.2 by 14.0
ArxivMath: Claude Opus 5 (91.33) beat Claude Fable 5 by 12.73
LMArena Text Arena: Claude Opus 4.6 (Thinking) (1512.42) beat Claude Opus 4.6 by 12.18
AudioMC: Gemini 3.1 Pro (Preview) (High) (66.8) beat Gemini 3 by 12.15
FrontierCode: Claude Opus 5 (53.4) beat Claude Fable 5 (xHigh) by 7.1
Harvey Legal Agent Benchmark: Claude Opus 5 (23.58) beat Claude Mythos 5 by 6.67
MMAU: Gemini 3.1 Pro (Preview) (High) (82.5) beat Audio-Thinker by 6.52
CAIS Risk Index: Claude Fable 5 (27.3) beat Claude Opus 4.7 by 5.6
Vals AI Harvey Legal Agent Bench: muse-spark-1.2 (25.42) beat Muse Spark 1.1 by 5.42
ZEROBench-Sub: Qwen 3.5 122B A10B (36.2) beat Seed1.5-VL (Thinking) by 5.4
MCP Atlas: Muse Spark 1.1 (xHigh) (88.1) beat Gemini 3.5 Flash by 4.5
ComplexFuncBench: GPT-4.1 (65.5) beat Claude 3.5 Sonnet by 4.5
MMVU: Kimi K2.5 (80.4) beat O1 by 4.3
Graphwalks BFS 256k F1: GPT-5.6 Sol (95.4) beat Claude Mythos 5 by 4.3
VideoMME w/o sub.: Qwen 3.5 122B A10B (83.9) beat Video-SALMONN 2 by 4.2
Graphwalks BFS 1M F1: GPT-5.6 Sol (83.4) beat Claude Mythos 5 by 4.0
PLawBench: Qwen 3.8 Max (73.2) beat GPT-5.2 by 3.53
WritingBench: Qwen 3 Next 80B A3B Instruct (87.3) beat GPT-5 by 3.43
ProteinGym Hard: Claude Opus 5 (47.7) beat Claude Mythos 5 by 2.7
Vals AI Finance Agent v2: muse-spark-1.2 (60.6) beat Claude Opus 5 by 1.97
PhiBench: Phi-4-reasoning-plus (Thinking) (74.2) beat GPT-4o by 1.8
CritPt: GPT-5.6 Sol (Max) (32.3) beat GPT-5.5 Pro by 1.7
Vals AI TaxEval v2: muse-spark-1.2 (80.38) beat Muse Spark 1.1 by 0.66
DeepSearchQA: Kimi K3 (Max) (95.0) beat Claude Mythos Preview by 0.6
LVBench: Seed 2.1 Pro (78.0) beat GPT-5.4 by 0.6
MedScribe: Muse Spark 1.1 (xHigh) (88.89) beat Claude Fable 5 (Max) by 0.37
MMEB Leaderboard: Tianmu-Emb-Uni (52.83) beat Ovis-Omni-Embedding-v0.1-3B by 0.28
MRCR-v2 128k: Gemini 3.6 Flash (91.8) beat Qwen 3.7 Plus by 0.1

The Aggregate Digest — 2026-08-06

2026-08-06T06:15:17.632202+00:00

Daily

New Benchmarks (11)

SWE-Touch (Counter-Edit Resolve Rate (%)): leader Claude Opus 4.8 (83.3), 9 models
SWE-Touch — tests whether a coding agent can still finish a repair after the user directly edits the code in the shared workspace mid-task. Counter-Edit resolve rate (%) on SWE-bench Verified: the same tasks as the agent's autonomous (Vanilla) run, but with a conflicting user commit landing during the run, which costs the 9 evaluated models a mean 7.7 points.
Computer Anthology Terminal Tasks (Claude Code) (pass@1 (%)): leader Claude Opus 5 (54.0), 4 models
Vetto's Computer Anthology Terminal Tasks v1.0 run under the Claude Code agent scaffold — 100 held-out terminal jobs graded by per-task sandboxed verifiers; pass@1 over 5 trials per task. Reported per model+scaffold pair because the scaffold moves the score substantially.
Computer Anthology Terminal Tasks (Codex CLI) (pass@1 (%)): leader GPT-5.6 Sol (58.0), 4 models
Vetto's Computer Anthology Terminal Tasks v1.0 run under the Codex CLI agent scaffold — 100 held-out terminal jobs graded by per-task sandboxed verifiers; pass@1 over 5 trials per task. Reported per model+scaffold pair because the scaffold moves the score substantially.
Computer Anthology Terminal Tasks (Terminus-2) (pass@1 (%)): leader Claude Opus 5 (61.8), 18 models
Vetto's Computer Anthology Terminal Tasks v1.0 — 100 held-out terminal jobs across 17 categories and 11 runtimes, weighted toward data science, ML, optimization and model training, each graded by its own sandboxed verifier (no LLM judge). This slice runs every model under the Terminus-2 scaffold; pass@1 over 5 trials per task.
BridgeBench Arena (Confidence-weighted Elo): leader Claude Fable 5 (1005.3), 28 models
Head-to-head coding arena: one unified confidence-weighted Elo ladder over seven task domains (reasoning, hallucination, security, refactoring, debugging, spec conformance, bullshit), decided by seeded blind matches judged 3-of-7 by a vendor-disjoint model panel. Domain filters slice the same match record rather than forming separate ladders.
OpenRouter GPQA Diamond (Accuracy (%)): leader Gemini 3.1 Pro (Preview) (94.3), 105 models
OpenRouter's own GPQA Diamond run: 198 graduate-level science questions executed by its native benchmark harness against the production endpoints it serves, so the score reflects the deployed model rather than a vendor-reported figure.
OpenRouter Tau2-Bench Airline (Accuracy (%)): leader Claude Fable 5 (82.0), 108 models
OpenRouter's own τ²-Bench Airline run: 50 multi-turn airline customer-service tasks where the model must use tools and follow policy, executed against the production endpoints it serves.
OpenRouter HLE (Search) (Best-configuration Accuracy (%)): leader Claude Opus 5 (72.7), 3 models
OpenRouter's Humanity's Last Exam run with web search: 2,158 expert-written questions answered through a search provider (Exa, Perplexity, Parallel, or the OpenAI/Anthropic native tools) under a 1, 5, or 25-turn budget. The board varies the search stack rather than the model, so each model is scored on its best configuration.
OpenRouter BrowseComp (Search) (Best-configuration Accuracy (%)): leader GPT-5.6 Sol (75.0), 3 models
OpenRouter's BrowseComp run: 1,266 questions built to be unfindable in a single search, rewarding persistent multi-step browsing. Scored across search providers and turn budgets; each model takes its best configuration.
OpenRouter DeepSearchQA (Search) (Best-configuration Accuracy (%)): leader GPT-5.6 Sol (77.0), 2 models
OpenRouter's DeepSearchQA run: research questions answered through a live search provider, graded by answer equivalence against a reference. Scored across search providers and turn budgets; each model takes its best configuration.
OpenRouter WideSearch (Search) (Best-configuration Exact Table Match (%)): leader GPT-5.6 Sol (15.0), 2 models
OpenRouter's WideSearch run: the model must assemble a complete reference table from the open web, scored strictly — credit only when every required row and cell matches. Scored across search providers and turn budgets; each model takes its best configuration.

New Models (1)

Muse Spark 1.2 — ELO 1727
- BenchLM: 60.2 (#49/200)
- LM Market Cap LMC Score: 80.6 (#55/346)

Top-10 New Scores (36)

Claude Fable 5 on Tau3-Bench Banking_Knowledge: 39.7 (#9)
Claude Opus 5 on Lean AI Formalization Leaderboard: 3.0 (#20)
Claude Opus 5 on Tau3-Bench Banking_Knowledge: 48.7 (#2)
GPT-5.6 Sol on Epoch AI - Mystery Game Puzzles: 49.0 (#6)
GPT-5.6 Sol on Tau3-Bench Banking_Knowledge: 46.9 (#4)
Kimi K3 on Tau3-Bench Banking_Knowledge: 37.1 (#11)
Qwen 3.8 Max on AA CritPt: 20.0 (#22)
Qwen 3.8 Max on AA GDPval: 1738.53 (#4)
Qwen 3.8 Max on AA GPQA Diamond: 92.73 (#14)
Qwen 3.8 Max on AA Humanity's Last Exam: 41.38 (#18)
Qwen 3.8 Max on AA Long Context Reasoning: 66.67 (#74)
Qwen 3.8 Max on AA MMMU-Pro: 82.31 (#10)
Qwen 3.8 Max on AA Omniscience: 4.08 (#51)
Qwen 3.8 Max on AA Omniscience - Business: 24.2 (#119)
Qwen 3.8 Max on AA Omniscience - Health: 30.4 (#90)
Qwen 3.8 Max on AA Omniscience - Humanities & Social Sciences: 30.3 (#100)
Qwen 3.8 Max on AA Omniscience - Law: 20.4 (#117)
Qwen 3.8 Max on AA Omniscience - Science, Engineering & Mathematics: 37.8 (#92)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE): 45.4 (#91)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - C: 62.0 (#98)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - Dart: 34.0 (#105)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - Go: 32.0 (#112)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - HTML: 50.0 (#98)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - Java: 30.0 (#101)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - JavaScript: 60.0 (#61)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - Julia: 16.0 (#198)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - Kotlin: 24.0 (#167)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - PHP: 38.0 (#134)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - Python: 43.5 (#86)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - R: 24.0 (#135)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - Rust: 70.0 (#75)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - Swift: 72.0 (#47)
Qwen 3.8 Max on AA Omniscience - Software Engineering (SWE) - TypeScript: 56.67 (#73)
Qwen 3.8 Max on AA SciCode: 52.89 (#29)
Qwen 3.8 Max on AA-Omniscience Accuracy: 31.42 (#96)
Qwen 3.8 Max on Artificial Analysis Intelligence Index: 56.22 (#9)

New #1 Leaders (5)

Tau3-Bench Banking_Knowledge: Qwen 3.8 Max (55.2) beat GPT-5.5 by 8.8
Tau3 Banking: Qwen 3.8 Max (42.47) beat Kimi K3 (Low) by 8.45
Vals AI Time Horizon Index: Claude Opus 5 (13.67) beat GPT-5.6 Sol by 0.67
ARC-AGI-1: Claude Fable 5 (Max) (98.5) beat Human Panel by 0.5
Deep20Bench: Claude Fable 5 (High) (12.06) beat Claude Opus 5 (High) by 0.28

The Aggregate Digest — 2026-08-05

2026-08-05T06:12:14.928861+00:00

Daily

New Benchmarks (3)

QuantCodeEval (Claude Code) (Full Pass Rate (%)): leader Claude Opus 5 (31.33), 3 models
QuantCodeEval (Codex CLI) (Full Pass Rate (%)): leader GPT-5.6 Sol (27.33), 2 models
QuantCodeEval (OpenCode) (Full Pass Rate (%)): leader Kimi K3 (26.0), 11 models

Top-10 New Scores (29)

Claude Fable 5 on SpacetimeDB LLM Benchmark (C#): 98.9 (#1)
Claude Fable 5 on SpacetimeDB LLM Benchmark (Rust): 96.6 (#6)
Claude Fable 5 on SpacetimeDB LLM Benchmark (TypeScript): 98.9 (#5)
Claude Opus 5 on BridgeBench Debugging: 1001.5 (#8)
Claude Opus 5 on BridgeBench Hallucination: 1001.5 (#8)
Claude Opus 5 on BridgeBench Refactoring: 1001.5 (#8)
Claude Opus 5 on BridgeBench Security: 1001.5 (#8)
Claude Opus 5 on SpacetimeDB LLM Benchmark (C#): 94.4 (#8)
Claude Opus 5 on SpacetimeDB LLM Benchmark (Rust): 96.6 (#4)
Claude Opus 5 on SpacetimeDB LLM Benchmark (TypeScript): 98.9 (#4)
GPT-5.6 Sol on Mercor APEX: 39.9 (#7)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (C#): 97.8 (#3)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (Rust): 97.8 (#3)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#1)
Kimi K3 on Epoch AI - Scicode: 58.68 (#3)
Kimi K3 on GameCraft-Bench: 56.96 (#3)
Kimi K3 on SpacetimeDB LLM Benchmark (C#): 93.3 (#9)
Kimi K3 on SpacetimeDB LLM Benchmark (Rust): 92.1 (#10)
Kimi K3 on SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#10)
Kimi K3 on TrackingAI IQ Test: 93.75 (#5)
Kimi K3 on TrackingAI IQ Test (Vision): 88.0 (#4)
Qwen 3.8 Max on AI Chess Leaderboard (Continuation): 1165.0 (#31)
Qwen 3.8 Max on AI Chess Leaderboard (Reasoning): 1521.0 (#20)
Qwen 3.8 Max on DeepSWE: 57.5 (#12)
Qwen 3.8 Max on KernelBench Hub - CUDA: 28.48 (#5)
Qwen 3.8 Max on KernelBench Hub - Hard: 33.89 (#8)
Qwen 3.8 Max on Multi-turn Debate (Lechmazur): 1587.5 (#11)
Qwen 3.8 Max on NYT Connections Extended: 88.9 (#15)
Qwen 3.8 Max on Vals AI Vibe Code Bench: 64.7 (#17)

New #1 Leaders (9)

MMOU: Gemini 3.1 Pro (Preview) (82.67) beat Gemini 2.5 Pro by 18.47
Design Arena (Video Editing): minimax-h3 (1395.0) beat Gemini 2.0 Flash by 14.0
SEAL - SWE Atlas - Test Writing: Claude Opus 5 (Claude Code) xHigh (62.22) beat Claude Fable 5 (Claude Code) xHigh by 6.62
MMEB Leaderboard: Ovis-Omni-Embedding-v0.1-3B (52.55) beat e5-omni-7B by 6.02
SEAL - SWE Atlas - Codebase QnA: Claude Opus 5 (Claude Code) xHigh (63.17) beat Claude Opus 4.8 (Claude Code) xhigh by 5.91
BridgeBench Security: Claude Fable 5 (1005.3) beat GPT-5.6 Sol by 0.2
BridgeBench Debugging: Claude Fable 5 (1005.3) beat GPT-5.6 Sol by 0.2
BridgeBench Refactoring: Claude Fable 5 (1005.3) beat GPT-5.6 Sol by 0.2
BridgeBench Hallucination: Claude Fable 5 (1005.3) beat GPT-5.6 Sol by 0.2