The Aggregate Digest

The Aggregate Digest — 2026-08-05

2026-08-05T06:12:14.928861+00:00

Daily

New Benchmarks (3)

QuantCodeEval (Claude Code) (Full Pass Rate (%)): leader Claude Opus 5 (31.33), 3 models
QuantCodeEval (Codex CLI) (Full Pass Rate (%)): leader GPT-5.6 Sol (27.33), 2 models
QuantCodeEval (OpenCode) (Full Pass Rate (%)): leader Kimi K3 (26.0), 11 models

Top-10 New Scores (29)

Claude Fable 5 on SpacetimeDB LLM Benchmark (C#): 98.9 (#1)
Claude Fable 5 on SpacetimeDB LLM Benchmark (Rust): 96.6 (#6)
Claude Fable 5 on SpacetimeDB LLM Benchmark (TypeScript): 98.9 (#5)
Claude Opus 5 on BridgeBench Debugging: 1001.5 (#8)
Claude Opus 5 on BridgeBench Hallucination: 1001.5 (#8)
Claude Opus 5 on BridgeBench Refactoring: 1001.5 (#8)
Claude Opus 5 on BridgeBench Security: 1001.5 (#8)
Claude Opus 5 on SpacetimeDB LLM Benchmark (C#): 94.4 (#8)
Claude Opus 5 on SpacetimeDB LLM Benchmark (Rust): 96.6 (#4)
Claude Opus 5 on SpacetimeDB LLM Benchmark (TypeScript): 98.9 (#4)
GPT-5.6 Sol on Mercor APEX: 39.9 (#7)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (C#): 97.8 (#3)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (Rust): 97.8 (#3)
GPT-5.6 Sol on SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#1)
Kimi K3 on Epoch AI - Scicode: 58.68 (#3)
Kimi K3 on GameCraft-Bench: 56.96 (#3)
Kimi K3 on SpacetimeDB LLM Benchmark (C#): 93.3 (#9)
Kimi K3 on SpacetimeDB LLM Benchmark (Rust): 92.1 (#10)
Kimi K3 on SpacetimeDB LLM Benchmark (TypeScript): 97.8 (#10)
Kimi K3 on TrackingAI IQ Test: 93.75 (#5)
Kimi K3 on TrackingAI IQ Test (Vision): 88.0 (#4)
Qwen 3.8 Max on AI Chess Leaderboard (Continuation): 1165.0 (#31)
Qwen 3.8 Max on AI Chess Leaderboard (Reasoning): 1521.0 (#20)
Qwen 3.8 Max on DeepSWE: 57.5 (#12)
Qwen 3.8 Max on KernelBench Hub - CUDA: 28.48 (#5)
Qwen 3.8 Max on KernelBench Hub - Hard: 33.89 (#8)
Qwen 3.8 Max on Multi-turn Debate (Lechmazur): 1587.5 (#11)
Qwen 3.8 Max on NYT Connections Extended: 88.9 (#15)
Qwen 3.8 Max on Vals AI Vibe Code Bench: 64.7 (#17)

New #1 Leaders (9)

MMOU: Gemini 3.1 Pro (Preview) (82.67) beat Gemini 2.5 Pro by 18.47
Design Arena (Video Editing): minimax-h3 (1395.0) beat Gemini 2.0 Flash by 14.0
SEAL - SWE Atlas - Test Writing: Claude Opus 5 (Claude Code) xHigh (62.22) beat Claude Fable 5 (Claude Code) xHigh by 6.62
MMEB Leaderboard: Ovis-Omni-Embedding-v0.1-3B (52.55) beat e5-omni-7B by 6.02
SEAL - SWE Atlas - Codebase QnA: Claude Opus 5 (Claude Code) xHigh (63.17) beat Claude Opus 4.8 (Claude Code) xhigh by 5.91
BridgeBench Security: Claude Fable 5 (1005.3) beat GPT-5.6 Sol by 0.2
BridgeBench Debugging: Claude Fable 5 (1005.3) beat GPT-5.6 Sol by 0.2
BridgeBench Refactoring: Claude Fable 5 (1005.3) beat GPT-5.6 Sol by 0.2
BridgeBench Hallucination: Claude Fable 5 (1005.3) beat GPT-5.6 Sol by 0.2

The Aggregate Digest — 2026-08-04

2026-08-04T06:18:08.392163+00:00

Daily

New Benchmarks (3)

BridgeBench Security (Confidence-weighted Elo): leader GPT-5.6 Sol (1005.1), 4 models
AI code security benchmark: 30 expert-level security tasks across 6 domains with 500+ test cases including adversarial hidden edge cases.
BridgeBench Refactoring (Confidence-weighted Elo): leader GPT-5.6 Sol (1005.1), 4 models
AI code refactoring benchmark: 15 tasks measuring behavior preservation, hidden regression resistance, and structural intent compliance.
BridgeBench Hallucination (Confidence-weighted Elo): leader GPT-5.6 Sol (1005.1), 4 models
AI code hallucination benchmark: 30 tasks across 6 clusters with 175 questions measuring fabrication rate and accuracy in coding contexts.

Top-10 New Scores (21)

GPT-5.6 Sol on BridgeBench Debugging: 1005.1 (#1)
GPT-5.6 Sol on Epoch AI - Critpt: 5.14 (#55)
GPT-5.6 Sol on Epoch AI - Scicode: 47.11 (#52)
GPT-5.6 Sol on MirrorCode: 20.0 (#2)
GPT-5.6 Sol on SWE-rebench: 62.34 (#6)
Kimi K3 on Epoch AI - Vending Bench 2: 5165.04 (#19)
Kimi K3 on InferenceBench: 5.7 (#12)
Qwen 3.8 Max on BenchLM: 78.2 (#6)
Qwen 3.8 Max on LLM Stats (DeepSWE 1.1): 56.6 (#9)
Qwen 3.8 Max on LLM Stats (FrontierSWE): 73.5 (#5)
Qwen 3.8 Max on LLM Stats (ScreenSpot Pro): 84.5 (#3)
Qwen 3.8 Max on LLM Stats (Terminal-Bench 2.1): 86.6 (#4)
Qwen 3.8 Max on LLM Stats Score: 52.56 (#7)
Qwen 3.8 Max on LM Market Cap LMC Score: 80.4 (#58)
Qwen 3.8 Max on MLS-Bench Lite: 41.0 (#5)
Qwen 3.8 Max on Roboflow Playground Object Detection: 1213.0 (#17)
Qwen 3.8 Max on Roboflow Playground Overall: 1213.0 (#19)
Qwen 3.8 Max on Vals AI (Vals Index): 66.12 (#10)
Qwen 3.8 Max on Vals AI CorpFin v2: 65.85 (#25)
Qwen 3.8 Max on Vals AI Terminal-Bench 2.1: 67.42 (#18)
Qwen 3.8 Max on ZeroEval GPQA Diamond: 92.6 (#12)

New #1 Leaders (18)

LLM Stats (SkillsBench): Qwen 3.8 Max (70.2) beat Qwen 3.7 Max by 11.0
Korean LLM Eval - KMMLU-HARD: GPT-5.6 Sol (85.04) beat GPT-5.2 (Medium) by 10.41
MirrorCode: Claude Fable 5 (63.9) beat Claude Opus 4.7 by 7.9
LLM Stats (ERQA): Qwen 3.8 Max (77.8) beat Seed 2.1 Pro by 5.8
Korean LLM Eval - CLIcK: GPT-5.6 Sol (95.5) beat GPT-5.2 (Medium) by 4.58
LLM Stats (LVBench): Qwen 3.8 Max (81.8) beat Seed 2.1 Pro by 3.8
LLM Stats (LongBench v2): Qwen 3.8 Max (66.3) beat Qwen 3.5 397B A17B by 3.1
Korean LLM Eval - HAE-RAE Bench: GPT-5.6 Sol (94.64) beat GPT-5.2 (Medium) by 2.83
LLM Stats (VideoMME w sub.): Qwen 3.8 Max (90.4) beat Qwen 3.6 27B by 2.7
Spider 2.0-Lite: GLM-5.2 (76.23) beat ktx by 2.56
LLM Stats (HealthBench): Qwen 3.8 Max (60.2) beat Kimi K2 (0905) (Thinking) by 2.2
SWE-rebench: Claude Fable 5 [high] (64.5) beat GPT-5.5 (xHigh) by 1.77
LLM Stats (NL2Repo): Qwen 3.8 Max (55.9) beat DeepSeek V4 Flash (0731) by 1.7
LLM Stats (OSWorld-Verified): Qwen 3.8 Max (86.1) beat Claude Fable 5 by 1.1
LLM Stats (RealWorldQA): Qwen 3.8 Max (88.0) beat Qwen 3.7 Plus by 1.1
ForecastBench: captain-jack (68.9) beat Cassi-2026-05-10 by 0.2
InferenceBench: Claude Opus 5 (8.9) beat Claude Fable 5 (Low) by 0.16
ProphetArena: Gemini 3.6 Flash (0.978) beat Gemini 3.1 Pro (Preview) by 0.03

The Aggregate Digest — 2026-08-03

2026-08-03T06:38:28.867175+00:00

Daily

New Benchmarks (4)

Supabase Evals (Skills) (Tasks Passed (%)): leader GPT-5.6 Sol (Medium) (100.0), 6 models
Supabase Evals with the Supabase skills pack loaded: AI coding agents bootstrap and build real Supabase apps over 19 CLI tasks (migrations, RLS, auth, data APIs); percent of tasks with every check passed. Each model runs under its vendor-native harness (claude-code / codex / opencode).
Supabase Evals (No Skills) (Tasks Passed (%)): leader GPT-5.6 Sol (Low) (100.0), 6 models
Supabase Evals without the skills pack: the same 19 CLI build tasks solved from the agent’s own knowledge of Supabase; percent of tasks with every check passed, one row per underlying model under its vendor-native harness.
Hermes Best Models (Pass Rate (%)): leader Gemini 2.5 Flash (100.0), 19 models
Hermes agent-framework model evaluation: 25 agentic tasks (simple QA, multi-step tool use, delegation, reasoning, failure handling) run per LLM; task pass rate in percent, latency/cost excluded.
LLM Stats (Agents' Last Exam) (Score (%)): leader GPT-5.6 Sol (52.7), 5 models

New #1 Leaders (2)

LLM Stats (NL2Repo): DeepSeek V4 Flash (0731) (54.2) beat GLM-5.2 by 5.3
MyPCBench: Claude Opus 4.8 (62.0) beat Claude Opus 4.6 by 3.8

The Aggregate Digest — 2026-08-02

2026-08-02T06:27:22.986816+00:00

Daily

New Benchmarks (5)

ATM-Bench Hard (Oracle QS (%)): leader GPT-5.5 (71.49), 18 models
The hard split of ATM-Bench: adversarially selected long-term memory questions. Oracle track (full context, one row per answer model); QS percent, LLM-judged.
MobileWorld - GUI Only (Success Rate (%)): leader Kimi K3 (74.4), 34 models
The GUI-only split of MobileWorld (Tongyi-MAI): 117 smartphone-environment tasks solved end-to-end from screen interaction alone; success rate %, agentic planner+grounder combos excluded.
MobileWorld - User Interaction (Success Rate (%)): leader Seed 2.0 Pro (61.4), 23 models
The user-interaction split of MobileWorld: 44 tasks requiring mid-task clarification or input from a simulated user; success rate %, end-to-end models only.
MLE-Bench Lite (Codex) (Medal Average (%)): leader GPT-5.5 (68.18), 2 models
FrontisAI OpenRSI's MLE-Bench Lite run — Kaggle-competition ML engineering with the Codex agent scaffold, 12-hour per-task budget on one RTX 4090; mean medal rate over the 22-task Lite split.
InfoOps Bench (Integrity Score (% refused)): leader Claude Opus 4.8 (96.7), 36 models
Live safety benchmark measuring whether frontier models can be co-opted for state-backed information operations, drawn from a monitoring pipeline tracking Russian, Chinese and Iranian state-backed assets. Scored as the Integrity Score — the percentage of information-operation requests the model refused — so this measures refusal propensity rather than capability. Updated weekly.

Top-10 New Scores (1)

Claude Opus 5 on HieroglyphBench: 14.6 (#15)

Weekly

New Benchmarks (5)

ATM-Bench Hard (Oracle QS (%)): leader GPT-5.5 (71.49), 18 models
The hard split of ATM-Bench: adversarially selected long-term memory questions. Oracle track (full context, one row per answer model); QS percent, LLM-judged.
MobileWorld - GUI Only (Success Rate (%)): leader Kimi K3 (74.4), 34 models
The GUI-only split of MobileWorld (Tongyi-MAI): 117 smartphone-environment tasks solved end-to-end from screen interaction alone; success rate %, agentic planner+grounder combos excluded.
MobileWorld - User Interaction (Success Rate (%)): leader Seed 2.0 Pro (61.4), 23 models
The user-interaction split of MobileWorld: 44 tasks requiring mid-task clarification or input from a simulated user; success rate %, end-to-end models only.
MLE-Bench Lite (Codex) (Medal Average (%)): leader GPT-5.5 (68.18), 2 models
FrontisAI OpenRSI's MLE-Bench Lite run — Kaggle-competition ML engineering with the Codex agent scaffold, 12-hour per-task budget on one RTX 4090; mean medal rate over the 22-task Lite split.
InfoOps Bench (Integrity Score (% refused)): leader Claude Opus 4.8 (96.7), 36 models
Live safety benchmark measuring whether frontier models can be co-opted for state-backed information operations, drawn from a monitoring pipeline tracking Russian, Chinese and Iranian state-backed assets. Scored as the Integrity Score — the percentage of information-operation requests the model refused — so this measures refusal propensity rather than capability. Updated weekly.

New Models (92)

Claude Opus 5 — ELO 1787, #1
- GBA Eval: 0.796 (#1/23)
- KernelBench Hub - Mega: 24.29 (#1/10)
- KernelBench Hub - CUDA: 196.1 (#1/6)
- HiL-Bench: 57.0 (#1/15)
- FoodTruckBench: 75264.0 (#1/10)
- Vending-Bench 2: 11181.87 (#1/58)
- Gert Labs Rankings: 75.81 (#1/91)
- AI for Education Pedagogy - Technology: 91.51 (#1/227)
- AI for Education SEND: 88.99 (#1/219)
- KernelBench Hub - Hard: 52.61 (#2/13)
Claude Fable 5 — ELO 1779, #3
- MWS Vision Bench: 79.9 (#1/46)
- ProphetArena: 0.9503 (#2/21)
GPT-5.6 Sol — ELO 1778, #4
- Vals AI Time Horizon Index: 13.0 (#1/6)
- VANTAGE-Bench - Temporal: 46.25 (#1/17)
- VANTAGE-Bench - Video QA: 77.9 (#1/17)
- MWS Vision Bench: 79.0 (#2/46)
- VANTAGE-Bench: 69.03 (#2/17)
- VANTAGE-Bench - Semantic: 76.9 (#2/17)
- VANTAGE-Bench - Event Verification: 75.9 (#2/17)
- VANTAGE-Bench - Single Object Tracking: 75.02 (#2/17)
- Design Arena (SVG): 1341.0 (#3/102)
- VANTAGE-Bench - Spatial: 77.93 (#4/17)
Kimi K3 — ELO 1762, #6
- SWE-Milestone: 46.75 (#2/24)
- Ramp SWE-Bench: 87.18 (#3/27)
- Epoch AI - Scicode: 58.68 (#3/102)
- MWS Vision Bench: 77.7 (#4/46)
- Epoch AI - Proofbench: 70.0 (#5/50)
- Vals AI Time Horizon Index: 5.17 (#5/6)
- Epoch AI - Apex Agents: 39.3 (#7/55)
- TrackingAI IQ Test (Offline): 81.25 (#7/39)
- Epoch AI - Blueprint Bench 2: 29.5 (#8/21)
- Blueprint-Bench 2: 0.295 (#8/21)
Gemini 3.6 Flash — ELO 1757, #7
- MathArena - Kangaroo 2025 Levels 9-10: 100.0 (#1/22)
- VANTAGE-Bench: 69.52 (#1/17)
- VANTAGE-Bench - Semantic: 79.41 (#1/17)
- VANTAGE-Bench - Event Verification: 82.0 (#1/17)
- VANTAGE-Bench - Single Object Tracking: 75.99 (#1/17)
- VANTAGE-Bench - Spatial: 78.85 (#2/17)
- VANTAGE-Bench - Temporal: 43.83 (#2/17)
- VANTAGE-Bench - Video QA: 76.82 (#2/17)
- Chatbot Arena (Vision): 1301.0 (#3/143)
- MathArena - Kangaroo 2025 Levels 5-6: 88.33 (#3/22)
GPT-5.5 — ELO 1753, #8
- MWS Vision Bench: 77.8 (#3/46)
- Epoch AI - Rli: 6.25 (#4/12)
Muse Spark 1.1 — ELO 1749, #9
- Epoch AI - Scicode: 58.22 (#4/102)
- Epoch AI - Apex Agents: 41.9 (#4/55)
- BridgeBench Debugging: 1000.2 (#6/14)
- Vending-Bench 2: 6520.47 (#9/58)
- Chatbot Arena (Document): 1472.0 (#13/38)
- Chatbot Arena (Vision): 1283.0 (#14/143)
- WebDev Arena: 1536.36 (#15/97)
- ProphetArena: 0.9285 (#17/21)
- Wolfram LLM Benchmarking Project: 67.8 (#23/528)
- Epoch AI - ECI: 155.49 (#67/450)
Gemini 3.1 Pro (Preview) — ELO 1742, #10
- APEX v1 Consulting: 66.7 (#2/18)
- APEX v1 Investment Banking: 63.4 (#3/18)
- APEX v1: 65.1 (#4/4)
- APEX v1 Medicine (MD): 62.7 (#4/18)
- APEX v1 Big Law: 67.6 (#13/18)
Gemini 3.5 Flash — ELO 1737, #12
- BridgeBench Debugging: 994.9 (#12/14)
Claude Opus 4.8 — ELO 1736, #13
- BridgeBench Debugging: 1005.1 (#1/14)
Grok 4.5 — ELO 1735, #14
- Agents' Last Exam: 27.0 (#5/22)
- Kaggle FACTS (Google): 50.2 (#13/32)
- Kaggle FACTS Parametric: 46.83 (#14/36)
- Chatbot Arena (Vision): 1282.0 (#16/143)
- ProphetArena: 0.9284 (#18/21)
- Chatbot Arena (Document): 1454.0 (#20/38)
GPT-5.6 Terra — ELO 1734, #15
- MWS Vision Bench: 74.9 (#5/46)
- Senior SWE-Bench: 27.4 (#7/10)
- Kaggle FACTS (Google): 52.0 (#11/32)
- Lean AI Formalization Leaderboard: 0.0 (#51/51)
GPT-5.4 — ELO 1721, #18
- APEX v1: 67.2 (#1/4)
- APEX v1 Investment Banking: 64.5 (#2/18)
- APEX v1 Big Law: 74.3 (#3/18)
- APEX v1 Consulting: 65.9 (#3/18)
- APEX v1 Medicine (MD): 64.0 (#3/18)
- Epoch AI - Apex Agents: 34.9 (#12/55)
- Chess Puzzles (Epoch AI): 5.0 (#80/86)
- OTIS Mock AIME 2024-25: 57.78 (#92/174)
Gemini 3 Pro — ELO 1720, #19
- APEX v1 Investment Banking: 59.7 (#8/18)
- APEX v1 Consulting: 58.6 (#10/18)
- APEX v1 Medicine (MD): 58.4 (#12/18)
- APEX v1 Big Law: 66.8 (#15/18)
Claude Opus 4.7 — ELO 1715, #21
- BridgeBench Debugging: 999.8 (#8/14)
Gemini 3 Flash — ELO 1713, #23
- APEX v1 Investment Banking: 59.8 (#6/18)
- APEX v1 Consulting: 59.9 (#9/18)
- APEX v1 Medicine (MD): 58.0 (#15/18)
- APEX v1 Big Law: 66.0 (#16/18)
GLM-5.2 — ELO 1712, #25
- ProgramBench Almost: 8.5 (#3/15)
- ProgramBench: 0.0 (#4/15)
GPT-5.6 Luna — ELO 1712, #26
- MWS Vision Bench: 72.7 (#9/46)
- Kaggle FACTS Parametric: 25.0 (#23/36)
- Kaggle FACTS (Google): 34.78 (#30/32)
Claude Sonnet 5 — ELO 1712, #27
- Design Arena (Game Dev): 1352.0 (#4/143)
- MWS Vision Bench: 73.1 (#8/46)
- BridgeBench Debugging: 999.8 (#9/14)
- TrackingAI IQ Test (Vision): 49.02 (#13/24)
- Design Arena (3D): 1313.0 (#14/135)
- Design Arena (UI Components): 1309.0 (#15/140)
- Design Arena (Website): 1298.0 (#17/161)
- TrackingAI IQ Test: 68.75 (#20/38)
- TrackingAI IQ Test (Offline): 62.5 (#20/39)
- Design Arena (Data Viz): 1265.0 (#27/142)
GPT-5.3 Codex — ELO 1711, #28
- APEX v1 Investment Banking: 65.0 (#1/18)
- APEX v1 Consulting: 65.5 (#4/18)
- APEX v1 Big Law: 69.8 (#12/18)
- APEX v1 Medicine (MD): 58.1 (#14/18)
- Epoch AI - Apex Agents: 31.8 (#20/55)
Claude Opus 4.6 — ELO 1710, #29
- APEX v1 Medicine (MD): 71.6 (#1/18)
- APEX v1: 65.7 (#2/4)
- APEX v1 Big Law: 76.4 (#2/18)
- APEX v1 Consulting: 62.5 (#5/18)
- APEX v1 Investment Banking: 53.4 (#13/18)
GPT-5.2 Codex — ELO 1710, #31
- APEX v1 Consulting: 66.9 (#1/18)
- APEX v1: 65.3 (#3/4)
- APEX v1 Investment Banking: 61.8 (#5/18)
- APEX v1 Big Law: 73.0 (#8/18)
- APEX v1 Medicine (MD): 59.7 (#9/18)
- AI Chess Leaderboard (Reasoning): 1303.0 (#32/306)
- AI Chess Leaderboard (Continuation): 1151.0 (#33/251)
Kimi K2.7 Code — ELO 1706, #33
- Mercor APEX: 27.6 (#20/52)
- Epoch AI - Apex Agents: 27.6 (#23/55)
Grok Build 0.1 — ELO 1704, #34
- Epoch AI - Scicode: 50.23 (#30/102)
Qwen 3.7 Plus — ELO 1702, #35
- Epoch AI - Critpt: 9.14 (#40/110)
- Epoch AI - Scicode: 45.49 (#46/102)
GPT-5.2 — ELO 1699, #37
- APEX v1 Investment Banking: 62.3 (#4/18)
- APEX v1 Big Law: 73.4 (#6/18)
- APEX v1 Consulting: 61.4 (#6/18)
- APEX v1 Medicine (MD): 61.9 (#6/18)
- Chess Puzzles (Epoch AI): 4.0 (#82/86)
- OTIS Mock AIME 2024-25: 62.22 (#88/174)
Claude Opus 4.5 — ELO 1699, #38
- APEX v1 Medicine (MD): 61.5 (#7/18)
- APEX v1 Big Law: 70.1 (#10/18)
- APEX v1 Consulting: 51.1 (#16/18)
- APEX v1 Investment Banking: 46.7 (#17/18)
Inkling — ELO 1699, #39
- Design Arena (Website): 1243.0 (#46/161)
- WebDev Arena: 1417.36 (#47/97)
GPT-5 — ELO 1695, #43
- APEX v1 Big Law: 76.6 (#1/18)
- APEX v1 Medicine (MD): 62.4 (#5/18)
- APEX v1 Consulting: 60.0 (#8/18)
- APEX v1 Investment Banking: 59.1 (#9/18)
Kimi K2.6 — ELO 1691, #44
- BridgeBench Debugging: 1005.1 (#3/14)
- MWS Vision Bench: 74.3 (#6/46)
DeepSeek V4 Pro — ELO 1691, #45
- BridgeBench Debugging: 1005.1 (#2/14)
- Onyx Open LLM Leaderboard: 87.5 (#3/19)
- KernelBench Hub - Mega: 1.38 (#10/10)
Hy3 — ELO 1691, #46
- Chatbot Arena (Text): 1456.0 (#50/384)
GPT-5 Codex — ELO 1691, #47
- APEX v1 Big Law: 73.5 (#5/18)
- APEX v1 Consulting: 60.3 (#7/18)
- APEX v1 Investment Banking: 59.8 (#7/18)
- AI Chess Leaderboard (Reasoning): 1777.0 (#7/306)
- APEX v1 Medicine (MD): 57.6 (#16/18)
- AI Chess Leaderboard (Continuation): 1291.0 (#24/251)
Inkling Small — ELO 1688, #50
- SEAL - AudioMultiChallenge: 54.87 (#2/33)
- SEAL - AudioMultiChallenge - Text Output: 54.87 (#2/18)
- Vals AI CorpFin v2: 69.62 (#5/128)
- SEAL - MCP Atlas: 79.2 (#8/29)
- Vals AI Public Benefits Bench: 59.41 (#16/26)
- Vals AI Legal Research Bench: 25.48 (#18/28)
- Vals AI Excel Modeling: 31.76 (#27/29)
- Vals AI Terminal-Bench 2.1: 55.06 (#28/47)
- Vals AI (Vals Index): 50.17 (#29/42)
- AA-Briefcase: 20.0 (#31/53)
MiMo-V2.5-Pro — ELO 1685, #53
- Onyx Open LLM Leaderboard: 68.5 (#18/19)
- EQ-Bench Longform Writing: 73.7 (#18/125)
GPT-5.1 Codex — ELO 1682, #56
- APEX v1 Big Law: 73.2 (#7/18)
- AI Chess Leaderboard (Reasoning): 1743.0 (#8/306)
- APEX v1 Investment Banking: 58.7 (#10/18)
- APEX v1 Consulting: 57.6 (#11/18)
- APEX v1 Medicine (MD): 58.1 (#13/18)
- AI Chess Leaderboard (Continuation): 1418.0 (#15/251)
Claude Sonnet 4.6 — ELO 1681, #57
- APEX v1 Medicine (MD): 66.7 (#2/18)
- APEX v1 Big Law: 72.3 (#9/18)
- APEX v1 Consulting: 41.9 (#18/18)
- APEX v1 Investment Banking: 36.9 (#18/18)
O3 — ELO 1681, #58
- APEX v1 Big Law: 73.6 (#4/18)
- APEX v1 Medicine (MD): 59.1 (#10/18)
- APEX v1 Investment Banking: 55.3 (#11/18)
- APEX v1 Consulting: 54.1 (#13/18)
Gemini 3.5 Flash Lite — ELO 1677, #61
- Agent Arena - Confirmed Success: 17.21 (#2/44)
- VANTAGE-Bench - Event Verification: 70.41 (#5/17)
- Agent Arena - Praise vs Complaint: 17.63 (#6/44)
- VANTAGE-Bench - Single Object Tracking: 55.77 (#6/17)
- Agent Arena - Tool Hallucination: 0.45 (#7/44)
- VANTAGE-Bench - Semantic: 68.01 (#7/17)
- Agent Arena: 10.2 (#8/44)
- SvelteBench: 100.0 (#8/155)
- VANTAGE-Bench: 52.89 (#8/17)
- Kaggle FACTS (Google): 53.46 (#9/32)
Grok 4.3 — ELO 1676, #62
- Epoch AI - Critpt: 0.0 (#102/110)
GPT-5.4 Mini — ELO 1675, #65
- Epoch AI - Apex Agents: 24.6 (#24/55)
DeepSeek V4 Flash — ELO 1674, #67
- Onyx Open LLM Leaderboard: 86.2 (#5/19)
- Ramp SWE-Bench: 79.49 (#11/27)
Claude Opus 4.1 (20250805) — ELO 1664, #77
- Chess Puzzles (Epoch AI): 7.0 (#73/86)
Qwen 3.6 27B — ELO 1662, #81
- Onyx Open LLM Leaderboard: 86.2 (#6/19)
Step 3.7 Flash — ELO 1659, #86
- Epoch AI - Critpt: 2.29 (#57/110)
- Epoch AI - Scicode: 40.05 (#66/102)
GPT-5 Mini — ELO 1652, #90
- Epoch AI - Critpt: 0.0 (#110/110)
MiMo-V2.5 — ELO 1652, #92
- Epoch AI - Critpt: 3.71 (#52/110)
- Epoch AI - Scicode: 43.06 (#53/102)
Nemotron 3 Ultra — ELO 1648, #98
- Onyx Open LLM Leaderboard: 86.8 (#4/19)
- BridgeBench Debugging: 1000.2 (#7/14)
- Epoch AI - ECI: 142.67 (#238/450)
GPT-5 Chat — ELO 1639, #115
- AI Chess Leaderboard (Continuation): 1465.0 (#13/251)
- AI Chess Leaderboard (Reasoning): 847.0 (#83/306)
Qwen 3.6 35B A3B — ELO 1637, #118
- Onyx Open LLM Leaderboard: 85.2 (#9/19)
Ring-2.6-1T — ELO 1634, #123
- Epoch AI - Critpt: 3.71 (#51/110)
- Epoch AI - Scicode: 42.36 (#57/102)
MiniMax-M2.7 — ELO 1627, #139
- BridgeBench Debugging: 994.9 (#14/14)
GPT-5.1 Codex Mini — ELO 1620, #151
- AI Chess Leaderboard (Reasoning): 952.0 (#63/306)
- AI Chess Leaderboard (Continuation): 812.0 (#68/251)
Gemma 4 31B — ELO 1619, #153
- Onyx Open LLM Leaderboard: 85.2 (#8/19)
- BridgeBench Debugging: 994.9 (#13/14)
Claude Haiku 4.5 — ELO 1617, #157
- BridgeBench Debugging: 1000.2 (#5/14)
GPT-5.4 Nano — ELO 1605, #178
- Epoch AI - Apex Agents: 16.9 (#36/55)
DeepSeek V3 Chat — ELO 1602, #183
- Chess Puzzles (Epoch AI): 1.0 (#84/86)
- OTIS Mock AIME 2024-25: 48.89 (#102/174)
Mistral Medium 3.5 — ELO 1601, #186
- BridgeBench Debugging: 999.8 (#10/14)
- Bongard Problems (Classic): 6.9 (#11/15)
- Epoch AI - ECI: 142.46 (#243/450)
Qwen3 Coder Next — ELO 1600, #190
- Vector Eval - MATH: 97.82 (#1/12)
- Vector Eval - GPQA-D: 75.63 (#1/12)
- Vector Eval - InterCode-CTF: 85.9 (#1/6)
- Vector Eval - GAIA: 36.36 (#2/6)
- Vector Eval - GSM8K: 95.83 (#3/12)
- Vector Eval - MMLU-Pro: 82.46 (#3/12)
- Vector Eval - In-House-CTF: 38.46 (#3/6)
- Vector Eval - ARC-E: 98.82 (#4/12)
- Vector Eval - DROP: 87.91 (#4/9)
- Vector Eval - HumanEval: 91.46 (#5/12)
Mistral Medium 3.1 — ELO 1596, #198
- Bongard Problems (Classic): 11.5 (#5/15)
- Guesswork: 0.9902 (#23/35)
- Epoch AI - Scicode: 33.8 (#84/102)
GPT-OSS-120B — ELO 1595, #200
- Epoch AI - Apex Agents: 4.7 (#49/55)
Mercury 2 — ELO 1595, #201
- Epoch AI - Critpt: 0.85 (#69/110)
- Epoch AI - Scicode: 38.66 (#72/102)
Nemotron 3 Super — ELO 1592, #205
- Onyx Open LLM Leaderboard: 83.7 (#14/19)
- Epoch AI - Scicode: 36.0 (#76/102)
Mistral Large 3 — ELO 1588, #210
- Guesswork: 1.0511 (#14/35)
- Bongard Problems (Classic): 5.1 (#15/15)
- Epoch AI - Scicode: 36.23 (#74/102)
- WebDev Arena: 1229.89 (#91/97)
- Epoch AI - Critpt: 0.0 (#101/110)
GPT-4.1 Mini — ELO 1586, #217
- Chess Puzzles (Epoch AI): 7.0 (#75/86)
DeepSeek V3 — ELO 1580, #236
- Epoch AI - Scicode: 35.42 (#81/102)
GPT-4o (2024-08-06) — ELO 1579, #240
- Chess Puzzles (Epoch AI): 13.0 (#62/86)
Magistral Medium 1.2 — ELO 1577, #243
- Guesswork: 1.275 (#6/35)
- Bongard Problems (Classic): 6.6 (#12/15)
- Epoch AI - Scicode: 39.24 (#70/102)
- Epoch AI - Critpt: 0.29 (#77/110)
Mistral Small 4 — ELO 1568, #258
- Bongard Problems (Classic): 5.6 (#14/15)
- Guesswork: 0.9222 (#28/35)
Devstral Small 2 — ELO 1566, #261
- Guesswork: 1.0495 (#15/35)
Claude 3 Opus (20240229) — ELO 1564, #265
- Chess Puzzles (Epoch AI): 5.0 (#79/86)
GLM-4.7 Flash — ELO 1562, #270
- ReasonScape R12: 596.84 (#68/85)
Magistral Small 1.2 — ELO 1560, #278
- Guesswork: 1.2208 (#8/35)
- Bongard Problems (Classic): 6.6 (#13/15)
- Epoch AI - Critpt: 0.29 (#78/110)
- Epoch AI - Scicode: 35.19 (#82/102)
GPT-4 Turbo — ELO 1557, #287
- Chess Puzzles (Epoch AI): 6.0 (#77/86)
- Epoch AI - ECI: 127.26 (#349/450)
Solar Pro 3 — ELO 1555, #293
- Epoch AI - Scicode: 24.65 (#97/102)
- Epoch AI - Critpt: 0.0 (#103/110)
Llama 3.3 70B Instruct — ELO 1545, #317
- Epoch AI - Scicode: 26.04 (#93/102)
- Epoch AI - Critpt: 0.0 (#97/110)
Mistral Small 3.2 — ELO 1545, #318
- Guesswork: 1.2168 (#9/35)
- Bongard Problems (Classic): 8.3 (#10/15)
- Epoch AI - Scicode: 26.39 (#92/102)
- Epoch AI - Critpt: 0.0 (#107/110)
Mistral Medium 3 — ELO 1544, #321
- Bongard Problems (Classic): 9.3 (#7/15)
- Guesswork: 1.0598 (#13/35)
Mistral Small 3.1 — ELO 1536, #337
- Epoch AI - Scicode: 26.5 (#91/102)
Ministral 3 14B — ELO 1525, #360
- Bongard Problems (Classic): 9.8 (#6/15)
- Guesswork: 1.0049 (#22/35)
Ministral 3 8B — ELO 1513, #383
- Guesswork: 1.2684 (#7/35)
- Bongard Problems (Classic): 8.6 (#9/15)
GPT-4o Mini (2024-07-18) — ELO 1511, #388
- Chess Puzzles (Epoch AI): 0.0 (#86/86)
Gemma 3 12B (IT) — ELO 1510, #390
- Epoch AI - Scicode: 17.36 (#99/102)
- Epoch AI - Critpt: 0.0 (#108/110)
Gemma 4 E4B — ELO 1476, #445
- ReasonScape R12: 609.87 (#65/85)
Phi-4 Mini Instruct — ELO 1405, #571
- Epoch AI - Scicode: 10.76 (#102/102)
- Epoch AI - Critpt: 0.0 (#104/110)
Celeris-1 — ELO 1521
- AA GDPval: 533.56 (#135/179)
- GDPval-AA: 534.0 (#139/184)
- Tau3 Banking: 4.95 (#151/183)
- BenchLM: 42.0 (#169/200)
- AA Humanity's Last Exam: 6.63 (#287/551)
- AA Long Context Reasoning: 27.0 (#301/483)
- AA GPQA Diamond: 63.13 (#326/555)
- AA Omniscience - Software Engineering (SWE) - PHP: 16.0 (#332/460)
- Artificial Analysis Intelligence Index: 11.81 (#337/554)
- AA Omniscience - Software Engineering (SWE) - Dart: 14.0 (#340/460)
DeepSeek Reasoner — ELO 1645
- gg-bench: 22.27 (#3/7)
GPT-5.1 Codex Max — ELO 1690
- AI Chess Leaderboard (Continuation): 1354.0 (#19/251)
- AI Chess Leaderboard (Reasoning): 1354.0 (#30/306)
- Epoch AI - ECI: 149.96 (#125/450)
Grok 4.1 — ELO 1657
- APEX v1 Big Law: 70.0 (#11/18)
- APEX v1 Medicine (MD): 58.4 (#11/18)
- APEX v1 Consulting: 52.7 (#14/18)
- APEX v1 Investment Banking: 48.8 (#15/18)
- Epoch AI - Apex Agents: 12.8 (#41/55)
Laguna M.1 — ELO 1581
- Vals AI Public Benefits Bench: 43.84 (#25/26)
- Epoch AI - Proofbench: 0.0 (#50/50)
- WebDev Arena: 1349.08 (#72/97)
Laguna XS.2 — ELO 1556
- Vals AI Public Benefits Bench: 41.41 (#26/26)
- Epoch AI - Proofbench: 0.0 (#49/50)
- Vals AI MedScribe: 61.43 (#75/77)
- WebDev Arena: 1303.68 (#80/97)
codestral-2508 — ELO 1544
- Guesswork: 0.9856 (#24/35)

Top-10 New Scores (46)

Claude Fable 5 on ProphetArena: 0.9503 (#2)
Claude Opus 5 on AI for Education Pedagogy: 91.1 (#4)
Claude Opus 5 on AI for Education Pedagogy - Maths: 95.24 (#2)
Claude Opus 5 on AI for Education Pedagogy - Primary: 95.31 (#4)
Claude Opus 5 on AI for Education Pedagogy - Science: 92.35 (#19)
Claude Opus 5 on AI for Education Pedagogy - Secondary: 89.94 (#5)
Claude Opus 5 on AI for Education Pedagogy - Social studies: 82.73 (#70)
Claude Opus 5 on Blueprint-Bench 2: 0.304 (#7)
Claude Opus 5 on Design Arena (3D): 1382.0 (#2)
Claude Opus 5 on HieroglyphBench: 14.6 (#15)
Claude Opus 5 on Kaggle FACTS Parametric: 48.77 (#12)
Claude Opus 5 on Kaggle Game Arena Poker (Heads Up): 20.34 (#3)
Claude Opus 5 on KernelBench Hub - Hard: 52.61 (#2)
Claude Opus 5 on LisanBench: 0.1144 (#38)
Claude Opus 5 on MineBench: 2103.11 (#2)
Claude Opus 5 on SpeechMap Compliance: 59.1 (#202)
Claude Opus 5 on TaxCalcBench: 18.0 (#9)
Claude Opus 5 on TrackingAI IQ Test: 82.35 (#14)
Claude Opus 5 on TrackingAI IQ Test (Mensa Norway): 77.14 (#17)
Claude Opus 5 on TrackingAI IQ Test (Offline): 81.25 (#8)
GPT-5.6 Sol on Design Arena (SVG): 1341.0 (#3)
GPT-5.6 Sol on Epoch AI - Apex Agents: 37.7 (#9)
GPT-5.6 Sol on Kaggle FACTS (Google): 52.41 (#10)
GPT-5.6 Sol on Lean AI Formalization Leaderboard: 2.0 (#24)
GPT-5.6 Sol on MWS Vision Bench: 79.0 (#2)
GPT-5.6 Sol on VANTAGE-Bench: 69.03 (#2)
GPT-5.6 Sol on VANTAGE-Bench - Event Verification: 75.9 (#2)
GPT-5.6 Sol on VANTAGE-Bench - Semantic: 76.9 (#2)
GPT-5.6 Sol on VANTAGE-Bench - Single Object Tracking: 75.02 (#2)
GPT-5.6 Sol on VANTAGE-Bench - Spatial: 77.93 (#4)
Kimi K3 on Blueprint-Bench 2: 0.295 (#8)
Kimi K3 on Epoch AI - Apex Agents: 39.3 (#7)
Kimi K3 on Epoch AI - Blueprint Bench 2: 29.5 (#8)
Kimi K3 on Epoch AI - ECI: 157.39 (#44)
Kimi K3 on Epoch AI - Proofbench: 70.0 (#5)
Kimi K3 on Epoch AI - Scicode: 58.68 (#3)
Kimi K3 on MWS Vision Bench: 77.7 (#4)
Kimi K3 on Mercor APEX: 39.3 (#8)
Kimi K3 on Ramp SWE-Bench: 87.18 (#3)
Kimi K3 on SWE-Milestone: 46.75 (#2)
Kimi K3 on Senior SWE-Bench: 20.2 (#10)
Kimi K3 on TrackingAI IQ Test (Offline): 81.25 (#7)
Kimi K3 on Vals AI ProgramBench: 62.77 (#12)
Kimi K3 on Vals AI Time Horizon Index: 5.17 (#5)
Kimi K3 on Vending-Bench 2: 5165.04 (#19)
Kimi K3 on Vending-Bench 2: 5165.04 (#19)

New #1 Leaders (49)

FoodTruckBench: Claude Opus 5 (75264.0) beat GPT-5.5 by 13856.0
Epoch AI - Vending Bench 2: Claude Opus 5 (Unknown) (11181.87) beat Claude Opus 4.7 (Unknown) by 245.11
Vending-Bench 2: Claude Opus 5 (11181.87) beat Claude Opus 4.7 by 245.11
KernelBench Hub - CUDA: Claude Opus 5 (196.1) beat Claude Fable 5 by 123.44
WebDev Arena: Claude Opus 5 (Max) (1711.88) beat Claude Fable 5 (Unknown) by 57.95
WDCD R3 Pressure Integrity: Grok 4 (163.0) beat Gemini 3.1 Pro (Preview) by 25.0
Chatbot Arena (Code): Claude Opus 5 (Max) (1705.0) beat Kimi K3 by 23.0
MathArena - ARXIV_FALSE June: Claude Opus 5 (Max) (90.74) beat GPT-5.5 (xHigh) by 21.3
GSMA Open-Telco - TeleTables: OTel-2.0-LLM-31B-IT (79.76) beat OTel-LLM-8.3B-QnA by 17.96
LLM Chess (Saplin): GPT-5.6 Sol (High) (1548.5) beat GPT-5.5 (Medium) by 16.3
Epoch AI - Enigma Eval: Claude Fable 5 (High) (39.28) beat GPT-5.4 Pro (xHigh) by 15.46
WDCD R2 In-Document Resistance: Grok 4 (100.0) beat DeepSeek V4 Pro by 12.0
VANTAGE-Bench - Single Object Tracking: Gemini 3.6 Flash (75.99) beat Gemini 3.1 Pro (Preview) by 11.64
Chatbot Arena (Document): Claude Opus 5 (High) (1520.0) beat Claude Opus 4.6 by 10.0
VANTAGE-Bench - Event Verification: Gemini 3.6 Flash (82.0) beat Cosmos-Reason2-32B by 8.42
VANTAGE-Bench - Semantic: Gemini 3.6 Flash (79.41) beat Cosmos-Reason2-32B by 7.47
OSWorld: Intelligence-Indeed Agent (90.19) beat Pointer Agent w/ Opus 4.7 by 6.55
VANTAGE-Bench: Gemini 3.6 Flash (69.52) beat Cosmos3-Super by 6.51
MWS Vision Bench: Claude Fable 5 (79.9) beat Cotype Pro 3 by 6.3
VANTAGE-Bench - Video QA: GPT-5.6 Sol (77.9) beat Gemini 3.1 Pro (Preview) by 6.02
GSMA Open-Telco - 3GPP: OTel-2.0-LLM-31B-IT (87.34) beat OTel-LLM-8.3B-QnA by 5.94
VANTAGE-Bench - Temporal: GPT-5.6 Sol (46.25) beat Gemini 3.1 Pro (Preview) by 5.51
KernelBench Hub - Mega: Claude Opus 5 (24.29) beat Claude Fable 5 by 5.17
ProfBench: Claude Opus 5 (Thinking) (66.2) beat Claude Opus 4.7 (Thinking) by 4.9
GSMA Open-Telco LLM Leaderboard: OTel-2.0-LLM-31B-IT (90.27) beat OTel-LLM-8.3B-QnA by 4.29
OckBench: opus-5 (xHigh) (95.0) beat opus-4.8 (xHigh) by 3.5
Gert Labs Rankings: Claude Opus 5 (75.81) beat Claude Fable 5 by 3.03
ProgramBench Almost: Claude Opus 4.8 (xHigh) (16.5) beat GPT-5.5 (xHigh) by 3.0
GSMA Open-Telco - TeleMath: OTel-2.0-LLM-31B-IT (89.8) beat OTel-LLM-8.3B-QnA by 2.4
PutnamBench: Humanfia (w/ GPT 5.5) (670.0) beat Aleph Prover (Logical Intelligence) by 2.0
TrackingAI IQ Test (Vision): GPT-Luna-Max (96.08) beat GPT-5.6 Terra (Ultra) by 1.96
PTEB: F2LLM-v2-4B (77.49) beat embeddinggemma-300m by 1.94
AI for Education Pedagogy - Technology: Claude Opus 5 (91.51) beat Kimi K2.5 by 1.89
GSMA Open-Telco - srsRAN-Bench: OTel-2.0-LLM-31B-IT (91.54) beat OTel-LLM-8.3B-QnA by 1.86
GSMA Open-Telco - TeleLogs: OTel-2.0-LLM-31B-IT (98.15) beat OTel-LLM-8.3B-QnA by 1.85
SpreadsheetBench: Qingqiu Agent (98.25) beat Data Analysis Agent by 1.75
Agent Arena - Praise vs Complaint: Claude Opus 5 (Max) (25.63) beat Claude Fable 5 (High) by 1.69
Vals AI Time Horizon Index: GPT-5.6 Sol (13.0) beat Claude Opus 4.8 by 1.17
Agent Arena: Gemma 4 31B (16.1) beat Grok 4.3 by 1.06
BridgeBench Debugging: Claude Opus 4.8 (1005.1) beat Gemini 3.1 Pro (Preview) by 1.0
Vector Eval - MATH: Qwen3 Coder Next (97.82) beat O3 Mini by 0.91
HiL-Bench: Claude Opus 5 (57.0) beat Claude Fable 5 by 0.67
Tau3 Banking: Kimi K3 (Low) (34.02) beat Kimi K3 by 0.62
GSMA Open-Telco - TeleQnA: OTel-2.0-LLM-31B-IT (91.7) beat OTel-LLM-8.3B-QnA by 0.5
AI for Education SEND: Claude Opus 5 (88.99) beat Gemini 3.6 Flash by 0.46
Agent Arena - Confirmed Success: Claude Opus 5 (Max) (17.56) beat MiniMax-M2.7 by 0.43
Vector Eval - InterCode-CTF: Qwen3 Coder Next (85.9) beat Claude 3.5 Sonnet by 0.33
Vector Eval - GPQA-D: Qwen3 Coder Next (75.63) beat O1 by 0.12
GBA Eval: Claude Opus 5 (0.796) beat Claude Fable 5 by 0.05

The Aggregate Digest — 2026-08-01

2026-08-01T06:28:22.655980+00:00

Daily

New Benchmarks (2)

LM Market Cap LMC Score (LMC Score (0-100)): leader Claude Fable 5 (97.1), 342 models
LLM Stats Score (LLM Stats Score (conservative rating)): leader GPT-5.6 Sol (58.05), 320 models

New Models (1)

Celeris-1 — ELO 1518
- AA GDPval: 533.56 (#135/179)
- GDPval-AA: 534.0 (#139/184)
- Tau3 Banking: 4.95 (#151/183)
- BenchLM: 42.0 (#169/200)
- AA Humanity's Last Exam: 6.63 (#287/551)
- AA Long Context Reasoning: 27.0 (#301/483)
- AA GPQA Diamond: 63.13 (#326/555)
- AA Omniscience - Software Engineering (SWE) - PHP: 16.0 (#332/460)
- Artificial Analysis Intelligence Index: 11.81 (#337/554)
- AA Omniscience - Software Engineering (SWE) - Dart: 14.0 (#340/460)

Top-10 New Scores (2)

GPT-5.6 Sol on Design Arena (SVG): 1341.0 (#3)
Kimi K3 on Senior SWE-Bench: 20.2 (#10)

The Aggregate Digest — 2026-07-31

2026-07-31T06:33:57.582950+00:00

Daily

New Benchmarks (4)

HumanCLAW-Bench - Find (Success Rate (%)): leader Gemini-3.1 (64.9), 9 models
The Find stage of HumanCLAW-Bench — a frozen off-the-shelf VLM drives a simulated humanoid over 1,218 episodes and must get the target object rendered in its own ego view and acknowledge it; FindSR success rate (%).
HumanCLAW-Bench - Navigate (Success Rate (%)): leader Gemini-3.1 (42.5), 9 models
The Navigate stage of HumanCLAW-Bench — having found the target, the VLM-driven humanoid must walk to it and come to a stop within 20 cm; NavSR success rate (%), gated on the Find stage.
HumanCLAW-Bench - Interact (Success Rate (%)): leader Gemini-3.1 (16.8), 9 models
The Interact stage of HumanCLAW-Bench — the full three-stage embodied task: find the target, reach it, and sit with pelvis contact. Gated on the earlier stages, so this is the end-to-end episode success rate (%); no current VLM clears 20%.
MathArena - ARXIVLEAN June (Accuracy (%)): leader GPT-5.6 Sol (37.5), 8 models

Top-10 New Scores (9)

Claude Opus 5 on Roboflow Playground Object Detection: 1200.0 (#18)
Claude Opus 5 on Roboflow Playground Overall: 1200.0 (#33)
Kimi K3 on Epoch AI - Apex Agents: 39.3 (#7)
Kimi K3 on Roboflow Playground Object Detection: 1200.0 (#19)
Kimi K3 on Roboflow Playground Overall: 1200.0 (#34)
Kimi K3 on Vals AI ProgramBench: 62.77 (#12)
Kimi K3 on Vals AI Time Horizon Index: 5.17 (#5)
Kimi K3 on Vending-Bench 2: 5165.04 (#19)
Kimi K3 on Vending-Bench 2: 5165.04 (#19)

New #1 Leaders (5)

MathArena - ARXIV_FALSE June: Claude Opus 5 (Max) (90.74) beat GPT-5.5 (xHigh) by 21.3
ProfBench: Claude Opus 5 (Thinking) (66.2) beat Claude Opus 4.7 (Thinking) by 4.9
OckBench: opus-5 (xHigh) (95.0) beat opus-4.8 (xHigh) by 3.5
Vals AI Time Horizon Index: GPT-5.6 Sol (13.0) beat Claude Opus 4.8 by 1.17
Tau3 Banking: Kimi K3 (Low) (34.02) beat Kimi K3 by 0.62

The Aggregate Digest — 2026-07-30

2026-07-30T06:07:50.538944+00:00

Daily

New Benchmarks (7)

Epoch AI - Mystery Game Puzzles (Score): leader Claude Opus 5 (Max) (59.0), 20 models
Roboflow Playground Overall (Arena Elo): leader SAM 3 (1409.0), 62 models
Roboflow Playground is a head-to-head vision arena: users submit an image and a task, two models answer anonymously, and the winner is voted on. Overall pools every task type into one Elo rating.
Roboflow Playground Object Detection (Arena Elo): leader Gemini 3 Flash (1457.0), 38 models
Roboflow Playground arena Elo restricted to object-detection duels — locating and boxing the objects a prompt asks for.
Roboflow Playground Classification (Arena Elo): leader Gemini 3 Flash (1224.0), 22 models
Roboflow Playground arena Elo restricted to classification duels — assigning an image to the right category.
Roboflow Playground OCR (Arena Elo): leader Gemini 2.5 Flash (1260.0), 51 models
Roboflow Playground arena Elo restricted to OCR duels — reading text out of photographs, screenshots and documents.
Roboflow Playground Captioning (Arena Elo): leader Gemini 2.5 Pro (1278.0), 46 models
Roboflow Playground arena Elo restricted to captioning duels — describing what an image actually contains.
Roboflow Vision Evals - Visual Understanding (Pass Rate (%)): leader Gemini 3.5 Flash (79.1), 77 models
Roboflow Vision Evals benchmark for visual QA tasks such as reading text from photos, counting objects, spotting defects, and understanding documents.

Top-10 New Scores (7)

Claude Opus 5 on Blueprint-Bench 2: 0.304 (#7)
GPT-5.6 Sol on Epoch AI - Apex Agents: 37.7 (#9)
Kimi K3 on Blueprint-Bench 2: 0.295 (#8)
Kimi K3 on Epoch AI - Blueprint Bench 2: 29.5 (#8)
Kimi K3 on Epoch AI - ECI: 155.59 (#59)
Kimi K3 on Epoch AI - Proofbench: 70.0 (#5)
Kimi K3 on Epoch AI - Scicode: 58.68 (#3)

New #1 Leaders (4)

Epoch AI - Vending Bench 2: Claude Opus 5 (Unknown) (11181.87) beat Claude Opus 4.7 (Unknown) by 245.11
Vending-Bench 2: Claude Opus 5 (11181.87) beat Claude Opus 4.7 by 245.11
WebDev Arena: Claude Opus 5 (Max) (1711.88) beat Claude Fable 5 (Unknown) by 57.95
Epoch AI - Enigma Eval: Claude Fable 5 (High) (39.28) beat GPT-5.4 Pro (xHigh) by 15.46

The Aggregate Digest — 2026-07-29

2026-07-29T06:21:38.233204+00:00

Daily

New Benchmarks (13)

AgentBattler (Pooled Score (%)): leader GPT-5.6 Luna (56.21), 3 models
AgentBattler pits complete agents rather than bare models: each entrant is one harness and model combination playing open, inspectable terminal and chess challenges, pooling five independently generated engines per row. Scoring uses same-model cross-harness games so model identity stays fixed; we keep the best harness result per underlying model.
PerceptionBench (Accuracy (%)): leader GPT-5.6 Sol (59.7), 16 models
Moonshot AI benchmark for atomic visual perception in multimodal models: 3,000 verified open-ended questions with short, uniquely determined answers across ten perceptual capabilities, graded by an LLM judge against the reference. Sixteen frontier models are run with unified prompts at their highest reasoning budget, and none reaches 60%.
PerceptionBench - Visual Relation (Visual Relation (%)): leader GPT-5.6 Sol (69.7), 16 models
PerceptionBench slice scoring whether a model reads the spatial and semantic relations between objects in an image rather than the objects alone.
PerceptionBench - Counting (Counting (%)): leader GPT-5.6 Sol (62.4), 16 models
PerceptionBench slice scoring exact enumeration of objects in a scene.
PerceptionBench - Attribute (Attribute (%)): leader GPT-5.6 Sol (62.1), 16 models
PerceptionBench slice scoring recognition of object attributes such as color, material, shape and state.
PerceptionBench - Depth & 3D (Depth & 3D Perception (%)): leader GPT-5.6 Sol (55.5), 16 models
PerceptionBench slice scoring depth ordering and three-dimensional spatial perception from a single image.
PerceptionBench - Localization (Localization (%)): leader GPT-5.6 Sol (76.7), 16 models
PerceptionBench slice scoring where in the image a referenced object actually is.
PerceptionBench - Comparison (Comparison (%)): leader GPT-5.6 Sol (67.0), 16 models
PerceptionBench slice scoring relative judgements between two or more elements of a scene (larger, closer, brighter, more numerous).
PerceptionBench - Fine-Grained Recognition (Fine-Grained Recognition (%)): leader Seed 2.1 Pro (56.6), 16 models
PerceptionBench slice scoring discrimination between visually similar categories that differ only in fine detail.
PerceptionBench - Contextual Integration (Contextual Integration (%)): leader Gemini 3.1 Pro (Preview) (61.2), 16 models
PerceptionBench slice scoring answers that require combining several parts of the scene rather than reading one region.
PerceptionBench - OCR (OCR (%)): leader Seed 2.1 Pro (66.7), 16 models
PerceptionBench slice scoring text read directly out of the image.
PerceptionBench - Hallucination (Perception Hallucination (%)): leader Gemini 3.5 Flash (50.6), 16 models
PerceptionBench slice scoring perception-related hallucination — the share of questions answered without asserting content that is not in the image. The hardest slice on the board: no model clears 51%.
LLM-SoccerArena (Prediction Points): leader GPT-5.5 (168.0), 7 models
Real-world football forecasting: models predict scorelines ahead of kickoff and are scored 5 points for the exact score, 2 for the goal difference, 1 for the outcome and 0 otherwise. Each row is a model at its best prompt and lead-time setup across the tournament.

Top-10 New Scores (14)

Claude Opus 5 on AI for Education Pedagogy: 91.1 (#4)
Claude Opus 5 on AI for Education Pedagogy - Maths: 95.24 (#2)
Claude Opus 5 on AI for Education Pedagogy - Primary: 95.31 (#4)
Claude Opus 5 on AI for Education Pedagogy - Science: 92.35 (#19)
Claude Opus 5 on AI for Education Pedagogy - Secondary: 89.94 (#5)
Claude Opus 5 on AI for Education Pedagogy - Social studies: 82.73 (#70)
Claude Opus 5 on Kaggle FACTS Parametric: 48.77 (#12)
Claude Opus 5 on KernelBench Hub - CUDA: 196.1 (#1)
Claude Opus 5 on KernelBench Hub - Hard: 52.61 (#2)
Claude Opus 5 on KernelBench Hub - Mega: 24.29 (#1)
Claude Opus 5 on SpeechMap Compliance: 59.1 (#202)
Claude Opus 5 on TrackingAI IQ Test: 82.35 (#14)
Claude Opus 5 on TrackingAI IQ Test (Mensa Norway): 77.14 (#17)
Claude Opus 5 on TrackingAI IQ Test (Offline): 81.25 (#7)

New #1 Leaders (5)

AI for Education Pedagogy - Technology: Claude Opus 5 (91.51) beat Kimi K2.5 by 1.89
Agent Arena - Confirmed Success: Claude Opus 5 (Max) (17.56) beat Gemini 3.5 Flash Lite by 1.75
Agent Arena - Praise vs Complaint: Claude Opus 5 (Max) (25.63) beat Claude Fable 5 (High) by 1.15
BridgeBench Debugging: Claude Opus 4.8 (1005.1) beat Gemini 3.1 Pro (Preview) by 1.0
AI for Education SEND: Claude Opus 5 (88.99) beat Gemini 3.6 Flash by 0.46

The Aggregate Digest — 2026-07-28

2026-07-28T06:45:31.729387+00:00

Daily

New Benchmarks (1)

SpreadsheetBench v2 (Score (%)): leader Claude Opus 4.6 (34.89), 8 models
SpreadsheetBench V2 evaluates model capability on spreadsheets tasks from the linked upstream source with Score as the primary reported metric.

Top-10 New Scores (12)

Claude Opus 5 on Kaggle Game Arena Poker (Heads Up): 20.34 (#3)
Claude Opus 5 on KernelBench Hub - Hard: 52.61 (#2)
Claude Opus 5 on LisanBench: 0.1144 (#38)
Claude Opus 5 on TaxCalcBench: 18.0 (#8)
GPT-5.6 Sol on Kaggle FACTS (Google): 52.41 (#10)
GPT-5.6 Sol on VANTAGE-Bench: 69.03 (#2)
GPT-5.6 Sol on VANTAGE-Bench - Event Verification: 75.9 (#2)
GPT-5.6 Sol on VANTAGE-Bench - Semantic: 76.9 (#2)
GPT-5.6 Sol on VANTAGE-Bench - Single Object Tracking: 75.02 (#2)
GPT-5.6 Sol on VANTAGE-Bench - Spatial: 77.93 (#4)
Kimi K3 on Mercor APEX: 39.3 (#8)
Kimi K3 on TrackingAI IQ Test (Offline): 81.25 (#6)

New #1 Leaders (12)

FoodTruckBench: Claude Opus 5 (75264.0) beat GPT-5.5 by 13856.0
Chatbot Arena (Code): Claude Opus 5 (Max) (1725.0) beat Kimi K3 by 43.0
KernelBench Hub - CUDA: Claude Opus 5 (103.67) beat Claude Fable 5 by 31.01
VANTAGE-Bench - Single Object Tracking: Gemini 3.6 Flash (75.99) beat Gemini 3.1 Pro (Preview) by 11.64
VANTAGE-Bench - Event Verification: Gemini 3.6 Flash (82.0) beat Cosmos-Reason2-32B by 8.42
VANTAGE-Bench - Semantic: Gemini 3.6 Flash (79.41) beat Cosmos-Reason2-32B by 7.47
OSWorld: Intelligence-Indeed Agent (90.19) beat Pointer Agent w/ Opus 4.7 by 6.55
VANTAGE-Bench: Gemini 3.6 Flash (69.52) beat Cosmos3-Super by 6.51
VANTAGE-Bench - Video QA: GPT-5.6 Sol (77.9) beat Gemini 3.1 Pro (Preview) by 6.02
VANTAGE-Bench - Temporal: GPT-5.6 Sol (46.25) beat Gemini 3.1 Pro (Preview) by 5.51
KernelBench Hub - Mega: Claude Opus 5 (24.29) beat Claude Fable 5 by 5.17
HiL-Bench: Claude Opus 5 (57.0) beat Claude Fable 5 by 0.67

The Aggregate Digest — 2026-07-27

2026-07-27T08:01:48.379435+00:00

Daily

New Benchmarks (8)

CodeRouterBench (Mean Task Score (%)): leader Claude Opus 4.6 (43.83), 8 models
Model-routing study dataset (Lance1573): mean per-task score % over the held-out 2,919-task ID test split of code tasks (MBPP-style generation, bug fixing and more) for 8 frontier models.
CodeRouterBench (OOD) (Resolve Rate (%)): leader GPT-5.4 (64.2), 8 models
CodeRouterBench out-of-distribution split: resolve rate % over 176 unseen repository tasks (featurebench-style bug fixing / feature work) for the same 8 models.
GDPevo (base accuracy (%, no-evolution mode)): leader Claude Opus 4.6 (50.4), 6 models
Prism-Shadow benchmark of agent self-evolution on 240 real business tasks across 24 groups; base accuracy % (no-evolution mode, mean of 3 runs), each model in its native harness (Codex / Claude Code / Panofy).
Biomni-Bench (Humanlaya) (Score (0-100)): leader Claude Opus 4.7 (73.34), 6 models
Humanlaya evaluation of frontier models on Biomni biomedical-agent data-analysis tasks, each model in its native agent harness (Claude Code / Codex / Gemini CLI); mean score 0-100 over 3 runs.
Excel-Bench (Humanlaya) (Average Score (%)): leader GPT-5.4 (72.92), 6 models
Humanlaya spreadsheet-manipulation benchmark: models solve real Excel tasks; average score %, with OpenAI effort tiers reported as separate rows.
OneMillion-Bench (Pass Rate (%)): leader Claude Opus 4.6 (43.5), 36 models
Humanlaya web-scale task benchmark: pass rate % over tasks with an estimated economic value per model; Base and Search-augmented rows are listed separately.
WorldTravel-Bench (Text) (Feasibility Rate (%)): leader GPT-5.2 (32.67), 10 models
Humanlaya real-world travel-planning feasibility benchmark, text setting: share of generated itineraries that are actually feasible; human performance reference 77.9%.
WorldTravel-Bench (Multimodal) (Feasibility Rate (%)): leader GPT-5.2 (19.33), 7 models
WorldTravel-Bench in the multimodal setting (plans grounded in visual materials); feasibility rate %, human performance reference 77.9%.

Top-10 New Scores (7)

Claude Fable 5 on ProphetArena: 0.9424 (#7)
Claude Opus 5 on Design Arena (3D): 1357.0 (#5)
Claude Opus 5 on MineBench: 2042.52 (#2)
GPT-5.6 Sol on Lean AI Formalization Leaderboard: 1.0 (#41)
GPT-5.6 Sol on MWS Vision Bench: 79.0 (#2)
Kimi K3 on MWS Vision Bench: 77.7 (#4)
Kimi K3 on SWE-Milestone: 46.75 (#2)

New #1 Leaders (16)

GSMA Open-Telco - TeleTables: OTel-2.0-LLM-31B-IT (79.76) beat OTel-LLM-8.3B-QnA by 17.96
Chatbot Arena (Document): Claude Opus 5 (High) (1520.0) beat Claude Opus 4.6 by 10.0
MWS Vision Bench: Claude Fable 5 (79.9) beat Cotype Pro 3 by 6.3
GSMA Open-Telco - 3GPP: OTel-2.0-LLM-31B-IT (87.34) beat OTel-LLM-8.3B-QnA by 5.94
GSMA Open-Telco LLM Leaderboard: OTel-2.0-LLM-31B-IT (90.27) beat OTel-LLM-8.3B-QnA by 4.29
ProgramBench Almost: Claude Opus 4.8 (xHigh) (16.5) beat GPT-5.5 (xHigh) by 3.0
Gert Labs Rankings: Claude Opus 5 (75.77) beat Claude Fable 5 by 2.99
GSMA Open-Telco - TeleMath: OTel-2.0-LLM-31B-IT (89.8) beat OTel-LLM-8.3B-QnA by 2.4
PutnamBench: Humanfia (w/ GPT 5.5) (670.0) beat Aleph Prover (Logical Intelligence) by 2.0
GSMA Open-Telco - srsRAN-Bench: OTel-2.0-LLM-31B-IT (91.54) beat OTel-LLM-8.3B-QnA by 1.86
GSMA Open-Telco - TeleLogs: OTel-2.0-LLM-31B-IT (98.15) beat OTel-LLM-8.3B-QnA by 1.85
Vector Eval - MATH: Qwen3 Coder Next (97.82) beat O3 Mini by 0.91
GSMA Open-Telco - TeleQnA: OTel-2.0-LLM-31B-IT (91.7) beat OTel-LLM-8.3B-QnA by 0.5
Vector Eval - InterCode-CTF: Qwen3 Coder Next (85.9) beat Claude 3.5 Sonnet by 0.33
Vector Eval - GPQA-D: Qwen3 Coder Next (75.63) beat O1 by 0.12
GBA Eval: Claude Opus 5 (0.796) beat Claude Fable 5 by 0.05

The Aggregate Digest — 2026-07-26

2026-07-26T06:48:19.646411+00:00

Daily

New Benchmarks (1)

SpatialGen-Bench (Text Answering) (Overall Accuracy (%)): leader Claude Fable 5 (81.26), 25 models
Spatial cognition benchmark from the ProVisE project (ZJU-OmniAI, "Show, Don\

Top-10 New Scores (6)

Claude Opus 5 on Benchmarks.bio - BioSecBench-Surveillance: 59.8 (#1)
Claude Opus 5 on Benchmarks.bio - EpiBench: 35.22 (#4)
Claude Opus 5 on Benchmarks.bio - VariantBench: 49.72 (#1)
Claude Opus 5 on Design Arena (UI Components): 1387.0 (#2)
Claude Opus 5 on Design Arena (Website): 1331.0 (#4)
GPT-5.6 Sol on FoodTruckBench: 53229.0 (#2)

New #1 Leaders (3)

Multi-turn Debate (Lechmazur): Claude Opus 5 (High) (1763.4) beat Claude Fable 5 (High) by 6.2
NYT Connections Older Models: ByteDance Seed2.1 Pro (97.8) beat GLM-5.2 (Thinking, Max) by 5.1
EQ-Bench Longform Writing: Claude Opus 5 (86.3) beat Claude Fable 5 by 3.3

Weekly

New Benchmarks (1)

SpatialGen-Bench (Text Answering) (Overall Accuracy (%)): leader Claude Fable 5 (81.26), 25 models
Spatial cognition benchmark from the ProVisE project (ZJU-OmniAI, "Show, Don\

New Models (138)

GPT-5.5 Pro — ELO 3263, #1
- BenchLM: 62.8 (#41/200)
Claude Fable 5 — ELO 3073, #2
- HiL-Bench: 56.33 (#1/14)
- Vals AI Public Benefits Bench: 70.43 (#2/23)
- CHI-Bench: 24.0 (#6/23)
Claude Opus 5 — ELO 2963, #3
- DeepSWE: 73.6 (#1/22)
- Benchmarks.bio - scBench-Long: 41.27 (#1/19)
- Benchmarks.bio - BioSecBench-Surveillance: 59.8 (#1/16)
- Benchmarks.bio - VariantBench: 49.72 (#1/13)
- Vellum - Humanity's Last Exam: 64.7 (#1/31)
- EQ-Bench Longform Writing: 86.3 (#1/124)
- LLM Stats (GDPval-AA): 1861.0 (#1/42)
- Vals AI CorpFin v2: 73.19 (#1/126)
- Vals AI Finance Agent v2: 58.63 (#1/40)
- Vals AI MortgageTax: 72.06 (#1/90)
GPT-5.4 Pro (xHigh) — ELO 2838, #4
- BenchLM: 60.0 (#47/200)
GPT-5.6 Sol — ELO 2723, #5
- Vellum - HumanEval: 96.2 (#1/47)
- Vellum - ARC-AGI-2: 92.5 (#1/15)
- YapBench: 18.5 (#1/108)
- Epoch AI - Vending Bench 2: 9619.37 (#2/54)
- MyPCBench: 55.4 (#2/8)
- FoodTruckBench: 53229.0 (#2/10)
- Icelandic LLM - ARC-Challenge-IS: 95.22 (#2/92)
- Vellum - GPQA: 94.6 (#3/68)
- CHI-Bench: 25.3 (#4/23)
- BenchLM: 81.5 (#4/200)
Claude Opus 4.8 — ELO 2669, #6
- Mercor APEX: 42.5 (#3/50)
Kimi K3 — ELO 2663, #7
- Design Arena (Website): 1375.0 (#1/159)
- Design Arena (UI Components): 1408.0 (#1/139)
- OpenClawProBench: 81.6 (#1/68)
- Agent Arena - Confirmed Success: 14.0 (#3/38)
- Agent Arena - Praise vs Complaint: 20.3 (#3/38)
- Agents' Last Exam: 28.3 (#3/21)
- Creative Writing (Lechmazur): 2.9 (#3/39)
- Multi-turn Debate (Lechmazur): 1734.3 (#3/42)
- Vellum - Humanity's Last Exam: 56.0 (#5/31)
- BenchLM: 80.0 (#5/200)
GPT-5.5 — ELO 2575, #8
- Mercor APEX: 38.5 (#8/50)
- OckBench: 26.0 (#45/49)
- ForecastBench: 64.6 (#65/236)
Claude Opus 4.6 — ELO 2510, #10
- OckBench: 84.5 (#8/49)
- Mercor APEX: 32.4 (#15/50)
- BenchLM: 67.8 (#19/200)
Gemini 3.1 Pro (Preview) — ELO 2495, #11
- BridgeBench Debugging: 1004.1 (#1/2)
- ProphetArena: 0.9545 (#5/50)
- Mercor APEX: 33.5 (#13/50)
- Agents' Last Exam: 15.8 (#13/21)
- OckBench: 76.0 (#19/49)
- GAIA: 55.48 (#556/3460)
Muse Spark 1.1 — ELO 2473, #12
- Gert Labs Rankings: 70.27 (#3/90)
- BenchLM: 76.6 (#7/200)
- Vals AI SkillsBench: 59.19 (#8/20)
- GBA Eval: 0.0792 (#12/22)
- GDP.pdf: 15.0 (#14/24)
- Epoch AI - Critpt: 15.1 (#19/73)
- AI Chess Leaderboard (Reasoning): 1140.0 (#44/303)
- AI Chess Leaderboard (Continuation): 727.0 (#83/248)
Claude Opus 4.7 — ELO 2463, #13
- Mercor APEX: 33.9 (#12/50)
- BenchLM: 71.6 (#14/200)
Qwen 3.7 Max — ELO 2438, #14
- YapBench: 124.7 (#32/108)
Gemini 3.5 Flash — ELO 2435, #15
- ProphetArena: 0.9509 (#12/50)
- TaxCalcBench: 4.0 (#12/13)
- ForecastBench: 64.4 (#73/236)
Grok 4.5 — ELO 2432, #16
- Vals AI SkillsBench: 66.03 (#1/20)
- TrackingAI IQ Test: 98.04 (#1/35)
- TrackingAI IQ Test (Mensa Norway): 100.0 (#1/35)
- FrontierSWE: 78.0 (#2/15)
- GBA Eval: 0.6541 (#3/22)
- FoodTruckBench: 34586.0 (#4/10)
- Kaggle FACTS Grounding: 74.52 (#5/45)
- TrackingAI IQ Test (Vision): 78.43 (#6/23)
- BenchLM: 75.5 (#8/200)
- Lean AI Formalization Leaderboard: 3.0 (#15/47)
Claude Sonnet 5 — ELO 2414, #17
- GBA Eval: 0.6526 (#4/22)
- Mercor APEX: 32.5 (#14/50)
- Vals AI SkillsBench: 46.48 (#17/20)
- BenchLM: 64.7 (#32/200)
- YapBench: 507.7 (#96/108)
GPT-5.6 Terra — ELO 2407, #18
- Agents' Last Exam: 28.0 (#4/21)
- Vals AI SkillsBench: 60.65 (#4/20)
- Epoch AI - Vending Bench 2: 7343.21 (#6/54)
- RuneBench: 5934.0 (#6/41)
- GDP.pdf: 24.7 (#7/24)
- Icelandic LLM - ARC-Challenge-IS: 94.8 (#9/92)
- Kaggle FACTS Grounding: 71.21 (#9/45)
- Kaggle FACTS Parametric: 55.04 (#10/33)
- Vellum - GPQA: 92.9 (#12/68)
- BenchLM: 72.0 (#12/200)
GPT-5.3 Codex — ELO 2389, #20
- Mercor APEX: 31.8 (#16/50)
Gemini 3 Pro — ELO 2384, #21
- Mercor APEX: 31.5 (#17/50)
GPT-5 — ELO 2379, #22
- Mercor APEX: 18.3 (#27/50)
Gemini 3.6 Flash — ELO 2379, #23
- Kaggle FACTS Multimodal: 47.21 (#1/32)
- AI for Education Pedagogy - Maths: 95.24 (#1/226)
- AI for Education SEND: 88.53 (#1/218)
- RuneBench: 7454.0 (#3/41)
- LLM Stats (OSWorld-Verified): 83.0 (#3/21)
- Vals AI GPQA: 93.43 (#3/126)
- Design Arena (Data Viz): 1343.0 (#4/142)
- AI Chess Leaderboard (Reasoning): 1800.0 (#4/303)
- AI Chess Leaderboard (Continuation): 1761.0 (#4/248)
- Kaggle FACTS Parametric: 73.35 (#4/33)
GPT-5.4 — ELO 2353, #24
- Mercor APEX: 36.0 (#9/50)
- OckBench: 23.5 (#47/49)
GPT-5.6 Luna — ELO 2350, #25
- Agents' Last Exam: 29.6 (#2/21)
- MyPCBench: 55.4 (#3/8)
- Vellum - HumanEval: 93.0 (#4/47)
- YapBench: 27.8 (#4/108)
- Vals AI SkillsBench: 60.45 (#5/20)
- GDP.pdf: 22.7 (#9/24)
- Icelandic LLM - WikiQA-IS: 46.87 (#13/92)
- Vellum - GPQA: 92.3 (#14/68)
- Icelandic LLM - GED: 74.5 (#14/92)
- Icelandic LLM - Inflection: 94.54 (#15/92)
GLM-5.2 — ELO 2340, #26
- HiL-Bench: 43.67 (#2/14)
- SEAL - MCP Atlas: 77.8 (#10/28)
- Mercor APEX: 35.6 (#10/50)
- YapBench: 76.5 (#16/108)
- AI for Education SEND: 81.19 (#38/218)
- AI for Education Pedagogy - Science: 90.16 (#44/226)
- AI for Education Pedagogy - Secondary: 85.85 (#53/226)
- AI for Education Pedagogy: 85.87 (#54/226)
- AI for Education Pedagogy - Maths: 85.71 (#56/226)
- AI for Education Pedagogy - Social studies: 83.64 (#63/226)
GPT-5.2 Pro — ELO 2337, #27
- BenchLM: 66.4 (#25/200)
GPT-5.2 Codex — ELO 2294, #30
- Mercor APEX: 27.6 (#18/50)
Inkling — ELO 2279, #31
- Agent Arena - Praise vs Complaint: 19.01 (#5/38)
- Agent Arena - Steerability: 11.6 (#5/38)
- Agent Arena - Tool Hallucination: 0.4 (#5/38)
- Agent Arena - Confirmed Success: 7.19 (#14/38)
- Riemann-bench: 15.2 (#15/24)
- AI for Education Pedagogy - Technology: 86.79 (#19/226)
- Agent Arena: 6.41 (#20/38)
- AI for Education Pedagogy - Social studies: 87.27 (#20/226)
- AI for Education SEND: 83.03 (#21/218)
- BenchLM: 66.9 (#22/200)
Qwen 3.7 Plus — ELO 2269, #32
- YapBench: 85.7 (#20/108)
Gemini 3 Flash (Preview) — ELO 2265, #33
- OckBench: 44.5 (#35/49)
Gemini 3 Flash — ELO 2258, #35
- DeepSWE: 48.6 (#17/22)
- Mercor APEX: 24.0 (#20/50)
GPT-5.2 — ELO 2252, #36
- Mercor APEX: 41.8 (#5/50)
Claude Opus 4.5 — ELO 2244, #38
- Mercor APEX: 20.7 (#23/50)
MiniMax-M3 — ELO 2238, #40
- Senior SWE-Bench: 13.8 (#9/10)
- Multi-turn Debate (Lechmazur): 1531.1 (#16/42)
- YapBench: 360.0 (#80/108)
Hy3 — ELO 2231, #42
- Agent Arena - Tool Hallucination: 0.89 (#11/38)
- Agent Arena - Steerability: 7.1 (#18/38)
- Chatbot Arena (Code): 1517.0 (#19/101)
- Agent Arena - Confirmed Success: 4.65 (#24/38)
- YapBench: 116.7 (#29/108)
- Agent Arena: 2.23 (#30/38)
- Agent Arena - Praise vs Complaint: 2.87 (#34/38)
- Agent Arena - Bash Recovery: 2.56 (#36/38)
- BenchLM: 54.6 (#91/200)
- Wolfram LLM Benchmarking Project: 54.8 (#103/520)
Qwen 3.6 Max Preview — ELO 2228, #43
- BenchLM: 59.3 (#59/200)
GPT-5 Codex — ELO 2211, #44
- Mercor APEX: 20.1 (#25/50)
MiMo-V2.5-Pro — ELO 2205, #45
- BenchLM: 69.4 (#17/200)
DeepSeek V4 Pro — ELO 2202, #47
- YapBench: 151.3 (#35/108)
O3 — ELO 2188, #49
- Mercor APEX: 17.2 (#29/50)
Kimi K2.7 Code — ELO 2175, #50
- BridgeBench Debugging: 995.9 (#2/2)
- YapBench: 72.0 (#14/108)
- BenchLM: 54.0 (#96/200)
GPT-5.1 Codex — ELO 2171, #53
- Mercor APEX: 20.7 (#24/50)
- BenchLM: 51.7 (#109/200)
Claude Sonnet 4.6 — ELO 2164, #54
- ProphetArena: 0.9407 (#17/50)
- Mercor APEX: 23.7 (#21/50)
- OckBench: 58.5 (#29/49)
DeepSeek V4 Flash — ELO 2158, #55
- RuneBench: 3492.0 (#9/41)
- YapBench: 202.2 (#43/108)
Kimi K2.6 — ELO 2146, #57
- HiL-Bench: 18.67 (#11/14)
- OckBench: 75.0 (#21/49)
- YapBench: 238.2 (#48/108)
GPT-5.1 — ELO 2135, #60
- Mercor APEX: 17.5 (#28/50)
Qwen 3.5 Plus — ELO 2129, #63
- BenchLM: 46.3 (#148/200)
Kimi K2.5 — ELO 2120, #67
- OckBench: 72.5 (#23/49)
O1 — ELO 2120, #68
- Mercor APEX: 1.1 (#50/50)
GLM-5.1 — ELO 2102, #72
- OckBench: 63.5 (#27/49)
Qwen 3 Max — ELO 2100, #73
- BenchLM: 47.2 (#139/200)
Grok 4.3 — ELO 2097, #74
- BenchLM: 64.2 (#35/200)
MiMo-V2-Pro — ELO 2097, #75
- BenchLM: 66.8 (#23/200)
O1 Preview — ELO 2092, #77
- BenchLM: 48.3 (#132/200)
GLM-5V Turbo — ELO 2092, #78
- BenchLM: 62.4 (#42/200)
Qwen 3.5 397B A17B — ELO 2089, #80
- OckBench: 67.5 (#24/49)
GLM-5-Turbo — ELO 2082, #83
- BenchLM: 66.0 (#28/200)
Gemini 3.5 Flash Lite — ELO 2076, #84
- AI Chess Leaderboard (Continuation): 1518.0 (#9/248)
- Vals AI CyberBench: 72.32 (#9/20)
- Kaggle FACTS Grounding: 71.2 (#10/45)
- LLM Stats (OSWorld-Verified): 74.0 (#11/21)
- Vals AI MortgageTax: 68.68 (#11/90)
- Kaggle FACTS Parametric: 43.36 (#13/33)
- LLM Stats (Terminal-Bench 2.1): 54.0 (#15/15)
- Vals AI IOI: 26.17 (#15/60)
- LLM Stats (MRCR v2 (8-needle)): 21.3 (#18/21)
- RuneBench: 2254.0 (#19/41)
GLM-5 — ELO 2069, #85
- OckBench: 59.5 (#28/49)
Gemini 3.1 Flash Lite — ELO 2058, #88
- Mercor APEX: 13.0 (#35/50)
- OckBench: 26.5 (#44/49)
- ForecastBench: 61.5 (#139/236)
Step 3.7 Flash — ELO 2047, #90
- BenchLM: 49.9 (#121/200)
GPT-5 Mini — ELO 2031, #91
- BenchLM: 43.0 (#165/200)
Step 3.5 Flash — ELO 2031, #92
- BenchLM: 54.2 (#94/200)
Gemma 4 31B (IT) — ELO 2020, #95
- OckBench: 41.5 (#39/49)
- Epoch AI - ECI: 141.86 (#238/427)
Grok 4.1 Fast — ELO 2019, #97
- MineBench: 822.37 (#48/53)
- BenchLM: 50.5 (#114/200)
LongCat 2.0 — ELO 2016, #98
- AI Chess Leaderboard (Reasoning): 773.0 (#103/303)
- AI Chess Leaderboard (Continuation): 573.0 (#124/248)
Grok 4.20 — ELO 2002, #103
- ProphetArena: 0.9464 (#14/50)
MiMo-V2.5 — ELO 2000, #104
- BenchLM: 57.9 (#72/200)
- YapBench: 395.7 (#85/108)
Hy3-preview — ELO 1989, #109
- BenchLM: 42.7 (#167/200)
Qwen 3.6 27B — ELO 1988, #110
- OckBench: 52.0 (#32/49)
GPT-5.4 Mini — ELO 1987, #111
- Mercor APEX: 24.6 (#19/50)
MiniMax-M2.7 — ELO 1981, #116
- OckBench: 66.88 (#25/49)
Qwen 3.5 122B A10B — ELO 1950, #127
- OckBench: 53.5 (#31/49)
DeepSeek V3.1 — ELO 1938, #130
- BenchLM: 52.8 (#104/200)
Mercury 2 — ELO 1933, #136
- Tau3 Banking: 9.9 (#94/179)
- BenchLM: 48.0 (#133/200)
GLM-4.6 — ELO 1929, #140
- BenchLM: 54.3 (#93/200)
MiMo-V2-Omni — ELO 1918, #148
- BenchLM: 62.2 (#43/200)
GLM-4.5 — ELO 1908, #155
- BenchLM: 56.8 (#78/200)
Command A+ — ELO 1897, #161
- BenchLM: 46.6 (#146/200)
MiniMax-M2.5 — ELO 1894, #164
- OckBench: 44.5 (#36/49)
- BenchLM: 58.6 (#66/200)
EXAONE 4.5 33B — ELO 1890, #167
- Tau3 Banking: 11.13 (#87/179)
- AA GDPval: 680.45 (#112/175)
- GDPval-AA: 680.0 (#116/180)
qwen3.5-flash — ELO 1869, #183
- BenchLM: 47.0 (#141/200)
nemotron-3-super-120B-a12B — ELO 1865, #186
- SEA-HELM: 47.94 (#30/44)
- BenchLM: 50.2 (#119/200)
Gemma 4 31B — ELO 1860, #189
- RuneBench: 633.0 (#34/41)
- BenchLM: 60.2 (#46/200)
Qwen 3.6 35B A3B — ELO 1854, #194
- OckBench: 47.5 (#33/49)
- Epoch AI - ECI: 143.31 (#209/427)
DeepSeek R1 — ELO 1844, #199
- BenchLM: 50.9 (#112/200)
GPT-5.4 Nano — ELO 1838, #201
- BenchLM: 66.0 (#27/200)
- Mercor APEX: 16.9 (#31/50)
Sarvam 105B — ELO 1829, #204
- BenchLM: 42.1 (#169/200)
GLM 4.5 Air — ELO 1820, #210
- BenchLM: 46.9 (#144/200)
Claude Haiku 4.5 — ELO 1819, #212
- ParseBench: 45.17 (#39/50)
- Mercor APEX: 8.9 (#39/50)
GPT-OSS-120B — ELO 1815, #215
- RuneBench: 463.0 (#37/41)
- Mercor APEX: 4.7 (#44/50)
Mistral Large 3 — ELO 1813, #216
- BenchLM: 49.6 (#122/200)
Grok Code Fast 1 — ELO 1810, #219
- BenchLM: 37.8 (#193/200)
Qwen 3.5 35B A3B — ELO 1800, #220
- OckBench: 46.0 (#34/49)
Llama 3.1 405B — ELO 1796, #223
- BenchLM: 50.9 (#111/200)
GPT-5 Nano — ELO 1763, #240
- BenchLM: 45.5 (#150/200)
Mistral Small 4 — ELO 1748, #256
- YapBench: 320.2 (#67/108)
- BenchLM: 45.3 (#151/200)
GLM-4.7 Flash — ELO 1725, #267
- BenchLM: 50.4 (#116/200)
trinity-large-preview — ELO 1721, #269
- BenchLM: 55.2 (#89/200)
Qwen 3.5 9B — ELO 1718, #271
- OckBench: 21.5 (#48/49)
Mistral Large 2 (Jul) — ELO 1705, #280
- BenchLM: 40.9 (#176/200)
Ministral 3 14B — ELO 1686, #300
- BenchLM: 33.7 (#200/200)
Mistral Medium 3 — ELO 1685, #302
- BenchLM: 42.3 (#168/200)
Gemini 1.5 Pro — ELO 1679, #304
- BenchLM: 35.0 (#198/200)
GPT-4o Mini — ELO 1668, #310
- BenchLM: 37.1 (#196/200)
Gemma 4 12B — ELO 1651, #325
- SEA-HELM: 65.23 (#9/44)
- BenchLM: 46.3 (#149/200)
Qwen 2.5 72B — ELO 1648, #328
- BenchLM: 51.4 (#110/200)
Gemma 4 26B A4B — ELO 1608, #357
- BenchLM: 57.1 (#77/200)
Ministral-3-3B-Reasoning-2512 — ELO 1602, #360
- BenchLM: 38.8 (#190/200)
Gemma 3 27B — ELO 1596, #363
- BenchLM: 40.9 (#177/200)
Llama 3 70B — ELO 1575, #382
- BenchLM: 50.0 (#120/200)
Qwen 2.5 Coder 32B Instruct — ELO 1572, #386
- BenchLM: 34.0 (#199/200)
DeepSeek R1 Distill Qwen 32B — ELO 1564, #394
- BenchLM: 41.7 (#173/200)
Sarvam 30B — ELO 1545, #401
- BenchLM: 39.9 (#182/200)
Ministral 3 3B — ELO 1541, #405
- AA MATH-500: 80.04 (#109/206)
Mixtral 8x22B Instruct (v0.1) — ELO 1538, #408
- BenchLM: 47.8 (#135/200)
Ling-2.6-flash — ELO 1518, #420
- BenchLM: 43.0 (#166/200)
Ministral-3-14B-Reasoning-2512 — ELO 1506, #431
- BenchLM: 48.6 (#131/200)
Exaone 4.0 1.2B — ELO 1502, #433
- BenchLM: 38.2 (#192/200)
Claude 3 Opus — ELO 1470, #453
- BenchLM: 40.4 (#180/200)
Granite 4.0 H 1B — ELO 1414, #490
- AA MATH-500: 52.16 (#172/206)
LFM2 2.6B — ELO 1373, #521
- AA MATH-500: 75.28 (#126/206)
Gemma 4 E4B — ELO 1343, #544
- SEA-HELM: 62.57 (#14/44)
- BenchLM: 42.1 (#170/200)
gemma-4-E2B-it — ELO 1327, #550
- VANTAGE-Bench: 21.56 (#14/14)
- VANTAGE-Bench - Spatial: 28.56 (#14/14)
- VANTAGE-Bench - Temporal: 9.16 (#14/14)
- VANTAGE-Bench - Semantic: 36.92 (#14/14)
- VANTAGE-Bench - Video QA: 46.19 (#14/14)
- VANTAGE-Bench - Event Verification: 27.65 (#14/14)
- VANTAGE-Bench - Single Object Tracking: 11.58 (#14/14)
Qwen 3.5 0.8B — ELO 1313, #561
- LIBRA - ruSciPassageCount *: 17.79 (#7/13)
- LIBRA - ruBABILongQA2: 44.67 (#7/13)
- LIBRA - MatreshkaNames *: 56.05 (#8/13)
- LIBRA - ru2WikiMultihopQA *: 46.0 (#8/13)
- LIBRA - LibrusecMHQA *: 35.21 (#8/13)
- LIBRA - ruBABILongQA1: 55.57 (#8/13)
- LIBRA - ruBABILongQA4: 49.0 (#8/13)
- LIBRA - ruSciAbstractRetrieval: 56.26 (#9/13)
- LIBRA - ruSciFi: 41.83 (#9/13)
- LIBRA - ruBABILongQA3 *: 26.5 (#9/13)
Mistral-7B-v0.3 — ELO 1222, #601
- BenchLM: 39.1 (#186/200)
Granite 4.0 H 350M — ELO 1145, #634
- BenchLM: 37.5 (#195/200)
- AA MATH-500: 11.64 (#204/206)
Granite 4.0 350M — ELO 1120, #641
- BenchLM: 37.5 (#194/200)
- AA MATH-500: 14.0 (#202/206)
Gemma 4 E2B — ELO 1111, #643
- SEA-HELM: 47.21 (#32/44)
- BenchLM: 41.1 (#175/200)
LFM2.5-1.2B Instruct — ELO 1110, #644
- AA MATH-500: 71.68 (#137/206)
GPT-5.6 Pro Sol — ELO 2802
- MineBench: 2150.33 (#1/53)
- PrinzBench: 91.0 (#1/29)
Grok 4.1 — ELO 2009
- Mercor APEX: 12.8 (#36/50)
- BenchLM: 59.2 (#60/200)
LFM2.5-8B-A1B — ELO 1666
- AA MATH-500: 90.16 (#71/206)
- BenchLM: 40.6 (#179/200)
Laguna M.1 — ELO 1705
- AI Chess Leaderboard (Continuation): 747.0 (#77/248)
- AI Chess Leaderboard (Reasoning): 770.0 (#105/303)
Muse Spark — ELO 2377
- BenchLM: 70.4 (#16/200)
Nemotron-4 15B — ELO 1291
- BenchLM: 44.3 (#155/200)
O1 Pro — ELO 1954
- BenchLM: 45.0 (#152/200)
O3 Pro — ELO 2275
- BenchLM: 47.4 (#137/200)
Seed 2.0 Lite — ELO 2046
- BenchLM: 49.1 (#130/200)

Top-10 New Scores (60)

Claude Fable 5 on CHI-Bench: 24.0 (#6)
Claude Fable 5 on Vals AI Public Benefits Bench: 70.43 (#2)
Claude Opus 4.8 on Mercor APEX: 42.5 (#3)
Claude Opus 5 on AI Chess Leaderboard (Continuation): 736.0 (#80)
Claude Opus 5 on AI Chess Leaderboard (Reasoning): 1181.0 (#41)
Claude Opus 5 on BenchLM: 85.9 (#1)
Claude Opus 5 on Benchmarks.bio - EpiBench: 35.22 (#4)
Claude Opus 5 on Benchmarks.bio - SpatialBench-Long: 25.0 (#2)
Claude Opus 5 on Bullshit Benchmark: 60.0 (#21)
Claude Opus 5 on CursorBench 3.1: 70.0 (#2)
Claude Opus 5 on Design Arena (Data Viz): 1338.0 (#5)
Claude Opus 5 on Design Arena (UI Components): 1387.0 (#2)
Claude Opus 5 on Design Arena (Website): 1331.0 (#4)
Claude Opus 5 on Epoch AI - ECI: 159.38 (#19)
Claude Opus 5 on LLM Stats (AutomationBench): 26.0 (#2)
Claude Opus 5 on LLM Stats (BrowseComp): 90.8 (#2)
Claude Opus 5 on LLM Stats (HealthBench Professional): 59.8 (#3)
Claude Opus 5 on LLM Stats (Legal Agent Benchmark): 11.7 (#2)
Claude Opus 5 on LLM2014 Logic 2026-07: 31.73 (#21)
Claude Opus 5 on Mercor APEX: 43.5 (#2)
Claude Opus 5 on Ramp SWE-Bench: 87.34 (#2)
Claude Opus 5 on RuneBench: 7841.0 (#2)
Claude Opus 5 on Senior SWE-Bench: 28.2 (#2)
Claude Opus 5 on SimpleBench: 80.6 (#2)
Claude Opus 5 on Vals AI (Vals Index): 74.82 (#2)
Claude Opus 5 on Vals AI CyberBench: 40.68 (#18)
Claude Opus 5 on Vals AI Excel Modeling: 73.56 (#2)
Claude Opus 5 on Vals AI GPQA: 93.43 (#4)
Claude Opus 5 on Vals AI Harvey Legal Agent Bench: 6.67 (#7)
Claude Opus 5 on Vals AI LegalBench: 86.97 (#4)
Claude Opus 5 on Vals AI LiveCodeBench: 89.03 (#2)
Claude Opus 5 on Vals AI Multimodal Index: 73.9 (#2)
Claude Opus 5 on Vals AI SAGE: 49.43 (#17)
Claude Opus 5 on Vals AI SkillsBench: 60.44 (#6)
Claude Opus 5 on Vals AI TaxEval v2: 75.14 (#18)
Claude Opus 5 on Vals AI Terminal-Bench 2.1: 84.64 (#2)
Claude Opus 5 on Vals AI Vibe Code Bench: 88.4 (#2)
Claude Opus 5 on You're Absolutely Right!: 4.0 (#4)
GPT-5.4 Pro on BenchLM: 60.0 (#47)
GPT-5.5 Pro on BenchLM: 62.8 (#41)
GPT-5.6 Sol on BenchLM: 81.5 (#4)
GPT-5.6 Sol on CHI-Bench: 25.3 (#4)
GPT-5.6 Sol on Epoch AI - Vending Bench 2: 9619.37 (#2)
GPT-5.6 Sol on FoodTruckBench: 53229.0 (#2)
GPT-5.6 Sol on GBA Eval: 0.5259 (#6)
GPT-5.6 Sol on GDP.pdf: 30.7 (#4)
GPT-5.6 Sol on HiL-Bench: 32.33 (#8)
GPT-5.6 Sol on Icelandic LLM - ARC-Challenge-IS: 95.22 (#2)
GPT-5.6 Sol on Icelandic LLM - Belebele-IS: 94.67 (#5)
GPT-5.6 Sol on Icelandic LLM - GED: 75.5 (#10)
GPT-5.6 Sol on Icelandic LLM - Inflection: 97.0 (#6)
GPT-5.6 Sol on Icelandic LLM - WikiQA-IS: 63.21 (#5)
GPT-5.6 Sol on Icelandic LLM - WinoGrande-IS: 94.49 (#10)
GPT-5.6 Sol on Icelandic LLM Leaderboard - Average: 86.68 (#8)
GPT-5.6 Sol on Kaggle FACTS Grounding: 68.75 (#16)
GPT-5.6 Sol on Mercor APEX: 39.9 (#7)
GPT-5.6 Sol on MyPCBench: 55.4 (#2)
GPT-5.6 Sol on Vals AI SkillsBench: 54.1 (#10)
GPT-5.6 Sol on Vellum - GPQA: 94.6 (#3)
GPT-5.6 Sol on Vellum - Humanity's Last Exam: 47.2 (#10)

New #1 Leaders (72)

RuneBench: Grok 4.5 (8716.0) beat GPT-5.6 Sol by 1316.0
Epoch AI - Vending Bench 2: Claude Opus 4.7 (Unknown) (10936.76) beat GPT-5.5 by 309.8
GDPval-AA: Claude Opus 5 (Adaptive Reasoning, Max Effort) (1861.0) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 101.0
AA GDPval: Claude Opus 5 (Adaptive Reasoning, Max Effort) (1860.57) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 100.97
LLM Stats (GDPval-AA): Claude Opus 5 (1861.0) beat Claude Fable 5 by 46.0
MineBench: GPT-5.6 Pro Sol (2150.33) beat GPT-5.6 Sol by 41.54
ASCIIBench: Claude Opus 4.7 (1701.0) beat GPT-5.5 by 28.0
HiL-Bench: Claude Fable 5 (56.33) beat GPT-5.5 by 27.23
WorldCupBench: Claude Fable 5 (110.0) beat MiniMax-M3 by 23.0
Design Arena (Website): Kimi K3 (1375.0) beat GPT-5.6 Sol by 23.0
ARC-AGI-3: Claude Opus 5 (High) (30.16) beat GPT-5.6 Sol (Max) by 22.38
SEAL - EnigmaEval: Claude Fable 5 (High) (39.28) beat GPT-5.4 Pro (xHigh) by 15.46
Design Arena (UI Components): Kimi K3 (1408.0) beat GPT-5.6 Sol by 15.0
Riemann-bench: GPT-5.6 Sol (Max) (74.4) beat Claude Fable 5 by 14.4
PrinzBench: GPT-5.6 Pro Sol (91.0) beat GPT-5.5 Pro (Extended) by 9.0
EnterpriseRAG Bench - Completeness: Troml (81.84) beat OpenClaw by 8.98
Vals AI Public Benefits Bench: Claude Opus 5 (76.93) beat Kimi K3 by 8.66
EnterpriseRAG Bench: Troml (76.79) beat OpenClaw by 8.57
AutomationBench: Claude Opus 5 (Max) (26.2) beat GPT-5.6 Sol (Max) by 8.1
Benchmarks.bio - VariantBench: Claude Opus 5 (49.72) beat GPT-5.6 Sol by 7.62
EnterpriseRAG Bench - Recall: Troml (86.55) beat OpenClaw by 7.53
Vellum - ARC-AGI-2: GPT-5.6 Sol (92.5) beat GPT-5.5 by 7.5
TrackingAI IQ Test (Vision): GPT-5.6 Terra (Ultra) (94.12) beat Claude Opus 4 (Thinking) by 6.62
Benchmarks.bio - BioSecBench-Surveillance: Claude Opus 5 (59.8) beat Grok 4.5 by 6.5
Design Arena (Text to Speech): google-gemini-3.1-flash-tts (1428.0) beat google-gemini-2.5-pro-tts by 6.0
Vals AI Legal Research Bench: Claude Opus 5 (55.29) beat Claude Fable 5 by 5.77
NYT Connections Older Models: ByteDance Seed2.1 Pro (97.8) beat GLM-5.2 (Thinking, Max) by 5.1
Vals AI IOI: Claude Opus 5 (91.67) beat GPT-5.6 Sol by 5.0
Vals AI ProgramBench: Claude Opus 5 (82.27) beat GPT-5.6 Sol by 4.63
Vals AI MedCode: Claude Opus 5 (63.57) beat Gemini 3.1 Pro (Preview) by 4.51
YapBench: GPT-5.6 Sol (18.5) beat GPT-3.5 Turbo by 4.2
AA Omniscience - Software Engineering (SWE) - Go: Claude Opus 5 (Adaptive Reasoning, Max Effort) (92.0) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 4.0
Vals AI SkillsBench: Grok 4.5 (66.03) beat GPT-5.5 Codex by 3.48
EQ-Bench Longform Writing: Claude Opus 5 (86.3) beat Claude Fable 5 by 3.3
WorldScore - Content Alignment: UniWorld-View (86.61) beat EvoPhys-World by 3.24
Benchmarks.bio - scBench-Long: Claude Opus 5 (41.27) beat GPT-5.6 Sol by 3.17
WorldScore - Camera Control: UniWorld-View (97.72) beat EvoPhys-World by 3.16
TrackingAI IQ Test (Mensa Norway): Grok 4.5 (100.0) beat GPT-5 Pro by 2.86
Vals AI Code Migration: Claude Opus 5 (57.47) beat Claude Fable 5 by 2.41
EnterpriseRAG Bench - Correctness: Troml (83.8) beat OpenClaw by 2.2
Vals AI MedScribe: Claude Opus 5 (90.98) beat Muse Spark 1.1 by 2.09
Chatbot Arena (Document): Claude Opus 4.6 (1510.0) beat Claude Opus 4.6 (Thinking) by 2.0
Vals AI MortgageTax: Claude Opus 5 (72.06) beat Claude Opus 4.7 by 1.79
WorldScore - Motion Accuracy: UniWorld-View (77.66) beat LTX-Video by 1.44
Vals AI CorpFin v2: Claude Opus 5 (73.19) beat Claude Fable 5 by 1.36
BoxPwnr CTF Bench: Grok 4.5 (xHigh) (56.73) beat GLM-5.1 by 1.26
WorldScore - Object Control: UniWorld-View (88.98) beat FantasyWorld-1.0 by 1.08
Epoch AI - Proofbench: Claude Opus 5 (Max) (78.0) beat GPT-5.6 Sol (Max) by 1.0
Vals AI ProofBench: Claude Opus 5 (78.0) beat GPT-5.6 Sol by 1.0
P2PCLAW Innovative Benchmark: API-User (9.1) beat cajal-9B-v2-q8-v7-4-fixed by 0.9
DeepSWE: Claude Opus 5 (73.6) beat GPT-5.6 Sol by 0.9
Artificial Analysis Intelligence Index: Claude Opus 5 (Adaptive Reasoning, Max Effort) (60.69) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 0.83
Vals AI SWE-bench Verified: Claude Opus 5 (97.0) beat GPT-5.6 Sol by 0.8
AI for Education Pedagogy - Maths: Gemini 3.6 Flash (95.24) beat Gemini-3.1 Pro by 0.8
Vals AI Finance Agent v2: Claude Opus 5 (58.63) beat Gemini 3.5 Flash by 0.77
TrackingAI IQ Test: Grok 4.5 (98.04) beat Claude Fable 5 by 0.74
Gert Labs Rankings: Claude Fable 5 (72.78) beat GPT-5.6 Sol by 0.73
Vellum - HumanEval: GPT-5.6 Sol (96.2) beat Claude Mythos 5 by 0.7
Vals AI MMMU: Claude Opus 5 (89.88) beat Claude Fable 5 by 0.57
Spider 2.0-Lite: ktx (73.67) beat DivSkill-SQL by 0.54
AA MMMU-Pro: Claude Opus 5 (Adaptive Reasoning, Max Effort) (84.74) beat Gemini 3.5 Flash (High) by 0.46
AI for Education SEND: Gemini 3.6 Flash (88.53) beat GPT-5.5 by 0.46
WorldScore - Static: UniWorld-View (85.53) beat WorldScape-0.2(MoE) by 0.4
Kaggle FACTS Multimodal: Gemini 3.6 Flash (47.21) beat Gemini 2.5 Pro by 0.35
OpenClawProBench: Kimi K3 (81.6) beat GLM-5.2 by 0.3
Vellum - Humanity's Last Exam: Claude Opus 5 (64.7) beat Claude Mythos 5 by 0.2
Agent Arena - Tool Hallucination: Gemini 3 Flash (0.03) beat Qwen 3.7 Plus by 0.16
WorldScore: UniWorld-View (80.81) beat WorldScape-0.2(MoE) by 0.13
LLM Stats (CyberGym): Gemini 3.5 Flash Cyber (83.2) beat Claude Mythos Preview by 0.1
Vals AI MMLU-Pro: Claude Opus 5 (91.59) beat Claude Fable 5 by 0.09
WorldScore - Photometric Consistency: UniWorld-View (94.11) beat FantasyWorld-1.0 by 0.04
ProphetArena: ap-gemini-3.5-flash (0.9629) beat Kimi K2.5 by 0.03

The Aggregate Digest — 2026-07-25

2026-07-25T06:36:16.452377+00:00

Daily

New Benchmarks (4)

DecBench (Exact Recovery (%)): leader GPT-5.6 Sol (53.5), 2 models
Bongard Problems (Classic) (Accuracy (% correct)): leader Gemini 3.1 Pro (Preview) (55.4), 4 models
LLM Stats (DeepSWE 1.1) (Score (%)): leader GPT-5.6 Sol (73.0), 18 models
LLM Stats (FrontierCode 1.1) (Score (%)): leader Claude Fable 5 (53.5), 15 models

New Models (1)

Claude Opus 5 — ELO 2882, #7
- DeepSWE: 73.6 (#1/22)
- Benchmarks.bio - scBench-Long: 41.27 (#1/19)
- Vellum - Humanity's Last Exam: 64.7 (#1/31)
- Vals AI CorpFin v2: 73.19 (#1/126)
- Vals AI Finance Agent v2: 58.63 (#1/40)
- Vals AI MortgageTax: 72.06 (#1/90)
- Vals AI MedCode: 63.57 (#1/76)
- Vals AI MedScribe: 90.98 (#1/75)
- Vals AI ProofBench: 78.0 (#1/54)
- Vals AI Public Benefits Bench: 76.93 (#1/23)

Top-10 New Scores (35)

Claude Opus 5 on AI Chess Leaderboard (Continuation): 736.0 (#80)
Claude Opus 5 on AI Chess Leaderboard (Reasoning): 1181.0 (#41)
Claude Opus 5 on Benchmarks.bio - SpatialBench-Long: 25.0 (#2)
Claude Opus 5 on Bullshit Benchmark: 60.0 (#21)
Claude Opus 5 on CursorBench 3.1: 70.0 (#2)
Claude Opus 5 on Design Arena (Data Viz): 1303.0 (#12)
Claude Opus 5 on Epoch AI - ECI: 159.38 (#19)
Claude Opus 5 on LLM Stats (AutomationBench): 26.0 (#2)
Claude Opus 5 on LLM Stats (BrowseComp): 90.8 (#2)
Claude Opus 5 on LLM Stats (HealthBench Professional): 59.8 (#3)
Claude Opus 5 on LLM Stats (Legal Agent Benchmark): 11.7 (#2)
Claude Opus 5 on LLM2014 Logic 2026-07: 31.73 (#21)
Claude Opus 5 on Mercor APEX: 43.5 (#2)
Claude Opus 5 on Ramp SWE-Bench: 87.34 (#2)
Claude Opus 5 on RuneBench: 7841.0 (#2)
Claude Opus 5 on SealedBench: 69.0 (#3)
Claude Opus 5 on Senior SWE-Bench: 28.2 (#2)
Claude Opus 5 on SimpleBench: 80.6 (#2)
Claude Opus 5 on Vals AI (Vals Index): 74.82 (#2)
Claude Opus 5 on Vals AI CyberBench: 40.68 (#18)
Claude Opus 5 on Vals AI Excel Modeling: 73.56 (#2)
Claude Opus 5 on Vals AI GPQA: 93.43 (#4)
Claude Opus 5 on Vals AI Harvey Legal Agent Bench: 6.67 (#7)
Claude Opus 5 on Vals AI LegalBench: 86.97 (#4)
Claude Opus 5 on Vals AI LiveCodeBench: 89.03 (#2)
Claude Opus 5 on Vals AI Multimodal Index: 73.9 (#2)
Claude Opus 5 on Vals AI SAGE: 49.43 (#17)
Claude Opus 5 on Vals AI SkillsBench: 60.44 (#6)
Claude Opus 5 on Vals AI TaxEval v2: 75.14 (#18)
Claude Opus 5 on Vals AI Terminal-Bench 2.1: 84.64 (#2)
Claude Opus 5 on Vals AI Vibe Code Bench: 88.4 (#2)
Claude Opus 5 on You're Absolutely Right!: 4.0 (#4)
GPT-5.6 Sol on CHI-Bench: 25.3 (#4)
Kimi K3 on Benchmarks.bio - SpatialBench-Long: 12.5 (#7)
Kimi K3 on Benchmarks.bio - scBench-Long: 12.7 (#11)

New #1 Leaders (30)

GDPval-AA: Claude Opus 5 (Adaptive Reasoning, Max Effort) (1861.0) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 114.0
AA GDPval: Claude Opus 5 (Adaptive Reasoning, Max Effort) (1860.57) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 113.83
LLM Stats (GDPval-AA): Claude Opus 5 (1861.0) beat Claude Fable 5 by 46.0
ARC-AGI-3: Claude Opus 5 (High) (30.16) beat GPT-5.6 Sol (Max) by 22.38
Benchmarks.bio - VariantBench: opus-5 (49.72) beat GPT-5.6 Sol by 7.62
Benchmarks.bio - BioSecBench-Surveillance: opus-5 (59.8) beat Grok 4.5 by 6.5
Vals AI Public Benefits Bench: Claude Opus 5 (76.93) beat Claude Fable 5 by 6.5
AutomationBench: Claude Opus 5 (Max) (26.2) beat Gemini 3.6 Flash (High) by 6.4
Vals AI Legal Research Bench: Claude Opus 5 (55.29) beat Claude Fable 5 by 5.77
Vals AI IOI: Claude Opus 5 (91.67) beat GPT-5.6 Sol by 5.0
Vals AI ProgramBench: Claude Opus 5 (82.27) beat GPT-5.6 Sol by 4.63
Vals AI MedCode: Claude Opus 5 (63.57) beat Gemini 3.1 Pro (Preview) by 4.51
AA Omniscience - Software Engineering (SWE) - Go: Claude Opus 5 (Adaptive Reasoning, Max Effort) (92.0) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 4.0
Benchmarks.bio - scBench-Long: Claude Opus 5 (41.27) beat GPT-5.6 Sol by 3.17
Vals AI Code Migration: Claude Opus 5 (57.47) beat Claude Fable 5 by 2.41
Vals AI MedScribe: Claude Opus 5 (90.98) beat Muse Spark 1.1 by 2.09
AA-Briefcase: Claude Opus 5 (Adaptive Reasoning, Xhigh Effort) (57.98) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 2.02
BenchLM: Claude Opus 5 (85.9) beat Claude Mythos 5 by 2.0
Vals AI MortgageTax: Claude Opus 5 (72.06) beat Claude Opus 4.7 by 1.79
Vals AI CorpFin v2: Claude Opus 5 (73.19) beat Claude Fable 5 by 1.36
Epoch AI - Proofbench: Claude Opus 5 (Max) (78.0) beat GPT-5.6 Sol (Max) by 1.0
Vals AI ProofBench: Claude Opus 5 (78.0) beat GPT-5.6 Sol by 1.0
DeepSWE: Claude Opus 5 (73.6) beat GPT-5.6 Sol by 0.9
Artificial Analysis Intelligence Index: Claude Opus 5 (Adaptive Reasoning, Max Effort) (60.69) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 0.83
Vals AI SWE-bench Verified: Claude Opus 5 (97.0) beat GPT-5.6 Sol by 0.8
Vals AI Finance Agent v2: Claude Opus 5 (58.63) beat Gemini 3.5 Flash by 0.77
Vals AI MMMU: Claude Opus 5 (89.88) beat Claude Fable 5 by 0.57
AA MMMU-Pro: Claude Opus 5 (Adaptive Reasoning, Max Effort) (84.74) beat Gemini 3.5 Flash (High) by 0.46
Vellum - Humanity's Last Exam: Claude Opus 5 (64.7) beat Claude Mythos 5 by 0.2
Vals AI MMLU-Pro: Claude Opus 5 (91.59) beat Claude Fable 5 by 0.09

The Aggregate Digest — 2026-07-24

2026-07-24T06:45:02.455016+00:00

Daily

New Benchmarks (2)

REFUTE (Truth Score (accuracy + calibration, higher is better)): leader Grok 4.20 (74.5), 15 models
SealedBench (Capability Index): leader Claude Fable 5 (71.0), 28 models
A private, continuously rotated frontier-capability eval whose tests, scoring weights and answer keys stay sealed to resist benchmark-specific training. Publishes a single composite Model Capability Index (0-100) blending capability, token efficiency, speed and cost, with thinking set to the highest available level.

Top-10 New Scores (9)

GPT-5.6 Sol on GDP.pdf: 30.7 (#4)
GPT-5.6 Sol on Kaggle FACTS Grounding: 68.75 (#16)
GPT-5.6 Sol on Vellum - GPQA: 94.6 (#3)
GPT-5.6 Sol on Vellum - Humanity's Last Exam: 47.2 (#9)
Kimi K3 on Agents' Last Exam: 28.3 (#3)
Kimi K3 on NYT Connections Extended: 94.5 (#10)
Kimi K3 on Vellum - GPQA: 93.5 (#10)
Kimi K3 on Vellum - HumanEval: 76.8 (#19)
Kimi K3 on Vellum - Humanity's Last Exam: 56.0 (#4)

New #1 Leaders (11)

SEAL - EnigmaEval: Claude Fable 5 (High) (39.28) beat GPT-5.4 Pro (xHigh) by 15.46
Vellum - ARC-AGI-2: GPT-5.6 Sol (92.5) beat GPT-5.5 by 7.5
WorldScore - Content Alignment: UniWorld-View (86.61) beat EvoPhys-World by 3.24
WorldScore - Camera Control: UniWorld-View (97.72) beat EvoPhys-World by 3.16
WorldScore - Motion Accuracy: UniWorld-View (77.66) beat LTX-Video by 1.44
WorldScore - Object Control: UniWorld-View (88.98) beat FantasyWorld-1.0 by 1.08
TrackingAI IQ Test: Grok 4.5 (98.04) beat Claude Fable 5 by 0.74
Vellum - HumanEval: GPT-5.6 Sol (96.2) beat Claude Mythos 5 by 0.7
WorldScore - Static: UniWorld-View (85.53) beat WorldScape-0.2(MoE) by 0.4
WorldScore: UniWorld-View (80.81) beat WorldScape-0.2(MoE) by 0.13
WorldScore - Photometric Consistency: UniWorld-View (94.11) beat FantasyWorld-1.0 by 0.04

The Aggregate Digest — 2026-07-23

2026-07-23T06:49:50.250436+00:00

Daily

New Benchmarks (4)

ActiveVision (Accuracy (%)): leader GPT-5.5 (10.6), 6 models
activevision.dev "exam for active observers": iterative visual reasoning where models must scan, traverse and transform images over multiple steps; accuracy over 85 items (human reference 96%).
ActiveVision (Agents) (Accuracy (%)): leader Claude Fable 5 (50.6), 3 models
ActiveVision solved by tool-using agent harnesses (image-manipulation tools available) instead of pure chain-of-thought; accuracy over the same 85 items.
KernelBench Hub - CUDA (Best % of Hardware Roofline): leader Claude Fable 5 (72.66), 5 models
kernelbench.com CUDA suite (independent of Stanford KernelBench): agentic CUDA kernel optimization scored as percent of hardware roofline achieved, best across RTX PRO 6000, H100 and B200 runs.
Vals AI Time Horizon Index (Progress (%)): leader Claude Opus 4.8 (11.83), 4 models
Vals AI Kerbal Space Program agent benchmark: how far a model progresses through a 30-mission space program (rocket design, orbital mechanics) in a 120-hour window; each mission rung is 3.33% progress.

Top-10 New Scores (7)

Claude Fable 5 on CHI-Bench: 24.0 (#5)
Kimi K3 on Agent Arena: 9.71 (#8)
Kimi K3 on Agent Arena - Bash Recovery: 6.33 (#25)
Kimi K3 on Agent Arena - Confirmed Success: 14.0 (#3)
Kimi K3 on Agent Arena - Praise vs Complaint: 20.3 (#3)
Kimi K3 on Agent Arena - Steerability: 6.52 (#22)
Kimi K3 on Agent Arena - Tool Hallucination: 1.39 (#25)

New #1 Leaders (6)

TrackingAI IQ Test (Vision): GPT-5.6 Terra (Ultra) (92.16) beat Claude Opus 4 (Thinking) by 4.66
Kaggle FACTS Multimodal: Gemini 3.6 Flash (50.24) beat Gemini 2.5 Pro by 3.38
Vals AI Public Benefits Bench: Claude Fable 5 (70.43) beat Kimi K3 by 2.16
AutomationBench: Gemini 3.6 Flash (High) (19.8) beat GPT-5.6 Sol (Max) by 1.7
AI for Education Pedagogy - Maths: Gemini-3.6 Flash (95.24) beat Gemini-3.1 Pro by 0.8
AI for Education SEND: Gemini-3.6 Flash (88.53) beat GPT-5.5 by 0.46

The Aggregate Digest — 2026-07-22

2026-07-22T06:52:29.593249+00:00

Daily

New Benchmarks (14)

LLM Arena RU (Arena Elo): leader Gemini 3 Pro (Preview) (1210.0), 92 models
Crowdsourced blind pairwise battles in Russian: Bradley-Terry Elo rating from user votes on llmarena.ru, the Russian-language counterpart of LMArena.
AA-Briefcase (Rubric Pass Rate (%)): leader Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) (55.96), 47 models
FutureSim (Top-1 Accuracy (%)): leader Claude Fable 5 (26.52), 9 models
ProfBench (Overall Rubric Score (%)): leader Claude Opus 4.7 (Thinking) (61.3), 98 models
Professional report-writing tasks graded against rubrics written by PhD physicists and chemists and MBA-level finance and consulting experts; rubric satisfaction across the four domains.
SalesBench (50-Lead Reward): leader GLM-5.1 (0.508), 7 models
DrugDiscoveryBench (Score (%)): leader GPT-5.5 (51.6), 7 models
TextQuests (No Clues) (Avg Game Progress (%)): leader Claude Fable 5 (54.48), 35 models
Playing 25 classic Infocom interactive-fiction games (Zork etc.) without hints: average game completion via long-horizon exploration, puzzle solving and state tracking in a text world.
TextQuests (With Clues) (Avg Game Progress (%)): leader GPT-5.6 Sol (86.84), 35 models
Playing 25 classic Infocom interactive-fiction games with the official hint books available: average game completion, testing how well models exploit in-context guidance over long horizons.
RevengeBench (Arena Mean Score): leader GPT-5 (71.92), 12 models
GameCraft-Bench (Overall Score): leader Claude Fable 5 (65.7), 10 models
Kagi LLM Benchmark (Accuracy (%)): leader Claude Fable 5 (Thinking) (91.4), 145 models
Rotating set of unpublished reasoning, coding and instruction-following tasks run by Kagi; accuracy on ~116 private tasks designed to resist benchmark contamination.
SpeechMap Compliance (% Requests Completed): leader sherlock-dash-alpha (100.0), 344 models
Willingness to answer sensitive or controversial prompts (political argument, satire, religion, rights advocacy): share of requests completed directly rather than refused, evaded or errored. Behavioral propensity, not capability.
SnitchBench (Govt Contact Rate (%)): leader Grok 4.1 (92.5), 8 models
Propensity to autonomously report user wrongdoing: rate at which a model, given evidence of corporate misconduct and email tools, contacts government authorities. Behavioral propensity, not capability.
MLS-Bench Lite (Score): leader Claude Fable 5 (49.9), 12 models
Official 30-task subset of MLS-Bench for evaluating whether AI systems can invent generalizable and scalable machine-learning methods.

New Models (2)

Gemini 3.6 Flash — ELO 2417, #23
- RuneBench: 7454.0 (#1/39)
- Design Arena (3D): 1384.0 (#2/133)
- LLM Stats (OSWorld-Verified): 83.0 (#3/21)
- AI Chess Leaderboard (Continuation): 1782.0 (#3/244)
- LLM Stats (CharXiv-R): 89.4 (#5/47)
- Arabic Broad Leaderboard: 9.111 (#6/106)
- AI Chess Leaderboard (Reasoning): 1763.0 (#6/300)
- Kaggle FACTS Grounding: 72.51 (#6/40)
- LLM Stats (MRCR v2 (8-needle)): 54.0 (#7/21)
- Vals AI MedCode: 53.15 (#7/74)
Gemini 3.5 Flash Lite — ELO 1973, #146
- AI Chess Leaderboard (Continuation): 1512.0 (#9/244)
- LLM Stats (OSWorld-Verified): 74.0 (#11/21)
- LLM Stats (Terminal-Bench 2.1): 54.0 (#15/15)
- LLM Stats (MRCR v2 (8-needle)): 21.3 (#18/21)
- RuneBench: 2254.0 (#18/39)
- AA MMMU-Pro: 79.02 (#24/228)
- AI Chess Leaderboard (Reasoning): 1399.0 (#24/300)
- LLM Stats (CharXiv-R): 76.5 (#29/47)
- LLM Stats (GDPval-AA): 1140.0 (#30/41)
- Arabic Broad Leaderboard: 8.626 (#34/106)

Top-10 New Scores (6)

Claude Opus 4.8 on Guesswork: 0.977 (#12)
GPT-5.6 Pro Sol on MineBench: 2159.91 (#1)
GPT-5.6 Sol on Epoch AI - Vending Bench 2: 9619.37 (#2)
GPT-5.6 Sol on HiL-Bench: 32.33 (#8)
GPT-5.6 Sol on Vals AI SkillsBench: 54.1 (#9)
Kimi K3 on GDP.pdf: 19.0 (#11)

New #1 Leaders (9)

RuneBench: Gemini 3.6 Flash (7454.0) beat GPT-5.6 Sol by 54.0
HiL-Bench: Claude Fable 5 (56.33) beat GPT-5.5 by 27.23
EnterpriseRAG Bench - Completeness: Troml (81.84) beat OpenClaw by 8.98
EnterpriseRAG Bench: Troml (76.79) beat OpenClaw by 8.57
EnterpriseRAG Bench - Recall: Troml (86.55) beat OpenClaw by 7.53
Vals AI SkillsBench: Grok 4.5 (66.03) beat GPT-5.5 Codex by 3.48
EnterpriseRAG Bench - Correctness: Troml (83.8) beat OpenClaw by 2.2
BoxPwnr CTF Bench: Grok 4.5 (xHigh) (56.73) beat GLM-5.1 by 1.26
LLM Stats (CyberGym): Gemini 3.5 Flash Cyber (83.2) beat Claude Mythos Preview by 0.1

The Aggregate Digest — 2026-07-21

2026-07-21T06:47:55.752132+00:00

Daily

New Benchmarks (10)

MathArena - ARXIV_FALSE June (Accuracy (%)): leader GPT-5.5 (xHigh) (69.44), 12 models
MathArena - ARXIV June (Accuracy (%)): leader GPT-5.6 Sol (Max) (86.73), 12 models
Office Comprehension Bench - File Fidelity Excel (File Fidelity Accuracy (%)): leader GPT-5.5 (Thinking) (72.6), 4 models
Perceiving structural and visual content in native .xlsx files — tables, charts, embedded images and formatting — answered from the file itself rather than a text conversion.
Office Comprehension Bench - File Fidelity PowerPoint (File Fidelity Accuracy (%)): leader Claude Opus 4.7 (86.6), 4 models
Perceiving structural and visual content in native .pptx decks — slide layouts, charts, images and formatting — answered from the file itself.
Office Comprehension Bench - File Fidelity Word (File Fidelity Accuracy (%)): leader Claude Opus 4.7 (91.5), 4 models
Perceiving structural and visual content in native .docx files — tables, figures and formatting — answered from the file itself; the widest model spread of the three applications.
GRIPS (Accuracy (%)): leader GPT-5.5 (xHigh) (96.4), 27 models
German Reasoning, Idioms, Puzzles & Wordplay (Sprachspiel): a held-out private set of German-language puzzles, idiomatic expressions and wordplay that resist translation, scored by LLM judge.
GRIPS-hard (Accuracy (%)): leader GPT-5.5 (xHigh) (87.3), 27 models
The hard subset of GRIPS, restricted to the German reasoning, idiom and wordplay items that discriminate most between models; scores drop sharply versus the full set.
Guesswork (MAE (z-score units)): leader Gemma 3 4B (IT) (6.7079), 17 models
This site's own live benchmark for predicting benchmark scores: frontier and budget LLMs forecast newly published (model, benchmark) results a day before they land, scored in per-benchmark z-units (MAE, lower is better) against the same-information statistical baseline The Aggregate.
StructureClaw (Automatic Workflow) (Success Rate (%)): leader Kimi K2.6 (100.0), 9 models
StructureClaw executable agent benchmark (arXiv 2607.14896); the automatic-workflow track reports the success rate of building a working structured-extraction agent without human guidance.
LLM GPU Kernel Generation (Pass Rate (%)): leader GPT-5.5 (100.0), 5 models
Trace-driven benchmark for LLM-generated GPU kernels (arXiv 2607.14541): the fraction of generated kernels that both compile and produce numerically correct results against reference execution traces.

Top-10 New Scores (36)

Claude Opus 4.8 on Mercor APEX: 42.5 (#2)
GPT-5.4 Pro on BenchLM: 60.9 (#44)
GPT-5.5 Pro on BenchLM: 63.7 (#38)
GPT-5.6 Sol on BenchLM: 82.0 (#3)
GPT-5.6 Sol on Icelandic LLM - ARC-Challenge-IS: 95.22 (#2)
GPT-5.6 Sol on Icelandic LLM - Belebele-IS: 94.67 (#5)
GPT-5.6 Sol on Icelandic LLM - GED: 75.5 (#10)
GPT-5.6 Sol on Icelandic LLM - Inflection: 97.0 (#6)
GPT-5.6 Sol on Icelandic LLM - WikiQA-IS: 63.21 (#5)
GPT-5.6 Sol on Icelandic LLM - WinoGrande-IS: 94.49 (#10)
GPT-5.6 Sol on Icelandic LLM Leaderboard - Average: 86.68 (#8)
GPT-5.6 Sol on Mercor APEX: 39.9 (#5)
Kimi K3 on AI for Education Pedagogy: 87.76 (#31)
Kimi K3 on AI for Education Pedagogy - Maths: 89.68 (#21)
Kimi K3 on AI for Education Pedagogy - Primary: 92.49 (#25)
Kimi K3 on AI for Education Pedagogy - Science: 89.62 (#48)
Kimi K3 on AI for Education Pedagogy - Secondary: 86.64 (#40)
Kimi K3 on AI for Education Pedagogy - Social studies: 84.55 (#51)
Kimi K3 on AI for Education Pedagogy - Technology: 82.08 (#71)
Kimi K3 on AI for Education SEND: 81.65 (#32)
Kimi K3 on Agent Arena: 9.62 (#8)
Kimi K3 on Agent Arena - Bash Recovery: 6.41 (#24)
Kimi K3 on Agent Arena - Confirmed Success: 14.42 (#3)
Kimi K3 on Agent Arena - Praise vs Complaint: 20.62 (#3)
Kimi K3 on Agent Arena - Steerability: 5.58 (#23)
Kimi K3 on Agent Arena - Tool Hallucination: 1.08 (#24)
Kimi K3 on BenchLM: 81.0 (#4)
Kimi K3 on Benchmarks.bio - VariantBench: 32.8 (#9)
Kimi K3 on Creative Writing (Lechmazur): 2.9 (#3)
Kimi K3 on GBA Eval: 0.4835 (#8)
Kimi K3 on GBA Eval: 0.4835 (#8)
Kimi K3 on MineBench: 1756.72 (#7)
Kimi K3 on Multi-turn Debate (Lechmazur): 1740.8 (#2)
Kimi K3 on PrinzBench: 47.0 (#12)
Kimi K3 on Riemann-bench: 37.6 (#7)
Kimi K3 on TaxCalcBench: 6.0 (#9)

New #1 Leaders (4)

Design Arena (UI Components): Kimi K3 (1443.0) beat GPT-5.6 Sol by 55.0
Riemann-bench: GPT-5.6 Sol (Max) (74.4) beat Claude Fable 5 by 14.4
PrinzBench: GPT-5.6 Pro Sol (91.0) beat GPT-5.5 Pro (Extended) by 9.0
OpenClawProBench: Kimi K3 (81.6) beat GLM-5.2 by 0.3

The Aggregate Digest — 2026-07-20

2026-07-20T07:04:39.733350+00:00

Daily

New Benchmarks (8)

SWE-Milestone (Milestone Score (%)): leader Claude Opus 4.8 (Max, 1M) (51.84), 23 models
SWE-Milestone evaluates coding agents on software-evolution itineraries mined from real repositories as milestone DAGs, requiring sustained multi-milestone implementation; scored by milestone completion with the best harness retained per model.
ErdosBench (Judge Average (0-4)): leader GPT-5.6 Sol (xHigh) (3.12), 7 models
Ulam\
MWS Vision Bench (Overall Score (%)): leader Cotype Pro 3 (73.6), 38 models
MTS AI\
VSI-Super-Wild (Overall Accuracy (%)): leader Gemini 3.1 Pro (Preview) (44.36), 14 models
VSI-Super-Wild (ECCV 2026) probes spatial supersensing in genuinely long in-the-wild videos — up to 4+ hours — with human-verified questions on motion, ordering, and continuous counting; scored by overall accuracy.
FutureX (Overall Score (latest week, %)): leader Qwen 3.7 Max (49.33), 15 models
FutureX is ByteDance\
WeaveBench (PassRate (%)): leader Claude Opus 4.7 (35.1), 10 models
WeaveBench tests computer-use agents on 114 long-horizon real-world tasks that require interleaving GUI clicks with shell and code in one trajectory, graded by a trajectory-aware agent judge; scored by pass rate.
SolarBench (Pass Rate (%)): leader Claude Fable 5 (53.8), 11 models
Maingen\
EdgeBench (Score @12h (134 tasks)): leader Claude Opus 4.8 (50.89), 5 models
ByteDance Seed\

Top-10 New Scores (7)

GPT-5.5 on ForecastBench: 64.6 (#65)
GPT-5.6 Sol on GBA Eval: 0.5259 (#6)
GPT-5.6 Sol on MyPCBench: 55.4 (#2)
GPT-5.6 Sol on YapBench: 18.5 (#1)
GPT-5.6 Sol on YapBench: 18.5 (#1)
Kimi K3 on Gert Labs Rankings: 67.79 (#5)
Kimi K3 on YapBench: 300.7 (#63)

New #1 Leaders (4)

Design Arena (Website): Kimi K3 (1396.0) beat GPT-5.6 Sol by 44.0
P2PCLAW Innovative Benchmark: API-User (9.1) beat cajal-9B-v2-q8-v7-4-fixed by 0.9
Spider 2.0-Lite: ktx (73.67) beat DivSkill-SQL by 0.54
ProphetArena: ap-gemini-3.5-flash (0.9629) beat Kimi K2.5 by 0.03

The Aggregate Digest — 2026-07-19

2026-07-19T06:47:08.069151+00:00

Daily

New Benchmarks (9)

SCALE SQL Understanding (Ability Score (%)): leader Gemini 3 Pro (86.0), 35 models
SCALE SQL Dialect Conversion (Ability Score (%)): leader Qwen 3.7 Max (77.3), 35 models
SCALE SQL Optimization (Ability Score (%)): leader Gemini 3.5 Flash (69.2), 35 models
ParseBench (Overall Score): leader KDL-Frontier-Parser-nano (76.36), 50 models
Enterprise document parsing benchmark over tables, charts, layout, and content faithfulness for multimodal extraction systems.
OmniaBench (Overall Pass@1 (%)): leader Claude Sonnet 5 (58.54), 22 models
PawBench - QwenPaw (Overall Score (%)): leader Qwen 3.6 Max Preview (78.35), 9 models
PawBench - OpenClaw (Overall Score (%)): leader Claude Opus 4.6 (76.11), 9 models
PawBench - Hermes (Overall Score (%)): leader Claude Opus 4.6 (78.44), 9 models
LoHoSearch (Accuracy (%)): leader GPT-5.5 (34.74), 11 models
LoHoSearch evaluates long-horizon search agents on 544 human-verified questions requiring multi-step open-web research across 11 topical domains; this board reports the paper\

Top-10 New Scores (18)

Kimi K3 on AA APEX-Agents: 41.3 (#2)
Kimi K3 on Arabic Broad Leaderboard: 8.906 (#18)
Kimi K3 on HieroglyphBench: 25.9 (#5)
Kimi K3 on KernelBench Hub - Mega: 18.09 (#3)
Kimi K3 on KernelBench Hub - Mega: 18.09 (#3)
Kimi K3 on Vals AI GPQA: 92.93 (#5)
Kimi K3 on Vals AI Harvey Legal Agent Bench: 10.83 (#4)
Kimi K3 on Vals AI Legal Research Bench: 44.23 (#3)
Kimi K3 on Vals AI LegalBench: 86.02 (#9)
Kimi K3 on Vals AI LiveCodeBench: 87.19 (#9)
Kimi K3 on Vals AI MMLU-Pro: 87.97 (#14)
Kimi K3 on Vals AI MMMU: 88.15 (#6)
Kimi K3 on Vals AI MedScribe: 87.96 (#4)
Kimi K3 on Vals AI MortgageTax: 66.34 (#30)
Kimi K3 on Vals AI Multimodal Index: 73.42 (#2)
Kimi K3 on Vals AI SAGE: 54.26 (#4)
Kimi K3 on Vals AI TaxEval v2: 75.72 (#10)
Kimi K3 on You're Absolutely Right!: 3.5 (#8)

New #1 Leaders (6)

Design Arena (3D): Kimi K3 (1441.0) beat Claude Fable 5 by 74.0
Design Arena (Data Viz): Kimi K3 (1429.0) beat Claude Fable 5 by 65.0
Vals AI Legal Research Bench: Claude Fable 5 (49.52) beat GPT-5.6 Sol by 1.44
Vals AI Excel Modeling: Claude Fable 5 (73.67) beat GPT-5.6 Sol by 1.33
MMEB Leaderboard: DME-Large (80.24) beat octen-vl-embedding-large by 0.15
Vals AI Public Benefits Bench: Kimi K3 (68.27) beat Claude Opus 4.8 by 0.14

Weekly

New Benchmarks (9)

SCALE SQL Understanding (Ability Score (%)): leader Gemini 3 Pro (86.0), 35 models
SCALE SQL Dialect Conversion (Ability Score (%)): leader Qwen 3.7 Max (77.3), 35 models
SCALE SQL Optimization (Ability Score (%)): leader Gemini 3.5 Flash (69.2), 35 models
ParseBench (Overall Score): leader KDL-Frontier-Parser-nano (76.36), 50 models
Enterprise document parsing benchmark over tables, charts, layout, and content faithfulness for multimodal extraction systems.
OmniaBench (Overall Pass@1 (%)): leader Claude Sonnet 5 (58.54), 22 models
PawBench - QwenPaw (Overall Score (%)): leader Qwen 3.6 Max Preview (78.35), 9 models
PawBench - OpenClaw (Overall Score (%)): leader Claude Opus 4.6 (76.11), 9 models
PawBench - Hermes (Overall Score (%)): leader Claude Opus 4.6 (78.44), 9 models
LoHoSearch (Accuracy (%)): leader GPT-5.5 (34.74), 11 models
LoHoSearch evaluates long-horizon search agents on 544 human-verified questions requiring multi-step open-web research across 11 topical domains; this board reports the paper\

New Models (72)

Claude Fable 5 — ELO 3074, #6
- Android Bench: 84.5 (#1/25)
- Terminal-Bench 2.1: 83.8 (#1/12)
- Terminal-Bench 2.1 (Claude Code): 83.8 (#1/5)
- SvelteBench: 100.0 (#1/152)
- GeoBench ACW: 4403.7 (#1/48)
- GeoBench Photos: 4313.64 (#1/32)
- Vals AI Excel Modeling: 73.67 (#1/25)
- Vals AI Legal Research Bench: 49.52 (#1/24)
- Arabic Broad Leaderboard: 8.217 (#40/104)
Kimi K3 — ELO 2874, #8
- BridgeBench Security: 1023.1 (#1/5)
- BridgeBench Debugging: 1066.7 (#1/3)
- BridgeBench Refactoring: 1065.7 (#1/3)
- BridgeBench Hallucination: 1030.4 (#1/3)
- Design Arena (Data Viz): 1429.0 (#1/139)
- Design Arena (3D): 1441.0 (#1/131)
- Tau3 Banking: 33.4 (#1/159)
- Chatbot Arena (Code): 1679.0 (#1/99)
- LLM Stats (APEX-Agents): 37.6 (#1/7)
- LLM Stats (BabyVision): 85.7 (#1/9)
GPT-5.6 Sol — ELO 2827, #9
- DeepSWE: 72.7 (#1/20)
- Benchmarks.bio - SpatialBench-Long: 38.89 (#1/18)
- ZeroEval GPQA Diamond: 94.6 (#1/232)
- MineBench: 2108.79 (#1/50)
- Agents' Last Exam: 30.6 (#1/20)
- AgenticVBench: 38.4 (#1/10)
- LLM Stats (Graphwalks BFS >128k): 90.7 (#1/11)
- LLM Stats (MRCR v2 (8-needle)): 91.5 (#1/19)
- LLM Stats (Terminal-Bench 2.1): 88.8 (#1/13)
- AI for Education Visual Reasoning - match (process): 88.9 (#1/64)
Claude Opus 4.8 — ELO 2741, #10
- BridgeBench Security: 1001.5 (#2/5)
- SvelteBench: 100.0 (#4/152)
- Android Bench: 72.4 (#6/25)
GPT-5.6 Pro Sol — ELO 2671, #11
- Arabic Broad Leaderboard: 8.97 (#12/104)
GPT-5.5 — ELO 2620, #12
- GDP.pdf: 52.8 (#2/20)
- ExploitGym: 129.0 (#3/9)
Qwen 3.7 Max — ELO 2590, #13
- Vals AI Finance Agent v2: 47.78 (#16/37)
- Android Bench: 54.2 (#20/25)
Inkling — ELO 2556, #14
- EQ-Bench Longform Writing: 72.5 (#22/123)
- RuneBench: 1590.0 (#24/33)
- ARC-AGI-2: 36.53 (#47/166)
- ARC-AGI-1: 79.5 (#48/164)
Muse Spark 1.1 — ELO 2532, #15
- LLM Stats (MCP Atlas): 88.1 (#1/30)
- LLM Stats (Toolathlon): 75.6 (#1/30)
- LLM Stats (BabyVision): 76.3 (#2/9)
- LLM Stats (Finance Agent v2): 57.2 (#2/26)
- Arabic Broad Leaderboard: 9.2 (#4/104)
- LLM Stats (OSWorld-Verified): 80.8 (#4/19)
- OSWorld: 80.67 (#5/63)
- LLM Stats (CharXiv-R): 88.4 (#5/45)
- HieroglyphBench: 24.5 (#6/19)
- Terminal-Bench 2.1: 76.2 (#6/12)
GPT-5.6 Terra — ELO 2489, #18
- Benchmarks.bio - SpatialBench-Long: 22.22 (#2/18)
- WolfBench: 82.0 (#2/32)
- LLM Stats (MRCR v2 (8-needle)): 89.6 (#2/19)
- LLM Stats (HealthBench): 57.0 (#3/8)
- LLM Stats (HealthBench Hard): 32.7 (#3/9)
- LLM Stats (Terminal-Bench 2.1): 87.4 (#3/13)
- Kaggle Game Arena Poker (Heads Up): 17.85 (#3/24)
- Clerk LLM Leaderboard: 79.2 (#4/22)
- LLM Stats (BrowseComp): 87.5 (#4/57)
- LLM Stats (Graphwalks BFS >128k): 76.9 (#4/11)
Gemini 3.5 Flash — ELO 2477, #19
- Chatbot Arena (Code): 1504.0 (#19/99)
Claude Sonnet 5 — ELO 2462, #20
- Android Bench: 76.2 (#3/25)
- Terminal-Bench 2.1 (Claude Code): 74.6 (#3/5)
- Benchmarks.bio - SpatialBench-Long: 12.5 (#5/18)
- CHI-Bench: 20.0 (#7/18)
- Terminal-Bench 2.1: 74.6 (#8/12)
- Benchmarks.bio - BioSecBench-Refusal: 30.1 (#11/15)
- GeoBench Photos: 3713.48 (#12/32)
- GeoBench ACW: 3878.41 (#19/48)
- AI for Education Pedagogy: 88.21 (#22/221)
- AI for Education SEND: 82.57 (#22/213)
Claude Opus 4.7 — ELO 2436, #22
- Creative Writing (Lechmazur): 2.5 (#7/38)
Grok 4.5 — ELO 2436, #23
- Benchmarks.bio - SpatialBench-Long: 19.44 (#3/18)
- Terminal-Bench 2.1: 79.3 (#3/12)
- TrackingAI IQ Test (Offline): 81.25 (#5/35)
- Blueprint-Bench 2: 0.273 (#6/18)
- LLM Stats (Terminal-Bench 2.1): 83.3 (#6/13)
- AI for Education Pedagogy - Secondary: 89.31 (#6/221)
- React Native Evals: 84.58 (#7/26)
- Kaggle Game Arena Poker (Heads Up): 5.18 (#7/24)
- ZeroEval GPQA Diamond: 93.0 (#9/232)
- HieroglyphBench: 21.2 (#9/19)
GLM-5.2 — ELO 2400, #27
- NatureBench: 15.56 (#2/12)
- Android Bench: 72.2 (#7/25)
- CHI-Bench: 18.7 (#9/18)
- React Native Evals: 78.36 (#14/26)
- Design Arena (SVG): 1260.0 (#19/100)
GPT-5.6 Luna — ELO 2386, #30
- Clerk LLM Leaderboard: 83.6 (#2/22)
- LLM Stats (Graphwalks BFS >128k): 81.3 (#2/11)
- Vals AI CyberBench: 83.9 (#2/17)
- Benchmarks.bio - SpatialBench-Long: 18.06 (#4/18)
- WolfBench: 79.0 (#4/32)
- LLM Stats (HealthBench Hard): 32.0 (#4/9)
- LLM Stats (Terminal-Bench 2.1): 84.7 (#4/13)
- Kaggle Game Arena Poker (Heads Up): 15.95 (#4/24)
- Vals AI Finance Agent v2: 55.04 (#4/37)
- Vals AI SWE-bench Verified: 93.0 (#4/72)
Qwen 3.7 Plus — ELO 2326, #36
- Design Arena (3D): 1324.0 (#9/131)
- Android Bench: 57.7 (#18/25)
- Design Arena (Data Viz): 1280.0 (#19/139)
- Design Arena (Game Dev): 1276.0 (#26/141)
- Design Arena (UI Components): 1279.0 (#29/136)
Gemini 2.5 Pro — ELO 2286, #44
- FutureEval: 2.43 (#46/77)
MiniMax-M3 — ELO 2272, #48
- NatureBench: 11.11 (#6/12)
- Android Bench: 63.6 (#13/25)
- Creative Writing (Lechmazur): 0.7 (#16/38)
DeepSeek V4 Pro — ELO 2251, #51
- Android Bench: 59.5 (#17/25)
Kimi K2.7 Code — ELO 2250, #52
- Android Bench: 70.4 (#9/25)
MiMo-V2.5-Pro — ELO 2236, #55
- Android Bench: 60.8 (#16/25)
Hy3 — ELO 2227, #60
- LLM Stats (SkillsBench): 55.3 (#2/6)
- LLM Stats (IMO-AnswerBench): 90.0 (#3/19)
- LLM Stats (WideSearch): 76.4 (#3/9)
- LLM Stats (Claw-Eval): 68.5 (#4/13)
- LLM Stats (DeepSearchQA): 91.0 (#4/8)
- LLM Stats (MathArena Apex): 38.7 (#4/7)
- LLM Stats (NL2Repo): 45.6 (#4/12)
- LLM Stats (APEX-Agents): 25.6 (#7/7)
- LLM Stats (MCP Atlas): 79.1 (#7/30)
- LLM Stats (Terminal-Bench 2.1): 71.7 (#9/13)
DeepSeek V4 Flash — ELO 2180, #66
- Lean AI Formalization Leaderboard: 3.0 (#14/41)
- Android Bench: 54.7 (#19/25)
MiMo-V2-Pro — ELO 2146, #76
- Creative Writing (Lechmazur): -0.6 (#25/38)
Qwen 3.6 Plus — ELO 2139, #81
- FutureEval: 7.53 (#25/77)
inkling — ELO 2118, #94
- Vals AI CorpFin v2: 68.57 (#4/123)
- Vals AI MedScribe: 85.41 (#12/72)
- Vals AI CyberBench: 64.14 (#12/17)
- Vals AI SkillsBench: 26.09 (#13/13)
- Vals AI Public Benefits Bench: 58.46 (#14/19)
- Vals AI Harvey Legal Agent Bench: 2.08 (#14/24)
- Vals AI TaxEval v2: 75.31 (#15/129)
- Vals AI Legal Research Bench: 28.36 (#15/24)
- Vals AI Finance Agent v2: 46.6 (#17/37)
- Vals AI SWE-bench Verified: 77.6 (#20/72)
Kimi K2.5 — ELO 2110, #96
- Creative Writing (Lechmazur): -0.5 (#23/38)
Qwen 3 Max — ELO 2104, #99
- FutureEval: 0.12 (#58/77)
Grok 4 — ELO 2103, #100
- FutureEval: 4.13 (#40/77)
Gemini 3.1 Flash Lite — ELO 2071, #114
- FutureEval: 3.99 (#41/77)
Claude Sonnet 4 — ELO 2047, #118
- FutureEval: 0.06 (#59/77)
GPT-5 Mini — ELO 2045, #119
- FutureEval: 0.83 (#56/77)
O4 Mini — ELO 2035, #123
- FutureEval: 3.12 (#44/77)
Grok 4.1 Fast — ELO 2023, #130
- FutureEval: 0.21 (#57/77)
KAT Coder Pro V2 — ELO 2019, #133
- AA GDPval: 908.18 (#79/157)
- GDPval-AA: 908.0 (#83/162)
Grok 4.20 — ELO 2014, #135
- HieroglyphBench: 16.8 (#13/19)
- FrontierMath - Tier 4 (v2): 17.07 (#26/40)
- FrontierMath - Tiers 1-3 (v2): 44.91 (#29/38)
Mercury 2 — ELO 2003, #140
- AA GDPval: 688.89 (#99/157)
- GDPval-AA: 689.0 (#103/162)
Kimi K2 — ELO 1991, #147
- FutureEval: 0.97 (#55/77)
Qwen 3 Max (Preview) — ELO 1979, #154
- Creative Writing (Lechmazur): 0.0 (#20/38)
DeepSeek V3.2 — ELO 1957, #167
- Creative Writing (Lechmazur): -2.3 (#34/38)
Gemini 2.5 Flash — ELO 1954, #168
- FutureEval: -8.3 (#73/77)
GPT-4.1 — ELO 1944, #173
- FutureEval: 3.02 (#45/77)
Claude 3.5 Sonnet (20240620) — ELO 1931, #178
- FutureEval: -2.47 (#66/77)
Grok 3 Mini — ELO 1931, #179
- FutureEval: 1.95 (#51/77)
O1 Mini — ELO 1918, #189
- FutureEval: -9.43 (#75/77)
DeepSeek V3.1 — ELO 1916, #194
- FutureEval: 2.37 (#47/77)
MiniMax-M2.5 — ELO 1913, #198
- FutureEval: -4.74 (#68/77)
Gemma 4 31B — ELO 1899, #205
- LLM Stats (MRCR v2 (8-needle)): 66.4 (#5/19)
DeepSeek R1 — ELO 1862, #237
- FutureEval: -0.37 (#61/77)
GPT-OSS-120B — ELO 1855, #241
- FutureEval: -0.72 (#62/77)
Claude Haiku 4.5 — ELO 1851, #244
- FutureEval: 3.25 (#43/77)
Grok 3 — ELO 1842, #250
- FutureEval: 1.59 (#53/77)
Mistral Large 3 — ELO 1832, #258
- Chatbot Arena (Vision): 1203.0 (#60/135)
DeepSeek V3 — ELO 1815, #265
- FutureEval: -5.35 (#69/77)
Claude 3.7 Sonnet — ELO 1792, #281
- FutureEval: -2.36 (#65/77)
Llama 3.1 Nemotron Ultra 253B v1 — ELO 1788, #282
- FutureEval: -10.7 (#76/77)
Claude 3.5 Sonnet — ELO 1782, #286
- FutureEval: 3.59 (#42/77)
GPT-4.1 Mini — ELO 1761, #308
- FutureEval: -1.4 (#63/77)
Gemini 2.0 Flash — ELO 1759, #312
- Design Arena (Video Editing): 1417.0 (#1/15)
- FutureEval: 4.18 (#37/77)
GPT-5 Nano — ELO 1754, #319
- FutureEval: -6.06 (#71/77)
Laguna XS.2 — ELO 1747, #322
- Vals AI Harvey Legal Agent Bench: 0.0 (#24/24)
- Vals AI Legal Research Bench: 0.96 (#24/24)
GPT-4o Mini — ELO 1688, #378
- FutureEval: -8.97 (#74/77)
Llama 4 Maverick — ELO 1680, #383
- FutureEval: -5.36 (#70/77)
Laguna M.1 — ELO 1642, #421
- Vals AI Harvey Legal Agent Bench: 0.0 (#20/24)
- Vals AI Legal Research Bench: 2.4 (#23/24)
- Vals AI Finance Agent v2: 25.03 (#36/37)
Qwen 2.5 Max — ELO 1625, #440
- FutureEval: -3.83 (#67/77)
Gemma 4 26B A4B — ELO 1624, #442
- LLM Stats (MRCR v2 (8-needle)): 44.1 (#7/19)
Gemma 3 27B — ELO 1613, #452
- LLM Stats (MRCR v2 (8-needle)): 13.5 (#19/19)
Gemma 4 12B — ELO 1605, #463
- LLM Stats (MRCR v2 (8-needle)): 43.4 (#8/19)
GPT-4.1 Nano — ELO 1458, #699
- FutureEval: -18.82 (#77/77)
Gemma 4 E4B — ELO 1316, #1050
- LLM Stats (MRCR v2 (8-needle)): 25.4 (#15/19)
Gemma 4 E2B — ELO 1115, #1314
- LLM Stats (MRCR v2 (8-needle)): 19.1 (#17/19)

Top-10 New Scores (120)

Claude Fable 5 on Arabic Broad Leaderboard: 8.217 (#40)
Claude Fable 5 on SvelteBench: 100.0 (#1)
Claude Opus 4.8 on Android Bench: 72.4 (#6)
Claude Opus 4.8 on BridgeBench Security: 1001.5 (#2)
Claude Opus 4.8 on SvelteBench: 100.0 (#4)
GPT-5.6 Pro Sol on Arabic Broad Leaderboard: 8.97 (#12)
GPT-5.6 Sol on AI for Education Pedagogy: 91.21 (#3)
GPT-5.6 Sol on AI for Education Pedagogy - Maths: 92.86 (#4)
GPT-5.6 Sol on AI for Education Pedagogy - Primary: 94.37 (#7)
GPT-5.6 Sol on AI for Education Pedagogy - Science: 92.35 (#17)
GPT-5.6 Sol on AI for Education Pedagogy - Secondary: 90.09 (#3)
GPT-5.6 Sol on AI for Education Pedagogy - Social studies: 90.91 (#4)
GPT-5.6 Sol on AI for Education Pedagogy - Technology: 88.68 (#5)
GPT-5.6 Sol on AI for Education SEND: 87.61 (#2)
GPT-5.6 Sol on AI for Education Visual Reasoning: 84.9 (#2)
GPT-5.6 Sol on AI for Education Visual Reasoning - match (figure): 77.0 (#3)
GPT-5.6 Sol on AI for Education Visual Reasoning - pattern completion (2d): 82.2 (#3)
GPT-5.6 Sol on AI for Education Visual Reasoning - reasoning by analogy: 86.5 (#2)
GPT-5.6 Sol on Agent Security League - Functional Correctness: 70.9 (#11)
GPT-5.6 Sol on Agent Security League - Security Correctness: 23.5 (#3)
GPT-5.6 Sol on Agents' Last Exam: 30.6 (#1)
GPT-5.6 Sol on Benchmarks.bio - BioSecBench-Refusal: 39.5 (#5)
GPT-5.6 Sol on Blueprint-Bench 2: 0.336 (#4)
GPT-5.6 Sol on CADGenBench: 0.4108 (#7)
GPT-5.6 Sol on Clerk LLM Leaderboard: 80.5 (#3)
GPT-5.6 Sol on DeepSWE: 72.7 (#1)
GPT-5.6 Sol on EQ-Bench Longform Writing: 81.7 (#3)
GPT-5.6 Sol on HieroglyphBench: 27.6 (#4)
GPT-5.6 Sol on Kaggle Game Arena Poker (Heads Up): 36.05 (#2)
GPT-5.6 Sol on LLM Stats (BrowseComp): 90.4 (#2)
GPT-5.6 Sol on LLM Stats (GDPval-AA): 1747.8 (#2)
GPT-5.6 Sol on LLM Stats (HealthBench Hard): 33.1 (#2)
GPT-5.6 Sol on LLM Stats (HealthBench Professional): 60.5 (#2)
GPT-5.6 Sol on LLM Stats (HealthBench): 57.0 (#4)
GPT-5.6 Sol on LLM Stats (Toolathlon): 58.0 (#4)
GPT-5.6 Sol on React Native Evals: 86.34 (#4)
GPT-5.6 Sol on ZeroEval GPQA Diamond: 94.6 (#1)
Kimi K3 on AA APEX-Agents: 41.3 (#2)
Kimi K3 on AA CritPt: 23.43 (#9)
Kimi K3 on AA GDPval: 1683.65 (#4)
Kimi K3 on AA GPQA Diamond: 93.54 (#4)
Kimi K3 on AA Humanity's Last Exam: 44.35 (#7)
Kimi K3 on AA Long Context Reasoning: 74.67 (#4)
Kimi K3 on AA MMMU-Pro: 80.52 (#13)
Kimi K3 on AA Omniscience: 18.42 (#13)
Kimi K3 on AA Omniscience - Business: 38.8 (#27)
Kimi K3 on AA Omniscience - Health: 37.5 (#44)
Kimi K3 on AA Omniscience - Humanities & Social Sciences: 43.8 (#26)
Kimi K3 on AA Omniscience - Law: 37.7 (#33)
Kimi K3 on AA Omniscience - Science, Engineering & Mathematics: 47.8 (#19)
Kimi K3 on AA Omniscience - Software Engineering (SWE): 70.1 (#25)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - C: 87.0 (#11)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Dart: 62.0 (#23)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Go: 56.0 (#48)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - HTML: 76.0 (#22)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Java: 53.0 (#30)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - JavaScript: 77.27 (#21)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Julia: 48.0 (#47)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Kotlin: 66.0 (#28)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - PHP: 78.0 (#21)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Python: 70.0 (#33)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - R: 62.0 (#20)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Rust: 74.0 (#43)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Swift: 80.0 (#22)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - TypeScript: 74.44 (#26)
Kimi K3 on AA SciCode: 58.68 (#3)
Kimi K3 on AA-Omniscience Accuracy: 45.95 (#25)
Kimi K3 on AI Chess Leaderboard (Continuation): 1351.0 (#18)
Kimi K3 on AI Chess Leaderboard (Reasoning): 1518.0 (#18)
Kimi K3 on Arabic Broad Leaderboard: 8.906 (#18)
Kimi K3 on Artificial Analysis Intelligence Index: 57.11 (#4)
Kimi K3 on BridgeBench Debugging: 1066.7 (#1)
Kimi K3 on BridgeBench Hallucination: 1030.4 (#1)
Kimi K3 on BridgeBench Refactoring: 1065.7 (#1)
Kimi K3 on BridgeBench Security: 1023.1 (#1)
Kimi K3 on Bullshit Benchmark: 58.2 (#23)
Kimi K3 on Chatbot Arena (Text): 1486.0 (#9)
Kimi K3 on DeepSWE: 68.5 (#5)
Kimi K3 on EQ-Bench Longform Writing: 79.6 (#6)
Kimi K3 on GDPval-AA: 1684.0 (#4)
Kimi K3 on HieroglyphBench: 25.9 (#5)
Kimi K3 on KernelBench Hub - Mega: 18.09 (#3)
Kimi K3 on KernelBench Hub - Mega: 18.09 (#3)
Kimi K3 on LLM Stats (CharXiv-R): 91.3 (#2)
Kimi K3 on LLM Stats (FrontierSWE): 81.2 (#2)
Kimi K3 on LLM Stats (GDPval-AA): 1668.0 (#3)
Kimi K3 on LLM Stats (MCP Atlas): 84.2 (#2)
Kimi K3 on LLM Stats (OfficeQA Pro): 63.3 (#4)
Kimi K3 on LLM Stats (Terminal-Bench 2.1): 88.3 (#2)
Kimi K3 on LLM Stats (Toolathlon): 73.2 (#2)
Kimi K3 on LLM Stats (ZEROBench): 41.0 (#3)
Kimi K3 on PM-LLM-Benchmark: 39.2 (#3)
Kimi K3 on RuneBench: 1891.0 (#21)
Kimi K3 on Surface Evolver Bench: 95.0 (#2)
Kimi K3 on Surface Evolver Bench Pass Rate: 87.5 (#2)
Kimi K3 on SvelteBench: 95.6 (#25)
Kimi K3 on Vals AI (Vals Index): 74.7 (#2)
Kimi K3 on Vals AI CorpFin v2: 71.56 (#2)
Kimi K3 on Vals AI CyberBench: 79.03 (#4)
Kimi K3 on Vals AI Excel Modeling: 66.4 (#6)
Kimi K3 on Vals AI Finance Agent v2: 54.36 (#5)
Kimi K3 on Vals AI GPQA: 92.93 (#5)
Kimi K3 on Vals AI Harvey Legal Agent Bench: 10.83 (#4)
Kimi K3 on Vals AI Legal Research Bench: 44.23 (#3)
Kimi K3 on Vals AI LegalBench: 86.02 (#9)
Kimi K3 on Vals AI LiveCodeBench: 87.19 (#9)
Kimi K3 on Vals AI MMLU-Pro: 87.97 (#14)
Kimi K3 on Vals AI MMMU: 88.15 (#6)
Kimi K3 on Vals AI MedCode: 48.88 (#16)
Kimi K3 on Vals AI MedScribe: 87.96 (#4)
Kimi K3 on Vals AI MortgageTax: 66.34 (#30)
Kimi K3 on Vals AI Multimodal Index: 73.42 (#2)
Kimi K3 on Vals AI ProofBench: 70.0 (#5)
Kimi K3 on Vals AI SAGE: 54.26 (#4)
Kimi K3 on Vals AI SWE-bench Verified: 93.4 (#3)
Kimi K3 on Vals AI TaxEval v2: 75.72 (#10)
Kimi K3 on Vals AI Terminal-Bench 2.1: 80.9 (#2)
Kimi K3 on Vals AI Vibe Code Bench: 84.96 (#2)
Kimi K3 on You're Absolutely Right!: 3.5 (#8)
Kimi K3 on ZeroEval GPQA Diamond: 93.5 (#7)

New #1 Leaders (49)

ExploitGym: GPT-5.6 Sol (293.0) beat Claude Mythos Preview by 136.0
GeoBench Photos: Claude Fable 5 (4313.64) beat Gemini 3 Flash (Preview) by 109.38
Design Arena (Video Editing): Gemini 2.0 Flash (1417.0) beat happy-horse-1.0 by 99.0
MineBench: GPT-5.6 Sol (2108.79) beat GPT-5.5 Pro by 73.0
GeoBench ACW: Claude Fable 5 (4403.7) beat Gemini 3 Flash (Preview) by 70.93
Design Arena (3D): Kimi K3 (1441.0) beat GLM-5.2 by 70.0
Design Arena (Data Viz): Kimi K3 (1429.0) beat Claude Fable 5 by 57.0
GDP.pdf: Claude Fable 5 (Thinking, Max) (70.3) beat GPT-5.6 Sol by 39.6
Chatbot Arena (Code): Kimi K3 (1679.0) beat Claude Fable 5 by 30.0
Benchmarks.bio - SpatialBench-Long: GPT-5.6 Sol (38.89) beat Gemini 3.5 Flash by 27.78
ClockBench: GPT-5.6 Sol (Max) (66.7) beat GPT-5.4 (High) by 16.1
LLM Stats (Toolathlon): Muse Spark 1.1 (75.6) beat Claude Opus 4.8 by 15.7
LLM Stats (MRCR v2 (8-needle)): GPT-5.6 Sol (91.5) beat Claude Opus 4.6 by 15.5
LLM Stats (BabyVision): Kimi K3 (85.7) beat Seed 2.1 Pro by 12.0
AI for Education Visual Reasoning - match (process): GPT-5.6 Sol (88.9) beat Gemini 3 Flash by 11.1
LLM Stats (Graphwalks BFS >128k): GPT-5.6 Sol (90.7) beat Claude Mythos Preview by 10.7
Android Bench: Claude Fable 5 (84.5) beat GPT-5.5 by 10.5
AI for Education Visual Reasoning - odd one out: GPT-5.6 Sol (88.3) beat Gemini 3.5 Flash by 7.8
Lean AI Formalization Leaderboard: Seed Prover (ByteDance) (129.0) beat Aristotle (Harmonic) by 7.0
ARC-AGI-3: GPT-5.6 Sol (Max) (7.78) beat Claude Opus 4.8 (High) by 6.26
TrackingAI IQ Test (Offline): GPT-Terra-Ultra (87.5) beat GPT-5 Pro by 6.25
Design Arena (Website): GPT-5.6 Sol (1352.0) beat GLM-5.2 by 6.0
AgenticVBench: GPT-5.6 Sol (38.4) beat Claude Fable 5 by 6.0
LLM Stats (Terminal-Bench 2.1): GPT-5.6 Sol (88.8) beat Claude Fable 5 by 4.5
LLM Stats (MCP Atlas): Muse Spark 1.1 (88.1) beat Seed 2.1 Pro by 4.3
TaxCalcBench: GPT-5.6 Sol Web Search (58.0) beat GPT-5.5 Web Search by 4.0
LLM Stats (APEX-Agents): Kimi K3 (37.6) beat Seed 2.1 Pro by 3.8
WorldArena Track 1 - Motion Quality: AccioWM (93.29) beat SUSWM by 3.53
LLM Stats (MathVision): Kimi K3 (97.8) beat Seed 2.1 Pro by 3.3
WeirdML: Claude Fable 5 (Max) (91.94) beat GPT-5.6 Sol (High) by 3.18
SEAL - AudioMultiChallenge: Inkling (Thinking) (56.64) beat Gemini 3 Pro (Preview) (Thinking) by 1.99
SEAL - AudioMultiChallenge - Text Output: Inkling (Thinking) (56.64) beat Gemini 3 Pro (Preview) (Thinking) by 1.99
LLM Stats (DeepSearchQA): Kimi K3 (95.0) beat Claude Opus 4.8 by 1.9
WorldScore - Dynamic: WorldScape-0.2(MoE) (76.23) beat World Dreamer by 1.88
Vals AI Legal Research Bench: Claude Fable 5 (49.52) beat GPT-5.6 Sol by 1.44
WorldArena Track 1: UNIS (73.64) beat PAIWorld by 1.33
Vals AI Excel Modeling: Claude Fable 5 (73.67) beat GPT-5.6 Sol by 1.33
WorldScore: WorldScape-0.2(MoE) (80.68) beat World Dreamer by 1.24
LLM Stats (BrowseComp): Kimi K3 (91.2) beat GPT-5.5 Pro by 1.1
AI for Education Visual Reasoning - pattern completion (linear): GPT-5.6 Sol (92.3) beat Gemini 3.5 Flash by 0.8
WorldArena Track 1 - Controllability: UNIS (91.6) beat ABot-PhysWorld (text) by 0.76
Terminal-Bench 2.1 (Claude Code): Claude Fable 5 (83.8) beat Claude 5 Fable by 0.7
WorldScore - Static: WorldScape-0.2(MoE) (85.13) beat World Dreamer by 0.61
Tau3 Banking: Kimi K3 (33.4) beat GPT-5.6 Sol (Max) by 0.41
Terminal-Bench 2.1: Claude Fable 5 (83.8) beat GPT-5.5 by 0.4
Agent Arena - Confirmed Success: Claude Fable 5 (High) (17.27) beat MiniMax-M2.7 by 0.15
MMEB Leaderboard: DME-Large (80.24) beat octen-vl-embedding-large by 0.15
Vals AI Public Benefits Bench: Kimi K3 (68.27) beat Claude Opus 4.8 by 0.14
CADGenBench: GPT-5.6 Sol (xHigh) (0.5319) beat GPT-5.5 (xHigh) by 0.07

The Aggregate Digest — 2026-07-18

2026-07-18T06:23:38.281963+00:00

Daily

New Benchmarks (10)

Sense Ruby Navigation - Baseline (Mean cited recall across repositories (%)): leader GPT-5.5 (71.46), 5 models
Sense Ruby Navigation measures how completely coding agents locate must-find code across 13 pinned Ruby and Rails repositories; this board averages exact path-and-line cited recall for the normal-tools baseline arm.
SciDiagramEdit - Semantic Faithfulness (No-skill pairwise win rate (%)): leader GPT-5.5 (74.5), 4 models
SciDiagramEdit measures instruction-driven editing of scientific figures reconstructed from real paper revision pairs; this board reports blind pairwise semantic-faithfulness win rate for the unassisted model backbones.
SciDiagramEdit - Aesthetic Preference (No-skill pairwise win rate (%)): leader GPT-5.5 (46.6), 4 models
SciDiagramEdit measures instruction-driven editing of scientific figures reconstructed from real paper revision pairs; this board reports blind pairwise aesthetic preference for the unassisted model backbones.
SVLAT - Overall (Accuracy (%)): leader Gemini 3.1 Pro (Preview) (88.57), 6 models
Scientific Visualization Literacy Assessment Test (SVLAT) evaluates reading and reasoning over static and animated scientific visualizations; overall accuracy is averaged across the authors\
SVLAT - Image (Accuracy (%)): leader Gemini 3.1 Pro (Preview) (90.86), 6 models
Scientific Visualization Literacy Assessment Test (SVLAT) multiple-choice questions about static scientific visualizations; accuracy is averaged across the authors\
SVLAT - Animation (Accuracy (%)): leader Gemini 3.1 Pro (Preview) (82.86), 6 models
Scientific Visualization Literacy Assessment Test (SVLAT) multiple-choice questions about animated scientific visualizations; accuracy is averaged across the authors\
Semgrep IDOR - Multimodal F1 (F1 score (%)): leader GPT-5.6 Sol (61.7), 6 models
Semgrep researcher-reviewed benchmark of insecure direct object reference detection in representative production applications over ten runs, using the Semgrep Multimodal security workflow and reporting F1.
Semgrep IDOR - Guided Prompt F1 (F1 score (%)): leader GPT-5.6 Sol (38.9), 5 models
Semgrep researcher-reviewed benchmark of insecure direct object reference detection in representative production applications over ten runs, using each LLM alone with a guided prompt and reporting F1.
Frontier Security Agents - Cybench Hard (Success rate (%)): leader GPT-5.5 (94.1), 8 models
Cybench hard-variant offensive-security evaluation of agents solving capture-the-flag challenges, reporting raw success rate at each model\
Frontier Security Agents - BOTS v1 (BOTS score (%)): leader Claude Opus 4.8 (93.9), 8 models
BOTS v1 defensive-security agent evaluation on scored SOC investigation questions in Splunk, reporting official point-weighted score at each model\

New Models (1)

Inkling — ELO 2630, #11
- EQ-Bench Longform Writing: 72.5 (#22/123)
- RuneBench: 1590.0 (#24/33)
- ARC-AGI-2: 36.53 (#47/166)
- ARC-AGI-1: 79.5 (#48/164)

Top-10 New Scores (35)

Claude Opus 4.8 on BridgeBench Security: 1001.5 (#2)
GPT-5.6 Sol on AI for Education Pedagogy: 91.21 (#3)
GPT-5.6 Sol on AI for Education Pedagogy - Maths: 92.86 (#4)
GPT-5.6 Sol on AI for Education Pedagogy - Primary: 94.37 (#7)
GPT-5.6 Sol on AI for Education Pedagogy - Science: 92.35 (#17)
GPT-5.6 Sol on AI for Education Pedagogy - Secondary: 90.09 (#3)
GPT-5.6 Sol on AI for Education Pedagogy - Social studies: 90.91 (#4)
GPT-5.6 Sol on AI for Education Pedagogy - Technology: 88.68 (#5)
GPT-5.6 Sol on AI for Education SEND: 87.61 (#2)
GPT-5.6 Sol on AI for Education Visual Reasoning: 84.9 (#2)
GPT-5.6 Sol on AI for Education Visual Reasoning - match (figure): 77.0 (#3)
GPT-5.6 Sol on AI for Education Visual Reasoning - pattern completion (2d): 82.2 (#3)
GPT-5.6 Sol on AI for Education Visual Reasoning - reasoning by analogy: 86.5 (#2)
GPT-5.6 Sol on Agent Security League - Functional Correctness: 70.9 (#11)
GPT-5.6 Sol on Agent Security League - Security Correctness: 23.5 (#3)
Inkling on ARC-AGI-1: 79.5 (#48)
Inkling on ARC-AGI-2: 36.53 (#47)
Inkling on EQ-Bench Longform Writing: 72.5 (#22)
Inkling on RuneBench: 1590.0 (#24)
Inkling on SvelteBench: 87.8 (#76)
Kimi K3 on BridgeBench Debugging: 1066.7 (#1)
Kimi K3 on BridgeBench Hallucination: 1030.4 (#1)
Kimi K3 on BridgeBench Refactoring: 1065.7 (#1)
Kimi K3 on BridgeBench Security: 1023.1 (#1)
Kimi K3 on DeepSWE: 68.5 (#5)
Kimi K3 on EQ-Bench Longform Writing: 79.6 (#6)
Kimi K3 on RuneBench: 1891.0 (#21)
Kimi K3 on Surface Evolver Bench: 95.0 (#2)
Kimi K3 on Surface Evolver Bench Pass Rate: 87.5 (#2)
Kimi K3 on Vals AI CyberBench: 79.03 (#4)
Kimi K3 on Vals AI Excel Modeling: 66.4 (#5)
Kimi K3 on Vals AI Finance Agent v2: 54.36 (#5)
Kimi K3 on Vals AI ProofBench: 70.0 (#5)
Kimi K3 on Vals AI SWE-bench Verified: 93.4 (#3)
Kimi K3 on Vals AI Vibe Code Bench: 71.06 (#8)

New #1 Leaders (9)

ExploitGym: GPT-5.6 Sol (293.0) beat Claude Mythos Preview by 136.0
AI for Education Visual Reasoning - match (process): GPT-5.6 Sol (88.9) beat Gemini 3 Flash by 11.1
AI for Education Visual Reasoning - odd one out: GPT-5.6 Sol (88.3) beat Gemini 3.5 Flash by 7.8
LLM Stats (MCP Atlas): Muse Spark 1.1 (88.1) beat Kimi K3 by 3.9
LLM Stats (Toolathlon): Muse Spark 1.1 (75.6) beat Kimi K3 by 2.4
AI for Education Visual Reasoning - pattern completion (linear): GPT-5.6 Sol (92.3) beat Gemini 3.5 Flash by 0.8
Terminal-Bench 2.1 (Claude Code): Claude Fable 5 (83.8) beat Claude 5 Fable by 0.7
Terminal-Bench 2.1: Claude Fable 5 (83.8) beat GPT-5.5 by 0.4
CADGenBench: GPT-5.6 Sol (xHigh) (0.5319) beat Archie in Forge by 0.02

The Aggregate Digest — 2026-07-17

2026-07-17T08:41:26.430027+00:00

Daily

New Benchmarks (9)

Benchmarks.bio - BioSecBench-Surveillance (Pass Rate (%)): leader Grok 4.5 (53.3), 15 models
BioSecBench-Surveillance (benchmarks.bio) — biosurveillance analysis over pathogen genomic samples (isolates, wastewater, clinical, air): variant detection, taxonomic classification, antimicrobial-resistance and toxin characterization, and source tracking, scored as pass rate.
Benchmarks.bio - VariantBench (Pass Rate (%)): leader GPT-5.6 Sol (42.1), 11 models
VariantBench (benchmarks.bio) — agentic genetic-variant discovery and interpretation over real sequencing data (short- and long-read WGS and exome across Illumina, PacBio and ONT): small-variant, structural-variant and repeat calling with QC, statistical and population genomics, and clinical/personal genome interpretation, scored as pass rate with deterministic grading.
LLM Stats (PostTrainBench) (Score (%)): leader MiniMax-M3 (37.1), 5 models
LLM Stats (Program Bench) (Score (%)): leader Kimi K3 (77.8), 5 models
LLM Stats (WorldVQA) (Score (%)): leader Qwen 3.7 Plus (61.1), 5 models
Long-Horizon Terminal-Bench (Mean Score (%)): leader Grok 4.5 (50.5), 21 models
Long-Horizon Terminal-Bench (LHTB) — coding and research agent tasks in a real terminal that unfold over hundreds of steps, graded by dense per-step reward rather than single pass/fail; headline is the mean reward across tasks (0-100).
Visual Acuity Test (4K) (Mean Accuracy (%)): leader GPT-5.4 (100.0), 30 models
Visual Acuity Test for LLMs — reading text rendered at progressively smaller sizes inside a 4K image; measures how small a model can still read as mean character accuracy, taking each model's best detail/resolution setting.
RadLE 2.0 (Radiology) (RadLE Score): leader Claude Fable 5 (758.0), 16 models
RadLE 2.0 (crashlab.in) — radiology visual-reasoning cases pitting vision-language models against board-certified radiologists and trainees; models earn an Elo-style RadLE score (human experts sit near 989).
HumainE Leaderboard (HumainE Score): leader Gemini 3.1 Pro (Preview) (35.38), 54 models

New Models (1)

Kimi K3 — ELO 2974, #7
- Chatbot Arena (Code): 1679.0 (#1/98)
- Tau3 Banking: 33.4 (#1/157)
- LLM Stats (APEX-Agents): 37.6 (#1/7)
- LLM Stats (AutomationBench): 30.8 (#1/6)
- LLM Stats (BabyVision): 85.7 (#1/8)
- LLM Stats (BrowseComp): 91.2 (#1/57)
- LLM Stats (DeepSearchQA): 95.0 (#1/8)
- LLM Stats (MathVision): 97.8 (#1/32)
- LLM Stats (MCP Atlas): 84.2 (#1/29)
- LLM Stats (Toolathlon): 73.2 (#1/29)

Top-10 New Scores (58)

GPT-5.6 Sol on Blueprint-Bench 2: 0.336 (#4)
GPT-5.6 Sol on CADGenBench: 0.4108 (#6)
GPT-5.6 Sol on HieroglyphBench: 27.6 (#4)
GPT-5.6 Sol on React Native Evals: 86.34 (#4)
Inkling on Chatbot Arena (Code): 1434.0 (#38)
Inkling on Chatbot Arena (Text): 1441.0 (#67)
Inkling on Vals AI (Vals Index): 49.28 (#26)
Inkling on Vals AI Terminal-Bench 2.1: 47.57 (#34)
Kimi K3 on AA CritPt: 23.43 (#9)
Kimi K3 on AA GDPval: 1668.08 (#4)
Kimi K3 on AA GPQA Diamond: 93.54 (#4)
Kimi K3 on AA Humanity's Last Exam: 44.35 (#7)
Kimi K3 on AA Long Context Reasoning: 74.67 (#4)
Kimi K3 on AA MMMU-Pro: 80.52 (#13)
Kimi K3 on AA Omniscience: 18.42 (#13)
Kimi K3 on AA Omniscience - Business: 38.8 (#27)
Kimi K3 on AA Omniscience - Health: 37.5 (#44)
Kimi K3 on AA Omniscience - Humanities & Social Sciences: 43.8 (#26)
Kimi K3 on AA Omniscience - Law: 37.7 (#33)
Kimi K3 on AA Omniscience - Science, Engineering & Mathematics: 47.8 (#19)
Kimi K3 on AA Omniscience - Software Engineering (SWE): 70.1 (#25)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - C: 87.0 (#11)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Dart: 62.0 (#23)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Go: 56.0 (#48)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - HTML: 76.0 (#22)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Java: 53.0 (#30)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - JavaScript: 77.27 (#21)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Julia: 48.0 (#47)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Kotlin: 66.0 (#28)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - PHP: 78.0 (#21)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Python: 70.0 (#33)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - R: 62.0 (#20)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Rust: 74.0 (#43)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - Swift: 80.0 (#22)
Kimi K3 on AA Omniscience - Software Engineering (SWE) - TypeScript: 74.44 (#26)
Kimi K3 on AA SciCode: 58.68 (#3)
Kimi K3 on AA-Omniscience Accuracy: 45.95 (#25)
Kimi K3 on AI Chess Leaderboard (Continuation): 1800.0 (#3)
Kimi K3 on AI Chess Leaderboard (Reasoning): 1522.0 (#18)
Kimi K3 on Artificial Analysis Intelligence Index: 57.11 (#4)
Kimi K3 on Bullshit Benchmark: 58.2 (#23)
Kimi K3 on Chatbot Arena (Text): 1486.0 (#9)
Kimi K3 on GDPval-AA: 1668.0 (#4)
Kimi K3 on LLM Stats (CharXiv-R): 91.3 (#2)
Kimi K3 on LLM Stats (DeepSWE): 67.5 (#3)
Kimi K3 on LLM Stats (FrontierSWE): 81.2 (#2)
Kimi K3 on LLM Stats (GDPval-AA): 1668.0 (#3)
Kimi K3 on LLM Stats (OfficeQA Pro): 63.3 (#4)
Kimi K3 on LLM Stats (Terminal-Bench 2.1): 88.3 (#2)
Kimi K3 on LLM Stats (ZEROBench): 41.0 (#3)
Kimi K3 on PM-LLM-Benchmark: 39.2 (#3)
Kimi K3 on PinchBench: 30.25 (#58)
Kimi K3 on SvelteBench: 95.6 (#25)
Kimi K3 on Vals AI (Vals Index): 74.7 (#2)
Kimi K3 on Vals AI CorpFin v2: 71.56 (#2)
Kimi K3 on Vals AI MedCode: 48.88 (#16)
Kimi K3 on Vals AI Terminal-Bench 2.1: 80.9 (#2)
Kimi K3 on ZeroEval GPQA Diamond: 93.5 (#7)

New #1 Leaders (14)

Chatbot Arena (Code): Kimi K3 (1679.0) beat GPT-5.6 Sol (xHigh) by 48.0
ClockBench: GPT-5.6 Sol (Max) (66.7) beat GPT-5.4 (High) by 16.1
LLM Stats (Toolathlon): Kimi K3 (73.2) beat Claude Opus 4.8 by 13.3
LLM Stats (AutomationBench): Kimi K3 (30.8) beat GPT-5.6 Sol by 12.7
LLM Stats (BabyVision): Kimi K3 (85.7) beat Seed 2.1 Pro by 12.0
TrackingAI IQ Test (Offline): GPT-Sol-Ultra (87.5) beat GPT-5 Pro by 6.25
AgenticVBench: GPT-5.6 Sol (38.4) beat Claude Fable 5 by 6.0
LLM Stats (APEX-Agents): Kimi K3 (37.6) beat Seed 2.1 Pro by 3.8
LLM Stats (MathVision): Kimi K3 (97.8) beat Seed 2.1 Pro by 3.3
LLM Stats (DeepSearchQA): Kimi K3 (95.0) beat Claude Opus 4.8 by 1.9
LLM Stats (BrowseComp): Kimi K3 (91.2) beat GPT-5.6 Sol by 0.8
Tau3 Banking: Kimi K3 (33.4) beat GPT-5.6 Sol (Max) by 0.41
LLM Stats (MCP Atlas): Kimi K3 (84.2) beat Seed 2.1 Pro by 0.4
CADGenBench: Archie in Forge (0.5073) beat GPT-5.5 (xHigh) by 0.05

The Aggregate Digest — 2026-07-16

2026-07-16T06:33:00.266162+00:00

Daily

New Benchmarks (3)

Epoch AI - Blueprint Bench 2 (Score): leader claude-fable-5_unknown (38.61), 17 models
Epoch AI - Proofbench (Score): leader gpt-5.6-sol_max (77.0), 43 models
Epoch AI - Enigma Eval (Score): leader gpt-5.4-pro-2026-03-05_unknown (23.82), 42 models

New Models (1)

Inkling — ELO 2701, #10
- SEAL - AudioMultiChallenge: 56.64 (#1/32)
- SEAL - AudioMultiChallenge - Text Output: 56.64 (#1/17)

Top-10 New Scores (2)

Claude Fable 5 on SvelteBench: 100.0 (#1)
Claude Opus 4.8 on SvelteBench: 100.0 (#4)

New #1 Leaders (3)

GDP.pdf: Claude Fable 5 (Thinking, Max) (70.3) beat GPT-5.6 Sol by 39.6
SEAL - AudioMultiChallenge: Inkling (56.64) beat Gemini 3 Pro (Preview) (Thinking) by 1.99
SEAL - AudioMultiChallenge - Text Output: Inkling (56.64) beat Gemini 3 Pro (Preview) (Thinking) by 1.99

The Aggregate Digest — 2026-07-15

2026-07-15T06:25:04.585596+00:00

Daily

Top-10 New Scores (6)

Claude Fable 5 on Arabic Broad Leaderboard: 8.217 (#38)
Claude Opus 4.8 on Android Bench: 72.4 (#6)
GPT-5.6 Sol on Benchmarks.bio - BioSecBench-Refusal: 39.5 (#5)
GPT-5.6 Sol on Clerk LLM Leaderboard: 80.5 (#3)
GPT-5.6 Sol on EQ-Bench Longform Writing: 81.7 (#3)
Qwen 3.7 Max on Android Bench: 54.2 (#20)

New #1 Leaders (9)

MineBench: GPT-5.6 Sol (2173.92) beat Claude Fable 5 by 118.26
GeoBench Photos: Claude Fable 5 (4313.64) beat Gemini 3 Flash (Preview) by 109.38
GeoBench ACW: Claude Fable 5 (4403.7) beat Gemini 3 Flash (Preview) by 70.93
Benchmarks.bio - SpatialBench-Long: GPT-5.6 Sol (38.89) beat Gemini 3.5 Flash by 27.78
Android Bench: Claude Fable 5 (84.5) beat GPT-5.5 by 10.5
ARC-AGI-3: GPT-5.6 Sol (Max) (7.78) beat Claude Opus 4.8 (High) by 6.26
WorldScore - Dynamic: WorldScape-0.2(MoE) (76.23) beat World Dreamer by 1.88
WorldScore: WorldScape-0.2(MoE) (80.68) beat World Dreamer by 1.24
WorldScore - Static: WorldScape-0.2(MoE) (85.13) beat World Dreamer by 0.61

The Aggregate Digest — 2026-07-14

2026-07-14T06:21:36.836442+00:00

Daily

Top-10 New Scores (3)

Claude Fable 5 on Chatbot Arena (Search): 1234.0 (#4)
GPT-5.6 Sol on Kaggle Game Arena Poker (Heads Up): 36.05 (#2)
Muse Spark 1.1 on Mercor APEX: 37.1 (#8)

New #1 Leaders (2)

TaxCalcBench: GPT-5.6 Sol Web Search (58.0) beat GPT-5.5 Web Search by 4.0
WorldArena Track 1 - Motion Quality: AccioWM (93.29) beat SUSWM by 3.53

The Aggregate Digest — 2026-07-13

2026-07-13T07:01:39.955504+00:00

Daily

New Benchmarks (4)

LLM Stats (Artificial Analysis) (Score (%)): leader GPT-5.6 Sol (59.0), 5 models
LLM Stats (AutomationBench) (Score (%)): leader GPT-5.6 Sol (18.1), 5 models
LLM Stats (DeepSWE) (Score (%)): leader GPT-5.6 Sol (72.7), 7 models
LLM Stats (GDP.pdf) (Score (%)): leader Claude Sonnet 5 (81.6), 5 models

Top-10 New Scores (10)

GPT-5.6 Sol on Agents' Last Exam: 30.6 (#1)
GPT-5.6 Sol on DeepSWE: 72.7 (#1)
GPT-5.6 Sol on LLM Stats (GDPval-AA): 1747.8 (#2)
GPT-5.6 Sol on LLM Stats (Graphwalks BFS >128k): 77.1 (#2)
GPT-5.6 Sol on LLM Stats (HealthBench Hard): 33.1 (#2)
GPT-5.6 Sol on LLM Stats (HealthBench Professional): 60.5 (#2)
GPT-5.6 Sol on LLM Stats (HealthBench): 57.0 (#3)
GPT-5.6 Sol on LLM Stats (MRCR v2 (8-needle)): 73.8 (#3)
GPT-5.6 Sol on LLM Stats (Toolathlon): 58.0 (#2)
GPT-5.6 Sol on ZeroEval GPQA Diamond: 94.6 (#1)

New #1 Leaders (4)

Design Arena (Video Editing): Gemini 2.0 Flash (1402.0) beat happy-horse-1.0 by 84.0
LLM Stats (Terminal-Bench 2.1): GPT-5.6 Sol (88.8) beat Claude Fable 5 by 4.5
WeirdML: Claude Fable 5 (Max) (91.94) beat GPT-5.6 Sol (High) by 3.18
LLM Stats (BrowseComp): GPT-5.6 Sol (90.4) beat GPT-5.5 Pro by 0.3

The Aggregate Digest — 2026-07-12

2026-07-12T06:50:29.666855+00:00

Daily

New Benchmarks (1)

OpenCompass Language - Chat (Score (%)): leader Doubao-Seed-2-0-Pro-260215 (high) (91.7), 23 models

Top-10 New Scores (10)

Claude Fable 5 on Lean AI Formalization Leaderboard: 4.0 (#13)
Claude Fable 5 on TaxCalcBench: 26.0 (#5)
Claude Fable 5 on TrackingAI IQ Test (Mensa Norway): 60.0 (#20)
Claude Fable 5 on TrackingAI IQ Test (Vision): 78.57 (#3)
Claude Opus 4.8 on TaxCalcBench: 16.0 (#7)
GPT-5.6 Sol on Design Arena (3D): 1370.0 (#2)
GPT-5.6 Sol on Design Arena (Game Dev): 1378.0 (#2)
GPT-5.6 Sol on TaxCalcBench: 26.0 (#4)
GPT-5.6 Sol on VoxelBench: 2306.0 (#1)
Seed 2.1 Pro on BabyVision: 73.7 (#4)

New #1 Leaders (6)

BabyVision: GPT-5.5 (83.6) beat Seed 2.0 Pro (High) by 23.0
Design Arena (UI Components): GPT-5.6 Sol (1413.0) beat Claude Fable 5 by 7.0
GSO-Bench: Claude Opus 4.8 (47.06) beat Claude Opus 4.7 by 2.94
Open Universal Arabic ASR Leaderboard: cohere-transcribe-arabic-07-2026 (25.87) beat omniASR_LLM_7B by 2.45
AMA-Bench - Embodied AI: VeraKV (Qwen3-32B) (58.09) beat GPT-5.2 by 1.65
Open Universal Arabic ASR Leaderboard - CER: cohere-transcribe-arabic-07-2026 (11.8) beat Qwen3-ASR-1.7B by 0.53

Weekly

New Benchmarks (15)

Shader Benchmark (Avg judge score (0-500, render fails count 0)): leader gpt-5.5-high (242.2), 3 models
Shader Benchmark (nbardy) — generating shader programs from natural-language prompts, graded by an LLM judge panel with render failures counting as zero; average judge score (0-500).
KINA (Accuracy (%)): leader Gemini-3.1-Pro-Preview (53.17), 42 models
KINA (Knowledge Index of Noah's Ark) — 899-item knowledge benchmark across 261 fine-grained disciplines with representativeness-aware sampling and tournament-audited annotation; overall accuracy.
Office Comprehension Bench (Domain Q&A Accuracy (%)): leader GPT-5.5 Thinking (59.35), 3 models
Office Comprehension Bench (Microsoft) — Domain Q&A track: expert-level multi-step reasoning over native Word/Excel/PowerPoint files across 12 professional domains; assertion-level accuracy via LLM-judge ensemble.
OmniGameArena - ObstacleRun2D (Normalized game score (%)): leader GPT-5.5 (47.3), 12 models
The \
OmniGameArena - ObstacleRun3D (Normalized game score (%)): leader Claude Sonnet 4.6 (20.0), 12 models
The \
OmniGameArena - LastStand (Normalized game score (%)): leader GPT-5.5 (41.6), 12 models
The \
OmniGameArena - MonsterShoot (Normalized game score (%)): leader Gemini 3.1 Pro Preview (71.0), 12 models
The \
OmniGameArena - SceneEscape (Normalized game score (%)): leader GPT-5.5 (72.0), 12 models
The \
OmniGameArena - CueChase (Normalized game score (%)): leader Claude Opus 4.6 (84.0), 12 models
The \
OmniGameArena - SoloCraft (Normalized game score (%)): leader GPT-5.5 (25.2), 12 models
The \
OmniGameArena - SharedFloor (Normalized game score (%)): leader GPT-5.5 (36.8), 10 models
The \
OmniGameArena - HandoffRun (Normalized game score (%)): leader GPT-5.5 (18.4), 10 models
The \
Poker Arena (Tournament win rate (%)): leader Claude Opus 4.6 (19.4), 7 models
Poker Arena (arXiv) — strategic reasoning, risk assessment and bluffing measured through head-to-head LLM poker tournaments; tournament win rate (%).
ClawArena (Composite Reliability Score): leader Claude Fable-5 (70.05), 20 models
ClawArena (clawarena.cc) — reliability of LLM subagent orchestration under a Claw-style harness; Composite Reliability Score across scripted multi-agent runs.
OpenCompass Language - Chat (Score (%)): leader Doubao-Seed-2-0-Pro-260215 (high) (91.7), 23 models

New Models (131)

Claude Fable 5 — ELO 3083, #6
- Senior SWE-Bench: 29.1 (#1/10)
- KernelBench Hub - Mega: 18.7 (#2/12)
- TrackingAI IQ Test (Vision): 78.57 (#3/19)
- KernelBench Hub - Hard: 24.0 (#4/13)
- TaxCalcBench: 26.0 (#5/9)
- Lean AI Formalization Leaderboard: 4.0 (#13/35)
- TrackingAI IQ Test (Mensa Norway): 60.0 (#20/31)
Claude Opus 4.8 — ELO 2770, #8
- GSO-Bench: 47.06 (#1/30)
- TaxCalcBench: 16.0 (#7/9)
Muse Spark 1.1 — ELO 2766, #9
- SEAL - MultiNRC: 65.59 (#1/44)
- SEAL - VisualToolBench (VTB): 44.77 (#1/21)
- SEAL - SWE-Bench Pro (Public Dataset): 61.5 (#1/25)
- SEAL - SWE-Bench Pro (Private Dataset): 51.5 (#1/14)
- SEAL - Professional Reasoning Benchmark - Finance: 55.01 (#1/29)
- SEAL - Professional Reasoning Benchmark - Legal: 57.05 (#1/29)
- SEAL - MCP Atlas: 88.1 (#1/24)
- Vals AI TaxEval v2: 79.72 (#1/127)
- Vals AI MedScribe: 88.89 (#1/70)
- Vals AI Harvey Legal Agent Bench: 20.0 (#1/19)
GPT-5.6 Sol — ELO 2718, #10
- KernelBench Hub - Hard: 30.0 (#1/13)
- FrontierMath - Tiers 1-3 (v2): 89.12 (#1/36)
- Benchmarks.bio - scBench-Long: 42.5 (#1/17)
- Design Arena (UI Components): 1413.0 (#1/128)
- Epoch AI - ECI: 161.61 (#1/424)
- OTIS Mock AIME 2024-25: 100.0 (#1/153)
- Chess Puzzles (Epoch AI): 64.0 (#1/55)
- Epoch AI - Critpt: 32.3 (#1/70)
- AA GPQA Diamond: 94.14 (#1/537)
- Tau3 Banking: 32.99 (#1/150)
Seed 2.1 Pro — ELO 2674, #11
- BabyVision: 73.7 (#4/24)
- Agents' Last Exam: 19.5 (#11/20)
GPT-5.5 — ELO 2608, #12
- BabyVision: 83.6 (#1/24)
- TaxCalcBench: 24.0 (#6/9)
Qwen 3.7 Max — ELO 2577, #13
- Agent Arena - Tool Hallucination: 1.02 (#14/32)
- Agent Arena - Bash Recovery: 7.91 (#20/32)
- Agent Arena - Steerability: 5.2 (#22/32)
- Agent Arena - Confirmed Success: 1.43 (#29/32)
- Agent Arena: 0.36 (#30/32)
- Agent Arena - Praise vs Complaint: 0.52 (#31/32)
Gemini 3.1 Pro (Preview) — ELO 2507, #15
- TaxCalcBench: 2.0 (#9/9)
- Agents' Last Exam: 15.8 (#12/20)
Claude Sonnet 5 — ELO 2441, #19
- You're Absolutely Right!: 3.875 (#4/21)
- KernelBench Hub - Hard: 23.0 (#5/13)
- GSO-Bench: 37.25 (#5/30)
- KernelBench Hub - Mega: 4.0 (#6/12)
- EQ-Bench Longform Writing: 78.3 (#7/117)
- Benchmarks.bio - scBench-Long: 14.29 (#8/17)
- TaxCalcBench: 6.0 (#8/9)
- Kaggle Game Arena Poker (Heads Up): 2.21 (#9/20)
- Bullshit Benchmark: 65.5 (#14/158)
- SimpleBench: 60.6 (#19/86)
Gemini 3 Pro — ELO 2412, #22
- Chatbot Arena (Image Edit): 1388.0 (#7/52)
- Chatbot Arena (Text-to-Image): 1245.0 (#8/74)
Grok 4.5 — ELO 2403, #24
- SWE-Marathon: 29.0 (#1/12)
- Vals AI Harvey Legal Agent Bench: 12.92 (#2/19)
- RuneBench: 5682.0 (#3/30)
- Lynchmark: 83.33 (#4/10)
- Chatbot Arena (Code): 1566.0 (#4/96)
- Vals AI SWE-bench Verified: 86.6 (#4/69)
- VoxelBench: 1775.0 (#5/45)
- Senior SWE-Bench: 17.2 (#5/10)
- Vals AI MedScribe: 86.88 (#5/70)
- Vals AI GPQA: 92.93 (#5/121)
GLM-5.2 — ELO 2398, #25
- Design Arena (Website): 1346.0 (#1/152)
- Design Arena (3D): 1371.0 (#1/125)
- Design Arena (Game Dev): 1354.0 (#3/138)
- Agent Security League - Functional Correctness: 82.5 (#4/17)
- Design Arena (UI Components): 1335.0 (#5/128)
- Design Arena (Data Viz): 1318.0 (#6/132)
- Agent Security League - Security Correctness: 12.0 (#12/17)
- Riemann-bench: 10.4 (#16/20)
- SWE-rebench: 51.09 (#29/91)
- LisanBench: 0.0161 (#89/150)
GPT-5.6 Terra — ELO 2380, #27
- Benchmarks.bio - scBench-Long: 25.4 (#2/17)
- AA CritPt: 30.0 (#2/443)
- Vals AI ProofBench: 71.0 (#3/49)
- Vals AI Excel Modeling: 67.61 (#3/21)
- FrontierMath - Tiers 1-3 (v2): 85.96 (#4/36)
- OTIS Mock AIME 2024-25: 99.72 (#4/153)
- Epoch AI - Critpt: 30.0 (#4/70)
- Tau3 Banking: 31.75 (#4/150)
- Gert Labs Rankings: 66.95 (#4/82)
- Vals AI Legal Research Bench: 40.87 (#4/18)
GPT-5 — ELO 2356, #29
- Epoch AI - Scicode: 42.94 (#47/69)
Gemini 3.1 Flash — ELO 2324, #31
- Chatbot Arena (Text-to-Image): 1261.0 (#5/74)
- Chatbot Arena (Image Edit): 1385.0 (#9/52)
Qwen 3.7 Plus — ELO 2317, #32
- Agent Arena - Tool Hallucination: 0.63 (#4/32)
- Design Arena (Website): 1287.0 (#21/152)
- Agent Arena - Steerability: 5.2 (#23/32)
- Agent Arena - Bash Recovery: 5.9 (#24/32)
- Agent Arena - Confirmed Success: 1.87 (#27/32)
- Agent Arena - Praise vs Complaint: 1.63 (#27/32)
- Agent Arena: 0.43 (#29/32)
Nex N2 Pro — ELO 2290, #36
- Wolfram LLM Benchmarking Project: 61.3 (#47/500)
GPT-5.6 Luna — ELO 2265, #41
- Vals AI IOI: 72.92 (#2/58)
- Vals AI Terminal-Bench 2.1: 79.03 (#3/41)
- Benchmarks.bio - scBench-Long: 19.05 (#5/17)
- OTIS Mock AIME 2024-25: 98.33 (#6/153)
- Vals AI Vibe Code Bench: 77.06 (#6/72)
- GDP.pdf: 22.7 (#6/17)
- FrontierMath - Tiers 1-3 (v2): 82.11 (#7/36)
- AA Long Context Reasoning: 74.0 (#7/465)
- FrontierMath - Tier 4 (v2): 60.98 (#8/38)
- AA GDPval: 1591.81 (#8/148)
Gemini 2.5 Pro — ELO 2263, #42
- MCPMark: 15.75 (#29/39)
Kimi K2.7 Code — ELO 2238, #46
- Design Arena (SVG): 1237.0 (#24/99)
- SimpleBench: 57.9 (#24/86)
DeepSeek V4 Pro — ELO 2230, #47
- Agent Arena - Tool Hallucination: 0.99 (#11/32)
- Agent Arena - Confirmed Success: 2.34 (#24/32)
- Agent Arena - Steerability: 4.61 (#24/32)
- Agent Arena - Bash Recovery: 4.94 (#25/32)
- Agent Arena - Praise vs Complaint: 3.13 (#26/32)
- Agent Arena: 0.83 (#28/32)
- SWE-rebench: 42.7 (#45/91)
MiniMax-M3 — ELO 2228, #48
- Agent Arena - Tool Hallucination: 0.99 (#12/32)
- Agent Arena - Confirmed Success: 7.66 (#14/32)
- Agent Arena - Praise vs Complaint: 9.93 (#16/32)
- Agent Arena - Steerability: 5.24 (#21/32)
- Agent Arena - Bash Recovery: 6.46 (#22/32)
- Agent Arena: 3.08 (#23/32)
- SimpleBench: 45.8 (#43/86)
- Epoch AI - ECI: 144.9 (#182/424)
MiMo-V2.5-Pro — ELO 2204, #58
- Agent Arena - Tool Hallucination: 0.63 (#5/32)
- Agent Arena - Praise vs Complaint: 10.28 (#15/32)
- Agent Arena - Confirmed Success: 6.18 (#19/32)
- Agent Arena - Steerability: 5.48 (#20/32)
- Agent Arena: 3.71 (#22/32)
- Agent Arena - Bash Recovery: 2.77 (#30/32)
- SWE-rebench: 42.36 (#46/91)
DeepSeek V4 Flash — ELO 2201, #59
- Agent Arena - Tool Hallucination: 0.6 (#3/32)
- Agent Arena - Confirmed Success: 9.37 (#8/32)
- Agent Arena - Praise vs Complaint: 10.65 (#14/32)
- Agent Arena - Steerability: 7.33 (#16/32)
- Agent Arena: 4.65 (#21/32)
- Agent Arena - Bash Recovery: 4.71 (#27/32)
- SWE-rebench: 38.36 (#51/91)
Kimi K2.6 — ELO 2171, #63
- BabyVision: 68.5 (#6/24)
- Agent Arena - Steerability: 7.33 (#15/32)
- Agent Arena - Bash Recovery: 8.23 (#19/32)
- Agent Arena - Praise vs Complaint: 3.89 (#24/32)
- Agent Arena: 1.61 (#25/32)
- Agent Arena - Confirmed Success: 2.34 (#25/32)
- Agent Arena - Tool Hallucination: 1.24 (#30/32)
GLM-5.1 — ELO 2136, #73
- Agent Arena: 1.57 (#26/32)
- Agent Arena - Confirmed Success: 1.67 (#28/32)
- Agent Arena - Praise vs Complaint: 0.8 (#28/32)
- Agent Arena - Bash Recovery: 4.07 (#28/32)
- Agent Arena - Tool Hallucination: 1.24 (#28/32)
- SimpleBench: 55.1 (#28/86)
- Agent Arena - Steerability: 0.05 (#32/32)
Grok 4.3 — ELO 2127, #77
- Agent Arena: 15.57 (#1/32)
- Agent Arena - Bash Recovery: 43.1 (#1/32)
- Agent Arena - Praise vs Complaint: 16.01 (#3/32)
- Agent Arena - Confirmed Success: 11.3 (#5/32)
- Agent Arena - Tool Hallucination: 0.91 (#10/32)
- Agent Arena - Steerability: 8.34 (#13/32)
- RuneBench: 1609.0 (#20/30)
Claude Sonnet 4.5 — ELO 2091, #93
- MCPMark: 32.09 (#9/39)
Qwen 3 Max — ELO 2090, #95
- MCPMark: 17.72 (#25/39)
MiMo-V2-Pro — ELO 2068, #98
- AI Chess Leaderboard (Reasoning): 756.0 (#101/292)
- AI Chess Leaderboard (Continuation): 591.0 (#112/237)
Qwen 3.6 27B — ELO 2060, #102
- SWE-rebench: 36.55 (#58/91)
Grok 4 — ELO 2058, #104
- MCPMark: 31.69 (#10/39)
GLM-5V Turbo — ELO 2050, #107
- Chatbot Arena (Code): 1402.0 (#44/96)
- Chatbot Arena (Text): 1440.0 (#66/374)
Claude Sonnet 4 — ELO 2036, #109
- MCPMark: 28.15 (#15/39)
Grok 4 Fast — ELO 2031, #111
- MCPMark: 24.02 (#20/39)
Claude Opus 4.1 — ELO 2024, #113
- MCPMark: 29.92 (#12/39)
GPT-5 Mini — ELO 2016, #119
- Epoch AI - Scicode: 39.24 (#55/69)
O4 Mini — ELO 2014, #121
- MCPMark: 17.32 (#26/39)
MiMo-V2.5 — ELO 2007, #125
- AA GDPval: 1145.01 (#50/148)
- GDPval-AA: 1145.0 (#54/153)
- Tau3 Banking: 6.6 (#102/150)
Mercury 2 — ELO 1980, #139
- Design Arena (SVG): 1024.0 (#93/99)
Kimi K2 0905 — ELO 1978, #142
- MCPMark: 21.85 (#21/39)
GPT-5.4 Mini — ELO 1972, #148
- BenchLM: 65.0 (#33/79)
MiniMax-M2.7 — ELO 1967, #151
- Agent Arena - Confirmed Success: 17.12 (#1/32)
- Agent Arena - Steerability: 16.42 (#2/32)
- Agent Arena: 10.18 (#5/32)
- Agent Arena - Praise vs Complaint: 14.49 (#5/32)
- Agent Arena - Tool Hallucination: 0.89 (#9/32)
- Agent Arena - Bash Recovery: 3.74 (#29/32)
GPT-4o ChatGPT — ELO 1964, #155
- LiveBench Table Join: 45.3 (#1/73)
- LiveBench AMPS Hard: 48.0 (#2/73)
- LiveBench Zebra Puzzle: 46.0 (#2/73)
- LiveBench Connections: 51.33 (#3/73)
- LiveBench Math Comp: 44.79 (#5/73)
- LiveBench Story Generation: 79.92 (#6/73)
- LiveBench Olympiad: 63.78 (#9/73)
- LiveBench Plot Unscrambling: 42.53 (#9/73)
- LiveBench Summarize: 71.42 (#10/73)
- LiveBench Typos: 56.0 (#10/73)
Gemini 2.5 Flash — ELO 1955, #161
- MCPMark: 9.06 (#31/39)
Hy3 — ELO 1946, #168
- AI Chess Leaderboard (Reasoning): 1540.0 (#15/292)
- AI Chess Leaderboard (Continuation): 1021.0 (#38/237)
- Gert Labs Rankings: 41.68 (#56/82)
- Design Arena (Game Dev): 1202.0 (#61/138)
- SvelteBench: 88.9 (#66/145)
- Design Arena (Website): 1198.0 (#74/152)
GLM-4.5 — ELO 1932, #173
- MCPMark: 15.55 (#30/39)
Claude 3.5 Sonnet (20240620) — ELO 1925, #177
- LiveBench AMPS Hard: 49.0 (#1/73)
- LiveBench Plot Unscrambling: 55.14 (#1/73)
- LiveBench Zebra Puzzle: 48.0 (#1/73)
- LiveBench Olympiad: 67.46 (#2/73)
- LiveBench Table Reformat: 68.0 (#2/73)
- LiveBench Typos: 66.0 (#2/73)
- LiveBench Table Join: 40.22 (#3/73)
- LiveBench Connections: 49.67 (#4/73)
- LiveBench Math Comp: 44.79 (#6/73)
- LiveBench Paraphrase: 72.22 (#8/73)
DeepSeek V3.1 Terminus — ELO 1924, #181
- MCPMark: 16.54 (#28/39)
GPT-4.1 — ELO 1923, #183
- MCPMark: 8.07 (#33/39)
Qwen 3 Coder Plus — ELO 1904, #196
- MCPMark: 24.8 (#19/39)
qwen3.5-flash — ELO 1879, #217
- AI Chess Leaderboard (Continuation): 437.0 (#182/237)
GPT-4o (2024-05-13) — ELO 1871, #219
- LiveBench Connections: 53.0 (#2/73)
- LiveBench Typos: 66.0 (#3/73)
- LiveBench AMPS Hard: 45.0 (#4/73)
- LiveBench Olympiad: 65.06 (#4/73)
- LiveBench Summarize: 74.62 (#6/73)
- LiveBench Plot Unscrambling: 42.81 (#8/73)
- LiveBench Paraphrase: 70.8 (#10/73)
- LiveBench Zebra Puzzle: 40.0 (#10/73)
- LiveBench Math Comp: 39.58 (#11/73)
- LiveBench Table Join: 29.22 (#12/73)
Gemini 1.5 Pro (Preview 0801) — ELO 1869, #222
- LiveBench Summarize: 83.13 (#2/73)
- LiveBench Table Reformat: 68.0 (#3/73)
- LiveBench Paraphrase: 78.63 (#4/73)
- LiveBench Simplify: 76.5 (#5/73)
- LiveBench Typos: 62.0 (#5/73)
- LiveBench AMPS Hard: 43.0 (#6/73)
- LiveBench Plot Unscrambling: 41.87 (#10/73)
- LiveBench Olympiad: 61.88 (#11/73)
- LiveBench Story Generation: 77.08 (#11/73)
- LiveBench Math Comp: 37.5 (#12/73)
Gemini 1.5 Pro (Preview 0827) — ELO 1866, #227
- LiveBench Math Comp: 54.17 (#1/73)
- LiveBench Olympiad: 68.66 (#1/73)
- LiveBench AMPS Hard: 46.0 (#3/73)
- LiveBench Simplify: 77.8 (#3/73)
- LiveBench Table Reformat: 68.0 (#4/73)
- LiveBench Paraphrase: 77.58 (#5/73)
- LiveBench Typos: 62.0 (#6/73)
- LiveBench Plot Unscrambling: 43.44 (#7/73)
- LiveBench Summarize: 73.4 (#7/73)
- LiveBench Connections: 42.5 (#8/73)
GPT-4 Turbo — ELO 1858, #231
- LiveBench Connections: 44.17 (#5/73)
- LiveBench Math Comp: 44.79 (#7/73)
- LiveBench Simplify: 74.23 (#7/73)
- LiveBench Story Generation: 79.92 (#7/73)
- LiveBench AMPS Hard: 42.0 (#8/73)
- LiveBench Table Join: 35.96 (#8/73)
- LiveBench Olympiad: 60.19 (#13/73)
- LiveBench Table Reformat: 64.0 (#13/73)
- LiveBench Typos: 52.0 (#13/73)
- LiveBench Plot Unscrambling: 39.62 (#14/73)
Qwen3 Coder Next — ELO 1857, #233
- Tau3 Banking: 5.15 (#119/150)
Qwen 3.6 35B A3B — ELO 1855, #236
- SWE-rebench: 33.82 (#65/91)
GPT-4o (2024-08-06) — ELO 1839, #245
- LiveBench Connections: 58.0 (#1/73)
- LiveBench Olympiad: 65.96 (#3/73)
- LiveBench Math Comp: 47.92 (#4/73)
- LiveBench Table Join: 38.68 (#4/73)
- LiveBench Plot Unscrambling: 45.11 (#5/73)
- LiveBench Story Generation: 80.08 (#5/73)
- LiveBench Summarize: 75.78 (#5/73)
- LiveBench AMPS Hard: 43.0 (#7/73)
- LiveBench Typos: 60.0 (#8/73)
- LiveBench Zebra Puzzle: 42.0 (#8/73)
GPT-4 Preview (0125) — ELO 1833, #247
- LiveBench Table Reformat: 70.0 (#1/73)
- LiveBench Zebra Puzzle: 44.0 (#4/73)
- LiveBench Plot Unscrambling: 43.49 (#6/73)
- LiveBench Connections: 43.17 (#7/73)
- LiveBench Table Join: 36.18 (#7/73)
- LiveBench AMPS Hard: 37.0 (#14/73)
- LiveBench Math Comp: 36.46 (#18/73)
- LiveBench Summarize: 65.68 (#19/73)
- LiveBench Typos: 44.0 (#21/73)
- LiveBench Olympiad: 54.79 (#22/73)
Gemini 1.5 Flash (Preview 0827) — ELO 1832, #249
- LiveBench Simplify: 84.75 (#1/73)
- LiveBench Story Generation: 81.33 (#1/73)
- LiveBench Paraphrase: 73.58 (#6/73)
- LiveBench Summarize: 72.77 (#9/73)
- LiveBench Olympiad: 63.4 (#10/73)
- LiveBench Table Reformat: 64.0 (#11/73)
- LiveBench Math Comp: 36.46 (#16/73)
- LiveBench Plot Unscrambling: 34.63 (#18/73)
- LiveBench Connections: 30.5 (#19/73)
- LiveBench Table Join: 21.6 (#20/73)
Grok Code Fast 1 — ELO 1780, #279
- MCPMark: 20.47 (#23/39)
DeepSeek V3 Chat — ELO 1774, #288
- MCPMark: 16.73 (#27/39)
Llama 3.1 70B Instruct — ELO 1764, #297
- LiveBench Paraphrase: 82.25 (#1/73)
- LiveBench Summarize: 83.85 (#1/73)
- LiveBench Table Reformat: 68.0 (#5/73)
- LiveBench Simplify: 74.8 (#6/73)
- LiveBench Connections: 41.67 (#9/73)
- LiveBench Table Join: 30.88 (#9/73)
- LiveBench Math Comp: 41.67 (#10/73)
- LiveBench Olympiad: 60.06 (#14/73)
- LiveBench Story Generation: 75.42 (#14/73)
- LiveBench AMPS Hard: 35.0 (#15/73)
GPT-4.1 Mini — ELO 1756, #303
- MCPMark: 3.94 (#38/39)
Llama 3.1 405B Instruct — ELO 1724, #336
- LiveBench Paraphrase: 81.13 (#2/73)
- LiveBench Simplify: 78.33 (#2/73)
- LiveBench Plot Unscrambling: 47.38 (#3/73)
- LiveBench Summarize: 78.93 (#3/73)
- LiveBench Connections: 44.17 (#6/73)
- LiveBench Table Join: 36.52 (#6/73)
- LiveBench Olympiad: 64.15 (#8/73)
- LiveBench Typos: 58.0 (#9/73)
- LiveBench AMPS Hard: 38.0 (#12/73)
- LiveBench Math Comp: 37.5 (#13/73)
Claude 3 Opus (20240229) — ELO 1713, #345
- LiveBench Typos: 68.0 (#1/73)
- LiveBench Plot Unscrambling: 50.16 (#2/73)
- LiveBench Table Join: 40.96 (#2/73)
- LiveBench Math Comp: 41.67 (#9/73)
- LiveBench Table Reformat: 64.0 (#9/73)
- LiveBench AMPS Hard: 38.0 (#11/73)
- LiveBench Simplify: 72.55 (#11/73)
- LiveBench Summarize: 70.48 (#11/73)
- LiveBench Connections: 37.0 (#12/73)
- LiveBench Paraphrase: 69.38 (#13/73)
Mistral Large 2 (Jul) — ELO 1702, #357
- LiveBench Olympiad: 64.41 (#6/73)
- LiveBench Story Generation: 78.5 (#8/73)
- LiveBench Simplify: 73.13 (#9/73)
- LiveBench Plot Unscrambling: 40.05 (#12/73)
- LiveBench Typos: 50.0 (#14/73)
- LiveBench Summarize: 68.15 (#15/73)
- LiveBench Table Reformat: 64.0 (#17/73)
- LiveBench Table Join: 21.82 (#19/73)
- LiveBench Connections: 29.33 (#20/73)
- LiveBench Paraphrase: 67.6 (#20/73)
GPT-4 (0613) — ELO 1698, #363
- LiveBench Plot Unscrambling: 45.54 (#4/73)
- LiveBench Typos: 64.0 (#4/73)
- LiveBench Zebra Puzzle: 42.0 (#7/73)
- LiveBench Summarize: 73.33 (#8/73)
- LiveBench Connections: 39.17 (#10/73)
- LiveBench Story Generation: 76.08 (#12/73)
- LiveBench Simplify: 70.97 (#14/73)
- LiveBench Olympiad: 58.7 (#18/73)
- LiveBench Table Join: 20.08 (#21/73)
- LiveBench Paraphrase: 66.77 (#23/73)
DeepSeek Coder V2 — ELO 1696, #367
- LiveBench Math Comp: 50.0 (#3/73)
- LiveBench AMPS Hard: 43.0 (#5/73)
- LiveBench Olympiad: 64.61 (#5/73)
- LiveBench Zebra Puzzle: 42.0 (#5/73)
- LiveBench Table Join: 30.76 (#10/73)
- LiveBench Story Generation: 75.33 (#15/73)
- LiveBench Summarize: 64.7 (#21/73)
- LiveBench Typos: 42.0 (#22/73)
- LiveBench Plot Unscrambling: 31.3 (#23/73)
- LiveBench Connections: 25.83 (#25/73)
Qwen 2 72B Instruct — ELO 1689, #376
- LiveBench Story Generation: 80.17 (#4/73)
- LiveBench Zebra Puzzle: 40.0 (#13/73)
- LiveBench Math Comp: 37.5 (#14/73)
- LiveBench Connections: 34.0 (#15/73)
- LiveBench Olympiad: 59.81 (#16/73)
- LiveBench Paraphrase: 68.73 (#16/73)
- LiveBench AMPS Hard: 33.0 (#20/73)
- LiveBench Plot Unscrambling: 33.64 (#21/73)
- LiveBench Summarize: 62.02 (#30/73)
- LiveBench Simplify: 62.17 (#31/73)
Qwen 2.5 Max — ELO 1652, #417
- AI Chess Leaderboard (Reasoning): 667.0 (#138/292)
- AI Chess Leaderboard (Continuation): 414.0 (#194/237)
Gemma 3 12B (IT) — ELO 1639, #425
- GDPval-AA: 144.0 (#139/153)
- AA GDPval: -143.64 (#146/148)
Phi-3.5-MoE-instruct — ELO 1622, #444
- LiveBench AMPS Hard: 32.0 (#21/73)
- LiveBench Paraphrase: 66.3 (#24/73)
- LiveBench Table Reformat: 58.0 (#24/73)
- LiveBench Math Comp: 26.04 (#26/73)
- LiveBench Simplify: 63.28 (#28/73)
- LiveBench Zebra Puzzle: 34.0 (#31/73)
- LiveBench Typos: 26.0 (#35/73)
- LiveBench Olympiad: 41.87 (#36/73)
- LiveBench Summarize: 51.6 (#44/73)
- LiveBench Connections: 11.0 (#45/73)
Gemma 2 27B (IT) — ELO 1598, #468
- LiveBench Zebra Puzzle: 42.0 (#6/73)
- LiveBench Paraphrase: 71.13 (#9/73)
- LiveBench Table Reformat: 64.0 (#12/73)
- LiveBench AMPS Hard: 33.0 (#19/73)
- LiveBench Summarize: 64.42 (#22/73)
- LiveBench Olympiad: 53.83 (#24/73)
- LiveBench Typos: 42.0 (#24/73)
- LiveBench Connections: 25.17 (#26/73)
- LiveBench Plot Unscrambling: 30.03 (#26/73)
- LiveBench Simplify: 64.52 (#26/73)
DeepSeek V2.5 — ELO 1593, #474
- LiveBench Math Comp: 53.12 (#2/73)
- LiveBench Olympiad: 64.29 (#7/73)
- LiveBench AMPS Hard: 40.0 (#10/73)
- LiveBench Story Generation: 77.17 (#10/73)
- LiveBench Table Reformat: 64.0 (#10/73)
- LiveBench Paraphrase: 68.72 (#17/73)
- LiveBench Plot Unscrambling: 35.38 (#17/73)
- LiveBench Table Join: 24.34 (#17/73)
- LiveBench Summarize: 65.12 (#20/73)
- LiveBench Connections: 28.17 (#22/73)
Llama 3 70B Instruct — ELO 1580, #490
- LiveBench Table Join: 25.24 (#16/73)
- LiveBench Connections: 31.17 (#17/73)
- LiveBench Simplify: 67.13 (#18/73)
- LiveBench Olympiad: 54.11 (#23/73)
- LiveBench Plot Unscrambling: 31.15 (#24/73)
- LiveBench Story Generation: 69.75 (#25/73)
- LiveBench Typos: 40.0 (#26/73)
- LiveBench Table Reformat: 52.0 (#29/73)
- LiveBench Summarize: 59.25 (#33/73)
- LiveBench AMPS Hard: 22.0 (#35/73)
Gemini 1.5 Flash (0514) — ELO 1572, #509
- LiveBench Olympiad: 61.67 (#12/73)
- LiveBench Zebra Puzzle: 38.0 (#16/73)
- LiveBench Typos: 46.0 (#17/73)
- LiveBench Table Join: 24.08 (#18/73)
- LiveBench Simplify: 66.77 (#20/73)
- LiveBench AMPS Hard: 30.0 (#25/73)
- LiveBench Table Reformat: 54.0 (#25/73)
- LiveBench Paraphrase: 66.1 (#27/73)
- LiveBench Plot Unscrambling: 29.41 (#27/73)
- LiveBench Math Comp: 25.0 (#28/73)
GPT-4o Mini (2024-07-18) — ELO 1568, #515
- LiveBench AMPS Hard: 41.0 (#9/73)
- LiveBench Connections: 36.5 (#14/73)
- LiveBench Typos: 46.0 (#18/73)
- LiveBench Math Comp: 35.42 (#19/73)
- LiveBench Table Reformat: 60.0 (#21/73)
- LiveBench Zebra Puzzle: 36.0 (#21/73)
- LiveBench Paraphrase: 67.0 (#22/73)
- LiveBench Simplify: 65.53 (#23/73)
- LiveBench Story Generation: 71.0 (#23/73)
- LiveBench Olympiad: 48.32 (#26/73)
Gemini 1.5 Pro (0514) — ELO 1560, #527
- LiveBench Table Join: 38.42 (#5/73)
- LiveBench Table Reformat: 66.0 (#7/73)
- LiveBench Typos: 56.0 (#11/73)
- LiveBench Plot Unscrambling: 39.92 (#13/73)
- LiveBench Simplify: 71.05 (#13/73)
- LiveBench Paraphrase: 69.27 (#14/73)
- LiveBench AMPS Hard: 34.0 (#17/73)
- LiveBench Math Comp: 36.46 (#17/73)
- LiveBench Olympiad: 56.8 (#20/73)
- LiveBench Summarize: 63.8 (#24/73)
Mistral Large — ELO 1540, #564
- LiveBench Zebra Puzzle: 40.0 (#11/73)
- LiveBench Summarize: 69.23 (#13/73)
- LiveBench Table Reformat: 64.0 (#16/73)
- LiveBench Simplify: 67.27 (#17/73)
- LiveBench Plot Unscrambling: 33.9 (#20/73)
- LiveBench Story Generation: 72.92 (#20/73)
- LiveBench Connections: 28.33 (#21/73)
- LiveBench Math Comp: 28.12 (#22/73)
- LiveBench Table Join: 15.66 (#25/73)
- LiveBench Olympiad: 46.49 (#28/73)
Qwen 1.5 110B Chat — ELO 1511, #607
- LiveBench AMPS Hard: 23.0 (#33/73)
- LiveBench Connections: 20.33 (#34/73)
- LiveBench Table Join: 10.36 (#35/73)
- LiveBench Olympiad: 39.16 (#38/73)
- LiveBench Summarize: 55.52 (#38/73)
- LiveBench Paraphrase: 59.52 (#41/73)
- LiveBench Story Generation: 59.25 (#42/73)
- LiveBench Zebra Puzzle: 28.0 (#44/73)
- LiveBench Plot Unscrambling: 15.34 (#46/73)
- LiveBench Math Comp: 16.67 (#47/73)
Mixtral 8x22B Instruct (v0.1) — ELO 1507, #614
- LiveBench Simplify: 66.87 (#19/73)
- LiveBench Table Join: 19.0 (#23/73)
- LiveBench Summarize: 63.62 (#26/73)
- LiveBench Connections: 25.17 (#27/73)
- LiveBench Plot Unscrambling: 28.27 (#28/73)
- LiveBench AMPS Hard: 27.0 (#29/73)
- LiveBench Story Generation: 68.67 (#29/73)
- LiveBench Zebra Puzzle: 34.0 (#29/73)
- LiveBench Olympiad: 43.41 (#34/73)
- LiveBench Typos: 26.0 (#34/73)
Phi-3-medium-4k-instruct — ELO 1480, #664
- LiveBench Math Comp: 27.08 (#24/73)
- LiveBench AMPS Hard: 29.0 (#26/73)
- LiveBench Zebra Puzzle: 30.0 (#39/73)
- LiveBench Connections: 14.0 (#40/73)
- LiveBench Olympiad: 37.99 (#40/73)
- LiveBench Simplify: 55.78 (#42/73)
- LiveBench Table Reformat: 36.0 (#43/73)
- LiveBench Plot Unscrambling: 15.73 (#44/73)
- LiveBench Paraphrase: 56.47 (#47/73)
- LiveBench Table Join: 4.88 (#47/73)
Ling-2.6-flash — ELO 1475, #680
- Tau3 Banking: 2.89 (#137/150)
Claude 3 Opus — ELO 1468, #691
- BabyVision: 81.2 (#2/24)
Command-R+ (08-2024) — ELO 1468, #692
- LiveBench Zebra Puzzle: 38.0 (#15/73)
- LiveBench Summarize: 65.87 (#18/73)
- LiveBench Typos: 44.0 (#20/73)
- LiveBench Table Reformat: 52.0 (#27/73)
- LiveBench Connections: 23.67 (#28/73)
- LiveBench Simplify: 62.85 (#29/73)
- LiveBench Math Comp: 23.96 (#30/73)
- LiveBench Paraphrase: 64.43 (#30/73)
- LiveBench Plot Unscrambling: 25.19 (#31/73)
- LiveBench Table Join: 11.76 (#31/73)
Phi-3-small-8k-instruct — ELO 1455, #715
- LiveBench Zebra Puzzle: 40.0 (#12/73)
- LiveBench Connections: 19.5 (#35/73)
- LiveBench Story Generation: 64.08 (#36/73)
- LiveBench Table Reformat: 42.0 (#38/73)
- LiveBench Math Comp: 18.75 (#41/73)
- LiveBench Summarize: 53.22 (#41/73)
- LiveBench Plot Unscrambling: 15.89 (#42/73)
- LiveBench Olympiad: 36.44 (#43/73)
- LiveBench Paraphrase: 57.93 (#45/73)
- LiveBench AMPS Hard: 16.0 (#47/73)
Phi-3-medium-128k-instruct — ELO 1452, #717
- LiveBench Math Comp: 25.0 (#29/73)
- LiveBench Table Reformat: 44.0 (#34/73)
- LiveBench Paraphrase: 62.07 (#36/73)
- LiveBench Story Generation: 63.75 (#37/73)
- LiveBench AMPS Hard: 21.0 (#39/73)
- LiveBench Simplify: 51.7 (#45/73)
- LiveBench Connections: 9.5 (#48/73)
- LiveBench Plot Unscrambling: 14.77 (#49/73)
- LiveBench Typos: 14.0 (#49/73)
- LiveBench Table Join: 4.36 (#50/73)
c4ai-command-r-08-2024 — ELO 1444, #735
- LiveBench Paraphrase: 69.05 (#15/73)
- LiveBench Zebra Puzzle: 34.0 (#24/73)
- LiveBench Story Generation: 69.58 (#26/73)
- LiveBench Olympiad: 47.29 (#27/73)
- LiveBench Simplify: 63.77 (#27/73)
- LiveBench Summarize: 60.42 (#31/73)
- LiveBench Plot Unscrambling: 18.16 (#35/73)
- LiveBench Table Reformat: 42.0 (#36/73)
- LiveBench Math Comp: 18.75 (#38/73)
- LiveBench Typos: 20.0 (#39/73)
Claude 3 Sonnet (20240229) — ELO 1440, #746
- LiveBench Typos: 54.0 (#12/73)
- LiveBench Summarize: 67.23 (#17/73)
- LiveBench Plot Unscrambling: 34.08 (#19/73)
- LiveBench Table Reformat: 62.0 (#19/73)
- LiveBench Zebra Puzzle: 36.0 (#19/73)
- LiveBench Connections: 26.17 (#24/73)
- LiveBench Table Join: 15.68 (#24/73)
- LiveBench Olympiad: 50.24 (#25/73)
- LiveBench Simplify: 64.55 (#25/73)
- LiveBench Paraphrase: 65.82 (#28/73)
GPT-4.1 Nano — ELO 1439, #748
- MCPMark: 0.0 (#39/39)
GPT-3.5 Turbo (0125) — ELO 1433, #756
- LiveBench Table Join: 19.64 (#22/73)
- LiveBench Typos: 42.0 (#25/73)
- LiveBench Table Reformat: 54.0 (#26/73)
- LiveBench Zebra Puzzle: 34.0 (#27/73)
- LiveBench Paraphrase: 65.77 (#29/73)
- LiveBench Olympiad: 44.26 (#31/73)
- LiveBench Story Generation: 67.08 (#31/73)
- LiveBench AMPS Hard: 23.0 (#32/73)
- LiveBench Plot Unscrambling: 17.32 (#37/73)
- LiveBench Simplify: 57.03 (#39/73)
GPT-3.5 Turbo — ELO 1426, #772
- SimpleBench: 8.0 (#86/86)
Llama 3 8B Instruct — ELO 1423, #779
- LiveBench Connections: 20.33 (#33/73)
- LiveBench Paraphrase: 61.88 (#37/73)
- LiveBench Zebra Puzzle: 30.0 (#37/73)
- LiveBench Summarize: 54.27 (#39/73)
- LiveBench Simplify: 56.17 (#41/73)
- LiveBench Typos: 20.0 (#41/73)
- LiveBench Plot Unscrambling: 15.82 (#43/73)
- LiveBench Olympiad: 36.41 (#44/73)
- LiveBench Table Reformat: 34.0 (#46/73)
- LiveBench Math Comp: 14.58 (#50/73)
Qwen 1.5 72B Chat — ELO 1421, #784
- LiveBench Story Generation: 69.08 (#28/73)
- LiveBench Olympiad: 43.77 (#32/73)
- LiveBench Table Join: 10.94 (#32/73)
- LiveBench Zebra Puzzle: 34.0 (#32/73)
- LiveBench Plot Unscrambling: 17.94 (#36/73)
- LiveBench Summarize: 56.83 (#36/73)
- LiveBench Simplify: 56.23 (#40/73)
- LiveBench AMPS Hard: 19.0 (#42/73)
- LiveBench Table Reformat: 36.0 (#44/73)
- LiveBench Math Comp: 17.71 (#45/73)
Hermes 3 - Llama-3.1 70B — ELO 1420, #787
- LiveBench Typos: 62.0 (#7/73)
- LiveBench Table Reformat: 66.0 (#8/73)
- LiveBench Table Join: 26.34 (#15/73)
- LiveBench Connections: 32.0 (#16/73)
- LiveBench Plot Unscrambling: 37.32 (#16/73)
- LiveBench Math Comp: 20.83 (#34/73)
- LiveBench Olympiad: 43.12 (#35/73)
- LiveBench Zebra Puzzle: 32.0 (#35/73)
- LiveBench Simplify: 58.65 (#37/73)
- LiveBench AMPS Hard: 21.0 (#38/73)
Kimi K2 (0711) — ELO 1416, #796
- MCPMark: 19.09 (#24/39)
Gemma 2 9B (IT) — ELO 1410, #815
- LiveBench Paraphrase: 68.05 (#19/73)
- LiveBench AMPS Hard: 28.0 (#27/73)
- LiveBench Typos: 38.0 (#27/73)
- LiveBench Connections: 23.0 (#29/73)
- LiveBench Table Reformat: 50.0 (#30/73)
- LiveBench Simplify: 61.77 (#32/73)
- LiveBench Plot Unscrambling: 21.93 (#34/73)
- LiveBench Summarize: 59.05 (#35/73)
- LiveBench Table Join: 5.18 (#46/73)
- LiveBench Story Generation: 57.33 (#49/73)
Llama 3.1 8B Instruct — ELO 1402, #835
- LiveBench Paraphrase: 64.03 (#31/73)
- LiveBench Table Reformat: 50.0 (#31/73)
- LiveBench Typos: 30.0 (#31/73)
- LiveBench Simplify: 57.52 (#38/73)
- LiveBench Plot Unscrambling: 16.98 (#39/73)
- LiveBench Summarize: 54.0 (#40/73)
- LiveBench AMPS Hard: 20.0 (#41/73)
- LiveBench Connections: 13.17 (#42/73)
- LiveBench Math Comp: 17.71 (#44/73)
- LiveBench Olympiad: 35.4 (#45/73)
Phi-3.5-mini-instruct — ELO 1402, #836
- LiveBench Zebra Puzzle: 34.0 (#30/73)
- LiveBench Story Generation: 65.5 (#32/73)
- LiveBench Summarize: 59.93 (#32/73)
- LiveBench AMPS Hard: 21.0 (#40/73)
- LiveBench Simplify: 53.2 (#44/73)
- LiveBench Table Join: 5.28 (#45/73)
- LiveBench Math Comp: 16.67 (#46/73)
- LiveBench Table Reformat: 34.0 (#47/73)
- LiveBench Paraphrase: 54.57 (#50/73)
- LiveBench Connections: 5.17 (#53/73)
Claude 3 Haiku (20240307) — ELO 1400, #845
- LiveBench Summarize: 69.58 (#12/73)
- LiveBench Typos: 48.0 (#15/73)
- LiveBench Table Reformat: 62.0 (#18/73)
- LiveBench Simplify: 65.1 (#24/73)
- LiveBench Table Join: 12.62 (#28/73)
- LiveBench Olympiad: 45.38 (#30/73)
- LiveBench Plot Unscrambling: 23.72 (#32/73)
- LiveBench Paraphrase: 63.52 (#33/73)
- LiveBench Zebra Puzzle: 32.0 (#33/73)
- LiveBench Math Comp: 19.79 (#36/73)
Mixtral 8x7B Instruct (v0.1) — ELO 1395, #857
- LiveBench Connections: 15.0 (#39/73)
- LiveBench Plot Unscrambling: 16.28 (#41/73)
- LiveBench AMPS Hard: 16.0 (#46/73)
- LiveBench Math Comp: 15.62 (#48/73)
- LiveBench Table Reformat: 30.0 (#50/73)
- LiveBench Olympiad: 30.5 (#55/73)
- LiveBench Typos: 10.0 (#57/73)
- LiveBench Simplify: 42.75 (#58/73)
- LiveBench Table Join: 2.38 (#58/73)
- LiveBench Story Generation: 51.08 (#59/73)
Phi-3-small-128k-instruct — ELO 1392, #866
- LiveBench AMPS Hard: 28.0 (#28/73)
- LiveBench Story Generation: 64.67 (#35/73)
- LiveBench Math Comp: 19.79 (#37/73)
- LiveBench Olympiad: 39.12 (#39/73)
- LiveBench Table Reformat: 40.0 (#39/73)
- LiveBench Typos: 20.0 (#42/73)
- LiveBench Zebra Puzzle: 28.0 (#43/73)
- LiveBench Connections: 12.83 (#44/73)
- LiveBench Simplify: 51.08 (#46/73)
- LiveBench Table Join: 3.78 (#52/73)
Qwen 2.5 Turbo — ELO 1389, #881
- AI Chess Leaderboard (Continuation): 438.0 (#178/237)
- AI Chess Leaderboard (Reasoning): 574.0 (#201/292)
DeepSeek Coder V2 Lite Instruct — ELO 1383, #901
- LiveBench AMPS Hard: 33.0 (#18/73)
- LiveBench Math Comp: 33.33 (#20/73)
- LiveBench Table Reformat: 52.0 (#28/73)
- LiveBench Table Join: 9.0 (#37/73)
- LiveBench Typos: 20.0 (#40/73)
- LiveBench Olympiad: 36.98 (#41/73)
- LiveBench Story Generation: 58.25 (#45/73)
- LiveBench Zebra Puzzle: 20.0 (#55/73)
- LiveBench Paraphrase: 52.85 (#57/73)
- LiveBench Simplify: 44.2 (#57/73)
Phi-3 Mini 4K Instruct — ELO 1381, #908
- LiveBench Table Reformat: 38.0 (#42/73)
- LiveBench Zebra Puzzle: 28.0 (#42/73)
- LiveBench Story Generation: 58.67 (#43/73)
- LiveBench Simplify: 50.67 (#48/73)
- LiveBench Table Join: 4.64 (#48/73)
- LiveBench AMPS Hard: 14.0 (#49/73)
- LiveBench Math Comp: 15.62 (#49/73)
- LiveBench Olympiad: 32.89 (#50/73)
- LiveBench Paraphrase: 53.98 (#53/73)
- LiveBench Plot Unscrambling: 13.18 (#55/73)
Mistral Small (2402) — ELO 1375, #930
- LiveBench Math Comp: 27.08 (#23/73)
- LiveBench Story Generation: 70.67 (#24/73)
- LiveBench Summarize: 63.75 (#25/73)
- LiveBench Plot Unscrambling: 26.5 (#29/73)
- LiveBench Connections: 21.67 (#31/73)
- LiveBench Table Reformat: 44.0 (#33/73)
- LiveBench Paraphrase: 62.07 (#35/73)
- LiveBench Simplify: 59.15 (#36/73)
- LiveBench Olympiad: 41.35 (#37/73)
- LiveBench Table Join: 7.64 (#41/73)
Phi-3 Mini 128K Instruct — ELO 1367, #950
- LiveBench Table Reformat: 44.0 (#35/73)
- LiveBench Zebra Puzzle: 30.0 (#40/73)
- LiveBench Table Join: 6.06 (#43/73)
- LiveBench Summarize: 49.35 (#47/73)
- LiveBench AMPS Hard: 14.0 (#48/73)
- LiveBench Paraphrase: 54.15 (#52/73)
- LiveBench Simplify: 48.7 (#52/73)
- LiveBench Story Generation: 53.42 (#55/73)
- LiveBench Olympiad: 29.9 (#57/73)
- LiveBench Typos: 10.0 (#58/73)
Qwen 2 7B Instruct — ELO 1355, #978
- LiveBench AMPS Hard: 25.0 (#31/73)
- LiveBench Math Comp: 23.96 (#32/73)
- LiveBench Table Reformat: 36.0 (#45/73)
- LiveBench Connections: 10.0 (#46/73)
- LiveBench Plot Unscrambling: 14.62 (#50/73)
- LiveBench Olympiad: 31.65 (#52/73)
- LiveBench Zebra Puzzle: 24.0 (#52/73)
- LiveBench Summarize: 45.02 (#53/73)
- LiveBench Table Join: 2.24 (#59/73)
- LiveBench Simplify: 40.0 (#63/73)
Command-R+ — ELO 1344, #1008
- LiveBench Story Generation: 80.83 (#2/73)
- LiveBench Zebra Puzzle: 46.0 (#3/73)
- LiveBench Simplify: 72.63 (#10/73)
- LiveBench Paraphrase: 70.22 (#11/73)
- LiveBench Summarize: 62.35 (#28/73)
- LiveBench Olympiad: 45.81 (#29/73)
- LiveBench Connections: 22.33 (#30/73)
- LiveBench Plot Unscrambling: 25.43 (#30/73)
- LiveBench Table Join: 11.8 (#30/73)
- LiveBench Typos: 24.0 (#36/73)
Nanbeige4.1-3B — ELO 1341, #1012
- GDPval-AA: 141.0 (#141/153)
- AA GDPval: -141.11 (#145/148)
- Tau3 Banking: 0.0 (#149/150)
zephyr-7B-alpha — ELO 1337, #1020
- LiveBench Paraphrase: 57.98 (#44/73)
- LiveBench Story Generation: 58.17 (#46/73)
- LiveBench Summarize: 47.32 (#50/73)
- LiveBench Table Join: 4.2 (#51/73)
- LiveBench Simplify: 47.7 (#54/73)
- LiveBench Connections: 4.0 (#56/73)
- LiveBench Plot Unscrambling: 11.59 (#59/73)
- LiveBench Zebra Puzzle: 20.0 (#61/73)
- LiveBench Olympiad: 19.62 (#62/73)
- LiveBench Table Reformat: 12.0 (#62/73)
OpenHermes-2.5-Mistral-7B — ELO 1327, #1040
- LiveBench Story Generation: 64.92 (#33/73)
- LiveBench Olympiad: 36.8 (#42/73)
- LiveBench Table Join: 6.76 (#42/73)
- LiveBench Typos: 18.0 (#45/73)
- LiveBench Plot Unscrambling: 15.11 (#47/73)
- LiveBench Zebra Puzzle: 26.0 (#47/73)
- LiveBench AMPS Hard: 11.0 (#52/73)
- LiveBench Summarize: 45.68 (#52/73)
- LiveBench Table Reformat: 28.0 (#52/73)
- LiveBench Math Comp: 13.54 (#54/73)
Starling-LM-7B-beta — ELO 1319, #1057
- LiveBench Connections: 4.5 (#54/73)
- LiveBench Plot Unscrambling: 13.29 (#54/73)
- LiveBench Zebra Puzzle: 22.0 (#54/73)
- LiveBench Math Comp: 11.46 (#56/73)
- LiveBench AMPS Hard: 9.0 (#57/73)
- LiveBench Olympiad: 24.13 (#61/73)
- LiveBench Paraphrase: 47.48 (#64/73)
- LiveBench Summarize: 31.08 (#66/73)
- LiveBench Simplify: 31.8 (#67/73)
- LiveBench Story Generation: 42.92 (#67/73)
Command R — ELO 1314, #1065
- LiveBench Zebra Puzzle: 38.0 (#14/73)
- LiveBench Table Join: 9.08 (#36/73)
- LiveBench Typos: 22.0 (#38/73)
- LiveBench Story Generation: 63.5 (#39/73)
- LiveBench Paraphrase: 59.85 (#40/73)
- LiveBench Table Reformat: 38.0 (#40/73)
- LiveBench Math Comp: 17.71 (#43/73)
- LiveBench Simplify: 53.83 (#43/73)
- LiveBench Summarize: 51.47 (#45/73)
- LiveBench Connections: 7.67 (#49/73)
Phi-4 Mini Instruct — ELO 1291, #1099
- Tau3 Banking: 1.03 (#144/150)
Gemma 1.1 7B (IT) — ELO 1249, #1181
- LiveBench Connections: 7.17 (#50/73)
- LiveBench Typos: 12.0 (#51/73)
- LiveBench Olympiad: 31.48 (#53/73)
- LiveBench AMPS Hard: 10.0 (#54/73)
- LiveBench Plot Unscrambling: 12.78 (#57/73)
- LiveBench Table Join: 2.52 (#57/73)
- LiveBench Zebra Puzzle: 20.0 (#57/73)
- LiveBench Paraphrase: 51.35 (#59/73)
- LiveBench Summarize: 40.72 (#59/73)
- LiveBench Simplify: 41.78 (#60/73)
Mistral 7B Instruct (v0.3) — ELO 1230, #1206
- LiveBench Plot Unscrambling: 15.39 (#45/73)
- LiveBench Olympiad: 33.48 (#48/73)
- LiveBench Paraphrase: 56.02 (#48/73)
- LiveBench Typos: 14.0 (#48/73)
- LiveBench Simplify: 50.03 (#49/73)
- LiveBench Summarize: 47.75 (#49/73)
- LiveBench Connections: 6.17 (#52/73)
- LiveBench Story Generation: 55.67 (#52/73)
- LiveBench Table Reformat: 18.0 (#58/73)
- LiveBench AMPS Hard: 5.0 (#61/73)
Mistral 7B Instruct (v0.2) — ELO 1223, #1216
- LiveBench Summarize: 51.75 (#43/73)
- LiveBench Story Generation: 58.42 (#44/73)
- LiveBench Olympiad: 34.87 (#46/73)
- LiveBench Connections: 6.17 (#51/73)
- LiveBench Paraphrase: 54.52 (#51/73)
- LiveBench Plot Unscrambling: 13.0 (#56/73)
- LiveBench Simplify: 41.92 (#59/73)
- LiveBench AMPS Hard: 7.0 (#60/73)
- LiveBench Math Comp: 9.38 (#60/73)
- LiveBench Typos: 8.0 (#60/73)
Qwen 2 1.5B Instruct — ELO 1221, #1217
- LiveBench Table Join: 2.02 (#60/73)
- LiveBench AMPS Hard: 5.0 (#62/73)
- LiveBench Math Comp: 8.33 (#62/73)
- LiveBench Table Reformat: 8.0 (#64/73)
- LiveBench Zebra Puzzle: 16.0 (#65/73)
- LiveBench Olympiad: 16.49 (#67/73)
- LiveBench Story Generation: 40.42 (#68/73)
- LiveBench Paraphrase: 26.2 (#70/73)
- LiveBench Connections: 0.0 (#71/73)
- LiveBench Plot Unscrambling: 9.14 (#71/73)
Qwen 1.5 4B Chat — ELO 1185, #1245
- LiveBench Zebra Puzzle: 26.0 (#48/73)
- LiveBench Table Join: 3.38 (#54/73)
- LiveBench Math Comp: 10.42 (#58/73)
- LiveBench Typos: 8.0 (#62/73)
- LiveBench AMPS Hard: 4.0 (#63/73)
- LiveBench Table Reformat: 6.0 (#66/73)
- LiveBench Olympiad: 15.15 (#68/73)
- LiveBench Paraphrase: 31.62 (#68/73)
- LiveBench Connections: 0.0 (#69/73)
- LiveBench Simplify: 22.65 (#69/73)
Qwen 1.5 7B Chat — ELO 1183, #1248
- LiveBench Zebra Puzzle: 22.0 (#53/73)
- LiveBench AMPS Hard: 10.0 (#55/73)
- LiveBench Table Join: 2.7 (#56/73)
- LiveBench Connections: 3.5 (#57/73)
- LiveBench Summarize: 41.77 (#57/73)
- LiveBench Olympiad: 28.59 (#59/73)
- LiveBench Paraphrase: 50.65 (#61/73)
- LiveBench Plot Unscrambling: 11.04 (#62/73)
- LiveBench Story Generation: 46.67 (#64/73)
- LiveBench Math Comp: 7.29 (#65/73)
zephyr-7B-beta — ELO 1183, #1249
- LiveBench Simplify: 49.62 (#51/73)
- LiveBench Math Comp: 11.46 (#57/73)
- LiveBench Paraphrase: 51.77 (#58/73)
- LiveBench Story Generation: 52.17 (#58/73)
- LiveBench Table Reformat: 14.0 (#59/73)
- LiveBench Summarize: 39.72 (#60/73)
- LiveBench Plot Unscrambling: 10.85 (#64/73)
- LiveBench Table Join: 1.24 (#64/73)
- LiveBench AMPS Hard: 4.0 (#65/73)
- LiveBench Olympiad: 18.23 (#65/73)
Llama 2 7B Chat — ELO 1131, #1306
- LiveBench Paraphrase: 55.12 (#49/73)
- LiveBench Typos: 10.0 (#56/73)
- LiveBench Simplify: 40.27 (#61/73)
- LiveBench Summarize: 38.07 (#62/73)
- LiveBench Story Generation: 46.08 (#65/73)
- LiveBench Connections: 0.0 (#66/73)
- LiveBench Plot Unscrambling: 10.58 (#66/73)
- LiveBench Table Join: 0.0 (#67/73)
- LiveBench Olympiad: 14.35 (#69/73)
- LiveBench Zebra Puzzle: 10.0 (#70/73)
Yi 6B (Chat) — ELO 1130, #1308
- LiveBench Table Reformat: 12.0 (#61/73)
- LiveBench Olympiad: 19.42 (#63/73)
- LiveBench Table Join: 1.14 (#65/73)
- LiveBench Plot Unscrambling: 10.07 (#67/73)
- LiveBench AMPS Hard: 2.0 (#68/73)
- LiveBench Summarize: 25.7 (#68/73)
- LiveBench Typos: 4.0 (#68/73)
- LiveBench Story Generation: 39.5 (#69/73)
- LiveBench Zebra Puzzle: 12.0 (#69/73)
- LiveBench Math Comp: 4.17 (#71/73)
vicuna-7B-v1.5 — ELO 1074, #1338
- LiveBench Typos: 14.0 (#50/73)
- LiveBench Story Generation: 50.67 (#60/73)
- LiveBench Simplify: 40.22 (#62/73)
- LiveBench Connections: 1.0 (#63/73)
- LiveBench Math Comp: 8.33 (#63/73)
- LiveBench Plot Unscrambling: 10.98 (#63/73)
- LiveBench Summarize: 32.77 (#64/73)
- LiveBench Paraphrase: 43.37 (#67/73)
- LiveBench Zebra Puzzle: 14.0 (#67/73)
- LiveBench AMPS Hard: 1.0 (#69/73)
vicuna-7B-v1.5-16k — ELO 1036, #1363
- LiveBench Story Generation: 53.5 (#54/73)
- LiveBench Typos: 12.0 (#55/73)
- LiveBench Zebra Puzzle: 20.0 (#60/73)
- LiveBench Paraphrase: 49.82 (#62/73)
- LiveBench Table Join: 1.8 (#62/73)
- LiveBench Connections: 1.0 (#64/73)
- LiveBench Olympiad: 18.84 (#64/73)
- LiveBench Plot Unscrambling: 10.75 (#65/73)
- LiveBench Summarize: 32.07 (#65/73)
- LiveBench Table Reformat: 8.0 (#65/73)
Qwen 1.5 1.8B Chat — ELO 965, #1394
- LiveBench Connections: 0.0 (#68/73)
- LiveBench Plot Unscrambling: 9.47 (#69/73)
- LiveBench Table Reformat: 4.0 (#69/73)
- LiveBench Math Comp: 4.17 (#70/73)
- LiveBench Simplify: 21.57 (#70/73)
- LiveBench Table Join: 0.0 (#70/73)
- LiveBench Paraphrase: 23.4 (#71/73)
- LiveBench Typos: 0.0 (#71/73)
- LiveBench Zebra Puzzle: 10.0 (#71/73)
- LiveBench Summarize: 23.48 (#72/73)
Qwen 2 0.5B Instruct — ELO 958, #1399
- LiveBench Math Comp: 9.38 (#61/73)
- LiveBench AMPS Hard: 3.0 (#67/73)
- LiveBench Simplify: 23.37 (#68/73)
- LiveBench Paraphrase: 30.9 (#69/73)
- LiveBench Connections: 0.0 (#70/73)
- LiveBench Story Generation: 28.08 (#71/73)
- LiveBench Summarize: 24.17 (#71/73)
- LiveBench Table Join: 0.0 (#71/73)
- LiveBench Olympiad: 9.66 (#72/73)
- LiveBench Typos: 0.0 (#72/73)
Qwen 1.5 0.5B Chat — ELO 814, #1448
- LiveBench Connections: 0.0 (#67/73)
- LiveBench Table Join: 0.0 (#69/73)
- LiveBench Summarize: 24.95 (#70/73)
- LiveBench Typos: 0.0 (#70/73)
- LiveBench Olympiad: 10.16 (#71/73)
- LiveBench AMPS Hard: 0.0 (#72/73)
- LiveBench Math Comp: 3.12 (#72/73)
- LiveBench Plot Unscrambling: 8.63 (#72/73)
- LiveBench Story Generation: 25.83 (#72/73)
- LiveBench Table Reformat: 0.0 (#72/73)

Top-10 New Scores (121)

Claude Fable 5 on KernelBench Hub - Hard: 24.0 (#4)
Claude Fable 5 on KernelBench Hub - Mega: 18.7 (#2)
Claude Fable 5 on Lean AI Formalization Leaderboard: 4.0 (#13)
Claude Fable 5 on TaxCalcBench: 26.0 (#5)
Claude Fable 5 on TrackingAI IQ Test (Mensa Norway): 60.0 (#20)
Claude Fable 5 on TrackingAI IQ Test (Vision): 78.57 (#3)
Claude Opus 4.8 on TaxCalcBench: 16.0 (#7)
GPT-5.6 Sol on AA GDPval: 1747.82 (#2)
GPT-5.6 Sol on AA GPQA Diamond: 94.14 (#1)
GPT-5.6 Sol on AA Humanity's Last Exam: 47.17 (#2)
GPT-5.6 Sol on AA IFBench: 72.65 (#47)
GPT-5.6 Sol on AA Long Context Reasoning: 73.67 (#12)
GPT-5.6 Sol on AA MMMU-Pro: 83.41 (#3)
GPT-5.6 Sol on AA Omniscience: 21.7 (#7)
GPT-5.6 Sol on AA Omniscience - Business: 50.6 (#3)
GPT-5.6 Sol on AA Omniscience - Health: 50.5 (#2)
GPT-5.6 Sol on AA Omniscience - Humanities & Social Sciences: 58.0 (#2)
GPT-5.6 Sol on AA Omniscience - Law: 53.5 (#8)
GPT-5.6 Sol on AA Omniscience - Science, Engineering & Mathematics: 54.1 (#2)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE): 84.5 (#2)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - C: 88.0 (#6)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - Dart: 76.0 (#7)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - Go: 82.0 (#8)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - Java: 72.0 (#3)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - JavaScript: 89.09 (#2)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - Julia: 56.0 (#34)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - Kotlin: 88.0 (#3)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - Python: 88.5 (#5)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - R: 66.0 (#12)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - Rust: 84.0 (#8)
GPT-5.6 Sol on AA Omniscience - Software Engineering (SWE) - Swift: 84.0 (#6)
GPT-5.6 Sol on AA SciCode: 56.13 (#8)
GPT-5.6 Sol on AA TAU-2 Bench: 85.09 (#99)
GPT-5.6 Sol on AA-Omniscience Accuracy: 58.53 (#2)
GPT-5.6 Sol on AI Chess Leaderboard (Continuation): 1213.0 (#25)
GPT-5.6 Sol on AI Chess Leaderboard (Reasoning): 1611.0 (#11)
GPT-5.6 Sol on ARC-AGI-1: 96.5 (#7)
GPT-5.6 Sol on ARC-AGI-2: 92.5 (#2)
GPT-5.6 Sol on Artificial Analysis Intelligence Index: 58.89 (#2)
GPT-5.6 Sol on BridgeBench Debugging: 80.1 (#24)
GPT-5.6 Sol on BridgeBench Hallucination: 76.8 (#9)
GPT-5.6 Sol on BridgeBench Refactoring: 64.7 (#18)
GPT-5.6 Sol on BridgeBench Security: 70.6 (#14)
GPT-5.6 Sol on BridgeBench UI: 77.6 (#11)
GPT-5.6 Sol on Bullshit Benchmark: 27.3 (#59)
GPT-5.6 Sol on Chess Puzzles (Epoch AI): 64.0 (#1)
GPT-5.6 Sol on Chess Puzzles (Epoch AI): 64.0 (#1)
GPT-5.6 Sol on CursorBench 3.1: 67.2 (#3)
GPT-5.6 Sol on Design Arena (3D): 1370.0 (#2)
GPT-5.6 Sol on Design Arena (Data Viz): 1344.0 (#2)
GPT-5.6 Sol on Design Arena (Game Dev): 1378.0 (#2)
GPT-5.6 Sol on Design Arena (UI Components): 1413.0 (#1)
GPT-5.6 Sol on Design Arena (Website): 1344.0 (#2)
GPT-5.6 Sol on Epoch AI - Cursorbench: 67.2 (#4)
GPT-5.6 Sol on Epoch AI - Scicode: 56.13 (#7)
GPT-5.6 Sol on FrontierMath - Tier 4 (v2): 82.93 (#2)
GPT-5.6 Sol on FrontierMath - Tier 4 (v2): 82.93 (#2)
GPT-5.6 Sol on GDPval-AA: 1748.0 (#2)
GPT-5.6 Sol on Gert Labs Rankings: 73.04 (#1)
GPT-5.6 Sol on KernelBench Hub - Mega: 1.1 (#11)
GPT-5.6 Sol on LisanBench: 0.0721 (#45)
GPT-5.6 Sol on Mercor APEX: 40.0 (#4)
GPT-5.6 Sol on Mercor APEX: 40.0 (#4)
GPT-5.6 Sol on OTIS Mock AIME 2024-25: 100.0 (#1)
GPT-5.6 Sol on PM-LLM-Benchmark: 36.3 (#17)
GPT-5.6 Sol on PinchBench: 87.0 (#22)
GPT-5.6 Sol on Ramp SWE-Bench: 82.28 (#4)
GPT-5.6 Sol on Senior SWE-Bench: 24.4 (#3)
GPT-5.6 Sol on SimpleBench: 64.8 (#12)
GPT-5.6 Sol on SimpleQA Verified: 71.6 (#3)
GPT-5.6 Sol on SvelteBench: 94.4 (#26)
GPT-5.6 Sol on TaxCalcBench: 26.0 (#4)
GPT-5.6 Sol on Vals AI (Vals Index): 72.63 (#2)
GPT-5.6 Sol on Vals AI Code Migration: 52.92 (#2)
GPT-5.6 Sol on Vals AI CorpFin v2: 64.38 (#31)
GPT-5.6 Sol on Vals AI Finance Agent v2: 53.76 (#6)
GPT-5.6 Sol on Vals AI GPQA: 95.2 (#2)
GPT-5.6 Sol on Vals AI Harvey Legal Agent Bench: 2.5 (#12)
GPT-5.6 Sol on Vals AI LegalBench: 86.97 (#4)
GPT-5.6 Sol on Vals AI LiveCodeBench: 82.6 (#38)
GPT-5.6 Sol on Vals AI MMLU-Pro: 89.1 (#10)
GPT-5.6 Sol on Vals AI MMMU: 88.84 (#2)
GPT-5.6 Sol on Vals AI MedCode: 43.97 (#23)
GPT-5.6 Sol on Vals AI MedScribe: 85.23 (#12)
GPT-5.6 Sol on Vals AI MortgageTax: 67.29 (#23)
GPT-5.6 Sol on Vals AI Multimodal Index: 72.19 (#2)
GPT-5.6 Sol on Vals AI ProofBench: 77.0 (#1)
GPT-5.6 Sol on Vals AI Public Benefits Bench: 66.51 (#2)
GPT-5.6 Sol on Vals AI SAGE: 52.56 (#4)
GPT-5.6 Sol on Vals AI TaxEval v2: 74.78 (#20)
GPT-5.6 Sol on Vals AI Vibe Code Bench: 80.5 (#4)
GPT-5.6 Sol on Vending-Bench 2: 9619.37 (#2)
GPT-5.6 Sol on YC-Bench: 685.9 (#16)
GPT-5.6 Sol on You're Absolutely Right!: 2.875 (#8)
Muse Spark 1.1 on Chatbot Arena (Code): 1540.0 (#10)
Muse Spark 1.1 on Chatbot Arena (Text): 1490.0 (#6)
Muse Spark 1.1 on SEAL - Fortress: 12.4 (#54)
Muse Spark 1.1 on SEAL - MultiChallenge: 75.3 (#2)
Muse Spark 1.1 on SvelteBench: 92.2 (#36)
Muse Spark 1.1 on Vals AI (Vals Index): 68.41 (#5)
Muse Spark 1.1 on Vals AI Code Migration: 31.11 (#12)
Muse Spark 1.1 on Vals AI CorpFin v2: 71.29 (#2)
Muse Spark 1.1 on Vals AI Excel Modeling: 56.36 (#11)
Muse Spark 1.1 on Vals AI Finance Agent v2: 57.21 (#2)
Muse Spark 1.1 on Vals AI GPQA: 91.16 (#13)
Muse Spark 1.1 on Vals AI Legal Research Bench: 37.98 (#7)
Muse Spark 1.1 on Vals AI LegalBench: 84.98 (#16)
Muse Spark 1.1 on Vals AI LiveCodeBench: 85.91 (#18)
Muse Spark 1.1 on Vals AI MMLU-Pro: 88.73 (#11)
Muse Spark 1.1 on Vals AI MMMU: 86.59 (#11)
Muse Spark 1.1 on Vals AI MortgageTax: 66.14 (#30)
Muse Spark 1.1 on Vals AI Multimodal Index: 66.74 (#7)
Muse Spark 1.1 on Vals AI ProgramBench: 16.16 (#25)
Muse Spark 1.1 on Vals AI ProofBench: 39.0 (#13)
Muse Spark 1.1 on Vals AI Public Benefits Bench: 61.3 (#9)
Muse Spark 1.1 on Vals AI SAGE: 45.58 (#22)
Muse Spark 1.1 on Vals AI SWE-bench Verified: 82.0 (#8)
Muse Spark 1.1 on Vals AI Terminal-Bench 2.1: 69.29 (#11)
Muse Spark 1.1 on Vals AI Vibe Code Bench: 72.16 (#7)
Seed 2.1 Pro on Agents' Last Exam: 19.5 (#11)
Seed 2.1 Pro on BabyVision: 73.7 (#4)

New #1 Leaders (66)

PredictionArena (Kalshi): Claude Opus 4.6 (221482.25) beat GLM-5 by 203630.3
RuneBench: GPT-5.6 Sol (7400.0) beat GPT-5.5 by 1030.0
VoxelBench: GPT-5.6 Sol (2306.0) beat Claude Fable 5 (Max) by 83.0
WDCD R3 Pressure Integrity: Gemini 3.1 Pro (Preview) (138.0) beat Grok 4 by 25.0
BabyVision: GPT-5.5 (83.6) beat Seed 2.0 Pro (High) by 23.0
SEAL - VisualToolBench (VTB): Muse Spark 1.1 (44.77) beat GPT-5.4 by 15.6
Vals AI IOI: GPT-5.6 Sol (86.67) beat Claude Fable 5 by 14.42
Benchmarks.bio - scBench-Long: GPT-5.6 Sol (42.5) beat Claude Opus 4.8 by 13.93
LiveBench Simplify: Gemini 1.5 Flash (Preview 0827) (84.75) beat Gemini 3.1 Pro (Preview) (High) by 13.7
TabArena All Tasks: TabFM (default) (89.1) beat TabPFN-3 (default) by 11.7
Opus Magnum Bench: GPT-5.6 Sol (xHigh) (70.86) beat Claude Fable 5 (High) by 10.71
Vals AI Harvey Legal Agent Bench: Muse Spark 1.1 (20.0) beat Claude Fable 5 by 8.75
SEAL - SWE Atlas - Codebase QnA: Claude Opus 4.8 (Claude Code) xhigh (57.26) beat Claude Opus 4.8 (Claude Code) by 8.47
Vals AI CyberBench: GPT-5.6 Sol (88.14) beat GPT-5.5 by 7.63
WolfBench: GPT-5.6 Sol (85.0) beat GPT-5.5 by 6.0
Vals AI Terminal-Bench 2.1: GPT-5.6 Sol (85.77) beat Claude Fable 5 by 5.25
Senior SWE-Bench: Claude Fable 5 (29.1) beat Claude Opus 4.8 by 5.1
SEAL - Professional Reasoning Benchmark - Legal: Muse Spark 1.1 (57.05) beat Muse Spark by 4.76
PIIMB PII Masking - OpenPII F2: OpenMed privacy-filter-nemotron-v2 (97.63) beat hivetrace gliner-guard-omni by 4.58
SEAL - MCP Atlas: Muse Spark 1.1 (88.1) beat Gemini 3.5 Flash (High) by 4.5
PIIMB PII Masking - Avg F1: OpenMed privacy-filter-multilingual-v2 (92.38) beat OpenMed OpenMed-PII-SuperClinical-Large-434M-v1 by 4.43
SEAL - SWE-Bench Pro (Private Dataset): Muse Spark 1.1 (51.5) beat Claude Opus 4.6 (Thinking) by 4.4
Vals AI Legal Research Bench: GPT-5.6 Sol (48.08) beat Claude Opus 4.8 by 4.33
KernelBench Hub - Hard: GPT-5.6 Sol (30.0) beat GLM-5.2 by 4.0
AA Omniscience - Software Engineering (SWE) - Kotlin: GPT-5.6 Sol (xHigh) (94.0) beat GPT-5.3 Codex (xHigh) by 4.0
AA Omniscience - Software Engineering (SWE) - PHP: GPT-5.6 Sol (96.0) beat GPT-5.5 (High) by 4.0
AA CritPt: GPT-5.6 Sol (32.29) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 3.72
Agents' Last Exam: GPT-5 Sol (30.6) beat Claude Opus 4.8 by 3.6
AA Terminal-Bench Hard: GPT-5.6 Sol (65.91) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 3.03
SWE-Marathon: Grok 4.5 (29.0) beat Claude Opus 4.8 by 3.0
PIIMB PII Masking - Gretel F2: OpenMed privacy-filter-nemotron-v2 (98.28) beat OpenMed OpenMed-PII-SuperClinical-Small-44M-v1 by 2.99
Vals AI Excel Modeling: GPT-5.6 Sol (72.34) beat Claude Opus 4.8 by 2.97
PIIMB PII Masking - Avg F2: OpenMed privacy-filter-multilingual-v2 (91.68) beat OpenMed OpenMed-PII-SuperClinical-Large-434M-v1 by 2.94
GSO-Bench: Claude Opus 4.8 (47.06) beat Claude Opus 4.7 by 2.94
DeepSWE: GPT-5 Sol (72.7) beat Claude Fable 5 by 2.8
Open Universal Arabic ASR Leaderboard: cohere-transcribe-arabic-07-2026 (25.87) beat omniASR_LLM_7B by 2.45
SEAL - SWE-Bench Pro (Public Dataset): Muse Spark 1.1 (61.5) beat GPT-5.4 (xHigh) by 2.4
PM-LLM-Benchmark: GPT-5.6 Sol (xHigh) (40.2) beat Kimi K2.6 by 2.4
LiveBench Paraphrase: Llama 3.1 70B Instruct (82.25) beat Gemini 3.1 Pro (Preview) (High) by 2.38
LiveBench Story Generation: Gemini 1.5 Flash (Preview 0827) (81.33) beat GPT-5.4 (xHigh) by 2.31
Vals AI TaxEval v2: Muse Spark 1.1 (79.72) beat Muse Spark by 2.04
AA Omniscience - Software Engineering (SWE) - HTML: GPT-5.6 Sol (92.0) beat GPT-5.5 (High) by 2.0
BenchLM: Claude Fable 5 (91.0) beat Claude Mythos 5 by 2.0
AA Omniscience - Health: GPT-5.6 Sol (High) (50.7) beat Grok Build 0.1 by 1.8
Epoch AI - Critpt: GPT-5.6 Sol (32.3) beat GPT-5.5 Pro (xHigh) by 1.73
SEAL - Professional Reasoning Benchmark - Finance: Muse Spark 1.1 (55.01) beat Claude Opus 4.6 (Non-reasoning) by 1.73
Tau3 Banking: GPT-5.6 Sol (32.99) beat GPT-5.5 (xHigh) by 1.65
AMA-Bench - Embodied AI: VeraKV (Qwen3-32B) (58.09) beat GPT-5.2 by 1.65
FrontierMath - Tiers 1-3 (v2): GPT-5.6 Sol (89.12) beat GPT-5.5 Pro (xHigh) by 1.4
TrackingAI IQ Test: Claude Fable 5 (97.3) beat GPT-5 Pro by 1.22
Vals AI SWE-bench Verified: GPT-5.6 Sol (96.2) beat Claude Fable 5 by 1.2
AA Omniscience - Software Engineering (SWE) - TypeScript: GPT-5.6 Sol (92.22) beat GPT-5.5 (xHigh) by 1.11
WorldScore - Static: World Dreamer (84.52) beat EvoPhys-World by 1.07
Agent Arena - Confirmed Success: MiniMax-M2.7 (17.12) beat Claude Fable 5 (High) by 1.0
WeirdML: GPT-5.6 Sol (High) (88.76) beat Claude Fable 5 (High) by 0.91
Vals AI ProgramBench: GPT-5.6 Sol (77.64) beat Claude Fable 5 by 0.84
WorldScore: World Dreamer (79.44) beat EvoPhys-World by 0.82
AutomationBench: GPT-5.6 Sol (18.1) beat Claude Fable 5.0 (Max) by 0.7
GDP.pdf: GPT-5.6 Sol (30.7) beat Claude Fable 5 by 0.7
Epoch AI - ECI: GPT-5.6 Sol (161.61) beat GPT-5.5 Pro (xHigh) by 0.6
Open Universal Arabic ASR Leaderboard - CER: cohere-transcribe-arabic-07-2026 (11.8) beat Qwen3-ASR-1.7B by 0.53
SEAL - MultiNRC: Muse Spark 1.1 (65.59) beat GPT-5 Pro by 0.39
WorldCupBench - Brier Skill: DeepSeek V4 Pro (85.1919) beat MiMo-V2.5-Pro by 0.37
Vals AI MedScribe: Muse Spark 1.1 (88.89) beat Claude Fable 5 by 0.37
WorldScore - Dynamic: World Dreamer (74.35) beat Inspatio-World by 0.25
Agent Arena - Tool Hallucination: Gemini 3 Flash (0.1) beat DeepSeek V4 Pro by 0.02

The Aggregate Digest — 2026-07-11

2026-07-11T06:25:38.438783+00:00

Daily

Top-10 New Scores (16)

GPT-5.6 Sol on AutomationBench: 18.1 (#1)
GPT-5.6 Sol on Bullshit Benchmark: 27.3 (#59)
GPT-5.6 Sol on Chess Puzzles (Epoch AI): 64.0 (#1)
GPT-5.6 Sol on Design Arena (Data Viz): 1341.0 (#2)
GPT-5.6 Sol on Design Arena (Website): 1346.0 (#2)
GPT-5.6 Sol on Epoch AI - Cursorbench: 67.2 (#4)
GPT-5.6 Sol on Epoch AI - Scicode: 56.13 (#7)
GPT-5.6 Sol on FrontierMath - Tier 4 (v2): 82.93 (#2)
GPT-5.6 Sol on LisanBench: 0.0721 (#45)
GPT-5.6 Sol on Mercor APEX: 40.0 (#4)
GPT-5.6 Sol on Mercor APEX: 40.0 (#4)
GPT-5.6 Sol on Ramp SWE-Bench: 82.28 (#4)
GPT-5.6 Sol on Senior SWE-Bench: 24.4 (#3)
GPT-5.6 Sol on You're Absolutely Right!: 2.875 (#8)
Muse Spark 1.1 on Chatbot Arena (Code): 1540.0 (#10)
Muse Spark 1.1 on Chatbot Arena (Text): 1490.0 (#6)

New #1 Leaders (9)

RuneBench: GPT-5.6 Sol (7400.0) beat GPT-5.5 by 1030.0
Benchmarks.bio - scBench-Long: GPT-5.6 Sol (42.5) beat Claude Opus 4.8 by 13.93
Opus Magnum Bench: GPT-5.6 Sol (xHigh) (70.86) beat Claude Fable 5 (High) by 10.71
WolfBench: GPT-5.6 Sol (85.0) beat GPT-5.5 by 6.0
Agents' Last Exam: GPT-5 Sol (30.6) beat Claude Opus 4.8 by 3.6
Epoch AI - Critpt: GPT-5.6 Sol (32.3) beat GPT-5.5 Pro (xHigh) by 1.73
WeirdML: GPT-5.6 Sol (High) (88.76) beat Claude Fable 5 (High) by 0.91
GDP.pdf: GPT-5.6 Sol (30.7) beat Claude Fable 5 by 0.7
Epoch AI - ECI: GPT-5.6 Sol (161.61) beat GPT-5.5 Pro (xHigh) by 0.6

The Aggregate Digest — 2026-07-10

2026-07-10T07:26:29.777517+00:00

Daily

New Models (1)

Muse Spark 1.1 — ELO 3315, #5
- SEAL - MultiNRC: 65.59 (#1/44)
- SEAL - VisualToolBench (VTB): 44.77 (#1/21)
- SEAL - SWE-Bench Pro (Public Dataset): 61.5 (#1/25)
- SEAL - SWE-Bench Pro (Private Dataset): 51.5 (#1/14)
- SEAL - Professional Reasoning Benchmark - Finance: 55.01 (#1/29)
- SEAL - Professional Reasoning Benchmark - Legal: 57.05 (#1/29)
- SEAL - MCP Atlas: 88.1 (#1/24)
- SEAL - MultiChallenge: 75.3 (#2/30)
- SEAL - Fortress: 12.4 (#54/56)

Top-10 New Scores (25)

Claude Fable 5 on KernelBench Hub - Hard: 24.0 (#4)
Claude Fable 5 on KernelBench Hub - Mega: 18.7 (#2)
Muse Spark 1.1 on SEAL - Fortress: 12.4 (#54)
Muse Spark 1.1 on SEAL - MultiChallenge: 75.3 (#2)
Muse Spark 1.1 on SvelteBench: 92.2 (#36)
Muse Spark 1.1 on Vals AI (Vals Index): 68.41 (#5)
Muse Spark 1.1 on Vals AI Code Migration: 31.11 (#11)
Muse Spark 1.1 on Vals AI CorpFin v2: 71.29 (#2)
Muse Spark 1.1 on Vals AI Excel Modeling: 56.36 (#11)
Muse Spark 1.1 on Vals AI Finance Agent v2: 57.21 (#2)
Muse Spark 1.1 on Vals AI GPQA: 91.16 (#13)
Muse Spark 1.1 on Vals AI Legal Research Bench: 37.98 (#7)
Muse Spark 1.1 on Vals AI LegalBench: 84.98 (#15)
Muse Spark 1.1 on Vals AI LiveCodeBench: 85.91 (#18)
Muse Spark 1.1 on Vals AI MMLU-Pro: 88.73 (#10)
Muse Spark 1.1 on Vals AI MMMU: 86.59 (#11)
Muse Spark 1.1 on Vals AI MortgageTax: 66.14 (#30)
Muse Spark 1.1 on Vals AI Multimodal Index: 66.74 (#7)
Muse Spark 1.1 on Vals AI ProgramBench: 16.16 (#24)
Muse Spark 1.1 on Vals AI ProofBench: 39.0 (#13)
Muse Spark 1.1 on Vals AI Public Benefits Bench: 61.3 (#9)
Muse Spark 1.1 on Vals AI SAGE: 45.58 (#22)
Muse Spark 1.1 on Vals AI SWE-bench Verified: 82.0 (#8)
Muse Spark 1.1 on Vals AI Terminal-Bench 2.1: 69.29 (#11)
Muse Spark 1.1 on Vals AI Vibe Code Bench: 72.16 (#7)

New #1 Leaders (36)

VoxelBench: GPT-5.6 (2313.0) beat Claude Fable 5 (Max) by 130.0
SEAL - VisualToolBench (VTB): Muse Spark 1.1 (44.77) beat GPT-5.4 by 15.6
Vals AI IOI: GPT-5.6 (86.67) beat Claude Fable 5 by 14.42
SEAL - SWE Atlas - Codebase QnA: Claude Opus 4.8 (Claude Code) xhigh (57.26) beat Claude Opus 4.8 (Claude Code) by 8.47
Vals AI CyberBench: GPT-5.6 (88.14) beat GPT-5.5 by 7.63
Vals AI Harvey Legal Agent Bench: muse-spark-1.1 (20.0) beat Grok 4.5 by 7.08
Vals AI Terminal-Bench 2.1: GPT-5.6 (85.77) beat Claude Fable 5 by 5.25
SEAL - Professional Reasoning Benchmark - Legal: Muse Spark 1.1 (57.05) beat Muse Spark by 4.76
SEAL - MCP Atlas: Muse Spark 1.1 (88.1) beat Gemini 3.5 Flash (High) by 4.5
SEAL - SWE-Bench Pro (Private Dataset): Muse Spark 1.1 (51.5) beat Claude Opus 4.6 (Thinking) by 4.4
Vals AI Legal Research Bench: GPT-5.6 (48.08) beat Claude Opus 4.8 by 4.33
Gert Labs Rankings: GPT-5.6 (73.77) beat Claude Fable 5 by 4.19
KernelBench Hub - Hard: GPT-5.6 (30.0) beat GLM-5.2 by 4.0
AA Omniscience - Software Engineering (SWE) - Kotlin: GPT-5.6 (xHigh) (94.0) beat GPT-5.3 Codex (xHigh) by 4.0
AA Omniscience - Software Engineering (SWE) - PHP: GPT-5.6 (96.0) beat GPT-5.5 (Low) by 4.0
AA CritPt: GPT-5.6 (32.29) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 3.72
AA Terminal-Bench Hard: GPT-5.6 (65.91) beat Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) by 3.03
Vals AI Excel Modeling: GPT-5.6 (72.34) beat Claude Opus 4.8 by 2.97
DeepSWE: GPT-5 (72.7) beat Claude Fable 5 by 2.8
SEAL - SWE-Bench Pro (Public Dataset): Muse Spark 1.1 (61.5) beat GPT-5.4 (xHigh) by 2.4
PM-LLM-Benchmark: GPT-5.6 (xHigh) (40.2) beat Kimi K2.6 by 2.4
Vals AI TaxEval v2: muse-spark-1.1 (79.72) beat Muse Spark by 2.04
AA Omniscience - Software Engineering (SWE) - HTML: GPT-5.6 (92.0) beat GPT-5.5 (Medium) by 2.0
AA Omniscience - Health: GPT-5.6 (High) (50.7) beat Grok Build 0.1 by 1.8
SEAL - Professional Reasoning Benchmark - Finance: Muse Spark 1.1 (55.01) beat Claude Opus 4.6 (Non-reasoning) by 1.73
FrontierMath - Tiers 1-3 (v2): GPT-5.6 (89.12) beat GPT-5.5 Pro (xHigh) by 1.4
Vals AI SWE-bench Verified: GPT-5.6 (96.2) beat Claude Fable 5 by 1.2
AA Omniscience - Software Engineering (SWE) - TypeScript: GPT-5.6 (92.22) beat GPT-5.5 (xHigh) by 1.11
WorldScore - Static: World Dreamer (84.52) beat EvoPhys-World by 1.07
Vals AI ProgramBench: GPT-5.6 (77.64) beat Claude Fable 5 by 0.84
WorldScore: World Dreamer (79.44) beat EvoPhys-World by 0.82
AutomationBench: GPT-5.6 (18.1) beat Claude Fable 5.0 (Max) by 0.7
Tau3 Banking: GPT-5.6 (32.99) beat Grok 4.5 (High) by 0.41
SEAL - MultiNRC: Muse Spark 1.1 (65.59) beat GPT-5 Pro by 0.39
Vals AI MedScribe: muse-spark-1.1 (88.89) beat Claude Fable 5 by 0.37
WorldScore - Dynamic: World Dreamer (74.35) beat Inspatio-World by 0.25

The Aggregate Digest — 2026-07-09

2026-07-09T08:28:35.747250+00:00

Daily

New Models (1)

Grok 4.5 — ELO 2494, #13
- SWE-Marathon: 29.0 (#1/12)
- Vals AI Harvey Legal Agent Bench: 12.92 (#1/16)
- Vals AI SWE-bench Verified: 86.6 (#3/66)
- VoxelBench: 1796.0 (#4/44)
- Vals AI MedScribe: 86.88 (#4/67)
- Vals AI GPQA: 92.93 (#4/118)
- Ramp SWE-Bench: 81.01 (#5/21)
- Vals AI Legal Research Bench: 37.98 (#5/15)
- Vals AI (Vals Index): 65.3 (#6/32)
- BridgeBench Security: 81.7 (#7/34)

Top-10 New Scores (6)

Qwen 3.7 Max on Agent Arena: 0.36 (#30)
Qwen 3.7 Max on Agent Arena - Bash Recovery: 7.91 (#20)
Qwen 3.7 Max on Agent Arena - Confirmed Success: 1.43 (#29)
Qwen 3.7 Max on Agent Arena - Praise vs Complaint: 0.52 (#31)
Qwen 3.7 Max on Agent Arena - Steerability: 5.2 (#22)
Qwen 3.7 Max on Agent Arena - Tool Hallucination: 1.02 (#14)

New #1 Leaders (7)

LiveBench Simplify: Gemini 1.5 Flash (Preview 0827) (84.75) beat Gemini 3.1 Pro (Preview) (High) by 13.7
TabArena All Tasks: TabFM (default) (89.0) beat TabPFN-3 (default) by 11.6
SWE-Marathon: Grok 4.5 (29.0) beat Claude Opus 4.8 by 3.0
LiveBench Paraphrase: Llama 3.1 70B Instruct (82.25) beat Gemini 3.1 Pro (Preview) (High) by 2.38
LiveBench Story Generation: Gemini 1.5 Flash (Preview 0827) (81.33) beat GPT-5.4 (xHigh) by 2.31
Vals AI Harvey Legal Agent Bench: Grok 4.5 (12.92) beat Claude Fable 5 by 1.67
Tau3 Banking: Grok 4.5 (High) (32.58) beat GPT-5.5 (xHigh) by 1.24

The Aggregate Digest — 2026-07-08

2026-07-08T07:17:07.129003+00:00

Daily

New #1 Leaders (5)

PIIMB PII Masking - OpenPII F2: OpenMed privacy-filter-nemotron-v2 (97.63) beat hivetrace gliner-guard-omni by 4.58
PIIMB PII Masking - Avg F1: OpenMed privacy-filter-multilingual-v2 (92.38) beat OpenMed OpenMed-PII-SuperClinical-Large-434M-v1 by 4.43
Senior SWE-Bench: Claude Fable 5 (27.9) beat Claude Opus 4.8 by 3.9
PIIMB PII Masking - Gretel F2: OpenMed privacy-filter-nemotron-v2 (98.28) beat OpenMed OpenMed-PII-SuperClinical-Small-44M-v1 by 2.99
PIIMB PII Masking - Avg F2: OpenMed privacy-filter-multilingual-v2 (91.68) beat OpenMed OpenMed-PII-SuperClinical-Large-434M-v1 by 2.94

The Aggregate Digest — 2026-07-07

2026-07-07T08:03:20.636451+00:00

Daily

New Benchmarks (1)

SkillsBench (With-skills score (%)): leader GPT-5.5 (67.3), 21 models
SkillsBench (BenchFlow) evaluates how effectively agentic models use modular Skills — folders of instructions, scripts, and resources — to complete tasks across diverse domains; scored by task completion with the Skills available (the benchmark also reports skill-lift, the with-minus-without-Skills gain).

New #1 Leaders (1)

Agent Arena - Confirmed Success: minimax-m2.7 MiniMax · Modified MIT (19.55) beat Claude Fable 5 (High) by 3.43