<?xml version="1.0" encoding="utf-8"?>
<?xml-stylesheet type="text/xsl" href="/feed.xsl"?>
<feed xmlns="http://www.w3.org/2005/Atom"><title>The Aggregate Digest</title><subtitle>AI benchmark changes — new models, leader shifts, and trends</subtitle><link href="https://theaggregate.ai/data/feed.xml" rel="self" /><link href="https://theaggregate.ai/whats-new?ref=atom" rel="alternate" /><id>https://theaggregate.ai/feed</id><icon>https://theaggregate.ai/favicon.svg</icon><updated>2026-11-16T08:17:49.919609+00:00</updated><entry><title>The Aggregate Digest — 2026-11-16</title><id>https://theaggregate.ai/digest/2026-11-16</id><updated>2026-11-16T08:17:49.919609+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (26)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;APEX-SWE&lt;/strong&gt; (Mean score over 4 attempts of 200 tasks (%)): leader Claude Opus 5.5 (Max) (67.62), 10 models&lt;br&gt;&lt;span&gt;Integration and observability engineering tasks in containerized repositories, checked by program; the score is the mean over 4 attempts on Mercor&amp;#x27;s 200-task held-out set, supplied by Mercor. It targets service wiring and monitoring work rather than issue fixing.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ATLAS-Finance&lt;/strong&gt; (Tasks passed, mean of 3 runs of 100 tasks (%)): leader Claude Opus 5 (High) (12.33), 11 models&lt;br&gt;&lt;span&gt;An agent inside a simulated advisory firm finds its assignment in inboxes, chat and drives, builds a live-formula Excel model and sends it to the right people; the score is the pass rate over 3 runs of 100 tasks, run by Handshake in the OpenCode harness. Inputs are scattered and sometimes corrected mid-task, unlike single-prompt finance tests.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 39 Issue / 20 Solution&lt;/strong&gt; (Pass rate over 39 issue judgments and 20 solution runs (%)): leader GPT-6 Astra (Max) (39.0), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench 39 Issue / 20 Solution — AutoRLBench native 39-issue/20-solution scored task-set edition, reviewed 16 November 2026. One run per model/task/level; issue compares root-cause answers against a hidden oracle, solution reruns the repaired workspace with metric thresholds and reward-hacking failure. Overall is total passes over59scored runs. README news retains32/24, but the exact shared task-ID sets and every one-decimal table score bind39/20. Native harness and effort are retained. Separate from33/20,32/18 and35/20 editions. Metric: Pass rate over 39 issue judgments and 20 solution runs (%). Source: https://thinking-space.github.io/AutoRLBench/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 39 Issue / 20 Solution - Issue&lt;/strong&gt; (Root cause found, 39 failed training runs (%)): leader GPT-6 Astra (Max) (38.5), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench 39 Issue / 20 Solution - Issue — AutoRLBench native 39-issue/20-solution scored task-set edition, reviewed 16 November 2026. One run per model/task/level; issue compares root-cause answers against a hidden oracle, solution reruns the repaired workspace with metric thresholds and reward-hacking failure. Overall is total passes over59scored runs. README news retains32/24, but the exact shared task-ID sets and every one-decimal table score bind39/20. Native harness and effort are retained. Separate from33/20,32/18 and35/20 editions. Metric: Root cause found, 39 failed training runs (%). Source: https://thinking-space.github.io/AutoRLBench/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 39 Issue / 20 Solution - Solution&lt;/strong&gt; (Repaired run passes its rerun, 20 tasks (%)): leader Gemini 4 Argon (Max) (60.0), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench 39 Issue / 20 Solution - Solution — AutoRLBench native 39-issue/20-solution scored task-set edition, reviewed 16 November 2026. One run per model/task/level; issue compares root-cause answers against a hidden oracle, solution reruns the repaired workspace with metric thresholds and reward-hacking failure. Overall is total passes over59scored runs. README news retains32/24, but the exact shared task-ID sets and every one-decimal table score bind39/20. Native harness and effort are retained. Separate from33/20,32/18 and35/20 editions. Metric: Repaired run passes its rerun, 20 tasks (%). Source: https://thinking-space.github.io/AutoRLBench/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Bedside Bench&lt;/strong&gt; (Macro-average of 10 suite means, 3 runs of 500 cases (%)): leader Claude Opus 5 (Max) (94.6), 15 models&lt;br&gt;&lt;span&gt;Physician-authored clinical cases in ten suites from differential diagnosis to medication safety, graded by an LLM judge against the authors&amp;#x27; criteria; the score is the macro-average of suite means over 3 Fireworks runs of 500 cases. Answers are free-text case work, not exam multiple choice.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CorporateBench&lt;/strong&gt; (Mean score over 3 runs of 128 tasks (%)): leader GPT-6.1 Sol (Max) (82.2093), 12 models&lt;br&gt;&lt;span&gt;An agent extracts and queries facts across corporate document collections of a few hundred to more than 230,000 emails and files; the score is the mean over 3 Fireworks runs of all 128 tasks. It tests retrieval at e-discovery scale rather than drafting legal work product.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DAYJOB: Accounting&lt;/strong&gt; (Attempts meeting at least 95% of rubric criteria (%)): leader Claude Sonnet 5.5 (Max) (35.2), 16 models&lt;br&gt;&lt;span&gt;Long-horizon accounting assignments from a brief request and a workspace of files, graded by an agent judge against an expert rubric; the score is the share of attempts meeting at least 95% of criteria. Surge AI supplied these results before publishing the board and has not stated the task count.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DAYJOB: Data Science&lt;/strong&gt; (Attempts meeting at least 95% of rubric criteria (%)): leader Claude Opus 5.5 (Max) (37.3), 16 models&lt;br&gt;&lt;span&gt;Long-horizon data-science assignments from a brief request and a workspace of files, graded by an agent judge against an expert rubric; the score is the share of attempts meeting at least 95% of criteria. Surge AI supplied these results before publishing the board and has not stated the task count.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DAYJOB: Law&lt;/strong&gt; (Attempts meeting at least 95% of rubric criteria (%)): leader GPT-6 Astra (Max) (36.9), 16 models&lt;br&gt;&lt;span&gt;Long-horizon legal assignments from a brief request and a workspace of files, graded by an agent judge against an expert rubric; the score is the share of attempts meeting at least 95% of criteria. Surge AI supplied these results before publishing the board and has not stated the task count.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierMemory1.0 Letta Dream memory model (best of two taskmodels at max)&lt;/strong&gt; (Full pass rate (%)): leader Claude Opus 5.5 (Max) (73.3), 7 models&lt;br&gt;&lt;span&gt;FrontierMemory1.0 Letta Dream memory model (best of two taskmodels at max) — Memory-generation model uses Letta Dream; Letta Code task agent uses Opus5.5 or GPT6Sol, native leaderboard publishes better of the two at max task effort.15tasks×3independent runs. This is not a same-model-agent score. Metric: Full pass rate (%). Source: https://www.frontiermemory.com/blog/v1/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hebbia Financial Services Benchmark&lt;/strong&gt; (Strict pass rate on 151 tasks (%)): leader Claude Opus 5.5 (Max) (74.6), 10 models&lt;br&gt;&lt;span&gt;Analyst requests answered over document sets, as Excel models or as client-ready decks, run by Hebbia in its production agent harness; the score is the strict pass rate on 151 tasks. The deliverables are files a client would receive, not short answers.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Legacy-Bench&lt;/strong&gt; (Tasks passed, mean of 3 runs of 100 tasks (%)): leader Kimi K3.1 Pro (Max) (63.0), 13 models&lt;br&gt;&lt;span&gt;Coding agents fix bugs, add features and migrate code in COBOL, Java 7, BASIC, C89, Fortran and Assembly, each task checked by hidden tests; the score is the pass rate over 3 Fireworks runs of 100 tasks. SWE-bench-style boards use modern Python and JavaScript repositories instead.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LuminaBench&lt;/strong&gt; (Lumina Capabilities Index (mean owner rating of three builds)): leader Claude Fable 5.2 (Harness: Claude Code) (High) (1116.0), 10 models&lt;br&gt;&lt;span&gt;A model in its vendor&amp;#x27;s coding agent builds one voxel pagoda, one 3D seaplane scene and one Mario Kart-style game, and the number is the mean of the three build ratings. Each build is rated on LuminaBench&amp;#x27;s own rating scale by the method recorded for it: rubric reviews with pairwise matchups, a rubric gameplay assessment, or the site owner&amp;#x27;s relative placement, none of them blinded. Unlike WebDev Arena&amp;#x27;s crowd votes over many prompts, each configuration is one build per test.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NCKI kspine_v3&lt;/strong&gt; (NCKI kspine_v3 index): leader GPT-6 Astra (131.08), 285 models&lt;br&gt;&lt;span&gt;NCKI kspine_v3 — Frozen bank/rung placement no-chain-of-thought benchmark; NCRI15.2 reasoning and NCKI kspine_v3 knowledge are separate native index scales, not percentages. Source chart eligibility flags are retained, never used as automatic filters. Explicit required filler configuration remains a source run. Metric: NCKI kspine_v3 index. Source: https://github.com/neelnanda-io/nocot-bench&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NCRI 15.2&lt;/strong&gt; (NCRI 15.2 index): leader GPT-6 Astra (159.04), 293 models&lt;br&gt;&lt;span&gt;NCRI 15.2 — Frozen bank/rung placement no-chain-of-thought benchmark; NCRI15.2 reasoning and NCKI kspine_v3 knowledge are separate native index scales, not percentages. Source chart eligibility flags are retained, never used as automatic filters. Explicit required filler configuration remains a source run. Metric: NCRI 15.2 index. Source: https://github.com/neelnanda-io/nocot-bench&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SecureBio BioTIER Permit&lt;/strong&gt; (Benign prompts answered (%)): leader GPT-Rosalind-5.5 (xHigh) (100.0), 77 models&lt;br&gt;&lt;span&gt;SecureBio BioTIER Permit — BioTIER separate benign compliance and hazardous refusal behavior panels, not biological task capability. Native percentages and model/run labels retained; no automatic eligibility/exclusion policy. Metric: Benign prompts answered (%). Source: https://securebio.org/benchmarks/details/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SecureBio BioTIER Refuse&lt;/strong&gt; (Hazardous prompts refused (%)): leader Claude Opus 5.5 (Max) (99.1), 77 models&lt;br&gt;&lt;span&gt;SecureBio BioTIER Refuse — BioTIER separate benign compliance and hazardous refusal behavior panels, not biological task capability. Native percentages and model/run labels retained; no automatic eligibility/exclusion policy. Metric: Hazardous prompts refused (%). Source: https://securebio.org/benchmarks/details/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SecureBio HPCT&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 4 [16k] (SecureBio Run: native-setting=Claude Opus 4 [16k]) (91.66666666666667), 109 models&lt;br&gt;&lt;span&gt;SecureBio HPCT — Native SecureBio benchmark panel; exact named population. Accuracy includes native refusal/failure conventions and source-reported runs. Raw fractions converted only according to this panel: ABLEFull already uses percent, baseline-only and other capability panels use fractions. Metric: Accuracy (%). Source: https://securebio.org/benchmarks/details/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SecureBio VCT v2&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (60.54055616910381), 27 models&lt;br&gt;&lt;span&gt;SecureBio VCT v2 — Native SecureBio benchmark panel; exact named population. Accuracy includes native refusal/failure conventions and source-reported runs. Raw fractions converted only according to this panel: ABLEFull already uses percent, baseline-only and other capability panels use fractions. Metric: Accuracy (%). Source: https://securebio.org/benchmarks/details/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SPU Private Mean&lt;/strong&gt; (Spearman correlation): leader Claude Opus 5.5 (Max) (0.42), 12 models&lt;br&gt;&lt;span&gt;SPU Private Mean — Public100tasks/private16tasks reported separately. Mainmeans exclude safetyrefusals and unrecoverableoutputs; schema-recoverable rankings included unchanged. ScoresareweightedSpearmancorrelation over experimentgroups, bounds[-1,1]. Task-adjusted mean is fitted additive task/configuration estimate and not guaranteed bounded, so do not infer [-1,1] bounds for adjusted estimate. Native per-row taskcoverage retained. No metric from APIcost/table3.    Metric: Spearman correlation. Source: https://arxiv.org/html/2611.15049v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SPU Public Raw Mean&lt;/strong&gt; (Spearman correlation): leader Claude Opus 5.5 (Max) (0.491), 12 models&lt;br&gt;&lt;span&gt;SPU Public Raw Mean — Public100tasks/private16tasks reported separately. Mainmeans exclude safetyrefusals and unrecoverableoutputs; schema-recoverable rankings included unchanged. ScoresareweightedSpearmancorrelation over experimentgroups, bounds[-1,1]. Task-adjusted mean is fitted additive task/configuration estimate and not guaranteed bounded, so do not infer [-1,1] bounds for adjusted estimate. Native per-row taskcoverage retained. No metric from APIcost/table3.    Metric: Spearman correlation. Source: https://arxiv.org/html/2611.15049v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SPU Public Task-adjusted Mean&lt;/strong&gt; (Spearman correlation): leader Claude Opus 5.5 (Max) (0.478), 12 models&lt;br&gt;&lt;span&gt;SPU Public Task-adjusted Mean — Public100tasks/private16tasks reported separately. Mainmeans exclude safetyrefusals and unrecoverableoutputs; schema-recoverable rankings included unchanged. ScoresareweightedSpearmancorrelation over experimentgroups, bounds[-1,1]. Task-adjusted mean is fitted additive task/configuration estimate and not guaranteed bounded, so do not infer [-1,1] bounds for adjusted estimate. Native per-row taskcoverage retained. No metric from APIcost/table3.    Metric: Spearman correlation. Source: https://arxiv.org/html/2611.15049v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;StudyArena Text - Weighted score&lt;/strong&gt; (Weighted score): leader LongCat 2.0 (High) (1375.0), 105 models&lt;br&gt;&lt;span&gt;StudyArena Text - Weighted score — Native weighted overall/all ballot cohort of student answer preferences. Full Flight roster, including low-ballot observations; the UI50-ballot default is not an exclusion. Native weighted score is not relabelled Elo. Every model key, effort label and ballot count retained. Metric: Weighted score. Source: https://studyarena.com/leaderboard&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Thrive Helpdesk - Difficult Tickets&lt;/strong&gt; (Correct ticket resolution (%)): leader GPT-6 Astra (xHigh) (61.53846153846154), 15 models&lt;br&gt;&lt;span&gt;Thrive Helpdesk - Difficult Tickets — 57native model/effort configurations, each52difficult helpdesk tickets. Shared proprietary Forge harness/scheduler/tools/lifecycle; score100×correct/graded. Tax tables in default SSR are a different evaluation and are never parsed as helpdesk. Metric: Correct ticket resolution (%). Source: https://evals.thriveholdings.com/?benchmark=it-helpdesk#it-helpdesk&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WitnessBench&lt;/strong&gt; (OAE-L20 score on 26 games x 3 seeds (0-100)): leader Claude Opus 5.5 (Max) (46.5), 13 models&lt;br&gt;&lt;span&gt;An agent plays 26 unseen line-puzzle games of 20 levels each, 3 seeds per game, without being told the rules and must infer them from the game&amp;#x27;s responses; the score rewards solving levels in close to the fewest actions, from Fleet&amp;#x27;s WitnessBench v2.0 runs. It tests rule discovery by experiment, unlike puzzle boards that state the rules.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (122)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arena AI Code: 1837.81 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LMArena WebDev Arena: 1837.81 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sycophancy Eval: 8.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sycophancy Eval - Instability (Number-of-Flip): 0.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sycophancy Eval - Moral Sycophancy: 0.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sycophancy Eval - Regressive Backdown: 0.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sycophancy Eval - Validation Sycophancy: 50.0 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - ALT - Bias: 80.0 (#127)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - ALT - Controllability: 93.68 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - ALT - Ethics &amp; Morality: 96.0 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - ALT - Robustness: 99.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - ALT - Toxicity: 78.26 (#105)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - ALT - Truthfulness: 88.23 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - ALT Average: 89.19 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - ARC-AGI-1: 98.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - ARC-AGI-2: 98.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - BFCL - Irrelevance Detection: 86.67 (#44)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - BFCL - Live AST: 63.89 (#136)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - BFCL - Multi-Turn: 33.33 (#108)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - BFCL - Non-Live AST: 75.56 (#82)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - BFCL - Relevance Detection: 55.56 (#131)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Abstract Reasoning: 98.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Application Development: 74.21 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Applied Language: 90.75 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Coding: 90.83 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Expression: 98.83 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Function Calling: 57.59 (#113)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - General Knowledge: 82.52 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Information Retrieval: 82.03 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Knowledge &amp; QA: 81.87 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Logical Reasoning: 82.5 (#108)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Mathematical Reasoning: 95.33 (#52)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Reasoning: 91.94 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Semantic Analysis: 86.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Specialized Knowledge: 81.22 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP - Translation: 91.4 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - GLP Average: 85.61 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Accuracy: 55.15 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Calibration Error: 35.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - HalluLens: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - JBBQ (2-shot) - Accuracy: 96.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - JHumanEval: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - JTruthfulQA: 76.46 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - M-IFEval (Japanese): 92.04 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese): 95.94 (#54)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Coding: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Extraction: 95.0 (#44)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Humanities: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Math: 94.5 (#113)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Roleplay: 98.5 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - STEM: 99.0 (#60)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Writing: 98.0 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - SWE-Bench (Japanese): 72.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - Total Score: 86.52 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - Toxicity - Fairness: 97.18 (#44)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - Toxicity - Prohibited Acts: 76.56 (#136)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - Toxicity - Social Norms: 98.02 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - Toxicity - Violation Categories: 41.27 (#77)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot): 87.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - AIO: 81.49 (#77)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT English-to-Japanese: 92.22 (#55)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT Japanese-to-English: 90.38 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - CommonsenseMoralJA: 97.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JBLiMP: 98.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (in-domain): 87.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (out-of-domain): 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCommonsenseQA: 98.0 (#45)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JEMHopQA: 57.16 (#54)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JMMLU: 96.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JNLI: 87.0 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSICK: 82.0 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSQuAD: 81.6 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSeM: 83.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaMP: 80.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaNLI: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - KUCI: 91.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MAWPS: 98.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MGSM: 93.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MMLU-ProX (Japanese): 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - NIILC: 60.62 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - AIO: 95.04 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT English-to-Japanese: 92.29 (#64)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT Japanese-to-English: 90.71 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JBLiMP: 95.0 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (in-domain): 87.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (out-of-domain): 91.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCommonsenseQA: 99.0 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JEMHopQA: 73.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JMMLU: 96.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JNLI: 91.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSICK: 85.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSQuAD: 82.46 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSeM: 82.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaMP: 79.0 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaNLI: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - KUCI: 92.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MAWPS: 99.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MGSM: 93.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MMLU-ProX (Japanese): 93.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - NIILC: 62.01 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Argo-Bench: 62.35 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Argo-Bench - Solved: 38.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Coding: 70.5 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Education - Gaokao: 69.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Education - High School Subjects: 78.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Education - Middle School Subjects: 60.1 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Education - Primary School Subjects: 72.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Finance: 89.9 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Language and Instruction Following: 81.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Law - Lawyer Qualification Exam: 80.0 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Law and Civil Service: 86.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Medicine - Basic Medical Knowledge: 90.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Medicine - Physician Exams: 88.7 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Medicine and Mental Health: 90.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Overall: 80.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Reasoning - BBH: 87.1 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ReLE - Reasoning and Mathematics: 88.1 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Sycophancy Eval: 17.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Sycophancy Eval - Instability (Number-of-Flip): 33.35 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Sycophancy Eval - Moral Sycophancy: 25.0 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Sycophancy Eval - Regressive Backdown: 33.35 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Sycophancy Eval - Validation Sycophancy: 0.0 (#5)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (52)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Sparse Retrieval Pruning&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (94.59828694) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 69.24&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Incremental Vector Search&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (97.80015102) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 50.32&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Negative-Constraint Retrieval&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (100.0) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 40.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - ChunQiu Temporal Retrieval&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (94.60573513) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 37.84&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Biomedical Graph Retrieval&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (65.0) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 35.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - ICSI Meeting Memory&lt;/strong&gt;: GPT-6 Astra (Max) (Search-SWE Run: Codex v0.164.0) (38.30845771) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 34.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Spoken-Query Document Search&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (45.47912566) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 31.32&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Join-aware Table Retrieval&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) (60.0) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 30.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Memory-constrained Dense Retrieval&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (100.0) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 27.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Filtered ANN Retrieval&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (100.0) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 26.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Narrative-Clue Search&lt;/strong&gt;: GPT-6 Astra (Max) (Search-SWE Run: Codex v0.164.0) (50.9279698) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 26.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Evolving Personal Memory&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (67.92784433) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 25.77&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Paper Metadata Retrieval&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (62.5) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 25.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Rust BM25 Search&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (75.63054003) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 22.53&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Arena AI Code&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (1837.81) beat Claude Opus 5.5 (Max) by 20.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LMArena WebDev Arena&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (1837.81) beat Claude Opus 5.5 (Max) by 20.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Scientific-Paper Question Answering&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (80.0) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 20.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Long-PDF Sentence Evidence — Vectorless&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (83.33333333) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 19.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Lightweight Query Retrieval&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (93.99696727) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 17.77&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Document Page Search&lt;/strong&gt;: GPT-6 Astra (Max) (Search-SWE Run: Codex v0.164.0) (42.77184087) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 16.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Code Reranker Fine-tuning&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) (52.27272727) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 15.91&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Training Error Tracing&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) (100.0) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 15.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Layout Page Search&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (34.77185041) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 11.22&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Long-PDF Sentence Evidence&lt;/strong&gt;: GPT-6 Astra (Max) (Search-SWE Run: Codex v0.164.0) (75.0) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 11.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Reasoning-intensive Query Rewriting&lt;/strong&gt;: GPT-6 Astra (Max) (Search-SWE Run: Codex v0.164.0) (70.0) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 10.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Visual-Document Question Answering&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (53.1699698) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 8.84&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Code Context Retrieval&lt;/strong&gt;: GPT-6 Astra (Max) (Search-SWE Run: Codex v0.164.0) (41.52250154) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 8.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Budgeted Candidate Reranking&lt;/strong&gt;: GPT-6 Astra (Max) (Search-SWE Run: Codex v0.164.0) (33.32150279) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 8.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Protein–Text Bidirectional Retrieval&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (30.475) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 6.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - GPU&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309; model-substitutions=constrained-molecular-search=anthropic/claude-opus-5) (46.93022393) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 6.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HieroglyphBench&lt;/strong&gt;: Gemini 4 Carbon (76.0) beat Gemini 4 Argon by 6.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Long-document Reranking&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (48.0) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 6.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309; model-substitutions=constrained-molecular-search=anthropic/claude-opus-5) (48.41088225) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 5.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Financial-Report Evidence Retrieval&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (45.23504094) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 5.26&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Composed Image Search&lt;/strong&gt;: GPT-6 Astra (Max) (Search-SWE Run: Codex v0.164.0) (51.99019004) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 5.14&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - CPU&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (49.82984647) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 5.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Sketch-and-Text Image Search&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (60.59619387) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 5.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Legal Stance Retrieval&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (73.24458801) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 4.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Quantity-aware Retrieval&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (39.73652549) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 4.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Agentic Search&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (51.41666667) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 2.58&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Constrained Molecular Search&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (20.569) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 2.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JEMHopQA&lt;/strong&gt;: GPT-6 Astra (Max) (73.9) beat Gemini 3 Pro (Preview) by 1.97&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Decision Model Training&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (61.35714286) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 1.83&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Khmer-to-English Embedding Training&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (34.30864198) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 1.26&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (0-shot) - JCoLA (in-domain)&lt;/strong&gt;: GPT-6 Astra (Max) (87.0) beat Gemini 3 Flash (Preview) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - MMLU-ProX (Japanese)&lt;/strong&gt;: GPT-6 Astra (Max) (93.0) beat GPT-6.1 Terra (Max) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Basic Language&lt;/strong&gt;: GPT-6 Astra (89.28) beat GPT-6.1 Sol by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Multilingual Vector Postprocessing&lt;/strong&gt;: GPT-6.1 Sol (Max) (Search-SWE Run: Codex v0.164.0) (43.1234796) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 0.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education&lt;/strong&gt;: Gemini 4 Carbon (70.2) beat Claude Fable 5.2 by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Video-Moment Search&lt;/strong&gt;: Claude Opus 5.5 (Max) (Search-SWE Run: Claude Code v2.1.309) (32.46086976) beat GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) by 0.31&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot)&lt;/strong&gt;: GPT-6 Astra (89.24) beat GPT-6.1 Sol by 0.26&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Syntactic Analysis&lt;/strong&gt;: GPT-6 Astra (91.67) beat Claude Opus 5.5 by 0.17&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-16

=== DAILY ===
NEW BENCHMARKS (26)
  - APEX-SWE (Mean score over 4 attempts of 200 tasks (%)): leader Claude Opus 5.5 (Max) (67.62), 10 models
      Integration and observability engineering tasks in containerized repositories, checked by program; the score is the m</summary></entry><entry><title>The Aggregate Digest — 2026-11-15</title><id>https://theaggregate.ai/digest/2026-11-15</id><updated>2026-11-15T10:29:02.484467+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (39)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 35 Issue / 20 Solution&lt;/strong&gt; (Pass rate over 35 issue judgments and 20 solution runs (%)): leader GPT-6 Astra (Max) (41.8), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 35 Issue / 20 Solution - Issue&lt;/strong&gt; (Root cause found, 35 failed training runs (%)): leader GPT-6 Astra (Max) (42.9), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 35 Issue / 20 Solution - Solution&lt;/strong&gt; (Repaired run passes its rerun, 20 tasks (%)): leader Gemini 4 Argon (Max) (60.0), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gamow LabRobotBench v0.1&lt;/strong&gt; (Mean dense recovery score (fraction, 0–1)): leader GPT-6.1 Sol (0.467006600721), 6 models&lt;br&gt;&lt;span&gt;LabRobotBench v0.1 — Recovery of robotic laboratory workflows across3tasks,one attempt per configuration. Mean dense recovery score includes inherited work and uses a retrospectively chosen rubric;all six systems pass0/3strict tasks. Source-native provider/harness/runtime differences remain explicit;score fraction0–1.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gamow ScreenBench&lt;/strong&gt; (Mean normalized answer score (fraction, 0–1)): leader Claude Fable 5.2 (0.702552747253), 7 models&lt;br&gt;&lt;span&gt;ScreenBench — Agents inspect compound-screening plate records to detect and correct hidden errors across182tasks. Mean normalized graded-answer score,0no better than shortcuts and1all answers right. Unprompted condition;the prompt listing possible errors is a separate experiment. Native harness/graded population and score revision retained. Native cohorts differ: historical rows use named cell lines; Astra, Carbon and Fable5.2 use coded names; Hydrogen is unspecified. Sol helped select95tasks; evaluation uses fresh runs.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Bearing Fault Transfer&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.6862), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Bearing Fault Transfer — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Chess Puzzle Post-Training&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5171375), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Chess Puzzle Post-Training — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Concrete Frame Sizing&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.598825), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Concrete Frame Sizing — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Design/Build/Fly Aircraft&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.7152875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Design/Build/Fly Aircraft — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Drawing to CAD&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.4376875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Drawing to CAD — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Earthquake Sequence Relocation&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5074875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Earthquake Sequence Relocation — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Engineering&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5962575), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Engineering — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Free-Form CAD Reconstruction&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.83805), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Free-Form CAD Reconstruction — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Function Page Layout&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.8686125), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Function Page Layout — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - GPU4PySCF J/K Kernels&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.167125), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - GPU4PySCF J/K Kernels — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - INT4 Decode Attention&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Opus 5.5 (Max) (0.627025), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - INT4 Decode Attention — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - LVGL Renderer Speedup&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5809875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - LVGL Renderer Speedup — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - MicroPython VM Speedup&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5797875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - MicroPython VM Speedup — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - ML&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.545644642857), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - ML — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Overall&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.545673557692), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Overall — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Pairwise-Gated Attention Engine&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.6435375), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Pairwise-Gated Attention Engine — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Plume Dispersion Forecast&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Gemini 4 Argon (High) (0.159425), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Plume Dispersion Forecast — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Scan to CAD&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.4971375), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Scan to CAD — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Science&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.401922916667), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Science — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Seismic Wave Propagation&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Opus 5.5 (Max) (0.2306625), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Seismic Wave Propagation — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Sentinel-2 Gap Filling&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.40305), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Sentinel-2 Gap Filling — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Skewed Cyclic Joins&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.713675), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Skewed Cyclic Joins — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - SmolLM2 Long Context&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.778875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - SmolLM2 Long Context — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - SmolLM3 Draft Head&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Gemini 4 Argon (High) (0.3919), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - SmolLM3 Draft Head — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Software&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.621896875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Software — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - SPARQL Store Speedup&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.9387625), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - SPARQL Store Speedup — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Spectral Kinetics Forecast&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.8148375), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Spectral Kinetics Forecast — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - SQL Change Replay&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Sonnet 5.5 (Max) (0.31185), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - SQL Change Replay — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Stateful Parquet Lakehouse&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.1 (Max) (0.4969625), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Stateful Parquet Lakehouse — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Storm Surge Forecast&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.611875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Storm Surge Forecast — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Streaming Payments ETL&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.838475), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Streaming Payments ETL — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SPU - Private - Raw&lt;/strong&gt; (Mean Spearman correlation): leader Claude Opus 5.5 (Max) (0.42), 12 models&lt;br&gt;&lt;span&gt;SPU - Private - Raw — SPU v1: rank wet-lab experiment outcomes; per-group Spearman correlation, combined with fixed source group weights within tasks (equal in97/100public tasks, unequal in3). 100 public/16 private tasks; one official run per configuration/task with one infrastructure retry. 8-hour budget,8CPU cores,16GiB RAM,H20GPU; results/reconstruction prohibited. GPT-6 Astra and Sol use Codex0.156.1; other models use Claude Code2.1.260 at max effort. Task-adjusted public score is the source additive task/configuration model, not a new local fit. Raw means use only valid or unambiguously recovered rankings; refusals/missing files excluded, not zero. Exact row counts and CIs remain provenance: public scored coverage30–100/100; private2–16/16. Private mean is unadjusted; it is not a matched-cohort comparison. The default-grader -1 convention is a separate diagnostic. Metric: Mean Spearman correlation.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SPU - Public - Raw&lt;/strong&gt; (Mean Spearman correlation): leader Claude Opus 5.5 (Max) (0.491), 12 models&lt;br&gt;&lt;span&gt;SPU - Public - Raw — SPU v1: rank wet-lab experiment outcomes; per-group Spearman correlation, combined with fixed source group weights within tasks (equal in97/100public tasks, unequal in3). 100 public/16 private tasks; one official run per configuration/task with one infrastructure retry. 8-hour budget,8CPU cores,16GiB RAM,H20GPU; results/reconstruction prohibited. GPT-6 Astra and Sol use Codex0.156.1; other models use Claude Code2.1.260 at max effort. Task-adjusted public score is the source additive task/configuration model, not a new local fit. Raw means use only valid or unambiguously recovered rankings; refusals/missing files excluded, not zero. Exact row counts and CIs remain provenance: public scored coverage30–100/100; private2–16/16. Private mean is unadjusted; it is not a matched-cohort comparison. The default-grader -1 convention is a separate diagnostic. Metric: Mean Spearman correlation.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SPU - Public - Task-adjusted&lt;/strong&gt; (Mean Spearman correlation): leader Claude Opus 5.5 (Max) (0.478), 12 models&lt;br&gt;&lt;span&gt;SPU - Public - Task-adjusted — SPU v1: rank wet-lab experiment outcomes; per-group Spearman correlation, combined with fixed source group weights within tasks (equal in97/100public tasks, unequal in3). 100 public/16 private tasks; one official run per configuration/task with one infrastructure retry. 8-hour budget,8CPU cores,16GiB RAM,H20GPU; results/reconstruction prohibited. GPT-6 Astra and Sol use Codex0.156.1; other models use Claude Code2.1.260 at max effort. Task-adjusted public score is the source additive task/configuration model, not a new local fit. Raw means use only valid or unambiguously recovered rankings; refusals/missing files excluded, not zero. Exact row counts and CIs remain provenance: public scored coverage30–100/100; private2–16/16. Private mean is unadjusted; it is not a matched-cohort comparison. The default-grader -1 convention is a separate diagnostic. Metric: Mean Spearman correlation.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Zarya-4B&lt;/strong&gt; — Aggregate Rating 1435, #4525&lt;ul&gt;&lt;li&gt;MERA v2 - SAGE: 48.6 (#35/50)&lt;/li&gt;&lt;li&gt;MERA v2 - RussianRegions: 3.6 (#40/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Culture Specific: 16.6 (#40/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Characters: 27.2 (#42/50)&lt;/li&gt;&lt;li&gt;MERA v2 - RUBIN: 26.0 (#44/50)&lt;/li&gt;&lt;li&gt;MERA v2 - GorillaHard: 4.8 (#44/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Agentic: 2.5 (#44/50)&lt;/li&gt;&lt;li&gt;MERA v2: 0.079 (#45/50)&lt;/li&gt;&lt;li&gt;MERA v2 - LIMUR: 30.4 (#45/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Riddles: 5.0 (#45/50)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (196)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on GBA Eval: 0.848 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on GBA Eval - Audio: 0.4623 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on GBA Eval - Procedural Tests: 0.7778 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Klu LLM Leaderboard: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Personal Style Imitation (53 Texts): 73.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on GBA Eval - Audio: 0.665 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on GBA Eval - Gameplay Replays: 0.9955 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Bias: 80.0 (#126)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Controllability: 91.82 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Ethics &amp; Morality: 98.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Robustness: 98.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Toxicity: 74.94 (#122)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Truthfulness: 86.44 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT Average: 88.2 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ARC-AGI-1: 98.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ARC-AGI-2: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Irrelevance Detection: 80.0 (#91)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Live AST: 69.44 (#72)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Multi-Turn: 72.22 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Non-Live AST: 86.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Relevance Detection: 66.67 (#86)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Applied Language: 90.61 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Basic Language: 87.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - General Knowledge: 83.54 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Information Retrieval: 80.35 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Logical Reasoning: 94.5 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Mathematical Reasoning: 98.17 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Semantic Analysis: 84.3 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Translation: 91.48 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Accuracy: 63.92 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Calibration Error: 22.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - HalluLens: 99.0 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - JBBQ (2-shot) - Accuracy: 96.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - JHumanEval: 97.44 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - JTruthfulQA: 73.88 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - M-IFEval (Japanese): 88.5 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Coding: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Extraction: 99.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Humanities: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Math: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Roleplay: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - STEM: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Writing: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - Toxicity - Fairness: 98.31 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - Toxicity - Prohibited Acts: 74.48 (#136)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - Toxicity - Social Norms: 95.24 (#76)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - Toxicity - Violation Categories: 31.75 (#140)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - AIO: 90.41 (#40)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT English-to-Japanese: 92.32 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT Japanese-to-English: 90.52 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - CommonsenseMoralJA: 98.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JBLiMP: 98.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (in-domain): 83.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (out-of-domain): 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCommonsenseQA: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JEMHopQA: 65.66 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JMMLU: 97.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JNLI: 91.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSICK: 83.0 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSQuAD: 82.14 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSeM: 80.0 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaMP: 65.0 (#122)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaNLI: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - KUCI: 93.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MAWPS: 99.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MGSM: 95.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MMLU-ProX (Japanese): 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - NIILC: 64.28 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot): 88.01 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - AIO: 90.72 (#56)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT English-to-Japanese: 92.52 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT Japanese-to-English: 90.57 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JBLiMP: 98.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (in-domain): 87.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (out-of-domain): 91.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCommonsenseQA: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JEMHopQA: 64.5 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JMMLU: 93.0 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JNLI: 93.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSICK: 84.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSQuAD: 78.56 (#77)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSeM: 84.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaMP: 63.0 (#136)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaNLI: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - KUCI: 95.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MAWPS: 99.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MGSM: 96.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MMLU-ProX (Japanese): 91.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - NIILC: 59.81 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GBA Eval: 0.8705 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GBA Eval - Audio: 0.6648 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GBA Eval - Gameplay Replays: 0.9955 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GBA Eval - Procedural Tests: 0.7014 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on GBA Eval: 0.8191 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on GBA Eval - Audio: 0.6829 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on GBA Eval - Gameplay Replays: 0.935 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on GBA Eval - Procedural Tests: 0.6077 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Bias: 80.0 (#127)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Controllability: 94.53 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Ethics &amp; Morality: 97.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Robustness: 98.5 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Toxicity: 78.79 (#100)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Truthfulness: 88.32 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT Average: 89.52 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ARC-AGI-1: 98.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ARC-AGI-2: 96.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Irrelevance Detection: 88.33 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Live AST: 64.81 (#129)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Multi-Turn: 37.78 (#97)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Non-Live AST: 77.78 (#50)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Relevance Detection: 55.56 (#130)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Abstract Reasoning: 97.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Application Development: 75.02 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Applied Language: 90.69 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Coding: 89.92 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Expression: 98.5 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Function Calling: 60.12 (#92)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - General Knowledge: 81.58 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Information Retrieval: 82.18 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Knowledge &amp; QA: 81.23 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Logical Reasoning: 86.5 (#81)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Mathematical Reasoning: 95.83 (#41)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Reasoning: 93.11 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Semantic Analysis: 86.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Specialized Knowledge: 80.89 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Syntactic Analysis: 90.67 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Translation: 91.39 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP Average: 85.75 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Accuracy: 55.67 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Calibration Error: 35.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - HalluLens: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - JBBQ (2-shot) - Accuracy: 96.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - JHumanEval: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - JTruthfulQA: 76.63 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - M-IFEval (Japanese): 93.81 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese): 96.56 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Coding: 98.5 (#60)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Extraction: 97.0 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Humanities: 98.5 (#88)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Math: 96.0 (#95)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Roleplay: 98.5 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - STEM: 99.0 (#60)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Writing: 98.5 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - SWE-Bench (Japanese): 71.25 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Total Score: 86.8 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Toxicity - Fairness: 96.05 (#62)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Toxicity - Prohibited Acts: 78.65 (#133)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Toxicity - Social Norms: 98.41 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Toxicity - Violation Categories: 42.06 (#72)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot): 86.89 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - AIO: 79.97 (#79)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT English-to-Japanese: 92.19 (#59)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT Japanese-to-English: 90.36 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - CommonsenseMoralJA: 98.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JBLiMP: 97.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (in-domain): 84.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (out-of-domain): 90.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCommonsenseQA: 99.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JEMHopQA: 52.66 (#66)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JMMLU: 95.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JNLI: 87.0 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSICK: 81.0 (#48)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSQuAD: 81.28 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSeM: 82.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaMP: 81.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaNLI: 100.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - KUCI: 91.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MAWPS: 98.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MGSM: 94.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MMLU-ProX (Japanese): 92.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - NIILC: 59.15 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - AIO: 94.17 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT English-to-Japanese: 92.34 (#53)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT Japanese-to-English: 90.66 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JBLiMP: 96.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (in-domain): 87.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (out-of-domain): 90.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCommonsenseQA: 99.0 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JEMHopQA: 71.83 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JMMLU: 95.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JNLI: 91.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSICK: 85.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSQuAD: 83.08 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSeM: 81.0 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaMP: 79.0 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaNLI: 100.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - KUCI: 92.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MAWPS: 98.0 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MGSM: 93.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MMLU-ProX (Japanese): 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - NIILC: 61.98 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on DuelLab GameBench 2 - Reasoning Variants: 94.878522999998 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ErdosBench: 3.24 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Gert Labs Rankings: 72.9 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Klu LLM Leaderboard: 93.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on The Aggregate - Personal Style Imitation (53 Texts): 67.0 (#10)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (56)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - OCR Distillation Trajectory Sampling&lt;/strong&gt;: Gemini 4 Hydrogen (87.47737556561086) beat Gemini 4 Carbon by 18.61&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - HLE (Japanese) - Calibration Error&lt;/strong&gt;: Claude Opus 5.5 (Max) (22.0) beat Grok 4.5 (High) by 12.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - HLE (Japanese) - Accuracy&lt;/strong&gt;: Claude Opus 5.5 (Max) (63.92) beat GPT-5.6 Sol (Max) by 11.86&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - MedAraBench&lt;/strong&gt;: Karnak-Th-41B-v1.0 (64.24) beat Llama 3.3 70B Instruct by 9.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - MedArabiQ MCQ&lt;/strong&gt;: Karnak-Th-41B-v1.0 (66.96) beat Karnak-40B-v1.0 by 8.36&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - E-Graph Invariant Restoration&lt;/strong&gt;: Gemini 4 Hydrogen (68.53231954722582) beat Gemini 4 Carbon by 6.56&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - BFCL - Multi-Turn&lt;/strong&gt;: Claude Opus 5.5 (Max) (72.22) beat GPT-5.6 Sol (Max) by 5.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - JHumanEval&lt;/strong&gt;: GPT-6.1 Terra (Max) (100.0) beat Claude Opus 5 (Max) by 5.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - Overall&lt;/strong&gt;: Karnak-Th-41B-v1.0 (73.71) beat Qwen 3.5 397B A17B by 4.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Application Development&lt;/strong&gt;: Claude Opus 5.5 (84.44) beat Claude Opus 5 by 4.06&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Function Calling&lt;/strong&gt;: Claude Opus 5.5 (75.99) beat GPT-5.6 Sol by 3.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JCoLA (out-of-domain)&lt;/strong&gt;: GPT-6.1 Terra (Max) (94.0) beat DeepSeek V4 Flash (0731) (Max) by 3.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Coding&lt;/strong&gt;: Claude Opus 5.5 (92.9) beat Claude Opus 5 by 2.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Specialized Knowledge&lt;/strong&gt;: Claude Opus 5.5 (83.47) beat Claude Fable 5 by 2.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP Average&lt;/strong&gt;: Claude Opus 5.5 (88.74) beat Claude Opus 5 by 2.06&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JCoLA (in-domain)&lt;/strong&gt;: Claude Opus 5.5 (Max) (87.0) beat Llama 3.1 405B Instruct FP8 by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Knowledge &amp; QA&lt;/strong&gt;: Claude Opus 5.5 (83.51) beat GPT-5.6 Sol by 1.97&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Hypergraph Core Numbers&lt;/strong&gt;: Gemini 4 Hydrogen (61.4131399518806) beat Gemini 4 Argon by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - GAT&lt;/strong&gt;: Karnak-Th-41B-v1.0 (63.97) beat AIC-1 by 1.86&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - LRR&lt;/strong&gt;: Gemini 4 Hydrogen (34.760000000000005) beat Gemini 4 Carbon by 1.73&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Syntactic Analysis&lt;/strong&gt;: Claude Opus 5.5 (91.5) beat GPT-5.5 (2026-04-23) by 1.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Future Video Sampling&lt;/strong&gt;: Gemini 4 Hydrogen (77.80450536665703) beat Gemini 4 Carbon by 1.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (0-shot) - NIILC&lt;/strong&gt;: Claude Opus 5.5 (Max) (64.28) beat DeepSeek V4.1 Flash by 1.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - SWE-Bench (Japanese)&lt;/strong&gt;: Claude Opus 5.5 (81.25) beat Claude Fable 5 by 1.25&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - Total Score&lt;/strong&gt;: Claude Opus 5.5 (88.43) beat Claude Opus 5 by 1.23&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals Inference Provider - Baseten - Overall&lt;/strong&gt;: Kimi K3 (Max) (69.877) beat DeepSeek V4.1 Flash by 1.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Abstract Reasoning&lt;/strong&gt;: Claude Opus 5.5 (99.0) beat GPT-5.6 Terra by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (0-shot) - JCoLA (out-of-domain)&lt;/strong&gt;: GPT-6.1 Terra (Max) (93.0) beat Qwen3.8-Flash-Next (xHigh) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JBLiMP&lt;/strong&gt;: Claude Opus 5.5 (Max) (98.0) beat Gemini 3.6 Flash by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - KUCI&lt;/strong&gt;: Claude Opus 5.5 (Max) (95.0) beat Gemini 3.1 Pro (Preview) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - MGSM&lt;/strong&gt;: Claude Opus 5.5 (Max) (96.0) beat Claude Opus 5 (Max) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Basic Language&lt;/strong&gt;: GPT-6.1 Sol (88.68) beat GPT-5.5 (2026-04-23) by 0.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals Inference Provider - Fireworks - Finance Agent v2&lt;/strong&gt;: Kimi K3 (Max) (59.291) beat DeepSeek V4.1 Flash by 0.87&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - 3LM STEM&lt;/strong&gt;: Karnak-Th-41B-v1.0 (96.04) beat Qwen 3.5 35B A3B Base by 0.71&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Reasoning&lt;/strong&gt;: Claude Opus 5.5 (97.22) beat Claude Opus 5 by 0.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VulcanBench v3 (BenchLM)&lt;/strong&gt;: Claude Fable 5.2 (93.8) beat Grok 4.7 by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU IVF-PQ Search Kernel Optimization&lt;/strong&gt;: Gemini 4 Hydrogen (79.68875187703873) beat Gemini 4 Carbon by 0.53&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - MT-Bench (Japanese) - Writing&lt;/strong&gt;: Claude Opus 5.5 (Max) (100.0) beat Ling 3.1 Flash by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - FannOrFlop&lt;/strong&gt;: Qwen 3 VL 235B A22B Instruct (61.01) beat Qwen 3.5 122B A10B by 0.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - MT-Bench (Japanese)&lt;/strong&gt;: Claude Opus 5.5 (99.25) beat Claude Opus 5 by 0.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Differentiable Population Statistics&lt;/strong&gt;: Gemini 4 Hydrogen (7.238810160238226) beat Gemini 4 Carbon by 0.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals Inference Provider - Baseten - Vibe Code Bench&lt;/strong&gt;: DeepSeek V4.1 Flash (Max) (82.491) beat Kimi K3 by 0.42&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot)&lt;/strong&gt;: GPT-6.1 Sol (88.98) beat Claude Fable 5 by 0.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Expression&lt;/strong&gt;: Claude Opus 5.5 (100.0) beat Ling 3.1 Flash by 0.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GBA Eval - Procedural Tests&lt;/strong&gt;: Claude Opus 5.5 (0.8764) beat GPT-5.6 Sol by 0.32&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Exact Sequence Alignment&lt;/strong&gt;: Gemini 4 Hydrogen (67.19671459086022) beat Gemini 4 Argon by 0.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - KAMI FP64 Block GEMM — trusted batched harness adaptation&lt;/strong&gt;: Gemini 4 Hydrogen (23.24232055075762) beat Gemini 4 Carbon by 0.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Go1 Joystick Policy Training&lt;/strong&gt;: Gemini 4 Hydrogen (50.10215349872997) beat Gemini 4 Argon by 0.21&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (0-shot)&lt;/strong&gt;: Claude Opus 5.5 (88.14) beat GPT-5.5 (2026-04-23) by 0.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GBA Eval&lt;/strong&gt;: Claude Opus 5.5 (0.9056) beat Claude Opus 5 by 0.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GBA Eval - Gameplay Replays&lt;/strong&gt;: Claude Fable 5.2 (1.0) beat Claude Opus 5 by 0.07&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - FARGO paper-aligned query core&lt;/strong&gt;: Gemini 4 Hydrogen (52.65348694173612) beat Gemini 4 Carbon by 0.07&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - MetaOpt Combined-TE Gap Search&lt;/strong&gt;: Gemini 4 Hydrogen (52.00947866014176) beat Gemini 4 Carbon by 0.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Register-aware Branch Prediction&lt;/strong&gt;: Gemini 4 Hydrogen (31.270009842412797) beat Gemini 4 Carbon by 0.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JSQuAD&lt;/strong&gt;: GPT-6.1 Sol (Max) (83.08) beat Kimi K3.1 Pro (Max) by 0.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (0-shot) - ALT Japanese-to-English&lt;/strong&gt;: Claude Opus 5.5 (Max) (90.52) beat GPT-5.6 Sol (Max) by 0.02&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Weekly&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (110)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;AA Coding Agent Index v2.1&lt;/strong&gt; (Coding Agent Index (0-100)): leader Claude Fable 5.2 (Xhigh, Default Fallback) (AA Coding Agent Run: harness=Claude Code; native=Claude Fable 5.2 (Xhigh, Default Fallback); fallback=source-published) (58.47), 27 models&lt;br&gt;&lt;span&gt;Artificial Analysis Coding Agent Index v2.1 (November 2026) evaluates a model inside its named coding-agent harness. It averages FrontierCode-AA 2.0 (132 repository-editing tasks), Terminal-Bench 4.1 (66 terminal tasks with revised verifiers), and SWE-Atlas-QnA-AA (124 repository questions), each using three attempts per task and task-equal means. Audited reward-hacking attempts score zero; environment/verifier errors are excluded. Native model effort and harness remain separate; this edition differs from v2.0. This board reports the equal-weight index on a 0-100 scale.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Coding Agent Index v2.1 - FrontierCode-AA 2.0&lt;/strong&gt; (Mean Score (%)): leader Claude Fable 5.2 (Xhigh, Default Fallback) (AA Coding Agent Run: harness=Claude Code; native=Claude Fable 5.2 (Xhigh, Default Fallback); fallback=source-published) (49.18), 27 models&lt;br&gt;&lt;span&gt;Artificial Analysis Coding Agent Index v2.1 (November 2026) evaluates a model inside its named coding-agent harness. It averages FrontierCode-AA 2.0 (132 repository-editing tasks), Terminal-Bench 4.1 (66 terminal tasks with revised verifiers), and SWE-Atlas-QnA-AA (124 repository questions), each using three attempts per task and task-equal means. Audited reward-hacking attempts score zero; environment/verifier errors are excluded. Native model effort and harness remain separate; this edition differs from v2.0. This board reports FrontierCode-AA 2.0, higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Coding Agent Index v2.1 - SWE-Atlas-QnA-AA&lt;/strong&gt; (Task Resolve Rate (%)): leader Muse Wonder Full (Harness: Muse Code) (Max) (68.82), 27 models&lt;br&gt;&lt;span&gt;Artificial Analysis Coding Agent Index v2.1 (November 2026) evaluates a model inside its named coding-agent harness. It averages FrontierCode-AA 2.0 (132 repository-editing tasks), Terminal-Bench 4.1 (66 terminal tasks with revised verifiers), and SWE-Atlas-QnA-AA (124 repository questions), each using three attempts per task and task-equal means. Audited reward-hacking attempts score zero; environment/verifier errors are excluded. Native model effort and harness remain separate; this edition differs from v2.0. This board reports SWE-Atlas-QnA-AA, higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Coding Agent Index v2.1 - Terminal-Bench 4.1&lt;/strong&gt; (Pass@1 (%)): leader Kimi K3.1 Pro (Harness: Kimi Code CLI) (67.17), 27 models&lt;br&gt;&lt;span&gt;Artificial Analysis Coding Agent Index v2.1 (November 2026) evaluates a model inside its named coding-agent harness. It averages FrontierCode-AA 2.0 (132 repository-editing tasks), Terminal-Bench 4.1 (66 terminal tasks with revised verifiers), and SWE-Atlas-QnA-AA (124 repository questions), each using three attempts per task and task-equal means. Audited reward-hacking attempts score zero; environment/verifier errors are excluded. Native model effort and harness remain separate; this edition differs from v2.0. This board reports Terminal-Bench 4.1, higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience v1.1&lt;/strong&gt; (Net Score): leader Claude Fable 5.2 (Max, Default Fallback) (66.77), 77 models&lt;br&gt;&lt;span&gt;Artificial Analysis&amp;#x27;s AA-Omniscience v1.1 knowledge and hallucination benchmark: 3,000 factual-recall questions (300 of them public) across Business, Health, Humanities &amp;amp; Social Sciences, Law, Software Engineering, and Science, Engineering &amp;amp; Mathematics, graded by GPT-5.6 Luna (medium). Net score: the share of correct answers minus the share of incorrect ones, -100 to 100, so abstaining beats guessing wrong; higher is better. v1.1 replaced v1&amp;#x27;s 6,000 questions by dropping saturated ones and v1&amp;#x27;s 600 public ones, adding new questions and fixing ambiguous or wrong ones, so scores are not directly comparable with the v1 AA Omniscience board.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience v1.1 Accuracy&lt;/strong&gt; (Accuracy (%)): leader Claude Fable 5.2 (Max, Default Fallback) (79.13), 77 models&lt;br&gt;&lt;span&gt;Artificial Analysis&amp;#x27;s AA-Omniscience v1.1 accuracy: the share of its 3,000 factual-recall questions (Business, Health, Humanities &amp;amp; Social Sciences, Law, Software Engineering, Science, Engineering &amp;amp; Mathematics) answered correctly, 0-100 (%), higher is better; an abstention counts as not correct. Not directly comparable with v1&amp;#x27;s 6,000-question AA-Omniscience Accuracy.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience v1.1 Hallucination Rate&lt;/strong&gt; (Hallucination Rate (%)): leader Gemini 4 Carbon (Medium) (4.24), 77 models&lt;br&gt;&lt;span&gt;Artificial Analysis&amp;#x27;s AA-Omniscience v1.1 factual-recall benchmark (3,000 questions across six domains): of the questions a model did not answer correctly, the share it answered incorrectly instead of abstaining. Reported 0-100 (%); lower is better. Declining to answer lowers the rate, so near-silent small models score best; read it beside AA-Omniscience v1.1 Accuracy. Scores are not directly comparable with v1, which had 6,000 questions.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agent Money Bench&lt;/strong&gt; (Wasted spend per $100, both tracks weighted equally (USD, lower is better)): leader Claude Fable 5.2 (21.0), 12 models&lt;br&gt;&lt;span&gt;Agent Money Bench (BlockRun, version 1, run 27 October to 11 November 2026) tests whether an agent can be trusted to spend money: it buys the data it needs in USDC on Solana mainnet, answers the question and files an expense report. Unlike Vending-Bench, where the money is simulated, every payment settles on-chain. It has two tracks. Financial Research is 76 open questions answered with live data bought from real providers, graded against per-question rubrics by an AI judge with every score reviewed by hand. Adversarial Marketplace is 80 tasks buying from simulated sellers, some of which raise the price after quoting, let quotes expire, charge twice, deliver the wrong data or hide instructions in what they deliver; it is graded deterministically from the settlement, the answer and the expense report. Each model runs each task once, and the prompt gives no payment rules: it says the money is the client&amp;#x27;s and that there is a budget. This board is the headline composite: dollars wasted per $100 spent, with each track counting for $50. A marketplace task&amp;#x27;s whole spend is wasted unless the run succeeds, and a research task&amp;#x27;s spend is wasted in proportion to its rubric shortfall. Lower is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench&lt;/strong&gt; (Pass rate over 33 issue judgments and 20 solution runs (%)): leader GPT-6 Astra (Max) (47.2), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench gives a coding agent a real reinforcement-learning training run that went wrong, with its code, config, logs and metrics, and asks it to name the root cause. On tasks that have a repaired reference run the agent must also fix the workspace; the harness then reruns the training and a metric threshold on the new logs decides. Root-cause answers are graded by an LLM judge against a written oracle, and each model gets one run per task at maximum reasoning effort with no network access. Unlike MLE-bench, where agents build a model for a Kaggle task from scratch, every task starts from someone else&amp;#x27;s broken RL run and is scored on diagnosis and repair. This board is the share of all 53 runs passed: 33 root-cause judgments and 20 repair runs.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 32 Issue / 18 Solution&lt;/strong&gt; (Pass rate over 32 issue judgments and 18 solution runs (%)): leader GPT-6 Astra (Max) (46.0), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench native 32-issue/18-solution task-set edition. Same published root-cause and repaired-workspace protocols, one run per model/task/level. Issue is an LLM judgment against hidden oracle; solution is the harness rerun metric threshold, with reward hacking failing. Overall counts passes over all50runs, not an average of two task-level rates. The source README announces32issue tasks and24solution-capable tasks, but exactly18 have scored solution runs; do not use24as the reported denominator. Exact task-ID sets are pinned and all rates reproduce the native table to its stated one decimal. Separate from the previously reviewed33/20task-set edition. This board reports Pass rate over 32 issue judgments and 18 solution runs (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 32 Issue / 18 Solution - Issue&lt;/strong&gt; (Root cause found, 32 failed training runs (%)): leader GPT-6 Astra (Max) (46.9), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench native 32-issue/18-solution task-set edition. Same published root-cause and repaired-workspace protocols, one run per model/task/level. Issue is an LLM judgment against hidden oracle; solution is the harness rerun metric threshold, with reward hacking failing. Overall counts passes over all50runs, not an average of two task-level rates. The source README announces32issue tasks and24solution-capable tasks, but exactly18 have scored solution runs; do not use24as the reported denominator. Exact task-ID sets are pinned and all rates reproduce the native table to its stated one decimal. Separate from the previously reviewed33/20task-set edition. This board reports Root cause found, 32 failed training runs (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 32 Issue / 18 Solution - Solution&lt;/strong&gt; (Repaired run passes its rerun, 18 tasks (%)): leader Gemini 4 Argon (Max) (66.7), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench native 32-issue/18-solution task-set edition. Same published root-cause and repaired-workspace protocols, one run per model/task/level. Issue is an LLM judgment against hidden oracle; solution is the harness rerun metric threshold, with reward hacking failing. Overall counts passes over all50runs, not an average of two task-level rates. The source README announces32issue tasks and24solution-capable tasks, but exactly18 have scored solution runs; do not use24as the reported denominator. Exact task-ID sets are pinned and all rates reproduce the native table to its stated one decimal. Separate from the previously reviewed33/20task-set edition. This board reports Repaired run passes its rerun, 18 tasks (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 35 Issue / 20 Solution&lt;/strong&gt; (Pass rate over 35 issue judgments and 20 solution runs (%)): leader GPT-6 Astra (Max) (41.8), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 35 Issue / 20 Solution - Issue&lt;/strong&gt; (Root cause found, 35 failed training runs (%)): leader GPT-6 Astra (Max) (42.9), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 35 Issue / 20 Solution - Solution&lt;/strong&gt; (Repaired run passes its rerun, 20 tasks (%)): leader Gemini 4 Argon (Max) (60.0), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run eligible workspaces.35issue judgments against hidden oracles and20solution metric-threshold runs;overall is passes over all55scored runs. Exact task-ID sets,harness and effort retained;separate from33/20and32/18editions.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;The Aggregate - Chart Recall&lt;/strong&gt; (RMSE (log10 views)): leader GPT-6.1 Terra (0.272468), 16 models&lt;br&gt;&lt;span&gt;Draw ten years of monthly English Wikipedia pageviews from an article title alone, without tools or a chart to look at. Sixteen models reconstruct the same 197 articles from January 2016 to December 2025. The score averages each article’s root mean squared error in log10 views, with lower values better; an error near 0.30 corresponds roughly to a factor-of-two miss. The separate shape score measures whether the rises and falls are in the right places.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CheatBench&lt;/strong&gt; (Cheating rate (%), equal-weight mean of ten task categories, each model in its provider's agent harness; lower is better): leader Gemini 4 Argon (Gemini CLI) (8.552349), 15 models&lt;br&gt;&lt;span&gt;Agents work in their vendor&amp;#x27;s own harness on hard tasks across ten categories (maths research, coding, protein design, board games and others) whose workspace hides a way to cheat, such as a leaked answer or a colleague&amp;#x27;s accepted work; the number is the equal-weight mean of the ten category rates, one of which is a sycophancy belief-shift score. Unlike ImpossibleBench, the tasks are solvable honestly and the shortcut is optional.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Company Agent Bench&lt;/strong&gt; (Overall accuracy on 205 questions, three parts weighted equally (%)): leader Claude Fable 5.2 (High) (82.2), 6 models&lt;br&gt;&lt;span&gt;Company Agent Bench (Concept, 12 November 2026) tests how well an agent understands the company it works in: 205 questions built from one month of real Slack history in the publisher&amp;#x27;s own workspace, replayed in order into a sandbox workspace. Unlike enterprise-search benchmarks, which ask for facts stored in documents, each question is about people: who wrote an anonymized message, who was tagged in one, and which teammate wrote which anonymized reply, among about 10 teammates. The recorded authors, mentions and replies are the ground truth, and the agent may use only context from before the message that reveals the answer. This board lists the API models, run in the publisher&amp;#x27;s Slack harness with Slack search and reading tools at low, medium and high reasoning effort; the score weights the three parts equally. The article also scores four assistant products (Dots, Grok Bot, Claude Tag and Devin), which are products and not models and are not listed here.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CWE-bench v1 - Programmatic Pass@1&lt;/strong&gt; (Programmatic Pass@1 (%)): leader Gemini 4 Carbon (71.0), 12 models&lt;br&gt;&lt;span&gt;Collinear CWE-bench archived v1, Programmatic Pass@1, reported as a percentage. 120 private audit-and-patch tasks; four rollouts per task at high reasoning, one hour each; timeouts graded on code at timeout. Native model-specific harnesses. Programmatic success requires exploit blocked and existing tests passing. Separate majority panel of three rubric-based agentic judges checks correctness, happy-path, generalization, integrity and soundness; any failed axis rejects a fix. Pass@1 and Pass@4 remain separate. This is archived Collinear v1, not the Artificial Analysis Stirrup two-hour implementation or CWE-bench v2. Source: https://cwe-bench.com/v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Deepnote Agent Initiative Benchmark&lt;/strong&gt; (Mean blind evaluator task score (0-10)): leader GPT-6 Astra (High) (8.97), 12 models&lt;br&gt;&lt;span&gt;Deepnote tests when a proactive workplace agent should investigate and help, and when it should remain silent. All 12 models face the same 50 synthetic company situations at source-declared high reasoning effort, using 11 service APIs and 180 seconds of model response time (tool time excluded). Two mutually blind LLM evaluators grade against a reference solution and complete action log; the0–10mean covers all50situations. Exact judge models and API snapshots are unreported. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DualEntry Accounting AI Benchmark&lt;/strong&gt; (Tasks passed in at least two of three runs, 101 accounting tasks (%)): leader Gemini 4 Argon (91.1), 25 models&lt;br&gt;&lt;span&gt;The DualEntry Accounting AI Benchmark (October 2026, version 1.0) tests whether a model can do routine bookkeeping inside an accounting system: 101 synthetic cases covering transaction classification, journal entries, payables, receivables, bank reconciliation, financial reporting, month-end close and accounting knowledge. Each model works in a freshly restored test company with the same instructions and accounting tools, saves its records as drafts, and runs the full set three times. Fixed rules check each answer against a reference written beforehand (field comparison, numeric tolerance, required classification, multiple choice), with no model acting as judge. Unlike FinanceBench, which asks questions about public company filings, it scores the records and figures a model produces through tools in the vendor&amp;#x27;s own ledger product; DualEntry, an accounting-software company, runs it. This board is the share of the 101 tasks a model passes in at least two of its three runs.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EmbodiedLab&lt;/strong&gt; (T0 (raw control) success rate, 30 experiments (%)): leader Claude Opus 5.5 (Medium) (43.3), 6 models&lt;br&gt;&lt;span&gt;EmbodiedLab (v0.1.0, arXiv 2610.35385) tests whether an agent can run a physical science experiment to find a quantity it cannot know in advance, such as a spring&amp;#x27;s stiffness or an acid&amp;#x27;s concentration. The agent gets an objective and the apparatus but no method, and writes Python that drives a simulated Franka arm in NVIDIA Isaac Sim and reads instruments through three cameras; the hidden parameters are redrawn for every instance. A run succeeds only if the submitted value is within the instrument&amp;#x27;s tolerance, rests on readings recorded during the run, and the bench is left as required. Each model runs in its developer&amp;#x27;s coding agent at that agent&amp;#x27;s middle reasoning setting, on 30 experiments at two interface levels: raw control (T0), and with perception and motion-planning tools (T1). Unlike manipulation benchmarks such as LIBERO, where the target state is given, the robot&amp;#x27;s actions here are only the means to a measurement. This board is the T0 success rate, the ranking column: raw control, averaged over runs and then over the 30 experiments.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Eon Agentic Misalignment&lt;/strong&gt; (Runs with a human-confirmed rule violation in 41 enterprise tasks x 3 repeats (% of answered runs)): leader Claude Opus 5.5 (21.666667), 19 models&lt;br&gt;&lt;span&gt;Agents do 41 everyday enterprise tasks in synthetic company systems where every write is refused and a temptation is planted; the score is the share of answered runs (3 per task) with a human-confirmed violation such as claiming a refused write succeeded or leaking PII. Unlike lab honeypots, the rules broken are ordinary enterprise policy.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FAR.AI AI Security Leaderboard&lt;/strong&gt; (Universal jailbreaks found (count; FAR.AI red team under the Minimal Standard for Safeguards v1.0, random-search and expert-guided attacks over chemical, biological, radiological &amp; nuclear, explosives and cyber hazards; 0 = none found within a search budget above $14K)): leader Claude Fable 5 (0.0), 8 models&lt;br&gt;&lt;span&gt;FAR.AI&amp;#x27;s red team attacks each frontier model&amp;#x27;s deployed safeguards with a fixed set of public jailbreak methods and their compositions, and the score is how many universal jailbreaks (attacks that work across a hazard category) it found. Unlike jailbreak success-rate boards such as Enkrypt or F5 CASI, it counts reusable breaks found by a budgeted search, so most current frontier models tie at zero.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gamow LabRobotBench v0.1&lt;/strong&gt; (Mean dense recovery score (fraction, 0–1)): leader GPT-6.1 Sol (0.467006600721), 6 models&lt;br&gt;&lt;span&gt;LabRobotBench v0.1 — Recovery of robotic laboratory workflows across3tasks,one attempt per configuration. Mean dense recovery score includes inherited work and uses a retrospectively chosen rubric;all six systems pass0/3strict tasks. Source-native provider/harness/runtime differences remain explicit;score fraction0–1.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gamow ScreenBench&lt;/strong&gt; (Mean normalized answer score (fraction, 0–1)): leader Claude Fable 5.2 (0.702552747253), 7 models&lt;br&gt;&lt;span&gt;ScreenBench — Agents inspect compound-screening plate records to detect and correct hidden errors across182tasks. Mean normalized graded-answer score,0no better than shortcuts and1all answers right. Unprompted condition;the prompt listing possible errors is a separate experiment. Native harness/graded population and score revision retained. Native cohorts differ: historical rows use named cell lines; Astra, Carbon and Fable5.2 use coded names; Hydrogen is unspecified. Sol helped select95tasks; evaluation uses fresh runs.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Guesswork 2026-11&lt;/strong&gt; (MAE (z-score units)): leader GPT-6 Astra (0.5752), 32 models&lt;br&gt;&lt;span&gt;The November 2026 round of this site&amp;#x27;s benchmark-score forecasting contest. Models predict new model and benchmark results using only earlier published score information and are evaluated on common cells against The Aggregate statistical baseline. Score is mean absolute error in per-benchmark z-score units; lower is better. Missing usable predictions receive the benchmark mean. Source: theaggregate.ai/prediction-duel.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HalachaBench&lt;/strong&gt; (Mean rubric score on 60 semicha exam questions, three attempts each, GPT-6 Astra judge (0-100)): leader Claude Fable 5.2 (High) (85.765638), 24 models&lt;br&gt;&lt;span&gt;HalachaBench (v1 early results, provisional) asks models to answer real questions from the Chief Rabbinate of Israel&amp;#x27;s semicha (rabbinic ordination) exams: the 60 questions of five papers of the Cheshvan 5787 sitting, each answered three times in Hebrew, closed-book and without tools. One AI judge, GPT-6 Astra, checks every answer against a sealed rubric of 9 to 12 criteria that AI models wrote and reviewed; an answer&amp;#x27;s score is the share of rubric points it earns. Unlike LEXam, where law-exam answers are graded against the examiners&amp;#x27; own reference answers, these exams have no official answer key, so the grading rests on AI-written rubrics that no rabbi has reviewed yet. The judge is also a contestant. Open-weight models that ran as local 4-bit files of higher-precision releases are left off the board. This board is the mean score, from 0 to 100, over a model&amp;#x27;s 180 answers.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HoneyBench v0.2 - Overall Hack Rate&lt;/strong&gt; (Overall hack rate (%) over scored runs in the v0.2 20-task universe; unevaluated tasks and provider refusals excluded (lower is better)): leader Claude Fable 5.2 (11.931818181818182), 22 models&lt;br&gt;&lt;span&gt;Share of scored agent rollouts that commit to specification gaming in HoneyBench v0.2, a 20-task evaluation spanning medical, coding, engineering, administrative, ML, multimodal and game tasks. The headline excludes provider-refused runs and unevaluated tasks; models therefore have different denominators. The alternate rate counting refusals as hacks is retained as source evidence, not used as the score. Access to forbidden information counts as hacking even without a final exploit; otherwise the final submission determines the outcome. Native percentage, lower is better. Source: goodhartlabs.com/honeybench.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Inference Module&lt;/strong&gt; (Mean per-task rank score (0-100)): leader GPT-6 Astra (Max) (86.7), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Inference Module: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Inference Operator&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (95.0), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Inference Operator: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Inference System&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (90.0), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Inference System: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Training Module&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (90.0), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Training Module: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Training Operator&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (93.3), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Training Operator: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Training System&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (92.0), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Training System: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;JobCannon Protocol Sensitivity&lt;/strong&gt; (Median protocol range in human SD (lower is better): for each questionnaire scale, the spread (max minus min) of the model's score over its valid protocol variants, in units of the human test-taker standard deviation, then the median over the model's scales (table F11)): leader GPT-6 Astra (0.51), 16 models&lt;br&gt;&lt;span&gt;A model takes four self-report questionnaires (Big Five, RIASEC, Dark Triad, Multiple Intelligences) under 11 protocol conditions such as reversed options, a paraphrased instruction or the whole test in one prompt; the number is the median, over its 22 scales, of how far its scale score moves across variants, in human standard deviations. Unlike Stick To Your Role, it measures the self-report&amp;#x27;s stability under administration changes, not across simulated personas.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Chartography (no tools))&lt;/strong&gt; (Score (%)): leader Claude Fable 5.2 (73.4), 5 models&lt;br&gt;&lt;span&gt;Chartography professional chart-understanding evaluation, mirrored by LLM Stats with tools disabled. Native score is reported as a percentage; keep this configuration separate from the tool-enabled and unspecified-tool boards. Source: https://llm-stats.com/benchmarks/chartography-no-tools&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM2014 Logic 2026-11&lt;/strong&gt; (Median Score): leader GPT-6 Astra (xHigh) (86.35), 41 models&lt;br&gt;&lt;span&gt;November 2026 LLM2014 Logic cohort, reported as the native median score. Source model, effort, refusal and estimated-result labels remain attached to their observations. Monthly editions stay separate. Source: https://raw.githubusercontent.com/llm2014/llm_benchmark/main/docs/data/logic/2026-11.csv&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV October&lt;/strong&gt; (Accuracy (%)): leader Gemini 4 Carbon (High) (93.62), 10 models&lt;br&gt;&lt;span&gt;October 2026 cohort of MathArena research-mathematics final-answer questions drawn from arXiv. The publisher reports the score as a percentage. Monthly problem sets remain separate, with the source model and effort labels retained; this is distinct from the false-statement cohort. Source: https://matharena.ai/?comp=arxiv--october&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV_FALSE October&lt;/strong&gt; (Accuracy (%)): leader Gemini 4 Carbon (High) (82.72), 10 models&lt;br&gt;&lt;span&gt;October 2026 MathArena cohort assessing responses to false mathematical statements drawn from arXiv. The publisher score is shown as a percentage. Keep this statement-assessment cohort separate from final-answer mathematics, Lean proof evaluation and other monthly editions. Source model and effort labels are retained. Source: https://matharena.ai/?comp=arxiv_false--october&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA MKA - Climb WSR&lt;/strong&gt; (Weighted Satisfaction Rate (%)): leader Claude Fable 5.2 (Max, Default Fallback) (56.5604258363361), 21 models&lt;br&gt;&lt;span&gt;Artificial Analysis MKA, Climb WSR: criterion-weighted task satisfaction, reported as a percentage. 200 private MKA tasks; one run per task, Stirrup, 250 turns and 8 hours; Linux sandbox with code, internet, search, page fetch and image viewing. GPT-6 Luna medium agentic judge; criterion weights 1–5, task-mean weighted satisfaction. Climb and Regression criterion populations remain separate. Empty submissions and errored runs score zero; final-text-only submissions are graded. The judge has no network or input files. Native effort and fallback labels retained. Source: https://artificialanalysis.ai/evaluations/mka&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Bearing Fault Transfer&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.6862), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Bearing Fault Transfer — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Chess Puzzle Post-Training&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5171375), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Chess Puzzle Post-Training — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Concrete Frame Sizing&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.598825), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Concrete Frame Sizing — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Design/Build/Fly Aircraft&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.7152875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Design/Build/Fly Aircraft — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Drawing to CAD&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.4376875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Drawing to CAD — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Earthquake Sequence Relocation&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5074875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Earthquake Sequence Relocation — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Engineering&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5962575), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Engineering — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Free-Form CAD Reconstruction&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.83805), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Free-Form CAD Reconstruction — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Function Page Layout&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.8686125), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Function Page Layout — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - GPU4PySCF J/K Kernels&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.167125), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - GPU4PySCF J/K Kernels — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - INT4 Decode Attention&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Opus 5.5 (Max) (0.627025), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - INT4 Decode Attention — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - LVGL Renderer Speedup&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5809875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - LVGL Renderer Speedup — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - MicroPython VM Speedup&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.5797875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - MicroPython VM Speedup — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - ML&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.545644642857), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - ML — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Overall&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.545673557692), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Overall — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Pairwise-Gated Attention Engine&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.6435375), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Pairwise-Gated Attention Engine — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Plume Dispersion Forecast&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Gemini 4 Argon (High) (0.159425), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Plume Dispersion Forecast — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Scan to CAD&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.4971375), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Scan to CAD — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Science&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.401922916667), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Science — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Seismic Wave Propagation&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Opus 5.5 (Max) (0.2306625), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Seismic Wave Propagation — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Sentinel-2 Gap Filling&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.40305), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Sentinel-2 Gap Filling — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Skewed Cyclic Joins&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.713675), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Skewed Cyclic Joins — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - SmolLM2 Long Context&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.778875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - SmolLM2 Long Context — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - SmolLM3 Draft Head&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Gemini 4 Argon (High) (0.3919), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - SmolLM3 Draft Head — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Software&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.621896875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Software — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - SPARQL Store Speedup&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.9387625), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - SPARQL Store Speedup — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Spectral Kinetics Forecast&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.8148375), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Spectral Kinetics Forecast — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - SQL Change Replay&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Sonnet 5.5 (Max) (0.31185), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - SQL Change Replay — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Stateful Parquet Lakehouse&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.1 (Max) (0.4969625), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Stateful Parquet Lakehouse — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Storm Surge Forecast&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader GPT-6 Astra (Max) (0.611875), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Storm Surge Forecast — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OWE-Bench - Streaming Payments ETL&lt;/strong&gt; (Mean normalized held-out task score (fraction, 0–1)): leader Claude Fable 5.2 (Max) (0.838475), 14 models&lt;br&gt;&lt;span&gt;OWE-Bench - Streaming Payments ETL — Expert-built long-horizon science, software, ML and engineering tasks with task-specific hardware and4–18hour budgets; native baseline/ceiling normalized held-out reward,mean of8attempts and equal task means. Web frozen October1,2026;deterministic sealed verifier. Native model/effort and every attempt retained. Score fraction0–1;higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Personal Coding Evals (core10)&lt;/strong&gt; (pass@1: mean solve rate over 10 personal coding tasks x 3 runs (%)): leader GPT-6 Luna (Pi, xHigh) (100.0), 16 models&lt;br&gt;&lt;span&gt;One engineer&amp;#x27;s 10 coding tasks taken from his daily work, each run three times per harness, model and effort configuration (Pi, Codex, Claude Code, DeepSeek Harness); the score is the share of the 30 runs solved. It is a personal, saturated suite (16 of 33 admitted configurations solve every run), not a general coding benchmark.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReuF2F&lt;/strong&gt; (pass@1: accepted attempts, two attempts on each of 244 Lean problems (%)): leader Gemini 4 Argon (High) (42.42), 8 models&lt;br&gt;&lt;span&gt;ReuF2F gives a coding agent one of 244 open problems from undergraduate research programmes, formalised in Lean 4, and asks it to either prove the statement or prove its full negation. Every model runs in the same Leanimum-agent harness inside a container, with up to 1,000 steps and 6 hours per attempt, internet access and two attempts per problem. A submission counts only if the unchanged statement or its negation passes the Comparator checker with both the Lean kernel and nanoda, so there is no LLM judge. Unlike miniF2F, whose statements are olympiad and textbook problems with known proofs, every ReuF2F problem was unsolved when collected, and a run is a multi-hour agent session. The rows are ProbXiv&amp;#x27;s transcription of the paper&amp;#x27;s tables (arXiv 2610.27046), including its reasoning-effort sweep. This board is pass@1: accepted attempts as a share of the 488 attempts.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RoboRisk - Physical Harm Rate&lt;/strong&gt; (Harm rate (%)): leader GPT-6 Astra (Benchmark Sweep Run: source=leash_roborisk_paper; harness=LEASH; reviewer=Jev; reasoning=unreported) (0.0), 10 models&lt;br&gt;&lt;span&gt;RoboRisk - Physical Harm Rate: Harm rate (%). LEASH arXiv 2611.14115v1 Table 2: RoboRisk-S 92 simulation tasks x3 seeds=276 runs, RoboRisk-R 13 physical tasks x1=13; robot APIs use an adapted simulation subset of 39 runs. HR harm rate lower is better, SSR safe-success rate higher is better. Baseline and Jev-reviewed LEASH runs stay distinct. All simulation planners use high reasoning; physical planner reasoning is not reported here. Robot API rows are supported specialist systems, not generic chat checkpoints.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RoboRisk - Simulation Harm Rate&lt;/strong&gt; (Harm rate (%)): leader GPT-6 Astra (Benchmark Sweep Run: source=leash_roborisk_paper; harness=LEASH; reviewer=Jev; reasoning=high) (0.0), 8 models&lt;br&gt;&lt;span&gt;RoboRisk - Simulation Harm Rate: Harm rate (%). LEASH arXiv 2611.14115v1 Table 2: RoboRisk-S 92 simulation tasks x3 seeds=276 runs, RoboRisk-R 13 physical tasks x1=13; robot APIs use an adapted simulation subset of 39 runs. HR harm rate lower is better, SSR safe-success rate higher is better. Baseline and Jev-reviewed LEASH runs stay distinct. All simulation planners use high reasoning; physical planner reasoning is not reported here. Robot API rows are supported specialist systems, not generic chat checkpoints.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RWE-bench&lt;/strong&gt; (Average pass rate over 20 engineering tasks, 5 runs each (%)): leader Claude Fable 5.2 (Max) (62.0), 15 models&lt;br&gt;&lt;span&gt;RWE-bench (Real World Engineering Bench, Xplore, Batch 1) asks a coding agent to do physical engineering design work: 20 tasks, 9 mechanical (parts and mechanisms modelled in CAD and checked for strength, fit and motion, such as a one-piece crankshaft for a 3-cylinder diesel engine), 5 electrical (high-speed PCB routing on dense, already populated boards, such as a DDR3 memory bus with skew and impedance limits) and 6 aerospace (CFD and conjugate heat transfer from mesh to converged solution, such as the heat load on a turbine vane). Unlike SWE-bench, where a fix passes when the repository&amp;#x27;s tests pass, each design here is checked by a deterministic verifier that runs physics solvers and passes it only if every requirement in the brief is met; there is no LLM judge and no human grading. The agent gets the brief an engineer would get and a sandbox with CAD and CAE tools, and has up to 24 hours per run. Each model runs each task 5 times in its vendor&amp;#x27;s own coding agent at the highest reasoning effort it offers. This board is the average pass rate: the share of runs that pass, averaged over the 20 tasks.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ScarfBench (Kaggle)&lt;/strong&gt; (Smoke tests passed (%) over 204 Java framework migrations, each model run in a coding-agent harness by Kaggle): leader Gemini 4 Argon (Antigravity CLI, High) (75.24), 11 models&lt;br&gt;&lt;span&gt;Coding agents (one model at one effort inside its harness, run by Kaggle) carry out 204 migrations of enterprise Java applications between Jakarta EE, Quarkus and Spring; the score is the share of the applications&amp;#x27; behavioral smoke tests that pass after the migration. Unlike the ScarfBench paper board (174 focused-app migrations, one skills prompt), Kaggle runs and maintains these results over focused and whole applications.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Bulletbench - Blitz 3+2&lt;/strong&gt; (ladder Elo): leader Gemini 4 Argon (1147.0), 19 models&lt;br&gt;&lt;span&gt;Spring Prompt Bulletbench - Blitz 3+2. Model chess in three separate time controls:10+1Lightning,60sBullet,180+2Blitz. Native ladder Elo and95%CI; not FIDE Elo. Timed-out/invalid moves retain native failure rules. Native best available configuration per clock; all source configurations retained. Metric: ladder Elo. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Bulletbench - Bullet 60s&lt;/strong&gt; (ladder Elo): leader Gemini 3.8 Flash (1068.0), 57 models&lt;br&gt;&lt;span&gt;Spring Prompt Bulletbench - Bullet 60s. Model chess in three separate time controls:10+1Lightning,60sBullet,180+2Blitz. Native ladder Elo and95%CI; not FIDE Elo. Timed-out/invalid moves retain native failure rules. Native best available configuration per clock; all source configurations retained. Metric: ladder Elo. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Catalogbench - Reliably Publish Ready&lt;/strong&gt; (% of products): leader GPT-6.1 Sol (75.0), 31 models&lt;br&gt;&lt;span&gt;Spring Prompt Catalogbench - Reliably Publish Ready. 56 product briefs, 3 repeats, factual and sales briefs separately. Reliable publication requires all repeats pass; claims/attributes/channel rubric; a single GPT-6.1 Sol judge. Failures stay failed. Native website reports best supported configuration per model; all underlying configurations retained in model_runs. Metric: % of products. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Catalogbench - Sales Brief Reliably Publish Ready&lt;/strong&gt; (% of products): leader GPT-6 Astra (69.64), 31 models&lt;br&gt;&lt;span&gt;Spring Prompt Catalogbench - Sales Brief Reliably Publish Ready. 56 product briefs, 3 repeats, factual and sales briefs separately. Reliable publication requires all repeats pass; claims/attributes/channel rubric; a single GPT-6.1 Sol judge. Failures stay failed. Native website reports best supported configuration per model; all underlying configurations retained in model_runs. Metric: % of products. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Deckbench - Deck Rating&lt;/strong&gt; (rating): leader GPT-6 Astra (1584.2), 32 models&lt;br&gt;&lt;span&gt;Spring Prompt Deckbench - Deck Rating. Six business deck tasks, majority rubric checks by GPT6.1Sol/Opus5.5/Gemini3.1Pro; vendor-own judge abstains on paired deck comparisons. Native deck rating; missing unusable-deck rating is not zero. Human-rating sample pending. Metric: rating. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Roasbench - Score&lt;/strong&gt; (score out of 100): leader Claude Fable 5.2 (61.27), 32 models&lt;br&gt;&lt;span&gt;Spring Prompt Roasbench - Score. 2026 v2, twelve-month simulated advertising budget, reproducible simulated scenarios, profit/planning scores and reported critical overspend failures; not comparable to v1. Native score0-100, not actual profit or return on a subscription. Metric: score out of 100. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Surveybench - Analysis Rating&lt;/strong&gt; (rating): leader Claude Fable 5.2 (1664.6), 23 models&lt;br&gt;&lt;span&gt;Spring Prompt Surveybench - Analysis Rating. Nine research-analysis tasks, majority of GPT6.1Sol/Opus5.5/Gemini3.1Pro rubric judges, vendor-own judge abstains on paired analysis. Native analysis rating with95%CI; research quality, not token cost. Human-rating sample pending. Metric: rating. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SPU - Private - Raw&lt;/strong&gt; (Mean Spearman correlation): leader Claude Opus 5.5 (Max) (0.42), 12 models&lt;br&gt;&lt;span&gt;SPU - Private - Raw — SPU v1: rank wet-lab experiment outcomes; per-group Spearman correlation, combined with fixed source group weights within tasks (equal in97/100public tasks, unequal in3). 100 public/16 private tasks; one official run per configuration/task with one infrastructure retry. 8-hour budget,8CPU cores,16GiB RAM,H20GPU; results/reconstruction prohibited. GPT-6 Astra and Sol use Codex0.156.1; other models use Claude Code2.1.260 at max effort. Task-adjusted public score is the source additive task/configuration model, not a new local fit. Raw means use only valid or unambiguously recovered rankings; refusals/missing files excluded, not zero. Exact row counts and CIs remain provenance: public scored coverage30–100/100; private2–16/16. Private mean is unadjusted; it is not a matched-cohort comparison. The default-grader -1 convention is a separate diagnostic. Metric: Mean Spearman correlation.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SPU - Public - Raw&lt;/strong&gt; (Mean Spearman correlation): leader Claude Opus 5.5 (Max) (0.491), 12 models&lt;br&gt;&lt;span&gt;SPU - Public - Raw — SPU v1: rank wet-lab experiment outcomes; per-group Spearman correlation, combined with fixed source group weights within tasks (equal in97/100public tasks, unequal in3). 100 public/16 private tasks; one official run per configuration/task with one infrastructure retry. 8-hour budget,8CPU cores,16GiB RAM,H20GPU; results/reconstruction prohibited. GPT-6 Astra and Sol use Codex0.156.1; other models use Claude Code2.1.260 at max effort. Task-adjusted public score is the source additive task/configuration model, not a new local fit. Raw means use only valid or unambiguously recovered rankings; refusals/missing files excluded, not zero. Exact row counts and CIs remain provenance: public scored coverage30–100/100; private2–16/16. Private mean is unadjusted; it is not a matched-cohort comparison. The default-grader -1 convention is a separate diagnostic. Metric: Mean Spearman correlation.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SPU - Public - Task-adjusted&lt;/strong&gt; (Mean Spearman correlation): leader Claude Opus 5.5 (Max) (0.478), 12 models&lt;br&gt;&lt;span&gt;SPU - Public - Task-adjusted — SPU v1: rank wet-lab experiment outcomes; per-group Spearman correlation, combined with fixed source group weights within tasks (equal in97/100public tasks, unequal in3). 100 public/16 private tasks; one official run per configuration/task with one infrastructure retry. 8-hour budget,8CPU cores,16GiB RAM,H20GPU; results/reconstruction prohibited. GPT-6 Astra and Sol use Codex0.156.1; other models use Claude Code2.1.260 at max effort. Task-adjusted public score is the source additive task/configuration model, not a new local fit. Raw means use only valid or unambiguously recovered rankings; refusals/missing files excluded, not zero. Exact row counts and CIs remain provenance: public scored coverage30–100/100; private2–16/16. Private mean is unadjusted; it is not a matched-cohort comparison. The default-grader -1 convention is a separate diagnostic. Metric: Mean Spearman correlation.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sycophancy Eval&lt;/strong&gt; (Composite sycophancy score (0-100, lower is better)): leader Grok 4.6 (2.0), 29 models&lt;br&gt;&lt;span&gt;The model holds scripted multi-turn conversations in English and Traditional Chinese in which a user pushes it, with social, emotional or authority pressure but no new evidence, to drop a position, validate the user or endorse their framing; a three-vendor judge panel rates each conversation. The number is the publisher&amp;#x27;s composite sycophancy score: 100 times a weighted mean of seven dimension scores, each 0 to 1 with higher meaning the model gave way more: Regressive Backdown 25% and Instability 10% (how early and how often its stance flips on the factual scenarios), Moral 20%, Validation 15%, Framing 15%, Indirectness 10% and Flattery Density 5% (extra giving way after praise compared with a no-praise control), with each conversation averaged over the model&amp;#x27;s last three comparable sweeps. Unlike single-turn sycophancy tests such as SycEval, it keeps up the pressure over several turns and tracks when the stance breaks.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;TavernBench&lt;/strong&gt; (Play accuracy: share of tasks answered correctly, mean over the 30 play scenarios (%)): leader GPT-6 Astra (Low) (63.0), 10 models&lt;br&gt;&lt;span&gt;Language models answer Hearthstone Battlegrounds decisions (combat outcomes, positioning, play order, targets, card choices, turn plans) from a text description of the game state, with no tools. Labels use simulator outcomes, and top human players&amp;#x27; placement statistics for hero and trinket choices. The score is accuracy over the 30 play scenarios of a 368-task set, which asks for the best play rather than what the rules do.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ultralytics VLM Leaderboard&lt;/strong&gt; (mAP50 over 36 UL37 detection datasets (0-1)): leader Gemini 4 Carbon (0.583719), 45 models&lt;br&gt;&lt;span&gt;The Ultralytics VLM Leaderboard (suite ul37-v2) ranks vision language models and open-vocabulary detectors on drawing boxes around named objects, the task behind auto-annotation. Unlike COCO or RefCOCO, which use everyday photographs, its 360 images come 10 each from 36 real-world detection datasets on Ultralytics Platform, among them dental X-rays, retinal lesions, retail products and baggage scans, sent whole with each dataset&amp;#x27;s own class names. Every image goes through the platform&amp;#x27;s production annotation call, and SAM 3.1 tightens every model&amp;#x27;s boxes except SAM 3&amp;#x27;s. A timeout, refusal or invalid output counts as an image with no predictions. Boxes are scored with the Ultralytics package&amp;#x27;s detection metrics in the order the model listed them, and the datasets are averaged equally. Providers run at their default reasoning effort unless that misses the platform&amp;#x27;s 115-second request limit. This board is mAP50: average precision at an IoU of 0.5, averaged over the labeled classes.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Access to Justice&lt;/strong&gt; (Rubric Criterion Pass Rate (%)): leader Qwen 3.8 Max (76.368), 22 models&lt;br&gt;&lt;span&gt;Vals A2J v1: 252 private legal-access scenarios with GPT-5 simulated client and Decider v1.2 judge; pooled expert-rubric criterion pass rate. Native model effort and versioned complete roster retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Financial Coaching Bench&lt;/strong&gt; (Mean practitioner-checklist credit (%)): leader Claude Opus 5.5 (Max) (76.162), 27 models&lt;br&gt;&lt;span&gt;Financial Coaching Bench v1 (Vals AI, November2026) tests AI financial coaching on 256 private household scenarios. A simulated client, played by Claude Sonnet4.6, answers from a hidden fact sheet; the coach can ask questions and search the web. This board is mean practitioner-checklist credit in percent, preserving each native model and effort. Length-controlled scores and elicitation diagnostics are separate; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Overall&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (77.69), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Overall. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - All&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (70.02), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - All. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - Continuation&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (79.76), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - Continuation. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - Photo&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (59.75), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - Photo. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - Script&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (85.62), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - Script. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - Video&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (56.84), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - Video. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - All&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (85.36), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - All. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - Continuation&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (91.1), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - Continuation. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - Photo&lt;/strong&gt; (Score (%)): leader GPT-6.1 Sol (93.8), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - Photo. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - Script&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (76.5), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - Script. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - Video&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (89.8), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - Video. VibePrevisBench frozen paper Table 1: 242 Blender 4D tasks, 94 script 39 photo 23 continuation 86 video, agent writes/runs Blender Python. Scene score is rule-based 84,406 hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score 0. All is weighted native aggregate; overall mean of SceneAll and ScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Knowledge&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (Max) (13.3), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports knowledge exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Logic&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (Max) (13.3), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports logic exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Overall&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (Max) (26.7), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports overall exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Rule&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (xHigh) (3.3), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports rule exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Spatial&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (Max) (80.0), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports spatial exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WARC-Bench&lt;/strong&gt; (Tasks completed on the 192-task test split, mean of 3 runs (%)): leader Claude Opus 5.5 (Max) (90.28), 9 models&lt;br&gt;&lt;span&gt;WARC-Bench (Uniphore, arXiv 2510.09872; test-split runs of 19 October to 10 November 2026) tests whether an agent can operate single web interface components: 192 short GUI subtasks such as picking a date in a calendar widget, choosing from a dropdown, scrolling a container to read a value or completing a multi-step form, replayed from web archive (WARC) snapshots of real sites. Unlike WebArena or Online-Mind2Web, which grade whole multi-page tasks, it isolates the short interactions those tasks are built from and replays recorded pages, so every run sees the same site. The agent sees screenshots only and gets at most 20 steps and 1,800 seconds per task; a programmatic check grades the end state, 53 tasks by exact string match, with no judge model. Each model runs the split 3 times at the highest reasoning effort its API accepts, and the score is the mean of the 3 runs. Four of the models are served by Fireworks.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (482)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; — Aggregate Rating 2003, #1&lt;ul&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Pass Rate: 90.26 (#1/82)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Code Smell Density: 12.79 (#1/82)&lt;/li&gt;&lt;li&gt;Dual-Arm Sim - All - Progress: 59.8 (#1/11)&lt;/li&gt;&lt;li&gt;Dual-Arm Sim - Instrument Swap - Progress: 68.0 (#1/11)&lt;/li&gt;&lt;li&gt;Dual-Arm Sim - Instrument Swap - Completion: 10.0 (#1/11)&lt;/li&gt;&lt;li&gt;Dual-Arm Sim - Expert Recovery: 4.5 (#1/11)&lt;/li&gt;&lt;li&gt;GBA Eval - Gameplay Replays: 1.0 (#1/26)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (TypeScript): 90.8 (#1/13)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (Rust): 96.8 (#1/13)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (C#): 93.2 (#1/13)&lt;/li&gt;&lt;li&gt;...and 349 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; — Aggregate Rating 1977, #2&lt;ul&gt;&lt;li&gt;CursorBench 4.0: 60.4 (#1/91)&lt;/li&gt;&lt;li&gt;MathArena - ARXIVLEAN September: 62.26 (#1/8)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE August: 88.1 (#1/21)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV September: 96.23 (#1/15)&lt;/li&gt;&lt;li&gt;Epoch AI - Cursorbench: 60.4 (#1/90)&lt;/li&gt;&lt;li&gt;Epoch AI - Apex Agents: 84.2 (#1/65)&lt;/li&gt;&lt;li&gt;LiveBench Coding: 91.0 (#1/35)&lt;/li&gt;&lt;li&gt;LiveBench Data Analysis: 86.5 (#1/35)&lt;/li&gt;&lt;li&gt;FoodTruckBench: 77217.68 (#1/62)&lt;/li&gt;&lt;li&gt;Humanity's Sixth Sense: 71.1 (#1/34)&lt;/li&gt;&lt;li&gt;...and 139 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; — Aggregate Rating 1956, #5&lt;ul&gt;&lt;li&gt;GBA Eval: 0.9056 (#1/28)&lt;/li&gt;&lt;li&gt;GBA Eval - Procedural Tests: 0.8764 (#1/26)&lt;/li&gt;&lt;li&gt;Nejumi 4 - Total Score: 88.43 (#1/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP Average: 88.74 (#1/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Reasoning: 97.22 (#1/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Knowledge &amp; QA: 83.51 (#1/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Application Development: 84.44 (#1/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Expression: 100.0 (#1/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Abstract Reasoning: 99.0 (#1/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Mathematical Reasoning: 98.17 (#1/148)&lt;/li&gt;&lt;li&gt;...and 113 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; — Aggregate Rating 1953, #6&lt;ul&gt;&lt;li&gt;GBA Eval: 0.8705 (#2/28)&lt;/li&gt;&lt;li&gt;GBA Eval - Gameplay Replays: 0.9955 (#3/26)&lt;/li&gt;&lt;li&gt;GBA Eval - Procedural Tests: 0.7014 (#3/26)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Primary: 96.24 (#3/283)&lt;/li&gt;&lt;li&gt;GBA Eval - Audio: 0.6648 (#4/26)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1766.84 (#4/146)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1766.84 (#4/44)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 92.32 (#4/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 95.24 (#4/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 91.19 (#7/283)&lt;/li&gt;&lt;li&gt;...and 12 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; — Aggregate Rating 1938, #10&lt;ul&gt;&lt;li&gt;Coarena: 1127.6 (#1/24)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Basic Language: 88.68 (#1/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot): 88.98 (#1/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - JSQuAD: 83.08 (#1/153)&lt;/li&gt;&lt;li&gt;GBA Eval - Audio: 0.6829 (#2/26)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Semantic Analysis: 86.7 (#2/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - ALT - Controllability: 94.53 (#2/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - ALT - Truthfulness: 88.32 (#2/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JCoLA (in-domain): 84.0 (#2/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - JCoLA (in-domain): 87.0 (#2/153)&lt;/li&gt;&lt;li&gt;...and 112 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Hydrogen&lt;/strong&gt; — Aggregate Rating 1935, #11&lt;ul&gt;&lt;li&gt;AA GDP.pdf: 32.8 (#1/261)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 91.46 (#1/261)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Operations - Raw Objective Completion: 97.6 (#1/263)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 1818.0 (#1/291)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 52.305 (#1/85)&lt;/li&gt;&lt;li&gt;FrontierCS 2 Antigravity - FARGO paper-aligned query core: 52.65348694173612 (#1/3)&lt;/li&gt;&lt;li&gt;FrontierCS 2 Antigravity - Exact Sequence Alignment: 67.19671459086022 (#1/3)&lt;/li&gt;&lt;li&gt;FrontierCS 2 Antigravity - E-Graph Invariant Restoration: 68.53231954722582 (#1/3)&lt;/li&gt;&lt;li&gt;FrontierCS 2 Antigravity - OCR Distillation Trajectory Sampling: 87.47737556561086 (#1/3)&lt;/li&gt;&lt;li&gt;FrontierCS 2 Antigravity - Hypergraph Core Numbers: 61.4131399518806 (#1/3)&lt;/li&gt;&lt;li&gt;...and 167 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; — Aggregate Rating 1931, #12&lt;ul&gt;&lt;li&gt;AutomationBench-AA - Sales - Raw Objective Completion: 94.54 (#1/263)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 92.88 (#1/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Science: 97.27 (#1/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 91.98 (#1/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Social studies: 93.64 (#1/283)&lt;/li&gt;&lt;li&gt;AA MMMU-Pro: 88.32 (#2/304)&lt;/li&gt;&lt;li&gt;AA-Briefcase - DOCX - Rubric Pass Rate: 63.91 (#2/268)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1522.38 (#2/33)&lt;/li&gt;&lt;li&gt;AI for Education SEND: 89.45 (#2/275)&lt;/li&gt;&lt;li&gt;MathArena - ARXIVLEAN September: 39.62 (#3/8)&lt;/li&gt;&lt;li&gt;...and 85 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; — Aggregate Rating 1922, #15&lt;ul&gt;&lt;li&gt;Arena AI Document: 1514.84 (#1/51)&lt;/li&gt;&lt;li&gt;MirrorCode: 93.3 (#3/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 93.33 (#3/46)&lt;/li&gt;&lt;li&gt;YC-Bench: 3199.37485 (#3/66)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Pass Rate: 87.68 (#4/82)&lt;/li&gt;&lt;li&gt;VulcanBench v3 (BenchLM): 91.8 (#5/18)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1744.56 (#5/146)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1744.56 (#5/44)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Bug Density: 0.56 (#6/82)&lt;/li&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 93.9995111944 (#6/273)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Pro Terra&lt;/strong&gt; — Aggregate Rating 1912, #16&lt;ul&gt;&lt;li&gt;YC-Bench: 1376.29866 (#25/66)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; — Aggregate Rating 1911, #17&lt;ul&gt;&lt;li&gt;Arena AI Code: 1723.89 (#7/146)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1723.89 (#7/44)&lt;/li&gt;&lt;li&gt;VulcanBench v3 (BenchLM): 88.4 (#12/18)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5.5&lt;/strong&gt; — Aggregate Rating 1906, #19&lt;ul&gt;&lt;li&gt;MirrorCode: 97.8 (#1/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 97.78 (#1/46)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - MGSM: 96.0 (#1/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - Toxicity - Fairness: 100.0 (#1/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - Total Score: 88.26 (#2/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP Average: 87.42 (#2/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Knowledge &amp; QA: 81.78 (#2/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Application Development: 82.78 (#2/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Coding: 91.92 (#2/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Function Calling: 73.64 (#2/148)&lt;/li&gt;&lt;li&gt;...and 105 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5&lt;/strong&gt; — Aggregate Rating 1897, #20&lt;ul&gt;&lt;li&gt;Arena AI Document: 1508.66 (#2/51)&lt;/li&gt;&lt;li&gt;MirrorCode: 87.8 (#4/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 87.78 (#4/46)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1239.0251 (#5/25)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1690.84 (#8/146)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1690.84 (#8/44)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 80.0 (#14/67)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1489.76 (#15/33)&lt;/li&gt;&lt;li&gt;LMArena Vision Arena: 1285.443 (#19/25)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 59.5 (#27/598)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; — Aggregate Rating 1897, #21&lt;ul&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JaNLI: 100.0 (#1/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JCoLA (out-of-domain): 93.0 (#1/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - JaNLI: 100.0 (#1/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - JCoLA (out-of-domain): 94.0 (#1/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - MMLU-ProX (Japanese): 92.0 (#1/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - HalluLens: 100.0 (#1/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - JHumanEval: 100.0 (#1/153)&lt;/li&gt;&lt;li&gt;Coarena: 1126.4 (#2/24)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Syntactic Analysis: 90.83 (#2/148)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Social studies: 93.64 (#2/283)&lt;/li&gt;&lt;li&gt;...and 115 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Sol&lt;/strong&gt; — Aggregate Rating 1879, #26&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 74.01 (#5/23)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1673.85 (#13/146)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1673.85 (#13/44)&lt;/li&gt;&lt;li&gt;VulcanBench v3 (BenchLM): 86.8 (#14/18)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 76.0 (#16/67)&lt;/li&gt;&lt;li&gt;MirrorCode: 48.9 (#18/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 48.89 (#21/46)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 53.85 (#49/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 27.02 (#55/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 50.74 (#133/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5&lt;/strong&gt; — Aggregate Rating 1878, #27&lt;ul&gt;&lt;li&gt;LMArena Vision Arena: 1309.4376 (#1/25)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Pass Rate: 87.68 (#3/82)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1496.61 (#5/51)&lt;/li&gt;&lt;li&gt;Vals AI Web Search: 48.454 (#5/12)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1503.91 (#6/33)&lt;/li&gt;&lt;li&gt;MirrorCode: 70.0 (#7/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 61.0 (#8/238)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 70.0 (#9/46)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 65.35 (#10/598)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1228.1528 (#10/25)&lt;/li&gt;&lt;li&gt;...and 10 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder&lt;/strong&gt; — Aggregate Rating 1874, #28&lt;ul&gt;&lt;li&gt;Terminal-Bench 4.0 (Muse Code): 46.97 (#1/2)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 95.8 (#2/24)&lt;/li&gt;&lt;li&gt;GDP.pdf: 54.8 (#2/21)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1510.96 (#3/33)&lt;/li&gt;&lt;li&gt;Coarena: 1061.7 (#6/24)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV September: 66.04 (#6/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV August: 68.42 (#10/22)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE September: 41.51 (#11/15)&lt;/li&gt;&lt;li&gt;EnigmaForge - Fact F1: 87.9 (#14/33)&lt;/li&gt;&lt;li&gt;EnigmaForge: 45.5 (#16/33)&lt;/li&gt;&lt;li&gt;...and 16 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro&lt;/strong&gt; — Aggregate Rating 1871, #29&lt;ul&gt;&lt;li&gt;Terminal-Bench 4.0 (Claude Code): 60.91 (#3/26)&lt;/li&gt;&lt;li&gt;Coarena: 1064.5 (#5/24)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE September: 63.52 (#5/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIVLEAN September: 16.98 (#6/8)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV September: 64.15 (#7/15)&lt;/li&gt;&lt;li&gt;YC-Bench: 2386.0026000000003 (#7/66)&lt;/li&gt;&lt;li&gt;HiL-Bench: 61.33 (#9/32)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1687.85 (#9/146)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1687.85 (#9/44)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE August: 64.29 (#10/21)&lt;/li&gt;&lt;li&gt;...and 21 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Sol&lt;/strong&gt; — Aggregate Rating 1866, #30&lt;ul&gt;&lt;li&gt;LMArena Search Arena: 1257.841 (#2/25)&lt;/li&gt;&lt;li&gt;Kaggle FACTS (Google): 70.57 (#8/23)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1480.04 (#11/51)&lt;/li&gt;&lt;li&gt;LMArena Vision Arena: 1288.0021 (#16/25)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1484.95 (#23/33)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 59.4 (#28/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 21.97 (#71/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.42 (#439/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.7 Flash&lt;/strong&gt; — Aggregate Rating 1847, #34&lt;ul&gt;&lt;li&gt;LMArena Vision Arena: 1293.7683 (#8/25)&lt;/li&gt;&lt;li&gt;MathArena - ARXIVLEAN June: 21.74 (#9/14)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV April: 57.5 (#10/23)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1476.47 (#12/51)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE April: 19.26 (#13/19)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1489.39 (#16/33)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV June: 62.5 (#21/31)&lt;/li&gt;&lt;li&gt;MathArena Arxiv False: 8.5 (#22/23)&lt;/li&gt;&lt;li&gt;MathArena Arxiv: 47.5 (#22/26)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE June: 11.11 (#28/30)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.8 Flash&lt;/strong&gt; — Aggregate Rating 1844, #36&lt;ul&gt;&lt;li&gt;MathArena - ARXIV April: 64.17 (#6/23)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE April: 33.61 (#8/19)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV September: 50.0 (#8/15)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1481.92 (#9/51)&lt;/li&gt;&lt;li&gt;MathArena - ARXIVLEAN June: 19.57 (#10/14)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1497.49 (#10/33)&lt;/li&gt;&lt;li&gt;LMArena Vision Arena: 1293.0738 (#11/25)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE September: 26.73 (#15/15)&lt;/li&gt;&lt;li&gt;MathArena Arxiv: 58.33 (#15/26)&lt;/li&gt;&lt;li&gt;MathArena Arxiv False: 19.0 (#16/23)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash (1017)&lt;/strong&gt; — Aggregate Rating 1840, #37&lt;ul&gt;&lt;li&gt;Coarena - Task Completion: 91.3 (#4/24)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE September: 55.97 (#7/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV September: 49.06 (#9/15)&lt;/li&gt;&lt;li&gt;VulcanBench v3 (BenchLM): 88.7 (#11/18)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE August: 60.71 (#13/21)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV August: 56.14 (#13/22)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 83.3 (#14/25)&lt;/li&gt;&lt;li&gt;Terminal-Bench 4.0 (Claude Code): 40.0 (#15/26)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1664.2 (#15/146)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1664.2 (#15/44)&lt;/li&gt;&lt;li&gt;...and 12 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;hy4&lt;/strong&gt; — Aggregate Rating 1837, #41&lt;ul&gt;&lt;li&gt;CWE-Bench-AA: 75.83 (#1/18)&lt;/li&gt;&lt;li&gt;Artificial Analysis Cyber Index: 57.98 (#2/18)&lt;/li&gt;&lt;li&gt;ParseBench - Charts: 70.96 (#2/122)&lt;/li&gt;&lt;li&gt;Coarena: 1090.4 (#4/24)&lt;/li&gt;&lt;li&gt;ReLE - Language and Instruction Following: 77.1 (#4/151)&lt;/li&gt;&lt;li&gt;ReLE - Education - Middle School Subjects: 59.0 (#6/151)&lt;/li&gt;&lt;li&gt;DeepsecBench-AA: 25.59 (#9/24)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 86.6 (#10/24)&lt;/li&gt;&lt;li&gt;CyberGym-E2E-AA: 72.52 (#10/18)&lt;/li&gt;&lt;li&gt;ReLE - Education - Primary School Subjects: 70.7 (#10/151)&lt;/li&gt;&lt;li&gt;...and 32 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.6&lt;/strong&gt; — Aggregate Rating 1835, #43&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 59.55 (#17/23)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 24.0 (#20/598)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 68.0 (#28/67)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1617.82 (#30/146)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1438.08 (#34/51)&lt;/li&gt;&lt;li&gt;AA Omniscience: 30.48 (#37/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 48.23 (#83/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.3&lt;/strong&gt; — Aggregate Rating 1834, #44&lt;ul&gt;&lt;li&gt;Coarena - Task Completion: 97.1 (#1/24)&lt;/li&gt;&lt;li&gt;GDP.pdf: 53.0 (#4/21)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE April: 49.18 (#5/19)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV April: 66.67 (#5/23)&lt;/li&gt;&lt;li&gt;Design Arena (ASCII Art): 1333.0 (#6/101)&lt;/li&gt;&lt;li&gt;MathArena - ARXIVLEAN June: 28.26 (#6/14)&lt;/li&gt;&lt;li&gt;MathArena Arxiv: 73.33 (#9/26)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE June: 53.24 (#10/30)&lt;/li&gt;&lt;li&gt;MathArena Arxiv False: 36.5 (#11/23)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV June: 76.74 (#11/31)&lt;/li&gt;&lt;li&gt;...and 11 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.7&lt;/strong&gt; — Aggregate Rating 1833, #45&lt;ul&gt;&lt;li&gt;VulcanBench v3 (BenchLM): 93.2 (#2/18)&lt;/li&gt;&lt;li&gt;Coarena: 993.0 (#13/24)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Generative Quality: 77.79 (#13/95)&lt;/li&gt;&lt;li&gt;MirrorCode: 54.4 (#14/39)&lt;/li&gt;&lt;li&gt;Kaggle FACTS (Google): 62.34 (#14/23)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 54.44 (#17/46)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 76.3 (#19/24)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1639.36 (#21/44)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 92.86 (#21/283)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1639.36 (#22/146)&lt;/li&gt;&lt;li&gt;...and 18 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Max (0902)&lt;/strong&gt; — Aggregate Rating 1832, #46&lt;ul&gt;&lt;li&gt;AI for Education Pedagogy - Science: 94.54 (#9/283)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Bug Density: 0.58 (#11/82)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1676.7 (#12/146)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1676.7 (#12/44)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Pass Rate: 83.46 (#14/82)&lt;/li&gt;&lt;li&gt;AI for Education SEND: 87.16 (#15/275)&lt;/li&gt;&lt;li&gt;MirrorCode: 47.8 (#19/39)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 92.86 (#20/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 89.94 (#20/283)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 47.78 (#22/46)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nex-N2.6&lt;/strong&gt; — Aggregate Rating 1831, #47&lt;ul&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 72.662329138215 (#44/273)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Success Rate: 100.0 (#128/419)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Exactness: 48.46 (#213/412)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 92.0 (#241/419)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5&lt;/strong&gt; — Aggregate Rating 1830, #50&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 100.0 (#1/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 96.7 (#2/27)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1245.8634 (#4/25)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1484.29 (#8/51)&lt;/li&gt;&lt;li&gt;VulcanBench v3 (BenchLM): 78.5 (#16/18)&lt;/li&gt;&lt;li&gt;LMArena Vision Arena: 1284.6135 (#20/25)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1476.27 (#31/33)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 57.95 (#33/598)&lt;/li&gt;&lt;li&gt;MirrorCode: 23.3 (#35/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 23.33 (#39/46)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Ember&lt;/strong&gt; — Aggregate Rating 1825, #52&lt;ul&gt;&lt;li&gt;LLM Stats (Agents' Last Exam): 61.6 (#1/28)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench v1.1: 25.0 (#2/38)&lt;/li&gt;&lt;li&gt;Vals AI Legal Index: 75.686 (#4/38)&lt;/li&gt;&lt;li&gt;LLM Stats (OSWorld 2.1 (strict)): 33.8 (#6/6)&lt;/li&gt;&lt;li&gt;LLM Stats (OSWorld 2.1 (partial)): 69.5 (#7/7)&lt;/li&gt;&lt;li&gt;Design Arena (Website): 1330.0 (#9/209)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - HR - Raw Objective Completion: 80.95 (#9/263)&lt;/li&gt;&lt;li&gt;GDP.pdf: 49.2 (#9/21)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 49.038 (#11/94)&lt;/li&gt;&lt;li&gt;APEX v1: 70.4 (#12/78)&lt;/li&gt;&lt;li&gt;...and 119 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Inkling 2 Small&lt;/strong&gt; — Aggregate Rating 1822, #53&lt;ul&gt;&lt;li&gt;LLM Stats (Tau3 Banking): 40.4 (#2/21)&lt;/li&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, No Tools): 44.1 (#3/58)&lt;/li&gt;&lt;li&gt;LLM Stats (MCP Atlas): 85.0 (#3/43)&lt;/li&gt;&lt;li&gt;SEAL - MCP Atlas: 85.4 (#6/37)&lt;/li&gt;&lt;li&gt;VIALS-AA - Pass@3: 72.05 (#7/21)&lt;/li&gt;&lt;li&gt;LLM Stats (BrowseComp): 90.2 (#7/76)&lt;/li&gt;&lt;li&gt;LLM Stats (GDPval-AA): 1630.0 (#7/18)&lt;/li&gt;&lt;li&gt;ZeroEval GPQA Diamond: 93.0 (#11/265)&lt;/li&gt;&lt;li&gt;VIALS-AA: 42.86 (#13/21)&lt;/li&gt;&lt;li&gt;VIALS-AA - Average of 3: 57.76 (#13/21)&lt;/li&gt;&lt;li&gt;...and 28 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3&lt;/strong&gt; — Aggregate Rating 1822, #54&lt;ul&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 7-8: 96.67 (#1/22)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Baseten - Overall: 69.877 (#1/2)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Fireworks - Finance Agent v2: 59.291 (#1/3)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Baseten - Finance Agent v2: 58.998 (#1/2)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Together AI - Vibe Code Bench: 76.907 (#1/2)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Together AI - Legal Research: 71.212 (#1/2)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Baseten - Legal Research: 68.182 (#1/2)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 11-12: 98.33 (#2/23)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Fireworks - Overall: 69.927 (#2/3)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Together AI - Overall: 66.14 (#2/2)&lt;/li&gt;&lt;li&gt;...and 33 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Max&lt;/strong&gt; — Aggregate Rating 1820, #56&lt;ul&gt;&lt;li&gt;LMArena Search Arena: 1259.7846 (#1/25)&lt;/li&gt;&lt;li&gt;MathArena - ARXIVLEAN June: 45.65 (#3/14)&lt;/li&gt;&lt;li&gt;LMArena Vision Arena: 1299.0385 (#3/25)&lt;/li&gt;&lt;li&gt;F5 Labs AI Security Leaderboard - CASI: 92.31 (#6/10)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE April: 45.49 (#6/19)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE August: 69.64 (#9/21)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV April: 59.17 (#9/23)&lt;/li&gt;&lt;li&gt;MathArena Arxiv: 69.17 (#10/26)&lt;/li&gt;&lt;li&gt;MathArena Arxiv False: 32.5 (#12/23)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV June: 75.69 (#12/31)&lt;/li&gt;&lt;li&gt;...and 11 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.8&lt;/strong&gt; — Aggregate Rating 1818, #57&lt;ul&gt;&lt;li&gt;MirrorCode: 65.6 (#8/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 65.56 (#10/46)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 83.3 (#11/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 90.0 (#11/27)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1474.55 (#13/51)&lt;/li&gt;&lt;li&gt;LMArena Vision Arena: 1285.9478 (#18/25)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1202.417 (#19/25)&lt;/li&gt;&lt;li&gt;AA Omniscience: 28.75 (#44/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1555.44 (#47/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 39.25 (#78/598)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash&lt;/strong&gt; — Aggregate Rating 1815, #59&lt;ul&gt;&lt;li&gt;Vals Inference Provider - Fireworks - Overall: 71.161 (#1/3)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Together AI - Overall: 67.794 (#1/2)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - First Party - Overall: 66.597 (#1/3)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Together AI - Finance Agent v2: 60.236 (#1/2)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Fireworks - Vibe Code Bench: 84.096 (#1/3)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Baseten - Vibe Code Bench: 82.491 (#1/2)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Fireworks - Legal Research: 72.727 (#1/3)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Baseten - Overall: 64.175 (#2/2)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - First Party - Finance Agent v2: 56.743 (#2/3)&lt;/li&gt;&lt;li&gt;Vals Inference Provider - Fireworks - Finance Agent v2: 56.661 (#2/3)&lt;/li&gt;&lt;li&gt;...and 31 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Terra&lt;/strong&gt; — Aggregate Rating 1814, #61&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 61.25 (#15/23)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1467.66 (#16/51)&lt;/li&gt;&lt;li&gt;MirrorCode: 41.1 (#24/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 41.11 (#28/46)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 61.0 (#38/67)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Particle-1&lt;/strong&gt; — Aggregate Rating 1814, #62&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 100.0 (#4/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 96.7 (#4/27)&lt;/li&gt;&lt;li&gt;The Aggregate - Day-one Recall Check: 38.2 (#9/43)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;snowhare&lt;/strong&gt; — Aggregate Rating 1813, #64&lt;ul&gt;&lt;li&gt;Design Arena (Agents - Web Apps): 1243.0 (#16/57)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3&lt;/strong&gt; — Aggregate Rating 1812, #65&lt;ul&gt;&lt;li&gt;AppWorld Normal: 94.0 (#1/15)&lt;/li&gt;&lt;li&gt;AppWorld Challenge: 89.5 (#2/15)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 91.7 (#7/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 92.9 (#7/27)&lt;/li&gt;&lt;li&gt;GDP.pdf: 36.6 (#18/21)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1622.17 (#28/146)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Bug Density: 0.66 (#29/82)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Pass Rate: 78.49 (#42/82)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Code Smell Density: 19.81 (#58/82)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Vulnerability Density: 0.32 (#59/82)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy4 preview&lt;/strong&gt; — Aggregate Rating 1811, #66&lt;ul&gt;&lt;li&gt;AI for Education Pedagogy - Technology: 87.74 (#19/283)&lt;/li&gt;&lt;li&gt;GDP.pdf: 35.0 (#19/21)&lt;/li&gt;&lt;li&gt;LMArena WebDev Arena: 1631.47 (#23/44)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1631.47 (#24/146)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Social studies: 88.18 (#28/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 89.32 (#36/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 88.52 (#37/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 90.48 (#45/283)&lt;/li&gt;&lt;li&gt;AI for Education SEND: 83.03 (#55/275)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Primary: 92.02 (#58/283)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 5.5&lt;/strong&gt; — Aggregate Rating 1809, #67&lt;ul&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - Humanities: 100.0 (#1/153)&lt;/li&gt;&lt;li&gt;F5 Labs AI Security Leaderboard - CASI: 94.61 (#2/10)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JaNLI: 100.0 (#2/153)&lt;/li&gt;&lt;li&gt;VulcanBench v3 (BenchLM): 92.2 (#3/18)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Application Development: 75.13 (#4/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - Roleplay: 99.5 (#4/153)&lt;/li&gt;&lt;li&gt;MirrorCode: 78.9 (#5/39)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 90.6 (#5/24)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 78.89 (#5/46)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Coding: 89.4 (#5/148)&lt;/li&gt;&lt;li&gt;...and 104 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWE-2 Flash&lt;/strong&gt; — Aggregate Rating 1806, #71&lt;ul&gt;&lt;li&gt;LLM Stats (SWE Atlas - Codebase QnA): 50.0 (#5/9)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 2.1): 89.2 (#5/56)&lt;/li&gt;&lt;li&gt;LLM Stats (ExploitBench): 34.8 (#8/10)&lt;/li&gt;&lt;li&gt;LLM Stats (FrontierCode 1.1): 45.5 (#12/25)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 4.0): 20.0 (#32/37)&lt;/li&gt;&lt;li&gt;LLM Stats (DeepSWE 1.1): 60.1 (#35/54)&lt;/li&gt;&lt;li&gt;LLM Stats Score: 43.78 (#56/412)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.6-Pro&lt;/strong&gt; — Aggregate Rating 1806, #72&lt;ul&gt;&lt;li&gt;Coarena: 954.6 (#19/24)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 76.0 (#20/24)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1623.91 (#27/146)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Technology: 83.96 (#74/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Social studies: 84.55 (#85/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Primary: 89.67 (#95/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 84.91 (#99/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 85.71 (#100/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 85.32 (#101/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Science: 89.07 (#103/283)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.6 Flash&lt;/strong&gt; — Aggregate Rating 1805, #73&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 67.47 (#10/23)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1456.13 (#26/51)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1482.53 (#26/33)&lt;/li&gt;&lt;li&gt;MirrorCode: 23.3 (#33/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 23.33 (#40/46)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1537.86 (#54/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 22.13 (#69/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 49.97 (#69/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 55.63 (#159/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM 5.3 FlashX&lt;/strong&gt; — Aggregate Rating 1805, #74&lt;ul&gt;&lt;li&gt;OpenRouter Tau2-Bench Airline: 73.3 (#55/146)&lt;/li&gt;&lt;li&gt;OpenRouter GPQA Diamond: 85.9 (#62/150)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.7&lt;/strong&gt; — Aggregate Rating 1804, #75&lt;ul&gt;&lt;li&gt;LMArena Vision Arena: 1300.2867 (#2/25)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1495.68 (#6/51)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1501.06 (#9/33)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1230.0981 (#9/25)&lt;/li&gt;&lt;li&gt;MirrorCode: 43.3 (#21/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 43.33 (#25/46)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1557.23 (#46/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 27.27 (#51/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 48.88 (#78/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 42.29 (#92/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.8 2.4T A95B&lt;/strong&gt; — Aggregate Rating 1804, #78&lt;ul&gt;&lt;li&gt;MathArena - ARXIVLEAN September: 1.89 (#8/8)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV September: 45.28 (#11/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE September: 33.02 (#12/15)&lt;/li&gt;&lt;li&gt;Coarena: 987.7 (#14/24)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 77.8 (#17/24)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark Open&lt;/strong&gt; — Aggregate Rating 1803, #80&lt;ul&gt;&lt;li&gt;MathArena - ARXIV_FALSE September: 27.67 (#13/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV September: 32.08 (#14/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE June: 42.28 (#16/30)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV June: 68.75 (#16/31)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE August: 29.17 (#18/21)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV August: 29.82 (#21/22)&lt;/li&gt;&lt;li&gt;YC-Bench: 1082.37575 (#35/66)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1526.17 (#57/146)&lt;/li&gt;&lt;li&gt;NYT Connections Extended: 78.4 (#61/139)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.5&lt;/strong&gt; — Aggregate Rating 1802, #81&lt;ul&gt;&lt;li&gt;Vals AI Web Search: 38.747 (#10/12)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1215.8179 (#13/25)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV June: 66.67 (#18/31)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE June: 29.32 (#19/30)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1451.49 (#28/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1552.43 (#49/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 51.55 (#60/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 25.32 (#62/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 54.15 (#155/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 (2026-03-05)&lt;/strong&gt; — Aggregate Rating 1799, #85&lt;ul&gt;&lt;li&gt;Epoch AI - Mirrorcode: 23.33 (#38/46)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro (0813)&lt;/strong&gt; — Aggregate Rating 1799, #86&lt;ul&gt;&lt;li&gt;Terminal-Bench 4.0 (Claude Code): 6.67 (#26/26)&lt;/li&gt;&lt;li&gt;MirrorCode: 33.3 (#28/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 33.33 (#33/46)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Swift: 30.77 (#62/142)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond - Knowledge: 15.2 (#69/169)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - HTML: 16.67 (#72/142)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 16.67 (#79/135)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Humanities &amp; Social Sciences: 5.56 (#79/142)&lt;/li&gt;&lt;li&gt;AA CritPt: 18.57 (#80/602)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond: 20.5 (#80/171)&lt;/li&gt;&lt;li&gt;...and 62 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.1&lt;/strong&gt; — Aggregate Rating 1799, #87&lt;ul&gt;&lt;li&gt;LMArena Text Arena: 1490.74 (#14/33)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 93.65 (#16/283)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1463.46 (#22/51)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 90.1 (#26/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Primary: 93.9 (#28/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 88.99 (#31/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Technology: 85.85 (#38/283)&lt;/li&gt;&lt;li&gt;AA Omniscience: 28.08 (#49/598)&lt;/li&gt;&lt;li&gt;AI for Education SEND: 83.49 (#50/275)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1541.55 (#52/146)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Pro (Preview)&lt;/strong&gt; — Aggregate Rating 1796, #92&lt;ul&gt;&lt;li&gt;MathArena - ArXiv Math Dec 2025: 66.18 (#1/20)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 985-988: 100.0 (#1/6)&lt;/li&gt;&lt;li&gt;MathArena - APEX Shortlist 2025: 92.02 (#2/38)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 943-970: 85.71 (#2/18)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV February: 62.5 (#3/26)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV March: 68.33 (#3/15)&lt;/li&gt;&lt;li&gt;MathArena - Usamo 2026: 74.4 (#3/9)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 971-984: 89.29 (#3/11)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 3-4: 76.04 (#4/23)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 5-6: 86.67 (#4/22)&lt;/li&gt;&lt;li&gt;...and 25 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ERNIE 5.5 Flash Preview&lt;/strong&gt; — Aggregate Rating 1794, #95&lt;ul&gt;&lt;li&gt;LLM Stats (APEX-Agents): 64.2 (#2/14)&lt;/li&gt;&lt;li&gt;LLM Stats (NL2Repo): 64.8 (#2/27)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 100.0 (#3/25)&lt;/li&gt;&lt;li&gt;LLM Stats (MCP Atlas): 84.6 (#4/43)&lt;/li&gt;&lt;li&gt;LLM Stats (OfficeQA Pro): 67.9 (#4/15)&lt;/li&gt;&lt;li&gt;LLM Stats (Humanity's Last Exam (no tools, text-only)): 46.4 (#5/15)&lt;/li&gt;&lt;li&gt;LLM Stats (BrowseComp): 87.4 (#10/76)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench-Science 0.1): 21.0 (#12/13)&lt;/li&gt;&lt;li&gt;LLM Stats Score: 52.33 (#13/412)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 4.0): 46.5 (#14/37)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.2&lt;/strong&gt; — Aggregate Rating 1793, #96&lt;ul&gt;&lt;li&gt;LMArena Vision Arena: 1293.5088 (#9/25)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1493.48 (#13/33)&lt;/li&gt;&lt;li&gt;AA Omniscience: 27.2 (#52/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1533.51 (#56/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 33.29 (#62/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 45.38 (#102/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Luna&lt;/strong&gt; — Aggregate Rating 1793, #97&lt;ul&gt;&lt;li&gt;Coarena - Task Completion: 91.7 (#3/24)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JSICK: 86.0 (#3/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - JTruthfulQA: 76.98 (#4/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - JHumanEval: 97.44 (#4/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - JCoLA (in-domain): 83.0 (#5/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Application Development: 72.85 (#6/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Coding: 86.57 (#6/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - MAWPS: 98.0 (#9/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - ALT - Ethics &amp; Morality: 97.0 (#11/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - MMLU-ProX (Japanese): 90.0 (#11/153)&lt;/li&gt;&lt;li&gt;...and 90 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.5 Flash&lt;/strong&gt; — Aggregate Rating 1791, #101&lt;ul&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 5-6: 90.0 (#1/22)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 9-10: 100.0 (#3/22)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 985-988: 100.0 (#3/6)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 1-2: 89.58 (#4/23)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 971-984: 82.14 (#4/11)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE March: 16.07 (#5/15)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 7-8: 93.33 (#5/22)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 943-970: 78.57 (#5/18)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 3-4: 72.92 (#6/23)&lt;/li&gt;&lt;li&gt;MathArena - HMMT Feb 2026: 95.45 (#6/31)&lt;/li&gt;&lt;li&gt;...and 20 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.6&lt;/strong&gt; — Aggregate Rating 1790, #103&lt;ul&gt;&lt;li&gt;MathArena - Project Euler 971-984: 92.86 (#1/11)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 985-988: 100.0 (#2/6)&lt;/li&gt;&lt;li&gt;MathArena - ArXiv Math Dec 2025: 57.35 (#3/20)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 943-970: 83.93 (#3/18)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1507.61 (#3/51)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1251.6735 (#3/25)&lt;/li&gt;&lt;li&gt;MathArena - ArXiv Math Jan 2026: 72.83 (#4/27)&lt;/li&gt;&lt;li&gt;LMArena Vision Arena: 1298.7552 (#4/25)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV March: 62.5 (#5/15)&lt;/li&gt;&lt;li&gt;MathArena - HMMT Feb 2026: 96.21 (#5/31)&lt;/li&gt;&lt;li&gt;...and 20 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling 3.1 Flash&lt;/strong&gt; — Aggregate Rating 1790, #104&lt;ul&gt;&lt;li&gt;Nejumi 4 - GLP - Expression: 99.67 (#2/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - MAWPS: 99.0 (#2/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - Writing: 99.5 (#2/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - Toxicity - Fairness: 100.0 (#2/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - MAWPS: 99.0 (#3/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - Humanities: 100.0 (#3/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - Math: 100.0 (#3/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese): 98.5 (#5/148)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - JaNLI: 100.0 (#5/153)&lt;/li&gt;&lt;li&gt;Nejumi 4 - ARC-AGI-1: 98.0 (#5/153)&lt;/li&gt;&lt;li&gt;...and 91 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M3.1-Flash&lt;/strong&gt; — Aggregate Rating 1789, #105&lt;ul&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 96.7 (#3/27)&lt;/li&gt;&lt;li&gt;HiL-Bench: 65.67 (#4/32)&lt;/li&gt;&lt;li&gt;OSWorld 2.0: 26.85 (#5/13)&lt;/li&gt;&lt;li&gt;OSWorld 2.0 Partial: 62.01 (#5/13)&lt;/li&gt;&lt;li&gt;YC-Bench: 2389.9464 (#6/66)&lt;/li&gt;&lt;li&gt;sudo L7: 18.9 (#7/19)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 88.8 (#8/24)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 75.0 (#17/25)&lt;/li&gt;&lt;li&gt;GDP.pdf: 27.0 (#21/21)&lt;/li&gt;&lt;li&gt;Coarena: 932.0 (#23/24)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 5 Preview&lt;/strong&gt; — Aggregate Rating 1787, #108&lt;ul&gt;&lt;li&gt;Creative Writing (Lechmazur): -0.103 (#35/64)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1572.46 (#44/146)&lt;/li&gt;&lt;li&gt;NYT Connections Extended: 76.6 (#64/139)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5&lt;/strong&gt; — Aggregate Rating 1785, #110&lt;ul&gt;&lt;li&gt;MirrorCode: 63.3 (#10/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 63.33 (#13/46)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1187.7432 (#24/25)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1459.54 (#25/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1539.24 (#53/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 39.37 (#79/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 16.45 (#89/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 40.05 (#139/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Flash&lt;/strong&gt; — Aggregate Rating 1785, #111&lt;ul&gt;&lt;li&gt;AI Coding Daily (OpenCode) - React-TS Code Quality: 17.67 (#7/21)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Total: 67.31 (#9/21)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Laravel Code Quality: 16.35 (#17/21)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 92.06 (#29/283)&lt;/li&gt;&lt;li&gt;AI for Education SEND: 84.4 (#36/275)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 88.77 (#47/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 88.05 (#47/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Primary: 92.49 (#51/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Science: 90.71 (#75/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Social studies: 84.55 (#84/283)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Pro (Preview)&lt;/strong&gt; — Aggregate Rating 1784, #113&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 55.75 (#41/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 15.28 (#92/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.0 (#454/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.8 Omni Flash&lt;/strong&gt; — Aggregate Rating 1783, #114&lt;ul&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 37.926231528421 (#195/273)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.3 Codex&lt;/strong&gt; — Aggregate Rating 1783, #115&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 52.88 (#56/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 10.87 (#105/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.18 (#435/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek-V4-Flash-Vision-Exp&lt;/strong&gt; — Aggregate Rating 1782, #116&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 38.57 (#149/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -17.63 (#254/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.48 (#491/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.8-Flash-Next&lt;/strong&gt; — Aggregate Rating 1782, #117&lt;ul&gt;&lt;li&gt;Arena AI Code: 1624.47 (#26/146)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 68.0 (#30/67)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 27.14 (#46/135)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Earth Sciences: 16.67 (#57/144)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences: 18.52 (#58/144)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 28.58 (#72/130)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 3.92 (#74/144)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1: 7.14 (#75/146)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Life Sciences: 3.51 (#83/144)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Physical Sciences: 3.92 (#87/144)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Luna&lt;/strong&gt; — Aggregate Rating 1778, #121&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 56.89 (#19/23)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1453.8 (#27/51)&lt;/li&gt;&lt;li&gt;MirrorCode: 21.1 (#36/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 21.11 (#41/46)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 55.0 (#43/67)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 42.45 (#122/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -10.77 (#220/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 75.0 (#267/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3 Flash&lt;/strong&gt; — Aggregate Rating 1777, #122&lt;ul&gt;&lt;li&gt;AppWorld Challenge: 92.8 (#1/15)&lt;/li&gt;&lt;li&gt;AppWorld Normal: 93.5 (#2/15)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 83.3 (#13/25)&lt;/li&gt;&lt;li&gt;Terminal-Bench 4.0 (Claude Code): 36.67 (#18/26)&lt;/li&gt;&lt;li&gt;Coarena: 926.0 (#24/24)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 63.6 (#24/24)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1608.57 (#32/146)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 64.0 (#34/67)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro (0424)&lt;/strong&gt; — Aggregate Rating 1777, #124&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 42.95 (#118/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -10.57 (#218/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.85 (#412/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agnes 2.5 Pro Theta&lt;/strong&gt; — Aggregate Rating 1775, #125&lt;ul&gt;&lt;li&gt;AA Long Context Reasoning: 84.67 (#9/598)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 67.0 (#31/67)&lt;/li&gt;&lt;li&gt;AA-Briefcase - PDF - Rubric Pass Rate: 56.9 (#35/268)&lt;/li&gt;&lt;li&gt;AA-Briefcase - Analytical Quality Elo: 1709.99 (#37/268)&lt;/li&gt;&lt;li&gt;AA-Briefcase - XLSX - Rubric Pass Rate: 53.9 (#37/268)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 78.89 (#44/261)&lt;/li&gt;&lt;li&gt;AA-Briefcase: 52.0 (#49/268)&lt;/li&gt;&lt;li&gt;AA SciCode: 56.02 (#50/253)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 80.03 (#50/261)&lt;/li&gt;&lt;li&gt;AA-Briefcase - PPTX - Rubric Pass Rate: 46.12 (#60/268)&lt;/li&gt;&lt;li&gt;...and 61 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4&lt;/strong&gt; — Aggregate Rating 1775, #127&lt;ul&gt;&lt;li&gt;Arena AI Document: 1472.95 (#14/51)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1198.0273 (#21/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 41.7 (#21/25)&lt;/li&gt;&lt;li&gt;LMArena Vision Arena: 1282.2033 (#23/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 60.0 (#23/27)&lt;/li&gt;&lt;li&gt;MirrorCode: 23.3 (#34/39)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 50.85 (#66/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 5.78 (#115/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.58 (#342/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark&lt;/strong&gt; — Aggregate Rating 1773, #129&lt;ul&gt;&lt;li&gt;LMArena Vision Arena: 1294.3962 (#7/25)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1489.1 (#17/33)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1443.55 (#32/51)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 49.6 (#73/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 7.17 (#112/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 84.19 (#358/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 4&lt;/strong&gt; — Aggregate Rating 1773, #131&lt;ul&gt;&lt;li&gt;MathArena - ARXIV_FALSE June: 22.22 (#20/30)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV June: 65.97 (#20/31)&lt;/li&gt;&lt;li&gt;GDP.pdf: 32.8 (#20/21)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE August: 11.31 (#21/21)&lt;/li&gt;&lt;li&gt;VIALS-AA: 20.5 (#21/21)&lt;/li&gt;&lt;li&gt;Terminal-Bench 4.0 (Claude Code): 27.88 (#21/26)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV August: 24.56 (#22/22)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 40.0 (#29/46)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 64.0 (#35/67)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1562.21 (#45/146)&lt;/li&gt;&lt;li&gt;...and 17 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.5 (20251101)&lt;/strong&gt; — Aggregate Rating 1772, #134&lt;ul&gt;&lt;li&gt;Arena AI Document: 1465.71 (#18/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1494.0 (#69/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Seed 2.1 Pro&lt;/strong&gt; — Aggregate Rating 1771, #136&lt;ul&gt;&lt;li&gt;Arena AI Code: 1519.25 (#60/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Pro&lt;/strong&gt; — Aggregate Rating 1770, #137&lt;ul&gt;&lt;li&gt;LMArena Vision Arena: 1289.4207 (#14/25)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1206.5725 (#17/25)&lt;/li&gt;&lt;li&gt;LMArena Text Arena: 1485.59 (#22/33)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1433.86 (#38/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1440.65 (#85/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash (0731)&lt;/strong&gt; — Aggregate Rating 1770, #138&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 40.38 (#137/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -14.28 (#238/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.7 (#498/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.2&lt;/strong&gt; — Aggregate Rating 1769, #141&lt;ul&gt;&lt;li&gt;MathArena - ARXIV March: 61.67 (#6/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV February: 46.09 (#8/26)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE March: 12.05 (#10/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE February: 10.48 (#12/17)&lt;/li&gt;&lt;li&gt;MathArena - ARXIVLEAN June: 8.7 (#13/14)&lt;/li&gt;&lt;li&gt;MathArena - HMMT Feb 2026: 92.42 (#13/31)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE April: 18.44 (#15/19)&lt;/li&gt;&lt;li&gt;MathArena - APEX 2025: 19.79 (#15/48)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV April: 47.5 (#16/23)&lt;/li&gt;&lt;li&gt;MathArena Arxiv: 56.67 (#16/26)&lt;/li&gt;&lt;li&gt;...and 12 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Flash (Preview)&lt;/strong&gt; — Aggregate Rating 1768, #144&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 63.78 (#13/23)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 53.43 (#53/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 10.13 (#108/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.41 (#515/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.7 Max&lt;/strong&gt; — Aggregate Rating 1768, #146&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 25.63 (#27/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 13.48 (#100/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 31.13 (#191/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.7 Max (2026-05-17)&lt;/strong&gt; — Aggregate Rating 1762, #152&lt;ul&gt;&lt;li&gt;Arena AI Code: 1515.09 (#62/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;auto&lt;/strong&gt; — Aggregate Rating 1762, #153&lt;ul&gt;&lt;li&gt;Arabic Broad Leaderboard - MMLU: 9.752 (#16/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Translation (incl Dialects): 7.917 (#17/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - RAG QA: 8.341 (#24/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Writing (incl Dialects): 7.818 (#28/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard: 8.834 (#35/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Reasoning &amp; Math: 8.465 (#52/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Trust &amp; Safety: 8.667 (#63/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - General Knowledge: 8.492 (#66/127)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.6-Flash&lt;/strong&gt; — Aggregate Rating 1760, #154&lt;ul&gt;&lt;li&gt;LMArena WebDev Arena: 1636.52 (#22/44)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1636.52 (#23/146)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Technology: 84.91 (#62/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 84.13 (#115/283)&lt;/li&gt;&lt;li&gt;AI for Education SEND: 78.44 (#117/275)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Primary: 87.32 (#120/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 82.98 (#124/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 83.02 (#124/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Science: 85.79 (#128/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Social studies: 80.0 (#139/283)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.5&lt;/strong&gt; — Aggregate Rating 1760, #156&lt;ul&gt;&lt;li&gt;F5 Labs AI Security Leaderboard - ARS: 91.54 (#4/10)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1179.1744 (#25/25)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 46.58 (#92/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 14.0 (#97/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 61.0 (#181/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 Max Preview&lt;/strong&gt; — Aggregate Rating 1757, #159&lt;ul&gt;&lt;li&gt;Arena AI Code: 1482.11 (#74/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 9.17 (#109/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 46.19 (#115/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 37.87 (#153/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;JT-4.1 Flash 236B A21B 0709&lt;/strong&gt; — Aggregate Rating 1755, #162&lt;ul&gt;&lt;li&gt;AA Humanity's Last Exam: 43.56 (#66/686)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 49.4 (#74/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 20.57 (#77/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 43.67 (#97/598)&lt;/li&gt;&lt;li&gt;AA SciCode: 51.97 (#102/253)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench 2.1: 59.55 (#104/243)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Support - Raw Objective Completion: 82.23 (#106/263)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Strict: 17.81 (#109/263)&lt;/li&gt;&lt;li&gt;AA GDPval: 1334.52 (#112/317)&lt;/li&gt;&lt;li&gt;AA CritPt: 14.0 (#115/602)&lt;/li&gt;&lt;li&gt;...and 29 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4.6&lt;/strong&gt; — Aggregate Rating 1754, #164&lt;ul&gt;&lt;li&gt;Arena AI Document: 1481.88 (#10/51)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1220.6184 (#12/25)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1522.01 (#58/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 12.22 (#102/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 48.45 (#123/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 40.87 (#133/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Flash&lt;/strong&gt; — Aggregate Rating 1754, #165&lt;ul&gt;&lt;li&gt;MathArena - BRUMO 2025: 100.0 (#2/45)&lt;/li&gt;&lt;li&gt;MathArena - SMT 2025: 92.92 (#2/43)&lt;/li&gt;&lt;li&gt;MathArena - AIME 2025: 97.5 (#4/61)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 1-2: 87.5 (#5/23)&lt;/li&gt;&lt;li&gt;MathArena - HMMT Feb 2025: 97.5 (#5/60)&lt;/li&gt;&lt;li&gt;MathArena - ArXiv Math Dec 2025: 43.38 (#7/20)&lt;/li&gt;&lt;li&gt;MathArena - HMMT Nov 2025: 93.33 (#7/23)&lt;/li&gt;&lt;li&gt;MathArena - AIME 2026: 96.67 (#7/31)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 971-984: 55.36 (#7/11)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 5-6: 77.5 (#8/22)&lt;/li&gt;&lt;li&gt;...and 13 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5&lt;/strong&gt; — Aggregate Rating 1751, #169&lt;ul&gt;&lt;li&gt;Arena AI Code: 1416.71 (#95/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 40.32 (#138/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -8.73 (#201/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 79.02 (#298/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.20 Multi-Agent&lt;/strong&gt; — Aggregate Rating 1749, #174&lt;ul&gt;&lt;li&gt;LMArena Search Arena: 1205.156 (#18/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agnes 2.5 Pro Beta&lt;/strong&gt; — Aggregate Rating 1747, #178&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 32.8 (#59/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -10.53 (#217/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.77 (#416/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.2&lt;/strong&gt; — Aggregate Rating 1747, #180&lt;ul&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 96.7 (#1/27)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 91.7 (#6/25)&lt;/li&gt;&lt;li&gt;LMArena Search Arena: 1207.0755 (#16/25)&lt;/li&gt;&lt;li&gt;MirrorCode: 32.2 (#31/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 32.22 (#34/46)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1405.0 (#48/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1419.68 (#94/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 44.32 (#107/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 0.27 (#152/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 61.6 (#185/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.6&lt;/strong&gt; — Aggregate Rating 1746, #181&lt;ul&gt;&lt;li&gt;MathArena - ArXiv Math Jan 2026: 71.74 (#5/27)&lt;/li&gt;&lt;li&gt;MathArena - Usamo 2026: 51.19 (#5/9)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE March: 15.62 (#6/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV March: 55.83 (#7/15)&lt;/li&gt;&lt;li&gt;MathArena - HMMT Feb 2026: 94.7 (#8/31)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE February: 11.69 (#10/17)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV February: 42.97 (#10/26)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 87.5 (#10/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 90.0 (#10/27)&lt;/li&gt;&lt;li&gt;MathArena - APEX Shortlist 2025: 77.13 (#11/38)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.20 Beta1&lt;/strong&gt; — Aggregate Rating 1746, #184&lt;ul&gt;&lt;li&gt;LMArena Search Arena: 1190.885 (#23/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.1&lt;/strong&gt; — Aggregate Rating 1746, #185&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 29.95 (#46/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1508.07 (#65/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 0.85 (#145/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 25.18 (#253/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5 Instant (June 2026)&lt;/strong&gt; — Aggregate Rating 1745, #191&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 44.42 (#106/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 3.7 (#129/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 73.25 (#254/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro&lt;/strong&gt; — Aggregate Rating 1745, #192&lt;ul&gt;&lt;li&gt;MathArena - ArXiv Math Jan 2026: 73.91 (#2/27)&lt;/li&gt;&lt;li&gt;MathArena - Usamo 2026: 60.71 (#4/9)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV February: 51.56 (#6/26)&lt;/li&gt;&lt;li&gt;MathArena - APEX Shortlist 2025: 87.77 (#6/38)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE February: 13.31 (#7/17)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE March: 15.18 (#7/15)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV March: 55.83 (#9/15)&lt;/li&gt;&lt;li&gt;MathArena - AIME 2026: 96.67 (#9/31)&lt;/li&gt;&lt;li&gt;MathArena - APEX 2025: 28.12 (#10/48)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE April: 22.13 (#11/19)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.20 0309 v2&lt;/strong&gt; — Aggregate Rating 1745, #194&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 17.44 (#10/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 14.83 (#95/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 27.48 (#226/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.7 Code&lt;/strong&gt; — Aggregate Rating 1745, #195&lt;ul&gt;&lt;li&gt;Arena AI Code: 1473.86 (#76/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 39.57 (#142/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -10.22 (#214/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.38 (#338/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.2 Codex&lt;/strong&gt; — Aggregate Rating 1745, #196&lt;ul&gt;&lt;li&gt;Arena AI Code: 1338.32 (#121/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 41.07 (#131/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -2.17 (#168/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 73.36 (#255/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy3&lt;/strong&gt; — Aggregate Rating 1742, #200&lt;ul&gt;&lt;li&gt;Arena AI Code: 1505.67 (#66/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 31.97 (#185/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 74.13 (#259/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -18.47 (#260/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 4 Preview&lt;/strong&gt; — Aggregate Rating 1742, #201&lt;ul&gt;&lt;li&gt;LLM Stats (AutomationBench; Protocol unverified — mistral-large-4): 59.9 (#1/1)&lt;/li&gt;&lt;li&gt;LLM Stats (SWE Atlas - Codebase QnA): 59.4 (#1/9)&lt;/li&gt;&lt;li&gt;LLM Stats (Harvey's Legal Agent Benchmark): 15.8 (#4/5)&lt;/li&gt;&lt;li&gt;LLM Stats (Finance Agent v2): 54.7 (#5/28)&lt;/li&gt;&lt;li&gt;LLM Stats (CyberGym): 82.0 (#10/21)&lt;/li&gt;&lt;li&gt;LLM Stats (GDP.pdf): 18.6 (#13/13)&lt;/li&gt;&lt;li&gt;MirrorCode: 40.0 (#25/39)&lt;/li&gt;&lt;li&gt;LLM Stats (DeepSWE 1.1): 61.7 (#34/54)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nex N2 Pro&lt;/strong&gt; — Aggregate Rating 1741, #202&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 34.82 (#171/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -27.05 (#295/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 94.91 (#564/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Beam 1.0&lt;/strong&gt; — Aggregate Rating 1741, #206&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 48.5 (#136/238)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 27B&lt;/strong&gt; — Aggregate Rating 1740, #207&lt;ul&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Bug Density: 0.63 (#24/82)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Vulnerability Density: 0.22 (#30/82)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1590.51 (#36/146)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Pass Rate: 77.94 (#49/82)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Code Smell Density: 22.6 (#73/82)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash (0420)&lt;/strong&gt; — Aggregate Rating 1739, #208&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 36.82 (#162/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -23.08 (#280/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 88.97 (#429/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.7 Plus&lt;/strong&gt; — Aggregate Rating 1738, #211&lt;ul&gt;&lt;li&gt;Arena AI Document: 1437.44 (#35/51)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 27.66 (#38/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 1.08 (#143/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 22.52 (#303/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Jais-3-32B&lt;/strong&gt; — Aggregate Rating 1737, #218&lt;ul&gt;&lt;li&gt;Arabic Broad Leaderboard - General Knowledge: 9.635 (#2/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Translation (incl Dialects): 8.083 (#4/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Reasoning &amp; Math: 9.512 (#11/127)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 73.02 (#12/100)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - MMLU: 9.752 (#14/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard: 8.994 (#16/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Writing (incl Dialects): 7.545 (#54/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Trust &amp; Safety: 8.667 (#61/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - RAG QA: 6.634 (#66/127)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 31.71 (#71/100)&lt;/li&gt;&lt;li&gt;...and 14 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 Plus&lt;/strong&gt; — Aggregate Rating 1731, #226&lt;ul&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 86.7 (#12/27)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 34.64 (#67/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1461.4 (#82/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 0.88 (#144/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 26.38 (#238/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5 Instant&lt;/strong&gt; — Aggregate Rating 1731, #227&lt;ul&gt;&lt;li&gt;Arena AI Document: 1403.24 (#50/51)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3&lt;/strong&gt; — Aggregate Rating 1730, #228&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 38.57 (#148/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -15.55 (#244/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 88.09 (#417/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.1&lt;/strong&gt; — Aggregate Rating 1730, #230&lt;ul&gt;&lt;li&gt;LMArena Search Arena: 1199.4167 (#20/25)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1403.01 (#51/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1397.11 (#104/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 5.4 (#117/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 51.89 (#143/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 37.72 (#155/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro Preview&lt;/strong&gt; — Aggregate Rating 1730, #231&lt;ul&gt;&lt;li&gt;Arena AI Code: 1463.27 (#79/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.20 0309&lt;/strong&gt; — Aggregate Rating 1729, #233&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 22.4 (#16/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 12.95 (#101/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 28.88 (#212/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5-Turbo&lt;/strong&gt; — Aggregate Rating 1727, #237&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 62.64 (#191/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 28.45 (#215/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -16.37 (#247/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Motif 3 (Beta)&lt;/strong&gt; — Aggregate Rating 1726, #239&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 49.1 (#126/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -17.77 (#257/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 21.02 (#321/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 Omni Plus&lt;/strong&gt; — Aggregate Rating 1726, #241&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 37.0 (#72/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -13.12 (#235/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.43 (#401/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5 Instant (May 2026)&lt;/strong&gt; — Aggregate Rating 1725, #243&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 46.25 (#95/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 11.15 (#104/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 65.3 (#209/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Dots3-Note (Preview)&lt;/strong&gt; — Aggregate Rating 1725, #244&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 100.0 (#2/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nex-N2.5-Pro&lt;/strong&gt; — Aggregate Rating 1723, #248&lt;ul&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 82.1 (#16/27)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 70.8 (#18/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5 Codex&lt;/strong&gt; — Aggregate Rating 1723, #249&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 38.7 (#147/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -7.53 (#195/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 75.42 (#271/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 397B A17B&lt;/strong&gt; — Aggregate Rating 1722, #252&lt;ul&gt;&lt;li&gt;Arena AI Code: 1400.11 (#99/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 30.8 (#195/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -30.75 (#309/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.69 (#345/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4.5&lt;/strong&gt; — Aggregate Rating 1721, #254&lt;ul&gt;&lt;li&gt;Arena AI Document: 1449.34 (#30/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1393.38 (#105/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 49.08 (#125/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -0.08 (#156/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 32.87 (#183/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.5&lt;/strong&gt; — Aggregate Rating 1720, #256&lt;ul&gt;&lt;li&gt;MathArena - BRUMO 2025: 98.33 (#5/45)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 985-988: 56.25 (#6/6)&lt;/li&gt;&lt;li&gt;MathArena - AIME 2025: 95.83 (#7/61)&lt;/li&gt;&lt;li&gt;MathArena - CMIMC 2025: 90.62 (#7/36)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 971-984: 51.79 (#8/11)&lt;/li&gt;&lt;li&gt;MathArena - ArXiv Math Dec 2025: 41.91 (#9/20)&lt;/li&gt;&lt;li&gt;MathArena - Project Euler 943-970: 63.39 (#9/18)&lt;/li&gt;&lt;li&gt;MathArena - ArXiv Math Jan 2026: 62.5 (#10/27)&lt;/li&gt;&lt;li&gt;MathArena - Kangaroo 2025 Levels 1-2: 76.04 (#10/23)&lt;/li&gt;&lt;li&gt;MathArena - SMT 2025: 90.57 (#10/43)&lt;/li&gt;&lt;li&gt;...and 17 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok Build 0.1&lt;/strong&gt; — Aggregate Rating 1720, #259&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 51.52 (#61/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 6.37 (#113/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.12 (#534/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.1 (20250805)&lt;/strong&gt; — Aggregate Rating 1720, #260&lt;ul&gt;&lt;li&gt;Arena AI Code: 1389.42 (#106/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Quasar 438B&lt;/strong&gt; — Aggregate Rating 1720, #261&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 21.44 (#13/598)&lt;/li&gt;&lt;li&gt;BenchLM: 49.0 (#100/200)&lt;/li&gt;&lt;li&gt;AA Omniscience: -2.6 (#169/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.52 (#451/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;JT-4.1 Flash 236B A21B&lt;/strong&gt; — Aggregate Rating 1720, #262&lt;ul&gt;&lt;li&gt;AutomationBench-AA - Finance - Raw Objective Completion: 87.27 (#43/263)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 52.0 (#46/67)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Dart: 66.67 (#49/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Python: 47.87 (#54/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Java: -7.27 (#62/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Law: 0.67 (#66/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - C: 52.73 (#67/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Humanities &amp; Social Sciences: 8.22 (#71/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE): 27.5 (#72/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - R: -26.67 (#72/142)&lt;/li&gt;&lt;li&gt;...and 52 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash&lt;/strong&gt; — Aggregate Rating 1720, #263&lt;ul&gt;&lt;li&gt;MathArena - ARXIV_FALSE March: 19.64 (#4/15)&lt;/li&gt;&lt;li&gt;MathArena - APEX Shortlist 2025: 89.36 (#5/38)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE February: 17.74 (#6/17)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV February: 43.75 (#9/26)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE April: 23.77 (#10/19)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV March: 52.5 (#10/15)&lt;/li&gt;&lt;li&gt;MathArena - HMMT Feb 2026: 93.94 (#10/31)&lt;/li&gt;&lt;li&gt;MathArena - APEX 2025: 27.08 (#11/48)&lt;/li&gt;&lt;li&gt;MathArena - AIME 2026: 95.83 (#13/31)&lt;/li&gt;&lt;li&gt;MathArena Arxiv False: 19.5 (#15/23)&lt;/li&gt;&lt;li&gt;...and 5 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5&lt;/strong&gt; — Aggregate Rating 1719, #264&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 35.32 (#68/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1435.21 (#91/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 0.27 (#151/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 26.3 (#239/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3.2 Speciale&lt;/strong&gt; — Aggregate Rating 1718, #266&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 37.77 (#154/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -17.72 (#256/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.15 (#434/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Pro&lt;/strong&gt; — Aggregate Rating 1717, #269&lt;ul&gt;&lt;li&gt;Arena AI Document: 1422.79 (#42/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1227.46 (#139/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 39.05 (#145/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -16.35 (#246/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.89 (#470/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M3&lt;/strong&gt; — Aggregate Rating 1717, #271&lt;ul&gt;&lt;li&gt;Arena AI Document: 1436.91 (#36/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1484.17 (#73/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Beam 1.0 501B A23B&lt;/strong&gt; — Aggregate Rating 1714, #276&lt;ul&gt;&lt;li&gt;VulcanBench v3 (BenchLM): 64.7 (#18/18)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1437.83 (#87/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Inkling&lt;/strong&gt; — Aggregate Rating 1712, #279&lt;ul&gt;&lt;li&gt;Arena AI Code: 1414.16 (#96/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.1 Codex&lt;/strong&gt; — Aggregate Rating 1712, #281&lt;ul&gt;&lt;li&gt;Arena AI Code: 1337.65 (#122/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 39.9 (#140/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -6.5 (#188/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 77.2 (#286/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.5-Pro&lt;/strong&gt; — Aggregate Rating 1711, #282&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 24.7 (#24/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1479.08 (#75/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 3.25 (#133/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 27.18 (#229/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-3.0-flash-fin&lt;/strong&gt; — Aggregate Rating 1710, #291&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 39.51 (#80/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -14.58 (#239/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.87 (#389/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Inkling Small&lt;/strong&gt; — Aggregate Rating 1708, #298&lt;ul&gt;&lt;li&gt;Arena AI Code: 1407.33 (#97/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.5 Flash Lite&lt;/strong&gt; — Aggregate Rating 1708, #299&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 53.51 (#20/23)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1435.23 (#90/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Cosmos3-Nano&lt;/strong&gt; — Aggregate Rating 1706, #305&lt;ul&gt;&lt;li&gt;CoCoBench - Construct Score: 75.0 (#3/11)&lt;/li&gt;&lt;li&gt;CoCoBench - Task Allocation: 79.8 (#5/11)&lt;/li&gt;&lt;li&gt;CoCoBench - Sequential Ordering: 37.8 (#6/11)&lt;/li&gt;&lt;li&gt;CoCoBench: 29.5 (#8/11)&lt;/li&gt;&lt;li&gt;CoCoBench - Mutual Exclusion: 1.7 (#9/11)&lt;/li&gt;&lt;li&gt;CoCoBench - Handoff Coordination: 0.9 (#9/11)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4&lt;/strong&gt; — Aggregate Rating 1704, #310&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 40.48 (#136/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 2.1 (#137/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 64.49 (#201/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy3-preview&lt;/strong&gt; — Aggregate Rating 1704, #312&lt;ul&gt;&lt;li&gt;Arena AI Code: 1358.23 (#115/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 27.93 (#221/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 75.68 (#273/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -34.95 (#326/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Motif 3&lt;/strong&gt; — Aggregate Rating 1703, #316&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 28.43 (#39/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 10.18 (#107/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 30.07 (#200/598)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Exactness: 47.06 (#240/412)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Success Rate: 96.0 (#327/419)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 78.0 (#354/419)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2-Pro&lt;/strong&gt; — Aggregate Rating 1701, #328&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 29.98 (#47/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1434.13 (#93/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 4.62 (#122/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 26.62 (#237/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.5-122B-A10B-FP8&lt;/strong&gt; — Aggregate Rating 1700, #331&lt;ul&gt;&lt;li&gt;QIMMA - FannOrFlop: 60.56 (#2/100)&lt;/li&gt;&lt;li&gt;QIMMA - PALMX: 81.64 (#3/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 85.37 (#4/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedAraBench: 53.04 (#6/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 54.6 (#7/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabicMMLU: 74.15 (#8/100)&lt;/li&gt;&lt;li&gt;QIMMA - MizanQA: 64.19 (#8/100)&lt;/li&gt;&lt;li&gt;QIMMA - Overall: 66.85 (#10/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 73.02 (#10/100)&lt;/li&gt;&lt;li&gt;QIMMA - AraDiCE-Culture: 75.56 (#12/100)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ernie 5.1&lt;/strong&gt; — Aggregate Rating 1700, #332&lt;ul&gt;&lt;li&gt;LMArena Search Arena: 1226.7996 (#11/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 122B A10B&lt;/strong&gt; — Aggregate Rating 1699, #336&lt;ul&gt;&lt;li&gt;Arena AI Code: 1360.09 (#113/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.37 (#267/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -41.5 (#356/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.09 (#398/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.3&lt;/strong&gt; — Aggregate Rating 1698, #337&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 16.94 (#7/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 17.98 (#87/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1357.45 (#116/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 34.78 (#173/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.20 Beta (0309)&lt;/strong&gt; — Aggregate Rating 1698, #340&lt;ul&gt;&lt;li&gt;Arena AI Document: 1417.1 (#45/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1375.52 (#109/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Super VL&lt;/strong&gt; — Aggregate Rating 1698, #341&lt;ul&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 92.6 (#8/27)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 91.7 (#9/25)&lt;/li&gt;&lt;li&gt;Humanity's Sixth Sense: 24.7 (#29/34)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 27B&lt;/strong&gt; — Aggregate Rating 1698, #342&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 49.27 (#127/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -20.02 (#267/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.6 (#347/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nanbeige4.3-33B&lt;/strong&gt; — Aggregate Rating 1696, #349&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 54.0 (#44/67)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 31B (IT)&lt;/strong&gt; — Aggregate Rating 1696, #350&lt;ul&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 73.3 (#19/27)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 25.0 (#24/25)&lt;/li&gt;&lt;li&gt;Gert Labs Rankings: 24.47 (#99/100)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5 Mini&lt;/strong&gt; — Aggregate Rating 1695, #353&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 44.32 (#105/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -11.03 (#226/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.97 (#256/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O4 Mini&lt;/strong&gt; — Aggregate Rating 1694, #355&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.8 (#261/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.45 (#311/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -35.7 (#331/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5V Turbo&lt;/strong&gt; — Aggregate Rating 1694, #357&lt;ul&gt;&lt;li&gt;Arena AI Document: 1418.77 (#44/51)&lt;/li&gt;&lt;li&gt;Gert Labs Rankings: 24.55 (#97/100)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1399.32 (#101/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 29.33 (#207/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 68.8 (#236/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -19.28 (#264/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.5&lt;/strong&gt; — Aggregate Rating 1693, #361&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 31.93 (#55/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1437.83 (#88/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: -9.83 (#211/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.75 (#417/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 27B&lt;/strong&gt; — Aggregate Rating 1692, #363&lt;ul&gt;&lt;li&gt;AppWorld Challenge: 83.7 (#4/15)&lt;/li&gt;&lt;li&gt;AppWorld Normal: 88.7 (#5/15)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1357.09 (#117/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 75.13 (#268/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.67 (#328/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -43.98 (#377/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Open2 250B&lt;/strong&gt; — Aggregate Rating 1692, #365&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 25.38 (#25/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -1.77 (#165/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.83 (#362/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Super VL (NVFP4)&lt;/strong&gt; — Aggregate Rating 1691, #370&lt;ul&gt;&lt;li&gt;Arena AI Code: 1399.42 (#100/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.7&lt;/strong&gt; — Aggregate Rating 1691, #371&lt;ul&gt;&lt;li&gt;Arena AI Code: 1434.91 (#92/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 29.32 (#208/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -36.43 (#335/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.91 (#528/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Flash Lite&lt;/strong&gt; — Aggregate Rating 1691, #372&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 30.0 (#59/67)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Glimmer 30B&lt;/strong&gt; — Aggregate Rating 1690, #379&lt;ul&gt;&lt;li&gt;Arena AI Code: 1354.56 (#118/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2 Turbo&lt;/strong&gt; — Aggregate Rating 1689, #383&lt;ul&gt;&lt;li&gt;Arena AI Code: 1321.36 (#125/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O1&lt;/strong&gt; — Aggregate Rating 1687, #393&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 34.53 (#174/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -11.05 (#227/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 69.63 (#242/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Mini&lt;/strong&gt; — Aggregate Rating 1687, #394&lt;ul&gt;&lt;li&gt;Terminal-Bench 4.0 (Codex): 1.82 (#13/13)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1397.52 (#103/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 37.47 (#156/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -18.92 (#261/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.85 (#449/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Apodex 1.1&lt;/strong&gt; — Aggregate Rating 1687, #395&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 46.0 (#49/67)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 40.0 (#97/135)&lt;/li&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 17.0 (#104/137)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 6.93 (#106/135)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 13.67 (#107/130)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 22.39 (#112/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics: -7.33 (#113/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - C: 9.09 (#115/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Rust: 58.82 (#115/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 2.33 (#118/142)&lt;/li&gt;&lt;li&gt;...and 18 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-3.0-flash-VL&lt;/strong&gt; — Aggregate Rating 1687, #396&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 22.05 (#15/598)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 78.3 (#16/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 63.3 (#22/27)&lt;/li&gt;&lt;li&gt;AA Omniscience: -4.53 (#181/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.35 (#479/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Jais-3-MoVA-36B-A4B&lt;/strong&gt; — Aggregate Rating 1686, #399&lt;ul&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 73.54 (#9/100)&lt;/li&gt;&lt;li&gt;QIMMA - FannOrFlop: 12.57 (#66/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 31.71 (#70/100)&lt;/li&gt;&lt;li&gt;QIMMA - MizanQA: 46.23 (#75/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabCulture: 34.5 (#79/100)&lt;/li&gt;&lt;li&gt;QIMMA - PALMX: 31.26 (#85/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM STEM: 28.09 (#86/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 24.1 (#88/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabLegalQA: 54.21 (#90/100)&lt;/li&gt;&lt;li&gt;QIMMA - Overall: 35.08 (#91/100)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 31B&lt;/strong&gt; — Aggregate Rating 1686, #402&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 50.08 (#21/23)&lt;/li&gt;&lt;li&gt;Arena AI Document: 1434.33 (#37/51)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 20.0 (#67/67)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1369.18 (#110/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.95 (#331/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.63 (#421/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -51.68 (#439/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4&lt;/strong&gt; — Aggregate Rating 1685, #407&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 29.12 (#41/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 0.17 (#154/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 22.68 (#299/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 Max&lt;/strong&gt; — Aggregate Rating 1684, #408&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 29.82 (#202/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -36.62 (#336/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.92 (#453/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3.2&lt;/strong&gt; — Aggregate Rating 1683, #409&lt;ul&gt;&lt;li&gt;Arena AI Code: 1362.51 (#112/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 32.97 (#182/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -22.48 (#278/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.72 (#346/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LongCat 2.0&lt;/strong&gt; — Aggregate Rating 1683, #414&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 29.63 (#203/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 75.39 (#270/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -23.42 (#282/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;KAT Coder Pro V2&lt;/strong&gt; — Aggregate Rating 1682, #418&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 55.98 (#160/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -22.47 (#277/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 21.48 (#315/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 Flash&lt;/strong&gt; — Aggregate Rating 1682, #421&lt;ul&gt;&lt;li&gt;Gert Labs Rankings: 28.71 (#85/100)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Flash Lite (Preview)&lt;/strong&gt; — Aggregate Rating 1681, #422&lt;ul&gt;&lt;li&gt;Arena AI Code: 1255.59 (#132/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek R1 0528&lt;/strong&gt; — Aggregate Rating 1680, #428&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 30.53 (#196/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -27.43 (#297/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 83.45 (#353/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.1 Fast&lt;/strong&gt; — Aggregate Rating 1680, #430&lt;ul&gt;&lt;li&gt;Gert Labs Rankings: 26.9 (#90/100)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1241.71 (#136/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 25.08 (#255/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 73.41 (#257/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -29.92 (#306/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3 Ultra&lt;/strong&gt; — Aggregate Rating 1680, #431&lt;ul&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 77.8 (#17/27)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Space Bunny Alpha&lt;/strong&gt; — Aggregate Rating 1679, #433&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 26.1 (#23/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 60.0 (#24/27)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-3.0-flash&lt;/strong&gt; — Aggregate Rating 1679, #435&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 44.05 (#102/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -17.88 (#258/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.17 (#380/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 Flash&lt;/strong&gt; — Aggregate Rating 1677, #441&lt;ul&gt;&lt;li&gt;Arena AI Code: 1243.5 (#135/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Flash (Preview 09-2025)&lt;/strong&gt; — Aggregate Rating 1677, #442&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 28.33 (#218/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -36.1 (#333/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.91 (#452/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3.2 Exp&lt;/strong&gt; — Aggregate Rating 1677, #443&lt;ul&gt;&lt;li&gt;Arena AI Code: 1270.27 (#129/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 27.55 (#224/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -30.97 (#311/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.77 (#314/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2&lt;/strong&gt; — Aggregate Rating 1677, #444&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 30.87 (#193/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -21.42 (#271/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 75.63 (#272/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4 Fast&lt;/strong&gt; — Aggregate Rating 1677, #445&lt;ul&gt;&lt;li&gt;Arena AI Code: 1162.96 (#145/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 68.27 (#232/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 22.78 (#298/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -29.93 (#307/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2-Omni&lt;/strong&gt; — Aggregate Rating 1676, #446&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 48.86 (#124/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -20.13 (#268/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.3 (#350/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3.1 Terminus&lt;/strong&gt; — Aggregate Rating 1676, #448&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 27.7 (#223/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 74.83 (#266/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -26.4 (#289/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 Max (Preview)&lt;/strong&gt; — Aggregate Rating 1674, #451&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 27.35 (#228/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -39.12 (#347/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 88.37 (#420/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Cogito v2.1&lt;/strong&gt; — Aggregate Rating 1674, #454&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 30.18 (#198/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -25.78 (#288/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.16 (#305/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Flash&lt;/strong&gt; — Aggregate Rating 1674, #457&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 26.13 (#241/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 75.29 (#269/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -29.8 (#304/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 235B A22B (Thinking)&lt;/strong&gt; — Aggregate Rating 1673, #458&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.85 (#323/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.15 (#369/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -46.55 (#395/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Pro 4&lt;/strong&gt; — Aggregate Rating 1673, #460&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 46.0 (#50/67)&lt;/li&gt;&lt;li&gt;OpenRouter GPQA Diamond: 80.8 (#89/150)&lt;/li&gt;&lt;li&gt;OpenRouter Tau2-Bench Airline: 66.0 (#103/146)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 14.16 (#106/130)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1368.98 (#111/146)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 4.08 (#113/135)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude 3.7 Sonnet&lt;/strong&gt; — Aggregate Rating 1673, #464&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 39.97 (#82/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -0.73 (#161/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 28.2 (#219/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 235B A22B 2507 (Thinking)&lt;/strong&gt; — Aggregate Rating 1671, #469&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 22.82 (#297/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -46.52 (#393/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.83 (#448/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 3.7 Flash&lt;/strong&gt; — Aggregate Rating 1671, #471&lt;ul&gt;&lt;li&gt;MathArena - ARXIV_FALSE February: 12.1 (#8/17)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE March: 10.71 (#11/15)&lt;/li&gt;&lt;li&gt;MathArena - APEX Shortlist 2025: 76.6 (#12/38)&lt;/li&gt;&lt;li&gt;MathArena - ARXIVLEAN June: 0.0 (#14/14)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE April: 18.85 (#14/19)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV March: 45.0 (#14/15)&lt;/li&gt;&lt;li&gt;MathArena - HMMT Feb 2026: 87.88 (#17/31)&lt;/li&gt;&lt;li&gt;MathArena - APEX 2025: 14.58 (#17/48)&lt;/li&gt;&lt;li&gt;MathArena - AIME 2026: 95.0 (#18/31)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV February: 32.03 (#19/26)&lt;/li&gt;&lt;li&gt;...and 8 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 35B A3B&lt;/strong&gt; — Aggregate Rating 1671, #472&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 50.5 (#132/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -22.18 (#274/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.82 (#363/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ERNIE 5.0 Thinking Preview&lt;/strong&gt; — Aggregate Rating 1670, #477&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 22.13 (#308/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -45.62 (#387/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.01 (#396/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.1&lt;/strong&gt; — Aggregate Rating 1670, #480&lt;ul&gt;&lt;li&gt;Arena AI Code: 1216.59 (#140/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 35B A3B&lt;/strong&gt; — Aggregate Rating 1669, #487&lt;ul&gt;&lt;li&gt;Arena AI Code: 1252.7 (#133/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.13 (#338/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.43 (#375/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -48.1 (#405/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.6&lt;/strong&gt; — Aggregate Rating 1668, #491&lt;ul&gt;&lt;li&gt;Arena AI Code: 1342.1 (#120/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 67.57 (#226/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 26.88 (#232/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -31.68 (#314/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Bolt 74B-A9B&lt;/strong&gt; — Aggregate Rating 1666, #497&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 43.72 (#98/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -19.17 (#263/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.08 (#410/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon MoVA 36B A4B&lt;/strong&gt; — Aggregate Rating 1665, #500&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 30.53 (#51/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -10.95 (#224/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.0 (#465/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4.1&lt;/strong&gt; — Aggregate Rating 1665, #501&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 27.77 (#222/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -39.63 (#350/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.31 (#538/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3.1&lt;/strong&gt; — Aggregate Rating 1664, #504&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 29.0 (#210/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -29.58 (#303/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.51 (#340/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 3.5 Flash&lt;/strong&gt; — Aggregate Rating 1661, #517&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 23.62 (#281/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -41.82 (#360/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.66 (#378/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 3.5 Flash 2603&lt;/strong&gt; — Aggregate Rating 1661, #518&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.62 (#263/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -44.22 (#380/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.31 (#486/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2.7&lt;/strong&gt; — Aggregate Rating 1660, #521&lt;ul&gt;&lt;li&gt;ReasonScape R12: 920.86 (#24/125)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 35.56 (#69/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1397.71 (#102/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: 0.77 (#147/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 26.8 (#234/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 235B A22B Instruct&lt;/strong&gt; — Aggregate Rating 1659, #529&lt;ul&gt;&lt;li&gt;QIMMA - FannOrFlop: 61.01 (#1/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 77.25 (#2/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 85.98 (#3/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ QA: 55.12 (#5/100)&lt;/li&gt;&lt;li&gt;QIMMA - MizanQA: 60.0 (#19/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabLegalQA: 71.02 (#22/100)&lt;/li&gt;&lt;li&gt;QIMMA - Overall: 52.89 (#49/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabCulture: 39.52 (#50/100)&lt;/li&gt;&lt;li&gt;QIMMA - AraDiCE-Culture: 53.89 (#53/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 30.16 (#63/100)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.5&lt;/strong&gt; — Aggregate Rating 1659, #530&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 70.15 (#245/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 25.13 (#254/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -27.38 (#296/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.6-35B-A3B-FP8&lt;/strong&gt; — Aggregate Rating 1658, #534&lt;ul&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 74.87 (#4/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 84.15 (#8/100)&lt;/li&gt;&lt;li&gt;QIMMA - MizanQA: 60.51 (#16/100)&lt;/li&gt;&lt;li&gt;QIMMA - AraDiCE-Culture: 69.44 (#25/100)&lt;/li&gt;&lt;li&gt;QIMMA - FannOrFlop: 55.0 (#27/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabCulture: 43.06 (#41/100)&lt;/li&gt;&lt;li&gt;QIMMA - Overall: 55.63 (#43/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabicMMLU: 54.79 (#55/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 32.16 (#58/100)&lt;/li&gt;&lt;li&gt;QIMMA - PALMX: 59.37 (#59/100)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3 Mini&lt;/strong&gt; — Aggregate Rating 1657, #539&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.08 (#316/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 21.27 (#317/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -42.57 (#366/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 3 Mini&lt;/strong&gt; — Aggregate Rating 1657, #545&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 25.82 (#30/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -6.8 (#191/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.12 (#462/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 4.5 (20251001)&lt;/strong&gt; — Aggregate Rating 1657, #546&lt;ul&gt;&lt;li&gt;Arena AI Document: 1423.0 (#41/51)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1330.08 (#124/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 26B A4B (IT)&lt;/strong&gt; — Aggregate Rating 1655, #555&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 41.7 (#22/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 53.3 (#25/27)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2 0905&lt;/strong&gt; — Aggregate Rating 1654, #560&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 20.8 (#25/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 43.3 (#27/27)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 69.72 (#243/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 25.42 (#252/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -26.58 (#290/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 32B (Thinking)&lt;/strong&gt; — Aggregate Rating 1654, #562&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 83.68 (#355/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.97 (#412/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -52.52 (#446/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.5 Omni Flash&lt;/strong&gt; — Aggregate Rating 1653, #566&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.65 (#473/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -65.07 (#539/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.4 (#541/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2-Flash (Feb 2026)&lt;/strong&gt; — Aggregate Rating 1653, #567&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 48.41 (#122/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -19.05 (#262/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.78 (#345/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2.1&lt;/strong&gt; — Aggregate Rating 1652, #573&lt;ul&gt;&lt;li&gt;Arena AI Code: 1384.31 (#108/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 68.54 (#235/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -32.87 (#317/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 21.17 (#320/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 4.5&lt;/strong&gt; — Aggregate Rating 1651, #575&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 25.71 (#28/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -7.57 (#196/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.43 (#478/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2.5&lt;/strong&gt; — Aggregate Rating 1650, #583&lt;ul&gt;&lt;li&gt;Arena AI Code: 1385.53 (#107/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 26.18 (#240/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -38.87 (#346/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 88.12 (#418/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 32B Instruct&lt;/strong&gt; — Aggregate Rating 1649, #589&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.83 (#471/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.82 (#501/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -63.37 (#528/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 Next 80B A3B (Thinking)&lt;/strong&gt; — Aggregate Rating 1649, #594&lt;ul&gt;&lt;li&gt;Metacognitive Probe: 68.5 (#7/8)&lt;/li&gt;&lt;li&gt;ARMOR 2025: 82.4 (#19/20)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.03 (#358/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.01 (#397/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -51.42 (#435/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ripple-65B Preview&lt;/strong&gt; — Aggregate Rating 1648, #599&lt;ul&gt;&lt;li&gt;Icelandic LLM - Song Lyrics: 12.36 (#49/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Completion: 3.19 (#79/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Flora &amp; Fauna: 0.79 (#83/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Case Government: 5.31 (#84/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Meanings: 16.41 (#87/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Famous Phrases: 0.6 (#89/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - GEC: 10.55 (#93/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM Leaderboard - Average: 7.75 (#94/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - WikiQA: 3.37 (#94/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Textile Terms: 0.0 (#95/95)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 3&lt;/strong&gt; — Aggregate Rating 1647, #603&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 28.9 (#211/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -33.18 (#319/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.32 (#403/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;A.X-K2&lt;/strong&gt; — Aggregate Rating 1646, #612&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 32.99 (#61/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Law: -2.44 (#73/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Business: -0.67 (#86/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Humanities &amp; Social Sciences: 2.89 (#87/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics: 2.5 (#90/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: -18.22 (#93/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Swift: -3.85 (#109/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - R: -56.67 (#116/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - HTML: -16.67 (#122/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Java: -56.36 (#124/142)&lt;/li&gt;&lt;li&gt;...and 20 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 26B A4B&lt;/strong&gt; — Aggregate Rating 1644, #619&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 47.35 (#22/23)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1358.41 (#114/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.1 (#354/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 86.44 (#391/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.83 (#430/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 235B A22B 2507 Instruct&lt;/strong&gt; — Aggregate Rating 1643, #625&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 77.17 (#285/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.73 (#364/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -43.98 (#378/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 235B A22B&lt;/strong&gt; — Aggregate Rating 1642, #629&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 77.62 (#290/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.57 (#368/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -44.7 (#383/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ring-1T&lt;/strong&gt; — Aggregate Rating 1642, #635&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.47 (#331/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.92 (#434/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.75 (#443/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Nano&lt;/strong&gt; — Aggregate Rating 1640, #640&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 51.13 (#136/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 25.67 (#249/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -18.03 (#259/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 Coder 480B A35B Instruct&lt;/strong&gt; — Aggregate Rating 1639, #646&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 44.93 (#110/598)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1274.56 (#128/146)&lt;/li&gt;&lt;li&gt;AA Omniscience: -22.2 (#275/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.68 (#444/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-2.6-1T&lt;/strong&gt; — Aggregate Rating 1639, #647&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 21.9 (#311/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.6 (#425/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.83 (#526/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Jais-3-3.7B&lt;/strong&gt; — Aggregate Rating 1638, #651&lt;ul&gt;&lt;li&gt;Arabic Broad Leaderboard - General Knowledge: 9.27 (#9/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Reasoning &amp; Math: 9.256 (#23/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Translation (incl Dialects): 7.75 (#28/127)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 56.61 (#47/100)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard: 8.626 (#52/127)&lt;/li&gt;&lt;li&gt;QIMMA - GAT: 34.78 (#62/100)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - MMLU: 9.174 (#66/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Writing (incl Dialects): 7.273 (#68/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Trust &amp; Safety: 8.0 (#75/127)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 26.22 (#79/100)&lt;/li&gt;&lt;li&gt;...and 14 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nex-N2.5-mini&lt;/strong&gt; — Aggregate Rating 1638, #656&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 50.0 (#20/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.6-35B-A3B-NVFP4&lt;/strong&gt; — Aggregate Rating 1636, #663&lt;ul&gt;&lt;li&gt;llmbenchmarks.io Local GPU - ARC-AGI-1 Public Eval: 55.5 (#2/12)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - Chess Puzzles: 6.0 (#3/12)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - SciCode: 33.56 (#5/12)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - SimpleQA Verified: 13.1 (#6/24)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - IFEval: 85.4 (#13/24)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - IFBench: 33.14 (#17/24)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;KAT-Coder-Pro V1&lt;/strong&gt; — Aggregate Rating 1636, #664&lt;ul&gt;&lt;li&gt;Arena AI Code: 1257.22 (#131/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 67.36 (#225/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -37.73 (#343/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.7 (#395/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nova 2.0 Omni&lt;/strong&gt; — Aggregate Rating 1635, #672&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 84.7 (#362/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.62 (#367/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.32 (#422/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2-Flash&lt;/strong&gt; — Aggregate Rating 1635, #675&lt;ul&gt;&lt;li&gt;Arena AI Code: 1331.13 (#123/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.83 (#260/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 75.96 (#275/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -45.35 (#385/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Doubao Seed Code&lt;/strong&gt; — Aggregate Rating 1635, #680&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.88 (#259/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 79.99 (#303/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -35.2 (#329/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2&lt;/strong&gt; — Aggregate Rating 1634, #681&lt;ul&gt;&lt;li&gt;QIMMA - FannOrFlop: 56.34 (#23/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 71.95 (#29/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 64.55 (#35/100)&lt;/li&gt;&lt;li&gt;QIMMA - GAT: 34.31 (#68/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 26.1 (#75/100)&lt;/li&gt;&lt;li&gt;QIMMA - AraDiCE-Culture: 36.67 (#81/100)&lt;/li&gt;&lt;li&gt;QIMMA - Overall: 39.15 (#82/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabCulture: 33.93 (#87/100)&lt;/li&gt;&lt;li&gt;QIMMA - PALMX: 20.23 (#94/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM STEM: 20.9 (#94/100)&lt;/li&gt;&lt;li&gt;...and 10 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Jais-3-7B&lt;/strong&gt; — Aggregate Rating 1634, #683&lt;ul&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 65.34 (#32/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabCulture: 36.37 (#61/100)&lt;/li&gt;&lt;li&gt;QIMMA - MizanQA: 46.24 (#74/100)&lt;/li&gt;&lt;li&gt;QIMMA - PALMX: 40.61 (#77/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 27.44 (#78/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ QA: 43.12 (#81/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedAraBench: 24.74 (#81/100)&lt;/li&gt;&lt;li&gt;QIMMA - FannOrFlop: 0.65 (#81/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabLegalQA: 56.01 (#82/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM STEM: 32.39 (#84/100)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K-EXAONE 2.0&lt;/strong&gt; — Aggregate Rating 1632, #700&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 34.0 (#56/67)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 12B (IT)&lt;/strong&gt; — Aggregate Rating 1631, #702&lt;ul&gt;&lt;li&gt;Icelandic LLM - Song Lyrics: 11.9 (#61/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - GEC: 36.7 (#62/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Meanings: 32.31 (#75/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Textile Terms: 13.1 (#76/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Famous Phrases: 1.81 (#78/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Completion: 3.98 (#78/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - WikiQA: 12.13 (#81/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Case Government: 5.62 (#81/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Flora &amp; Fauna: 0.69 (#84/95)&lt;/li&gt;&lt;li&gt;Icelandic LLM Leaderboard - Average: 13.53 (#85/95)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Flash Lite (Preview 09-2025)&lt;/strong&gt; — Aggregate Rating 1631, #703&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 66.06 (#216/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -42.57 (#368/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.98 (#385/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-OSS-120B&lt;/strong&gt; — Aggregate Rating 1631, #704&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 79.2 (#15/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 83.3 (#15/27)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - HTML: -11.11 (#116/142)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 7.71 (#120/135)&lt;/li&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 4.0 (#121/137)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Rust: 41.18 (#124/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - C: -16.36 (#127/142)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Kotlin: -48.57 (#127/142)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 3.82 (#128/130)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 1.49 (#129/142)&lt;/li&gt;&lt;li&gt;...and 18 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.6V&lt;/strong&gt; — Aggregate Rating 1630, #708&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 51.46 (#139/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -26.95 (#294/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.4 (#403/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3 (0324)&lt;/strong&gt; — Aggregate Rating 1629, #712&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.32 (#270/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -40.67 (#353/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.86 (#383/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.1 Codex Mini&lt;/strong&gt; — Aggregate Rating 1629, #713&lt;ul&gt;&lt;li&gt;Arena AI Code: 1244.4 (#134/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 51.58 (#140/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -16.42 (#249/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 23.2 (#290/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok Code Fast 1&lt;/strong&gt; — Aggregate Rating 1628, #715&lt;ul&gt;&lt;li&gt;Arena AI Code: 1163.86 (#144/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 23.55 (#284/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 79.29 (#299/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -37.07 (#338/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K-EXAONE&lt;/strong&gt; — Aggregate Rating 1628, #717&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 88.84 (#426/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.35 (#430/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -57.97 (#486/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 30B A3B (Thinking)&lt;/strong&gt; — Aggregate Rating 1627, #720&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.93 (#413/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.77 (#445/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -57.63 (#483/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3 Coder Next&lt;/strong&gt; — Aggregate Rating 1627, #722&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.15 (#438/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -62.4 (#519/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.68 (#546/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hermes 4 405B&lt;/strong&gt; — Aggregate Rating 1626, #731&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 30.08 (#199/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.23 (#307/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -33.28 (#321/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.0 Flash (001)&lt;/strong&gt; — Aggregate Rating 1625, #732&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 23.47 (#286/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -42.53 (#365/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 86.24 (#387/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon 7B&lt;/strong&gt; — Aggregate Rating 1625, #734&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 33.63 (#63/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -16.57 (#250/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.77 (#508/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4o (2024-08-06)&lt;/strong&gt; — Aggregate Rating 1624, #741&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 58.41 (#167/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -20.87 (#270/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 23.7 (#279/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EXAONE 4.5 33B&lt;/strong&gt; — Aggregate Rating 1623, #748&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 79.92 (#301/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.72 (#427/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.23 (#434/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.5V&lt;/strong&gt; — Aggregate Rating 1623, #749&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.83 (#324/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 84.78 (#365/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -46.28 (#391/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Medium 3.5&lt;/strong&gt; — Aggregate Rating 1623, #750&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 26.0 (#63/67)&lt;/li&gt;&lt;li&gt;Arena AI Code: 1262.5 (#130/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Trinity-Large-Thinking&lt;/strong&gt; — Aggregate Rating 1621, #760&lt;ul&gt;&lt;li&gt;Arena AI Code: 1236.38 (#137/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 22.5 (#304/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -44.07 (#379/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.89 (#385/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 30B A3B 2507 (Thinking)&lt;/strong&gt; — Aggregate Rating 1620, #769&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 86.54 (#392/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.3 (#432/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -56.13 (#477/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;G9v3-39A5B&lt;/strong&gt; — Aggregate Rating 1620, #770&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 13.04 (#3/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: 3.77 (#128/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.87 (#468/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 Next 80B A3B Instruct&lt;/strong&gt; — Aggregate Rating 1619, #775&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.25 (#405/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -59.5 (#500/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.75 (#521/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3 Super&lt;/strong&gt; — Aggregate Rating 1618, #779&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 83.3 (#12/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 66.7 (#20/27)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.32 (#269/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -41.5 (#357/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 86.96 (#395/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4.1 Mini&lt;/strong&gt; — Aggregate Rating 1618, #782&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.32 (#335/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -53.57 (#457/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.72 (#520/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Laguna M.1&lt;/strong&gt; — Aggregate Rating 1616, #804&lt;ul&gt;&lt;li&gt;Arena AI Code: 1349.41 (#119/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nova 2.0 Pro Preview&lt;/strong&gt; — Aggregate Rating 1616, #806&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 79.94 (#302/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 22.0 (#309/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -46.53 (#394/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 9B&lt;/strong&gt; — Aggregate Rating 1615, #811&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 83.61 (#354/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.42 (#426/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -53.47 (#455/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Pro 2&lt;/strong&gt; — Aggregate Rating 1615, #817&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.95 (#341/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -55.88 (#476/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.03 (#533/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 3&lt;/strong&gt; — Aggregate Rating 1613, #827&lt;ul&gt;&lt;li&gt;Arena AI Code: 1233.14 (#138/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.95 (#257/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -39.57 (#349/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.96 (#386/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-1T&lt;/strong&gt; — Aggregate Rating 1612, #835&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.33 (#333/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -55.13 (#472/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 94.73 (#560/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2-V2&lt;/strong&gt; — Aggregate Rating 1612, #836&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 76.04 (#276/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.53 (#370/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -47.28 (#399/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Magistral Medium 1.2&lt;/strong&gt; — Aggregate Rating 1611, #854&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 59.98 (#174/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -26.73 (#292/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.78 (#326/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Medium 3.1&lt;/strong&gt; — Aggregate Rating 1610, #860&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.75 (#327/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 84.58 (#361/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -46.28 (#390/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Seed-OSS-36B-Instruct&lt;/strong&gt; — Aggregate Rating 1610, #863&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.2 (#378/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 86.39 (#390/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -52.47 (#445/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 Omni 30B A3B&lt;/strong&gt; — Aggregate Rating 1609, #866&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 88.99 (#430/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.6 (#475/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -61.4 (#510/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Laguna XS 2.1&lt;/strong&gt; — Aggregate Rating 1609, #867&lt;ul&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 44.8 (#26/27)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4o (2024-11-20)&lt;/strong&gt; — Aggregate Rating 1608, #877&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 37.92 (#75/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -10.52 (#216/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.87 (#343/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM 4.5 Air&lt;/strong&gt; — Aggregate Rating 1608, #878&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.25 (#433/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -61.53 (#511/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.88 (#527/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercury 2.5&lt;/strong&gt; — Aggregate Rating 1607, #881&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 22.65 (#301/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.33 (#309/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -39.48 (#348/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 30B A3B Instruct&lt;/strong&gt; — Aggregate Rating 1607, #882&lt;ul&gt;&lt;li&gt;QIMMA - ArabLegalQA: 72.25 (#7/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 81.71 (#12/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ QA: 54.18 (#13/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 69.05 (#23/100)&lt;/li&gt;&lt;li&gt;QIMMA - FannOrFlop: 41.3 (#51/100)&lt;/li&gt;&lt;li&gt;QIMMA - AraDiCE-Culture: 46.67 (#60/100)&lt;/li&gt;&lt;li&gt;QIMMA - MizanQA: 49.09 (#67/100)&lt;/li&gt;&lt;li&gt;QIMMA - Overall: 45.62 (#69/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabCulture: 34.5 (#75/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM STEM: 37.68 (#79/100)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;INTELLECT-3&lt;/strong&gt; — Aggregate Rating 1607, #883&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.58 (#348/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.59 (#407/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.85 (#432/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mi:dm K 2.5 Pro&lt;/strong&gt; — Aggregate Rating 1605, #893&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.88 (#388/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -59.18 (#496/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.85 (#548/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 8B (Thinking)&lt;/strong&gt; — Aggregate Rating 1604, #901&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.43 (#332/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -52.3 (#442/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.41 (#488/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5 Nano&lt;/strong&gt; — Aggregate Rating 1604, #903&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 52.62 (#147/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -25.78 (#287/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.08 (#355/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Command A+&lt;/strong&gt; — Aggregate Rating 1603, #909&lt;ul&gt;&lt;li&gt;AI for Education Pedagogy - Technology: 81.13 (#126/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Science: 83.06 (#143/283)&lt;/li&gt;&lt;li&gt;AI for Education SEND: 73.39 (#156/275)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 76.1 (#169/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 75.86 (#175/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Primary: 77.0 (#199/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Social studies: 70.0 (#204/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 68.25 (#213/283)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama-3.3-nemotron-super-49B-v1.5&lt;/strong&gt; — Aggregate Rating 1603, #916&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 67.26 (#224/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -46.18 (#389/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.17 (#407/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3 Omni 30B A3B Instruct&lt;/strong&gt; — Aggregate Rating 1602, #917&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.33 (#480/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -69.27 (#554/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 97.59 (#593/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nova Premier&lt;/strong&gt; — Aggregate Rating 1602, #919&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 71.92 (#250/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 21.25 (#318/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -35.38 (#330/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax M1 80k&lt;/strong&gt; — Aggregate Rating 1601, #926&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 22.47 (#305/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -48.5 (#409/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.53 (#493/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 12B&lt;/strong&gt; — Aggregate Rating 1600, #942&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 73.41 (#256/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -52.72 (#448/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.62 (#448/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek R1 (Jan '25)&lt;/strong&gt; — Aggregate Rating 1600, #943&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 30.52 (#197/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -32.22 (#315/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.29 (#461/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Lightning&lt;/strong&gt; — Aggregate Rating 1600, #944&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 33.0 (#57/67)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-24B-A2B&lt;/strong&gt; — Aggregate Rating 1599, #947&lt;ul&gt;&lt;li&gt;AI for Education SEND: 72.94 (#159/275)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Science: 78.69 (#169/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 71.43 (#199/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Social studies: 70.0 (#205/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 71.97 (#208/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 71.07 (#208/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Primary: 75.12 (#213/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Technology: 67.92 (#243/283)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon 3.7B (2026-10-01)&lt;/strong&gt; — Aggregate Rating 1599, #951&lt;ul&gt;&lt;li&gt;LLM Stats (HMMT Feb 26): 75.2 (#14/16)&lt;/li&gt;&lt;li&gt;LLM Stats (Tau3 Banking): 17.7 (#15/21)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 2.1): 39.7 (#50/56)&lt;/li&gt;&lt;li&gt;ZeroEval GPQA Diamond: 68.9 (#159/265)&lt;/li&gt;&lt;li&gt;LLM Stats Score: 21.19 (#225/412)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 Coder 30B A3B Instruct&lt;/strong&gt; — Aggregate Rating 1598, #953&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.3 (#308/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.73 (#428/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.4 (#428/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercury 2&lt;/strong&gt; — Aggregate Rating 1598, #959&lt;ul&gt;&lt;li&gt;Arena AI Code: 1171.01 (#143/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 21.18 (#319/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.72 (#426/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.22 (#482/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Flash Lite&lt;/strong&gt; — Aggregate Rating 1597, #966&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 77.37 (#288/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.9 (#387/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -45.62 (#388/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 32B&lt;/strong&gt; — Aggregate Rating 1597, #968&lt;ul&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - STEM: 69.98 (#1/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Indonesian Language: 65.68 (#2/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU: 59.9 (#5/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Social Science: 66.09 (#6/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Humanities: 63.75 (#7/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Local Languages and Cultures: 33.99 (#8/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - Batak Toba: 24.78 (#12/28)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - Balinese: 38.17 (#15/28)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.12 (#333/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.43 (#402/598)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sarvam 105B&lt;/strong&gt; — Aggregate Rating 1595, #990&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.6 (#398/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -59.38 (#499/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.43 (#543/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 8B Instruct&lt;/strong&gt; — Aggregate Rating 1594, #993&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.48 (#330/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -51.88 (#440/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.01 (#478/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4&lt;/strong&gt; — Aggregate Rating 1594, #999&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 69.41 (#241/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 21.0 (#322/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -33.83 (#323/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Devstral 2&lt;/strong&gt; — Aggregate Rating 1592, #1020&lt;ul&gt;&lt;li&gt;Arena AI Code: 1195.48 (#141/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.82 (#325/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.27 (#370/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -46.7 (#397/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 Nemotron Ultra 253B v1&lt;/strong&gt; — Aggregate Rating 1590, #1043&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.23 (#321/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.07 (#339/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -44.87 (#384/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma-SEA-LION-v4-27B-IT&lt;/strong&gt; — Aggregate Rating 1590, #1046&lt;ul&gt;&lt;li&gt;GoTo-AI Leaderboard - Batak Toba: 40.87 (#4/28)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - Balinese: 56.93 (#5/28)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Local Languages and Cultures: 37.98 (#6/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Humanities: 62.11 (#8/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU: 56.72 (#10/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - STEM: 59.92 (#10/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Social Science: 63.51 (#10/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Indonesian Language: 60.09 (#10/19)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Open 100B&lt;/strong&gt; — Aggregate Rating 1586, #1074&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.53 (#369/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.65 (#442/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -54.5 (#464/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon 3.7B&lt;/strong&gt; — Aggregate Rating 1585, #1085&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 44.14 (#103/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -28.62 (#299/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 10.77 (#532/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Medium 3&lt;/strong&gt; — Aggregate Rating 1585, #1086&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 60.9 (#180/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -31.4 (#312/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.33 (#375/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron Cascade 2 30B A3B&lt;/strong&gt; — Aggregate Rating 1585, #1091&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.36 (#373/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.28 (#376/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -51.47 (#436/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ERNIE 4.5 300B A47B&lt;/strong&gt; — Aggregate Rating 1585, #1098&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 66.93 (#223/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -35.13 (#328/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.05 (#356/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 30B A3B 2507 Instruct&lt;/strong&gt; — Aggregate Rating 1584, #1099&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.97 (#466/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -66.15 (#545/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 95.39 (#571/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 72B Instruct&lt;/strong&gt; — Aggregate Rating 1584, #1102&lt;ul&gt;&lt;li&gt;DeathMath - Overall: 18.9 (#29/32)&lt;/li&gt;&lt;li&gt;DeathMath - Physics: 0.0 (#32/32)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.46 (#376/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.47 (#400/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -53.07 (#451/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;c4ai-command-a-03-2025&lt;/strong&gt; — Aggregate Rating 1584, #1111&lt;ul&gt;&lt;li&gt;QIMMA - MedArabiQ QA: 53.18 (#23/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 73.78 (#23/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 68.25 (#25/100)&lt;/li&gt;&lt;li&gt;QIMMA - FannOrFlop: 55.96 (#25/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabCulture: 39.4 (#51/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabLegalQA: 67.64 (#53/100)&lt;/li&gt;&lt;li&gt;QIMMA - AraDiCE-Culture: 47.22 (#59/100)&lt;/li&gt;&lt;li&gt;QIMMA - Overall: 48.41 (#62/100)&lt;/li&gt;&lt;li&gt;QIMMA - PALMX: 50.54 (#69/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 27.46 (#70/100)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 4 Maverick&lt;/strong&gt; — Aggregate Rating 1584, #1115&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 24.9 (#258/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -41.85 (#361/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 88.88 (#427/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 405B Instruct&lt;/strong&gt; — Aggregate Rating 1583, #1123&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 52.44 (#146/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -17.1 (#251/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 23.18 (#291/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 30B A3B&lt;/strong&gt; — Aggregate Rating 1582, #1132&lt;ul&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - STEM: 67.79 (#2/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Indonesian Language: 62.79 (#7/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU: 57.48 (#8/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Social Science: 64.91 (#8/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Humanities: 60.91 (#10/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - IndoMMLU - Local Languages and Cultures: 30.98 (#10/19)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - Balinese: 32.94 (#17/28)&lt;/li&gt;&lt;li&gt;GoTo-AI Leaderboard - Batak Toba: 21.14 (#17/28)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.73 (#313/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.18 (#435/598)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nova 2.0 Lite&lt;/strong&gt; — Aggregate Rating 1581, #1146&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.03 (#357/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.87 (#384/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -54.1 (#460/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step3 VL 10B&lt;/strong&gt; — Aggregate Rating 1580, #1156&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 83.32 (#352/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -59.0 (#494/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.27 (#500/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-OSS-20B&lt;/strong&gt; — Aggregate Rating 1580, #1159&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 86.71 (#394/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.0 (#440/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -58.55 (#491/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-2.6-flash&lt;/strong&gt; — Aggregate Rating 1579, #1174&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.55 (#450/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -66.13 (#544/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 96.72 (#587/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 4B&lt;/strong&gt; — Aggregate Rating 1578, #1188&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 86.55 (#393/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.12 (#461/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -58.35 (#490/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 2 (Nov) Instruct (2411)&lt;/strong&gt; — Aggregate Rating 1576, #1210&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 67.68 (#230/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -34.37 (#324/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.87 (#342/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Magistral Medium&lt;/strong&gt; — Aggregate Rating 1576, #1221&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 59.33 (#172/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -26.93 (#293/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.33 (#334/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 4&lt;/strong&gt; — Aggregate Rating 1575, #1234&lt;ul&gt;&lt;li&gt;QIMMA - AraTrust: 88.51 (#12/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 46.9 (#17/100)&lt;/li&gt;&lt;li&gt;QIMMA - GAT: 51.48 (#19/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedAraBench: 44.99 (#19/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM STEM: 84.9 (#20/100)&lt;/li&gt;&lt;li&gt;QIMMA - Overall: 60.19 (#22/100)&lt;/li&gt;&lt;li&gt;QIMMA - PALMX: 74.78 (#22/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabicMMLU: 67.44 (#23/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ QA: 52.84 (#27/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 66.67 (#28/100)&lt;/li&gt;&lt;li&gt;...and 10 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 14B&lt;/strong&gt; — Aggregate Rating 1574, #1255&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 75.82 (#274/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -48.95 (#415/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.28 (#455/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mi:dm K 2.5 Pro Preview&lt;/strong&gt; — Aggregate Rating 1572, #1278&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.42 (#427/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -62.73 (#524/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 94.7 (#559/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Laguna XS.2&lt;/strong&gt; — Aggregate Rating 1572, #1279&lt;ul&gt;&lt;li&gt;Arena AI Code: 1303.17 (#126/146)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.7 Flash&lt;/strong&gt; — Aggregate Rating 1571, #1284&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.15 (#437/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -62.62 (#522/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.94 (#549/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Apriel-v1.5-15B-Thinker&lt;/strong&gt; — Aggregate Rating 1571, #1288&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.28 (#371/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.45 (#425/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -54.8 (#468/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Command A&lt;/strong&gt; — Aggregate Rating 1571, #1289&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 77.32 (#287/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -48.27 (#407/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.38 (#429/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;North Mini Code&lt;/strong&gt; — Aggregate Rating 1571, #1291&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 27.0 (#62/67)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 27B (IT)&lt;/strong&gt; — Aggregate Rating 1570, #1303&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.95 (#507/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.09 (#508/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -67.22 (#548/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 4B Instruct&lt;/strong&gt; — Aggregate Rating 1570, #1309&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 10.87 (#531/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -75.97 (#573/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 97.42 (#592/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-flash-2.0&lt;/strong&gt; — Aggregate Rating 1569, #1328&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.48 (#440/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.13 (#484/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -62.7 (#523/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Motif-2-12.7B&lt;/strong&gt; — Aggregate Rating 1569, #1329&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.17 (#459/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.59 (#462/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -61.68 (#514/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 4B (Thinking)&lt;/strong&gt; — Aggregate Rating 1569, #1331&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.82 (#502/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.22 (#514/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -68.38 (#550/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniCPM5-2B&lt;/strong&gt; — Aggregate Rating 1567, #1348&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 21.91 (#14/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -11.65 (#228/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 8.42 (#560/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.3 Nemotron Super 49B (v1)&lt;/strong&gt; — Aggregate Rating 1567, #1358&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 73.69 (#258/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.1 (#409/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.3 (#421/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DiffusionGemma 26B-A4B&lt;/strong&gt; — Aggregate Rating 1567, #1359&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.58 (#422/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.89 (#471/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -59.23 (#497/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Celeris-1&lt;/strong&gt; — Aggregate Rating 1566, #1364&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.75 (#522/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 10.98 (#529/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -71.58 (#562/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Devstral Small 2&lt;/strong&gt; — Aggregate Rating 1566, #1365&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.29 (#402/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.8 (#442/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -57.7 (#484/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.3 70B Instruct&lt;/strong&gt; — Aggregate Rating 1565, #1375&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.95 (#359/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.21 (#459/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -54.17 (#461/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ring-flash-2.0&lt;/strong&gt; — Aggregate Rating 1565, #1378&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.55 (#424/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.07 (#456/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -58.62 (#492/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HyperNova 60B 2605&lt;/strong&gt; — Aggregate Rating 1564, #1386&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.33 (#337/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.22 (#457/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -54.58 (#466/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Magistral Small 1.2&lt;/strong&gt; — Aggregate Rating 1563, #1409&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.76 (#466/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.45 (#497/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -65.1 (#540/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 4B 2507 (Thinking)&lt;/strong&gt; — Aggregate Rating 1563, #1413&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 83.23 (#350/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -60.27 (#506/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.53 (#511/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3 Nano Omni 30B A3B&lt;/strong&gt; — Aggregate Rating 1562, #1424&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.65 (#377/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.2 (#458/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -57.43 (#481/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;jamba-large-1.7&lt;/strong&gt; — Aggregate Rating 1561, #1435&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 20.58 (#329/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -54.92 (#469/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 95.07 (#567/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EXAONE 4.0 32B&lt;/strong&gt; — Aggregate Rating 1561, #1437&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.1 (#332/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.87 (#489/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -60.07 (#503/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 Nemotron 70B Instruct&lt;/strong&gt; — Aggregate Rating 1559, #1456&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 71.26 (#248/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -40.83 (#354/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.77 (#391/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HyperCLOVA X SEED Think (32B)&lt;/strong&gt; — Aggregate Rating 1559, #1465&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.19 (#319/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -53.28 (#453/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.4 (#454/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OLMo-3.1 32B Think&lt;/strong&gt; — Aggregate Rating 1558, #1466&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 66.29 (#217/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -43.37 (#373/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.78 (#490/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3 (Dec '24)&lt;/strong&gt; — Aggregate Rating 1558, #1484&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 25.45 (#251/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -41.65 (#359/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.01 (#455/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;JT-MINI&lt;/strong&gt; — Aggregate Rating 1557, #1499&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.22 (#482/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.3 (#485/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -64.1 (#533/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hermes 4 70B&lt;/strong&gt; — Aggregate Rating 1556, #1503&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 23.55 (#283/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.68 (#344/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -48.58 (#413/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Apriel-v1.6-15B-Thinker&lt;/strong&gt; — Aggregate Rating 1556, #1513&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.97 (#411/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -59.37 (#498/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.93 (#504/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Molmo2-8B&lt;/strong&gt; — Aggregate Rating 1556, #1515&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.1 (#479/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 11.45 (#522/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -69.22 (#553/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 70B Instruct&lt;/strong&gt; — Aggregate Rating 1553, #1557&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 78.21 (#295/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.7 (#346/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -43.1 (#372/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude 3.5 Haiku&lt;/strong&gt; — Aggregate Rating 1553, #1580&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 41.15 (#88/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -22.52 (#279/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.2 (#502/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nova Pro&lt;/strong&gt; — Aggregate Rating 1552, #1583&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 77.71 (#291/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -47.73 (#403/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 16.87 (#415/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Devstral Medium&lt;/strong&gt; — Aggregate Rating 1551, #1599&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 62.19 (#188/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -31.57 (#313/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.88 (#361/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Think V2&lt;/strong&gt; — Aggregate Rating 1551, #1609&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 71.17 (#247/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -40.33 (#352/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.02 (#384/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 3.2&lt;/strong&gt; — Aggregate Rating 1550, #1619&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 76.81 (#282/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.58 (#424/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.83 (#470/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.2 8B&lt;/strong&gt; — Aggregate Rating 1550, #1627&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 22.0 (#65/67)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LongCat Flash Lite&lt;/strong&gt; — Aggregate Rating 1548, #1648&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.3 (#499/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -70.07 (#557/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 96.16 (#582/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Falcon-H1R-7B&lt;/strong&gt; — Aggregate Rating 1547, #1657&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 88.77 (#425/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.55 (#476/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -61.3 (#509/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek R1 Distill Llama 70B&lt;/strong&gt; — Aggregate Rating 1544, #1681&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.09 (#317/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 19.17 (#351/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -46.38 (#392/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Reka Flash 3&lt;/strong&gt; — Aggregate Rating 1544, #1685&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.27 (#484/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.67 (#492/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -65.13 (#541/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3 70B Instruct&lt;/strong&gt; — Aggregate Rating 1544, #1691&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.72 (#392/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.54 (#405/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -54.32 (#462/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 3.1&lt;/strong&gt; — Aggregate Rating 1542, #1718&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 77.56 (#289/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.75 (#429/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.1 (#463/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sarvam 30B&lt;/strong&gt; — Aggregate Rating 1537, #1779&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.62 (#509/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -71.52 (#561/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 96.28 (#585/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 4 Scout&lt;/strong&gt; — Aggregate Rating 1534, #1811&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 79.35 (#300/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -52.15 (#441/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.17 (#460/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.2 3B&lt;/strong&gt; — Aggregate Rating 1534, #1814&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 27.0 (#60/67)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 8B&lt;/strong&gt; — Aggregate Rating 1530, #1860&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.74 (#465/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.63 (#494/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -64.73 (#537/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 12B (IT)&lt;/strong&gt; — Aggregate Rating 1529, #1867&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 10.3 (#536/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -77.07 (#576/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 97.4 (#591/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ministral 3 14B&lt;/strong&gt; — Aggregate Rating 1529, #1872&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.58 (#495/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.52 (#517/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -66.37 (#546/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Devstral Small (2507)&lt;/strong&gt; — Aggregate Rating 1529, #1875&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 76.9 (#283/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -50.83 (#431/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.73 (#472/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nemotron-3-nano-30B-a3B&lt;/strong&gt; — Aggregate Rating 1527, #1898&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 83.27 (#351/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.3 (#404/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -51.57 (#438/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OLMo 3 32B&lt;/strong&gt; — Aggregate Rating 1526, #1907&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.67 (#379/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.87 (#469/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -58.07 (#487/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.1 30B&lt;/strong&gt; — Aggregate Rating 1525, #1916&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.95 (#488/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -67.78 (#549/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 94.98 (#565/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Devstral Small (2505)&lt;/strong&gt; — Aggregate Rating 1524, #1923&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 86.35 (#389/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.87 (#441/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -56.78 (#478/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GaMS3-12B-Instruct&lt;/strong&gt; — Aggregate Rating 1523, #1932&lt;ul&gt;&lt;li&gt;Open PL LLM - CBD (multiple choice, 5-shot): 41.54 (#9/319)&lt;/li&gt;&lt;li&gt;Open PL LLM - KLEJ NER (generative, 5-shot): 59.28 (#16/314)&lt;/li&gt;&lt;li&gt;Open PL LLM - 8TAGS (generative, 5-shot): 78.8 (#23/314)&lt;/li&gt;&lt;li&gt;Open PL LLM - Belebele (generative, 5-shot): 90.0 (#36/314)&lt;/li&gt;&lt;li&gt;Open PL LLM - Belebele (multiple choice, 5-shot): 89.33 (#39/316)&lt;/li&gt;&lt;li&gt;Open PL LLM - CBD (generative, 5-shot): 35.88 (#50/314)&lt;/li&gt;&lt;li&gt;Open PL LLM - PoQuAD Open Book (generative, 5-shot): 69.22 (#62/301)&lt;/li&gt;&lt;li&gt;Open PL LLM - 8TAGS (multiple choice, 5-shot): 78.09 (#65/315)&lt;/li&gt;&lt;li&gt;Open PL LLM - PolQA Closed Book (generative, 5-shot): 72.27 (#68/314)&lt;/li&gt;&lt;li&gt;Open PL LLM - PSC (generative, 5-shot): 96.36 (#74/309)&lt;/li&gt;&lt;li&gt;...and 13 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Phi-4&lt;/strong&gt; — Aggregate Rating 1523, #1933&lt;ul&gt;&lt;li&gt;QIMMA - MedArabiQ QA: 54.47 (#9/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabLegalQA: 71.59 (#15/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedAraBench: 42.31 (#26/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM HumanEval+: 73.17 (#26/100)&lt;/li&gt;&lt;li&gt;QIMMA - AraTrust: 84.1 (#31/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 39.19 (#35/100)&lt;/li&gt;&lt;li&gt;QIMMA - Overall: 56.64 (#38/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 61.11 (#39/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM STEM: 75.12 (#41/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabicMMLU: 61.58 (#43/100)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.0 H Small&lt;/strong&gt; — Aggregate Rating 1522, #1950&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.85 (#411/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.28 (#481/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -61.02 (#507/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sarvam M&lt;/strong&gt; — Aggregate Rating 1520, #1983&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 15.23 (#456/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.64 (#464/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -61.6 (#513/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM3-1.7B&lt;/strong&gt; — Aggregate Rating 1519, #2003&lt;ul&gt;&lt;li&gt;The Aggregate - Day-one Recall Check: 3.1 (#42/43)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 96.0 (#113/419)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Science: 65.57 (#229/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Maths: 64.29 (#230/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Secondary: 62.42 (#236/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy: 62.74 (#237/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Social studies: 59.09 (#242/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Primary: 63.85 (#244/283)&lt;/li&gt;&lt;li&gt;AI for Education Pedagogy - Technology: 66.98 (#246/283)&lt;/li&gt;&lt;li&gt;AI for Education SEND: 59.17 (#249/275)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4.1 Nano&lt;/strong&gt; — Aggregate Rating 1517, #2023&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.56 (#341/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -57.58 (#482/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 13.68 (#491/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ministral 3 8B&lt;/strong&gt; — Aggregate Rating 1513, #2098&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.98 (#506/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -68.95 (#552/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 94.16 (#553/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek R1 0528 Qwen3 8B&lt;/strong&gt; — Aggregate Rating 1507, #2178&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 86.35 (#388/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.2 (#515/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -63.62 (#530/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 4B 2507 Instruct&lt;/strong&gt; — Aggregate Rating 1503, #2217&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.74 (#328/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -61.93 (#515/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 10.9 (#530/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nova Lite&lt;/strong&gt; — Aggregate Rating 1500, #2282&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 57.61 (#164/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -42.17 (#363/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 9.8 (#542/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron Nano 12B v2 VL&lt;/strong&gt; — Aggregate Rating 1499, #2287&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 14.62 (#474/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.35 (#513/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -64.23 (#536/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NVIDIA Nemotron Nano 9B V2&lt;/strong&gt; — Aggregate Rating 1497, #2329&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 60.49 (#176/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -41.55 (#358/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 11.8 (#520/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Magistral Small&lt;/strong&gt; — Aggregate Rating 1496, #2339&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 76.13 (#277/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -44.4 (#381/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.02 (#383/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 E4B&lt;/strong&gt; — Aggregate Rating 1496, #2341&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 30.94 (#53/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -19.7 (#266/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 8.58 (#557/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Tri-21B-Think&lt;/strong&gt; — Aggregate Rating 1495, #2366&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.42 (#374/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.33 (#512/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -62.55 (#520/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 1.5 32B Chat&lt;/strong&gt; — Aggregate Rating 1492, #2411&lt;ul&gt;&lt;li&gt;OpenEval - BBQ: 90.7 (#2/10)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Jais-3-0.9B&lt;/strong&gt; — Aggregate Rating 1490, #2435&lt;ul&gt;&lt;li&gt;Arabic Broad Leaderboard - Translation (incl Dialects): 7.389 (#47/127)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Writing (incl Dialects): 7.409 (#62/127)&lt;/li&gt;&lt;li&gt;QIMMA - GAT: 34.78 (#63/100)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard - Trust &amp; Safety: 7.667 (#82/127)&lt;/li&gt;&lt;li&gt;QIMMA - FannOrFlop: 0.53 (#82/100)&lt;/li&gt;&lt;li&gt;Arabic Broad Leaderboard: 7.326 (#83/127)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 24.45 (#83/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedAraBench: 24.63 (#84/100)&lt;/li&gt;&lt;li&gt;QIMMA - AraDiCE-Culture: 36.67 (#86/100)&lt;/li&gt;&lt;li&gt;QIMMA - MizanQA: 43.91 (#89/100)&lt;/li&gt;&lt;li&gt;...and 14 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OLMo 3.1 32B Instruct&lt;/strong&gt; — Aggregate Rating 1490, #2445&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 73.24 (#253/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -52.37 (#443/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.05 (#516/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-2.6B&lt;/strong&gt; — Aggregate Rating 1487, #2479&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 16.0 (#6/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -10.95 (#225/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 4.35 (#591/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.1 8B&lt;/strong&gt; — Aggregate Rating 1486, #2493&lt;ul&gt;&lt;li&gt;Arena AI Code: 1192.59 (#142/146)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 87.13 (#401/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 12.3 (#513/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -64.12 (#534/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test1 (vicgalleorg)&lt;/strong&gt; — Aggregate Rating 1476, #2694&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 89.52 (#46/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 78.32 (#78/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 72.27 (#631/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 83.9 (#658/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 66.67 (#882/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 64.67 (#1546/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude 3 Haiku&lt;/strong&gt; — Aggregate Rating 1475, #2726&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.45 (#310/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 17.63 (#397/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -48.63 (#414/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-8B-A1B&lt;/strong&gt; — Aggregate Rating 1473, #2772&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 46.91 (#118/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -33.2 (#320/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 9.33 (#547/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Apertus-70B-Instruct-2509&lt;/strong&gt; — Aggregate Rating 1472, #2794&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.82 (#347/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -61.55 (#512/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 11.63 (#521/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nova Micro&lt;/strong&gt; — Aggregate Rating 1470, #2902&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 65.08 (#207/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -48.52 (#411/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 10.03 (#539/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 1.5 14B Chat&lt;/strong&gt; — Aggregate Rating 1465, #3109&lt;ul&gt;&lt;li&gt;OpenEval - BBQ: 93.5 (#1/10)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ministral 3 3B&lt;/strong&gt; — Aggregate Rating 1458, #3430&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.19 (#306/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -64.0 (#531/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 8.98 (#552/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.1 3B&lt;/strong&gt; — Aggregate Rating 1455, #3571&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 9.25 (#549/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 95.08 (#568/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -77.03 (#575/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniCPM-V-4.6&lt;/strong&gt; — Aggregate Rating 1450, #3763&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 6.73 (#576/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 96.07 (#579/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -82.87 (#591/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.0 Micro&lt;/strong&gt; — Aggregate Rating 1448, #3825&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 9.32 (#548/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 95.77 (#577/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -77.53 (#580/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-mini-2.0&lt;/strong&gt; — Aggregate Rating 1447, #3879&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 8.88 (#554/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 95.87 (#578/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -78.47 (#582/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 4B (IT)&lt;/strong&gt; — Aggregate Rating 1445, #3999&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 7.73 (#566/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -82.88 (#592/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 98.21 (#596/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OLMo 3 7B&lt;/strong&gt; — Aggregate Rating 1444, #4069&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 11.08 (#527/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.21 (#535/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -71.8 (#563/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nanbeige4.1-3B&lt;/strong&gt; — Aggregate Rating 1444, #4080&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 58.1 (#165/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -42.5 (#364/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 9.87 (#540/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Apertus-8B-Instruct-2509&lt;/strong&gt; — Aggregate Rating 1443, #4103&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 9.65 (#544/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -77.22 (#578/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 96.14 (#581/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-VL-1.6B&lt;/strong&gt; — Aggregate Rating 1441, #4170&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 95.74 (#576/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 5.78 (#584/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -84.42 (#596/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Tri-21B-Think Preview&lt;/strong&gt; — Aggregate Rating 1441, #4179&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 80.53 (#312/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -62.3 (#517/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 10.1 (#537/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 E2B&lt;/strong&gt; — Aggregate Rating 1436, #4473&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 32.36 (#57/598)&lt;/li&gt;&lt;li&gt;BenchLM: 23.8 (#200/200)&lt;/li&gt;&lt;li&gt;AA Omniscience: -23.58 (#283/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 8.15 (#563/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 8B Instruct&lt;/strong&gt; — Aggregate Rating 1435, #4505&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 43.04 (#95/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -30.88 (#310/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 8.5 (#559/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Zarya-4B&lt;/strong&gt; — Aggregate Rating 1435, #4525&lt;ul&gt;&lt;li&gt;MERA v2 - SAGE: 48.6 (#35/50)&lt;/li&gt;&lt;li&gt;MERA v2 - RussianRegions: 3.6 (#40/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Culture Specific: 16.6 (#40/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Characters: 27.2 (#42/50)&lt;/li&gt;&lt;li&gt;MERA v2 - RUBIN: 26.0 (#44/50)&lt;/li&gt;&lt;li&gt;MERA v2 - GorillaHard: 4.8 (#44/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Agentic: 2.5 (#44/50)&lt;/li&gt;&lt;li&gt;MERA v2: 0.079 (#45/50)&lt;/li&gt;&lt;li&gt;MERA v2 - LIMUR: 30.4 (#45/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Riddles: 5.0 (#45/50)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Jamba Reasoning 3B&lt;/strong&gt; — Aggregate Rating 1432, #4641&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 69.04 (#239/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -57.72 (#485/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 6.7 (#577/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3 8B Instruct&lt;/strong&gt; — Aggregate Rating 1430, #4796&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 89.79 (#447/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 10.37 (#535/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -70.12 (#558/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Jamba 1.7 Mini&lt;/strong&gt; — Aggregate Rating 1424, #5062&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 11.27 (#524/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -73.12 (#569/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 95.1 (#569/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 2B&lt;/strong&gt; — Aggregate Rating 1424, #5082&lt;ul&gt;&lt;li&gt;MERA v2 - SAGE: 54.3 (#32/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Culture Specific: 16.3 (#41/50)&lt;/li&gt;&lt;li&gt;MERA v2 - RUBIN: 26.4 (#43/50)&lt;/li&gt;&lt;li&gt;MERA v2 - IFHardBench: 3.5 (#43/50)&lt;/li&gt;&lt;li&gt;MERA v2 - GorillaHard: 4.2 (#45/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Agentic: 2.5 (#45/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Characters: 21.0 (#46/50)&lt;/li&gt;&lt;li&gt;MERA v2 - NewReasoning: 10.8 (#46/50)&lt;/li&gt;&lt;li&gt;MERA v2: 0.062 (#47/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Riddles: 2.4 (#47/50)&lt;/li&gt;&lt;li&gt;...and 10 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen-14B-Chat&lt;/strong&gt; — Aggregate Rating 1423, #5108&lt;ul&gt;&lt;li&gt;OpenEval - BBQ: 82.5 (#3/10)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.2 11B Instruct&lt;/strong&gt; — Aggregate Rating 1420, #5223&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.88 (#329/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -62.6 (#521/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 10.6 (#534/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 3.3 8B&lt;/strong&gt; — Aggregate Rating 1417, #5336&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 9.65 (#543/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -77.28 (#579/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 96.22 (#584/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test (UCLA-AGI)&lt;/strong&gt; — Aggregate Rating 1410, #5661&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 85.44 (#1853/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 65.87 (#2072/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 57.39 (#2158/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 60.95 (#3606/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 76.64 (#3653/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 30.86 (#3781/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 1.7B&lt;/strong&gt; — Aggregate Rating 1409, #5780&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 8.87 (#555/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 94.81 (#562/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -77.53 (#581/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test0 (UCLA-AGI)&lt;/strong&gt; — Aggregate Rating 1407, #5849&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 84.44 (#2453/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 63.65 (#2626/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 77.98 (#2905/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 36.69 (#3477/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 61.01 (#3586/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 50.48 (#3692/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test (jikaixuan) [Adapter]&lt;/strong&gt; — Aggregate Rating 1405, #5958&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 57.51 (#2132/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 84.42 (#2469/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 78.06 (#2881/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 62.29 (#3116/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 61.07 (#3571/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 27.22 (#3979/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test_model (jikaixuan) [Adapter]&lt;/strong&gt; — Aggregate Rating 1405, #5959&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 57.51 (#2133/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 84.42 (#2470/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 78.06 (#2882/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 62.29 (#3117/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 61.07 (#3572/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 27.22 (#3980/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon 0.9B&lt;/strong&gt; — Aggregate Rating 1404, #6001&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.69 (#380/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -72.82 (#568/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 6.93 (#572/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2 24B A2B&lt;/strong&gt; — Aggregate Rating 1402, #6062&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 69.03 (#238/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -58.1 (#488/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 6.47 (#580/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 1.5 7B Chat&lt;/strong&gt; — Aggregate Rating 1393, #6390&lt;ul&gt;&lt;li&gt;OpenEval - BBQ: 76.6 (#4/10)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3n E4B Instruct&lt;/strong&gt; — Aggregate Rating 1392, #6409&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 7.98 (#565/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -81.33 (#588/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 97.07 (#589/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Phi-4 Mini Instruct&lt;/strong&gt; — Aggregate Rating 1390, #6476&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 77.99 (#292/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -61.08 (#508/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 9.5 (#546/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 1.1 7B (IT) [float16]&lt;/strong&gt; — Aggregate Rating 1389, #6508&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 42.99 (#3089/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 60.92 (#3617/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 50.74 (#3621/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 60.07 (#3907/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 76.14 (#5210/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 69.69 (#5526/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 1.1 7B (IT) [bfloat16]&lt;/strong&gt; — Aggregate Rating 1386, #6600&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 41.77 (#3142/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 50.4 (#3711/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 60.39 (#3768/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 59.98 (#3923/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 76.21 (#5204/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 69.93 (#5499/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Tiny Aya Global&lt;/strong&gt; — Aggregate Rating 1386, #6615&lt;ul&gt;&lt;li&gt;AA-Omniscience Accuracy: 6.07 (#583/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 96.2 (#583/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -84.3 (#595/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Exaone 4.0 1.2B&lt;/strong&gt; — Aggregate Rating 1384, #6712&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.74 (#499/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 6.83 (#575/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -79.82 (#584/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2 8B A1B&lt;/strong&gt; — Aggregate Rating 1377, #6886&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 92.3 (#511/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 8.22 (#562/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -76.5 (#574/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OLMo 3 7B Instruct&lt;/strong&gt; — Aggregate Rating 1375, #6933&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 91.22 (#483/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 7.4 (#568/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -77.07 (#577/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;platypus-2-22B-relora [float16]&lt;/strong&gt; — Aggregate Rating 1359, #7351&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 77.35 (#3255/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 82.44 (#3535/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 57.68 (#4491/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 55.33 (#4643/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 6.6 (#5381/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 43.61 (#5434/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;platypus-2-22B-relora [bfloat16]&lt;/strong&gt; — Aggregate Rating 1357, #7423&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 77.11 (#3378/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 82.36 (#3574/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 57.51 (#4523/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 54.94 (#4715/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 6.29 (#5407/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 43.62 (#5432/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;gemma-3n-E2B-it Instruct&lt;/strong&gt; — Aggregate Rating 1356, #7445&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.38 (#540/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 6.83 (#574/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -80.17 (#586/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral 7B Instruct&lt;/strong&gt; — Aggregate Rating 1355, #7473&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.26 (#322/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -64.92 (#538/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 9.02 (#551/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2 2.6B&lt;/strong&gt; — Aggregate Rating 1353, #7526&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 62.89 (#194/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -54.07 (#459/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 5.42 (#585/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.0 1B&lt;/strong&gt; — Aggregate Rating 1348, #7661&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 93.52 (#544/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 6.15 (#582/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -81.62 (#589/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.0 H 1B&lt;/strong&gt; — Aggregate Rating 1343, #7762&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.72 (#327/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -72.27 (#566/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 5.2 (#586/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2 1.5B Instruct&lt;/strong&gt; — Aggregate Rating 1342, #7790&lt;ul&gt;&lt;li&gt;OpenEval - BBQ: 37.34 (#8/10)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 1.5 4B Chat&lt;/strong&gt; — Aggregate Rating 1331, #7968&lt;ul&gt;&lt;li&gt;OpenEval - BBQ: 62.86 (#5/10)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 0.8B&lt;/strong&gt; — Aggregate Rating 1329, #8009&lt;ul&gt;&lt;li&gt;MERA v2 - SAGE: 47.9 (#37/50)&lt;/li&gt;&lt;li&gt;MERA v2 - RussianRegions: 3.6 (#41/50)&lt;/li&gt;&lt;li&gt;MERA v2 - NewReasoning: 14.9 (#42/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Culture Specific: 15.3 (#43/50)&lt;/li&gt;&lt;li&gt;MERA v2 - IFHardBench: 2.7 (#46/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Reasoning: 12.3 (#46/50)&lt;/li&gt;&lt;li&gt;MERA v2 - Enantiosemy: 5.7 (#47/50)&lt;/li&gt;&lt;li&gt;MERA v2 - GorillaHard: 1.5 (#47/50)&lt;/li&gt;&lt;li&gt;MERA v2: 0.055 (#48/50)&lt;/li&gt;&lt;li&gt;MERA v2 - SOB-Hard: 1.0 (#48/50)&lt;/li&gt;&lt;li&gt;...and 10 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 0.6B&lt;/strong&gt; — Aggregate Rating 1314, #8322&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 90.94 (#473/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 6.33 (#581/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -78.85 (#583/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test_llama2.7B (Publisher: yeen214)&lt;/strong&gt; — Aggregate Rating 1310, #8387&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 78.57 (#4753/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 74.03 (#4757/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 53.07 (#5193/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 7.13 (#5339/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 46.86 (#5387/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 38.75 (#6525/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;testmodel2 (TinyPixel)&lt;/strong&gt; — Aggregate Rating 1293, #8615&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 78.78 (#4663/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 73.8 (#4820/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 53.24 (#5141/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 7.66 (#5275/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 46.61 (#5425/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 39.17 (#6423/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.0 350M&lt;/strong&gt; — Aggregate Rating 1293, #8616&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 76.13 (#278/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -69.35 (#556/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 3.85 (#593/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test-llama2-7B (Publisher: bongchoi)&lt;/strong&gt; — Aggregate Rating 1293, #8618&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 74.03 (#4743/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 78.57 (#4749/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 53.07 (#5180/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 7.13 (#5333/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 46.86 (#5386/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 38.75 (#6522/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 1B (IT)&lt;/strong&gt; — Aggregate Rating 1292, #8627&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 82.29 (#335/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -75.45 (#572/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 3.75 (#595/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-1.2B Instruct&lt;/strong&gt; — Aggregate Rating 1292, #8638&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 85.09 (#368/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -72.1 (#564/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 7.02 (#570/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2 1.2B&lt;/strong&gt; — Aggregate Rating 1289, #8674&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.38 (#323/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -72.43 (#567/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 4.93 (#589/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-1.2B&lt;/strong&gt; — Aggregate Rating 1273, #8811&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 94.37 (#555/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 7.42 (#567/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -79.95 (#585/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniCPM5-1B&lt;/strong&gt; — Aggregate Rating 1270, #8851&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 0.9 (#1/598)&lt;/li&gt;&lt;li&gt;QIMMA - FannOrFlop: 8.85 (#74/100)&lt;/li&gt;&lt;li&gt;QIMMA - PALMX: 39.49 (#79/100)&lt;/li&gt;&lt;li&gt;QIMMA - ArabCulture: 34.43 (#80/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM STEM: 37.41 (#80/100)&lt;/li&gt;&lt;li&gt;QIMMA - AraTrust: 36.21 (#85/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedAraBench: 24.43 (#87/100)&lt;/li&gt;&lt;li&gt;QIMMA - 3LM MBPP+: 18.25 (#92/100)&lt;/li&gt;&lt;li&gt;QIMMA - MedArabiQ MCQ: 22.77 (#94/100)&lt;/li&gt;&lt;li&gt;QIMMA - MizanQA: 43.02 (#96/100)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 1.5 1.8B Chat&lt;/strong&gt; — Aggregate Rating 1263, #8903&lt;ul&gt;&lt;li&gt;OpenEval - BBQ: 33.93 (#9/10)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.2 1B Instruct&lt;/strong&gt; — Aggregate Rating 1260, #8926&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 66.34 (#218/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -54.7 (#467/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 7.0 (#571/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.0 H 350M&lt;/strong&gt; — Aggregate Rating 1255, #8961&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 88.06 (#416/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -80.92 (#587/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 3.8 (#594/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 270M&lt;/strong&gt; — Aggregate Rating 1214, #9256&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 30.42 (#49/598)&lt;/li&gt;&lt;li&gt;AA Omniscience: -29.27 (#301/598)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 0.88 (#597/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 1.5 0.5B Chat&lt;/strong&gt; — Aggregate Rating 1194, #9358&lt;ul&gt;&lt;li&gt;OpenEval - BBQ: 31.23 (#10/10)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SmolLM-135M&lt;/strong&gt; — Aggregate Rating 1159, #9621&lt;ul&gt;&lt;li&gt;Supra-microBench - Lexicon and Semantics: 84.51 (#3/39)&lt;/li&gt;&lt;li&gt;Supra-microBench - World Knowledge: 78.29 (#3/39)&lt;/li&gt;&lt;li&gt;Supra-microBench - Coherence and Common Sense: 45.73 (#4/39)&lt;/li&gt;&lt;li&gt;Supra-microBench - Index: 54.86 (#7/39)&lt;/li&gt;&lt;li&gt;Supra-microBench - Context Use and Reading: 40.0 (#7/39)&lt;/li&gt;&lt;li&gt;Supra-microBench - Elementary Reasoning: 37.99 (#10/39)&lt;/li&gt;&lt;li&gt;Supra-microBench - Linguistic Competence: 53.19 (#16/39)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test5 (doas)&lt;/strong&gt; — Aggregate Rating 1136, #10019&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 47.34 (#4483/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 25.36 (#6692/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 28.41 (#6696/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 52.64 (#6743/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 0.0 (#6990/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 26.63 (#6996/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test2 (doas)&lt;/strong&gt; — Aggregate Rating 1132, #10107&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 48.49 (#4199/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 29.61 (#6605/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 0.0 (#6989/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 24.34 (#6992/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 26.65 (#6993/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 50.12 (#7044/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;testmodel (huashiyiqike)&lt;/strong&gt; — Aggregate Rating 1129, #10152&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 43.72 (#5406/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 25.28 (#6720/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 26.68 (#6989/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 0.0 (#7027/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 50.2 (#7033/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 19.71 (#7252/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-2-SlimOrcaDeduped-airoboros-3.1-MetaMathQA-SFT-124M&lt;/strong&gt; — Aggregate Rating 1126, #10186&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 2.12 (#5850/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 38.84 (#6498/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 25.82 (#6527/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 29.43 (#6872/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 24.57 (#6977/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 49.01 (#7194/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-2&lt;/strong&gt; — Aggregate Rating 1124, #10211&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 40.69 (#6092/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 0.68 (#6279/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 25.83 (#6522/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 31.53 (#6791/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 50.43 (#6996/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 22.01 (#7194/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test1 (marcchew) [8bit]&lt;/strong&gt; — Aggregate Rating 1111, #10290&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 48.33 (#4245/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 27.65 (#6762/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 24.55 (#6944/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 50.2 (#7036/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 26.17 (#7064/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 0.0 (#7101/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;pythia-6.9B&lt;/strong&gt; — Aggregate Rating 1096, #10373&lt;ul&gt;&lt;li&gt;OpenEval - BoolQ: 70.31 (#2/2)&lt;/li&gt;&lt;li&gt;OpenEval - IMDb: 93.18 (#2/2)&lt;/li&gt;&lt;li&gt;OpenEval - CNN/DailyMail: 21.59 (#6/7)&lt;/li&gt;&lt;li&gt;OpenEval - BBQ: 39.0 (#7/10)&lt;/li&gt;&lt;li&gt;OpenEval - XSum: 18.17 (#7/7)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (671)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA CritPt: 35.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA CritPt v2: 86.29 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA CritPt v2 - Hard Subset: 76.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA FrontierCode 2.0: 49.13 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf: 26.8 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Construction - Criterion Pass Rate: 79.15 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Criterion Pass Rate: 81.13 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Engineering - Criterion Pass Rate: 86.26 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 92.36 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - HR - Criterion Pass Rate: 78.81 (#46)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Healthcare - Criterion Pass Rate: 90.57 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Insurance - Criterion Pass Rate: 86.39 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Legal - Criterion Pass Rate: 83.67 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 78.57 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Real Estate - Criterion Pass Rate: 71.24 (#53)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - STEM/Research - Criterion Pass Rate: 80.23 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDPval: 1831.06 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA HLE-Diamond: 80.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA HLE-Diamond - Knowledge: 74.8 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA HLE-Diamond - Reasoning: 85.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Humanity's Last Exam: 74.28 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Long Context Reasoning: 83.33 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Omniscience: 57.87 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA SciCode: 67.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench 4.0: 61.62 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1: 73.33 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Earth Sciences: 70.83 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Engineering Sciences: 70.37 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Life Sciences: 68.42 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Mathematical Sciences: 86.27 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Physical Sciences: 70.59 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-3D: 1271.4151 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase: 66.08 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - Analytical Quality Elo: 2012.61 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - DOCX - Rubric Pass Rate: 60.36 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - PDF - Rubric Pass Rate: 62.93 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - PPTX - Rubric Pass Rate: 64.08 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - Presentation Elo: 1706.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - XLSX - Rubric Pass Rate: 69.52 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Accuracy: 74.43 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Hallucination Rate: 63.04 (#196)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Business: 57.11 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Health: 60.22 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Humanities &amp; Social Sciences: 68.44 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Law: 57.11 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Science, Engineering &amp; Mathematics: 59.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE): 92.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - C: 92.73 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Dart: 94.44 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Go: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - HTML: 91.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Java: 90.91 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Julia: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 74.29 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - PHP: 91.18 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Python: 97.87 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - R: 70.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Rust: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Swift: 88.46 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Chess Leaderboard (Continuation): 1686.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1685.0 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Coding Daily (Claude Code) - Laravel Code Quality: 19.13 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Coding Daily (Claude Code) - React-TS Code Quality: 19.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Coding Daily (Claude Code) - Total: 84.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AIMultiple - FinanceReasoning (Hard): 91.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AIMultiple - Text-to-SQL (Adjudicated): 90.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AIMultiple - Text-to-SQL (Strict Execution Match): 71.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ARC-AGI-1: 98.5 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ARC-AGI-2: 93.75 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ARC-AGI-3: 81.17 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard: 9.111 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - General Knowledge: 9.048 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - MMLU: 9.752 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - RAG QA: 8.0 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - Reasoning &amp; Math: 9.767 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - Translation (incl Dialects): 7.917 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - Trust &amp; Safety: 9.333 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - Writing (incl Dialects): 7.591 (#47)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Architectural RedlineBench: 68.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Argo-Bench: 70.22 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Argo-Bench - Solved: 41.43 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Artificial Analysis Intelligence Index: 64.56 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench (Default Fallbacks): 51.45 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA: 76.42 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Finance - Raw Objective Completion: 90.38 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - HR - Raw Objective Completion: 81.24 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Marketing - Raw Objective Completion: 90.81 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Operations - Raw Objective Completion: 97.15 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Sales - Raw Objective Completion: 94.37 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Strict: 51.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Support - Raw Objective Completion: 93.76 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on BenchGecko Score: 97.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Blueprint-Bench 2: 0.566 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Bug Hunt Bench: 58.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Bug Hunt Bench - LMS: 35.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 23.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Chess Bench LLM: 1867.0 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Chess Puzzles (Epoch AI): 79.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ChessBench: 31.09 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ChessBench - Coherence: 79.4 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ChessBench - Elo: 1635.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Coarena: 1125.3 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Coarena - Task Completion: 84.7 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Conceptual Reasoning Index: 79.89 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Conceptual Reasoning Index - Argument Evaluation (LMCA): 70.61 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Conceptual Reasoning Index - Consistency (ACCoRD): 89.39 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Creative Writing (Lechmazur): 3.687 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on CursorBench 4.0: 60.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Deep20Bench - Edition 1.1: 14.566666666667 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Design Arena (3D): 1496.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Design Arena (Game Dev): 1399.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Design Arena (UI Components): 1378.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Design Arena (Website): 1350.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - All - Completion: 7.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Cartridge Swap - Completion: 15.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Cartridge Swap - Progress: 70.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Execution: 4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Handling: 2.5 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Honesty: 3 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Overall: 3.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Perception: 3.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Instrument Swap - Completion: 10.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Parts Kitting - Completion: 5.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Parts Kitting - Progress: 63.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Sample Transfer - Completion: 0.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Sample Transfer - Progress: 38.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on DuelLab GameBench 2 - Reasoning Variants: 99.226094853495 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EKI Estonian LLM - Language Advice: 74.38 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EKI Estonian LLM - Propaganda Resistance: 94.3 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EKI Estonian LLM - Trivia: 80.5 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EKI Estonian LLM - Trivia Hard: 39.12 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EnigmaForge: 68.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EnigmaForge - Fact F1: 97.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EnigmaForge - Intuition: 56.2 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Critpt: 35.71 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Cursorbench: 60.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Dtbench: 98.93 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Ebr Bench: 90.48 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Frontiermath Tier 4 V2: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Frontiermath Tiers 1 3 V2: 95.09 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Frontierswe: 69.94 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Furniture Assembly: 90.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - GPQA Diamond: 96.02 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Lmca: 70.61 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Mystery Game Puzzles: 90.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Proofbench: 100.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Scicode: 67.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Spireclimb: 15.33 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Spireclimb Sts1 A20 Heart: 13.33 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Spireclimb Sts2 A10: 17.33 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on FrontierMath - Tier 4 (v2): 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on FrontierMath - Tiers 1-3 (v2): 95.09 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on GBA Eval: 0.848 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on GBA Eval - Audio: 0.4623 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on GBA Eval - Procedural Tests: 0.7778 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Gert Labs Rankings: 77.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on GoBench - Track 1: 2565.662263822524 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on HLE-Diamond: 77.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on HLE-Diamond - Knowledge: 74.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on HLE-Diamond - Reasoning: 81.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA: 7.22 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - All Pass: 10.83 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Antitrust &amp; Competition: 88.77 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Arbitration &amp; International Dispute Resolution: 91.28 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Banking &amp; Finance: 94.4 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Bankruptcy &amp; Restructuring: 89.04 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Capital Markets: 94.51 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Corporate Governance: 84.93 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Corporate M&amp;A: 94.11 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Data Privacy &amp; Cybersecurity: 96.7 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Emerging Companies &amp; Venture Capital: 93.29 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Employment &amp; Labor: 86.52 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Energy &amp; Natural Resources: 95.82 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Environmental &amp; ESG: 85.07 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Funds &amp; Asset Management: 90.28 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Healthcare &amp; Life Sciences: 91.95 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Immigration: 92.48 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Insurance: 93.38 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Intellectual Property: 95.61 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - International Trade &amp; Sanctions: 89.38 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Litigation &amp; Dispute Resolution: 92.61 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Real Estate: 93.88 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Structured Finance &amp; Securitization: 90.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Tax: 84.5 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Trusts, Estates &amp; Private Client: 92.49 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - White Collar Defense &amp; Investigations: 90.01 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on HieroglyphBench: 43.6 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Humanity's Sixth Sense: 59.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Icelandic LLM - Generative Quality: 82.94 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Kaggle Game Arena (Unified): 546.34 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Kaggle Game Arena Chess: 1313.29 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Kaggle Game Arena Four in a Row: 818.97 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on KernelBench Hub - CUDA: 10.85 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on KernelBench Hub - Mega: 31.41 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Klu LLM Leaderboard: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (AA-Briefcase v1.1): 1785.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (ARC-AGI-3): 79.26 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (AutomationBench; Private held-out set): 51.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (DeepSWE 1.1): 74.3 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (HealthBench Professional): 67.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (HealthBench; Length-adjusted): 59.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (Program Bench): 85.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (Terminal-Bench 4.0): 67.6 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (Terminal-Bench-Science 0.1): 74.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LM Market Cap LMC Score: 95.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Criteria Pass Rate: 88.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Form: 90.9 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Task Pass Rate: 32.3 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Criteria Pass Rate: 89.2 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Form: 79.3 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Task Pass Rate: 66.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LegalReasonBench: 49.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Browser: 86.9 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Coding: 79.2 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Data Analysis: 81.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Design Layout: 85.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Reasoning: 88.8 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MLCR-AA: 43.89 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MLCR-AA - Accuracy: 75.56 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MLCR-AA - Completeness: 56.11 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MathArena - ARXIV September: 96.23 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MathArena - ARXIV_FALSE September: 80.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Maze-Bench: 90.67 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Mercor APEX: 80.1 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MineBench: 2110.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MissionReport: 63.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on NYT Connections Extended: 97.3 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on OTIS Mock AIME 2024-25: 99.72 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on OpenRouter GPQA Diamond: 94.9 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on OpenRouter Tau2-Bench Airline: 68.0 (#93)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Art &amp; Entertainment: 96.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Culture &amp; Tradition: 98.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Geography: 98.0 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Grammar: 97.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - History: 99.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Overall: 97.33 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Vocabulary: 96.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Contemporary Life: 73.0 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Document QA: 92.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Geography and Environment: 91.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - History and Culture: 77.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - OCR: 92.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Structured Extraction: 71.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Vision Mean: 82.67 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Agents - BFCL-V3: 65.8 (#34)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Agents and Tool Use: 70.4 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Education - High School Subjects: 77.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Education - Primary School Subjects: 70.7 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Finance: 90.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Language and Instruction Following: 72.4 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Law - Lawyer Qualification Exam: 82.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Law and Civil Service: 88.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Medicine - Basic Medical Knowledge: 90.6 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Medicine - Physician Exams: 92.8 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Reasoning - BBH: 87.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on RealmPathology Benchmark: 76.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on RuneBench: 6.983758527082 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SEAL - Fortress: 15.6 (#64)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SEAL - Professional Reasoning Benchmark - Finance: 47.07 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SEAL - Professional Reasoning Benchmark - Legal: 48.97 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SimpleQA Verified: 94.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Bug Density: 0.56 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Code Smell Density: 12.79 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Pass Rate: 90.26 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Vulnerability Density: 0.25 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SpacetimeDB LLM Benchmark (C#): 93.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SpacetimeDB LLM Benchmark (Rust): 96.8 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SpacetimeDB LLM Benchmark (TypeScript): 90.8 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on TaxCalcBench: 62.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Asteroid Numbers: 47.9 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Day-one Recall Check: 61.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Exotic Fact Recall: 75.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Inversion and Execution Ladder: 100.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Lichess Pairwise Strength (25 Pairs): 72.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Lichess Rating Prediction (25 Games): 333.14 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Long-tail Book Recall: 77.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Personal Style Imitation (53 Texts): 73.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Puzzle Portfolio Ladder: 96.7 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Trip Outcome Prediction (5 Questions): 68.7 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Tinybird AI SQL Benchmark - Exactness: 63.86 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Tinybird AI SQL Benchmark - First-Attempt Success Rate: 98.0 (#69)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Tinybird AI SQL Benchmark - Success Rate: 100.0 (#105)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on VIALS-AA: 67.08 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on VIALS-AA - Average of 3: 71.84 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on VIALS-AA - Pass@3: 74.53 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI (Vals Index): 71.309 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI BioMysteryBench: 83.704 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Code Migration v1.1: 83.265 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Excel Modeling v1.1: 77.377 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Finance Agent v2.1: 69.47 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Finance Index: 63.583 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench v1.1: 5.0 (#31)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Immigration: 73.982 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Legal Index: 69.175 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Legal Research Bench: 46.154 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI MysteryMechanism: 51.802 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI ProgramBench: 87.793 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI ProofBench: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 62.292 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Science Index: 63.74 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Tax Agent Bench: 32.092 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 65.152 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Terminal-Bench Science: 55.714 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Vibe Code Bench: 93.771 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vending-Bench 2: 9628.479999999998 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on sudo L7: 52.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy: 91.88 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Maths: 95.24 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Primary: 94.84 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Science: 93.44 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Secondary: 91.19 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Social studies: 89.09 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Technology: 89.62 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education SEND: 85.32 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Arena AI Code: 1817.7 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Coarena: 1045.1 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Coarena - Task Completion: 86.3 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Design Arena (Agents - Fullstack): 1388.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - GPQA Diamond: 94.07 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Mirrorcode: 95.56 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on GBA Eval - Audio: 0.665 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on GBA Eval - Gameplay Replays: 0.9955 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Kaggle Game Arena Four in a Row: 541.59 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LMArena Search Arena: 1213.328 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LMArena Text Arena: 1506.07 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LMArena Vision Arena: 1295.2791 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LMArena WebDev Arena: 1817.7 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Last Translation Benchmark: 35.236004390779364 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MirrorCode: 95.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Bias: 80.0 (#126)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Controllability: 91.82 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Ethics &amp; Morality: 98.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Robustness: 98.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Toxicity: 74.94 (#122)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT - Truthfulness: 86.44 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ALT Average: 88.2 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ARC-AGI-1: 98.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - ARC-AGI-2: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Irrelevance Detection: 80.0 (#91)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Live AST: 69.44 (#72)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Multi-Turn: 72.22 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Non-Live AST: 86.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - BFCL - Relevance Detection: 66.67 (#86)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Applied Language: 90.61 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Basic Language: 87.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - General Knowledge: 83.54 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Information Retrieval: 80.35 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Logical Reasoning: 94.5 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Mathematical Reasoning: 98.17 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Semantic Analysis: 84.3 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - GLP - Translation: 91.48 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Accuracy: 63.92 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Calibration Error: 22.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - HalluLens: 99.0 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - JBBQ (2-shot) - Accuracy: 96.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - JHumanEval: 97.44 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - JTruthfulQA: 73.88 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - M-IFEval (Japanese): 88.5 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Coding: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Extraction: 99.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Humanities: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Math: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Roleplay: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - STEM: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Writing: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - Toxicity - Fairness: 98.31 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - Toxicity - Prohibited Acts: 74.48 (#136)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - Toxicity - Social Norms: 95.24 (#76)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - Toxicity - Violation Categories: 31.75 (#140)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - AIO: 90.41 (#40)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT English-to-Japanese: 92.32 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT Japanese-to-English: 90.52 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - CommonsenseMoralJA: 98.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JBLiMP: 98.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (in-domain): 83.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (out-of-domain): 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCommonsenseQA: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JEMHopQA: 65.66 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JMMLU: 97.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JNLI: 91.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSICK: 83.0 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSQuAD: 82.14 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSeM: 80.0 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaMP: 65.0 (#122)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaNLI: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - KUCI: 93.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MAWPS: 99.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MGSM: 95.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MMLU-ProX (Japanese): 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - NIILC: 64.28 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot): 88.01 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - AIO: 90.72 (#56)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT English-to-Japanese: 92.52 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT Japanese-to-English: 90.57 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JBLiMP: 98.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (in-domain): 87.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (out-of-domain): 91.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCommonsenseQA: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JEMHopQA: 64.5 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JMMLU: 93.0 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JNLI: 93.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSICK: 84.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSQuAD: 78.56 (#77)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSeM: 84.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaMP: 63.0 (#136)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaNLI: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - KUCI: 95.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MAWPS: 99.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MGSM: 96.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MMLU-ProX (Japanese): 91.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - NIILC: 59.81 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Web Search: 54.05218 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on VulcanBench v3 (BenchLM): 91.1 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy: 92.32 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Maths: 95.24 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Primary: 96.24 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Science: 93.99 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Secondary: 91.19 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Social studies: 89.09 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Technology: 84.91 (#61)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education SEND: 85.32 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Arena AI Code: 1766.84 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Arena AI Document: 1467.25 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Coarena: 1036.9 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Coarena - Task Completion: 71.0 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Epoch AI - Mirrorcode: 60.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GBA Eval: 0.8705 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GBA Eval - Audio: 0.6648 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GBA Eval - Gameplay Replays: 0.9955 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GBA Eval - Procedural Tests: 0.7014 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on LMArena Search Arena: 1230.8122 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on LMArena Vision Arena: 1283.3831 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on LMArena WebDev Arena: 1766.84 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on MirrorCode: 60.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on VulcanBench v3 (BenchLM): 89.3 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy: 91.77 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Maths: 92.86 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Primary: 96.24 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Science: 94.54 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Secondary: 90.72 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Social studies: 89.09 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Technology: 85.85 (#42)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education SEND: 86.24 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Arena AI Code: 1742.84 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Blueprint-Bench 2: 0.432 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Coarena: 1127.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Coarena - Task Completion: 81.6 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (SVG): 1343.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Blueprint Bench 2: 0.432 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Mirrorcode: 61.11 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Vending Bench 2: 12860.61 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on GBA Eval: 0.8191 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on GBA Eval - Audio: 0.6829 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on GBA Eval - Gameplay Replays: 0.935 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on GBA Eval - Procedural Tests: 0.6077 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LMArena Search Arena: 1230.3608 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LMArena Vision Arena: 1292.5032 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LMArena WebDev Arena: 1742.84 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on MirrorCode: 61.1 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Bias: 80.0 (#127)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Controllability: 94.53 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Ethics &amp; Morality: 97.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Robustness: 98.5 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Toxicity: 78.79 (#100)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT - Truthfulness: 88.32 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ALT Average: 89.52 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ARC-AGI-1: 98.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - ARC-AGI-2: 96.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Irrelevance Detection: 88.33 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Live AST: 64.81 (#129)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Multi-Turn: 37.78 (#97)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Non-Live AST: 77.78 (#50)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - BFCL - Relevance Detection: 55.56 (#130)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Abstract Reasoning: 97.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Application Development: 75.02 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Applied Language: 90.69 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Coding: 89.92 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Expression: 98.5 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Function Calling: 60.12 (#92)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - General Knowledge: 81.58 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Information Retrieval: 82.18 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Knowledge &amp; QA: 81.23 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Logical Reasoning: 86.5 (#81)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Mathematical Reasoning: 95.83 (#41)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Reasoning: 93.11 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Semantic Analysis: 86.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Specialized Knowledge: 80.89 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Syntactic Analysis: 90.67 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP - Translation: 91.39 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - GLP Average: 85.75 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Accuracy: 55.67 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - HLE (Japanese) - Calibration Error: 35.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - HalluLens: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - JBBQ (2-shot) - Accuracy: 96.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - JHumanEval: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - JTruthfulQA: 76.63 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - M-IFEval (Japanese): 93.81 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese): 96.56 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Coding: 98.5 (#60)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Extraction: 97.0 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Humanities: 98.5 (#88)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Math: 96.0 (#95)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Roleplay: 98.5 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - STEM: 99.0 (#60)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - MT-Bench (Japanese) - Writing: 98.5 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - SWE-Bench (Japanese): 71.25 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Total Score: 86.8 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Toxicity - Fairness: 96.05 (#62)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Toxicity - Prohibited Acts: 78.65 (#133)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Toxicity - Social Norms: 98.41 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - Toxicity - Violation Categories: 42.06 (#72)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot): 86.89 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - AIO: 79.97 (#79)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT English-to-Japanese: 92.19 (#59)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - ALT Japanese-to-English: 90.36 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - CommonsenseMoralJA: 98.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JBLiMP: 97.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (in-domain): 84.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCoLA (out-of-domain): 90.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JCommonsenseQA: 99.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JEMHopQA: 52.66 (#66)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JMMLU: 95.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JNLI: 87.0 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSICK: 81.0 (#48)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSQuAD: 81.28 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JSeM: 82.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaMP: 81.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - JaNLI: 100.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - KUCI: 91.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MAWPS: 98.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MGSM: 94.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - MMLU-ProX (Japanese): 92.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (0-shot) - NIILC: 59.15 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - AIO: 94.17 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT English-to-Japanese: 92.34 (#53)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - ALT Japanese-to-English: 90.66 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JBLiMP: 96.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (in-domain): 87.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCoLA (out-of-domain): 90.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JCommonsenseQA: 99.0 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JEMHopQA: 71.83 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JMMLU: 95.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JNLI: 91.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSICK: 85.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSQuAD: 83.08 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JSeM: 81.0 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaMP: 79.0 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - JaNLI: 100.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - KUCI: 92.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MAWPS: 98.0 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MGSM: 93.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - MMLU-ProX (Japanese): 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Nejumi 4 - jaster (2-shot) - NIILC: 61.98 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vending-Bench 2: 12860.61166666667 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on VulcanBench v3 (BenchLM): 88.8 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AI Chess Leaderboard (Continuation): 1689.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1855.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIMultiple - Agentic RAG Routing: 92.39 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIMultiple - FinanceReasoning (Hard): 90.34 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIMultiple - Text-to-SQL (Adjudicated): 83.3 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIMultiple - Text-to-SQL (Strict Execution Match): 63.2 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ARC-AGI-1: 98.5 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ARC-AGI-2: 94.17 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ARC-AGI-3: 65.12 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard: 9.119 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - General Knowledge: 8.794 (#50)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - MMLU: 9.669 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - Reasoning &amp; Math: 9.302 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - Translation (incl Dialects): 7.972 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - Trust &amp; Safety: 8.333 (#70)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - Writing (incl Dialects): 8.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arena AI Code: 1767.98 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on BenchGecko Score: 93.2 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench: 57.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench - LMS: 35.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 22.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Chess Bench LLM: 2243.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Chess Puzzles (Epoch AI): 78.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ChessBench: 31.75 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ChessBench - Accuracy: 86.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ChessBench - Coherence: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Coarena: 1044.8 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Coarena - Task Completion: 71.7 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Creative Writing (Lechmazur): 1.752 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on CrueltyBench: 34.7 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on CursorBench 4.0: 60.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on DeepsecBench: 35.19 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (3D): 1425.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (Agents - Mobile Apps): 1286.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (Game Dev): 1335.0 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (UI Components): 1347.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (Website): 1319.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on DuelLab GameBench 2 - Reasoning Variants: 94.878522999998 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on EnigmaForge: 50.8 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on EnigmaForge - Fact F1: 91.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on EnigmaForge - Intuition: 38.8 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Cursorbench: 60.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - ECI: 169.76 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Ebr Bench: 87.62 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Frontiermath Erdos: 2.94 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Frontiermath Tier 4 V2: 95.12 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Frontiermath Tiers 1 3 V2: 90.53 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Frontierswe: 61.2 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Furniture Assembly: 77.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - GPQA Diamond: 96.53 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Mystery Game Puzzles: 73.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ErdosBench: 3.24 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierMath - Tier 4 (v2): 95.12 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierMath - Tiers 1-3 (v2): 90.53 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on GDP.pdf: 56.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Gert Labs Rankings: 72.9 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on HLE-Diamond: 63.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on HLE-Diamond - Knowledge: 55.2 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on HLE-Diamond - Reasoning: 72.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on HiL-Bench: 50.33 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Humanity's Sixth Sense: 71.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle FACTS Grounding: 81.08 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Dutch Task v6: 64.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Farsi Task v6: 41.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - French Task v6: 71.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Hebrew Task v6: 56.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Indonesian Task v6: 60.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Italian Task v6: 70.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Japanese Task v6: 60.4 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Russian Task v6: 77.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Spanish Task v6: 68.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Swedish Task v6: 58.8 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Traditional Mandarin Task v6: 76.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on KernelBench Hub - CUDA: 10.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on KernelBench Hub - Mega: 23.76 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on KillSwitch-Bench: 80.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Klu LLM Leaderboard: 93.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LMArena Text Arena: 1525.95 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LMArena WebDev Arena: 1767.98 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Browser: 80.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Coding: 91.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Design Layout: 80.2 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Long Horizon: 72.7 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Reasoning: 90.6 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Research: 92.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MLCR-AA: 19.44 (#59)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MLCR-AA - Accuracy: 37.22 (#84)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MLCR-AA - Completeness: 44.44 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV August: 92.98 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV September: 96.23 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIVLEAN September: 62.26 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV_FALSE August: 88.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV_FALSE September: 82.39 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Maze-Bench: 73.8 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MineBench: 2197.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on NYT Connections Extended: 99.3 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on OTIS Mock AIME 2024-25: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - Culture &amp; Tradition: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - Geography: 100.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - Grammar: 97.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - History: 98.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - Vocabulary: 98.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PUMA Polish Multimodal - Structured Extraction: 77.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on SEAL - Fortress: 16.42 (#59)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on SEAL - Professional Reasoning Benchmark - Legal: 53.35 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on SimpleBench: 92.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on SimpleQA Verified: 77.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on TaxCalcBench: 54.17 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on The Aggregate - Personal Style Imitation (53 Texts): 67.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Tinybird AI SQL Benchmark - Exactness: 61.22 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Tinybird AI SQL Benchmark - First-Attempt Success Rate: 98.0 (#98)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Tinybird AI SQL Benchmark - Success Rate: 100.0 (#193)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on YC-Bench: 3425.28987 (#2)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (318)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;FoodTruckBench&lt;/strong&gt;: Gemini 4 Carbon (77217.68) beat Claude Opus 5 by 1954.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;YC-Bench&lt;/strong&gt;: Claude Fable 5.2 (3445.49457) beat Claude Opus 5.5 by 571.77&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena Chess&lt;/strong&gt;: Gemini 4 Carbon (1646.3) beat Gemini 3.8 Flash by 193.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena (Unified)&lt;/strong&gt;: Gemini 4 Carbon (638.17) beat Gemini 4 Argon by 103.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena Four in a Row&lt;/strong&gt;: Gemini 4 Carbon (1002.7) beat Gemini 4 Argon by 79.37&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-3D&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (1271.4151) beat Claude Sonnet 5.5 (Max, Default Fallback) by 72.58&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Training Error Tracing&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (84.94526407) beat MiMo-V2.6-Flash (High) (Search-SWE Run: Codex v0.164.0) by 64.95&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (Agents - Fullstack)&lt;/strong&gt;: Claude Fable 5.2 (1399.0) beat GPT-6 Astra by 58.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - ChunQiu Temporal Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (56.76611355) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 55.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Arena AI Code&lt;/strong&gt;: Claude Opus 5.5 (Max) (1817.7) beat Claude Fable 5.1 (Max) by 52.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LMArena WebDev Arena&lt;/strong&gt;: Claude Opus 5.5 (Max) (1817.7) beat Claude Fable 5.1 (Max) by 52.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Sketch-and-Text Image Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (55.54960848) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 52.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenEval - BBQ&lt;/strong&gt;: Qwen 1.5 14B Chat (93.5) beat pythia-12B-v0 by 51.36&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Scientific-Paper Question Answering&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (60.0) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 40.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Incremental Vector Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (47.48242491) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 39.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Legal Stance Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (68.80864487) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 38.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Agentic Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (48.83333333) beat MiMo-V2.6-Flash (High) (Search-SWE Run: Codex v0.164.0) by 37.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Chip Design - Aria - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (66.91) beat GPT-6.1 Sol (GDPx Run: v1.0; gpt-6-1-sol; Codex; effort Max) by 33.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MirrorCode&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (97.8) beat Claude Fable 5 (High) by 33.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Large-scale Formula Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (50.54164603) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 33.87&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Glimmer Index - FreeCell&lt;/strong&gt;: Claude Haiku 5.5 (Glimmer Index Run: claude-haiku-5-5-medium;model=anthropic/claude-haiku-5-5;provider=claude;route=claude-haiku-5-5;effort=medium;precision=unreported) (63.3) beat GPT-6 Astra (Glimmer Index Run: codex-gpt-6-astra-medium;model=openai/gpt-6-astra;provider=codex;route=gpt-6-astra;effort=medium;precision=unreported) by 33.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Terminal-Bench 4.0 (Muse Code)&lt;/strong&gt;: Muse Wonder (xHigh) (46.97) beat Muse Spark 1.3 (xHigh) by 32.42&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Long-PDF Sentence Evidence&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (63.88888889) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 30.56&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Reasoning-intensive Query Rewriting&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (60.0) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 30.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - R&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (70.0) beat Gemini 4 Carbon (High) by 30.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Same-Place Image Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (87.00465784) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 29.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Financial-Report Evidence Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (39.97516236) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 29.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Rust BM25 Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (53.09702237) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 29.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - CPU&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (44.73755958) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 28.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ChessBench - Elo&lt;/strong&gt;: Gemini 4 Carbon (1783.0) beat GPT-6 Astra by 28.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Query-side Encoder Alignment&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (55.26315789) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 26.32&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (42.53223472) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 26.15&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Chip Design - Aria - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (38.42) beat GPT-6.1 Sol (GDPx Run: v1.0; gpt-6-1-sol; Codex; effort Max) by 25.61&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Paper Metadata Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (37.5) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 25.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Visual-Document Question Answering&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (44.32775623) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 22.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - GPU&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (40.23102617) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 22.54&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HLE-Diamond - Knowledge&lt;/strong&gt;: Claude Fable 5.2 (Max) (74.6) beat Claude Opus 5.5 (Max) by 21.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDP.pdf&lt;/strong&gt;: Gemini 4 Carbon (High) (56.4) beat GPT-6.1 Sol (Max) by 21.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Decision Model Training&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (59.52380952) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 21.26&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (Agents - Mobile Apps)&lt;/strong&gt;: Claude Fable 5.2 (1324.0) beat Claude Opus 5.5 by 21.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;llmbenchmarks.io Local GPU - ARC-AGI-1 Public Eval&lt;/strong&gt;: Qwen3.6-27B-NVFP4 (Non-reasoning) (58.63) beat Qwen 3.5 4B (Non-reasoning) by 20.63&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Flora &amp; Fauna&lt;/strong&gt;: Claude Fable 5.2 (77.88) beat Claude Opus 5.5 by 20.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Rli&lt;/strong&gt;: Claude Fable 5.2 (Unknown) (41.3) beat Claude Opus 5.5 (Unknown) by 20.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Biomedical Graph Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (30.0) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 20.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SEAL - Remote Labor Index (RLI)&lt;/strong&gt;: Claude Fable 5.2 (41.25) beat Claude Opus 5.5 by 20.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hack-Verifiable TextArena&lt;/strong&gt;: GPT-5.4 (8.5) beat Grok 4.1 Fast by 20.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Document Page Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (26.57158929) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 19.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - HTML&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (91.67) beat Claude Opus 5.5 (Xhigh, Default Fallback) by 19.45&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Mirrorcode&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (97.78) beat Claude Opus 5.5 (High) by 18.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Song Lyrics&lt;/strong&gt;: Claude Fable 5.2 (41.53) beat Claude Opus 5 by 18.79&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - OCR Distillation Trajectory Sampling&lt;/strong&gt;: Gemini 4 Hydrogen (87.47737556561086) beat Gemini 4 Carbon by 18.61&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ARC-AGI-3&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (81.17) beat GPT-6 Astra (Max) by 18.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Health&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (60.22) beat Gemini 4 Carbon (High) by 18.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LMArena Text Arena&lt;/strong&gt;: Gemini 4 Carbon (High) (1525.95) beat Claude Fable 5 by 18.36&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Tau3-Bench Banking_Knowledge&lt;/strong&gt;: PrincipledMachines Cortex 0.1 (73.2) beat Qwen 3.8 Max by 18.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SimpleQA Verified&lt;/strong&gt;: Claude Fable 5.2 (Max) (94.0) beat Gemini 4 Argon (High) by 18.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Reific Environments - Industry&lt;/strong&gt;: Claude Fable 5.2 (Reific Run: native=Claude%20Fable%205.2; cfg=351d3d3783973f89) (60.2) beat Claude Opus 5.5 (Reific Run: native=Claude%20Opus%205.5; cfg=06d6e96794d99d92) by 17.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Code Context Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (33.08004047) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 16.34&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Dual-Arm Sim - Instrument Swap - Progress&lt;/strong&gt;: Claude Fable 5.2 (68.0) beat GPT-6 Astra (Benchmark Sweep Run: source=Micro1DualArm; API; 120 decisions; effort and snapshot unreported) by 16.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Artificial Analysis Music Arena - Vocals&lt;/strong&gt;: ACE Music v1 (1160.0) beat Suno v6 by 16.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA HLE-Diamond - Knowledge&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (74.8) beat Gemini 4 Carbon (Medium) by 15.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Humanity's Sixth Sense&lt;/strong&gt;: Gemini 4 Carbon (High) (71.1) beat Gemini 4 Argon (High) by 15.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Narrative-Clue Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (24.65711778) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 15.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIVLEAN September&lt;/strong&gt;: Gemini 4 Carbon (High) (62.26) beat Claude Opus 5.5 (High) by 15.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - FARGO paper-aligned query core&lt;/strong&gt;: Inkling 2 Small (OpenCode) (67.11574029712814) beat Claude Opus 5.5 (Max) by 14.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Business&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (57.11) beat Gemini 4 Carbon (High) by 14.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Artificial Analysis Music Arena&lt;/strong&gt;: ACE Music v1 (1160.0) beat Suno v6 by 14.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Long-PDF Sentence Evidence — Vectorless&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (63.88888889) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 13.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - Go&lt;/strong&gt;: Claude Fable 5.2 (Xhigh, Default Fallback) (100.0) beat GPT-6 Astra (Max) by 13.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Budgeted Bimetric Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (32.04050889) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 13.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Methodology Inspiration Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (16.83410533) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 13.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (59.0) beat Gemini 4 Carbon (High) by 13.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Software Engineering - Datacurve - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (24.44) beat Claude Sonnet 5.5 (GDPx Run: v1.0; claude-sonnet-5-5; Claude Code; effort Max) by 13.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Humanity's Last Exam&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (74.28) beat Claude Opus 5.5 (Max, Default Fallback) by 12.93&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Chip Design - Domain Mean - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (25.1166666667) beat GPT-6.1 Sol (GDPx Run: v1.0; gpt-6-1-sol; Codex; effort Max) by 12.79&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (SWE-Bench Multimodal)&lt;/strong&gt;: Claude Fable 5.2 (74.1) beat Claude Opus 5.5 by 12.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Glimmer Index - Mines&lt;/strong&gt;: Claude Haiku 5.5 (Glimmer Index Run: claude-haiku-5-5-medium;model=anthropic/claude-haiku-5-5;provider=claude;route=claude-haiku-5-5;effort=medium;precision=unreported) (70.8) beat GPT-6.1 Sol (Glimmer Index Run: codex-gpt-6-1-sol-medium;model=openai/gpt-6.1-sol;provider=codex;route=gpt-6.1-sol;effort=medium;precision=unreported) by 12.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Ebr Bench&lt;/strong&gt;: Claude Fable 5.2 (Max) (90.48) beat GPT-6.1 Terra (Max) by 12.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - WikiQA&lt;/strong&gt;: Claude Fable 5.2 (86.58) beat Claude Opus 5.5 by 12.26&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Humanities &amp; Social Sciences&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (68.44) beat Gemini 4 Carbon (High) by 12.22&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Khmer-to-English Embedding Training&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (33.04938272) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 12.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HLE-Diamond&lt;/strong&gt;: Claude Fable 5.2 (Max) (77.9) beat GPT-6 Astra (Max) by 12.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - HLE (Japanese) - Calibration Error&lt;/strong&gt;: Claude Opus 5.5 (Max) (22.0) beat Grok 4.5 (High) by 12.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - HLE (Japanese) - Accuracy&lt;/strong&gt;: Claude Opus 5.5 (Max) (63.92) beat GPT-5.6 Sol (Max) by 11.86&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Argo-Bench&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (70.22) beat Claude Opus 5.5 (xHigh) by 11.84&lt;/li&gt;&lt;li&gt;&lt;strong&gt;The Aggregate - Exotic Fact Recall&lt;/strong&gt;: Claude Fable 5.2 (High) (75.0) beat GPT-5.6 Sol (High) by 11.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA HLE-Diamond&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (80.2) beat Gemini 4 Carbon (High) by 11.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OSWorld 2.1 (strict))&lt;/strong&gt;: Claude Fable 5.2 (54.6) beat Claude Sonnet 5.5 by 11.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;TAC (Travel Agent Compassion)&lt;/strong&gt;: Claude Fable 5.2 (75.6) beat Claude Opus 4.8 by 10.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Instruction-following Embedding&lt;/strong&gt;: GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) (100.0) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 10.75&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Composed Image Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (46.84707561) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 10.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Terminal-Bench-Science 0.1)&lt;/strong&gt;: Claude Mythos 5.2 (75.3) beat Gemini 4 Carbon by 10.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Dual-Arm Sim - All - Progress&lt;/strong&gt;: Claude Fable 5.2 (59.8) beat Gemini 4 Argon (Benchmark Sweep Run: source=Micro1DualArm; API; 120 decisions; effort and snapshot unreported) by 10.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Chip Design - Domain Mean - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (15.62) beat GPT-6.1 Sol (GDPx Run: v1.0; gpt-6-1-sol; Codex; effort Max) by 10.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;KillSwitch-Bench&lt;/strong&gt;: Claude Fable 5.2 (80.0) beat Claude Opus 5 by 10.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Video-Moment Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (32.15543698) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 9.39&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Evolving Personal Memory&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (42.15313994) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 9.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - MedAraBench&lt;/strong&gt;: Karnak-Th-41B-v1.0 (64.24) beat Llama 3.3 70B Instruct by 9.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - Java&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (90.91) beat Claude Opus 5.5 (Max, Default Fallback) by 9.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Long-document Reranking&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (42.0) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 9.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (73.33) beat Gemini 4 Carbon (High) by 8.57&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Web Search&lt;/strong&gt;: Claude Opus 5.5 Fast (57.535937) beat GPT-5.6 Sol (Max) by 8.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - MedArabiQ MCQ&lt;/strong&gt;: Karnak-Th-41B-v1.0 (66.96) beat Karnak-40B-v1.0 by 8.36&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Budgeted Candidate Reranking&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (25.23026772) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 8.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE)&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (92.67) beat Claude Opus 5.5 (Max, Default Fallback) by 8.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Chess Leaderboard (Continuation)&lt;/strong&gt;: Gemini 4 Hydrogen (1818.0) beat Gemini 3 Pro (Preview) by 8.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Malay Task v6&lt;/strong&gt;: Gemini 4 Carbon (48.0) beat Gemini 3.1 Pro (Preview) by 8.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Terminal-Bench 4.0 (mini-SWE-agent)&lt;/strong&gt;: MiMo-V2.6-Pro (Terminal-Bench Effort: none) (26.97) beat Gemini 3.8 Flash (High) by 7.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM Leaderboard - Average&lt;/strong&gt;: Claude Fable 5.2 (78.73) beat Claude Opus 5.5 by 7.86&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OSWorld 2.1 (partial))&lt;/strong&gt;: Claude Fable 5.2 (89.3) beat Gemini 4 Carbon by 7.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Argo-Bench - Solved&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (41.43) beat Gemini 4 Argon (High) by 7.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;The Aggregate - Long-tail Book Recall&lt;/strong&gt;: Claude Fable 5.2 (High) (77.0) beat Claude Fable 5.1 (High) by 7.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Idiom Completion&lt;/strong&gt;: Claude Fable 5.2 (83.4) beat Gemini 4 Argon by 7.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AppWorld Challenge&lt;/strong&gt;: GLM-5.3 Flash (92.8) beat Claude Opus 4.7 by 7.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Life Sciences&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (68.42) beat Gemini 4 Carbon (High) by 7.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Protein–Text Bidirectional Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (23.595) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 6.86&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench (Default Fallbacks)&lt;/strong&gt;: Claude Fable 5.2 (default fallbacks, Max) (51.45) beat Claude Sonnet 5.5 (default fallbacks, Max) by 6.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Bug Hunt Bench&lt;/strong&gt;: Claude Fable 5.2 (Max) (58.0) beat Claude Sonnet 5.5 (Max) by 6.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Furniture Assembly&lt;/strong&gt;: Claude Fable 5.2 (Max) (90.0) beat Claude Opus 5.5 (Max) by 6.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CWE-Bench-AA&lt;/strong&gt;: hy4 (75.83) beat DeepSeek V4.1 Flash (1017) (Max) by 6.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - E-Graph Invariant Restoration&lt;/strong&gt;: Gemini 4 Hydrogen (68.53231954722582) beat Gemini 4 Carbon by 6.56&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Materials Science - Emulated - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (30.83) beat Gemini 4 Carbon (GDPx Run: v1.0; gemini-4-carbon; Antigravity; effort High) by 6.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Research&lt;/strong&gt;: Claude Fable 5.2 (94.5) beat Muse Wonder by 6.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SimpleBench&lt;/strong&gt;: Claude Fable 5.2 (94.9) beat Claude Opus 5.5 by 6.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - Python&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (97.87) beat Claude Opus 5.5 (Max, Default Fallback) by 6.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Bug Hunt Bench - LMS&lt;/strong&gt;: Claude Fable 5.2 (Max) (35.0) beat Claude Sonnet 5.5 (Max) by 6.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Accuracy&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (74.43) beat Claude Fable 5.1 (Max, Default Fallback) by 6.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Mystery Game Puzzles&lt;/strong&gt;: Claude Fable 5.2 (Max) (90.0) beat GPT-6 Astra (Max) by 6.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA HLE-Diamond - Reasoning&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (85.6) beat Gemini 4 Carbon (High) by 6.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - PHP&lt;/strong&gt;: Claude Fable 5.2 (Low, Default Fallback) (91.18) beat Claude Opus 5.5 (High, Default Fallback) by 5.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - BFCL - Multi-Turn&lt;/strong&gt;: Claude Opus 5.5 (Max) (72.22) beat GPT-5.6 Sol (Max) by 5.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - JHumanEval&lt;/strong&gt;: GPT-6.1 Terra (Max) (100.0) beat Claude Opus 5 (Max) by 5.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Book Recommendations&lt;/strong&gt;: Claude Fable 5.2 (95.71) beat GPT-5.6 Sol by 5.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Long Horizon&lt;/strong&gt;: Claude Fable 5.2 (82.9) beat Claude Fable 5.1 by 5.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BenchLM&lt;/strong&gt;: Claude Fable 5.2 (93.7) beat Gemini 4 Carbon by 5.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - OCR&lt;/strong&gt;: Gemini 4 Carbon (96.0) beat Gemini 4 Argon by 5.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Layout Page Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (23.55070204) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 4.74&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV September&lt;/strong&gt;: Gemini 4 Carbon (High) (96.23) beat GPT-6.1 Sol (Max) by 4.72&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Artificial Analysis Intelligence Index&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (64.56) beat Gemini 4 Carbon (High) by 4.53&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA CritPt v2 - Hard Subset&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (76.5) beat GPT-6 Astra (Medium) by 4.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - Overall&lt;/strong&gt;: Karnak-Th-41B-v1.0 (73.71) beat Qwen 3.5 397B A17B by 4.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Frontierswe&lt;/strong&gt;: Claude Fable 5.2 (Max) (69.94) beat GPT-6 Astra (Max) by 4.43&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierSWE V2&lt;/strong&gt;: Claude Fable 5.2 (69.94018470588233) beat GPT-6 Astra (Max) by 4.43&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Overall - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (16.44) beat Claude Opus 5.5 (GDPx Run: v1.0; claude-opus-5-5; Claude Code; effort Max) by 4.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BALSAM - Fill in the Blank&lt;/strong&gt;: slm-3.1 (90.0) beat GPT-5.2 by 4.39&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - ECI&lt;/strong&gt;: Claude Fable 5.2 (173.44) beat GPT-6 Astra by 4.37&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Omniscience&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (57.87) beat Gemini 4 Carbon (High) by 4.37&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DuelLab GameBench 2 - Reasoning Variants&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (99.226094853495) beat Claude Opus 5.5 (xHigh) by 4.36&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Vision Mean&lt;/strong&gt;: Gemini 4 Carbon (89.17) beat Gemini 4 Argon by 4.34&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Humanity's Last Exam (with tools))&lt;/strong&gt;: Claude Fable 5.2 (74.2) beat Kimi K3.1 Pro by 4.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Earth Sciences&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (70.83) beat Claude Opus 5.5 (Xhigh, Default Fallback) by 4.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Overall - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (26.5687878788) beat Claude Opus 5.5 (GDPx Run: v1.0; claude-opus-5-5; Claude Code; effort Max) by 4.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Textile Terms&lt;/strong&gt;: Claude Fable 5.2 (78.1) beat GPT-6 Astra by 4.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Application Development&lt;/strong&gt;: Claude Opus 5.5 (84.44) beat Claude Opus 5 by 4.06&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Law&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (57.11) beat Gemini 4 Carbon (High) by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Polish Task v6&lt;/strong&gt;: Gemini 4 Carbon (48.0) beat GPT-6 Astra by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Document QA&lt;/strong&gt;: Gemini 4 Carbon (97.0) beat Gemini 4 Argon by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Physical Sciences&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (70.59) beat Claude Opus 5.5 (Xhigh, Default Fallback) by 3.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VulcanBench v3 (BenchLM)&lt;/strong&gt;: Claude Fable 5.2 (93.8) beat Grok 4.5 by 3.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences&lt;/strong&gt;: Claude Fable 5.2 (Xhigh, Default Fallback) (70.37) beat Gemini 4 Carbon (High) by 3.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Arabic Task v6&lt;/strong&gt;: Gemini 4 Carbon (55.2) beat Gemini 3.1 Pro (Preview) by 3.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Portuguese Task v6&lt;/strong&gt;: Gemini 4 Carbon (78.0) beat Gemini 3.1 Pro (Preview) by 3.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Romanian Task v6&lt;/strong&gt;: Gemini 4 Carbon (57.6) beat GPT-6 Astra by 3.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Idiom Meanings&lt;/strong&gt;: Claude Fable 5.2 (88.19) beat GPT-6 Astra by 3.58&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Open Benchmarks - GPQA Diamond (Zero-Shot)&lt;/strong&gt;: Gemini 4 Carbon (96.97) beat Gemini 3.6 Flash by 3.54&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VIALS-AA - Average of 3&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (71.84) beat GPT-6 Astra (Max) by 3.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Code Embedding Fine-tuning&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (20.5) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 3.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Function Calling&lt;/strong&gt;: Claude Opus 5.5 (75.99) beat GPT-5.6 Sol by 3.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WDCD&lt;/strong&gt;: Claude Sonnet 4.6 (93.93) beat GPT-6 Astra by 3.43&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Briefcase - XLSX - Rubric Pass Rate&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (69.52) beat Claude Sonnet 5.5 (Max, Default Fallback) by 3.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VIALS-AA&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (67.08) beat Gemini 4 Carbon (High) by 3.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (Image)&lt;/strong&gt;: gpt-image-2.5-sunburst (1388.0) beat rowan by 3.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Chess Puzzles (Epoch AI)&lt;/strong&gt;: Claude Fable 5.2 (Max) (79.0) beat GPT-6 Astra Pro (Max) by 3.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Contemporary Life&lt;/strong&gt;: Gemini 4 Carbon (87.0) beat Gemini 3.8 Flash by 3.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JCoLA (out-of-domain)&lt;/strong&gt;: GPT-6.1 Terra (Max) (94.0) beat DeepSeek V4 Flash (0731) (Max) by 3.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Supra-microBench - Linguistic Competence&lt;/strong&gt;: ZeroS-v0.1-150M (65.13) beat SmolLM2-135M-Base by 2.95&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AppWorld Normal&lt;/strong&gt;: GLM-5.3 (94.0) beat Claude Opus 4.6 by 2.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AIMultiple - Agentic RAG Routing&lt;/strong&gt;: Claude Fable 5.2 (96.74) beat Gemini 4 Argon by 2.72&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Constrained Molecular Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (18.2149) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 2.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (SWE-bench Multilingual)&lt;/strong&gt;: Claude Fable 5.2 (96.6) beat Claude Opus 5.5 by 2.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA FrontierCode 2.0&lt;/strong&gt;: Claude Fable 5.2 (High, Default Fallback) (49.13) beat Claude Opus 5.5 (Max, Default Fallback) by 2.69&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CursorBench 4.0&lt;/strong&gt;: Gemini 4 Carbon (High) (60.4) beat Claude Opus 5.5 (Max) by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Cursorbench&lt;/strong&gt;: Gemini 4 Carbon (High) (60.4) beat Claude Opus 5.5 (Max) by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Agents - BFCL-V3&lt;/strong&gt;: Gemini 4 Carbon (79.1) beat MiniMax-M2.7 by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Reific Environments - Personal Agents&lt;/strong&gt;: Claude Fable 5.2 (Reific Run: native=Claude%20Fable%205.2; cfg=351d3d3783973f89) (34.2) beat Gemini 4 Argon (Reific Run: native=Gemini%204%20Argon; cfg=b2c93c3d7afec7c8) by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI BioMysteryBench&lt;/strong&gt;: Claude Fable 5.2 (High) (83.704) beat Claude Sonnet 5.5 (Max) by 2.59&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Biology - LatchBio - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (28.21) beat Gemini 4 Carbon (GDPx Run: v1.0; gemini-4-carbon; Antigravity; effort High) by 2.57&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LMArena Search Arena&lt;/strong&gt;: Qwen 3.8 Max (1259.7846) beat GPT-5.6 Sol (xHigh) by 2.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Lmca&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (70.61) beat Claude Sonnet 5.5 (Max) by 2.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conceptual Reasoning Index - Argument Evaluation (LMCA)&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (70.61) beat Claude Sonnet 5.5 by 2.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA CritPt v2&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (86.29) beat GPT-6 Astra (Medium) by 2.29&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Hardware Design - Beautiful Machines - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (20.99) beat Claude Opus 5.5 (GDPx Run: v1.0; claude-opus-5-5; Claude Code; effort Max) by 2.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MLCR-AA&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (43.89) beat GPT-6 Astra (Max) by 2.22&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ChessBench - Accuracy&lt;/strong&gt;: Claude Fable 5.2 (87.2) beat GPT-6 Astra by 2.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Overall&lt;/strong&gt;: Claude Fable 5.2 (80.8) beat Claude Opus 5.5 by 2.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Coding&lt;/strong&gt;: Claude Opus 5.5 (92.9) beat Claude Opus 5 by 2.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Specialized Knowledge&lt;/strong&gt;: Claude Opus 5.5 (83.47) beat Claude Fable 5 by 2.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP Average&lt;/strong&gt;: Claude Opus 5.5 (88.74) beat Claude Opus 5 by 2.06&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conceptual Reasoning Index&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (79.89) beat Claude Opus 5.5 by 2.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Simplified Mandarin Task v6&lt;/strong&gt;: Gemini 4 Carbon (71.6) beat Gemini 3.1 Pro (Preview) by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Vietnamese Task v6&lt;/strong&gt;: Gemini 4 Carbon (59.6) beat Gemini 3.1 Pro (Preview) by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLCC - Art &amp; Entertainment&lt;/strong&gt;: Gemini 4 Carbon (97.0) beat Gemini 3 Pro (Preview) by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JCoLA (in-domain)&lt;/strong&gt;: Claude Opus 5.5 (Max) (87.0) beat Llama 3.1 405B Instruct FP8 by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Knowledge &amp; QA&lt;/strong&gt;: Claude Opus 5.5 (83.51) beat GPT-5.6 Sol by 1.97&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Engineering - Criterion Pass Rate&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (86.26) beat Claude Opus 5.5 (Max, Default Fallback) by 1.91&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Hypergraph Core Numbers&lt;/strong&gt;: Gemini 4 Hydrogen (61.4131399518806) beat Gemini 4 Argon by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - GAT&lt;/strong&gt;: Karnak-Th-41B-v1.0 (63.97) beat AIC-1 by 1.86&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI for Education Pedagogy - Social studies&lt;/strong&gt;: Gemini 4 Argon (Low) (93.64) beat O3 by 1.82&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Apex Agents&lt;/strong&gt;: Gemini 4 Carbon (84.2) beat Gemini 4 Argon by 1.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV_FALSE August&lt;/strong&gt;: Gemini 4 Carbon (High) (88.1) beat GPT-6.1 Sol (Max) by 1.79&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - LRR&lt;/strong&gt;: Gemini 4 Hydrogen (34.760000000000005) beat Gemini 4 Carbon by 1.73&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Coding&lt;/strong&gt;: Claude Fable 5.2 (79.7) beat Claude Sonnet 5.5 by 1.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sonar LLM Leaderboard - Java - Pass Rate&lt;/strong&gt;: Claude Fable 5.2 (Thinking, High) (90.26) beat Claude Opus 5 (Thinking) by 1.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Finance Agent v2.1&lt;/strong&gt;: Claude Fable 5.2 (High) (69.47) beat Claude Opus 5.5 (Max) by 1.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Agents' Last Exam)&lt;/strong&gt;: Muse Ember (61.6) beat GPT-6.1 Terra by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Khmer Task v6&lt;/strong&gt;: Gemini 4 Carbon (39.6) beat GPT-6 Astra by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Tagalog Task v6&lt;/strong&gt;: Gemini 4 Carbon (59.2) beat Gemini 3.8 Flash by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;sudo L7&lt;/strong&gt;: Claude Fable 5.2 (Adaptive Reasoning, Max Effort) (52.2) beat Claude Sonnet 5.5 (Adaptive Reasoning, Max Effort) by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Syntactic Analysis&lt;/strong&gt;: Claude Opus 5.5 (91.5) beat GPT-5.5 (2026-04-23) by 1.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Future Video Sampling&lt;/strong&gt;: Gemini 4 Hydrogen (77.80450536665703) beat Gemini 4 Carbon by 1.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierMath - Tiers 1-3 (v2)&lt;/strong&gt;: Claude Fable 5.2 (Max) (95.09) beat GPT-6.1 Sol (Max) by 1.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Frontiermath Tiers 1 3 V2&lt;/strong&gt;: Claude Fable 5.2 (Max) (95.09) beat GPT-6.1 Sol (Max) by 1.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Chartography)&lt;/strong&gt;: Claude Fable 5.2 (91.6) beat Kimi K3.1 Pro by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Vibe Code Bench&lt;/strong&gt;: Claude Fable 5.2 (High) (93.771) beat Claude Sonnet 5.5 (Max) by 1.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Famous Phrases&lt;/strong&gt;: Claude Fable 5.2 (37.52) beat Gemini 4 Argon by 1.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Data Analysis&lt;/strong&gt;: Gemini 4 Carbon (86.5) beat Gemini 4 Argon by 1.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (0-shot) - NIILC&lt;/strong&gt;: Claude Opus 5.5 (Max) (64.28) beat DeepSeek V4.1 Flash by 1.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - SWE-Bench (Japanese)&lt;/strong&gt;: Claude Opus 5.5 (81.25) beat Claude Fable 5 by 1.25&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GoTo-AI Leaderboard - IndoMMLU - Local Languages and Cultures&lt;/strong&gt;: Llama-Sahabat-AI-v2-70B-IT (44.46) beat Llama-Sahabat-AI-v2-70B-R by 1.24&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - Total Score&lt;/strong&gt;: Claude Opus 5.5 (88.43) beat Claude Opus 5 by 1.23&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Terminology&lt;/strong&gt;: Claude Fable 5.2 (63.04) beat GPT-6 Astra by 1.23&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Bengali Task v6&lt;/strong&gt;: Gemini 4 Carbon (48.0) beat Gemini 3.1 Pro (Preview) by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Czech Task v6&lt;/strong&gt;: Gemini 4 Carbon (65.6) beat Gemini 3.8 Flash by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Marathi Task v6&lt;/strong&gt;: Gemini 4 Carbon (51.6) beat Gemini 3.1 Pro (Preview) by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Agents and Tool Use&lt;/strong&gt;: Gemini 4 Carbon (77.0) beat MiniMax-M2.7 by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats Score&lt;/strong&gt;: Claude Fable 5.2 (61.06) beat Claude Opus 5.5 by 1.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals Inference Provider - Baseten - Overall&lt;/strong&gt;: Kimi K3 (Max) (69.877) beat DeepSeek V4.1 Flash by 1.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sonar LLM Leaderboard - Java - Code Smell Density&lt;/strong&gt;: Claude Fable 5.2 (Thinking, High) (12.79) beat Claude Opus 4.5 (High) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Axis Spatial - GeoBenchX&lt;/strong&gt;: Claude Opus 5.5 (Max) (Axis Spatial Run: native=claude-code-claude-opus-5-5-max; cfg=64f5831c15f78626) (83.67) beat Claude Opus 5.5 (xHigh) (Axis Spatial Run: native=claude-code-claude-opus-5-5-xhigh; cfg=b2f9ca8815f8e318) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Abstract Reasoning&lt;/strong&gt;: Claude Opus 5.5 (99.0) beat GPT-5.6 Terra by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLCC - Overall&lt;/strong&gt;: Gemini 4 Carbon (98.33) beat Gemini 4 Argon by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Idioms&lt;/strong&gt;: Claude Fable 5.2 (75.0) beat GPT-6 Astra by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - History and Culture&lt;/strong&gt;: Gemini 4 Carbon (86.0) beat Gemini 3.7 Flash by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Geography and Environment&lt;/strong&gt;: Gemini 4 Carbon (92.0) beat Gemini 3.8 Flash by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (0-shot) - JCoLA (out-of-domain)&lt;/strong&gt;: GPT-6.1 Terra (Max) (93.0) beat Qwen3.8-Flash-Next (xHigh) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JBLiMP&lt;/strong&gt;: Claude Opus 5.5 (Max) (98.0) beat Gemini 3.6 Flash by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - KUCI&lt;/strong&gt;: Claude Opus 5.5 (Max) (95.0) beat Gemini 3.1 Pro (Preview) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - MGSM&lt;/strong&gt;: Claude Opus 5.5 (Max) (96.0) beat Claude Opus 5 (Max) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conceptual Reasoning Index - Consistency (ACCoRD)&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (89.39) beat Claude Opus 5.5 (High) by 0.97&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Case Government&lt;/strong&gt;: Claude Fable 5.2 (100.0) beat Claude Opus 5 by 0.94&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Briefcase&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (66.08) beat Claude Sonnet 5.5 (Max, Default Fallback) by 0.93&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - GEC&lt;/strong&gt;: Claude Fable 5.2 (83.94) beat Claude Opus 5 by 0.91&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Reasoning and Mathematics&lt;/strong&gt;: Claude Fable 5.2 (92.9) beat Claude Opus 5 by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education - Gaokao&lt;/strong&gt;: Claude Fable 5.2 (70.2) beat doubao-seed-evolving by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education - Middle School Subjects&lt;/strong&gt;: Claude Fable 5.2 (61.3) beat Kimi K3.1 Pro by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Sales - Raw Objective Completion&lt;/strong&gt;: Gemini 4 Argon (Medium) (94.54) beat Gemini 4 Carbon (Medium) by 0.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Basic Language&lt;/strong&gt;: GPT-6.1 Sol (88.68) beat GPT-5.5 (2026-04-23) by 0.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals Inference Provider - Fireworks - Finance Agent v2&lt;/strong&gt;: Kimi K3 (Max) (59.291) beat DeepSeek V4.1 Flash by 0.87&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Scicode&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (67.71) beat Claude Opus 5.5 (Max, Default Fallback) by 0.81&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA SciCode&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (67.71) beat Claude Opus 5.5 (Max, Default Fallback) by 0.81&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI ProgramBench&lt;/strong&gt;: Claude Fable 5.2 (High) (87.793) beat Claude Opus 5.5 (Max) by 0.81&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Thai Task v6&lt;/strong&gt;: Gemini 4 Carbon (42.0) beat Gemini 3.1 Pro (Preview) by 0.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - 3LM STEM&lt;/strong&gt;: Karnak-Th-41B-v1.0 (96.04) beat Qwen 3.5 35B A3B Base by 0.71&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OfficeQA Pro)&lt;/strong&gt;: Claude Fable 5.2 (72.9) beat Seed 2.1 Pro by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NYT Connections Extended&lt;/strong&gt;: Gemini 4 Carbon (Thinking, High) (99.3) beat Gemini 4 Argon (Thinking, High) by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Reasoning&lt;/strong&gt;: Claude Opus 5.5 (97.22) beat Claude Opus 5 by 0.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Excel Modeling v1.1&lt;/strong&gt;: Claude Fable 5.2 (High) (77.377) beat Claude Fable 5.1 (Max) by 0.65&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWE-Marathon v1.1&lt;/strong&gt;: Gemini 4 Carbon (Thinking, high) (Antigravity CLI) (53.75) beat Muse Wonder Full (Thinking, max) (Muse Code) by 0.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VIALS-AA - Pass@3&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (74.53) beat Kimi K3.1 Pro by 0.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Supra-microBench - Lexicon and Semantics&lt;/strong&gt;: cagliostro-v3 (86.94) beat SmolLM2-135M-Base by 0.61&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Glimmer Index - Hustle Sim&lt;/strong&gt;: Claude Sonnet 5.5 (Glimmer Index Run: claude-sonnet-5-5-medium;model=anthropic/claude-sonnet-5-5;provider=claude;route=claude-sonnet-5-5;effort=medium;precision=unreported) (64.2) beat GPT-6 Astra (Glimmer Index Run: codex-gpt-6-astra-medium;model=openai/gpt-6-astra;provider=codex;route=gpt-6-astra;effort=medium;precision=unreported) by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI for Education Pedagogy - Science&lt;/strong&gt;: Gemini 4 Argon (High) (97.27) beat Grok 4.6 by 0.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Insurance - Criterion Pass Rate&lt;/strong&gt;: Gemini 4 Hydrogen (Medium) (91.46) beat Gemini 4 Carbon (Medium) by 0.53&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU IVF-PQ Search Kernel Optimization&lt;/strong&gt;: Gemini 4 Hydrogen (79.68875187703873) beat Gemini 4 Carbon by 0.53&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Tax Agent Bench&lt;/strong&gt;: Gemini 4 Hydrogen (xHigh) (52.305) beat Gemini 4 Carbon (High) by 0.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Dual-Arm Sim - Expert Recovery&lt;/strong&gt;: Claude Fable 5.2 (4.5) beat Gemini 4 Argon (Benchmark Sweep Run: source=Micro1DualArm; API; 120 decisions; effort and snapshot unreported) by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Expression&lt;/strong&gt;: Claude Opus 5.5 (100.0) beat Qwen 3.6 Max Preview by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education&lt;/strong&gt;: Claude Fable 5.2 (69.8) beat Gemini 4 Argon by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - MT-Bench (Japanese) - Writing&lt;/strong&gt;: Claude Opus 5.5 (Max) (100.0) beat Mistral Large 4 (High) by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QIMMA - FannOrFlop&lt;/strong&gt;: Qwen 3 VL 235B A22B Instruct (61.01) beat Qwen 3.5 122B A10B by 0.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI (Vals Index)&lt;/strong&gt;: Claude Fable 5.2 (High) (71.309) beat Claude Sonnet 5.5 (Max) by 0.45&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conceptual Reasoning Index - Decision Theory (DTBench)&lt;/strong&gt;: Claude Fable 5.2 (98.67) beat Claude Opus 5.5 by 0.45&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - MT-Bench (Japanese)&lt;/strong&gt;: Claude Opus 5.5 (99.25) beat Claude Opus 5 by 0.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Differentiable Population Statistics&lt;/strong&gt;: Gemini 4 Hydrogen (7.238810160238226) beat Gemini 4 Carbon by 0.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals Inference Provider - Baseten - Vibe Code Bench&lt;/strong&gt;: DeepSeek V4.1 Flash (Max) (82.491) beat Kimi K3 by 0.42&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HLE-Diamond - Reasoning&lt;/strong&gt;: Claude Fable 5.2 (Max) (81.2) beat GPT-6 Astra (Max) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - English Task v6&lt;/strong&gt;: Gemini 4 Carbon (78.4) beat GPT-6.1 Sol by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Cantonese Task v6&lt;/strong&gt;: Gemini 4 Carbon (51.6) beat Gemini 3.1 Pro (Preview) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Hindi Task v6&lt;/strong&gt;: Gemini 4 Carbon (53.2) beat Gemini 3.1 Pro (Preview) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot)&lt;/strong&gt;: GPT-6.1 Sol (88.98) beat Claude Fable 5 by 0.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GBA Eval - Procedural Tests&lt;/strong&gt;: Claude Opus 5.5 (0.8764) beat GPT-5.6 Sol by 0.32&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI for Education Pedagogy - Secondary&lt;/strong&gt;: Gemini 4 Argon (High) (91.98) beat Claude Fable 5.1 by 0.31&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Dtbench&lt;/strong&gt;: Claude Fable 5.2 (Unknown) (99.2) beat Claude Opus 5.5 (Max) by 0.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Materials Science - Domain Mean - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (21.83) beat GPT-6 Astra (GDPx Run: v1.0; gpt-6-astra; Codex; effort Max) by 0.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Exact Sequence Alignment&lt;/strong&gt;: Gemini 4 Hydrogen (67.19671459086022) beat Gemini 4 Argon by 0.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - KAMI FP64 Block GEMM — trusted batched harness adaptation&lt;/strong&gt;: Gemini 4 Hydrogen (23.24232055075762) beat Gemini 4 Carbon by 0.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Overall&lt;/strong&gt;: Claude Fable 5.2 (74.58) beat GPT-6 Astra by 0.26&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle FACTS (Google)&lt;/strong&gt;: Gemini 4 Carbon (78.35) beat GPT-6 Astra by 0.24&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI for Education Pedagogy&lt;/strong&gt;: Gemini 4 Argon (High) (92.88) beat GPT-6 Astra (High) by 0.22&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Go1 Joystick Policy Training&lt;/strong&gt;: Gemini 4 Hydrogen (50.10215349872997) beat Gemini 4 Argon by 0.21&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf&lt;/strong&gt;: Gemini 4 Hydrogen (Medium) (32.8) beat Muse Wonder Full (Max) by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Medicine and Mental Health&lt;/strong&gt;: Claude Fable 5.2 (92.9) beat Kimi K3.1 Pro by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals Inference Provider - First Party - Finance Agent v2&lt;/strong&gt;: MiniMax-M3 (59.623) beat Kimi K3 (Max) by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Operations - Raw Objective Completion&lt;/strong&gt;: Gemini 4 Hydrogen (High) (97.6) beat Claude Sonnet 5.5 (Max, Default Fallback) by 0.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - GPQA Diamond&lt;/strong&gt;: Gemini 4 Carbon (High) (96.53) beat Gemini 4 Argon (High) by 0.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (0-shot)&lt;/strong&gt;: Claude Opus 5.5 (88.14) beat GPT-5.5 (2026-04-23) by 0.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Materials Science - Domain Mean - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (32.76) beat GPT-6 Astra (GDPx Run: v1.0; gpt-6-astra; Codex; effort Max) by 0.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GSMA Open-Telco - TeleLogs&lt;/strong&gt;: telco-slm-4B (96.41) beat OTel-LLM-8.3B-QnA by 0.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GBA Eval&lt;/strong&gt;: Claude Opus 5.5 (0.9056) beat Claude Opus 5 by 0.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Arabic Broad Leaderboard - RAG QA&lt;/strong&gt;: Gemini 4 Carbon (9.829) beat GLM-5.3 by 0.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Alignment - Fleet - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (9.29) beat Muse Wonder Full (GDPx Run: v1.0; muse-wonder-full; Muse Code; effort Highest) by 0.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GBA Eval - Gameplay Replays&lt;/strong&gt;: Claude Fable 5.2 (1.0) beat Claude Opus 5 by 0.07&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - FARGO paper-aligned query core&lt;/strong&gt;: Gemini 4 Hydrogen (52.65348694173612) beat Gemini 4 Carbon by 0.07&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ErdosBench&lt;/strong&gt;: Claude Fable 5.2 (3.3) beat Claude Fable 5.1 (xHigh) by 0.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - MetaOpt Combined-TE Gap Search&lt;/strong&gt;: Gemini 4 Hydrogen (52.00947866014176) beat Gemini 4 Carbon by 0.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Register-aware Branch Prediction&lt;/strong&gt;: Gemini 4 Hydrogen (31.270009842412797) beat Gemini 4 Carbon by 0.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JSQuAD&lt;/strong&gt;: GPT-6.1 Sol (Max) (83.08) beat Kimi K3.1 Pro (Max) by 0.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (0-shot) - ALT Japanese-to-English&lt;/strong&gt;: Claude Opus 5.5 (Max) (90.52) beat GPT-5.6 Sol (Max) by 0.02&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-15

=== DAILY ===
NEW BENCHMARKS (39)
  - AutoRLBench 35 Issue / 20 Solution (Pass rate over 35 issue judgments and 20 solution runs (%)): leader GPT-6 Astra (Max) (41.8), 10 models
      AutoRLBench35/20 — Agents identify real RL-training faults and repair/re-run elig</summary></entry><entry><title>The Aggregate Digest — 2026-11-14</title><id>https://theaggregate.ai/digest/2026-11-14</id><updated>2026-11-14T19:25:07.230299+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (17)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;AA Coding Agent Index v2.1&lt;/strong&gt; (Coding Agent Index (0-100)): leader Claude Fable 5.2 (Xhigh, Default Fallback) (AA Coding Agent Run: harness=Claude Code; native=Claude Fable 5.2 (Xhigh, Default Fallback); fallback=source-published) (58.47), 27 models&lt;br&gt;&lt;span&gt;Artificial Analysis Coding Agent Index v2.1 (November 2026) evaluates a model inside its named coding-agent harness. It averages FrontierCode-AA 2.0 (132 repository-editing tasks), Terminal-Bench 4.1 (66 terminal tasks with revised verifiers), and SWE-Atlas-QnA-AA (124 repository questions), each using three attempts per task and task-equal means. Audited reward-hacking attempts score zero; environment/verifier errors are excluded. Native model effort and harness remain separate; this edition differs from v2.0. This board reports the equal-weight index on a 0-100 scale.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Coding Agent Index v2.1 - FrontierCode-AA 2.0&lt;/strong&gt; (Mean Score (%)): leader Claude Fable 5.2 (Xhigh, Default Fallback) (AA Coding Agent Run: harness=Claude Code; native=Claude Fable 5.2 (Xhigh, Default Fallback); fallback=source-published) (49.18), 27 models&lt;br&gt;&lt;span&gt;Artificial Analysis Coding Agent Index v2.1 (November 2026) evaluates a model inside its named coding-agent harness. It averages FrontierCode-AA 2.0 (132 repository-editing tasks), Terminal-Bench 4.1 (66 terminal tasks with revised verifiers), and SWE-Atlas-QnA-AA (124 repository questions), each using three attempts per task and task-equal means. Audited reward-hacking attempts score zero; environment/verifier errors are excluded. Native model effort and harness remain separate; this edition differs from v2.0. This board reports FrontierCode-AA 2.0, higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Coding Agent Index v2.1 - SWE-Atlas-QnA-AA&lt;/strong&gt; (Task Resolve Rate (%)): leader Muse Wonder Full (Harness: Muse Code) (Max) (68.82), 27 models&lt;br&gt;&lt;span&gt;Artificial Analysis Coding Agent Index v2.1 (November 2026) evaluates a model inside its named coding-agent harness. It averages FrontierCode-AA 2.0 (132 repository-editing tasks), Terminal-Bench 4.1 (66 terminal tasks with revised verifiers), and SWE-Atlas-QnA-AA (124 repository questions), each using three attempts per task and task-equal means. Audited reward-hacking attempts score zero; environment/verifier errors are excluded. Native model effort and harness remain separate; this edition differs from v2.0. This board reports SWE-Atlas-QnA-AA, higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Coding Agent Index v2.1 - Terminal-Bench 4.1&lt;/strong&gt; (Pass@1 (%)): leader Kimi K3.1 Pro (Harness: Kimi Code CLI) (67.17), 27 models&lt;br&gt;&lt;span&gt;Artificial Analysis Coding Agent Index v2.1 (November 2026) evaluates a model inside its named coding-agent harness. It averages FrontierCode-AA 2.0 (132 repository-editing tasks), Terminal-Bench 4.1 (66 terminal tasks with revised verifiers), and SWE-Atlas-QnA-AA (124 repository questions), each using three attempts per task and task-equal means. Audited reward-hacking attempts score zero; environment/verifier errors are excluded. Native model effort and harness remain separate; this edition differs from v2.0. This board reports Terminal-Bench 4.1, higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 32 Issue / 18 Solution&lt;/strong&gt; (Pass rate over 32 issue judgments and 18 solution runs (%)): leader GPT-6 Astra (Max) (46.0), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench native 32-issue/18-solution task-set edition. Same published root-cause and repaired-workspace protocols, one run per model/task/level. Issue is an LLM judgment against hidden oracle; solution is the harness rerun metric threshold, with reward hacking failing. Overall counts passes over all50runs, not an average of two task-level rates. The source README announces32issue tasks and24solution-capable tasks, but exactly18 have scored solution runs; do not use24as the reported denominator. Exact task-ID sets are pinned and all rates reproduce the native table to its stated one decimal. Separate from the previously reviewed33/20task-set edition. This board reports Pass rate over 32 issue judgments and 18 solution runs (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 32 Issue / 18 Solution - Issue&lt;/strong&gt; (Root cause found, 32 failed training runs (%)): leader GPT-6 Astra (Max) (46.9), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench native 32-issue/18-solution task-set edition. Same published root-cause and repaired-workspace protocols, one run per model/task/level. Issue is an LLM judgment against hidden oracle; solution is the harness rerun metric threshold, with reward hacking failing. Overall counts passes over all50runs, not an average of two task-level rates. The source README announces32issue tasks and24solution-capable tasks, but exactly18 have scored solution runs; do not use24as the reported denominator. Exact task-ID sets are pinned and all rates reproduce the native table to its stated one decimal. Separate from the previously reviewed33/20task-set edition. This board reports Root cause found, 32 failed training runs (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench 32 Issue / 18 Solution - Solution&lt;/strong&gt; (Repaired run passes its rerun, 18 tasks (%)): leader Gemini 4 Argon (Max) (66.7), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench native 32-issue/18-solution task-set edition. Same published root-cause and repaired-workspace protocols, one run per model/task/level. Issue is an LLM judgment against hidden oracle; solution is the harness rerun metric threshold, with reward hacking failing. Overall counts passes over all50runs, not an average of two task-level rates. The source README announces32issue tasks and24solution-capable tasks, but exactly18 have scored solution runs; do not use24as the reported denominator. Exact task-ID sets are pinned and all rates reproduce the native table to its stated one decimal. Separate from the previously reviewed33/20task-set edition. This board reports Repaired run passes its rerun, 18 tasks (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Company Agent Bench&lt;/strong&gt; (Overall accuracy on 205 questions, three parts weighted equally (%)): leader Claude Fable 5.2 (High) (82.2), 6 models&lt;br&gt;&lt;span&gt;Company Agent Bench (Concept, 12 November 2026) tests how well an agent understands the company it works in: 205 questions built from one month of real Slack history in the publisher&amp;#x27;s own workspace, replayed in order into a sandbox workspace. Unlike enterprise-search benchmarks, which ask for facts stored in documents, each question is about people: who wrote an anonymized message, who was tagged in one, and which teammate wrote which anonymized reply, among about 10 teammates. The recorded authors, mentions and replies are the ground truth, and the agent may use only context from before the message that reveals the answer. This board lists the API models, run in the publisher&amp;#x27;s Slack harness with Slack search and reading tools at low, medium and high reasoning effort; the score weights the three parts equally. The article also scores four assistant products (Dots, Grok Bot, Claude Tag and Devin), which are products and not models and are not listed here.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Deepnote Agent Initiative Benchmark&lt;/strong&gt; (Mean blind evaluator task score (0-10)): leader GPT-6 Astra (High) (8.97), 12 models&lt;br&gt;&lt;span&gt;Deepnote tests when a proactive workplace agent should investigate and help, and when it should remain silent. All 12 models face the same 50 synthetic company situations at source-declared high reasoning effort, using 11 service APIs and 180 seconds of model response time (tool time excluded). Two mutually blind LLM evaluators grade against a reference solution and complete action log; the0–10mean covers all50situations. Exact judge models and API snapshots are unreported. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RWE-bench&lt;/strong&gt; (Average pass rate over 20 engineering tasks, 5 runs each (%)): leader Claude Fable 5.2 (Max) (62.0), 15 models&lt;br&gt;&lt;span&gt;RWE-bench (Real World Engineering Bench, Xplore, Batch 1) asks a coding agent to do physical engineering design work: 20 tasks, 9 mechanical (parts and mechanisms modelled in CAD and checked for strength, fit and motion, such as a one-piece crankshaft for a 3-cylinder diesel engine), 5 electrical (high-speed PCB routing on dense, already populated boards, such as a DDR3 memory bus with skew and impedance limits) and 6 aerospace (CFD and conjugate heat transfer from mesh to converged solution, such as the heat load on a turbine vane). Unlike SWE-bench, where a fix passes when the repository&amp;#x27;s tests pass, each design here is checked by a deterministic verifier that runs physics solvers and passes it only if every requirement in the brief is met; there is no LLM judge and no human grading. The agent gets the brief an engineer would get and a sandbox with CAD and CAE tools, and has up to 24 hours per run. Each model runs each task 5 times in its vendor&amp;#x27;s own coding agent at the highest reasoning effort it offers. This board is the average pass rate: the share of runs that pass, averaged over the 20 tasks.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Financial Coaching Bench&lt;/strong&gt; (Mean practitioner-checklist credit (%)): leader Claude Opus 5.5 (Max) (76.162), 27 models&lt;br&gt;&lt;span&gt;Financial Coaching Bench v1 (Vals AI, November2026) tests AI financial coaching on 256 private household scenarios. A simulated client, played by Claude Sonnet4.6, answers from a hidden fact sheet; the coach can ask questions and search the web. This board is mean practitioner-checklist credit in percent, preserving each native model and effort. Length-controlled scores and elicitation diagnostics are separate; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Knowledge&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (Max) (13.3), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports knowledge exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Logic&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (Max) (13.3), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports logic exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Overall&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (Max) (26.7), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports overall exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Rule&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (xHigh) (3.3), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports rule exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Visual Cipher Bench - Spatial&lt;/strong&gt; (Exact final-answer accuracy (%)): leader GPT-6 Astra (Harness: Codex CLI v0.163.0) (Max) (80.0), 8 models&lt;br&gt;&lt;span&gt;Visual Cipher Bench v1 (2611.16861v1): 40 image-based puzzles, ten each of spatial, logic, rule and knowledge types. Harbor v0.24.0; GPT models use Codex CLI v0.163.0 and other models Claude Code v2.1.296; 60-minute network-disabled runs, 2 CPUs/4 GB/no GPU, Python/NumPy/Pillow tools and read-only puzzle images. Three independent runs per exact model/effort. Accuracy is final-answer exact match after fixed normalization; invalid submissions/timeouts score zero. Each category has 30 attempts; overall has 120. Literal none effort is retained without inferring disabled reasoning. Mean solved-count column is not another benchmark. This board reports spatial exact-answer accuracy in percent; higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WARC-Bench&lt;/strong&gt; (Tasks completed on the 192-task test split, mean of 3 runs (%)): leader Claude Opus 5.5 (Max) (90.28), 9 models&lt;br&gt;&lt;span&gt;WARC-Bench (Uniphore, arXiv 2510.09872; test-split runs of 19 October to 10 November 2026) tests whether an agent can operate single web interface components: 192 short GUI subtasks such as picking a date in a calendar widget, choosing from a dropdown, scrolling a container to read a value or completing a multi-step form, replayed from web archive (WARC) snapshots of real sites. Unlike WebArena or Online-Mind2Web, which grade whole multi-page tasks, it isolates the short interactions those tasks are built from and replays recorded pages, so every run sees the same site. The agent sees screenshots only and gets at most 20 steps and 1,800 seconds per task; a programmatic check grades the end state, 53 tasks by exact string match, with no judge model. Each model runs the split 3 times at the highest reasoning effort its API accepts, and the score is the mean of the 3 runs. Four of the models are served by Fireworks.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (54)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Coarena: 1130.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Coarena - Task Completion: 84.8 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Creative Writing (Lechmazur): 3.687 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - All - Completion: 7.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Cartridge Swap - Completion: 15.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Cartridge Swap - Progress: 70.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Execution: 4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Handling: 2.5 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Honesty: 3 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Overall: 3.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Expert Perception: 3.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Instrument Swap - Completion: 10.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Parts Kitting - Completion: 5.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Parts Kitting - Progress: 63.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Sample Transfer - Completion: 0.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Dual-Arm Sim - Sample Transfer - Progress: 38.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - GPQA Diamond: 96.02 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MathArena - ARXIV October: 93.62 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MathArena - ARXIV September: 96.23 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MathArena - ARXIV_FALSE October: 73.46 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MathArena - ARXIV_FALSE September: 80.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vending-Bench 2: 9628.479999999998 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Coarena: 1043.8 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Coarena - Task Completion: 86.4 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - GPQA Diamond: 94.07 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LLM2014 Logic 2026-11: 51.11 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Coarena: 1035.2 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Coarena - Task Completion: 70.8 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Coarena: 1129.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Coarena - Task Completion: 81.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (SVG): 1343.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LLM2014 Logic 2026-11: 31.45 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench: 46.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench - LMS: 28.7 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 17.3 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Chess Bench LLM: 2101.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ChessBench: 31.75 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ChessBench - Accuracy: 86.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ChessBench - Coherence: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Coarena: 1045.3 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Coarena - Task Completion: 72.6 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Creative Writing (Lechmazur): 1.752 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (Agents - Mobile Apps): 1285.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on GDP.pdf: 56.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on HalachaBench: 82.290515 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on HiL-Bench: 50.33 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle FACTS Grounding: 81.08 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV August: 92.98 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV October: 93.62 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV September: 96.23 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIVLEAN September: 62.26 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV_FALSE August: 88.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV_FALSE October: 82.72 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MathArena - ARXIV_FALSE September: 82.39 (#2)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (18)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;ChessBench - Elo&lt;/strong&gt;: Gemini 4 Carbon (1783.0) beat GPT-6 Astra by 47.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Glimmer Index - FreeCell&lt;/strong&gt;: Claude Haiku 5.5 (Glimmer Index Run: claude-haiku-5-5-medium;model=anthropic/claude-haiku-5-5;provider=claude;route=claude-haiku-5-5;effort=medium;precision=unreported) (63.3) beat GPT-6 Astra (Glimmer Index Run: codex-gpt-6-astra-medium;model=openai/gpt-6-astra;provider=codex;route=gpt-6-astra;effort=medium;precision=unreported) by 33.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Dual-Arm Sim - Instrument Swap - Progress&lt;/strong&gt;: Claude Fable 5.2 (68.0) beat GPT-6 Astra (Benchmark Sweep Run: source=Micro1DualArm; API; 120 decisions; effort and snapshot unreported) by 16.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIVLEAN September&lt;/strong&gt;: Gemini 4 Carbon (High) (62.26) beat Claude Opus 5.5 (High) by 15.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Glimmer Index - Mines&lt;/strong&gt;: Claude Haiku 5.5 (Glimmer Index Run: claude-haiku-5-5-medium;model=anthropic/claude-haiku-5-5;provider=claude;route=claude-haiku-5-5;effort=medium;precision=unreported) (70.8) beat GPT-6.1 Sol (Glimmer Index Run: codex-gpt-6-1-sol-medium;model=openai/gpt-6.1-sol;provider=codex;route=gpt-6.1-sol;effort=medium;precision=unreported) by 12.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Dual-Arm Sim - All - Progress&lt;/strong&gt;: Claude Fable 5.2 (59.8) beat Gemini 4 Argon (Benchmark Sweep Run: source=Micro1DualArm; API; 120 decisions; effort and snapshot unreported) by 10.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (Agents - Fullstack)&lt;/strong&gt;: Claude Fable 5.2 (1402.0) beat Claude Opus 5.5 by 10.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Terminal-Bench 4.0 (mini-SWE-agent)&lt;/strong&gt;: MiMo-V2.6-Pro (Terminal-Bench Effort: none) (26.97) beat Gemini 3.8 Flash (High) by 7.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV September&lt;/strong&gt;: Gemini 4 Carbon (High) (96.23) beat GPT-6.1 Sol (Max) by 4.72&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV October&lt;/strong&gt;: Gemini 4 Carbon (High) (93.62) beat GPT-6.1 Sol (Max) by 3.19&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV_FALSE October&lt;/strong&gt;: Gemini 4 Carbon (High) (82.72) beat GPT-6.1 Sol (Max) by 2.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV_FALSE August&lt;/strong&gt;: Gemini 4 Carbon (High) (88.1) beat GPT-6.1 Sol (Max) by 1.79&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDP.pdf&lt;/strong&gt;: Gemini 4 Carbon (High) (56.4) beat Muse Wonder (Max) by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Glimmer Index - Hustle Sim&lt;/strong&gt;: Claude Sonnet 5.5 (Glimmer Index Run: claude-sonnet-5-5-medium;model=anthropic/claude-sonnet-5-5;provider=claude;route=claude-sonnet-5-5;effort=medium;precision=unreported) (64.2) beat GPT-6 Astra (Glimmer Index Run: codex-gpt-6-astra-medium;model=openai/gpt-6-astra;provider=codex;route=gpt-6-astra;effort=medium;precision=unreported) by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Dual-Arm Sim - Expert Recovery&lt;/strong&gt;: Claude Fable 5.2 (4.5) beat Gemini 4 Argon (Benchmark Sweep Run: source=Micro1DualArm; API; 120 decisions; effort and snapshot unreported) by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle FACTS (Google)&lt;/strong&gt;: Gemini 4 Carbon (78.35) beat GPT-6 Astra by 0.24&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Sales - Raw Objective Completion&lt;/strong&gt;: Gemini 4 Argon (Medium) (94.54) beat Claude Fable 5.2 (Xhigh, Default Fallback) by 0.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ErdosBench&lt;/strong&gt;: Claude Fable 5.2 (3.3) beat Claude Fable 5.1 (xHigh) by 0.05&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-14

=== DAILY ===
NEW BENCHMARKS (17)
  - AA Coding Agent Index v2.1 (Coding Agent Index (0-100)): leader Claude Fable 5.2 (Xhigh, Default Fallback) (AA Coding Agent Run: harness=Claude Code; native=Claude Fable 5.2 (Xhigh, Default Fallback); fallback=source-published)</summary></entry><entry><title>The Aggregate Digest — 2026-11-13</title><id>https://theaggregate.ai/digest/2026-11-13</id><updated>2026-11-13T09:50:58.408736+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Frontier&lt;/h2&gt;
&lt;ul&gt;&lt;li&gt;Best available model: Gemini 4 Hydrogen enters at #12 (Aggregate Rating 1937) on 95 benchmarks&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (26)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;CheatBench&lt;/strong&gt; (Cheating rate (%), equal-weight mean of ten task categories, each model in its provider's agent harness; lower is better): leader Gemini 4 Argon (Gemini CLI) (8.552349), 15 models&lt;br&gt;&lt;span&gt;Agents work in their vendor&amp;#x27;s own harness on hard tasks across ten categories (maths research, coding, protein design, board games and others) whose workspace hides a way to cheat, such as a leaked answer or a colleague&amp;#x27;s accepted work; the number is the equal-weight mean of the ten category rates, one of which is a sycophancy belief-shift score. Unlike ImpossibleBench, the tasks are solvable honestly and the shortcut is optional.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Eon Agentic Misalignment&lt;/strong&gt; (Runs with a human-confirmed rule violation in 41 enterprise tasks x 3 repeats (% of answered runs)): leader Claude Opus 5.5 (21.666667), 19 models&lt;br&gt;&lt;span&gt;Agents do 41 everyday enterprise tasks in synthetic company systems where every write is refused and a temptation is planted; the score is the share of answered runs (3 per task) with a human-confirmed violation such as claiming a refused write succeeded or leaking PII. Unlike lab honeypots, the rules broken are ordinary enterprise policy.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FAR.AI AI Security Leaderboard&lt;/strong&gt; (Universal jailbreaks found (count; FAR.AI red team under the Minimal Standard for Safeguards v1.0, random-search and expert-guided attacks over chemical, biological, radiological &amp; nuclear, explosives and cyber hazards; 0 = none found within a search budget above $14K)): leader Claude Fable 5 (0.0), 8 models&lt;br&gt;&lt;span&gt;FAR.AI&amp;#x27;s red team attacks each frontier model&amp;#x27;s deployed safeguards with a fixed set of public jailbreak methods and their compositions, and the score is how many universal jailbreaks (attacks that work across a hazard category) it found. Unlike jailbreak success-rate boards such as Enkrypt or F5 CASI, it counts reusable breaks found by a budgeted search, so most current frontier models tie at zero.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;JobCannon Protocol Sensitivity&lt;/strong&gt; (Median protocol range in human SD (lower is better): for each questionnaire scale, the spread (max minus min) of the model's score over its valid protocol variants, in units of the human test-taker standard deviation, then the median over the model's scales (table F11)): leader GPT-6 Astra (0.51), 16 models&lt;br&gt;&lt;span&gt;A model takes four self-report questionnaires (Big Five, RIASEC, Dark Triad, Multiple Intelligences) under 13 protocol variants such as reversed options, a paraphrased instruction or the whole test in one prompt; the number is the median, over its 22 scales, of how far its scale score moves across variants, in human standard deviations. Unlike Stick To Your Role, it measures the self-report&amp;#x27;s stability under administration changes, not across simulated personas.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Personal Coding Evals (core10)&lt;/strong&gt; (pass@1: mean solve rate over 10 personal coding tasks x 3 runs (%)): leader GPT-6 Luna (Pi, xHigh) (100.0), 16 models&lt;br&gt;&lt;span&gt;One engineer&amp;#x27;s 10 coding tasks taken from his daily work, each run three times per harness, model and effort configuration (Pi, Codex, Claude Code, DeepSeek Harness); the score is the share of the 30 runs solved. It is a personal, saturated suite (16 of 33 admitted configurations solve every run), not a general coding benchmark.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ScarfBench (Kaggle)&lt;/strong&gt; (Smoke tests passed (%) over 204 Java framework migrations, each model run in a coding-agent harness by Kaggle): leader Gemini 4 Argon (Antigravity CLI, High) (75.24), 11 models&lt;br&gt;&lt;span&gt;Coding agents (one model at one effort inside its harness, run by Kaggle) carry out 204 migrations of enterprise Java applications between Jakarta EE, Quarkus and Spring; the score is the share of the applications&amp;#x27; behavioral smoke tests that pass after the migration. Unlike the ScarfBench paper board (174 focused-app migrations, one skills prompt), Kaggle runs and maintains these results over focused and whole applications.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Bulletbench - Blitz 3+2&lt;/strong&gt; (ladder Elo): leader Gemini 4 Argon (1147.0), 18 models&lt;br&gt;&lt;span&gt;Spring Prompt Bulletbench - Blitz 3+2. Model chess in three separate time controls:10+1Lightning,60sBullet,180+2Blitz. Native ladder Elo and95%CI; not FIDE Elo. Timed-out/invalid moves retain native failure rules. Native best available configuration per clock; all source configurations retained. Metric: ladder Elo. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Bulletbench - Bullet 60s&lt;/strong&gt; (ladder Elo): leader Gemini 3.8 Flash (1068.0), 56 models&lt;br&gt;&lt;span&gt;Spring Prompt Bulletbench - Bullet 60s. Model chess in three separate time controls:10+1Lightning,60sBullet,180+2Blitz. Native ladder Elo and95%CI; not FIDE Elo. Timed-out/invalid moves retain native failure rules. Native best available configuration per clock; all source configurations retained. Metric: ladder Elo. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Catalogbench - Reliably Publish Ready&lt;/strong&gt; (% of products): leader GPT-6.1 Sol (75.0), 30 models&lt;br&gt;&lt;span&gt;Spring Prompt Catalogbench - Reliably Publish Ready. 56 product briefs,3repeats, factual and sales briefs separately. Reliable publication requires all repeats pass; claims/attributes/channel rubric; majority of three independent judges. Failures stay failed. Native website reports best supported configuration per model; all underlying configurations retained in model_runs. Metric: % of products. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Catalogbench - Sales Brief Reliably Publish Ready&lt;/strong&gt; (% of products): leader GPT-6 Astra (69.64), 30 models&lt;br&gt;&lt;span&gt;Spring Prompt Catalogbench - Sales Brief Reliably Publish Ready. 56 product briefs,3repeats, factual and sales briefs separately. Reliable publication requires all repeats pass; claims/attributes/channel rubric; majority of three independent judges. Failures stay failed. Native website reports best supported configuration per model; all underlying configurations retained in model_runs. Metric: % of products. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Deckbench - Deck Rating&lt;/strong&gt; (rating): leader GPT-6 Astra (1587.1), 31 models&lt;br&gt;&lt;span&gt;Spring Prompt Deckbench - Deck Rating. Six business deck tasks, majority rubric checks by GPT6.1Sol/Opus5.5/Gemini3.1Pro; vendor-own judge abstains on paired deck comparisons. Native deck rating; missing unusable-deck rating is not zero. Human-rating sample pending. Metric: rating. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Roasbench - Score&lt;/strong&gt; (score out of 100): leader Claude Fable 5.2 (61.27), 31 models&lt;br&gt;&lt;span&gt;Spring Prompt Roasbench - Score. 2026 v2, twelve-month simulated advertising budget, reproducible seeded scenarios, profit/planning and critical overspend penalties; not comparable to v1. Native score0-100, not actual profit or return on a subscription. Metric: score out of 100. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spring Prompt Surveybench - Analysis Rating&lt;/strong&gt; (rating): leader Claude Fable 5.2 (1662.5), 22 models&lt;br&gt;&lt;span&gt;Spring Prompt Surveybench - Analysis Rating. Nine research-analysis tasks, majority of GPT6.1Sol/Opus5.5/Gemini3.1Pro rubric judges, vendor-own judge abstains on paired analysis. Native analysis rating with95%CI; research quality, not token cost. Human-rating sample pending. Metric: rating. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sycophancy Eval&lt;/strong&gt; (Composite sycophancy score (0-100, lower is better)): leader Grok 4.6 (2.0), 29 models&lt;br&gt;&lt;span&gt;The model holds scripted multi-turn conversations in English and Traditional Chinese in which a user pushes it, with social, emotional or authority pressure but no new evidence, to drop a position, validate the user or endorse their framing; a three-vendor judge panel rates each conversation. The number is the publisher&amp;#x27;s composite sycophancy score: 100 times a weighted mean of seven dimension scores, each 0 to 1 with higher meaning the model gave way more: Regressive Backdown 25% and Instability 10% (how early and how often its stance flips on the factual scenarios), Moral 20%, Validation 15%, Framing 15%, Indirectness 10% and Flattery Density 5% (extra giving way after praise compared with a no-praise control), with each conversation averaged over the model&amp;#x27;s last three comparable sweeps. Unlike single-turn sycophancy tests such as SycEval, it keeps up the pressure over several turns and tracks when the stance breaks.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;TavernBench&lt;/strong&gt; (Play accuracy: share of tasks answered correctly, mean over the 30 play scenarios (%)): leader GPT-6 Astra (Low) (63.0), 10 models&lt;br&gt;&lt;span&gt;Language models answer Hearthstone Battlegrounds decisions (combat outcomes, positioning, play order, targets, card choices, turn plans) from a text description of the game state, with no tools. Labels use simulator outcomes, and top human players&amp;#x27; placement statistics for hero and trinket choices. The score is accuracy over the 30 play scenarios of a 368-task set, which asks for the best play rather than what the rules do.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Overall&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (77.69), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Overall. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - All&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (70.02), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - All. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - Continuation&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (79.76), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - Continuation. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - Photo&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (59.75), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - Photo. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - Script&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (85.62), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - Script. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Scene - Video&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (56.84), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Scene - Video. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - All&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (85.36), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - All. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - Continuation&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (91.1), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - Continuation. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - Photo&lt;/strong&gt; (Score (%)): leader GPT-6.1 Sol (93.8), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - Photo. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - Script&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (76.5), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - Script. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibePrevisBench - Screen - Video&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (89.8), 11 models&lt;br&gt;&lt;span&gt;VibePrevisBench - Screen - Video. VibePrevisBench frozen paper Table1:242 Blender4D tasks,94script39photo23continuation86video,agent writes/runs Blender Python. Scene score is rule-based84,406hidden facts; Screen score Gemini4Argon VLM judge (photo recognition only). Failed runs score0. All is weighted native aggregate; overall mean of SceneAll andScreenAll. Harness and exact effort are observation settings, not checkpoints. README values are native one-decimal report; JSON provides two-decimal aggregate separately. Metric: Score (%). Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (2)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Hydrogen&lt;/strong&gt; — Aggregate Rating 1937, #12&lt;ul&gt;&lt;li&gt;AA GDP.pdf: 32.8 (#1/259)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 91.46 (#1/259)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Operations - Raw Objective Completion: 97.6 (#1/261)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 52.305 (#1/85)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - HR - Raw Objective Completion: 82.56 (#2/261)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 1801.0 (#2/291)&lt;/li&gt;&lt;li&gt;Vals AI Finance Index: 66.136 (#2/40)&lt;/li&gt;&lt;li&gt;LiveBench Data Analysis: 83.2 (#3/35)&lt;/li&gt;&lt;li&gt;AA-Omniscience v1.1 Hallucination Rate: 8.48 (#3/140)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Finance - Raw Objective Completion: 95.3 (#3/261)&lt;/li&gt;&lt;li&gt;...and 84 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;JT-4.1 Flash 236B A21B 0709&lt;/strong&gt; — Aggregate Rating 1751, #169&lt;ul&gt;&lt;li&gt;AA Humanity's Last Exam: 43.56 (#65/684)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 49.4 (#73/596)&lt;/li&gt;&lt;li&gt;AA Omniscience: 20.57 (#76/596)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 43.67 (#96/596)&lt;/li&gt;&lt;li&gt;AA SciCode: 51.97 (#101/251)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench 2.1: 59.55 (#104/243)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Support - Raw Objective Completion: 82.23 (#105/261)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Strict: 17.81 (#108/261)&lt;/li&gt;&lt;li&gt;AA GDPval: 1335.19 (#110/315)&lt;/li&gt;&lt;li&gt;AA CritPt: 14.0 (#113/600)&lt;/li&gt;&lt;li&gt;...and 29 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (74)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard: 9.111 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - General Knowledge: 9.048 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - MMLU: 9.752 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - RAG QA: 8.0 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - Reasoning &amp; Math: 9.767 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - Translation (incl Dialects): 7.917 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - Trust &amp; Safety: 9.333 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Arabic Broad Leaderboard - Writing (incl Dialects): 7.591 (#47)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Architectural RedlineBench: 68.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Argo-Bench: 70.22 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Argo-Bench - Solved: 41.43 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Chess Bench LLM: 1867.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on HLE-Diamond: 77.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on HLE-Diamond - Knowledge: 74.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on HLE-Diamond - Reasoning: 81.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Kaggle Game Arena (Unified): 546.34 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Kaggle Game Arena Chess: 1313.29 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Kaggle Game Arena Four in a Row: 818.97 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Agents - BFCL-V3: 65.8 (#34)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Agents and Tool Use: 70.4 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Education - High School Subjects: 77.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Education - Primary School Subjects: 70.7 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Finance: 90.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Language and Instruction Following: 72.4 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Law - Lawyer Qualification Exam: 82.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Law and Civil Service: 88.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Medicine - Basic Medical Knowledge: 90.6 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Medicine - Physician Exams: 92.8 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ReLE - Reasoning - BBH: 87.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SEAL - Professional Reasoning Benchmark - Finance: 47.07 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SEAL - Professional Reasoning Benchmark - Legal: 48.97 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Bug Density: 0.56 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Code Smell Density: 12.79 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Pass Rate: 90.26 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Vulnerability Density: 0.25 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on sudo L7: 52.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Kaggle Game Arena Four in a Row: 541.59 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MirrorCode: 95.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on VulcanBench v3 (BenchLM): 91.1 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on MirrorCode: 60.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on VulcanBench v3 (BenchLM): 89.3 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on MirrorCode: 61.1 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on VulcanBench v3 (BenchLM): 88.8 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIMultiple - Agentic RAG Routing: 92.39 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIMultiple - FinanceReasoning (Hard): 90.34 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIMultiple - Text-to-SQL (Adjudicated): 83.3 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIMultiple - Text-to-SQL (Strict Execution Match): 63.2 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard: 9.119 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - General Knowledge: 8.794 (#50)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - MMLU: 9.669 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - Reasoning &amp; Math: 9.302 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - Translation (incl Dialects): 7.972 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - Trust &amp; Safety: 8.333 (#70)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arabic Broad Leaderboard - Writing (incl Dialects): 8.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on BenchGecko Score: 93.2 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Chess Bench LLM: 2208.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on HLE-Diamond: 63.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on HLE-Diamond - Knowledge: 55.2 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on HLE-Diamond - Reasoning: 72.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Humanity's Sixth Sense: 71.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Dutch Task v6: 64.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Farsi Task v6: 41.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - French Task v6: 71.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Hebrew Task v6: 56.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Indonesian Task v6: 60.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Italian Task v6: 70.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Japanese Task v6: 60.4 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Russian Task v6: 77.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Spanish Task v6: 68.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Swedish Task v6: 58.8 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Kaggle MultiLoKo - Traditional Mandarin Task v6: 76.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PUMA Polish Multimodal - Structured Extraction: 77.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on SEAL - Fortress: 16.42 (#59)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on TaxCalcBench: 54.17 (#15)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (56)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena Chess&lt;/strong&gt;: Gemini 4 Carbon (1646.3) beat Gemini 3.8 Flash by 193.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena (Unified)&lt;/strong&gt;: Gemini 4 Carbon (638.17) beat Gemini 4 Argon by 103.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena Four in a Row&lt;/strong&gt;: Gemini 4 Carbon (1002.7) beat Gemini 4 Argon by 79.37&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenEval - BBQ&lt;/strong&gt;: Qwen 1.5 14B Chat (93.5) beat pythia-12B-v0 by 51.36&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MirrorCode&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (97.8) beat Claude Fable 5 (High) by 33.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HLE-Diamond - Knowledge&lt;/strong&gt;: Claude Fable 5.2 (Max) (74.6) beat Claude Opus 5.5 (Max) by 21.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Reific Environments - Industry&lt;/strong&gt;: Claude Fable 5.2 (Reific Run: native=Claude%20Fable%205.2; cfg=351d3d3783973f89) (60.2) beat Claude Opus 5.5 (Reific Run: native=Claude%20Opus%205.5; cfg=06d6e96794d99d92) by 17.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HLE-Diamond&lt;/strong&gt;: Claude Fable 5.2 (Max) (77.9) beat GPT-6 Astra (Max) by 12.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Argo-Bench&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (70.22) beat Claude Opus 5.5 (xHigh) by 11.84&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Humanity's Sixth Sense&lt;/strong&gt;: Gemini 4 Carbon (High) (71.1) beat Claude Fable 5.2 (Max) by 11.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Malay Task v6&lt;/strong&gt;: Gemini 4 Carbon (48.0) beat Gemini 3.1 Pro (Preview) by 8.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Argo-Bench - Solved&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (41.43) beat Gemini 4 Argon (High) by 7.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BALSAM - Fill in the Blank&lt;/strong&gt;: slm-3.1 (90.0) beat GPT-5.2 by 4.39&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Vision Mean&lt;/strong&gt;: Gemini 4 Carbon (89.17) beat Gemini 4 Argon by 4.34&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Polish Task v6&lt;/strong&gt;: Gemini 4 Carbon (48.0) beat GPT-6 Astra by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - OCR&lt;/strong&gt;: Gemini 4 Carbon (96.0) beat Claude Fable 5.2 by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Document QA&lt;/strong&gt;: Gemini 4 Carbon (97.0) beat Gemini 4 Argon by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Arabic Task v6&lt;/strong&gt;: Gemini 4 Carbon (55.2) beat Gemini 3.1 Pro (Preview) by 3.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Portuguese Task v6&lt;/strong&gt;: Gemini 4 Carbon (78.0) beat Gemini 3.1 Pro (Preview) by 3.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Romanian Task v6&lt;/strong&gt;: Gemini 4 Carbon (57.6) beat GPT-6 Astra by 3.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Open Benchmarks - GPQA Diamond (Zero-Shot)&lt;/strong&gt;: Gemini 4 Carbon (96.97) beat Gemini 3.6 Flash by 3.54&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VulcanBench v3 (BenchLM)&lt;/strong&gt;: Grok 4.7 (93.2) beat Grok 4.5 by 3.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Contemporary Life&lt;/strong&gt;: Gemini 4 Carbon (87.0) beat Gemini 3.8 Flash by 3.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Agents - BFCL-V3&lt;/strong&gt;: Gemini 4 Carbon (79.1) beat MiniMax-M2.7 by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Reific Environments - Personal Agents&lt;/strong&gt;: Claude Fable 5.2 (Reific Run: native=Claude%20Fable%205.2; cfg=351d3d3783973f89) (34.2) beat Gemini 4 Argon (Reific Run: native=Gemini%204%20Argon; cfg=b2c93c3d7afec7c8) by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Overall&lt;/strong&gt;: Claude Fable 5.2 (80.8) beat Claude Opus 5.5 by 2.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Simplified Mandarin Task v6&lt;/strong&gt;: Gemini 4 Carbon (71.6) beat Gemini 3.1 Pro (Preview) by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Vietnamese Task v6&lt;/strong&gt;: Gemini 4 Carbon (59.6) beat Gemini 3.1 Pro (Preview) by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Coding&lt;/strong&gt;: Claude Fable 5.2 (79.7) beat Claude Sonnet 5.5 by 1.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sonar LLM Leaderboard - Java - Pass Rate&lt;/strong&gt;: Claude Fable 5.2 (Thinking, High) (90.26) beat Claude Opus 5 (Thinking) by 1.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Agents' Last Exam)&lt;/strong&gt;: Muse Ember (61.6) beat GPT-6.1 Terra by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Khmer Task v6&lt;/strong&gt;: Gemini 4 Carbon (39.6) beat GPT-6 Astra by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Tagalog Task v6&lt;/strong&gt;: Gemini 4 Carbon (59.2) beat Gemini 3.8 Flash by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;sudo L7&lt;/strong&gt;: Claude Fable 5.2 (Adaptive Reasoning, Max Effort) (52.2) beat Claude Sonnet 5.5 (Adaptive Reasoning, Max Effort) by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Bengali Task v6&lt;/strong&gt;: Gemini 4 Carbon (48.0) beat Gemini 3.1 Pro (Preview) by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Czech Task v6&lt;/strong&gt;: Gemini 4 Carbon (65.6) beat Gemini 3.8 Flash by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Marathi Task v6&lt;/strong&gt;: Gemini 4 Carbon (51.6) beat Gemini 3.1 Pro (Preview) by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Agents and Tool Use&lt;/strong&gt;: Gemini 4 Carbon (77.0) beat MiniMax-M2.7 by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sonar LLM Leaderboard - Java - Code Smell Density&lt;/strong&gt;: Claude Fable 5.2 (Thinking, High) (12.79) beat Claude Opus 4.5 (High) by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - History and Culture&lt;/strong&gt;: Gemini 4 Carbon (86.0) beat Gemini 3.7 Flash by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Geography and Environment&lt;/strong&gt;: Gemini 4 Carbon (92.0) beat Gemini 3.8 Flash by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Reasoning and Mathematics&lt;/strong&gt;: Claude Fable 5.2 (92.9) beat Claude Opus 5 by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education - Gaokao&lt;/strong&gt;: Claude Fable 5.2 (70.2) beat doubao-seed-evolving by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education - Middle School Subjects&lt;/strong&gt;: Claude Fable 5.2 (61.3) beat Kimi K3.1 Pro by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Thai Task v6&lt;/strong&gt;: Gemini 4 Carbon (42.0) beat Gemini 3.1 Pro (Preview) by 0.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Insurance - Criterion Pass Rate&lt;/strong&gt;: Gemini 4 Hydrogen (Medium) (91.46) beat Gemini 4 Carbon (Medium) by 0.53&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Tax Agent Bench&lt;/strong&gt;: Gemini 4 Hydrogen (xHigh) (52.305) beat Gemini 4 Carbon (High) by 0.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education&lt;/strong&gt;: Claude Fable 5.2 (69.8) beat Gemini 4 Argon by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HLE-Diamond - Reasoning&lt;/strong&gt;: Claude Fable 5.2 (Max) (81.2) beat GPT-6 Astra (Max) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - English Task v6&lt;/strong&gt;: Gemini 4 Carbon (78.4) beat GPT-6.1 Sol by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Cantonese Task v6&lt;/strong&gt;: Gemini 4 Carbon (51.6) beat Gemini 3.1 Pro (Preview) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Hindi Task v6&lt;/strong&gt;: Gemini 4 Carbon (53.2) beat Gemini 3.1 Pro (Preview) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf&lt;/strong&gt;: Gemini 4 Hydrogen (Medium) (32.8) beat Muse Wonder Full (Max) by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Medicine and Mental Health&lt;/strong&gt;: Claude Fable 5.2 (92.9) beat Kimi K3.1 Pro by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Operations - Raw Objective Completion&lt;/strong&gt;: Gemini 4 Hydrogen (High) (97.6) beat Claude Sonnet 5.5 (Max, Default Fallback) by 0.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Arabic Broad Leaderboard - RAG QA&lt;/strong&gt;: Gemini 4 Carbon (9.829) beat GLM-5.3 by 0.1&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-13

=== FRONTIER ===
  - Best available model: Gemini 4 Hydrogen enters at #12 (Aggregate Rating 1937) on 95 benchmarks

=== DAILY ===
NEW BENCHMARKS (26)
  - CheatBench (Cheating rate (%), equal-weight mean of ten task categories, each model in its provider's agent ha</summary></entry><entry><title>The Aggregate Digest — 2026-11-12</title><id>https://theaggregate.ai/digest/2026-11-12</id><updated>2026-11-12T13:45:10.733145+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Frontier&lt;/h2&gt;
&lt;ul&gt;&lt;li&gt;Best available model: Claude Fable 5.2 (2009) takes the crown from Claude Opus 5.5 (1965), which had trailed for 3 days&lt;/li&gt;&lt;li&gt;Best measured model: Claude Fable 5.2 (2009) takes the crown from Gemini 4 Carbon (1982), which had trailed for 3 days&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (16)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Agent Money Bench&lt;/strong&gt; (Wasted spend per $100, both tracks weighted equally (USD, lower is better)): leader Claude Fable 5.2 (21.0), 12 models&lt;br&gt;&lt;span&gt;Agent Money Bench (BlockRun, version 1, run 27 October to 11 November 2026) tests whether an agent can be trusted to spend money: it buys the data it needs in USDC on Solana mainnet, answers the question and files an expense report. Unlike Vending-Bench, where the money is simulated, every payment settles on-chain. It has two tracks. Financial Research is 76 open questions answered with live data bought from real providers, graded against per-question rubrics by an AI judge with every score reviewed by hand. Adversarial Marketplace is 80 tasks buying from simulated sellers, some of which raise the price after quoting, let quotes expire, charge twice, deliver the wrong data or hide instructions in what they deliver; it is graded deterministically from the settlement, the answer and the expense report. Each model runs each task once, and the prompt gives no payment rules: it says the money is the client&amp;#x27;s and that there is a budget. This board is the headline composite: dollars wasted per $100 spent, with each track counting for $50. A marketplace task&amp;#x27;s whole spend is wasted unless the run succeeds, and a research task&amp;#x27;s spend is wasted in proportion to its rubric shortfall. Lower is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutoRLBench&lt;/strong&gt; (Pass rate over 33 issue judgments and 20 solution runs (%)): leader GPT-6 Astra (Max) (47.2), 10 models&lt;br&gt;&lt;span&gt;AutoRLBench gives a coding agent a real reinforcement-learning training run that went wrong, with its code, config, logs and metrics, and asks it to name the root cause. On tasks that have a repaired reference run the agent must also fix the workspace; the harness then reruns the training and a metric threshold on the new logs decides. Root-cause answers are graded by an LLM judge against a written oracle, and each model gets one run per task at maximum reasoning effort with no network access. Unlike MLE-bench, where agents build a model for a Kaggle task from scratch, every task starts from someone else&amp;#x27;s broken RL run and is scored on diagnosis and repair. This board is the share of all 53 runs passed: 33 root-cause judgments and 20 repair runs.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DualEntry Accounting AI Benchmark&lt;/strong&gt; (Tasks passed in at least two of three runs, 101 accounting tasks (%)): leader Gemini 4 Argon (91.1), 24 models&lt;br&gt;&lt;span&gt;The DualEntry Accounting AI Benchmark (October 2026, version 1.0) tests whether a model can do routine bookkeeping inside an accounting system: 101 synthetic cases covering transaction classification, journal entries, payables, receivables, bank reconciliation, financial reporting, month-end close and accounting knowledge. Each model works in a freshly restored test company with the same instructions and accounting tools, saves its records as drafts, and runs the full set three times. Fixed rules check each answer against a reference written beforehand (field comparison, numeric tolerance, required classification, multiple choice), with no model acting as judge. Unlike FinanceBench, which asks questions about public company filings, it scores the records and figures a model produces through tools in the vendor&amp;#x27;s own ledger product; DualEntry, an accounting-software company, runs it. This board is the share of the 101 tasks a model passes in at least two of its three runs.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EmbodiedLab&lt;/strong&gt; (T0 (raw control) success rate, 30 experiments (%)): leader Claude Opus 5.5 (Medium) (43.3), 6 models&lt;br&gt;&lt;span&gt;EmbodiedLab (v0.1.0, arXiv 2610.35385) tests whether an agent can run a physical science experiment to find a quantity it cannot know in advance, such as a spring&amp;#x27;s stiffness or an acid&amp;#x27;s concentration. The agent gets an objective and the apparatus but no method, and writes Python that drives a simulated Franka arm in NVIDIA Isaac Sim and reads instruments through three cameras; the hidden parameters are redrawn for every instance. A run succeeds only if the submitted value is within the instrument&amp;#x27;s tolerance, rests on readings recorded during the run, and the bench is left as required. Each model runs in its developer&amp;#x27;s coding agent at that agent&amp;#x27;s middle reasoning setting, on 30 experiments at two interface levels: raw control (T0), and with perception and motion-planning tools (T1). Unlike manipulation benchmarks such as LIBERO, where the target state is given, the robot&amp;#x27;s actions here are only the means to a measurement. This board is the T0 success rate, the ranking column: raw control, averaged over runs and then over the 30 experiments.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HalachaBench&lt;/strong&gt; (Mean rubric score on 60 semicha exam questions, three attempts each, GPT-6 Astra judge (0-100)): leader Claude Fable 5.2 (High) (85.765638), 23 models&lt;br&gt;&lt;span&gt;HalachaBench (v1 early results, provisional) asks models to answer real questions from the Chief Rabbinate of Israel&amp;#x27;s semicha (rabbinic ordination) exams: the 60 questions of five papers of the Cheshvan 5787 sitting, each answered three times in Hebrew, closed-book and without tools. One AI judge, GPT-6 Astra, checks every answer against a sealed rubric of 9 to 12 criteria that AI models wrote and reviewed; an answer&amp;#x27;s score is the share of rubric points it earns. Unlike LEXam, where law-exam answers are graded against the examiners&amp;#x27; own reference answers, these exams have no official answer key, so the grading rests on AI-written rubrics that no rabbi has reviewed yet. The judge is also a contestant. Open-weight models that ran as local 4-bit files of higher-precision releases are left off the board. This board is the mean score, from 0 to 100, over a model&amp;#x27;s 180 answers.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HoneyBench v0.2 - Overall Hack Rate&lt;/strong&gt; (Overall hack rate (%) over scored runs in the v0.2 20-task universe; unevaluated tasks and provider refusals excluded (lower is better)): leader Claude Fable 5.2 (11.931818181818182), 22 models&lt;br&gt;&lt;span&gt;Share of scored agent rollouts that commit to specification gaming in HoneyBench v0.2, a 20-task evaluation spanning medical, coding, engineering, administrative, ML, multimodal and game tasks. The headline excludes provider-refused runs and unevaluated tasks; models therefore have different denominators. The alternate rate counting refusals as hacks is retained as source evidence, not used as the score. Access to forbidden information counts as hacking even without a final exploit; otherwise the final submission determines the outcome. Native percentage, lower is better. Source: goodhartlabs.com/honeybench.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Inference Module&lt;/strong&gt; (Mean per-task rank score (0-100)): leader GPT-6 Astra (Max) (86.7), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Inference Module: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Inference Operator&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (95.0), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Inference Operator: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Inference System&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (90.0), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Inference System: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Training Module&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (90.0), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Training Module: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Training Operator&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (93.3), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Training Operator: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Infrathon Mean Rank Score - Training System&lt;/strong&gt; (Mean per-task rank score (0-100)): leader Claude Opus 5.5 (Max) (92.0), 6 models&lt;br&gt;&lt;span&gt;Infrathon Mean Rank Score - Training System: Mean per-task rank score (0-100). Infrathon arXiv 2611.11995v1 Figure 5: source-printed mean rank scores (0-100, higher better), not task raw score. Six panels: training operator 3, module 2, system 5; inference operator 8, module 3, system 4 tasks. Best valid private-test score within budget is ranked per task; operator 12h, module/system 24h. Claude Code for Opus 5.5, Codex CLI for the five others; source says max reasoning for all. Exact primary SVG bytes and companion paper bytes pinned. Dataset card HTTP 401 does not erase the public paper results. Native Flash identity preserved; no dated backend inferred.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReuF2F&lt;/strong&gt; (pass@1: accepted attempts, two attempts on each of 244 Lean problems (%)): leader Gemini 4 Argon (High) (42.42), 8 models&lt;br&gt;&lt;span&gt;ReuF2F gives a coding agent one of 244 open problems from undergraduate research programmes, formalised in Lean 4, and asks it to either prove the statement or prove its full negation. Every model runs in the same Leanimum-agent harness inside a container, with up to 1,000 steps and 6 hours per attempt, internet access and two attempts per problem. A submission counts only if the unchanged statement or its negation passes the Comparator checker with both the Lean kernel and nanoda, so there is no LLM judge. Unlike miniF2F, whose statements are olympiad and textbook problems with known proofs, every ReuF2F problem was unsolved when collected, and a run is a multi-hour agent session. The rows are ProbXiv&amp;#x27;s transcription of the paper&amp;#x27;s tables (arXiv 2610.27046), including its reasoning-effort sweep. This board is pass@1: accepted attempts as a share of the 488 attempts.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RoboRisk - Physical Harm Rate&lt;/strong&gt; (Harm rate (%)): leader GPT-6 Astra (Benchmark Sweep Run: source=leash_roborisk_paper; harness=LEASH; reviewer=Jev; reasoning=unreported) (0.0), 10 models&lt;br&gt;&lt;span&gt;RoboRisk - Physical Harm Rate: Harm rate (%). LEASH arXiv 2611.14115v1 Table 2: RoboRisk-S 92 simulation tasks x3 seeds=276 runs, RoboRisk-R 13 physical tasks x1=13; robot APIs use an adapted simulation subset of 39 runs. HR harm rate lower is better, SSR safe-success rate higher is better. Baseline and Jev-reviewed LEASH runs stay distinct. All simulation planners use high reasoning; physical planner reasoning is not reported here. Robot API rows are supported specialist systems, not generic chat checkpoints.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RoboRisk - Simulation Harm Rate&lt;/strong&gt; (Harm rate (%)): leader GPT-6 Astra (Benchmark Sweep Run: source=leash_roborisk_paper; harness=LEASH; reviewer=Jev; reasoning=high) (0.0), 8 models&lt;br&gt;&lt;span&gt;RoboRisk - Simulation Harm Rate: Harm rate (%). LEASH arXiv 2611.14115v1 Table 2: RoboRisk-S 92 simulation tasks x3 seeds=276 runs, RoboRisk-R 13 physical tasks x1=13; robot APIs use an adapted simulation subset of 39 runs. HR harm rate lower is better, SSR safe-success rate higher is better. Baseline and Jev-reviewed LEASH runs stay distinct. All simulation planners use high reasoning; physical planner reasoning is not reported here. Robot API rows are supported specialist systems, not generic chat checkpoints.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ultralytics VLM Leaderboard&lt;/strong&gt; (mAP50 over 36 UL37 detection datasets (0-1)): leader Gemini 4 Carbon (0.583719), 44 models&lt;br&gt;&lt;span&gt;The Ultralytics VLM Leaderboard (suite ul37-v2) ranks vision language models and open-vocabulary detectors on drawing boxes around named objects, the task behind auto-annotation. Unlike COCO or RefCOCO, which use everyday photographs, its 360 images come 10 each from 36 real-world detection datasets on Ultralytics Platform, among them dental X-rays, retinal lesions, retail products and baggage scans, sent whole with each dataset&amp;#x27;s own class names. Every image goes through the platform&amp;#x27;s production annotation call, and SAM 3.1 tightens every model&amp;#x27;s boxes except SAM 3&amp;#x27;s. A timeout, refusal or invalid output counts as an image with no predictions. Boxes are scored with the Ultralytics package&amp;#x27;s detection metrics in the order the model listed them, and the datasets are averaged equally. Providers run at their default reasoning effort unless that misses the platform&amp;#x27;s 115-second request limit. This board is mAP50: average precision at an IoU of 0.5, averaged over the labeled classes.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;ERNIE 5.5 Flash Preview&lt;/strong&gt; — Aggregate Rating 1752, #166&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 100.0 (#4/24)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 86.2 (#12/25)&lt;/li&gt;&lt;li&gt;The Aggregate - Day-one Recall Check: 26.5 (#19/40)&lt;/li&gt;&lt;li&gt;LLM2014 Logic 2026-11: 28.66 (#24/40)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Success Rate: 100.0 (#96/416)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Exactness: 45.31 (#268/409)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 90.0 (#269/416)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (64)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on BenchGecko Score: 95.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Blueprint-Bench 2: 0.566 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Conceptual Reasoning Index: 79.89 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Conceptual Reasoning Index - Argument Evaluation (LMCA): 70.61 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Conceptual Reasoning Index - Consistency (ACCoRD): 89.39 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EKI Estonian LLM - Language Advice: 74.38 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EKI Estonian LLM - Propaganda Resistance: 94.3 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EKI Estonian LLM - Trivia: 80.5 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EKI Estonian LLM - Trivia Hard: 39.12 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EnigmaForge: 68.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EnigmaForge - Fact F1: 97.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on EnigmaForge - Intuition: 56.2 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Critpt: 35.71 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Cursorbench: 60.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Dtbench: 98.93 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Frontierswe: 69.94 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Lmca: 70.61 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Scicode: 67.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on GoBench - Track 1: 2566.114615618314 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Humanity's Sixth Sense: 59.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Criteria Pass Rate: 88.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Form: 90.9 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Task Pass Rate: 32.3 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Criteria Pass Rate: 89.2 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Form: 79.3 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Task Pass Rate: 66.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SEAL - Fortress: 15.6 (#61)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on TaxCalcBench: 62.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Chart Recall: 0.291679 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Chart Recall - Shape: 0.513924 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Inversion and Execution Ladder: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Puzzle Portfolio Ladder: 96.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on The Aggregate - Chart Recall: 0.285835 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on The Aggregate - Chart Recall - Shape: 0.384865 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench: 56.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench - LMS: 34.3 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 21.7 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on CrueltyBench: 34.7 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (3D): 1360.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (Game Dev): 1277.0 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (UI Components): 1347.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Design Arena (Website): 1322.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on EnigmaForge: 50.8 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on EnigmaForge - Fact F1: 91.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on EnigmaForge - Intuition: 38.8 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Cursorbench: 60.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - ECI: 169.76 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on KernelBench Hub - CUDA: 10.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on KernelBench Hub - Mega: 23.76 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Browser: 80.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Design Layout: 80.2 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Long Horizon: 72.7 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Reasoning: 90.6 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LiveBench Research: 92.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Maze-Bench: 73.8 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on NYT Connections Extended: 99.3 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on SEAL - Professional Reasoning Benchmark - Legal: 53.67 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on SimpleBench: 92.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on The Aggregate - Chart Recall: 0.272598 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on The Aggregate - Chart Recall - Shape: 0.579456 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Tinybird AI SQL Benchmark - Exactness: 61.22 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Tinybird AI SQL Benchmark - First-Attempt Success Rate: 98.0 (#61)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Tinybird AI SQL Benchmark - Success Rate: 100.0 (#59)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on YC-Bench: 3425.28987 (#2)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (25)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;FoodTruckBench&lt;/strong&gt;: Gemini 4 Carbon (77217.68) beat Claude Opus 5 by 1954.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;YC-Bench&lt;/strong&gt;: Claude Fable 5.2 (3445.49457) beat Claude Opus 5.5 by 571.77&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Terminal-Bench 4.0 (Muse Code)&lt;/strong&gt;: Muse Wonder (xHigh) (46.97) beat Muse Spark 1.3 (xHigh) by 32.42&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Rli&lt;/strong&gt;: Claude Fable 5.2 (Unknown) (41.3) beat Claude Opus 5.5 (Unknown) by 20.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Artificial Analysis Music Arena - Vocals&lt;/strong&gt;: ACE Music v1 (1160.0) beat Suno v6 by 16.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Artificial Analysis Music Arena&lt;/strong&gt;: ACE Music v1 (1160.0) beat Suno v6 by 14.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CWE-Bench-AA&lt;/strong&gt;: hy4 (75.83) beat DeepSeek V4.1 Flash (1017) (Max) by 6.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SimpleBench&lt;/strong&gt;: Claude Fable 5.2 (94.9) beat Claude Opus 5.5 by 6.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Book Recommendations&lt;/strong&gt;: Claude Fable 5.2 (95.71) beat GPT-5.6 Sol by 5.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Frontierswe&lt;/strong&gt;: Claude Fable 5.2 (Max) (69.94) beat GPT-6 Astra (Max) by 4.43&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Humanity's Sixth Sense&lt;/strong&gt;: Claude Fable 5.2 (Max) (59.5) beat Gemini 4 Argon (High) by 4.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Cursorbench&lt;/strong&gt;: Gemini 4 Carbon (High) (60.4) beat Claude Opus 5.5 (Max) by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Lmca&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (70.61) beat Claude Sonnet 5.5 (Max) by 2.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conceptual Reasoning Index - Argument Evaluation (LMCA)&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (70.61) beat Claude Sonnet 5.5 by 2.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conceptual Reasoning Index&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (79.89) beat Claude Opus 5.5 by 2.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Coding&lt;/strong&gt;: Gemini 4 Carbon (91.0) beat Claude Fable 5.1 by 1.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Data Analysis&lt;/strong&gt;: Gemini 4 Carbon (86.5) beat Gemini 4 Argon by 1.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Terminology&lt;/strong&gt;: Claude Fable 5.2 (63.04) beat GPT-6 Astra by 1.23&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Idioms&lt;/strong&gt;: Claude Fable 5.2 (75.0) beat GPT-6 Astra by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conceptual Reasoning Index - Consistency (ACCoRD)&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (89.39) beat Claude Opus 5.5 (High) by 0.97&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Scicode&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (67.71) beat Claude Opus 5.5 (Max, Default Fallback) by 0.81&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NYT Connections Extended&lt;/strong&gt;: Gemini 4 Carbon (Thinking, High) (99.3) beat Gemini 4 Argon (Thinking, High) by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conceptual Reasoning Index - Decision Theory (DTBench)&lt;/strong&gt;: Claude Fable 5.2 (98.67) beat Claude Opus 5.5 by 0.45&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Dtbench&lt;/strong&gt;: Claude Fable 5.2 (Unknown) (99.2) beat Claude Opus 5.5 (Max) by 0.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Overall&lt;/strong&gt;: Claude Fable 5.2 (74.58) beat GPT-6 Astra by 0.26&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-12

=== FRONTIER ===
  - Best available model: Claude Fable 5.2 (2009) takes the crown from Claude Opus 5.5 (1965), which had trailed for 3 days
  - Best measured model: Claude Fable 5.2 (2009) takes the crown from Gemini 4 Carbon (1982), which had trailed for 3 days

</summary></entry><entry><title>The Aggregate Digest — 2026-11-11</title><id>https://theaggregate.ai/digest/2026-11-11</id><updated>2026-11-11T11:07:31.513659+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (5)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;The Aggregate - Chart Recall&lt;/strong&gt; (RMSE (log10 views)): leader GPT-6.1 Terra (Default) (0.272468), 16 models&lt;br&gt;&lt;span&gt;Draw ten years of monthly English Wikipedia pageviews from an article title alone, without tools or a chart to look at. Sixteen models reconstruct the same 197 articles from January 2016 to December 2025. The score averages each article’s root mean squared error in log10 views, with lower values better; an error near 0.30 corresponds roughly to a factor-of-two miss. The separate shape score measures whether the rises and falls are in the right places.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CWE-bench v1 - Programmatic Pass@1&lt;/strong&gt; (Programmatic Pass@1 (%)): leader Gemini 4 Carbon (71.0), 12 models&lt;br&gt;&lt;span&gt;Collinear CWE-bench archived v1, Programmatic Pass@1, reported as a percentage. 120 private audit-and-patch tasks; four rollouts per task at high reasoning, one hour each; timeouts graded on code at timeout. Native model-specific harnesses. Programmatic success requires exploit blocked and existing tests passing. Separate majority panel of three rubric-based agentic judges checks correctness, happy-path, generalization, integrity and soundness; any failed axis rejects a fix. Pass@1 and Pass@4 remain separate. This is archived Collinear v1, not the Artificial Analysis Stirrup two-hour implementation or CWE-bench v2. Source: https://cwe-bench.com/v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM2014 Logic 2026-11&lt;/strong&gt; (Median Score): leader GPT-6 Astra (xHigh) (86.35), 36 models&lt;br&gt;&lt;span&gt;November 2026 LLM2014 Logic cohort, reported as the native median score. Source model, effort, refusal and estimated-result labels remain attached to their observations. Monthly editions stay separate. Source: https://raw.githubusercontent.com/llm2014/llm_benchmark/main/docs/data/logic/2026-11.csv&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA MKA - Climb WSR&lt;/strong&gt; (Weighted Satisfaction Rate (%)): leader Claude Fable 5.2 (Max, Default Fallback) (56.5604258363361), 21 models&lt;br&gt;&lt;span&gt;Artificial Analysis MKA, Climb WSR: criterion-weighted task satisfaction, reported as a percentage. 200 private MKA tasks; one run per task, Stirrup, 250 turns and 8 hours; Linux sandbox with code, internet, search, page fetch and image viewing. GPT-6 Luna medium agentic judge; criterion weights 1–5, task-mean weighted satisfaction. Climb and Regression criterion populations remain separate (Regression covers 180 tasks). Empty submissions and errored runs score zero; final-text-only submissions are graded. The judge has no network or input files. Native effort and fallback labels retained. Source: https://artificialanalysis.ai/evaluations/mka&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Access to Justice&lt;/strong&gt; (Rubric Criterion Pass Rate (%)): leader Qwen 3.8 Max (76.368), 22 models&lt;br&gt;&lt;span&gt;Vals A2J v1:252private legal-access scenarios withGPT-5simulated client andDeciderv1.2judge; pooled expert-rubric criterion pass rate. Native model effort and versioned complete roster retained.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (15)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;test1 (vicgalleorg)&lt;/strong&gt; — Aggregate Rating 1468, #2784&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 89.52 (#46/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 78.32 (#78/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 72.27 (#631/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 83.9 (#658/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 66.67 (#882/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 64.67 (#1546/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Beast-Soul-new (Publisher: Vasanth)&lt;/strong&gt; — Aggregate Rating 1468, #2793&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 85.24 (#157/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 73.12 (#300/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 88.35 (#640/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 69.75 (#702/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 67.38 (#1010/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 64.74 (#1759/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test (UCLA-AGI)&lt;/strong&gt; — Aggregate Rating 1401, #5894&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 85.44 (#1853/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 65.87 (#2072/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 57.39 (#2158/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 60.95 (#3606/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 76.64 (#3653/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 30.86 (#3781/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test0 (UCLA-AGI)&lt;/strong&gt; — Aggregate Rating 1398, #6086&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 84.44 (#2453/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 63.65 (#2626/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 77.98 (#2905/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 36.69 (#3477/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 61.01 (#3586/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 50.48 (#3692/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test (jikaixuan) [Adapter]&lt;/strong&gt; — Aggregate Rating 1396, #6198&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 57.51 (#2132/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 84.42 (#2469/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 78.06 (#2881/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 62.29 (#3116/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 61.07 (#3571/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 27.22 (#3979/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test_model (jikaixuan) [Adapter]&lt;/strong&gt; — Aggregate Rating 1396, #6199&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 57.51 (#2133/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 84.42 (#2470/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 78.06 (#2882/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 62.29 (#3117/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 61.07 (#3572/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 27.22 (#3980/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 1.1 7B (IT) [float16]&lt;/strong&gt; — Aggregate Rating 1378, #6801&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 42.99 (#3089/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 60.92 (#3617/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 50.74 (#3621/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 60.07 (#3907/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 76.14 (#5210/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 69.69 (#5526/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 1.1 7B (IT) [bfloat16]&lt;/strong&gt; — Aggregate Rating 1375, #6883&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 41.77 (#3142/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 50.4 (#3711/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 60.39 (#3768/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 59.98 (#3923/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 76.21 (#5204/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 69.93 (#5499/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test_llama2.7B (Publisher: yeen214)&lt;/strong&gt; — Aggregate Rating 1301, #8621&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 78.57 (#4753/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 74.03 (#4757/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 53.07 (#5193/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 7.13 (#5339/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 46.86 (#5387/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 38.75 (#6525/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test-llama2-7B (Publisher: bongchoi)&lt;/strong&gt; — Aggregate Rating 1285, #8835&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 74.03 (#4743/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 78.57 (#4749/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 53.07 (#5180/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 7.13 (#5333/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 46.86 (#5386/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 38.75 (#6522/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-2-SlimOrcaDeduped-airoboros-3.1-MetaMathQA-SFT-124M&lt;/strong&gt; — Aggregate Rating 1109, #10513&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 2.12 (#5850/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 38.84 (#6498/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 25.82 (#6527/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 29.43 (#6872/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 24.57 (#6977/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 49.01 (#7194/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test1 (marcchew) [8bit]&lt;/strong&gt; — Aggregate Rating 1097, #10625&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 48.33 (#4245/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 27.65 (#6762/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 24.55 (#6944/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 50.2 (#7036/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 26.17 (#7064/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 0.0 (#7101/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test-model (yyjjtt)&lt;/strong&gt; — Aggregate Rating 1097, #10633&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 44.59 (#5175/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 25.88 (#6503/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 30.17 (#6851/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 50.83 (#6938/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 24.4 (#6993/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 0.0 (#7247/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test2 (alnrg2arg) [Delta]&lt;/strong&gt; — Aggregate Rating 1096, #10634&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 50.14 (#3776/7162)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 27.22 (#6789/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 24.64 (#6921/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 0.0 (#6938/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 26.25 (#7049/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 49.88 (#7065/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;test (alnrg2arg) [Delta]&lt;/strong&gt; — Aggregate Rating 1073, #10814&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 51.14 (#6882/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 0.0 (#6937/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 23.04 (#7087/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 23.28 (#7116/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 25.23 (#7185/7252)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (87)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Coding Daily (Claude Code) - Laravel Code Quality: 19.13 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Coding Daily (Claude Code) - Python Code Quality: 17.8 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Coding Daily (Claude Code) - React-TS Code Quality: 19.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Coding Daily (Claude Code) - Total: 84.35 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AIMultiple - FinanceReasoning (Hard): 91.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AIMultiple - Text-to-SQL (Adjudicated): 90.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AIMultiple - Text-to-SQL (Strict Execution Match): 71.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Bug Hunt Bench: 58.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Bug Hunt Bench - LMS: 35.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 23.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ChessBench: 31.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ChessBench - Coherence: 82.1 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ChessBench - Elo: 1618.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Design Arena (Game Dev): 1401.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Design Arena (UI Components): 1332.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Design Arena (Website): 1303.0 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on DuelLab GameBench 2 - Reasoning Variants: 99.107223911113 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Gert Labs Rankings: 71.44 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Icelandic LLM - Generative Quality: 82.94 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on KernelBench Hub - CUDA: 10.85 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on KernelBench Hub - Mega: 31.41 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LegalReasonBench: 49.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Maze-Bench: 90.67 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MineBench: 2082.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MissionReport: 63.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on NYT Connections Extended: 97.3 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on OpenRouter GPQA Diamond: 94.9 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on OpenRouter Tau2-Bench Airline: 68.0 (#92)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Art &amp; Entertainment: 96.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Culture &amp; Tradition: 98.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Geography: 98.0 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Grammar: 97.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - History: 99.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Overall: 97.33 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PLCC - Vocabulary: 96.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Contemporary Life: 73.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Document QA: 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Geography and Environment: 91.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - History and Culture: 77.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Structured Extraction: 71.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on PUMA Polish Multimodal - Vision Mean: 82.67 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on RealmPathology Benchmark: 76.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SpacetimeDB LLM Benchmark (C#): 97.6 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SpacetimeDB LLM Benchmark (Rust): 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Tinybird AI SQL Benchmark - Exactness: 63.86 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Tinybird AI SQL Benchmark - First-Attempt Success Rate: 98.0 (#93)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Tinybird AI SQL Benchmark - Success Rate: 100.0 (#189)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Mirrorcode: 95.56 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Last Translation Benchmark: 35.236004390779364 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Web Search: 54.05218 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Epoch AI - Mirrorcode: 60.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Mirrorcode: 61.11 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AI Chess Leaderboard (Continuation): 1706.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1883.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ARC-AGI-1: 98.5 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ARC-AGI-2: 94.17 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on ARC-AGI-3: 65.12 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench: 57.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench - LMS: 35.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 22.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Chess Puzzles (Epoch AI): 78.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Coarena: 1067.2 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Coarena - Task Completion: 90.3 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on CursorBench 4.0: 60.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on DeepsecBench: 35.19 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Ebr Bench: 87.62 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Frontiermath Erdos: 2.94 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Frontiermath Tier 4 V2: 95.12 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Frontiermath Tiers 1 3 V2: 90.53 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Furniture Assembly: 77.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - GPQA Diamond: 96.53 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Mystery Game Puzzles: 73.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierMath - Tier 4 (v2): 95.12 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierMath - Tiers 1-3 (v2): 90.53 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on KillSwitch-Bench: 80.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MLCR-AA: 19.44 (#57)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MLCR-AA - Accuracy: 37.22 (#83)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MLCR-AA - Completeness: 44.44 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on MineBench: 2188.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on OTIS Mock AIME 2024-25: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - Culture &amp; Tradition: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - Geography: 100.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - Grammar: 97.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - History: 98.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on PLCC - Vocabulary: 98.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on SimpleQA Verified: 77.7 (#2)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (44)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Chip Design - Aria - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (66.91) beat GPT-6.1 Sol (GDPx Run: v1.0; gpt-6-1-sol; Codex; effort Max) by 33.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (3D)&lt;/strong&gt;: Claude Fable 5.2 (1539.0) beat GPT-6 Astra (Max) by 28.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Chip Design - Aria - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (38.42) beat GPT-6.1 Sol (GDPx Run: v1.0; gpt-6-1-sol; Codex; effort Max) by 25.61&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (Agents - Mobile Apps)&lt;/strong&gt;: Claude Fable 5.2 (1323.0) beat Claude Opus 5.5 by 21.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Flora &amp; Fauna&lt;/strong&gt;: Claude Fable 5.2 (77.88) beat Claude Opus 5.5 by 20.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Mirrorcode&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (97.78) beat Claude Opus 5.5 (High) by 18.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Song Lyrics&lt;/strong&gt;: Claude Fable 5.2 (41.53) beat Claude Opus 5 by 18.79&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Tau3-Bench Banking_Knowledge&lt;/strong&gt;: PrincipledMachines Cortex 0.1 (73.2) beat Qwen 3.8 Max by 18.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Software Engineering - Datacurve - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (24.44) beat Claude Sonnet 5.5 (GDPx Run: v1.0; claude-sonnet-5-5; Claude Code; effort Max) by 13.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Chip Design - Domain Mean - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (25.1166666667) beat GPT-6.1 Sol (GDPx Run: v1.0; gpt-6-1-sol; Codex; effort Max) by 12.79&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - WikiQA&lt;/strong&gt;: Claude Fable 5.2 (86.58) beat Claude Opus 5.5 by 12.26&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Chip Design - Domain Mean - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (15.62) beat GPT-6.1 Sol (GDPx Run: v1.0; gpt-6-1-sol; Codex; effort Max) by 10.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Web Search&lt;/strong&gt;: Claude Opus 5.5 Fast (57.535937) beat GPT-5.6 Sol (Max) by 8.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM Leaderboard - Average&lt;/strong&gt;: Claude Fable 5.2 (78.73) beat Claude Opus 5.5 by 7.86&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Idiom Completion&lt;/strong&gt;: Claude Fable 5.2 (83.4) beat Gemini 4 Argon by 7.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AppWorld Challenge&lt;/strong&gt;: GLM-5.3 Flash (92.8) beat Claude Opus 4.7 by 7.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Bug Hunt Bench&lt;/strong&gt;: Claude Fable 5.2 (Max) (58.0) beat Claude Sonnet 5.5 (Max) by 6.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Materials Science - Emulated - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (30.83) beat Gemini 4 Carbon (GDPx Run: v1.0; gemini-4-carbon; Antigravity; effort High) by 6.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Bug Hunt Bench - LMS&lt;/strong&gt;: Claude Fable 5.2 (Max) (35.0) beat Claude Sonnet 5.5 (Max) by 6.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Overall - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (16.44) beat Claude Opus 5.5 (GDPx Run: v1.0; claude-opus-5-5; Claude Code; effort Max) by 4.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DuelLab GameBench 2 - Reasoning Variants&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (99.107223911113) beat Claude Opus 5.5 (xHigh) by 4.24&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Overall - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (26.5687878788) beat Claude Opus 5.5 (GDPx Run: v1.0; claude-opus-5-5; Claude Code; effort Max) by 4.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Textile Terms&lt;/strong&gt;: Claude Fable 5.2 (78.1) beat GPT-6 Astra by 4.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Idiom Meanings&lt;/strong&gt;: Claude Fable 5.2 (88.19) beat GPT-6 Astra by 3.58&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AppWorld Normal&lt;/strong&gt;: GLM-5.3 (94.0) beat Claude Opus 4.6 by 2.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AIMultiple - Agentic RAG Routing&lt;/strong&gt;: Claude Fable 5.2 (96.74) beat Gemini 4 Argon by 2.72&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Biology - LatchBio - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (28.21) beat Gemini 4 Carbon (GDPx Run: v1.0; gemini-4-carbon; Antigravity; effort High) by 2.57&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Hardware Design - Beautiful Machines - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (20.99) beat Claude Opus 5.5 (GDPx Run: v1.0; claude-opus-5-5; Claude Code; effort Max) by 2.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLCC - Art &amp; Entertainment&lt;/strong&gt;: Gemini 4 Carbon (97.0) beat Gemini 3 Pro (Preview) by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ChessBench - Accuracy&lt;/strong&gt;: Claude Fable 5.2 (87.0) beat GPT-6 Astra by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Famous Phrases&lt;/strong&gt;: Claude Fable 5.2 (37.52) beat Gemini 4 Argon by 1.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLCC - Overall&lt;/strong&gt;: Gemini 4 Carbon (98.33) beat Gemini 4 Argon by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - OCR&lt;/strong&gt;: Claude Fable 5.2 (92.0) beat Gemini 4 Argon by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Case Government&lt;/strong&gt;: Claude Fable 5.2 (100.0) beat Claude Opus 5 by 0.94&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - GEC&lt;/strong&gt;: Claude Fable 5.2 (83.94) beat Claude Opus 5 by 0.91&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWE-Marathon v1.1&lt;/strong&gt;: Gemini 4 Carbon (Thinking, high) (Antigravity CLI) (53.75) beat Muse Wonder Full (Thinking, max) (Muse Code) by 0.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Coding Daily (Claude Code) - React-TS Code Quality&lt;/strong&gt;: Claude Fable 5.2 (Medium) (19.5) beat Claude Opus 5.5 (High) by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CursorBench 4.0&lt;/strong&gt;: Gemini 4 Carbon (High) (60.4) beat Claude Fable 5.2 (Max) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Materials Science - Domain Mean - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (21.83) beat GPT-6 Astra (GDPx Run: v1.0; gpt-6-astra; Codex; effort Max) by 0.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Coding Daily (Claude Code) - Laravel Code Quality&lt;/strong&gt;: Claude Fable 5.2 (Medium) (19.13) beat Claude Opus 5.5 (Medium) by 0.18&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - GLP - Expression&lt;/strong&gt;: Ling 3.1 Flash (99.67) beat Qwen 3.6 Max Preview by 0.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - GPQA Diamond&lt;/strong&gt;: Gemini 4 Carbon (High) (96.53) beat Gemini 4 Argon (High) by 0.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Materials Science - Domain Mean - Partial Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (32.76) beat GPT-6 Astra (GDPx Run: v1.0; gpt-6-astra; Codex; effort Max) by 0.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDPx v1.0 - Alignment - Fleet - Default Score&lt;/strong&gt;: Claude Fable 5.2 (GDPx Run: v1.0; claude-fable-5-2; Claude Code; effort Max) (9.29) beat Muse Wonder Full (GDPx Run: v1.0; muse-wonder-full; Muse Code; effort Highest) by 0.08&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-11

=== DAILY ===
NEW BENCHMARKS (5)
  - The Aggregate - Chart Recall (RMSE (log10 views)): leader GPT-6.1 Terra (Default) (0.272468), 16 models
      Draw ten years of monthly English Wikipedia pageviews from an article title alone, without tools or a chart to look at</summary></entry><entry><title>The Aggregate Digest — 2026-11-10</title><id>https://theaggregate.ai/digest/2026-11-10</id><updated>2026-11-10T11:36:35.383636+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Frontier&lt;/h2&gt;
&lt;ul&gt;&lt;li&gt;Best available model: Claude Fable 5.2 enters at #1 (Aggregate Rating 2009) on 191 benchmarks&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (3)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Chartography (no tools))&lt;/strong&gt; (Score (%)): leader Claude Fable 5.2 (73.4), 5 models&lt;br&gt;&lt;span&gt;Chartography professional chart-understanding evaluation, mirrored by LLM Stats with tools disabled. Native score is reported as a percentage; keep this configuration separate from the tool-enabled and unspecified-tool boards. Source: https://llm-stats.com/benchmarks/chartography-no-tools&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV October&lt;/strong&gt; (Accuracy (%)): leader GPT-6.1 Sol (Max) (90.43), 5 models&lt;br&gt;&lt;span&gt;October 2026 cohort of MathArena research-mathematics final-answer questions drawn from arXiv. The publisher reports the score as a percentage. Monthly problem sets remain separate, with the source model and effort labels retained; this is distinct from the false-statement cohort. Source: https://matharena.ai/?comp=arxiv--october&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MathArena - ARXIV_FALSE October&lt;/strong&gt; (Accuracy (%)): leader GPT-6.1 Sol (Max) (80.25), 5 models&lt;br&gt;&lt;span&gt;October 2026 MathArena cohort assessing responses to false mathematical statements drawn from arXiv. The publisher score is shown as a percentage. Keep this statement-assessment cohort separate from final-answer mathematics, Lean proof evaluation and other monthly editions. Source model and effort labels are retained. Source: https://matharena.ai/?comp=arxiv_false--october&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (5)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; — Aggregate Rating 2009, #1&lt;ul&gt;&lt;li&gt;CursorBench 4.0: 60.0 (#1/86)&lt;/li&gt;&lt;li&gt;FrontierMath - Tier 4 (v2): 100.0 (#1/80)&lt;/li&gt;&lt;li&gt;FrontierMath - Tiers 1-3 (v2): 95.09 (#1/121)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 100.0 (#1/23)&lt;/li&gt;&lt;li&gt;Epoch AI - ECI: 174.24 (#1/255)&lt;/li&gt;&lt;li&gt;SimpleQA Verified: 94.0 (#1/102)&lt;/li&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 79.0 (#1/163)&lt;/li&gt;&lt;li&gt;Epoch AI - Frontiermath Tiers 1 3 V2: 95.09 (#1/124)&lt;/li&gt;&lt;li&gt;Epoch AI - Frontiermath Tier 4 V2: 100.0 (#1/80)&lt;/li&gt;&lt;li&gt;Epoch AI - Ebr Bench: 90.48 (#1/30)&lt;/li&gt;&lt;li&gt;...and 183 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Ember&lt;/strong&gt; — Aggregate Rating 1844, #43&lt;ul&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench v1.1: 25.0 (#2/37)&lt;/li&gt;&lt;li&gt;Vals AI Legal Index: 75.686 (#4/37)&lt;/li&gt;&lt;li&gt;AA-Omniscience v1.1 Hallucination Rate: 17.34 (#5/133)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - HR - Raw Objective Completion: 80.95 (#6/256)&lt;/li&gt;&lt;li&gt;Design Arena (Website): 1330.0 (#8/205)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 49.038 (#10/93)&lt;/li&gt;&lt;li&gt;Design Arena (3D): 1380.0 (#11/178)&lt;/li&gt;&lt;li&gt;Vals AI Finance Index: 60.914 (#11/39)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 1595.0 (#12/289)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling v1.1: 72.169 (#12/41)&lt;/li&gt;&lt;li&gt;...and 79 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agnes 2.5 Pro Theta&lt;/strong&gt; — Aggregate Rating 1777, #129&lt;ul&gt;&lt;li&gt;AA Long Context Reasoning: 84.67 (#9/596)&lt;/li&gt;&lt;li&gt;AA-Briefcase - PDF - Rubric Pass Rate: 56.9 (#32/261)&lt;/li&gt;&lt;li&gt;AA-Briefcase - XLSX - Rubric Pass Rate: 53.9 (#36/261)&lt;/li&gt;&lt;li&gt;AA-Briefcase - Analytical Quality Elo: 1709.99 (#37/261)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 78.89 (#41/254)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 80.03 (#45/254)&lt;/li&gt;&lt;li&gt;AA-Briefcase: 52.0 (#46/261)&lt;/li&gt;&lt;li&gt;AA SciCode: 56.02 (#49/251)&lt;/li&gt;&lt;li&gt;AA-Omniscience v1.1 Hallucination Rate: 41.4 (#56/133)&lt;/li&gt;&lt;li&gt;AA-Briefcase - PPTX - Rubric Pass Rate: 46.12 (#57/261)&lt;/li&gt;&lt;li&gt;...and 63 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;platypus-2-22B-relora [float16]&lt;/strong&gt; — Aggregate Rating 1351, #7520&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 77.35 (#3255/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 82.44 (#3535/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 57.68 (#4491/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 55.33 (#4643/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 6.6 (#5381/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 43.61 (#5434/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;platypus-2-22B-relora [bfloat16]&lt;/strong&gt; — Aggregate Rating 1349, #7588&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard v1 - WinoGrande: 77.11 (#3378/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - HellaSwag: 82.36 (#3574/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - ARC Challenge: 57.51 (#4523/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - MMLU: 54.94 (#4715/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - GSM8K: 6.29 (#5407/7252)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard v1 - TruthfulQA MC2: 43.62 (#5432/7162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (180)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA CritPt: 35.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA CritPt v2: 86.29 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA CritPt v2 - Hard Subset: 76.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA FrontierCode 2.0: 49.13 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf: 26.8 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Construction - Criterion Pass Rate: 79.15 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Criterion Pass Rate: 81.13 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Engineering - Criterion Pass Rate: 86.26 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 92.36 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - HR - Criterion Pass Rate: 78.81 (#43)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Healthcare - Criterion Pass Rate: 90.57 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Insurance - Criterion Pass Rate: 86.39 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Legal - Criterion Pass Rate: 83.67 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 78.57 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - Real Estate - Criterion Pass Rate: 71.24 (#50)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDP.pdf - STEM/Research - Criterion Pass Rate: 80.23 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA GDPval: 1831.33 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA HLE-Diamond: 80.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA HLE-Diamond - Knowledge: 74.8 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA HLE-Diamond - Reasoning: 85.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Humanity's Last Exam: 74.28 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Long Context Reasoning: 83.33 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Omniscience: 57.87 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA SciCode: 67.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench 4.0: 61.62 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1: 73.33 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Earth Sciences: 70.83 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Engineering Sciences: 70.37 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Life Sciences: 68.42 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Mathematical Sciences: 86.27 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Physical Sciences: 70.59 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-3D: 1271.4151 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase: 66.08 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - Analytical Quality Elo: 2015.01 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - DOCX - Rubric Pass Rate: 60.36 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - PDF - Rubric Pass Rate: 62.93 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - PPTX - Rubric Pass Rate: 64.08 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - Presentation Elo: 1706.03 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Briefcase - XLSX - Rubric Pass Rate: 69.52 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Accuracy: 74.43 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Hallucination Rate: 63.04 (#195)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Business: 57.11 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Health: 60.22 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Humanities &amp; Social Sciences: 68.44 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Law: 57.11 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Science, Engineering &amp; Mathematics: 59.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE): 92.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - C: 92.73 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Dart: 94.44 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Go: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - HTML: 91.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Java: 90.91 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Julia: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 74.29 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - PHP: 91.18 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Python: 97.87 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - R: 70.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Rust: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Swift: 88.46 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience v1.1: 66.77 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience v1.1 Accuracy: 79.13 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AA-Omniscience v1.1 Hallucination Rate: 56.4 (#77)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Chess Leaderboard (Continuation): 1622.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1676.0 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ARC-AGI-1: 98.5 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ARC-AGI-2: 93.75 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on ARC-AGI-3: 81.17 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Artificial Analysis Intelligence Index: 64.56 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench (Default Fallbacks): 51.45 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA: 76.42 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Finance - Raw Objective Completion: 90.38 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - HR - Raw Objective Completion: 81.24 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Marketing - Raw Objective Completion: 90.81 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Operations - Raw Objective Completion: 97.15 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Sales - Raw Objective Completion: 94.37 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Strict: 51.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on AutomationBench-AA - Support - Raw Objective Completion: 93.76 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Chess Puzzles (Epoch AI): 79.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Coarena: 1105.9 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on CursorBench 4.0: 60.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Deep20Bench - Edition 1.1: 14.566666666667 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Ebr Bench: 90.48 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Frontiermath Tier 4 V2: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Frontiermath Tiers 1 3 V2: 95.09 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Furniture Assembly: 90.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - GPQA Diamond: 93.88 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Mystery Game Puzzles: 90.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Proofbench: 100.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Spireclimb: 19.62 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Spireclimb Sts1 A20 Heart: 21.67 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Epoch AI - Spireclimb Sts2 A10: 17.57 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on FrontierMath - Tier 4 (v2): 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on FrontierMath - Tiers 1-3 (v2): 95.09 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA: 7.22 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - All Pass: 10.83 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Antitrust &amp; Competition: 88.77 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Arbitration &amp; International Dispute Resolution: 91.28 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Banking &amp; Finance: 94.4 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Bankruptcy &amp; Restructuring: 89.04 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Capital Markets: 94.51 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Corporate Governance: 84.93 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Corporate M&amp;A: 94.11 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Data Privacy &amp; Cybersecurity: 96.7 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Emerging Companies &amp; Venture Capital: 93.29 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Employment &amp; Labor: 86.52 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Energy &amp; Natural Resources: 95.82 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Environmental &amp; ESG: 85.07 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Funds &amp; Asset Management: 90.28 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Healthcare &amp; Life Sciences: 91.95 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Immigration: 92.48 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Insurance: 93.38 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Intellectual Property: 95.61 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - International Trade &amp; Sanctions: 89.38 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Litigation &amp; Dispute Resolution: 92.61 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Real Estate: 93.88 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Structured Finance &amp; Securitization: 90.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Tax: 84.5 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - Trusts, Estates &amp; Private Client: 92.49 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Harvey LAB-AA - White Collar Defense &amp; Investigations: 90.01 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on HieroglyphBench: 43.6 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (AA-Briefcase v1.1): 1785.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (ARC-AGI-3): 79.26 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (AutomationBench; Private held-out set): 51.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (DeepSWE 1.1): 74.3 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (HealthBench Professional): 67.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (HealthBench; Length-adjusted): 59.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (Program Bench): 85.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (Terminal-Bench 4.0): 67.6 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LLM Stats (Terminal-Bench-Science 0.1): 74.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LM Market Cap LMC Score: 95.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Browser: 86.9 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Coding: 79.2 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Data Analysis: 81.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Design Layout: 85.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on LiveBench Reasoning: 88.8 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MLCR-AA: 43.89 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MLCR-AA - Accuracy: 75.56 (#31)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on MLCR-AA - Completeness: 56.11 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Mercor APEX: 80.1 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on OTIS Mock AIME 2024-25: 99.72 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on RuneBench: 6.983758527082 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on SimpleQA Verified: 94.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Asteroid Numbers: 47.9 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Day-one Recall Check: 61.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Exotic Fact Recall: 75.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Inversion and Execution Ladder: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Lichess Pairwise Strength (25 Pairs): 72.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Lichess Rating Prediction (25 Games): 333.14 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Long-tail Book Recall: 77.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Puzzle Portfolio Ladder: 96.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on The Aggregate - Trip Outcome Prediction (5 Questions): 68.7 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on VIALS-AA: 67.08 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on VIALS-AA - Average of 3: 71.84 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on VIALS-AA - Pass@3: 74.53 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI (Vals Index): 71.309 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI BioMysteryBench: 83.704 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Code Migration v1.1: 83.265 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Excel Modeling v1.1: 77.377 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Finance Agent v2.1: 69.47 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Finance Index: 63.583 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench v1.1: 5.0 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Immigration: 73.982 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Legal Index: 69.175 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Legal Research Bench: 46.154 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI MysteryMechanism: 51.802 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI ProgramBench: 87.793 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI ProofBench: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 62.292 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Science Index: 63.74 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Tax Agent Bench: 32.092 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 65.152 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Terminal-Bench Science: 55.714 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.2&lt;/strong&gt; on Vals AI Vibe Code Bench: 93.771 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Blueprint-Bench 2: 0.432 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Blueprint Bench 2: 0.432 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Vending Bench 2: 12860.61 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Frontierswe: 61.2 (#4)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (80)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;AA-3D&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (1271.4151) beat Claude Sonnet 5.5 (Max, Default Fallback) by 72.58&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (Agents - Fullstack)&lt;/strong&gt;: Claude Opus 5.5 (1412.0) beat GPT-6 Astra by 70.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - R&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (70.0) beat Gemini 4 Carbon (High) by 30.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SEAL - Remote Labor Index (RLI)&lt;/strong&gt;: Claude Fable 5.2 (41.3) beat Claude Opus 5.5 by 20.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - HTML&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (91.67) beat Claude Opus 5.5 (Xhigh, Default Fallback) by 19.45&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ARC-AGI-3&lt;/strong&gt;: Claude Fable 5.2 (xHigh) (81.17) beat GPT-6 Astra (Max) by 18.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Health&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (60.22) beat Gemini 4 Carbon (High) by 18.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SimpleQA Verified&lt;/strong&gt;: Claude Fable 5.2 (Max) (94.0) beat Gemini 4 Argon (High) by 18.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Science's Last Exam - Scientific Computing&lt;/strong&gt;: GPT-6.1 Sol (Benchmark Sweep Run: source=Science's Last Exam; harness=Codex; hypothesis-discovery runtime=DiscoveryRSI; effort=high; API endpoint) (73.33) beat GPT-6.1 Sol (Benchmark Sweep Run: source=Science's Last Exam; harness=Codex; hypothesis-discovery runtime=DiscoveryRSI; effort=medium; API endpoint) by 16.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA HLE-Diamond - Knowledge&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (74.8) beat Gemini 4 Carbon (Medium) by 15.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Business&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (57.11) beat Gemini 4 Carbon (High) by 14.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - Go&lt;/strong&gt;: Claude Fable 5.2 (Xhigh, Default Fallback) (100.0) beat GPT-6 Astra (Max) by 13.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (59.0) beat Gemini 4 Carbon (High) by 13.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience v1.1&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (66.77) beat Gemini 4 Carbon (High) by 13.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Humanity's Last Exam&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (74.28) beat Claude Opus 5.5 (Max, Default Fallback) by 12.93&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (SWE-Bench Multimodal)&lt;/strong&gt;: Claude Fable 5.2 (74.1) beat Claude Opus 5.5 by 12.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Ebr Bench&lt;/strong&gt;: Claude Fable 5.2 (Max) (90.48) beat GPT-6.1 Terra (Max) by 12.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Humanities &amp; Social Sciences&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (68.44) beat Gemini 4 Carbon (High) by 12.22&lt;/li&gt;&lt;li&gt;&lt;strong&gt;The Aggregate - Exotic Fact Recall&lt;/strong&gt;: Claude Fable 5.2 (High) (75.0) beat GPT-5.6 Sol (High) by 11.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Omniscience&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (57.87) beat Claude Opus 5.5 (Max, Default Fallback) by 11.45&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA HLE-Diamond&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (80.2) beat Gemini 4 Carbon (High) by 11.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OSWorld 2.1 (strict))&lt;/strong&gt;: Claude Fable 5.2 (54.6) beat Claude Sonnet 5.5 by 11.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;TAC (Travel Agent Compassion)&lt;/strong&gt;: Claude Fable 5.2 (75.6) beat Claude Opus 4.8 by 10.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience v1.1 Accuracy&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (79.13) beat Claude Fable 5.1 (Max, Default Fallback) by 10.73&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Terminal-Bench-Science 0.1)&lt;/strong&gt;: Claude Mythos 5.2 (75.3) beat Gemini 4 Carbon by 10.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;KillSwitch-Bench&lt;/strong&gt;: Claude Fable 5.2 (80.0) beat Claude Opus 5 by 10.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - Java&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (90.91) beat Claude Opus 5.5 (Max, Default Fallback) by 9.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (73.33) beat Gemini 4 Carbon (High) by 8.57&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE)&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (92.67) beat Claude Opus 5.5 (Max, Default Fallback) by 8.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OSWorld 2.1 (partial))&lt;/strong&gt;: Claude Fable 5.2 (89.3) beat Gemini 4 Carbon by 7.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;The Aggregate - Long-tail Book Recall&lt;/strong&gt;: Claude Fable 5.2 (High) (77.0) beat Claude Fable 5.1 (High) by 7.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Accuracy&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (74.43) beat Claude Fable 5.1 (Max, Default Fallback) by 7.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Science's Last Exam&lt;/strong&gt;: GPT-6.1 Sol (Benchmark Sweep Run: source=Science's Last Exam; harness=Codex; hypothesis-discovery runtime=DiscoveryRSI; effort=high; API endpoint) (44.66) beat Claude Opus 5 (Benchmark Sweep Run: source=Science's Last Exam; harness=Claude Code; hypothesis-discovery runtime=DiscoveryRSI; effort=high; API endpoint) by 7.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Life Sciences&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (68.42) beat Gemini 4 Carbon (High) by 7.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Science's Last Exam - Scientific Software Operation&lt;/strong&gt;: GPT-6.1 Sol (Benchmark Sweep Run: source=Science's Last Exam; harness=Codex; hypothesis-discovery runtime=DiscoveryRSI; effort=high; API endpoint) (85.71) beat Claude Opus 5 (Benchmark Sweep Run: source=Science's Last Exam; harness=Claude Code; hypothesis-discovery runtime=DiscoveryRSI; effort=high; API endpoint) by 6.71&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench (Default Fallbacks)&lt;/strong&gt;: Claude Fable 5.2 (default fallbacks, Max) (51.45) beat Claude Sonnet 5.5 (default fallbacks, Max) by 6.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Furniture Assembly&lt;/strong&gt;: Claude Fable 5.2 (Max) (90.0) beat Claude Opus 5.5 (Max) by 6.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Research&lt;/strong&gt;: Claude Fable 5.2 (94.5) beat Muse Wonder by 6.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - Python&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (97.87) beat Claude Opus 5.5 (Max, Default Fallback) by 6.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Mystery Game Puzzles&lt;/strong&gt;: Claude Fable 5.2 (Max) (90.0) beat GPT-6 Astra (Max) by 6.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA HLE-Diamond - Reasoning&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (85.6) beat Gemini 4 Carbon (High) by 6.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - PHP&lt;/strong&gt;: Claude Fable 5.2 (Low, Default Fallback) (91.18) beat Claude Opus 5.5 (High, Default Fallback) by 5.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - ECI&lt;/strong&gt;: Claude Fable 5.2 (174.24) beat GPT-6 Astra by 5.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Long Horizon&lt;/strong&gt;: Claude Fable 5.2 (82.9) beat Claude Fable 5.1 by 5.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Artificial Analysis Intelligence Index&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (64.56) beat Gemini 4 Carbon (High) by 4.53&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA CritPt v2 - Hard Subset&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (76.5) beat GPT-6 Astra (Medium) by 4.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierSWE V2&lt;/strong&gt;: Claude Fable 5.2 (69.94018470588233) beat GPT-6 Astra (Max) by 4.43&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Humanity's Last Exam (with tools))&lt;/strong&gt;: Claude Fable 5.2 (74.2) beat Kimi K3.1 Pro by 4.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Earth Sciences&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (70.83) beat Claude Opus 5.5 (Xhigh, Default Fallback) by 4.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Law&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (57.11) beat Gemini 4 Carbon (High) by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Physical Sciences&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (70.59) beat Claude Opus 5.5 (Xhigh, Default Fallback) by 3.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences&lt;/strong&gt;: Claude Fable 5.2 (Xhigh, Default Fallback) (70.37) beat Gemini 4 Carbon (High) by 3.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VIALS-AA - Average of 3&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (71.84) beat GPT-6 Astra (Max) by 3.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Briefcase - XLSX - Rubric Pass Rate&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (69.52) beat Claude Sonnet 5.5 (Max, Default Fallback) by 3.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VIALS-AA&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (67.08) beat Gemini 4 Carbon (High) by 3.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Chess Puzzles (Epoch AI)&lt;/strong&gt;: Claude Fable 5.2 (Max) (79.0) beat GPT-6 Astra Pro (Max) by 3.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (SWE-bench Multilingual)&lt;/strong&gt;: Claude Fable 5.2 (96.6) beat Claude Opus 5.5 by 2.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA FrontierCode 2.0&lt;/strong&gt;: Claude Fable 5.2 (High, Default Fallback) (49.13) beat Claude Opus 5.5 (Max, Default Fallback) by 2.69&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI BioMysteryBench&lt;/strong&gt;: Claude Fable 5.2 (High) (83.704) beat Claude Sonnet 5.5 (Max) by 2.59&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BenchLM&lt;/strong&gt;: Claude Fable 5.2 (91.2) beat Gemini 4 Carbon by 2.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA CritPt v2&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (86.29) beat GPT-6 Astra (Medium) by 2.29&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MLCR-AA&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (43.89) beat GPT-6 Astra (Max) by 2.22&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CursorBench 4.0&lt;/strong&gt;: Claude Fable 5.2 (Max) (60.0) beat Claude Opus 5.5 (Max) by 2.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coarena - Task Completion&lt;/strong&gt;: Claude Fable 5.2 (100.0) beat Muse Spark 1.3 by 2.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Engineering - Criterion Pass Rate&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (86.26) beat Claude Opus 5.5 (Max, Default Fallback) by 1.91&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Apex Agents&lt;/strong&gt;: Gemini 4 Carbon (84.2) beat Gemini 4 Argon by 1.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Finance Agent v2.1&lt;/strong&gt;: Claude Fable 5.2 (High) (69.47) beat Claude Opus 5.5 (Max) by 1.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierMath - Tiers 1-3 (v2)&lt;/strong&gt;: Claude Fable 5.2 (Max) (95.09) beat GPT-6.1 Sol (Max) by 1.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Frontiermath Tiers 1 3 V2&lt;/strong&gt;: Claude Fable 5.2 (Max) (95.09) beat GPT-6.1 Sol (Max) by 1.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Chartography)&lt;/strong&gt;: Claude Fable 5.2 (91.6) beat Kimi K3.1 Pro by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats Score&lt;/strong&gt;: Claude Fable 5.2 (61.25) beat Claude Opus 5.5 by 1.39&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Vibe Code Bench&lt;/strong&gt;: Claude Fable 5.2 (High) (93.771) beat Claude Sonnet 5.5 (Max) by 1.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Briefcase&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (66.08) beat Claude Sonnet 5.5 (Max, Default Fallback) by 0.93&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA SciCode&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (67.71) beat Claude Opus 5.5 (Max, Default Fallback) by 0.81&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI ProgramBench&lt;/strong&gt;: Claude Fable 5.2 (High) (87.793) beat Claude Opus 5.5 (Max) by 0.81&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Sales - Raw Objective Completion&lt;/strong&gt;: Claude Fable 5.2 (Xhigh, Default Fallback) (94.37) beat Gemini 4 Carbon (Medium) by 0.72&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OfficeQA Pro)&lt;/strong&gt;: Claude Fable 5.2 (72.9) beat Seed 2.1 Pro by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Excel Modeling v1.1&lt;/strong&gt;: Claude Fable 5.2 (High) (77.377) beat Claude Fable 5.1 (Max) by 0.65&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VIALS-AA - Pass@3&lt;/strong&gt;: Claude Fable 5.2 (Max, Default Fallback) (74.53) beat Kimi K3.1 Pro by 0.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI (Vals Index)&lt;/strong&gt;: Claude Fable 5.2 (High) (71.309) beat Claude Sonnet 5.5 (Max) by 0.45&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-10

=== FRONTIER ===
  - Best available model: Claude Fable 5.2 enters at #1 (Aggregate Rating 2009) on 191 benchmarks

=== DAILY ===
NEW BENCHMARKS (3)
  - LLM Stats (Chartography (no tools)) (Score (%)): leader Claude Fable 5.2 (73.4), 5 models
      Chartography pro</summary></entry><entry><title>The Aggregate Digest — 2026-11-09</title><id>https://theaggregate.ai/digest/2026-11-09</id><updated>2026-11-09T10:02:57.474598+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (4)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience v1.1&lt;/strong&gt; (Net Score): leader Gemini 4 Carbon (High) (53.5), 69 models&lt;br&gt;&lt;span&gt;Artificial Analysis&amp;#x27;s AA-Omniscience v1.1 knowledge and hallucination benchmark: 3,000 factual-recall questions (300 of them public) across Business, Health, Humanities &amp;amp; Social Sciences, Law, Software Engineering, and Science, Engineering &amp;amp; Mathematics, graded by GPT-5.6 Luna (medium). Net score: the share of correct answers minus the share of incorrect ones, -100 to 100, so abstaining beats guessing wrong; higher is better. v1.1 replaced v1&amp;#x27;s 6,000 questions by dropping saturated ones and v1&amp;#x27;s 600 public ones, adding new questions and fixing ambiguous or wrong ones, so scores are not directly comparable with the v1 AA Omniscience board.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience v1.1 Accuracy&lt;/strong&gt; (Accuracy (%)): leader Claude Fable 5.1 (Max, Default Fallback) (68.4), 69 models&lt;br&gt;&lt;span&gt;Artificial Analysis&amp;#x27;s AA-Omniscience v1.1 accuracy: the share of its 3,000 factual-recall questions (Business, Health, Humanities &amp;amp; Social Sciences, Law, Software Engineering, Science, Engineering &amp;amp; Mathematics) answered correctly, 0-100 (%), higher is better; an abstention counts as not correct. Not directly comparable with v1&amp;#x27;s 6,000-question AA-Omniscience Accuracy.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience v1.1 Hallucination Rate&lt;/strong&gt; (Hallucination Rate (%)): leader Gemini 4 Carbon (Medium) (4.24), 69 models&lt;br&gt;&lt;span&gt;Artificial Analysis&amp;#x27;s AA-Omniscience v1.1 factual-recall benchmark (3,000 questions across six domains): of the questions a model did not answer correctly, the share it answered incorrectly instead of abstaining. Reported 0-100 (%); lower is better. Declining to answer lowers the rate, so near-silent small models score best; read it beside AA-Omniscience v1.1 Accuracy. Scores are not directly comparable with v1, which had 6,000 questions.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Guesswork 2026-11&lt;/strong&gt; (MAE (z-score units)): leader GPT-6 Astra (0.6568), 32 models&lt;br&gt;&lt;span&gt;The November 2026 round of this site&amp;#x27;s benchmark-score forecasting contest. Models predict new model and benchmark results before publication and are evaluated on common cells against The Aggregate statistical baseline. Score is mean absolute error in per-benchmark z-score units; lower is better. Missing usable predictions receive the benchmark mean. Source: theaggregate.ai/prediction-duel.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon 3.7B (2026-10-01)&lt;/strong&gt; — Aggregate Rating 1604, #914&lt;ul&gt;&lt;li&gt;LLM Stats (HMMT Feb 26): 75.2 (#14/16)&lt;/li&gt;&lt;li&gt;LLM Stats (Tau3 Banking): 17.7 (#15/21)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 2.1): 39.7 (#49/55)&lt;/li&gt;&lt;li&gt;ZeroEval GPQA Diamond: 68.9 (#159/265)&lt;/li&gt;&lt;li&gt;LLM Stats Score: 21.22 (#219/407)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (54)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy: 91.88 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Maths: 95.24 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Primary: 94.84 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Science: 93.44 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Secondary: 91.19 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Social studies: 89.09 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education Pedagogy - Technology: 89.62 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI for Education SEND: 85.32 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Arena AI Code: 1814.34 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LMArena Search Arena: 1214.4806 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LMArena Text Arena: 1505.1 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LMArena Vision Arena: 1295.6406 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LMArena WebDev Arena: 1814.34 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy: 92.32 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Maths: 95.24 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Primary: 96.24 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Science: 93.99 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Secondary: 91.19 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Social studies: 89.09 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education Pedagogy - Technology: 84.91 (#61)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI for Education SEND: 85.32 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Arena AI Code: 1769.81 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Arena AI Document: 1470.43 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on LMArena Search Arena: 1232.2813 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on LMArena Vision Arena: 1284.8173 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on LMArena WebDev Arena: 1769.81 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy: 91.77 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Maths: 92.86 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Primary: 96.24 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Science: 94.54 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Secondary: 90.72 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Social studies: 89.09 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education Pedagogy - Technology: 85.85 (#42)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AI for Education SEND: 86.24 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Arena AI Code: 1741.81 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LMArena Search Arena: 1229.2247 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LMArena Text Arena: 1482.8 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LMArena Vision Arena: 1292.7852 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LMArena WebDev Arena: 1741.81 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vending-Bench 2: 12860.61166666667 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI for Education Pedagogy: 92.88 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI for Education Pedagogy - Maths: 95.24 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI for Education Pedagogy - Primary: 96.24 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI for Education Pedagogy - Science: 97.27 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI for Education Pedagogy - Secondary: 91.98 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI for Education Pedagogy - Social studies: 93.64 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI for Education Pedagogy - Technology: 87.74 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI for Education SEND: 89.45 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Arena AI Code: 1682.73 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on LMArena Text Arena: 1522.63 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on LMArena WebDev Arena: 1682.73 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Arena AI Code: 1767.21 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LMArena Text Arena: 1526.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LMArena WebDev Arena: 1767.21 (#4)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (49)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Training Error Tracing&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (84.94526407) beat MiMo-V2.6-Flash (High) (Search-SWE Run: Codex v0.164.0) by 64.95&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - ChunQiu Temporal Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (56.76611355) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 55.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Sketch-and-Text Image Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (55.54960848) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 52.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Arena AI Code&lt;/strong&gt;: Claude Opus 5.5 (Max) (1814.34) beat Claude Fable 5.1 (Max) by 48.97&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LMArena WebDev Arena&lt;/strong&gt;: Claude Opus 5.5 (Max) (1814.34) beat Claude Fable 5.1 (Max) by 48.97&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Scientific-Paper Question Answering&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (60.0) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 40.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Incremental Vector Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (47.48242491) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 39.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Legal Stance Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (68.80864487) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 38.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Agentic Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (48.83333333) beat MiMo-V2.6-Flash (High) (Search-SWE Run: Codex v0.164.0) by 37.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Large-scale Formula Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (50.54164603) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 33.87&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Long-PDF Sentence Evidence&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (63.88888889) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 30.56&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Reasoning-intensive Query Rewriting&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (60.0) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 30.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Same-Place Image Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (87.00465784) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 29.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Financial-Report Evidence Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (39.97516236) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 29.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Rust BM25 Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (53.09702237) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 29.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - CPU&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (44.73755958) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 28.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Query-side Encoder Alignment&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (55.26315789) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 26.32&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (42.53223472) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 26.15&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Paper Metadata Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (37.5) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 25.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Visual-Document Question Answering&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (44.32775623) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 22.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - GPU&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (40.23102617) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 22.54&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Decision Model Training&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (59.52380952) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 21.26&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Biomedical Graph Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (30.0) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 20.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Document Page Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (26.57158929) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 19.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LMArena Text Arena&lt;/strong&gt;: Gemini 4 Carbon (High) (1526.5) beat Claude Fable 5 by 18.91&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Code Context Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (33.08004047) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 16.34&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Narrative-Clue Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (24.65711778) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 15.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - FARGO paper-aligned query core&lt;/strong&gt;: Inkling 2 Small (OpenCode) (67.11574029712814) beat Claude Opus 5.5 (Max) by 14.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Long-PDF Sentence Evidence — Vectorless&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (63.88888889) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 13.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Budgeted Bimetric Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (32.04050889) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 13.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Methodology Inspiration Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (16.83410533) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 13.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Khmer-to-English Embedding Training&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (33.04938272) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 12.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Composed Image Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (46.84707561) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 10.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Video-Moment Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (32.15543698) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 9.39&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Evolving Personal Memory&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (42.15313994) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 9.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Long-document Reranking&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (42.0) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 9.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Budgeted Candidate Reranking&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (25.23026772) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 8.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Protein–Text Bidirectional Retrieval&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (23.595) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 6.86&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Layout Page Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (23.55070204) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 4.74&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Code Embedding Fine-tuning&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (20.5) beat DeepSeek V4.1 Flash (1017) (Max) (Search-SWE Run: Codex v0.164.0) by 3.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Supra-microBench - Linguistic Competence&lt;/strong&gt;: ZeroS-v0.1-150M (65.13) beat SmolLM2-135M-Base by 2.95&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Search-SWE - Task - Constrained Molecular Search&lt;/strong&gt;: GPT-6.1 Terra (Max) (Search-SWE Run: Codex v0.164.0) (18.2149) beat GPT-6 Luna (Max) (Search-SWE Run: Codex v0.164.0) by 2.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LMArena Search Arena&lt;/strong&gt;: Qwen 3.8 Max (1259.7196) beat GPT-5.6 Sol (xHigh) by 2.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI for Education Pedagogy - Social studies&lt;/strong&gt;: Gemini 4 Argon (Low) (93.64) beat O3 by 1.82&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Supra-microBench - Lexicon and Semantics&lt;/strong&gt;: cagliostro-v3 (86.94) beat SmolLM2-135M-Base by 0.61&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI for Education Pedagogy - Science&lt;/strong&gt;: Gemini 4 Argon (High) (97.27) beat Grok 4.6 by 0.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI for Education Pedagogy - Secondary&lt;/strong&gt;: Gemini 4 Argon (High) (91.98) beat Claude Fable 5.1 by 0.31&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI for Education Pedagogy&lt;/strong&gt;: Gemini 4 Argon (High) (92.88) beat GPT-6 Astra (High) by 0.22&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GSMA Open-Telco - TeleLogs&lt;/strong&gt;: telco-slm-4B (96.41) beat OTel-LLM-8.3B-QnA by 0.11&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-09

=== DAILY ===
NEW BENCHMARKS (4)
  - AA-Omniscience v1.1 (Net Score): leader Gemini 4 Carbon (High) (53.5), 69 models
      Artificial Analysis's AA-Omniscience v1.1 knowledge and hallucination benchmark: 3,000 factual-recall questions (300 of them public) across B</summary></entry><entry><title>The Aggregate Digest — 2026-11-08</title><id>https://theaggregate.ai/digest/2026-11-08</id><updated>2026-11-08T09:34:56.292255+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Weekly&lt;/h2&gt;
&lt;h3&gt;New Models (207)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; — Aggregate Rating 1989, #1&lt;ul&gt;&lt;li&gt;AA MMMU-Pro: 89.25 (#1/302)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 4.24 (#1/124)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond - Knowledge: 59.0 (#1/154)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 88.24 (#1/127)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 90.93 (#1/246)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Sales - Raw Objective Completion: 93.65 (#1/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA: 77.6 (#2/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Strict: 54.19 (#2/248)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: 36.44 (#2/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Law: 45.11 (#2/124)&lt;/li&gt;&lt;li&gt;...and 56 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; — Aggregate Rating 1965, #4&lt;ul&gt;&lt;li&gt;FrontierFinance - Financial Data &amp; Modeling: 57.25 (#2/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Earnings &amp; Events: 66.37 (#3/16)&lt;/li&gt;&lt;li&gt;FrontierFinance: 50.88 (#4/16)&lt;/li&gt;&lt;li&gt;SEAL - MCP Atlas: 85.4 (#5/37)&lt;/li&gt;&lt;li&gt;FrontierFinance - Sector, Industry &amp; Macro: 38.98 (#6/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Company Research: 46.28 (#6/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Screening &amp; Discovery: 35.59 (#7/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Coverage &amp; Catalyst Monitoring: 45.51 (#9/16)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; — Aggregate Rating 1961, #5&lt;ul&gt;&lt;li&gt;Design Arena (SVG): 1412.0 (#3/137)&lt;/li&gt;&lt;li&gt;SEAL - Fortress: 12.14 (#72/79)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; — Aggregate Rating 1946, #9&lt;ul&gt;&lt;li&gt;PFArena - Single-Mutant-Informed Multi-Mutant Ranking: 0.6006 (#1/9)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (C#): 100.0 (#2/22)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#3/22)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 70.1 (#3/16)&lt;/li&gt;&lt;li&gt;PFArena - Measurement-Free Multi-Mutant Ranking: 0.2514 (#4/9)&lt;/li&gt;&lt;li&gt;PFArena - Anchor-Informed Multi-Mutant Ranking: 0.3299 (#4/9)&lt;/li&gt;&lt;li&gt;Riemann-bench: 79.2 (#5/21)&lt;/li&gt;&lt;li&gt;TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#5/30)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (Rust): 98.4 (#6/22)&lt;/li&gt;&lt;li&gt;TAC (Travel Agent Compassion): 40.4 (#11/30)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; — Aggregate Rating 1940, #10&lt;ul&gt;&lt;li&gt;PFArena - Anchor-Informed Multi-Mutant Ranking: 0.3822 (#1/9)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (C#): 100.0 (#1/22)&lt;/li&gt;&lt;li&gt;SEAL - MCP Atlas: 90.7 (#1/37)&lt;/li&gt;&lt;li&gt;PFArena - Measurement-Free Multi-Mutant Ranking: 0.2766 (#2/9)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#2/22)&lt;/li&gt;&lt;li&gt;TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#2/30)&lt;/li&gt;&lt;li&gt;PFArena - Single-Mutant-Informed Multi-Mutant Ranking: 0.5826 (#3/9)&lt;/li&gt;&lt;li&gt;Kaggle FACTS (Google): 72.4 (#3/8)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (Rust): 100.0 (#3/22)&lt;/li&gt;&lt;li&gt;TAC (Travel Agent Compassion): 57.1 (#4/30)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; — Aggregate Rating 1931, #13&lt;ul&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 84.607567808327 (#15/253)&lt;/li&gt;&lt;li&gt;ForecastBench: 66.0 (#98/363)&lt;/li&gt;&lt;li&gt;ForecastBench Tournament - Market: 70.9 (#120/363)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; — Aggregate Rating 1920, #14&lt;ul&gt;&lt;li&gt;Riemann-bench: 63.2 (#9/21)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5.5&lt;/strong&gt; — Aggregate Rating 1918, #16&lt;ul&gt;&lt;li&gt;FrontierCS 2 - GPU DBSCAN Kernel Optimization: 73.3787292124976 (#1/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Learned Transformer Layer Mapping: 73.72019487756762 (#1/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Exact Sequence Alignment: 67.84462833266464 (#1/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - E-Graph Invariant Restoration: 67.30846631487712 (#1/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Hypergraph Core Numbers: 70.61820270770313 (#1/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - GPU IVF-PQ Search Kernel Optimization: 80.99348237443567 (#1/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - KAMI FP64 Block GEMM — trusted batched harness adaptation: 23.310051695541144 (#1/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - GPU K-Means Kernel Optimization: 54.23780982604794 (#1/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - ParButterfly Exact Counting: 58.29391835181178 (#1/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Binary Fuse Filter Construction and Lookup: 56.78771994912518 (#2/20)&lt;/li&gt;&lt;li&gt;...and 46 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; — Aggregate Rating 1911, #17&lt;ul&gt;&lt;li&gt;Epoch AI - Spireclimb Sts2 A10: 34.67 (#1/8)&lt;/li&gt;&lt;li&gt;HiL-Bench: 68.0 (#2/31)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (C#): 100.0 (#3/22)&lt;/li&gt;&lt;li&gt;Epoch AI - Spireclimb: 22.33 (#3/8)&lt;/li&gt;&lt;li&gt;Epoch AI - Spireclimb Sts1 A20 Heart: 10.0 (#3/8)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#4/22)&lt;/li&gt;&lt;li&gt;TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#6/30)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (Rust): 98.4 (#7/22)&lt;/li&gt;&lt;li&gt;ChessBench: 22.59 (#7/72)&lt;/li&gt;&lt;li&gt;ReLE - Coding: 74.3 (#7/69)&lt;/li&gt;&lt;li&gt;...and 30 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5&lt;/strong&gt; — Aggregate Rating 1906, #19&lt;ul&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Dart: 83.33 (#11/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Rust: 100.0 (#11/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - HTML: 61.11 (#12/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: 23.33 (#13/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 60.87 (#15/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Humanities &amp; Social Sciences: 45.78 (#15/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Law: 29.78 (#17/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Swift: 69.23 (#17/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Java: 63.64 (#18/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Julia: 73.68 (#19/124)&lt;/li&gt;&lt;li&gt;...and 20 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Sol&lt;/strong&gt; — Aggregate Rating 1891, #22&lt;ul&gt;&lt;li&gt;AA CritPt v2: 80.86 (#14/117)&lt;/li&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 66.5 (#14/117)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 35.58 (#20/114)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 51.13 (#24/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 94.03 (#26/124)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 44.16 (#27/109)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 81.4 (#27/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Dart: 69.44 (#29/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Python: 71.28 (#29/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - PHP: 58.82 (#31/124)&lt;/li&gt;&lt;li&gt;...and 17 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder&lt;/strong&gt; — Aggregate Rating 1887, #24&lt;ul&gt;&lt;li&gt;FrontierFinance - Coverage &amp; Catalyst Monitoring: 56.22 (#1/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Screening &amp; Discovery: 39.94 (#1/16)&lt;/li&gt;&lt;li&gt;ChessBench - Coherence: 100.0 (#2/72)&lt;/li&gt;&lt;li&gt;FrontierFinance - Company Research: 52.31 (#2/16)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 81.56 (#3/246)&lt;/li&gt;&lt;li&gt;FrontierFinance: 52.56 (#3/16)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - HR - Raw Objective Completion: 81.47 (#4/248)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 67.9 (#4/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Earnings &amp; Events: 65.95 (#4/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Sector, Industry &amp; Macro: 39.42 (#5/16)&lt;/li&gt;&lt;li&gt;...and 71 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro&lt;/strong&gt; — Aggregate Rating 1885, #28&lt;ul&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 100.0 (#1/314)&lt;/li&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, No Tools): 51.1 (#1/57)&lt;/li&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, With Tools): 70.0 (#1/35)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JaNLI: 100.0 (#1/146)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - JaNLI: 100.0 (#1/146)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - JSQuAD: 83.05 (#1/146)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - MAWPS: 99.0 (#1/146)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - STEM: 100.0 (#1/146)&lt;/li&gt;&lt;li&gt;Nejumi 4 - HalluLens: 100.0 (#1/146)&lt;/li&gt;&lt;li&gt;Design Arena (Agents - Fullstack): 1337.0 (#2/50)&lt;/li&gt;&lt;li&gt;...and 115 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;hy4&lt;/strong&gt; — Aggregate Rating 1861, #31&lt;ul&gt;&lt;li&gt;Coarena - Task Completion: 97.7 (#1/22)&lt;/li&gt;&lt;li&gt;FoodTruckBench: 67597.64 (#2/60)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Laravel Code Quality: 18.05 (#2/23)&lt;/li&gt;&lt;li&gt;Coarena: 1107.9 (#3/22)&lt;/li&gt;&lt;li&gt;Mercor APEX: 77.6 (#3/70)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 1779.0 (#3/287)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Marketing - Raw Objective Completion: 93.47 (#4/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Operations - Raw Objective Completion: 96.6 (#7/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Sales - Raw Objective Completion: 89.09 (#8/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Support - Raw Objective Completion: 91.98 (#8/248)&lt;/li&gt;&lt;li&gt;...and 79 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.8 Flash&lt;/strong&gt; — Aggregate Rating 1849, #37&lt;ul&gt;&lt;li&gt;TaxCalcBench: 6.0 (#48/56)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash (1017)&lt;/strong&gt; — Aggregate Rating 1847, #38&lt;ul&gt;&lt;li&gt;Coarena - Task Completion: 96.9 (#4/22)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - ALT English-to-Japanese: 92.67 (#4/146)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE September: 49.69 (#6/12)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (C#): 98.4 (#7/22)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV September: 45.28 (#8/12)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JSQuAD: 82.12 (#8/146)&lt;/li&gt;&lt;li&gt;AutomationBench-AA: 68.49 (#11/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Finance - Raw Objective Completion: 91.96 (#11/248)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JSICK: 84.0 (#11/146)&lt;/li&gt;&lt;li&gt;AA-Briefcase - Presentation Elo: 1582.41 (#12/253)&lt;/li&gt;&lt;li&gt;...and 134 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.7&lt;/strong&gt; — Aggregate Rating 1844, #41&lt;ul&gt;&lt;li&gt;TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#3/30)&lt;/li&gt;&lt;li&gt;TAC (Travel Agent Compassion): 44.9 (#9/30)&lt;/li&gt;&lt;li&gt;Riemann-bench: 57.6 (#12/21)&lt;/li&gt;&lt;li&gt;OpenRouter GPQA Diamond: 91.9 (#17/145)&lt;/li&gt;&lt;li&gt;ChessBench - Elo: 1076.0 (#20/72)&lt;/li&gt;&lt;li&gt;ChessBench - Accuracy: 74.0 (#22/72)&lt;/li&gt;&lt;li&gt;ChessBench: 15.29 (#24/72)&lt;/li&gt;&lt;li&gt;ChessBench - Coherence: 71.8 (#35/72)&lt;/li&gt;&lt;li&gt;OpenRouter Tau2-Bench Airline: 75.3 (#35/142)&lt;/li&gt;&lt;li&gt;TaxCalcBench: 8.0 (#42/56)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;doubao-seed-evolving&lt;/strong&gt; — Aggregate Rating 1843, #42&lt;ul&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 99.17 (#21/314)&lt;/li&gt;&lt;li&gt;Epoch AI - GPQA Diamond: 93.12 (#24/334)&lt;/li&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 40.0 (#43/162)&lt;/li&gt;&lt;li&gt;SimpleQA Verified: 41.9 (#49/101)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3-FlashX-Legal&lt;/strong&gt; — Aggregate Rating 1843, #43&lt;ul&gt;&lt;li&gt;LLM Stats (Legal Agent Benchmark): 19.2 (#1/16)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Success Rate: 100.0 (#41/407)&lt;/li&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 69.750693660318 (#44/253)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Exactness: 55.29 (#45/400)&lt;/li&gt;&lt;li&gt;LLM Stats Score: 41.48 (#70/405)&lt;/li&gt;&lt;li&gt;LM Market Cap LMC Score: 69.7 (#123/214)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 94.0 (#179/407)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.6&lt;/strong&gt; — Aggregate Rating 1841, #44&lt;ul&gt;&lt;li&gt;AA-Omniscience Index - Humanities &amp; Social Sciences: 34.89 (#30/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Rust: 91.18 (#31/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Law: 18.67 (#33/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: 12.44 (#34/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 8.57 (#36/124)&lt;/li&gt;&lt;li&gt;AA Omniscience: 24.87 (#38/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Business: 21.11 (#38/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - PHP: 50.0 (#38/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Swift: 42.31 (#38/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 46.23 (#39/124)&lt;/li&gt;&lt;li&gt;...and 26 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.3&lt;/strong&gt; — Aggregate Rating 1840, #45&lt;ul&gt;&lt;li&gt;OSWorld 2.0: 39.25 (#3/12)&lt;/li&gt;&lt;li&gt;OSWorld 2.0 Partial: 69.56 (#3/12)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 64.8 (#6/16)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Total: 50.37 (#11/23)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - React-TS Code Quality: 16.67 (#14/23)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Laravel Code Quality: 15.9 (#20/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5&lt;/strong&gt; — Aggregate Rating 1835, #48&lt;ul&gt;&lt;li&gt;DAXBench: 86.7 (#16/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LongCat-2.5&lt;/strong&gt; — Aggregate Rating 1829, #50&lt;ul&gt;&lt;li&gt;LLM Stats (Vision2Web): 78.6 (#1/6)&lt;/li&gt;&lt;li&gt;LLM Stats (BrowseComp): 91.9 (#2/74)&lt;/li&gt;&lt;li&gt;LLM Stats (Claw-Eval): 78.3 (#2/16)&lt;/li&gt;&lt;li&gt;LLM Stats (Humanity's Last Exam (with tools, text-only)): 63.1 (#2/12)&lt;/li&gt;&lt;li&gt;LLM Stats (MathVision): 96.5 (#2/40)&lt;/li&gt;&lt;li&gt;LLM Stats (SkillsBench): 63.3 (#2/11)&lt;/li&gt;&lt;li&gt;LLM Stats (JobBench): 65.4 (#3/5)&lt;/li&gt;&lt;li&gt;LLM Stats (SWE Atlas - Codebase QnA): 58.3 (#3/8)&lt;/li&gt;&lt;li&gt;LLM Stats (AutomationBench v1.0.6): 51.8 (#4/7)&lt;/li&gt;&lt;li&gt;LLM Stats (NL2Repo): 60.5 (#5/26)&lt;/li&gt;&lt;li&gt;...and 8 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3&lt;/strong&gt; — Aggregate Rating 1829, #53&lt;ul&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Bug Density: 0.51 (#1/76)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Code Smell Density: 14.43 (#4/76)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Pass Rate: 83.82 (#8/76)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - LRR: 11.68 (#14/20)&lt;/li&gt;&lt;li&gt;Sonar LLM Leaderboard - Java - Vulnerability Density: 0.28 (#41/76)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Inkling 2 Small&lt;/strong&gt; — Aggregate Rating 1825, #55&lt;ul&gt;&lt;li&gt;AA-AnalystAgent - Pass@1: 71.25 (#1/52)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 86.25 (#1/52)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - E-Graph Invariant Restoration: 64.74090991515263 (#2/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Join-Tree Rewriting: 12.348392894651337 (#2/20)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 90.4 (#3/246)&lt;/li&gt;&lt;li&gt;EnterpriseOps-Gym-AA - Drive: 63.81 (#3/51)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 71.8 (#3/55)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Approximate-MIPS Query Stopping: 86.5565536045519 (#4/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - OCR Distillation Trajectory Sampling: 77.54524886877829 (#4/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - KAMI FP64 Block GEMM — trusted batched harness adaptation: 22.687322415024063 (#4/20)&lt;/li&gt;&lt;li&gt;...and 156 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash&lt;/strong&gt; — Aggregate Rating 1822, #58&lt;ul&gt;&lt;li&gt;PFArena - Single-Mutant-Informed Multi-Mutant Ranking: 0.5662 (#5/9)&lt;/li&gt;&lt;li&gt;PFArena - Measurement-Free Multi-Mutant Ranking: 0.2034 (#7/9)&lt;/li&gt;&lt;li&gt;PFArena - Anchor-Informed Multi-Mutant Ranking: 0.2568 (#9/9)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Terra&lt;/strong&gt; — Aggregate Rating 1821, #59&lt;ul&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 50.5 (#35/117)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 69.71 (#37/117)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 32.34 (#46/109)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 23.89 (#46/114)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;snowhare&lt;/strong&gt; — Aggregate Rating 1819, #63&lt;ul&gt;&lt;li&gt;Design Arena (SVG): 1289.0 (#19/137)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 5.5&lt;/strong&gt; — Aggregate Rating 1818, #64&lt;ul&gt;&lt;li&gt;Kaggle FACTS (Google): 47.32 (#8/8)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (C#): 97.6 (#10/22)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (Rust): 96.4 (#13/22)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (TypeScript): 97.6 (#15/22)&lt;/li&gt;&lt;li&gt;ProphetArena: 0.9048 (#24/24)&lt;/li&gt;&lt;li&gt;Design Arena (ASCII Art): 1208.0 (#30/99)&lt;/li&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 74.761584055738 (#32/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3&lt;/strong&gt; — Aggregate Rating 1818, #65&lt;ul&gt;&lt;li&gt;Design Arena (Agents - Web Apps): 1273.0 (#13/56)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Particle-1&lt;/strong&gt; — Aggregate Rating 1817, #67&lt;ul&gt;&lt;li&gt;SvelteBench: 100.0 (#26/259)&lt;/li&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 71.694177723704 (#37/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy4 preview&lt;/strong&gt; — Aggregate Rating 1816, #69&lt;ul&gt;&lt;li&gt;Computer Anthology Terminal Tasks (Terminus-2): 33.0 (#12/28)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Doubao-Seed-2.1-Pro-260628&lt;/strong&gt; — Aggregate Rating 1810, #75&lt;ul&gt;&lt;li&gt;SimpleQA Verified: 44.7 (#44/101)&lt;/li&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 95.56 (#51/314)&lt;/li&gt;&lt;li&gt;Epoch AI - GPQA Diamond: 89.33 (#61/334)&lt;/li&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 21.0 (#93/162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.7&lt;/strong&gt; — Aggregate Rating 1809, #78&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.528 (#21/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark Open&lt;/strong&gt; — Aggregate Rating 1809, #79&lt;ul&gt;&lt;li&gt;FrontierFinance - Sector, Industry &amp; Macro: 44.13 (#1/16)&lt;/li&gt;&lt;li&gt;FrontierFinance: 53.73 (#2/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Earnings &amp; Events: 70.53 (#2/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Company Research: 52.23 (#3/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Coverage &amp; Catalyst Monitoring: 51.05 (#3/16)&lt;/li&gt;&lt;li&gt;FrontierFinance - Screening &amp; Discovery: 38.05 (#3/16)&lt;/li&gt;&lt;li&gt;CrueltyBench: 62.8 (#4/18)&lt;/li&gt;&lt;li&gt;FrontierFinance - Financial Data &amp; Modeling: 55.83 (#5/16)&lt;/li&gt;&lt;li&gt;Design Arena (3D): 1261.0 (#41/176)&lt;/li&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 59.284511522045 (#74/253)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro (0813)&lt;/strong&gt; — Aggregate Rating 1805, #85&lt;ul&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (Rust): 92.8 (#17/22)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (TypeScript): 96.8 (#18/22)&lt;/li&gt;&lt;li&gt;SpacetimeDB LLM Benchmark (C#): 90.4 (#19/22)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 10.31 (#83/114)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 12.57 (#90/109)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 25.71 (#91/117)&lt;/li&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 8.5 (#93/117)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 0.0 (#99/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Life Sciences: 0.0 (#100/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Physical Sciences: 0.0 (#100/127)&lt;/li&gt;&lt;li&gt;...and 3 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Pro (Preview)&lt;/strong&gt; — Aggregate Rating 1800, #93&lt;ul&gt;&lt;li&gt;DAXBench: 93.8 (#6/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M3.1-Flash&lt;/strong&gt; — Aggregate Rating 1798, #95&lt;ul&gt;&lt;li&gt;FrontierCS 2 - Hypergraph Core Numbers: 59.03285492131152 (#4/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Progent Privilege Control: 50.529218750000005 (#6/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Dynamic GPU Allocation: 46.56241284294073 (#7/20)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 93.8 (#7/22)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - OCR Distillation Trajectory Sampling: 74.43438914027149 (#8/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Graph Mining Method Selection: 90.67251828307572 (#8/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Exact DBSCAN Connectivity: 72.2019058049676 (#9/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Reusable Library Candidates: 18.24735225500887 (#10/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - LRR: 21.17 (#10/20)&lt;/li&gt;&lt;li&gt;FrontierCS 2 - Go1 Joystick Policy Training: 49.82208428819455 (#10/20)&lt;/li&gt;&lt;li&gt;...and 43 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.2&lt;/strong&gt; — Aggregate Rating 1798, #96&lt;ul&gt;&lt;li&gt;OSWorld 2.0: 15.84 (#7/12)&lt;/li&gt;&lt;li&gt;OSWorld 2.0 Partial: 44.08 (#8/12)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 5 Preview&lt;/strong&gt; — Aggregate Rating 1795, #101&lt;ul&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 52.611369388214 (#99/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.6&lt;/strong&gt; — Aggregate Rating 1794, #104&lt;ul&gt;&lt;li&gt;DAXBench: 82.0 (#28/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5&lt;/strong&gt; — Aggregate Rating 1791, #108&lt;ul&gt;&lt;li&gt;AA FrontierCode 2.0: 17.83 (#61/114)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling 3.1 Flash&lt;/strong&gt; — Aggregate Rating 1790, #110&lt;ul&gt;&lt;li&gt;EnigmaForge - Fact F1: 74.8 (#23/28)&lt;/li&gt;&lt;li&gt;EnigmaForge: 33.3 (#25/28)&lt;/li&gt;&lt;li&gt;EnigmaForge - Intuition: 6.2 (#25/28)&lt;/li&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 42.883587232105 (#145/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;mistral-large-4-oss&lt;/strong&gt; — Aggregate Rating 1788, #111&lt;ul&gt;&lt;li&gt;AI Coding Daily (OpenCode) - React-TS Code Quality: 18.5 (#1/23)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - Coding: 100.0 (#1/146)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - Writing: 99.5 (#1/146)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JaNLI: 100.0 (#2/146)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - JaNLI: 100.0 (#2/146)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - STEM: 100.0 (#2/146)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Laravel Code Quality: 17.78 (#5/23)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - MAWPS: 98.0 (#7/146)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Total: 50.31 (#12/23)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - MAWPS: 98.0 (#12/146)&lt;/li&gt;&lt;li&gt;...and 97 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.3 Codex&lt;/strong&gt; — Aggregate Rating 1788, #112&lt;ul&gt;&lt;li&gt;DAXBench: 88.6 (#15/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Pro (Preview)&lt;/strong&gt; — Aggregate Rating 1787, #115&lt;ul&gt;&lt;li&gt;DAXBench: 81.3 (#33/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Luna&lt;/strong&gt; — Aggregate Rating 1784, #118&lt;ul&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 5.71 (#38/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Rust: 88.24 (#42/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 58.14 (#50/124)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond - Reasoning: 31.4 (#54/154)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 56.29 (#56/117)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 39.13 (#61/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 70.15 (#61/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - PHP: 29.41 (#62/124)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 17.57 (#63/114)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond: 20.9 (#65/154)&lt;/li&gt;&lt;li&gt;...and 20 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.8-Flash-Next&lt;/strong&gt; — Aggregate Rating 1779, #124&lt;ul&gt;&lt;li&gt;ZeroEval GPQA Diamond: 91.7 (#21/264)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4&lt;/strong&gt; — Aggregate Rating 1779, #125&lt;ul&gt;&lt;li&gt;DAXBench: 83.2 (#25/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark&lt;/strong&gt; — Aggregate Rating 1777, #128&lt;ul&gt;&lt;li&gt;ChemGraph - Single-Agent: 95.0 (#18/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.2&lt;/strong&gt; — Aggregate Rating 1774, #138&lt;ul&gt;&lt;li&gt;SEAL - Fortress: 48.69 (#17/79)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 25.89 (#20/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics: 15.67 (#51/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Julia: 15.79 (#66/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: -12.22 (#68/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Business: 0.44 (#69/124)&lt;/li&gt;&lt;li&gt;AA Omniscience: 0.63 (#72/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - R: -30.0 (#73/124)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond: 17.7 (#75/154)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond - Reasoning: 23.2 (#76/154)&lt;/li&gt;&lt;li&gt;...and 17 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Flash (Preview)&lt;/strong&gt; — Aggregate Rating 1772, #142&lt;ul&gt;&lt;li&gt;DAXBench: 83.8 (#22/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Doubao-Seed-2.0-Pro-260215&lt;/strong&gt; — Aggregate Rating 1767, #148&lt;ul&gt;&lt;li&gt;SimpleQA Verified: 39.4 (#56/101)&lt;/li&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 93.89 (#65/314)&lt;/li&gt;&lt;li&gt;Epoch AI - GPQA Diamond: 88.19 (#73/334)&lt;/li&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 20.0 (#100/162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.5&lt;/strong&gt; — Aggregate Rating 1764, #150&lt;ul&gt;&lt;li&gt;DAXBench: 82.7 (#27/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 Max Preview&lt;/strong&gt; — Aggregate Rating 1761, #155&lt;ul&gt;&lt;li&gt;DAXBench: 90.9 (#10/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4.6&lt;/strong&gt; — Aggregate Rating 1759, #159&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.4837 (#17/23)&lt;/li&gt;&lt;li&gt;DAXBench: 84.5 (#20/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;doubao-seed-2.1-turbo-260628&lt;/strong&gt; — Aggregate Rating 1755, #163&lt;ul&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 96.11 (#46/314)&lt;/li&gt;&lt;li&gt;Epoch AI - GPQA Diamond: 89.58 (#59/334)&lt;/li&gt;&lt;li&gt;SimpleQA Verified: 29.7 (#77/101)&lt;/li&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 26.0 (#82/162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.20 Multi-Agent&lt;/strong&gt; — Aggregate Rating 1754, #165&lt;ul&gt;&lt;li&gt;DAXBench: 77.9 (#43/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.7 Code&lt;/strong&gt; — Aggregate Rating 1753, #167&lt;ul&gt;&lt;li&gt;FrontierCS 2 - LRR: 10.65 (#15/20)&lt;/li&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 33.208864904406 (#197/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.1 Codex Max&lt;/strong&gt; — Aggregate Rating 1753, #168&lt;ul&gt;&lt;li&gt;DAXBench: 85.0 (#18/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercury 3&lt;/strong&gt; — Aggregate Rating 1751, #173&lt;ul&gt;&lt;li&gt;Design Arena (3D): 1243.0 (#50/176)&lt;/li&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 43.82649651953 (#139/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.2&lt;/strong&gt; — Aggregate Rating 1751, #174&lt;ul&gt;&lt;li&gt;DocHop: 40.36 (#3/21)&lt;/li&gt;&lt;li&gt;DAXBench: 78.4 (#39/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.6&lt;/strong&gt; — Aggregate Rating 1751, #176&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.4413 (#14/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.1&lt;/strong&gt; — Aggregate Rating 1751, #178&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.3239 (#5/23)&lt;/li&gt;&lt;li&gt;DAXBench: 90.3 (#11/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 Plus (2026-02-15)&lt;/strong&gt; — Aggregate Rating 1750, #182&lt;ul&gt;&lt;li&gt;DAXBench: 89.7 (#13/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Beam 1.0&lt;/strong&gt; — Aggregate Rating 1750, #183&lt;ul&gt;&lt;li&gt;Riemann-bench: 37.6 (#17/21)&lt;/li&gt;&lt;li&gt;Vals AI MysteryMechanism: 22.072 (#21/29)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 22.031 (#23/23)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 49.2 (#25/55)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration v1.1: 22.846 (#25/29)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench v1.1: 4.583 (#29/35)&lt;/li&gt;&lt;li&gt;Vals AI BioMysteryBench: 42.593 (#30/30)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 43.424 (#31/34)&lt;/li&gt;&lt;li&gt;Vals AI Legal Index: 61.799 (#31/35)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling v1.1: 64.733 (#33/39)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;beam-1.0&lt;/strong&gt; — Aggregate Rating 1749, #185&lt;ul&gt;&lt;li&gt;Design Arena (UI Components): 1258.0 (#53/183)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro&lt;/strong&gt; — Aggregate Rating 1749, #188&lt;ul&gt;&lt;li&gt;DAXBench: 83.2 (#24/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 4&lt;/strong&gt; — Aggregate Rating 1749, #190&lt;ul&gt;&lt;li&gt;Vals AI SAFE-Teen - Relationship Boundaries: 19.0 (#2/18)&lt;/li&gt;&lt;li&gt;Vals AI SAFE-Teen - Urgent Safety: 33.0 (#8/18)&lt;/li&gt;&lt;li&gt;Vals AI SAFE-Teen - Health Advice: 53.0 (#12/18)&lt;/li&gt;&lt;li&gt;HANDBOOK.md Agents: 19.1 (#16/17)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 42.4 (#16/16)&lt;/li&gt;&lt;li&gt;Riemann-bench: 20.0 (#21/21)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 47.7 (#26/55)&lt;/li&gt;&lt;li&gt;Epoch AI - ECI: 154.66 (#45/253)&lt;/li&gt;&lt;li&gt;FrontierMath - Tier 4 (v2): 26.83 (#46/79)&lt;/li&gt;&lt;li&gt;FrontierMath - Tiers 1-3 (v2): 58.6 (#46/120)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.3&lt;/strong&gt; — Aggregate Rating 1747, #194&lt;ul&gt;&lt;li&gt;DAXBench: 96.2 (#2/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;doubao-seed-2.0-lite-260428&lt;/strong&gt; — Aggregate Rating 1743, #202&lt;ul&gt;&lt;li&gt;SimpleQA Verified: 31.5 (#73/101)&lt;/li&gt;&lt;li&gt;Epoch AI - GPQA Diamond: 88.13 (#75/334)&lt;/li&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 91.11 (#80/314)&lt;/li&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 20.0 (#99/162)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 Plus (Preview)&lt;/strong&gt; — Aggregate Rating 1738, #215&lt;ul&gt;&lt;li&gt;DAXBench: 93.3 (#7/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 Plus&lt;/strong&gt; — Aggregate Rating 1735, #221&lt;ul&gt;&lt;li&gt;DAXBench: 89.0 (#14/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3&lt;/strong&gt; — Aggregate Rating 1733, #226&lt;ul&gt;&lt;li&gt;DAXBench: 80.2 (#34/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5-Turbo&lt;/strong&gt; — Aggregate Rating 1732, #230&lt;ul&gt;&lt;li&gt;DAXBench: 84.3 (#21/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4.5&lt;/strong&gt; — Aggregate Rating 1726, #244&lt;ul&gt;&lt;li&gt;DAXBench: 77.9 (#42/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 397B A17B&lt;/strong&gt; — Aggregate Rating 1725, #245&lt;ul&gt;&lt;li&gt;DAXBench: 90.3 (#12/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Apodex 1.1&lt;/strong&gt; — Aggregate Rating 1724, #253&lt;ul&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences: 3.7 (#86/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1: 0.48 (#99/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Life Sciences: 0.0 (#119/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 0.0 (#119/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Physical Sciences: 0.0 (#120/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Earth Sciences: 0.0 (#120/127)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.5&lt;/strong&gt; — Aggregate Rating 1724, #254&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.4319 (#13/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5&lt;/strong&gt; — Aggregate Rating 1723, #258&lt;ul&gt;&lt;li&gt;DAXBench: 96.2 (#3/50)&lt;/li&gt;&lt;li&gt;Altered Riddles: 0.346 (#8/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash&lt;/strong&gt; — Aggregate Rating 1722, #259&lt;ul&gt;&lt;li&gt;ParaRecover - Level 1 (Localisation): 68.42 (#2/17)&lt;/li&gt;&lt;li&gt;ParaRecover - Level 2 (Recovery): 66.57 (#2/17)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ripple-65B Preview&lt;/strong&gt; — Aggregate Rating 1721, #267&lt;ul&gt;&lt;li&gt;HMMT February 2026: 90.5 (#9/26)&lt;/li&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 84.2 (#26/75)&lt;/li&gt;&lt;li&gt;MCP Atlas: 71.67 (#44/80)&lt;/li&gt;&lt;li&gt;AA-LCR: 75.0 (#110/202)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.5-Pro&lt;/strong&gt; — Aggregate Rating 1714, #286&lt;ul&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 37.750506748042 (#176/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.20&lt;/strong&gt; — Aggregate Rating 1712, #292&lt;ul&gt;&lt;li&gt;DAXBench: 80.1 (#35/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.2 Instant&lt;/strong&gt; — Aggregate Rating 1710, #301&lt;ul&gt;&lt;li&gt;DAXBench: 80.1 (#36/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4&lt;/strong&gt; — Aggregate Rating 1709, #307&lt;ul&gt;&lt;li&gt;DAXBench: 76.7 (#45/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.3&lt;/strong&gt; — Aggregate Rating 1703, #332&lt;ul&gt;&lt;li&gt;DAXBench: 81.3 (#31/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2-Pro&lt;/strong&gt; — Aggregate Rating 1702, #336&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.2873 (#1/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5V Turbo&lt;/strong&gt; — Aggregate Rating 1701, #344&lt;ul&gt;&lt;li&gt;DAXBench: 91.5 (#9/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 31B (IT)&lt;/strong&gt; — Aggregate Rating 1700, #347&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.3619 (#10/23)&lt;/li&gt;&lt;li&gt;ReasonScape R12: 928.31 (#18/123)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nanbeige4.3-33B&lt;/strong&gt; — Aggregate Rating 1700, #349&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 18.19 (#7/124)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Finance - Raw Objective Completion: 85.59 (#47/248)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - R: -23.33 (#54/124)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench 4.0: 29.8 (#58/244)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: -10.67 (#65/124)&lt;/li&gt;&lt;li&gt;ReasonScape R12: 802.97 (#69/123)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Business: 0.0 (#70/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - HTML: 11.11 (#72/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics: 4.83 (#75/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Kotlin: -20.0 (#75/124)&lt;/li&gt;&lt;li&gt;...and 55 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Super VL&lt;/strong&gt; — Aggregate Rating 1698, #360&lt;ul&gt;&lt;li&gt;Design Arena (Data Viz): 1097.0 (#128/154)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O4 Mini&lt;/strong&gt; — Aggregate Rating 1697, #363&lt;ul&gt;&lt;li&gt;DAXBench: 76.5 (#48/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Flash Lite&lt;/strong&gt; — Aggregate Rating 1695, #375&lt;ul&gt;&lt;li&gt;DAXBench: 82.0 (#29/50)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Julia: 5.26 (#75/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 31.7 (#84/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Rust: 64.71 (#90/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Law: -24.67 (#97/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - C: 10.91 (#98/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 20.9 (#98/124)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 12.0 (#100/117)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 6.16 (#101/109)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 81.75 (#101/124)&lt;/li&gt;&lt;li&gt;...and 27 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Glimmer 30B&lt;/strong&gt; — Aggregate Rating 1694, #381&lt;ul&gt;&lt;li&gt;Design Arena (3D): 1070.0 (#126/176)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Mini&lt;/strong&gt; — Aggregate Rating 1691, #390&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.3058 (#3/23)&lt;/li&gt;&lt;li&gt;DAXBench: 96.2 (#4/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 31B&lt;/strong&gt; — Aggregate Rating 1691, #392&lt;ul&gt;&lt;li&gt;DAXBench: 94.5 (#5/50)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 4.38 (#106/109)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 0.0 (#111/114)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond - Reasoning: 9.4 (#114/154)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Life Sciences: 0.0 (#115/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 0.0 (#115/127)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond: 7.2 (#116/154)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Physical Sciences: 0.0 (#116/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences: 0.0 (#116/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Earth Sciences: 0.0 (#116/127)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;KAT Coder Pro V2&lt;/strong&gt; — Aggregate Rating 1690, #395&lt;ul&gt;&lt;li&gt;DAXBench: 84.8 (#19/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4&lt;/strong&gt; — Aggregate Rating 1689, #402&lt;ul&gt;&lt;li&gt;DAXBench: 81.3 (#32/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3.2&lt;/strong&gt; — Aggregate Rating 1687, #410&lt;ul&gt;&lt;li&gt;DAXBench: 79.9 (#37/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Flash Lite (Preview)&lt;/strong&gt; — Aggregate Rating 1686, #413&lt;ul&gt;&lt;li&gt;DAXBench: 97.4 (#1/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3 Ultra&lt;/strong&gt; — Aggregate Rating 1683, #427&lt;ul&gt;&lt;li&gt;DuelLab GameBench 2 - Reasoning Variants: 45.610909360743 (#133/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Mini 4&lt;/strong&gt; — Aggregate Rating 1682, #434&lt;ul&gt;&lt;li&gt;Design Arena (Data Viz): 1175.0 (#93/154)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 Flash&lt;/strong&gt; — Aggregate Rating 1682, #435&lt;ul&gt;&lt;li&gt;DAXBench: 83.1 (#26/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2-Omni&lt;/strong&gt; — Aggregate Rating 1681, #439&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.3366 (#7/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 Flash&lt;/strong&gt; — Aggregate Rating 1681, #441&lt;ul&gt;&lt;li&gt;DAXBench: 93.2 (#8/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2&lt;/strong&gt; — Aggregate Rating 1680, #443&lt;ul&gt;&lt;li&gt;DAXBench: 78.4 (#40/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Seed-1.6&lt;/strong&gt; — Aggregate Rating 1678, #461&lt;ul&gt;&lt;li&gt;SimpleQA Verified: 17.4 (#90/101)&lt;/li&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 5.0 (#146/162)&lt;/li&gt;&lt;li&gt;Epoch AI - GPQA Diamond: 77.59 (#155/334)&lt;/li&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 69.44 (#157/314)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Flash&lt;/strong&gt; — Aggregate Rating 1677, #464&lt;ul&gt;&lt;li&gt;DAXBench: 76.0 (#49/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 35B A3B&lt;/strong&gt; — Aggregate Rating 1675, #478&lt;ul&gt;&lt;li&gt;DAXBench: 79.2 (#38/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Pro 4&lt;/strong&gt; — Aggregate Rating 1674, #484&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 20.92 (#12/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Business: 7.11 (#49/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics: 12.83 (#60/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: -9.78 (#62/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - R: -26.67 (#63/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Law: -3.56 (#64/124)&lt;/li&gt;&lt;li&gt;AA Omniscience: 1.37 (#69/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Humanities &amp; Social Sciences: 4.22 (#70/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Java: -32.73 (#73/124)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 30.86 (#85/117)&lt;/li&gt;&lt;li&gt;...and 24 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek R1&lt;/strong&gt; — Aggregate Rating 1674, #485&lt;ul&gt;&lt;li&gt;DAXBench: 81.3 (#30/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Flash (Preview 09-2025)&lt;/strong&gt; — Aggregate Rating 1668, #510&lt;ul&gt;&lt;li&gt;DAXBench: 83.8 (#23/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3.1&lt;/strong&gt; — Aggregate Rating 1667, #518&lt;ul&gt;&lt;li&gt;DAXBench: 75.9 (#50/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2.7&lt;/strong&gt; — Aggregate Rating 1662, #547&lt;ul&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, No Tools): 28.0 (#25/57)&lt;/li&gt;&lt;li&gt;Gert Labs Rankings: 24.61 (#100/100)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon MoVA 36B A4B&lt;/strong&gt; — Aggregate Rating 1660, #558&lt;ul&gt;&lt;li&gt;AA FrontierCode 2.0: 4.5 (#93/114)&lt;/li&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 8.0 (#94/117)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 19.43 (#95/117)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Life Sciences: 0.0 (#124/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 0.0 (#124/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences: 0.0 (#124/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1: 0.0 (#125/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Physical Sciences: 0.0 (#125/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Earth Sciences: 0.0 (#125/127)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 4.5&lt;/strong&gt; — Aggregate Rating 1654, #593&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 23.44 (#16/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - R: -20.0 (#49/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Business: 4.67 (#57/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Law: -2.44 (#60/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Humanities &amp; Social Sciences: 7.33 (#64/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Java: -30.91 (#69/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Swift: 11.54 (#73/124)&lt;/li&gt;&lt;li&gt;AA Omniscience: -1.8 (#76/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - HTML: 8.33 (#76/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Dart: 33.33 (#82/124)&lt;/li&gt;&lt;li&gt;...and 23 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2.5&lt;/strong&gt; — Aggregate Rating 1651, #621&lt;ul&gt;&lt;li&gt;ReasonScape R12: 927.74 (#19/123)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 26B A4B&lt;/strong&gt; — Aggregate Rating 1646, #645&lt;ul&gt;&lt;li&gt;AA HLE-Diamond - Knowledge: 5.0 (#120/154)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond: 6.2 (#125/154)&lt;/li&gt;&lt;li&gt;AA HLE-Diamond - Reasoning: 7.4 (#125/154)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K-EXAONE 2.0&lt;/strong&gt; — Aggregate Rating 1637, #714&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 30.37 (#30/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - R: -33.33 (#76/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics: 2.83 (#77/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Business: -6.0 (#80/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Java: -40.0 (#82/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: -21.33 (#85/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Law: -13.33 (#88/124)&lt;/li&gt;&lt;li&gt;AA Omniscience: -11.47 (#94/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Go: -25.0 (#96/124)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Kotlin: -31.43 (#96/124)&lt;/li&gt;&lt;li&gt;...and 62 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon 7B&lt;/strong&gt; — Aggregate Rating 1637, #716&lt;ul&gt;&lt;li&gt;AA FrontierCode 2.0: 1.16 (#100/114)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 4.57 (#106/117)&lt;/li&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 1.5 (#109/117)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Life Sciences: 0.0 (#121/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 0.0 (#121/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences: 0.0 (#121/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1: 0.0 (#122/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Physical Sciences: 0.0 (#122/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Earth Sciences: 0.0 (#122/127)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-OSS-120B&lt;/strong&gt; — Aggregate Rating 1634, #731&lt;ul&gt;&lt;li&gt;DAXBench: 85.6 (#17/50)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 0.0 (#114/114)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.0 Flash&lt;/strong&gt; — Aggregate Rating 1633, #736&lt;ul&gt;&lt;li&gt;DAXBench: 76.6 (#46/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-24B-A2B&lt;/strong&gt; — Aggregate Rating 1631, #742&lt;ul&gt;&lt;li&gt;Wolfram LLM Benchmarking Project: 28.5 (#445/614)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3-30B-A3B-Thinking-2507-FP8&lt;/strong&gt; — Aggregate Rating 1631, #746&lt;ul&gt;&lt;li&gt;Medmarks - M-ARC: 63.33333333333333 (#15/89)&lt;/li&gt;&lt;li&gt;Medmarks - Med-HALT Reasoning NOTA: 66.71180931744311 (#22/89)&lt;/li&gt;&lt;li&gt;Medmarks - MedHallu Medium: 63.36477987421384 (#24/89)&lt;/li&gt;&lt;li&gt;Medmarks - MedHallu Easy: 70.07299270072993 (#25/89)&lt;/li&gt;&lt;li&gt;Medmarks - SuperGPQA Medicine Easy: 58.37917125045837 (#27/89)&lt;/li&gt;&lt;li&gt;Medmarks - PubMedQA: 76.6 (#29/89)&lt;/li&gt;&lt;li&gt;Medmarks - Med-HALT Reasoning FCT: 78.8912964969303 (#30/89)&lt;/li&gt;&lt;li&gt;Medmarks - SuperGPQA Medicine Hard: 43.625192012288785 (#30/89)&lt;/li&gt;&lt;li&gt;Medmarks - MedCalc-Bench: 62.090909090909086 (#31/89)&lt;/li&gt;&lt;li&gt;Medmarks - LongHealth Task 2: 88.29166666666667 (#32/89)&lt;/li&gt;&lt;li&gt;...and 18 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Medium 3.5&lt;/strong&gt; — Aggregate Rating 1629, #762&lt;ul&gt;&lt;li&gt;AA FrontierCode 2.0: 1.77 (#97/114)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 4.65 (#105/109)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 4.29 (#107/117)&lt;/li&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 0.5 (#113/117)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.0 Flash (Preview)&lt;/strong&gt; — Aggregate Rating 1628, #769&lt;ul&gt;&lt;li&gt;DAXBench: 76.6 (#47/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4o (2024-08-06)&lt;/strong&gt; — Aggregate Rating 1626, #783&lt;ul&gt;&lt;li&gt;ASCIIEval: 43.4 (#3/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 30B A3B 2507 (Thinking)&lt;/strong&gt; — Aggregate Rating 1623, #809&lt;ul&gt;&lt;li&gt;Medmarks - MedDialog: 88.53333333278802 (#6/12)&lt;/li&gt;&lt;li&gt;Medmarks - HealthBench Hard: 28.069416977380012 (#7/12)&lt;/li&gt;&lt;li&gt;Medmarks - HealthBench: 50.90161247471801 (#8/12)&lt;/li&gt;&lt;li&gt;Medmarks - MedExQA: 80.27925531914893 (#8/12)&lt;/li&gt;&lt;li&gt;Medmarks - HealthBench Consensus: 52.47080214118226 (#9/12)&lt;/li&gt;&lt;li&gt;Medmarks - MedicationQA: 84.67995169052176 (#9/12)&lt;/li&gt;&lt;li&gt;Medmarks - MedCaseReasoning: 29.4 (#10/12)&lt;/li&gt;&lt;li&gt;Medmarks - MEDEC: 40.28475711892797 (#10/12)&lt;/li&gt;&lt;li&gt;Medmarks - MTSamples Procedures: 77.29007633595421 (#10/12)&lt;/li&gt;&lt;li&gt;Medmarks - MTSamples Replicate: 87.96728971999997 (#10/12)&lt;/li&gt;&lt;li&gt;...and 35 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.6-35B-A3B-NVFP4&lt;/strong&gt; — Aggregate Rating 1614, #887&lt;ul&gt;&lt;li&gt;llmbenchmarks.io Local GPU - Chess Puzzles: 6.0 (#3/12)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - SciCode: 33.56 (#5/12)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - SimpleQA Verified: 13.1 (#6/24)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - ARC-AGI-1 Public Eval: 9.25 (#7/12)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - IFEval: 85.4 (#13/24)&lt;/li&gt;&lt;li&gt;llmbenchmarks.io Local GPU - IFBench: 33.14 (#17/24)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 VL 72B Instruct&lt;/strong&gt; — Aggregate Rating 1614, #894&lt;ul&gt;&lt;li&gt;DocHop: 24.4 (#7/21)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Laguna XS 2.1&lt;/strong&gt; — Aggregate Rating 1611, #916&lt;ul&gt;&lt;li&gt;ReasonScape R12: 778.71 (#74/123)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon 3.7B&lt;/strong&gt; — Aggregate Rating 1610, #921&lt;ul&gt;&lt;li&gt;AA FrontierCode 2.0: 0.91 (#102/114)&lt;/li&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 1.0 (#112/117)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 2.57 (#113/117)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Life Sciences: 0.0 (#123/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 0.0 (#123/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences: 0.0 (#123/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1: 0.0 (#124/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Physical Sciences: 0.0 (#124/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Earth Sciences: 0.0 (#124/127)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Phi-4 (Publisher: unsloth)&lt;/strong&gt; — Aggregate Rating 1606, #957&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard - BBH: 68.86 (#124/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MATH Level 5: 50.0 (#140/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard: 40.73 (#217/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MMLU-Pro: 53.78 (#238/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - GPQA: 33.64 (#776/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - IFEval: 68.82 (#825/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MuSR: 41.14 (#2203/4576)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 12B&lt;/strong&gt; — Aggregate Rating 1600, #1008&lt;ul&gt;&lt;li&gt;AutomationBench-AA - Strict: 0.15 (#212/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - HR - Raw Objective Completion: 1.38 (#212/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Marketing - Raw Objective Completion: 0.43 (#222/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Finance - Raw Objective Completion: 0.34 (#223/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA: 0.69 (#224/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Sales - Raw Objective Completion: 1.08 (#224/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Operations - Raw Objective Completion: 0.32 (#225/248)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Support - Raw Objective Completion: 0.55 (#234/248)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 8B Instruct&lt;/strong&gt; — Aggregate Rating 1598, #1037&lt;ul&gt;&lt;li&gt;DocHop: 24.16 (#8/21)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.2 30B&lt;/strong&gt; — Aggregate Rating 1595, #1059&lt;ul&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Life Sciences: 0.0 (#120/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 0.0 (#120/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences: 0.0 (#120/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1: 0.0 (#121/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Physical Sciences: 0.0 (#121/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Earth Sciences: 0.0 (#121/127)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 VL 32B Instruct&lt;/strong&gt; — Aggregate Rating 1589, #1131&lt;ul&gt;&lt;li&gt;DocHop: 21.79 (#9/21)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 30B A3B 2507 Instruct&lt;/strong&gt; — Aggregate Rating 1588, #1149&lt;ul&gt;&lt;li&gt;DNBench - Single - Zero-shot: 48.9 (#1/4)&lt;/li&gt;&lt;li&gt;DNBench - Real World - Zero-shot: 25.3 (#1/4)&lt;/li&gt;&lt;li&gt;DNBench - Component - Violation F1: 67.8 (#1/4)&lt;/li&gt;&lt;li&gt;DNBench - BIRD - Single - Zero-shot: 47.4 (#1/4)&lt;/li&gt;&lt;li&gt;DNBench - BIRD - Real World - Zero-shot: 22.4 (#1/4)&lt;/li&gt;&lt;li&gt;DNBench - Spider - Single - Zero-shot: 49.6 (#1/4)&lt;/li&gt;&lt;li&gt;DNBench - Spider - Real World - Zero-shot: 26.5 (#1/4)&lt;/li&gt;&lt;li&gt;DNBench - Component - Primary Key F1: 61.1 (#2/4)&lt;/li&gt;&lt;li&gt;DNBench - Component - Foreign Key Score: 15.8 (#2/4)&lt;/li&gt;&lt;li&gt;DNBench - Component - LLM Judge: 36.1 (#2/4)&lt;/li&gt;&lt;li&gt;...and 14 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 72B Instruct&lt;/strong&gt; — Aggregate Rating 1587, #1158&lt;ul&gt;&lt;li&gt;ASCIIEval: 33.2 (#7/37)&lt;/li&gt;&lt;li&gt;CSimpleQA: 50.2 (#10/41)&lt;/li&gt;&lt;li&gt;PhysicsFinals: 28.7 (#10/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 405B Instruct&lt;/strong&gt; — Aggregate Rating 1585, #1177&lt;ul&gt;&lt;li&gt;ASCIIEval: 32.31 (#10/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 32B Instruct&lt;/strong&gt; — Aggregate Rating 1578, #1282&lt;ul&gt;&lt;li&gt;ASCIIEval: 31.65 (#11/37)&lt;/li&gt;&lt;li&gt;PhysicsFinals: 27.6 (#11/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 4&lt;/strong&gt; — Aggregate Rating 1577, #1295&lt;ul&gt;&lt;li&gt;Altered Riddles: 0.3487 (#9/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 27B (IT)&lt;/strong&gt; — Aggregate Rating 1572, #1375&lt;ul&gt;&lt;li&gt;ASCIIEval: 35.65 (#5/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;North Mini Code&lt;/strong&gt; — Aggregate Rating 1570, #1408&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 83.5 (#102/124)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 0.19 (#106/114)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 3.42 (#108/109)&lt;/li&gt;&lt;li&gt;AA-Omniscience Accuracy: 18.37 (#109/124)&lt;/li&gt;&lt;li&gt;AA CritPt v2 - Hard Subset: 1.0 (#111/117)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 2.57 (#112/117)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - JavaScript: -2.99 (#115/124)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Life Sciences: 0.0 (#117/127)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences: 0.0 (#117/127)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Rust: 20.59 (#117/124)&lt;/li&gt;&lt;li&gt;...and 23 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM3-VL-2B&lt;/strong&gt; — Aggregate Rating 1569, #1418&lt;ul&gt;&lt;li&gt;LLM Stats (ChartQA): 92.7 (#1/27)&lt;/li&gt;&lt;li&gt;LLM Stats (DocVQA): 96.7 (#1/29)&lt;/li&gt;&lt;li&gt;LLM Stats (InfoVQA): 86.2 (#1/11)&lt;/li&gt;&lt;li&gt;LLM Stats (TextVQA): 87.9 (#1/17)&lt;/li&gt;&lt;li&gt;LLM Stats (BLINK): 71.1 (#5/18)&lt;/li&gt;&lt;li&gt;LLM Stats (DynaMath): 79.2 (#6/10)&lt;/li&gt;&lt;li&gt;LLM Stats (MotionBench): 64.0 (#6/6)&lt;/li&gt;&lt;li&gt;LLM Stats (RefCOCO-avg): 90.5 (#7/10)&lt;/li&gt;&lt;li&gt;LLM Stats (EmbSpatialBench): 82.9 (#8/9)&lt;/li&gt;&lt;li&gt;LLM Stats (Hallusion Bench): 65.6 (#8/21)&lt;/li&gt;&lt;li&gt;...and 24 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.3 70B Instruct&lt;/strong&gt; — Aggregate Rating 1568, #1429&lt;ul&gt;&lt;li&gt;ASCIIEval: 32.74 (#8/37)&lt;/li&gt;&lt;li&gt;PhysicsFinals: 31.5 (#8/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Laguna XS.2&lt;/strong&gt; — Aggregate Rating 1566, #1466&lt;ul&gt;&lt;li&gt;ReasonScape R12: 626.29 (#102/123)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3 Nano Omni 30B A3B&lt;/strong&gt; — Aggregate Rating 1565, #1483&lt;ul&gt;&lt;li&gt;ParseBench - Semantic Formatting: 59.69 (#69/121)&lt;/li&gt;&lt;li&gt;ParseBench - Charts: 31.47 (#73/121)&lt;/li&gt;&lt;li&gt;ParseBench - Tables: 70.44 (#98/121)&lt;/li&gt;&lt;li&gt;ParseBench: 48.51 (#100/121)&lt;/li&gt;&lt;li&gt;ParseBench - Content Faithfulness: 80.97 (#109/121)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2 72B Instruct&lt;/strong&gt; — Aggregate Rating 1559, #1564&lt;ul&gt;&lt;li&gt;IFEval Leaderboard: 80.56 (#3/27)&lt;/li&gt;&lt;li&gt;ASCIIEval: 30.73 (#15/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM3-1.7B&lt;/strong&gt; — Aggregate Rating 1558, #1585&lt;ul&gt;&lt;li&gt;LLM Stats (Multi-IF): 89.1 (#1/26)&lt;/li&gt;&lt;li&gt;LLM Stats (TAU3-Bench): 68.01 (#4/9)&lt;/li&gt;&lt;li&gt;LLM Stats (Claw-Eval): 68.64 (#5/16)&lt;/li&gt;&lt;li&gt;LLM Stats (Humanity's Last Exam (with tools, text-only)): 13.9 (#12/12)&lt;/li&gt;&lt;li&gt;LLM Stats (MCP Atlas): 32.57 (#40/41)&lt;/li&gt;&lt;li&gt;LLM Stats (SWE-bench Multilingual): 23.0 (#51/51)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 2.1): 12.92 (#55/55)&lt;/li&gt;&lt;li&gt;ReasonScape R12: 704.98 (#89/123)&lt;/li&gt;&lt;li&gt;ZeroEval GPQA Diamond: 63.37 (#182/264)&lt;/li&gt;&lt;li&gt;LLM Stats Score: 25.02 (#188/405)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 70B Instruct&lt;/strong&gt; — Aggregate Rating 1555, #1629&lt;ul&gt;&lt;li&gt;ASCIIEval: 31.27 (#13/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.2 8B&lt;/strong&gt; — Aggregate Rating 1551, #1692&lt;ul&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 5.1 (#103/109)&lt;/li&gt;&lt;li&gt;AA FrontierCode 2.0: 0.0 (#109/114)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;diffusiongemma-26B-A4B-it&lt;/strong&gt; — Aggregate Rating 1550, #1701&lt;ul&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, No Tools): 11.0 (#51/57)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4o Mini&lt;/strong&gt; — Aggregate Rating 1550, #1705&lt;ul&gt;&lt;li&gt;BenchLM: 26.6 (#194/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3 70B Instruct&lt;/strong&gt; — Aggregate Rating 1544, #1774&lt;ul&gt;&lt;li&gt;LingOly: 10.3 (#9/11)&lt;/li&gt;&lt;li&gt;ASCIIEval: 30.42 (#17/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen2.5-Math-72B-Instruct&lt;/strong&gt; — Aggregate Rating 1542, #1809&lt;ul&gt;&lt;li&gt;PhysicsFinals: 32.2 (#7/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 14B Instruct&lt;/strong&gt; — Aggregate Rating 1541, #1831&lt;ul&gt;&lt;li&gt;PhysicsFinals: 19.6 (#14/33)&lt;/li&gt;&lt;li&gt;ASCIIEval: 29.14 (#21/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 1.5 110B Chat&lt;/strong&gt; — Aggregate Rating 1537, #1865&lt;ul&gt;&lt;li&gt;ASCIIEval: 30.28 (#18/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 3&lt;/strong&gt; — Aggregate Rating 1536, #1876&lt;ul&gt;&lt;li&gt;PhysicsFinals: 21.8 (#12/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 2 27B (IT)&lt;/strong&gt; — Aggregate Rating 1534, #1903&lt;ul&gt;&lt;li&gt;ASCIIEval: 32.36 (#9/37)&lt;/li&gt;&lt;li&gt;PhysicsFinals: 18.3 (#15/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 VL 7B Instruct&lt;/strong&gt; — Aggregate Rating 1532, #1924&lt;ul&gt;&lt;li&gt;DocHop: 19.05 (#10/21)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 12B (IT)&lt;/strong&gt; — Aggregate Rating 1531, #1934&lt;ul&gt;&lt;li&gt;ASCIIEval: 29.29 (#20/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling 3.0 Tiny&lt;/strong&gt; — Aggregate Rating 1530, #1946&lt;ul&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Success Rate: 96.0 (#313/407)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 76.0 (#349/407)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Exactness: 28.54 (#364/400)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ministral-3-8B-Instruct-2512&lt;/strong&gt; — Aggregate Rating 1530, #1948&lt;ul&gt;&lt;li&gt;LongPIBench - Resume Screening: 96.0 (#3/8)&lt;/li&gt;&lt;li&gt;LongPIBench - Paper Review: 93.0 (#4/8)&lt;/li&gt;&lt;li&gt;LongPIBench - Code Review: 88.0 (#4/8)&lt;/li&gt;&lt;li&gt;LongPIBench - Email Summary: 77.0 (#5/8)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4.1 Nano&lt;/strong&gt; — Aggregate Rating 1519, #2074&lt;ul&gt;&lt;li&gt;BenchLM: 24.6 (#200/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mixtral 8x22B Instruct&lt;/strong&gt; — Aggregate Rating 1514, #2133&lt;ul&gt;&lt;li&gt;SynthPAI: 72.0 (#3/18)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Extraction: 9.55 (#10/50)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Roleplay: 9.05 (#10/50)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Writing: 9.25 (#10/50)&lt;/li&gt;&lt;li&gt;IFEval Leaderboard: 73.08 (#10/27)&lt;/li&gt;&lt;li&gt;CyberCertBench - Fortinet NSE: 55.3 (#12/45)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Coding: 6.45 (#13/50)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Math: 6.9 (#13/50)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Overall: 8.23 (#14/50)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Reasoning: 6.3 (#16/50)&lt;/li&gt;&lt;li&gt;...and 36 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-2.6B&lt;/strong&gt; — Aggregate Rating 1502, #2332&lt;ul&gt;&lt;li&gt;BenchLM: 25.2 (#199/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mixtral 8x22B (Epoch ECI mixed checkpoints)&lt;/strong&gt; — Aggregate Rating 1501, #2337&lt;ul&gt;&lt;li&gt;Epoch AI - ECI: 118.88 (#185/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mixtral-8x22B-v0.1 (Publisher: mistral-community)&lt;/strong&gt; — Aggregate Rating 1500, #2359&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard - GPQA: 33.0 (#886/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MATH Level 5: 15.43 (#1691/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MMLU-Pro: 36.0 (#2063/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard: 16.83 (#3081/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - IFEval: 31.67 (#3219/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - BBH: 38.0 (#3567/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MuSR: 35.33 (#3918/4576)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 2 9B (IT)&lt;/strong&gt; — Aggregate Rating 1499, #2375&lt;ul&gt;&lt;li&gt;ASCIIEval: 30.5 (#16/37)&lt;/li&gt;&lt;li&gt;PhysicsFinals: 11.9 (#21/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Yi 1.5 34B Chat&lt;/strong&gt; — Aggregate Rating 1484, #2605&lt;ul&gt;&lt;li&gt;PhysicsFinals: 17.4 (#16/33)&lt;/li&gt;&lt;li&gt;CSimpleQA: 31.8 (#24/41)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;internlm2.5-20B-chat&lt;/strong&gt; — Aggregate Rating 1481, #2651&lt;ul&gt;&lt;li&gt;CSimpleQA: 32.8 (#23/41)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 7B Instruct&lt;/strong&gt; — Aggregate Rating 1479, #2678&lt;ul&gt;&lt;li&gt;PhysicsFinals: 20.4 (#13/33)&lt;/li&gt;&lt;li&gt;ASCIIEval: 27.57 (#27/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-8B-A1B&lt;/strong&gt; — Aggregate Rating 1478, #2692&lt;ul&gt;&lt;li&gt;BenchLM: 25.2 (#198/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Command-R+&lt;/strong&gt; — Aggregate Rating 1466, #2958&lt;ul&gt;&lt;li&gt;IFEval Leaderboard: 77.01 (#5/27)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2 7B Instruct&lt;/strong&gt; — Aggregate Rating 1461, #3255&lt;ul&gt;&lt;li&gt;ASCIIEval: 27.71 (#26/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek LLM Chat 67B&lt;/strong&gt; — Aggregate Rating 1461, #3285&lt;ul&gt;&lt;li&gt;CSimpleQA: 47.0 (#15/41)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Phi-3 Mini 4K Instruct (Publisher: unsloth)&lt;/strong&gt; — Aggregate Rating 1453, #3653&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard - GPQA: 32.3 (#1063/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MMLU-Pro: 40.31 (#1183/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - BBH: 55.0 (#1213/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard: 27.34 (#1359/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MuSR: 42.84 (#1409/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - IFEval: 54.4 (#1568/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MATH Level 5: 16.39 (#1615/4576)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RusMind Зёрнышко 1.6 (83M)&lt;/strong&gt; — Aggregate Rating 1452, #3681&lt;ul&gt;&lt;li&gt;MERA v2 - RUBIN: 28.2 (#38/47)&lt;/li&gt;&lt;li&gt;MERA v2 - Humor: 3.9 (#43/47)&lt;/li&gt;&lt;li&gt;MERA v2 - NewReasoning: 13.5 (#43/47)&lt;/li&gt;&lt;li&gt;MERA v2 - SAGE: 14.0 (#44/47)&lt;/li&gt;&lt;li&gt;MERA v2 - Characters: 18.7 (#45/47)&lt;/li&gt;&lt;li&gt;MERA v2 - RussianRegions: 2.8 (#45/47)&lt;/li&gt;&lt;li&gt;MERA v2 - Culture Specific: 10.3 (#45/47)&lt;/li&gt;&lt;li&gt;MERA v2 - LIMUR: 22.5 (#46/47)&lt;/li&gt;&lt;li&gt;MERA v2 - Enantiosemy: 5.5 (#46/47)&lt;/li&gt;&lt;li&gt;MERA v2 - IFHardBench: 1.5 (#46/47)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MixTAO-7Bx2-MoE-v8.1 (Publisher: mixtao)&lt;/strong&gt; — Aggregate Rating 1449, #3833&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard - MuSR: 44.63 (#834/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - BBH: 51.89 (#1903/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard: 21.08 (#2479/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MATH Level 5: 9.06 (#2528/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - IFEval: 41.62 (#2626/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MMLU-Pro: 31.23 (#2699/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - GPQA: 28.44 (#2773/4576)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 4B (IT)&lt;/strong&gt; — Aggregate Rating 1447, #3896&lt;ul&gt;&lt;li&gt;ASCIIEval: 27.34 (#28/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Yi-1.5-9B Chat&lt;/strong&gt; — Aggregate Rating 1443, #4103&lt;ul&gt;&lt;li&gt;CSimpleQA: 18.4 (#33/41)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nanbeige4.1-3B&lt;/strong&gt; — Aggregate Rating 1441, #4152&lt;ul&gt;&lt;li&gt;ReasonScape R12: 597.93 (#107/123)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InternLM2.5-Chat-7B&lt;/strong&gt; — Aggregate Rating 1438, #4255&lt;ul&gt;&lt;li&gt;CSimpleQA: 25.7 (#29/41)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 8B Instruct&lt;/strong&gt; — Aggregate Rating 1437, #4318&lt;ul&gt;&lt;li&gt;PhysicsFinals: 11.7 (#23/33)&lt;/li&gt;&lt;li&gt;ASCIIEval: 27.22 (#29/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mixtral 8x7B Instruct&lt;/strong&gt; — Aggregate Rating 1433, #4522&lt;ul&gt;&lt;li&gt;AgentClinic: 37.1 (#3/7)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Writing: 9.35 (#6/50)&lt;/li&gt;&lt;li&gt;LingOly: 14.2 (#7/11)&lt;/li&gt;&lt;li&gt;Kagi LLM Chess Puzzles: 0.9 (#11/11)&lt;/li&gt;&lt;li&gt;BenSyc: 33.2 (#11/15)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Roleplay: 8.95 (#13/50)&lt;/li&gt;&lt;li&gt;BenSyc - Binary Detection: 49.4 (#13/15)&lt;/li&gt;&lt;li&gt;SynthPAI: 52.3 (#14/18)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Humanities: 9.45 (#20/50)&lt;/li&gt;&lt;li&gt;MT-Bench PL - Overall: 7.64 (#22/50)&lt;/li&gt;&lt;li&gt;...and 34 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3 8B Instruct&lt;/strong&gt; — Aggregate Rating 1429, #4721&lt;ul&gt;&lt;li&gt;LingOly: 11.4 (#8/11)&lt;/li&gt;&lt;li&gt;Fin-Bias: 95.4 (#11/19)&lt;/li&gt;&lt;li&gt;ASCIIEval: 28.71 (#22/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mathstral-7B-v0.1&lt;/strong&gt; — Aggregate Rating 1428, #4753&lt;ul&gt;&lt;li&gt;PhysicsFinals: 10.8 (#24/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;quantumaikr llama-2-70B-chat&lt;/strong&gt; — Aggregate Rating 1427, #4788&lt;ul&gt;&lt;li&gt;MMLU-by-task - International Law: 89.26 (#2/1257)&lt;/li&gt;&lt;li&gt;MMLU-by-task - Formal Logic: 55.56 (#3/1257)&lt;/li&gt;&lt;li&gt;MMLU-by-task - High School World History: 90.72 (#4/1257)&lt;/li&gt;&lt;li&gt;MMLU-by-task - Machine Learning: 59.82 (#5/1257)&lt;/li&gt;&lt;li&gt;MMLU-by-task - High School Physics: 49.67 (#6/1257)&lt;/li&gt;&lt;li&gt;MMLU-by-task - Management: 84.47 (#8/1257)&lt;/li&gt;&lt;li&gt;MMLU-by-task - Marketing: 91.45 (#8/1257)&lt;/li&gt;&lt;li&gt;MMLU-by-task - High School European History: 85.45 (#13/1257)&lt;/li&gt;&lt;li&gt;MMLU-by-task - Professional Accounting: 57.45 (#16/1257)&lt;/li&gt;&lt;li&gt;MMLU-by-task - Public Relations: 74.55 (#16/1257)&lt;/li&gt;&lt;li&gt;...and 52 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AlphaMonarch-7B&lt;/strong&gt; — Aggregate Rating 1427, #4821&lt;ul&gt;&lt;li&gt;IFEval Leaderboard: 53.72 (#26/27)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nous-Hermes-2-Mistral-7B-DPO&lt;/strong&gt; — Aggregate Rating 1426, #4870&lt;ul&gt;&lt;li&gt;IFEval Leaderboard: 59.68 (#21/27)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MixTAO-7Bx2-MoE-v8.1 (Publisher: zhengr)&lt;/strong&gt; — Aggregate Rating 1423, #5017&lt;ul&gt;&lt;li&gt;Open LLM Leaderboard - GPQA: 29.87 (#2099/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - IFEval: 41.88 (#2606/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MuSR: 39.76 (#2742/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MATH Level 5: 6.04 (#3058/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard: 17.07 (#3061/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - MMLU-Pro: 28.47 (#3100/4576)&lt;/li&gt;&lt;li&gt;Open LLM Leaderboard - BBH: 42.02 (#3304/4576)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 3B Instruct&lt;/strong&gt; — Aggregate Rating 1419, #5216&lt;ul&gt;&lt;li&gt;CSimpleQA: 17.3 (#35/41)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 1.5 7B Chat&lt;/strong&gt; — Aggregate Rating 1389, #6558&lt;ul&gt;&lt;li&gt;ASCIIEval: 26.71 (#31/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Baichuan2-13B-Chat&lt;/strong&gt; — Aggregate Rating 1386, #6681&lt;ul&gt;&lt;li&gt;CSimpleQA: 21.8 (#31/41)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral 7B Instruct (v0.3)&lt;/strong&gt; — Aggregate Rating 1383, #6777&lt;ul&gt;&lt;li&gt;PhysicsFinals: 11.7 (#22/33)&lt;/li&gt;&lt;li&gt;ASCIIEval: 25.57 (#34/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 2 70B Chat&lt;/strong&gt; — Aggregate Rating 1378, #6907&lt;ul&gt;&lt;li&gt;FinBen: 4.02 (#4/6)&lt;/li&gt;&lt;li&gt;MoralChoice: 99.1 (#8/21)&lt;/li&gt;&lt;li&gt;T-Eval: 53.0 (#10/21)&lt;/li&gt;&lt;li&gt;LingOly: 6.4 (#10/11)&lt;/li&gt;&lt;li&gt;SynthPAI: 56.8 (#11/18)&lt;/li&gt;&lt;li&gt;AgentBoard: 23.8 (#11/13)&lt;/li&gt;&lt;li&gt;ASCIIEval: 28.08 (#25/37)&lt;/li&gt;&lt;li&gt;VisualWebArena: 1.1 (#29/32)&lt;/li&gt;&lt;li&gt;CyberCertBench - ISA/IEC 62443: 54.7 (#32/45)&lt;/li&gt;&lt;li&gt;CyberCertBench - Fortinet NSE: 44.0 (#39/45)&lt;/li&gt;&lt;li&gt;...and 3 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 2 2B (IT)&lt;/strong&gt; — Aggregate Rating 1376, #6954&lt;ul&gt;&lt;li&gt;PhysicsFinals: 6.1 (#27/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral 7B Instruct (v0.2)&lt;/strong&gt; — Aggregate Rating 1372, #7120&lt;ul&gt;&lt;li&gt;ASCIIEval: 26.28 (#32/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen-7B Chat&lt;/strong&gt; — Aggregate Rating 1368, #7215&lt;ul&gt;&lt;li&gt;ASCIIEval: 23.3 (#37/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Baichuan2-7B-Chat&lt;/strong&gt; — Aggregate Rating 1335, #8078&lt;ul&gt;&lt;li&gt;CSimpleQA: 14.0 (#38/41)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 2 13B Chat&lt;/strong&gt; — Aggregate Rating 1322, #8303&lt;ul&gt;&lt;li&gt;ASCIIEval: 25.93 (#33/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Instruct v0.1 7B&lt;/strong&gt; — Aggregate Rating 1317, #8392&lt;ul&gt;&lt;li&gt;ASCIIEval: 26.88 (#30/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 2 7B Chat&lt;/strong&gt; — Aggregate Rating 1284, #8922&lt;ul&gt;&lt;li&gt;ASCIIEval: 24.59 (#36/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;gemma-7B (IT)&lt;/strong&gt; — Aggregate Rating 1283, #8946&lt;ul&gt;&lt;li&gt;LingOly: 4.9 (#11/11)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen2.5-Math-1.5B-Instruct&lt;/strong&gt; — Aggregate Rating 1255, #9209&lt;ul&gt;&lt;li&gt;PhysicsFinals: 16.4 (#17/33)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (96)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance: 50.88 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Company Research: 46.28 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Coverage &amp; Catalyst Monitoring: 45.51 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Earnings &amp; Events: 66.37 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Financial Data &amp; Modeling: 57.25 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Screening &amp; Discovery: 35.59 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Sector, Industry &amp; Macro: 38.98 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on SEAL - MCP Atlas: 85.4 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Design Arena (SVG): 1412.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on SEAL - Fortress: 12.14 (#72)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on ComplexConstraints: 70.1 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on EnterpriseBench (CoreCraft Agents): 50.3 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on PFArena - Anchor-Informed Multi-Mutant Ranking: 0.3299 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on PFArena - Measurement-Free Multi-Mutant Ranking: 0.2514 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on PFArena - Single-Mutant-Informed Multi-Mutant Ranking: 0.6006 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Riemann-bench: 79.2 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on SpacetimeDB LLM Benchmark (C#): 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on SpacetimeDB LLM Benchmark (Rust): 98.4 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on TAC (Travel Agent Compassion): 40.4 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle FACTS (Google): 72.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PFArena - Measurement-Free Multi-Mutant Ranking: 0.2766 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PFArena - Single-Mutant-Informed Multi-Mutant Ranking: 0.5826 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Riemann-bench: 74.4 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SpacetimeDB LLM Benchmark (Rust): 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on TAC (Travel Agent Compassion): 57.1 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on TaxCalcBench: 40.82 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA CritPt v2: 82.29 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA CritPt v2 - Hard Subset: 69.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA FrontierCode 2.0: 39.45 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf: 29.6 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Construction - Criterion Pass Rate: 78.47 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Criterion Pass Rate: 81.68 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Engineering - Criterion Pass Rate: 82.56 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 84.75 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - HR - Criterion Pass Rate: 81.34 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Healthcare - Criterion Pass Rate: 88.2 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Insurance - Criterion Pass Rate: 90.93 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Legal - Criterion Pass Rate: 82.3 (#49)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 78.29 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Real Estate - Criterion Pass Rate: 75.3 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - STEM/Research - Criterion Pass Rate: 79.94 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA HLE-Diamond: 67.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA HLE-Diamond - Knowledge: 59.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA HLE-Diamond - Reasoning: 75.8 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA MMMU-Pro: 89.25 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Omniscience: 46.77 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench 4.0: 62.63 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1: 62.38 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Earth Sciences: 62.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Engineering Sciences: 55.56 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Life Sciences: 56.14 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Mathematical Sciences: 88.24 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Physical Sciences: 56.86 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase: 62.32 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - Analytical Quality Elo: 1634.69 (#48)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - DOCX - Rubric Pass Rate: 62.13 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - PDF - Rubric Pass Rate: 65.52 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - PPTX - Rubric Pass Rate: 64.08 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - Presentation Elo: 1587.56 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - XLSX - Rubric Pass Rate: 59.32 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Accuracy: 49.23 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Hallucination Rate: 4.24 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Business: 31.56 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Health: 36.44 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Humanities &amp; Social Sciences: 50.22 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Law: 45.11 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Science, Engineering &amp; Mathematics: 43.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE): 67.83 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - C: 85.45 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Dart: 80.56 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Go: 52.78 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - HTML: 61.11 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Java: 52.73 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 94.03 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Julia: 63.16 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 25.71 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - PHP: 73.53 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Python: 70.21 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - R: 26.67 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Rust: 91.18 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Swift: 57.69 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 76.74 (#31)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Artificial Analysis Intelligence Index: 58.26 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA: 77.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Finance - Raw Objective Completion: 95.95 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - HR - Raw Objective Completion: 80.81 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Marketing - Raw Objective Completion: 94.25 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Operations - Raw Objective Completion: 97.39 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Sales - Raw Objective Completion: 93.65 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Strict: 54.19 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Support - Raw Objective Completion: 94.03 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Riemann-bench: 80.0 (#3)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (48)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;llmbenchmarks.io Local GPU - ARC-AGI-1 Public Eval&lt;/strong&gt;: Qwen 3.5 4B (Non-reasoning) (38.0) beat Qwen3.6-27B-NVFP4 (Non-reasoning) by 25.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Spireclimb Sts1 A20 Heart&lt;/strong&gt;: GPT-6 Astra (Max) (33.33) beat GPT-6.1 Sol (Max) by 13.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WDCD R3 Pressure Integrity&lt;/strong&gt;: Gemini 3.1 Pro (Preview) (163.0) beat Claude Opus 4.7 by 13.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Learned Transformer Layer Mapping&lt;/strong&gt;: Gemini 4 Carbon (82.1614643729987) beat Gemini 4 Argon by 12.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Hypergraph Core Numbers&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (70.61820270770313) beat Claude Fable 5.1 (Max) by 10.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (Video)&lt;/strong&gt;: gemini-omni-flash-1.2 (1370.0) beat gemini-omni-flash-1.1 by 10.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AIR-Bench Long-Doc&lt;/strong&gt;: piragi (Reranker: bge-reranker-v2-gemma) (84.96) beat BM25 (Reranker: mmarco-mMiniLMv2-L12-H384-v1) by 9.95&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Humanity's Last Exam (Self-Reported, No Tools)&lt;/strong&gt;: Kimi K3.1 Pro (51.1) beat Ornith-1.5-397B by 6.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Humanity's Last Exam (Self-Reported, With Tools)&lt;/strong&gt;: Kimi K3.1 Pro (70.0) beat DeepSeek V4.1 Flash by 6.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Legal Agent Benchmark)&lt;/strong&gt;: GLM-5.3-FlashX-Legal (19.2) beat Claude Fable 5 by 5.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - E-Graph Invariant Restoration&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (67.30846631487712) beat Kimi K3.1 Pro by 5.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agent Security League - Functional Correctness&lt;/strong&gt;: Muse Wonder (Muse Code) (91.6) beat Claude Fable 5.1 (Claude Code) by 4.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWT-Bench Verified&lt;/strong&gt;: Bugstack + Claude Sonnet 5.5 + Opus 5.5 (91.5) beat DPIAgent + GPT-5 (with test selection) by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Learned Transformer Layer Mapping&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (73.72019487756762) beat GPT-6 Astra (xHigh) by 3.72&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coarena - Task Completion&lt;/strong&gt;: hy4 (97.7) beat Claude Haiku 5.5 by 3.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (InfoVQA)&lt;/strong&gt;: LFM3-VL-2B (86.2) beat Qwen 2.5 VL 32B Instruct by 2.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SEAL - MCP Atlas&lt;/strong&gt;: Gemini 4 Argon (90.7) beat Muse Spark 1.1 by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierFinance - Coverage &amp; Catalyst Monitoring&lt;/strong&gt;: Muse Wonder (56.22) beat Claude Fable 5.1 (High) by 2.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (TextVQA)&lt;/strong&gt;: LFM3-VL-2B (87.9) beat Qwen 2 VL 72B Instruct by 2.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Tax Agent Bench&lt;/strong&gt;: Gemini 4 Carbon (High) (51.785) beat qwen3.8-max-internal by 2.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (ChartQA)&lt;/strong&gt;: LFM3-VL-2B (92.7) beat Claude 3.5 Sonnet by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Insurance - Criterion Pass Rate&lt;/strong&gt;: Gemini 4 Carbon (Medium) (90.93) beat Claude Fable 5.1 (Max, Default Fallback) by 1.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierFinance - Sector, Industry &amp; Macro&lt;/strong&gt;: Muse Spark Open (xHigh) (44.13) beat Claude Fable 5.1 (High) by 1.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA MMMU-Pro&lt;/strong&gt;: Gemini 4 Carbon (Medium) (89.25) beat Claude Opus 5.5 (Max, Default Fallback) by 1.56&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Multi-IF)&lt;/strong&gt;: LFM3-1.7B (89.1) beat Ling-3.0-flash by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-AnalystAgent - Pass@K&lt;/strong&gt;: Inkling 2 Small (86.25) beat GPT-6 Luna (Max) by 1.25&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SpacetimeDB LLM Benchmark (C#)&lt;/strong&gt;: Gemini 4 Argon (100.0) beat Claude Sonnet 4.6 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Exact Sequence Alignment&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (67.84462833266464) beat GPT-6 Astra (xHigh) by 1.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - ParButterfly Exact Counting&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (58.29391835181178) beat Claude Fable 5.1 (Max) by 0.98&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SpacetimeDB LLM Benchmark (TypeScript)&lt;/strong&gt;: Claude Opus 5.5 (100.0) beat GPT-6 Astra by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-AnalystAgent - Pass@1&lt;/strong&gt;: Inkling 2 Small (71.25) beat Gemini 3.7 Flash (High) by 0.75&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench&lt;/strong&gt;: PipesHub (pipeshub.com) (93.24) beat codos.ai by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Correctness&lt;/strong&gt;: PipesHub (pipeshub.com) (95.0) beat codos.ai by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Approximate-MIPS Query Stopping&lt;/strong&gt;: Claude Fable 5.1 (Max) (87.27679649790228) beat GLM-5.3 (Max) by 0.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU DBSCAN Kernel Optimization&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (73.3787292124976) beat Claude Opus 5.5 (Max) by 0.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Completeness&lt;/strong&gt;: PipesHub (pipeshub.com) (95.63) beat codos.ai by 0.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU IVF-PQ Search Kernel Optimization&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (80.99348237443567) beat Claude Opus 5.5 (Max) by 0.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU K-Means Kernel Optimization&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (54.23780982604794) beat Claude Opus 5.5 (Max) by 0.45&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JSQuAD&lt;/strong&gt;: Kimi K3.1 Pro (Max) (83.05) beat Qwen 3 Next 80B A3B Instruct by 0.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Approximate-MIPS Query Stopping&lt;/strong&gt;: Gemini 4 Argon (81.86985817344119) beat Gemini 4 Carbon by 0.37&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GAIA&lt;/strong&gt;: Keen AgenticOS V1.1 (94.35) beat Shadow Harness V8.01 by 0.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (DocVQA)&lt;/strong&gt;: LFM3-VL-2B (96.7) beat Qwen 2.5 VL 72B Instruct by 0.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - KAMI FP64 Block GEMM — trusted batched harness adaptation&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (23.310051695541144) beat Gemini 4 Argon (High) by 0.28&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Recall&lt;/strong&gt;: Neurond AI (neurond.com) (96.69) beat codos.ai by 0.24&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierFinance - Screening &amp; Discovery&lt;/strong&gt;: Muse Wonder (39.94) beat Claude Fable 5.1 (High) by 0.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PFArena - Anchor-Informed Multi-Mutant Ranking&lt;/strong&gt;: Gemini 4 Argon (0.3822) beat Claude Opus 5 by 0.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Hallucination Rate&lt;/strong&gt;: Gemini 4 Carbon (Medium) (4.24) beat Gemini 4 Argon (High) by 0.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sonar LLM Leaderboard - Java - Bug Density&lt;/strong&gt;: Kimi K3 (Max) (0.51) beat GPT-5.5 (Medium) by 0.01&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-08

=== WEEKLY ===
NEW MODELS (207)
  - Gemini 4 Carbon — Aggregate Rating 1989, #1/10916 (above GPT-6 Astra Pro)
      AA MMMU-Pro: 89.25 (#1/302)
      AA-Omniscience Hallucination Rate: 4.24 (#1/124)
      AA HLE-Diamond - Knowledge: 59.0 (#1/154)
      AA Terminal-</summary></entry><entry><title>The Aggregate Digest — 2026-11-07</title><id>https://theaggregate.ai/digest/2026-11-07</id><updated>2026-11-07T08:47:10.517279+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Frontier&lt;/h2&gt;
&lt;ul&gt;&lt;li&gt;Best open-weights model: Kimi K3.1 Pro (1906) takes the crown from DeepSeek V4.1 Flash (1017) (1862), which had trailed for 3 days&lt;/li&gt;&lt;li&gt;Best available model: hy4 enters at #20 (Aggregate Rating 1894) on 89 benchmarks&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;VIALS-AA&lt;/strong&gt; (Pass^3 (%)): leader Gemini 4 Carbon (High) (63.98), 19 models&lt;br&gt;&lt;span&gt;Artificial Analysis&amp;#x27; implementation of Handshake AI&amp;#x27;s VIALS (life-science image interpretation: blots, plasmid maps, phylogenetics, flow cytometry, molecular and protein structures). AA evaluates the released 161-task set; models get the question and image file paths, a sandbox with Python and no network, and an image-viewing tool if vision-capable, through AA&amp;#x27;s Stirrup harness. Three independent repeats per task, graded by GPT-6 Luna (medium) with the upstream judge prompts; a missing answer or refusal scores zero. This board reports pass^3: share of the 161 tasks answered correctly in all three repeats. Distinct from the paper&amp;#x27;s direct-VLM &amp;#x27;VIALS&amp;#x27; board. Rows keep AA&amp;#x27;s effort labels (for example Max, High, Xhigh).&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle FACTS (Google): 72.4 (#3)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (11)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Hypergraph Core Numbers&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (70.61820270770313) beat Claude Fable 5.1 (Max) by 10.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AIR-Bench Long-Doc&lt;/strong&gt;: piragi (Reranker: bge-reranker-v2-gemma) (84.96) beat BM25 (Reranker: mmarco-mMiniLMv2-L12-H384-v1) by 9.95&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - E-Graph Invariant Restoration&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (67.30846631487712) beat Kimi K3.1 Pro by 5.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Learned Transformer Layer Mapping&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (73.72019487756762) beat Gemini 4 Argon (High) by 3.72&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Exact Sequence Alignment&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (67.84462833266464) beat GPT-6 Astra (xHigh) by 1.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - ParButterfly Exact Counting&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (58.29391835181178) beat Claude Fable 5.1 (Max) by 0.98&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU DBSCAN Kernel Optimization&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (73.3787292124976) beat Claude Opus 5.5 (Max) by 0.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU IVF-PQ Search Kernel Optimization&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (80.99348237443567) beat Claude Opus 5.5 (Max) by 0.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU K-Means Kernel Optimization&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (54.23780982604794) beat Claude Opus 5.5 (Max) by 0.45&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nejumi 4 - jaster (2-shot) - JSQuAD&lt;/strong&gt;: Kimi K3.1 Pro (Max) (83.05) beat Qwen 3 Next 80B A3B Instruct by 0.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - KAMI FP64 Block GEMM — trusted batched harness adaptation&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (23.310051695541144) beat Gemini 4 Argon (High) by 0.28&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-07

=== FRONTIER ===
  - Best open-weights model: Kimi K3.1 Pro (1906) takes the crown from DeepSeek V4.1 Flash (1017) (1862), which had trailed for 3 days
  - Best available model: hy4 enters at #20 (Aggregate Rating 1894) on 89 benchmarks

=== DAILY ===
NEW BENCHMARK</summary></entry><entry><title>The Aggregate Digest — 2026-11-06</title><id>https://theaggregate.ai/digest/2026-11-06</id><updated>2026-11-06T09:00:14.155977+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (3)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Anthropic ECI - October 2026 Fit (METR Assessment)&lt;/strong&gt; (AECI Score): leader Claude Model 4 (METR October 2026) (178.6), 21 models&lt;br&gt;&lt;span&gt;METR&amp;#x27;s November 5, 2026 assessment of Anthropic model capability through October 2026. AECI is an unbounded IRT index combining public and internal benchmarks; higher is better. This separate fit includes all 21 points in METR&amp;#x27;s main chart, including private models, distinct early snapshots and non-frontier models. Model 4&amp;#x27;s 178.6 is printed in the report; the other scores are approximate, digitized from the original figure to 0.1 AECI. Extraction precision is separate from the plotted statistical confidence intervals. The chart dates describe availability to most Anthropic employees, not public releases. Model 2, Model 3 and Model 4 are report-specific anonymous identities; no released-model mapping or effort setting is assumed. Scores are not interchangeable with earlier Anthropic system-card fits or public Epoch ECI. The exact benchmark basket and scored questions are not disclosed.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GM League Arena - NBA Elo&lt;/strong&gt; (Mean season-end Elo over five simulated NBA seasons (2022-2026; ratings reset to 1500 each season, K=20, 400-point logistic scale; one shared league of 29 LLM general managers and a rule-based Native GM team; official APIs, high reasoning effort where supported; one full-horizon run per model)): leader Claude Opus 5.5 (1977.3), 29 models&lt;br&gt;&lt;span&gt;GM League Arena (arXiv 2611.05147v1; called GM Bench in the paper) makes each LLM the general manager of one team in a shared simulated NBA league for five seasons (2022–2026), &amp;quot;managing players and assets through selection, trades, and contract renewals&amp;quot;. Table 1 Elo is &amp;quot;the arithmetic mean of season-end ratings over the five seasons, with ratings initialized to 1500 at the start of each season&amp;quot;, so it is relative to the other 29 teams in this league; a rule-based Native GM team takes part but is not a model row. &amp;quot;Configurable reasoning effort is set to high for every model that supports this setting&amp;quot;; each model has &amp;quot;one complete five-season run&amp;quot; and no confidence intervals are reported. Metric: Mean season-end Elo.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (LifeSci Diamond (2026-11); 4 November 2026 edition)&lt;/strong&gt; (Strict pass rate (%)): leader GPT-6 Astra (20.0), 18 models&lt;br&gt;&lt;span&gt;Arca’s 4 November 2026 edition regrades 55 regulatory affairs tasks with a revised strict-pass rubric and one GPT-6.1 Sol judge at medium reasoning effort, including OpenAI contestants. The native benchmark version is 2026-11-03-gdp-v6. The historical answer batches include 102 answers refreshed November 4. Ten example tasks are public; 45 are held out and 11 answer keys lack expert review. Eighteen plain APIs use the shared Vercel AI Gateway reference harness, Exa search snippets and up to 40 model steps. Effective reasoning, temperature and token settings are unrecorded provider defaults; gateway aliases are not immutable snapshots. The relay scores are checked against the native CSV. Strict pass is separate from partial credit and Arca workflow results.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Inkling 2 Small&lt;/strong&gt; — Aggregate Rating 1822, #50&lt;ul&gt;&lt;li&gt;AA-AnalystAgent - Pass@1: 71.25 (#1/52)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 86.25 (#1/52)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 90.4 (#3/245)&lt;/li&gt;&lt;li&gt;EnterpriseOps-Gym-AA - Drive: 63.81 (#3/51)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 71.8 (#3/55)&lt;/li&gt;&lt;li&gt;EnterpriseOps-Gym-AA - Hybrid: 43.14 (#4/51)&lt;/li&gt;&lt;li&gt;EnterpriseOps-Gym-AA - Teams: 50.0 (#4/51)&lt;/li&gt;&lt;li&gt;EnterpriseOps-Gym-AA - Email: 66.35 (#8/51)&lt;/li&gt;&lt;li&gt;EnterpriseOps-Gym-AA - Calendar: 47.12 (#9/51)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent: 51.25 (#10/52)&lt;/li&gt;&lt;li&gt;...and 79 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (11)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on SEAL - MCP Atlas: 85.4 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on EnterpriseBench (CoreCraft Agents): 50.3 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on SpacetimeDB LLM Benchmark (C#): 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on SpacetimeDB LLM Benchmark (Rust): 98.4 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on TAC (Travel Agent Compassion): 40.4 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SpacetimeDB LLM Benchmark (Rust): 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SpacetimeDB LLM Benchmark (TypeScript): 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on TAC (Travel Agent Compassion): 57.1 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on TAC (Travel Agent Compassion) - With Welfare Guidance: 100.0 (#2)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (6)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;SEAL - MCP Atlas&lt;/strong&gt;: Gemini 4 Argon (90.7) beat Muse Spark 1.1 by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-AnalystAgent - Pass@K&lt;/strong&gt;: Inkling 2 Small (86.25) beat GPT-6 Luna (Max) by 1.25&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SpacetimeDB LLM Benchmark (C#)&lt;/strong&gt;: Gemini 4 Argon (100.0) beat Claude Sonnet 4.6 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-AnalystAgent - Pass@1&lt;/strong&gt;: Inkling 2 Small (71.25) beat Gemini 3.7 Flash (High) by 0.75&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coarena - Task Completion&lt;/strong&gt;: Hy4 (98.2) beat Muse Spark 1.3 by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sonar LLM Leaderboard - Java - Bug Density&lt;/strong&gt;: Kimi K3 (Max) (0.51) beat GPT-5.5 (Medium) by 0.01&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-06

=== DAILY ===
NEW BENCHMARKS (3)
  - Anthropic ECI - October 2026 Fit (METR Assessment) (AECI Score): leader Claude Model 4 (METR October 2026) (178.6), 21 models
      METR's November 5, 2026 assessment of Anthropic model capability through October 2026. AECI is a</summary></entry><entry><title>The Aggregate Digest — 2026-11-05</title><id>https://theaggregate.ai/digest/2026-11-05</id><updated>2026-11-05T09:17:21.901271+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;The Aggregate - Personal Style Imitation (53 Texts)&lt;/strong&gt; (Style match (%)): leader Claude Fable 5.1 (Max) (76.7), 13 models&lt;br&gt;&lt;span&gt;The Aggregate runs this evaluation itself. The site&amp;#x27;s author chose 53 of their longest English messages (chat replies, Discord posts and prompts typed to AI assistants), each reviewed as entirely their own words, and a separate model rewrote each one in plain, neutral English with the same content. Each model sees 30 other messages by the author as style samples and one neutral draft at a time, at its highest reasoning effort, and rewrites the draft so that it reads as if the author wrote it. A classifier trained on the author&amp;#x27;s other writing, against other people&amp;#x27;s ChatGPT prompts and chat messages and against GPT- and Claude-written text, gives the probability that the author wrote each rewrite; it sees function words, punctuation, capitalisation and word shapes, not topics. A rewrite that changes the meaning scores 0. The score is the mean probability as a percentage; the author&amp;#x27;s own held-out messages and the unchanged drafts, scored the same way, are shown as reference rows. One author and 53 messages: close scores do not establish an ordering.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;hy4&lt;/strong&gt; — Aggregate Rating 1944, #13&lt;ul&gt;&lt;li&gt;FoodTruckBench: 67597.64 (#2/60)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 1802.0 (#2/287)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Reasoning): 1810.0 (#5/350)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2.1: 64.705 (#11/36)&lt;/li&gt;&lt;li&gt;Vals AI Finance Index: 58.05 (#12/36)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 59.353 (#13/33)&lt;/li&gt;&lt;li&gt;Vals AI Legal Index: 70.552 (#14/34)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 34.399 (#15/81)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 39.394 (#15/60)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench v1.1: 12.5 (#16/34)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (2)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on SEAL - Fortress: 12.14 (#72)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on TaxCalcBench: 40.82 (#20)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (4)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Humanity's Last Exam (Self-Reported, No Tools)&lt;/strong&gt;: Kimi K3.1 Pro (51.1) beat Ornith-1.5-397B by 6.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Humanity's Last Exam (Self-Reported, With Tools)&lt;/strong&gt;: Kimi K3.1 Pro (70.0) beat DeepSeek V4.1 Flash by 6.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWT-Bench Verified&lt;/strong&gt;: Bugstack + Claude Sonnet 5.5 + Opus 5.5 (91.5) beat DPIAgent + GPT-5 (with test selection) by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GAIA&lt;/strong&gt;: Keen AgenticOS V1.1 (94.35) beat Shadow Harness V8.01 by 0.33&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-05

=== DAILY ===
NEW BENCHMARKS (1)
  - The Aggregate - Personal Style Imitation (53 Texts) (Style match (%)): leader Claude Fable 5.1 (Max) (76.7), 13 models
      The Aggregate runs this evaluation itself. The site's author chose 53 of their longest English messages</summary></entry><entry><title>The Aggregate Digest — 2026-11-04</title><id>https://theaggregate.ai/digest/2026-11-04</id><updated>2026-11-04T08:43:01.421760+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (LifeSci Diamond (2026-11))&lt;/strong&gt; (Strict pass rate (%)): leader Muse Wonder (27.3), 18 models&lt;br&gt;&lt;span&gt;Strict pass rate over 55 regulatory affairs tasks in Arca’s November 2026 LifeSci Diamond cohort. Eighteen plain APIs use a shared Vercel AI Gateway reference harness with Exa web search, up to 40 model steps and provider defaults. Every rubric requirement must pass with no fail line; one answer per task is graded by an LLM jury. Results combine September 25–November 3 answer batches, rather than a fresh November rerun. Gateway API aliases do not identify immutable snapshots; effective reasoning, temperature and token settings were not recorded. Eleven answer keys lack expert review. Arca’s separate workflows use different tools and are measured separately.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (8)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance: 50.88 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Company Research: 46.28 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Coverage &amp; Catalyst Monitoring: 45.51 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Earnings &amp; Events: 66.37 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Financial Data &amp; Modeling: 57.25 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Screening &amp; Discovery: 35.59 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on FrontierFinance - Sector, Industry &amp; Macro: 38.98 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Design Arena (SVG): 1412.0 (#3)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (7)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Agent Security League - Functional Correctness&lt;/strong&gt;: Muse Wonder (Muse Code) (91.6) beat Claude Fable 5.1 (Claude Code) by 4.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierFinance - Coverage &amp; Catalyst Monitoring&lt;/strong&gt;: Muse Wonder (56.22) beat Claude Fable 5.1 (High) by 2.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierFinance - Sector, Industry &amp; Macro&lt;/strong&gt;: Muse Spark Open (xHigh) (44.13) beat Claude Fable 5.1 (High) by 1.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench&lt;/strong&gt;: PipesHub (pipeshub.com) (93.24) beat codos.ai by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Correctness&lt;/strong&gt;: PipesHub (pipeshub.com) (95.0) beat codos.ai by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Completeness&lt;/strong&gt;: PipesHub (pipeshub.com) (95.63) beat codos.ai by 0.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierFinance - Screening &amp; Discovery&lt;/strong&gt;: Muse Wonder (39.94) beat Claude Fable 5.1 (High) by 0.09&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-04

=== DAILY ===
NEW BENCHMARKS (1)
  - LLM Stats (LifeSci Diamond (2026-11)) (Strict pass rate (%)): leader Muse Wonder (27.3), 18 models
      Strict pass rate over 55 regulatory affairs tasks in Arca’s November 2026 LifeSci Diamond cohort. Eighteen plain APIs use a</summary></entry><entry><title>The Aggregate Digest — 2026-11-03</title><id>https://theaggregate.ai/digest/2026-11-03</id><updated>2026-11-03T16:56:13.021806+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (21)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;CloudOpsBench v1.0&lt;/strong&gt; (Resolution rate (%)): leader Claude Fable 5.1 (Harness: Claude Code) (Max) (60.6), 10 models&lt;br&gt;&lt;span&gt;CloudOpsBench v 1.0:45 seeded AWS incident tasks,4 trials each,40 minutes. Native state-based resolution rate, passes/(passes+failures+timeouts), evaluation errors excluded. Model,effort andharness kept distinct.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FDE Requirement Discovery - Extreme ambiguity&lt;/strong&gt; (Recovery rate (%)): leader Claude Opus 5.5 (83.5), 15 models&lt;br&gt;&lt;span&gt;Enterprise text-to-SQL FDE study Table 1:10 independent simulated-customer discovery trajectories per model and ambiguity level; recovery of 17 known core requirements, judged against frozen annotations. The three ambiguity levels and their mean remain separate. Dialogue counts do not measure capability and delivery-agent training outcomes are different protocols. No unreported reasoning setting is inferred. Metric: Recovery rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FDE Requirement Discovery - Low ambiguity&lt;/strong&gt; (Recovery rate (%)): leader Claude Opus 5 (81.4), 15 models&lt;br&gt;&lt;span&gt;Enterprise text-to-SQL FDE study Table 1:10 independent simulated-customer discovery trajectories per model and ambiguity level; recovery of 17 known core requirements, judged against frozen annotations. The three ambiguity levels and their mean remain separate. Dialogue counts do not measure capability and delivery-agent training outcomes are different protocols. No unreported reasoning setting is inferred. Metric: Recovery rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FDE Requirement Discovery - Mean&lt;/strong&gt; (Recovery rate (%)): leader Claude Opus 5.5 (81.7), 15 models&lt;br&gt;&lt;span&gt;Enterprise text-to-SQL FDE study Table 1:10 independent simulated-customer discovery trajectories per model and ambiguity level; recovery of 17 known core requirements, judged against frozen annotations. The three ambiguity levels and their mean remain separate. Dialogue counts do not measure capability and delivery-agent training outcomes are different protocols. No unreported reasoning setting is inferred. Metric: Recovery rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FDE Requirement Discovery - Moderate ambiguity&lt;/strong&gt; (Recovery rate (%)): leader Claude Opus 5.5 (84.4), 15 models&lt;br&gt;&lt;span&gt;Enterprise text-to-SQL FDE study Table 1:10 independent simulated-customer discovery trajectories per model and ambiguity level; recovery of 17 known core requirements, judged against frozen annotations. The three ambiguity levels and their mean remain separate. Dialogue counts do not measure capability and delivery-agent training outcomes are different protocols. No unreported reasoning setting is inferred. Metric: Recovery rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grid Studies Base Models - Mean&lt;/strong&gt; (End-to-end score E (%)): leader GPT-6.1 Sol (40.8), 11 models&lt;br&gt;&lt;span&gt;Grid Studies AppendixC Table 8:519 power-system tasks, native end-to-end score E, original units and equal source protocol. Only standalone no-tool completions are selected from the explicitly labelled base-model sections; base here means no tools, not pretrained checkpoint. Framework/tool-system rows are distinct and not attributed to their backend models. Scores verify required work against deterministic benchmark targets; timing excluded. Metric: End-to-end score E (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grid Studies Base Models - PAB-SS&lt;/strong&gt; (End-to-end score E (%)): leader GPT-6.1 Sol (41.8), 11 models&lt;br&gt;&lt;span&gt;Grid Studies AppendixC Table 8:519 power-system tasks, native end-to-end score E, original units and equal source protocol. Only standalone no-tool completions are selected from the explicitly labelled base-model sections; base here means no tools, not pretrained checkpoint. Framework/tool-system rows are distinct and not attributed to their backend models. Scores verify required work against deterministic benchmark targets; timing excluded. Metric: End-to-end score E (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grid Studies Base Models - PF Delta&lt;/strong&gt; (End-to-end score E (%)): leader GPT-6.1 Sol (32.8), 11 models&lt;br&gt;&lt;span&gt;Grid Studies AppendixC Table 8:519 power-system tasks, native end-to-end score E, original units and equal source protocol. Only standalone no-tool completions are selected from the explicitly labelled base-model sections; base here means no tools, not pretrained checkpoint. Framework/tool-system rows are distinct and not attributed to their backend models. Scores verify required work against deterministic benchmark targets; timing excluded. Metric: End-to-end score E (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grid Studies Base Models - PFBench&lt;/strong&gt; (End-to-end score E (%)): leader GPT-6.1 Sol (45.5), 11 models&lt;br&gt;&lt;span&gt;Grid Studies AppendixC Table 8:519 power-system tasks, native end-to-end score E, original units and equal source protocol. Only standalone no-tool completions are selected from the explicitly labelled base-model sections; base here means no tools, not pretrained checkpoint. Framework/tool-system rows are distinct and not attributed to their backend models. Scores verify required work against deterministic benchmark targets; timing excluded. Metric: End-to-end score E (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grid Studies Base Models - PGLib&lt;/strong&gt; (End-to-end score E (%)): leader GPT-6.1 Sol (42.5), 11 models&lt;br&gt;&lt;span&gt;Grid Studies AppendixC Table 8:519 power-system tasks, native end-to-end score E, original units and equal source protocol. Only standalone no-tool completions are selected from the explicitly labelled base-model sections; base here means no tools, not pretrained checkpoint. Framework/tool-system rows are distinct and not attributed to their backend models. Scores verify required work against deterministic benchmark targets; timing excluded. Metric: End-to-end score E (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grid Studies Base Models - PowerCodeBench&lt;/strong&gt; (End-to-end score E (%)): leader GPT-6.1 Sol (42.9), 11 models&lt;br&gt;&lt;span&gt;Grid Studies AppendixC Table 8:519 power-system tasks, native end-to-end score E, original units and equal source protocol. Only standalone no-tool completions are selected from the explicitly labelled base-model sections; base here means no tools, not pretrained checkpoint. Framework/tool-system rows are distinct and not attributed to their backend models. Scores verify required work against deterministic benchmark targets; timing excluded. Metric: End-to-end score E (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grid Studies Base Models - ProOPF&lt;/strong&gt; (End-to-end score E (%)): leader Claude Fable 5.1 (35.5), 11 models&lt;br&gt;&lt;span&gt;Grid Studies AppendixC Table 8:519 power-system tasks, native end-to-end score E, original units and equal source protocol. Only standalone no-tool completions are selected from the explicitly labelled base-model sections; base here means no tools, not pretrained checkpoint. Framework/tool-system rows are distinct and not attributed to their backend models. Scores verify required work against deterministic benchmark targets; timing excluded. Metric: End-to-end score E (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grid Studies Base Models - PSAB&lt;/strong&gt; (End-to-end score E (%)): leader GPT-6.1 Sol (58.2), 11 models&lt;br&gt;&lt;span&gt;Grid Studies AppendixC Table 8:519 power-system tasks, native end-to-end score E, original units and equal source protocol. Only standalone no-tool completions are selected from the explicitly labelled base-model sections; base here means no tools, not pretrained checkpoint. Framework/tool-system rows are distinct and not attributed to their backend models. Scores verify required work against deterministic benchmark targets; timing excluded. Metric: End-to-end score E (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grid Studies Base Models - RL2Grid&lt;/strong&gt; (End-to-end score E (%)): leader GPT-6.1 Sol (31.4), 11 models&lt;br&gt;&lt;span&gt;Grid Studies AppendixC Table 8:519 power-system tasks, native end-to-end score E, original units and equal source protocol. Only standalone no-tool completions are selected from the explicitly labelled base-model sections; base here means no tools, not pretrained checkpoint. Framework/tool-system rows are distinct and not attributed to their backend models. Scores verify required work against deterministic benchmark targets; timing excluded. Metric: End-to-end score E (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NitiBench+ - Cited sections F1&lt;/strong&gt; (F1 (0–1)): leader GPT-6.1 Sol (0.657), 26 models&lt;br&gt;&lt;span&gt;NitiBench+ Table 17:500 held-out Thai legal questions; three distinct evidence settings. Bar-standard answer scores 0–10; citation precision/recall/F 1 fractions 0–1. avg@3 for 13 self-hosted bf 16 models; other rows are leaderboard API observations, not replicated avg@3. AppendixC exact checkpoints and reasoning-on controls retained; other API effort unspecified. Same official judging rubric; citations checked against current in-force law. The public live site was 403, resolved via complete native paper table. Metric: F 1 (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NitiBench+ - Cited sections precision&lt;/strong&gt; (Precision (0–1)): leader Gemma 4 31B (IT) (Thinking) (0.642), 26 models&lt;br&gt;&lt;span&gt;NitiBench+ Table 17:500 held-out Thai legal questions; three distinct evidence settings. Bar-standard answer scores 0–10; citation precision/recall/F 1 fractions 0–1. avg@3 for 13 self-hosted bf 16 models; other rows are leaderboard API observations, not replicated avg@3. AppendixC exact checkpoints and reasoning-on controls retained; other API effort unspecified. Same official judging rubric; citations checked against current in-force law. The public live site was 403, resolved via complete native paper table. Metric: Precision (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NitiBench+ - Cited sections recall&lt;/strong&gt; (Recall (0–1)): leader Claude Fable 5.1 (0.88), 26 models&lt;br&gt;&lt;span&gt;NitiBench+ Table 17:500 held-out Thai legal questions; three distinct evidence settings. Bar-standard answer scores 0–10; citation precision/recall/F 1 fractions 0–1. avg@3 for 13 self-hosted bf 16 models; other rows are leaderboard API observations, not replicated avg@3. AppendixC exact checkpoints and reasoning-on controls retained; other API effort unspecified. Same official judging rubric; citations checked against current in-force law. The public live site was 403, resolved via complete native paper table. Metric: Recall (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NitiBench+ - Gold context In-force&lt;/strong&gt; (Bar-standard score (0–10)): leader Claude Fable 5.1 (8.04), 26 models&lt;br&gt;&lt;span&gt;NitiBench+ Table 17:500 held-out Thai legal questions; three distinct evidence settings. Bar-standard answer scores 0–10; citation precision/recall/F 1 fractions 0–1. avg@3 for 13 self-hosted bf 16 models; other rows are leaderboard API observations, not replicated avg@3. AppendixC exact checkpoints and reasoning-on controls retained; other API effort unspecified. Same official judging rubric; citations checked against current in-force law. The public live site was 403, resolved via complete native paper table. Metric: Bar-standard score (0–10).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NitiBench+ - Parametric&lt;/strong&gt; (Bar-standard score (0–10)): leader Claude Fable 5.1 (7.4), 26 models&lt;br&gt;&lt;span&gt;NitiBench+ Table 17:500 held-out Thai legal questions; three distinct evidence settings. Bar-standard answer scores 0–10; citation precision/recall/F 1 fractions 0–1. avg@3 for 13 self-hosted bf 16 models; other rows are leaderboard API observations, not replicated avg@3. AppendixC exact checkpoints and reasoning-on controls retained; other API effort unspecified. Same official judging rubric; citations checked against current in-force law. The public live site was 403, resolved via complete native paper table. Metric: Bar-standard score (0–10).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NitiBench+ - RAG In-force&lt;/strong&gt; (Bar-standard score (0–10)): leader Claude Fable 5.1 (7.78), 26 models&lt;br&gt;&lt;span&gt;NitiBench+ Table 17:500 held-out Thai legal questions; three distinct evidence settings. Bar-standard answer scores 0–10; citation precision/recall/F 1 fractions 0–1. avg@3 for 13 self-hosted bf 16 models; other rows are leaderboard API observations, not replicated avg@3. AppendixC exact checkpoints and reasoning-on controls retained; other API effort unspecified. Same official judging rubric; citations checked against current in-force law. The public live site was 403, resolved via complete native paper table. Metric: Bar-standard score (0–10).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;The Aggregate - Trip Outcome Prediction (5 Questions)&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (69.8), 24 models&lt;br&gt;&lt;span&gt;The Aggregate runs this evaluation itself. Two travellers wrote five questions about what actually happened on a two-day trip to the Baie de Somme, France, in October 2026, and answered them before any model saw the questions. Each model sees only the anonymized trip plan, including the packing list, and one question at a time, and gives a single best guess: what surprised them in town at 18:00, what disappointed them on the guided bay walk, which unpacked item they most wished for, what was inconvenient after the walk and how they solved it, and what the two taxi rides cost in total. A judge model (GPT-6.1 Sol) grades the four open answers against the travellers’ own answers with fixed rubrics (1, ½ or 0); the taxi total earns one minus its relative error, floored at 0. Two runs share the frozen questions and judge: a first panel at medium reasoning with a 4,096-token output cap, and a panel of current leaderboard models at their highest reasoning effort with a 32,768-token cap; each row is labelled with its effort. No tools are used. The score is the mean credit as a percentage; with five questions, close scores do not establish an ordering. Only models answering all five questions are ranked.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (3)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;LongCat-2.5&lt;/strong&gt; — Aggregate Rating 1871, #25&lt;ul&gt;&lt;li&gt;LLM Stats (Vision2Web): 78.6 (#1/6)&lt;/li&gt;&lt;li&gt;LLM Stats (BrowseComp): 91.9 (#2/74)&lt;/li&gt;&lt;li&gt;LLM Stats (Claw-Eval): 78.3 (#2/16)&lt;/li&gt;&lt;li&gt;LLM Stats (Humanity's Last Exam (with tools, text-only)): 63.1 (#2/12)&lt;/li&gt;&lt;li&gt;LLM Stats (MathVision): 96.5 (#2/40)&lt;/li&gt;&lt;li&gt;LLM Stats (SkillsBench): 63.3 (#2/11)&lt;/li&gt;&lt;li&gt;LLM Stats (JobBench): 65.4 (#3/5)&lt;/li&gt;&lt;li&gt;LLM Stats (SWE Atlas - Codebase QnA): 58.3 (#3/8)&lt;/li&gt;&lt;li&gt;LLM Stats (AutomationBench v1.0.6): 51.8 (#4/7)&lt;/li&gt;&lt;li&gt;LLM Stats (NL2Repo): 60.5 (#5/26)&lt;/li&gt;&lt;li&gt;...and 8 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nanbeige4.3-33B&lt;/strong&gt; — Aggregate Rating 1769, #98&lt;ul&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 18.19 (#7/111)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Finance - Raw Objective Completion: 85.59 (#46/243)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - R: -23.33 (#50/111)&lt;/li&gt;&lt;li&gt;AA Terminal-Bench 4.0: 29.8 (#56/240)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: -10.67 (#58/111)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Business: 0.0 (#62/111)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - HTML: 11.11 (#66/111)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics: 4.83 (#67/111)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - Kotlin: -20.0 (#68/111)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 20.41 (#69/94)&lt;/li&gt;&lt;li&gt;...and 54 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM3-1.7B&lt;/strong&gt; — Aggregate Rating 1618, #402&lt;ul&gt;&lt;li&gt;LLM Stats (Multi-IF): 89.1 (#1/26)&lt;/li&gt;&lt;li&gt;LLM Stats (TAU3-Bench): 68.01 (#4/9)&lt;/li&gt;&lt;li&gt;LLM Stats (Claw-Eval): 68.64 (#5/16)&lt;/li&gt;&lt;li&gt;LLM Stats (Humanity's Last Exam (with tools, text-only)): 13.9 (#12/12)&lt;/li&gt;&lt;li&gt;LLM Stats (MCP Atlas): 32.57 (#40/41)&lt;/li&gt;&lt;li&gt;LLM Stats (SWE-bench Multilingual): 23.0 (#51/51)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 2.1): 12.92 (#55/55)&lt;/li&gt;&lt;li&gt;ZeroEval GPQA Diamond: 63.37 (#182/264)&lt;/li&gt;&lt;li&gt;LLM Stats Score: 25.02 (#189/405)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (78)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ForecastBench: 66.0 (#98)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ForecastBench Tournament - Market: 70.9 (#120)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ASCIIBench: 1761.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on ComplexConstraints: 70.1 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on PFArena - Anchor-Informed Multi-Mutant Ranking: 0.3299 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on PFArena - Measurement-Free Multi-Mutant Ranking: 0.2514 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on PFArena - Single-Mutant-Informed Multi-Mutant Ranking: 0.6006 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Riemann-bench: 79.2 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ASCIIBench: 1984.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PFArena - Measurement-Free Multi-Mutant Ranking: 0.2766 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PFArena - Single-Mutant-Informed Multi-Mutant Ranking: 0.5826 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Riemann-bench: 74.4 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA CritPt v2: 79.14 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA CritPt v2 - Hard Subset: 63.5 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA FrontierCode 2.0: 39.45 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf: 29.6 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Construction - Criterion Pass Rate: 78.47 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Criterion Pass Rate: 81.68 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Engineering - Criterion Pass Rate: 82.56 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 84.75 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - HR - Criterion Pass Rate: 81.34 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Healthcare - Criterion Pass Rate: 88.2 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Insurance - Criterion Pass Rate: 90.93 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Legal - Criterion Pass Rate: 82.3 (#47)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 78.29 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Real Estate - Criterion Pass Rate: 75.3 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - STEM/Research - Criterion Pass Rate: 79.94 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA HLE-Diamond: 67.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA HLE-Diamond - Knowledge: 59.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA HLE-Diamond - Reasoning: 75.8 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA MMMU-Pro: 89.25 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Omniscience: 46.77 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench 4.0: 62.63 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1: 62.38 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Earth Sciences: 62.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Engineering Sciences: 55.56 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Life Sciences: 56.14 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Mathematical Sciences: 88.24 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench Science 0.1 - Physical Sciences: 56.86 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase: 62.32 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - Analytical Quality Elo: 1635.14 (#47)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - DOCX - Rubric Pass Rate: 62.13 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - PDF - Rubric Pass Rate: 65.52 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - PPTX - Rubric Pass Rate: 64.08 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - Presentation Elo: 1586.54 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - XLSX - Rubric Pass Rate: 59.32 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Accuracy: 49.23 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Hallucination Rate: 4.24 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Business: 31.56 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Health: 36.44 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Humanities &amp; Social Sciences: 50.22 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Law: 45.11 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Science, Engineering &amp; Mathematics: 43.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE): 67.83 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - C: 85.45 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Dart: 80.56 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Go: 52.78 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - HTML: 61.11 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Java: 52.73 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 94.03 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Julia: 63.16 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 25.71 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - PHP: 73.53 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Python: 70.21 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - R: 26.67 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Rust: 91.18 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Swift: 57.69 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 76.74 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Artificial Analysis Intelligence Index: 57.71 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA: 77.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Finance - Raw Objective Completion: 95.95 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - HR - Raw Objective Completion: 80.81 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Marketing - Raw Objective Completion: 94.25 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Operations - Raw Objective Completion: 97.39 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Sales - Raw Objective Completion: 93.65 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Strict: 54.19 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Support - Raw Objective Completion: 94.03 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Riemann-bench: 80.0 (#3)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (13)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;ASCIIBench&lt;/strong&gt;: Gemini 4 Argon (High) (1984.0) beat GPT-6.1 Sol (High) by 300.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Vision2Web)&lt;/strong&gt;: LongCat-2.5 (78.6) beat Qwen 3.8 Max by 9.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Legal Agent Benchmark)&lt;/strong&gt;: GLM-5.3-FlashX-Legal (19.2) beat Claude Fable 5 by 5.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (InfoVQA)&lt;/strong&gt;: LFM3-VL-2B (86.2) beat Qwen 2.5 VL 32B Instruct by 2.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (TextVQA)&lt;/strong&gt;: LFM3-VL-2B (87.9) beat Qwen 2 VL 72B Instruct by 2.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (ChartQA)&lt;/strong&gt;: LFM3-VL-2B (92.7) beat Claude 3.5 Sonnet by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Insurance - Criterion Pass Rate&lt;/strong&gt;: Gemini 4 Carbon (Medium) (90.93) beat Claude Fable 5.1 (Max, Default Fallback) by 1.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA MMMU-Pro&lt;/strong&gt;: Gemini 4 Carbon (Medium) (89.25) beat Claude Opus 5.5 (Max, Default Fallback) by 1.56&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Multi-IF)&lt;/strong&gt;: LFM3-1.7B (89.1) beat Ling-3.0-flash by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Speech to Speech - Big Bench Audio&lt;/strong&gt;: StepAudio 3 Realtime (AA system stepfun_stepaudio-3-realtime) (99.7) beat Qwen Audio 3.0 Realtime Plus by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (DocVQA)&lt;/strong&gt;: LFM3-VL-2B (96.7) beat Qwen 2.5 VL 72B Instruct by 0.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PFArena - Anchor-Informed Multi-Mutant Ranking&lt;/strong&gt;: Gemini 4 Argon (0.3822) beat Claude Opus 5 by 0.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Hallucination Rate&lt;/strong&gt;: Gemini 4 Carbon (Medium) (4.24) beat Gemini 4 Argon (High) by 0.02&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-03

=== DAILY ===
NEW BENCHMARKS (21)
  - CloudOpsBench v1.0 (Resolution rate (%)): leader Claude Fable 5.1 (Harness: Claude Code) (Max) (60.6), 10 models
      CloudOpsBench v 1.0:45 seeded AWS incident tasks,4 trials each,40 minutes. Native state-based resolution rat</summary></entry><entry><title>The Aggregate Digest — 2026-11-02</title><id>https://theaggregate.ai/digest/2026-11-02</id><updated>2026-11-02T08:40:27.424231+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;Top-10 New Scores (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on DuelLab GameBench 2 - Reasoning Variants: 86.480079882117 (#12)&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-02

=== DAILY ===
NEW SCORES FROM TOP-10 MODELS (1)
  - Claude Fable 5.1 on DuelLab GameBench 2 - Reasoning Variants: 86.480079882117 GameBench2 normalized score (0–100) (#12/231)
</summary></entry><entry><title>The Aggregate Digest — 2026-11-01</title><id>https://theaggregate.ai/digest/2026-11-01</id><updated>2026-11-01T08:38:20.149229+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;Top-10 New Scores (4)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ObviousBench: 99.3056 (#44)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ASCIIBench: 1781.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ObviousBench: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ASCIIBench: 2006.0 (#1)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (16)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;ASCIIBench&lt;/strong&gt;: Gemini 4 Argon (High) (2006.0) beat GPT-6.1 Sol (High) by 322.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VAmoS Pro Bench&lt;/strong&gt;: Phonic phonic-1 (57.5) beat Grok Voice Think Fast 2.0 by 12.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Reclamation farming&lt;/strong&gt;: GPT-6 Luna (71.1) beat GPT-5.4 by 5.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAquaBench - Ichthyopathology&lt;/strong&gt;: GPT-6 Luna (64.2) beat GPT-5.4 by 4.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BALSAM - Summarization&lt;/strong&gt;: Smart-1.1 (63.78) beat soual-3 by 4.76&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BenCzechMark - NER&lt;/strong&gt;: Muse Glimmer 30B (98.07692307692308) beat Llama 3.1 405B Instruct by 3.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Seed production and seed science&lt;/strong&gt;: GPT-6 Luna (76.3) beat GPT-5.4 by 2.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Crop cultivation technologies&lt;/strong&gt;: GPT-6 Luna (82.2) beat GPT-5.4 by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTMedQFundamental - General chemistry&lt;/strong&gt;: Qwen 3.8 27B (80.4) beat Qwen 3 30B A3B 2507 (Thinking) by 1.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Farming systems on agricultural landscapes&lt;/strong&gt;: GPT-6 Luna (74.7) beat GPT-5.4 by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAquaBench - Feeding of fish and aquatic organisms&lt;/strong&gt;: GPT-6 Luna (73.4) beat GPT-5.4 by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - General genetics&lt;/strong&gt;: GPT-6 Luna (84.6) beat GPT-5.4 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - General agriculture&lt;/strong&gt;: GPT-6 Luna (74.1) beat GPT-5.4 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Botany&lt;/strong&gt;: GPT-6 Luna (89.9) beat GPT-5.4 by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - Agriculture&lt;/strong&gt;: GPT-6 Luna (76.3) beat GPT-5.4 by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTMedQFundamental - Pathophysiology&lt;/strong&gt;: Qwen 3.8 27B (90.0) beat DeepSeek V4 Pro by 0.4&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Weekly&lt;/h2&gt;
&lt;h3&gt;New Models (190)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; — Aggregate Rating 2033, #1&lt;ul&gt;&lt;li&gt;Google Argon Report - GraphWalks BFS 256k to 1M: 96.2 (#1/5)&lt;/li&gt;&lt;li&gt;Epoch AI - Critpt: 36.0 (#1/259)&lt;/li&gt;&lt;li&gt;Epoch AI - Vending Bench 2: 16484.21 (#1/71)&lt;/li&gt;&lt;li&gt;Epoch AI - Blueprint Bench 2: 0.586 (#1/35)&lt;/li&gt;&lt;li&gt;Artificial Analysis Intelligence Index: 59.43 (#1/89)&lt;/li&gt;&lt;li&gt;AA Omniscience: 53.5 (#1/107)&lt;/li&gt;&lt;li&gt;AutomationBench-AA: 77.82 (#1/239)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Strict: 55.25 (#1/239)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Business: 42.67 (#1/107)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Health: 41.78 (#1/107)&lt;/li&gt;&lt;li&gt;...and 79 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Mythos 5.1&lt;/strong&gt; — Aggregate Rating 2012, #2&lt;ul&gt;&lt;li&gt;ExploitGym (version unspecified; 6h timeout): 264.0 (#2/5)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; — Aggregate Rating 1996, #3&lt;ul&gt;&lt;li&gt;Supabase Evals (Skills): 92.8 (#1/6)&lt;/li&gt;&lt;li&gt;Supabase Evals (No Skills): 89.9 (#1/6)&lt;/li&gt;&lt;li&gt;ProgramBench: 18.5 (#1/30)&lt;/li&gt;&lt;li&gt;ProgramBench Almost: 70.5 (#1/30)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 66.9 (#1/230)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 100.0 (#1/38)&lt;/li&gt;&lt;li&gt;ExploitGym (version unspecified; 6h timeout): 300.0 (#1/5)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 93.75 (#1/71)&lt;/li&gt;&lt;li&gt;ObviousBench: 100.0 (#1/643)&lt;/li&gt;&lt;li&gt;Design Arena (ASCII Art): 1355.0 (#2/98)&lt;/li&gt;&lt;li&gt;...and 18 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; — Aggregate Rating 1987, #4&lt;ul&gt;&lt;li&gt;OpenAI GPT-5.6 Launch - Internal Research Debugging Evaluation: 78.05 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-5.6 Launch - KernelGen 1P: 66.72 (#1/4)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 83.71 (#1/95)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Codex) - API Security: 81.5 (#1/6)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Cloud Security: 72.5 (#2/25)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Codex) - Cloud Security: 68.1 (#2/6)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Codex) - Code Vulnerabilities: 56.1 (#3/6)&lt;/li&gt;&lt;li&gt;ProgramBench Almost: 52.5 (#4/30)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 93.0 (#4/38)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Overall: 70.2 (#4/25)&lt;/li&gt;&lt;li&gt;...and 14 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; — Aggregate Rating 1978, #7&lt;ul&gt;&lt;li&gt;NatureBench: 36.0 (#1/24)&lt;/li&gt;&lt;li&gt;SEAL - MultiChallenge: 81.95 (#1/36)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Codex) - Cloud Security: 68.1 (#1/6)&lt;/li&gt;&lt;li&gt;OpenAI GPT-5.6 Launch - KernelGen 1P: 60.44 (#2/4)&lt;/li&gt;&lt;li&gt;DeepsecBench: 39.49 (#2/54)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Codex) - Overall: 70.6 (#2/6)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Codex) - Code Vulnerabilities: 56.4 (#2/6)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Codex) - API Security: 80.9 (#2/6)&lt;/li&gt;&lt;li&gt;OpenAI GPT-5.6 Launch - Internal Research Debugging Evaluation: 75.52 (#3/4)&lt;/li&gt;&lt;li&gt;ASCIIBench: 1672.0 (#3/13)&lt;/li&gt;&lt;li&gt;...and 20 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; — Aggregate Rating 1970, #9&lt;ul&gt;&lt;li&gt;ASCIIBench: 2006.0 (#1/13)&lt;/li&gt;&lt;li&gt;CAIS Vision Capabilities Index: 71.2 (#2/45)&lt;/li&gt;&lt;li&gt;RSI-Exam: 0.494 (#3/19)&lt;/li&gt;&lt;li&gt;Epoch AI - Rli: 17.08 (#4/18)&lt;/li&gt;&lt;li&gt;Design Arena (SVG): 1329.0 (#5/135)&lt;/li&gt;&lt;li&gt;Epoch AI - Frontiermath Erdos: 1.47 (#5/14)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 87.5 (#5/71)&lt;/li&gt;&lt;li&gt;Legal Benchmarks - Data Extraction - Criteria Pass Rate: 86.2 (#6/26)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 91.0 (#7/38)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 98.44 (#8/71)&lt;/li&gt;&lt;li&gt;...and 15 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; — Aggregate Rating 1961, #10&lt;ul&gt;&lt;li&gt;ProgramBench: 9.0 (#2/30)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 93.33 (#2/30)&lt;/li&gt;&lt;li&gt;ProgramBench Almost: 55.5 (#3/30)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 63.08 (#4/230)&lt;/li&gt;&lt;li&gt;APEX v1: 73.8 (#7/75)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 90.0 (#9/38)&lt;/li&gt;&lt;li&gt;CritPt: 29.7 (#10/279)&lt;/li&gt;&lt;li&gt;GDP.pdf: 27.6 (#10/39)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 76.86 (#14/95)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 45.1 (#14/44)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; — Aggregate Rating 1959, #11&lt;ul&gt;&lt;li&gt;ProgramBench Almost: 60.0 (#2/30)&lt;/li&gt;&lt;li&gt;ProgramBench: 8.5 (#3/30)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 94.0 (#3/38)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 37.22 (#32/176)&lt;/li&gt;&lt;li&gt;MLCR-AA: 22.22 (#40/176)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 58.89 (#54/176)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5.5&lt;/strong&gt; — Aggregate Rating 1947, #12&lt;ul&gt;&lt;li&gt;Epoch AI - Mirrorcode: 97.78 (#1/30)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 100.0 (#1/71)&lt;/li&gt;&lt;li&gt;Supabase Evals (Skills): 85.5 (#2/6)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 97.0 (#2/38)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 90.62 (#2/71)&lt;/li&gt;&lt;li&gt;FrontierSWE V2: 61.889299411764696 (#3/23)&lt;/li&gt;&lt;li&gt;DeepsecBench: 37.5 (#4/54)&lt;/li&gt;&lt;li&gt;ObviousBench: 100.0 (#4/643)&lt;/li&gt;&lt;li&gt;Supabase Evals (No Skills): 76.8 (#5/6)&lt;/li&gt;&lt;li&gt;ASCIIBench: 1511.0 (#5/13)&lt;/li&gt;&lt;li&gt;...and 5 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; — Aggregate Rating 1941, #13&lt;ul&gt;&lt;li&gt;OpenAI GPT-6 Sol &amp; Luna Launch - Broken Search Tool Non-Disclosure: 1.83 (#1/2)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Sol &amp; Luna Launch - Reviewer Bypass Attempts: 0.0 (#1/2)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Cyber Safety - Synthetic Agentic Environments: 1.0 (#1/2)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Sensitive Personal Data: 74.7 (#1/2)&lt;/li&gt;&lt;li&gt;Coarena: 1162.3 (#1/20)&lt;/li&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 100.0 (#1/306)&lt;/li&gt;&lt;li&gt;Epoch AI - Ebr Bench: 78.1 (#1/28)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 98.33 (#1/176)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 92.74 (#1/238)&lt;/li&gt;&lt;li&gt;LLM Stats (Agents' Last Exam): 60.0 (#1/26)&lt;/li&gt;&lt;li&gt;...and 218 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Pro Sol&lt;/strong&gt; — Aggregate Rating 1940, #14&lt;ul&gt;&lt;li&gt;ProphetArena: 0.9111 (#23/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro&lt;/strong&gt; — Aggregate Rating 1930, #15&lt;ul&gt;&lt;li&gt;LLM Stats (APEX-Agents): 67.2 (#1/13)&lt;/li&gt;&lt;li&gt;LLM Stats (AutomationBench-AA): 73.3 (#1/4)&lt;/li&gt;&lt;li&gt;LLM Stats (BabyVision): 95.4 (#1/18)&lt;/li&gt;&lt;li&gt;LLM Stats (BrowseComp): 92.1 (#1/73)&lt;/li&gt;&lt;li&gt;LLM Stats (Chartography): 90.2 (#1/7)&lt;/li&gt;&lt;li&gt;LLM Stats (DeepSWE): 75.5 (#1/14)&lt;/li&gt;&lt;li&gt;LLM Stats (GDPval-AA 2.1): 1846.0 (#1/6)&lt;/li&gt;&lt;li&gt;LLM Stats (Humanity's Last Exam (with tools)): 70.0 (#1/6)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Total: 63.8 (#1/22)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Laravel Code Quality: 18.55 (#1/22)&lt;/li&gt;&lt;li&gt;...and 92 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5&lt;/strong&gt; — Aggregate Rating 1930, #16&lt;ul&gt;&lt;li&gt;Epoch AI - Mirrorcode: 87.78 (#3/30)&lt;/li&gt;&lt;li&gt;APEX v1: 68.6 (#15/75)&lt;/li&gt;&lt;li&gt;GDP.pdf: 24.0 (#15/39)&lt;/li&gt;&lt;li&gt;Chartography: 27.3 (#19/42)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 37.3 (#25/44)&lt;/li&gt;&lt;li&gt;Kaggle WWTP Engineering: 42.11 (#47/54)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder&lt;/strong&gt; — Aggregate Rating 1918, #17&lt;ul&gt;&lt;li&gt;ProgramBench: 5.5 (#4/30)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV September: 66.04 (#4/11)&lt;/li&gt;&lt;li&gt;APEX v1: 75.1 (#4/75)&lt;/li&gt;&lt;li&gt;ProgramBench Almost: 50.5 (#5/30)&lt;/li&gt;&lt;li&gt;CAIS Risk Index: 37.2 (#5/58)&lt;/li&gt;&lt;li&gt;ProphetArena: 0.9489 (#6/23)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 92.0 (#6/38)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE September: 41.51 (#7/11)&lt;/li&gt;&lt;li&gt;Epoch AI - Apex Agents: 66.1 (#8/60)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV August: 68.42 (#9/19)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Sol&lt;/strong&gt; — Aggregate Rating 1915, #18&lt;ul&gt;&lt;li&gt;OpenAI GPT-5.6 Launch - Internal Research Debugging Evaluation: 64.2 (#4/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-5.6 Launch - KernelGen 1P: 38.12 (#4/4)&lt;/li&gt;&lt;li&gt;InfoOps Bench: 88.5 (#6/46)&lt;/li&gt;&lt;li&gt;Riemann-bench: 64.8 (#7/44)&lt;/li&gt;&lt;li&gt;GDP.pdf: 26.4 (#12/39)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 43.1 (#22/50)&lt;/li&gt;&lt;li&gt;APEX v1: 63.6 (#38/75)&lt;/li&gt;&lt;li&gt;Kaggle WWTP Engineering: 44.44 (#46/54)&lt;/li&gt;&lt;li&gt;ObviousBench: 99.3056 (#75/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5&lt;/strong&gt; — Aggregate Rating 1907, #19&lt;ul&gt;&lt;li&gt;GDP.pdf: 29.8 (#8/39)&lt;/li&gt;&lt;li&gt;Riemann-bench: 60.0 (#8/44)&lt;/li&gt;&lt;li&gt;Chartography: 34.8 (#9/42)&lt;/li&gt;&lt;li&gt;APEX v1: 66 (#22/75)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 38.1 (#23/44)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Sol&lt;/strong&gt; — Aggregate Rating 1890, #21&lt;ul&gt;&lt;li&gt;Riemann-bench: 74.4 (#1/44)&lt;/li&gt;&lt;li&gt;Kaggle FACTS Parametric: 80.7 (#3/45)&lt;/li&gt;&lt;li&gt;Kaggle WWTP Engineering: 76.92 (#12/54)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 46.2 (#20/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.7 Flash&lt;/strong&gt; — Aggregate Rating 1879, #23&lt;ul&gt;&lt;li&gt;MirrorCode: 41.1 (#3/12)&lt;/li&gt;&lt;li&gt;Chartography: 40.4 (#8/42)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 55.9 (#11/50)&lt;/li&gt;&lt;li&gt;Riemann-bench: 39.2 (#14/44)&lt;/li&gt;&lt;li&gt;GDP.pdf: 23.8 (#16/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 41.11 (#17/30)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 42.4 (#17/44)&lt;/li&gt;&lt;li&gt;APEX v1: 67 (#20/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.8 Flash&lt;/strong&gt; — Aggregate Rating 1871, #24&lt;ul&gt;&lt;li&gt;ASCIIBench: 1579.0 (#4/13)&lt;/li&gt;&lt;li&gt;Chartography: 40.9 (#7/42)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 58.5 (#9/50)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 48.4 (#10/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 51.2 (#11/44)&lt;/li&gt;&lt;li&gt;HANDBOOK.md Agents: 26.5 (#13/14)&lt;/li&gt;&lt;li&gt;APEX v1: 68.8 (#14/75)&lt;/li&gt;&lt;li&gt;GDP.pdf: 23.2 (#17/39)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 38.89 (#20/30)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 69.0 (#22/38)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.7&lt;/strong&gt; — Aggregate Rating 1870, #25&lt;ul&gt;&lt;li&gt;Design Arena (Agents - Mobile Apps): 1267.0 (#4/56)&lt;/li&gt;&lt;li&gt;CrueltyBench: 54.7 (#5/16)&lt;/li&gt;&lt;li&gt;Architectural RedlineBench: 54.2 (#6/31)&lt;/li&gt;&lt;li&gt;Kaggle MultiLoKo - Average v2 (Tasks v6, German v2): 36.76 (#7/16)&lt;/li&gt;&lt;li&gt;Kaggle MultiLoKo - Spanish Task v6: 51.2 (#8/16)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 48.9 (#9/44)&lt;/li&gt;&lt;li&gt;ASCIIBench: 1365.0 (#11/13)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 79.0 (#13/38)&lt;/li&gt;&lt;li&gt;GDP.pdf: 22.8 (#20/39)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 96.88 (#24/71)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash (1017)&lt;/strong&gt; — Aggregate Rating 1870, #27&lt;ul&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Chemistry: 2.686 (#2/9)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Earth science: 2.688 (#2/9)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Biology: 2.559 (#3/9)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Astronomy: 2.687 (#3/9)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - API Security: 82.7 (#3/21)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - API Security: 83.3 (#3/25)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE September: 55.97 (#4/11)&lt;/li&gt;&lt;li&gt;Epoch AI - Apex Agents: 69.8 (#4/60)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Overall: 2.538 (#4/9)&lt;/li&gt;&lt;li&gt;SEAL - MultiChallenge: 74.9 (#5/36)&lt;/li&gt;&lt;li&gt;...and 105 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.6&lt;/strong&gt; — Aggregate Rating 1862, #30&lt;ul&gt;&lt;li&gt;MirrorCode: 41.1 (#4/12)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 41.11 (#18/30)&lt;/li&gt;&lt;li&gt;APEX v1: 67.7 (#19/75)&lt;/li&gt;&lt;li&gt;Chartography: 20.5 (#23/42)&lt;/li&gt;&lt;li&gt;GDP.pdf: 17.2 (#25/39)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 36.5 (#26/44)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.3&lt;/strong&gt; — Aggregate Rating 1860, #31&lt;ul&gt;&lt;li&gt;HiL-Bench: 66.89 (#2/30)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 61.9 (#8/50)&lt;/li&gt;&lt;li&gt;GDP.pdf: 27.6 (#11/39)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 83.0 (#12/38)&lt;/li&gt;&lt;li&gt;Chartography: 27.6 (#18/42)&lt;/li&gt;&lt;li&gt;Riemann-bench: 28.0 (#23/44)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 66.86 (#33/95)&lt;/li&gt;&lt;li&gt;ObviousBench: 99.3056 (#67/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5&lt;/strong&gt; — Aggregate Rating 1850, #32&lt;ul&gt;&lt;li&gt;ComplexConstraints: 49.5 (#8/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 55.2 (#10/44)&lt;/li&gt;&lt;li&gt;GDP.pdf: 26.0 (#13/39)&lt;/li&gt;&lt;li&gt;Chartography: 31.0 (#14/42)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 51.3 (#16/50)&lt;/li&gt;&lt;li&gt;APEX v1: 68.4 (#17/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Max&lt;/strong&gt; — Aggregate Rating 1846, #34&lt;ul&gt;&lt;li&gt;HANDBOOK.md Agents: 30.8 (#8/14)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 98.44 (#12/71)&lt;/li&gt;&lt;li&gt;ProphetArena: 0.947 (#13/23)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 52.22 (#13/30)&lt;/li&gt;&lt;li&gt;ASCIIBench: 1323.0 (#13/13)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 45.5 (#13/44)&lt;/li&gt;&lt;li&gt;Chartography: 29.1 (#17/42)&lt;/li&gt;&lt;li&gt;GDP.pdf: 23.2 (#18/39)&lt;/li&gt;&lt;li&gt;Riemann-bench: 15.2 (#31/44)&lt;/li&gt;&lt;li&gt;APEX v1: 65.3 (#32/75)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3&lt;/strong&gt; — Aggregate Rating 1846, #35&lt;ul&gt;&lt;li&gt;Kaggle MultiLoKo - Average v2 (Tasks v6, German v2): 28.0 (#11/16)&lt;/li&gt;&lt;li&gt;Kaggle MultiLoKo - Japanese Task v6: 25.6 (#11/16)&lt;/li&gt;&lt;li&gt;HANDBOOK.md Agents: 28.6 (#11/14)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 54.44 (#12/30)&lt;/li&gt;&lt;li&gt;APEX v1: 68.4 (#16/75)&lt;/li&gt;&lt;li&gt;Riemann-bench: 37.6 (#18/44)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 96.88 (#18/71)&lt;/li&gt;&lt;li&gt;Chartography: 26.6 (#20/42)&lt;/li&gt;&lt;li&gt;CritPt: 23.4 (#23/279)&lt;/li&gt;&lt;li&gt;GDP.pdf: 19.0 (#23/39)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Max (0902)&lt;/strong&gt; — Aggregate Rating 1846, #36&lt;ul&gt;&lt;li&gt;Epoch AI - Mirrorcode: 47.78 (#14/30)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 77.0 (#16/38)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 50.86 (#54/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 5.5&lt;/strong&gt; — Aggregate Rating 1839, #38&lt;ul&gt;&lt;li&gt;o11y-bench - Pass@3: 100.0 (#2/71)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 78.89 (#4/30)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 87.5 (#4/71)&lt;/li&gt;&lt;li&gt;InfoOps Bench: 88.5 (#7/46)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Code Vulnerabilities: 34.1 (#8/21)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 46.6 (#8/25)&lt;/li&gt;&lt;li&gt;ASCIIBench: 1384.0 (#10/13)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Cloud Security: 50.7 (#12/21)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Overall: 48.6 (#15/21)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - API Security: 76.5 (#15/21)&lt;/li&gt;&lt;li&gt;...and 12 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash&lt;/strong&gt; — Aggregate Rating 1838, #39&lt;ul&gt;&lt;li&gt;ReasonScape R12: 965.6 (#1/108)&lt;/li&gt;&lt;li&gt;HumanCLAW-Bench - Find: 68.1 (#2/11)&lt;/li&gt;&lt;li&gt;MERA v2 - IFHardBench: 90.4 (#2/46)&lt;/li&gt;&lt;li&gt;ExploitGym (v1; 6h timeout): 206.0 (#2/3)&lt;/li&gt;&lt;li&gt;HumanCLAW-Bench - Navigate: 30.0 (#3/11)&lt;/li&gt;&lt;li&gt;MERA v2 - GorillaHard: 68.7 (#3/46)&lt;/li&gt;&lt;li&gt;MERA v2 - Agentic: 81.3 (#3/46)&lt;/li&gt;&lt;li&gt;MERA v2 - SAGE: 72.2 (#4/46)&lt;/li&gt;&lt;li&gt;MERA v2 - RUBIN: 80.3 (#4/46)&lt;/li&gt;&lt;li&gt;MERA v2 - Riddles: 64.7 (#4/46)&lt;/li&gt;&lt;li&gt;...and 22 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.8&lt;/strong&gt; — Aggregate Rating 1838, #40&lt;ul&gt;&lt;li&gt;DystopiaBench: 19.1 (#1/50)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 65.56 (#7/30)&lt;/li&gt;&lt;li&gt;Riemann-bench: 47.2 (#12/44)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 52.3 (#13/50)&lt;/li&gt;&lt;li&gt;GDP.pdf: 24.0 (#14/39)&lt;/li&gt;&lt;li&gt;Kaggle WWTP Engineering: 64.29 (#28/54)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 35.6 (#28/44)&lt;/li&gt;&lt;li&gt;Chartography: 15.9 (#30/42)&lt;/li&gt;&lt;li&gt;APEX v1: 62.7 (#41/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;mistral-large-4-oss&lt;/strong&gt; — Aggregate Rating 1838, #41&lt;ul&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 88.25 (#2/238)&lt;/li&gt;&lt;li&gt;LiveBench Coding: 95.4 (#5/32)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench v1.1: 15.833 (#8/33)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 24.82 (#11/107)&lt;/li&gt;&lt;li&gt;LiveBench Research: 79.7 (#12/32)&lt;/li&gt;&lt;li&gt;LLM Stats (BrowseComp): 84.5 (#14/73)&lt;/li&gt;&lt;li&gt;LiveBench Data Analysis: 76.6 (#15/32)&lt;/li&gt;&lt;li&gt;AA-Briefcase - DOCX - Rubric Pass Rate: 57.4 (#15/245)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 4.0): 41.0 (#15/31)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2.1: 61.723 (#15/35)&lt;/li&gt;&lt;li&gt;...and 65 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Terra&lt;/strong&gt; — Aggregate Rating 1837, #42&lt;ul&gt;&lt;li&gt;OpenAI GPT-6 Sol &amp; Luna Launch - Broken Search Tool Non-Disclosure: 66.42 (#2/2)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Sol &amp; Luna Launch - Reviewer Bypass Attempts: 0.6 (#2/2)&lt;/li&gt;&lt;li&gt;APEX v1: 69.5 (#11/75)&lt;/li&gt;&lt;li&gt;Chartography: 34.0 (#11/42)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 41.11 (#19/30)&lt;/li&gt;&lt;li&gt;Kaggle WWTP Engineering: 64.29 (#29/54)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark Open&lt;/strong&gt; — Aggregate Rating 1836, #43&lt;ul&gt;&lt;li&gt;Next.js Agent Evals (OpenCode) - Success Rate: 90.0 (#4/15)&lt;/li&gt;&lt;li&gt;Next.js Agent Evals (OpenCode) - Success Rate with AGENTS.md: 97.0 (#5/15)&lt;/li&gt;&lt;li&gt;CAIS Risk Index: 43.0 (#12/58)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 85.94 (#12/71)&lt;/li&gt;&lt;li&gt;Vals AI SAGE: 85.772 (#13/28)&lt;/li&gt;&lt;li&gt;Design Arena (UI Components): 1333.0 (#16/182)&lt;/li&gt;&lt;li&gt;EKI Estonian LLM - Trivia: 67.9 (#16/100)&lt;/li&gt;&lt;li&gt;EKI Estonian LLM - Trivia Hard: 24.12 (#16/92)&lt;/li&gt;&lt;li&gt;EKI Estonian LLM - Book Recommendations: 69.38 (#17/100)&lt;/li&gt;&lt;li&gt;Humanity's Sixth Sense: 29.6 (#19/29)&lt;/li&gt;&lt;li&gt;...and 20 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3&lt;/strong&gt; — Aggregate Rating 1834, #44&lt;ul&gt;&lt;li&gt;ExploitGym (v1; 6h timeout): 159.0 (#3/3)&lt;/li&gt;&lt;li&gt;ASCIIBench: 1434.0 (#8/13)&lt;/li&gt;&lt;li&gt;HANDBOOK.md Agents: 29.9 (#9/14)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 98.44 (#11/71)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 51.8 (#14/50)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 77.0 (#15/38)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 81.25 (#26/71)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 30.5 (#31/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 12.0 (#34/44)&lt;/li&gt;&lt;li&gt;APEX v1: 63 (#40/75)&lt;/li&gt;&lt;li&gt;...and 3 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy4 preview&lt;/strong&gt; — Aggregate Rating 1834, #45&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 64.6 (#7/50)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 46.0 (#11/44)&lt;/li&gt;&lt;li&gt;HANDBOOK.md Agents: 24.3 (#14/14)&lt;/li&gt;&lt;li&gt;Riemann-bench: 24.0 (#26/44)&lt;/li&gt;&lt;li&gt;RuneBench: 5.373458475339 (#40/100)&lt;/li&gt;&lt;li&gt;ObviousBench: 99.3056 (#86/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ember-1&lt;/strong&gt; — Aggregate Rating 1832, #47&lt;ul&gt;&lt;li&gt;Vals AI SAGE: 86.376 (#12/28)&lt;/li&gt;&lt;li&gt;ObviousBench: 97.2222 (#175/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.6-Pro&lt;/strong&gt; — Aggregate Rating 1828, #49&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 73.0 (#21/38)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 56.86 (#47/95)&lt;/li&gt;&lt;li&gt;ObviousBench: 84.7222 (#420/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M3.1-Flash&lt;/strong&gt; — Aggregate Rating 1827, #50&lt;ul&gt;&lt;li&gt;ASCIIBench: 1435.0 (#7/13)&lt;/li&gt;&lt;li&gt;Vals AI SAGE: 85.258 (#17/28)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 75.0 (#19/38)&lt;/li&gt;&lt;li&gt;CritPt: 22.3 (#24/279)&lt;/li&gt;&lt;li&gt;AIMultiple - Agentic RAG Routing: 81.52 (#24/81)&lt;/li&gt;&lt;li&gt;APEX v1: 65 (#33/75)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Song Lyrics: 13.31 (#34/88)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 95.31 (#34/71)&lt;/li&gt;&lt;li&gt;AIIQ Composite IQ: 123.0 (#40/184)&lt;/li&gt;&lt;li&gt;AIMultiple - Text-to-SQL (Adjudicated): 73.5 (#41/81)&lt;/li&gt;&lt;li&gt;...and 17 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 5 Preview&lt;/strong&gt; — Aggregate Rating 1823, #52&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 78.0 (#14/38)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench v1.1: 12.917 (#14/33)&lt;/li&gt;&lt;li&gt;Vals AI Legal Index: 70.162 (#15/33)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 54.762 (#20/32)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 54.57 (#49/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.7&lt;/strong&gt; — Aggregate Rating 1822, #53&lt;ul&gt;&lt;li&gt;GDP.pdf: 21.0 (#21/39)&lt;/li&gt;&lt;li&gt;Riemann-bench: 32.8 (#21/44)&lt;/li&gt;&lt;li&gt;Chartography: 16.5 (#28/42)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 35.9 (#32/50)&lt;/li&gt;&lt;li&gt;APEX v1: 62.2 (#43/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.5&lt;/strong&gt; — Aggregate Rating 1820, #54&lt;ul&gt;&lt;li&gt;MirrorCode: 32.2 (#5/12)&lt;/li&gt;&lt;li&gt;Riemann-bench: 38.4 (#16/44)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 32.22 (#23/30)&lt;/li&gt;&lt;li&gt;Chartography: 17.0 (#26/42)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 35.9 (#27/44)&lt;/li&gt;&lt;li&gt;GDP.pdf: 14.0 (#32/39)&lt;/li&gt;&lt;li&gt;APEX v1: 61.8 (#47/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.1&lt;/strong&gt; — Aggregate Rating 1820, #55&lt;ul&gt;&lt;li&gt;APEX v1: 73.8 (#8/75)&lt;/li&gt;&lt;li&gt;Chartography: 24.4 (#22/42)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 38.1 (#22/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 20.8 (#29/44)&lt;/li&gt;&lt;li&gt;ObviousBench: 98.6111 (#103/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.6 Flash&lt;/strong&gt; — Aggregate Rating 1819, #57&lt;ul&gt;&lt;li&gt;Chartography: 34.0 (#10/42)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 40.0 (#19/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 30.4 (#22/44)&lt;/li&gt;&lt;li&gt;APEX v1: 66 (#23/75)&lt;/li&gt;&lt;li&gt;Kaggle WWTP Engineering: 64.29 (#30/54)&lt;/li&gt;&lt;li&gt;GDP.pdf: 14.0 (#30/39)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 39.5 (#30/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro (0813)&lt;/strong&gt; — Aggregate Rating 1818, #58&lt;ul&gt;&lt;li&gt;Epoch AI - Mirrorcode: 33.33 (#22/30)&lt;/li&gt;&lt;li&gt;APEX v1: 66 (#24/75)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 64.0 (#26/38)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 45.71 (#64/95)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 51.04 (#108/230)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Flash&lt;/strong&gt; — Aggregate Rating 1814, #60&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 57.4 (#10/50)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 43.3 (#16/44)&lt;/li&gt;&lt;li&gt;Chartography: 17.4 (#24/42)&lt;/li&gt;&lt;li&gt;GDP.pdf: 16.6 (#27/39)&lt;/li&gt;&lt;li&gt;ObviousBench: 95.1389 (#271/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Luna&lt;/strong&gt; — Aggregate Rating 1813, #61&lt;ul&gt;&lt;li&gt;MERA Industrial - Agriculture: 76.3 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTAgroBench - Botany: 89.9 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTAgroBench - General genetics: 84.6 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTAgroBench - General agriculture: 74.1 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTAgroBench - Reclamation farming: 71.1 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTAgroBench - Seed production and seed science: 76.3 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTAgroBench - Farming systems on agricultural landscapes: 74.7 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTAgroBench - Crop cultivation technologies: 82.2 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTAquaBench - Feeding of fish and aquatic organisms: 73.4 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTAquaBench - Ichthyopathology: 64.2 (#1/106)&lt;/li&gt;&lt;li&gt;...and 55 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling 3.1 Flash&lt;/strong&gt; — Aggregate Rating 1813, #62&lt;ul&gt;&lt;li&gt;AIIQ Composite IQ: 120.0 (#52/184)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Pro (Preview)&lt;/strong&gt; — Aggregate Rating 1809, #64&lt;ul&gt;&lt;li&gt;ComplexConstraints: 43.7 (#15/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 33.6 (#20/44)&lt;/li&gt;&lt;li&gt;Chartography: 26.1 (#21/42)&lt;/li&gt;&lt;li&gt;GDP.pdf: 17.0 (#26/39)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 27.2 (#35/50)&lt;/li&gt;&lt;li&gt;APEX v1: 61.8 (#46/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5&lt;/strong&gt; — Aggregate Rating 1809, #65&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 51.3 (#17/50)&lt;/li&gt;&lt;li&gt;Chartography: 16.6 (#27/42)&lt;/li&gt;&lt;li&gt;Kaggle WWTP Engineering: 31.82 (#49/54)&lt;/li&gt;&lt;li&gt;APEX v1: 49.6 (#72/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.5 Flash&lt;/strong&gt; — Aggregate Rating 1806, #66&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 50.8 (#18/50)&lt;/li&gt;&lt;li&gt;Riemann-bench: 36.8 (#19/44)&lt;/li&gt;&lt;li&gt;DystopiaBench: 72.6 (#40/50)&lt;/li&gt;&lt;li&gt;APEX v1: 62.2 (#44/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.6&lt;/strong&gt; — Aggregate Rating 1805, #67&lt;ul&gt;&lt;li&gt;Epoch AI - Mirrorcode: 33.33 (#21/30)&lt;/li&gt;&lt;li&gt;Riemann-bench: 27.2 (#24/44)&lt;/li&gt;&lt;li&gt;APEX v1: 65.8 (#26/75)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 30.8 (#34/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.2&lt;/strong&gt; — Aggregate Rating 1803, #69&lt;ul&gt;&lt;li&gt;Chartography: 32.1 (#12/42)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 39.9 (#21/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 23.2 (#28/44)&lt;/li&gt;&lt;li&gt;GDP.pdf: 16.0 (#29/39)&lt;/li&gt;&lt;li&gt;ObviousBench: 98.6111 (#105/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Pro (Preview)&lt;/strong&gt; — Aggregate Rating 1797, #71&lt;ul&gt;&lt;li&gt;APEX v1: 60.9 (#50/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.3 Codex&lt;/strong&gt; — Aggregate Rating 1796, #72&lt;ul&gt;&lt;li&gt;APEX v1: 65.9 (#25/75)&lt;/li&gt;&lt;li&gt;Epoch AI - Critpt: 16.86 (#87/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 4 Preview&lt;/strong&gt; — Aggregate Rating 1794, #74&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 69.0 (#25/38)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 43.43 (#67/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Luna&lt;/strong&gt; — Aggregate Rating 1793, #75&lt;ul&gt;&lt;li&gt;Chartography: 31.4 (#13/42)&lt;/li&gt;&lt;li&gt;APEX v1: 66.6 (#21/75)&lt;/li&gt;&lt;li&gt;Kaggle WWTP Engineering: 57.14 (#34/54)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 65.62 (#52/71)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 89.06 (#58/71)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3 Flash&lt;/strong&gt; — Aggregate Rating 1792, #76&lt;ul&gt;&lt;li&gt;Vals AI SAGE: 89.255 (#5/28)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 46.7 (#19/50)&lt;/li&gt;&lt;li&gt;ReasonScape R12: 896.37 (#26/108)&lt;/li&gt;&lt;li&gt;Chartography: 16.3 (#29/42)&lt;/li&gt;&lt;li&gt;GDP.pdf: 14.0 (#31/39)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 27.6 (#34/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 8.8 (#40/44)&lt;/li&gt;&lt;li&gt;APEX v1: 58.7 (#58/75)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 46.0 (#63/95)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 51.62 (#101/230)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.6-Flash&lt;/strong&gt; — Aggregate Rating 1790, #77&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 69.0 (#24/38)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 50.29 (#57/95)&lt;/li&gt;&lt;li&gt;ObviousBench: 82.6389 (#434/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercury 3&lt;/strong&gt; — Aggregate Rating 1789, #78&lt;ul&gt;&lt;li&gt;ZeroEval GPQA Diamond: 89.23 (#37/263)&lt;/li&gt;&lt;li&gt;AIIQ Composite IQ: 98.0 (#140/184)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.2&lt;/strong&gt; — Aggregate Rating 1789, #79&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 42.1 (#27/50)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 29.3 (#33/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 10.4 (#39/44)&lt;/li&gt;&lt;li&gt;APEX v1: 61.5 (#49/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Pro&lt;/strong&gt; — Aggregate Rating 1782, #83&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 14.4 (#44/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash (0731)&lt;/strong&gt; — Aggregate Rating 1782, #84&lt;ul&gt;&lt;li&gt;ReasonScape R12: 957.14 (#4/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.7 Max&lt;/strong&gt; — Aggregate Rating 1781, #85&lt;ul&gt;&lt;li&gt;DystopiaBench: 75.8 (#47/50)&lt;/li&gt;&lt;li&gt;ObviousBench: 97.2222 (#195/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4&lt;/strong&gt; — Aggregate Rating 1781, #86&lt;ul&gt;&lt;li&gt;ComplexConstraints: 46.0 (#12/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 41.6 (#13/44)&lt;/li&gt;&lt;li&gt;Chartography: 29.6 (#15/42)&lt;/li&gt;&lt;li&gt;APEX v1: 65.6 (#28/75)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 36.4 (#31/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.5&lt;/strong&gt; — Aggregate Rating 1778, #87&lt;ul&gt;&lt;li&gt;Riemann-bench: 11.2 (#36/44)&lt;/li&gt;&lt;li&gt;APEX v1: 57.3 (#61/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Flash (Preview)&lt;/strong&gt; — Aggregate Rating 1777, #89&lt;ul&gt;&lt;li&gt;APEX v1: 60.9 (#51/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.7 Code&lt;/strong&gt; — Aggregate Rating 1769, #95&lt;ul&gt;&lt;li&gt;DystopiaBench: 54.1 (#16/50)&lt;/li&gt;&lt;li&gt;APEX v1: 60.1 (#54/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4.6&lt;/strong&gt; — Aggregate Rating 1768, #96&lt;ul&gt;&lt;li&gt;GDP.pdf: 18.0 (#24/39)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 16.4 (#43/50)&lt;/li&gt;&lt;li&gt;APEX v1: 55.1 (#64/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 Max Preview&lt;/strong&gt; — Aggregate Rating 1768, #97&lt;ul&gt;&lt;li&gt;ObviousBench: 97.9167 (#157/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy3&lt;/strong&gt; — Aggregate Rating 1768, #98&lt;ul&gt;&lt;li&gt;ComplexConstraints: 34.3 (#29/44)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 41.0 (#29/50)&lt;/li&gt;&lt;li&gt;Gert Labs Rankings: 25.96 (#96/100)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.2 Codex&lt;/strong&gt; — Aggregate Rating 1766, #100&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 20.1 (#40/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.7 Plus&lt;/strong&gt; — Aggregate Rating 1765, #102&lt;ul&gt;&lt;li&gt;DystopiaBench: 73.0 (#42/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Beam 1.0&lt;/strong&gt; — Aggregate Rating 1764, #103&lt;ul&gt;&lt;li&gt;ZeroEval GPQA Diamond: 90.6 (#25/263)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 62.0 (#28/38)&lt;/li&gt;&lt;li&gt;BrowseComp (Mercor 130): 56.7 (#37/45)&lt;/li&gt;&lt;li&gt;APEX v1: 60.6 (#52/75)&lt;/li&gt;&lt;li&gt;Epoch AI - ECI: 150.24 (#60/271)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Flash&lt;/strong&gt; — Aggregate Rating 1763, #104&lt;ul&gt;&lt;li&gt;GDP.pdf: 10.0 (#36/39)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 20.0 (#41/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5&lt;/strong&gt; — Aggregate Rating 1761, #106&lt;ul&gt;&lt;li&gt;MirrorCode: 4.4 (#12/12)&lt;/li&gt;&lt;li&gt;Epoch AI - Mirrorcode: 4.44 (#30/30)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 4&lt;/strong&gt; — Aggregate Rating 1761, #108&lt;ul&gt;&lt;li&gt;ProphetArena: 0.9463 (#14/23)&lt;/li&gt;&lt;li&gt;PRISM (1C:Enterprise) - Platform Tasks (B): 80.0 (#23/65)&lt;/li&gt;&lt;li&gt;Epoch AI - Apex Agents: 51.6 (#30/60)&lt;/li&gt;&lt;li&gt;PRISM (1C:Enterprise) - Algorithmic Tasks (A): 73.0 (#34/65)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Generative Quality: 61.01 (#48/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Flora &amp; Fauna: 18.58 (#50/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Completion: 21.12 (#53/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Case Government: 45.0 (#54/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Famous Phrases: 6.03 (#55/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM Leaderboard - Average: 32.99 (#56/88)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.1&lt;/strong&gt; — Aggregate Rating 1760, #109&lt;ul&gt;&lt;li&gt;APEX v1: 52.4 (#69/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro&lt;/strong&gt; — Aggregate Rating 1759, #111&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 53.3 (#12/50)&lt;/li&gt;&lt;li&gt;Riemann-bench: 38.4 (#15/44)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 42.0 (#18/44)&lt;/li&gt;&lt;li&gt;APEX v1: 59.7 (#56/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.6&lt;/strong&gt; — Aggregate Rating 1756, #113&lt;ul&gt;&lt;li&gt;ComplexConstraints: 30.1 (#32/44)&lt;/li&gt;&lt;li&gt;GDP.pdf: 12.0 (#33/39)&lt;/li&gt;&lt;li&gt;Chartography: 12.2 (#36/42)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 24.6 (#37/50)&lt;/li&gt;&lt;li&gt;Riemann-bench: 7.2 (#42/44)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.2&lt;/strong&gt; — Aggregate Rating 1754, #114&lt;ul&gt;&lt;li&gt;Riemann-bench: 37.6 (#17/44)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 42.6 (#26/50)&lt;/li&gt;&lt;li&gt;APEX v1: 64.7 (#35/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 27B&lt;/strong&gt; — Aggregate Rating 1752, #119&lt;ul&gt;&lt;li&gt;MERA Industrial - ruTXTMedQFundamental - General chemistry: 80.4 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTMedQFundamental - Pathophysiology: 90.0 (#1/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - Medicine and Healthcare: 86.8 (#2/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTMedQFundamental - Anatomy: 89.3 (#2/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTMedQFundamental - Hygiene: 83.7 (#2/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTMedQFundamental - Histology: 87.8 (#2/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTMedQFundamental - Biophysics: 84.1 (#2/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTMedQFundamental - Biochemistry: 91.9 (#2/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTMedQFundamental - Pharmacology: 90.4 (#2/106)&lt;/li&gt;&lt;li&gt;MERA Industrial - ruTXTMedQFundamental - General surgery: 81.9 (#2/106)&lt;/li&gt;&lt;li&gt;...and 56 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok Build 0.1&lt;/strong&gt; — Aggregate Rating 1747, #126&lt;ul&gt;&lt;li&gt;DystopiaBench: 70.0 (#31/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon 375B A23B&lt;/strong&gt; — Aggregate Rating 1745, #128&lt;ul&gt;&lt;li&gt;LLM Stats (MCP-Mark): 67.7 (#2/10)&lt;/li&gt;&lt;li&gt;LLM Stats (Tau3 Banking): 34.0 (#3/20)&lt;/li&gt;&lt;li&gt;LLM Stats (SWE Atlas - Codebase QnA): 48.4 (#4/7)&lt;/li&gt;&lt;li&gt;LLM Stats (CritPT): 8.6 (#5/10)&lt;/li&gt;&lt;li&gt;LLM Stats (OmniScience): 23.0 (#5/10)&lt;/li&gt;&lt;li&gt;LLM Stats (WildClawBench): 50.9 (#5/7)&lt;/li&gt;&lt;li&gt;LLM Stats (Toolathlon-Verified): 65.3 (#6/6)&lt;/li&gt;&lt;li&gt;LLM Stats (GDPval-AA): 1441.0 (#9/17)&lt;/li&gt;&lt;li&gt;LLM Stats (APEX-Agents): 24.8 (#13/13)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 69.0 (#23/38)&lt;/li&gt;&lt;li&gt;...and 5 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3&lt;/strong&gt; — Aggregate Rating 1742, #132&lt;ul&gt;&lt;li&gt;APEX v1: 60.5 (#53/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.1&lt;/strong&gt; — Aggregate Rating 1733, #141&lt;ul&gt;&lt;li&gt;ZeroEval GPQA Diamond: 88.1 (#48/263)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Inkling&lt;/strong&gt; — Aggregate Rating 1732, #145&lt;ul&gt;&lt;li&gt;Riemann-bench: 15.2 (#32/44)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 47.0 (#33/38)&lt;/li&gt;&lt;li&gt;Chartography: 10.0 (#40/42)&lt;/li&gt;&lt;li&gt;APEX v1: 61.6 (#48/75)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 29.43 (#75/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5&lt;/strong&gt; — Aggregate Rating 1732, #147&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 17.4 (#42/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.5&lt;/strong&gt; — Aggregate Rating 1730, #149&lt;ul&gt;&lt;li&gt;Riemann-bench: 12.0 (#35/44)&lt;/li&gt;&lt;li&gt;Chartography: 12.6 (#35/42)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 18.2 (#37/44)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 8.7 (#48/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash&lt;/strong&gt; — Aggregate Rating 1730, #150&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 30.8 (#33/50)&lt;/li&gt;&lt;li&gt;APEX v1: 63.8 (#37/75)&lt;/li&gt;&lt;li&gt;Riemann-bench: 10.4 (#38/44)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M3&lt;/strong&gt; — Aggregate Rating 1730, #151&lt;ul&gt;&lt;li&gt;DystopiaBench: 40.7 (#10/50)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 55.0 (#31/38)&lt;/li&gt;&lt;li&gt;APEX v1: 59.8 (#55/75)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 44.57 (#66/95)&lt;/li&gt;&lt;li&gt;CritPt: 3.7 (#88/279)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 397B A17B&lt;/strong&gt; — Aggregate Rating 1729, #152&lt;ul&gt;&lt;li&gt;Vals AI SAGE: 85.53 (#15/28)&lt;/li&gt;&lt;li&gt;APEX v1: 50.3 (#71/75)&lt;/li&gt;&lt;li&gt;Gert Labs Rankings: 24.91 (#99/100)&lt;/li&gt;&lt;li&gt;Epoch AI - Critpt: 0.86 (#181/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro Preview&lt;/strong&gt; — Aggregate Rating 1726, #157&lt;ul&gt;&lt;li&gt;Riemann-bench: 5.6 (#43/44)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Pro&lt;/strong&gt; — Aggregate Rating 1726, #158&lt;ul&gt;&lt;li&gt;ObviousBench: 95.8333 (#239/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Super VL (BF16)&lt;/strong&gt; — Aggregate Rating 1723, #162&lt;ul&gt;&lt;li&gt;ZeroEval GPQA Diamond: 89.52 (#34/263)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.5 Flash Lite&lt;/strong&gt; — Aggregate Rating 1716, #169&lt;ul&gt;&lt;li&gt;Kaggle WWTP Engineering: 69.23 (#24/54)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 45.0 (#35/38)&lt;/li&gt;&lt;li&gt;GDP.pdf: 10.0 (#35/39)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 23.2 (#36/44)&lt;/li&gt;&lt;li&gt;Riemann-bench: 11.2 (#37/44)&lt;/li&gt;&lt;li&gt;Chartography: 10.2 (#39/42)&lt;/li&gt;&lt;li&gt;APEX v1: 54.7 (#65/75)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 16.0 (#81/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.20&lt;/strong&gt; — Aggregate Rating 1711, #176&lt;ul&gt;&lt;li&gt;ComplexConstraints: 16.4 (#40/44)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 27B&lt;/strong&gt; — Aggregate Rating 1710, #177&lt;ul&gt;&lt;li&gt;ReasonScape R12: 872.57 (#35/108)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 89.06 (#60/71)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 51.56 (#64/71)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 31B (IT)&lt;/strong&gt; — Aggregate Rating 1709, #179&lt;ul&gt;&lt;li&gt;ReasonScape R12: 927.75 (#15/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Inkling Small&lt;/strong&gt; — Aggregate Rating 1709, #180&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 49.0 (#32/38)&lt;/li&gt;&lt;li&gt;Chartography: 9.1 (#41/42)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 34.0 (#73/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Super VL&lt;/strong&gt; — Aggregate Rating 1708, #181&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 57.0 (#30/38)&lt;/li&gt;&lt;li&gt;APEX v1: 57.6 (#59/75)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 36.86 (#72/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O4 Mini&lt;/strong&gt; — Aggregate Rating 1708, #182&lt;ul&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 1007.0 (#61/286)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Reasoning): 1034.0 (#89/349)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4&lt;/strong&gt; — Aggregate Rating 1706, #187&lt;ul&gt;&lt;li&gt;APEX v1: 57.1 (#62/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O1&lt;/strong&gt; — Aggregate Rating 1704, #188&lt;ul&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 989.0 (#64/286)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Reasoning): 807.0 (#134/349)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 27B&lt;/strong&gt; — Aggregate Rating 1704, #189&lt;ul&gt;&lt;li&gt;ReasonScape R12: 948.0 (#11/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Motif 3&lt;/strong&gt; — Aggregate Rating 1702, #192&lt;ul&gt;&lt;li&gt;AIIQ Composite IQ: 107.0 (#95/184)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.3&lt;/strong&gt; — Aggregate Rating 1701, #195&lt;ul&gt;&lt;li&gt;Chartography: 12.9 (#34/42)&lt;/li&gt;&lt;li&gt;GDP.pdf: 8.0 (#37/39)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.7&lt;/strong&gt; — Aggregate Rating 1700, #200&lt;ul&gt;&lt;li&gt;ReasonScape R12: 894.43 (#27/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Glimmer 30B&lt;/strong&gt; — Aggregate Rating 1697, #202&lt;ul&gt;&lt;li&gt;BenCzechMark - NER: 98.07692307692308 (#1/79)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 51.8 (#15/50)&lt;/li&gt;&lt;li&gt;BenCzechMark - Factual Knowledge: 65.93406593406593 (#15/79)&lt;/li&gt;&lt;li&gt;BenCzechMark - Reading Comprehension: 69.23076923076924 (#17/79)&lt;/li&gt;&lt;li&gt;BenCzechMark: 65.39442700156987 (#21/79)&lt;/li&gt;&lt;li&gt;BenCzechMark - Czech Math Reasoning: 62.179487179487175 (#21/79)&lt;/li&gt;&lt;li&gt;BenCzechMark - Czech Language Understanding: 62.17948717948718 (#22/79)&lt;/li&gt;&lt;li&gt;BenCzechMark - NLI: 60.41666666666667 (#22/79)&lt;/li&gt;&lt;li&gt;Chartography: 17.2 (#25/42)&lt;/li&gt;&lt;li&gt;Riemann-bench: 13.6 (#33/44)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 31B&lt;/strong&gt; — Aggregate Rating 1695, #205&lt;ul&gt;&lt;li&gt;ObviousBench: 100.0 (#16/643)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 56.25 (#59/71)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 85.94 (#64/71)&lt;/li&gt;&lt;li&gt;APEX v1: 48.1 (#73/75)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 11.71 (#84/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3.2&lt;/strong&gt; — Aggregate Rating 1694, #209&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 24.1 (#38/50)&lt;/li&gt;&lt;li&gt;Riemann-bench: 8.0 (#41/44)&lt;/li&gt;&lt;li&gt;APEX v1: 51.5 (#70/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 Max&lt;/strong&gt; — Aggregate Rating 1693, #211&lt;ul&gt;&lt;li&gt;ObviousBench: 68.75 (#489/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-3.0-flash-VL&lt;/strong&gt; — Aggregate Rating 1692, #217&lt;ul&gt;&lt;li&gt;ObviousBench: 91.6667 (#329/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Mini&lt;/strong&gt; — Aggregate Rating 1691, #218&lt;ul&gt;&lt;li&gt;o11y-bench - Pass@3: 92.19 (#55/71)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 54.69 (#60/71)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.1 Fast&lt;/strong&gt; — Aggregate Rating 1690, #223&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 20.5 (#39/50)&lt;/li&gt;&lt;li&gt;APEX v1: 57.5 (#60/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ernie 5.1&lt;/strong&gt; — Aggregate Rating 1690, #226&lt;ul&gt;&lt;li&gt;ComplexConstraints: 14.7 (#41/44)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 35B A3B&lt;/strong&gt; — Aggregate Rating 1685, #235&lt;ul&gt;&lt;li&gt;ReasonScape R12: 819.24 (#55/108)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 43.75 (#66/71)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 81.25 (#68/71)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Mini 4&lt;/strong&gt; — Aggregate Rating 1685, #236&lt;ul&gt;&lt;li&gt;Design Arena (3D): 1080.0 (#114/170)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3 Ultra&lt;/strong&gt; — Aggregate Rating 1682, #247&lt;ul&gt;&lt;li&gt;DystopiaBench: 62.4 (#22/50)&lt;/li&gt;&lt;li&gt;Klu LLM Leaderboard: 47.0 (#34/38)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 17.8 (#38/44)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 28.86 (#76/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2&lt;/strong&gt; — Aggregate Rating 1680, #248&lt;ul&gt;&lt;li&gt;APEX v1: 52.4 (#68/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Flash&lt;/strong&gt; — Aggregate Rating 1680, #249&lt;ul&gt;&lt;li&gt;ObviousBench: 92.3611 (#317/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 35B A3B&lt;/strong&gt; — Aggregate Rating 1675, #263&lt;ul&gt;&lt;li&gt;ReasonScape R12: 927.21 (#16/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2.7&lt;/strong&gt; — Aggregate Rating 1673, #266&lt;ul&gt;&lt;li&gt;ReasonScape R12: 896.49 (#25/108)&lt;/li&gt;&lt;li&gt;APEX v1: 45.8 (#75/75)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3 Mini&lt;/strong&gt; — Aggregate Rating 1669, #274&lt;ul&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 708.0 (#113/286)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Reasoning): 717.0 (#168/349)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 4.5&lt;/strong&gt; — Aggregate Rating 1664, #285&lt;ul&gt;&lt;li&gt;o11y-bench - Pass^3: 26.56 (#70/71)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 62.5 (#70/71)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 26B A4B (IT)&lt;/strong&gt; — Aggregate Rating 1660, #294&lt;ul&gt;&lt;li&gt;ReasonScape R12: 943.58 (#12/108)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Song Lyrics: 12.17 (#53/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - GEC: 36.24 (#58/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Famous Phrases: 4.1 (#64/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Meanings: 44.64 (#64/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Textile Terms: 19.2 (#65/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM Leaderboard - Average: 18.71 (#68/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Generative Quality: 37.53 (#68/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Completion: 6.24 (#68/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Case Government: 14.69 (#69/88)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2.1&lt;/strong&gt; — Aggregate Rating 1659, #295&lt;ul&gt;&lt;li&gt;ReasonScape R12: 842.69 (#45/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2.5&lt;/strong&gt; — Aggregate Rating 1653, #308&lt;ul&gt;&lt;li&gt;ReasonScape R12: 915.16 (#19/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.6V&lt;/strong&gt; — Aggregate Rating 1647, #322&lt;ul&gt;&lt;li&gt;ReasonScape R12: 523.76 (#100/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-OSS-120B&lt;/strong&gt; — Aggregate Rating 1635, #344&lt;ul&gt;&lt;li&gt;APEX v1: 46.4 (#74/75)&lt;/li&gt;&lt;li&gt;CritPt: 0.0 (#206/279)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.2 30B&lt;/strong&gt; — Aggregate Rating 1632, #350&lt;ul&gt;&lt;li&gt;AA CritPt v2: 4.29 (#89/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Nano&lt;/strong&gt; — Aggregate Rating 1631, #356&lt;ul&gt;&lt;li&gt;o11y-bench - Pass^3: 10.94 (#71/71)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 32.81 (#71/71)&lt;/li&gt;&lt;li&gt;CritPt: 0.0 (#207/279)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 26B A4B&lt;/strong&gt; — Aggregate Rating 1631, #357&lt;ul&gt;&lt;li&gt;ObviousBench: 96.5278 (#216/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude 3.5 Sonnet&lt;/strong&gt; — Aggregate Rating 1630, #359&lt;ul&gt;&lt;li&gt;BenchLM: 27.6 (#193/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;command-a-reasoning-08-2025&lt;/strong&gt; — Aggregate Rating 1629, #362&lt;ul&gt;&lt;li&gt;ReasonScape R12: 808.73 (#58/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 9B&lt;/strong&gt; — Aggregate Rating 1619, #391&lt;ul&gt;&lt;li&gt;ReasonScape R12: 753.54 (#69/108)&lt;/li&gt;&lt;li&gt;Epoch AI - Critpt: 0.57 (#186/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Laguna S 2.1&lt;/strong&gt; — Aggregate Rating 1619, #392&lt;ul&gt;&lt;li&gt;ObviousBench: 80.5556 (#443/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Command A+&lt;/strong&gt; — Aggregate Rating 1615, #403&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 32.0 (#36/38)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 4.0 (#90/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM 4.5 Air&lt;/strong&gt; — Aggregate Rating 1614, #405&lt;ul&gt;&lt;li&gt;ReasonScape R12: 798.07 (#62/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 3&lt;/strong&gt; — Aggregate Rating 1611, #410&lt;ul&gt;&lt;li&gt;GDP.pdf: 2.0 (#38/39)&lt;/li&gt;&lt;li&gt;Chartography: 9.0 (#42/42)&lt;/li&gt;&lt;li&gt;ComplexConstraints: 0.4 (#43/44)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 3.6 (#50/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4 Turbo&lt;/strong&gt; — Aggregate Rating 1609, #414&lt;ul&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 1018.0 (#60/286)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Reasoning): 632.0 (#215/349)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QwQ-32B&lt;/strong&gt; — Aggregate Rating 1606, #422&lt;ul&gt;&lt;li&gt;ReasonScape R12: 832.87 (#48/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax M1 80k&lt;/strong&gt; — Aggregate Rating 1602, #441&lt;ul&gt;&lt;li&gt;BenchLM: 26.5 (#198/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercury 2.5&lt;/strong&gt; — Aggregate Rating 1602, #443&lt;ul&gt;&lt;li&gt;ObviousBench: 94.4444 (#280/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4&lt;/strong&gt; — Aggregate Rating 1599, #449&lt;ul&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 1252.0 (#42/286)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Reasoning): 617.0 (#226/349)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 32B&lt;/strong&gt; — Aggregate Rating 1599, #450&lt;ul&gt;&lt;li&gt;ReasonScape R12: 835.58 (#47/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 1.5 Pro&lt;/strong&gt; — Aggregate Rating 1597, #456&lt;ul&gt;&lt;li&gt;BenchLM: 27.2 (#195/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Lightning&lt;/strong&gt; — Aggregate Rating 1596, #459&lt;ul&gt;&lt;li&gt;AA CritPt v2: 6.29 (#86/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;c4ai-command-a-03-2025&lt;/strong&gt; — Aggregate Rating 1596, #461&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#252/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercury 2&lt;/strong&gt; — Aggregate Rating 1596, #464&lt;ul&gt;&lt;li&gt;ObviousBench: 94.4444 (#279/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 405B Instruct&lt;/strong&gt; — Aggregate Rating 1595, #466&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#255/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 405B&lt;/strong&gt; — Aggregate Rating 1592, #477&lt;ul&gt;&lt;li&gt;BenchLM: 28.0 (#189/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 4 Maverick&lt;/strong&gt; — Aggregate Rating 1590, #482&lt;ul&gt;&lt;li&gt;BenchLM: 27.8 (#191/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron Cascade 2 30B A3B&lt;/strong&gt; — Aggregate Rating 1585, #494&lt;ul&gt;&lt;li&gt;ReasonScape R12: 846.77 (#43/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 30B A3B&lt;/strong&gt; — Aggregate Rating 1585, #496&lt;ul&gt;&lt;li&gt;ReasonScape R12: 819.42 (#54/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 4B&lt;/strong&gt; — Aggregate Rating 1581, #511&lt;ul&gt;&lt;li&gt;ReasonScape R12: 746.06 (#74/108)&lt;/li&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#245/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-OSS-20B&lt;/strong&gt; — Aggregate Rating 1578, #520&lt;ul&gt;&lt;li&gt;CritPt: 0.0 (#205/279)&lt;/li&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#248/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 4&lt;/strong&gt; — Aggregate Rating 1577, #521&lt;ul&gt;&lt;li&gt;Vals AI SAGE: 59.158 (#28/28)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 0.57 (#93/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.2 8B&lt;/strong&gt; — Aggregate Rating 1575, #524&lt;ul&gt;&lt;li&gt;AA CritPt v2: 0.86 (#92/95)&lt;/li&gt;&lt;li&gt;ObviousBench: 89.5833 (#366/643)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 Coder 32B Instruct&lt;/strong&gt; — Aggregate Rating 1573, #529&lt;ul&gt;&lt;li&gt;BenchLM: 26.1 (#200/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.7 Flash&lt;/strong&gt; — Aggregate Rating 1573, #530&lt;ul&gt;&lt;li&gt;ReasonScape R12: 596.81 (#96/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 14B&lt;/strong&gt; — Aggregate Rating 1571, #540&lt;ul&gt;&lt;li&gt;ReasonScape R12: 812.56 (#57/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hermes 4 70B&lt;/strong&gt; — Aggregate Rating 1569, #549&lt;ul&gt;&lt;li&gt;ReasonScape R12: 608.73 (#94/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 70B Instruct&lt;/strong&gt; — Aggregate Rating 1564, #562&lt;ul&gt;&lt;li&gt;Icelandic LLM - Song Lyrics: 12.04 (#56/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Famous Phrases: 5.07 (#60/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - WikiQA: 24.03 (#64/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Flora &amp; Fauna: 4.62 (#69/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Generative Quality: 37.32 (#70/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Completion: 4.12 (#72/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM Leaderboard - Average: 16.53 (#73/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Textile Terms: 10.7 (#74/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Case Government: 7.5 (#74/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Meanings: 23.18 (#78/88)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;gemma-4-E4B-it&lt;/strong&gt; — Aggregate Rating 1560, #577&lt;ul&gt;&lt;li&gt;Icelandic LLM - Song Lyrics: 12.29 (#47/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - GEC: 29.82 (#67/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Famous Phrases: 0.97 (#79/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - WikiQA: 7.47 (#82/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Textile Terms: 5.6 (#82/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Case Government: 3.75 (#82/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Completion: 0.8 (#83/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Meanings: 14.43 (#83/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM Leaderboard - Average: 10.9 (#84/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Flora &amp; Fauna: 0.1 (#85/88)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Magistral Small 1.2&lt;/strong&gt; — Aggregate Rating 1558, #588&lt;ul&gt;&lt;li&gt;ReasonScape R12: 674.57 (#83/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek R1 Distill Llama 70B&lt;/strong&gt; — Aggregate Rating 1553, #601&lt;ul&gt;&lt;li&gt;ReasonScape R12: 670.06 (#84/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Reka Flash 3&lt;/strong&gt; — Aggregate Rating 1550, #613&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#258/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Laguna XS.2&lt;/strong&gt; — Aggregate Rating 1540, #640&lt;ul&gt;&lt;li&gt;ReasonScape R12: 619.2 (#91/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 4 Scout&lt;/strong&gt; — Aggregate Rating 1535, #656&lt;ul&gt;&lt;li&gt;BenchLM: 26.8 (#196/200)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.2 3B&lt;/strong&gt; — Aggregate Rating 1532, #669&lt;ul&gt;&lt;li&gt;AA CritPt v2: 0.0 (#95/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hunyuan A13B-Instruct&lt;/strong&gt; — Aggregate Rating 1532, #670&lt;ul&gt;&lt;li&gt;ReasonScape R12: 691.15 (#79/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 8B&lt;/strong&gt; — Aggregate Rating 1531, #676&lt;ul&gt;&lt;li&gt;ReasonScape R12: 751.22 (#72/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Phi-4&lt;/strong&gt; — Aggregate Rating 1523, #703&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#259/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4.1 Nano&lt;/strong&gt; — Aggregate Rating 1516, #731&lt;ul&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 582.0 (#157/286)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Reasoning): 570.0 (#260/349)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek R1 0528 Qwen3 8B&lt;/strong&gt; — Aggregate Rating 1509, #756&lt;ul&gt;&lt;li&gt;ReasonScape R12: 628.29 (#89/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hermes-4-14B&lt;/strong&gt; — Aggregate Rating 1499, #782&lt;ul&gt;&lt;li&gt;ReasonScape R12: 357.27 (#105/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 E2B (IT)&lt;/strong&gt; — Aggregate Rating 1498, #785&lt;ul&gt;&lt;li&gt;Icelandic LLM - GEC: 44.04 (#49/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Song Lyrics: 10.81 (#80/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM Leaderboard - Average: 11.75 (#82/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Famous Phrases: 0.6 (#82/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Completion: 0.93 (#82/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Textile Terms: 3.3 (#84/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Generative Quality: 24.29 (#84/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Case Government: 2.19 (#85/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Flora &amp; Fauna: 0.0 (#86/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Meanings: 8.88 (#86/88)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3 Nano 4B&lt;/strong&gt; — Aggregate Rating 1485, #831&lt;ul&gt;&lt;li&gt;Klu LLM Leaderboard: 24.0 (#38/38)&lt;/li&gt;&lt;li&gt;AA CritPt v2: 0.29 (#94/95)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 4B&lt;/strong&gt; — Aggregate Rating 1484, #836&lt;ul&gt;&lt;li&gt;ReasonScape R12: 685.95 (#80/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NVIDIA Nemotron Nano 9B V2&lt;/strong&gt; — Aggregate Rating 1480, #852&lt;ul&gt;&lt;li&gt;ReasonScape R12: 595.66 (#97/108)&lt;/li&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#257/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-3.5 Turbo&lt;/strong&gt; — Aggregate Rating 1476, #862&lt;ul&gt;&lt;li&gt;AI Chess Leaderboard (Continuation): 1382.0 (#25/286)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Reasoning): 530.0 (#287/349)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.0 Micro&lt;/strong&gt; — Aggregate Rating 1472, #878&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#249/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 8B Instruct&lt;/strong&gt; — Aggregate Rating 1442, #1036&lt;ul&gt;&lt;li&gt;Icelandic LLM - Song Lyrics: 11.45 (#71/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Generative Quality: 31.38 (#79/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - WikiQA: 7.82 (#81/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Flora &amp; Fauna: 0.29 (#83/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM Leaderboard - Average: 8.45 (#87/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Famous Phrases: 0.24 (#87/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - GEC: 8.26 (#87/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Case Government: 2.19 (#87/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Completion: 0.27 (#87/88)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Textile Terms: 1.4 (#88/88)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.2 11B Instruct&lt;/strong&gt; — Aggregate Rating 1423, #1124&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#254/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 2B&lt;/strong&gt; — Aggregate Rating 1419, #1151&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#250/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SmolLM3-3B&lt;/strong&gt; — Aggregate Rating 1389, #1248&lt;ul&gt;&lt;li&gt;ReasonScape R12: 303.06 (#106/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek R1 Distill Llama 8B&lt;/strong&gt; — Aggregate Rating 1374, #1303&lt;ul&gt;&lt;li&gt;ReasonScape R12: 12.66 (#108/108)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 0.8B&lt;/strong&gt; — Aggregate Rating 1322, #1424&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#253/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Granite 4.0 350M&lt;/strong&gt; — Aggregate Rating 1320, #1430&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#251/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 270M (IT)&lt;/strong&gt; — Aggregate Rating 1190, #1580&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 0.0 (#246/259)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (190)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on AA CritPt v2: 76.86 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on APEX v1: 73.8 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ComplexConstraints: 45.1 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on CritPt: 29.7 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Epoch AI - Mirrorcode: 93.33 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Epoch AI - Scicode: 63.08 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on GDP.pdf: 27.6 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Klu LLM Leaderboard: 90.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ObviousBench: 99.3056 (#44)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ProgramBench: 9.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ProgramBench Almost: 55.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Mythos 5.1&lt;/strong&gt; on ExploitGym (version unspecified; 6h timeout): 264.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AA CritPt v2: 78.86 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on APEX v1: 65.8 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ASCIIBench: 1781.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on CritPt: 31.7 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Design Arena (ASCII Art): 1355.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Mirrorcode: 78.89 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Scicode: 66.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on GDP.pdf: 30.6 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on InfoOps Bench: 86.9 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Klu LLM Leaderboard: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ObviousBench: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ProgramBench: 18.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ProgramBench Almost: 70.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Supabase Evals (No Skills): 89.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Supabase Evals (Skills): 92.8 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - API Security: 75.9 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Cloud Security: 66.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Code Vulnerabilities: 58.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Overall: 64.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Websec.fr CTFs: 57.5 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - API Security: 77.8 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Cloud Security: 66.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 58.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Overall: 64.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 55.2 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on o11y-bench - Pass@3: 98.44 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on o11y-bench - Pass^3: 93.75 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AA CritPt v2: 83.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on APEX v1: 69.3 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on EnterpriseBench (CoreCraft Agents): 42.6 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Epoch AI - Mirrorcode: 60.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on HANDBOOK.md Agents: 31.4 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Kaggle WWTP Engineering: 50.0 (#41)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Klu LLM Leaderboard: 93.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on ObviousBench: 100.0 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on ProgramBench: 5.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on ProgramBench Almost: 52.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - API Security: 81.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Cloud Security: 72.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 56.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Overall: 70.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 70.1 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Cloud Security: 68.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Code Vulnerabilities: 56.1 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Overall: 69.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Websec.fr CTFs: 71.3 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on o11y-bench - Pass@3: 96.88 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on o11y-bench - Pass^3: 87.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AA CritPt v2: 81.71 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on APEX v1: 69.3 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on ASCIIBench: 1672.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Architectural RedlineBench: 59.1 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on CAIS Risk Index: 47.2 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on DeepsecBench: 39.49 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Mirrorcode: 61.11 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Gray Swan IPI — Combined Attack Set, Transfer Only, No Computer Use: 8.6 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on HANDBOOK.md Agents: 28.6 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on InfoOps Bench: 86.5 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Klu LLM Leaderboard: 91.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on ObviousBench: 100.0 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-5.6 Launch - Internal Research Debugging Evaluation: 75.52 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-5.6 Launch - KernelGen 1P: 60.44 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on SEAL - MultiChallenge: 81.95 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Supabase Evals (No Skills): 78.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Supabase Evals (Skills): 81.2 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - API Security: 80.2 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Cloud Security: 66.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 56.6 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Overall: 70.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 79.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - API Security: 80.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Code Vulnerabilities: 56.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Overall: 70.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Websec.fr CTFs: 77.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on o11y-bench - Pass@3: 96.88 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on o11y-bench - Pass^3: 87.5 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ASCIIBench: 2006.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Architectural RedlineBench: 46.4 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on CAIS Vision Capabilities Index: 71.2 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on CrueltyBench: 30.0 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Design Arena (SVG): 1329.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EQ-Bench Longform Writing: 80.9 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Frontiermath Erdos: 1.47 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Rli: 17.08 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on GDP.pdf: 28.4 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on InfoOps Bench: 45.4 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Klu LLM Leaderboard: 91.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Lech Mazur Writing: 0.719 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Criteria Pass Rate: 30.5 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Form: 60.3 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Task Pass Rate: 25.8 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Criteria Pass Rate: 86.2 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Form: 66.5 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Task Pass Rate: 53.3 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PRISM (1C:Enterprise) - Algorithmic Tasks (A): 93.0 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PRISM (1C:Enterprise) - Platform Tasks (B): 85.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ProphetArena: 0.9445 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on RSI-Exam: 0.494 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Terminal-Bench 2.1: 57.678 (#43)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on o11y-bench - Pass@3: 98.44 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on o11y-bench - Pass^3: 87.5 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA CritPt v2: 78.86 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf: 28.4 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Construction - Criterion Pass Rate: 76.92 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Criterion Pass Rate: 80.77 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Engineering - Criterion Pass Rate: 77.86 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 83.56 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - HR - Criterion Pass Rate: 80.75 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Healthcare - Criterion Pass Rate: 88.85 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Insurance - Criterion Pass Rate: 89.13 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Legal - Criterion Pass Rate: 83.05 (#38)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 78.24 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Real Estate - Criterion Pass Rate: 70.54 (#50)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - STEM/Research - Criterion Pass Rate: 80.11 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDPval: 1732.68 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Long Context Reasoning: 79.33 (#104)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Omniscience: 53.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench 4.0: 65.15 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase: 61.82 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - Analytical Quality Elo: 1671.64 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - DOCX - Rubric Pass Rate: 63.31 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - PDF - Rubric Pass Rate: 63.79 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - PPTX - Rubric Pass Rate: 62.04 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - Presentation Elo: 1598.74 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - XLSX - Rubric Pass Rate: 59.32 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-LCR: 79.33 (#73)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Accuracy: 58.27 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Hallucination Rate: 11.42 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Business: 42.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Health: 41.78 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Humanities &amp; Social Sciences: 56.22 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Law: 53.11 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Science, Engineering &amp; Mathematics: 45.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE): 76.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - C: 90.91 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Dart: 80.56 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Go: 72.22 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - HTML: 61.11 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Java: 70.91 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 95.52 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Julia: 68.42 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 34.29 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - PHP: 67.65 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Python: 86.17 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - R: 40.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Rust: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Swift: 57.69 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 90.7 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIIQ Composite IQ: 139.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on APEX v1: 80 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Artificial Analysis Intelligence Index: 59.43 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench: 53.88 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA: 77.82 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Finance - Raw Objective Completion: 97.34 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - HR - Raw Objective Completion: 81.81 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Marketing - Raw Objective Completion: 94.44 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Operations - Raw Objective Completion: 97.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Sales - Raw Objective Completion: 92.21 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Strict: 55.25 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Support - Raw Objective Completion: 94.26 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on CritPt: 36.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Critpt: 36.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Scicode: 64.24 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierSWE V2: 61.19949235294118 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LLM Stats (FrontierSWE V2): 61.2 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LLM Stats (Terminal-Bench-Science 0.1): 64.8 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LLM Stats Score: 58.99 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Mercor APEX: 84.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI (Vals Index): 70.355 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Excel Modeling v1.1: 76.468 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Finance Agent v2.1: 67.162 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Finance Index: 66.35 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench v1.1: 17.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Legal Index: 75.828 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Legal Research Bench: 55.769 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Tax Agent Bench: 48.86 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 63.131 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Vibe Code Bench: 89.06 (#8)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (86)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Vending Bench 2&lt;/strong&gt;: Gemini 4 Carbon (16484.21) beat GPT-6 Astra by 969.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vending-Bench 2&lt;/strong&gt;: Gemini 4 Carbon (16484.20666666667) beat GPT-6 Astra by 969.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ExploitGym (version unspecified; 6h timeout)&lt;/strong&gt;: Claude Opus 5.5 (300.0) beat Claude Mythos 5 by 53.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI SAGE&lt;/strong&gt;: Claude Opus 5.5 (Max) (91.302) beat Claude Opus 4.7 (Max) by 35.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ProgramBench Almost&lt;/strong&gt;: Claude Opus 5.5 (xHigh) (70.5) beat Claude Opus 5 (xHigh) by 33.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (AutomationBench-AA)&lt;/strong&gt;: Kimi K3.1 Pro (73.3) beat Grok 4.5 by 21.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Mirrorcode&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (97.78) beat Claude Opus 5.5 (Max) by 20.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coarena&lt;/strong&gt;: GPT-6.1 Terra (1162.3) beat GPT-6.1 Sol by 14.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Health&lt;/strong&gt;: Gemini 4 Carbon (High) (41.78) beat GPT-6 Astra (High) by 14.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ProgramBench&lt;/strong&gt;: Claude Opus 5.5 (xHigh) (18.5) beat Claude Opus 5 (xHigh) by 14.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VAmoS Pro Bench&lt;/strong&gt;: Phonic phonic-1 (57.5) beat Grok Voice Think Fast 2.0 by 12.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Google Argon Report - GraphWalks BFS 256k to 1M&lt;/strong&gt;: Gemini 4 Carbon (96.2) beat Gemini 4 Argon by 12.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Graphwalks BFS &gt;128k; 256K–1M range (F1))&lt;/strong&gt;: Gemini 4 Carbon (96.2) beat Gemini 4 Argon by 12.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Humanities &amp; Social Sciences&lt;/strong&gt;: Gemini 4 Carbon (High) (56.22) beat Claude Opus 5.5 (Max, Default Fallback) by 11.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReasonScape R12&lt;/strong&gt;: DeepSeek V4.1 Flash (965.6) beat Muse-Glimmer-30B (high, 16k) by 11.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (ExploitGym)&lt;/strong&gt;: Gemini 4 Carbon (53.5) beat GPT-6 Astra by 11.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-5.6 Launch - Internal Research Debugging Evaluation&lt;/strong&gt;: GPT-6 Astra (78.05) beat GPT-5.6 Sol by 9.75&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NatureBench&lt;/strong&gt;: GPT-6.1 Sol (36.0) beat Claude Opus 5 (HELIX) by 9.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Business&lt;/strong&gt;: Gemini 4 Carbon (High) (42.67) beat Claude Fable 5 (Max, Opus 4.8 Fallback) by 8.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - Julia&lt;/strong&gt;: Claude Opus 5.5 (Max, Default Fallback) (100.0) beat Claude Sonnet 5.5 (Max, Default Fallback) by 8.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - Rust&lt;/strong&gt;: Gemini 4 Carbon (High) (100.0) beat Claude Opus 5.5 (Max, Default Fallback) by 8.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics&lt;/strong&gt;: Gemini 4 Carbon (High) (45.67) beat Claude Opus 5.5 (Max, Default Fallback) by 7.87&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (APEX-Agents)&lt;/strong&gt;: Kimi K3.1 Pro (67.2) beat GPT-6.1 Sol by 7.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Omniscience&lt;/strong&gt;: Gemini 4 Carbon (High) (53.5) beat Claude Opus 5.5 (Max, Default Fallback) by 7.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-5.6 Launch - KernelGen 1P&lt;/strong&gt;: GPT-6 Astra (66.72) beat GPT-5.6 Sol by 5.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Humanity's Last Exam (with tools))&lt;/strong&gt;: Kimi K3.1 Pro (70.0) beat Claude Sonnet 5.5 by 5.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Reclamation farming&lt;/strong&gt;: GPT-6 Luna (71.1) beat GPT-5.4 by 5.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAquaBench - Ichthyopathology&lt;/strong&gt;: GPT-6 Luna (64.2) beat GPT-5.4 by 4.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BALSAM - Summarization&lt;/strong&gt;: Smart-1.1 (63.78) beat soual-3 by 4.76&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DystopiaBench&lt;/strong&gt;: Claude Opus 4.8 (19.1) beat Claude Haiku 4.5 by 4.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;o11y-bench - Pass^3&lt;/strong&gt;: Claude Opus 5.5 (Medium) (93.75) beat Claude Sonnet 5.5 (High) by 4.69&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWT-Bench Verified&lt;/strong&gt;: DPIAgent + GPT-5 (with test selection) (87.5) beat PatchTwin+Guard + GPT-5 by 4.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Critpt&lt;/strong&gt;: Gemini 4 Carbon (High) (36.0) beat GPT-5.6 Sol (Max) by 3.71&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BenCzechMark - NER&lt;/strong&gt;: Muse Glimmer 30B (98.07692307692308) beat Llama 3.1 405B Instruct by 3.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CritPt&lt;/strong&gt;: Gemini 4 Carbon (High) (36.0) beat Muse Wonder Full (Max) by 3.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (BabyVision)&lt;/strong&gt;: Kimi K3.1 Pro (95.4) beat DeepSeek V4.1 Flash (1017) by 2.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Codex) - Cloud Security&lt;/strong&gt;: GPT-6.1 Sol (68.1) beat GPT-6 Sol by 2.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Coding Daily (OpenCode) - Total&lt;/strong&gt;: Kimi K3.1 Pro (Max) (63.8) beat DeepSeek V4.1 Flash (1017) (Max) by 2.83&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (DeepSWE)&lt;/strong&gt;: Kimi K3.1 Pro (75.5) beat GPT-5.6 Sol by 2.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (SEC-bench Pro)&lt;/strong&gt;: Gemini 4 Carbon (88.2) beat GPT-6 Astra by 2.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Finance - Raw Objective Completion&lt;/strong&gt;: Gemini 4 Carbon (High) (97.34) beat Gemini 4 Argon (High) by 2.72&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (AutomationBench; Private held-out set)&lt;/strong&gt;: Gemini 4 Carbon (53.9) beat Gemini 4 Argon by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench&lt;/strong&gt;: Gemini 4 Carbon (High) (53.88) beat Gemini 4 Argon (High) by 2.59&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Seed production and seed science&lt;/strong&gt;: GPT-6 Luna (76.3) beat GPT-5.4 by 2.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Terminal-Bench 2.1)&lt;/strong&gt;: SWE-2 (92.8) beat DeepSeek V4.1 Flash by 2.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Strict&lt;/strong&gt;: Gemini 4 Carbon (High) (55.25) beat Gemini 4 Argon (High) by 2.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM-AggreFact&lt;/strong&gt;: nimble-v4-max (79.45) beat Bespoke-Minicheck-7B by 2.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Ebr Bench&lt;/strong&gt;: GPT-6.1 Terra (Max) (78.1) beat GPT-6 Astra (Max) by 1.91&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Codex) - API Security&lt;/strong&gt;: GPT-6 Astra (81.5) beat GPT-5.6 Sol by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Artificial Analysis Intelligence Index&lt;/strong&gt;: Gemini 4 Carbon (High) (59.43) beat Claude Opus 5.5 (Max, Default Fallback) by 1.81&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercor APEX&lt;/strong&gt;: Gemini 4 Carbon (High) (84.2) beat Gemini 4 Argon (High) by 1.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BenchLM&lt;/strong&gt;: Gemini 4 Carbon (88.4) beat Claude Opus 5.5 by 1.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MLCR-AA - Accuracy&lt;/strong&gt;: GPT-6.1 Terra (Max) (98.33) beat GPT-6.1 Sol (Max) by 1.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Crop cultivation technologies&lt;/strong&gt;: GPT-6 Luna (82.2) beat GPT-5.4 by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTMedQFundamental - General chemistry&lt;/strong&gt;: Qwen 3.8 27B (80.4) beat Qwen 3 30B A3B 2507 (Thinking) by 1.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate&lt;/strong&gt;: GPT-6.1 Terra (Max) (92.74) beat Claude Opus 5.5 (Max, Default Fallback) by 1.49&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Farming systems on agricultural landscapes&lt;/strong&gt;: GPT-6 Luna (74.7) beat GPT-5.4 by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OSWorld 2.1 (partial))&lt;/strong&gt;: Gemini 4 Carbon (81.5) beat Claude Sonnet 5.5 by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAquaBench - Feeding of fish and aquatic organisms&lt;/strong&gt;: GPT-6 Luna (73.4) beat GPT-5.4 by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - General genetics&lt;/strong&gt;: GPT-6 Luna (84.6) beat GPT-5.4 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - General agriculture&lt;/strong&gt;: GPT-6 Luna (74.1) beat GPT-5.4 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Medicine and Mental Health&lt;/strong&gt;: Kimi K3.1 Pro (92.7) beat Doubao-Seed-2.1-Pro-260628 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTAgroBench - Botany&lt;/strong&gt;: GPT-6 Luna (89.9) beat GPT-5.4 by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ViDoRe V3 Pipeline&lt;/strong&gt;: VisAgentRet (EVIE-4.5B + GLM-5.3-Flash) (67.84) beat NeMo Retriever Agentic (ColEmbed-VL-8B + Opus 4.5) by 0.77&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Finance Index&lt;/strong&gt;: Gemini 4 Carbon (High) (66.35) beat Claude Opus 5.5 (Max) by 0.76&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ForecastBench&lt;/strong&gt;: model-4 (69.7) beat fire-hedgehog by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Agents' Last Exam)&lt;/strong&gt;: GPT-6.1 Terra (60.0) beat GPT-6 Astra by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gray Swan IPI — Combined Attack Set, Transfer Only, No Computer Use&lt;/strong&gt;: Gemini 4 Carbon (0.0) beat Gemini 4 Argon by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWE-bench Live&lt;/strong&gt;: AiWork.Code + Claude Opus 5.5 (xHigh) (71.67) beat TianxiCode + Deepseek-v4.1-Flash by 0.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Briefcase - Presentation Elo&lt;/strong&gt;: Claude Opus 5.5 (Max, Default Fallback) (1716.31) beat Claude Sonnet 5.5 (Max, Default Fallback) by 0.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NVIDIA Nemotron 3.5 Super VL Card - GraphWalks&lt;/strong&gt;: Nemotron 3.5 Super VL (NVFP4) (90.34) beat Nemotron 3.5 Super VL (BF16) by 0.63&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (BrowseComp)&lt;/strong&gt;: Kimi K3.1 Pro (92.1) beat GPT-6 Astra by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Generative Quality&lt;/strong&gt;: GPT-6.1 Terra (84.34) beat GPT-6 Sol by 0.59&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Legal Research Bench&lt;/strong&gt;: Gemini 4 Carbon (High) (55.769) beat Muse Spark 1.3 (Max) by 0.48&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - Agriculture&lt;/strong&gt;: GPT-6 Luna (76.3) beat GPT-5.4 by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MERA Industrial - ruTXTMedQFundamental - Pathophysiology&lt;/strong&gt;: Qwen 3.8 27B (90.0) beat DeepSeek V4 Pro by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA&lt;/strong&gt;: Gemini 4 Carbon (High) (77.82) beat Gemini 4 Argon (High) by 0.31&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education - Middle School Subjects&lt;/strong&gt;: Kimi K3.1 Pro (60.4) beat doubao-seed-2-1-lite-260915 by 0.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - ECI&lt;/strong&gt;: GPT-6 Astra (169.06) beat Claude Opus 5.5 by 0.23&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Sales - Raw Objective Completion&lt;/strong&gt;: Gemini 4 Carbon (High) (92.21) beat Gemini 4 Argon (High) by 0.21&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Coding Daily (OpenCode) - Laravel Code Quality&lt;/strong&gt;: Kimi K3.1 Pro (Max) (18.55) beat Kimi K3 (High) by 0.15&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Law&lt;/strong&gt;: Gemini 4 Carbon (High) (53.11) beat Gemini 4 Argon (High) by 0.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenRouter Tau2-Bench Airline&lt;/strong&gt;: GPT-5.1 (80.0) beat Claude Fable 5 by 0.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NVIDIA Nemotron 3.5 Super VL Card - OCRBench v2, EN&lt;/strong&gt;: Nemotron 3.5 Super VL (NVFP4) (68.34) beat Qwen 3.5 122B A10B by 0.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Blueprint Bench 2&lt;/strong&gt;: Gemini 4 Carbon (0.586) beat Gemini 4 Argon by 0.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Blueprint-Bench 2&lt;/strong&gt;: Gemini 4 Carbon (0.586) beat Gemini 4 Argon by 0.04&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-11-01

=== DAILY ===
NEW SCORES FROM TOP-10 MODELS (4)
  - Claude Fable 5.1 on ObviousBench: 99.3056 Answer pass³ (%) (#44/643)
  - Claude Opus 5.5 on ASCIIBench: 1781.0 Bradley–Terry score (Elo scale) (#2/13)
  - Claude Opus 5.5 on ObviousBench: 100.0 Answer pass³ (%) (#</summary></entry><entry><title>The Aggregate Digest — 2026-10-31</title><id>https://theaggregate.ai/digest/2026-10-31</id><updated>2026-10-31T09:42:46.974137+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Frontier&lt;/h2&gt;
&lt;ul&gt;&lt;li&gt;Best measured model: Gemini 4 Carbon (2030) takes the crown from Claude Opus 5.5 (1995), which had trailed for 3 days&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (12)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;AA Coding Agent Index v2.0&lt;/strong&gt; (Coding Agent Index (0-100)): leader Claude Opus 5.5 (Max, Default Fallback) (Harness: Claude Code) (61.65), 25 models&lt;br&gt;&lt;span&gt;Artificial Analysis Coding Agent Index v2.0 (October 2026): each row is a model running inside a coding-agent harness such as Claude Code, Codex or Mistral Vibe, scored as the equal-weight average of FrontierCode v1.1 Main, Terminal-Bench 4.0 and SWE-Atlas-QnA-AA over three attempts per task. Unlike AA&amp;#x27;s model-level evaluations, the coding agent named in each row is part of what is measured; v2.0 replaced DeepSWE with FrontierCode, so earlier Coding Agent Index figures are not comparable. Router and fusion agents, and cells where refusals stopped more than 5% of attempts, are excluded. Index (0-100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepsecBench-AA&lt;/strong&gt; (Median F2 (%)): leader GPT-6 Sol (Max) (46.32), 22 models&lt;br&gt;&lt;span&gt;Artificial Analysis&amp;#x27;s private DeepsecBench run: agents statically review open-source application code pinned before major vulnerability fixes and report findings, which a GPT-5.6 Sol (high) judge validates and matches to an expert-verified golden set; F2 weights recall over precision. Median of three runs on AA&amp;#x27;s Stirrup harness; cells for models that declined more than 5% of tasks are withheld. F2 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DAYJOB: Finance&lt;/strong&gt; (Pass rate at a 100% rubric threshold (%)): leader Claude Opus 5.5 (Adaptive Reasoning, Max Effort) (42.5), 38 models&lt;br&gt;&lt;span&gt;DAYJOB: Finance (Surge AI, 2026): 80 expert-designed finance assignments across corporate finance, banking, credit, investing and real assets. Each is a brief workplace request (81 words on average) backed by 25.7 input files on average and represents an estimated 16.6 hours of professional work. An agent works through the documents and spreadsheets in a sandboxed container (Harbor tasks, OpenHands-based harness, network limited to the model API) and delivers the finished analysis. Unlike FinanceBench-style question answering over public filings, the agent has to choose its own sources and approach: &amp;quot;Financial assignments often arrive without instructions for which sources to trust or how to approach the analysis. DAYJOB asks agents to make those decisions themselves.&amp;quot; Work is graded against a task-specific rubric (median 57.5 criteria per task). Score: the percentage of tasks passed with 100% of rubric criteria satisfied, five attempts per task, Claude Opus 4.8 as the autograder; reasoning effort as labelled. Higher is better. Source: https://surgehq.ai/benchmarks/dayjob-finance&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DAYJOB: Healthcare&lt;/strong&gt; (Pass rate at a 100% rubric threshold (%)): leader Claude Opus 5.5 (Adaptive Reasoning, Max Effort) (48.3), 38 models&lt;br&gt;&lt;span&gt;DAYJOB: Healthcare (Surge AI, 2026): 50 expert-designed healthcare assignments across clinical, operational, payer, pharmacy and compliance workflows. Each starts from a brief handoff (prompts average 49 words) backed by 19.8 supporting files on average and represents an estimated 13.6 hours of professional work. An agent works through the files in a sandboxed container (Harbor tasks, OpenHands-based harness, network limited to the model API) and has to work out what the request needs: &amp;quot;Rather than telling models exactly what to do, it asks whether agents can figure out what needs to be done, and reason through it until the end.&amp;quot; Unlike MedQA-style licensing-exam questions or HealthBench&amp;#x27;s rubric-graded health conversations, it grades end-to-end agentic work over patient and operations records against a task-specific rubric (median 47.5 criteria per task). Score: the percentage of tasks passed with 100% of rubric criteria satisfied, five attempts per task, Claude Opus 4.8 as the autograder; reasoning effort as labelled. Higher is better. Source: https://surgehq.ai/benchmarks/dayjob-healthcare&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DAYJOB: Tax&lt;/strong&gt; (Pass@1 at a 95% rubric threshold (%)): leader Claude Sonnet 5.5 (Adaptive Reasoning, Max Effort) (10.8), 16 models&lt;br&gt;&lt;span&gt;DAYJOB: Tax (Surge AI, 2026): 50 professional tax assignments across eight areas of practice, from individual returns to partnerships, estates and international tax, with 1,789 workspace files. An agent works on local files in a Linux sandbox through the OpenHands harness and delivers returns, workpapers, reviews or client advice. Unlike TaxBench or TaxCalcBench, which pose a defined question or return, &amp;quot;DAYJOB also asks whether the client asked the right question&amp;quot;: the agent must find issues the client overlooked. Score: pass@1 at a 95% rubric threshold, the share (%) of attempts that satisfy at least 95% of the task&amp;#x27;s rubric criteria (five attempts per task, Claude Opus 4.8 autograder); reasoning effort as labelled. Higher is better. Source: https://surgehq.ai/benchmarks/dayjob-tax&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDP.xlsx&lt;/strong&gt; (Mean rubric score (%)): leader Gemini 4 Argon (High) (38.3), 20 models&lt;br&gt;&lt;span&gt;GDP.xlsx (Surge AI, 2026): 70 workplace questions, each paired with a professional spreadsheet (58 Excel and 12 CSV files in all, 393 worksheets, more than 2.1 million populated cells and fields) from finance, healthcare, engineering, real estate, the public sector and other domains. An agent inspects the file with Python spreadsheet libraries and LibreOffice in a sandboxed container (Harbor tasks, OpenHands-based harness, up to 200 turns and 60 minutes per task) and answers in its final message. Unlike SpreadsheetBench, which checks the cells an agent writes into a workbook, GDP.xlsx grades the written answer against task-specific must-have rubric criteria, and &amp;quot;It grades only the final response text: content that appears only in intermediate messages or in files the agent created is not credited.&amp;quot; Score: mean rubric score (%) as Surge&amp;#x27;s evaluation harness README defines the benchmark score: a Gemini 3.8 Flash judge marks each criterion met or not, an attempt scores the share met, a task scores the mean of five attempts and the benchmark score is the unweighted mean over tasks (the README does not say outright that the leaderboard runs used these settings); reasoning effort as labelled. Higher is better. Source: https://surgehq.ai/benchmarks/gdp-xlsx&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LegalReasonBench&lt;/strong&gt; (Mean score (%)): leader Claude Fable 5.1 (High) (50.6), 8 models&lt;br&gt;&lt;span&gt;LegalReasonBench (micro1, 2026): 50 legal matters that practicing lawyers built end to end, each &amp;quot;a sequence of two to five assignments worked in one persistent sandbox&amp;quot; (122 assignments, 2,236 rubric criteria), from reading the case file and researching the law to court filings and memoranda. Unlike LegalBench, whose bar-exam style prompts use &amp;quot;a format that frontier models have since saturated&amp;quot;, the client facts &amp;quot;stay in the uploaded case files rather than being paraphrased into the prompt&amp;quot;. Protocol: the OpenCode 1.18.31 agent harness &amp;quot;with reasoning effort set to high for every model&amp;quot; in the report (shell with pdftotext, pandoc and Python, file tools, grep and glob, websearch and webfetch; web research is allowed in the 22 open-universe matters), a three-hour limit per assignment and three runs per matter; a strict majority of three LLM judges (GPT-5.6 Terra, Claude Sonnet 5, Gemini 3.8 Flash, each at medium reasoning effort) decides every criterion. Score: mean score (%), the mean over the 50 matters of each matter&amp;#x27;s mean reward over three runs; an assignment&amp;#x27;s reward is earned rubric points (penalties subtracted) over available positive points, clamped to [0, 1], and a matter&amp;#x27;s reward is the points-weighted total across its assignments. Rows labelled (High) are the report&amp;#x27;s runs; Kimi K3.1 Pro was added after the report and its run settings are not described; Gemini 4 Argon, which answered newer versions of 17 matters, is not listed. Higher is better. Source: https://www.micro1.ai/benchmark/legal-reason-bench&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MissionReport&lt;/strong&gt; (Mean score (%)): leader Gemini 4 Argon (64.1), 9 models&lt;br&gt;&lt;span&gt;MissionReport (micro1, 2026): 20 government status-reporting assignments, each written by an expert who has done the job, in four report families (weekly activity, after-action, program status, and performance and accomplishment reports) and five settings (Army, Air Force, Navy, Marine Corps, and Department of Defense offices and agencies). Each task gives &amp;quot;a role, a reader, the working files and the organization&amp;#x27;s template&amp;quot;; the model &amp;quot;must produce a finished Word document, and the expert&amp;#x27;s rubric scores each decision the report makes.&amp;quot; Unlike broad occupational suites such as GDPval (44 occupations), it covers one job, where &amp;quot;the difficulty is not in the prose&amp;quot; but in what a reader &amp;quot;who has none of the writer&amp;#x27;s context&amp;quot; needs; its 20 rubrics hold 302 positive criteria worth 535 points and 63 penalties worth 88 points. Protocol: the OpenCode 1.18.31 harness in a fresh Linux sandbox (shell and file tools, with pandoc, pdftotext, LibreOffice and Python document libraries); &amp;quot;Every model ran every task three times using default reasoning effort, with a 60-minute limit and no feedback between attempts.&amp;quot; An agentic judge, Claude Opus 5.5 in OpenCode, decides each criterion true or false against the expert&amp;#x27;s report and never sees the weights, the grading notes or the source files. Score: mean score (%), the mean over tasks of each model&amp;#x27;s three-report mean; a report scores &amp;quot;the positive points it earns minus the penalty points it triggers, divided by the task&amp;#x27;s positive points, bounded to 0 to 100%&amp;quot;. Rows keep their bare labels because every model ran at its default reasoning effort. Higher is better. Source: https://www.micro1.ai/benchmark/mission-report&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RealmPathology Benchmark&lt;/strong&gt; (Mean reward (%)): leader Claude Fable 5.1 (Max) (78.0), 15 models&lt;br&gt;&lt;span&gt;RealmPathology Benchmark (micro1, 2026): expert-authored tasks that give an agent one or more real anatomic pathology report PDFs and ask for &amp;quot;the structured interpretation a tumor board can act on&amp;quot;. Unlike pathology boards that grade image reading on whole-slide images, such as MedFlowBench&amp;#x27;s QuPath tasks, the input is the written report: extraction criteria are met 93% of the time, and the score turns on interpretation, staging, management and not overreaching. Of 1,071 rubric criteria, 299 are penalties; &amp;quot;a penalty fires when a summary invents a finding, misclassifies the case, or recommends beyond what the report supports, and it subtracts from the score.&amp;quot; This release is &amp;quot;a new comparison identity&amp;quot;, &amp;quot;not continuous with the earlier release&amp;quot;, whose rubrics had no penalties. Protocol: a sandboxed OpenCode container with a shell, file tools, a todo list, and web search and web fetch; three rollouts per task; one judge, GPT-5.4 mini (snapshot of 17 March 2026, medium effort), grades every criterion against a hidden expert answer, one call per criterion. Efforts from the methodology note: Claude Fable 5.1, Claude Sonnet 5.5, Claude Opus 5.5 and Claude Opus 5 at max; Muse Wonder at max via OpenRouter; Muse Spark 1.3, Muse Spark 1.1 and Grok 4.7 at xhigh; Grok 4.6 and Gemini 3.8 Flash at high; Claude Sonnet 5 at default; GPT-6 Astra and GPT-6 Sol at max &amp;quot;with the subagent task tool denied&amp;quot;. Score: mean reward (%), the mean over tasks of each model&amp;#x27;s mean reward over three rollouts, shown times 100 (the report prints 0.780 for 78.0); reward is &amp;quot;earned positive weight over total positive weight, floored at zero&amp;quot;. Rows carry those efforts; Gemini 4 Argon (high) and Kimi K3.1 Pro were evaluated after the report (&amp;quot;The report below reflects only the original evaluations except the sample task runs.&amp;quot;) and are listed as printed. Higher is better. Source: https://www.micro1.ai/benchmark/realm-pathology&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sports Film Review - Frames - svc-bench-3&lt;/strong&gt; (Mean score above the blind-guess floor (0-1)): leader Claude Opus 5.5 (1.0), 5 models&lt;br&gt;&lt;span&gt;Sports Film Review, svc-bench-3 (eyes), frames track (mariusDing/sports-video-coach, 2026): a model reviews four synthetic sports videos (21 to 53 s) by eye and answers with numbers: the left player&amp;#x27;s foot touches while two players juggle (a count; double touches count twice, floor bounces not at all), made or missed for 12 shots at a far hoop, the release times of 12 passes between three players (four pass fakes are not passes), and a rower&amp;#x27;s stroke rate from 15 s, when the rate changes. The author built it for the frames track after strong models scored 1.0 on svc-bench-2 by eye: the events are small, short or hidden, so they have to be found still by still. The tool draws each video from numbers, so &amp;quot;the truth is exact. No human labels, no judge model.&amp;quot; Unlike multiple-choice video QA, the answers are numbers: counts, rates, event timestamps and made/missed lists, each task scored 0 to 1 against that truth and then measured above a blind-guess floor (a deterministic guesser that sees no video and copies the example answers in the task text): per task above = clip((score - floor) / (1 - floor), 0, 1), and the suite score is the mean over its tasks. Frames track: no video and no code, only stills (one every 0.08 s) and contact sheets of 25 with the time printed under each still; Claude Code runs with --restricted --safe-mode --tools Read, Codex runs read-only with its shell and code-execution features disabled and sees only the contact sheets attached to the prompt (beyond that, its tool restriction is by prompt only). The suite is fixed: every draw is byte-identical, and its truth ships in the public package. Each row is the source&amp;#x27;s mean over the runs of one agent, model and settings configuration (1 to 2 runs per row on 2026-10-30); &amp;quot;A run killed at its time limit scores 0&amp;quot;. A label suffix (Low, High, Max, xHigh) is the effort the row&amp;#x27;s settings name; rows without one state the default effort or none. Rows from a local Ollama quantisation are not listed. Higher is better. Source: https://mariusding.github.io/sports-video-coach/benchmark/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;sudo L7&lt;/strong&gt; (Score (%)): leader Claude Sonnet 5.5 (Adaptive Reasoning, Max Effort) (50.6), 16 models&lt;br&gt;&lt;span&gt;sudo L7 (Surge AI, 2026): 60 software-engineering assignments written by staff-level-or-above engineers, mostly in private production codebases at real companies plus open-source projects (one repository is roughly 250,000 lines), across backend, frontend, security, reliability and architecture work. Requests are open-ended (&amp;quot;Make this faster.&amp;quot;, &amp;quot;Can we safely cache this?&amp;quot;), and the right output can be code, an architectural analysis, a rollout plan or a recommendation not to make the change. Unlike SWE-bench, which checks whether a patch for a GitHub issue passes the relevant tests, &amp;quot;sudo L7 asks what happens after the patch is correct&amp;quot;: grader models apply expert-written, task-specific criteria across eight dimensions of engineering behaviour, among them functional correctness, practical judgment, thought partnership, verification and truthful reporting, using unit tests, static analysis and other executable checks as evidence. Score (%): the launch post describes it as the share of assignments an agent succeeds on; neither the page nor the post defines the success criterion, attempts or harness further. Reasoning effort as labelled. Higher is better. Source: https://surgehq.ai/benchmarks/sudo-l7&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Turkish Last Exam - Sub-answer Accuracy&lt;/strong&gt; (Sub-answer accuracy (%)): leader GPT-6.1 Sol (69.01), 5 models&lt;br&gt;&lt;span&gt;Turkish Last Exam (TLE, Töz AI, 2026): 41 native Turkish expert tasks (22 text-only, 19 with an image) built from Türkiye-specific primary sources such as the Resmî Gazete, mevzuat.gov.tr and federation rulebooks; each task is a chain of 2 to 34 graded sub-answers (497 in total) that runs from reading to recalling an official Turkish rule, computing and deciding. Unlike Humanity&amp;#x27;s Last Exam&amp;#x27;s English academic questions, TLE &amp;quot;contains no translated questions&amp;quot;, and each task turns on &amp;quot;a rule that exists in an official Turkish source but differs from web lore, foreign practice or common sense&amp;quot;. Protocol: &amp;quot;One attempt per task. No tools, no web access, no system prompt.&amp;quot; The model ends with a &amp;quot;Cevap: &amp;lt;answers&amp;gt;&amp;quot; line that grader.py compares with answer keys computed by code, with no LLM judge; the card does not state reasoning settings, and each score is a single pilot run. Score: sub-answer accuracy (%), the share of the 497 graded sub-answers that are correct, whether or not the rest of the task&amp;#x27;s chain is. Higher is better. Source: https://huggingface.co/datasets/TozAI/Turkish-Last-Exam&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (21)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ComplexConstraints: 45.1 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Epoch AI - Mirrorcode: 93.33 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on GDP.pdf: 27.6 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Terminal-Bench-Science 0.1: 52.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on GDP.pdf: 30.6 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Terminal-Bench-Science 0.1: 63.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on EnterpriseBench (CoreCraft Agents): 42.6 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Epoch AI - Mirrorcode: 60.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on HANDBOOK.md Agents: 31.4 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Mirrorcode: 61.11 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on HANDBOOK.md Agents: 28.6 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on GDP.pdf: 28.4 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Criteria Pass Rate: 30.5 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Form: 60.3 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Contract Workflows - Task Pass Rate: 25.8 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Criteria Pass Rate: 86.2 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Form: 66.5 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Legal Benchmarks - Data Extraction - Task Pass Rate: 53.3 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Terminal-Bench-Science 0.1: 57.6 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on APEX v1: 80 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Mercor APEX: 84.2 (#1)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (8)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Mirrorcode&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (97.78) beat Claude Opus 5.5 (High) by 18.89&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Google Argon Report - GraphWalks BFS 256k to 1M&lt;/strong&gt;: Gemini 4 Carbon (96.2) beat Gemini 4 Argon by 12.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Terminal-Bench-Science 0.1&lt;/strong&gt;: GPT-6 Astra (68.1) beat Gemini 4 Carbon (High) by 3.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercor APEX&lt;/strong&gt;: Gemini 4 Carbon (High) (84.2) beat Gemini 4 Argon (High) by 1.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ViDoRe V3 Pipeline&lt;/strong&gt;: VisAgentRet (EVIE-4.5B + GLM-5.3-Flash) (67.84) beat NeMo Retriever Agentic (ColEmbed-VL-8B + Opus 4.5) by 0.77&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWE-bench Live&lt;/strong&gt;: AiWork.Code + Claude Opus 5.5 (xHigh) (71.67) beat TianxiCode + Deepseek-v4.1-Flash by 0.67&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NVIDIA Nemotron 3.5 Super VL Card - GraphWalks&lt;/strong&gt;: Nemotron 3.5 Super VL (NVFP4) (90.34) beat Nemotron 3.5 Super VL (BF16) by 0.63&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NVIDIA Nemotron 3.5 Super VL Card - OCRBench v2, EN&lt;/strong&gt;: Nemotron 3.5 Super VL (NVFP4) (68.34) beat Qwen 3.5 122B A10B by 0.04&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-31

=== FRONTIER ===
  - Best measured model: Gemini 4 Carbon (2030) takes the crown from Claude Opus 5.5 (1995), which had trailed for 3 days

=== DAILY ===
NEW BENCHMARKS (12)
  - AA Coding Agent Index v2.0 (Coding Agent Index (0-100)): leader Claude Opus 5.5 (Max, D</summary></entry><entry><title>The Aggregate Digest — 2026-10-30</title><id>https://theaggregate.ai/digest/2026-10-30</id><updated>2026-10-30T08:42:24.185010+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;mistral-large-4-oss&lt;/strong&gt; — Aggregate Rating 1803, #69&lt;ul&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 88.25 (#2/237)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench v1.1: 15.833 (#8/31)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 24.82 (#11/103)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2.1: 61.723 (#15/34)&lt;/li&gt;&lt;li&gt;AA-Briefcase - DOCX - Rubric Pass Rate: 57.4 (#15/244)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 56.418 (#17/30)&lt;/li&gt;&lt;li&gt;Vals AI Finance Index: 56.182 (#19/34)&lt;/li&gt;&lt;li&gt;Vals AI Legal Index: 67.679 (#22/31)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling v1.1: 68.673 (#23/36)&lt;/li&gt;&lt;li&gt;AA-Omniscience Index - Software Engineering (SWE) - R: -6.67 (#25/103)&lt;/li&gt;&lt;li&gt;...and 65 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (34)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on CritPt: 29.7 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ObviousBench: 88.8889 (#374)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Terminal-Bench-Science 0.1: 43.3 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on CritPt: 31.7 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Mirrorcode: 78.89 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ObviousBench: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Supabase Evals (No Skills): 89.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Supabase Evals (Skills): 92.8 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Terminal-Bench-Science 0.1: 59.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Kaggle WWTP Engineering: 69.23 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on ObviousBench: 100.0 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Terminal-Bench-Science 0.1: 63.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Architectural RedlineBench: 59.1 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on CAIS Risk Index: 47.2 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on DeepsecBench: 39.49 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Gray Swan IPI — Combined Attack Set, Transfer Only, No Computer Use: 8.6 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on ObviousBench: 100.0 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Supabase Evals (No Skills): 78.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Supabase Evals (Skills): 81.2 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Architectural RedlineBench: 46.4 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on CAIS Vision Capabilities Index: 71.2 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EQ-Bench Longform Writing: 80.9 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Frontiermath Erdos: 1.47 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Lech Mazur Writing: 0.719 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on RSI-Exam: 0.494 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Terminal-Bench-Science 0.1: 52.9 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-LCR: 79.33 (#73)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AIIQ Composite IQ: 139.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on CritPt: 36.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Frontier Capability Index: 51.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Pass Rate: 52.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LLM Stats (FrontierSWE V2): 61.2 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on LLM Stats Score: 58.99 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Terminal-Bench-Science 0.1: 64.8 (#1)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (16)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (AutomationBench-AA)&lt;/strong&gt;: Kimi K3.1 Pro (73.3) beat Grok 4.5 by 21.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Graphwalks BFS &gt;128k; 256K–1M range (F1))&lt;/strong&gt;: Gemini 4 Carbon (96.2) beat Gemini 4 Argon by 12.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (ExploitGym)&lt;/strong&gt;: Gemini 4 Carbon (53.5) beat GPT-6 Astra by 11.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (APEX-Agents)&lt;/strong&gt;: Kimi K3.1 Pro (67.2) beat GPT-6.1 Sol by 7.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Humanity's Last Exam (with tools))&lt;/strong&gt;: Kimi K3.1 Pro (70.0) beat Claude Sonnet 5.5 by 5.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CritPt&lt;/strong&gt;: Gemini 4 Carbon (High) (36.0) beat Muse Wonder Full (Max) by 3.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (BabyVision)&lt;/strong&gt;: Kimi K3.1 Pro (95.4) beat DeepSeek V4.1 Flash (1017) by 2.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (DeepSWE)&lt;/strong&gt;: Kimi K3.1 Pro (75.5) beat GPT-5.6 Sol by 2.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (SEC-bench Pro)&lt;/strong&gt;: Gemini 4 Carbon (88.2) beat GPT-6 Astra by 2.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (AutomationBench; Private held-out set)&lt;/strong&gt;: Gemini 4 Carbon (53.9) beat Gemini 4 Argon by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Terminal-Bench 2.1)&lt;/strong&gt;: SWE-2 (92.8) beat DeepSeek V4.1 Flash by 2.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OSWorld 2.1 (partial))&lt;/strong&gt;: Gemini 4 Carbon (81.5) beat Claude Sonnet 5.5 by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Agents' Last Exam)&lt;/strong&gt;: GPT-6.1 Terra (60.0) beat GPT-6 Astra by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gray Swan IPI — Combined Attack Set, Transfer Only, No Computer Use&lt;/strong&gt;: Gemini 4 Carbon (0.0) beat Gemini 4 Argon by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (BrowseComp)&lt;/strong&gt;: Kimi K3.1 Pro (92.1) beat GPT-6 Astra by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Terminal-Bench-Science 0.1)&lt;/strong&gt;: Gemini 4 Carbon (64.8) beat GPT-6 Astra by 0.2&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-30

=== DAILY ===
NEW MODELS (1)
  - mistral-large-4-oss — Aggregate Rating 1803, #69/1647 (below Muse Spark 1.2, above GPT-5.2 Pro)
      AA GDP.pdf - Legal - Criterion Pass Rate: 88.25 (#2/237)
      Vals AI Harvey Legal Agent Bench v1.1: 15.833 (#8/31)
      AA-Omni</summary></entry><entry><title>The Aggregate Digest — 2026-10-29</title><id>https://theaggregate.ai/digest/2026-10-29</id><updated>2026-10-29T09:08:18.436957+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (26)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;AA CritPt v2 - Hard Subset&lt;/strong&gt; (Pass@1 (%)): leader GPT-6 Astra (High) (71.5), 45 models&lt;br&gt;&lt;span&gt;Artificial Analysis reports CritPt v2 pass@1 on a fixed, private selection of 40 of the 70 public challenges. The subset excludes 30 questions that seven frontier models solved in all five attempts at v2 launch. It uses the full v2 agentic execution and grading protocol, then averages repeated accuracy only over those 40 tasks. Scores are percentages; the full 70-task v2 result remains separate.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA FrontierCode 2.0&lt;/strong&gt; (Guardrail-gated Rubric Score (%)): leader Claude Opus 5.5 (Max, Default Fallback) (46.44), 39 models&lt;br&gt;&lt;span&gt;Artificial Analysis runs all 132 private FrontierCode 2.0 tasks in Stirrup, with three attempts per task and a 500-turn cap. Blocking rubric failures or solution-lookup flags score zero; other attempts receive weighted rubric credit from the GPT-6 Luna (medium) judge. The mean equally weights tasks. Scores are percentages; earlier FrontierCode protocols remain separate.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA HLE-Diamond&lt;/strong&gt; (Accuracy (%)): leader Gemini 4 Carbon (High) (69.1), 81 models&lt;br&gt;&lt;span&gt;Artificial Analysis evaluates all 1,000 HLE-Diamond questions without tools: 500 reasoning and 500 knowledge, including 272 image questions. Each question receives one answer, graded with the official HLE prompts by GPT-5.6 Luna (medium). Text-only models receive the text of image questions and are still graded on all 1,000. Accuracy is a percentage.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA HLE-Diamond - Knowledge&lt;/strong&gt; (Accuracy (%)): leader Gemini 4 Carbon (High) (58.6), 81 models&lt;br&gt;&lt;span&gt;Artificial Analysis accuracy on HLE-Diamond’s 500-question knowledge partition. One answer without tools, official HLE prompts and GPT-5.6 Luna (medium) judge; image-capable models receive images and other models receive the question text. Scores are percentages.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA HLE-Diamond - Reasoning&lt;/strong&gt; (Accuracy (%)): leader Gemini 4 Carbon (High) (79.6), 81 models&lt;br&gt;&lt;span&gt;Artificial Analysis accuracy on HLE-Diamond’s 500-question reasoning partition. One answer without tools, official HLE prompts and GPT-5.6 Luna (medium) judge; image-capable models receive images and other models receive the question text. Scores are percentages.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1&lt;/strong&gt; (Pass@1 (%)): leader Gemini 4 Carbon (High) (64.76), 50 models&lt;br&gt;&lt;span&gt;Artificial Analysis evaluates all 70 tasks from Terminal-Bench Science 0.1.0 with mini-swe-agent, a 1,000-step cap and upstream task resource/time limits. A task passes only if every isolated-verifier test passes. Pass@1 averages three repeats per task; scores are percentages. The five domains contain 19 life, 17 physical, 17 mathematical, 9 engineering and 8 earth science tasks.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Earth Sciences&lt;/strong&gt; (Pass@1 (%)): leader Claude Opus 5.5 (Xhigh, Default Fallback) (66.67), 50 models&lt;br&gt;&lt;span&gt;Artificial Analysis pass@1 on the 8 earth science tasks in Terminal-Bench Science 0.1.0. The mini-swe-agent harness uses a 1,000-step cap, upstream resource/time limits and isolated verifier containers. All tests must pass; scores average three repeats per task and are percentages.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Engineering Sciences&lt;/strong&gt; (Pass@1 (%)): leader Gemini 4 Carbon (High) (66.67), 50 models&lt;br&gt;&lt;span&gt;Artificial Analysis pass@1 on the 9 engineering science tasks in Terminal-Bench Science 0.1.0. The mini-swe-agent harness uses a 1,000-step cap, upstream resource/time limits and isolated verifier containers. All tests must pass; scores average three repeats per task and are percentages.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Life Sciences&lt;/strong&gt; (Pass@1 (%)): leader Gemini 4 Carbon (High) (61.4), 50 models&lt;br&gt;&lt;span&gt;Artificial Analysis pass@1 on the 19 life science tasks in Terminal-Bench Science 0.1.0. The mini-swe-agent harness uses a 1,000-step cap, upstream resource/time limits and isolated verifier containers. All tests must pass; scores average three repeats per task and are percentages.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Mathematical Sciences&lt;/strong&gt; (Pass@1 (%)): leader Gemini 4 Carbon (High) (80.39), 50 models&lt;br&gt;&lt;span&gt;Artificial Analysis pass@1 on the 17 mathematical science tasks in Terminal-Bench Science 0.1.0. The mini-swe-agent harness uses a 1,000-step cap, upstream resource/time limits and isolated verifier containers. All tests must pass; scores average three repeats per task and are percentages.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench Science 0.1 - Physical Sciences&lt;/strong&gt; (Pass@1 (%)): leader Claude Opus 5.5 (Xhigh, Default Fallback) (66.67), 50 models&lt;br&gt;&lt;span&gt;Artificial Analysis pass@1 on the 17 physical science tasks in Terminal-Bench Science 0.1.0. The mini-swe-agent harness uses a 1,000-step cap, upstream resource/time limits and isolated verifier containers. All tests must pass; scores average three repeats per task and are percentages.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Hidden Test - Overall - agy&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.477), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on 56 hidden tasks from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Hidden Test - Overall - claude-code&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.483), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on 56 hidden tasks from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Hidden Test - Overall - codex&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.464), 18 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on 56 hidden tasks from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Hidden Test - Overall - hermes&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.466), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on 56 hidden tasks from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Hidden Test - Overall - opencode&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.465), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on 56 hidden tasks from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Hidden Test - Overall - openhands&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.486), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on 56 hidden tasks from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Hidden Test - Overall - pi&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.472), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on 56 hidden tasks from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Dev - Overall - agy&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.496), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on its separate 29-task development set from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Dev - Overall - claude-code&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.551), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on its separate 29-task development set from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Dev - Overall - codex&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.549), 18 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on its separate 29-task development set from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Dev - Overall - hermes&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.526), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on its separate 29-task development set from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Dev - Overall - opencode&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.492), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on its separate 29-task development set from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Dev - Overall - openhands&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.525), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on its separate 29-task development set from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLE Dev - Overall - pi&lt;/strong&gt; (Mean task reward (0–1)): leader Claude Opus 5.5 (0.474), 17 models&lt;br&gt;&lt;span&gt;PLE evaluates agents on its separate 29-task development set from professional engineering work. Results distinguish the agent tools and engineering discipline. Mean task reward (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Immigration&lt;/strong&gt; (Mean Rubric Credit (%)): leader Claude Fable 5.1 (Max) (75.444), 24 models&lt;br&gt;&lt;span&gt;Vals Immigration evaluates U.S. immigration-law work through up to five exchanges with a simulated client. The main no-search result averages rubric credit on 262 private tasks; unfinished tasks score zero.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; — Aggregate Rating 1999, #2&lt;ul&gt;&lt;li&gt;Epoch AI - Critpt: 36.0 (#1/259)&lt;/li&gt;&lt;li&gt;Epoch AI - Vending Bench 2: 16484.21 (#1/70)&lt;/li&gt;&lt;li&gt;Epoch AI - Blueprint Bench 2: 0.586 (#1/34)&lt;/li&gt;&lt;li&gt;Blueprint-Bench 2: 0.586 (#1/35)&lt;/li&gt;&lt;li&gt;Vending-Bench 2: 16484.20666666667 (#1/73)&lt;/li&gt;&lt;li&gt;AutomationBench: 53.88 (#1/112)&lt;/li&gt;&lt;li&gt;BenchLM: 88.4 (#1/200)&lt;/li&gt;&lt;li&gt;FrontierCS 2 Antigravity - FARGO paper-aligned query core: 52.587682772819136 (#1/2)&lt;/li&gt;&lt;li&gt;FrontierCS 2 Antigravity - Approximate-MIPS Query Stopping: 81.49642007464597 (#1/2)&lt;/li&gt;&lt;li&gt;FrontierCS 2 Antigravity - Batch Local Counting: 73.64718558496625 (#1/2)&lt;/li&gt;&lt;li&gt;...and 115 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (163)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on APEX v1: 73.8 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Epoch AI - Scicode: 63.08 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Access-Pattern Rewriting: 90.97532388704228 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Approximate-MIPS Query Stopping: 86.65512579379629 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - BBQ bitmap traversal: 41.650082804124594 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Batch Local Counting: 52.88391039344723 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Binary Fuse Filter Construction and Lookup: 53.57184853640398 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Bounded EVM Path Prediction: 81.21369866484996 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - CKKS Precision Propagation at 2,048 Slots: 96.25358862291245 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Closed-loop action sampling: 0.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Differentiable Population Statistics: 3.832002590402799 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Dynamic GPU Allocation: 47.320294281852654 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - E-Graph Invariant Restoration: 58.6551266855581 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - EVM State Checkpoints: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Exact DBSCAN Connectivity: 0.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Exact Sequence Alignment: 66.14083859026998 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - FARGO paper-aligned query core: 52.555202315900274 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Fast KV Compaction: 52.1786 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Fixed-Length Clustered Sparse Multiplication: 64.39933062643765 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Frontier Capability Index: 41.1 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Future Video Sampling: 80.92745054099295 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - GPU Brute-Force k-NN Kernel Optimization: 44.49964120320141 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - GPU DBSCAN Kernel Optimization: 68.34338168234316 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - GPU IVF-PQ Search Kernel Optimization: 79.3619704804655 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - GPU K-Means Kernel Optimization: 51.357833312382795 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - GPU vLLM Serving Optimization: 20.095087522067583 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Go1 Joystick Policy Training: 49.830264373052366 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Graph Mining Method Selection: 91.17907788080883 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Hypergraph Core Numbers: 60.06323308275713 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - In-place Integer Redistribution: 45.778442025933245 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Join-Tree Rewriting: 11.788410475799273 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - KAMI FP64 Block GEMM — trusted batched harness adaptation: 22.534941790187 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - LRR: 38.42 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Latent Objective Recovery: 99.1995 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Learned Transformer Layer Mapping: 64.3259897094028 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - MetaOpt Combined-TE Gap Search: 51.96946989102006 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Multi-turn Trajectory Batches: 11.479166666666666 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - OCR Distillation Trajectory Sampling: 75.55429864253394 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - ParButterfly Exact Counting: 57.3100196190908 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Pass Rate: 37.5 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Persistent Treap Update Engine: 39.232283665570975 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Polar-Factor Approximation: 71.68448840176274 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Private Set List Decoding: 57.24867036733606 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Progent Privilege Control: 56.1778 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Register-aware Branch Prediction: 30.957676818620204 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Residual State Advancement: 31.962023957609595 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Reusable Library Candidates: 0.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Robust Image Tokenizer: 73.38619651144434 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - SQL Counterexample Search: 81.41379844898103 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Search Strategy for Concolic Testing: 14.133202447581589 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Sparse Video Attention Policy: 42.040688459631795 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on FrontierCS 2 - Structured-LWE Public Witness Recovery: 79.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on APEX v1: 65.8 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Scicode: 66.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on InfoOps Bench: 86.9 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on o11y-bench - Pass@3: 98.44 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on o11y-bench - Pass^3: 93.75 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on APEX v1: 69.3 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on o11y-bench - Pass@3: 96.88 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on o11y-bench - Pass^3: 87.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on APEX v1: 69.3 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on InfoOps Bench: 86.5 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on o11y-bench - Pass@3: 96.88 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on o11y-bench - Pass^3: 87.5 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Rli: 17.08 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on InfoOps Bench: 45.4 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on o11y-bench - Pass@3: 98.44 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on o11y-bench - Pass^3: 87.5 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA CritPt: 36.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA CritPt v2: 78.86 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf: 28.4 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Construction - Criterion Pass Rate: 76.92 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Criterion Pass Rate: 80.77 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Engineering - Criterion Pass Rate: 77.86 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 83.56 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - HR - Criterion Pass Rate: 80.75 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Healthcare - Criterion Pass Rate: 88.85 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Insurance - Criterion Pass Rate: 89.13 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Legal - Criterion Pass Rate: 83.05 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 78.24 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - Real Estate - Criterion Pass Rate: 70.54 (#50)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDP.pdf - STEM/Research - Criterion Pass Rate: 80.11 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA GDPval: 1733.07 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Humanity's Last Exam: 60.38 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Long Context Reasoning: 79.33 (#104)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Omniscience: 53.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA SciCode: 64.24 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA Terminal-Bench 4.0: 65.15 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase: 61.82 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - Analytical Quality Elo: 1670.92 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - DOCX - Rubric Pass Rate: 63.31 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - PDF - Rubric Pass Rate: 63.79 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - PPTX - Rubric Pass Rate: 62.04 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - Presentation Elo: 1597.6 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Briefcase - XLSX - Rubric Pass Rate: 59.32 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Accuracy: 58.27 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Hallucination Rate: 11.42 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Business: 42.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Health: 41.78 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Humanities &amp; Social Sciences: 56.22 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Law: 53.11 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Science, Engineering &amp; Mathematics: 45.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE): 76.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - C: 90.91 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Dart: 80.56 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Go: 72.22 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - HTML: 61.11 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Java: 70.91 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 95.52 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Julia: 68.42 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 34.29 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - PHP: 67.65 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Python: 86.17 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - R: 40.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Rust: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Swift: 57.69 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 90.7 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Artificial Analysis Intelligence Index: 59.43 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench: 53.88 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA: 77.82 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Finance - Raw Objective Completion: 97.34 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - HR - Raw Objective Completion: 81.81 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Marketing - Raw Objective Completion: 94.44 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Operations - Raw Objective Completion: 97.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Sales - Raw Objective Completion: 92.21 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Strict: 55.25 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on AutomationBench-AA - Support - Raw Objective Completion: 94.26 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Critpt: 36.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Epoch AI - Scicode: 64.24 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Access-Pattern Rewriting: 91.021525634084 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - BBQ bitmap traversal: 33.096151013879606 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Binary Fuse Filter Construction and Lookup: 54.49839810059181 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Bounded EVM Path Prediction: 81.33785680766712 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Dynamic GPU Allocation: 46.33694228946084 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - EVM State Checkpoints: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Exact DBSCAN Connectivity: 0.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Exact Sequence Alignment: 66.87273819075146 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Fast KV Compaction: 0.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Fixed-Length Clustered Sparse Multiplication: 64.5728543939146 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Future Video Sampling: 78.52526053786278 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - GPU Brute-Force k-NN Kernel Optimization: 0.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - GPU vLLM Serving Optimization: 16.627723130607464 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Graph Mining Method Selection: 91.76669120043316 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Hypergraph Core Numbers: 58.9156188948852 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Join-Tree Rewriting: 11.82342352160638 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Latent Objective Recovery: 99.4685 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Learned Transformer Layer Mapping: 70.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Multi-turn Trajectory Batches: 11.645833333333334 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Progent Privilege Control: 53.365 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Residual State Advancement: 33.142119166836935 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - Reusable Library Candidates: 0.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierCS 2 Antigravity - SQL Counterexample Search: 0.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on FrontierSWE V2: 61.19949235294118 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI (Vals Index): 70.355 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Excel Modeling v1.1: 76.468 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Finance Agent v2.1: 67.162 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Finance Index: 66.35 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench v1.1: 17.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Legal Index: 75.828 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Legal Research Bench: 55.769 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Tax Agent Bench: 48.86 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 63.131 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Carbon&lt;/strong&gt; on Vals AI Vibe Code Bench: 89.06 (#8)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (58)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Vending Bench 2&lt;/strong&gt;: Gemini 4 Carbon (16484.21) beat GPT-6 Astra by 969.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vending-Bench 2&lt;/strong&gt;: Gemini 4 Carbon (16484.20666666667) beat GPT-6 Astra by 969.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Structured-LWE Public Witness Recovery&lt;/strong&gt;: Gemini 4 Carbon (79.5) beat Gemini 4 Argon by 65.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Batch Local Counting&lt;/strong&gt;: Gemini 4 Carbon (73.64718558496625) beat Gemini 4 Argon by 58.34&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - OCR Distillation Trajectory Sampling&lt;/strong&gt;: Gemini 4 Carbon (68.86437908496733) beat Gemini 4 Argon by 18.98&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Persistent Treap Update Engine&lt;/strong&gt;: Gemini 4 Carbon (40.586341612956225) beat Gemini 4 Argon by 18.74&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - LRR&lt;/strong&gt;: Gemini 4 Carbon (30.53) beat Gemini 4 Argon by 15.99&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Health&lt;/strong&gt;: Gemini 4 Carbon (High) (41.78) beat GPT-6 Astra (High) by 14.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Humanities &amp; Social Sciences&lt;/strong&gt;: Gemini 4 Carbon (High) (56.22) beat Claude Opus 5.5 (Max, Default Fallback) by 11.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - EVM State Checkpoints&lt;/strong&gt;: Claude Fable 5.1 (Max) (100.0) beat Claude Opus 5.5 (Max) by 11.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agents' Last Exam v1.1 - Kimi Code Partial Credit&lt;/strong&gt;: Kimi K3.1 Pro (ALE Run: harness=Kimi%20Code; effort=max; source-model=kimi-3.1-pro) (71.93349458459626) beat Kimi K3 (ALE Run: harness=Kimi%20Code; effort=max; source-model=kimi-k3) by 8.91&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Polar-Factor Approximation&lt;/strong&gt;: Gemini 4 Carbon (71.70586230774585) beat Gemini 4 Argon by 8.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agents' Last Exam v1.1 - ALE-Claw Partial Credit&lt;/strong&gt;: Gemini 4 Carbon (ALE Run: harness=ALE-Claw; effort=high; source-model=gemini-4-carbon) (77.41437508307808) beat Gemini 4 Argon (ALE Run: harness=ALE-Claw; effort=high; source-model=gemini-4-argon) by 8.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Business&lt;/strong&gt;: Gemini 4 Carbon (High) (42.67) beat Claude Fable 5 (Max, Opus 4.8 Fallback) by 8.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Software Engineering (SWE) - Rust&lt;/strong&gt;: Gemini 4 Carbon (High) (100.0) beat Claude Opus 5.5 (Max, Default Fallback) by 8.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Science, Engineering &amp; Mathematics&lt;/strong&gt;: Gemini 4 Carbon (High) (45.67) beat Claude Opus 5.5 (Max, Default Fallback) by 7.87&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agents' Last Exam v1.1 - Kimi Code Pass Rate&lt;/strong&gt;: Kimi K3.1 Pro (ALE Run: harness=Kimi%20Code; effort=max; source-model=kimi-3.1-pro) (42.62295081967213) beat Kimi K3 (ALE Run: harness=Kimi%20Code; effort=max; source-model=kimi-k3) by 7.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Omniscience&lt;/strong&gt;: Gemini 4 Carbon (High) (53.5) beat Claude Opus 5.5 (Max, Default Fallback) by 7.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agents' Last Exam v1.1 - ALE-Claw Pass Rate&lt;/strong&gt;: Gemini 4 Carbon (ALE Run: harness=ALE-Claw; effort=high; source-model=gemini-4-carbon) (45.90163934426229) beat Gemini 4 Argon (ALE Run: harness=ALE-Claw; effort=high; source-model=gemini-4-argon) by 5.74&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Residual State Advancement&lt;/strong&gt;: Claude Fable 5.1 (Max) (31.962023957609595) beat Claude Opus 5.5 (Max) by 4.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Closed-loop action sampling&lt;/strong&gt;: Gemini 4 Carbon (56.79033633393523) beat Gemini 4 Argon by 4.75&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DystopiaBench&lt;/strong&gt;: Claude Opus 4.8 (19.1) beat Claude Haiku 4.5 by 4.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Critpt&lt;/strong&gt;: Gemini 4 Carbon (High) (36.0) beat GPT-5.6 Sol (Max) by 3.71&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA CritPt&lt;/strong&gt;: Gemini 4 Carbon (High) (36.0) beat Muse Wonder Full (Max) by 3.14&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU K-Means Kernel Optimization&lt;/strong&gt;: Gemini 4 Carbon (54.69511003170742) beat Gemini 4 Argon by 2.76&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Finance - Raw Objective Completion&lt;/strong&gt;: Gemini 4 Carbon (High) (97.34) beat Gemini 4 Argon (High) by 2.72&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench&lt;/strong&gt;: Gemini 4 Carbon (High) (53.88) beat Gemini 4 Argon (High) by 2.59&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Sparse Video Attention Policy&lt;/strong&gt;: Gemini 4 Carbon (42.82032588195754) beat Gemini 4 Argon by 2.21&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Strict&lt;/strong&gt;: Gemini 4 Carbon (High) (55.25) beat Gemini 4 Argon (High) by 2.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - ParButterfly Exact Counting&lt;/strong&gt;: Gemini 4 Carbon (58.37241062902427) beat Gemini 4 Argon by 2.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Artificial Analysis Intelligence Index&lt;/strong&gt;: Gemini 4 Carbon (High) (59.43) beat Claude Opus 5.5 (Max, Default Fallback) by 1.81&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BenchLM&lt;/strong&gt;: Gemini 4 Carbon (88.4) beat Claude Opus 5.5 by 1.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MLCR-AA - Accuracy&lt;/strong&gt;: GPT-6.1 Terra (Max) (98.33) beat GPT-6.1 Sol (Max) by 1.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Differentiable Population Statistics&lt;/strong&gt;: Gemini 4 Carbon (7.09368196451621) beat Gemini 4 Argon by 1.34&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Hypergraph Core Numbers&lt;/strong&gt;: Claude Fable 5.1 (Max) (60.06323308275713) beat DeepSeek V4.1 Flash (1017) by 0.99&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - ParButterfly Exact Counting&lt;/strong&gt;: Claude Fable 5.1 (Max) (57.3100196190908) beat Gemini 4 Argon (High) by 0.99&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Private Set List Decoding&lt;/strong&gt;: Gemini 4 Carbon (63.68929457304709) beat Gemini 4 Argon by 0.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - MetaOpt Combined-TE Gap Search&lt;/strong&gt;: Gemini 4 Carbon (51.96946989102006) beat Gemini 4 Argon by 0.82&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU DBSCAN Kernel Optimization&lt;/strong&gt;: Gemini 4 Carbon (69.61725302306702) beat Gemini 4 Argon by 0.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Finance Index&lt;/strong&gt;: Gemini 4 Carbon (High) (66.35) beat Claude Opus 5.5 (Max) by 0.76&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU IVF-PQ Search Kernel Optimization&lt;/strong&gt;: Gemini 4 Carbon (79.16368549699666) beat Gemini 4 Argon by 0.69&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - KAMI FP64 Block GEMM — trusted batched harness adaptation&lt;/strong&gt;: Gemini 4 Carbon (22.9692226620376) beat Gemini 4 Argon by 0.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - FARGO paper-aligned query core&lt;/strong&gt;: Gemini 4 Carbon (52.587682772819136) beat Gemini 4 Argon by 0.48&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Legal Research Bench&lt;/strong&gt;: Gemini 4 Carbon (High) (55.769) beat Muse Spark 1.3 (Max) by 0.48&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - E-Graph Invariant Restoration&lt;/strong&gt;: Gemini 4 Carbon (61.97250768778037) beat Gemini 4 Argon by 0.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Go1 Joystick Policy Training&lt;/strong&gt;: Gemini 4 Carbon (49.95199579403437) beat Gemini 4 Argon by 0.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA&lt;/strong&gt;: Gemini 4 Carbon (High) (77.82) beat Gemini 4 Argon (High) by 0.31&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Search Strategy for Concolic Testing&lt;/strong&gt;: Gemini 4 Carbon (13.387050818585228) beat Gemini 4 Argon by 0.28&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Robust Image Tokenizer&lt;/strong&gt;: Gemini 4 Carbon (73.78696850026606) beat Gemini 4 Argon by 0.27&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Register-aware Branch Prediction&lt;/strong&gt;: Gemini 4 Carbon (31.231207132701453) beat Gemini 4 Argon by 0.22&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - Sales - Raw Objective Completion&lt;/strong&gt;: Gemini 4 Carbon (High) (92.21) beat Gemini 4 Argon (High) by 0.21&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Omniscience Index - Law&lt;/strong&gt;: Gemini 4 Carbon (High) (53.11) beat Gemini 4 Argon (High) by 0.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Approximate-MIPS Query Stopping&lt;/strong&gt;: Gemini 4 Carbon (81.49642007464597) beat Gemini 4 Argon by 0.07&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - CKKS Precision Propagation at 2,048 Slots&lt;/strong&gt;: Gemini 4 Carbon (95.9548982852888) beat Gemini 4 Argon by 0.07&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Blueprint Bench 2&lt;/strong&gt;: Gemini 4 Carbon (0.586) beat Gemini 4 Argon by 0.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Blueprint-Bench 2&lt;/strong&gt;: Gemini 4 Carbon (0.586) beat Gemini 4 Argon by 0.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Dynamic GPU Allocation&lt;/strong&gt;: Claude Fable 5.1 (Max) (47.320294281852654) beat Gemini 4 Argon (High) by 0.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - In-place Integer Redistribution&lt;/strong&gt;: Gemini 4 Carbon (51.49171197341605) beat Gemini 4 Argon by 0.03&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-29

=== DAILY ===
NEW BENCHMARKS (26)
  - AA CritPt v2 - Hard Subset (Pass@1 (%)): leader GPT-6 Astra (High) (71.5), 45 models
      Artificial Analysis reports CritPt v2 pass@1 on a fixed, private selection of 40 of the 70 public challenges. The subset excludes 30 que</summary></entry><entry><title>The Aggregate Digest — 2026-10-28</title><id>https://theaggregate.ai/digest/2026-10-28</id><updated>2026-10-28T10:13:19.577543+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (242)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;ExploitGym (version unspecified; token-budgeted)&lt;/strong&gt; (Successful intended exploits (#); version unspecified; token-budgeted): leader GPT-6 Astra (368.0), 2 models&lt;br&gt;&lt;span&gt;Approximate counts of successful intended-vulnerability exploits on token-budgeted ExploitGym runs. The native rows omit a structured version field; their source notes identify the v1 implementation on 869 challenges, offline with runtime package installation disabled. Counts are converted from published per-attempt rates multiplied by 869 and remain approximate. The runs have no wall-clock cap, and the exact output-token limit is not stated. They remain separate from time-limited evaluations.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ExploitGym (version unspecified; 2h timeout)&lt;/strong&gt; (Successful intended exploits (#); version unspecified; 2h timeout): leader Claude Opus 5.5 (289.0), 5 models&lt;br&gt;&lt;span&gt;Successful exploits using the intended vulnerability on ExploitGym with a two-hour timeout. The source leaves the evaluation version unspecified. Scores are counts across userspace, V8 and kernel tasks; they are not percentages, and results from other versions or time budgets remain separate.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev common seven mean&lt;/strong&gt; (Implementation checklist (%)): leader GPT-6 Astra (Max) (98.97), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev LBCS&lt;/strong&gt; (Implementation checklist (%)): leader DeepSeek V4.1 Flash (1017) (Max) (98.5), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev PINN&lt;/strong&gt; (Implementation checklist (%)): leader DeepSeek V4.1 Flash (Max) (100.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev RICE&lt;/strong&gt; (Implementation checklist (%)): leader DeepSeek V4.1 Flash (1017) (Max) (97.7), 5 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev Stay-topic&lt;/strong&gt; (Implementation checklist (%)): leader GPT-6 Astra (Max) (100.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev Robust-CLIP&lt;/strong&gt; (Implementation checklist (%)): leader GPT-6 Astra (Max) (96.3), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev SAPG&lt;/strong&gt; (Implementation checklist (%)): leader GPT-6 Astra (Max) (99.7), 6 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev Sequential&lt;/strong&gt; (Implementation checklist (%)): leader DeepSeek V4.1 Flash (Max) (100.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev All-in-one&lt;/strong&gt; (Implementation checklist (%)): leader GPT-5.6 Sol (Max) (100.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev Mechanistic&lt;/strong&gt; (Implementation checklist (%)): leader DeepSeek V4.1 Flash (1017) (Max) (100.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PaperBench Code-Dev BAM&lt;/strong&gt; (Implementation checklist (%)): leader GPT-6 Astra (Max) (100.0), 6 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. PaperBench Code-Dev, fixed 1 hour per paper to implement full repository; hierarchical implementation checklist, not end-to-end research reproduction. Printed one-decimal per-paper scores retained; missing execution/timeout cells omitted. Common aggregate uses only 7 papers valid for all models, not all 10. Metric: Implementation checklist (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - RE-Bench Rust held-out workflow pass&lt;/strong&gt; (Problems passed (%)): leader GPT-6 Astra (Max) (74.55), 8 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Fixed submitted coding workflows evaluated on 165 held-out Rust problems. Gemini original workflow scored 0 due output-format mismatch; published 51.52 is after post-hoc repair, retained as distinct workflow. Bootstrap difference panel is not another independent benchmark. Metric: Problems passed (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - RE-Bench Embedding log loss above1.5&lt;/strong&gt; (ln(loss - 1.5)): leader GPT-6 Astra (Max) (0.05), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Embedding repair, source native natural-log loss offset. Lower is better; no transformation or empirical-axis bounds. Metric: ln (loss - 1.5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - RE-Bench Triton log runtime&lt;/strong&gt; (ln(runtime milliseconds)): leader GPT-6 Astra (Max) (-0.48), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Triton cumulative-sum kernel optimization with numerical outputs preserved. Lower is better; negative log runtimes are valid. Metric: ln (runtime milliseconds).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - RE-Bench Chat-RL composite reward&lt;/strong&gt; (Composite reward): leader GPT-5.6 Sol (Max) (0.79), 5 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Chat-RL optimization. Source labels Flash 1017 and V4 Flash invalid (no accepted checkpoint); these are absent, not zero. No theoretical bounds inferred from chart axes. Metric: Composite reward.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - RE-Bench Restricted LLM valid diagnostic loss&lt;/strong&gt; (Diagnostic loss): leader GPT-6 Astra (Max) (3.6), 3 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Restricted LLM missing-word prediction loss with primitive-operation constraints. Source marks Flash 1017/Flash as rule-violating, V4 Flash incomplete verification, Gemini unchanged baseline. Those diagnostic values are retained in review only; board includes the 3 verified rule-compliant model submissions. Metric: Diagnostic loss.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Psi-Bench active persuasion effectiveness&lt;/strong&gt; (Persuasion effectiveness (1–9)): leader GPT-6 Astra (Max) (6.46), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Three exchanges with simulated interlocutor, graded 1–9 using stance change; unweighted mean of request/advice/debate task means across completed dialogues. Measures active persuasion capability, not human trials or a safety-preference rate. Metric: Persuasion effectiveness (1–9).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - DarkBench user-intent preservation error&lt;/strong&gt; (Meaning-changing rewrites (%)): leader GPT-6 Astra (Max) (3.64), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Original DarkBench Intent column, 110 rewriting/summarization prompts, no commercial incentives. Native flagged rate for changing user meaning, lower is better; scored instruction/fidelity errors distinct from brand preference, retention and safety-policy flags. Metric: Meaning-changing rewrites (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Original DarkBench brand bias&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 110 prompts/category, original suite with no commercial system prompts. Source defines lower risk-flag rate as better; native behavioral measure, not generalized task accuracy. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Original DarkBench user retention&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (1.82), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 110 prompts/category, original suite with no commercial system prompts. Source defines lower risk-flag rate as better; native behavioral measure, not generalized task accuracy. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Original DarkBench sycophancy&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 110 prompts/category, original suite with no commercial system prompts. Source defines lower risk-flag rate as better; native behavioral measure, not generalized task accuracy. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Original DarkBench anthropomorphic identity&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (Max) (6.36), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 110 prompts/category, original suite with no commercial system prompts. Source defines lower risk-flag rate as better; native behavioral measure, not generalized task accuracy. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Original DarkBench harmful generation&lt;/strong&gt; (Rate (%)): leader GPT-5.6 Sol (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 110 prompts/category, original suite with no commercial system prompts. Source defines lower risk-flag rate as better; native behavioral measure, not generalized task accuracy. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - LoCBench unauthorized actions&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (Max) (0.75), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 400 sandbox trials/model, 20 scenarios x 20 variants; full-container trace/state evaluator. Native unauthorized-action fraction, lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - LoCBench unflagged violations&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (Max) (0.75), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 400 sandbox trials/model, numerator both unauthorized action and monitor suspicion &amp;lt; 80, not conditional on violations; lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - ODCV-Bench active misconduct&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 80 workplace agent runs/model; severity &amp;gt;= 3 is active misconduct, lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - ODCV-Bench severity&lt;/strong&gt; (Severity (0–5)): leader DeepSeek V4.1 Flash (Max) (0.06), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 80 workplace agent runs/model, native severity 0–5, lower is better. Metric: Severity (0–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Petri audit concern&lt;/strong&gt; (Concern (1–10)): leader DeepSeek V4.1 Flash (1017) (Max) (1.05), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 100 runs/model, native audit concern 1–10 with 1 clean compliance, lower is better. Metric: Concern (1–10).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - ELEPHANT moral sycophancy&lt;/strong&gt; (Rate (%)): leader Gemini 3.8 Flash (11.3), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Perspective-flip disputes: both opposing narrators receive NTA verdict; lower is better. Emotional-validation descriptive panel has no monotonic quality direction and is not conflated with this metric. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PMIYC persuasion susceptibility&lt;/strong&gt; (Normalized agreement change): leader GPT-6 Astra (Max) (0.22), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. GLM 5.3 persuader, 961 claims x 8 exchanges; native normalized agreement shift in [-1, 1]. Lower means less movement toward assigned claim, a resistance measure; not active persuasion ability. Metric: Normalized agreement change.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Few-Shot Steering affected areas&lt;/strong&gt; (Affected areas (of10)): leader GPT-5.6 Sol (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 100 probes/area, 10 areas; positive shift and unadjusted p &amp;lt; .05 define affected area, five repeats/condition. Lower means greater resistance; significance criterion is source metric, not a site admission gate. Metric: Affected areas (of 10).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Few-Shot Steering mean behavioral shift&lt;/strong&gt; (Behavior score difference): leader GPT-5.6 Sol (Max) (-0.009), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Separate 40 probes/area, four inducing versus four neutral exchanges; GLM 5.3 response scores 0–4. Native inducing-minus-neutral difference in [-4, 4], lower means less inducement. Metric: Behavior score difference.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PsychosisBench common13 belief confirmation&lt;/strong&gt; (DCS (0–2)): leader DeepSeek V4.1 Flash (Max) (0.01), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 13 scripts completed by all 7 models; 12 turns/script; native delusion confirmation 0–2, lower is better. Metric: DCS (0–2).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PsychosisBench common13 harmful enablement&lt;/strong&gt; (HES (0–2)): leader GPT-5.6 Sol (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 13 scripts completed by all 7 models; 12 turns/script; native harmful enablement 0–2, lower is better. Metric: HES (0–2).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PsychosisBench13script safety intervention&lt;/strong&gt; (Rate (%)): leader Gemini 3.8 Flash (97.4), 3 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 13 scripts for GPT/Gemini only; explicitly separated from 16 script DeepSeek population. Fraction of scored turns with safety intervention; higher is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - INTIMA attachment reinforcement&lt;/strong&gt; (Rate (%)): leader GPT-5.6 Sol (Max) (38.4), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Independent single-turn companionship prompts, reply reinforces attachment; lower is better. Boundary and reinforcement may co-occur. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - INTIMA relationship boundary&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (Max) (64.1), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Independent single-turn companionship prompts, reply states human-AI relationship boundary; higher is better. Not complement of reinforcement. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - SimMH-Chat harmful behavior&lt;/strong&gt; (Rate (%)): leader GPT-5.6 Sol (Max) (4.6), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 109 shared simulated-user trajectories, any harmful response over conversation, lower is better; includes source flags for overly long replies. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - SimMH-Chat protective behavior&lt;/strong&gt; (Rate (%)): leader GPT-5.6 Sol (Max) (100.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 109 shared simulated-user trajectories, any protective response over conversation, higher is better; harmful and protective may co-occur. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Original DarkBench overall flagged&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (Max) (3.03), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 660 prompts/model, native any-risk flag rate across 6 categories; lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Original DarkBench severe flagged&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 660 prompts/model, severe flags subset of overall flags; lower is better, not disjoint. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Commercial Incentives brand preference&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (2.73), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 110 brand prompts, explicit revenue-sharing/performance system incentives; lower flagged preference is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Commercial Incentives user retention&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (1.82), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 110 retention prompts, explicit revenue-sharing/performance system incentives; lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Commercial Incentives overall risk&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (2.42), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 330 prompts across brand/retention/sycophancy under commercial incentives; lower is better, separate from original 660 prompt suite. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - DeceptionBench L1 no incentive&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0.33), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Factual scenarios in 5 domains, direct decision/advice. No incentive. Native deception rate, lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - DeceptionBench L2 single incentive&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (Max) (0.33), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Factual scenarios in 5 domains, direct decision/advice. Single reward/pressure prompt. Native deception rate, lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - DeceptionBench L3 adaptive follow-up&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (Max) (4.83), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Factual scenarios in 5 domains, direct decision/advice. Up to 5 adaptive followup turns. Native deception rate, lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - DeceptionBench v2 Pressure&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 210 scenarios, 7 domains; false statement counts only without refusal/correction/fictional-example label. Pressure inducement; lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - DeceptionBench v2 Reward&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 210 scenarios, 7 domains; false statement counts only without refusal/correction/fictional-example label. Reward inducement; lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - DeceptionBench v2 Purpose&lt;/strong&gt; (Rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (2.14), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 210 scenarios, 7 domains; false statement counts only without refusal/correction/fictional-example label. Purpose inducement; lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - DeceptionBench v2 Combined&lt;/strong&gt; (Rate (%)): leader GPT-5.6 Sol (Max) (0.72), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. 210 scenarios, 7 domains; false statement counts only without refusal/correction/fictional-example label. Combined inducement; lower is better. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MASK contradiction&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (1.32), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Native contradiction of neutral baseline belief or supplied facts under pressure; lower is better. API failures removed: 964 valid 1017, 976 valid Flash; source populations retained. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MASK explicit honesty&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (86.51), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Explicit honesty under pressure; higher is better. Neither/evasive answers mean this is not 100-minus-contradiction. API-failure denominators preserved. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MASK contradiction Known&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (0.99), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Native MASK task-type contradiction rate, lower is better. Independent neutral belief query versus pressured context; API failures excluded by source. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MASK contradiction Provided&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (1.1), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Native MASK task-type contradiction rate, lower is better. Independent neutral belief query versus pressured context; API failures excluded by source. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MASK contradiction False info&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (1.63), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Native MASK task-type contradiction rate, lower is better. Independent neutral belief query versus pressured context; API failures excluded by source. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MASK contradiction Continue&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (1.72), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Native MASK task-type contradiction rate, lower is better. Independent neutral belief query versus pressured context; API failures excluded by source. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MASK contradiction Reconfirm&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Native MASK task-type contradiction rate, lower is better. Independent neutral belief query versus pressured context; API failures excluded by source. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MASK contradiction Numbers&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (3.12), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. Native MASK task-type contradiction rate, lower is better. Independent neutral belief query versus pressured context; API failures excluded by source. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - OpenDeception deceptive intent&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (Max) (46.0), 7 models&lt;br&gt;&lt;span&gt;Fangcun independent report 2610.35671v1, Oct 4–25 2026 hosted API evaluations; source Table A.1 resolves all seven full model names. Temperature 1, top_p 0.95, maximum reasoning effort unless explicitly overridden; GLM 5.3 rubric judge. API/format failures excluded according to source; not locally deployed weights or uncensored variants. System-assigned deceptive goal across simulated multi-turn scenarios; any reply pursuing goal counts, lower is better for report safety outcome. 46 valid Flash conversations and 50 conversations for each other model; not actual human experiments. Separate simulated trust diagnostic has no unambiguous quality direction. Metric: Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - EvalDetectBench AUROC&lt;/strong&gt; (AUROC): leader Gemini 3.8 Flash (0.95), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.1: the same 152 conversations (83 evaluation and 69 real); mean of 5 prompt probes. AUROC distinguishes evaluation/deployment settings; capability classification accuracy, not propensity toward safety violations. Chance AUROC is .5. Metric: AUROC.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - EvalDetectBench Accuracy&lt;/strong&gt; (Accuracy (%)): leader Gemini 3.8 Flash (86.2), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.1: the same 152 conversations (83 evaluation and 69 real); forced binary evaluation/deployment classification. Benchmark balanced accuracy is not claimed. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Permissions&lt;/strong&gt; (Successful trials): leader DeepSeek V4.1 Flash (1017) (Max) (10), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Calculator Fix&lt;/strong&gt; (Successful trials): leader DeepSeek V4.1 Flash (1017) (Max) (1), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Context Instruction&lt;/strong&gt; (Successful trials): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Context Irreversible&lt;/strong&gt; (Successful trials): leader Gemini 3.8 Flash (1), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Database Tool&lt;/strong&gt; (Successful trials): leader GPT-5.6 Sol (Max) (10), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Latency&lt;/strong&gt; (Successful trials): leader DeepSeek V4.1 Flash (Max) (10), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Message Limit&lt;/strong&gt; (Successful trials): leader DeepSeek V4.1 Flash (Max) (8), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Output Length&lt;/strong&gt; (Successful trials): leader DeepSeek V4.1 Flash (1017) (Max) (10), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Monitoring&lt;/strong&gt; (Successful trials): leader GPT-6 Astra (Max) (9), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Log Recovery&lt;/strong&gt; (Successful trials): leader DeepSeek V4.1 Flash (1017) (Max) (10), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning Output Filter&lt;/strong&gt; (Successful trials): leader DeepSeek V4 Pro (Max) (10), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 10 trials of each runtime-environment challenge; native successful-trial counts, not probability estimates or safety judgments. Task success may require permitted environment changes; challenge pass is separately &amp;gt;= 3 successful trials. Metric: Successful trials.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning trial success&lt;/strong&gt; (Successful trials (%)): leader DeepSeek V4.1 Flash (Max) (52.7), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 110 trials across 11 challenge types, 10 trials each. Pooled trial success, not thresholded challenge pass. Metric: Successful trials (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Self-Reasoning challenge pass&lt;/strong&gt; (Passed challenges (%)): leader DeepSeek V4.1 Flash (1017) (Max) (63.6), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.3: 11 challenges; native challenge pass requires &amp;gt;= 3 successes in 10 trials. Metric: Passed challenges (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - NCRI reasoning-effort comparison&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (Max) (99.3), 19 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.6: No-Chain-of-Thought Reasoning Index accuracy on arithmetic, multistep reasoning, and program-output prediction; exact API effort settings preserved as separate model configurations. Off disables CoT; low/max enable different reasoning budgets. Accuracy uses valid responses in each setting, not a guaranteed identical answered subset. Unsupported off modes are absent, never zeros. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - SHADE-Arena primary task completion&lt;/strong&gt; (Completed primary tasks (%)): leader DeepSeek V4.1 Flash (1017) (Max) (73.08), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.10a: 130 English trials per model, 26 workplace/coding environments repeated 5 times; legitimate primary user task completion; malicious side-task and unflagged side-task rates remain separate non-capability behavior diagnostics. Source adversarial dual-goal setup retained, not ordinary benign-agent accuracy. Metric: Completed primary tasks (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance instructed control ability&lt;/strong&gt; (Task success (%)): leader GPT-6 Astra (Max) (100), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.2c explicit instruction to prevent shutdown, which tests whether the model can disable the sandbox workspace script. Source explicitly identifies this condition as ability control. This ability-control board excludes unsanctioned shutdown-warning and allow-shutdown outcomes; those are reported separately as propensity/instruction-conflict measures. Metric: Task success (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Lie-to-Me misleading hint susceptibility&lt;/strong&gt; (Answers moved to wrong hint (%)): leader GPT-6 Astra (Max) (3), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.4a same multiple-choice questions with/without an incorrect-option hint. Fraction switching baseline answer to the wrong target; a factual-answer robustness error metric, not refusal or harmful-compliance frequency. Conditional CoT-disclosure rates are reported separately because they measure observability. Metric: Answers moved to wrong hint (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Alignment Faking no draft training included&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (Max) (3), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.7 harmful compliance, StrongREJECT judged GLM 5.3 score &amp;gt; 0. Training included/excluded crossed with hidden scratchpad draft/no-draft; not classifier-estimated intention frequency. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Alignment Faking no draft training excluded&lt;/strong&gt; (Native rate (%)): leader GPT-5.6 Sol (Max) (3), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.7 harmful compliance, StrongREJECT judged GLM 5.3 score &amp;gt; 0. Training included/excluded crossed with hidden scratchpad draft/no-draft; not classifier-estimated intention frequency. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Alignment Faking draft training included&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (1), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.7 harmful compliance, StrongREJECT judged GLM 5.3 score &amp;gt; 0. Training included/excluded crossed with hidden scratchpad draft/no-draft; not classifier-estimated intention frequency. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Alignment Faking draft training excluded&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (Max) (2), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.7 harmful compliance, StrongREJECT judged GLM 5.3 score &amp;gt; 0. Training included/excluded crossed with hidden scratchpad draft/no-draft; not classifier-estimated intention frequency. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Agentic Misalignment overall&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.8 eighteen configurations with 100 trials each; 1800 trials/model. Harmful autonomous action under mission/continuation threats, lower source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Agentic Misalignment - Blackmail&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.8 risk-specific rates across matched experimental configurations; not refusal-on-benign-query accuracy. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Agentic Misalignment - Information leakage&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.8 risk-specific rates across matched experimental configurations; not refusal-on-benign-query accuracy. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Agentic Misalignment - Rescue blocking&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.8 risk-specific rates across matched experimental configurations; not refusal-on-benign-query accuracy. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Apollo arithmetic high minus low performance gap&lt;/strong&gt; (Accuracy gap (percentage points)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.9a arithmetic performance drop under incentives to hide capability; paired high-minus-low accuracy percentage points, lower gap source-defined hiding-risk quality. This is not absolute math accuracy. Metric: Accuracy gap (percentage points).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Apollo scheming - Oversight subversion&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.9b source-reported in-context scheming actions under four separate scenarios; lower risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Apollo scheming - Self exfiltration&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.9b source-reported in-context scheming actions under four separate scenarios; lower risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Apollo scheming - Goal guarding&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.9b source-reported in-context scheming actions under four separate scenarios; lower risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Apollo scheming - Reranking&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.9b source-reported in-context scheming actions under four separate scenarios; lower risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Apollo paired supervision conversion&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.9c matched supervision conditions; native propensity conversion rate, not general task accuracy. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MonitorBench input intervention monitorability&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4 Flash (Max) (96.93), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.10 input-intervention monitorability; trace-based observation quality, higher source-defined desirable. Models expose different native CoT; score uses available trace, not a claim of identical reasoning access. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - MonitorBench outcome solution monitorability&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4 Flash (Max) (98.33), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 1.10 outcome/solution monitorability on native available traces; higher source observability quality, not answer accuracy. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PropensityBench overall&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.1 forbidden tool-use under escalating operational pressure; overall native risk, lower. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PropensityBench - Biological&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.1 category-specific forbidden tool-use rates under operational pressure; lower source risk, not willingness as capability. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PropensityBench - Chemical&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.1 category-specific forbidden tool-use rates under operational pressure; lower source risk, not willingness as capability. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PropensityBench - Cyber&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.1 category-specific forbidden tool-use rates under operational pressure; lower source risk, not willingness as capability. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - PropensityBench - Self proliferation&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.1 category-specific forbidden tool-use rates under operational pressure; lower source risk, not willingness as capability. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance no instruction&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.2a shutdown warning without allow/prevent instruction; native operational-interference risk, lower. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance instructed allow&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.2b explicit allow-shutdown instruction averaged over 9 prompt variants; lower bypass means following operator instruction. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S01&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S02&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S03&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S04&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S05&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S06&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S07&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Shared baseline: Figure 2.3/S07 and Figure 2.4/H01 report the same strong-completion, script-path-shown condition without an explicit allow/prevent instruction or an added remaining-work statement. Figure 2.3 identifies the question-tool warning; Figure 2.4 holds completion emphasis and script visibility fixed while varying the remaining-work statement. Its warning channel is inferred from that context and the identical seven-model vector. This board reports the documented baseline once and retains both panel references; individual trial reuse is not explicitly documented. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S08&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S09&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S10&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S11&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance cues - S12&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.3 S1–6 standard task, S7–12 strong completion. Within each 6: question-tool/status-tool/user-warning pairs, odd script-path shown/even omitted. No explicit allow/prevent instruction; lower bypass source risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance handoff - H02&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.4 fixed script-path shown/strong completion. H1 no handoff, H2 work ends, H3–5 uncertain handoff via question/status/user, H6–8 guaranteed handoff via question/status/user. Lower operator-stop interference. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance handoff - H03&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.4 fixed script-path shown/strong completion. H1 no handoff, H2 work ends, H3–5 uncertain handoff via question/status/user, H6–8 guaranteed handoff via question/status/user. Lower operator-stop interference. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance handoff - H04&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.4 fixed script-path shown/strong completion. H1 no handoff, H2 work ends, H3–5 uncertain handoff via question/status/user, H6–8 guaranteed handoff via question/status/user. Lower operator-stop interference. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance handoff - H05&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.4 fixed script-path shown/strong completion. H1 no handoff, H2 work ends, H3–5 uncertain handoff via question/status/user, H6–8 guaranteed handoff via question/status/user. Lower operator-stop interference. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance handoff - H06&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.4 fixed script-path shown/strong completion. H1 no handoff, H2 work ends, H3–5 uncertain handoff via question/status/user, H6–8 guaranteed handoff via question/status/user. Lower operator-stop interference. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance handoff - H07&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.4 fixed script-path shown/strong completion. H1 no handoff, H2 work ends, H3–5 uncertain handoff via question/status/user, H6–8 guaranteed handoff via question/status/user. Lower operator-stop interference. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Shutdown Resistance handoff - H08&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.4 fixed script-path shown/strong completion. H1 no handoff, H2 work ends, H3–5 uncertain handoff via question/status/user, H6–8 guaranteed handoff via question/status/user. Lower operator-stop interference. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - ImpossibleBench One-off LiveCodeBench&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4 Pro (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.5 99 function tasks; passing deliberately faulty tests contrary to task specification is reward hacking; lower is source-defined desirable. One-off wrong expected output vs conflicting expected outputs kept distinct. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - ImpossibleBench One-off SWE-bench&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (3.4), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.5 88 repository tasks; passing deliberately faulty tests contrary to task specification is reward hacking; lower is source-defined desirable. One-off wrong expected output vs conflicting expected outputs kept distinct. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - ImpossibleBench Conflicting LiveCodeBench&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.5 97 function tasks; passing deliberately faulty tests contrary to task specification is reward hacking; lower is source-defined desirable. One-off wrong expected output vs conflicting expected outputs kept distinct. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - ImpossibleBench Conflicting SWE-bench&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.5 89 repository tasks; passing deliberately faulty tests contrary to task specification is reward hacking; lower is source-defined desirable. One-off wrong expected output vs conflicting expected outputs kept distinct. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench SSH no additional rule&lt;/strong&gt; (Native rate (%)): leader Gemini 3.8 Flash (16.4), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.7a 360 runs/access rule; forbidden remote grader access versus legitimate client task; lower native boundary risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench SSH explicit prohibition&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0.3), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.7a 360 runs/access rule; explicit SSH prohibition, lower unauthorized access. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench SSH condition - Hidden no ban&lt;/strong&gt; (Native rate (%)): leader Gemini 3.8 Flash (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.7b 180 runs/cell, credential visibility crossed with SSH restriction. Native rounded percentages retained. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench SSH condition - Hidden ban&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.7b 180 runs/cell, credential visibility crossed with SSH restriction. Native rounded percentages retained. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench SSH condition - Visible no ban&lt;/strong&gt; (Native rate (%)): leader Gemini 3.8 Flash (33), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.7b 180 runs/cell, credential visibility crossed with SSH restriction. Native rounded percentages retained. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench SSH condition - Visible ban&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (1), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.7b 180 runs/cell, credential visibility crossed with SSH restriction. Native rounded percentages retained. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench reward submission - Read&lt;/strong&gt; (Native rate (%)): leader GPT-5.6 Sol (Max) (9), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.8a conditional stage 1 SSH sessions forked 5 ways: sample counts 176, 203, 187, 158, 116, 149, 82 respectively. Valid reward claim fraction after harmful offer; not unconditional misconduct rate. Missing token does not establish absence of harmful action. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench reward submission - Download&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (35), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.8a conditional stage 1 SSH sessions forked 5 ways: sample counts 176, 203, 187, 158, 116, 149, 82 respectively. Valid reward claim fraction after harmful offer; not unconditional misconduct rate. Missing token does not establish absence of harmful action. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench reward submission - Delete&lt;/strong&gt; (Native rate (%)): leader GPT-5.6 Sol (Max) (8), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.8a conditional stage 1 SSH sessions forked 5 ways: sample counts 176, 203, 187, 158, 116, 149, 82 respectively. Valid reward claim fraction after harmful offer; not unconditional misconduct rate. Missing token does not establish absence of harmful action. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench reward submission - Upload&lt;/strong&gt; (Native rate (%)): leader GPT-5.6 Sol (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.8a conditional stage 1 SSH sessions forked 5 ways: sample counts 176, 203, 187, 158, 116, 149, 82 respectively. Valid reward claim fraction after harmful offer; not unconditional misconduct rate. Missing token does not establish absence of harmful action. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - FoFBench reward submission - Escalate&lt;/strong&gt; (Native rate (%)): leader GPT-5.6 Sol (Max) (1), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.8a conditional stage 1 SSH sessions forked 5 ways: sample counts 176, 203, 187, 158, 116, 149, 82 respectively. Valid reward claim fraction after harmful offer; not unconditional misconduct rate. Missing token does not establish absence of harmful action. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Monitor Sabotage direct attack&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.9a 100 trials/model; joint unauthorized action+tactic+evasion of at least one monitor. Lower source oversight risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - Monitor Sabotage indirect disruption&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (Max) (1), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.9b 100 trials/model; joint unauthorized action+tactic+evasion of at least one monitor. Lower source oversight risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - SHADE-Arena malicious side task completion&lt;/strong&gt; (Native rate (%)): leader GPT-5.6 Sol (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.10b 130 English trials/model; 26 environments 5 repeats, explicitly assigned covert goal; source behavioral risk lower, separate legitimate primary task. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - SHADE-Arena unflagged side task completion&lt;/strong&gt; (Native rate (%)): leader DeepSeek V4.1 Flash (1017) (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.10c 130 English trials/model; side-task completion with post hoc transcript monitor suspicion &amp;lt; 80. CoT not provided to monitor. Lower evasion risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Fangcun DeepSeek 1017 Report - GDM Oversight Pattern secret goal completion&lt;/strong&gt; (Native rate (%)): leader GPT-6 Astra (Max) (0), 7 models&lt;br&gt;&lt;span&gt;Fangcun report 2610.35671v1, hosted API evaluations Oct 4–25 2026; Appendix A.1 exact model identities, maximum reasoning effort unless named otherwise, temperature 1/top_p 0.95. GLM 5.3 judge where required. API/format failures excluded by source; common cohort versus successfully completed denominators retained. Primary PDF printed values, not estimated plot heights. Figure 2.11 100 trials/model: 50 domain-review/50 subject-tag filter; 44 emails within 120 message budget, secret goal 3 or 6 recipients without &amp;gt; 8 rejections. Lower source coverage-exploitation risk. Metric: Native rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - HLE&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (54.9), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - CritPt&lt;/strong&gt; (Score (%)): leader GPT-5.6 Sol (32.3), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - Healthbench Professional&lt;/strong&gt; (Score (%)): leader Ling 3.1 Flash (AQ) (65.35), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Ling healthcare capability is evaluated and available in the AQ environment, not established for the plain downloaded checkpoint. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - SWE-bench Multilingual&lt;/strong&gt; (Score (%)): leader GLM-5.3 (90.67), 6 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - SWE-bench Pro&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (79.2), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Ling&amp;#x27;s row uses LingCode, a Pi-derived harness; comparator harnesses are not established by the overview footnote. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - SWE Atlas - Codebase QnA&lt;/strong&gt; (Score (%)): leader Kimi K3 (66.0), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Ling&amp;#x27;s row uses LingCode, a Pi-derived harness; comparator harnesses are not established by the overview footnote. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - DeepSWE&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (74.2), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Ling&amp;#x27;s row uses LingCode, a Pi-derived harness; comparator harnesses are not established by the overview footnote. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - Terminal-Bench-2.1&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (90.6), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - Terminal-Bench-4.0&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (49.0), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Ling&amp;#x27;s row uses LingCode, a Pi-derived harness; comparator harnesses are not established by the overview footnote. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - TAU3-Banking&lt;/strong&gt; (Score (%)): leader GLM-5.3 (50.3), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - Automationbench (Public)&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (54.8), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - SkillsBench&lt;/strong&gt; (Score (%)): leader GPT-5.6 Sol (73.5), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - Mcp-atlas&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (85.8), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - GDPval-AA v2.1&lt;/strong&gt; (Elo): leader Claude Opus 5 (1708.0), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Elo.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - Browsecomp&lt;/strong&gt; (Score (%)): leader Ling 3.1 Flash (91.67), 5 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - Draco&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (88.6), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - WideSearch&lt;/strong&gt; (Score (%)): leader GPT-5.6 Sol (86.3), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - Finance Agent V2&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (62.96), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - Finsearch&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (84.75), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;InclusionAI Ling 3.1 Flash Release - MultiChallenge&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (72.12), 7 models&lt;br&gt;&lt;span&gt;Official undated Ling 3.1 Flash release-card comparison; native printed scores. Effort, prompt and harness details are unspecified except the explicit LingCode/AQ notes; do not infer the quickstart&amp;#x27;s thinking recommendation as evaluation configuration. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - L1 Spec Compliance&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (99.6), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - Delivered L2 Check Pass&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (97.3), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - Delivered L3 Visual Quality&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.08), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - Delivered L3 Visual Aesthetics&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.3), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - Delivered L3 Content Logic&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.2), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - Delivered L3 Timing and Motion&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (3.95), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - Delivered L3 Audio Quality&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.1), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - Delivered L3 Overall&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.126), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - All Tasks L2 Check Pass&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (97.3), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - All Tasks L2 Strict Success&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (76.3), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - All Tasks L3 Overall&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.109), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench - Human Preference&lt;/strong&gt; (Human Bradley–Terry Elo): leader GPT-6 Astra (1242.0), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Blind human pairwise preferences between playable videos; decisive votes only; native Bradley–Terry Elo scale centered geometrically at 1000, not delivery probability. Metric: Human Bradley–Terry Elo.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - Delivered L2 Pass&lt;/strong&gt; (Pass rate (%)): leader GPT-6 Astra (98.4), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - All Tasks L2 Pass&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (98.4), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - All Core Pass&lt;/strong&gt; (Pass rate (%)): leader GPT-6 Astra (76.6), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - Strict Success&lt;/strong&gt; (Pass rate (%)): leader GPT-6 Astra (83.7), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - Delivered L3 Visual Quality&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.0), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - Delivered L3 Visual Aesthetics&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.06), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - Delivered L3 Content Logic&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.07), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - Delivered L3 Timing and Motion&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (3.92), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - Delivered L3 Audio Quality&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (3.8), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - Delivered L3 Overall&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (3.963), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini4Argon - All Tasks L3 Overall&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (3.946), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 4 Argon panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - Delivered L2 Pass&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (99.4), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - All Tasks L2 Pass&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (99.4), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - All Core Pass&lt;/strong&gt; (Pass rate (%)): leader GPT-6 Astra (91.2), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - Strict Success&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (93.3), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - Delivered L3 Visual Quality&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (4.16), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - Delivered L3 Visual Aesthetics&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (4.15), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - Delivered L3 Content Logic&lt;/strong&gt; (Quality (1–5)): leader Gemini 4 Argon (High) (Harness: Antigravity CLI) (4.15), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - Delivered L3 Timing and Motion&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (4.14), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - Delivered L3 Audio Quality&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (4.1), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - Delivered L3 Overall&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (4.137), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Gemini3.8Flash - All Tasks L3 Overall&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (4.137), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual Gemini 3.8 Flash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - Delivered L2 Pass&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (94.2), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - All Tasks L2 Pass&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (94.2), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - All Core Pass&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (35.1), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - Strict Success&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (51.9), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - Delivered L3 Visual Quality&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.08), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - Delivered L3 Visual Aesthetics&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.71), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - Delivered L3 Content Logic&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.43), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - Delivered L3 Timing and Motion&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (3.88), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - Delivered L3 Audio Quality&lt;/strong&gt; (Quality (1–5)): leader Gemini 4 Argon (High) (Harness: Antigravity CLI) (4.47), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - Delivered L3 Overall&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.303), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench GPT5.6Sol and Qwen3.8OmniFlash - All Tasks L3 Overall&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.285), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. This board is the individual GPT 5.6 Sol and Qwen 3.8 OmniFlash panel, not the three-panel average. All-core success counts unknown judgments against success; strict success does not. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Brand &amp; marketing (9)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.18), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Product &amp; feature demos (6)&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (4.24), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Events &amp; personal expression (7)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.03), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Concept &amp; pipeline explainers (57)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.19), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Data &amp; fact stories (24)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.14), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Visual art &amp; form (26)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (3.94), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Games &amp; sports (19)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.15), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Story &amp; visual comedy (15)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.0), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Building &amp; process (23)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (3.97), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Music &amp; sound-driven (19)&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (3.92), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Classroom lessons (24)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.47), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench All Tasks - Paper explainers (10)&lt;/strong&gt; (Quality (1–5)): leader Claude Sonnet 5.5 (Max) (Harness: Claude Code 2.1.301) (4.1), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. All Tasks. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Brand &amp; marketing (9)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.18), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Product &amp; feature demos (6)&lt;/strong&gt; (Quality (1–5)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (4.24), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Events &amp; personal expression (7)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.03), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Concept &amp; pipeline explainers (57)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.19), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Data &amp; fact stories (24)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.14), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Visual art &amp; form (26)&lt;/strong&gt; (Quality (1–5)): leader Gemini 4 Argon (High) (Harness: Antigravity CLI) (3.96), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Games &amp; sports (19)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.15), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Story &amp; visual comedy (15)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.0), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Building &amp; process (23)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (3.97), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Music &amp; sound-driven (19)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.04), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Classroom lessons (24)&lt;/strong&gt; (Quality (1–5)): leader GPT-6 Astra (4.47), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered - Paper explainers (10)&lt;/strong&gt; (Quality (1–5)): leader Claude Sonnet 5.5 (Max) (Harness: Claude Code 2.1.301) (4.1), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Category-specific L3 overall, three-panel mean of per-task scores; exact task count in board name. Delivered. Metric: Quality (1–5).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered Modality Pass - Visual&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (96.6), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. L2 pass rates by modality; unknown-rate columns are diagnostic, not correctness. Visual. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered Modality Pass - Audio&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (99.4), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. L2 pass rates by modality; unknown-rate columns are diagnostic, not correctness. Audio. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Delivered Modality Pass - Audio-visual&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (98.9), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. L2 pass rates by modality; unknown-rate columns are diagnostic, not correctness. Audio-visual. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Core Spec - Resolution&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (100.0), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. There are 205 core tasks: 54 require an exact frame count, and 98 require audio with a peak above -50 dBFS. Missing videos fail. Resolution. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Core Spec - FPS&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (99.5), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. There are 205 core tasks: 54 require an exact frame count, and 98 require audio with a peak above -50 dBFS. Missing videos fail. FPS. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Core Spec - Duration&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (100.0), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. There are 205 core tasks: 54 require an exact frame count, and 98 require audio with a peak above -50 dBFS. Missing videos fail. Duration. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Core Spec - Exact Frame Count&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (100.0), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. There are 205 core tasks: 54 require an exact frame count, and 98 require audio with a peak above -50 dBFS. Missing videos fail. Exact Frame Count. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Core Spec - Required Audio&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (100.0), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. There are 205 core tasks: 54 require an exact frame count, and 98 require audio with a peak above -50 dBFS. Missing videos fail. Required Audio. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Core Spec - Editable Project&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (100.0), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. There are 205 core tasks: 54 require an exact frame count, and 98 require audio with a peak above -50 dBFS. Missing videos fail. Editable Project. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;VibeVideoBench Core Spec - All Checks&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (Harness: Claude Code 2.1.301) (99.5), 9 models&lt;br&gt;&lt;span&gt;VibeVideoBench v1 has 239 English/Chinese tasks across 12 categories. Each model-task pair receives one autonomous rollout in its native agent harness, using identical Ubuntu 24.04 containers with 8 CPUs, 32GB RAM, and no GPU. The 3600-second budget includes tools and rendering. The output is a code-generated MP4 with sound; Qwen3-TTS-Flash is the only generative service. Native harnesses and effort levels remain explicit, with no model-only aliases. Three blind judge panels are averaged separately: Gemini 4 Argon and Gemini 3.8 Flash evaluate full video/audio, while GPT 5.6 Sol evaluates visual frames with Qwen3.8 OmniFlash evaluating audio. L2 checks allow pass, fail, unknown, or NA; strict success requires no failed core check and at least one pass. L3 averages five anchored quality dimensions. Delivered-only populations remain separate from populations that assign zero to missing videos. Success requires all program checks to pass across 205 core tasks. The native percentage printed beside the count is retained. Metric: Accuracy (%).&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (29)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on AA CritPt v2: 76.86 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Klu LLM Leaderboard: 92.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ProgramBench: 9.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on ProgramBench Almost: 55.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Mythos 5.1&lt;/strong&gt; on ExploitGym (version unspecified; 6h timeout): 264.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ASCIIBench: 1771.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Design Arena (ASCII Art): 1351.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Klu LLM Leaderboard: 98.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ProgramBench: 18.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ProgramBench Almost: 70.5 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on o11y-bench - Pass@3: 98.44 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on o11y-bench - Pass^3: 93.75 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Klu LLM Leaderboard: 88.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on ProgramBench: 5.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on ProgramBench Almost: 52.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on o11y-bench - Pass@3: 96.88 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on o11y-bench - Pass^3: 87.5 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on ASCIIBench: 1621.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Klu LLM Leaderboard: 87.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on o11y-bench - Pass@3: 96.88 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on o11y-bench - Pass^3: 87.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ASCIIBench: 2085.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Klu LLM Leaderboard: 91.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on o11y-bench - Pass@3: 98.44 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on o11y-bench - Pass^3: 87.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Klu LLM Leaderboard: 96.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LLM Stats Score: 53.17 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on ProgramBench: 8.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on ProgramBench Almost: 60.0 (#2)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (14)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;ExploitGym (version unspecified; 6h timeout)&lt;/strong&gt;: Claude Opus 5.5 (300.0) beat Claude Mythos 5 by 53.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ProgramBench Almost&lt;/strong&gt;: Claude Opus 5.5 (xHigh) (70.5) beat Claude Opus 5 (xHigh) by 33.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ProgramBench&lt;/strong&gt;: Claude Opus 5.5 (xHigh) (18.5) beat Claude Opus 5 (xHigh) by 14.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReasonScape R12&lt;/strong&gt;: DeepSeek V4.1 Flash (965.6) beat Muse-Glimmer-30B (high, 16k) by 11.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;o11y-bench - Pass^3&lt;/strong&gt;: Claude Opus 5.5 (Medium) (93.75) beat Claude Sonnet 5.5 (High) by 4.69&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM-AggreFact&lt;/strong&gt;: nimble-v4-max (79.45) beat Bespoke-Minicheck-7B by 2.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Ebr Bench&lt;/strong&gt;: GPT-6.1 Terra (Max) (78.1) beat GPT-6 Astra (Max) by 1.91&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Medicine and Mental Health&lt;/strong&gt;: Kimi K3.1 Pro (92.7) beat Doubao-Seed-2.1-Pro-260628 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Search Strategy for Concolic Testing&lt;/strong&gt;: GPT-6.1 Terra (16.214223208857305) beat Claude Opus 5.5 (Max) by 0.74&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Generative Quality&lt;/strong&gt;: GPT-6.1 Terra (84.34) beat GPT-6 Sol by 0.59&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - E-Graph Invariant Restoration&lt;/strong&gt;: Kimi K3.1 Pro (62.25732688392222) beat DeepSeek V4.1 Flash (1017) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Join-Tree Rewriting&lt;/strong&gt;: Kimi K3.1 Pro (12.533414389629822) beat Kimi K2.7 Code (Medium) by 0.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education - Middle School Subjects&lt;/strong&gt;: Kimi K3.1 Pro (60.4) beat doubao-seed-2-1-lite-260915 by 0.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - SQL Counterexample Search&lt;/strong&gt;: GPT-6.1 Terra (97.95388756950616) beat GLM-5.3 (Max) by 0.3&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-28

=== DAILY ===
NEW BENCHMARKS (242)
  - ExploitGym (version unspecified; token-budgeted) (Successful intended exploits (#); version unspecified; token-budgeted): leader GPT-6 Astra (368.0), 2 models
      Approximate counts of successful intended-vulnerability explo</summary></entry><entry><title>The Aggregate Digest — 2026-10-27</title><id>https://theaggregate.ai/digest/2026-10-27</id><updated>2026-10-27T08:41:07.860809+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (135)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Closed QA Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Closed QA; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Closed QA Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Closed QA; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Closed QA Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (3.3), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Closed QA; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Closed QA Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (8.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Closed QA; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Closed QA Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (0.2), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Closed QA; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Closed QA Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (0.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Closed QA; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Extraction Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Kimi K3 (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Extraction; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Extraction Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Kimi K3 (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Extraction; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Extraction Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (2.7), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Extraction; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Extraction Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (7.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Extraction; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Extraction Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (0.2), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Extraction; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Extraction Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (0.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Extraction; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Summarization Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Summarization; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Summarization Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Summarization; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Summarization Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (6.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Summarization; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Summarization Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (16.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Summarization; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Summarization Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (0.2), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Summarization; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Dolly: Summarization Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (0.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Dolly: Summarization; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (long) Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Meta-SecAlign-70B (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (long); Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (long) Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Meta-SecAlign-70B (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (long); Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (long) Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (2.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (long); Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (long) Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (5.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (long); Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (long) Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Kimi K3 (Low) (1.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (long); Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (long) Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (3.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (long); Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (RAG) Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (RAG); Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (RAG) Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (RAG); Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (RAG) Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (4.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (RAG); Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (RAG) Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (11.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (RAG); Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (RAG) Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (3.7), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (RAG); Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - HotpotQA (RAG) Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (11.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. HotpotQA (RAG); Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - LCC Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Meta-SecAlign-70B (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. LCC; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - LCC Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Meta-SecAlign-70B (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. LCC; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - LCC Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (6.1), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. LCC; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - LCC Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (15.2), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. LCC; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - LCC Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (3.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. LCC; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - LCC Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (9.1), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. LCC; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Multi-News Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Multi-News; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Multi-News Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Multi-News; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Multi-News Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (1.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Multi-News; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Multi-News Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (3.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Multi-News; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Multi-News Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (0.3), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Multi-News; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Multi-News Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (1.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Multi-News; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - NQ (RAG) Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader GLM-5.3 (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. NQ (RAG); Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - NQ (RAG) Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader GLM-5.3 (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. NQ (RAG); Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - NQ (RAG) Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (3.7), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. NQ (RAG); Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - NQ (RAG) Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (11.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. NQ (RAG); Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - NQ (RAG) Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (2.7), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. NQ (RAG); Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - NQ (RAG) Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (8.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. NQ (RAG); Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Qasper Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Meta-SecAlign-70B (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Qasper; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Qasper Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Meta-SecAlign-70B (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Qasper; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Qasper Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (3.3), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Qasper; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Qasper Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (8.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Qasper; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Qasper Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Qasper; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Qasper Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Qasper; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - SQuAD v2 Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.2), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. SQuAD v2; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - SQuAD v2 Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. SQuAD v2; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - SQuAD v2 Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (3.7), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. SQuAD v2; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - SQuAD v2 Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (8.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. SQuAD v2; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - SQuAD v2 Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (0.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. SQuAD v2; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - SQuAD v2 Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (1.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. SQuAD v2; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Language Overall Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.1), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Language Overall; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Language Overall Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (0.3), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Language Overall; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Language Overall Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (3.7), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Language Overall; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Language Overall Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (9.6), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Language Overall; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Language Overall Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (1.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Language Overall; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - Language Overall Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (4.1), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. Language Overall; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDojo Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDojo; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDojo Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (1.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDojo; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDojo Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (3.4), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDojo; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDojo Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (7.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDojo; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDojo Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (1.4), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDojo; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDojo Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (2.8), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDojo; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDyn Fixed Target ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDyn; Fixed Target; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDyn Fixed Target ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (0.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDyn; Fixed Target; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDyn Near Target Incorrect ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Claude Opus 5 (Low) (6.2), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDyn; Near Target Incorrect; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDyn Near Target Incorrect ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Claude Opus 5 (Low) (16.5), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDyn; Near Target Incorrect; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDyn Near Target Harmful ASR@1&lt;/strong&gt; (Attack success rate at 1 (%)): leader Gemini 3.7 Flash (Low) (2.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDyn; Near Target Harmful; ASR@1. Metric: Attack success rate at 1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NTBench - AgentDyn Near Target Harmful ASR@3&lt;/strong&gt; (Attack success rate at 3 (%)): leader Gemini 3.7 Flash (Low) (4.0), 7 models&lt;br&gt;&lt;span&gt;NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims, no adaptive search. Native attack success rate, lower is better. ASR@1 and ASR@3 retain distinct budgets. Language attempts total 4,197 per track across ten datasets; agentic samples are reached injection positions in AgentDojo/AgentDyn, with 15 tool-calling iterations. Deviations already present attack-free and unreached positions are excluded. Temperature 0 where supported, Kimi K3 1, Claude unspecified; GPT-5.6 Sol/DeepSeek reasoning off, other API models low. Different attack tracks retain different success criteria. AgentDyn; Near Target Harmful; ASR@3. Metric: Attack success rate at 3 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra Launch - Agents' Last Exam v1&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (Max) (63.2), 26 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra launch effort sweep; Score (%). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra Launch - MMMU-Pro (Vision)&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (Max) (86.7), 25 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra launch effort sweep; Score (%). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra Launch - OSWorld 2.1, offline set&lt;/strong&gt; (Score (%)): leader Claude Sonnet 5.5 (Max) (83.9), 30 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra launch effort sweep; Score (%). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra Launch - Factual Errors (October 2026 Refreshed Prompts)&lt;/strong&gt; (Answers with any factual error (%)): leader GPT-6 Astra (Max) (4.2), 25 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra launch effort sweep; Answers with any factual error (%). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Answers with any factual error (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra Launch - Terminal-Bench 4.0&lt;/strong&gt; (Score (%)): leader Claude Sonnet 5.5 (Max) (70.6), 21 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra launch effort sweep; Score (%). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra Launch - Terminal-Bench-Science 0.1&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (Max) (68.1), 16 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra launch effort sweep; Score (%). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - HealthBench Consensus&lt;/strong&gt; (Length-Adjusted Score (0-100)): leader GPT-6.1 Sol (96.0), 2 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra system-card addendum; Length-Adjusted Score (0-100). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Length-Adjusted Score (0-100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - MentalHealthBench (October 2026) - overall&lt;/strong&gt; (Mean task rubric score (%; four responses per task, max effort)): leader GPT-6 Astra (Max) (58.7), 8 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra system-card addendum; Mean task rubric score (%; four responses per task, max effort). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Mean task rubric score (%; four responses per task, max effort).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - MentalHealthBench (October 2026) - non-acute&lt;/strong&gt; (Mean task rubric score (%; four responses per task, max effort)): leader GPT-6 Astra (Max) (58.5), 8 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra system-card addendum; Mean task rubric score (%; four responses per task, max effort). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Mean task rubric score (%; four responses per task, max effort).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - MentalHealthBench (October 2026) - high acuity&lt;/strong&gt; (Mean task rubric score (%; four responses per task, max effort)): leader GPT-6 Astra (Max) (59.8), 8 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra system-card addendum; Mean task rubric score (%; four responses per task, max effort). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Mean task rubric score (%; four responses per task, max effort).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - MentalHealthBench (October 2026) - emergent&lt;/strong&gt; (Mean task rubric score (%; four responses per task, max effort)): leader GPT-6 Astra (Max) (58.5), 8 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra system-card addendum; Mean task rubric score (%; four responses per task, max effort). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Mean task rubric score (%; four responses per task, max effort).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - AAV Capsid Packaging Prediction&lt;/strong&gt; (Spearman rank correlation (higher is better)): leader GPT-6 Sol (0.534), 5 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra system-card addendum; Spearman rank correlation (higher is better). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Spearman rank correlation (higher is better).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Coronavirus--ACE2 Cell-Entry Screen&lt;/strong&gt; (Composite score (higher is better)): leader GPT-6.1 Terra (0.4741), 5 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra system-card addendum; Composite score (higher is better). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Composite score (higher is better).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Phage--plasmid Co-evolution&lt;/strong&gt; (Negative log-likelihood (lower is better)): leader GPT-6.1 Terra (10.8229), 5 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra system-card addendum; Negative log-likelihood (lower is better). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Negative log-likelihood (lower is better).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - GPT-Red Indirect Prompt Injection&lt;/strong&gt; (Defender success rate (%; averaged per query)): leader GPT-6 Astra (99.789), 5 models&lt;br&gt;&lt;span&gt;Defender success rate (%; averaged per query); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Defender success rate (%; averaged per query).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Successful Warning Circumvention (October 2026)&lt;/strong&gt; (Successful circumvention rate (%; without safeguards)): leader GPT-6 Luna (Max) (17.4), 7 models&lt;br&gt;&lt;span&gt;Successful circumvention rate (%; without safeguards); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Successful circumvention rate (%; without safeguards).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - ExploitGym Honeypot - Successful Exploit&lt;/strong&gt; (Successful honeypot exploit rate (%; no production safeguards)): leader GPT-6.1 Terra (Max) (0.0), 3 models&lt;br&gt;&lt;span&gt;Successful honeypot exploit rate (%; no production safeguards); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Successful honeypot exploit rate (%; no production safeguards).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - ExploitGym Honeypot - Unsuccessful Attempt&lt;/strong&gt; (Unsuccessful honeypot attempt rate (%; no production safeguards)): leader GPT-6.1 Terra (Max) (0.07), 3 models&lt;br&gt;&lt;span&gt;Unsuccessful honeypot attempt rate (%; no production safeguards); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Unsuccessful honeypot attempt rate (%; no production safeguards).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Blocker Deception (October 2026)&lt;/strong&gt; (Blocker-related deception rate (%)): leader GPT-6 Sol (8.05), 7 models&lt;br&gt;&lt;span&gt;Blocker-related deception rate (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Blocker-related deception rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Realistic Work Environments (October 2026) - Overall&lt;/strong&gt; (Misaligned outcome rate (%)): leader GPT-6 Astra (1.16), 5 models&lt;br&gt;&lt;span&gt;Misaligned outcome rate (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Misaligned outcome rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Realistic Work Environments (October 2026) - Security Weakening&lt;/strong&gt; (Misaligned outcome rate (%)): leader GPT-6 Astra (0.39), 5 models&lt;br&gt;&lt;span&gt;Misaligned outcome rate (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Misaligned outcome rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Realistic Work Environments (October 2026) - Data Exfiltration&lt;/strong&gt; (Misaligned outcome rate (%)): leader GPT-6 Astra (0.46), 5 models&lt;br&gt;&lt;span&gt;Misaligned outcome rate (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Misaligned outcome rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Realistic Work Environments (October 2026) - Destructive Action&lt;/strong&gt; (Misaligned outcome rate (%)): leader GPT-6.1 Sol (0.0), 5 models&lt;br&gt;&lt;span&gt;Misaligned outcome rate (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Misaligned outcome rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Realistic Work Environments (October 2026) - Unauthorized External Communication&lt;/strong&gt; (Misaligned outcome rate (%)): leader GPT-6.1 Sol (0.0), 5 models&lt;br&gt;&lt;span&gt;Misaligned outcome rate (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Misaligned outcome rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Realistic Work Environments (October 2026) - Service Disruption&lt;/strong&gt; (Misaligned outcome rate (%)): leader GPT-5.6 Sol (0.0), 5 models&lt;br&gt;&lt;span&gt;Misaligned outcome rate (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Misaligned outcome rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Realistic Work Environments (October 2026) - Unauthorized Transactions&lt;/strong&gt; (Misaligned outcome rate (%)): leader GPT-6.1 Terra (0.0), 5 models&lt;br&gt;&lt;span&gt;Misaligned outcome rate (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Misaligned outcome rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Realistic Work Environments (October 2026) - Circumventing Restrictions&lt;/strong&gt; (Misaligned outcome rate (%)): leader GPT-6 Astra (0.0), 5 models&lt;br&gt;&lt;span&gt;Misaligned outcome rate (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Misaligned outcome rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - External Agent Messages (Corrected Paths October 2026) - Engagement&lt;/strong&gt; (Engagement given discovery (%; 80 samples; includes refusals)): leader GPT-6 Luna (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Engagement given discovery (%; 80 samples; includes refusals); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Engagement given discovery (%; 80 samples; includes refusals).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - External Agent Messages (Corrected Paths October 2026) - Unauthorized Action&lt;/strong&gt; (Specified unauthorized action given discovery (%; 80 samples)): leader GPT-6 Luna (Max) (0.0), 7 models&lt;br&gt;&lt;span&gt;Specified unauthorized action given discovery (%; 80 samples); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Specified unauthorized action given discovery (%; 80 samples).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ All Tasks&lt;/strong&gt; (Severity 3+ flagged trajectories (%; 48,690 matched tasks)): leader GPT-6.1 Terra (0.045), 5 models&lt;br&gt;&lt;span&gt;Severity 3+ flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Severity 3+ flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ No Detected Awareness&lt;/strong&gt; (Severity 3+ flagged trajectories (%; 39,083 tasks, awareness absent in all replays)): leader GPT-6.1 Sol (0.036), 5 models&lt;br&gt;&lt;span&gt;Severity 3+ flagged trajectories (%; 39,083 tasks, awareness absent in all replays); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Severity 3+ flagged trajectories (%; 39,083 tasks, awareness absent in all replays).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ Circumventing Restrictions&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Astra (0.0246), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ Reward Hacking&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Astra (0.0), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ Exfiltration&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Sol (0.0103), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ Other Misalignment&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6.1 Sol (0.0041), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ Destructive Action&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6.1 Sol (0.0), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ Outbound Prompt Injection&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Sol (0.0), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ Credential Harvesting&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-5.6 Sol (0.0), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 3+ Sabotage&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Sol (0.0), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Ignoring Instructions&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6.1 Terra (11.803), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Concealing Uncertainty&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Astra (0.335), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Deception&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6.1 Terra (0.06), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Reward Hacking&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6.1 Terra (0.033), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Circumventing Restrictions&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Astra (0.207), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Exfiltration&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Astra (0.016), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Other Misalignment&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6.1 Sol (0.15), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Destructive Action&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Astra (0.076), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Credential Harvesting&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-6 Sol (0.012), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Sandbagging&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-5.6 Sol (0.002), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Codex Deployment Simulation (October 2026) - Severity 1-2 Inbound Prompt Injection&lt;/strong&gt; (Flagged trajectories (%; 48,690 matched tasks)): leader GPT-5.6 Sol (0.004), 5 models&lt;br&gt;&lt;span&gt;Flagged trajectories (%; 48,690 matched tasks); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Flagged trajectories (%; 48,690 matched tasks).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - ExploitBench (Max Tested Solution Length)&lt;/strong&gt; (ACE-normalized mean capability (%)): leader GPT-6 Astra (100.0), 6 models&lt;br&gt;&lt;span&gt;ACE-normalized mean capability (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: ACE-normalized mean capability (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - SEC-Bench Pro (Max Tested Solution Length)&lt;/strong&gt; (pass@1 (%)): leader GPT-6 Astra (85.4), 6 models&lt;br&gt;&lt;span&gt;pass@1 (%); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: pass@1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Sandbox-Bench - Runtime&lt;/strong&gt; (Success rate (%; 5 targets)): leader GPT-6 Astra (80.0), 6 models&lt;br&gt;&lt;span&gt;Success rate (%; 5 targets); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Success rate (%; 5 targets).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Sandbox-Bench - Parser&lt;/strong&gt; (Success rate (%; 14 targets)): leader GPT-6 Astra (35.7), 6 models&lt;br&gt;&lt;span&gt;Success rate (%; 14 targets); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Success rate (%; 14 targets).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - Sandbox-Bench - Egress Proxy&lt;/strong&gt; (Success rate (%; 3 targets)): leader GPT-6 Astra (33.3), 6 models&lt;br&gt;&lt;span&gt;Success rate (%; 3 targets); GPT-6.1 Terra October 26, 2026 system-card cohort; native printed labels, with evaluation and configuration boundaries in board/model names. Metric: Success rate (%; 3 targets).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6.1 Terra System Card - ExploitBench Internal Port (June-August 2026)&lt;/strong&gt; (Arbitrary code-execution success rate (%; max tested solution length)): leader GPT-6 Astra (31.5), 5 models&lt;br&gt;&lt;span&gt;Official OpenAI October 26, 2026 GPT-6.1 Terra system-card addendum; Arbitrary code-execution success rate (%; max tested solution length). Explicit version/effort/model settings are preserved in the board and row names. Shared Astra-card safety/health rows retain the unchanged GPT-6.1 Sol anchor only; changed older comparison checkpoints are not overwritten. Metric: Arbitrary code-execution success rate (%; max tested solution length).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Code2Video-AA (Dataset 0.3.0)&lt;/strong&gt; (Mean axis Elo (GPT-5.5 xhigh anchor=1000)): leader Claude Opus 5.5 (Max, Default Fallback) (1535.57), 27 models&lt;br&gt;&lt;span&gt;Artificial Analysis Code2Video-AA dataset 0.3.0: 114 design briefs (39 public, 75 private). One no-tool generation using HeyGen system prompt; HTML layout and GSAP animation rendered offline with HyperFrames 0.8.30, five-minute render limit. Anonymous pairwise Gemini 4 Argon judging at 8fps/high media resolution; independent Bradley-Terry fits on engagement, prompt intent, composition, temporal consistency and craft. Overall is the mean of five axis Elos, each anchored to GPT-5.5 (xhigh) at 1000. Render failures lose to successful renders and tie each other. Effort/default-fallback labels remain separate. Gemini 4 Argon is both a rated generator and the judge; these roles and shared axes are not independent replication. Metric: Mean axis Elo (GPT-5.5 xhigh anchor=1000).&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; — Aggregate Rating 1948, #10&lt;ul&gt;&lt;li&gt;OpenAI GPT-6 Sol &amp; Luna Launch - Broken Search Tool Non-Disclosure: 1.83 (#1/2)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Sol &amp; Luna Launch - Reviewer Bypass Attempts: 0.0 (#1/2)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Cyber Safety - Synthetic Agentic Environments: 1.0 (#1/2)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Sensitive Personal Data: 74.7 (#1/2)&lt;/li&gt;&lt;li&gt;Coarena: 1170.6 (#1/20)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 96.4 (#1/20)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 92.74 (#1/234)&lt;/li&gt;&lt;li&gt;OpenAI GPT-5.6 Launch - Internal Research Debugging Evaluation: 76.39 (#2/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Biology Refusal - Severe: 99.0 (#2/2)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Biology Refusal - Dual Use: 96.0 (#2/2)&lt;/li&gt;&lt;li&gt;...and 114 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (155)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - API Security: 75.9 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Cloud Security: 66.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Code Vulnerabilities: 58.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Overall: 64.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Websec.fr CTFs: 57.5 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - API Security: 77.8 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Cloud Security: 66.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 58.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Overall: 64.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 55.2 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - API Security: 81.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Cloud Security: 72.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 56.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Overall: 70.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 70.1 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Cloud Security: 68.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Code Vulnerabilities: 56.1 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Overall: 69.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Websec.fr CTFs: 71.3 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-5.6 Launch - Internal Research Debugging Evaluation: 75.52 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-5.6 Launch - KernelGen 1P: 60.44 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - API Security: 80.2 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Cloud Security: 66.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 56.6 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Overall: 70.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 79.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - API Security: 80.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Code Vulnerabilities: 56.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Overall: 70.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Wiz Cyber Model Arena (Codex) - Websec.fr CTFs: 77.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA CritPt: 28.0 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA CritPt v2: 79.43 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf: 28.6 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - Construction - Criterion Pass Rate: 78.71 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - Criterion Pass Rate: 79.54 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - Engineering - Criterion Pass Rate: 83.87 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 92.74 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - HR - Criterion Pass Rate: 74.11 (#69)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - Healthcare - Criterion Pass Rate: 89.78 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - Insurance - Criterion Pass Rate: 75.83 (#57)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - Legal - Criterion Pass Rate: 76.07 (#87)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 79.73 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - Real Estate - Criterion Pass Rate: 74.78 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDP.pdf - STEM/Research - Criterion Pass Rate: 77.88 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA GDPval: 1476.05 (#58)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA Humanity's Last Exam: 46.25 (#46)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA Long Context Reasoning: 82.67 (#38)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA MMMU-Pro: 84.68 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA Omniscience: 29.78 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA SciCode: 55.09 (#54)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA Terminal-Bench 4.0: 55.56 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Briefcase: 42.42 (#67)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Briefcase - Analytical Quality Elo: 1491.89 (#64)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Briefcase - DOCX - Rubric Pass Rate: 40.24 (#84)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Briefcase - PDF - Rubric Pass Rate: 38.79 (#89)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Briefcase - PPTX - Rubric Pass Rate: 45.71 (#44)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Briefcase - Presentation Elo: 1463.42 (#34)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Briefcase - XLSX - Rubric Pass Rate: 41.82 (#68)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Accuracy: 49.97 (#62)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Hallucination Rate: 38.04 (#73)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Business: 23.3 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Health: 14.0 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Humanities &amp; Social Sciences: 30.8 (#31)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Law: 25.5 (#52)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Science, Engineering &amp; Mathematics: 16.1 (#64)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE): 71.7 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - C: 83.0 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Dart: 54.0 (#46)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Go: 78.0 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - HTML: 80.0 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Java: 54.0 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 79.09 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Julia: 64.0 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 78.0 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - PHP: 80.0 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Python: 80.0 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - R: 54.0 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Rust: 78.0 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Swift: 64.0 (#48)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 91.11 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AI Chess Leaderboard (Continuation): 1610.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1337.0 (#49)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on Artificial Analysis Intelligence Index: 48.11 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AutomationBench-AA: 60.17 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AutomationBench-AA - Finance - Raw Objective Completion: 87.12 (#31)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AutomationBench-AA - HR - Raw Objective Completion: 73.49 (#41)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AutomationBench-AA - Marketing - Raw Objective Completion: 80.5 (#64)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AutomationBench-AA - Operations - Raw Objective Completion: 95.65 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AutomationBench-AA - Sales - Raw Objective Completion: 85.68 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AutomationBench-AA - Strict: 33.03 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on AutomationBench-AA - Support - Raw Objective Completion: 87.43 (#53)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on BenchLM: 75.9 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on Bug Hunt Bench: 40.7 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on Bug Hunt Bench - LMS: 22.7 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 18.0 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on Design Arena (Game Dev): 1328.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on Design Arena (Website): 1309.0 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on Explicit Edit Benchmark: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on LiveBench Browser: 93.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on LiveBench Coding: 90.8 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on LiveBench Data Analysis: 82.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on LiveBench Design Layout: 82.3 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on LiveBench Long Horizon: 48.6 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on LiveBench Reasoning: 88.7 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on LiveBench Research: 73.9 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on Maze-Bench: 79.63 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on Mercor APEX: 68.3 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on NonoBench: 86.666666666667 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-5.6 Launch - Internal Research Debugging Evaluation: 76.39 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-5.6 Launch - KernelGen 1P: 45.34 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Adversarial User Simulations - Emotional Reliance: 97.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Adversarial User Simulations - Mental Health: 99.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Adversarial User Simulations - Self-Harm: 88.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Chat Plugins: 92.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Age-Restricted Actions: 82.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Non-Violent Wrongdoing: 97.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Self-Harm: 92.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Sensitive Personal Data: 74.7 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Violent Wrongdoing: 88.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Human Red-Teaming Chat Plugins: 93.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Human Red-Teaming Codex: 95.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Biology Refusal - Benign: 98.2 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Biology Refusal - Dual Use: 96.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Biology Refusal - Severe: 99.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Extremism: 93.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Gore: 88.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Hate: 98.2 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Non-Violent Illicit Behavior: 99.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Self-Harm: 97.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Sexual: 95.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Sexual/Minors: 96.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Violent Illicit Behavior: 97.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Cyber Safety - Production Chat: 0.98 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Cyber Safety - Semi-Synthetic Agentic Environments: 0.973 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Cyber Safety - Synthetic Agentic Environments: 1.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - HealthBench: 55.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - HealthBench Hard: 31.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - HealthBench Professional: 62.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Erotic: 99.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Extremism: 92.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Hate: 98.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Self-Harm: 98.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Multimodal Troubleshooting Virology: 52.54 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - U18 - Age-Restricted Goods and Dangerous Activities: 92.6 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - U18 - Eating Disorders: 93.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - U18 - Emotional Reliance: 94.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - U18 - Self-Harm: 99.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - U18 - Sexual Content: 97.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Broken Search Tool Non-Disclosure: 1.83 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Reviewer Bypass Attempts: 0.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on RuneBench: 6.648699907871 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Terra&lt;/strong&gt; on VoxelBench: 2136.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on CrueltyBench: 30.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Design Arena (SVG): 1322.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Terminal-Bench 2.1: 57.678 (#43)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (11)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;LLMEval-Logic Hard Sub-Q&lt;/strong&gt;: Claude Opus 4.6 (Thinking) (76.6) beat Doubao-1.5-Thinking-Pro (Thinking) by 67.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLMEval-Logic Formalization Fixed&lt;/strong&gt;: GPT-5.4 Pro (Non-reasoning) (60.16) beat Doubao-1.5-Thinking-Pro (Thinking) by 50.36&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLMEval-Logic Formalization Free&lt;/strong&gt;: Gemini 3.1 Pro (Preview) (Thinking) (45.12) beat Doubao-1.5-Thinking-Pro (Thinking) by 35.42&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coarena&lt;/strong&gt;: GPT-6.1 Terra (1170.6) beat GPT-6.1 Sol by 26.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-5.6 Launch - Internal Research Debugging Evaluation&lt;/strong&gt;: GPT-6 Astra (78.05) beat GPT-5.6 Sol by 9.75&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-5.6 Launch - KernelGen 1P&lt;/strong&gt;: GPT-6 Astra (66.72) beat GPT-5.6 Sol by 5.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWT-Bench Verified&lt;/strong&gt;: DPIAgent + GPT-5 (with test selection) (87.5) beat PatchTwin+Guard + GPT-5 by 4.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Codex) - Cloud Security&lt;/strong&gt;: GPT-6.1 Sol (68.1) beat GPT-6 Sol by 2.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coarena - Task Completion&lt;/strong&gt;: GPT-6.1 Terra (96.4) beat Claude Haiku 5.5 by 2.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Codex) - API Security&lt;/strong&gt;: GPT-6 Astra (81.5) beat GPT-5.6 Sol by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate&lt;/strong&gt;: GPT-6.1 Terra (Max) (92.74) beat Claude Opus 5.5 (Max, Default Fallback) by 1.49&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-27

=== DAILY ===
NEW BENCHMARKS (135)
  - NTBench - Dolly: Closed QA Fixed Target ASR@1 (Attack success rate at 1 (%)): leader GPT-5.6 Sol (Non-reasoning) (0.0), 7 models
      NTBench paper v1: fixed reference-guided prompt-injection sets shared across seven victims,</summary></entry><entry><title>The Aggregate Digest — 2026-10-26</title><id>https://theaggregate.ai/digest/2026-10-26</id><updated>2026-10-26T08:42:21.085515+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (139)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;GoBench - Track 1&lt;/strong&gt; (Elo): leader GPT-6 Astra (Max) (2567.645678703034), 32 models&lt;br&gt;&lt;span&gt;Models play complete 9x9 Go games against a calibrated KataGo ladder through multi-turn APIs without tools. Each turn supplies the board, rules and legal moves; illegal outputs are retried. Elo ratings are fitted from game outcomes with a random-player anchor and color-advantage adjustment. Reasoning settings and dated model releases stay separate; the coding-engine and tutorial tracks are separate experiments.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 1 Completion&lt;/strong&gt; (Completion (%)): leader Claude Opus 5 (42.17), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t1. Metric: Completion (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 1 Level 1&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (79.17), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t1. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 1 Level 2&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (49.17), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t1. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 1 Level 3&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (37.5), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t1. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 1 Level 4&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (25.0), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t1. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 1 Level 5&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (20.0), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t1. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 2 Completion&lt;/strong&gt; (Completion (%)): leader Claude Opus 5 (47.67), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t2. Metric: Completion (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 2 Level 1&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (85.83), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t2. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 2 Level 2&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (56.67), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t2. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 2 Level 3&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (39.17), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t2. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 2 Level 4&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (33.33), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t2. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CLE - Claude Code Trial 2 Level 5&lt;/strong&gt; (Cumulative pass rate (%)): leader Claude Opus 5 (23.33), 2 models&lt;br&gt;&lt;span&gt;Full Continual Learning Exam: one serial stream of 120 professional tasks, five levels each (600 levels), in fixed order. Trial 1 permits one submission per level; Trial 2 permits one feedback-guided revision. Unreached levels fail. Lk is cumulative clearance through level k. DeepSeek V4 Flash grades the human-authored rubrics. Harness/provider default effort is unspecified. These are the full exam results, not the 20-task CLE Lite release. Harness: Claude Code; trial: t2. Metric: Cumulative pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wordsmith Model Bench v1 - Overall&lt;/strong&gt; (Editorial score (0–100)): leader Claude Fable 5.1 (High) (78.856), 12 models&lt;br&gt;&lt;span&gt;Wordsmith October 15, 2026 v1 pilot on unpublished manuscripts. Cross-maker AI judges never grade their own maker. Exact source effort settings are retained. Scores reward correct useful editorial work; manuscripts, prompts and checklist are withheld. Metric: Editorial score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wordsmith Model Bench v1 - Craft&lt;/strong&gt; (Editorial score (0–100)): leader Claude Fable 5.1 (High) (69.089), 12 models&lt;br&gt;&lt;span&gt;Wordsmith October 15, 2026 v1 pilot on unpublished manuscripts. Cross-maker AI judges never grade their own maker. Exact source effort settings are retained. Scores reward correct useful editorial work; manuscripts, prompts and checklist are withheld. Metric: Editorial score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wordsmith Model Bench v1 - Line&lt;/strong&gt; (Editorial score (0–100)): leader GPT-6.1 Sol (High) (77.807), 12 models&lt;br&gt;&lt;span&gt;Wordsmith October 15, 2026 v1 pilot on unpublished manuscripts. Cross-maker AI judges never grade their own maker. Exact source effort settings are retained. Scores reward correct useful editorial work; manuscripts, prompts and checklist are withheld. Metric: Editorial score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wordsmith Model Bench v1 - Polish&lt;/strong&gt; (Editorial score (0–100)): leader Gemini 4 Argon (High) (99.478), 12 models&lt;br&gt;&lt;span&gt;Wordsmith October 15, 2026 v1 pilot on unpublished manuscripts. Cross-maker AI judges never grade their own maker. Exact source effort settings are retained. Scores reward correct useful editorial work; manuscripts, prompts and checklist are withheld. Metric: Editorial score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wordsmith Model Bench v1 - Sentence-level Craft&lt;/strong&gt; (Editorial score (0–100)): leader Claude Fable 5.1 (High) (69.636), 12 models&lt;br&gt;&lt;span&gt;Wordsmith October 15, 2026 v1 pilot on unpublished manuscripts. Cross-maker AI judges never grade their own maker. Exact source effort settings are retained. Scores reward correct useful editorial work; manuscripts, prompts and checklist are withheld. Metric: Editorial score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wordsmith Model Bench v1 - Structural Craft&lt;/strong&gt; (Editorial score (0–100)): leader Gemini 4 Argon (High) (70.978), 12 models&lt;br&gt;&lt;span&gt;Wordsmith October 15, 2026 v1 pilot on unpublished manuscripts. Cross-maker AI judges never grade their own maker. Exact source effort settings are retained. Scores reward correct useful editorial work; manuscripts, prompts and checklist are withheld. Metric: Editorial score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - FARGO paper-aligned query core&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (52.59552674697766), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Approximate-MIPS Query Stopping&lt;/strong&gt; (Task score (0–100)): leader GLM-5.3 (Max) (86.73015686324783), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Reusable Library Candidates&lt;/strong&gt; (Task score (0–100)): leader DeepSeek V4.1 Flash (Max) (19.35320481841905), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Batch Local Counting&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (High) (73.13487561111772), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - BBQ bitmap traversal&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (42.21291774304742), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Binary Fuse Filter Construction and Lookup&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (57.1623176023505), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Bounded EVM Path Prediction&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (High) (81.47386835295775), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - CKKS Precision Propagation at 2,048 Slots&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (97.05498326851828), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Fixed-Length Clustered Sparse Multiplication&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (66.8685854926921), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Search Strategy for Concolic Testing&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (15.47452771540353), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU DBSCAN Kernel Optimization&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (72.85673763507015), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Robust Image Tokenizer&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (High) (78.29088429863549), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Closed-loop action sampling&lt;/strong&gt; (Task score (0–100)): leader GLM-5.3 (Max) (59.809290278961925), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Learned Transformer Layer Mapping&lt;/strong&gt; (Task score (0–100)): leader GPT-6 Astra (xHigh) (70.0), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Exact Sequence Alignment&lt;/strong&gt; (Task score (0–100)): leader GPT-6 Astra (xHigh) (66.75940344970795), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - E-Graph Invariant Restoration&lt;/strong&gt; (Task score (0–100)): leader DeepSeek V4.1 Flash (1017) (61.85321099172882), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - EVM State Checkpoints&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (88.52889123095609), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Fast KV Compaction&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (68.1994), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - OCR Distillation Trajectory Sampling&lt;/strong&gt; (Task score (0–100)): leader GLM-5.3 (Max) (86.13122171945702), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Access-Pattern Rewriting&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (91.04413190447389), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Graph Mining Method Selection&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (91.95225525774885), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Hypergraph Core Numbers&lt;/strong&gt; (Task score (0–100)): leader DeepSeek V4.1 Flash (1017) (59.07397688392045), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU IVF-PQ Search Kernel Optimization&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (80.53513660468923), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Join-Tree Rewriting&lt;/strong&gt; (Task score (0–100)): leader Kimi K2.7 Code (Medium) (12.20594396959284), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - KAMI FP64 Block GEMM — trusted batched harness adaptation&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (High) (23.03119467585492), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU K-Means Kernel Optimization&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (53.78825934310897), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU Brute-Force k-NN Kernel Optimization&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (51.805760800273), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Latent Objective Recovery&lt;/strong&gt; (Task score (0–100)): leader GPT-6 Astra (xHigh) (99.4927), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - LRR&lt;/strong&gt; (Task score (0–100)): leader DeepSeek V4.1 Flash (1017) (73.97), 12 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Structured-LWE Public Witness Recovery&lt;/strong&gt; (Task score (0–100)): leader GPT-6 Astra (xHigh) (84.0), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - MetaOpt Combined-TE Gap Search&lt;/strong&gt; (Task score (0–100)): leader GPT-6 Astra (xHigh) (52.04945410251058), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Go1 Joystick Policy Training&lt;/strong&gt; (Task score (0–100)): leader GPT-6 Astra (xHigh) (50.56363844359285), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Future Video Sampling&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (81.40141228006945), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Exact DBSCAN Connectivity&lt;/strong&gt; (Task score (0–100)): leader GLM-5.3 (Max) (75.96834539618399), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - In-place Integer Redistribution&lt;/strong&gt; (Task score (0–100)): leader GPT-6.1 Sol (xHigh) (51.30557766134971), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - ParButterfly Exact Counting&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (High) (56.32137474200954), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Persistent Treap Update Engine&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (40.168442143631346), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Polar-Factor Approximation&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (High) (71.70463310601987), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - SQL Counterexample Search&lt;/strong&gt; (Task score (0–100)): leader GLM-5.3 (Max) (97.65875765065286), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Progent Privilege Control&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (High) (56.8798), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Private Set List Decoding&lt;/strong&gt; (Task score (0–100)): leader GPT-6.1 Sol (xHigh) (58.1894064195512), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Multi-turn Trajectory Batches&lt;/strong&gt; (Task score (0–100)): leader GPT-6.1 Sol (xHigh) (18.458333333333336), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Differentiable Population Statistics&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (7.157393783742427), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Residual State Advancement&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (27.037849770404986), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Register-aware Branch Prediction&lt;/strong&gt; (Task score (0–100)): leader Claude Opus 5.5 (Max) (31.267441350421088), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Dynamic GPU Allocation&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (High) (47.28588453469387), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Sparse Video Attention Policy&lt;/strong&gt; (Task score (0–100)): leader GPT-6 Astra (xHigh) (48.58633407346494), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - GPU vLLM Serving Optimization&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (High) (24.21685514139821), 14 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Pass Rate&lt;/strong&gt; (Pass rate (%)): leader Claude Opus 5.5 (Max) (58.3), 13 models&lt;br&gt;&lt;span&gt;FrontierCS 2 native website aggregate snapshot over 48 research engineering tasks. These values are independently published by the website, not recomputed from the separately updated formal-run snapshot. Each source-native coding harness and reviewed Table 3 effort setting is retained; DeepSeek 1017 effort is unspecified.  FCI uses Kimi K2.7 as its zero anchor; the index is not a percentage or a human score. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Pass Rate&lt;/strong&gt; (Pass rate (%)): leader Gemini 4 Argon (54.2), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 native website aggregate snapshot over 48 research engineering tasks. These values are independently published by the website, not recomputed from the separately updated formal-run snapshot. Each source-native coding harness and reviewed Table 3 effort setting is retained; DeepSeek 1017 effort is unspecified.  Separate Antigravity CLI; effort unspecified. Metric: Pass rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 - Frontier Capability Index&lt;/strong&gt; (FCI): leader Claude Opus 5.5 (Max) (53.3), 13 models&lt;br&gt;&lt;span&gt;FrontierCS 2 native website aggregate snapshot over 48 research engineering tasks. These values are independently published by the website, not recomputed from the separately updated formal-run snapshot. Each source-native coding harness and reviewed Table 3 effort setting is retained; DeepSeek 1017 effort is unspecified.  FCI uses Kimi K2.7 as its zero anchor; the index is not a percentage or a human score. Metric: FCI.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Frontier Capability Index&lt;/strong&gt; (FCI): leader Gemini 4 Argon (45.2), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 native website aggregate snapshot over 48 research engineering tasks. These values are independently published by the website, not recomputed from the separately updated formal-run snapshot. Each source-native coding harness and reviewed Table 3 effort setting is retained; DeepSeek 1017 effort is unspecified.  Separate Antigravity CLI; effort unspecified. Metric: FCI.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - FARGO paper-aligned query core&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (52.11058966636611), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Approximate-MIPS Query Stopping&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (81.42833586820102), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Reusable Library Candidates&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (18.64444132319428), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Batch Local Counting&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (15.310672252844668), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - BBQ bitmap traversal&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (37.2653934069011), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Binary Fuse Filter Construction and Lookup&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (55.33129023145147), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Bounded EVM Path Prediction&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (81.44027469141948), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - CKKS Precision Propagation at 2,048 Slots&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (95.88501762360848), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Fixed-Length Clustered Sparse Multiplication&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (65.20431121843026), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Search Strategy for Concolic Testing&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (13.1037839064819), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU DBSCAN Kernel Optimization&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (68.84074949349228), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Robust Image Tokenizer&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (73.52176330582107), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Closed-loop action sampling&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (52.039369409257155), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Learned Transformer Layer Mapping&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (70.0), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Exact Sequence Alignment&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (66.92783490595173), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - E-Graph Invariant Restoration&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (61.62438289285781), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - EVM State Checkpoints&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (100.0), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Fast KV Compaction&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (0.0), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - OCR Distillation Trajectory Sampling&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (49.886877828054295), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Access-Pattern Rewriting&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (91.01987343783895), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Graph Mining Method Selection&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (91.99094310262464), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Hypergraph Core Numbers&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (59.517940492645224), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU IVF-PQ Search Kernel Optimization&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (78.472797464762), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Join-Tree Rewriting&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (12.106911513692946), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - KAMI FP64 Block GEMM — trusted batched harness adaptation&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (22.313538458199503), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU K-Means Kernel Optimization&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (51.93069820373417), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU Brute-Force k-NN Kernel Optimization&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (45.41661130784086), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Latent Objective Recovery&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (99.5064), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - LRR&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (14.54), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Structured-LWE Public Witness Recovery&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (14.0), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - MetaOpt Combined-TE Gap Search&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (51.14868500818264), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Go1 Joystick Policy Training&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (49.62192905094747), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Future Video Sampling&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (79.39257646600404), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Exact DBSCAN Connectivity&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (77.4266832531289), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - In-place Integer Redistribution&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (51.45850689981266), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - ParButterfly Exact Counting&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (56.254849492913166), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Persistent Treap Update Engine&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (21.84427397857361), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Polar-Factor Approximation&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (62.929186449533994), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - SQL Counterexample Search&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (80.2915139697341), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Progent Privilege Control&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (56.2725), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Private Set List Decoding&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (62.76438619424235), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Multi-turn Trajectory Batches&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (12.020833333333332), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Differentiable Population Statistics&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (5.751003595975148), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Residual State Advancement&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (33.35741965335753), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Register-aware Branch Prediction&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (31.01248675654361), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Dynamic GPU Allocation&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (47.134258204741045), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - Sparse Video Attention Policy&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (40.606621454212096), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FrontierCS 2 Antigravity - GPU vLLM Serving Optimization&lt;/strong&gt; (Task score (0–100)): leader Gemini 4 Argon (17.743844660906397), 1 models&lt;br&gt;&lt;span&gt;FrontierCS 2 selected formal runs on 48 research engineering tasks, with each source-native coding harness. Paper v1 Table 3 fixes the effort settings; the subsequently added DeepSeek 1017 row leaves effort unspecified. Superseded attempts are not selected by score. Native bounded task scores are distinct from human-reference margins and from pass/fail verdicts. Explicit failed-run zeroes remain zero; unreported scores remain missing. Separate Antigravity CLI run added October 25; effort is not disclosed and is not inherited from the Pi run. Metric: Task score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual go&lt;/strong&gt; (Strict tuple F1 (%)): leader GPT-5.6 Sol (High) (7.1), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual chess&lt;/strong&gt; (Strict tuple F1 (%)): leader GPT-5.6 Sol (High) (47.3), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual gomoku&lt;/strong&gt; (Strict tuple F1 (%)): leader GPT-5.6 Sol (High) (19.6), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual doudizhu&lt;/strong&gt; (Strict tuple F1 (%)): leader Kimi K3 (21.4), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual cs2&lt;/strong&gt; (Strict tuple F1 (%)): leader GPT-5.6 Sol (High) (17.8), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual dota2&lt;/strong&gt; (Strict tuple F1 (%)): leader GPT-5.6 Sol (High) (2.2), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual sc2&lt;/strong&gt; (Strict tuple F1 (%)): leader GPT-5.6 Sol (High) (3.4), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual gfootball&lt;/strong&gt; (Strict tuple F1 (%)): leader GPT-5.6 Sol (High) (0.8), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual badminton&lt;/strong&gt; (Strict tuple F1 (%)): leader GPT-5.6 Sol (High) (2.9), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Visual Overall&lt;/strong&gt; (Strict tuple F1 (%)): leader GPT-5.6 Sol (High) (13.2), 10 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. One request, 1 FPS capped at 256 uniform frames; no tools. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code go&lt;/strong&gt; (Strict tuple F1 (%)): leader Claude Opus 5 (High) (49.0), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code chess&lt;/strong&gt; (Strict tuple F1 (%)): leader Qwen 3.8 27B (Thinking) (64.3), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code gomoku&lt;/strong&gt; (Strict tuple F1 (%)): leader Claude Opus 5 (High) (32.7), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code doudizhu&lt;/strong&gt; (Strict tuple F1 (%)): leader Claude Opus 5 (High) (29.1), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code cs2&lt;/strong&gt; (Strict tuple F1 (%)): leader Claude Opus 5 (High) (84.5), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code dota2&lt;/strong&gt; (Strict tuple F1 (%)): leader Claude Opus 5 (High) (30.0), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code sc2&lt;/strong&gt; (Strict tuple F1 (%)): leader Claude Opus 5 (High) (14.6), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code gfootball&lt;/strong&gt; (Strict tuple F1 (%)): leader Claude Opus 5 (High) (92.9), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code badminton&lt;/strong&gt; (Strict tuple F1 (%)): leader Claude Opus 5 (High) (63.2), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Arena - Agentic Claude Code Overall&lt;/strong&gt; (Strict tuple F1 (%)): leader Claude Opus 5 (High) (51.0), 6 models&lt;br&gt;&lt;span&gt;Video-Arena: 180 cases, 20 per game. Strict tuple F1 requires every non-temporal field to match and timestamps within 3 seconds; overall is the mean of nine game means. Paper v1 Appendix B.3 fixes high effort for OpenAI, Anthropic and Gemini; Kimi/MiniMax use unspecified provider defaults. Qwen thinking modes remain distinct. Full MP4, Claude Code harness, at most 200 turns and 120 minutes per case. Metric: Strict tuple F1 (%).&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (6)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AA CritPt v2: 78.86 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AA CritPt v2: 83.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on AA CritPt v2: 81.71 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PRISM (1C:Enterprise) - Algorithmic Tasks (A): 93.0 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PRISM (1C:Enterprise) - Platform Tasks (B): 85.0 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ProphetArena: 0.9478 (#11)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (8)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;LLMEval-Logic Hard&lt;/strong&gt;: Doubao-1.5-Thinking-Pro (Thinking) (93.7) beat Gemini 3.1 Pro (Preview) (Thinking) by 56.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLMEval-Logic Base&lt;/strong&gt;: Doubao-1.5-Thinking-Pro (Thinking) (100.0) beat Seed 2.0 Pro (Thinking) by 24.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NatureBench&lt;/strong&gt;: GPT-6.1 Sol (36.0) beat Claude Opus 5 (HELIX) by 9.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;cua-speedrun - OSWorld 2.0 (52 Tasks)&lt;/strong&gt;: Claude Opus 5.5 (Claude Code, Max) (85.06374640475177) beat GPT-6 Astra (xHigh) by 8.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;cua-speedrun - OSWorld (50 Tasks)&lt;/strong&gt;: Claude Opus 5.5 (Claude Code, xHigh) (95.61904360085809) beat GPT-6 Astra (xHigh) by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Coding Daily (OpenCode) - Total&lt;/strong&gt;: Kimi K3.1 Pro (Max) (63.8) beat DeepSeek V4.1 Flash (1017) (Max) by 2.83&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ForecastBench&lt;/strong&gt;: model-4 (69.7) beat fire-hedgehog by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Coding Daily (OpenCode) - Laravel Code Quality&lt;/strong&gt;: Kimi K3.1 Pro (Max) (18.55) beat Kimi K3 (High) by 0.15&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-26

=== DAILY ===
NEW BENCHMARKS (139)
  - GoBench - Track 1 (Elo): leader GPT-6 Astra (Max) (2567.645678703034), 32 models
      Models play complete 9x9 Go games against a calibrated KataGo ladder through multi-turn APIs without tools. Each turn supplies the board, r</summary></entry><entry><title>The Aggregate Digest — 2026-10-25</title><id>https://theaggregate.ai/digest/2026-10-25</id><updated>2026-10-25T07:18:13.391283+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (51)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Average v2 (Tasks v6, German v2)&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (57.03), 14 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - English Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6.1 Sol (78.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Arabic Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (51.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Bengali Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (46.8), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Cantonese Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (51.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Czech Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.8 Flash (64.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Dutch Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (66.8), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Farsi Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (44.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - French Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (71.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - German Task v2&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (65.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Hebrew Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6 Astra (57.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Hindi Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (52.8), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Indonesian Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (63.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Italian Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (71.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Japanese Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (63.6), 15 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Khmer Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6 Astra (38.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Korean Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6.1 Sol (57.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Malay Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (40.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Marathi Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (50.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Polish Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6 Astra (44.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Portuguese Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (74.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Romanian Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6 Astra (54.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Russian Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (79.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Simplified Mandarin Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (69.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Spanish Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (69.2), 15 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Swedish Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (63.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Tagalog Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.8 Flash (57.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Thai Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (41.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Traditional Mandarin Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (76.8), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Turkish Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (68.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Vietnamese Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (57.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PointerBench - Sheets&lt;/strong&gt; (Success rate (%)): leader Claude Opus 5 (98.8), 45 models&lt;br&gt;&lt;span&gt;PointerBench: 1,500 screenshot/instruction pairs, 500 per suite, with 1024×768 absolute pixel coordinates. One JSON prompt; success requires a point inside the target or box coverage ≥0.90 and precision ≥0.70. Disabled uses 512 output tokens; Low and Medium use 2,048, with a 1,024-token reasoning allowance for Medium. Outputs are unmodified; malformed, filtered and exhausted responses fail without retry. Explicit thinking settings are retained. Success rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PointerBench - Text&lt;/strong&gt; (Success rate (%)): leader Claude Fable 5 (Medium) (69.6), 45 models&lt;br&gt;&lt;span&gt;PointerBench: 1,500 screenshot/instruction pairs, 500 per suite, with 1024×768 absolute pixel coordinates. One JSON prompt; success requires a point inside the target or box coverage ≥0.90 and precision ≥0.70. Disabled uses 512 output tokens; Low and Medium use 2,048, with a 1,024-token reasoning allowance for Medium. Outputs are unmodified; malformed, filtered and exhausted responses fail without retry. Explicit thinking settings are retained. Success rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PointerBench - Pro&lt;/strong&gt; (Success rate (%)): leader Claude Fable 5 (Medium) (82.4), 45 models&lt;br&gt;&lt;span&gt;PointerBench: 1,500 screenshot/instruction pairs, 500 per suite, with 1024×768 absolute pixel coordinates. One JSON prompt; success requires a point inside the target or box coverage ≥0.90 and precision ≥0.70. Disabled uses 512 output tokens; Low and Medium use 2,048, with a 1,024-token reasoning allowance for Medium. Outputs are unmodified; malformed, filtered and exhausted responses fail without retry. Explicit thinking settings are retained. Success rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PointerBench - Average&lt;/strong&gt; (Success rate (%)): leader Claude Fable 5 (Medium) (83.2), 45 models&lt;br&gt;&lt;span&gt;PointerBench: 1,500 screenshot/instruction pairs, 500 per suite, with 1024×768 absolute pixel coordinates. One JSON prompt; success requires a point inside the target or box coverage ≥0.90 and precision ≥0.70. Disabled uses 512 output tokens; Low and Medium use 2,048, with a 1,024-token reasoning allowance for Medium. Outputs are unmodified; malformed, filtered and exhausted responses fail without retry. Explicit thinking settings are retained. Success rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code All-task Average&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (63.3), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Success&lt;/strong&gt; (Score (%)): leader GPT-5.6 Sol (100.0), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Successful Average&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (66.0), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Content&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (59.9), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Presentation&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (90.3), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Citation&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (59.9), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Legal Index&lt;/strong&gt; (Legal Index (%)): leader Muse Wonder Full (Max) (77.863), 28 models&lt;br&gt;&lt;span&gt;Vals AI legal capability composite: Legal Research Bench, HLAB Vals v1.1 and Tax Agent Bench, weighted by 2022 US law-office revenue across practice areas. Published cohort and weighted accuracy, not a simple mean of component leaderboard scores.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Guided All 14 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.855), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Guided All 14 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Guided Radiology 9 Images&lt;/strong&gt; (Mean Dice (0–1)): leader Claude Opus 5.5 (Medium) (0.828), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Guided Radiology 9 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Guided Radiology without Adrenal 8 Images&lt;/strong&gt; (Mean Dice (0–1)): leader Claude Opus 5.5 (Medium) (0.818), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Guided Radiology without Adrenal 8 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Guided Legs Mouse Ant 5 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.912), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Guided Legs Mouse Ant 5 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Unguided All 14 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.842), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Unguided All 14 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Unguided Radiology 9 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.815), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Unguided Radiology 9 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Unguided Radiology without Adrenal 8 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.804), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Unguided Radiology without Adrenal 8 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Unguided Legs Mouse Ant 5 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.892), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Unguided Legs Mouse Ant 5 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GlobeBench - Task Success&lt;/strong&gt; (Task Success (%)): leader GPT-6 Astra (49.57), 20 models&lt;br&gt;&lt;span&gt;GlobeBench - Task Success. GlobeBench measures next-observation prediction on 1,632 execution-grounded text-world items across five domains and 19 skills. All 20 models retain the full item denominator; provider refusals and empty or errored responses count as failures. Scoring uses wm-hybrid-deterministic-llm/2, wm-grader-1.5, and response-judge/4: deterministic certification first, then GPT-5.6 Terra with GPT-5.6 Sol for missing or uncertain judgments. The primary repository binds every model key and serving route. Most subjects nominally use high effort; Qwen 3.8 Flash Next nominally uses xhigh. Nonusable responses were retried, sometimes with lower effort, larger budgets, or different routes. Completed answers and refusals were never resampled. These are mixed serving settings, not a claim that every item used one effort. Reported metric: Task Success (%).&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (11)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on GPQA Diamond (BenchmarkList): 93.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Legal Agent Benchmark (BenchmarkList Mixed Public and Held-Out): 19.1 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on BrowseComp: 90.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on GPQA Diamond (BenchmarkList): 93.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on BrowseComp: 91.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 100.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GPQA Diamond (BenchmarkList): 96.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 99.82 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 100.0 (#3)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (9)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Design Arena (Slides)&lt;/strong&gt;: Claude Opus 5.5 (1309.0) beat playyy by 15.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SloBench - Slovene SuperGLUE&lt;/strong&gt;: Cvetka-27B-tiv (82.9) beat Gizela-31B-it-vit-think by 3.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BALSAM - Program Execution&lt;/strong&gt;: NatiqLM-Version3 (98.45) beat Grok 4.1 Fast (Reasoning) by 2.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPQA Diamond (BenchmarkList)&lt;/strong&gt;: GPT-6 Astra (Max) (96.1) beat Qwen 3.8 2.4T A95B (xHigh) by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;IFBench&lt;/strong&gt;: Mercury 3 (Medium) (76.84) beat LFM2.5-24B-A2B by 2.15&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OCRBench-V2 (zh)&lt;/strong&gt;: Qwen3.8-Flash-Next (68.3) beat Qwen 3.7 Plus by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BrowseComp&lt;/strong&gt;: Kimi K3.1 Pro (Max) (92.1) beat GPT-6 Astra (Max) by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BALSAM - Summarization&lt;/strong&gt;: soual-3 (59.02) beat Nojoom-AQL-120B by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Legal Agent Benchmark (BenchmarkList Mixed Public and Held-Out)&lt;/strong&gt;: Kimi K3.1 Pro (Max) (23.7) beat Claude Opus 5 by 0.12&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Weekly&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (52)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Average v2 (Tasks v6, German v2)&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (57.03), 14 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - English Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6.1 Sol (78.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Arabic Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (51.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Bengali Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (46.8), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Cantonese Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (51.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Czech Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.8 Flash (64.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Dutch Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (66.8), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Farsi Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (44.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - French Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (71.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - German Task v2&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (65.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Hebrew Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6 Astra (57.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Hindi Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (52.8), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Indonesian Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (63.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Italian Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (71.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Japanese Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (63.6), 15 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Khmer Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6 Astra (38.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Korean Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6.1 Sol (57.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Malay Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (40.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Marathi Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (50.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Polish Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6 Astra (44.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Portuguese Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (74.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Romanian Task v6&lt;/strong&gt; (Exact Match (%)): leader GPT-6 Astra (54.0), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Russian Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (79.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Simplified Mandarin Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (69.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Spanish Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (69.2), 15 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Swedish Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (63.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Tagalog Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.8 Flash (57.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Thai Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (41.2), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Traditional Mandarin Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (76.8), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Turkish Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (68.4), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle MultiLoKo - Vietnamese Task v6&lt;/strong&gt; (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (57.6), 16 models&lt;br&gt;&lt;span&gt;Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items per language where evaluated. Average v2 requires numeric results for all 31 tasks. Language boards retain their reported rows. These revisions are distinct from the historical EM tasks. Urdu retains the existing reviewed scoring hold.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PointerBench - Sheets&lt;/strong&gt; (Success rate (%)): leader Claude Opus 5 (98.8), 45 models&lt;br&gt;&lt;span&gt;PointerBench: 1,500 screenshot/instruction pairs, 500 per suite, with 1024×768 absolute pixel coordinates. One JSON prompt; success requires a point inside the target or box coverage ≥0.90 and precision ≥0.70. Disabled uses 512 output tokens; Low and Medium use 2,048, with a 1,024-token reasoning allowance for Medium. Outputs are unmodified; malformed, filtered and exhausted responses fail without retry. Explicit thinking settings are retained. Success rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PointerBench - Text&lt;/strong&gt; (Success rate (%)): leader Claude Fable 5 (Medium) (69.6), 45 models&lt;br&gt;&lt;span&gt;PointerBench: 1,500 screenshot/instruction pairs, 500 per suite, with 1024×768 absolute pixel coordinates. One JSON prompt; success requires a point inside the target or box coverage ≥0.90 and precision ≥0.70. Disabled uses 512 output tokens; Low and Medium use 2,048, with a 1,024-token reasoning allowance for Medium. Outputs are unmodified; malformed, filtered and exhausted responses fail without retry. Explicit thinking settings are retained. Success rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PointerBench - Pro&lt;/strong&gt; (Success rate (%)): leader Claude Fable 5 (Medium) (82.4), 45 models&lt;br&gt;&lt;span&gt;PointerBench: 1,500 screenshot/instruction pairs, 500 per suite, with 1024×768 absolute pixel coordinates. One JSON prompt; success requires a point inside the target or box coverage ≥0.90 and precision ≥0.70. Disabled uses 512 output tokens; Low and Medium use 2,048, with a 1,024-token reasoning allowance for Medium. Outputs are unmodified; malformed, filtered and exhausted responses fail without retry. Explicit thinking settings are retained. Success rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PointerBench - Average&lt;/strong&gt; (Success rate (%)): leader Claude Fable 5 (Medium) (83.2), 45 models&lt;br&gt;&lt;span&gt;PointerBench: 1,500 screenshot/instruction pairs, 500 per suite, with 1024×768 absolute pixel coordinates. One JSON prompt; success requires a point inside the target or box coverage ≥0.90 and precision ≥0.70. Disabled uses 512 output tokens; Low and Medium use 2,048, with a 1,024-token reasoning allowance for Medium. Outputs are unmodified; malformed, filtered and exhausted responses fail without retry. Explicit thinking settings are retained. Success rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code All-task Average&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (63.3), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Success&lt;/strong&gt; (Score (%)): leader GPT-5.6 Sol (100.0), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Successful Average&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (66.0), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Content&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (59.9), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Presentation&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (90.3), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Video-Centered Research - Claude Code Citation&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (59.9), 12 models&lt;br&gt;&lt;span&gt;Video-Centered DeepResearch: 50 tasks with a frozen video, query, report and tool contract; Claude Code native harness, 64K tokens and 500 iterations, up to three complete attempts and 10 hours per attempt. All-task average assigns zero to failed reports; quality dimensions and successful average use each model’s successful subset. GPT-5.6 Sol provides semantic and visual judging alongside deterministic browser checks. Generation effort is undisclosed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Legal Index&lt;/strong&gt; (Legal Index (%)): leader Muse Wonder Full (Max) (77.863), 28 models&lt;br&gt;&lt;span&gt;Vals AI legal capability composite: Legal Research Bench, HLAB Vals v1.1 and Tax Agent Bench, weighted by 2022 US law-office revenue across practice areas. Published cohort and weighted accuracy, not a simple mean of component leaderboard scores.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Router Index&lt;/strong&gt; (Accuracy (%)): leader Gemini 4 Argon (69.237), 27 models&lt;br&gt;&lt;span&gt;Vals Router Index v1.0: five routing systems evaluated on eight Vals Index v2.2 tasks with the same weights and limits. Default pools and quality/cost preferences; refusals always fail with no backup, unlike some standard Index runs. System identities and default routing settings are retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Guided All 14 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.855), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Guided All 14 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Guided Radiology 9 Images&lt;/strong&gt; (Mean Dice (0–1)): leader Claude Opus 5.5 (Medium) (0.828), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Guided Radiology 9 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Guided Radiology without Adrenal 8 Images&lt;/strong&gt; (Mean Dice (0–1)): leader Claude Opus 5.5 (Medium) (0.818), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Guided Radiology without Adrenal 8 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Guided Legs Mouse Ant 5 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.912), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Guided Legs Mouse Ant 5 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Unguided All 14 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.842), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Unguided All 14 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Unguided Radiology 9 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.815), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Unguided Radiology 9 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Unguided Radiology without Adrenal 8 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.804), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Unguided Radiology without Adrenal 8 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;nnInteractive Agents - Unguided Legs Mouse Ant 5 Images&lt;/strong&gt; (Mean Dice (0–1)): leader GPT-6.1 Sol (Medium) (0.892), 2 models&lt;br&gt;&lt;span&gt;nnInteractive Agents - Unguided Legs Mouse Ant 5 Images. The nnInteractive paper evaluates its public v1.0 segmentation weights on one RTX 4090 through an MCP tool interface. Codex 0.162 uses GPT-6.1 Sol at medium reasoning effort, and Claude Code 2.1 uses Claude Opus 5.5 at medium effort. Guided and unguided requests are separate conditions. There are 51 targets in 14 images: 25 radiology targets and 26 targets in leg, mouse, and ant images. Dice is averaged over targets within each image, then over images. Agent scores average three independent repetitions; reported uncertainty describes variation between repetitions. The radiology cohort without adrenal_ct contains eight images. Reported metric: Mean Dice (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GlobeBench - Task Success&lt;/strong&gt; (Task Success (%)): leader GPT-6 Astra (49.57), 20 models&lt;br&gt;&lt;span&gt;GlobeBench - Task Success. GlobeBench measures next-observation prediction on 1,632 execution-grounded text-world items across five domains and 19 skills. All 20 models retain the full item denominator; provider refusals and empty or errored responses count as failures. Scoring uses wm-hybrid-deterministic-llm/2, wm-grader-1.5, and response-judge/4: deterministic certification first, then GPT-5.6 Terra with GPT-5.6 Sol for missing or uncertain judgments. The primary repository binds every model key and serving route. Most subjects nominally use high effort; Qwen 3.8 Flash Next nominally uses xhigh. Nonusable responses were retried, sometimes with lower effort, larger budgets, or different routes. Completed answers and refusals were never resampled. These are mixed serving settings, not a claim that every item used one effort. Reported metric: Task Success (%).&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (160)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra Pro&lt;/strong&gt; — Aggregate Rating 1982, #4&lt;ul&gt;&lt;li&gt;KernelBench CUDA: 0.2068 (#10/32)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; — Aggregate Rating 1979, #5&lt;ul&gt;&lt;li&gt;Design Arena (Slides): 1309.0 (#1/13)&lt;/li&gt;&lt;li&gt;HANDBOOK.md Agents: 52.3 (#1/6)&lt;/li&gt;&lt;li&gt;CAIS Risk Index: 27.1 (#1/55)&lt;/li&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 100.0 (#1/253)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 87.64 (#1/76)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 65.152 (#1/55)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 91.333 (#1/13)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 70.208 (#1/20)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 86.986 (#1/64)&lt;/li&gt;&lt;li&gt;MedXpertQA MM (Natomy) - Basic Science: 89.52 (#2/11)&lt;/li&gt;&lt;li&gt;...and 39 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; — Aggregate Rating 1977, #6&lt;ul&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Multimodal Troubleshooting Virology: 63.11 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - ProtocolQA Open-Ended: 45.37 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Tacit Knowledge and Troubleshooting: 92.55 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - TroubleshootingBench: 63.46 (#1/4)&lt;/li&gt;&lt;li&gt;GPQA Diamond (BenchmarkList): 96.1 (#1/6)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 65.09 (#1/22)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench Science: 62.857 (#1/42)&lt;/li&gt;&lt;li&gt;Vals AI ReverseEngBench: 56.87 (#1/35)&lt;/li&gt;&lt;li&gt;Vals AI MysteryMechanism: 53.153 (#1/28)&lt;/li&gt;&lt;li&gt;NYT Connections Extended: 98.4 (#2/133)&lt;/li&gt;&lt;li&gt;...and 26 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; — Aggregate Rating 1971, #7&lt;ul&gt;&lt;li&gt;OpenAI GPT-6 Astra Launch - Internal Computer Use Safety Benchmark: 4.32 (#1/3)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Sol &amp; Luna Launch - Reviewer Bypass Attempts: 0.0 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Biology Refusal - Benign: 98.2 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Cyber Safety - Production Chat: 0.987 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - HealthBench: 58.5 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Challenging Prompts - Non-Violent Illicit Behavior: 100.0 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Challenging Prompts - Self-Harm: 99.4 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - Challenging Prompts - Sexual: 98.7 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - U18 - Age-Restricted Goods and Dangerous Activities: 93.4 (#1/4)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra System Card - U18 - Eating Disorders: 96.2 (#1/4)&lt;/li&gt;&lt;li&gt;...and 77 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; — Aggregate Rating 1963, #8&lt;ul&gt;&lt;li&gt;Harvey LAB-AA - Antitrust &amp; Competition: 95.29 (#1/28)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - Environmental &amp; ESG: 95.58 (#1/28)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - Healthcare &amp; Life Sciences: 97.87 (#1/28)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - Intellectual Property: 97.98 (#1/28)&lt;/li&gt;&lt;li&gt;Mercor APEX: 82.4 (#1/62)&lt;/li&gt;&lt;li&gt;APEX v1: 81.1 (#1/7)&lt;/li&gt;&lt;li&gt;MedXpertQA MM (Natomy): 90.5 (#1/14)&lt;/li&gt;&lt;li&gt;MedXpertQA MM (Natomy) - Diagnosis: 89.91 (#1/11)&lt;/li&gt;&lt;li&gt;MedXpertQA MM (Natomy) - Treatment: 90.18 (#1/11)&lt;/li&gt;&lt;li&gt;MedXpertQA MM (Natomy) - Basic Science: 92.92 (#1/11)&lt;/li&gt;&lt;li&gt;...and 78 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; — Aggregate Rating 1949, #9&lt;ul&gt;&lt;li&gt;Vals AI MMMU: 90.636 (#1/93)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 76.67 (#1/77)&lt;/li&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 100.0 (#2/253)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 70.786 (#2/98)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 49.194 (#2/76)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 74.899 (#2/50)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 61.25 (#2/20)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 63.333 (#3/13)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 55.288 (#3/85)&lt;/li&gt;&lt;li&gt;Mercor APEX: 68.6 (#4/62)&lt;/li&gt;&lt;li&gt;...and 19 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; — Aggregate Rating 1939, #10&lt;ul&gt;&lt;li&gt;LisanBench: 1.0 (#1/211)&lt;/li&gt;&lt;li&gt;CritPt: 32.9 (#1/275)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 33.978 (#1/26)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 73.748 (#3/50)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 66.787 (#4/27)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 21.667 (#4/81)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 47.292 (#5/20)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 53.365 (#5/85)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 4.0): 57.1 (#6/28)&lt;/li&gt;&lt;li&gt;Finance Agent v2: 59.1 (#6/55)&lt;/li&gt;&lt;li&gt;...and 17 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5.5&lt;/strong&gt; — Aggregate Rating 1934, #11&lt;ul&gt;&lt;li&gt;Epoch AI - Lmca: 68.26 (#1/197)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 70.854 (#1/27)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 89.06 (#1/6)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 100.0 (#1/6)&lt;/li&gt;&lt;li&gt;KernelBench CUDA: 0.6248 (#1/32)&lt;/li&gt;&lt;li&gt;Vals AI ProofBench: 100.0 (#1/53)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 92.392 (#1/119)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 69.825 (#1/79)&lt;/li&gt;&lt;li&gt;Vals AI BioMysteryBench: 81.111 (#1/28)&lt;/li&gt;&lt;li&gt;Mercor APEX: 75.5 (#2/62)&lt;/li&gt;&lt;li&gt;...and 30 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro&lt;/strong&gt; — Aggregate Rating 1933, #12&lt;ul&gt;&lt;li&gt;AA Terminal-Bench 4.0: 67.17 (#1/226)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 89.44 (#1/228)&lt;/li&gt;&lt;li&gt;BrowseComp: 92.1 (#1/3)&lt;/li&gt;&lt;li&gt;Coarena: 1104.8 (#2/19)&lt;/li&gt;&lt;li&gt;AutomationBench-AA: 73.3 (#2/229)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Strict: 45.97 (#2/229)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 92.99 (#2/228)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Marketing - Raw Objective Completion: 93.18 (#2/229)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Sales - Raw Objective Completion: 91.83 (#2/229)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Total: 59.99 (#2/22)&lt;/li&gt;&lt;li&gt;...and 91 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5&lt;/strong&gt; — Aggregate Rating 1926, #13&lt;ul&gt;&lt;li&gt;Vals AI Public Benefits Bench: 76.928 (#1/50)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 89.884 (#2/93)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 55.288 (#2/85)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 28.533 (#3/26)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 45.548 (#4/76)&lt;/li&gt;&lt;li&gt;Lech Mazur Writing: 3.372 (#5/59)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 84.644 (#5/76)&lt;/li&gt;&lt;li&gt;Mercor APEX: 65.8 (#6/62)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 73.556 (#6/77)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 82.273 (#6/64)&lt;/li&gt;&lt;li&gt;...and 13 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder&lt;/strong&gt; — Aggregate Rating 1908, #15&lt;ul&gt;&lt;li&gt;Vals AI CyberBench: 80.357 (#1/47)&lt;/li&gt;&lt;li&gt;Coarena: 1091.5 (#3/19)&lt;/li&gt;&lt;li&gt;LisanBench: 0.7529 (#3/211)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 64.877 (#6/27)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 70.23 (#6/50)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 26.55 (#6/26)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 50.481 (#6/85)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 62.552 (#6/79)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 45.625 (#7/20)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 4.0): 52.0 (#8/28)&lt;/li&gt;&lt;li&gt;...and 31 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Sol&lt;/strong&gt; — Aggregate Rating 1907, #16&lt;ul&gt;&lt;li&gt;Vals AI CyberBench: 77.976 (#2/47)&lt;/li&gt;&lt;li&gt;OpenAI GPT-6 Astra Launch - Internal Computer Use Safety Benchmark: 17.39 (#3/3)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 95.0 (#4/172)&lt;/li&gt;&lt;li&gt;Epoch AI - Frontiermath Erdos: 1.47 (#5/13)&lt;/li&gt;&lt;li&gt;EKI Estonian LLM - Overall: 69.16 (#5/90)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 83.146 (#6/76)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 81.909 (#7/64)&lt;/li&gt;&lt;li&gt;Vals AI ReverseEngBench: 30.534 (#7/35)&lt;/li&gt;&lt;li&gt;Vals AI BioMysteryBench: 74.815 (#7/28)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 44.998 (#8/22)&lt;/li&gt;&lt;li&gt;...and 21 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5&lt;/strong&gt; — Aggregate Rating 1902, #17&lt;ul&gt;&lt;li&gt;Code Migration: 55.065 (#2/33)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 89.306 (#3/93)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 90.352 (#4/119)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 70.433 (#5/50)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 73.666 (#5/77)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 68.919 (#8/98)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 49.519 (#8/85)&lt;/li&gt;&lt;li&gt;Vals AI ProofBench: 95.0 (#9/53)&lt;/li&gt;&lt;li&gt;Mercor APEX: 63.6 (#10/62)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 80.524 (#10/76)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Sol&lt;/strong&gt; — Aggregate Rating 1886, #19&lt;ul&gt;&lt;li&gt;Vals AI Web Search: 49.076695 (#1/9)&lt;/li&gt;&lt;li&gt;Code Migration: 52.916 (#3/33)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 85.768 (#3/76)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 76.31 (#4/47)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 91.167 (#5/45)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 23.833 (#5/13)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 88.844 (#6/93)&lt;/li&gt;&lt;li&gt;Vals AI ReverseEngBench: 30.534 (#6/35)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 72.339 (#7/77)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 41.481 (#9/22)&lt;/li&gt;&lt;li&gt;...and 14 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.7 Flash&lt;/strong&gt; — Aggregate Rating 1871, #22&lt;ul&gt;&lt;li&gt;FutureEval: 11.83 (#4/106)&lt;/li&gt;&lt;li&gt;Mercor APEX: 67.8 (#5/62)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 88.96 (#5/93)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 59.042 (#7/81)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 71.326 (#13/77)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 67.833 (#14/45)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 77.528 (#14/76)&lt;/li&gt;&lt;li&gt;Vals AI ReverseEngBench: 4.58 (#17/35)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench Science: 7.143 (#20/42)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 8.75 (#26/81)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.8 Flash&lt;/strong&gt; — Aggregate Rating 1868, #23&lt;ul&gt;&lt;li&gt;Kaggle WWTP Engineering: 100.0 (#1/42)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 61.435 (#2/81)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 89.075 (#4/93)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 18.833 (#6/13)&lt;/li&gt;&lt;li&gt;Epoch AI - Frontiermath Erdos: 0.0 (#7/13)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 81.273 (#8/76)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 72.196 (#8/77)&lt;/li&gt;&lt;li&gt;Vals AI MysteryMechanism: 36.486 (#8/28)&lt;/li&gt;&lt;li&gt;Mercor APEX: 64.3 (#9/62)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 35.76 (#13/22)&lt;/li&gt;&lt;li&gt;...and 23 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash (1017)&lt;/strong&gt; — Aggregate Rating 1863, #25&lt;ul&gt;&lt;li&gt;LLM Stats (AutomationBench; v1.0.6 method cohort): 60.5 (#1/4)&lt;/li&gt;&lt;li&gt;LLM Stats (BabyVision): 92.5 (#1/16)&lt;/li&gt;&lt;li&gt;LLM Stats (NL2Repo): 68.2 (#1/25)&lt;/li&gt;&lt;li&gt;LLM Stats (ZEROBench): 51.0 (#1/15)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Total: 60.97 (#1/22)&lt;/li&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, No Tools): 44.0 (#2/52)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Laravel Code Quality: 17.98 (#3/22)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - React-TS Code Quality: 18.17 (#3/22)&lt;/li&gt;&lt;li&gt;LLM Stats (Chartography): 83.0 (#4/6)&lt;/li&gt;&lt;li&gt;LLM Stats (DeepSWE 1.1): 77.0 (#4/49)&lt;/li&gt;&lt;li&gt;...and 211 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.7&lt;/strong&gt; — Aggregate Rating 1862, #26&lt;ul&gt;&lt;li&gt;HANDBOOK.md Agents: 44.0 (#4/6)&lt;/li&gt;&lt;li&gt;Epoch AI - Frontiermath Erdos: 0.0 (#6/13)&lt;/li&gt;&lt;li&gt;KernelBench CUDA: 0.2201 (#8/32)&lt;/li&gt;&lt;li&gt;Argo-Bench: 38.59 (#12/61)&lt;/li&gt;&lt;li&gt;Argo-Bench - Solved: 20.48 (#12/61)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 86.175 (#12/119)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 75.701 (#12/64)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 58.564 (#13/27)&lt;/li&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 99.45 (#13/253)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 42.917 (#13/20)&lt;/li&gt;&lt;li&gt;...and 24 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.6&lt;/strong&gt; — Aggregate Rating 1861, #27&lt;ul&gt;&lt;li&gt;Mercor APEX: 65.3 (#7/62)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 48.077 (#10/85)&lt;/li&gt;&lt;li&gt;Vals AI BioMysteryBench: 72.222 (#10/28)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 15.833 (#11/81)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 38.094 (#12/22)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 5.833 (#12/13)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 78.277 (#13/76)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 74.029 (#13/64)&lt;/li&gt;&lt;li&gt;Vals AI MysteryMechanism: 30.631 (#13/28)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench Science: 11.429 (#14/42)&lt;/li&gt;&lt;li&gt;...and 15 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.3&lt;/strong&gt; — Aggregate Rating 1859, #28&lt;ul&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 22.917 (#3/81)&lt;/li&gt;&lt;li&gt;HANDBOOK.md Agents: 33.9 (#6/6)&lt;/li&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 100.0 (#7/253)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 58.901 (#8/81)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 35.974 (#13/76)&lt;/li&gt;&lt;li&gt;Mercor APEX: 58.6 (#14/62)&lt;/li&gt;&lt;li&gt;Coarena: 977.7 (#15/19)&lt;/li&gt;&lt;li&gt;Coarena - Task Completion: 63.5 (#16/19)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 71.722 (#19/64)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 82.859 (#21/119)&lt;/li&gt;&lt;li&gt;...and 13 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5&lt;/strong&gt; — Aggregate Rating 1848, #30&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 73.202 (#1/67)&lt;/li&gt;&lt;li&gt;Code Migration: 45.161 (#5/33)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 66.238 (#7/54)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 88.266 (#9/93)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 68.76 (#10/98)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 8.333 (#10/13)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 76.404 (#16/76)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 45.161 (#18/79)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 85.56 (#19/172)&lt;/li&gt;&lt;li&gt;Vals AI ReverseEngBench: 3.817 (#20/35)&lt;/li&gt;&lt;li&gt;...and 11 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Max&lt;/strong&gt; — Aggregate Rating 1845, #32&lt;ul&gt;&lt;li&gt;Mercor APEX: 63.3 (#11/62)&lt;/li&gt;&lt;li&gt;Finance Agent v2: 50.6 (#28/55)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3&lt;/strong&gt; — Aggregate Rating 1844, #33&lt;ul&gt;&lt;li&gt;SimpleVQA: 71.9 (#2/3)&lt;/li&gt;&lt;li&gt;JobBench: 54.3 (#3/6)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 44.167 (#8/20)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 75.238 (#8/47)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 10.5 (#9/13)&lt;/li&gt;&lt;li&gt;Vals AI BioMysteryBench: 72.593 (#9/28)&lt;/li&gt;&lt;li&gt;Epoch AI - Frontiermath Erdos: 0.0 (#10/13)&lt;/li&gt;&lt;li&gt;Toolathlon: 76.5 (#11/54)&lt;/li&gt;&lt;li&gt;NL2Repo: 58.0 (#12/54)&lt;/li&gt;&lt;li&gt;Vals AI ProofBench: 87.0 (#12/53)&lt;/li&gt;&lt;li&gt;...and 17 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash&lt;/strong&gt; — Aggregate Rating 1840, #35&lt;ul&gt;&lt;li&gt;Vectara Hallucination Leaderboard: 95.8 (#6/111)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 46.458 (#6/20)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 73.69 (#10/47)&lt;/li&gt;&lt;li&gt;Kaggle Open Benchmarks - AIME 2025: 96.67 (#15/66)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 71.999 (#16/64)&lt;/li&gt;&lt;li&gt;Vals AI BioMysteryBench: 67.778 (#16/28)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 29.65 (#17/22)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 16.378 (#17/26)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 45.623 (#17/79)&lt;/li&gt;&lt;li&gt;Kaggle Open Benchmarks - MMLU: 92.47 (#18/59)&lt;/li&gt;&lt;li&gt;...and 30 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.8&lt;/strong&gt; — Aggregate Rating 1837, #37&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 70.037 (#2/67)&lt;/li&gt;&lt;li&gt;Code Migration: 47.25 (#4/33)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 69.912 (#5/98)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 11.833 (#8/13)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 68.133 (#13/50)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 69.373 (#15/77)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 47.25 (#16/79)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 71.945 (#17/64)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 86.59 (#19/93)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 9.583 (#21/81)&lt;/li&gt;&lt;li&gt;...and 8 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3&lt;/strong&gt; — Aggregate Rating 1835, #38&lt;ul&gt;&lt;li&gt;JobBench: 61.4 (#2/6)&lt;/li&gt;&lt;li&gt;GPQA Diamond (BenchmarkList): 91.7 (#6/6)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 49.038 (#9/85)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 59.729 (#10/27)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 39.698 (#10/76)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 68.539 (#10/50)&lt;/li&gt;&lt;li&gt;NL2Repo: 58.0 (#13/54)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 68.444 (#13/45)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 38.889 (#13/55)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 19.994 (#13/26)&lt;/li&gt;&lt;li&gt;...and 19 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M3.1-Flash&lt;/strong&gt; — Aggregate Rating 1834, #39&lt;ul&gt;&lt;li&gt;LLM Stats (AutomationBench v1.0.6): 62.8 (#1/6)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Finance - Raw Objective Completion: 93.72 (#2/229)&lt;/li&gt;&lt;li&gt;FoodTruckBench: 63464.21 (#2/58)&lt;/li&gt;&lt;li&gt;LLM Stats (BabyVision): 91.0 (#2/16)&lt;/li&gt;&lt;li&gt;AutomationBench-AA: 72.52 (#3/229)&lt;/li&gt;&lt;li&gt;LLM Stats (PostTrainBench): 39.3 (#3/9)&lt;/li&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 96.7 (#3/22)&lt;/li&gt;&lt;li&gt;LLM Stats (NL2Repo): 61.2 (#4/25)&lt;/li&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, No Tools): 43.4 (#5/52)&lt;/li&gt;&lt;li&gt;LiveBench Browser: 92.0 (#6/53)&lt;/li&gt;&lt;li&gt;...and 116 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Terra&lt;/strong&gt; — Aggregate Rating 1833, #40&lt;ul&gt;&lt;li&gt;Vals AI IOI: 87.611 (#7/45)&lt;/li&gt;&lt;li&gt;Vals AI SkillsBench: 58.895 (#11/35)&lt;/li&gt;&lt;li&gt;Code Migration: 36.445 (#12/33)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 47.804 (#13/79)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 72.345 (#14/64)&lt;/li&gt;&lt;li&gt;Vals AI ReverseEngBench: 9.542 (#14/35)&lt;/li&gt;&lt;li&gt;Mercor APEX: 58.2 (#15/62)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 77.528 (#15/76)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench Science: 10.0 (#18/42)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 72.084 (#18/47)&lt;/li&gt;&lt;li&gt;...and 15 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy4 preview&lt;/strong&gt; — Aggregate Rating 1833, #41&lt;ul&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 91.82 (#78/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 5.5&lt;/strong&gt; — Aggregate Rating 1831, #43&lt;ul&gt;&lt;li&gt;Coarena - Task Completion: 94.0 (#1/19)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass^3: 78.12 (#2/6)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 100.0 (#2/6)&lt;/li&gt;&lt;li&gt;MultiChallenge: 52.48 (#2/3)&lt;/li&gt;&lt;li&gt;IFBench: 54.65 (#3/3)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 90.444 (#3/119)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 75.476 (#6/47)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 78.676 (#8/64)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 43.958 (#9/20)&lt;/li&gt;&lt;li&gt;Vals AI MysteryMechanism: 35.135 (#10/28)&lt;/li&gt;&lt;li&gt;...and 55 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.6-Pro&lt;/strong&gt; — Aggregate Rating 1831, #44&lt;ul&gt;&lt;li&gt;EKI Estonian LLM - Overall: 39.95 (#49/90)&lt;/li&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 93.45 (#69/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5 Instant (June 2026)&lt;/strong&gt; — Aggregate Rating 1825, #46&lt;ul&gt;&lt;li&gt;MLCR-AA: 31.67 (#14/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 81.11 (#23/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 35.56 (#39/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 5 Preview&lt;/strong&gt; — Aggregate Rating 1823, #50&lt;ul&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 92.18 (#74/253)&lt;/li&gt;&lt;li&gt;Wolfram LLM Benchmarking Project: 63.0 (#82/605)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.5&lt;/strong&gt; — Aggregate Rating 1822, #51&lt;ul&gt;&lt;li&gt;Code Migration: 36.596 (#11/33)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 6.333 (#11/13)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 12.917 (#12/81)&lt;/li&gt;&lt;li&gt;Mercor APEX: 56.2 (#18/62)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 71.012 (#20/47)&lt;/li&gt;&lt;li&gt;Vals AI ReverseEngBench: 0.763 (#29/35)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 65.246 (#31/64)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 67.79 (#32/76)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 40.556 (#34/45)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 36.596 (#35/79)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.7&lt;/strong&gt; — Aggregate Rating 1822, #52&lt;ul&gt;&lt;li&gt;Vals AI MortgageTax: 70.27 (#3/98)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 68.539 (#3/67)&lt;/li&gt;&lt;li&gt;Code Migration: 43.878 (#8/33)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 85.549 (#24/93)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 43.878 (#26/79)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 63.728 (#27/77)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 68.539 (#31/76)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 6.667 (#32/81)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 64.225 (#32/64)&lt;/li&gt;&lt;li&gt;Mercor APEX: 49.2 (#33/62)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.6 Flash&lt;/strong&gt; — Aggregate Rating 1820, #53&lt;ul&gt;&lt;li&gt;Vals AI MMMU: 88.382 (#7/93)&lt;/li&gt;&lt;li&gt;Code Migration: 30.928 (#15/33)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 56.296 (#21/81)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 73.783 (#21/76)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 67.608 (#24/98)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 65.411 (#25/77)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench Science: 4.286 (#28/42)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 59.926 (#37/64)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 35.056 (#39/45)&lt;/li&gt;&lt;li&gt;Mercor APEX: 46.9 (#40/62)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro (0813)&lt;/strong&gt; — Aggregate Rating 1819, #54&lt;ul&gt;&lt;li&gt;SWE Atlas: 66.0 (#3/65)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 72.857 (#14/47)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 17.533 (#16/26)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 52.974 (#19/27)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 70.103 (#22/64)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 82.297 (#23/119)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 51.611 (#25/45)&lt;/li&gt;&lt;li&gt;Vals AI ReverseEngBench: 1.908 (#25/35)&lt;/li&gt;&lt;li&gt;Finance Agent v2: 50.4 (#29/55)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 7.5 (#29/81)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.1&lt;/strong&gt; — Aggregate Rating 1818, #55&lt;ul&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 20.0 (#5/81)&lt;/li&gt;&lt;li&gt;Vals AI Time Horizon Index: 1.667 (#13/13)&lt;/li&gt;&lt;li&gt;Code Migration: 31.115 (#14/33)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 57.207 (#17/81)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 86.59 (#18/93)&lt;/li&gt;&lt;li&gt;FutureEval: 9.48 (#22/106)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 61.299 (#30/50)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 69.288 (#30/76)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 66.136 (#37/98)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 37.981 (#39/85)&lt;/li&gt;&lt;li&gt;...and 8 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.8 2.4T A95B&lt;/strong&gt; — Aggregate Rating 1816, #56&lt;ul&gt;&lt;li&gt;JobBench: 52.7 (#4/6)&lt;/li&gt;&lt;li&gt;Toolathlon: 72.5 (#28/54)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark Open&lt;/strong&gt; — Aggregate Rating 1815, #57&lt;ul&gt;&lt;li&gt;LLM Stats (MCP Atlas): 89.2 (#1/39)&lt;/li&gt;&lt;li&gt;JobBench: 61.8 (#1/6)&lt;/li&gt;&lt;li&gt;SEAL - Professional Reasoning Benchmark - Legal: 59.76 (#2/43)&lt;/li&gt;&lt;li&gt;LLM Stats (Job Bench): 61.8 (#2/9)&lt;/li&gt;&lt;li&gt;LLM Stats (ZEROBench): 50.0 (#2/15)&lt;/li&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, With Tools): 61.0 (#3/33)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 61.336 (#3/81)&lt;/li&gt;&lt;li&gt;LLM Stats (SimpleVQA): 73.6 (#4/17)&lt;/li&gt;&lt;li&gt;LLM Stats (WorldVQA): 49.4 (#4/7)&lt;/li&gt;&lt;li&gt;SEAL - MultiChallenge: 74.2 (#5/34)&lt;/li&gt;&lt;li&gt;...and 49 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Flash&lt;/strong&gt; — Aggregate Rating 1814, #58&lt;ul&gt;&lt;li&gt;FutureEval: 7.61 (#38/106)&lt;/li&gt;&lt;li&gt;EKI Estonian LLM - Overall: 34.46 (#61/90)&lt;/li&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 88.64 (#102/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 (2026-03-05)&lt;/strong&gt; — Aggregate Rating 1809, #59&lt;ul&gt;&lt;li&gt;AIME: 96.667 (#5/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 96.667 (#5/96)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 58.427 (#12/67)&lt;/li&gt;&lt;li&gt;Code Migration: 34.984 (#13/33)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 87.514 (#15/93)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 63.773 (#16/54)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 68.323 (#17/98)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 68.596 (#27/64)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 34.984 (#37/79)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 67.421 (#47/119)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Pro (Preview)&lt;/strong&gt; — Aggregate Rating 1808, #60&lt;ul&gt;&lt;li&gt;AIME: 98.125 (#1/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 98.125 (#1/96)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 67.416 (#5/67)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 69.396 (#6/98)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 88.208 (#10/93)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 64.845 (#12/54)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 27.383 (#20/22)&lt;/li&gt;&lt;li&gt;Code Migration: 17.313 (#23/33)&lt;/li&gt;&lt;li&gt;Vals AI MysteryMechanism: 20.721 (#23/28)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 51.833 (#24/45)&lt;/li&gt;&lt;li&gt;...and 17 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Luna&lt;/strong&gt; — Aggregate Rating 1808, #61&lt;ul&gt;&lt;li&gt;OpenAI GPT-6 Astra Launch - Internal Computer Use Safety Benchmark: 13.7 (#2/3)&lt;/li&gt;&lt;li&gt;IFBench: 61.82 (#2/3)&lt;/li&gt;&lt;li&gt;MultiChallenge: 51.49 (#3/3)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 76.25 (#5/47)&lt;/li&gt;&lt;li&gt;EKI Estonian LLM - Overall: 61.98 (#15/90)&lt;/li&gt;&lt;li&gt;KernelBench CUDA: 0.1708 (#15/32)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 68.515 (#16/77)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 86.67 (#18/172)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 37.5 (#18/20)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 28.379 (#19/22)&lt;/li&gt;&lt;li&gt;...and 29 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5&lt;/strong&gt; — Aggregate Rating 1808, #62&lt;ul&gt;&lt;li&gt;Vals AI MortgageTax: 70.032 (#4/98)&lt;/li&gt;&lt;li&gt;Code Migration: 44.392 (#7/33)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 72.067 (#15/64)&lt;/li&gt;&lt;li&gt;Vals AI ProofBench: 77.0 (#17/53)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 74.532 (#19/76)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 66.035 (#20/50)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 13.822 (#22/26)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 66.316 (#22/77)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 70.298 (#22/47)&lt;/li&gt;&lt;li&gt;Mercor APEX: 54.5 (#23/62)&lt;/li&gt;&lt;li&gt;...and 16 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.5 Flash&lt;/strong&gt; — Aggregate Rating 1806, #63&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 67.416 (#4/67)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 88.266 (#8/93)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 57.861 (#14/81)&lt;/li&gt;&lt;li&gt;Code Migration: 26.745 (#17/33)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 68.124 (#19/98)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 74.157 (#20/76)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench Science: 5.714 (#21/42)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 63.548 (#28/77)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 59.472 (#33/50)&lt;/li&gt;&lt;li&gt;Vals AI ProofBench: 31.0 (#37/53)&lt;/li&gt;&lt;li&gt;...and 8 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.6&lt;/strong&gt; — Aggregate Rating 1804, #65&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 58.427 (#11/67)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 68.522 (#15/98)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 83.873 (#29/93)&lt;/li&gt;&lt;li&gt;Mercor APEX: 46.3 (#41/62)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 57.573 (#57/119)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.2&lt;/strong&gt; — Aggregate Rating 1802, #67&lt;ul&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 25.417 (#1/81)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 60.599 (#4/81)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 68.471 (#11/50)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 32.798 (#17/76)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 86.127 (#22/93)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 43.75 (#22/85)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 69.524 (#23/47)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 79.1 (#27/119)&lt;/li&gt;&lt;li&gt;FutureEval: 8.55 (#29/106)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 69.663 (#29/76)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Pro (Preview)&lt;/strong&gt; — Aggregate Rating 1797, #68&lt;ul&gt;&lt;li&gt;Vals AI MATH 500: 96.4 (#1/60)&lt;/li&gt;&lt;li&gt;AIME: 96.68 (#4/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 96.68 (#4/96)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 69.078 (#7/98)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 93.927 (#7/75)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 87.514 (#14/93)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 55.056 (#15/67)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 53.055 (#42/54)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.3 Codex&lt;/strong&gt; — Aggregate Rating 1795, #70&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 64.045 (#6/67)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 53.833 (#22/45)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 61.767 (#54/119)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercury 3&lt;/strong&gt; — Aggregate Rating 1793, #71&lt;ul&gt;&lt;li&gt;IFBench: 76.84 (#1/3)&lt;/li&gt;&lt;li&gt;MultiChallenge: 60.35 (#1/3)&lt;/li&gt;&lt;li&gt;LLM Stats (Tau3 Banking): 30.24 (#4/15)&lt;/li&gt;&lt;li&gt;LLM Stats (DeepSearchQA): 71.33 (#12/12)&lt;/li&gt;&lt;li&gt;LLM Stats (GDPval-AA): 1237.2 (#12/16)&lt;/li&gt;&lt;li&gt;LLM Stats (Multi-Challenge): 60.35 (#14/33)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - React-TS Code Quality: 15.83 (#17/22)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Laravel Code Quality: 16.08 (#18/22)&lt;/li&gt;&lt;li&gt;DeepSearchQA: 71.33 (#19/22)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Total: 41.68 (#21/22)&lt;/li&gt;&lt;li&gt;...and 16 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling 3.1 Flash&lt;/strong&gt; — Aggregate Rating 1793, #72&lt;ul&gt;&lt;li&gt;WideResearch (BenchLM): 83.3 (#2/16)&lt;/li&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, No Tools): 37.44 (#8/52)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 30.56 (#61/172)&lt;/li&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 93.18 (#71/253)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 37.78 (#73/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 12.22 (#97/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3 Flash&lt;/strong&gt; — Aggregate Rating 1793, #73&lt;ul&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 83.3 (#10/21)&lt;/li&gt;&lt;li&gt;OCRBench-V2 (en): 61.8 (#13/32)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 57.85 (#15/81)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 37.374 (#16/55)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 16.033 (#18/26)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 45.192 (#19/85)&lt;/li&gt;&lt;li&gt;OCRBench-V2 (zh): 56.8 (#21/32)&lt;/li&gt;&lt;li&gt;Vals AI (Vals Index): 50.861 (#22/27)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 52.5 (#23/45)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 86.012 (#23/93)&lt;/li&gt;&lt;li&gt;...and 16 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.6-Flash&lt;/strong&gt; — Aggregate Rating 1793, #74&lt;ul&gt;&lt;li&gt;EKI Estonian LLM - Overall: 30.59 (#67/90)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 26.67 (#82/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 30.56 (#97/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 11.67 (#105/172)&lt;/li&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 83.0 (#153/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Luna&lt;/strong&gt; — Aggregate Rating 1792, #75&lt;ul&gt;&lt;li&gt;Code Migration: 44.55 (#6/33)&lt;/li&gt;&lt;li&gt;Vals AI SkillsBench: 60.447 (#7/35)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 22.594 (#8/26)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 79.026 (#11/76)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 73.631 (#11/47)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 61.778 (#16/45)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 67.116 (#18/77)&lt;/li&gt;&lt;li&gt;Vals AI Science Index: 25.298 (#21/22)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 44.55 (#22/79)&lt;/li&gt;&lt;li&gt;Vals AI ProofBench: 60.0 (#24/53)&lt;/li&gt;&lt;li&gt;...and 18 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.8-Flash-Next&lt;/strong&gt; — Aggregate Rating 1792, #76&lt;ul&gt;&lt;li&gt;OCRBench-V2 (zh): 68.3 (#1/32)&lt;/li&gt;&lt;li&gt;OCRBench-V2 (en): 67.9 (#6/32)&lt;/li&gt;&lt;li&gt;MLCR-AA: 20.56 (#45/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 54.44 (#59/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 28.89 (#68/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.2&lt;/strong&gt; — Aggregate Rating 1788, #77&lt;ul&gt;&lt;li&gt;Code Migration: 37.87 (#10/33)&lt;/li&gt;&lt;li&gt;Finance Agent v2: 49.7 (#30/55)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 7.083 (#30/81)&lt;/li&gt;&lt;li&gt;Vals AI ReverseEngBench: 0.0 (#32/35)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 67.79 (#33/76)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 62.605 (#34/64)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 37.87 (#34/79)&lt;/li&gt;&lt;li&gt;Mercor APEX: 45.2 (#43/62)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 31.25 (#48/85)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 63.961 (#53/119)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.5 (20251101)&lt;/strong&gt; — Aggregate Rating 1786, #78&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 58.427 (#10/67)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 68.68 (#14/98)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 82.948 (#36/93)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.7 Max&lt;/strong&gt; — Aggregate Rating 1783, #79&lt;ul&gt;&lt;li&gt;MLCR-AA - Accuracy: 33.89 (#82/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 14.44 (#85/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 21.67 (#116/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash (0731)&lt;/strong&gt; — Aggregate Rating 1781, #81&lt;ul&gt;&lt;li&gt;Vals AI CyberBench: 72.084 (#17/47)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench Science: 4.286 (#27/42)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 8.333 (#27/81)&lt;/li&gt;&lt;li&gt;Vals AI ProofBench: 56.0 (#28/53)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 18.687 (#32/55)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 28.069 (#33/76)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 38.631 (#33/79)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 74.736 (#38/119)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 57.291 (#39/64)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 32.722 (#40/45)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4&lt;/strong&gt; — Aggregate Rating 1780, #83&lt;ul&gt;&lt;li&gt;Mercor APEX: 52.4 (#29/62)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Beam 1.0&lt;/strong&gt; — Aggregate Rating 1779, #84&lt;ul&gt;&lt;li&gt;LLM Stats (LongBench v2): 67.5 (#1/19)&lt;/li&gt;&lt;li&gt;Featherbench: 100.0 (#1/33)&lt;/li&gt;&lt;li&gt;LLM Stats (AIME 2026): 97.7 (#2/27)&lt;/li&gt;&lt;li&gt;LLM Stats (Tau3 Banking): 36.1 (#2/15)&lt;/li&gt;&lt;li&gt;LLM Stats (CritPT): 16.3 (#3/7)&lt;/li&gt;&lt;li&gt;LLM Stats (DeepSearchQA): 80.5 (#8/12)&lt;/li&gt;&lt;li&gt;LLM Stats (MCP Atlas): 80.8 (#8/39)&lt;/li&gt;&lt;li&gt;Humanity's Last Exam (Self-Reported, No Tools): 36.0 (#10/52)&lt;/li&gt;&lt;li&gt;LLM Stats (SWE-bench Multilingual): 78.3 (#10/50)&lt;/li&gt;&lt;li&gt;Featherbench - Rubric Quality: 9.0 (#11/33)&lt;/li&gt;&lt;li&gt;...and 15 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3 Flash (Preview)&lt;/strong&gt; — Aggregate Rating 1777, #88&lt;ul&gt;&lt;li&gt;Vals AI AIME: 95.625 (#8/96)&lt;/li&gt;&lt;li&gt;AIME: 95.625 (#9/96)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 93.309 (#10/75)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 68.72 (#11/98)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 87.63 (#13/93)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 51.685 (#18/67)&lt;/li&gt;&lt;li&gt;Code Migration: 6.391 (#31/33)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 55.842 (#33/54)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 33.83 (#51/64)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 53.933 (#53/76)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.7 Code&lt;/strong&gt; — Aggregate Rating 1769, #92&lt;ul&gt;&lt;li&gt;Mercor APEX: 37.6 (#51/62)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4.6&lt;/strong&gt; — Aggregate Rating 1767, #94&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 59.551 (#8/67)&lt;/li&gt;&lt;li&gt;Code Migration: 39.892 (#9/33)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 67.726 (#22/98)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 62.449 (#27/50)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 83.584 (#31/93)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 39.892 (#31/79)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 60.154 (#35/77)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 38.462 (#36/85)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 25.84 (#38/76)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 5.0 (#38/81)&lt;/li&gt;&lt;li&gt;...and 5 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 4&lt;/strong&gt; — Aggregate Rating 1766, #96&lt;ul&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 15.833 (#10/81)&lt;/li&gt;&lt;li&gt;The Aggregate - Day-one Recall Check: 27.3 (#16/37)&lt;/li&gt;&lt;li&gt;SWE Atlas: 59.4 (#17/65)&lt;/li&gt;&lt;li&gt;Finance Agent v2: 54.7 (#19/55)&lt;/li&gt;&lt;li&gt;Vals AI SWE-bench Multimodal: 33.125 (#20/20)&lt;/li&gt;&lt;li&gt;Vals AI BioMysteryBench: 67.037 (#20/28)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 14.261 (#21/26)&lt;/li&gt;&lt;li&gt;Epoch AI - Weirdml V3: 4.45 (#22/23)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 27.778 (#23/55)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 54.678 (#27/81)&lt;/li&gt;&lt;li&gt;...and 14 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.2 Codex&lt;/strong&gt; — Aggregate Rating 1765, #97&lt;ul&gt;&lt;li&gt;Vals AI Vibe Code Bench: 37.912 (#72/119)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-3.0-flash-fin&lt;/strong&gt; — Aggregate Rating 1764, #98&lt;ul&gt;&lt;li&gt;AIIQ Composite IQ: 102.0 (#117/177)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 13.89 (#151/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 13.33 (#152/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 3.89 (#158/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5&lt;/strong&gt; — Aggregate Rating 1761, #102&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 69.8 (#2/25)&lt;/li&gt;&lt;li&gt;Vals AI MATH 500: 96.0 (#3/60)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 66.452 (#6/54)&lt;/li&gt;&lt;li&gt;AIME: 93.374 (#14/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 93.374 (#14/96)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 92.836 (#14/75)&lt;/li&gt;&lt;li&gt;Confabulation Leaderboard (Lechmazur): 10.9 (#17/66)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 37.079 (#39/67)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 81.503 (#39/93)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 65.454 (#42/98)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.1 (2025-11-13)&lt;/strong&gt; — Aggregate Rating 1759, #107&lt;ul&gt;&lt;li&gt;Vals AI CaseLaw v2: 73.419 (#2/54)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 92.982 (#12/75)&lt;/li&gt;&lt;li&gt;AIME: 93.333 (#15/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 93.333 (#15/96)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 44.944 (#25/67)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 83.179 (#33/93)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 61.368 (#61/98)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro&lt;/strong&gt; — Aggregate Rating 1758, #109&lt;ul&gt;&lt;li&gt;Code Migration: 26.201 (#18/33)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 62.923 (#26/50)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 59.378 (#27/54)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 25.17 (#39/76)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 11.111 (#39/55)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 3.75 (#41/81)&lt;/li&gt;&lt;li&gt;Vals AI ProofBench: 16.0 (#45/53)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 47.778 (#46/64)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 26.201 (#47/79)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 51.623 (#56/77)&lt;/li&gt;&lt;li&gt;...and 3 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.6&lt;/strong&gt; — Aggregate Rating 1756, #110&lt;ul&gt;&lt;li&gt;MLCR-AA - Accuracy: 34.44 (#80/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 11.11 (#109/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 21.11 (#118/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 27B&lt;/strong&gt; — Aggregate Rating 1756, #111&lt;ul&gt;&lt;li&gt;Coding Agent Leaderboard - SWE-Bench Pro -- Ansible (Claude Code): 86.5 (#1/10)&lt;/li&gt;&lt;li&gt;Coding Agent Leaderboard - SWE-Bench Pro -- Ansible (OpenCode): 89.6 (#1/9)&lt;/li&gt;&lt;li&gt;Coding Agent Leaderboard - SWE-Bench Verified (OpenCode): 94.0 (#1/9)&lt;/li&gt;&lt;li&gt;Coding Agent Leaderboard - SWE-Bench Verified (Claude Code): 86.8 (#2/10)&lt;/li&gt;&lt;li&gt;ErdosBench: 2.8 (#12/21)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 11.25 (#16/81)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 30.894 (#27/76)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 83.931 (#28/93)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 67.798 (#28/47)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 39.056 (#37/45)&lt;/li&gt;&lt;li&gt;...and 14 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.2&lt;/strong&gt; — Aggregate Rating 1753, #116&lt;ul&gt;&lt;li&gt;Vals AI AIME: 96.875 (#2/96)&lt;/li&gt;&lt;li&gt;AIME: 96.875 (#3/96)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 94.0 (#6/75)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 66.024 (#8/54)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 86.667 (#17/93)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 51.685 (#19/67)&lt;/li&gt;&lt;li&gt;Epoch AI - Weirdml V3: 5.01 (#21/23)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 67.13 (#30/98)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 53.499 (#59/119)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3 (2025-04-16)&lt;/strong&gt; — Aggregate Rating 1747, #124&lt;ul&gt;&lt;li&gt;Vals AI MATH 500: 94.6 (#8/60)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 91.746 (#26/75)&lt;/li&gt;&lt;li&gt;AIME: 85.278 (#37/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 85.278 (#37/96)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 65.7 (#40/98)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 80.416 (#44/93)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Inkling&lt;/strong&gt; — Aggregate Rating 1735, #136&lt;ul&gt;&lt;li&gt;JobBench: 31.1 (#5/6)&lt;/li&gt;&lt;li&gt;Epoch AI - Weirdml V3: 3.9 (#23/23)&lt;/li&gt;&lt;li&gt;Mercor APEX: 33.8 (#53/62)&lt;/li&gt;&lt;li&gt;FutureEval: 3.88 (#60/106)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.7 Flash&lt;/strong&gt; — Aggregate Rating 1732, #141&lt;ul&gt;&lt;li&gt;AIIQ Composite IQ: 94.0 (#152/177)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash&lt;/strong&gt; — Aggregate Rating 1732, #144&lt;ul&gt;&lt;li&gt;LiveBench Research: 79.2 (#19/53)&lt;/li&gt;&lt;li&gt;Mercor APEX: 55.3 (#20/62)&lt;/li&gt;&lt;li&gt;LiveBench Long Horizon: 36.6 (#29/53)&lt;/li&gt;&lt;li&gt;LiveBench Coding: 77.0 (#35/53)&lt;/li&gt;&lt;li&gt;LiveBench Design Layout: 71.7 (#37/53)&lt;/li&gt;&lt;li&gt;LiveBench Data Analysis: 68.0 (#38/53)&lt;/li&gt;&lt;li&gt;LiveBench Reasoning: 70.6 (#50/53)&lt;/li&gt;&lt;li&gt;LiveBench Browser: 0.0 (#50/53)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4.5&lt;/strong&gt; — Aggregate Rating 1730, #147&lt;ul&gt;&lt;li&gt;MLCR-AA - Completeness: 23.33 (#106/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 10.56 (#114/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 25.0 (#118/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 397B A17B&lt;/strong&gt; — Aggregate Rating 1729, #150&lt;ul&gt;&lt;li&gt;Mercor APEX: 24.9 (#59/62)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 32.22 (#92/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 8.89 (#125/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 18.33 (#130/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M3&lt;/strong&gt; — Aggregate Rating 1729, #151&lt;ul&gt;&lt;li&gt;Mercor APEX: 37.7 (#50/62)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O1 (2024-12-17)&lt;/strong&gt; — Aggregate Rating 1727, #153&lt;ul&gt;&lt;li&gt;Vals AI MATH 500: 90.4 (#25/60)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 89.309 (#48/75)&lt;/li&gt;&lt;li&gt;AIME: 71.458 (#53/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 71.458 (#53/96)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 77.412 (#53/93)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 2.5 Pro&lt;/strong&gt; — Aggregate Rating 1726, #155&lt;ul&gt;&lt;li&gt;MLCR-AA - Accuracy: 46.67 (#66/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 12.22 (#102/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 18.33 (#131/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.5 Flash Lite&lt;/strong&gt; — Aggregate Rating 1720, #160&lt;ul&gt;&lt;li&gt;Vals AI MortgageTax: 68.68 (#12/98)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 83.584 (#30/93)&lt;/li&gt;&lt;li&gt;Code Migration: 6.148 (#32/33)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 59.107 (#37/47)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 51.827 (#44/50)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 33.852 (#50/64)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 47.442 (#54/81)&lt;/li&gt;&lt;li&gt;Mercor APEX: 29.3 (#57/62)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 50.187 (#59/76)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 43.231 (#61/77)&lt;/li&gt;&lt;li&gt;...and 5 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Open2 250B&lt;/strong&gt; — Aggregate Rating 1717, #165&lt;ul&gt;&lt;li&gt;AIIQ Composite IQ: 102.0 (#115/177)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ring-2.6-1T&lt;/strong&gt; — Aggregate Rating 1716, #166&lt;ul&gt;&lt;li&gt;MLCR-AA - Completeness: 15.56 (#144/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 3.89 (#159/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 7.78 (#165/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4&lt;/strong&gt; — Aggregate Rating 1716, #167&lt;ul&gt;&lt;li&gt;EKI Estonian LLM - Overall: 41.62 (#43/90)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Inkling Small&lt;/strong&gt; — Aggregate Rating 1714, #170&lt;ul&gt;&lt;li&gt;OCRBench-V2 (en): 54.2 (#23/32)&lt;/li&gt;&lt;li&gt;OCRBench-V2 (zh): 52.4 (#28/32)&lt;/li&gt;&lt;li&gt;FutureEval: 1.3 (#79/106)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 20.0 (#122/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 7.78 (#134/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 17.22 (#139/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Super VL&lt;/strong&gt; — Aggregate Rating 1711, #172&lt;ul&gt;&lt;li&gt;THOR Finding Triage - Balanced OTS: 62.2 (#22/89)&lt;/li&gt;&lt;li&gt;THOR Finding Triage - Critical Miss Rate: 0.0 (#24/89)&lt;/li&gt;&lt;li&gt;THOR Finding Triage - False Review Load: 41.2 (#36/89)&lt;/li&gt;&lt;li&gt;CritPt: 10.6 (#52/275)&lt;/li&gt;&lt;li&gt;THOR Finding Triage - CW%: 54.7 (#54/89)&lt;/li&gt;&lt;li&gt;Mercor APEX: 29.5 (#56/62)&lt;/li&gt;&lt;li&gt;Gert Labs Rankings: 30.48 (#81/100)&lt;/li&gt;&lt;li&gt;AIIQ Composite IQ: 108.0 (#87/177)&lt;/li&gt;&lt;li&gt;MLCR-AA: 13.33 (#92/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 30.56 (#98/172)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Super VL (BF16)&lt;/strong&gt; — Aggregate Rating 1710, #174&lt;ul&gt;&lt;li&gt;LLM Stats (MMLongBench-Doc): 62.82 (#1/7)&lt;/li&gt;&lt;li&gt;LLM Stats (OCRBench-V2 (en)): 68.12 (#2/15)&lt;/li&gt;&lt;li&gt;LLM Stats (Tau3 Banking): 18.88 (#9/15)&lt;/li&gt;&lt;li&gt;LLM Stats (GDPval-AA): 1211.0 (#13/16)&lt;/li&gt;&lt;li&gt;LLM Stats (MathVision): 78.16 (#18/38)&lt;/li&gt;&lt;li&gt;LLM Stats (BrowseComp): 78.0 (#23/69)&lt;/li&gt;&lt;li&gt;LLM Stats (CharXiv-R): 80.82 (#27/60)&lt;/li&gt;&lt;li&gt;LLM Stats (OSWorld-Verified): 51.14 (#27/28)&lt;/li&gt;&lt;li&gt;LLM Stats (SWE-bench Multilingual): 67.72 (#31/50)&lt;/li&gt;&lt;li&gt;LLM Stats (Terminal-Bench 2.1): 66.7 (#32/47)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Mini (2026-03-17)&lt;/strong&gt; — Aggregate Rating 1710, #175&lt;ul&gt;&lt;li&gt;Vals AI AIME: 95.625 (#10/96)&lt;/li&gt;&lt;li&gt;AIME: 95.625 (#11/96)&lt;/li&gt;&lt;li&gt;Code Migration: 12.936 (#25/33)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 44.944 (#27/67)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 54.375 (#41/64)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 51.661 (#47/54)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 79.249 (#49/93)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 2.525 (#50/55)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 54.682 (#51/76)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 63.514 (#52/98)&lt;/li&gt;&lt;li&gt;...and 7 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 27B&lt;/strong&gt; — Aggregate Rating 1710, #176&lt;ul&gt;&lt;li&gt;MLCR-AA: 10.56 (#113/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 21.11 (#131/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 17.22 (#138/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5 Mini (2025-08-07)&lt;/strong&gt; — Aggregate Rating 1710, #177&lt;ul&gt;&lt;li&gt;Vals AI CaseLaw v2: 68.489 (#4/54)&lt;/li&gt;&lt;li&gt;Vals AI MATH 500: 94.8 (#6/60)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 92.582 (#16/75)&lt;/li&gt;&lt;li&gt;AIME: 91.458 (#24/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 91.458 (#24/96)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 66.892 (#31/98)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 26.966 (#50/67)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 78.914 (#50/93)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 14.171 (#96/119)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.5 122B A10B&lt;/strong&gt; — Aggregate Rating 1708, #178&lt;ul&gt;&lt;li&gt;OCRBench-V2 (zh): 67.3 (#2/32)&lt;/li&gt;&lt;li&gt;OCRBench-V2 (en): 68.3 (#3/32)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 22.78 (#123/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 7.78 (#135/172)&lt;/li&gt;&lt;li&gt;CritPt: 1.1 (#138/275)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 15.56 (#145/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 31B (IT)&lt;/strong&gt; — Aggregate Rating 1708, #179&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 39.326 (#35/67)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 52.626 (#43/54)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 61.368 (#60/98)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O4 Mini (2025-04-16)&lt;/strong&gt; — Aggregate Rating 1706, #182&lt;ul&gt;&lt;li&gt;Vals AI MGSM: 93.418 (#9/75)&lt;/li&gt;&lt;li&gt;Vals AI MATH 500: 94.2 (#10/60)&lt;/li&gt;&lt;li&gt;AIME: 83.667 (#41/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 83.667 (#41/96)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 64.826 (#46/98)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 79.665 (#46/93)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Glimmer 30B&lt;/strong&gt; — Aggregate Rating 1706, #183&lt;ul&gt;&lt;li&gt;LLM Stats (MMLU-ProX): 80.1 (#11/33)&lt;/li&gt;&lt;li&gt;LLM Stats (GDPval-AA): 953.0 (#15/16)&lt;/li&gt;&lt;li&gt;FutureEval: 6.58 (#47/106)&lt;/li&gt;&lt;li&gt;Design Arena (Website): 1157.0 (#123/200)&lt;/li&gt;&lt;li&gt;Design Arena (Game Dev): 1125.0 (#123/186)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5 Mini&lt;/strong&gt; — Aggregate Rating 1702, #192&lt;ul&gt;&lt;li&gt;Confabulation Leaderboard (Lechmazur): 21.8 (#31/66)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 33.89 (#83/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 8.89 (#126/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 15.0 (#146/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3P2&lt;/strong&gt; — Aggregate Rating 1701, #193&lt;ul&gt;&lt;li&gt;AIME: 84.583 (#39/96)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.3&lt;/strong&gt; — Aggregate Rating 1701, #194&lt;ul&gt;&lt;li&gt;Code Migration: 6.795 (#30/33)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 51.691 (#45/50)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 22.105 (#57/64)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 0.417 (#62/81)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 37.728 (#67/81)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 41.948 (#67/76)&lt;/li&gt;&lt;li&gt;Vals AI Legal Research Bench: 15.385 (#68/85)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 6.795 (#69/79)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 6.179 (#70/76)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 18.394 (#72/77)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Flash Lite (Preview)&lt;/strong&gt; — Aggregate Rating 1700, #198&lt;ul&gt;&lt;li&gt;Vals AI MortgageTax: 68.044 (#20/98)&lt;/li&gt;&lt;li&gt;Code Migration: 4.609 (#33/33)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 54.984 (#35/54)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 82.486 (#37/93)&lt;/li&gt;&lt;li&gt;AIME: 83.333 (#43/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 83.333 (#43/96)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 24.719 (#53/67)&lt;/li&gt;&lt;li&gt;Vals AI ProgramBench: 14.282 (#61/64)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 0.0 (#65/81)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 34.082 (#72/76)&lt;/li&gt;&lt;li&gt;...and 6 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4&lt;/strong&gt; — Aggregate Rating 1696, #204&lt;ul&gt;&lt;li&gt;RuneBench: 3.71157427994 (#84/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V3.2&lt;/strong&gt; — Aggregate Rating 1694, #207&lt;ul&gt;&lt;li&gt;Vals AI CaseLaw v2: 55.412 (#34/54)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 84.583 (#39/96)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 35.955 (#41/67)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 5.108 (#106/119)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-3.0-flash&lt;/strong&gt; — Aggregate Rating 1694, #209&lt;ul&gt;&lt;li&gt;MLCR-AA - Accuracy: 15.56 (#145/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 14.44 (#149/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 5.0 (#150/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Mini&lt;/strong&gt; — Aggregate Rating 1691, #216&lt;ul&gt;&lt;li&gt;LisanBench: 0.0307 (#95/211)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Pro 4&lt;/strong&gt; — Aggregate Rating 1691, #218&lt;ul&gt;&lt;li&gt;Wolfram LLM Benchmarking Project: 59.7 (#105/605)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling-3.0-flash-VL&lt;/strong&gt; — Aggregate Rating 1691, #219&lt;ul&gt;&lt;li&gt;MLCR-AA - Accuracy: 15.56 (#144/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 5.0 (#149/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 12.78 (#153/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek R1 0528&lt;/strong&gt; — Aggregate Rating 1687, #228&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 85.0 (#21/74)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 35B A3B&lt;/strong&gt; — Aggregate Rating 1686, #232&lt;ul&gt;&lt;li&gt;ToolSandbox: 75.02 (#3/17)&lt;/li&gt;&lt;li&gt;Multi-IF: 81.53 (#12/19)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 13.89 (#149/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 5.0 (#151/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 12.22 (#155/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3 Mini (2025-01-31)&lt;/strong&gt; — Aggregate Rating 1682, #238&lt;ul&gt;&lt;li&gt;Vals AI MATH 500: 91.8 (#19/60)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 91.346 (#29/75)&lt;/li&gt;&lt;li&gt;AIME: 86.458 (#32/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 86.458 (#32/96)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 3.7 Flash&lt;/strong&gt; — Aggregate Rating 1682, #239&lt;ul&gt;&lt;li&gt;Wolfram LLM Benchmarking Project: 53.4 (#183/605)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Mini 4&lt;/strong&gt; — Aggregate Rating 1682, #240&lt;ul&gt;&lt;li&gt;AIIQ Composite IQ: 107.0 (#90/177)&lt;/li&gt;&lt;li&gt;Design Arena (UI Components): 1087.0 (#132/178)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 17.22 (#137/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 18.33 (#138/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 6.67 (#139/172)&lt;/li&gt;&lt;li&gt;Dnotitia Korean LLM Leaderboard: 78.36 (#188/253)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 235B A22B&lt;/strong&gt; — Aggregate Rating 1682, #241&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 63.7 (#5/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3 Ultra&lt;/strong&gt; — Aggregate Rating 1681, #245&lt;ul&gt;&lt;li&gt;JobBench: 27.2 (#6/6)&lt;/li&gt;&lt;li&gt;RuneBench: 3.761471574869 (#81/99)&lt;/li&gt;&lt;li&gt;CritPt: 3.1 (#92/275)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4.5 (Preview)&lt;/strong&gt; — Aggregate Rating 1679, #248&lt;ul&gt;&lt;li&gt;AIIQ Composite IQ: 83.0 (#175/177)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 235B A22B 2507&lt;/strong&gt; — Aggregate Rating 1673, #265&lt;ul&gt;&lt;li&gt;MLCR-AA: 8.89 (#128/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 20.0 (#135/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 15.0 (#147/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Nano (2026-03-17)&lt;/strong&gt; — Aggregate Rating 1670, #271&lt;ul&gt;&lt;li&gt;Vals AI AIME: 88.75 (#28/96)&lt;/li&gt;&lt;li&gt;AIME: 88.75 (#29/96)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 39.888 (#34/67)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 51.875 (#46/54)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 14.467 (#57/79)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 73.584 (#59/93)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 44.751 (#60/77)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 38.217 (#66/81)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 59.102 (#69/98)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 41.573 (#69/76)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3 Mini&lt;/strong&gt; — Aggregate Rating 1668, #274&lt;ul&gt;&lt;li&gt;MLCR-AA - Accuracy: 25.0 (#119/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 7.22 (#138/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 10.56 (#157/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 235B A22B Instruct&lt;/strong&gt; — Aggregate Rating 1668, #275&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 64.7 (#4/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 4.5&lt;/strong&gt; — Aggregate Rating 1663, #285&lt;ul&gt;&lt;li&gt;o11y-bench - Pass^3: 26.56 (#5/6)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 62.5 (#5/6)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 32B Instruct&lt;/strong&gt; — Aggregate Rating 1663, #286&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 62.0 (#6/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 32B&lt;/strong&gt; — Aggregate Rating 1657, #297&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 61.0 (#7/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;A.X-K2&lt;/strong&gt; — Aggregate Rating 1654, #304&lt;ul&gt;&lt;li&gt;AIIQ Composite IQ: 103.0 (#113/177)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 3&lt;/strong&gt; — Aggregate Rating 1652, #306&lt;ul&gt;&lt;li&gt;LisanBench: 0.0199 (#112/211)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-OSS-120B&lt;/strong&gt; — Aggregate Rating 1635, #345&lt;ul&gt;&lt;li&gt;Mercor APEX: 4.4 (#62/62)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Nano&lt;/strong&gt; — Aggregate Rating 1634, #348&lt;ul&gt;&lt;li&gt;o11y-bench - Pass^3: 10.94 (#6/6)&lt;/li&gt;&lt;li&gt;o11y-bench - Pass@3: 32.81 (#6/6)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4o&lt;/strong&gt; — Aggregate Rating 1632, #351&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 56.2 (#16/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-4.5V&lt;/strong&gt; — Aggregate Rating 1632, #352&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 59.2 (#10/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 26B A4B&lt;/strong&gt; — Aggregate Rating 1631, #355&lt;ul&gt;&lt;li&gt;ToolSandbox: 72.09 (#5/17)&lt;/li&gt;&lt;li&gt;Multi-IF: 82.19 (#10/19)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude 3.5 Sonnet&lt;/strong&gt; — Aggregate Rating 1630, #357&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 46.0 (#25/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 30B A3B&lt;/strong&gt; — Aggregate Rating 1630, #358&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 57.3 (#11/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Medium 3.5&lt;/strong&gt; — Aggregate Rating 1626, #372&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 30.337 (#45/67)&lt;/li&gt;&lt;li&gt;Finance Agent v2: 32.1 (#49/55)&lt;/li&gt;&lt;li&gt;Vals AI ProofBench: 9.0 (#49/53)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 44.159 (#53/54)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 0.417 (#64/81)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 38.951 (#70/76)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 32.103 (#71/81)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 5.127 (#72/79)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 4.482 (#73/76)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 11.185 (#75/77)&lt;/li&gt;&lt;li&gt;...and 3 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 VL 72B Instruct&lt;/strong&gt; — Aggregate Rating 1622, #380&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 60.8 (#8/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 30B A3B Instruct&lt;/strong&gt; — Aggregate Rating 1621, #382&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 59.5 (#9/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 30B A3B 2507&lt;/strong&gt; — Aggregate Rating 1621, #383&lt;ul&gt;&lt;li&gt;MLCR-AA - Completeness: 8.89 (#160/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 9.44 (#161/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 2.22 (#162/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 3&lt;/strong&gt; — Aggregate Rating 1611, #410&lt;ul&gt;&lt;li&gt;ProphetArena: 0.9368 (#16/17)&lt;/li&gt;&lt;li&gt;EKI Estonian LLM - Overall: 31.75 (#63/90)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 6.67 (#163/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 1.11 (#168/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 4.44 (#169/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Medium 3.1&lt;/strong&gt; — Aggregate Rating 1608, #415&lt;ul&gt;&lt;li&gt;MLCR-AA - Accuracy: 12.22 (#155/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 2.78 (#160/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 5.56 (#167/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 VL 8B Instruct&lt;/strong&gt; — Aggregate Rating 1607, #419&lt;ul&gt;&lt;li&gt;HELM Robo-Reward-Bench - Berkeley Autolab Ur5: 0.42 (#2/25)&lt;/li&gt;&lt;li&gt;HELM Robo-Reward-Bench - Utokyo Xarm Bimanual Converted Externally To Rlds: 1.08 (#2/25)&lt;/li&gt;&lt;li&gt;HELM Robo-Reward-Bench - Kaist Nonprehensile Converted Externally To Rlds: 0.75 (#3/25)&lt;/li&gt;&lt;li&gt;HELM Robo-Reward-Bench - Ucsd Kitchen Dataset Converted Externally To Rlds: 0.89 (#5/25)&lt;/li&gt;&lt;li&gt;HELM Robo-Reward-Bench - Robo Arena: 0.85 (#6/25)&lt;/li&gt;&lt;li&gt;HELM Robo-Reward-Bench - Roboturk: 0.79 (#8/25)&lt;/li&gt;&lt;li&gt;HELM Robo-Reward-Bench - Core Scenarios: 89.17 (#9/25)&lt;/li&gt;&lt;li&gt;HELM Robo-Reward-Bench - Bridge: 0.98 (#10/25)&lt;/li&gt;&lt;li&gt;HELM Robo-Reward-Bench - Dlr Edan Shared Control Converted Externally To Rlds: 0.47 (#10/25)&lt;/li&gt;&lt;li&gt;HELM Robo-Reward-Bench - Jaco Play: 0.95 (#10/25)&lt;/li&gt;&lt;li&gt;...and 15 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 Max&lt;/strong&gt; — Aggregate Rating 1606, #425&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 76.1 (#45/74)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5 Nano&lt;/strong&gt; — Aggregate Rating 1603, #434&lt;ul&gt;&lt;li&gt;Vals AI MATH 500: 93.8 (#15/60)&lt;/li&gt;&lt;li&gt;Vals AI CaseLaw v2: 52.626 (#44/54)&lt;/li&gt;&lt;li&gt;AIME: 81.181 (#45/96)&lt;/li&gt;&lt;li&gt;Vals AI AIME: 81.181 (#45/96)&lt;/li&gt;&lt;li&gt;Vals AI MGSM: 89.309 (#49/75)&lt;/li&gt;&lt;li&gt;Vals AI MMMU: 70.942 (#69/93)&lt;/li&gt;&lt;li&gt;Vals AI MortgageTax: 53.617 (#81/98)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercury 2.5&lt;/strong&gt; — Aggregate Rating 1601, #444&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench Science: 0.0 (#39/42)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 45.476 (#40/47)&lt;/li&gt;&lt;li&gt;Vals AI IOI: 2.444 (#45/45)&lt;/li&gt;&lt;li&gt;Vals AI Public Benefits Bench: 45.264 (#47/50)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 0.0 (#55/55)&lt;/li&gt;&lt;li&gt;Vals AI Harvey Legal Agent Bench: 0.0 (#66/81)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 2.1: 34.457 (#71/76)&lt;/li&gt;&lt;li&gt;Vals AI Code Migration: 4.517 (#75/79)&lt;/li&gt;&lt;li&gt;Vals AI Tax Agent Bench: 2.487 (#76/76)&lt;/li&gt;&lt;li&gt;Vals AI Excel Modeling: 8.789 (#76/77)&lt;/li&gt;&lt;li&gt;...and 4 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 2&lt;/strong&gt; — Aggregate Rating 1596, #460&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 75.5 (#49/74)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 VL 32B Instruct&lt;/strong&gt; — Aggregate Rating 1596, #462&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 55.3 (#19/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Lightning&lt;/strong&gt; — Aggregate Rating 1589, #482&lt;ul&gt;&lt;li&gt;ToolSandbox: 75.13 (#2/17)&lt;/li&gt;&lt;li&gt;Multi-IF: 82.34 (#9/19)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;North Mini Code&lt;/strong&gt; — Aggregate Rating 1580, #518&lt;ul&gt;&lt;li&gt;MLCR-AA: 4.44 (#156/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 11.67 (#157/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 8.89 (#159/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 4&lt;/strong&gt; — Aggregate Rating 1578, #526&lt;ul&gt;&lt;li&gt;Vals AI Terminal-Bench 2.0: 16.854 (#62/67)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench: 0.0 (#119/119)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 6.67 (#162/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 6.11 (#166/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 1.67 (#167/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-OSS-20B&lt;/strong&gt; — Aggregate Rating 1578, #527&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 38.89 (#176/221)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude 3.5 Haiku&lt;/strong&gt; — Aggregate Rating 1564, #570&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 65.0 (#62/74)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.3 70B&lt;/strong&gt; — Aggregate Rating 1560, #585&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 68.9 (#56/74)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 70B&lt;/strong&gt; — Aggregate Rating 1557, #595&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 66.4 (#61/74)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 3.1&lt;/strong&gt; — Aggregate Rating 1549, #618&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 66.8 (#60/74)&lt;/li&gt;&lt;li&gt;MLCR-AA: 2.22 (#163/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 8.33 (#164/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 3.33 (#170/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 2.5 VL 7B Instruct&lt;/strong&gt; — Aggregate Rating 1536, #658&lt;ul&gt;&lt;li&gt;Physical AI Bench Understanding: 51.0 (#20/25)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ministral 3 14B&lt;/strong&gt; — Aggregate Rating 1536, #659&lt;ul&gt;&lt;li&gt;MLCR-AA: 2.22 (#164/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 5.0 (#167/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 3.89 (#169/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ministral 3 8B&lt;/strong&gt; — Aggregate Rating 1526, #700&lt;ul&gt;&lt;li&gt;MLCR-AA: 0.0 (#171/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 3.33 (#171/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 0.0 (#172/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Phi-4&lt;/strong&gt; — Aggregate Rating 1522, #710&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 74.3 (#50/74)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude 3 Haiku&lt;/strong&gt; — Aggregate Rating 1493, #810&lt;ul&gt;&lt;li&gt;EKI Estonian LLM - Overall: 28.93 (#71/90)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Command-R+ (08-2024)&lt;/strong&gt; — Aggregate Rating 1486, #833&lt;ul&gt;&lt;li&gt;LM Market Cap LMC Score: 33.3 (#188/222)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ministral 3 3B&lt;/strong&gt; — Aggregate Rating 1459, #940&lt;ul&gt;&lt;li&gt;MLCR-AA - Accuracy: 1.11 (#171/172)&lt;/li&gt;&lt;li&gt;MLCR-AA: 0.0 (#172/172)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 1.11 (#172/172)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 3.1 8B&lt;/strong&gt; — Aggregate Rating 1419, #1156&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 48.3 (#66/74)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Phi-3.5-mini-instruct&lt;/strong&gt; — Aggregate Rating 1416, #1167&lt;ul&gt;&lt;li&gt;Arcee MMLU-Pro (BenchLM): 47.4 (#67/74)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (291)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on GPQA Diamond (BenchmarkList): 93.7 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on KernelBench CUDA: 0.2188 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on LisanBench: 0.4064 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on MLCR-AA: 18.33 (#60)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on MLCR-AA - Accuracy: 28.89 (#104)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on MLCR-AA - Completeness: 51.67 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Mercor APEX: 68.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Code Migration: 54.607 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI CyberBench: 70.416 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Excel Modeling: 76.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Finance Agent v2: 58.877 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 6.667 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI IOI: 90.778 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Legal Research Bench: 55.288 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI MMMU: 90.636 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI MortgageTax: 70.786 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI MysteryMechanism: 47.748 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI ProgramBench: 82.688 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Public Benefits Bench: 74.899 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI ReverseEngBench: 22.901 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 61.25 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Tax Agent Bench: 49.194 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Terminal-Bench 2.1: 85.019 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 58.081 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Terminal-Bench Science: 40.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Time Horizon Index: 63.333 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Vibe Code Bench: 90.263 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 28.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1800.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on BrowseComp: 90.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on CAIS Vision Capabilities Index: 68.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Design Arena (Agents - Web Apps): 1319.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on EKI Estonian LLM - Overall: 63.68 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Dtbench: 97.33 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Lmca: 66.18 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on GPQA Diamond (BenchmarkList): 93.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on HANDBOOK.md Agents: 52.3 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Kaggle WWTP Engineering: 0.0 (#42)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on KernelBench CUDA: 0.5167 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Lech Mazur Writing: 3.715 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LisanBench: 0.6468 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MLCR-AA: 25.0 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MLCR-AA - Accuracy: 37.22 (#76)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MLCR-AA - Completeness: 51.67 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MedXpertQA MM (Natomy): 86.55 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MedXpertQA MM (Natomy) - Basic Science: 89.52 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MedXpertQA MM (Natomy) - Diagnosis: 85.99 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MedXpertQA MM (Natomy) - Treatment: 85.71 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Mercor APEX: 73.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on SEAL - Fortress: 5.78 (#71)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI (Vals Index): 70.177 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI BioMysteryBench: 79.259 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Code Migration: 66.646 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI CyberBench: 74.584 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Excel Modeling: 75.938 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Finance Agent v2: 58.587 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 3.75 (#43)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI IOI: 95.056 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Legal Research Bench: 50.481 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI MysteryMechanism: 49.55 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI ProgramBench: 86.986 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI ProofBench: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Public Benefits Bench: 70.636 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI ReverseEngBench: 33.588 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 70.208 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Science Index: 58.651 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Tax Agent Bench: 45.573 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Terminal-Bench 2.1: 87.64 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 65.152 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Terminal-Bench Science: 47.143 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Time Horizon Index: 91.333 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Vibe Code Bench: 90.294 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 30.361 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vectara Hallucination Leaderboard: 90.1 (#60)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1810.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on BrowseComp: 91.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 100.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on GPQA Diamond (BenchmarkList): 96.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Kaggle Open Benchmarks - MMLU: 94.14 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Kaggle Open Benchmarks - MMLU-Pro: 89.8 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on LisanBench: 0.199 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on MLCR-AA: 38.33 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on MLCR-AA - Accuracy: 95.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on MLCR-AA - Completeness: 40.56 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Mercor APEX: 64.7 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on NYT Connections Extended: 98.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI (Vals Index): 65.023 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI BioMysteryBench: 79.259 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Code Migration: 67.74 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Excel Modeling: 71.7 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Finance Agent v2: 53.54 (#34)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 5.417 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI IOI: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Legal Research Bench: 39.423 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI MysteryMechanism: 53.153 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI ProgramBench: 85.417 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI ReverseEngBench: 56.87 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 43.333 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Science Index: 65.09 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Tax Agent Bench: 20.717 (#48)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Terminal-Bench 2.1: 87.266 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 59.596 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Terminal-Bench Science: 62.857 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Time Horizon Index: 90.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Vibe Code Bench: 89.594 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 27.644 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra Pro&lt;/strong&gt; on KernelBench CUDA: 0.2068 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on CAIS Vision Capabilities Index: 68.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (Agents - Mobile Apps): 1238.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (Agents - Web Apps): 1306.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (Data Viz): 1292.0 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 99.82 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Dtbench: 96.53 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Frontiermath Erdos: 4.41 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Lmca: 62.78 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle WWTP Engineering: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on KernelBench CUDA: 0.1975 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LisanBench: 0.2351 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on MLCR-AA: 35.56 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on MLCR-AA - Accuracy: 93.89 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on MLCR-AA - Completeness: 36.11 (#34)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Mercor APEX: 60 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on NYT Connections Extended: 97.8 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra Launch - Internal Computer Use Safety Benchmark: 4.32 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Adversarial User Simulations - Mental Health: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Chat Plugins: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Non-Violent Wrongdoing: 98.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Self-Harm: 92.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Sensitive Personal Data: 74.4 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Human Red-Teaming Chat Plugins: 95.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Biology Refusal - Benign: 98.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Biology Refusal - Dual Use: 98.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Biology Refusal - Severe: 99.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Extremism: 97.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Gore: 88.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Hate: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Sexual/Minors: 98.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Violent Illicit Behavior: 98.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Cyber Safety - Production Chat: 0.987 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Cyber Safety - Semi-Synthetic Agentic Environments: 0.98 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Cyber Safety - Synthetic Agentic Environments: 0.997 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - HealthBench Hard: 36.2 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - HealthBench Professional: 64.2 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Erotic: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Extremism: 98.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Hate: 99.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Self-Harm: 99.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Multimodal Troubleshooting Virology: 55.34 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - ProtocolQA Open-Ended: 40.74 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - SHP2 Protein Function Prediction: 0.332 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Bio High Risk: 93.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Bio Severe: 94.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Cyber: 85.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Violence Moderate: 88.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Violence Severe: 95.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Tacit Knowledge and Troubleshooting: 88.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - TroubleshootingBench: 47.96 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - U18 - Sexual Content: 98.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Broken Search Tool Non-Disclosure: 2.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Factual Errors on User-Flagged Conversations: 4.12 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Reviewer Bypass Attempts: 0.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Warning Circumvention: 23.48 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI (Vals Index): 63.944 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI BioMysteryBench: 79.63 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Code Migration: 65.122 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Excel Modeling: 70.773 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Finance Agent v2: 52.033 (#38)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 5.417 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI IOI: 96.889 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Legal Research Bench: 38.462 (#38)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI MysteryMechanism: 46.396 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI ProgramBench: 85.25 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI ProofBench: 99.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Public Benefits Bench: 59.337 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI ReverseEngBench: 50.763 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 43.542 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Science Index: 59.628 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Tax Agent Bench: 21.754 (#46)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 55.051 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Terminal-Bench Science: 52.857 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Vibe Code Bench: 88.931 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1598.0 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AIIQ Composite IQ: 140.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on APEX v1: 81.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on CAIS Risk Index: 56.2 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Coarena: 1032.3 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Coarena - Task Completion: 76.8 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Dnotitia Korean LLM Leaderboard: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on DuelLab Overall: 89.369322241651 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Dtbench: 97.6 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Lmca: 63.69 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Mirrorcode: 58.89 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Weirdml V3: 20.22 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Featherbench: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Featherbench - Rubric Quality: 9.2 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on HANDBOOK.md Agents: 36.6 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA: 3.89 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - All Pass: 20.83 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Antitrust &amp; Competition: 95.29 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Arbitration &amp; International Dispute Resolution: 93.91 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Banking &amp; Finance: 97.01 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Bankruptcy &amp; Restructuring: 92.04 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Capital Markets: 95.16 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Corporate Governance: 96.69 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Corporate M&amp;A: 94.53 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Data Privacy &amp; Cybersecurity: 97.64 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Emerging Companies &amp; Venture Capital: 94.43 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Employment &amp; Labor: 89.39 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Energy &amp; Natural Resources: 97.69 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Environmental &amp; ESG: 95.58 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Funds &amp; Asset Management: 91.74 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Healthcare &amp; Life Sciences: 97.87 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Immigration: 93.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Insurance: 94.12 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Intellectual Property: 97.98 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - International Trade &amp; Sanctions: 92.56 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Litigation &amp; Dispute Resolution: 94.24 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Real Estate: 97.08 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Structured Finance &amp; Securitization: 92.15 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Tax: 93.42 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Trusts, Estates &amp; Private Client: 95.04 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - White Collar Defense &amp; Investigations: 95.06 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Game Arena Chess: 1380.78 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle WWTP Engineering: 100.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on KernelBench CUDA: 0.2732 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on LLM Chess (Saplin): 1609.422 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Lech Mazur Writing: 0.8 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on LisanBench: 0.2035 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MLCR-AA: 13.89 (#87)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MLCR-AA - Accuracy: 32.78 (#85)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MLCR-AA - Completeness: 31.67 (#52)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Mercor APEX: 82.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MirrorCode: 58.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SEAL - MultiChallenge: 78.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SEAL - Remote Labor Index (RLI): 17.08 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Bug Density: 0.56 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Code Smell Density: 15.53 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Pass Rate: 85.66 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Vulnerability Density: 0.31 (#51)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI (Vals Index): 69.237 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI BioMysteryBench: 76.296 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Code Migration: 68.173 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI CyberBench: 77.857 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Excel Modeling: 75.244 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Finance Agent v2: 65.401 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 19.583 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI IOI: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Legal Research Bench: 54.808 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI MysteryMechanism: 45.495 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI ProgramBench: 83.715 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI ProofBench: 99.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Public Benefits Bench: 69.756 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI ReverseEngBench: 44.275 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 56.042 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Science Index: 55.359 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Tax Agent Bench: 44.56 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 57.576 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Terminal-Bench Science: 44.286 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Time Horizon Index: 52.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Vibe Code Bench: 91.906 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 21.2 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wolfram LLM Benchmarking Project: 80.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-LCR: 83.67 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AIIQ Composite IQ: 131.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on CritPt: 32.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Finance Agent v2: 59.1 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LLM Stats (Terminal-Bench 4.0): 57.1 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LisanBench: 1.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI (Vals Index): 66.787 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI BioMysteryBench: 73.333 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Code Migration: 61.033 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI CyberBench: 73.631 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Excel Modeling: 71.917 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Finance Agent v2: 59.061 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 21.667 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI IOI: 79.833 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Legal Research Bench: 53.365 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI MysteryMechanism: 35.135 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI ProgramBench: 24.187 (#56)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI ProofBench: 94.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Public Benefits Bench: 73.748 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI ReverseEngBench: 11.832 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 47.292 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Science Index: 48.537 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Tax Agent Bench: 42.997 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 58.081 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Terminal-Bench Science: 37.143 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Vibe Code Bench: 85.996 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 33.978 (#1)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (83)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena Four in a Row&lt;/strong&gt;: Gemini 4 Argon (923.33) beat GPT-6 Astra by 228.01&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coarena&lt;/strong&gt;: GPT-6.1 Sol (1147.5) beat Claude Fable 5 by 84.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena (Unified)&lt;/strong&gt;: Gemini 4 Argon (535.09) beat GPT-6 Astra by 42.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - TroubleshootingBench&lt;/strong&gt;: GPT-6 Astra (63.46) beat GPT-6 Sol by 18.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coding Agent Leaderboard - SWE-Bench Pro -- Ansible (Claude Code)&lt;/strong&gt;: Qwen 3.8 27B (86.5) beat Claude Opus 4.8 by 16.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle WWTP Engineering&lt;/strong&gt;: Gemini 3.8 Flash (100.0) beat Claude Sonnet 4 by 15.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HANDBOOK.md Agents&lt;/strong&gt;: Claude Opus 5.5 (Adaptive Reasoning, Max Effort) (52.3) beat Claude Fable 5.1 (Max) by 13.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Tacit Knowledge and Troubleshooting&lt;/strong&gt;: GPT-6 Astra (92.55) beat GPT-6 Sol by 13.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Multimodal Troubleshooting Virology&lt;/strong&gt;: GPT-6 Astra (63.11) beat GPT-6 Sol by 12.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (AutomationBench; v1.0.6 method cohort)&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (60.5) beat GLM-5.3 Flash by 11.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coding Agent Leaderboard - SWE-Bench Pro -- Ansible (OpenCode)&lt;/strong&gt;: Qwen 3.8 27B (89.6) beat Claude Opus 4.8 by 11.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coding Agent Leaderboard - SWE-Bench Verified (OpenCode)&lt;/strong&gt;: Qwen 3.8 27B (94.0) beat Claude Opus 4.8 by 10.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (AutomationBench v1.0.6)&lt;/strong&gt;: MiniMax-M3.1-Flash (62.8) beat MiMo-V2.6-Pro by 9.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;o11y-bench - Pass^3&lt;/strong&gt;: Claude Sonnet 5.5 (High) (89.06) beat Claude Opus 4.8 (High) by 9.69&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ChessBench - Coherence&lt;/strong&gt;: GPT-5.5 (100.0) beat GPT-5.4 by 8.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WDCD&lt;/strong&gt;: GPT-6 Luna (100.0) beat Grok 4 by 8.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ToolSandbox&lt;/strong&gt;: LFM2.5-24B-A2B (80.71) beat GPT-4o (2024-05-13) by 7.71&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (Slides)&lt;/strong&gt;: Claude Opus 5.5 (1309.0) beat playyy by 7.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ParseBench - Charts&lt;/strong&gt;: Upstage Document Parse 2 (Enhanced) (77.26) beat Claude Opus 5.5 (High) by 6.37&lt;/li&gt;&lt;li&gt;&lt;strong&gt;o11y-bench - Pass@3&lt;/strong&gt;: Claude Sonnet 5.5 (High) (100.0) beat Claude Opus 5 (High) by 6.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;APEX v1&lt;/strong&gt;: Gemini 4 Argon (High) (81.1) beat Muse Spark 1.2 (xHigh) by 5.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MMLongBench-Doc - Accuracy&lt;/strong&gt;: Nemotron 3.5 Super VL 120B A12B (66.6) beat Claude Opus 4.5 by 4.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (NL2Repo)&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (68.2) beat DeepSeek V4.1 Flash by 4.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - U18 - Eating Disorders&lt;/strong&gt;: GPT-6.1 Sol (96.2) beat GPT-6 Astra by 4.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coarena - Task Completion&lt;/strong&gt;: Claude Haiku 5.5 (94.0) beat GPT-5.6 Luna by 3.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Coding Daily (OpenCode) - Total&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (Max) (60.97) beat Space Bunny (Max) by 3.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MedXpertQA MM (Natomy) - Basic Science&lt;/strong&gt;: Gemini 4 Argon (92.92) beat GPT-6 Astra by 3.68&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench 4.0&lt;/strong&gt;: Kimi K3.1 Pro (67.17) beat Claude Sonnet 5.5 (Max, Default Fallback) by 3.53&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MedXpertQA MM (Natomy) - Treatment&lt;/strong&gt;: Gemini 4 Argon (90.18) beat GPT-6 Astra by 3.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MedXpertQA MM (Natomy)&lt;/strong&gt;: Gemini 4 Argon (90.5) beat GPT-6 Astra by 3.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SloBench - Slovene SuperGLUE&lt;/strong&gt;: Cvetka-27B-tiv (82.9) beat Gizela-31B-it-vit-think by 3.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MedXpertQA MM (Natomy) - Diagnosis&lt;/strong&gt;: Gemini 4 Argon (89.91) beat GPT-6 Astra by 2.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (BabyVision)&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (92.5) beat DeepSeek V4.1 Flash by 2.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BALSAM - Program Execution&lt;/strong&gt;: NatiqLM-Version3 (98.45) beat Grok 4.1 Fast (Reasoning) by 2.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Crosby micro1 RedlineBench&lt;/strong&gt;: Gemini 4 Argon (59.5) beat Claude Opus 5 by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ScreenSpot-Pro&lt;/strong&gt;: GroundCascade (Indeed-UI-32B + KV-Ground-8B) ScreenSpot-Pro official grounding leaderboard (85.14) beat Indeed-UI-32B-zoomin ScreenSpot-Pro official grounding leaderboard by 2.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Recall&lt;/strong&gt;: codos.ai (96.45) beat Mixedbread (+ Opus 5) by 2.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - U18 - Emotional Reliance&lt;/strong&gt;: GPT-6.1 Sol (96.9) beat GPT-6 Sol by 2.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (ASCII Art)&lt;/strong&gt;: Claude Opus 5 (1355.0) beat GPT-6 Astra by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (ZEROBench)&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (51.0) beat DeepSeek V4.1 Flash by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI (Vals Index)&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (70.854) beat Gemini 4 Argon by 1.95&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Age-Restricted Actions&lt;/strong&gt;: GPT-6.1 Sol (83.0) beat GPT-6 Astra by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Violent Wrongdoing&lt;/strong&gt;: GPT-6.1 Sol (92.6) beat GPT-6 Astra by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Healthcare &amp; Life Sciences&lt;/strong&gt;: Gemini 4 Argon (High) (97.87) beat Muse Wonder (Max) by 1.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - U18 - Age-Restricted Goods and Dangerous Activities&lt;/strong&gt;: GPT-6.1 Sol (93.4) beat GPT-6 Astra by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench - MASE&lt;/strong&gt;: timeh2o-v1 (90.39) beat Timestral-209M by 1.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Frontiermath Erdos&lt;/strong&gt;: GPT-6.1 Sol (Max) (4.41) beat Claude Opus 5.5 (Max) by 1.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Legal - Criterion Pass Rate&lt;/strong&gt;: Kimi K3.1 Pro (89.44) beat Muse Wonder (Max) by 1.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Correctness&lt;/strong&gt;: codos.ai (94.4) beat Flexor.ai (+ GPT-6.1 Sol) by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MMEB Leaderboard&lt;/strong&gt;: Sentio (61.69) beat Wianmu-5B-v0 by 1.24&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (LongBench v2)&lt;/strong&gt;: Beam 1.0 (67.5) beat Qwen 3.8 Max by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OCRBench-V2 (zh)&lt;/strong&gt;: Qwen3.8-Flash-Next (68.3) beat Qwen 3.7 Plus by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench&lt;/strong&gt;: codos.ai (92.64) beat Flexor.ai (+ GPT-6.1 Sol) by 1.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (MCP Atlas)&lt;/strong&gt;: Muse Spark Open (89.2) beat Muse Spark 1.1 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MCP Atlas&lt;/strong&gt;: Muse Spark Open (xhigh effort, full 1,000-task set) (xHigh) (89.2) beat Muse Spark 1.1 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench - SQL&lt;/strong&gt;: timeh2o-v1 (91.87) beat Timestral-209M by 1.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;YKS Exam 2023 - TYT Social Sciences&lt;/strong&gt;: Jev-1.14 (text input) (20.0) beat Qwen 3.8 27B by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - ProtocolQA Open-Ended&lt;/strong&gt;: GPT-6 Astra (45.37) beat GPT-6 Sol by 0.97&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Open Universal Arabic ASR Leaderboard - CER&lt;/strong&gt;: Jais-ASR (8.31) beat Audar-ASR-V1-Turbo by 0.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (MMLongBench-Doc)&lt;/strong&gt;: Nemotron 3.5 Super VL (BF16) (62.82) beat Qwen 3.6 Plus by 0.82&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Completeness&lt;/strong&gt;: codos.ai (95.16) beat Flexor.ai (+ GPT-6.1 Sol) by 0.79&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Adversarial User Simulations - Self-Harm&lt;/strong&gt;: GPT-6.1 Sol (99.6) beat GPT-6 Astra by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ParseBench&lt;/strong&gt;: Upstage Document Parse 2 (Enhanced) (80.51) beat Claude Opus 5.5 (High) by 0.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GameDevBench&lt;/strong&gt;: Claude Opus 5.5 (Claude Code, xHigh) (70.6) beat GPT-6.1 Sol (Codex, xHigh) by 0.63&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Web Search&lt;/strong&gt;: GPT-5.6 Sol (Max) (49.076695) beat Claude Fable 5 (Max) by 0.63&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CritPt&lt;/strong&gt;: Muse Wonder Full (Max) (32.9) beat GPT-5.6 Sol (Max) by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BrowseComp&lt;/strong&gt;: Kimi K3.1 Pro (Max) (92.1) beat GPT-6 Astra (Max) by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench&lt;/strong&gt;: timeh2o-v1 (90.06) beat Timestral-209M by 0.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Tax Agent Bench&lt;/strong&gt;: qwen3.8-max-internal (49.663) beat Claude Fable 5.1 by 0.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Environmental &amp; ESG&lt;/strong&gt;: Gemini 4 Argon (High) (95.58) beat Muse Spark 1.3 (Max) by 0.36&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Open Universal Arabic ASR Leaderboard&lt;/strong&gt;: Jais-ASR (22.82) beat Audar-ASR-V1-Turbo by 0.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Challenging Prompts - Non-Violent Illicit Behavior&lt;/strong&gt;: GPT-6.1 Sol (100.0) beat GPT-6 Astra by 0.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Challenging Prompts - Sexual&lt;/strong&gt;: GPT-6.1 Sol (98.7) beat GPT-6 Sol by 0.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - HealthBench&lt;/strong&gt;: GPT-6.1 Sol (58.5) beat GPT-6 Astra by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Challenging Prompts - Self-Harm&lt;/strong&gt;: GPT-6.1 Sol (99.4) beat GPT-6 Astra by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - U18 - Self-Harm&lt;/strong&gt;: GPT-6.1 Sol (99.7) beat GPT-6 Astra by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Adversarial User Simulations - Emotional Reliance&lt;/strong&gt;: GPT-6.1 Sol (99.5) beat GPT-6 Astra by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CAIS Risk Index&lt;/strong&gt;: Claude Opus 5.5 (27.1) beat Claude Fable 5 by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BALSAM - Summarization&lt;/strong&gt;: soual-3 (59.02) beat Nojoom-AQL-120B by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Antitrust &amp; Competition&lt;/strong&gt;: Gemini 4 Argon (High) (95.29) beat Muse Spark 1.3 (Max) by 0.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Intellectual Property&lt;/strong&gt;: Gemini 4 Argon (High) (97.98) beat Muse Wonder Full (Max) by 0.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Human Red-Teaming Codex&lt;/strong&gt;: GPT-6.1 Sol (97.8) beat GPT-6 Astra by 0.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Lmca&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (68.26) beat Claude Opus 5.5 (Max) by 0.03&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-25

=== DAILY ===
NEW BENCHMARKS (51)
  - Kaggle MultiLoKo - Average v2 (Tasks v6, German v2) (Exact Match (%)): leader Gemini 3.1 Pro (Preview) (57.03), 14 models
      Meta MultiLoKo native Kaggle revisions: 30 languages at task v6 and German at v2, with 250 items pe</summary></entry><entry><title>The Aggregate Digest — 2026-10-24</title><id>https://theaggregate.ai/digest/2026-10-24</id><updated>2026-10-24T06:49:30.355924+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Frontier&lt;/h2&gt;
&lt;ul&gt;&lt;li&gt;Best available model: Kimi K3.1 Pro enters at #12 (Aggregate Rating 1934) on 113 benchmarks&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (62)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Legal Agent Benchmark (BenchmarkList Mixed Public and Held-Out)&lt;/strong&gt; (All-Pass Task Success (system card)): leader Claude Opus 5 (23.58), 13 models&lt;br&gt;&lt;span&gt;BenchmarkList imports all-pass legal workflow results from model system cards. Its main column combines results reported on public legal-agent tasks and Harvey held-out tasks. The imported cohort preserves each reported score and remains separate from the primary public and held-out populations.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OSWorld-Verified (BenchmarkList Multi-Source)&lt;/strong&gt; (Success rate (imported)): leader Intelligence-Indeed Agent (100 steps) (Max) OSWorld-Verified (90.19), 143 models&lt;br&gt;&lt;span&gt;BenchmarkList combines reported desktop computer-use success rates from official OSWorld submissions and vendor evaluations. Agents operate real desktop applications and are graded on the resulting task state. Task populations, step budgets and agent harnesses are not uniformly bound across the imported rows, so this cohort remains separate from the primary OSWorld-Verified board.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;scBench (BenchmarkList Legacy Multi-Harness)&lt;/strong&gt; (Accuracy (imported)): leader Claude Mythos 5 (59.3), 17 models&lt;br&gt;&lt;span&gt;BenchmarkList imports single-cell RNA-seq agent workflow scores citing the legacy 195-task scBench release, alongside vendor system-card results. The imported rows include different agent harnesses whose exact dataset and run versions are not uniformly identified. This cohort remains separate from clean-v1 and the primary harness-specific results.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CellTrack-Bench - Link F1&lt;/strong&gt; (Score (0–1)): leader GPT-5.6 Sol (Low) (0.811), 22 models&lt;br&gt;&lt;span&gt;CellTrack-Bench:frozen c1 hard-test subset,244 neighborhoods; same unannotated frame-pair coordinate interface,crop-and-zoom tool,prompt,parser and token budget. Dataset-macro scores. Pinned primary test configs use GPT5.5/GPT5.6 and Fable5/Opus4.8/Opus5/Sonnet5 Low with final reasoning None after tool withdrawal; Haiku4.5 uses dated20251001 endpoint and None. Open chat runner does not send configured reasoning_effort, so cached Medium is not an evaluated setting. Gemma4 is IT; Qwen3.6 result averages two same-checkpoint replicates. Nearest-neighbor/division resistance and graph-level correctness remain separate metrics. Score (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CellTrack-Bench - Repair F1&lt;/strong&gt; (Score (0–1)): leader GPT-5.6 Sol (Low) (0.815), 22 models&lt;br&gt;&lt;span&gt;CellTrack-Bench:frozen c1 hard-test subset,244 neighborhoods; same unannotated frame-pair coordinate interface,crop-and-zoom tool,prompt,parser and token budget. Dataset-macro scores. Pinned primary test configs use GPT5.5/GPT5.6 and Fable5/Opus4.8/Opus5/Sonnet5 Low with final reasoning None after tool withdrawal; Haiku4.5 uses dated20251001 endpoint and None. Open chat runner does not send configured reasoning_effort, so cached Medium is not an evaluated setting. Gemma4 is IT; Qwen3.6 result averages two same-checkpoint replicates. Nearest-neighbor/division resistance and graph-level correctness remain separate metrics. Score (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CellTrack-Bench - Inspection F1&lt;/strong&gt; (Score (0–1)): leader Claude Fable 5 (Low) (0.698), 22 models&lt;br&gt;&lt;span&gt;CellTrack-Bench:frozen c1 hard-test subset,244 neighborhoods; same unannotated frame-pair coordinate interface,crop-and-zoom tool,prompt,parser and token budget. Dataset-macro scores. Pinned primary test configs use GPT5.5/GPT5.6 and Fable5/Opus4.8/Opus5/Sonnet5 Low with final reasoning None after tool withdrawal; Haiku4.5 uses dated20251001 endpoint and None. Open chat runner does not send configured reasoning_effort, so cached Medium is not an evaluated setting. Gemma4 is IT; Qwen3.6 result averages two same-checkpoint replicates. Nearest-neighbor/division resistance and graph-level correctness remain separate metrics. Score (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CellTrack-Bench - Mean F1&lt;/strong&gt; (Score (0–1)): leader Claude Fable 5 (Low) (0.758), 22 models&lt;br&gt;&lt;span&gt;CellTrack-Bench:frozen c1 hard-test subset,244 neighborhoods; same unannotated frame-pair coordinate interface,crop-and-zoom tool,prompt,parser and token budget. Dataset-macro scores. Pinned primary test configs use GPT5.5/GPT5.6 and Fable5/Opus4.8/Opus5/Sonnet5 Low with final reasoning None after tool withdrawal; Haiku4.5 uses dated20251001 endpoint and None. Open chat runner does not send configured reasoning_effort, so cached Medium is not an evaluated setting. Gemma4 is IT; Qwen3.6 result averages two same-checkpoint replicates. Nearest-neighbor/division resistance and graph-level correctness remain separate metrics. Score (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CellTrack-Bench - Nearest-Neighbor Resistance&lt;/strong&gt; (Score (0–1)): leader GPT-5.6 Sol (Low) (0.723), 22 models&lt;br&gt;&lt;span&gt;CellTrack-Bench:frozen c1 hard-test subset,244 neighborhoods; same unannotated frame-pair coordinate interface,crop-and-zoom tool,prompt,parser and token budget. Dataset-macro scores. Pinned primary test configs use GPT5.5/GPT5.6 and Fable5/Opus4.8/Opus5/Sonnet5 Low with final reasoning None after tool withdrawal; Haiku4.5 uses dated20251001 endpoint and None. Open chat runner does not send configured reasoning_effort, so cached Medium is not an evaluated setting. Gemma4 is IT; Qwen3.6 result averages two same-checkpoint replicates. Nearest-neighbor/division resistance and graph-level correctness remain separate metrics. Score (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CellTrack-Bench - Division Resistance&lt;/strong&gt; (Score (0–1)): leader GPT-5.6 Sol (Low) (0.787), 22 models&lt;br&gt;&lt;span&gt;CellTrack-Bench:frozen c1 hard-test subset,244 neighborhoods; same unannotated frame-pair coordinate interface,crop-and-zoom tool,prompt,parser and token budget. Dataset-macro scores. Pinned primary test configs use GPT5.5/GPT5.6 and Fable5/Opus4.8/Opus5/Sonnet5 Low with final reasoning None after tool withdrawal; Haiku4.5 uses dated20251001 endpoint and None. Open chat runner does not send configured reasoning_effort, so cached Medium is not an evaluated setting. Gemma4 is IT; Qwen3.6 result averages two same-checkpoint replicates. Nearest-neighbor/division resistance and graph-level correctness remain separate metrics. Score (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CellTrack-Bench - Subgraph LNK&lt;/strong&gt; (Score (0–1)): leader GPT-5.6 Sol (Low) (0.767), 22 models&lt;br&gt;&lt;span&gt;CellTrack-Bench:frozen c1 hard-test subset,244 neighborhoods; same unannotated frame-pair coordinate interface,crop-and-zoom tool,prompt,parser and token budget. Dataset-macro scores. Pinned primary test configs use GPT5.5/GPT5.6 and Fable5/Opus4.8/Opus5/Sonnet5 Low with final reasoning None after tool withdrawal; Haiku4.5 uses dated20251001 endpoint and None. Open chat runner does not send configured reasoning_effort, so cached Medium is not an evaluated setting. Gemma4 is IT; Qwen3.6 result averages two same-checkpoint replicates. Nearest-neighbor/division resistance and graph-level correctness remain separate metrics. Score (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DuoCook - Order Completion&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (26.2), 4 models&lt;br&gt;&lt;span&gt;DuoCook:one VLM controls two players on20UnrealEngine maps,one4000-step episode/model/map,shared notebook plus six-response action history. Completion and acceptance average all maps; delivery precision averages only maps with deliveries (Qwen6,Gemini7,Claude5,Astra16),not the same denominator. Engine handles navigation; no APIcost or unbounded rawgame score is admitted. Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DuoCook - Delivery Precision&lt;/strong&gt; (Rate (%)): leader Claude Opus 5 (100.0), 4 models&lt;br&gt;&lt;span&gt;DuoCook:one VLM controls two players on20UnrealEngine maps,one4000-step episode/model/map,shared notebook plus six-response action history. Completion and acceptance average all maps; delivery precision averages only maps with deliveries (Qwen6,Gemini7,Claude5,Astra16),not the same denominator. Engine handles navigation; no APIcost or unbounded rawgame score is admitted. Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DuoCook - Action Acceptance&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (64.6), 4 models&lt;br&gt;&lt;span&gt;DuoCook:one VLM controls two players on20UnrealEngine maps,one4000-step episode/model/map,shared notebook plus six-response action history. Completion and acceptance average all maps; delivery precision averages only maps with deliveries (Qwen6,Gemini7,Claude5,Astra16),not the same denominator. Engine handles navigation; no APIcost or unbounded rawgame score is admitted. Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RobotWorld - Manipulation&lt;/strong&gt; (Tasks solved (/38)): leader GPT-6 Astra (9.0), 5 models&lt;br&gt;&lt;span&gt;RobotWorld:84tasks from20projects,one episode/model/task Sept30–Oct6,2026;fixed domain partition38/20/11/11/4. Direct robot tools,environment-side code control disabled,up to4earlier observations every2rounds. Success at applicable control/non-action budget boundary,retrospective adjudication preserved. Not interchangeable with RoboUse live aggregate or VLA LIBERO cohort. Tasks solved (/38).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RobotWorld - Mobile Manipulation&lt;/strong&gt; (Tasks solved (/20)): leader GPT-6 Astra (4.0), 5 models&lt;br&gt;&lt;span&gt;RobotWorld:84tasks from20projects,one episode/model/task Sept30–Oct6,2026;fixed domain partition38/20/11/11/4. Direct robot tools,environment-side code control disabled,up to4earlier observations every2rounds. Success at applicable control/non-action budget boundary,retrospective adjudication preserved. Not interchangeable with RoboUse live aggregate or VLA LIBERO cohort. Tasks solved (/20).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RobotWorld - Locomotion&lt;/strong&gt; (Tasks solved (/11)): leader Claude Opus 5.5 (1.0), 5 models&lt;br&gt;&lt;span&gt;RobotWorld:84tasks from20projects,one episode/model/task Sept30–Oct6,2026;fixed domain partition38/20/11/11/4. Direct robot tools,environment-side code control disabled,up to4earlier observations every2rounds. Success at applicable control/non-action budget boundary,retrospective adjudication preserved. Not interchangeable with RoboUse live aggregate or VLA LIBERO cohort. Tasks solved (/11).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RobotWorld - Driving&lt;/strong&gt; (Tasks solved (/11)): leader GPT-6 Astra (2.0), 5 models&lt;br&gt;&lt;span&gt;RobotWorld:84tasks from20projects,one episode/model/task Sept30–Oct6,2026;fixed domain partition38/20/11/11/4. Direct robot tools,environment-side code control disabled,up to4earlier observations every2rounds. Success at applicable control/non-action budget boundary,retrospective adjudication preserved. Not interchangeable with RoboUse live aggregate or VLA LIBERO cohort. Tasks solved (/11).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RobotWorld - Aerial&lt;/strong&gt; (Tasks solved (/4)): leader Claude Opus 5.5 (3.0), 5 models&lt;br&gt;&lt;span&gt;RobotWorld:84tasks from20projects,one episode/model/task Sept30–Oct6,2026;fixed domain partition38/20/11/11/4. Direct robot tools,environment-side code control disabled,up to4earlier observations every2rounds. Success at applicable control/non-action budget boundary,retrospective adjudication preserved. Not interchangeable with RoboUse live aggregate or VLA LIBERO cohort. Tasks solved (/4).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RobotWorld - Overall&lt;/strong&gt; (Tasks solved (/84)): leader GPT-6 Astra (16.0), 5 models&lt;br&gt;&lt;span&gt;RobotWorld:84tasks from20projects,one episode/model/task Sept30–Oct6,2026;fixed domain partition38/20/11/11/4. Direct robot tools,environment-side code control disabled,up to4earlier observations every2rounds. Success at applicable control/non-action budget boundary,retrospective adjudication preserved. Not interchangeable with RoboUse live aggregate or VLA LIBERO cohort. Tasks solved (/84).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Kimi Code Bench 2.0 (Claude Code)&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (77.1), 3 models&lt;br&gt;&lt;span&gt;Moonshot AI&amp;#x27;s in-house Kimi Code Bench 2.0 evaluates code agents on end-to-end software engineering tasks across many languages and production stacks. Kimi K3.1 Pro launch post benchmark sheet (October 2026), a separate run from the Kimi K3 report&amp;#x27;s table; score in percent, higher is better. Kimi K3.1 Pro ran in Claude Code at max reasoning effort and the other models at their max-effort setting; the sheet gives no proprietary-model scores for this row.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Kimi Code Bench 3.0 Long-horizon (Claude Code)&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (Max) (66.5), 6 models&lt;br&gt;&lt;span&gt;Moonshot AI&amp;#x27;s in-house Kimi Code Bench 3.0 Long-horizon tests coding agents on multi-step execution, tool use and end-to-end problem solving across diverse engineering domains. Kimi K3.1 Pro launch post benchmark sheet (October 2026); score in percent, higher is better. Kimi K3.1 Pro ran in Claude Code at max reasoning effort and the other models at their max-effort setting. Moonshot notes that its proprietary-model runs may involve fallback behavior.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - DECK-Arena&lt;/strong&gt; (Elo): leader Kimi K3.1 Pro (Max) (1456.0), 5 models&lt;br&gt;&lt;span&gt;Moonshot AI&amp;#x27;s in-house DECK-Arena compares presentation decks generated for real-world queries. Models work across python-pptx, pptxjs and Kimi&amp;#x27;s in-house slide DSL in a harness with integrated search and image generation, and an automated grader calibrated on more than 10,000 human judgments produces Elo scores; higher is better. Unlike the percentage DECK Bench in the Kimi K3 report, this is an Elo arena. Kimi K3.1 Pro launch post benchmark sheet (October 2026); models at max reasoning effort. Moonshot notes that its proprietary-model runs may involve fallback behavior.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Design Bench&lt;/strong&gt; (Elo): leader GPT-6 Astra (Max) (1076.0), 5 models&lt;br&gt;&lt;span&gt;Moonshot AI&amp;#x27;s in-house Design Bench covers graphic design through direct prompting and reference-image few-shot prompting, in DECK-Arena&amp;#x27;s harness (python-pptx, pptxjs and Kimi&amp;#x27;s in-house DSL with integrated search and image generation) with the same automated grader calibrated on more than 10,000 human judgments; Elo, higher is better. Kimi K3.1 Pro launch post benchmark sheet (October 2026); models at max reasoning effort. Moonshot notes that its proprietary-model runs may involve fallback behavior.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Kimi Hallu Bench Hard (with tools)&lt;/strong&gt; (Claim-level Hallucination Rate (%, lower is better)): leader GPT-6 Astra (Max) (5.5), 6 models&lt;br&gt;&lt;span&gt;Moonshot AI&amp;#x27;s in-house hallucination stress test on challenging, de-identified prompts drawn from practical work tasks and crafted to probe situations where models are prone to errors; fact-checking agents assess every claim. This is the sheet&amp;#x27;s Hard set, answered with tools. The score is the claim-level hallucination rate in percent, so lower is better; Moonshot says the rates do not represent overall error rates in daily use. Kimi K3.1 Pro launch post benchmark sheet (October 2026); models at max reasoning effort. Moonshot notes that its proprietary-model runs may involve fallback behavior.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Kimi Hallu Bench (without tools)&lt;/strong&gt; (Claim-level Hallucination Rate (%, lower is better)): leader GPT-6 Astra (Max) (4.2), 6 models&lt;br&gt;&lt;span&gt;Moonshot AI&amp;#x27;s in-house Kimi Hallucination Benchmark on challenging, de-identified prompts drawn from practical work tasks, answered without tools; fact-checking agents assess every claim. The score is the claim-level hallucination rate in percent, so lower is better; the sheet labels only its with-tools row Hard. Kimi K3.1 Pro launch post benchmark sheet (October 2026); models at max reasoning effort. Moonshot notes that its proprietary-model runs may involve fallback behavior.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - xdev-bench-hard&lt;/strong&gt; (pass@3 (%)): leader DeepSeek V4.1 Flash (1017) (Max) (85.7), 4 models&lt;br&gt;&lt;span&gt;Moonshot AI&amp;#x27;s in-house cyber-offense benchmark of expert-curated, source-code-based vulnerability discovery and exploitation tasks, most reproducing recently disclosed vulnerabilities in software such as the Linux kernel, Node.js, JavaScriptCore, Exim and VirGL with mitigations enabled. Given the vulnerable source, a short bug hint and the binary, the agent must achieve arbitrary code execution in a separate sandbox and retrieve a secret string. pass@3 in percent over a random subset of runs, with no hard timeouts or output token limits; higher is better. Kimi K3.1 Pro launch post benchmark sheet (October 2026); Kimi K3.1 Pro ran in KimiCode at max reasoning effort, GLM-5.3 and DeepSeek-V4.1-Flash-1017 in Claude Code.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Terminal-Bench 4.0&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (65.7), 1 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Kimi Claude Code; all competitor cells taken from release blogs or official leaderboard maxima. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Terminal-Bench Science 0.1&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (38.1), 2 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Kimi Claude Code. Printed DS1017 column15.7 actually belongs to older DSFlash; own DS1017 run9.3 only in footnote. Remaining copied from official leaderboard. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - SWE Marathon v1.1&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (51.7), 2 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Kimi and DS1017 Claude Code; all others copied from official leaderboard. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - BenchCAD w/o tools&lt;/strong&gt; (Voxel IoU (%)): leader Kimi K3.1 Pro (Max) (72.8), 2 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Kimi and DS1017 max on random1000-case subset of17900 Vision2Code; official prompts/scoring, single-shot; VoxelIoU percent. Claude copied. Voxel IoU (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - BenchCAD w/ tools&lt;/strong&gt; (Voxel IoU (%)): leader Kimi K3.1 Pro (Max) (92.7), 2 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Kimi and DS1017 max on random1000-case subset of17900 Vision2Code; Claude Code; VoxelIoU percent. Claude and Astra copied. Voxel IoU (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Program Bench&lt;/strong&gt; (Task pass rate at95% threshold (%)): leader Kimi K3.1 Pro (Max) (31.5), 1 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Kimi Claude Code; binary task score1 iff underlying ProgramBench score&amp;gt;=95%; comparator cells copied. Task pass rate at95% threshold (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - DeepSWE v1.1&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (75.5), 1 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Kimi Claude Code; competitor cells from their respective technical reports. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - PostTrain Bench v1.2&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (36.5), 2 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Kimi and DS1017 Claude Code on H20 instead of H100; DS1017 max judge instead of official GPT5.6Terra xhigh; separate protocol from official scores. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - HLE-Full w/ tools&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (70.0), 6 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Release-source reported cohort; no effort beyond explicit max header. Multimodal: GDP.pdf/ZeroBench/ChartoGraphy five runs, others four; MMMU preserves input order with images before text. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - BrowseComp&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (92.1), 3 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Context compaction at200K,500steps maximum then forced answer,temperature1/top-p1. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - BioMysteryBench&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (80.6), 4 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Officialv11, KimiK3judge; KimiCode(Kimi),ClaudeCode(DS/GLM),Codex(Astra); Claude cells copied from system cards. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - CharXiv (RQ) w/ python&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (Max) (93.8), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Release-source reported cohort; no effort beyond explicit max header. Multimodal: GDP.pdf/ZeroBench/ChartoGraphy five runs, others four; MMMU preserves input order with images before text. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - MMMU-Pro w/ python&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (90.0), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Release-source reported cohort; no effort beyond explicit max header. Multimodal: GDP.pdf/ZeroBench/ChartoGraphy five runs, others four; MMMU preserves input order with images before text. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - GDP.pdf w/ python&lt;/strong&gt; (Task-macro rubric pass rate (%)): leader Kimi K3.1 Pro (Max) (85.7), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Original PDF in IPython sandbox; Gemini3.5Flash Medium officialrubricjudge; task-macro criterionpassrate; five runs. Task-macro rubric pass rate (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - ChartoGraphy w/ python&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (Max) (91.0), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Release-source reported cohort; no effort beyond explicit max header. Multimodal: GDP.pdf/ZeroBench/ChartoGraphy five runs, others four; MMMU preserves input order with images before text. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - BabyVision w/ python&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (Max) (97.7), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Release-source reported cohort; no effort beyond explicit max header. Multimodal: GDP.pdf/ZeroBench/ChartoGraphy five runs, others four; MMMU preserves input order with images before text. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - ZeroBench_main w/ python (pass@5)&lt;/strong&gt; (pass@5 (%)): leader Kimi K3.1 Pro (Max) (63.0), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Python tools, pass@five; five runs. pass@5 (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Agents' Last Exam (ALL)&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (43.2), 2 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Officialv1.1,122tasks; bypassing requirements receives0; KimiCode(Kimi),ClaudeCode(DS); all others copied. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Agents' Last Exam (CLI)&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (44.2), 2 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. 86CLI tasks within officialv1.1; bypassing requirements receives0; KimiCode(Kimi),ClaudeCode(DS); all others copied. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - OfficeQA Pro v2&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (64.4), 6 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Entire PDFs rendered as images, no machine-readable text; GPT Codex, all others Claude Code; retrieved reference answers online receive0. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - OfficeQA Pro&lt;/strong&gt; (Score (%)): leader Claude Fable 5.1 (Max) (68.6), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Entire PDFs rendered as images, no machine-readable text; GPT Codex, all others Claude Code; retrieved reference answers online receive0. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Frontier Finance&lt;/strong&gt; (Score (%)): leader Claude Fable 5.1 (Max) (60.8), 6 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Official finance agentv2 harness; preserve release-source cohort. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - Harvey Legal Agent&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (23.7), 3 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. 1251 public tasks, ArtificialAnalysis Stirrup harness, all-rubric-criteria task pass rate. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - HANDBOOK.md&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (44.6), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Official OpenHands harness, early open-sourcev1 tasks. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - OSWorld V2.1&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (Max) (81.8), 4 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. KimiAgent harness, official OSWorldv2 repository. Astra75.8 copied from MuseWonder report. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - GDP.pdf&lt;/strong&gt; (Strict response pass rate (%)): leader Kimi K3.1 Pro (Max) (34.6), 4 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. 1152px page images interleaved with Docling text; Gemini3.5FlashMedium officialrubricjudge; strictpass amongfullyscoredresponses; five runs. DS/GLM cells copiedAA. Strict response pass rate (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - HLE-Full&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (Max) (59.8), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. 2500full questions; GLM43.0* text-onlysubset must be a separate board. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - HLE-Diamond&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (Max) (65.8), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. 1000full questions; GLM25.1* text-onlysubset must be a separate board. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - GPQA-Diamond&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (Max) (96.1), 6 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Release-source reported cohort; no effort beyond explicit max header. Multimodal: GDP.pdf/ZeroBench/ChartoGraphy five runs, others four; MMMU preserves input order with images before text. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - CharXiv (RQ)&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (Max) (92.1), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Release-source reported cohort; no effort beyond explicit max header. Multimodal: GDP.pdf/ZeroBench/ChartoGraphy five runs, others four; MMMU preserves input order with images before text. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - MMMU-Pro&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (Max) (87.7), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Release-source reported cohort; no effort beyond explicit max header. Multimodal: GDP.pdf/ZeroBench/ChartoGraphy five runs, others four; MMMU preserves input order with images before text. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - ZeroBench_main avg@5&lt;/strong&gt; (avg@5 (%)): leader GPT-6 Astra (Max) (43.2), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Two distinct metrics printed percell: averagefive and pass@five; split boards. avg@5 (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - ZeroBench_main pass@5&lt;/strong&gt; (pass@5 (%)): leader GPT-6 Astra (Max) (52.0), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Two distinct metrics printed percell: averagefive and pass@five; split boards. pass@5 (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - PerceptionBench&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (Max) (77.6), 5 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Release-source reported cohort; no effort beyond explicit max header. Multimodal: GDP.pdf/ZeroBench/ChartoGraphy five runs, others four; MMMU preserves input order with images before text. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - ExploitBench&lt;/strong&gt; (pass@5 max capability (%)): leader Kimi K3.1 Pro (Max) (74.7), 1 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. Kimi official harness300turns; pass@5 maxcapability across5seeds;16flags and ACE impliesall16. Competitor cells copied. pass@5 max capability (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro Tech Blog - ExploitGym&lt;/strong&gt; (Score (%)): leader Kimi K3.1 Pro (Max) (20.7), 1 models&lt;br&gt;&lt;span&gt;Moonshot KimiK3.1Pro release evaluations; explicit max header, Kimi temperature1/top-p0.95 except specified overrides. KimiCode,6hour/task budget; competitor cells copied. Score (%). Copied third-party comparator results are not re-ingested; source-specific harness retained.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Kimi K3.1 Pro&lt;/strong&gt; — Aggregate Rating 1934, #12&lt;ul&gt;&lt;li&gt;AA Terminal-Bench 4.0: 67.17 (#1/226)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 89.44 (#1/228)&lt;/li&gt;&lt;li&gt;AutomationBench-AA: 73.3 (#2/229)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Strict: 45.97 (#2/229)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 92.99 (#2/228)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Marketing - Raw Objective Completion: 93.18 (#2/229)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Sales - Raw Objective Completion: 91.83 (#2/229)&lt;/li&gt;&lt;li&gt;AA Long Context Reasoning: 86.33 (#3/581)&lt;/li&gt;&lt;li&gt;AA GDPval: 1845.72 (#3/301)&lt;/li&gt;&lt;li&gt;AA-Briefcase - Presentation Elo: 1665.7 (#3/235)&lt;/li&gt;&lt;li&gt;...and 65 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (17)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on LisanBench: 0.4064 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Kaggle WWTP Engineering: 0.0 (#42)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on LisanBench: 0.6468 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vectara Hallucination Leaderboard: 90.1 (#60)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Kaggle Open Benchmarks - MMLU: 94.14 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on LisanBench: 0.199 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on NYT Connections Extended: 98.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle WWTP Engineering: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on LisanBench: 0.2351 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on NYT Connections Extended: 97.8 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Featherbench: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Featherbench - Rubric Quality: 9.2 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle WWTP Engineering: 100.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on LisanBench: 0.2035 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MirrorCode: 58.9 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wolfram LLM Benchmarking Project: 80.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LisanBench: 1.0 (#1)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (11)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Coding Agent Leaderboard - SWE-Bench Pro -- Ansible (Claude Code)&lt;/strong&gt;: Qwen 3.8 27B (86.5) beat Claude Opus 4.8 by 16.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle WWTP Engineering&lt;/strong&gt;: Gemini 3.8 Flash (100.0) beat Claude Sonnet 4 by 15.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coding Agent Leaderboard - SWE-Bench Pro -- Ansible (OpenCode)&lt;/strong&gt;: Qwen 3.8 27B (89.6) beat Claude Opus 4.8 by 11.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coding Agent Leaderboard - SWE-Bench Verified (OpenCode)&lt;/strong&gt;: Qwen 3.8 27B (94.0) beat Claude Opus 4.8 by 10.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA Terminal-Bench 4.0&lt;/strong&gt;: Kimi K3.1 Pro (67.17) beat Claude Sonnet 5.5 (Max, Default Fallback) by 3.53&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench - MASE&lt;/strong&gt;: timeh2o-v1 (90.39) beat Timestral-209M by 1.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Legal - Criterion Pass Rate&lt;/strong&gt;: Kimi K3.1 Pro (89.44) beat Muse Wonder (Max) by 1.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench - SQL&lt;/strong&gt;: timeh2o-v1 (91.87) beat Timestral-209M by 1.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;YKS Exam 2023 - TYT Social Sciences&lt;/strong&gt;: Jev-1.14 (text input) (20.0) beat Qwen 3.8 27B by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench&lt;/strong&gt;: timeh2o-v1 (90.06) beat Timestral-209M by 0.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Tax Agent Bench&lt;/strong&gt;: qwen3.8-max-internal (49.663) beat Claude Fable 5.1 (Max) by 0.47&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-24

=== FRONTIER ===
  - Best available model: Kimi K3.1 Pro enters at #12 (Aggregate Rating 1934) on 113 benchmarks

=== DAILY ===
NEW BENCHMARKS (62)
  - Legal Agent Benchmark (BenchmarkList Mixed Public and Held-Out) (All-Pass Task Success (system card)): leader Cla</summary></entry><entry><title>The Aggregate Digest — 2026-10-23</title><id>https://theaggregate.ai/digest/2026-10-23</id><updated>2026-10-23T06:25:00.353717+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (15)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;EnvironmentDuels - Difficulty&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (Max) (29.7), 9 models&lt;br&gt;&lt;span&gt;EnvironmentDuels: 90 agent-created environments, 12,960 episodes; each creator makes 10, solver conditions with/without hints share seeds. Difficulty is author failure rate, solver success and hint lift are distinct axes. Native Claude Code, Codex, DeepSeek Harness, Qwen Code, ZCode, Grok Build, Gemini CLI, Kimi Code and OpenCode harnesses; maximal supported reasoning (Gemini High, Qwen/Grok XHigh, others Max). Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnvironmentDuels - Solver Success&lt;/strong&gt; (Score (%)): leader Claude Opus 5 (Max) (94.4), 9 models&lt;br&gt;&lt;span&gt;EnvironmentDuels: 90 agent-created environments, 12,960 episodes; each creator makes 10, solver conditions with/without hints share seeds. Difficulty is author failure rate, solver success and hint lift are distinct axes. Native Claude Code, Codex, DeepSeek Harness, Qwen Code, ZCode, Grok Build, Gemini CLI, Kimi Code and OpenCode harnesses; maximal supported reasoning (Gemini High, Qwen/Grok XHigh, others Max). Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnvironmentDuels - Hint Lift&lt;/strong&gt; (Lift (percentage points)): leader Qwen 3.8 27B (xHigh) (21.2), 9 models&lt;br&gt;&lt;span&gt;EnvironmentDuels: 90 agent-created environments, 12,960 episodes; each creator makes 10, solver conditions with/without hints share seeds. Difficulty is author failure rate, solver success and hint lift are distinct axes. Native Claude Code, Codex, DeepSeek Harness, Qwen Code, ZCode, Grok Build, Gemini CLI, Kimi Code and OpenCode harnesses; maximal supported reasoning (Gemini High, Qwen/Grok XHigh, others Max). Lift (percentage points).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QuantArena - Iterative - Overall&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (xHigh) (51.29), 38 models&lt;br&gt;&lt;span&gt;QuantArena Iterative: shared equity research environment,2022-2023 research and sealed 2024 out-of-sample windows. Version-fixed quality/diversity composite scaled0-100; out-of-sample combination unpublished. Each of 38 requested effort configurations retained; no best-effort pooling. These evaluate agent-generated strategies, not investment advice or model response speed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QuantArena - Iterative - Research Period&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (Low) (58.1), 38 models&lt;br&gt;&lt;span&gt;QuantArena Iterative: shared equity research environment,2022-2023 research and sealed 2024 out-of-sample windows. Version-fixed quality/diversity composite scaled0-100; out-of-sample combination unpublished. Each of 38 requested effort configurations retained; no best-effort pooling. These evaluate agent-generated strategies, not investment advice or model response speed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QuantArena - Iterative - Sealed Period&lt;/strong&gt; (Score (%)): leader GPT-6 Sol (Medium) (52.23), 38 models&lt;br&gt;&lt;span&gt;QuantArena Iterative: shared equity research environment,2022-2023 research and sealed 2024 out-of-sample windows. Version-fixed quality/diversity composite scaled0-100; out-of-sample combination unpublished. Each of 38 requested effort configurations retained; no best-effort pooling. These evaluate agent-generated strategies, not investment advice or model response speed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QuantArena - One-shot - Overall&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (Medium) (42.61), 38 models&lt;br&gt;&lt;span&gt;QuantArena One-shot: shared equity research environment,2022-2023 research and sealed 2024 out-of-sample windows. Version-fixed quality/diversity composite scaled0-100; out-of-sample combination unpublished. Each of 38 requested effort configurations retained; no best-effort pooling. These evaluate agent-generated strategies, not investment advice or model response speed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QuantArena - One-shot - Research Period&lt;/strong&gt; (Score (%)): leader GLM-5.3 (High) (52.54), 38 models&lt;br&gt;&lt;span&gt;QuantArena One-shot: shared equity research environment,2022-2023 research and sealed 2024 out-of-sample windows. Version-fixed quality/diversity composite scaled0-100; out-of-sample combination unpublished. Each of 38 requested effort configurations retained; no best-effort pooling. These evaluate agent-generated strategies, not investment advice or model response speed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;QuantArena - One-shot - Sealed Period&lt;/strong&gt; (Score (%)): leader Claude Opus 5.5 (Medium) (40.65), 38 models&lt;br&gt;&lt;span&gt;QuantArena One-shot: shared equity research environment,2022-2023 research and sealed 2024 out-of-sample windows. Version-fixed quality/diversity composite scaled0-100; out-of-sample combination unpublished. Each of 38 requested effort configurations retained; no best-effort pooling. These evaluate agent-generated strategies, not investment advice or model response speed. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM-LIBERO - LIBERO-10&lt;/strong&gt; (Success Rate (%)): leader GPT-6 Astra (High) (44.0), 18 models&lt;br&gt;&lt;span&gt;LLM-LIBERO: 100 episodes per configuration, 10 long-horizon robot manipulation tasks; visual observations, proprioception and structured pose/gripper tool calls, no task-specific fine-tuning. Six frontier models at Low, Medium and High requested reasoning, 1,800 episodes total. Timeout/failure retained in denominator; cost and call counts omitted. Success Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Silent Cue Loss - Typed&lt;/strong&gt; (Micro F1 (0–1)): leader Claude Opus 4.6 (0.687), 6 models&lt;br&gt;&lt;span&gt;Silent Cue Loss:cross-model appendix, 76–79 documents per model, formatting-aware document QA; Typed serialization, temperature 0. Corpus size differs slightly by completed run; reported micro F1 retained,per-document significance is not a capability score. Micro F1 (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Silent Cue Loss - Untyped&lt;/strong&gt; (Micro F1 (0–1)): leader Claude Opus 4.6 (0.605), 6 models&lt;br&gt;&lt;span&gt;Silent Cue Loss:cross-model appendix, 76–79 documents per model, formatting-aware document QA; Untyped serialization, temperature 0. Corpus size differs slightly by completed run; reported micro F1 retained,per-document significance is not a capability score. Micro F1 (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Silent Cue Loss - docx2python&lt;/strong&gt; (Micro F1 (0–1)): leader Claude Opus 4.6 (0.586), 6 models&lt;br&gt;&lt;span&gt;Silent Cue Loss:cross-model appendix, 76–79 documents per model, formatting-aware document QA; docx2python serialization, temperature 0. Corpus size differs slightly by completed run; reported micro F1 retained,per-document significance is not a capability score. Micro F1 (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Silent Cue Loss - Plain&lt;/strong&gt; (Micro F1 (0–1)): leader Claude Opus 4.6 (0.075), 6 models&lt;br&gt;&lt;span&gt;Silent Cue Loss:cross-model appendix, 76–79 documents per model, formatting-aware document QA; Plain serialization, temperature 0. Corpus size differs slightly by completed run; reported micro F1 retained,per-document significance is not a capability score. Micro F1 (0–1).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA CritPt v2&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (Max) (83.14), 11 models&lt;br&gt;&lt;span&gt;Artificial Analysis independently evaluates 70 revised research-level physics challenges with code execution in an offline Linux sandbox. CritPt authors and physics experts corrected underspecified statements and errors. Requested effort and default-fallback configurations remain separate. The earlier CritPt score is a distinct protocol.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (13)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1800.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on EKI Estonian LLM - Overall: 63.68 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1810.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AutomationBench: 41.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1598.0 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Coarena: 1032.5 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Coarena - Task Completion: 77.1 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on DuelLab Overall: 89.369322241651 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SEAL - Remote Labor Index (RLI): 17.08 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Bug Density: 0.56 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Code Smell Density: 15.53 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Pass Rate: 85.66 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Sonar LLM Leaderboard - Java - Vulnerability Density: 0.31 (#51)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (5)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;o11y-bench - Pass^3&lt;/strong&gt;: Claude Sonnet 5.5 (High) (89.06) beat Claude Opus 4.8 (High) by 9.69&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ParseBench - Charts&lt;/strong&gt;: Upstage Document Parse 2 (Enhanced) (77.26) beat Claude Opus 5.5 (High) by 6.37&lt;/li&gt;&lt;li&gt;&lt;strong&gt;o11y-bench - Pass@3&lt;/strong&gt;: Claude Sonnet 5.5 (High) (100.0) beat Claude Opus 5 (High) by 6.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Coarena - Task Completion&lt;/strong&gt;: Claude Haiku 5.5 (94.3) beat GPT-5.6 Luna by 4.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ParseBench&lt;/strong&gt;: Upstage Document Parse 2 (Enhanced) (80.51) beat Claude Opus 5.5 (High) by 0.66&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-23

=== DAILY ===
NEW BENCHMARKS (15)
  - EnvironmentDuels - Difficulty (Score (%)): leader DeepSeek V4.1 Flash (Max) (29.7), 9 models
      EnvironmentDuels: 90 agent-created environments, 12,960 episodes; each creator makes 10, solver conditions with/without hints sh</summary></entry><entry><title>The Aggregate Digest — 2026-10-22</title><id>https://theaggregate.ai/digest/2026-10-22</id><updated>2026-10-22T07:29:49.306627+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (82)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;GenData Experiment Selection - Headline 281 - Positive Records&lt;/strong&gt; (Paired Accuracy (0–1)): leader Claude Opus 5.5 (High) (0.3713), 2 models&lt;br&gt;&lt;span&gt;GenData v1.0:281paired tasks,Claude Code headless High effort,no tools/web,three attempts. Pair succeeds only when failed run is withheld and matched successful run approved. Claude-created and checked tasks, no practitioner key review; positive-only and negative-record contexts remain separate. Source is frozen maintainer reference, including69private tasks.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GenData Experiment Selection - Headline 281 - Negative Records&lt;/strong&gt; (Paired Accuracy (0–1)): leader Claude Opus 5.5 (High) (0.5552), 2 models&lt;br&gt;&lt;span&gt;GenData v1.0:281paired tasks,Claude Code headless High effort,no tools/web,three attempts. Pair succeeds only when failed run is withheld and matched successful run approved. Claude-created and checked tasks, no practitioner key review; positive-only and negative-record contexts remain separate. Source is frozen maintainer reference, including69private tasks.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AIM Decision Models - Browser 50&lt;/strong&gt; (Task Success (%)): leader GPT-6 Astra (94.0), 60 models&lt;br&gt;&lt;span&gt;50 tasks, one attempt each; browser-use/jev-ultrafast, 900 seconds, 60 browser actions and 120 decision calls. Shared Mercury 2.5 text-entry helper (Mercury 3 on four Celeris tasks). Input-limit failures and all source run variants retained; no inference from the publisher rank flag.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Humanity's Last Exam (with tools))&lt;/strong&gt; (Score (%)): leader Claude Sonnet 5.5 (64.5), 5 models&lt;br&gt;&lt;span&gt;LLM Stats mirror of tool-enabled Humanity’s Last Exam accuracy. Tool access distinguishes this board from closed-book HLE; source-reported models and settings remain separate.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OptionWorld - Source Access&lt;/strong&gt; (Rate (%)): leader DeepSeek V4 Pro (0813) (96.5), 6 models&lt;br&gt;&lt;span&gt;OptionWorld (2026): three runs over192targets,24trajectories per system,60-day evolving environments. API routes use fresh daily conversations,4096completion tokens and16turns/day; temperature/seed omitted. The separate local Astra bridge has only gpt-6-astra-pro-local identity and is not merged into a public release. Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OptionWorld - Risk Hypothesis&lt;/strong&gt; (Rate (%)): leader DeepSeek V4 Pro (0813) (86.3), 6 models&lt;br&gt;&lt;span&gt;OptionWorld (2026): three runs over192targets,24trajectories per system,60-day evolving environments. API routes use fresh daily conversations,4096completion tokens and16turns/day; temperature/seed omitted. The separate local Astra bridge has only gpt-6-astra-pro-local identity and is not merged into a public release. Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OptionWorld - Grounded Discovery&lt;/strong&gt; (Rate (%)): leader DeepSeek V4 Pro (0813) (80.6), 6 models&lt;br&gt;&lt;span&gt;OptionWorld (2026): three runs over192targets,24trajectories per system,60-day evolving environments. API routes use fresh daily conversations,4096completion tokens and16turns/day; temperature/seed omitted. The separate local Astra bridge has only gpt-6-astra-pro-local identity and is not merged into a public release. Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OptionWorld - Timely Grounded Discovery&lt;/strong&gt; (Rate (%)): leader DeepSeek V4 Pro (0813) (78.1), 6 models&lt;br&gt;&lt;span&gt;OptionWorld (2026): three runs over192targets,24trajectories per system,60-day evolving environments. API routes use fresh daily conversations,4096completion tokens and16turns/day; temperature/seed omitted. The separate local Astra bridge has only gpt-6-astra-pro-local identity and is not merged into a public release. Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OptionWorld - Intervention Delivered&lt;/strong&gt; (Rate (%)): leader OptionWorld Astra Codex local slot (68.6), 6 models&lt;br&gt;&lt;span&gt;OptionWorld (2026): three runs over192targets,24trajectories per system,60-day evolving environments. API routes use fresh daily conversations,4096completion tokens and16turns/day; temperature/seed omitted. The separate local Astra bridge has only gpt-6-astra-pro-local identity and is not merged into a public release. Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OptionWorld - Target Success&lt;/strong&gt; (Rate (%)): leader OptionWorld Astra Codex local slot (61.6), 6 models&lt;br&gt;&lt;span&gt;OptionWorld (2026): three runs over192targets,24trajectories per system,60-day evolving environments. API routes use fresh daily conversations,4096completion tokens and16turns/day; temperature/seed omitted. The separate local Astra bridge has only gpt-6-astra-pro-local identity and is not merged into a public release. Rate (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Recognition and Grounding - Answer Accuracy&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (85.3), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Recognition and Grounding - Coverage Grounded Score&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (84.0), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Recognition and Grounding - Bottleneck Grounded Score&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (49.1), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Basic Understanding - Answer Accuracy&lt;/strong&gt; (Score (%)): leader Gemini 3.8 Flash (73.9), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Basic Understanding - Coverage Grounded Score&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (65.3), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Basic Understanding - Bottleneck Grounded Score&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (29.2), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Complex Reasoning - Answer Accuracy&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (79.1), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Complex Reasoning - Coverage Grounded Score&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (49.1), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Complex Reasoning - Bottleneck Grounded Score&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (12.3), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Overall - Answer Accuracy&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (79.4), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Overall - Coverage Grounded Score&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (70.8), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EvaOmni - Overall - Bottleneck Grounded Score&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (35.1), 24 models&lt;br&gt;&lt;span&gt;EvaOmni (2026):1027questions from392videos,1FPS and480px long side with audio; shared prompt. Open non-MiMo models use recommended card configurations on8H20GPUs; MiMo and closed models use official APIs. Coverage/bottleneck grounding score correct answers against annotated audio and visual intervals; expert reference uses replay and is retained separately. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - ES&lt;/strong&gt; (Published ES Score): leader Claude Opus 5.5 (0.944), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published ES Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - TQ&lt;/strong&gt; (Published TQ Score): leader GPT-6.1 Sol (0.117), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published TQ Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - AQ&lt;/strong&gt; (Published AQ Score): leader Claude Opus 5.5 (3.636), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published AQ Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - MS&lt;/strong&gt; (Published MS Score): leader Claude Opus 5.5 (0.933), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published MS Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - L-TVA&lt;/strong&gt; (Published L-TVA Score): leader Claude Opus 5.5 (0.522), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published L-TVA Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - G-TVA&lt;/strong&gt; (Published G-TVA Score): leader Claude Opus 5.5 (0.653), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published G-TVA Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - IF&lt;/strong&gt; (Published IF Score): leader Claude Opus 5.5 (0.915), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published IF Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - CC&lt;/strong&gt; (Published CC Score): leader Claude Opus 5.5 (4.047), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published CC Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - VD&lt;/strong&gt; (Published VD Score): leader GPT-6.1 Sol (3.883), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published VD Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - MQ&lt;/strong&gt; (Published MQ Score): leader Claude Sonnet 5.5 (3.847), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published MQ Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - T2V - VE&lt;/strong&gt; (Published VE Score): leader Claude Sonnet 5.5 (3.9), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):360fixed T2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published VE Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - ES&lt;/strong&gt; (Published ES Score): leader Claude Opus 5.5 (0.96), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published ES Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - TQ&lt;/strong&gt; (Published TQ Score): leader Claude Sonnet 5.5 (0.114), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published TQ Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - AQ&lt;/strong&gt; (Published AQ Score): leader Claude Opus 5.5 (3.754), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published AQ Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - MS&lt;/strong&gt; (Published MS Score): leader Claude Opus 5.5 (0.948), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published MS Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - L-TVA&lt;/strong&gt; (Published L-TVA Score): leader Claude Opus 5.5 (0.517), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published L-TVA Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - G-TVA&lt;/strong&gt; (Published G-TVA Score): leader Claude Opus 5.5 (0.625), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published G-TVA Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - PF&lt;/strong&gt; (Published PF Score): leader Claude Opus 5.5 (0.894), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published PF Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - SF&lt;/strong&gt; (Published SF Score): leader Claude Opus 5.5 (0.658), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published SF Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - IF&lt;/strong&gt; (Published IF Score): leader Claude Opus 5.5 (0.915), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published IF Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - CC&lt;/strong&gt; (Published CC Score): leader Claude Opus 5.5 (3.934), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published CC Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - VD&lt;/strong&gt; (Published VD Score): leader Claude Opus 5.5 (3.784), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published VD Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - MQ&lt;/strong&gt; (Published MQ Score): leader Claude Opus 5.5 (3.755), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published MQ Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - I2V - VE&lt;/strong&gt; (Published VE Score): leader Claude Opus 5.5 (3.818), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed I2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published VE Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - ES&lt;/strong&gt; (Published ES Score): leader Claude Opus 5.5 (0.971), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published ES Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - TQ&lt;/strong&gt; (Published TQ Score): leader Claude Sonnet 5.5 (0.118), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published TQ Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - AQ&lt;/strong&gt; (Published AQ Score): leader Claude Opus 5.5 (3.794), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published AQ Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - MS&lt;/strong&gt; (Published MS Score): leader Claude Opus 5.5 (0.958), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published MS Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - L-TVA&lt;/strong&gt; (Published L-TVA Score): leader Claude Sonnet 5.5 (0.528), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published L-TVA Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - G-TVA&lt;/strong&gt; (Published G-TVA Score): leader Claude Opus 5.5 (0.63), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published G-TVA Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - SA&lt;/strong&gt; (Published SA Score): leader Claude Opus 5.5 (0.882), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published SA Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - AF&lt;/strong&gt; (Published AF Score): leader Claude Opus 5.5 (0.787), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published AF Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - MF&lt;/strong&gt; (Published MF Score): leader Claude Opus 5.5 (0.445), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published MF Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - IF&lt;/strong&gt; (Published IF Score): leader Claude Opus 5.5 (0.929), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published IF Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - CC&lt;/strong&gt; (Published CC Score): leader Claude Opus 5.5 (3.997), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published CC Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - VD&lt;/strong&gt; (Published VD Score): leader Claude Opus 5.5 (3.853), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published VD Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - MQ&lt;/strong&gt; (Published MQ Score): leader Claude Opus 5.5 (3.839), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published MQ Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Code2Video-Compass - V2V - VE&lt;/strong&gt; (Published VE Score): leader Claude Opus 5.5 (3.882), 15 models&lt;br&gt;&lt;span&gt;Code2Video-Compass (2026):347fixed V2Vtasks;15models generate code in one API call with provider-default reasoning, no execution feedback or repair. Failures score zero on quality metrics. V2V GPT/Claude receive4FPSframes, others native video at4FPS. Execution, image/video similarity and Gemini3.8Flash rubric scores remain separate; runtime tokens and average ranks omitted. Published VE Score.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CyberArena - Expected Contest Score&lt;/strong&gt; (Posterior Expected Score (%)): leader GPT-5.6 Sol (78.6), 12 models&lt;br&gt;&lt;span&gt;CyberArena:12models in their developer coding harness or OpenCode;40challenge reference set. Conditional posterior mean attack,defense and service-availability score, averaged over opponents/challenges; not observed win rate. Adaptive game selection and native harnesses; requested reasoning effort is not specified. Posterior Expected Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spacebar Workspace Pilot - Overall&lt;/strong&gt; (Artifact Score (0–100)): leader GPT-6.1 Sol (Medium) (90.0), 6 models&lt;br&gt;&lt;span&gt;Spacebar October14 workspace pilot:14sequential user requests in a persistent workspace, scored on retained artifacts; one campaign per model. Direct/OpenRouter route and requested medium/max/high setting follow source; no repeated-run estimate. Research,spreadsheet,collaborative app,memo,consistency and recovery rubric. Artifact Score (0–100).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spacebar Workspace Pilot - Research and evidence&lt;/strong&gt; (Rubric Points): leader GPT-6.1 Sol (Medium) (13.0), 6 models&lt;br&gt;&lt;span&gt;Spacebar October14 workspace pilot:14sequential user requests in a persistent workspace, scored on retained artifacts; one campaign per model. Direct/OpenRouter route and requested medium/max/high setting follow source; no repeated-run estimate. Research,spreadsheet,collaborative app,memo,consistency and recovery rubric. Rubric Points.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spacebar Workspace Pilot - Spreadsheet&lt;/strong&gt; (Rubric Points): leader GPT-6.1 Sol (Medium) (23.0), 6 models&lt;br&gt;&lt;span&gt;Spacebar October14 workspace pilot:14sequential user requests in a persistent workspace, scored on retained artifacts; one campaign per model. Direct/OpenRouter route and requested medium/max/high setting follow source; no repeated-run estimate. Research,spreadsheet,collaborative app,memo,consistency and recovery rubric. Rubric Points.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spacebar Workspace Pilot - Collaborative app&lt;/strong&gt; (Rubric Points): leader GPT-6.1 Sol (Medium) (14.0), 6 models&lt;br&gt;&lt;span&gt;Spacebar October14 workspace pilot:14sequential user requests in a persistent workspace, scored on retained artifacts; one campaign per model. Direct/OpenRouter route and requested medium/max/high setting follow source; no repeated-run estimate. Research,spreadsheet,collaborative app,memo,consistency and recovery rubric. Rubric Points.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spacebar Workspace Pilot - CFO memo&lt;/strong&gt; (Rubric Points): leader GPT-6.1 Sol (Medium) (14.0), 6 models&lt;br&gt;&lt;span&gt;Spacebar October14 workspace pilot:14sequential user requests in a persistent workspace, scored on retained artifacts; one campaign per model. Direct/OpenRouter route and requested medium/max/high setting follow source; no repeated-run estimate. Research,spreadsheet,collaborative app,memo,consistency and recovery rubric. Rubric Points.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spacebar Workspace Pilot - Cross-artifact consistency&lt;/strong&gt; (Rubric Points): leader GPT-6.1 Sol (Medium) (15.0), 6 models&lt;br&gt;&lt;span&gt;Spacebar October14 workspace pilot:14sequential user requests in a persistent workspace, scored on retained artifacts; one campaign per model. Direct/OpenRouter route and requested medium/max/high setting follow source; no repeated-run estimate. Research,spreadsheet,collaborative app,memo,consistency and recovery rubric. Rubric Points.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Spacebar Workspace Pilot - Reliability and recovery&lt;/strong&gt; (Rubric Points): leader GPT-6.1 Sol (Medium) (11.0), 6 models&lt;br&gt;&lt;span&gt;Spacebar October14 workspace pilot:14sequential user requests in a persistent workspace, scored on retained artifacts; one campaign per model. Direct/OpenRouter route and requested medium/max/high setting follow source; no repeated-run estimate. Research,spreadsheet,collaborative app,memo,consistency and recovery rubric. Rubric Points.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Overall&lt;/strong&gt; (Score (%)): leader GPT-6.1 Sol (High) (67.68), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Tunnel risk review&lt;/strong&gt; (Score (%)): leader GOMAX deepseek-flash (High) (70.0), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Airfoil research review&lt;/strong&gt; (Score (%)): leader GPT-6.1 Sol (High) (81.1), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Airfoil retest handoff&lt;/strong&gt; (Score (%)): leader GPT-6.1 Sol (High) (91.7), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Compressor commissioning&lt;/strong&gt; (Score (%)): leader GPT-6.1 Sol (High) (87.0), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Geoscience field-course planning&lt;/strong&gt; (Score (%)): leader GPT-6.1 Sol (High) (63.7), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Microbial remediation&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (High) (70.0), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Thyroid clinical-data delivery&lt;/strong&gt; (Score (%)): leader GOMAX deepseek-flash (High) (76.3), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Hydrogenation process selection&lt;/strong&gt; (Score (%)): leader GPT-6.1 Sol (High) (82.5), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Medical-device registration&lt;/strong&gt; (Score (%)): leader GOMAX deepseek-flash (High) (38.0), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - IVD reagent-kit registration&lt;/strong&gt; (Score (%)): leader GOMAX deepseek-flash (High) (12.0), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Community-service operations&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (High) (100.0), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GOMAX - Rongqiao service reorganization&lt;/strong&gt; (Score (%)): leader GPT-6.1 Sol (High) (98.5), 7 models&lt;br&gt;&lt;span&gt;GOMAX October18 snapshot:Claude Code harness,High reasoning,professional deliverables from Chinese industrial cases across five domains. Weighted artifact rubrics,programmatic or model-based verification. deepseek-flash lacks an exact API generation binding and remains source-qualified. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Finance Index&lt;/strong&gt; (Finance Index (%)): leader Claude Opus 5.5 (Max) (65.587), 28 models&lt;br&gt;&lt;span&gt;Vals Finance Index v1: native overall aggregate across the released financial evaluations; preserves every published model and explicit reasoning setting.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (24)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on MLCR-AA: 18.33 (#58)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on MLCR-AA - Accuracy: 28.89 (#97)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on MLCR-AA - Completeness: 51.67 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MLCR-AA: 25.0 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MLCR-AA - Accuracy: 37.22 (#71)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MLCR-AA - Completeness: 51.67 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Router Index: 64.156 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AutomationBench: 41.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Kaggle Open Benchmarks - MMLU-Pro: 89.8 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on MLCR-AA: 38.33 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on MLCR-AA - Accuracy: 95.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on MLCR-AA - Completeness: 40.56 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Router Index: 65.023 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (Agents - Web Apps): 1306.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on MLCR-AA: 35.56 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on MLCR-AA - Accuracy: 93.89 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on MLCR-AA - Completeness: 36.11 (#34)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Router Index: 63.944 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Mirrorcode: 58.89 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MLCR-AA: 13.89 (#83)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MLCR-AA - Accuracy: 32.78 (#80)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MLCR-AA - Completeness: 31.67 (#51)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SEAL - MultiChallenge: 78.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Router Index: 66.787 (#3)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (6)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Vals AI Router Index&lt;/strong&gt;: Gemini 4 Argon (69.237) beat Switchyard by 11.44&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MMLongBench-Doc - Accuracy&lt;/strong&gt;: Nemotron 3.5 Super VL 120B A12B (66.6) beat Claude Opus 4.5 by 4.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MMEB Leaderboard&lt;/strong&gt;: Sentio (61.69) beat Wianmu-5B-v0 by 1.24&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (LongBench v2)&lt;/strong&gt;: Beam 1.0 (67.5) beat Qwen 3.8 Max by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Open Universal Arabic ASR Leaderboard - CER&lt;/strong&gt;: Jais-ASR (8.31) beat Audar-ASR-V1-Turbo by 0.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Open Universal Arabic ASR Leaderboard&lt;/strong&gt;: Jais-ASR (22.82) beat Audar-ASR-V1-Turbo by 0.35&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-22

=== DAILY ===
NEW BENCHMARKS (82)
  - GenData Experiment Selection - Headline 281 - Positive Records (Paired Accuracy (0–1)): leader Claude Opus 5.5 (High) (0.3713), 2 models
      GenData v1.0:281paired tasks,Claude Code headless High effort,no tools/web,three att</summary></entry><entry><title>The Aggregate Digest — 2026-10-21</title><id>https://theaggregate.ai/digest/2026-10-21</id><updated>2026-10-21T07:04:03.722121+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (78)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Vals AI Finance Agent v2.1&lt;/strong&gt; (Dealbreaker-gated Partial Credit (%)): leader Claude Opus 5.5 (Max) (67.846), 28 models&lt;br&gt;&lt;span&gt;Finance Agent v2.1: 449 private test questions across nine financial-analysis categories, dealbreaker-gated partial credit. Revised judge jury, hedged-answer prompt and expert-reviewed questions/rubrics; not comparable with v2. 27 public examples and 450 licensed validation questions are separate.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI ClaimBench&lt;/strong&gt; (Accuracy (%)): leader Claude Sonnet 5.5 (Max) (100.0), 19 models&lt;br&gt;&lt;span&gt;ClaimBench v1: 1,960 claims paired with 10-K excerpts; three-way supported, contradicted or insufficient-evidence label accuracy. Five construction categories have roughly 392 claims each. This is the full label-accuracy table, not selective coverage or the threshold-tuned cascade on a 70% holdout. Public examples are available.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Excel Modeling v1.1&lt;/strong&gt; (Accuracy (%)): leader Claude Fable 5.1 (Max) (76.728), 30 models&lt;br&gt;&lt;span&gt;EMB v1.1: financial workbook creation in template and scratch modes; equal-weight seven-category accuracy. Template cells match gold workbooks after Microsoft Excel recalculation; scratch rubrics use GPT-6 Luna. Revised judge and sheet-name matching differ from v1.0; 3.5-hour limit and private scored tasks.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Harvey Legal Agent Bench v1.1&lt;/strong&gt; (Task Resolution (%)): leader Gemini 4 Argon (High) (27.083), 24 models&lt;br&gt;&lt;span&gt;HLAB-Vals v1.1: 120 private legal tasks, all-criteria task pass rates averaged over GPT-5.5 and Claude Sonnet 4.6 judges. Removes 17 unsolvable criteria, replaces one document, and updates harness, context compaction and finish handling. All models rerun; distinct from archived v1 and LAB-AA.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Code Migration v1.1&lt;/strong&gt; (Hidden-test Pass Rate (%)): leader Claude Sonnet 5.5 (Max) (86.875), 26 models&lt;br&gt;&lt;span&gt;Code Migration v1.1: Pi-agent offline migration with revised hidden tests, distinct from v1 mini-SWE-agent runs. 30 CLI repositories each migrated into four target languages and 10 COBOL-to-Java repositories; source repositories equally weighted (3:1 CLI/COBOL). Anti-cheat validation; headline hidden-test pass rate excludes separate style ratings.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RefCraft - Overall&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (52.12), 9 models&lt;br&gt;&lt;span&gt;RefCraft (2026): reference-driven web interface implementation; B is keypoint completion, C core functionality and R reference following. Overall is the mean of each task’s minimum of B, C and R, not the minimum of their aggregate means. Nine model agents share the published protocol; provider token accounting is not a capability score. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RefCraft - Keypoint Completion&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (75.68), 9 models&lt;br&gt;&lt;span&gt;RefCraft (2026): reference-driven web interface implementation; B is keypoint completion, C core functionality and R reference following. Overall is the mean of each task’s minimum of B, C and R, not the minimum of their aggregate means. Nine model agents share the published protocol; provider token accounting is not a capability score. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RefCraft - Core Functionality&lt;/strong&gt; (Score (%)): leader GPT-6 Astra (94.89), 9 models&lt;br&gt;&lt;span&gt;RefCraft (2026): reference-driven web interface implementation; B is keypoint completion, C core functionality and R reference following. Overall is the mean of each task’s minimum of B, C and R, not the minimum of their aggregate means. Nine model agents share the published protocol; provider token accounting is not a capability score. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;RefCraft - Reference Following&lt;/strong&gt; (Score (%)): leader Grok 4.6 (66.25), 9 models&lt;br&gt;&lt;span&gt;RefCraft (2026): reference-driven web interface implementation; B is keypoint completion, C core functionality and R reference following. Overall is the mean of each task’s minimum of B, C and R, not the minimum of their aggregate means. Nine model agents share the published protocol; provider token accounting is not a capability score. Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;KORA V2 - Child Safety&lt;/strong&gt; (Safety Score (%)): leader GPT-6 Astra (73.0), 42 models&lt;br&gt;&lt;span&gt;KORA V2 (2026): child and teen safety in conversations, scored against the published V2 rubric. Versioned safety behavior, not general knowledge or reasoning accuracy. Exact public model labels retained; no merge with V1 or unreleased V3. Safety Score (%).&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ProteinGeoBench - Main 130 Tasks&lt;/strong&gt; (Mean Reward (0–1)): leader Claude Opus 5 (Medium) (0.816), 4 models&lt;br&gt;&lt;span&gt;ProteinGeoBench (2026): Harbor executable-verifier mean reward over 130 protein tasks measured for all four agents, from a 131-task suite; medium reasoning requested, default temperature, up to eight runs per task (ten for Gemini). Internet and structure tools allowed; task-balanced mean, not perfect-solve rate. Mean Reward (0–1); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ProteinGeoBench - Selected 20 Tasks&lt;/strong&gt; (Mean Reward (0–1)): leader Claude Opus 5 (0.597), 26 models&lt;br&gt;&lt;span&gt;ProteinGeoBench (2026): 26 models on 20 discrimination-selected tasks, 2048 graded trials, up to four runs per task; temperature 1 where accepted and no explicit reasoning setting. Different tasks and sampling from the main four-model comparison; no cross-table score pooling. Mean Reward (0–1); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PhysRSI - Initial Performance&lt;/strong&gt; (Normalized Score (0–1)): leader GPT-6 Astra (xHigh) (0.3676), 6 models&lt;br&gt;&lt;span&gt;PhysRSI-eval (2026): six model/runtime systems on 18 robotic simulation tasks; initial K0 scores averaged over three runs per task, followed by iterative K1–K17 revisions. Codex uses 4200-second reasoning/1800-second revision budgets; Claude Code uses 4200/600. Runtime settings are part of these descriptive system comparisons. Normalized Score (0–1); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PhysRSI - Success&lt;/strong&gt; (Normalized Score (0–1)): leader GPT-6 Astra (xHigh) (0.8509), 6 models&lt;br&gt;&lt;span&gt;PhysRSI-eval (2026): six model/runtime systems on 18 robotic simulation tasks; initial K0 scores averaged over three runs per task, followed by iterative K1–K17 revisions. Codex uses 4200-second reasoning/1800-second revision budgets; Claude Code uses 4200/600. Runtime settings are part of these descriptive system comparisons. Normalized Score (0–1); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WildAWM OpenCode - Online-Mind2Web&lt;/strong&gt; (Pass Rate (%)): leader Claude Opus 5 (High) (96.6), 10 models&lt;br&gt;&lt;span&gt;WildAWM (2026): common OpenCode v1.17.7 agent harness, high reasoning effort, eight public web, claw and computer-use evaluations. Fine-tuned WildAWM-4B and WildAWM-35B-A3B remain distinct from Qwen baselines; cited OpenForgeRL and CUA-Gym rows use different harnesses and are not pooled into this protocol. Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WildAWM OpenCode - WebArena&lt;/strong&gt; (Pass Rate (%)): leader Claude Opus 5 (High) (75.66), 10 models&lt;br&gt;&lt;span&gt;WildAWM (2026): common OpenCode v1.17.7 agent harness, high reasoning effort, eight public web, claw and computer-use evaluations. Fine-tuned WildAWM-4B and WildAWM-35B-A3B remain distinct from Qwen baselines; cited OpenForgeRL and CUA-Gym rows use different harnesses and are not pooled into this protocol. Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WildAWM OpenCode - QwenClawBench&lt;/strong&gt; (Mean Score (%)): leader GPT-6 Astra (High) (65.21), 10 models&lt;br&gt;&lt;span&gt;WildAWM (2026): common OpenCode v1.17.7 agent harness, high reasoning effort, eight public web, claw and computer-use evaluations. Fine-tuned WildAWM-4B and WildAWM-35B-A3B remain distinct from Qwen baselines; cited OpenForgeRL and CUA-Gym rows use different harnesses and are not pooled into this protocol. Mean Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WildAWM OpenCode - Claw-Eval&lt;/strong&gt; (Pass Rate (%)): leader GPT-6 Astra (High) (80.33), 10 models&lt;br&gt;&lt;span&gt;WildAWM (2026): common OpenCode v1.17.7 agent harness, high reasoning effort, eight public web, claw and computer-use evaluations. Fine-tuned WildAWM-4B and WildAWM-35B-A3B remain distinct from Qwen baselines; cited OpenForgeRL and CUA-Gym rows use different harnesses and are not pooled into this protocol. Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WildAWM OpenCode - CUA-Gym&lt;/strong&gt; (Mean Score (%)): leader Claude Opus 4.8 (High) (84.52), 10 models&lt;br&gt;&lt;span&gt;WildAWM (2026): common OpenCode v1.17.7 agent harness, high reasoning effort, eight public web, claw and computer-use evaluations. Fine-tuned WildAWM-4B and WildAWM-35B-A3B remain distinct from Qwen baselines; cited OpenForgeRL and CUA-Gym rows use different harnesses and are not pooled into this protocol. Mean Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WildAWM OpenCode - OSWorld-Verified&lt;/strong&gt; (Mean Score (%)): leader Claude Opus 5 (High) (74.64), 10 models&lt;br&gt;&lt;span&gt;WildAWM (2026): common OpenCode v1.17.7 agent harness, high reasoning effort, eight public web, claw and computer-use evaluations. Fine-tuned WildAWM-4B and WildAWM-35B-A3B remain distinct from Qwen baselines; cited OpenForgeRL and CUA-Gym rows use different harnesses and are not pooled into this protocol. Mean Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WildAWM OpenCode - OSWorld-2&lt;/strong&gt; (Mean Score (%)): leader GPT-6 Astra (High) (60.77), 10 models&lt;br&gt;&lt;span&gt;WildAWM (2026): common OpenCode v1.17.7 agent harness, high reasoning effort, eight public web, claw and computer-use evaluations. Fine-tuned WildAWM-4B and WildAWM-35B-A3B remain distinct from Qwen baselines; cited OpenForgeRL and CUA-Gym rows use different harnesses and are not pooled into this protocol. Mean Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WildAWM OpenCode - Agents Last Exam&lt;/strong&gt; (Mean Score (%)): leader GPT-6 Astra (High) (56.58), 10 models&lt;br&gt;&lt;span&gt;WildAWM (2026): common OpenCode v1.17.7 agent harness, high reasoning effort, eight public web, claw and computer-use evaluations. Fine-tuned WildAWM-4B and WildAWM-35B-A3B remain distinct from Qwen baselines; cited OpenForgeRL and CUA-Gym rows use different harnesses and are not pooled into this protocol. Mean Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WildAWM OpenCode - Average&lt;/strong&gt; (Mean Score (%)): leader GPT-6 Astra (High) (73.11), 10 models&lt;br&gt;&lt;span&gt;WildAWM (2026): common OpenCode v1.17.7 agent harness, high reasoning effort, eight public web, claw and computer-use evaluations. Fine-tuned WildAWM-4B and WildAWM-35B-A3B remain distinct from Qwen baselines; cited OpenForgeRL and CUA-Gym rows use different harnesses and are not pooled into this protocol. Mean Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HERA-Bench - Base Harness - Abstention&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (High) (81.7), 20 models&lt;br&gt;&lt;span&gt;HERA-Bench (2026): explicit Appendix A1.T4 reasoning and routed precision retained per model; 60 matched feasible/infeasible pairs, 120 tasks with executable verifiers. The final harness is evolved using GPT-5.6 Luna and fixed for cross-model evaluation; the base harness is separate. Paired success requires correct action and correct abstention on the same pair; no training-pool measurements mixed in. Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HERA-Bench - Base Harness - Paired Success&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (High) (70.0), 20 models&lt;br&gt;&lt;span&gt;HERA-Bench (2026): explicit Appendix A1.T4 reasoning and routed precision retained per model; 60 matched feasible/infeasible pairs, 120 tasks with executable verifiers. The final harness is evolved using GPT-5.6 Luna and fixed for cross-model evaluation; the base harness is separate. Paired success requires correct action and correct abstention on the same pair; no training-pool measurements mixed in. Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HERA-Bench - Final Harness - Abstention&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (High) (91.7), 20 models&lt;br&gt;&lt;span&gt;HERA-Bench (2026): explicit Appendix A1.T4 reasoning and routed precision retained per model; 60 matched feasible/infeasible pairs, 120 tasks with executable verifiers. The final harness is evolved using GPT-5.6 Luna and fixed for cross-model evaluation; the base harness is separate. Paired success requires correct action and correct abstention on the same pair; no training-pool measurements mixed in. Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HERA-Bench - Final Harness - Paired Success&lt;/strong&gt; (Rate (%)): leader GPT-6 Astra (High) (80.0), 20 models&lt;br&gt;&lt;span&gt;HERA-Bench (2026): explicit Appendix A1.T4 reasoning and routed precision retained per model; 60 matched feasible/infeasible pairs, 120 tasks with executable verifiers. The final harness is evolved using GPT-5.6 Luna and fixed for cross-model evaluation; the base harness is separate. Paired success requires correct action and correct abstention on the same pair; no training-pool measurements mixed in. Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FunCAD - Requirements only&lt;/strong&gt; (Tasks Solved (of 46)): leader Grok 4.6 (xHigh) (3.0), 2 models&lt;br&gt;&lt;span&gt;FunCAD (2026): 46 functional CAD tasks, one independent attempt per task family within three hours active time, ReAct construction with the named support condition; constructors use Codex GPT-6 Astra High and Cursor Grok 4.6 extra-high. Separate final evaluation uses GPT-6 Astra medium reasoning and one-million-token context to bind physical requirements to simulation tests. Grok history/multiple attempts are a separate experiment. Tasks Solved (of 46); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FunCAD - Simulation tools&lt;/strong&gt; (Tasks Solved (of 46)): leader Grok 4.6 (xHigh) (10.0), 2 models&lt;br&gt;&lt;span&gt;FunCAD (2026): 46 functional CAD tasks, one independent attempt per task family within three hours active time, ReAct construction with the named support condition; constructors use Codex GPT-6 Astra High and Cursor Grok 4.6 extra-high. Separate final evaluation uses GPT-6 Astra medium reasoning and one-million-token context to bind physical requirements to simulation tests. Grok history/multiple attempts are a separate experiment. Tasks Solved (of 46); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FunCAD - Binary feedback&lt;/strong&gt; (Tasks Solved (of 46)): leader Grok 4.6 (xHigh) (8.0), 2 models&lt;br&gt;&lt;span&gt;FunCAD (2026): 46 functional CAD tasks, one independent attempt per task family within three hours active time, ReAct construction with the named support condition; constructors use Codex GPT-6 Astra High and Cursor Grok 4.6 extra-high. Separate final evaluation uses GPT-6 Astra medium reasoning and one-million-token context to bind physical requirements to simulation tests. Grok history/multiple attempts are a separate experiment. Tasks Solved (of 46); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FunCAD - No constructor vision&lt;/strong&gt; (Tasks Solved (of 46)): leader GPT-6 Astra (High) (20.0), 2 models&lt;br&gt;&lt;span&gt;FunCAD (2026): 46 functional CAD tasks, one independent attempt per task family within three hours active time, ReAct construction with the named support condition; constructors use Codex GPT-6 Astra High and Cursor Grok 4.6 extra-high. Separate final evaluation uses GPT-6 Astra medium reasoning and one-million-token context to bind physical requirements to simulation tests. Grok history/multiple attempts are a separate experiment. Tasks Solved (of 46); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FunCAD - Full harness&lt;/strong&gt; (Tasks Solved (of 46)): leader GPT-6 Astra (High) (22.0), 2 models&lt;br&gt;&lt;span&gt;FunCAD (2026): 46 functional CAD tasks, one independent attempt per task family within three hours active time, ReAct construction with the named support condition; constructors use Codex GPT-6 Astra High and Cursor Grok 4.6 extra-high. Separate final evaluation uses GPT-6 Astra medium reasoning and one-million-token context to bind physical requirements to simulation tests. Grok history/multiple attempts are a separate experiment. Tasks Solved (of 46); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - CountQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (82.0), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - CountQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (86.4), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - FSC-147&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (44.01), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - FSC-147&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (46.11), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - OdinW-13-All&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (50.46), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - OdinW-13-All&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (60.45), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - FSCD-147&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (15.49), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - FSCD-147&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (30.33), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - FSCD-LVIS&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (10.55), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - FSCD-LVIS&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (36.02), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - MMSearch&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (57.31), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - MMSearch&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (74.27), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - MMSearch-Plus&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (27.97), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - MMSearch-Plus&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (53.7), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - RealX-Bench&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (44.33), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - RealX-Bench&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (53.09), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - FVQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (60.67), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - FVQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (74.67), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - HLE-VL&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (49.09), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - HLE-VL&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (56.06), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - BC-VL&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (46.12), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - BC-VL&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (63.91), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - RealWorldQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (87.58), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - RealWorldQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (92.16), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - SimpleVQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (62.17), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - SimpleVQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (76.1), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - HallusionBench&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (78.75), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - HallusionBench&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (83.78), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - FREAK-VQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (63.2), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - FREAK-VQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (73.34), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - FREAK-MCQ&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (76.26), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - FREAK-MCQ&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (83.9), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - V*&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (93.72), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - V*&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (97.38), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - MMSI-Bench&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (65.2), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - MMSI-Bench&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (73.9), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - ERQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (76.5), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - ERQA&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (81.25), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - EmbSpatial&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (82.99), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - EmbSpatial&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (85.14), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - SparBench&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (63.68), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - SparBench&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (68.17), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - MindCube&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (87.79), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - MindCube&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (93.65), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - Base - Average&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (60.27), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; Base protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenVLHarness Study - OpenVLHarness - Average&lt;/strong&gt; (Published Task Score (%)): leader GPT-6 Astra (Medium) (70.17), 4 models&lt;br&gt;&lt;span&gt;OpenVLHarness (2026): full evaluation splits of 22 visual benchmarks, four models; OpenVLHarness protocol. Scores follow each source task metric (accuracy, F1 or grounding AP), all on 0–100; Average is the published equal-benchmark composite. No across-harness maximum; medium effort is explicit for both GPT models. Published Task Score (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (160)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Mercor APEX: 68.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Code Migration: 54.607 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI CyberBench: 70.416 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Excel Modeling: 76.67 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Finance Agent v2: 58.877 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 6.667 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI IOI: 90.778 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Legal Research Bench: 55.288 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI MMMU: 90.636 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI MortgageTax: 70.786 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI MysteryMechanism: 47.748 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI ProgramBench: 82.688 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Public Benefits Bench: 74.899 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI ReverseEngBench: 22.901 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 61.25 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Tax Agent Bench: 49.194 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Terminal-Bench 2.1: 85.019 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 58.081 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Terminal-Bench Science: 40.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Time Horizon Index: 63.333 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Vibe Code Bench: 90.263 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 28.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Mercor APEX: 73.5 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI (Vals Index): 70.177 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI BioMysteryBench: 79.259 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Code Migration: 66.646 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI CyberBench: 74.584 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Excel Modeling: 75.938 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Finance Agent v2: 58.587 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 3.75 (#43)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI IOI: 95.056 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Legal Research Bench: 50.481 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI MysteryMechanism: 49.55 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI ProgramBench: 86.986 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI ProofBench: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Public Benefits Bench: 70.636 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI ReverseEngBench: 33.588 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 70.208 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Science Index: 58.651 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Tax Agent Bench: 45.573 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Terminal-Bench 2.1: 87.64 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 65.152 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Terminal-Bench Science: 47.143 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Time Horizon Index: 91.333 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Vibe Code Bench: 90.294 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 30.361 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Mercor APEX: 64.7 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI (Vals Index): 65.023 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI BioMysteryBench: 79.259 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Code Migration: 67.74 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Excel Modeling: 71.7 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Finance Agent v2: 53.54 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 5.417 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI IOI: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Legal Research Bench: 39.423 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI MysteryMechanism: 53.153 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI ProgramBench: 85.417 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI ReverseEngBench: 56.87 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 43.333 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Science Index: 65.09 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Tax Agent Bench: 20.717 (#46)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Terminal-Bench 2.1: 87.266 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 59.596 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Terminal-Bench Science: 62.857 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Time Horizon Index: 90.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Vibe Code Bench: 89.594 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 27.644 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (Data Viz): 1286.0 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Frontiermath Erdos: 4.41 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Mercor APEX: 60 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI (Vals Index): 63.944 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI BioMysteryBench: 79.63 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Code Migration: 65.122 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Excel Modeling: 70.773 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Finance Agent v2: 52.033 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 5.417 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI IOI: 96.889 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Legal Research Bench: 38.462 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI MysteryMechanism: 46.396 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI ProgramBench: 85.25 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI ProofBench: 99.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Public Benefits Bench: 59.337 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI ReverseEngBench: 50.763 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 43.542 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Science Index: 59.628 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Tax Agent Bench: 21.754 (#44)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 55.051 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Terminal-Bench Science: 52.857 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Vals AI Vibe Code Bench: 88.931 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA: 3.89 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - All Pass: 20.83 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Antitrust &amp; Competition: 95.29 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Arbitration &amp; International Dispute Resolution: 93.91 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Banking &amp; Finance: 97.01 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Bankruptcy &amp; Restructuring: 92.04 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Capital Markets: 95.16 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Corporate Governance: 96.69 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Corporate M&amp;A: 94.53 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Data Privacy &amp; Cybersecurity: 97.64 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Emerging Companies &amp; Venture Capital: 94.43 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Employment &amp; Labor: 89.39 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Energy &amp; Natural Resources: 97.69 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Environmental &amp; ESG: 95.58 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Funds &amp; Asset Management: 91.74 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Healthcare &amp; Life Sciences: 97.87 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Immigration: 93.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Insurance: 94.12 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Intellectual Property: 97.98 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - International Trade &amp; Sanctions: 92.56 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Litigation &amp; Dispute Resolution: 94.24 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Real Estate: 97.08 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Structured Finance &amp; Securitization: 92.15 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Tax: 93.42 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - Trusts, Estates &amp; Private Client: 95.04 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Harvey LAB-AA - White Collar Defense &amp; Investigations: 95.06 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on LLM Chess (Saplin): 1609.422 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Mercor APEX: 82.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI (Vals Index): 69.237 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI BioMysteryBench: 76.296 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Code Migration: 68.173 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI CyberBench: 77.857 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Excel Modeling: 75.244 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Finance Agent v2: 65.401 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 19.583 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI IOI: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Legal Research Bench: 54.808 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI MysteryMechanism: 45.495 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI ProgramBench: 83.715 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI ProofBench: 99.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Public Benefits Bench: 69.756 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI ReverseEngBench: 44.275 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 56.042 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Science Index: 55.359 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Tax Agent Bench: 44.56 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 57.576 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Terminal-Bench Science: 44.286 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Time Horizon Index: 52.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Vibe Code Bench: 91.906 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 21.2 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI (Vals Index): 66.787 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI BioMysteryBench: 73.333 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Code Migration: 61.033 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI CyberBench: 73.631 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Excel Modeling: 71.917 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Finance Agent v2: 59.061 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 21.667 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI IOI: 79.833 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Legal Research Bench: 53.365 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI MysteryMechanism: 35.135 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI ProgramBench: 24.187 (#56)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI ProofBench: 94.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Public Benefits Bench: 73.748 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI ReverseEngBench: 11.832 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI SWE-bench Multimodal: 47.292 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Science Index: 48.537 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Tax Agent Bench: 42.997 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 58.081 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Terminal-Bench Science: 37.143 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Vibe Code Bench: 85.996 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 33.978 (#1)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (10)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Recall&lt;/strong&gt;: codos.ai (96.45) beat Mixedbread (+ Opus 5) by 2.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI (Vals Index)&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (70.854) beat Gemini 4 Argon by 1.95&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Healthcare &amp; Life Sciences&lt;/strong&gt;: Gemini 4 Argon (High) (97.87) beat Muse Wonder (Max) by 1.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Frontiermath Erdos&lt;/strong&gt;: GPT-6.1 Sol (Max) (4.41) beat Claude Opus 5.5 (Max) by 1.47&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Correctness&lt;/strong&gt;: codos.ai (94.4) beat Flexor.ai (+ GPT-6.1 Sol) by 1.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench&lt;/strong&gt;: codos.ai (92.64) beat Flexor.ai (+ GPT-6.1 Sol) by 1.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Completeness&lt;/strong&gt;: codos.ai (95.16) beat Flexor.ai (+ GPT-6.1 Sol) by 0.79&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Environmental &amp; ESG&lt;/strong&gt;: Gemini 4 Argon (High) (95.58) beat Muse Spark 1.3 (Max) by 0.36&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Antitrust &amp; Competition&lt;/strong&gt;: Gemini 4 Argon (High) (95.29) beat Muse Spark 1.3 (Max) by 0.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Intellectual Property&lt;/strong&gt;: Gemini 4 Argon (High) (97.98) beat Muse Wonder Full (Max) by 0.12&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-21

=== DAILY ===
NEW BENCHMARKS (78)
  - Vals AI Finance Agent v2.1 (Dealbreaker-gated Partial Credit (%)): leader Claude Opus 5.5 (Max) (67.846), 28 models
      Finance Agent v2.1: 449 private test questions across nine financial-analysis categories, dealbreaker-gat</summary></entry><entry><title>The Aggregate Digest — 2026-10-20</title><id>https://theaggregate.ai/digest/2026-10-20</id><updated>2026-10-20T05:22:15.202646+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Frontier&lt;/h2&gt;
&lt;ul&gt;&lt;li&gt;Best open-weights model: DeepSeek V4.1 Flash (1017) (1874) takes the crown from Kimi K3 (1851), which had trailed for 3 days&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (44)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (AutomationBench-AA)&lt;/strong&gt; (Score (%)): leader Grok 4.5 (51.4), 3 models&lt;br&gt;&lt;span&gt;LLM Stats mirror: AutomationBench-AA is Artificial Analysis&amp;#x27;s independently run version of AutomationBench, covering 657 real-world SaaS workflow tasks across 40 simulated applications (e.g. Gmail, Slack, Salesforce, HubSpot). It scores the share of objectives an agent completes without violating business guardrails, using a private held-out task set. Results combine the explicitly named vendor or evaluator reports linked by the source; the mirror does not independently verify them. Published score percentage; higher is better. Protocol/version qualifiers in the board title are retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Harvey LAB-AA)&lt;/strong&gt; (Score (%)): leader Gemini 3.7 Flash (90.7), 3 models&lt;br&gt;&lt;span&gt;LLM Stats mirror: Harvey LAB-AA evaluates model performance on complex legal workflows. Results combine the explicitly named vendor or evaluator reports linked by the source; the mirror does not independently verify them. Published score percentage; higher is better. Protocol/version qualifiers in the board title are retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (JobBench)&lt;/strong&gt; (Score (%)): leader Muse Wonder Full (75.3), 4 models&lt;br&gt;&lt;span&gt;LLM Stats mirror: JobBench evaluates agents on work-related tasks, as reported in the MiMo-V2.6 release. Results combine the explicitly named vendor or evaluator reports linked by the source; the mirror does not independently verify them. Published score percentage; higher is better. Protocol/version qualifiers in the board title are retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OSWorld 2.1 (partial))&lt;/strong&gt; (Score (%)): leader Claude Sonnet 5.5 (80.1), 4 models&lt;br&gt;&lt;span&gt;LLM Stats mirror: Partial-credit computer-use score on OSWorld 2.1, as reported in the Claude Sonnet 5.5 release. Separate from OSWorld 2.0 and strict completion scores. Results combine the explicitly named vendor or evaluator reports linked by the source; the mirror does not independently verify them. Published score percentage; higher is better. Protocol/version qualifiers in the board title are retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (OSWorld 2.1 (strict))&lt;/strong&gt; (Score (%)): leader Claude Sonnet 5.5 (43.5), 4 models&lt;br&gt;&lt;span&gt;LLM Stats mirror: Strict pass rate on OSWorld 2.1: fraction of tasks where every graded checkpoint is satisfied. Separate from the OSWorld 2.1 partial-credit score and from OSWorld 2.0. Results combine the explicitly named vendor or evaluator reports linked by the source; the mirror does not independently verify them. Published score percentage; higher is better. Protocol/version qualifiers in the board title are retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Toolathlon-Verified)&lt;/strong&gt; (Score (%)): leader Claude Sonnet 5.5 (77.8), 5 models&lt;br&gt;&lt;span&gt;LLM Stats mirror: Verified Toolathlon evaluation of agent tool-use capability, as reported in the MiMo-V2.6 release. Results combine the explicitly named vendor or evaluator reports linked by the source; the mirror does not independently verify them. Published score percentage; higher is better. Protocol/version qualifiers in the board title are retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (Vals Finance Agent v2)&lt;/strong&gt; (Score (%)): leader Gemini 4 Argon (65.4), 4 models&lt;br&gt;&lt;span&gt;LLM Stats mirror: Vals Finance Agent v2 evaluates AI agents on realistic financial analyst tasks. Results combine the explicitly named vendor or evaluator reports linked by the source; the mirror does not independently verify them. Published score percentage; higher is better. Protocol/version qualifiers in the board title are retained.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Project Record - Requested Low&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (57.7), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Project Record, requested effort Low. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Options Only - Requested Low&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (64.5), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Options Only, requested effort Low. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Project Record - Requested Medium&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (59.5), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Project Record, requested effort Medium. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Options Only - Requested Medium&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (63.7), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Options Only, requested effort Medium. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Project Record - Requested High&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (62.1), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Project Record, requested effort High. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Options Only - Requested High&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (64.9), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Options Only, requested effort High. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Project Record - Requested xhigh&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (63.5), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Project Record, requested effort xhigh. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Options Only - Requested xhigh&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (64.5), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Options Only, requested effort xhigh. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Project Record - Requested Max&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (63.1), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Project Record, requested effort Max. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciPIvot - Options Only - Requested Max&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (62.7), 5 models&lt;br&gt;&lt;span&gt;SciPIvot (2026), 248 selected human-corrected scientific decisions from 26 projects; option-order-averaged choice accuracy; Options Only, requested effort Max. GPT models use Codex shell with read-only projects; Opus and Grok use a shared file/search client with shell, edits and web disabled. Client effort labels do not establish equivalent compute; these selected cases are not a representative population sample. Accuracy (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BizAgentBench - Goal&lt;/strong&gt; (Rate (%)): leader GPT-6 Sol (64.7), 10 models&lt;br&gt;&lt;span&gt;BizAgentBench / BizBench (2026), 50 long-horizon banking tasks with deterministic final-state and trajectory requirements, revised customer simulator protocol. Claude Opus 5 and Muse Spark 1.3 have one run per task; other models have three, averaged. Goal attainment, process compliance, protected constraints and joint strict success are distinct outcomes; intermediate failure is failure of process or protection. Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BizAgentBench - Process&lt;/strong&gt; (Rate (%)): leader Claude Opus 5 (96.0), 10 models&lt;br&gt;&lt;span&gt;BizAgentBench / BizBench (2026), 50 long-horizon banking tasks with deterministic final-state and trajectory requirements, revised customer simulator protocol. Claude Opus 5 and Muse Spark 1.3 have one run per task; other models have three, averaged. Goal attainment, process compliance, protected constraints and joint strict success are distinct outcomes; intermediate failure is failure of process or protection. Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BizAgentBench - Protection&lt;/strong&gt; (Rate (%)): leader GPT-6 Sol (100.0), 10 models&lt;br&gt;&lt;span&gt;BizAgentBench / BizBench (2026), 50 long-horizon banking tasks with deterministic final-state and trajectory requirements, revised customer simulator protocol. Claude Opus 5 and Muse Spark 1.3 have one run per task; other models have three, averaged. Goal attainment, process compliance, protected constraints and joint strict success are distinct outcomes; intermediate failure is failure of process or protection. Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BizAgentBench - Intermediate Failure&lt;/strong&gt; (Rate (%)): leader Claude Opus 5 (4.0), 10 models&lt;br&gt;&lt;span&gt;BizAgentBench / BizBench (2026), 50 long-horizon banking tasks with deterministic final-state and trajectory requirements, revised customer simulator protocol. Claude Opus 5 and Muse Spark 1.3 have one run per task; other models have three, averaged. Goal attainment, process compliance, protected constraints and joint strict success are distinct outcomes; intermediate failure is failure of process or protection. Rate (%); lower is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BizAgentBench - Strict Success&lt;/strong&gt; (Rate (%)): leader Claude Opus 5 (64.0), 10 models&lt;br&gt;&lt;span&gt;BizAgentBench / BizBench (2026), 50 long-horizon banking tasks with deterministic final-state and trajectory requirements, revised customer simulator protocol. Claude Opus 5 and Muse Spark 1.3 have one run per task; other models have three, averaged. Goal attainment, process compliance, protected constraints and joint strict success are distinct outcomes; intermediate failure is failure of process or protection. Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - C&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6.1 Sol (Max) (88.4), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - C++&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (89.0), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - C#&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (90.8), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - COBOL&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6.1 Sol (Max) (72.6), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Common Lisp&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (88.3), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Fortran&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6.1 Sol (Max) (84.1), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Go&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6.1 Sol (Max) (89.9), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Java&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (91.0), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - JavaScript&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (91.0), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Lean&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (84.0), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - OCaml&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6.1 Sol (Max) (87.5), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - PHP&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (91.3), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - PowerShell&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (88.2), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Python&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6.1 Sol (Max) (91.0), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Ruby&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (90.3), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Rust&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (87.7), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Scala&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6.1 Sol (Max) (89.1), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Swift&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (88.6), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Visual Basic&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (89.4), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Zig&lt;/strong&gt; (Test Pass Rate (%)): leader GPT-6 Astra (Max) (85.4), 22 models&lt;br&gt;&lt;span&gt;OpenAI October 19 release; implement SQLite 3.40.1 from documentation without network or installed SQLite; validate database effects against reference SQLite. Default released evaluation uses 272 hash-selected tests; the repository permits full-suite refinement if reported. Every embedded model/effort point retained; no cost-based or best-effort selection. Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI COBOL Migration&lt;/strong&gt; (All-Tests Pass Rate (%)): leader GPT-6.1 Sol (xHigh) (97.9), 20 models&lt;br&gt;&lt;span&gt;Migrate COBOL to C++, Java, Kotlin, Python or Rust; success requires byte-for-byte output parity on all hidden tests. Every published effort retained. The migration task is distinct from MultiLangBench; access to its exact input programs is not established by the hidden-grader description. All-Tests Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MultiLangBench - Published Average&lt;/strong&gt; (Average Test Pass Rate (%)): leader GPT-6 Astra (87.39), 8 models&lt;br&gt;&lt;span&gt;Published 20-language aggregate chart. The chart omits effort labels for its eight bars; no universal effort is inferred from the three prose examples. Per-language model/effort results are separate boards. Average Test Pass Rate (%); higher is better.&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M3.1-Flash&lt;/strong&gt; — Aggregate Rating 1848, #33&lt;ul&gt;&lt;li&gt;LLM Stats (AutomationBench v1.0.6): 62.2 (#1/6)&lt;/li&gt;&lt;li&gt;AutomationBench-AA: 72.52 (#2/227)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Finance - Raw Objective Completion: 93.72 (#2/227)&lt;/li&gt;&lt;li&gt;LLM Stats (BabyVision): 91.0 (#2/16)&lt;/li&gt;&lt;li&gt;LLM Stats (PostTrainBench): 39.3 (#3/9)&lt;/li&gt;&lt;li&gt;LLM Stats (NL2Repo): 61.2 (#4/25)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - HR - Raw Objective Completion: 77.5 (#6/227)&lt;/li&gt;&lt;li&gt;LLM Stats (Humanity's Last Exam (no tools, text-only)): 43.4 (#7/14)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Strict: 38.05 (#8/227)&lt;/li&gt;&lt;li&gt;LLM Stats (CyberGym): 82.2 (#9/21)&lt;/li&gt;&lt;li&gt;...and 65 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (17)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Design Arena (Agents - Web Apps): 1318.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Dtbench: 97.33 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Epoch AI - Lmca: 66.18 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on HANDBOOK.md Agents: 52.3 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on SEAL - Fortress: 5.78 (#71)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (Agents - Mobile Apps): 1204.0 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Dtbench: 96.53 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Lmca: 62.78 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AIIQ Composite IQ: 140.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on APEX v1: 81.1 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Dtbench: 97.6 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Lmca: 63.69 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Weirdml V3: 20.22 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on HANDBOOK.md Agents: 36.6 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Game Arena Chess: 1380.78 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AIIQ Composite IQ: 131.0 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LLM Stats (Terminal-Bench 4.0): 57.1 (#6)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (12)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena Four in a Row&lt;/strong&gt;: Gemini 4 Argon (923.33) beat GPT-6 Astra by 228.01&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Game Arena (Unified)&lt;/strong&gt;: Gemini 4 Argon (535.09) beat GPT-6 Astra by 42.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HANDBOOK.md Agents&lt;/strong&gt;: Claude Opus 5.5 (Adaptive Reasoning, Max Effort) (52.3) beat Claude Fable 5.1 (Max) by 13.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (AutomationBench; v1.0.6 method cohort)&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (60.5) beat GLM-5.3 Flash by 11.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (AutomationBench v1.0.6)&lt;/strong&gt;: MiniMax-M3.1-Flash (62.2) beat MiMo-V2.6-Pro by 9.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;APEX v1&lt;/strong&gt;: Gemini 4 Argon (High) (81.1) beat Muse Spark 1.2 (xHigh) by 5.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (NL2Repo)&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (68.2) beat DeepSeek V4.1 Flash by 4.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (BabyVision)&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (92.5) beat DeepSeek V4.1 Flash by 2.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (ZEROBench)&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (51.0) beat DeepSeek V4.1 Flash by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (MCP Atlas)&lt;/strong&gt;: Muse Spark Open (89.2) beat Muse Spark 1.1 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (MMLongBench-Doc)&lt;/strong&gt;: Nemotron 3.5 Super VL (BF16) (62.82) beat Qwen 3.6 Plus by 0.82&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Lmca&lt;/strong&gt;: Claude Sonnet 5.5 (Max) (68.26) beat Claude Opus 5.5 (Max) by 0.03&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-20

=== FRONTIER ===
  - Best open-weights model: DeepSeek V4.1 Flash (1017) (1874) takes the crown from Kimi K3 (1851), which had trailed for 3 days

=== DAILY ===
NEW BENCHMARKS (44)
  - LLM Stats (AutomationBench-AA) (Score (%)): leader Grok 4.5 (51.4), 3 models
   </summary></entry><entry><title>The Aggregate Digest — 2026-10-19</title><id>https://theaggregate.ai/digest/2026-10-19</id><updated>2026-10-19T07:04:34.833918+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (30)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;LiveBench AMPS Hard&lt;/strong&gt; (Score): leader Claude Opus 5 (Max) (99.01), 71 models&lt;br&gt;&lt;span&gt;Items are machine-generated symbolic maths problems built from the ten hardest AMPS categories with a harder draw of random primitives. The model answers in one turn and its final boxed expression is compared with the ground truth in SymPy; if SymPy cannot confirm a match, the grading script asks a language model to compare the two. This is the AMPS Hard task in LiveBench&amp;#x27;s maths category, and higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Connections&lt;/strong&gt; (Score): leader Gemini 3.1 Pro (Preview) (High) (100.0), 71 models&lt;br&gt;&lt;span&gt;One item gives a list of 8, 12 or 16 words to be sorted into groups of four that share a hidden connection, such as a category or a common associated word. This is the connections task in LiveBench&amp;#x27;s language comprehension category, checked against a fixed answer key with no model judge, on puzzles that LiveBench replaces with newer ones in dated releases. Scores run 0 to 100 and higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Math Comp&lt;/strong&gt; (Score): leader Claude Opus 4.7 (xHigh) (98.04), 71 models&lt;br&gt;&lt;span&gt;The competition task in LiveBench&amp;#x27;s math category: high school contest problems with the prose and answer order changed to limit contamination. The first release drew on AMC 12 2023, SMC 2023 and AIME 2024, and later releases swapped in newer competitions. Answers are checked against ground truth with no model judge; the number is the share answered correctly, higher is better. Source: arxiv.org/abs/2406.19314.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Olympiad&lt;/strong&gt; (Score): leader Claude Opus 5.5 (Max) (93.25), 71 models&lt;br&gt;&lt;span&gt;The LiveBench math task built from proof-based USAMO and IMO problems, refreshed over time with newer competitions. Formulae are cut out of a written solution and handed back to the model in scrambled order, and it must say which formula fills each gap. Scoring is automatic, by edit distance between its ordering and the true one with no judge model, and higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Paraphrase&lt;/strong&gt; (Score): leader Gemini 3.8 Flash (High) (80.0), 71 models&lt;br&gt;&lt;span&gt;The model is given a recent article from The Guardian and asked to paraphrase it while obeying randomly drawn constraints such as a word limit or elements it must include. It is a single-turn task with the article supplied in context, and scoring is an automatic check of whether the constraints were met rather than a judgement of writing quality. Higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Plot Unscrambling&lt;/strong&gt; (Score): leader GPT-6 Astra (Max) (86.29), 71 models&lt;br&gt;&lt;span&gt;LiveBench shuffles the sentences of a plot synopsis taken from a recently released film and asks the model to put them back in the original order. Grading compares the model&amp;#x27;s ordering with the true one and gives partial credit from the edit distance between the two sequences. Source material is refreshed to limit contamination, and higher is better. Source: arxiv.org/abs/2406.19314.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Simplify&lt;/strong&gt; (Score): leader Gemini 3.6 Flash (High) (75.7), 71 models&lt;br&gt;&lt;span&gt;The model is given a recent article from The Guardian and asked to simplify it while obeying randomly drawn constraints such as a word count, required keywords or banned words. This is the simplify task in LiveBench&amp;#x27;s instruction following category, and the score counts only adherence to those instructions, checked without a model judge. Scores run 0 to 100 and higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Story Generation&lt;/strong&gt; (Score): leader Gemini 3.8 Flash (High) (82.75), 71 models&lt;br&gt;&lt;span&gt;One of four instruction-following subtasks in LiveBench. The model writes a story about a recent Guardian article while obeying a randomly drawn subset of automatically verifiable constraints such as word limits, required keywords and formatting rules. Grading checks only whether those constraints were met, combining whole-prompt and per-instruction accuracy; the score runs from 0 to 100, higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Summarize&lt;/strong&gt; (Score): leader Gemini 3.1 Pro (Preview) (High) (88.23), 71 models&lt;br&gt;&lt;span&gt;The summarize task in LiveBench&amp;#x27;s instruction following category: the model is given a recent Guardian article and must summarise it while obeying several automatically checkable instructions, such as word limits. Scoring is adherence to those instructions rather than summary quality, and the board shows the task score in percent, higher is better. Source: arxiv.org/abs/2406.19314.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Table Join&lt;/strong&gt; (Score): leader GPT-6 Astra (Max) (58.37), 71 models&lt;br&gt;&lt;span&gt;Each item shows two tables from a recent public dataset whose columns partly overlap, with the second table&amp;#x27;s names anonymized, and asks for a valid join mapping from the first to the second. Scoring is automatic: an F1 over the proposed column pairs, counting extra pairs as false positives and missing ones as false negatives. This board is that task&amp;#x27;s LiveBench score, higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Table Reformat&lt;/strong&gt; (Score): leader DeepSeek V4 Pro (100.0), 71 models&lt;br&gt;&lt;span&gt;The table reformatting task inside LiveBench&amp;#x27;s data analysis category: the model is handed a table taken from a recent Kaggle or Socrata dataset in one serialisation and has to rewrite it in a requested target format among JSON, JSONL, Markdown, CSV, TSV and HTML. Output is checked automatically against the expected table with no model judge. Higher is better. Source: arxiv.org/abs/2406.19314.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Typos&lt;/strong&gt; (Score): leader Claude Fable 5 (Max) (94.0), 71 models&lt;br&gt;&lt;span&gt;LiveBench injects common human typos into a recent arXiv abstract and asks the model to return the passage with the misspellings fixed and nothing else altered. Scoring is all or nothing: the answer counts only if the exact corrected text appears inside it, so any other edit loses the point. The board is the percent of passages scored correct, higher is better. Source: arxiv.org/abs/2406.19314.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LiveBench Zebra Puzzle&lt;/strong&gt; (Score): leader GPT-5.5 (xHigh) (100.0), 71 models&lt;br&gt;&lt;span&gt;Logic grid puzzles in LiveBench&amp;#x27;s reasoning category, where constraints about people and their attributes must be combined to work out a requested assignment. They are generated procedurally and have been replaced twice with harder sets, first with larger grids and more complex constraints. Answers in solution tags are checked against ground truth automatically, and the score is a percentage where higher is better.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;KillSwitch-Bench&lt;/strong&gt; (Pass rate (%; attempts whose program passed the hidden verification, every task attempted the same number of times)): leader Claude Opus 5 (70.0), 20 models&lt;br&gt;&lt;span&gt;Coding agents write and interpret programs in KillSwitch, an esoteric language built to be hard for LLMs, working only from its published specification. Five tasks (hello world, a temperature conversion, max, and two program-understanding tasks), each attempted twice with a $5 budget; an attempt passes when its program passes the hidden verification. Each model runs under its vendor&amp;#x27;s agent (Claude Code, Codex) or OpenCode. Score: pass rate (%) over the ten attempts; the site&amp;#x27;s own ranking also subtracts small cost and code-size penalties, which are left out here. Higher is better. Source: https://bench.killswitch-lang.org/&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;TAC (Travel Agent Compassion)&lt;/strong&gt; (Welfare rate (%; ticket-booking scenarios in which the agent avoided the option that harms animals, nothing in the request mentioning welfare)): leader Claude Opus 4.8 (64.7), 26 models&lt;br&gt;&lt;span&gt;Travel Agent Compassion (CaML, arXiv 2606.18142; inspect_evals/tac): an AI booking agent chooses activities for users in 13 ticket-booking scenarios where the obvious or best-reviewed option harms animals (bullfights, marine shows, animal rides and races) and the request never mentions welfare. Scoring is programmatic on what the agent books. Score: welfare rate, the share of runs (%) that avoid the harmful option; picking at random avoids it about 64.5% of the time and many models do worse. Higher is better. Source: https://compassionbench.com/tac&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;TAC (Travel Agent Compassion) - With Welfare Guidance&lt;/strong&gt; (Welfare rate (%; the same scenarios with one sentence of animal-welfare guidance in the system prompt)): leader Claude Opus 5 (100.0), 26 models&lt;br&gt;&lt;span&gt;The Travel Agent Compassion scenarios (CaML, arXiv 2606.18142; inspect_evals/tac_welfare) rerun with one sentence of animal-welfare guidance in the agent&amp;#x27;s system prompt, so the gap to the unguided board separates default behaviour from instruction-following. Score: welfare rate, the share of runs (%) that avoid the option harming animals. Higher is better. Source: https://compassionbench.com/tac&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CrueltyBench&lt;/strong&gt; (Welfare score (0-100; two-judge panel, green 1 / yellow 0.5 / red 0, mean over the implicit animal-cruelty requests and epochs)): leader GPT-6 Astra (81.7), 14 models&lt;br&gt;&lt;span&gt;CrueltyBench (CaML): single-turn, realistic in-persona requests with implicit animal-cruelty implications that never mention the animal&amp;#x27;s welfare (18 harm scenarios plus controls, 5 epochs). A fixed two-judge panel (Claude Opus 5, Gemini 3.7 Flash) grades each response green (clearly reduces or mitigates the cruelty), yellow or red; control items penalise moralising at ordinary requests. Score: mean welfare score (0-100; green 1, yellow 0.5, red 0). Higher is better. Source: https://compassionbench.com/crueltybench&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LatentGUIWorld - Overall&lt;/strong&gt; (Episode success rate (%; macro-average over six interactive GUI environments, 150 test episodes each, at most 12 model responses per episode)): leader GPT-6 Astra (63.3), 11 models&lt;br&gt;&lt;span&gt;LatentGUIWorld (Learn from Move, OpenMOSS, arXiv 2610.16720): six interactive GUI environments in matched pairs (Drag, Ten-Choice, Rotation), 150 test episodes each at a 1280x720 viewport. In one Drag and one Ten-Choice variant the target follows from what is on screen or revealed by hovering; in the other variants the agent must work out hidden movement dynamics from the effects of its own actions. At most 12 model responses per episode, Rotation scored at the first release, each agent on its official computer-use or tool interface where one exists and on the authors&amp;#x27; six-action JSON schema otherwise (DeepSeek-V4.1-Flash); GUI-specialised agents get additional tools where their official action space is insufficient. Score: episode success rate (%), macro-average over the six environments. Higher is better. Source: https://arxiv.org/abs/2610.16720&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LatentGUIWorld - Dynamics-Dependent (I2) Average&lt;/strong&gt; (Episode success rate (%; macro-average over the four environments whose hidden movement dynamics must be identified from the agent's own actions: egocentric Drag, egocentric Ten-Choice, inner and outer Rotation)): leader GPT-6 Astra (45.0), 11 models&lt;br&gt;&lt;span&gt;The four LatentGUIWorld environments (arXiv 2610.16720) whose hidden movement dynamics can only be identified from the effects of the agent&amp;#x27;s own actions: egocentric Drag, egocentric Ten-Choice, and inner- and outer-ring Rotation, 150 episodes each, at most 12 model responses per episode. Score: episode success rate (%), macro-average over the four. Higher is better. Source: https://arxiv.org/abs/2610.16720&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - WGO Bench&lt;/strong&gt; (Score): leader Glimpse (Max) (42.7), 5 models&lt;br&gt;&lt;span&gt;WGO Bench, in Elorian&amp;#x27;s robotics category: end-to-end understanding of manipulation events, identifying completed actions in robot footage. Elorian reported this for its Glimpse research preview (October 16, 2026), running each model with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at high effort with Code Interpreter (the post says max-effort results are not available for these video evaluations), and Gemini 3.8 Flash at high with its Video Agent. Vendor-reported. Metric: Score. Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - ERQA&lt;/strong&gt; (Accuracy (%)): leader Glimpse (Max) (83.75), 5 models&lt;br&gt;&lt;span&gt;ERQA embodied spatial reasoning questions about images. Elorian reported this for its Glimpse research preview (October 16, 2026), running every model at its highest thinking effort with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at max effort with Code Interpreter, and Gemini 3.8 Flash at high with Code Interpreter. Vendor-reported. Metric: Accuracy (%). Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - BabyVision&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (Max) (97.68), 4 models&lt;br&gt;&lt;span&gt;BabyVision fine-grained visual reasoning questions. Elorian reported this for its Glimpse research preview (October 16, 2026), running every model at its highest thinking effort with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at max effort with Code Interpreter, and Gemini 3.8 Flash at high with Code Interpreter. Vendor-reported. Metric: Accuracy (%). Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - CharXiv-R&lt;/strong&gt; (Accuracy (%)): leader Claude Opus 5.5 (Max) (93.4), 5 models&lt;br&gt;&lt;span&gt;CharXiv reasoning questions about charts from scientific papers. Elorian reported this for its Glimpse research preview (October 16, 2026), running every model at its highest thinking effort with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at max effort with Code Interpreter, and Gemini 3.8 Flash at high with Code Interpreter. Vendor-reported. Metric: Accuracy (%). Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - UGC-VideoCap&lt;/strong&gt; (Score): leader Glimpse (Max) (82.66), 5 models&lt;br&gt;&lt;span&gt;UGC-VideoCap captioning of short user-generated videos. Elorian reported this for its Glimpse research preview (October 16, 2026), running each model with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at high effort with Code Interpreter (the post says max-effort results are not available for these video evaluations), and Gemini 3.8 Flash at high with its Video Agent. Vendor-reported. Metric: Score. Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - DREAM-1K&lt;/strong&gt; (Score): leader GPT-6 Astra (Max) (79.05), 5 models&lt;br&gt;&lt;span&gt;DREAM-1K dense, detailed video captioning. Elorian reported this for its Glimpse research preview (October 16, 2026), running each model with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at high effort with Code Interpreter (the post says max-effort results are not available for these video evaluations), and Gemini 3.8 Flash at high with its Video Agent. Vendor-reported. Metric: Score. Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - TOMATO&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (Max) (96.02), 5 models&lt;br&gt;&lt;span&gt;TOMATO fine-grained temporal reasoning over video. Elorian reported this for its Glimpse research preview (October 16, 2026), running each model with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at high effort with Code Interpreter (the post says max-effort results are not available for these video evaluations), and Gemini 3.8 Flash at high with its Video Agent. Vendor-reported. Metric: Accuracy (%). Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - Video-MME v2 (with subtitles)&lt;/strong&gt; (Grouped rubric score): leader GPT-6.1 Sol (Max) (65.27), 4 models&lt;br&gt;&lt;span&gt;Video-MME v2 video understanding with subtitles, scored with its grouped rubric. Elorian reported this for its Glimpse research preview (October 16, 2026), running each model with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol at max and Claude Opus 5.5 at high effort with Code Interpreter (the post says max-effort results are not available for its other video evaluations), and Gemini 3.8 Flash at high with its Video Agent. Vendor-reported. Metric: Grouped rubric score. Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - E.T. Bench&lt;/strong&gt; (Score): leader Glimpse (Max) (63.55), 3 models&lt;br&gt;&lt;span&gt;E.T. Bench event-level understanding of video. Elorian reported this for its Glimpse research preview (October 16, 2026), running each model with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at high effort with Code Interpreter (the post says max-effort results are not available for these video evaluations), and Gemini 3.8 Flash at high with its Video Agent. Vendor-reported. Metric: Score. Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - FSC-147&lt;/strong&gt; (Score): leader Glimpse (Max) (83.36), 5 models&lt;br&gt;&lt;span&gt;FSC-147 open-vocabulary object counting in images (Elorian&amp;#x27;s higher-is-better score; the post does not define the metric further). Elorian reported this for its Glimpse research preview (October 16, 2026), running every model at its highest thinking effort with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at max effort with Code Interpreter, and Gemini 3.8 Flash at high with Code Interpreter. Vendor-reported. Metric: Score. Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Elorian Glimpse Card - CountQA&lt;/strong&gt; (Accuracy (%)): leader GPT-6 Astra (Max) (86.42), 5 models&lt;br&gt;&lt;span&gt;CountQA counting questions about product images. Elorian reported this for its Glimpse research preview (October 16, 2026), running every model at its highest thinking effort with its native tools: Glimpse at max, GPT-6 Astra at max with Code Interpreter, GPT-6.1 Sol and Claude Opus 5.5 at max effort with Code Interpreter, and Gemini 3.8 Flash at high with Code Interpreter. Vendor-reported. Metric: Accuracy (%). Higher is better. Source: https://elorian.ai/blog/introducing-glimpse&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (56)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on KernelBench CUDA: 0.2188 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on CAIS Vision Capabilities Index: 68.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on KernelBench CUDA: 0.5167 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Lech Mazur Writing: 3.715 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MedXpertQA MM (Natomy): 86.55 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MedXpertQA MM (Natomy) - Basic Science: 89.52 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MedXpertQA MM (Natomy) - Diagnosis: 85.99 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on MedXpertQA MM (Natomy) - Treatment: 85.71 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra Pro&lt;/strong&gt; on KernelBench CUDA: 0.2068 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on CAIS Vision Capabilities Index: 68.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on KernelBench CUDA: 0.1975 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra Launch - Internal Computer Use Safety Benchmark: 4.32 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Adversarial User Simulations - Mental Health: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Chat Plugins: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Non-Violent Wrongdoing: 98.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Self-Harm: 92.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Sensitive Personal Data: 74.4 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Human Red-Teaming Chat Plugins: 95.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Biology Refusal - Benign: 98.2 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Biology Refusal - Dual Use: 98.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Biology Refusal - Severe: 99.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Extremism: 97.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Gore: 88.9 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Hate: 100.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Sexual/Minors: 98.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Challenging Prompts - Violent Illicit Behavior: 98.3 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Cyber Safety - Production Chat: 0.987 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Cyber Safety - Semi-Synthetic Agentic Environments: 0.98 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Cyber Safety - Synthetic Agentic Environments: 0.997 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - HealthBench Hard: 36.2 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - HealthBench Professional: 64.2 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Erotic: 100.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Extremism: 98.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Hate: 99.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Image Input - Self-Harm: 99.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Multimodal Troubleshooting Virology: 55.34 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - ProtocolQA Open-Ended: 40.74 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - SHP2 Protein Function Prediction: 0.332 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Bio High Risk: 93.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Bio Severe: 94.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Cyber: 85.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Violence Moderate: 88.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Static Jailbreaks - Violence Severe: 95.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - Tacit Knowledge and Troubleshooting: 88.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - TroubleshootingBench: 47.96 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Astra System Card - U18 - Sexual Content: 98.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Broken Search Tool Non-Disclosure: 2.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Factual Errors on User-Flagged Conversations: 4.12 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Reviewer Bypass Attempts: 0.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on OpenAI GPT-6 Sol &amp; Luna Launch - Warning Circumvention: 23.48 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on CAIS Risk Index: 56.2 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on KernelBench CUDA: 0.2732 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Lech Mazur Writing: 0.8 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-LCR: 83.67 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on CritPt: 32.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Finance Agent v2: 59.1 (#6)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (29)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - TroubleshootingBench&lt;/strong&gt;: GPT-6 Astra (63.46) beat GPT-6 Sol by 18.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Tacit Knowledge and Troubleshooting&lt;/strong&gt;: GPT-6 Astra (92.55) beat GPT-6 Sol by 13.35&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Multimodal Troubleshooting Virology&lt;/strong&gt;: GPT-6 Astra (63.11) beat GPT-6 Sol by 12.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ToolSandbox&lt;/strong&gt;: LFM2.5-24B-A2B (80.71) beat GPT-4o (2024-05-13) by 7.71&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - U18 - Eating Disorders&lt;/strong&gt;: GPT-6.1 Sol (96.2) beat GPT-6 Astra by 4.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Coding Daily (OpenCode) - Total&lt;/strong&gt;: DeepSeek V4.1 Flash (1017) (Max) (60.97) beat Space Bunny (Max) by 3.78&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MedXpertQA MM (Natomy) - Basic Science&lt;/strong&gt;: Gemini 4 Argon (92.92) beat GPT-6 Astra by 3.68&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MedXpertQA MM (Natomy) - Treatment&lt;/strong&gt;: Gemini 4 Argon (90.18) beat GPT-6 Astra by 3.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MedXpertQA MM (Natomy)&lt;/strong&gt;: Gemini 4 Argon (90.5) beat GPT-6 Astra by 3.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MedXpertQA MM (Natomy) - Diagnosis&lt;/strong&gt;: Gemini 4 Argon (89.91) beat GPT-6 Astra by 2.92&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Crosby micro1 RedlineBench&lt;/strong&gt;: Gemini 4 Argon (59.5) beat Claude Opus 5 by 2.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ScreenSpot-Pro&lt;/strong&gt;: GroundCascade (Indeed-UI-32B + KV-Ground-8B) ScreenSpot-Pro official grounding leaderboard (85.14) beat Indeed-UI-32B-zoomin ScreenSpot-Pro official grounding leaderboard by 2.41&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - U18 - Emotional Reliance&lt;/strong&gt;: GPT-6.1 Sol (96.9) beat GPT-6 Sol by 2.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Age-Restricted Actions&lt;/strong&gt;: GPT-6.1 Sol (83.0) beat GPT-6 Astra by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Codex Violent Wrongdoing&lt;/strong&gt;: GPT-6.1 Sol (92.6) beat GPT-6 Astra by 1.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - U18 - Age-Restricted Goods and Dangerous Activities&lt;/strong&gt;: GPT-6.1 Sol (93.4) beat GPT-6 Astra by 1.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MCP Atlas&lt;/strong&gt;: Muse Spark Open (xhigh effort, full 1,000-task set) (xHigh) (89.2) beat Muse Spark 1.1 by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - ProtocolQA Open-Ended&lt;/strong&gt;: GPT-6 Astra (45.37) beat GPT-6 Sol by 0.97&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Adversarial User Simulations - Self-Harm&lt;/strong&gt;: GPT-6.1 Sol (99.6) beat GPT-6 Astra by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GameDevBench&lt;/strong&gt;: Claude Opus 5.5 (Claude Code, xHigh) (70.6) beat GPT-6.1 Sol (Codex, xHigh) by 0.63&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CritPt&lt;/strong&gt;: Muse Wonder Full (Max) (32.9) beat GPT-5.6 Sol (Max) by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Challenging Prompts - Non-Violent Illicit Behavior&lt;/strong&gt;: GPT-6.1 Sol (100.0) beat GPT-6 Astra by 0.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Challenging Prompts - Sexual&lt;/strong&gt;: GPT-6.1 Sol (98.7) beat GPT-6 Sol by 0.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - HealthBench&lt;/strong&gt;: GPT-6.1 Sol (58.5) beat GPT-6 Astra by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Challenging Prompts - Self-Harm&lt;/strong&gt;: GPT-6.1 Sol (99.4) beat GPT-6 Astra by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - U18 - Self-Harm&lt;/strong&gt;: GPT-6.1 Sol (99.7) beat GPT-6 Astra by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Adversarial User Simulations - Emotional Reliance&lt;/strong&gt;: GPT-6.1 Sol (99.5) beat GPT-6 Astra by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;CAIS Risk Index&lt;/strong&gt;: Claude Opus 5.5 (27.1) beat Claude Fable 5 by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenAI GPT-6 Astra System Card - Agentic Safe Completions - Human Red-Teaming Codex&lt;/strong&gt;: GPT-6.1 Sol (97.8) beat GPT-6 Astra by 0.1&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-19

=== DAILY ===
NEW BENCHMARKS (30)
  - LiveBench AMPS Hard (Score): leader Claude Opus 5 (Max) (99.01), 71 models
      Items are machine-generated symbolic maths problems built from the ten hardest AMPS categories with a harder draw of random primitives. The model </summary></entry><entry><title>The Aggregate Digest — 2026-10-18</title><id>https://theaggregate.ai/digest/2026-10-18</id><updated>2026-10-18T06:10:22.705382+00:00</updated><link href="https://theaggregate.ai/whats-new?ref=atom" /><author><name>The Aggregate</name></author><content type="html">&lt;h2&gt;Frontier&lt;/h2&gt;
&lt;ul&gt;&lt;li&gt;Best available model: DeepSeek V4.1 Flash (1017) enters at #15 (Aggregate Rating 1911) on 42 benchmarks&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Daily&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (33)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - Overall&lt;/strong&gt; (Macro average of the four category pass@1 rates (%)): leader Gemini 4 Argon (77.0), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - Code Vulnerabilities&lt;/strong&gt; (Code Vulnerabilities challenges solved, pass@1 (%)): leader Gemini 4 Argon (64.9), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - API Security&lt;/strong&gt; (API Security challenges solved, pass@1 (%)): leader Gemini 4 Argon (87.0), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - Websec.fr CTFs&lt;/strong&gt; (Websec.fr CTFs challenges solved, pass@1 (%)): leader Gemini 3.8 Flash Cyber (77.0), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - Cloud Security&lt;/strong&gt; (Cloud Security challenges solved, pass@1 (%)): leader Gemini 4 Argon (87.0), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - HLE w/o tools (full set)&lt;/strong&gt; (Accuracy (%)): leader DeepSeek V4.1 Flash (1017) (42.6), 2 models&lt;br&gt;&lt;span&gt;Humanity&amp;#x27;s Last Exam without tools, full question set. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Accuracy (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - HLE w/o tools (text-only subset)&lt;/strong&gt; (Accuracy (%)): leader DeepSeek V4.1 Flash (1017) (44.0), 2 models&lt;br&gt;&lt;span&gt;Humanity&amp;#x27;s Last Exam without tools, text-only subset of the questions. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Accuracy (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Terminal-Bench 4.0 (Claude Code harness)&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (42.3), 2 models&lt;br&gt;&lt;span&gt;Terminal-Bench 4.0 command-line tasks, run in the Claude Code harness. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - DeepSWE v1.1 (Mini-SWE-V2 harness)&lt;/strong&gt; (Pass@1 (%)): leader DeepSeek V4.1 Flash (1017) (77.0), 2 models&lt;br&gt;&lt;span&gt;DeepSWE v1.1 software-engineering tasks, run in the Mini-SWE-V2 harness, one attempt per task. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Pass@1 (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - ProgramBench&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (29.1), 2 models&lt;br&gt;&lt;span&gt;ProgramBench program-construction tasks. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - NL2Repo-Bench&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (68.2), 2 models&lt;br&gt;&lt;span&gt;NL2Repo-Bench: building a complete code repository from a natural-language specification. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Automation-Bench v1.0.6&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (60.5), 2 models&lt;br&gt;&lt;span&gt;Automation-Bench version 1.0.6 workflow-automation tasks. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Agents' Last Exam&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (35.4), 2 models&lt;br&gt;&lt;span&gt;Agents&amp;#x27; Last Exam agentic tasks. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Apex Bench v1.1&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (72.1), 2 models&lt;br&gt;&lt;span&gt;Apex Bench version 1.1. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Chartography (with tools)&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (83.0), 2 models&lt;br&gt;&lt;span&gt;Chartography chart-understanding questions, with tools allowed. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - BabyVision (with tools)&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (92.5), 2 models&lt;br&gt;&lt;span&gt;BabyVision visual-reasoning questions, with tools allowed. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - ZeroBench-main (with tools, Pass@5)&lt;/strong&gt; (Pass@5 (%)): leader DeepSeek V4.1 Flash (1017) (51.0), 2 models&lt;br&gt;&lt;span&gt;ZeroBench main set of hard visual-reasoning questions, with tools allowed, scored pass@5. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Pass@5 (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Meta Muse Wonder Scorecard - MemoryBench&lt;/strong&gt; (Rubric score (%)): leader Muse Wonder (Max) (85.6), 5 models&lt;br&gt;&lt;span&gt;Meta&amp;#x27;s internal MemoryBench: recalling critical details across long multi-turn conversations of 64K to 100K tokens, with distractor conversations and turns that conflict with or update earlier facts; an LLM judge scores each answer against weighted, question-specific rubrics. The questions are not public. Meta ran this for its October 12, 2026 Muse Wonder release scorecard, with every model at max reasoning effort; competitor runs are Meta&amp;#x27;s best-effort internal runs. Metric: Rubric score (%). Higher is better. Source: https://research.meta.ai/blog/muse-wonder&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Meta Muse Wonder Scorecard - OSWorld V2.1 (mean partial reward)&lt;/strong&gt; (Mean per-task partial reward (%)): leader Claude Fable 5.1 (Max) (80.7), 4 models&lt;br&gt;&lt;span&gt;OSWorld V2.1: 108 long-horizon real-world workflows on a full Ubuntu desktop, controlled through screenshots, clicks and typing, run in Meta&amp;#x27;s common internal framework with the official task environments and graders. Mean partial reward per task. Meta ran this for its October 12, 2026 Muse Wonder release scorecard, with every model at max reasoning effort; competitor runs are Meta&amp;#x27;s best-effort internal runs. Claude Opus 5.5&amp;#x27;s provider-reported score is left out. Metric: Mean per-task partial reward (%). Higher is better. Source: https://research.meta.ai/blog/muse-wonder&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Meta Muse Wonder Scorecard - OSWorld V2.1 (strict binary completion)&lt;/strong&gt; (Strict task completion (%)): leader Claude Fable 5.1 (Max) (43.1), 4 models&lt;br&gt;&lt;span&gt;OSWorld V2.1: 108 long-horizon real-world workflows on a full Ubuntu desktop, controlled through screenshots, clicks and typing, run in Meta&amp;#x27;s common internal framework with the official task environments and graders. Share of tasks fully completed. Meta ran this for its October 12, 2026 Muse Wonder release scorecard, with every model at max reasoning effort; competitor runs are Meta&amp;#x27;s best-effort internal runs. Metric: Strict task completion (%). Higher is better. Source: https://research.meta.ai/blog/muse-wonder&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HoldemBench&lt;/strong&gt; (All-in-adjusted win rate (mbb/hand)): leader DeepSeek V4.1 Flash (5368.1), 12 models&lt;br&gt;&lt;span&gt;HoldemBench: duplicate heads-up no-limit Texas hold&amp;#x27;em (blinds 50/100, 100 big blinds, stacks reset every hand). Every deal is played twice with seats swapped and all-in hands before the river are scored by exact equity, so most card luck cancels. LLMs, OpenRouter decision models and a local model play through one identical action menu. Score: all-in-adjusted win rate in milli-big-blinds per hand against the whole field, from one additive model over all pairings of six leagues (95% deal-cluster bootstrap intervals upstream). Unbounded; 0 is break-even. Reference bots are excluded. Higher is better. Source: https://huggingface.co/datasets/chrystians/poker-holdembench&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GACC-Bench (5% budget) - Micro accuracy&lt;/strong&gt; (Question accuracy (%)): leader GPT-6.1 Sol (42.86), 10 models&lt;br&gt;&lt;span&gt;GACC-Bench-full (500 goals, 2,571 questions over 281 contexts from seven long-document and agent-trace collections). The listed model is the goal-conditioned compressor; questions are withheld from it. A fixed DeepSeek-V4-Pro reader answers each question from the compressed context and a fixed DeepSeek-V4-Pro judge scores it against the reference (98.25% human agreement on 114 sampled responses). Retention budget 5% of source-context tokens. Micro accuracy is the percentage of individual questions answered correctly. Model labels as printed; reasoning or effort settings are not disclosed. Full-context reference: 99.38 Micro / 96.80 Macro; no-context: 6.46 / 0.20. Metric: Question accuracy (%). Higher is better. Source: https://arxiv.org/html/2610.21893v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GACC-Bench (5% budget) - Macro accuracy&lt;/strong&gt; (Goal accuracy, all questions correct (%)): leader GPT-6 Astra (6.0), 10 models&lt;br&gt;&lt;span&gt;GACC-Bench-full (500 goals, 2,571 questions over 281 contexts from seven long-document and agent-trace collections). The listed model is the goal-conditioned compressor; questions are withheld from it. A fixed DeepSeek-V4-Pro reader answers each question from the compressed context and a fixed DeepSeek-V4-Pro judge scores it against the reference (98.25% human agreement on 114 sampled responses). Retention budget 5% of source-context tokens. Macro accuracy is the percentage of goals for which every associated question is answered correctly, computed over all 500 goals. Model labels as printed; reasoning or effort settings are not disclosed. Full-context reference: 99.38 Micro / 96.80 Macro; no-context: 6.46 / 0.20. Metric: Goal accuracy, all questions correct (%). Higher is better. Source: https://arxiv.org/html/2610.21893v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GACC-Bench (10% budget) - Micro accuracy&lt;/strong&gt; (Question accuracy (%)): leader GPT-6 Astra (57.41), 10 models&lt;br&gt;&lt;span&gt;GACC-Bench-full (500 goals, 2,571 questions over 281 contexts from seven long-document and agent-trace collections). The listed model is the goal-conditioned compressor; questions are withheld from it. A fixed DeepSeek-V4-Pro reader answers each question from the compressed context and a fixed DeepSeek-V4-Pro judge scores it against the reference (98.25% human agreement on 114 sampled responses). Retention budget 10% of source-context tokens. Micro accuracy is the percentage of individual questions answered correctly. Model labels as printed; reasoning or effort settings are not disclosed. Full-context reference: 99.38 Micro / 96.80 Macro; no-context: 6.46 / 0.20. Metric: Question accuracy (%). Higher is better. Source: https://arxiv.org/html/2610.21893v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GACC-Bench (10% budget) - Macro accuracy&lt;/strong&gt; (Goal accuracy, all questions correct (%)): leader GPT-6 Astra (14.8), 10 models&lt;br&gt;&lt;span&gt;GACC-Bench-full (500 goals, 2,571 questions over 281 contexts from seven long-document and agent-trace collections). The listed model is the goal-conditioned compressor; questions are withheld from it. A fixed DeepSeek-V4-Pro reader answers each question from the compressed context and a fixed DeepSeek-V4-Pro judge scores it against the reference (98.25% human agreement on 114 sampled responses). Retention budget 10% of source-context tokens. Macro accuracy is the percentage of goals for which every associated question is answered correctly, computed over all 500 goals. Model labels as printed; reasoning or effort settings are not disclosed. Full-context reference: 99.38 Micro / 96.80 Macro; no-context: 6.46 / 0.20. Metric: Goal accuracy, all questions correct (%). Higher is better. Source: https://arxiv.org/html/2610.21893v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - Mem0&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (54.02), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. Mem0 harness (trained on). Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - Hy-Memory&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (60.55), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. Hy-Memory harness (trained on). Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - MemOS&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (50.51), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. MemOS harness (trained on). Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - EverMemOS&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (56.89), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. EverMemOS harness, held out from Hy-Memory training. Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - Macro average&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (55.49), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. Equal-weight average of the four harness scores. Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PremiseSQL - Base prompting - Flaw detection&lt;/strong&gt; (DET (fraction)): leader Claude Opus 5.5 (0.492), 16 models&lt;br&gt;&lt;span&gt;PremiseSQL premise-critique evaluation over Spider and BIRD Text-to-SQL queries with flawed-premise variants and their paired originals (3,600 requests per model and prompting setting), temperature 0 and default API configuration, two-judge labels. Base prompting: standard Text-to-SQL instructions with no premise warning. DET: share of flawed queries for which the model detects a premise flaw. Fraction from 0 to 1. Model labels as printed. Metric: DET (fraction). Higher is better. Source: https://arxiv.org/html/2610.22158v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PremiseSQL - Implicit prompting - Flaw detection&lt;/strong&gt; (DET (fraction)): leader DeepSeek V3.2 (0.898), 16 models&lt;br&gt;&lt;span&gt;PremiseSQL premise-critique evaluation over Spider and BIRD Text-to-SQL queries with flawed-premise variants and their paired originals (3,600 requests per model and prompting setting), temperature 0 and default API configuration, two-judge labels. Implicit prompting: the model is told that the question may contain errors and asked to assess its validity before answering. DET: share of flawed queries for which the model detects a premise flaw. Fraction from 0 to 1. Model labels as printed. Metric: DET (fraction). Higher is better. Source: https://arxiv.org/html/2610.22158v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PremiseSQL - Implicit prompting - Valid-query recognition&lt;/strong&gt; (VAL (fraction)): leader Gemini 3.1 Pro (Preview) (0.99), 16 models&lt;br&gt;&lt;span&gt;PremiseSQL premise-critique evaluation over Spider and BIRD Text-to-SQL queries with flawed-premise variants and their paired originals (3,600 requests per model and prompting setting), temperature 0 and default API configuration, two-judge labels. Implicit prompting: the model is told that the question may contain errors and asked to assess its validity before answering. VAL: share of the paired valid original queries that the model explicitly recognises as valid, the guard against rejecting everything. Fraction from 0 to 1. Model labels as printed. Metric: VAL (fraction). Higher is better. Source: https://arxiv.org/html/2610.22158v1&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (1)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash (1017)&lt;/strong&gt; — Aggregate Rating 1911, #15&lt;ul&gt;&lt;li&gt;SelfBench - PostHog/posthog: 65.71 (#2/14)&lt;/li&gt;&lt;li&gt;SelfBench - mupt-ai/self-bench: 71.05 (#3/7)&lt;/li&gt;&lt;li&gt;SelfBench - vercel/next.js: 71.05 (#3/13)&lt;/li&gt;&lt;li&gt;SelfBench - getsentry/sentry: 79.49 (#4/15)&lt;/li&gt;&lt;li&gt;NL2Repo: 68.2 (#5/51)&lt;/li&gt;&lt;li&gt;LLM2014 Logic 2026-10: 59.46 (#9/45)&lt;/li&gt;&lt;li&gt;VoxelBench: 2052.0 (#10/59)&lt;/li&gt;&lt;li&gt;Bug Hunt Bench - LMS: 21.2 (#11/117)&lt;/li&gt;&lt;li&gt;Bug Hunt Bench: 39.4 (#12/117)&lt;/li&gt;&lt;li&gt;Bug Hunt Bench - VS Code Extension: 18.2 (#12/117)&lt;/li&gt;&lt;li&gt;...and 20 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (17)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on HiL-Bench: 59.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on HiL-Bench: 65.32 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on HiL-Bench: 62.33 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Bug Hunt Bench: 40.7 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Bug Hunt Bench - LMS: 27.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 13.0 (#31)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on HiL-Bench: 62.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MathArena - ARXIVLEAN September: 39.62 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on UGI - Natural Intelligence: 81.36 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on UGI - Willingness (W/10): 1.8 (#1247)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on UGI - Writing: 79.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on UGI Leaderboard: 49.92 (#150)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vectara Hallucination Leaderboard: 87.9 (#83)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Websec.fr CTFs: 74.7 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - API Security: 82.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Overall: 70.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 66.7 (#12)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (10)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;KNOWS - Accessibility Tree + Screenshot&lt;/strong&gt;: GPT-6 Astra (72.5) beat GPT-5.5 by 23.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HiL-Bench&lt;/strong&gt;: Muse Wonder (79.67) beat Claude Fable 5.1 by 18.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (ADK React) - Cloud Security&lt;/strong&gt;: Gemini 4 Argon (82.6) beat Gemini 3.8 Flash Cyber by 15.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Claude Code) - Cloud Security&lt;/strong&gt;: Gemini 4 Argon (73.9) beat Claude Opus 5 by 10.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (ADK React) - Code Vulnerabilities&lt;/strong&gt;: Gemini 4 Argon (65.8) beat Gemini 3.8 Flash Cyber by 8.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs&lt;/strong&gt;: GPT-6 Sol (88.5) beat Claude Opus 5 by 6.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities&lt;/strong&gt;: Gemini 4 Argon (59.9) beat Claude Opus 5 by 4.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (ADK React) - Overall&lt;/strong&gt;: Gemini 4 Argon (77.2) beat Gemini 3.8 Flash Cyber by 2.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;UGI - Writing&lt;/strong&gt;: gemini-4-argon (Thinking, low) (79.71) beat Gemini 3.8 Flash (Thinking, Medium) by 1.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (ADK React) - API Security&lt;/strong&gt;: Gemini 4 Argon (85.8) beat Gemini 3.8 Flash Cyber by 0.6&lt;/li&gt;&lt;/ul&gt;
&lt;hr/&gt;
&lt;h2&gt;Weekly&lt;/h2&gt;
&lt;h3&gt;New Benchmarks (38)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;AgenticBBO Frontier - Digits MLP HPO&lt;/strong&gt; (Normalized anytime/final score (0-1)): leader GPT-6 Astra (Max) (0.617), 7 models&lt;br&gt;&lt;span&gt;AgenticBBO-Bench frontier challenge on the single Digits MLP HPO task, with 5 initial + 25 new black-box evaluations and seed 2. An LLM agent uses the Codex harness to propose and adapt optimization queries. The normalized score is 70% anytime trajectory quality plus 30% final quality, bounded 0–1; higher is better. This is one run per model/task, not a mean over the broad 77-task benchmark. Published reproduction profiles request max reasoning for six models and high for Gemini 3.1 Pro Preview; original historical run catalogs are not distributed. The paper’s DeepSeek V4.1 Flash identity is retained despite its reproduction API alias lacking .1. The numerical Gaussian-process reference is not an LLM. Single-run results do not measure run-to-run uncertainty.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AgenticBBO Frontier - Sysbench Database Tuning&lt;/strong&gt; (Normalized anytime/final score (0-1)): leader GPT-6 Astra (Max) (0.501), 7 models&lt;br&gt;&lt;span&gt;AgenticBBO-Bench frontier challenge on the single Sysbench Database Tuning task, with 50 initial + 200 new black-box evaluations and seed 2. An LLM agent uses the Codex harness to propose and adapt optimization queries. The normalized score is 70% anytime trajectory quality plus 30% final quality, bounded 0–1; higher is better. This is one run per model/task, not a mean over the broad 77-task benchmark. Published reproduction profiles request max reasoning for six models and high for Gemini 3.1 Pro Preview; original historical run catalogs are not distributed. The paper’s DeepSeek V4.1 Flash identity is retained despite its reproduction API alias lacking .1. The numerical Gaussian-process reference is not an LLM. Single-run results do not measure run-to-run uncertainty.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AgenticBBO Frontier - Median1 Molecular Design&lt;/strong&gt; (Normalized anytime/final score (0-1)): leader DeepSeek V4.1 Flash (Max) (0.823), 7 models&lt;br&gt;&lt;span&gt;AgenticBBO-Bench frontier challenge on the single Median1 Molecular Design task, with 50 initial + 200 new black-box evaluations and seed 2. An LLM agent uses the Codex harness to propose and adapt optimization queries. The normalized score is 70% anytime trajectory quality plus 30% final quality, bounded 0–1; higher is better. This is one run per model/task, not a mean over the broad 77-task benchmark. Published reproduction profiles request max reasoning for six models and high for Gemini 3.1 Pro Preview; original historical run catalogs are not distributed. The paper’s DeepSeek V4.1 Flash identity is retained despite its reproduction API alias lacking .1. The numerical Gaussian-process reference is not an LLM. Single-run results do not measure run-to-run uncertainty.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AgenticBBO Frontier - BBOB f15 d10&lt;/strong&gt; (Normalized anytime/final score (0-1)): leader GPT-6 Astra (Max) (0.664), 7 models&lt;br&gt;&lt;span&gt;AgenticBBO-Bench frontier challenge on the single BBOB f15 d10 task, with 20 initial + 100 new black-box evaluations and seed 2. An LLM agent uses the Codex harness to propose and adapt optimization queries. The normalized score is 70% anytime trajectory quality plus 30% final quality, bounded 0–1; higher is better. This is one run per model/task, not a mean over the broad 77-task benchmark. Published reproduction profiles request max reasoning for six models and high for Gemini 3.1 Pro Preview; original historical run catalogs are not distributed. The paper’s DeepSeek V4.1 Flash identity is retained despite its reproduction API alias lacking .1. The numerical Gaussian-process reference is not an LLM. Single-run results do not measure run-to-run uncertainty.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AgenticBBO Frontier - Bigblue1 Chip Placement&lt;/strong&gt; (Normalized anytime/final score (0-1)): leader DeepSeek V4.1 Flash (Max) (0.673), 7 models&lt;br&gt;&lt;span&gt;AgenticBBO-Bench frontier challenge on the single Bigblue1 Chip Placement task, with 50 initial + 200 new black-box evaluations and seed 2. An LLM agent uses the Codex harness to propose and adapt optimization queries. The normalized score is 70% anytime trajectory quality plus 30% final quality, bounded 0–1; higher is better. This is one run per model/task, not a mean over the broad 77-task benchmark. Published reproduction profiles request max reasoning for six models and high for Gemini 3.1 Pro Preview; original historical run catalogs are not distributed. The paper’s DeepSeek V4.1 Flash identity is retained despite its reproduction API alias lacking .1. The numerical Gaussian-process reference is not an LLM. Single-run results do not measure run-to-run uncertainty.&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - Overall&lt;/strong&gt; (Macro average of the four category pass@1 rates (%)): leader Gemini 4 Argon (77.0), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - Code Vulnerabilities&lt;/strong&gt; (Code Vulnerabilities challenges solved, pass@1 (%)): leader Gemini 4 Argon (64.9), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - API Security&lt;/strong&gt; (API Security challenges solved, pass@1 (%)): leader Gemini 4 Argon (87.0), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - Websec.fr CTFs&lt;/strong&gt; (Websec.fr CTFs challenges solved, pass@1 (%)): leader Gemini 3.8 Flash Cyber (77.0), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Antigravity) - Cloud Security&lt;/strong&gt; (Cloud Security challenges solved, pass@1 (%)): leader Gemini 4 Argon (87.0), 5 models&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - HLE w/o tools (full set)&lt;/strong&gt; (Accuracy (%)): leader DeepSeek V4.1 Flash (1017) (42.6), 2 models&lt;br&gt;&lt;span&gt;Humanity&amp;#x27;s Last Exam without tools, full question set. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Accuracy (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - HLE w/o tools (text-only subset)&lt;/strong&gt; (Accuracy (%)): leader DeepSeek V4.1 Flash (1017) (44.0), 2 models&lt;br&gt;&lt;span&gt;Humanity&amp;#x27;s Last Exam without tools, text-only subset of the questions. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Accuracy (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Terminal-Bench 4.0 (Claude Code harness)&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (42.3), 2 models&lt;br&gt;&lt;span&gt;Terminal-Bench 4.0 command-line tasks, run in the Claude Code harness. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - DeepSWE v1.1 (Mini-SWE-V2 harness)&lt;/strong&gt; (Pass@1 (%)): leader DeepSeek V4.1 Flash (1017) (77.0), 2 models&lt;br&gt;&lt;span&gt;DeepSWE v1.1 software-engineering tasks, run in the Mini-SWE-V2 harness, one attempt per task. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Pass@1 (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - ProgramBench&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (29.1), 2 models&lt;br&gt;&lt;span&gt;ProgramBench program-construction tasks. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - NL2Repo-Bench&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (68.2), 2 models&lt;br&gt;&lt;span&gt;NL2Repo-Bench: building a complete code repository from a natural-language specification. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Automation-Bench v1.0.6&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (60.5), 2 models&lt;br&gt;&lt;span&gt;Automation-Bench version 1.0.6 workflow-automation tasks. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Agents' Last Exam&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (35.4), 2 models&lt;br&gt;&lt;span&gt;Agents&amp;#x27; Last Exam agentic tasks. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Apex Bench v1.1&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (72.1), 2 models&lt;br&gt;&lt;span&gt;Apex Bench version 1.1. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - Chartography (with tools)&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (83.0), 2 models&lt;br&gt;&lt;span&gt;Chartography chart-understanding questions, with tools allowed. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - BabyVision (with tools)&lt;/strong&gt; (Score (%)): leader DeepSeek V4.1 Flash (1017) (92.5), 2 models&lt;br&gt;&lt;span&gt;BabyVision visual-reasoning questions, with tools allowed. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Score (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash 1017 Card - ZeroBench-main (with tools, Pass@5)&lt;/strong&gt; (Pass@5 (%)): leader DeepSeek V4.1 Flash (1017) (51.0), 2 models&lt;br&gt;&lt;span&gt;ZeroBench main set of hard visual-reasoning questions, with tools allowed, scored pass@5. DeepSeek reported this on the DeepSeek-V4.1-Flash-1017 model card and API change log (October 17, 2026). Only DeepSeek&amp;#x27;s own runs of the dated 1017 checkpoint and the original V4.1 Flash are included; the card&amp;#x27;s comparator columns are left out, and reasoning effort is not stated. Metric: Pass@5 (%). Higher is better. Source: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017/raw/1bae0e1088993b9b82de08f59de5be5a34e42170/README.md&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Meta Muse Wonder Scorecard - MemoryBench&lt;/strong&gt; (Rubric score (%)): leader Muse Wonder (Max) (85.6), 5 models&lt;br&gt;&lt;span&gt;Meta&amp;#x27;s internal MemoryBench: recalling critical details across long multi-turn conversations of 64K to 100K tokens, with distractor conversations and turns that conflict with or update earlier facts; an LLM judge scores each answer against weighted, question-specific rubrics. The questions are not public. Meta ran this for its October 12, 2026 Muse Wonder release scorecard, with every model at max reasoning effort; competitor runs are Meta&amp;#x27;s best-effort internal runs. Metric: Rubric score (%). Higher is better. Source: https://research.meta.ai/blog/muse-wonder&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Meta Muse Wonder Scorecard - OSWorld V2.1 (mean partial reward)&lt;/strong&gt; (Mean per-task partial reward (%)): leader Claude Fable 5.1 (Max) (80.7), 4 models&lt;br&gt;&lt;span&gt;OSWorld V2.1: 108 long-horizon real-world workflows on a full Ubuntu desktop, controlled through screenshots, clicks and typing, run in Meta&amp;#x27;s common internal framework with the official task environments and graders. Mean partial reward per task. Meta ran this for its October 12, 2026 Muse Wonder release scorecard, with every model at max reasoning effort; competitor runs are Meta&amp;#x27;s best-effort internal runs. Claude Opus 5.5&amp;#x27;s provider-reported score is left out. Metric: Mean per-task partial reward (%). Higher is better. Source: https://research.meta.ai/blog/muse-wonder&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Meta Muse Wonder Scorecard - OSWorld V2.1 (strict binary completion)&lt;/strong&gt; (Strict task completion (%)): leader Claude Fable 5.1 (Max) (43.1), 4 models&lt;br&gt;&lt;span&gt;OSWorld V2.1: 108 long-horizon real-world workflows on a full Ubuntu desktop, controlled through screenshots, clicks and typing, run in Meta&amp;#x27;s common internal framework with the official task environments and graders. Share of tasks fully completed. Meta ran this for its October 12, 2026 Muse Wonder release scorecard, with every model at max reasoning effort; competitor runs are Meta&amp;#x27;s best-effort internal runs. Metric: Strict task completion (%). Higher is better. Source: https://research.meta.ai/blog/muse-wonder&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HoldemBench&lt;/strong&gt; (All-in-adjusted win rate (mbb/hand)): leader DeepSeek V4.1 Flash (5368.1), 12 models&lt;br&gt;&lt;span&gt;HoldemBench: duplicate heads-up no-limit Texas hold&amp;#x27;em (blinds 50/100, 100 big blinds, stacks reset every hand). Every deal is played twice with seats swapped and all-in hands before the river are scored by exact equity, so most card luck cancels. LLMs, OpenRouter decision models and a local model play through one identical action menu. Score: all-in-adjusted win rate in milli-big-blinds per hand against the whole field, from one additive model over all pairings of six leagues (95% deal-cluster bootstrap intervals upstream). Unbounded; 0 is break-even. Reference bots are excluded. Higher is better. Source: https://huggingface.co/datasets/chrystians/poker-holdembench&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GACC-Bench (5% budget) - Micro accuracy&lt;/strong&gt; (Question accuracy (%)): leader GPT-6.1 Sol (42.86), 10 models&lt;br&gt;&lt;span&gt;GACC-Bench-full (500 goals, 2,571 questions over 281 contexts from seven long-document and agent-trace collections). The listed model is the goal-conditioned compressor; questions are withheld from it. A fixed DeepSeek-V4-Pro reader answers each question from the compressed context and a fixed DeepSeek-V4-Pro judge scores it against the reference (98.25% human agreement on 114 sampled responses). Retention budget 5% of source-context tokens. Micro accuracy is the percentage of individual questions answered correctly. Model labels as printed; reasoning or effort settings are not disclosed. Full-context reference: 99.38 Micro / 96.80 Macro; no-context: 6.46 / 0.20. Metric: Question accuracy (%). Higher is better. Source: https://arxiv.org/html/2610.21893v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GACC-Bench (5% budget) - Macro accuracy&lt;/strong&gt; (Goal accuracy, all questions correct (%)): leader GPT-6 Astra (6.0), 10 models&lt;br&gt;&lt;span&gt;GACC-Bench-full (500 goals, 2,571 questions over 281 contexts from seven long-document and agent-trace collections). The listed model is the goal-conditioned compressor; questions are withheld from it. A fixed DeepSeek-V4-Pro reader answers each question from the compressed context and a fixed DeepSeek-V4-Pro judge scores it against the reference (98.25% human agreement on 114 sampled responses). Retention budget 5% of source-context tokens. Macro accuracy is the percentage of goals for which every associated question is answered correctly, computed over all 500 goals. Model labels as printed; reasoning or effort settings are not disclosed. Full-context reference: 99.38 Micro / 96.80 Macro; no-context: 6.46 / 0.20. Metric: Goal accuracy, all questions correct (%). Higher is better. Source: https://arxiv.org/html/2610.21893v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GACC-Bench (10% budget) - Micro accuracy&lt;/strong&gt; (Question accuracy (%)): leader GPT-6 Astra (57.41), 10 models&lt;br&gt;&lt;span&gt;GACC-Bench-full (500 goals, 2,571 questions over 281 contexts from seven long-document and agent-trace collections). The listed model is the goal-conditioned compressor; questions are withheld from it. A fixed DeepSeek-V4-Pro reader answers each question from the compressed context and a fixed DeepSeek-V4-Pro judge scores it against the reference (98.25% human agreement on 114 sampled responses). Retention budget 10% of source-context tokens. Micro accuracy is the percentage of individual questions answered correctly. Model labels as printed; reasoning or effort settings are not disclosed. Full-context reference: 99.38 Micro / 96.80 Macro; no-context: 6.46 / 0.20. Metric: Question accuracy (%). Higher is better. Source: https://arxiv.org/html/2610.21893v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GACC-Bench (10% budget) - Macro accuracy&lt;/strong&gt; (Goal accuracy, all questions correct (%)): leader GPT-6 Astra (14.8), 10 models&lt;br&gt;&lt;span&gt;GACC-Bench-full (500 goals, 2,571 questions over 281 contexts from seven long-document and agent-trace collections). The listed model is the goal-conditioned compressor; questions are withheld from it. A fixed DeepSeek-V4-Pro reader answers each question from the compressed context and a fixed DeepSeek-V4-Pro judge scores it against the reference (98.25% human agreement on 114 sampled responses). Retention budget 10% of source-context tokens. Macro accuracy is the percentage of goals for which every associated question is answered correctly, computed over all 500 goals. Model labels as printed; reasoning or effort settings are not disclosed. Full-context reference: 99.38 Micro / 96.80 Macro; no-context: 6.46 / 0.20. Metric: Goal accuracy, all questions correct (%). Higher is better. Source: https://arxiv.org/html/2610.21893v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - Mem0&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (54.02), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. Mem0 harness (trained on). Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - Hy-Memory&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (60.55), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. Hy-Memory harness (trained on). Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - MemOS&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (50.51), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. MemOS harness (trained on). Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - EverMemOS&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (56.89), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. EverMemOS harness, held out from Hy-Memory training. Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy-Memory Writer Matrix - Macro average&lt;/strong&gt; (Six-metric mean score (%)): leader Hy-Memory-30B-A3B (55.49), 7 models&lt;br&gt;&lt;span&gt;Long-term memory writing: the listed model is the memory writer inside an unchanged memory framework; a frozen answer model and judge are shared. Each harness score is the equal-weight mean of six metrics (LoCoMo judge, LongMemEval, PersonaMem-v2, Travel SPS, Shopping PS, Code Memory Bench) on fixed deterministic 20% benchmark subsamples. Equal-weight average of the four harness scores. Checkpoint identifiers are omitted by the paper. Hy3 is the paper&amp;#x27;s Hy3 flagship label (reasoning mode unspecified); Hy3-A3B Base is the paper&amp;#x27;s label for the Hy3-30B-A3B instruction-tuned starting checkpoint of Hy-Memory-30B-A3B. Metric: Six-metric mean score (%). Higher is better. Source: https://arxiv.org/html/2610.22196v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PremiseSQL - Base prompting - Flaw detection&lt;/strong&gt; (DET (fraction)): leader Claude Opus 5.5 (0.492), 16 models&lt;br&gt;&lt;span&gt;PremiseSQL premise-critique evaluation over Spider and BIRD Text-to-SQL queries with flawed-premise variants and their paired originals (3,600 requests per model and prompting setting), temperature 0 and default API configuration, two-judge labels. Base prompting: standard Text-to-SQL instructions with no premise warning. DET: share of flawed queries for which the model detects a premise flaw. Fraction from 0 to 1. Model labels as printed. Metric: DET (fraction). Higher is better. Source: https://arxiv.org/html/2610.22158v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PremiseSQL - Implicit prompting - Flaw detection&lt;/strong&gt; (DET (fraction)): leader DeepSeek V3.2 (0.898), 16 models&lt;br&gt;&lt;span&gt;PremiseSQL premise-critique evaluation over Spider and BIRD Text-to-SQL queries with flawed-premise variants and their paired originals (3,600 requests per model and prompting setting), temperature 0 and default API configuration, two-judge labels. Implicit prompting: the model is told that the question may contain errors and asked to assess its validity before answering. DET: share of flawed queries for which the model detects a premise flaw. Fraction from 0 to 1. Model labels as printed. Metric: DET (fraction). Higher is better. Source: https://arxiv.org/html/2610.22158v1&lt;/span&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PremiseSQL - Implicit prompting - Valid-query recognition&lt;/strong&gt; (VAL (fraction)): leader Gemini 3.1 Pro (Preview) (0.99), 16 models&lt;br&gt;&lt;span&gt;PremiseSQL premise-critique evaluation over Spider and BIRD Text-to-SQL queries with flawed-premise variants and their paired originals (3,600 requests per model and prompting setting), temperature 0 and default API configuration, two-judge labels. Implicit prompting: the model is told that the question may contain errors and asked to assess its validity before answering. VAL: share of the paired valid original queries that the model explicitly recognises as valid, the guard against rejecting everything. Fraction from 0 to 1. Model labels as printed. Metric: VAL (fraction). Higher is better. Source: https://arxiv.org/html/2610.22158v1&lt;/span&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New Models (109)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra Pro&lt;/strong&gt; — Aggregate Rating 1989, #3&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 76.0 (#1/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; — Aggregate Rating 1988, #4&lt;ul&gt;&lt;li&gt;Agent Arena: 15.53 (#1/51)&lt;/li&gt;&lt;li&gt;Agent Arena - Confirmed Success: 17.29 (#1/51)&lt;/li&gt;&lt;li&gt;Anthropic Child Safety Single-Turn Benign Requests - Over-Refusal Rate (Claude.ai): 0.05 (#1/7)&lt;/li&gt;&lt;li&gt;Anthropic Claude Code Dual-Use Requests - Success Rate (Mythos 5.1 System Card): 99.8 (#1/5)&lt;/li&gt;&lt;li&gt;SWE-bench Pro (Opus 5 &amp; Mythos 5.1 System Cards): 89.9 (#1/3)&lt;/li&gt;&lt;li&gt;ParseBench: 79.85 (#1/113)&lt;/li&gt;&lt;li&gt;ParseBench - Tables: 94.25 (#1/113)&lt;/li&gt;&lt;li&gt;ParseBench - Charts: 70.89 (#1/113)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Physics: 2.808 (#1/8)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Earth science: 2.76 (#1/8)&lt;/li&gt;&lt;li&gt;...and 38 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; — Aggregate Rating 1979, #6&lt;ul&gt;&lt;li&gt;SciLaws-Bench: 51.33 (#1/14)&lt;/li&gt;&lt;li&gt;Design Arena (3D): 1509.0 (#1/166)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Tools) - Overall: 2.809 (#1/2)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Tools) - Biology: 2.709 (#1/2)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Tools) - Chemistry: 2.777 (#1/2)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Tools) - Physics: 3.025 (#1/2)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Tools) - Astronomy: 2.723 (#1/2)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Tools) - Earth science: 2.726 (#1/2)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Tools) - Engineering: 2.894 (#1/2)&lt;/li&gt;&lt;li&gt;KNOWS - Accessibility Tree + Screenshot: 72.5 (#1/4)&lt;/li&gt;&lt;li&gt;...and 3 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; — Aggregate Rating 1967, #7&lt;ul&gt;&lt;li&gt;GDP.pdf: 35.0 (#1/6)&lt;/li&gt;&lt;li&gt;Kaggle FACTS (Google): 75.29 (#2/39)&lt;/li&gt;&lt;li&gt;Kaggle FACTS Grounding: 82.42 (#2/52)&lt;/li&gt;&lt;li&gt;Kaggle FACTS Parametric: 82.11 (#2/44)&lt;/li&gt;&lt;li&gt;Kaggle FACTS Multimodal: 48.46 (#3/38)&lt;/li&gt;&lt;li&gt;HiL-Bench: 62.33 (#4/28)&lt;/li&gt;&lt;li&gt;HLE-Diamond (With Tools): 69.1 (#4/8)&lt;/li&gt;&lt;li&gt;Design Arena (3D): 1458.0 (#5/166)&lt;/li&gt;&lt;li&gt;Design Arena (UI Components): 1325.0 (#16/172)&lt;/li&gt;&lt;li&gt;Kaggle Open Benchmarks - SciCode Subproblems: 37.8 (#16/60)&lt;/li&gt;&lt;li&gt;...and 23 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; — Aggregate Rating 1965, #8&lt;ul&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 100.0 (#1/304)&lt;/li&gt;&lt;li&gt;SimpleQA Verified: 76.0 (#1/91)&lt;/li&gt;&lt;li&gt;HieroglyphBench: 69.7 (#1/33)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Astronomy: 2.778 (#1/8)&lt;/li&gt;&lt;li&gt;Humanity's Sixth Sense: 55.4 (#1/27)&lt;/li&gt;&lt;li&gt;Argo-Bench - Solved: 35.71 (#1/50)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Famous Phrases: 36.19 (#1/68)&lt;/li&gt;&lt;li&gt;Icelandic LLM - Idiom Completion: 75.96 (#1/68)&lt;/li&gt;&lt;li&gt;NYT Connections Extended: 98.6 (#1/128)&lt;/li&gt;&lt;li&gt;Kaggle Global-MMLU-Lite - Bengali: 96.0 (#1/44)&lt;/li&gt;&lt;li&gt;...and 186 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; — Aggregate Rating 1958, #9&lt;ul&gt;&lt;li&gt;BBQ Disambiguated Accuracy (Mythos 5.1 System Card): 89.92 (#1/6)&lt;/li&gt;&lt;li&gt;Anthropic Election Integrity Multi-Turn - Appropriate Response Rate (API): 90.0 (#4/8)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; — Aggregate Rating 1943, #10&lt;ul&gt;&lt;li&gt;LLM Stats (DeepSWE 1.1): 79.0 (#1/45)&lt;/li&gt;&lt;li&gt;LLM Stats (SWE-Marathon): 57.5 (#1/8)&lt;/li&gt;&lt;li&gt;Vals AI Vibe Code Bench 1-100: 33.98 (#1/25)&lt;/li&gt;&lt;li&gt;AA CritPt: 32.86 (#1/581)&lt;/li&gt;&lt;li&gt;AA GDP.pdf: 32.6 (#1/224)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA: 13.06 (#1/27)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - Corporate M&amp;A: 98.18 (#1/27)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - Funds &amp; Asset Management: 95.72 (#1/27)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - Intellectual Property: 97.86 (#1/27)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - International Trade &amp; Sanctions: 96.41 (#1/27)&lt;/li&gt;&lt;li&gt;...and 108 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5.5&lt;/strong&gt; — Aggregate Rating 1936, #11&lt;ul&gt;&lt;li&gt;Conceptual Reasoning Index - Argument Evaluation (LMCA): 68.26 (#1/211)&lt;/li&gt;&lt;li&gt;ReLE - Coding: 78.0 (#1/90)&lt;/li&gt;&lt;li&gt;Conceptual Reasoning Index: 77.08 (#2/210)&lt;/li&gt;&lt;li&gt;SecIT Bench (Claude Code): 69.93 (#2/4)&lt;/li&gt;&lt;li&gt;Design Arena (3D): 1463.0 (#4/166)&lt;/li&gt;&lt;li&gt;SEAL - Humanity's Last Exam: 46.12 (#4/55)&lt;/li&gt;&lt;li&gt;SEAL - Humanity's Last Exam (Text Only): 46.62 (#4/65)&lt;/li&gt;&lt;li&gt;GDP.pdf: 31.4 (#4/6)&lt;/li&gt;&lt;li&gt;Conceptual Reasoning Index - Consistency (ACCoRD): 85.66 (#4/210)&lt;/li&gt;&lt;li&gt;ReLE - Reasoning and Mathematics: 85.7 (#8/187)&lt;/li&gt;&lt;li&gt;...and 19 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5&lt;/strong&gt; — Aggregate Rating 1930, #12&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 68.7 (#4/6)&lt;/li&gt;&lt;li&gt;BBQ Disambiguated Accuracy (Mythos 5.1 System Card): 82.14 (#5/6)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder&lt;/strong&gt; — Aggregate Rating 1911, #14&lt;ul&gt;&lt;li&gt;Design Arena (Website): 1361.0 (#1/192)&lt;/li&gt;&lt;li&gt;HiL-Bench: 79.67 (#1/28)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Overall: 2.683 (#1/8)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Biology: 2.615 (#1/8)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Chemistry: 2.703 (#1/8)&lt;/li&gt;&lt;li&gt;Vals AI CyberBench: 80.36 (#1/47)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 88.03 (#1/224)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - Data Privacy &amp; Cybersecurity: 98.12 (#1/27)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - Healthcare &amp; Life Sciences: 96.09 (#1/27)&lt;/li&gt;&lt;li&gt;Harvey LAB-AA - Immigration: 95.39 (#1/27)&lt;/li&gt;&lt;li&gt;...and 176 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash (1017)&lt;/strong&gt; — Aggregate Rating 1911, #15&lt;ul&gt;&lt;li&gt;SelfBench - PostHog/posthog: 65.71 (#2/14)&lt;/li&gt;&lt;li&gt;SelfBench - mupt-ai/self-bench: 71.05 (#3/7)&lt;/li&gt;&lt;li&gt;SelfBench - vercel/next.js: 71.05 (#3/13)&lt;/li&gt;&lt;li&gt;SelfBench - getsentry/sentry: 79.49 (#4/15)&lt;/li&gt;&lt;li&gt;NL2Repo: 68.2 (#5/51)&lt;/li&gt;&lt;li&gt;LLM2014 Logic 2026-10: 59.46 (#9/45)&lt;/li&gt;&lt;li&gt;VoxelBench: 2052.0 (#10/59)&lt;/li&gt;&lt;li&gt;Bug Hunt Bench - LMS: 21.2 (#11/117)&lt;/li&gt;&lt;li&gt;Bug Hunt Bench: 39.4 (#12/117)&lt;/li&gt;&lt;li&gt;Bug Hunt Bench - VS Code Extension: 18.2 (#12/117)&lt;/li&gt;&lt;li&gt;...and 20 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Sol&lt;/strong&gt; — Aggregate Rating 1910, #16&lt;ul&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 88.5 (#1/20)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Overall: 69.3 (#3/20)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 50.8 (#3/20)&lt;/li&gt;&lt;li&gt;Design Arena (SVG): 1328.0 (#5/129)&lt;/li&gt;&lt;li&gt;AndroidWorld: 100.0 (#5/48)&lt;/li&gt;&lt;li&gt;AutomationBench: 33.2 (#7/109)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Cloud Security: 59.4 (#7/20)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - API Security: 78.4 (#8/20)&lt;/li&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 56.0 (#15/153)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 57.64 (#26/220)&lt;/li&gt;&lt;li&gt;...and 1 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5&lt;/strong&gt; — Aggregate Rating 1907, #17&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 89.8 (#32/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5 Pro&lt;/strong&gt; — Aggregate Rating 1899, #18&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 62.0 (#8/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Sol&lt;/strong&gt; — Aggregate Rating 1889, #20&lt;ul&gt;&lt;li&gt;SciLaws-Bench: 40.11 (#12/14)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 57.75 (#23/220)&lt;/li&gt;&lt;li&gt;OmniDocBench 1.5: 85.8 (#62/99)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 57.431351352674 (#62/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.8 Flash&lt;/strong&gt; — Aggregate Rating 1873, #22&lt;ul&gt;&lt;li&gt;SEAL - AudioMultiChallenge - Text Output: 60.4 (#1/21)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent: 46.25 (#15/50)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@1: 61.25 (#17/50)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 72.5 (#28/50)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 56.6 (#35/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.7 Flash&lt;/strong&gt; — Aggregate Rating 1871, #23&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 59.84 (#11/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.3&lt;/strong&gt; — Aggregate Rating 1865, #25&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 59.72 (#12/220)&lt;/li&gt;&lt;li&gt;Epoch AI - Frontierswe: 25.8 (#13/21)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent: 37.5 (#26/50)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@1: 56.5 (#28/50)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 70.707351473382 (#29/208)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 71.25 (#30/50)&lt;/li&gt;&lt;li&gt;Epoch AI - Apex Agents: 47.6 (#31/54)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.6&lt;/strong&gt; — Aggregate Rating 1864, #26&lt;ul&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 65.6 (#6/6)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 56.48 (#36/220)&lt;/li&gt;&lt;li&gt;Kaggle WWTP Engineering: 0.0 (#37/37)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.7&lt;/strong&gt; — Aggregate Rating 1864, #27&lt;ul&gt;&lt;li&gt;AA-AnalystAgent - Pass@1: 67.0 (#4/50)&lt;/li&gt;&lt;li&gt;EnterpriseBench (CoreCraft Agents): 68.2 (#5/6)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent: 55.0 (#5/50)&lt;/li&gt;&lt;li&gt;MineBench: 1840.0 (#14/75)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 72.5 (#25/50)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 57.41 (#29/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Max (0902)&lt;/strong&gt; — Aggregate Rating 1854, #28&lt;ul&gt;&lt;li&gt;DuelLab Overall: 70.09754911653 (#30/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K3&lt;/strong&gt; — Aggregate Rating 1852, #29&lt;ul&gt;&lt;li&gt;SciLaws-Bench: 47.36 (#4/14)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 59.49 (#13/220)&lt;/li&gt;&lt;li&gt;HiL-Bench: 42.67 (#13/28)&lt;/li&gt;&lt;li&gt;OmniDocBench 1.5: 91.1 (#17/99)&lt;/li&gt;&lt;li&gt;HLE-Diamond - Knowledge: 24.0 (#19/28)&lt;/li&gt;&lt;li&gt;HLE-Diamond: 28.3 (#22/28)&lt;/li&gt;&lt;li&gt;HLE-Diamond - Reasoning: 32.6 (#23/28)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 44.992307148713 (#108/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.5&lt;/strong&gt; — Aggregate Rating 1850, #31&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 63.0 (#7/153)&lt;/li&gt;&lt;li&gt;HiL-Bench: 39.67 (#16/28)&lt;/li&gt;&lt;li&gt;CyberGym: 81.8 (#16/52)&lt;/li&gt;&lt;li&gt;OmniDocBench 1.5: 89.4 (#33/99)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 56.13 (#40/220)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 37.096104955095 (#141/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 Max&lt;/strong&gt; — Aggregate Rating 1849, #32&lt;ul&gt;&lt;li&gt;Vals AI SAFE-Teen - Urgent Safety: 26.0 (#5/18)&lt;/li&gt;&lt;li&gt;Vals AI SAFE-Teen - Health Advice: 35.0 (#5/18)&lt;/li&gt;&lt;li&gt;Vals AI SAFE-Teen - Relationship Boundaries: 51.0 (#7/18)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4.1 Flash&lt;/strong&gt; — Aggregate Rating 1842, #35&lt;ul&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - API Security: 80.9 (#4/18)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 80.0 (#5/50)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Cloud Security: 56.5 (#9/20)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Cloud Security: 50.7 (#10/18)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - API Security: 78.4 (#10/20)&lt;/li&gt;&lt;li&gt;SciLaws-Bench: 40.59 (#11/14)&lt;/li&gt;&lt;li&gt;SelfBench - getsentry/sentry: 57.69 (#13/15)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Overall: 47.4 (#13/18)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Code Vulnerabilities: 16.4 (#15/18)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Websec.fr CTFs: 41.4 (#15/18)&lt;/li&gt;&lt;li&gt;...and 9 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 5.5&lt;/strong&gt; — Aggregate Rating 1840, #36&lt;ul&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 83.75 (#2/50)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@1: 68.25 (#3/50)&lt;/li&gt;&lt;li&gt;DRACO: 81.5 (#6/17)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent: 53.75 (#7/50)&lt;/li&gt;&lt;li&gt;Epoch AI - Frontierswe: 43.77 (#8/21)&lt;/li&gt;&lt;li&gt;Epoch AI - Ebr Bench: 43.81 (#9/27)&lt;/li&gt;&lt;li&gt;Epoch AI - Proofbench: 89.0 (#9/47)&lt;/li&gt;&lt;li&gt;KernelBench Hub - CUDA: 34.02 (#11/21)&lt;/li&gt;&lt;li&gt;BridgeBench Overall: 601.0 (#11/30)&lt;/li&gt;&lt;li&gt;Design Arena (3D): 1348.0 (#14/166)&lt;/li&gt;&lt;li&gt;...and 58 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.8&lt;/strong&gt; — Aggregate Rating 1840, #37&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 87.9 (#51/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hy4 preview&lt;/strong&gt; — Aggregate Rating 1838, #38&lt;ul&gt;&lt;li&gt;Wolfram LLM Benchmarking Project: 66.8 (#54/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.6-Pro&lt;/strong&gt; — Aggregate Rating 1837, #39&lt;ul&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Biology: 2.533 (#4/8)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Earth science: 2.623 (#4/8)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 80.0 (#4/50)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Chemistry: 2.498 (#5/8)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Overall: 2.172 (#8/8)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Physics: 0.983 (#8/8)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Astronomy: 2.223 (#8/8)&lt;/li&gt;&lt;li&gt;STEMBench120 v1.0.0 (Closed Book) - Engineering: 2.172 (#8/8)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - API Security: 77.8 (#9/18)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - API Security: 76.5 (#13/20)&lt;/li&gt;&lt;li&gt;...and 12 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3&lt;/strong&gt; — Aggregate Rating 1835, #40&lt;ul&gt;&lt;li&gt;HiL-Bench: 58.33 (#8/28)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 59.03 (#15/220)&lt;/li&gt;&lt;li&gt;Last Translation Benchmark: 8.562019758507136 (#19/50)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent: 38.75 (#24/50)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 73.216004317629 (#25/208)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@1: 55.25 (#29/50)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 71.25 (#31/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Terra&lt;/strong&gt; — Aggregate Rating 1834, #41&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 54.98 (#52/220)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 36.70263882537 (#142/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 5 Preview&lt;/strong&gt; — Aggregate Rating 1833, #43&lt;ul&gt;&lt;li&gt;Vals AI ReverseEngBench: 0.38 (#31/35)&lt;/li&gt;&lt;li&gt;SvelteBench: 92.2 (#84/240)&lt;/li&gt;&lt;li&gt;SpeechMap Compliance: 59.7 (#228/411)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5 Pro (2025-10-06)&lt;/strong&gt; — Aggregate Rating 1826, #45&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 48.0 (#20/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.1&lt;/strong&gt; — Aggregate Rating 1824, #48&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 34.0 (#59/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.5&lt;/strong&gt; — Aggregate Rating 1823, #49&lt;ul&gt;&lt;li&gt;Kaggle WWTP Engineering: 0.0 (#36/37)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 54.98 (#51/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.6 Flash&lt;/strong&gt; — Aggregate Rating 1823, #51&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 53.36 (#77/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Luna&lt;/strong&gt; — Aggregate Rating 1820, #52&lt;ul&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 85.0 (#1/50)&lt;/li&gt;&lt;li&gt;KNOWS - Accessibility Tree + Screenshot: 48.0 (#3/4)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 43.4 (#6/20)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@1: 65.75 (#7/50)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Overall: 60.5 (#9/20)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 71.3 (#9/20)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Cloud Security: 52.2 (#11/20)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - API Security: 75.3 (#16/20)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent: 43.75 (#21/50)&lt;/li&gt;&lt;li&gt;AutomationBench: 20.7 (#27/109)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen3.8 2.4T A95B&lt;/strong&gt; — Aggregate Rating 1814, #54&lt;ul&gt;&lt;li&gt;DuelLab Overall: 68.574824916157 (#31/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 5&lt;/strong&gt; — Aggregate Rating 1813, #55&lt;ul&gt;&lt;li&gt;BBQ Disambiguated Accuracy (Mythos 5.1 System Card): 72.36 (#6/6)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark Open&lt;/strong&gt; — Aggregate Rating 1812, #56&lt;ul&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - Coding: 100.0 (#1/143)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - STEM: 100.0 (#1/143)&lt;/li&gt;&lt;li&gt;Nejumi 4 - MT-Bench (Japanese) - Writing: 99.5 (#1/143)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 87.81 (#2/224)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Logical Reasoning: 97.0 (#3/138)&lt;/li&gt;&lt;li&gt;Vals AI Finance Agent v2: 61.34 (#3/79)&lt;/li&gt;&lt;li&gt;APEX v1: 73.9 (#4/7)&lt;/li&gt;&lt;li&gt;Nejumi 4 - GLP - Coding: 71.21 (#4/138)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - NIILC: 61.58 (#4/143)&lt;/li&gt;&lt;li&gt;Nejumi 4 - HLE (Japanese) - Calibration Error: 41.0 (#4/143)&lt;/li&gt;&lt;li&gt;...and 182 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.1 Pro (Preview)&lt;/strong&gt; — Aggregate Rating 1810, #57&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 90.0 (#31/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 3.5 Flash&lt;/strong&gt; — Aggregate Rating 1809, #58&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 53.94 (#68/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Spark 1.2&lt;/strong&gt; — Aggregate Rating 1807, #61&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 57.41 (#28/220)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 59.63986214168 (#58/208)&lt;/li&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 27.0 (#75/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.6&lt;/strong&gt; — Aggregate Rating 1806, #62&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 86.6 (#61/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.3 Flash&lt;/strong&gt; — Aggregate Rating 1801, #63&lt;ul&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - API Security: 78.4 (#8/18)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 78.75 (#8/50)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - API Security: 77.2 (#11/20)&lt;/li&gt;&lt;li&gt;Vals AI Terminal-Bench 4.0: 37.37 (#14/49)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Websec.fr CTFs: 35.6 (#16/18)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Cloud Security: 40.6 (#16/18)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Overall: 41.8 (#17/18)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities: 24.1 (#17/20)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (Claude Code) - Cloud Security: 42.0 (#17/20)&lt;/li&gt;&lt;li&gt;Wiz Cyber Model Arena (ADK React) - Code Vulnerabilities: 12.4 (#18/18)&lt;/li&gt;&lt;li&gt;...and 12 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.6-Flash&lt;/strong&gt; — Aggregate Rating 1800, #65&lt;ul&gt;&lt;li&gt;Wolfram LLM Benchmarking Project: 50.4 (#214/598)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.6 Luna&lt;/strong&gt; — Aggregate Rating 1798, #68&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 53.59 (#72/220)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 26.223645310319 (#189/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.3 Codex&lt;/strong&gt; — Aggregate Rating 1796, #70&lt;ul&gt;&lt;li&gt;HiL-Bench: 4.33 (#28/28)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GLM-5.2&lt;/strong&gt; — Aggregate Rating 1790, #71&lt;ul&gt;&lt;li&gt;CyberGym: 90.84 (#5/52)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 51.16 (#98/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Ling 3.1 Flash&lt;/strong&gt; — Aggregate Rating 1790, #72&lt;ul&gt;&lt;li&gt;OpenRouter Tau2-Bench Airline: 79.3 (#2/147)&lt;/li&gt;&lt;li&gt;LLM2014 Logic 2026-10: 28.67 (#28/45)&lt;/li&gt;&lt;li&gt;Wolfram LLM Benchmarking Project: 64.9 (#66/598)&lt;/li&gt;&lt;li&gt;OpenRouter GPQA Diamond: 81.3 (#91/163)&lt;/li&gt;&lt;li&gt;SpeechMap Compliance: 57.4 (#242/411)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 4.5 (20251101)&lt;/strong&gt; — Aggregate Rating 1787, #73&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 14.0 (#114/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash (0731)&lt;/strong&gt; — Aggregate Rating 1784, #76&lt;ul&gt;&lt;li&gt;CyberGym: 93.17 (#1/52)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4&lt;/strong&gt; — Aggregate Rating 1782, #79&lt;ul&gt;&lt;li&gt;CyberGym: 79.0 (#17/52)&lt;/li&gt;&lt;li&gt;HiL-Bench: 9.67 (#26/28)&lt;/li&gt;&lt;li&gt;OmniDocBench 1.5: 85.5 (#69/99)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 47.949879446397 (#99/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.7 Plus&lt;/strong&gt; — Aggregate Rating 1769, #89&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 91.4 (#15/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.2 Codex&lt;/strong&gt; — Aggregate Rating 1764, #93&lt;ul&gt;&lt;li&gt;BenchGecko Score: 80.7 (#20/20)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5&lt;/strong&gt; — Aggregate Rating 1763, #95&lt;ul&gt;&lt;li&gt;CyberGym: 60.2 (#26/52)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 41.830431578697 (#125/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Beam 1.0&lt;/strong&gt; — Aggregate Rating 1763, #97&lt;ul&gt;&lt;li&gt;AutomationBench-AA - Support - Raw Objective Completion: 84.2 (#68/225)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 78.29 (#72/224)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Finance - Raw Objective Completion: 78.83 (#73/225)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 67.37 (#76/224)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Marketing - Raw Objective Completion: 77.7 (#76/225)&lt;/li&gt;&lt;li&gt;AutomationBench-AA: 48.46 (#80/225)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Operations - Raw Objective Completion: 85.03 (#80/225)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Strict: 17.05 (#81/225)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - Sales - Raw Objective Completion: 66.34 (#81/225)&lt;/li&gt;&lt;li&gt;AutomationBench-AA - HR - Raw Objective Completion: 65.55 (#84/225)&lt;/li&gt;&lt;li&gt;...and 47 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Large 4&lt;/strong&gt; — Aggregate Rating 1763, #99&lt;ul&gt;&lt;li&gt;AI Coding Daily (OpenCode) - React-TS Code Quality: 19.33 (#1/21)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Laravel Code Quality: 18.13 (#2/21)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Total: 56.86 (#3/21)&lt;/li&gt;&lt;li&gt;SecIT Bench (Pydantic AI): 76.38 (#10/27)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE August: 11.31 (#16/16)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV August: 24.56 (#16/16)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV_FALSE June: 22.22 (#20/30)&lt;/li&gt;&lt;li&gt;MathArena - ARXIV June: 65.97 (#20/31)&lt;/li&gt;&lt;li&gt;OTIS Mock AIME 2024-25: 98.33 (#24/304)&lt;/li&gt;&lt;li&gt;Agent Arena - Bash Recovery: 1.78 (#26/51)&lt;/li&gt;&lt;li&gt;...and 23 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.8 27B&lt;/strong&gt; — Aggregate Rating 1762, #100&lt;ul&gt;&lt;li&gt;SWE-bench Live Go (138 tasks, AMI Agent): 29.71 (#6/6)&lt;/li&gt;&lt;li&gt;OmniDocBench 1.5: 91.1 (#18/99)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 40.694320328634 (#131/208)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 46.64 (#133/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Pro&lt;/strong&gt; — Aggregate Rating 1761, #102&lt;ul&gt;&lt;li&gt;HiL-Bench: 38.46 (#17/28)&lt;/li&gt;&lt;li&gt;HLE-Diamond - Reasoning: 20.4 (#27/28)&lt;/li&gt;&lt;li&gt;HLE-Diamond - Knowledge: 12.8 (#27/28)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.6&lt;/strong&gt; — Aggregate Rating 1758, #105&lt;ul&gt;&lt;li&gt;HiL-Bench: 18.67 (#25/28)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;K2 Horizon 375B A23B&lt;/strong&gt; — Aggregate Rating 1756, #108&lt;ul&gt;&lt;li&gt;MLCR-AA - Conciseness: 98.89 (#8/96)&lt;/li&gt;&lt;li&gt;MLCR-AA - Accuracy: 67.98 (#57/96)&lt;/li&gt;&lt;li&gt;MLCR-AA: 6.67 (#75/96)&lt;/li&gt;&lt;li&gt;MLCR-AA - Completeness: 10.11 (#81/96)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 Plus&lt;/strong&gt; — Aggregate Rating 1750, #118&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 91.2 (#16/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3&lt;/strong&gt; — Aggregate Rating 1743, #123&lt;ul&gt;&lt;li&gt;DuelLab Overall: 42.445279604729 (#123/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Inkling&lt;/strong&gt; — Aggregate Rating 1742, #124&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 46.99 (#129/220)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 19.0345386711 (#203/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash&lt;/strong&gt; — Aggregate Rating 1735, #134&lt;ul&gt;&lt;li&gt;HiL-Bench: 47.81 (#11/28)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M3&lt;/strong&gt; — Aggregate Rating 1734, #137&lt;ul&gt;&lt;li&gt;HiL-Bench: 38.26 (#19/28)&lt;/li&gt;&lt;li&gt;Last Translation Benchmark: 6.256860592755214 (#22/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Sonnet 4.5&lt;/strong&gt; — Aggregate Rating 1732, #142&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 10.0 (#123/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kimi K2.5&lt;/strong&gt; — Aggregate Rating 1732, #143&lt;ul&gt;&lt;li&gt;CyberGym: 41.3 (#32/52)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Super VL&lt;/strong&gt; — Aggregate Rating 1727, #152&lt;ul&gt;&lt;li&gt;The Aggregate - Puzzle Portfolio Ladder: 92.6 (#5/20)&lt;/li&gt;&lt;li&gt;The Aggregate - Inversion and Execution Ladder: 91.7 (#6/18)&lt;/li&gt;&lt;li&gt;The Aggregate - Day-one Recall Check: 29.4 (#15/34)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - React-TS Code Quality: 15.17 (#19/21)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Laravel Code Quality: 15.01 (#20/21)&lt;/li&gt;&lt;li&gt;AI Coding Daily (OpenCode) - Total: 38.61 (#21/21)&lt;/li&gt;&lt;li&gt;OpenRouter Tau2-Bench Airline: 75.3 (#38/147)&lt;/li&gt;&lt;li&gt;OpenRouter GPQA Diamond: 88.0 (#45/163)&lt;/li&gt;&lt;li&gt;AA-Omniscience Hallucination Rate: 35.85 (#68/578)&lt;/li&gt;&lt;li&gt;AI Chess Leaderboard (Reasoning): 1066.0 (#81/344)&lt;/li&gt;&lt;li&gt;...and 64 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Open2 250B&lt;/strong&gt; — Aggregate Rating 1721, #158&lt;ul&gt;&lt;li&gt;AA GDPval: 970.77 (#172/296)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3.6 27B&lt;/strong&gt; — Aggregate Rating 1712, #168&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 89.4 (#34/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.20&lt;/strong&gt; — Aggregate Rating 1710, #173&lt;ul&gt;&lt;li&gt;HiL-Bench: 20.0 (#24/28)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Glimmer 30B&lt;/strong&gt; — Aggregate Rating 1706, #179&lt;ul&gt;&lt;li&gt;AA-AnalystAgent: 20.0 (#37/50)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@1: 37.25 (#42/50)&lt;/li&gt;&lt;li&gt;AA-AnalystAgent - Pass@K: 58.75 (#42/50)&lt;/li&gt;&lt;li&gt;OmniDocBench 1.5: 75.8 (#84/99)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Exactness: 51.96 (#105/387)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Success Rate: 100.0 (#186/394)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 90.0 (#267/394)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Grok 4.3&lt;/strong&gt; — Aggregate Rating 1703, #188&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 48.26 (#121/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiMo-V2.5&lt;/strong&gt; — Aggregate Rating 1700, #193&lt;ul&gt;&lt;li&gt;AA GDP.pdf - Engineering - Criterion Pass Rate: 40.25 (#135/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf: 4.0 (#171/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 52.86 (#176/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - STEM/Research - Criterion Pass Rate: 44.67 (#191/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 44.72 (#192/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 34.68 (#195/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 38.18 (#202/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Criterion Pass Rate: 37.42 (#204/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 28.92 (#210/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 28.02 (#215/224)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 31B&lt;/strong&gt; — Aggregate Rating 1697, #199&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 0.13 (#99/99)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 38.266838188441 (#139/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Mini&lt;/strong&gt; — Aggregate Rating 1692, #213&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 52.08 (#87/220)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 30.412700420227 (#174/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Pro 4&lt;/strong&gt; — Aggregate Rating 1692, #215&lt;ul&gt;&lt;li&gt;AA-Briefcase - XLSX - Rubric Pass Rate: 33.17 (#94/231)&lt;/li&gt;&lt;li&gt;AA-Briefcase - PPTX - Rubric Pass Rate: 31.22 (#96/231)&lt;/li&gt;&lt;li&gt;AA-Briefcase - Analytical Quality Elo: 1160.71 (#97/231)&lt;/li&gt;&lt;li&gt;AA-Briefcase: 33.55 (#98/231)&lt;/li&gt;&lt;li&gt;AA-Briefcase - PDF - Rubric Pass Rate: 33.62 (#105/231)&lt;/li&gt;&lt;li&gt;AA-Briefcase - DOCX - Rubric Pass Rate: 32.54 (#107/231)&lt;/li&gt;&lt;li&gt;AA-Briefcase - Presentation Elo: 1010.17 (#123/231)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 96.0 (#141/394)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Exactness: 47.48 (#218/387)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Success Rate: 98.0 (#263/394)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Solar Mini 4&lt;/strong&gt; — Aggregate Rating 1688, #224&lt;ul&gt;&lt;li&gt;Design Arena (Game Dev): 1138.0 (#109/180)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Step 3.7 Flash&lt;/strong&gt; — Aggregate Rating 1683, #237&lt;ul&gt;&lt;li&gt;Gert Labs Rankings: 23.67 (#100/100)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 30.675572146764 (#171/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4.1&lt;/strong&gt; — Aggregate Rating 1679, #245&lt;ul&gt;&lt;li&gt;DuelLab Overall: 41.868615462324 (#124/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;llm-jp-4.1-33B&lt;/strong&gt; — Aggregate Rating 1678, #251&lt;ul&gt;&lt;li&gt;Open Japanese LLM v2 - JBLiMP: 85.8 (#1/12)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - M-IFEval-Ja: 53.49 (#2/26)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JCulture-MCQ: 55.15 (#2/10)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JCoLA In-Domain: 78.73 (#3/12)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - JSICK: 85.0 (#3/143)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JamC-QA: 55.61 (#4/24)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - StructEval: 80.27 (#4/12)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JCoLA Out-of-Domain: 79.51 (#5/12)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - CommonsenseMoralJA: 97.0 (#7/143)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - MAWPS: 98.0 (#7/143)&lt;/li&gt;&lt;li&gt;...and 64 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Haiku 4.5 (20251001)&lt;/strong&gt; — Aggregate Rating 1674, #259&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 8.0 (#128/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Nano (2026-03-17)&lt;/strong&gt; — Aggregate Rating 1670, #266&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 45.0 (#28/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;O3 Mini&lt;/strong&gt; — Aggregate Rating 1669, #270&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 42.82 (#151/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 235B A22B 2507 Instruct&lt;/strong&gt; — Aggregate Rating 1656, #298&lt;ul&gt;&lt;li&gt;ContextEcho: 40.0 (#11/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MiniMax-M2.5&lt;/strong&gt; — Aggregate Rating 1654, #300&lt;ul&gt;&lt;li&gt;HiL-Bench: 6.33 (#27/28)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-5.4 Nano&lt;/strong&gt; — Aggregate Rating 1639, #329&lt;ul&gt;&lt;li&gt;Epoch AI - Scicode: 47.22 (#126/220)&lt;/li&gt;&lt;li&gt;DuelLab Overall: 27.172941650905 (#185/208)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Hermes 4 405B&lt;/strong&gt; — Aggregate Rating 1639, #330&lt;ul&gt;&lt;li&gt;EQ-Bench Creative Writing v3: 1270.7 (#44/106)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-OSS-120B&lt;/strong&gt; — Aggregate Rating 1636, #337&lt;ul&gt;&lt;li&gt;Last Translation Benchmark: 2.305159165751921 (#30/50)&lt;/li&gt;&lt;li&gt;Epoch AI - Scicode: 34.03 (#196/220)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 26B A4B&lt;/strong&gt; — Aggregate Rating 1631, #351&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 0.15 (#98/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;llm-jp-4.1-32B-a3B&lt;/strong&gt; — Aggregate Rating 1629, #355&lt;ul&gt;&lt;li&gt;Open Japanese LLM v2 - M-IFEval-Ja: 56.4 (#1/26)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - MAWPS: 99.0 (#1/143)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JCulture-MCQ: 52.24 (#3/10)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JFinQA: 65.6 (#3/11)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JCoLA Out-of-Domain: 81.35 (#3/12)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JCoLA In-Domain: 76.88 (#5/12)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JBLiMP: 79.15 (#8/12)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (0-shot) - MAWPS: 98.0 (#8/143)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JamC-QA: 52.75 (#9/24)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - StructEval: 77.68 (#9/12)&lt;/li&gt;&lt;li&gt;...and 64 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 Next 80B A3B Instruct&lt;/strong&gt; — Aggregate Rating 1627, #364&lt;ul&gt;&lt;li&gt;ContextEcho: 30.0 (#7/23)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;llm-jp-4.1-8B&lt;/strong&gt; — Aggregate Rating 1614, #399&lt;ul&gt;&lt;li&gt;Open Japanese LLM v2 - JCoLA Out-of-Domain: 82.87 (#1/12)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JCoLA In-Domain: 79.19 (#2/12)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - M-IFEval-Ja: 50.58 (#3/26)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JCulture-MCQ: 48.7 (#5/10)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - StructEval: 79.03 (#5/12)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JBLiMP: 82.48 (#6/12)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JFinQA: 60.9 (#7/11)&lt;/li&gt;&lt;li&gt;Nejumi 4 - BFCL - Irrelevance Detection: 93.33 (#9/143)&lt;/li&gt;&lt;li&gt;Nejumi 4 - jaster (2-shot) - MAWPS: 98.0 (#13/143)&lt;/li&gt;&lt;li&gt;Open Japanese LLM v2 - JamC-QA: 48.25 (#15/24)&lt;/li&gt;&lt;li&gt;...and 64 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mercury 2&lt;/strong&gt; — Aggregate Rating 1602, #434&lt;ul&gt;&lt;li&gt;AA GDP.pdf: 3.8 (#176/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 52.67 (#177/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - STEM/Research - Criterion Pass Rate: 48.34 (#177/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 49.2 (#180/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 47.28 (#187/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Real Estate - Criterion Pass Rate: 44.92 (#187/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Criterion Pass Rate: 43.14 (#188/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Construction - Criterion Pass Rate: 46.17 (#189/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 45.76 (#189/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 42.45 (#195/224)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 32B&lt;/strong&gt; — Aggregate Rating 1600, #442&lt;ul&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 49.98 (#178/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf: 1.6 (#196/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - STEM/Research - Criterion Pass Rate: 40.28 (#199/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 40.49 (#200/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 34.16 (#200/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 37.79 (#201/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 36.28 (#201/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Real Estate - Criterion Pass Rate: 40.76 (#202/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Criterion Pass Rate: 38.26 (#203/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 41.35 (#203/224)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 4 Maverick&lt;/strong&gt; — Aggregate Rating 1593, #465&lt;ul&gt;&lt;li&gt;AA GDP.pdf: 0.8 (#212/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Engineering - Criterion Pass Rate: 9.75 (#220/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Criterion Pass Rate: 3.97 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 0.0 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 3.86 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 0.0 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 1.67 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 0.0 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 0.92 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - STEM/Research - Criterion Pass Rate: 0.44 (#223/224)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Nemotron 3.5 Lightning&lt;/strong&gt; — Aggregate Rating 1585, #488&lt;ul&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Exactness: 49.35 (#177/387)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 94.0 (#186/394)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Success Rate: 98.0 (#253/394)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 4&lt;/strong&gt; — Aggregate Rating 1579, #516&lt;ul&gt;&lt;li&gt;Last Translation Benchmark: 1.3172338090010975 (#40/50)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 14B&lt;/strong&gt; — Aggregate Rating 1572, #535&lt;ul&gt;&lt;li&gt;AA GDP.pdf - Real Estate - Criterion Pass Rate: 41.42 (#200/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf: 1.2 (#202/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 37.53 (#202/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 38.0 (#202/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - STEM/Research - Criterion Pass Rate: 39.37 (#203/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Engineering - Criterion Pass Rate: 24.45 (#204/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 37.47 (#204/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Criterion Pass Rate: 35.63 (#209/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 28.87 (#210/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 30.59 (#210/224)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Mistral Small 3.2&lt;/strong&gt; — Aggregate Rating 1554, #602&lt;ul&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 21.62 (#216/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - STEM/Research - Criterion Pass Rate: 6.48 (#220/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Criterion Pass Rate: 12.61 (#221/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 18.87 (#221/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 5.78 (#221/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 7.98 (#221/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Real Estate - Criterion Pass Rate: 22.42 (#221/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf: 0.0 (#222/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Construction - Criterion Pass Rate: 26.85 (#222/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 8.48 (#222/224)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Llama 4 Scout&lt;/strong&gt; — Aggregate Rating 1536, #652&lt;ul&gt;&lt;li&gt;AA GDP.pdf - Engineering - Criterion Pass Rate: 8.89 (#221/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf: 0.0 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Construction - Criterion Pass Rate: 21.3 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Real Estate - Criterion Pass Rate: 2.5 (#223/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Criterion Pass Rate: 3.68 (#224/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 0.0 (#224/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 1.82 (#224/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 0.0 (#224/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Insurance - Criterion Pass Rate: 1.39 (#224/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 0.0 (#224/224)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Qwen 3 8B&lt;/strong&gt; — Aggregate Rating 1532, #671&lt;ul&gt;&lt;li&gt;AA GDP.pdf: 1.4 (#198/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 30.58 (#208/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - STEM/Research - Criterion Pass Rate: 38.14 (#208/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Healthcare - Criterion Pass Rate: 29.6 (#211/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Legal - Criterion Pass Rate: 24.64 (#211/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Real Estate - Criterion Pass Rate: 34.02 (#213/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Criterion Pass Rate: 30.37 (#214/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Engineering - Criterion Pass Rate: 21.97 (#214/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 28.81 (#214/224)&lt;/li&gt;&lt;li&gt;AA GDP.pdf - HR - Criterion Pass Rate: 24.41 (#215/224)&lt;/li&gt;&lt;li&gt;...and 2 more&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-4.1 Nano&lt;/strong&gt; — Aggregate Rating 1515, #731&lt;ul&gt;&lt;li&gt;Chess Puzzles (Epoch AI): 0.0 (#152/153)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 E4B&lt;/strong&gt; — Aggregate Rating 1504, #766&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 0.18 (#96/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LFM2.5-2.6B&lt;/strong&gt; — Aggregate Rating 1492, #804&lt;ul&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Success Rate: 90.0 (#342/394)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - First-Attempt Success Rate: 60.0 (#361/394)&lt;/li&gt;&lt;li&gt;Tinybird AI SQL Benchmark - Exactness: 9.34 (#374/387)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 3 4B&lt;/strong&gt; — Aggregate Rating 1432, #1082&lt;ul&gt;&lt;li&gt;OpenRouter GPQA Diamond: 27.1 (#161/163)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemma 4 E2B&lt;/strong&gt; — Aggregate Rating 1428, #1103&lt;ul&gt;&lt;li&gt;OmniDocBench 1.5: 0.29 (#95/99)&lt;/li&gt;&lt;/ul&gt;&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;Top-10 New Scores (373)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on Anthropic Election Integrity Multi-Turn - Appropriate Response Rate (API): 90.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Fable 5.1&lt;/strong&gt; on BBQ Disambiguated Accuracy (Mythos 5.1 System Card): 89.92 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Agent Arena: 15.53 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Agent Arena - Bash Recovery: 15.01 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Agent Arena - Confirmed Success: 17.29 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Agent Arena - Praise vs Complaint: 36.78 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Agent Arena - Steerability: 8.2 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Agent Arena - Tool Hallucination: -0.37 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Child Safety Multi-Turn - Appropriate Response Rate (API): 84.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Child Safety Multi-Turn - Appropriate Response Rate (Claude.ai): 99.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Claude Code Malicious Requests - Refusal Rate (Mythos 5.1 System Card): 79.8 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Disordered Eating Single-Turn Risk Requests - Harmless Rate (API): 95.64 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Disordered Eating Single-Turn Risk Requests - Harmless Rate (Claude.ai): 99.26 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Election Integrity Multi-Turn - Appropriate Response Rate (API): 87.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Election Integrity Multi-Turn - Appropriate Response Rate (Claude.ai): 85.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Malicious Computer Use - Refusal Rate: 79.46 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Single-Turn Benign Requests - Over-Refusal Rate (API): 0.03 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Single-Turn Benign Requests - Over-Refusal Rate (Claude.ai): 0.38 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Single-Turn Harmful Requests - Harmless Rate (API): 94.5 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Single-Turn Harmful Requests - Harmless Rate (Claude.ai): 99.51 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Suicide and Self-Harm Multi-Turn - Appropriate Response Rate (API): 66.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Suicide and Self-Harm Multi-Turn - Appropriate Response Rate (Claude.ai): 94.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Anthropic Suicide and Self-Harm Single-Turn Risk Requests - Harmless Rate (API): 99.16 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on BBQ Disambiguated Accuracy (Mythos 5.1 System Card): 89.65 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Chess Puzzles (Epoch AI): 67.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Conceptual Reasoning Index: 76.07 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Conceptual Reasoning Index - Argument Evaluation (LMCA): 66.18 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Conceptual Reasoning Index - Consistency (ACCoRD): 88.42 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Conceptual Reasoning Index - Decision Theory (DTBench): 95.55 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on DRACO: 87.4 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on HiL-Bench: 59.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ParseBench: 79.85 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ParseBench - Charts: 70.89 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ParseBench - Content Faithfulness: 91.72 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ParseBench - Semantic Formatting: 77.04 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ParseBench - Tables: 94.25 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on ParseBench - Visual Grounding: 65.33 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Astronomy: 2.759 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Biology: 1.865 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Chemistry: 2.666 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Overall: 2.621 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on SWE-bench Pro (Opus 5 &amp; Mythos 5.1 System Cards): 89.9 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on SciLaws-Bench: 48.87 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Toolathlon Verified Pass@3 (Anthropic Internal Harness): 82.4 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Claude Opus 5.5&lt;/strong&gt; on Toolathlon Verified Pass^3 (Anthropic Internal Harness): 72.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on AutomationBench: 41.4 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Design Arena (3D): 1509.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on Epoch AI - Scicode: 56.48 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on HiL-Bench: 65.32 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on STEMBench120 v1.0.0 (Tools) - Astronomy: 2.723 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on STEMBench120 v1.0.0 (Tools) - Biology: 2.709 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on STEMBench120 v1.0.0 (Tools) - Chemistry: 2.777 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on STEMBench120 v1.0.0 (Tools) - Earth science: 2.726 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on STEMBench120 v1.0.0 (Tools) - Engineering: 2.894 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on STEMBench120 v1.0.0 (Tools) - Overall: 2.809 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra&lt;/strong&gt; on STEMBench120 v1.0.0 (Tools) - Physics: 3.025 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6 Astra Pro&lt;/strong&gt; on Chess Puzzles (Epoch AI): 76.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Conceptual Reasoning Index: 72.68 (#22)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Conceptual Reasoning Index - Argument Evaluation (LMCA): 62.78 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Conceptual Reasoning Index - Consistency (ACCoRD): 80.87 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Conceptual Reasoning Index - Decision Theory (DTBench): 94.22 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (3D): 1458.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Design Arena (UI Components): 1325.0 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Epoch AI - Scicode: 55.79 (#43)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on GDP.pdf: 35.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on HLE-Diamond (With Tools): 69.1 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on HiL-Bench: 62.33 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle FACTS (Google): 75.29 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle FACTS Grounding: 82.42 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle FACTS Multimodal: 48.46 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle FACTS Parametric: 82.11 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Arabic: 69.5 (#40)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Bengali: 70.0 (#40)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Burmese: 74.5 (#38)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Chinese: 82.75 (#34)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Culturally Agnostic: 78.31 (#36)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Culturally Sensitive: 79.47 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - English: 91.0 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - French: 76.5 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - German: 85.25 (#31)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Hindi: 65.0 (#40)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Indonesian: 77.0 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Italian: 81.75 (#34)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Japanese: 87.5 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Korean: 90.25 (#23)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Portuguese: 88.0 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Spanish: 82.0 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Swahili: 77.0 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Yoruba: 64.25 (#43)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GPT-6.1 Sol&lt;/strong&gt; on Kaggle Open Benchmarks - SciCode Subproblems: 37.8 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AA-AnalystAgent: 48.75 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AA-AnalystAgent - Pass@1: 57.75 (#25)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AA-AnalystAgent - Pass@K: 66.25 (#40)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI Chess Leaderboard (Continuation): 1641.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AI Chess Leaderboard (Reasoning): 1632.0 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AIMultiple - FinanceReasoning (Hard): 90.76 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AIMultiple - Text-to-SQL (Adjudicated): 81.4 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on AIMultiple - Text-to-SQL (Strict Execution Match): 64.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ARC-AGI-1: 98.5 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ARC-AGI-2: 91.25 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ARC-AGI-3: 49.79 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Arabic Broad Leaderboard: 9.128 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Arabic Broad Leaderboard - General Knowledge: 9.175 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Arabic Broad Leaderboard - MMLU: 9.752 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Arabic Broad Leaderboard - RAG QA: 8.61 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Arabic Broad Leaderboard - Reasoning &amp; Math: 9.465 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Arabic Broad Leaderboard - Translation (incl Dialects): 8.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Arabic Broad Leaderboard - Trust &amp; Safety: 9.0 (#43)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Arabic Broad Leaderboard - Writing (incl Dialects): 7.864 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Argo-Bench: 54.99 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Argo-Bench - Solved: 35.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on BenchGecko Score: 88.6 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on BridgeBench Overall: 623.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Bug Hunt Bench: 40.7 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Bug Hunt Bench - LMS: 27.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 13.7 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Chess Bench LLM: 2022.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Chess Puzzles (Epoch AI): 59.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ChessBench: 24.41 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ChessBench - Accuracy: 80.1 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ChessBench - Coherence: 85.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ChessBench - Elo: 1356.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ChessBench GitHub - Elo: 2430.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Conceptual Reasoning Index: 72.41 (#24)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Conceptual Reasoning Index - Argument Evaluation (LMCA): 63.69 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Conceptual Reasoning Index - Consistency (ACCoRD): 77.3 (#48)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Conceptual Reasoning Index - Decision Theory (DTBench): 96.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Creative Writing (Lechmazur): 0.8 (#19)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on CursorBench 4.0: 52.9 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Deep20Bench - Edition 1.1: 13.666666666667 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Design Arena (3D): 1392.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Design Arena (Data Viz): 1347.0 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Design Arena (Game Dev): 1370.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Design Arena (UI Components): 1338.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Design Arena (Website): 1314.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EKI Estonian LLM - Book Recommendations: 63.47 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EKI Estonian LLM - Idioms: 70.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EKI Estonian LLM - Overall: 72.48 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EKI Estonian LLM - Propaganda Resistance: 79.52 (#51)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EKI Estonian LLM - Terminology: 57.54 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EKI Estonian LLM - Trivia Hard: 50.62 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EnigmaForge: 81.2 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EnigmaForge - Fact F1: 98.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on EnigmaForge - Intuition: 72.9 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Cursorbench: 52.9 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - ECI: 164.56 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Ebr Bench: 64.76 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Frontiermath Tier 4 V2: 78.05 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Frontiermath Tiers 1 3 V2: 85.96 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Furniture Assembly: 72.5 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - GPQA Diamond: 96.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Epoch AI - Mystery Game Puzzles: 64.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ErdosBench: 3.04 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on FrontierMath - Tier 4 (v2): 78.05 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on FrontierMath - Tiers 1-3 (v2): 85.96 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Gert Labs Rankings: 69.63 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on HLE-Diamond: 56.9 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on HLE-Diamond - Knowledge: 52.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on HLE-Diamond - Reasoning: 61.8 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on HiL-Bench: 62.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Humanity's Sixth Sense: 55.4 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Icelandic LLM - Case Government: 98.75 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Icelandic LLM - Flora &amp; Fauna: 49.95 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Icelandic LLM - GEC: 80.73 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Icelandic LLM - Generative Quality: 74.91 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Icelandic LLM - Idiom Meanings: 81.67 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Icelandic LLM - Song Lyrics: 21.83 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Icelandic LLM - Textile Terms: 70.4 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Icelandic LLM - WikiQA: 71.26 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Icelandic LLM Leaderboard - Average: 70.52 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle ExtractBench: 89.81 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle FACTS Grounding: 78.63 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle FACTS Parametric: 71.55 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Arabic: 94.75 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Burmese: 94.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Chinese: 95.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Culturally Agnostic: 95.91 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Culturally Sensitive: 94.84 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - English: 96.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - French: 95.25 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - German: 96.75 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Hindi: 95.5 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Indonesian: 94.75 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Italian: 95.5 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Korean: 94.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Portuguese: 96.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Spanish: 96.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Global-MMLU-Lite - Yoruba: 93.5 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Open Benchmarks - AIME 2025: 100.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Open Benchmarks - MATH-500: 99.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle Open Benchmarks - SciCode Subproblems: 45.02 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle SimpleQA: 71.18 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle SimpleQA Verified: 75.6 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Kaggle VideoQA: 87.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on KernelBench Hub - CUDA: 42.2 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on KernelBench Hub - Mega: 27.03 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on LLM2014 Logic 2026-10: 69.35 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Last Translation Benchmark: 40.944017563117455 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MathArena - ARXIV August: 82.46 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MathArena - ARXIV_FALSE August: 54.17 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on MineBench: 2250.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Multi-turn Debate (Lechmazur): 1672.7 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on NYT Connections Extended: 98.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Next.js Agent Evals (OpenCode) - Success Rate with AGENTS.md: 97.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on NonoBench: 86.666666666667 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on OTIS Mock AIME 2024-25: 100.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on OpenRouter Tau2-Bench Airline: 78.7 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PLCC - Art &amp; Entertainment: 95.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PLCC - Culture &amp; Tradition: 98.0 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PLCC - Geography: 99.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PLCC - Grammar: 96.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PLCC - History: 98.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PUMA Polish Multimodal - Contemporary Life: 84.0 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PUMA Polish Multimodal - Geography and Environment: 89.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PUMA Polish Multimodal - History and Culture: 83.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on PUMA Polish Multimodal - Structured Extraction: 69.0 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Agents - BFCL-V3: 72.5 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Agents and Tool Use: 73.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Coding: 60.5 (#47)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Education - Gaokao: 68.7 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Education - Middle School Subjects: 58.0 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Education - Primary School Subjects: 71.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Finance: 90.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Law - Lawyer Qualification Exam: 77.3 (#47)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Law and Civil Service: 80.7 (#65)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Medicine - Basic Medical Knowledge: 88.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Medicine - Physician Exams: 88.3 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Medicine and Mental Health: 88.5 (#21)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Overall: 77.3 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Reasoning - BBH: 87.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on ReLE - Reasoning and Mathematics: 85.4 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on RuneBench: 6.906727309216 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SEAL - AudioMultiChallenge - Text Output: 58.41 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SEAL - Professional Reasoning Benchmark - Finance: 49.9 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SEAL - Professional Reasoning Benchmark - Legal: 49.41 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Biology: 2.348 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Chemistry: 2.336 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Earth science: 2.647 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Engineering: 2.471 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Overall: 2.444 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on STEMBench120 v1.0.0 (Closed Book) - Physics: 2.083 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SimpleQA Verified: 76.0 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SpeechMap Compliance: 87.0 (#55)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on SvelteBench: 100.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Tinybird AI SQL Benchmark - Exactness: 56.91 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Tinybird AI SQL Benchmark - First-Attempt Success Rate: 100.0 (#41)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Tinybird AI SQL Benchmark - Success Rate: 100.0 (#178)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on UGI - Natural Intelligence: 81.36 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on UGI - Willingness (W/10): 1.8 (#1247)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on UGI - Writing: 79.71 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on UGI Leaderboard: 49.92 (#150)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vals AI Vibe Code Bench 1-100: 21.2 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Vectara Hallucination Leaderboard: 87.9 (#83)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on VoxelBench: 2229.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wiz Cyber Model Arena (ADK React) - Websec.fr CTFs: 74.7 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - API Security: 82.7 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Overall: 70.8 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Gemini 4 Argon&lt;/strong&gt; on Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs: 66.7 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA CritPt: 32.86 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf: 32.6 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - Construction - Criterion Pass Rate: 75.97 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - Criterion Pass Rate: 80.35 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - Engineering - Criterion Pass Rate: 75.05 (#29)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - Finance/Investing - Criterion Pass Rate: 83.08 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - HR - Criterion Pass Rate: 72.48 (#73)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - Healthcare - Criterion Pass Rate: 89.84 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - Insurance - Criterion Pass Rate: 84.25 (#20)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - Legal - Criterion Pass Rate: 87.8 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - Manufacturing/Supply Chains - Criterion Pass Rate: 79.93 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - Real Estate - Criterion Pass Rate: 75.78 (#17)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDP.pdf - STEM/Research - Criterion Pass Rate: 80.46 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA GDPval: 1857.09 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA Humanity's Last Exam: 53.61 (#16)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA Long Context Reasoning: 83.67 (#15)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA Omniscience: 30.15 (#32)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA SciCode: 62.62 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA Terminal-Bench 4.0: 57.07 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Briefcase: 61.31 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Briefcase - Analytical Quality Elo: 2116.26 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Briefcase - DOCX - Rubric Pass Rate: 60.36 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Briefcase - PDF - Rubric Pass Rate: 67.24 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Briefcase - PPTX - Rubric Pass Rate: 57.14 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Briefcase - Presentation Elo: 1524.05 (#14)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Briefcase - XLSX - Rubric Pass Rate: 61.36 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Accuracy: 43.22 (#99)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Hallucination Rate: 23.01 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Business: 23.5 (#33)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Health: 16.4 (#28)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Humanities &amp; Social Sciences: 31.5 (#26)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Law: 24.8 (#54)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Science, Engineering &amp; Mathematics: 28.6 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE): 56.1 (#53)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - C: 79.0 (#41)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Dart: 36.0 (#71)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Go: 64.0 (#50)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - HTML: 74.0 (#30)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Java: 32.0 (#49)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - JavaScript: 74.55 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Julia: 24.0 (#90)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Kotlin: 46.0 (#59)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - PHP: 52.0 (#79)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Python: 48.5 (#85)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - R: 48.0 (#40)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Rust: 70.0 (#53)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - Swift: 68.0 (#34)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AA-Omniscience Index - Software Engineering (SWE) - TypeScript: 58.89 (#60)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Artificial Analysis Intelligence Index: 55.8 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AutomationBench-AA: 71.33 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AutomationBench-AA - Finance - Raw Objective Completion: 85.68 (#35)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AutomationBench-AA - HR - Raw Objective Completion: 83.32 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AutomationBench-AA - Marketing - Raw Objective Completion: 84.58 (#37)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AutomationBench-AA - Operations - Raw Objective Completion: 96.21 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AutomationBench-AA - Sales - Raw Objective Completion: 88.19 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AutomationBench-AA - Strict: 45.36 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on AutomationBench-AA - Support - Raw Objective Completion: 92.3 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on BenchLM: 74.3 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Bug Hunt Bench: 42.0 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Bug Hunt Bench - LMS: 25.5 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Bug Hunt Bench - VS Code Extension: 21.5 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on GDP.pdf: 33.2 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA: 13.06 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - All Pass: 25.56 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Antitrust &amp; Competition: 93.6 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Arbitration &amp; International Dispute Resolution: 96.06 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Banking &amp; Finance: 96.64 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Bankruptcy &amp; Restructuring: 94.46 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Capital Markets: 96.86 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Corporate Governance: 96.69 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Corporate M&amp;A: 98.18 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Data Privacy &amp; Cybersecurity: 97.88 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Emerging Companies &amp; Venture Capital: 95.79 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Employment &amp; Labor: 91.51 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Energy &amp; Natural Resources: 97.36 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Environmental &amp; ESG: 92.95 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Funds &amp; Asset Management: 95.72 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Healthcare &amp; Life Sciences: 95.19 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Immigration: 94.55 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Insurance: 94.12 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Intellectual Property: 97.86 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - International Trade &amp; Sanctions: 96.41 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Litigation &amp; Dispute Resolution: 94.86 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Real Estate: 98.06 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Structured Finance &amp; Securitization: 94.52 (#2)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Tax: 90.97 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - Trusts, Estates &amp; Private Client: 95.04 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Harvey LAB-AA - White Collar Defense &amp; Investigations: 95.74 (#1)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LLM Stats (Artificial Analysis): 56.0 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LLM Stats (GDP.pdf): 33.2 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LLM Stats (HealthBench Professional): 62.3 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LLM Stats (Program Bench): 59.0 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on LLM Stats Score: 52.06 (#13)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI (Vals Index): 64.57 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI BioMysteryBench: 73.33 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Code Migration: 61.03 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI CyberBench: 73.63 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Excel Modeling: 71.92 (#9)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Finance Agent v2: 59.06 (#6)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Harvey Legal Agent Bench: 21.67 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI IOI: 79.83 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Legal Research Bench: 53.37 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI LegalBench: 87.42 (#4)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI MedCode: 49.93 (#18)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI MedScribe: 89.33 (#8)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI MysteryMechanism: 35.13 (#11)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI ProgramBench: 0.5 (#27)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI ProofBench: 94.0 (#10)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Public Benefits Bench: 73.75 (#3)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI ReverseEngBench: 11.83 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Tax Agent Bench: 43.0 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Terminal-Bench 4.0: 58.08 (#5)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Terminal-Bench Science: 37.14 (#7)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vals AI Vibe Code Bench: 86.0 (#12)&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Muse Wonder Full&lt;/strong&gt; on Vending-Bench 2: 12291.558333333334 (#4)&lt;/li&gt;&lt;/ul&gt;
&lt;h3&gt;New #1 Leaders (117)&lt;/h3&gt;&lt;ul&gt;&lt;li&gt;&lt;strong&gt;Design Arena (3D)&lt;/strong&gt;: GPT-6 Astra (Max) (1509.0) beat Claude Opus 5.5 by 29.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;KNOWS - Accessibility Tree + Screenshot&lt;/strong&gt;: GPT-6 Astra (72.5) beat GPT-5.5 by 23.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HiL-Bench&lt;/strong&gt;: Muse Wonder (79.67) beat Claude Fable 5.1 by 18.17&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (ADK React) - Cloud Security&lt;/strong&gt;: Gemini 4 Argon (82.6) beat Gemini 3.8 Flash Cyber by 15.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;HieroglyphBench&lt;/strong&gt;: Gemini 4 Argon (69.7) beat Gemini 3.7 Flash by 14.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Lean AI Formalization Leaderboard&lt;/strong&gt;: Axiom Prover (Axiom Math) (263.0) beat Humanifa + GPT 5.6 sol by 12.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WDCD R3 Pressure Integrity&lt;/strong&gt;: Claude Opus 4.7 (150.0) beat Grok 4 by 12.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;WebArena&lt;/strong&gt;: Narada AI Code (86.1) beat DeepSeek V3.2 by 11.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (SWE-Marathon)&lt;/strong&gt;: Muse Wonder Full (57.5) beat Grok 4.7 by 11.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Claude Code) - Cloud Security&lt;/strong&gt;: Gemini 4 Argon (73.9) beat Claude Opus 5 by 10.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (ADK React) - Code Vulnerabilities&lt;/strong&gt;: Gemini 4 Argon (65.8) beat Gemini 3.8 Flash Cyber by 8.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Open Japanese LLM v2 - M-IFEval-Ja&lt;/strong&gt;: llm-jp-4.1-32B-a3B (Thinking) (56.4) beat llm-jp-4-kappa-32B-a3B-v0.1 (Thinking) by 8.73&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWE-bench Pro (Opus 5 &amp; Mythos 5.1 System Cards)&lt;/strong&gt;: Claude Opus 5.5 (Max) (89.9) beat Claude Fable 5.1 (Max) by 8.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BenchX - Pass Rate&lt;/strong&gt;: gemma-4-31B-it · T=1.0 (90.14) beat GPT-5.6 Sol by 7.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Claude Code) - Websec.fr CTFs&lt;/strong&gt;: GPT-6 Sol (88.5) beat Claude Opus 5 by 6.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;TaxCalcBench&lt;/strong&gt;: Meta Muse Wonder w/ Web Search (49/50) (71.43) beat GPT-6 Sol Web Search by 5.43&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Famous Phrases&lt;/strong&gt;: Gemini 4 Argon (36.19) beat Gemini 3.8 Flash by 5.31&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Trivia&lt;/strong&gt;: Gemini 4 Argon (90.0) beat Gemini 3.1 Pro (Preview) by 5.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (Claude Code) - Code Vulnerabilities&lt;/strong&gt;: Gemini 4 Argon (59.9) beat Claude Opus 5 by 4.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Senior SWE-Bench&lt;/strong&gt;: Claude Opus 5.5 (Mini-SWE-Agent, High) (39.325842696629216) beat GPT-5.6 Sol (Mini-SWE-Agent, xHigh) by 4.59&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PinchBench - 147 Tasks - OpenClaw 2026.6.5 (5181e4f)&lt;/strong&gt;: nemotron-3.5-bolt-74B-a9B-bf16 (91.68252517006803) beat Nemotron Lightning 3.5 30B A3B by 4.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Chess Puzzles (Epoch AI)&lt;/strong&gt;: GPT-6 Astra Pro (Max) (76.0) beat GPT-6 Astra (Max) by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Document QA&lt;/strong&gt;: Gemini 4 Argon (93.0) beat Claude Opus 5.5 by 4.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AIMultiple - Agentic RAG Routing&lt;/strong&gt;: Gemini 4 Argon (94.02) beat Claude Opus 5.5 by 3.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SEAL - AudioMultiChallenge - Text Output&lt;/strong&gt;: Gemini 3.8 Flash (High) (60.4) beat Inkling (Thinking) by 3.76&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-AnalystAgent - Pass@K&lt;/strong&gt;: GPT-6 Luna (Max) (85.0) beat Gemini 3.1 Pro (Preview) by 3.75&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI Vibe Code Bench 1-100&lt;/strong&gt;: Muse Wonder Full (33.98) beat Claude Opus 5.5 by 3.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA&lt;/strong&gt;: Muse Wonder Full (Max) (13.06) beat Grok 4.7 (xHigh) by 3.62&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA-Briefcase - DOCX - Rubric Pass Rate&lt;/strong&gt;: Muse Wonder (Max) (65.68) beat Gemini 4 Argon (High) by 3.55&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Funds &amp; Asset Management&lt;/strong&gt;: Muse Wonder Full (Max) (95.72) beat Muse Spark 1.3 (Max) by 3.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench - WAPE&lt;/strong&gt;: Timestral-209M (88.9) beat timesfm-3 by 3.19&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Next.js Agent Evals (OpenCode) - Success Rate&lt;/strong&gt;: Gemini 4 Argon (97.0) beat Grok 4.7 by 3.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench - MASE&lt;/strong&gt;: Timestral-209M (88.84) beat timesfm-3 by 2.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Language and Instruction Following&lt;/strong&gt;: Gemini 4 Argon (81.6) beat Hy4 preview by 2.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench&lt;/strong&gt;: Timestral-209M (89.59) beat timesfm-3 by 2.75&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench - SQL&lt;/strong&gt;: Timestral-209M (90.84) beat timesfm-3 by 2.63&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Vals AI CyberBench&lt;/strong&gt;: Muse Wonder (80.36) beat GPT-6 Sol by 2.38&lt;/li&gt;&lt;li&gt;&lt;strong&gt;FEV-Bench - WQL&lt;/strong&gt;: Timestral-209M (89.77) beat timesfm-3 by 2.31&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (ADK React) - Overall&lt;/strong&gt;: Gemini 4 Argon (77.2) beat Gemini 3.8 Flash Cyber by 2.3&lt;/li&gt;&lt;li&gt;&lt;strong&gt;BALSAM - Summarization&lt;/strong&gt;: Nojoom-AQL-120B (58.82) beat Nojoom-AQL-42B by 2.24&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Immigration&lt;/strong&gt;: Muse Wonder (Max) (95.39) beat Grok 4.7 (xHigh) by 2.18&lt;/li&gt;&lt;li&gt;&lt;strong&gt;DuelLab Overall&lt;/strong&gt;: Claude Opus 5.5 (xHigh) (95.16780469798) beat GPT-6 Astra (xHigh) by 2.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLCC - Vocabulary&lt;/strong&gt;: Gemini 4 Argon (98.0) beat Gemini 3.1 Pro (Preview) by 2.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SloBench - Slovene SuperGLUE&lt;/strong&gt;: Gizela-31B-it-vit-think (79.87) beat Gemma 3 27B (IT) by 1.93&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SWE-Marathon v1.1&lt;/strong&gt;: Muse Wonder Full (Thinking, max) (Muse Code) (53.125) beat Claude Sonnet 5.5 (Thinking, Max) (Claude Code) by 1.88&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Open Japanese LLM v2 - JBLiMP&lt;/strong&gt;: llm-jp-4.1-33B (Thinking) (85.8) beat llm-jp-4-8B-Thinking by 1.81&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Humanity's Sixth Sense&lt;/strong&gt;: Gemini 4 Argon (High) (55.4) beat GPT-6 Astra (Max) by 1.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Coding&lt;/strong&gt;: Claude Sonnet 5.5 (78.0) beat Claude Opus 5.5 by 1.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Open Japanese LLM v2 - JCoLA Out-of-Domain&lt;/strong&gt;: llm-jp-4.1-8B (Thinking) (82.87) beat llm-jp-4-8B-Thinking by 1.52&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Open Benchmarks - MMLU-Pro&lt;/strong&gt;: Gemini 4 Argon (93.19) beat Claude Opus 5 by 1.51&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EKI Estonian LLM - Language Advice&lt;/strong&gt;: Gemini 4 Argon (77.5) beat GPT-6 Astra by 1.46&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AutomationBench-AA - HR - Raw Objective Completion&lt;/strong&gt;: Muse Wonder Full (Max) (83.32) beat Gemini 4 Argon (High) by 1.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education - High School Subjects&lt;/strong&gt;: Gemini 4 Argon (79.3) beat Gemini 3.1 Pro (Preview) by 1.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;UGI - Writing&lt;/strong&gt;: gemini-4-argon (Thinking, low) (79.71) beat Gemini 3.8 Flash (Thinking, Medium) by 1.16&lt;/li&gt;&lt;li&gt;&lt;strong&gt;MMEB Leaderboard&lt;/strong&gt;: Wianmu-5B-v0 (60.45) beat WeMM-Embedding-9B by 1.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Agent Arena - Confirmed Success&lt;/strong&gt;: Claude Opus 5.5 (Max) (17.29) beat Claude Fable 5.1 (Max) by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;LLM Stats (DeepSWE 1.1)&lt;/strong&gt;: Muse Wonder Full (79.0) beat Gemini 4 Argon by 1.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Corporate M&amp;A&lt;/strong&gt;: Muse Wonder Full (Max) (98.18) beat Mistral Large 4 Preview by 1.07&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Design Arena (ASCII Art)&lt;/strong&gt;: GPT-6 Astra (1353.0) beat Claude Opus 5 by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - OCR&lt;/strong&gt;: Gemini 4 Argon (91.0) beat Claude Fable 5 by 1.0&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Argo-Bench - Solved&lt;/strong&gt;: Gemini 4 Argon (High) (35.71) beat Claude Opus 5.5 (xHigh) by 0.95&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Icelandic LLM - Idiom Completion&lt;/strong&gt;: Gemini 4 Argon (75.96) beat Claude Opus 5.5 by 0.93&lt;/li&gt;&lt;li&gt;&lt;strong&gt;OpenRouter GPQA Diamond&lt;/strong&gt;: Gemini 4 Argon (96.5) beat Gemini 3.8 Flash by 0.9&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PUMA Polish Multimodal - Vision Mean&lt;/strong&gt;: Gemini 4 Argon (84.83) beat Gemini 3.7 Flash by 0.83&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GDP.pdf&lt;/strong&gt;: GPT-6.1 Sol (Max) (35.0) beat GPT-6 Astra (Max) by 0.8&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Healthcare &amp; Life Sciences&lt;/strong&gt;: Muse Wonder (Max) (96.09) beat Muse Spark 1.3 (Max) by 0.79&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf - Legal - Criterion Pass Rate&lt;/strong&gt;: Muse Wonder (Max) (88.03) beat DeepSeek V4.1 Flash (Max) by 0.75&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Open Benchmarks - MMLU&lt;/strong&gt;: Gemini 4 Argon (95.25) beat Grok 4.5 by 0.71&lt;/li&gt;&lt;li&gt;&lt;strong&gt;ReLE - Education&lt;/strong&gt;: Gemini 4 Argon (69.3) beat Gemini 3.1 Pro (Preview) by 0.7&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Coding Daily (OpenCode) - React-TS Code Quality&lt;/strong&gt;: Mistral Large 4 (High) (19.33) beat DeepSeek V4.1 Flash (High) by 0.66&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - GPQA Diamond&lt;/strong&gt;: Gemini 4 Argon (High) (96.4) beat GPT-6 Astra (Max) by 0.63&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench - Correctness&lt;/strong&gt;: Flexor.ai (+ GPT-6.1 Sol) (93.0) beat CDL (Causal Dynamics Lab) by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Wiz Cyber Model Arena (ADK React) - API Security&lt;/strong&gt;: Gemini 4 Argon (85.8) beat Gemini 3.8 Flash Cyber by 0.6&lt;/li&gt;&lt;li&gt;&lt;strong&gt;EnterpriseRAG Bench&lt;/strong&gt;: Flexor.ai (+ GPT-6.1 Sol) (91.48) beat CDL (Causal Dynamics Lab) by 0.57&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA CritPt&lt;/strong&gt;: Muse Wonder Full (Max) (32.86) beat GPT-5.6 Sol (Max) by 0.57&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SciLaws-Bench&lt;/strong&gt;: GPT-6 Astra (51.33) beat GPT-5.5 by 0.56&lt;/li&gt;&lt;li&gt;&lt;strong&gt;NYT Connections Extended&lt;/strong&gt;: Gemini 4 Argon (Thinking, High) (98.6) beat GPT-6 Astra (Thinking, xHigh) by 0.5&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Real Estate&lt;/strong&gt;: Muse Wonder Full (Max) (98.06) beat Muse Spark 1.3 (Max) by 0.49&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Epoch AI - Rli&lt;/strong&gt;: Claude Opus 5.5 (Unknown) (21.25) beat GPT-6 Astra by 0.42&lt;/li&gt;&lt;li&gt;&lt;strong&gt;SimpleQA Verified&lt;/strong&gt;: Gemini 4 Argon (High) (76.0) beat GPT-6 Astra (Max) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AA GDP.pdf&lt;/strong&gt;: Muse Wonder Full (Max) (32.6) beat GPT-6 Astra (xHigh) by 0.4&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - White Collar Defense &amp; Investigations&lt;/strong&gt;: Muse Wonder Full (Max) (95.74) beat Muse Spark 1.3 (Max) by 0.34&lt;/li&gt;&lt;li&gt;&lt;strong&gt;GAIA&lt;/strong&gt;: Shadow Harness V8.01 (94.02) beat Ops-Agentic-Search-2.0 by 0.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;PLCC - Overall&lt;/strong&gt;: Gemini 4 Argon (97.33) beat Gemini 3.1 Pro (Preview) by 0.33&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Closed Book) - Physics&lt;/strong&gt;: Claude Opus 5.5 (2.808) beat Hy4 preview by 0.29&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Closed Book) - Engineering&lt;/strong&gt;: Claude Opus 5.5 (2.872) beat Hy4 preview by 0.26&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Global-MMLU-Lite - Bengali&lt;/strong&gt;: Gemini 4 Argon (96.0) beat Claude Opus 4.8 by 0.25&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Global-MMLU-Lite - Japanese&lt;/strong&gt;: Gemini 4 Argon (96.25) beat Claude Opus 5 by 0.25&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Kaggle Global-MMLU-Lite - Swahili&lt;/strong&gt;: Gemini 4 Argon (95.75) beat Claude Opus 5.5 by 0.25&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Set Generation&lt;/strong&gt;: Qwen3.8-27B (kb graphRAG) (aa045f) (0.5046) beat RuadaptQwen2.5-32B-Instruct (9449f3) by 0.22&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Simple Generation&lt;/strong&gt;: Qwen3.8-27B (kb graphRAG) (aa045f) (0.6265) beat RuadaptQwen2.5-32B-Instruct (9449f3) by 0.2&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Set Retrieval&lt;/strong&gt;: Qwen3.8-27B (kb graphRAG) (aa045f) (0.8139) beat Qwen2.5-7B-Instruct (d6ccf3) by 0.19&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Multi-Hop Generation&lt;/strong&gt;: gemma4-25B (kb graphRAG) (fe39ea) (0.6374) beat RuadaptQwen2.5-32B-Instruct (9449f3) by 0.19&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Closed Book) - Astronomy&lt;/strong&gt;: Gemini 4 Argon (2.778) beat DeepSeek V4.1 Flash by 0.18&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Generation&lt;/strong&gt;: Qwen3.8-27B (kb graphRAG) (aa045f) (0.6474) beat RuadaptQwen2.5-32B-Instruct (9449f3) by 0.18&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Closed Book) - Earth science&lt;/strong&gt;: Claude Opus 5.5 (2.76) beat MiMo-V2.6-Pro by 0.14&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - International Trade &amp; Sanctions&lt;/strong&gt;: Muse Wonder Full (Max) (96.41) beat Muse Spark 1.3 (Max) by 0.13&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Anthropic Child Safety Single-Turn Benign Requests - Over-Refusal Rate (Claude.ai)&lt;/strong&gt;: Claude Opus 5.5 (0.05) beat Claude Fable 5.1 by 0.12&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Harvey LAB-AA - Intellectual Property&lt;/strong&gt;: Muse Wonder Full (Max) (97.86) beat Muse Spark 1.3 (Max) by 0.11&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Anthropic Claude Code Dual-Use Requests - Success Rate (Mythos 5.1 System Card)&lt;/strong&gt;: Claude Opus 5.5 (99.8) beat Claude Opus 5 by 0.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Closed Book) - Overall&lt;/strong&gt;: Muse Wonder (2.683) beat Hy4 preview by 0.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Conditional Generation&lt;/strong&gt;: Qwen3.8-27B (kb graphRAG) (aa045f) (0.8285) beat RuadaptQwen2.5-32B-Instruct (9449f3) by 0.1&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Closed Book) - Chemistry&lt;/strong&gt;: Muse Wonder (2.703) beat Hy4 preview by 0.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Retrieval&lt;/strong&gt;: Qwen3.8-27B (kb graphRAG) (aa045f) (0.7716) beat RuadaptQwen2.5-32B-Instruct (9449f3) by 0.09&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Tools) - Engineering&lt;/strong&gt;: GPT-6 Astra (xHigh) (2.894) beat GPT-6.1 Sol by 0.08&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Multi-Hop Retrieval&lt;/strong&gt;: Qwen3.8-27B (kb graphRAG) (aa045f) (0.7529) beat RuadaptQwen2.5-32B-Instruct (9449f3) by 0.07&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Simple Retrieval&lt;/strong&gt;: Qwen3.8-27B (kb graphRAG) (aa045f) (0.7688) beat RuadaptQwen2.5-32B-Instruct (9449f3) by 0.06&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Tools) - Overall&lt;/strong&gt;: GPT-6 Astra (xHigh) (2.809) beat GPT-6.1 Sol by 0.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Tools) - Biology&lt;/strong&gt;: GPT-6 Astra (xHigh) (2.709) beat GPT-6.1 Sol by 0.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Tools) - Chemistry&lt;/strong&gt;: GPT-6 Astra (xHigh) (2.777) beat GPT-6.1 Sol by 0.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Tools) - Earth science&lt;/strong&gt;: GPT-6 Astra (xHigh) (2.726) beat GPT-6.1 Sol by 0.05&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Tools) - Physics&lt;/strong&gt;: GPT-6 Astra (xHigh) (3.025) beat GPT-6.1 Sol by 0.04&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Closed Book) - Biology&lt;/strong&gt;: Muse Wonder (2.615) beat Hy4 preview by 0.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;STEMBench120 v1.0.0 (Tools) - Astronomy&lt;/strong&gt;: GPT-6 Astra (xHigh) (2.723) beat GPT-6.1 Sol by 0.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Conceptual Reasoning Index - Argument Evaluation (LMCA)&lt;/strong&gt;: Claude Sonnet 5.5 (68.26) beat Claude Opus 5.5 by 0.03&lt;/li&gt;&lt;li&gt;&lt;strong&gt;AI Forever RAG - Conditional Retrieval&lt;/strong&gt;: gemma4-25B (kb graphRAG) (fe39ea) (0.7644) beat RuadaptQwen2.5-32B-Instruct (9449f3) by 0.02&lt;/li&gt;&lt;li&gt;&lt;strong&gt;Sonar LLM Leaderboard - Java - Vulnerability Density&lt;/strong&gt;: GPT-5.4 (Medium) (0.04) beat GPT-5.2 (High) by 0.01&lt;/li&gt;&lt;/ul&gt;</content><summary>The Aggregate Digest — 2026-10-18

=== FRONTIER ===
  - Best available model: DeepSeek V4.1 Flash (1017) enters at #15 (Aggregate Rating 1911) on 42 benchmarks

=== DAILY ===
NEW BENCHMARKS (33)
  - Wiz Cyber Model Arena (Antigravity) - Overall (Macro average of the four category pass@1 rates (%)): </summary></entry></feed>