SG-LegalClarify Mandatory Web enabled: leaderboard

SG-LegalClarify Mandatory Web enabled — Separateclosedbookandwebconditions; helpers/gateclosedbook, max8modelturns. Web8OpenAI/Anthropic/Geminimodelsnativeprovidersearch,remaining11Serper–Jinaloop; sameclosedbooksampling. AppendixBdetailsread andretainedinscale-paper.txt. 19models,1860scenarios (1100 originalquestions,760underspecifiedpairedwithfullyspecified),FS+UScombined. Mandatoryrubricpercentage macroaveragedacrosstrials andtheneligible3runs; nativeexceptions1/2runs retained. ThreejudgepanelDeepSeekV4Flash,GPT5.6Sol,KimiK3. Gemini3.1Pro gemini/gemini-3.1-pro-preview;Mistral mistral/mistral-large-latest;Qwen3.6 fireworks_ai/deployed/t5cvm6h4;Llama4ScoutBedrockclosedbookGroqweb;Gemma3Bedrockclosedbooklocalweb. Do not inventdatedmodelIDs absentpaper. Metric: Mandatory rubric criteria satisfied (%). Source: https://cdn.sanity.io/files/50zba0eo/production/aad2df6575c85900d3f862ae56e4bed01af76e4f.pdf

Metric: Mandatory rubric criteria satisfied (%). Source: cdn.sanity.io. Saturation forecast: Not forecast. 19 models tracked. Updated 16 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Kimi K3High77.8#56
2DeepSeek V4 FlashHigh74#259
3Claude Fable 5High73.3#27
4DeepSeek V4 ProHigh73.1#191
5Claude Opus 4.8High72.9#58
6Gemini 3.5 FlashHigh71.6#100
7GPT-5.6 SolHigh70.5#30
8Claude Sonnet 5High70.1#111
9Kimi K2.6High68.9#181
10Qwen 3.7 PlusHigh67.9#208
11GPT-5.6 TerraHigh67.1#64
12GPT-5.6 LunaHigh66.7#122
13Gemini 3.1 Pro (Preview)High59.9#93
14Qwen 3.6 27BHigh58.9#344
15Mistral Large 3Scale run: reasoning=none57.2#827

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=sg-legalclarify-mandatory-web-enabled · How It Works · Data refreshed daily, snapshot 2026-11-16.