SG-LegalClarify Mandatory Web enabled: leaderboard
SG-LegalClarify Mandatory Web enabled — Separateclosedbookandwebconditions; helpers/gateclosedbook, max8modelturns. Web8OpenAI/Anthropic/Geminimodelsnativeprovidersearch,remaining11Serper–Jinaloop; sameclosedbooksampling. AppendixBdetailsread andretainedinscale-paper.txt. 19models,1860scenarios (1100 originalquestions,760underspecifiedpairedwithfullyspecified),FS+UScombined. Mandatoryrubricpercentage macroaveragedacrosstrials andtheneligible3runs; nativeexceptions1/2runs retained. ThreejudgepanelDeepSeekV4Flash,GPT5.6Sol,KimiK3. Gemini3.1Pro gemini/gemini-3.1-pro-preview;Mistral mistral/mistral-large-latest;Qwen3.6 fireworks_ai/deployed/t5cvm6h4;Llama4ScoutBedrockclosedbookGroqweb;Gemma3Bedrockclosedbooklocalweb. Do not inventdatedmodelIDs absentpaper. Metric: Mandatory rubric criteria satisfied (%). Source: https://cdn.sanity.io/files/50zba0eo/production/aad2df6575c85900d3f862ae56e4bed01af76e4f.pdf
Metric: Mandatory rubric criteria satisfied (%). Source: cdn.sanity.io. Saturation forecast: Not forecast. 19 models tracked. Updated 16 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Kimi K3 | High | 77.8 | #56 |
| 2 | DeepSeek V4 Flash | High | 74 | #259 |
| 3 | Claude Fable 5 | High | 73.3 | #27 |
| 4 | DeepSeek V4 Pro | High | 73.1 | #191 |
| 5 | Claude Opus 4.8 | High | 72.9 | #58 |
| 6 | Gemini 3.5 Flash | High | 71.6 | #100 |
| 7 | GPT-5.6 Sol | High | 70.5 | #30 |
| 8 | Claude Sonnet 5 | High | 70.1 | #111 |
| 9 | Kimi K2.6 | High | 68.9 | #181 |
| 10 | Qwen 3.7 Plus | High | 67.9 | #208 |
| 11 | GPT-5.6 Terra | High | 67.1 | #64 |
| 12 | GPT-5.6 Luna | High | 66.7 | #122 |
| 13 | Gemini 3.1 Pro (Preview) | High | 59.9 | #93 |
| 14 | Qwen 3.6 27B | High | 58.9 | #344 |
| 15 | Mistral Large 3 | Scale run: reasoning=none | 57.2 | #827 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=sg-legalclarify-mandatory-web-enabled · How It Works · Data refreshed daily, snapshot 2026-11-16.