SciConvBench - Inconsistency Resolution - Materials Science - Final Resolution: leaderboard

SciConvBench (2026) asks an assistant to turn an ill-posed computational-science request into a correct task specification by questioning a simulated scientist, one clarification per turn, judged against the annotated missing or conflicting requirements. This board is the final resolution rate on materials science inconsistency resolution cases (requests with planted conflicting information), the share whose final specification resolves every issue; percent, higher is better.

Metric: Final Resolution Rate (%): share of materials science inconsistency resolution cases (requests with planted conflicting information) whose final specification resolves every annotated issue, the assistant may ask one clarification question per turn of a simulated scientist (Claude Sonnet 4.6, answering only from the hidden reference specification) for at most 11 turns before producing the final task specification, guided scientist-mode system prompt, judged by Gemini 2.5 Pro against the annotated issues; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 5 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Pro87.4#276
2Gemini 2.5 Flash75.3#463
3Claude Sonnet 4.671.3#173
4GPT-5.266.7#185
5GPT-OSS-120B54.6#704

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=sciconvbench-inconsistency-resolution-materials-science-final-resolution · How It Works · Data refreshed daily, snapshot 2026-11-21.