CNSL-bench: leaderboard

CNSL-bench (2026) evaluates multimodal LLMs on Chinese National Sign Language: each entry of the standardized National Common Sign Language Dictionary is posed as a four-way multiple-choice question from a single input modality. This board is accuracy in percent identifying the sign from its sign language video sampled at a nominal 10 frames per second, reduced when context-window or API frame limits require; higher is better, and chance is 25.

Metric: Accuracy (%) identifying the sign's meaning from its sign language video sampled at 10 frames per second (fewer where the context window or API frame cap requires), four-way multiple choice over the 6,707 sign entries of the National Common Sign Language Dictionary (distractors sampled at random), official sampling settings per model, accuracy by exact match of the chosen option; higher is better. Source: arxiv.org. Saturation forecast: Around December 2027. 18 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-556.72#185
2Gemini 2.5 ProMedium48.35#280
3Gemini 2.5 Flash42.63#466
4Qwen 3 VL Plus37.37#514
5Qwen-VL-Max34.43#1402
6Qwen-VL-Plus33.95#5893
7Qwen 3 VL 8B Instruct33.89#1043
8InternVL3.5-8B33.59#1618
9Qwen 3 VL 8B (Thinking)33#924
10Qwen 2 VL 7B Instruct31.19#2581
11Qwen 3 VL 2B (Thinking)30.68#2552
12Qwen 2.5 VL 3B Instruct30.34#2720
13Qwen 3 VL 2B Instruct30.18#2389
14Qwen 2.5 VL 7B Instruct30.15#1919
15GLM-4.1V-9B (Thinking)29.75

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=cnsl-bench · How It Works · Data refreshed daily, snapshot 2026-11-22.