CNSL-bench - Video (2 fps): leaderboard

CNSL-bench (2026) evaluates multimodal LLMs on Chinese National Sign Language: each entry of the standardized National Common Sign Language Dictionary is posed as a four-way multiple-choice question from a single input modality. This board is accuracy in percent identifying the sign from its sign language video sampled at 2 frames per second; higher is better, and chance is 25.

Metric: Accuracy (%) identifying the sign's meaning from its sign language video sampled at 2 frames per second, four-way multiple choice over the 6,707 sign entries of the National Common Sign Language Dictionary (distractors sampled at random), official sampling settings per model, accuracy by exact match of the chosen option; higher is better. Source: arxiv.org. Saturation forecast: Around September 2027. 20 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-553.42#178
2Gemini 2.5 ProMedium48.32#276
3Gemini 2.5 Flash42.28#463
4Qwen 3 VL PlusThinking35.34#512
5InternVL3.5-8B32.26#1604
6Qwen-VL-Plus32.06#5818
7GPT-4o31.26#751
8Qwen-VL-Max31.01#1400
9Qwen 3 VL 8B Instruct30.94#1029
10Qwen 3 VL 8B (Thinking)29.88#910
11Qwen 2 VL 7B Instruct29.24#2573
12Qwen 2.5 VL 7B Instruct29.1#1902
13Qwen 2.5 VL 3B Instruct28.36#2713
14GLM-4.1V-9B (Thinking)28.03
15Qwen 3 VL 2B Instruct27.78#2367

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=cnsl-bench-video-2-fps · How It Works · Data refreshed daily, snapshot 2026-11-21.