Seizure-Semiology-Suite - Report Generation: leaderboard

Narrative report generation on Seizure-Semiology-Suite (S3, 2026): a model watches an epilepsy monitoring unit seizure video and writes the semiology report, scored by the Seizure Report Quality Index against the clinician report (structure, symptom coverage, localizing features and temporal order, with penalties for hallucinated or hazardous content). Scale 0 to 100 over the held-out test videos; higher is better.

Metric: Seizure Report Quality Index, Seizure RQI (0-100): weighted structural completeness (15%), symptom coverage (35%), key localizing features (25%) and temporal-relation F1 (25%) of the generated semiology report against the clinician report, extracted by a Qwen3-Plus LLM, with multiplicative penalties for hallucinated features, off-topic content and excess length and a cap of 50 for hazardous statements; mean over the 82 held-out test videos, 2 FPS sliding windows merged by an LLM, temperature 0; higher is better. Source: arxiv.org. Saturation forecast: Around December 2027. 10 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Lingshu-32B39.8#1113
2Qwen 3 VL 32B Instruct39.13#606
3InternVL3.5-8B38.43#1604
4Qwen 3 VL 8B Instruct38.2#1029
5Qwen3 Omni 30B A3B Instruct37.52#930
6Qwen 2.5 VL 32B Instruct37.48#1062
7Qwen 2.5 VL 72B Instruct37.37#841
8Qwen 2.5 VL 7B Instruct36.94#1902
9InternVL3.5-38B36.14#1021
10Qwen2.5-Omni-7B35.91#1884

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=seizure-semiology-suite-report-generation · How It Works · Data refreshed daily, snapshot 2026-11-21.