Seizure-Semiology-Suite - Report Generation: leaderboard
Narrative report generation on Seizure-Semiology-Suite (S3, 2026): a model watches an epilepsy monitoring unit seizure video and writes the semiology report, scored by the Seizure Report Quality Index against the clinician report (structure, symptom coverage, localizing features and temporal order, with penalties for hallucinated or hazardous content). Scale 0 to 100 over the held-out test videos; higher is better.
Metric: Seizure Report Quality Index, Seizure RQI (0-100): weighted structural completeness (15%), symptom coverage (35%), key localizing features (25%) and temporal-relation F1 (25%) of the generated semiology report against the clinician report, extracted by a Qwen3-Plus LLM, with multiplicative penalties for hallucinated features, off-topic content and excess length and a cap of 50 for hazardous statements; mean over the 82 held-out test videos, 2 FPS sliding windows merged by an LLM, temperature 0; higher is better. Source: arxiv.org. Saturation forecast: Around December 2027. 10 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Lingshu-32B | 39.8 | #1113 |
| 2 | Qwen 3 VL 32B Instruct | 39.13 | #606 |
| 3 | InternVL3.5-8B | 38.43 | #1604 |
| 4 | Qwen 3 VL 8B Instruct | 38.2 | #1029 |
| 5 | Qwen3 Omni 30B A3B Instruct | 37.52 | #930 |
| 6 | Qwen 2.5 VL 32B Instruct | 37.48 | #1062 |
| 7 | Qwen 2.5 VL 72B Instruct | 37.37 | #841 |
| 8 | Qwen 2.5 VL 7B Instruct | 36.94 | #1902 |
| 9 | InternVL3.5-38B | 36.14 | #1021 |
| 10 | Qwen2.5-Omni-7B | 35.91 | #1884 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=seizure-semiology-suite-report-generation · How It Works · Data refreshed daily, snapshot 2026-11-21.