SciEval (K-12) - Quadratic Weighted Kappa: leaderboard
Automatic evaluation of K-12 science lessons on SciEval (2026): given an NGSS-aligned lesson and one EQuIP rubric criterion, the model returns a 0 to 3 score with supporting evidence, compared with expert annotations on the held-out test set. Quadratic weighted kappa between predicted and expert scores, times 100 (from -100 to 100); higher is better.
Metric: Quadratic weighted kappa (x100, -100 to 100) between the predicted and expert criterion scores on the held-out test set of SciEval, a K-12 science instructional-material evaluation benchmark: given an NGSS lesson as page-marked PDF text and one EQuIP rubric criterion, the model returns a 0-3 score and the supporting evidence as JSON, simplified prompt, temperature 0; failed predictions are left out; higher is better. Source: arxiv.org. Saturation forecast: Around 2030. 5 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 4B 2507 Instruct | 16.23 | #2221 |
| 2 | Gemini 2.0 Flash | 5.7 | #723 |
| 3 | Llama 3.1 8B Instruct | 4.58 | #4600 |
| 4 | Llama 3.2 3B Instruct | 4.52 | #7241 |
| 5 | GPT-4o Mini | 4.02 | #1667 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=scieval-k-12-quadratic-weighted-kappa · How It Works · Data refreshed daily, snapshot 2026-11-21.