Nepal Math Competency Grading (Teacher 2 Reference): leaderboard

LLMs as competency-based assessors (2026): each model assigns rubric competency levels to handwritten Grade 10 Optional Mathematics answers of students in Nepal, transcribed by OCR, and is compared with the levels the second senior mathematics teacher assigned. Quadratic weighted Cohen kappa from -1 to 1; higher is better.

Metric: Quadratic weighted Cohen's kappa (-1 to 1) between the model's competency-level assignments (Awareness, Application, Mastery, Influence for comprehension, knowledge, operational fluency and behavior and correlation) and those of the second senior mathematics teacher, over the handwritten Grade 10 Optional Mathematics answers of 33 students in Nepal (matrix, coordinate geometry, trigonometry, functions), transcribed by gemini-2.5-flash and graded with one master evaluation prompt at temperature 0.1; higher is better. Source: arxiv.org. Saturation forecast: Around January 2028. 4 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Flash0.3461#466
2Gemini 3 Pro0.2664#146
3Llama 3.1 8B0.1233#5193
4Llama 3.3 70B-0.0292#1523

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=nepal-math-competency-grading-teacher-2-reference · How It Works · Data refreshed daily, snapshot 2026-11-22.