MetaGAI - Model Card Quality: leaderboard
Writing model documentation from research papers on MetaGAI (2026): the model reads a paper and fills a structured model card, which three LLM judges rate from 1 to 5 on faithfulness, relevance, accuracy, consistency and usefulness on a 250-entry stratified sample. Mean Likert score; higher is better.
Metric: Qualitative average (1-5): mean judge Likert score of model cards generated zero-shot from the paper text alone, rated on faithfulness, relevance, accuracy, consistency and usefulness by GPT-OSS-120B, Llama-3.3-70B-Instruct and Qwen3-235B-A22B-2507 judges, on a stratified sample of 250 MetaGAI entries (2,541 verified paper, repository and Hugging Face triplets); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 30B A3B Instruct | 4.55 | #1053 |
| 2 | GPT-5 Mini | 4.28 | #364 |
| 3 | Gemini 2.5 Flash Lite | 4.03 | #964 |
| 4 | Gemini 2.5 Flash | 4 | #463 |
| 5 | Gemma 3 27B (IT) | 3.94 | #1324 |
| 6 | nemotron-3-nano-30B-a3B | 3.74 | #1946 |
| 7 | Mistral Small 3.2 | 3.63 | #1641 |
| 8 | Qwen 3 32B | 3.6 | #972 |
| 9 | GPT-OSS-20B | 3.38 | #1183 |
| 10 | GPT-5 Nano | 3.23 | #906 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=metagai-model-card-quality · How It Works · Data refreshed daily, snapshot 2026-11-21.