MetaGAI - Model Card Quality: leaderboard

Writing model documentation from research papers on MetaGAI (2026): the model reads a paper and fills a structured model card, which three LLM judges rate from 1 to 5 on faithfulness, relevance, accuracy, consistency and usefulness on a 250-entry stratified sample. Mean Likert score; higher is better.

Metric: Qualitative average (1-5): mean judge Likert score of model cards generated zero-shot from the paper text alone, rated on faithfulness, relevance, accuracy, consistency and usefulness by GPT-OSS-120B, Llama-3.3-70B-Instruct and Qwen3-235B-A22B-2507 judges, on a stratified sample of 250 MetaGAI entries (2,541 verified paper, repository and Hugging Face triplets); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3 30B A3B Instruct4.55#1053
2GPT-5 Mini4.28#364
3Gemini 2.5 Flash Lite4.03#964
4Gemini 2.5 Flash4#463
5Gemma 3 27B (IT)3.94#1324
6nemotron-3-nano-30B-a3B3.74#1946
7Mistral Small 3.23.63#1641
8Qwen 3 32B3.6#972
9GPT-OSS-20B3.38#1183
10GPT-5 Nano3.23#906

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=metagai-model-card-quality · How It Works · Data refreshed daily, snapshot 2026-11-21.