MADE - Extreme-Tail Labels: leaderboard

Very rare label classification on MADE (2026), a living benchmark of FDA medical device adverse event reports: macro-F1 over the 181 extreme-tail labels attached to fewer than 0.01 percent of training reports, with models prompted with ten kNN-retrieved training reports. Percent; higher is better.

Metric: Macro-F1 (%, the 0-1 score times 100) over the 181 extreme-tail labels (under 0.01 percent of training reports) with ten-shot prompting with kNN-retrieved training reports, the task instructions and the label list, on the 10,288-report stratified (truncated) MADE test set of FDA medical device adverse event reports from July 2024 to June 2025 with 1,154 hierarchical IMDRF product and patient problem labels, greedy decoding (GPT-5 at its default medium reasoning effort); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-534#178
2Qwen 3 235B A22BThinking33#628
3DeepSeek R1 052830#428
4Qwen 3 30B A3BThinking28#1156
5GLM 4.5 Air24#886
6Qwen 3 235B A22B Instruct24#691
7GPT-4.122#505
8Qwen 3 4BReasoning20#2348
9Llama-3.3-nemotron-super-49B-v1.519#920
10GPT-OSS-120B15#704
11Qwen 3 4B Instruct9#1696
12Llama 3.1 70B Instruct8#1571
13Qwen 3 30B A3B Instruct5#1053
14Llama 3.1 8B Instruct1#4600
15Kimi K21#443

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=made-extreme-tail-labels · How It Works · Data refreshed daily, snapshot 2026-11-21.