Llama-3.3-nemotron-super-49B-v1.5 Benchmark Results

Llama-3.3-nemotron-super-49B-v1.5: benchmark results

Provider: NVIDIA. Access: Open.

Aggregate Rating 1561 ± 33, rank #1535 of 10916 rated models, from 21 benchmark results.

Benchmark	Score	Metric	Percentile
CareTransition-Audit - Completeness Ranking	0.47	Spearman correlation (-1 to 1) between the per-summary docum	90
SpeechMap Compliance	87.8	% Requests Completed	88.7
MADE - Jaccard	46	Sample-level Jaccard index (%, the 0-1 score times 100) betw	57.1
MADE	42	Macro-F1 (%, the 0-1 score times 100) over all 1,154 labels	53.6
MADE - Head Labels	57	Macro-F1 (%, the 0-1 score times 100) over the 144 head labe	53.6
MADE - Medium Labels	46	Macro-F1 (%, the 0-1 score times 100) over the 481 medium la	53.6
IKP - T2	99	Accuracy (%)	51
CareTransition-Audit	0.37	Cohen kappa (-1 to 1) between the model labels and the clini	50
MADE - Tail Labels	38	Macro-F1 (%, the 0-1 score times 100) over the 348 tail labe	46.4
MADE - Extreme-Tail Labels	19	Macro-F1 (%, the 0-1 score times 100) over the 181 extreme-t	42.9
AI Chess Leaderboard (Reasoning)	619	Elo	35.8
CareTransition-Audit - Accuracy	71.9	Label accuracy (%) against the clinician labels on the 46-qu	30