Jamendo-MT-QA - Short-Answer Questions: leaderboard

Jamendo-MT-QA (2026) asks comparative questions about pairs of music tracks from Jamendo; models that take two audio inputs answer end to end on a 2,010-pair subset. This board covers the short-answer questions whose answer names one of the two tracks, scored by exact match. Accuracy in percent; higher is better.

Metric: Accuracy (%) on comparative short-answer questions whose answer names one of the two tracks, exact match on the track identifier; the 2,010-pair random subset on which every multi-audio model ran; each model hears both tracks and the question end to end; higher is better. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 4 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-4o Audio84.4#1129
2Jamendo-MT-QA Qwen2-Audio (checkpoint unspecified)77.7
3Qwen3-Omni75.5#1055
4GPT-4o-mini-Audio73.2#1795

Interactive version: theaggregate.ai/benchmark?slug=jamendo-mt-qa-short-answer-questions · How It Works · Data refreshed daily, snapshot 2026-11-21.