CodeMixQA: leaderboard
Code-switched factual question answering from CodeMixQA (Winata et al., 2026): SimpleQA Verified questions rewritten by GPT-5.2 into 16 code-switched variants (English mixed into Bengali, Chinese, French, Hindi, Indonesian, Italian, Japanese, Korean, Marathi, Spanish, Toba Batak and Urdu, with romanised forms of the four Indic languages) under four switching strategies. Mean over the four strategies of the SimpleQA F1 graded by a Qwen3-30B-A3B judge, 0-100; higher is better. The unperturbed English questions are not part of the mean.
Metric: SimpleQA F1 (%), mean of 4 code-switching perturbations. Source: arxiv.org. Saturation forecast: Estimated already saturated. 11 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4.1 | 36.83 | #492 |
| 2 | GPT-OSS-120B | 22.23 | #699 |
| 3 | Qwen 3 30B A3B (Thinking) | 20.84 | #1124 (Qwen 3 30B A3B) |
| 4 | Llama 3.3 70B Instruct | 17.21 | #1353 |
| 5 | GPT-OSS-20B | 16.48 | #1150 |
| 6 | Gemma 3 27B | 11.38 | #1638 |
| 7 | Qwen 3 4B | 8.38 | #2297 |
| 8 | Gemma 3 12B | 8.1 | #2013 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=codemixqa · How It Works · Data refreshed daily, snapshot 2026-11-12.