Emoji-Affect Decodability (Bangla): leaderboard
Emoji-affect decodability (Sadab and Sakhawat, 2026) measures how much of the emotion in a sentence survives when an LLM summarizes it in one to five emojis: a logistic-regression probe trained on the emoji output of each model must recover the emotion label of the source corpus. Macro-F1 in percent, 5-fold cross-validated over 4,206 single-label Bangla sentences and five emotions; higher is better.
Metric: Macro-F1 (%; emoji-affect decodability: 5-fold cross-validated l2-regularised logistic regression on sublinear TF-IDF features of each model's emoji graphemes, predicting the source emotion label of the 4,206 single-label Bangla sentences over the five emotions shared by the three corpora; one generation per sentence at temperature 0.7; majority-class floor 8.7). Source: arxiv.org. Saturation forecast: Around November 2028. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.0 Flash | 65.7 | #702 |
| 2 | Grok 4 Fast | 65.5 | #438 |
| 3 | Gemma 3 27B (IT) | 65.3 | #1306 |
| 4 | Mistral Large 3 | 63.9 | #819 |
| 5 | Qwen 3 VL 235B A22B | 59.7 | #477 |
| 6 | Claude 3 Haiku | 58.4 | #2629 |
| 7 | GPT-4.1 Nano | 56.2 | #1992 |
| 8 | DeepSeek V3.2 | 56 | #403 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=emoji-affect-decodability-bangla · How It Works · Data refreshed daily, snapshot 2026-11-12.