EditReward-Compass - Visual Consistency: leaderboard
Accuracy (%) of multimodal LLMs used as reward models on the preference pairs that differ in how well unedited content is preserved. EditReward-Compass (2026) holds 2,251 human-verified preference pairs of edited images produced by the same editing models under one instruction, as a reward model meets them during RL; higher is better.
Metric: Preference accuracy (%) on the pairs that differ in visual consistency on EditReward-Compass (2,251 human-verified preference pairs of edited images sampled from the same editing models under the same instruction), general MLLMs prompted with the Edit-Compass rubric; thinking disabled unless the row is a thinking run; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 18 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 60.02 | #102 | |
| 2 | Gemini 3 Flash | 59.81 | #180 | |
| 3 | Qwen 3.5 27B | Non-reasoning | 58.5 | #380 |
| 4 | Qwen 3.6 27B | 56.56 | #358 | |
| 5 | Qwen 3.5 35B A3B | Non-reasoning | 54.79 | #494 |
| 6 | Qwen 3.6 35B A3B | 53 | #485 | |
| 7 | Gemma 4 31B | 51.65 | #390 | |
| 8 | Qwen 3.5 9B | Non-reasoning | 50.75 | #838 |
| 9 | GPT-4.1 | 48.45 | #506 | |
| 10 | Gemma 4 26B A4B | 39.6 | #602 | |
| 11 | Qwen 3 VL 32B | 39.6 | #624 | |
| 12 | Gemma 3 27B | 33 | #1672 | |
| 13 | Qwen 3.5 2B | Non-reasoning | 32.79 | #5380 |
| 14 | Qwen 3 VL 30B A3B | 30.86 | #791 | |
| 15 | Gemma 3 12B | 28.71 | #2091 |
No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=editreward-compass-visual-consistency · How It Works · Data refreshed daily, snapshot 2026-11-22.