EditReward-Compass - Visual Consistency: leaderboard

Accuracy (%) of multimodal LLMs used as reward models on the preference pairs that differ in how well unedited content is preserved. EditReward-Compass (2026) holds 2,251 human-verified preference pairs of edited images produced by the same editing models under one instruction, as a reward model meets them during RL; higher is better.

Metric: Preference accuracy (%) on the pairs that differ in visual consistency on EditReward-Compass (2,251 human-verified preference pairs of edited images sampled from the same editing models under the same instruction), general MLLMs prompted with the Edit-Compass rubric; thinking disabled unless the row is a thinking run; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 18 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3.1 Pro (Preview)60.02#102
2Gemini 3 Flash59.81#180
3Qwen 3.5 27BNon-reasoning58.5#380
4Qwen 3.6 27B56.56#358
5Qwen 3.5 35B A3BNon-reasoning54.79#494
6Qwen 3.6 35B A3B53#485
7Gemma 4 31B51.65#390
8Qwen 3.5 9BNon-reasoning50.75#838
9GPT-4.148.45#506
10Gemma 4 26B A4B39.6#602
11Qwen 3 VL 32B39.6#624
12Gemma 3 27B33#1672
13Qwen 3.5 2BNon-reasoning32.79#5380
14Qwen 3 VL 30B A3B30.86#791
15Gemma 3 12B28.71#2091

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=editreward-compass-visual-consistency · How It Works · Data refreshed daily, snapshot 2026-11-22.