EditJudge-Bench - Main - Rot: leaderboard
EditJudge-Bench v1,1382curatedBlender before/after pairs,factual/counterfactual instructions. NativeAUROC or multiplechoice factual-instruction ranking percent. No parsing-failure dropping:asymmetric score2fail factual,8failcounterfactual;unsupportedlogitoperation.5;APIerrors score5;zero outputs retained. Exact publishedTOML backend/model/prompt/image-order/budget retained; source-unreported launcherflags remain unknown. Gemma31B is NVIDIA NVFP4;QwenThinking is its distinct checkpoint. Supervised judges retain own trainedartifact IDs,not their basemodels.
Metric: AUROC (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 25 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | gpt-5.6-sol (EditJudge Run: config=vie__gpt-5.6-sol;backend=openai,template=vie_tie_0shot,image_position=after,reasoning_effort=low,max_tokens=8192) | 80.1 | |
| 2 | claude-opus-5 (EditJudge Run: config=vie__claude-opus-5;backend=anthropic,template=vie_tie_0shot,image_position=after,thinking_effort=low,max_tokens=8192) | 70.8 | |
| 3 | gpt-5.4 (EditJudge Run: config=vie__gpt-5.4;backend=openai,template=vie_tie_0shot,image_position=after,reasoning_effort=low,max_tokens=8192) | 64.4 | |
| 4 | Gemma-4-31B-IT-NVFP4 (EditJudge Run: config=logprob__gemma-4-31b;backend=vllm,template=binary_q_images_last,logprobs=True,max_model_len=4096) | 63.8 | |
| 5 | Gemma-4-31B-IT-NVFP4 (EditJudge Run: config=vie__gemma-4-31b;backend=vllm,template=vie_tie_0shot,image_position=after,max_tokens=16,max_model_len=4096) | 61.5 | |
| 6 | llava-onevision-qwen2-7b-ov-hf (EditJudge Run: config=vie__llava-ov-7b;backend=vllm,template=vie_tie_0shot,image_position=after,max_tokens=16,max_model_len=8192) | 57.8 | |
| 7 | Gemma-4-31B-IT-NVFP4 (EditJudge Run: config=imgedit__gemma-4-31b;backend=vllm,template=imgedit,image_position=after,max_tokens=16,max_model_len=4096;unlogged_runtime_fields=max_tokens,image_position;values=reviewed_config_defaults_not_confirmed_launch) | 55.5 | |
| 8 | gpt-4.1-mini (EditJudge Run: config=vie__gpt-4.1-mini;backend=openai,template=vie_tie_0shot,image_position=after,max_tokens=1024) | 55 | |
| 9 | Gemma-4-31B-IT-NVFP4 (EditJudge Run: config=kris__gemma-4-31b;backend=vllm,template=kris_instruction,image_position=after,max_tokens=16,max_model_len=4096) | 54.6 | |
| 10 | gemini-2.5-flash (EditJudge Run: config=vie__gemini-2.5-flash;backend=gemini,template=vie_tie_0shot,image_position=after,thinking=True,max_tokens=128) | 54 |
Interactive version: theaggregate.ai/benchmark?slug=editjudge-bench-main-rot · How It Works · Data refreshed daily, snapshot 2026-11-12.