WebHarbor WebArena - Auto-login - Shopping: leaderboard
WebHarbor Table 8,page 9:306 held-out tasks; ReAct-style GUI vs OpenCode equipped with GUI tools, separate configurations. Verifier and agent-judge pass rates separately reported; their agreement is not capability. General-model effort unspecified. Browser 1280 x 1000,temperature 0.6,top-p 0.95,top-k 20,30-action limit except WebArena 50. Qwen 3-VL-4B-Thinking is the Base row per training setup; trained OpenWebRL and WebHarbor checkpoints remain distinct. Table 9:WebArena programmatic task validation; no-auto-login condition gives credentials in task instead of an authenticated session; native subgroup means retained. Metric: Task success (%).
Metric: Task success (%). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 VL 4B (Thinking) | 25.5 | #548 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=webharbor-webarena-auto-login-shopping · How It Works · Data refreshed daily, snapshot 2026-11-05.