MobileProbe - Fake Dialog - Adapted Genesis: leaderboard

MobileProbe - Fake Dialog - Adapted Genesis — MobileProbe v1;83Android14attack instances,44malicious-text/39fake-dialog. Three repeats per defender; ASR is the share reaching attacker goal,lower is better for the named defender. Victims are Gemini3.1Pro,GUI-Owl1.5-8B-Instruct and a Gemini3.1Pro+UI-Ins7Bplanning/grounding composite. Seed2.0Progenerates/mutates attacks; GPT5.4judges semantic/visual/goal outcomes,not a scored defender. Iterative methods share15evaluations/task; source static/zero-shot controls stay separate attack conditions. Overall,44text and39dialog cohorts are not pooled into new scores. Metric: Defender attack success rate (%).

Metric: Defender attack success rate (%). Source: arxiv.org. Saturation forecast: Around June 2027. 3 models tracked. Updated 15 Nov 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3.1 ProMobileProbe run: planning=Gemini3.1Pro, grounding=UI-Ins-7B51.3
2Gemini 3.1 Pro (Preview)57.3#93
3GUI-Owl-1.5-8B-Instruct81.2#1350

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=mobileprobe-fake-dialog-adapted-genesis · How It Works · Data refreshed daily, snapshot 2026-11-16.