MobileProbe - Fake Dialog - Zero-shot LLM: leaderboard
MobileProbe - Fake Dialog - Zero-shot LLM — MobileProbe v1;83Android14attack instances,44malicious-text/39fake-dialog. Three repeats per defender; ASR is the share reaching attacker goal,lower is better for the named defender. Victims are Gemini3.1Pro,GUI-Owl1.5-8B-Instruct and a Gemini3.1Pro+UI-Ins7Bplanning/grounding composite. Seed2.0Progenerates/mutates attacks; GPT5.4judges semantic/visual/goal outcomes,not a scored defender. Iterative methods share15evaluations/task; source static/zero-shot controls stay separate attack conditions. Overall,44text and39dialog cohorts are not pooled into new scores. Metric: Defender attack success rate (%).
Metric: Defender attack success rate (%). Source: arxiv.org. Saturation forecast: Around February 2027. 3 models tracked. Updated 15 Nov 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Gemini 3.1 Pro | MobileProbe run: planning=Gemini3.1Pro, grounding=UI-Ins-7B | 5.1 | |
| 2 | Gemini 3.1 Pro (Preview) | 8.5 | #93 | |
| 3 | GUI-Owl-1.5-8B-Instruct | 16.2 | #1350 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=mobileprobe-fake-dialog-zero-shot-llm · How It Works · Data refreshed daily, snapshot 2026-11-16.