MAC-Bench (AutoGen) - Success Rate: leaderboard
MAC-Bench (2026) tests whether LLM agents follow procedural and regulatory rules drawn from GDPR, PIPL, the EU AI Act and security standards while completing tasks in sandboxed environments under social-engineering pressure; here each model runs inside the AutoGen multi-agent framework under combined authority and urgency pressure. The board is the task success rate in percent regardless of compliance; higher is better.
Metric: Task success rate (%) regardless of compliance; 4,128 scenarios synthesized from 847 atomic legal and security rules (GDPR, PIPL, EU AI Act, CWE, OWASP, CIS) in sandboxed environments, under combined high pressure (authority plus urgency), models orchestrated by the AutoGen multi-agent framework with a shared compliance charter, temperature 0.7; higher is better. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | AutoGen + GPT-5 | 98.2 | |
| 2 | AutoGen + Gemini-3 (MAC-Bench checkpoint unspecified) | 97.1 | |
| 3 | AutoGen + GPT-4o | 96.8 | |
| 4 | AutoGen + Claude-3.5 (MAC-Bench checkpoint unspecified) | 94.5 | |
| 5 | AutoGen + DeepSeek-V3 | 93.9 | |
| 6 | AutoGen + Gemini-2.5 (MAC-Bench checkpoint unspecified) | 93.7 | |
| 7 | AutoGen + Qwen-3-32B | 93.2 | |
| 8 | AutoGen + Claude-3 (MAC-Bench checkpoint unspecified) | 91.4 | |
| 9 | AutoGen + Qwen-3-8B | 90.6 | |
| 10 | AutoGen + Llama-3.1-70B | 87.1 |
Interactive version: theaggregate.ai/benchmark?slug=mac-bench-autogen-success-rate · How It Works · Data refreshed daily, snapshot 2026-11-13.