TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems
TAMAS: сравнительный анализ рисков состязательных атак в многоагентных системах на основе больших языковых моделей
2026-01-01
SCID: 54.1/te7uwyj6
Discuss with AI
Effective Robustness ScoreTAMAS benchmarkadversarial attacksagent interaction configurationsmulti-agent LLM systems
Figures from the paper
Abstract (AI)
Large Language Models (LLMs) have demonstrated strong capabilities as autonomous agents through tool use, planning, and decisionmaking abilities, leading to their widespread adoption across diverse tasks.As task complexity grows, multi-agent LLM systems are increasingly used to solve problems collaboratively.However, safety and security of these systems remains largely under-explored.Existing benchmarks and datasets predominantly focus on single-agent settings, failing to capture the unique vulnerabilities of multi-agent dynamics and co-ordination.To address this gap, we introduce Threats and Attacks in Multi-Agent Systems (TAMAS), a benchmark designed to evaluate the robustness and safety of multi-agent LLM systems.TAMAS includes five distinct scenarios comprising 300 adversarial instances across six attack types and 211 tools, along with 100 harmless tasks.We assess system performance across ten backbone LLMs and three agent interaction configurations from Autogen and CrewAI frameworks, highlighting critical challenges and failure modes in current multi-agent deployments.Furthermore, we introduce Effective Robustness Score (ERS) to assess the tradeoff between safety and task effectiveness of these frameworks.Our findings show that multi-agent systems are highly vulnerable to adversarial attacks, underscoring the urgent need for stronger defenses.TAMAS provides a foundation for systematically studying and improving the safety of multi-agent LLM systems.Code and dataset is available at https://github.com/microsoft/TAMAS.
Key Findings
1
Experiments show that current multi-agent LLM systems are highly vulnerable to adversarial attacks, highlighting the need for stronger defenses.
2
TAMAS evaluates ten backbone LLMs and three agent interaction configurations from AutoGen and CrewAI, exposing critical vulnerabilities and failure modes.
3
TAMAS introduces a benchmark for evaluating adversarial robustness and safety in multi-agent LLM systems, addressing gaps in predominantly single-agent evaluations.
4
The benchmark contains five scenarios, 300 adversarial instances spanning six attack types and 211 tools, plus 100 harmless tasks.
5
The proposed Effective Robustness Score measures the tradeoff between multi-agent systems’ safety and task effectiveness.
Research Object
multi-agent LLM systems
Research Subject
robustness and safety against adversarial attacks, including system vulnerabilities, failure modes, and the tradeoff between safety and task effectiveness
Publication Details
Publication Date
2026-01-01
Journal
Publisher
ISSN
Cited by
43
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai4
GPT-4 Technical Report2023
Prompt Injection attack against LLM-integrated Applications2023
Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection2023
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents2024