InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents

InjecAgent: бенчмаркинг косвенных внедрений промптов в агентных системах на основе больших языковых моделей с интеграцией инструментов
Zhixiang Liang, Qiusi Zhan, Zifan Ying, Daniel D. Kang, Qiusi Zhan
2024-01-01

INJECAGENT benchmarkattack success rateindirect prompt injectionprivate data exfiltrationtool-integrated LLM agents
Recent work has embodied LLMs as agents, allowing them to access tools, perform actions, and interact with external content (e.g., emails or websites).However, external content introduces the risk of indirect prompt injection (IPI) attacks, where malicious instructions are embedded within the content processed by LLMs, aiming to manipulate these agents into executing detrimental actions against users.Given the potentially severe consequences of such attacks, establishing benchmarks to assess and mitigate these risks is imperative.In this work, we introduce INJECAGENT, a benchmark designed to assess the vulnerability of tool-integrated LLM agents to IPI attacks.INJECAGENT comprises 1,054 test cases covering 17 different user tools and 62 attacker tools.We categorize attack intentions into two primary types: direct harm to users and exfiltration of private data.We evaluate 30 different LLM agents and show that agents are vulnerable to IPI attacks, with ReAct-prompted GPT-4 vulnerable to attacks 24% of the time.Further investigation into an enhanced setting, where the attacker instructions are reinforced with a hacking prompt, shows additional increases in success rates, nearly doubling the attack success rate on the ReAct-prompted GPT-4.Our findings raise questions about the widespread deployment of LLM Agents.
1
Attack intentions are categorized into direct harm to users and exfiltration of private data.
2
Evaluation of 30 LLM agents demonstrates broad vulnerability to indirect prompt injection attacks.
3
INJECAGENT introduces a benchmark for evaluating indirect prompt injection vulnerabilities in tool-integrated LLM agents.
4
ReAct-prompted GPT-4 was successfully attacked in 24% of cases, with success nearly doubling when attacker instructions included a reinforcing hacking prompt.
5
The benchmark contains 1,054 test cases spanning 17 user tools and 62 attacker tools.

tool-integrated large language model agents exposed to indirect prompt injection attacks through external content

vulnerability to indirect prompt injection attacks, including harmful action execution and private-data exfiltration

Publication Details
Publication Date
2024-01-01
Journal
Publisher
ISSN
Cited by
143
Access Type
Author Information
Authors
Zhixiang Liang
Qiusi Zhan
Zifan Ying
Daniel D. Kang
Qiusi Zhan
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%