AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways
AI-агенты под угрозой: обзор ключевых проблем безопасности и перспективных направлений
2025-02-07
SCID: 54.1/5rvx3qbu
Discuss with AI
AI agentsinteractions with untrusted external entitiessecurity threatsunpredictability of multi-step user inputs
Figures from the paper
Abstract (AI)
An Artificial Intelligence (AI) agent is a software entity that autonomously performs tasks or makes decisions based on pre-defined objectives and data inputs. AI agents, capable of perceiving user inputs, reasoning and planning tasks, and executing actions, have seen remarkable advancements in algorithm development and task performance. However, the security challenges they pose remain under-explored and unresolved. This survey delves into the emerging security threats faced by AI agents, categorizing them into four critical knowledge gaps: unpredictability of multi-step user inputs, complexity in internal executions, variability of operational environments, and interactions with untrusted external entities. By systematically reviewing these threats, this article highlights both the progress made and the existing limitations in safeguarding AI agents. The insights provided aim to inspire further research into addressing the security threats associated with AI agents, thereby fostering the development of more robust and secure AI agent applications.
Key Findings
1
AI agents autonomously perform perception, reasoning, planning, and action, but their security challenges are under-explored and unresolved.
2
Insights from the survey are intended to motivate further research to develop more robust and secure AI agent applications.
3
The article systematically reviews existing threats and identifies both progress and persistent limitations in safeguarding AI agents.
4
The survey categorizes AI agent security threats into four critical knowledge gaps: unpredictability of multi-step user inputs, complexity in internal executions, variability of operational environments, and interactions with untrusted external entities.
Research Object
Artificial Intelligence (AI) agents (software entities that autonomously perceive, reason, plan, and act)
Research Subject
Security threats, vulnerabilities, and defense challenges affecting AI agents, including unpredictability from multi-step user inputs, complex internal executions, variable operational environments, and interactions with untrusted external entities
Publication Details
Publication Date
2025-02-07
Journal
Publisher
ISSN
Cited by
183
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai9
Training language models to follow instructions with human feedback2022
Generative Agents: Interactive Simulacra of Human Behavior2023
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly2024
From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy2023
ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health2023
Tree of Thoughts: Deliberate Problem Solving with Large Language Models2023
Retrieval Augmentation Reduces Hallucination in Conversation2021
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents2024
A Brief Survey of Vector Databases2023