LLM and AI Agents for Autonomous Systems: A Survey of Applications, Datasets, and Security Challenges
Большие языковые модели и ИИ-агенты для автономных систем: обзор приложений, наборов данных и проблем безопасности
2026-01-01
SCID: 54.1/bf4h6zsf
Discuss with AI
AI agentsAdversarial prompt injectionAutonomous drivingLarge Language Models (LLMs)UAV swarm autonomy
Figures from the paper
Abstract (AI)
The rapid integration of Large Language Models (LLMs) into autonomous systems marks a significant transition from modular, rule-based approaches to reasoning-driven, agent-based, and multimodal intelligence. LLM reasoning enables adaptive decision-making, context-aware planning, and human-aligned interaction, while AI agents extend these capabilities into structured autonomy pipelines that coordinate perception, reasoning, and control. These advancements are particularly critical in safety-sensitive domains such as autonomous driving (AD) and unmanned aerial vehicles (UAVs). This survey provides a comprehensive review of LLM reasoning and AI agents across scenario generation, decision-making, multimodal perception, cooperative V2X interactions, and UAV swarm autonomy. We examine the role of simulation platforms and datasets, including CARLA, Apollo ADS, AirSim, nuScenes, DriveLM, and emerging synthetic environments, in supporting reproducible evaluation and benchmarking. In addition, we analyze pressing security and robustness challenges, including adversarial prompt injection, data poisoning, multimodal perturbations, privacy leakage, and vulnerabilities in cooperative agent communication. Finally, we propose future research directions including adversarially robust pipelines, hybrid symbolic LLM planning, secure multimodal fusion, privacy-preserving human alignment, distributed trust mechanisms for swarm autonomy, and optimized Drone-LLM deployment across on-drone, edge, and cloud environments. By unifying applications, datasets, benchmarks, reasoning, agents, and security, this survey establishes a roadmap for developing robust, trustworthy, and secure LLM-enabled autonomous systems.
Key Findings
1
LLM reasoning supports adaptive decision-making, context-aware planning, and human-aligned interaction, while AI agents coordinate perception, reasoning, and control.
2
LLMs are shifting autonomous systems from modular rule-based designs toward reasoning-driven, agent-based, and multimodal intelligence.
3
Major security and robustness risks include prompt injection, data poisoning, multimodal perturbations, privacy leakage, and vulnerabilities in cooperative agent communication.
4
Simulation platforms and datasets including CARLA, Apollo ADS, AirSim, nuScenes, and DriveLM support reproducible evaluation and benchmarking of LLM-enabled autonomy.
5
The survey covers applications in scenario generation, decision-making, multimodal perception, cooperative V2X interaction, and UAV swarm autonomy, especially for autonomous driving and UAVs.
6
The survey recommends robust pipelines, hybrid symbolic-LLM planning, secure multimodal fusion, privacy-preserving alignment, distributed swarm trust, and optimized on-drone, edge, and cloud deployment.
Research Object
LLM-enabled autonomous systems, particularly autonomous driving and unmanned aerial vehicle systems
Research Subject
Applications, reasoning and agent capabilities, evaluation resources, and security and robustness challenges of integrating LLMs and AI agents into autonomous systems
Publication Details
Publication Date
2026-01-01
Journal
Publisher
ISSN
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest