LLM and AI Agents for Autonomous Systems: A Survey of Applications, Datasets, and Security Challenges

Большие языковые модели и ИИ-агенты для автономных систем: обзор приложений, наборов данных и проблем безопасности
Mohamed Amine Ferrag, Abderrahmane Lakas, Norbert Tihanyi, Merouane Abdelkader Debbah
2026-01-01

AI agentsAdversarial prompt injectionAutonomous drivingLarge Language Models (LLMs)UAV swarm autonomy
The rapid integration of Large Language Models (LLMs) into autonomous systems marks a significant transition from modular, rule-based approaches to reasoning-driven, agent-based, and multimodal intelligence. LLM reasoning enables adaptive decision-making, context-aware planning, and human-aligned interaction, while AI agents extend these capabilities into structured autonomy pipelines that coordinate perception, reasoning, and control. These advancements are particularly critical in safety-sensitive domains such as autonomous driving (AD) and unmanned aerial vehicles (UAVs). This survey provides a comprehensive review of LLM reasoning and AI agents across scenario generation, decision-making, multimodal perception, cooperative V2X interactions, and UAV swarm autonomy. We examine the role of simulation platforms and datasets, including CARLA, Apollo ADS, AirSim, nuScenes, DriveLM, and emerging synthetic environments, in supporting reproducible evaluation and benchmarking. In addition, we analyze pressing security and robustness challenges, including adversarial prompt injection, data poisoning, multimodal perturbations, privacy leakage, and vulnerabilities in cooperative agent communication. Finally, we propose future research directions including adversarially robust pipelines, hybrid symbolic LLM planning, secure multimodal fusion, privacy-preserving human alignment, distributed trust mechanisms for swarm autonomy, and optimized Drone-LLM deployment across on-drone, edge, and cloud environments. By unifying applications, datasets, benchmarks, reasoning, agents, and security, this survey establishes a roadmap for developing robust, trustworthy, and secure LLM-enabled autonomous systems.
1
LLM reasoning supports adaptive decision-making, context-aware planning, and human-aligned interaction, while AI agents coordinate perception, reasoning, and control.
2
LLMs are shifting autonomous systems from modular rule-based designs toward reasoning-driven, agent-based, and multimodal intelligence.
3
Major security and robustness risks include prompt injection, data poisoning, multimodal perturbations, privacy leakage, and vulnerabilities in cooperative agent communication.
4
Simulation platforms and datasets including CARLA, Apollo ADS, AirSim, nuScenes, and DriveLM support reproducible evaluation and benchmarking of LLM-enabled autonomy.
5
The survey covers applications in scenario generation, decision-making, multimodal perception, cooperative V2X interaction, and UAV swarm autonomy, especially for autonomous driving and UAVs.
6
The survey recommends robust pipelines, hybrid symbolic-LLM planning, secure multimodal fusion, privacy-preserving alignment, distributed swarm trust, and optimized on-drone, edge, and cloud deployment.

LLM-enabled autonomous systems, particularly autonomous driving and unmanned aerial vehicle systems

Applications, reasoning and agent capabilities, evaluation resources, and security and robustness challenges of integrating LLMs and AI agents into autonomous systems

Publication Details
Publication Date
2026-01-01
Journal
Publisher
ISSN
Access Type
Author Information
Authors
Mohamed Amine Ferrag
Abderrahmane Lakas
Norbert Tihanyi
Merouane Abdelkader Debbah
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%