From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review

От рассуждений больших языковых моделей к автономным ИИ-агентам: всесторонний обзор
Merouane Abdelkader Debbah, Mohamed Amine Ferrag, Norbert Tihanyi
2026-01-01

AI-agent frameworksautonomous AI agentsevaluation benchmarkslarge language modelsmulti-agent collaboration protocols
Large language models and autonomous AI agents have evolved rapidly, resulting in a diverse array of evaluation benchmarks, frameworks, and collaboration protocols. Driven by the growing need for standardized evaluation and integration, we systematically consolidate these fragmented efforts into a unified framework. However, the landscape remains fragmented and lacks a unified taxonomy or comprehensive survey. Therefore, we present a side-by-side comparison of benchmarks developed between 2019 and 2025 that evaluate these models and agents across multiple domains. In addition, we propose a taxonomy of approximately 60 benchmarks that cover general and academic knowledge reasoning, mathematical problem-solving, code generation and software engineering, factual grounding and retrieval, domain-specific evaluations, multimodal and embodied tasks, task orchestration, and interactive assessments. Furthermore, we review AI-agent frameworks introduced between 2023 and 2025 that integrate large language models with modular toolkits to enable autonomous decision-making and multi-step reasoning. Moreover, we present real-world applications of autonomous AI agents in materials science, biomedical research, academic ideation, software engineering, synthetic data generation, chemical reasoning, mathematical problem-solving, geographic information systems, multimedia, healthcare, and finance. We then survey key agent-to-agent collaboration protocols, namely the Agent Communication Protocol (ACP), the Model Context Protocol (MCP), and the Agent-to-Agent Protocol (A2A). Finally, we discuss recommendations for future research, focusing on advanced reasoning strategies, failure modes in multi-agent LLM systems, automated scientific discovery, dynamic tool integration via reinforcement learning, integrated search capabilities, and security vulnerabilities in agent protocols.
1
It provides a taxonomy of approximately 60 benchmarks spanning reasoning, mathematics, coding, factual grounding, specialized domains, multimodal tasks, orchestration, and interaction.
2
It surveys AI-agent frameworks introduced from 2023 to 2025 that combine large language models with modular toolkits for autonomous decision-making and multi-step reasoning.
3
The review consolidates fragmented evaluation benchmarks, agent frameworks, and collaboration protocols for large language models and autonomous AI agents.
4
The review identifies applications of autonomous agents across scientific research, software, healthcare, finance, multimedia, and other domains, and highlights future priorities including failure modes, tool integration, discovery, and protocol security.
5
The study compares benchmarks developed from 2019 to 2025 across multiple evaluation domains to support standardized assessment.

large language models and autonomous AI agents

their evaluation benchmarks, taxonomies, frameworks, applications, collaboration protocols, reasoning capabilities, and integration for autonomous decision-making

Publication Details
Publication Date
2026-01-01
Journal
Publisher
ISSN
Cited by
268
Access Type
Author Information
Authors
Merouane Abdelkader Debbah
Mohamed Amine Ferrag
Norbert Tihanyi
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%