From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review
От рассуждений больших языковых моделей к автономным ИИ-агентам: всесторонний обзор
2026-01-01
SCID: 54.1/yea5b3t5
Discuss with AI
AI-agent frameworksautonomous AI agentsevaluation benchmarkslarge language modelsmulti-agent collaboration protocols
Figures from the paper
Abstract (AI)
Large language models and autonomous AI agents have evolved rapidly, resulting in a diverse array of evaluation benchmarks, frameworks, and collaboration protocols. Driven by the growing need for standardized evaluation and integration, we systematically consolidate these fragmented efforts into a unified framework. However, the landscape remains fragmented and lacks a unified taxonomy or comprehensive survey. Therefore, we present a side-by-side comparison of benchmarks developed between 2019 and 2025 that evaluate these models and agents across multiple domains. In addition, we propose a taxonomy of approximately 60 benchmarks that cover general and academic knowledge reasoning, mathematical problem-solving, code generation and software engineering, factual grounding and retrieval, domain-specific evaluations, multimodal and embodied tasks, task orchestration, and interactive assessments. Furthermore, we review AI-agent frameworks introduced between 2023 and 2025 that integrate large language models with modular toolkits to enable autonomous decision-making and multi-step reasoning. Moreover, we present real-world applications of autonomous AI agents in materials science, biomedical research, academic ideation, software engineering, synthetic data generation, chemical reasoning, mathematical problem-solving, geographic information systems, multimedia, healthcare, and finance. We then survey key agent-to-agent collaboration protocols, namely the Agent Communication Protocol (ACP), the Model Context Protocol (MCP), and the Agent-to-Agent Protocol (A2A). Finally, we discuss recommendations for future research, focusing on advanced reasoning strategies, failure modes in multi-agent LLM systems, automated scientific discovery, dynamic tool integration via reinforcement learning, integrated search capabilities, and security vulnerabilities in agent protocols.
Key Findings
1
It provides a taxonomy of approximately 60 benchmarks spanning reasoning, mathematics, coding, factual grounding, specialized domains, multimodal tasks, orchestration, and interaction.
2
It surveys AI-agent frameworks introduced from 2023 to 2025 that combine large language models with modular toolkits for autonomous decision-making and multi-step reasoning.
3
The review consolidates fragmented evaluation benchmarks, agent frameworks, and collaboration protocols for large language models and autonomous AI agents.
4
The review identifies applications of autonomous agents across scientific research, software, healthcare, finance, multimedia, and other domains, and highlights future priorities including failure modes, tool integration, discovery, and protocol security.
5
The study compares benchmarks developed from 2019 to 2025 across multiple evaluation domains to support standardized assessment.
Research Object
large language models and autonomous AI agents
Research Subject
their evaluation benchmarks, taxonomies, frameworks, applications, collaboration protocols, reasoning capabilities, and integration for autonomous decision-making
Publication Details
Publication Date
2026-01-01
Journal
Publisher
ISSN
Cited by
268
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai10
Generative Agents: Interactive Simulacra of Human Behavior2023
GPT-4 Technical Report2023
Evaluating Large Language Models Trained on Code2021
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models2022
Retrieval-Augmented Generation for AI-Generated Content: A Survey2026
AgentClinic: a multimodal benchmark for tool-using clinical AI agents2026
AgentTuning: Enabling Generalized Agent Abilities for LLMs2024
A review of large language models and autonomous agents in chemistry2024
Accelerating scientific discovery with Co-Scientist2026
DataSciBench: An LLM Agent Benchmark for Data Science2026
Cited by7
From language to action: a review of large language models as autonomous agents and tool users2026
From Large AI Models to Agentic AI: A Tutorial on Future Intelligent Communications2026
Agentic AI in Healthcare and Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-Based Agents2026
Artificial Intelligence agents for biological research: a survey2026
A self-correcting Agentic Graph RAG for clinical decision support in hepatology2025
A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applications2026
LLM-Enabled Multi-Agent Systems: Empirical Evaluation and Insights into Emerging Design Patterns & Paradigms2026