A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness

Всесторонний обзор малых языковых моделей в эпоху больших языковых моделей: методы, усовершенствования, приложения, взаимодействие с большими языковыми моделями и надежность
Fali Wang, Zhiwei Zhang, Xianren Zhang, Zongyu Wu, Tzuhao Mo, Qiuhao Lu, Wanjing Wang, Rui Li, Junjie Xu, Xianfeng Tang, Qi He, Yao Ma, Ming S. Huang, Suhang Wang
2025-09-18

Large language modelsLightweight fine-tuningResource-constrained environmentsSmall language modelsTrustworthiness
Large language models (LLMs) have demonstrated emergent abilities in text generation, question answering, and reasoning, facilitating various tasks and domains. Despite their proficiency in various tasks, LLMs like PaLM 540B and Llama-3.1 405B face limitations due to large parameter sizes and computational demands, often requiring cloud API use, which raises privacy concerns, limits real-time applications on edge devices, and increases fine-tuning costs. Additionally, LLMs often underperform in specialized domains such as healthcare and law due to insufficient domain-specific knowledge, necessitating specialized models. Therefore, Small Language Models (SLMs) are increasingly favored for their low inference latency, cost-effectiveness, efficient development, and easy customization and adaptability. These models are particularly well-suited for resource-limited environments and domain knowledge acquisition, addressing LLMs’ challenges and proving ideal for applications that require localized data handling for privacy, minimal inference latency for efficiency, and domain knowledge acquisition through lightweight fine-tuning. The rising demand for SLMs has spurred extensive research and development. However, a comprehensive survey investigating issues related to the definition, acquisition, application, enhancement, and reliability of SLM remains lacking, prompting us to conduct a detailed survey on these topics. The definition of SLMs varies widely; thus, to standardize, we propose defining SLMs by their capability to perform specialized tasks and suitability for resource-constrained settings, setting boundaries based on the minimal size for emergent abilities and the maximum size sustainable under resource constraints. For other aspects, we provide a taxonomy of relevant models/methods and develop general frameworks for each category to enhance and utilize SLMs effectively. We have compiled the collected SLM models and related methods on GitHub: https://github.com/FairyFali/SLMs-Survey .
1
Small language models offer lower inference latency, reduced costs, easier customization, and better suitability for resource-constrained, privacy-sensitive, and domain-specific applications.
2
The authors compile small language models and related methods in a publicly available GitHub repository.
3
The paper proposes defining small language models by specialized-task capability and suitability for resource-constrained settings, with boundaries based on emergent-ability and sustainability limits.
4
The survey develops taxonomies and general frameworks covering small-language-model definition, acquisition, enhancement, applications, collaboration with LLMs, and trustworthiness.
5
The survey identifies computational cost, privacy risks, edge-device limitations, fine-tuning expense, and weak domain specialization as key limitations of large language models.

small language models (SLMs)

their definition, acquisition, applications, enhancement, collaboration with large language models, and trustworthiness, especially for specialized tasks and resource-constrained settings

Publication Details
Publication Date
2025-09-18
Journal
Publisher
ISSN
Access Type
Author Information
Authors
Fali Wang
Zhiwei Zhang
Xianren Zhang
Zongyu Wu
Tzuhao Mo
Qiuhao Lu
Wanjing Wang
Rui Li
Junjie Xu
Xianfeng Tang
Qi He
Yao Ma
Ming S. Huang
Suhang Wang
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%