Survey on LLM Safety: Attacks, Defenses, Alignment, Metrics, and Guardrails
Обзор безопасности больших языковых моделей: атаки, методы защиты, согласование, метрики и механизмы контроля
2026-05-20
SCID: 54.1/5xhjgv5p
Discuss with AI
LLM safetyadversarial attacksevaluation metricsguarding mechanismssafety alignment
Figures from the paper
Abstract (AI)
Abstract Large Language Models (LLM) have demonstrated remarkable capabilities across various applications, but their deployment raises critical safety concerns as potential misuse poses significant societal risks. This survey reviews the end-to-end security and safety pipeline of LLMs, focusing on the interaction between users and model responses. We categorize the system into five key components: attacks, defenses, safety alignment, metrics and guarding mechanisms. Attacks involve crafting adversarial inputs to exploit model vulnerabilities. Defenses act as countermeasures, aiming to detect and prevent such inputs before processing. Safety alignment ensures that, even when attacks reach the model, its responses remain consistent with ethical and policy-aligned behavior. Guarding mechanisms operate post-response to flag, filter, or block unsafe outputs. Each of these stages is subject to rigorous evaluation metrics to assess their effectiveness, robustness, and limitations. As LLMs evolve toward more general-purpose intelligence, these safety considerations become increasingly critical for the development of robust and trustworthy AI systems. Finally, we highlight open challenges and future research directions to advance the security and alignment of LLMs.
Key Findings
1
Attacks exploit model vulnerabilities through adversarial inputs, while defenses aim to detect and prevent these inputs before they reach the model.
2
Guarding mechanisms assess model outputs after generation to identify, filter, or block unsafe responses.
3
Safety alignment is intended to maintain ethical and policy-consistent model behavior even when adversarial inputs bypass preventive defenses.
4
The survey emphasizes that effectiveness, robustness, and limitations of every safety stage require rigorous evaluation, while identifying open challenges for trustworthy general-purpose LLMs.
5
The survey presents an end-to-end framework for LLM safety spanning attacks, defenses, safety alignment, evaluation metrics, and post-response guarding mechanisms.
Research Object
large language models (LLMs) and their end-to-end security and safety pipeline
Research Subject
attacks, defenses, safety alignment, evaluation metrics, and guarding mechanisms for maintaining robust, trustworthy, and policy-aligned model behavior
Publication Details
Publication Date
2026-05-20
Journal
Publisher
ISSN
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest