Survey on LLM Safety: Attacks, Defenses, Alignment, Metrics, and Guardrails

Обзор безопасности больших языковых моделей: атаки, методы защиты, согласование, метрики и механизмы контроля
Pratik Jalan, Vadivel Abishethvarman, Bhavik Chandna, Usman Naseem
2026-05-20

LLM safetyadversarial attacksevaluation metricsguarding mechanismssafety alignment
Abstract Large Language Models (LLM) have demonstrated remarkable capabilities across various applications, but their deployment raises critical safety concerns as potential misuse poses significant societal risks. This survey reviews the end-to-end security and safety pipeline of LLMs, focusing on the interaction between users and model responses. We categorize the system into five key components: attacks, defenses, safety alignment, metrics and guarding mechanisms. Attacks involve crafting adversarial inputs to exploit model vulnerabilities. Defenses act as countermeasures, aiming to detect and prevent such inputs before processing. Safety alignment ensures that, even when attacks reach the model, its responses remain consistent with ethical and policy-aligned behavior. Guarding mechanisms operate post-response to flag, filter, or block unsafe outputs. Each of these stages is subject to rigorous evaluation metrics to assess their effectiveness, robustness, and limitations. As LLMs evolve toward more general-purpose intelligence, these safety considerations become increasingly critical for the development of robust and trustworthy AI systems. Finally, we highlight open challenges and future research directions to advance the security and alignment of LLMs.
1
Attacks exploit model vulnerabilities through adversarial inputs, while defenses aim to detect and prevent these inputs before they reach the model.
2
Guarding mechanisms assess model outputs after generation to identify, filter, or block unsafe responses.
3
Safety alignment is intended to maintain ethical and policy-consistent model behavior even when adversarial inputs bypass preventive defenses.
4
The survey emphasizes that effectiveness, robustness, and limitations of every safety stage require rigorous evaluation, while identifying open challenges for trustworthy general-purpose LLMs.
5
The survey presents an end-to-end framework for LLM safety spanning attacks, defenses, safety alignment, evaluation metrics, and post-response guarding mechanisms.

large language models (LLMs) and their end-to-end security and safety pipeline

attacks, defenses, safety alignment, evaluation metrics, and guarding mechanisms for maintaining robust, trustworthy, and policy-aligned model behavior

Publication Details
Publication Date
2026-05-20
Journal
Publisher
ISSN
Access Type
Author Information
Authors
Pratik Jalan
Vadivel Abishethvarman
Bhavik Chandna
Usman Naseem
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%