A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Обзор безопасности и конфиденциальности больших языковых моделей (LLM): хорошее, плохое и ужасное
2024-02-29
SCID: 54.1/gg5c598t
Discuss with AI
LLM security and privacycode vulnerability detectiondata confidentiality protectionsafe instruction tuninguser-level attacks
Figures from the paper
Abstract (AI)
Large Language Models (LLMs), such as ChatGPT and Bard, have revolutionized natural language understanding and generation. They possess deep language comprehension, human-like text generation capabilities, contextual awareness, and robust problem-solving skills, making them invaluable in various domains (e.g., search engines, customer support, translation). In the meantime, LLMs have also gained traction in the security community, revealing security vulnerabilities and showcasing their potential in security-related tasks. This paper explores the intersection of LLMs with security and privacy. Specifically, we investigate how LLMs positively impact security and privacy, potential risks and threats associated with their use, and inherent vulnerabilities within LLMs. Through a comprehensive literature review, the paper categorizes the papers into “The Good” (beneficial LLM applications), “The Bad” (offensive applications), and “The Ugly” (vulnerabilities of LLMs and their defenses). We have some interesting findings. For example, LLMs have proven to enhance code security (code vulnerability detection) and data privacy (data confidentiality protection), outperforming traditional methods. However, they can also be harnessed for various attacks (particularly user-level attacks) due to their human-like reasoning abilities. We have identified areas that require further research efforts. For example, Research on model and parameter extraction attacks is limited and often theoretical, hindered by LLM parameter scale and confidentiality. Safe instruction tuning, a recent development, requires more exploration. We hope that our work can shed light on the LLMs’ potential to both bolster and jeopardize cybersecurity.
Key Findings
1
LLMs can improve code security through vulnerability detection and enhance data privacy through data confidentiality protection, outperforming traditional methods.
2
LLMs’ human-like reasoning capabilities enable various attacks, particularly user-level attacks, creating significant security and privacy risks.
3
Research on model and parameter extraction attacks remains limited and often theoretical because of LLM parameter scale and confidentiality.
4
Safe instruction tuning is an emerging defense area that requires substantially more investigation.
5
The survey categorizes LLM security and privacy research into beneficial applications (“The Good”), offensive uses (“The Bad”), and vulnerabilities with defenses (“The Ugly”).
Research Object
Large Language Models (LLMs)
Research Subject
their impacts on security and privacy, including beneficial applications, offensive uses, inherent vulnerabilities, and defenses
Publication Details
Publication Date
2024-02-29
Journal
Publisher
ISSN
Cited by
1078
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai20
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer2019
A Survey of Large Language Models2026
A Survey on Evaluation of Large Language Models2024
Training language models to follow instructions with human feedback2022
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT2023
BloombergGPT: A Large Language Model for Finance2023
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing2021
ChatGPT for Education and Research: Opportunities, Threats, and Strategies2023
Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers2023
ChatGPT in higher education: Considerations for academic integrity and student learning2023
The future landscape of large language models in medicine2023
Large Language Models for Software Engineering: A Systematic Literature Review2024
From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy2023
Prompt Injection attack against LLM-integrated Applications2023
Academic Integrity considerations of AI Large Language Models in the post-pandemic era: ChatGPT and beyond2023
A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage2023
Large Language Models for Software Engineering: Survey and Open Problems2023
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT2023
Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model2022
Cited by11
Unleashing the potential of prompt engineering for large language models2025
Large Language Models for Cyber Security: A Systematic Literature Review2025
When LLMs meet cybersecurity: a systematic literature review2025
Large Language Models for Mental Health Applications: Systematic Review2024
AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways2025
Application of Large Language Models in Cybersecurity: A Systematic Literature Review2024
The Importance of AI Data Governance in Large Language Models2025
Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Review2026
LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks2026
Bridging urban theory and artificial intelligence: a multi-agent recommendation system for sustainable city development2026
A Comparative Survey of Security Risks in AI Systems: From LLMs to AI Agents and Embodied Agents2026