From AI-generated content to agentic action: Security and safety threats in generative AI
От контента, сгенерированного ИИ, к агентным действиям: угрозы безопасности и надежности в генеративном искусственном интеллекте
2026-05-27
SCID: 54.1/5vr9z8ke
Discuss with AI
AI governanceagentic AIagentic threatsgenerative AI securitytool invocation
Figures from the paper
Abstract (AI)
Generative AI systems are increasingly used not only to produce content but also to retrieve data, invoke tools, and execute actions. This work examines the security and safety implications of that shift across content-level, model-level, and agentic threats. We analyze how attacker access requirements, system autonomy, and the scope of potential harm change as models move from generating artifacts to executing operations through tool chains and external APIs. We then assess technical countermeasures including detection, watermarking, alignment, and emerging agentic safeguards, and show that several depend on forms of institutional coordination that current governance arrangements do not yet provide. Across the cases examined, capability deployment and attack-surface expansion repeatedly outpace defensive responses as systems move from generating content to executing real-world actions.
Key Findings
1
Across examined cases, capability deployment and attack-surface expansion consistently outpace defensive responses as generative AI systems execute real-world actions.
2
Generative AI security risks span content-level, model-level, and agentic threats as systems increasingly retrieve data, invoke tools, and execute actions.
3
Moving from content generation to tool-mediated operations increases system autonomy, expands attack surfaces, and broadens the potential scope of harm.
4
Several proposed countermeasures depend on institutional coordination that existing governance arrangements do not yet provide.
5
The analysis evaluates detection, watermarking, alignment, and emerging agentic safeguards as technical countermeasures against these threats.
Research Object
Generative AI systems spanning content generation, data retrieval, tool invocation, and autonomous action execution
Research Subject
Security and safety threats, changing attacker access requirements, autonomy, harm scope, and the adequacy of technical and governance countermeasures across content-level, model-level, and agentic systems
Publication Details
Publication Date
2026-05-27
Journal
Publisher
ISSN
Cited by
0
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest