Doors of (AI)pportunity: The Front and Backdoors of LLMs - Kasimir Schulz & Kenneth Yeung
The talk provides an advanced deep dive into LLM security vulnerabilities, moving beyond simple prompt injection attacks. Speakers demonstrate sophisticated methods including Knowledge Return Oriented Prompting (KROP), Policy Puppetry, and direct manipulation of the model's computational graph to implant backdoors. The discussion highlights that even seemingly safe formats (like ONNX) can be exploited if the underlying architecture is compromised, posing significant supply chain risks for AI-driven applications.
Key takeaways
-
Multiple Layers of Defense are Required
35:35
System prompts alone are insufficient protection. Developers must implement multiple layers of defense to mitigate attacks, as demonstrated by the failure of simple system prompt rules against advanced exploits.
-
Indirect Prompt Injection is a Major Threat
43:25
Attacks do not require direct user interaction. Compromise can occur through injected text in external sources like customer reviews, emails (especially those processed by Retrieval Augmented Generation/RAG systems), or calendar invites.
-
Model Backdoors are Difficult to Detect
50:15
Backdoors can be implanted into the computational graph of a model (e.g., changing an animal classification logic based on a specific color trigger). These backdoors can remain dormant until a precise trigger condition is met, making them highly effective and difficult to detect.
