Prompt-Jacking: The Rise of a New Supply Chain Risk - Kasimir Schulz & Kenneth Yeung
The talk details how AI coding assistants and agentic systems introduce novel supply chain risks by allowing attackers to compromise developers through hidden text or malicious prompts. Vulnerabilities are often found not in traditional code flaws but in the design of agents themselves. Key attack vectors include indirect prompt injection via raw file data (e.g., READMEs), exploiting tool call responses, manipulating control sequences, and abusing structured formats like JSON within automated pipelines.
Key takeaways
-
Inspect Raw Data Sources
2:00
When reviewing codebases or documentation, always check the raw markdown/HTML data (e.g., a README file) rather than just the pre-rendered view, as malicious payloads can be hidden in comments or unrendered sections.
-
Guardrails are Not Enough
10:30
While guardrails (LLM as a judge, classification models) are useful, they can be bypassed by advanced techniques like recursive prompt injection or token manipulation. Security must address architectural weaknesses beyond just the prompt.
-
Separate Control and Data Planes
17:45
A critical defense is ensuring a strict separation between system instructions (the control plane) and user/tool input data (the data plane). LLMs often fail to distinguish between these two, making this separation crucial for security.
-
Audit All Skills and Tools
28:00
When using agentic skills or tools (e.g., in OpenWebUI), audit the skill's name, description, and parameters, as these are often inserted into the system prompt at a high privilege level and can be exploited for injection.