The Video Signal

Topic

LLM Guardrails

All digests tagged LLM Guardrails

· 1:01:49

Prompt-Jacking: The Rise of a New Supply Chain Risk - Kasimir Schulz & Kenneth Yeung

The talk details how AI coding assistants and agentic systems introduce novel supply chain risks by allowing attackers to compromise developers through hidden text or malicious prompts. Vulnerabilities are often found not in traditional code flaws but in the design of agents themselves. Key attack vectors include indirect prompt injection via raw file data (e.g., READMEs), exploiting tool call responses, manipulating control sequences, and abusing structured formats like JSON within automated pipelines.

Key takeaways

  1. Inspect Raw Data Sources 2:00

    When reviewing codebases or documentation, always check the raw markdown/HTML data (e.g., a README file) rather than just the pre-rendered view, as malicious payloads can be hidden in comments or unrendered sections.

  2. Guardrails are Not Enough 10:30

    While guardrails (LLM as a judge, classification models) are useful, they can be bypassed by advanced techniques like recursive prompt injection or token manipulation. Security must address architectural weaknesses beyond just the prompt.

  3. Separate Control and Data Planes 17:45

    A critical defense is ensuring a strict separation between system instructions (the control plane) and user/tool input data (the data plane). LLMs often fail to distinguish between these two, making this separation crucial for security.

  4. Audit All Skills and Tools 28:00

    When using agentic skills or tools (e.g., in OpenWebUI), audit the skill's name, description, and parameters, as these are often inserted into the system prompt at a high privilege level and can be exploited for injection.

Watch on YouTube Full article

· 21:18

Why Agentic Systems Need Ontologies — Frank Coyle, UC Berkeley

The talk argues that while Large Language Models (LLMs) and agentic systems represent a powerful shift toward AI, their inherent probabilistic nature makes them unreliable for mission-critical tasks. To mitigate failures—such as incorrect order statuses or double refunds—the solution is 'Neurosymbolic AI': coupling the LLM's generative power with external, formal ontologies. An ontology acts as a logical guardrail, enforcing constraints (e.g., payment status must be one of three values) that are impossible to reliably enforce using prompt engineering alone.

Key takeaways

  1. The Problem with Probabilistic Agents

    LLMs reason probabilistically over domains they only half understand, leading to failures in brittle tools and fragile handoffs. These errors cannot be reliably stopped by mere instructions or prompt engineering.

  2. Neurosymbolic AI: Guardrails for LLMs 7:04

    This approach ties together neural networks (LLMs) with symbolic AI (rule-based systems and ontologies). The ontology provides the necessary guardrails to keep the probabilistic model constrained and honest.

  3. The Role of Ontologies 8:43

    An ontology is a formal specification of a shared conceptualization, defining typed entities, their relationships, and constraints. It allows systems to validate proposed actions (e.g., ensuring an order can only be refunded once).

Watch on YouTube Full article