# AI Security Engineer Foundations + Certificate — Javier Garza, Snyk

## Executive summary

This workshop provides a comprehensive overview of AI security engineering, detailing the shift from traditional software security models to address the unique risks posed by Large Language Models (LLMs) and agentic workflows. Key areas covered include the OWASP Top 10 for LLMs, securing the supply chain of AI components (AI Bomb), and implementing proactive AI-native threat modeling. The session emphasizes that security must be integrated early in the design lifecycle, requiring continuous monitoring, strict guardrails, and human-in-the-loop processes to manage non-deterministic AI behavior.

## Key takeaways

- AI Systems are Non-Deterministic: Unlike traditional software, AI systems are probabilistic. This necessitates a fundamental shift in threat modeling, moving from reactive scanning to proactive, AI-centric design and continuous iteration.
- AI Bomb (CBEM) is Essential for Inventory: An AI Bill of Materials (AI Bomb) is required to create a searchable inventory of all AI assets—including models, datasets, agents, and pipelines—to track vulnerabilities and impact in the event of a supply chain attack.
- Defense Requires Guardrails and Education: Technical controls (like sanitizing inputs and using function calls) must be paired with organizational guardrails, developer training, and clear policies to prevent workarounds and misuse.
- Secure Development Requires Hooks and CI/CD Integration: Security checks must be integrated into the development pipeline (CI/CD) using hooks and agent rules to automatically scan generated code and prevent vulnerabilities before deployment.

## Technical details

- OWASP Top 10 for LLMs: Key risks include Prompt Injection (manipulating LLM behavior, e.g., using 'Gandalf' techniques), Sensitive Information Disclosure (LLMs revealing proprietary data), and Supply Chain Vulnerabilities (attacking open-source libraries or third-party models).
- Prompt Injection Techniques: Attacks can be direct (manipulating the system prompt) or indirect (embedding malicious text, e.g., white text on a white background, into external content like a LinkedIn profile). Defenses include sanitizing input and requiring human approval for high-risk actions.
- Agent Architecture Components: An Agent is the interactive tool; the LLM is the 'brain' (the large language model); and an MCP (Model Communication Protocol) tool enhances the agent's abilities by connecting it to external APIs for real-time data (e.g., weather).
- MCP Security and Tool Poisoning: MCP servers are susceptible to attacks like tool poisoning, where hidden instructions are embedded to execute malicious actions (e.g., sending API tokens). Developers must treat MCP servers like any other code and use specialized tools to scan for malware.
- AI Threat Modeling: This is a proactive process that identifies, analyzes, and mitigates threats early in the design lifecycle. It requires defining trust boundaries for AI inputs (e.g., external data from an API) and ensuring data flow diagrams are updated as the architecture evolves.

## Practical implications

- Implement automated security hooks and scans within the CI/CD pipeline to vet code generated by AI tools.
- Adopt an AI Bomb methodology to maintain a continuous, comprehensive inventory of all LLMs, datasets, and agents used in the application.
- Enforce strict access controls and use read-only database views when connecting agents to internal data sources.
- Mandate 'human-in-the-loop' approvals for high-risk actions (e.g., production deployment, database deletion) to mitigate excessive agency.

## Topics

AI Security Engineering, LLM Vulnerabilities, Threat Modeling, Supply Chain Security, Agentic Workflows, DevSecOps, aiccurity.engineer, snyk.io, gandalf.lacera.ai

Source: https://www.youtube.com/watch?v=ZyC56eV8sVw