# AI Security ### Architecture of LLMs and AI Agents <!-- .slide: class="title-slide" --> ## Overview ### About * Understanding Large Language Models (LLMs) * How AI agents interact with tools Notes: But first, we will explore the underlying architecture of LLMs and AI Agents and discover the security implications that come with them. ## Large Language Models ### Processing Natural Language * Models process input non-deterministically * Instructions and user data are passed together as natural language * No strict syntactic separation Notes: Unlike traditional web applications where code and data have a clear syntactic boundary, AI models process inputs non-deterministically. Both the developer's instructions and the user's data are passed as natural language, meaning there is no strict separation. This fundamental difference makes AI vulnerabilities harder to mitigate with traditional rules. ### Tokens * LLMs process natural language using tokens * Input is broken down into chunks for processing Notes: We will explore how Large Language Models process natural language using tokens. Tokens are the fundamental units of data the model understands. ### Training Models * Models memorize vast amounts of training data * Susceptible to data poisoning Notes: LLMs are trained on massive datasets. They memorize vast amounts of this data, which can sometimes be regurgitated if an attacker crafts a specific query. Training datasets can also be poisoned by attackers to introduce hidden backdoors that activate upon seeing specific trigger words. ## AI Agents ### What is an Agent? * Models wrapped in code * Enables interaction with external environments * Expands the attack surface significantly Notes: "Agents" are built by wrapping Large Language Models in code. This allows the models to interact with external tools, APIs, and databases. This capability is what transforms a static text generator into an active agent, which expands the attack surface significantly. ### Interacting with External Tools * Agents can be granted access to: * Databases * Web-browsing plugins * Internal APIs Notes: Agents can be equipped with plugins, such as web-browsing capabilities. This allows them to fetch external URLs or read internal network addresses, introducing severe risks like Server-Side Request Forgery (SSRF) if manipulated. ## Security Implications ### Blurred Lines * Traditional injection: Clear code/data boundary * AI injection: Natural language processing blurs the lines * The core failure is still **trusting user input** Notes: There are strong similarities between traditional vulnerabilities and AI security concerns, as both stem from failing to never trust user input. However, the primary difference is the blurred lines between functionality and data in the case of AI. ### Over-privileged Agents * Agents executing unauthorized, high-impact actions * Prompt injection can compromise the agent * Agent 'wears' privileges given to it Notes: If an AI agent is granted broad permissions to access internal databases or user accounts, an attacker who successfully compromises the AI can 'wear' the agent's privileges to execute unauthorized, high-impact actions. ## Epilogue ### Summary * LLMs process text as tokens non-deterministically * Agents connect LLMs to external tools and APIs * This architecture introduces blurred code/data boundaries Notes: To summarize, understanding how LLMs process tokens and how Agents are built to interact with APIs is crucial. These architectures blur the lines between instructions and data, creating complex new attack surfaces that are fundamentally different from traditional web vulnerabilities.