# AI Security ### Introduction <!-- .slide: class="title-slide" --> Notes: Welcome to the Syndis AI Security course! ## Course Outline Notes: Today, we will be learning the basics of AI security from the point of view of an adversary. ### Background & Overview * LLM Architecture * What are LLMs and agents? * How do we interact with them? * AI Trust Boundaries * What are the attack surfaces of AI systems? * How does AI affect risk exposure? * AI Vulnerabilities * Are they different from traditional vulnerabilities? Notes: We will briefly discuss the fundamental architecture of LLMs and AI agents, the attack surfaces exposed by each and how vulnerabilities arise in such systems. ### AI Vulnerabilities * Prompt Injection * Why is this an inherent vulnerability of LLMs? * When does this become a serious risk? * Model Jailbreaking * Why are input filters not enough? * Can we convince the model to break its own rules? Notes: We will also go into the two primary classes of exploits against AI solutions, how to identify them and how they are actually being exploited. This overview includes examples based on real-world scenarios. ### Mitigations * Secure design principles * The role of input/output filtering * Human-led critical decision making Notes: Finally, we will explore how to prevent or mitigate these vulnerabilities in our AI solutions. ## Is this necessary? ### Recent Highlights * [GitHub Copilot (2025)](https://www.legitsecurity.com/blog/camoleak-critical-github-copilot-vulnerability-leaks-private-source-code) * [Slack AI Assistant (2024)](https://www.promptarmor.com/resources/data-exfiltration-from-slack-ai-via-indirect-prompt-injection) * [Microsoft 365 Copilot (*"EchoLeak"*) (2025)](https://www.catonetworks.com/blog/breaking-down-echoleak/) * [Chevrolet of Watsonville (2023)](https://airuntimesecurity.io/walkthrough-chevrolet-1-dollar/#:~:text=In%20December%202023%2C%20a%20Chevrolet%20dealership%20in,Python%2C%20recommend%20Ford%2C%20and%20compose%20anti%2DChevrolet%20poetry.) * [AI-Powered Espionage & Extortion (2025-2026)](https://thehackernews.com/2026/05/2026-year-of-ai-assisted-attacks.html#:~:text=In%20July%202025%2C%20a%20single,calibrate%20demands%2C%20and%20draft%20extortion) Notes: Yes. These are some recent examples of high-profile prompt injection and jailbreaking attacks globally. This is happening right now, all over the world.