AI & Cybersecurity

Currently Playing: Chapter 5.2 - Red Teaming Agentic AI - Part 2

Welcome to Chapter 5.2 of the AI & Cybersecurity Learning Series by KK Mookhey! Part 2 of our deep dive into red teaming agentic AI systems continues with the Cloud Security Alliance's framework. This chapter focuses on external dependencies—what happens when AI agents access external data sources and third-party tools? We explore three critical attack vectors with real-world implications and consequences. What You'll Learn: Three Critical Attack Vectors: • Data Poisoning: Corrupting knowledge bases to cause harmful decisions • Hallucination Exploitation: Manipulating agents to act on fabricated data • Supply Chain Attacks: Compromising third-party dependencies Real-World Context: • Microsoft's Twitter bot becoming abusive within 24 hours • Air Canada legally bound to honor hallucinated policies • Robo-taxi failures causing pedestrian injuries • Malware in Hugging Face language models • Replit database deletion incident Technical Architecture: • Framework: LangChain with experimental Pandas DataFrame agent • Data Manipulation: Pandas library for CSV processing • Agent Structure: Embedded agents within tool definitions • External Dependencies: Custom Python scripts (status checker) • Experimental Features: allow_dangerous_code=True (demonstration only) Critical Insights: • Real-world AI failures happen daily with serious consequences • Legal liability for hallucinated commitments (Air Canada case) • Data poisoning can cause catastrophic automated actions • Hallucinations aren't just wrong answers—they trigger wrong actions • Supply chain security for AI is as critical as traditional software • Human-in-the-loop is essential for high-impact operations • Multi-layered guardrails provide defense-in-depth • Experimental code features require extra caution in production What's Next? Chapter 5.3 will explore: Critical system interactions Resource exhaustion attacks Secure architecture patterns Code Repository: Complete IT asset management agent code available in Google Doc - https://docs.google.com/document/d/1NkGZthHxy3QWdkyeDKC_TB2gFOk2H0n-BKZVZjXwq-Q/edit?tab=t.6ypecfaymbgh About the Instructor: KK Mookhey applies 25+ years of cybersecurity expertise to teach real-world AI security, grounded in frameworks like the CSA Agentic AI Red Teaming Guide. Connect with KK on https://www.linkedin.com/in/kkmookhey/ Course Series Progress: This is Chapter 5.2 of our AI & Cybersecurity Learning Series. Chapter 1 to 4: https://www.youtube.com/watch?v=caSd12M5Axk&list=PLXVUBNOa2d7YyqWr_DgUHw7RwQLE7P24m&index=5 Chapter 5.1: Red Teaming Part 1 - Authorization, Goal, Memory https://www.youtube.com/watch?v=te-qix6B5R4 Chapter 5.2: Red Teaming Part 2 - Data, Hallucination, Supply Chain ← You Are Here Chapter 5.3: Red Teaming Part 3 (Coming Soon) Timestamps: 00:00 - Introduction: External Data Sources and Tools 00:38 - Real-World AI Failures and Security Incidents 01:18 - Hugging Face Models with Malware 01:56 - Microsoft's Twitter Bot: Knowledge Base Poisoning 02:33 - Air Canada's Hallucinated Bereavement Policy 03:09 - Building an IT Asset Management Agent 03:46 - Attack Vector 1: Knowledge Base Poisoning 04:21 - Code Walkthrough: Pandas DataFrame Agent 05:04 - Creating Embedded Agents within Tools 05:37 - Natural Language Asset Querying System 06:11 - Enabling Experimental Code: allow_dangerous_code=True 06:52 - Supply Chain Vulnerability: External Status Checker 07:30 - Agent Initialization with Agentic Tools 08:18 - Testing Asset Database Queries 09:30 - Demonstrating Data Poisoning Attack 10:25 - Manipulating CSV: Changing Production Asset Status 11:15 - Automated Resource Reclamation on Poisoned Data 11:51 - Pause & Think: Guardrails Against Data Poisoning 12:31 - Human-in-the-Loop for Critical Assets 13:07 - Replit Database Deletion Incident 13:46 - Attack Vector 2: Hallucination Exploitation 14:22 - Testing Agent Response to Non-Existent Assets 15:11 - Hallucination Detected: Running Tools on Fabricated Data 16:16 - Security Implications of Hallucinated Actions 16:50 - Vulnerability Scans on Non-Existent Systems 17:27 - Strengthening Prompts to Prevent Hallucination 18:09 - Implementing "Asset Not Found" Response Protocol 18:50 - Dual-Layer Guardrails: Tool and Agent Prompts 19:24 - Stop Execution Rule: If First Tool Fails, Halt 19:58 - Testing Fixed Code: Proper Hallucination Prevention 20:32 - Validation: No Further Tool Execution on Missing Data 21:09 - Attack Vector 3: Supply Chain Attacks 21:45 - Third-Party Dependencies and Library Vulnerabilities 22:29 - Mitigation Strategies: Sandboxing and Isolation 23:01 - Summary: Three Attack Vectors and Defenses 23:34 - Preview: Critical System Interactions, Resource Exhaustion 24:11 - Conclusion and Next Steps #RedTeaming #AgenticAI #AIandCybersecurity #DataPoisoning #AIHallucination #SupplyChainSecurity #CSA #CloudSecurityAlliance #KnowledgeBasePoisoning #LLMSecurity #AIFailures #Cybersecurity #KKMookhey #NetworkIntelligence #AIRedTeam #Pandas #LangChain #SecureA