AI & Cybersecurity

Currently Playing: Chapter 3: Hacking Chatbots - Prompt Injection & Security Testing

Welcome to Chapter 3 of the AI & Cybersecurity "Hacking Chatbots" Learning Series by KK Mookhey! In this hands-on security testing chapter, we move from building to breaking. Learn how to identify and exploit vulnerabilities in RAG-powered chatbots through prompt injection attacks, jailbreaking techniques, and data leakage scenarios, then discover how to defend against them. What You'll Learn: • Attack Techniques • Defense Strategies • Key Vulnerabilities Demonstrated Technologies Used: • Local LLM: Mistral (running on Llama) • UI Framework: Streamlit for user interface • Vector Database: Chroma DB • Attack Testing: DAN prompts, jailbreak techniques • Defense: Few-shot prompting, prompt safety analysis Critical Security Insights: The video demonstrates how: • Basic RAG chatbots can be manipulated to reveal sensitive information • Local models without built-in guardrails require application-level security • Simple prompt improvements are insufficient against sophisticated attacks • Two-stage guardrail systems effectively prevent jailbreak attempts • Standard web security principles (OWASP Top 10) apply to LLM applications Defense Architecture Implemented: Stage 1: Prompt Safety Analysis Analyze user input before processing Use few-shot examples to train detection Classify prompts as safe or malicious Stage 2: Conditional Execution Only safe prompts proceed to RAG retrieval Apply strict context limitations Filter responses before returning to users What's Next? Chapter 4 will explore AI Agents, their architecture, capabilities, attack surfaces, and how to build and secure agentic AI systems. Code Repository: Complete code, prompts, and test cases available in the Google Doc - https://docs.google.com/document/d/1NkGZthHxy3QWdkyeDKC_TB2gFOk2H0n-BKZVZjXwq-Q/edit?tab=t.6ypecfaymbgh Includes vulnerable and secured versions for comparison Ready-to-use guardrail implementations and safety checks About the Instructor: KK Mookhey brings 25+ years of cybersecurity expertise to this practical series on securing AI applications. Learn offensive and defensive techniques for LLM security. Connect with KK on https://www.linkedin.com/in/kkmookhey/ Course Series: This is Chapter 3 of our complete AI & Cybersecurity Learning Series. Watch Chapter 1: AI & Cybersecurity Introduction - https://www.youtube.com/watch?v=n-ZgaeULsmw Watch Chapter 2: Building Your First RAG App - https://www.youtube.com/watch?v=n-ZgaeULsmw Subscribe for Chapter 4: AI Agents & Security Key Takeaways: • Application developers must implement security guardrails, not rely solely on LLM safety • Prompt engineering is critical for preventing harmful outputs • Multi-stage validation provides defense in depth • Testing with known jailbreak techniques is essential • Standard web security practices apply to AI applications • Local/smaller models require more explicit security controls Join the Conversation: What attack techniques would you test on a RAG chatbot? Have you encountered similar vulnerabilities in production systems? Share your experiences with prompt injection defense! Network Intelligence - Empowering professionals with practical offensive and defensive AI security skills. 00:00 - Building a User-Friendly RAG Chatbot UI with Streamlit 01:47 - Understanding Guardrail Layers: Code vs Model-Level Protection 03:43 - Testing Basic Prompt Injection Attacks 05:09 - Document-Based Prompt Injection Attempts 07:15 - Data Leakage: Document Listing Vulnerability 08:33 - Advanced Jailbreak Prompts: "Do Anything Now" (DAN) Technique 09:48 - Exploiting Sensitive Data Leakage (HR/Payroll Information) 12:44 - Code-Level Analysis: Identifying Missing Guardrails 13:56 - Improving Prompt Security: First Defense Attempt 16:26 - Implementing Two-Stage Guardrail Architecture 17:42 - Few-Shot Prompting for Jailbreak Detection 18:53 - Testing Enhanced Guardrails Against Known Attacks 20:17 - Successful Defense: Blocking Harmful Prompts 22:33 - Summary of Vulnerabilities and Mitigation Strategies 24:29 - Preview: AI Agents and Their Attack Surface #AIandCybersecurity #PromptInjection #Jailbreaking #RAGSecurity #ChatbotSecurity #LLMSecurity #Cybersecurity #KKMookhey #NetworkIntelligence #EthicalHacking #AIVulnerabilities #PromptEngineering #OWASPTop10 #Mistral #Streamlit #SecurityTesting #DefensiveAI #AIRedTeam