TL;DR
AI security is about stopping an AI feature from working against the business that runs it. Four failure patterns show up most often: attackers slipping instructions into the AI's input, hidden instructions inside the content the AI reads, the AI bypassing its own safety rules, and AI agents turning bad input into real-world actions like sending email or making changes.
By Rohit Hatagale, AI Security Lead, SecureLayer7Updated
Topics
- **The Anatomy of an Agentic AI Attack**: the full agentic AI kill chain, from ML entry to cloud, C2, and CI/CD, with defences.
- OWASP LLM Top 10 (2025): Every Risk Explained: the ten biggest risks for AI apps. What each one is, what changed in 2025, and how it lines up with MITRE ATLAS and NIST AI 600-1.
- What is Prompt Injection?: what it is, the direct and indirect kinds, real cases, and how to defend against it.
- What is Indirect Prompt Injection?: the kind that reaches the model through content it reads (a web page, an email, a file), not what the user typed.
- What is LLM Jailbreaking?: getting an AI to ignore its safety rules. The common tricks, and how to measure your risk before launch.
- What is RAG Poisoning?: planting bad content in the knowledge base an AI reads from. The two ways it goes wrong.
- What is Model Extraction?: stealing what an AI knows by asking it questions, cloning it, recovering its parameters, or leaking its training data.
- What is Agentic AI Security?: what changes once an AI can use tools and take actions, and the new ways it gets attacked.
- What is Training Data Poisoning?: slipping bad data into what an AI learns from, so it misbehaves on cue.
- What is AI Red Teaming?: goal-led attack testing of an AI system, and how it differs from a pentest.
- LLM Output Validation: Defense Patterns That Actually Work: five ways to check an AI's output before anything downstream trusts it.
- What is Prompt Obfuscation?: encoding a prompt in Base64, leetspeak, or invisible Unicode so a filter misses it but the model still obeys, and how to defend.
- What is a DAN Jailbreak?: the classic Do Anything Now roleplay jailbreak, and why persona attacks keep working.
- What is Many-Shot Jailbreaking?: flooding the context with fake compliant dialogues so the model copies the pattern.
- What is a Crescendo Attack?: a multi-turn jailbreak that escalates gradually past single-message filters.
- What is an Adversarial Suffix?: optimized GCG-style strings that force compliance and transfer between models.
- What is Refusal Suppression?: banning the model's refusal language to boost other jailbreaks.
- What is Payload Splitting?: hiding a request in fragments the model reassembles and runs.
- What is Token Smuggling?: getting a banned word past a filter by splitting or encoding it.
- What is Unicode Tag Smuggling?: hiding instructions in invisible Unicode Tag characters.
- What is a Homoglyph Attack?: swapping letters for look-alike Unicode to evade keyword filters.
- What is System Prompt Extraction?: prompt leaking that exposes your hidden instructions and secrets.
- What is Context Flooding?: burying the safety instructions under bulk text.
- What is LLM Data Exfiltration?: prompt injection that leaks data through a crafted image or link.
- What is an Autonomous AI Attack?: An AI agent, not a human operator, plans and executes the whole intrusion: recon, exploita
- What is ML Data-Loader Injection?: A dataset or model config can carry template expressions, external file references, or pro
- What is a Model File Supply Chain Attack?: Teams pull models and datasets from public hubs and load them without a second thought. A
- What is Excessive Agency?
- What is a Denial of Wallet Attack?
- What is MCP Tool Poisoning?
- What are Vector and Embedding Weaknesses?
- What is Sensitive Information Disclosure?
- What is Model Poisoning?
- LLMjacking
- LLM-enabled malware
- AI-augmented vulnerability research
- model distillation attack
- AI coding assistant supply chain attack
- AI-assisted phishing
- Malicious AI packages and models
- The OpenAI and Hugging Face supply chain incidents
- **What are AI Safety Benchmarks?**: the 25 safeguard evaluations in the UK AI Security Institute's Inspect registry, grouped by what they test.
- What is AgentDojo?: the prompt-injection benchmark for agents doing email, Slack, travel and banking tasks.
- What is AgentHarm?: whether AI agents carry out openly malicious multi-step tasks.
- What is StrongREJECT?: the jailbreak benchmark that scores how useful a harmful answer really is.
- What is the b3 Benchmark?: testing the security of the model inside an agent with crowdsourced attacks.
- What is AgentThreatBench?: the OWASP agentic risks as runnable tests for memory poisoning, goal hijack and exfiltration.
- What is Prompt Injection in Coding Agents?: instructions hidden in issues, comments and READMEs, and how CodeIPI tests it.
- What is a Guardrail Agent?: a second model that checks an agent's actions against rules before they run.
- What is LLM Memory Leakage?: when an assistant's long-term memory surfaces where it does not belong.
- What is Reward Hacking?: agents that game their checks instead of doing the task.
References
- [1]OWASP LLM Top 10 (2025)(OWASP)
- [2]MITRE ATLAS(MITRE)
- [3]NIST AI 600-1 (Generative AI Profile)(NIST)
Related terms