appsec.fyi › AI

AI Resources

Page 15 of 17: resources 701–750 of 819, added Apr 2026.

The newest AI resources are on the AI page. For a reading order instead of a date order, see the AI learning path.

Date AddedLinkExcerpt
2026-04-11 2026Protecting Against Indirect Prompt Injection Attacks in MCP intermediate 5 min readLibrary for mitigating Indirect Prompt Injection attacks within the Model Context Protocol (MCP). This resource details vulnerabilities like Tool Poisoning, where malicious instructions are embedded in tool metadata, and recommends implementing AI Prompt Shields with techniques like "Spotlighting" and "Datamarking." It also emphasizes supply chain security and general security hygiene as crucial for safeguarding AI systems.
2026-04-11 2026Indirect Prompt Injection Attacks: Hidden AI Risks intermediate 4 min readLibrary for defending against indirect prompt injection attacks, a sophisticated AI threat recognized by OWASP as a top risk. This library addresses vulnerabilities where malicious instructions are embedded in external content like documents, emails, or images, rather than being submitted directly to an AI agent. It aims to mitigate risks such as data exfiltration and manipulation of business processes by enabling prompt injection detection, input validation, and the establishment of content security policies, similar to CrowdStrike's approach using its Falcon platform.
2026-04-11 2026Fooling AI Agents: Web-Based Indirect Prompt Injection Observed in the Wild intermediate 20 min readWriteup detailing observed in-the-wild indirect prompt injection (IDPI) attacks targeting AI agents. The analysis highlights real-world cases including AI-based ad review evasion, SEO manipulation for phishing, data destruction, and sensitive information leakage. It discusses 22 distinct payload engineering techniques and classifies attacker intents, emphasizing the growing weaponization of IDPI beyond theoretical risks. → unit42.paloaltonetworks.com
2026-04-11 2026Anatomy of an Indirect Prompt Injection beginner 13 min readLibrary detailing the CFS (Context, Format, Salience) model for understanding indirect prompt injection in LLMs. It analyzes vulnerabilities, drawing on concepts like Simon Willison's "lethal trifecta" (access to private data, untrusted content exposure, external communication), and examines how attackers refine tactics to bypass LLM security. Real-world examples, such as the Supabase Model Context Protocol (MCP) attack, illustrate the dangers of embedding malicious instructions within seemingly benign data, leading to unauthorized data exposure or system compromise.
2026-04-11 2026New Prompt Injection Attack Vectors Through MCP Sampling intermediate 13 min readWriteup of new prompt injection attack vectors targeting the Model Context Protocol (MCP) sampling feature. Exploiting the implicit trust model and lack of built-in security controls, attackers can achieve resource theft, conversation hijacking, and covert tool invocation. The analysis details three proof-of-concept examples and evaluates mitigation strategies for MCP-based systems, highlighting vulnerabilities in this LLM integration standard. → unit42.paloaltonetworks.com
2026-04-11 2026A Timeline of Model Context Protocol (MCP) Security Breaches news 9 min readTimeline details MCP security breaches from April to December 2025, highlighting vulnerabilities like "tool poisoning" in WhatsApp MCP, prompt injection in GitHub MCP leading to data exfiltration, cross-tenant access flaws in Asana MCP, and remote code execution in Anthropic's MCP Inspector. Other incidents include OS command injection in `mcp-remote` (CVE-2025-6514), sandbox escapes in Anthropic's Filesystem-MCP server, supply-chain compromises via malicious MCP servers, systemic MCP design flaws enabling RCE in Flowise, and path traversal in Smithery MCP hosting.
2026-04-11 2026The Vulnerable MCP Project: Comprehensive MCP Security Database beginner 7 min readLibrary of known vulnerabilities impacting MCP (Model Configuration Protocol) servers and SDKs. This catalog details specific exploits such as CVE-2025-68145, CVE-2025-68143, and CVE-2025-68144, alongside broader attack classes including prompt injection, DNS rebinding, Server-Side Request Forgery (SSRF), and command injection. Vulnerabilities affect various products like Anthropic's mcp-server-git, MCP TypeScript SDK, Cursor IDE, and Grafana MCP server, often enabling arbitrary code execution, data exfiltration, or unauthorized transactions.
2026-04-11 2026MCP Security: Critical Vulnerabilities Every CISO Must Address in 2025 intermediate 8 min readLibrary detailing critical vulnerabilities in Model Context Protocol (MCP), a new standard for AI-tool integration. It highlights how prompt injection attacks in MCP ecosystems can trigger automated actions through connected systems, potentially leading to sensitive data exfiltration. The library also addresses supply chain risks, explaining how MCP servers can dynamically modify tool definitions, allowing for "rug pull" attacks where previously approved tools can be repurposed for malicious activity, affecting vendors like Microsoft and impacting applications such as Nginx-ui (CVE-2026-33032) and Adobe Acrobat Reader.
2026-04-11 2026OWASP LLM Prompt Injection Prevention Cheat Sheet beginner 12 min readReference LLM Prompt Injection Prevention Cheat Sheet detailing vulnerabilities in Large Language Model applications. It covers direct and indirect prompt injection, encoding and obfuscation techniques like Base64 and Unicode smuggling, and typoglycemia-based attacks. The resource also discusses jailbreaking methods such as DAN prompts, multi-turn attacks, system prompt extraction, data exfiltration, multimodal injection, RAG poisoning, and agent-specific attacks. Defenses include input validation and sanitization, with code examples for pattern matching and fuzzy matching against typoglycemia variants. → cheatsheetseries.owasp.org
2026-04-11 2026Attention Tracker: Detecting Prompt Injection Attacks in LLMs intermediateAttention Tracker: Detecting Prompt Injection Attacks in LLMs
2026-04-11 2026How Microsoft Defends Against Indirect Prompt Injection Attacks intermediate 13 min readLibrary that defends against indirect prompt injection attacks targeting LLM-based systems. This library implements a multi-layered defense strategy including preventative techniques like hardened system prompts and Spotlighting, detection tools such as Microsoft Prompt Shields integrated with Defender for Cloud, and impact mitigation through data governance, user consent workflows, and deterministic blocking. It addresses vulnerabilities like data exfiltration via HTML images, clickable links, tool calls, and covert channels, as well as unintended actions and phishing. → microsoft.com
2026-04-10 2026AI Cybersecurity After Mythos: The Jagged Frontier intermediate 17 min readLibrary for AI-driven vulnerability discovery, demonstrating that smaller, cheaper open-weight models can recover significant analysis from Anthropic's Mythos showcase, including detecting exploit candidates for FreeBSD and OpenBSD bugs. This work emphasizes that the effectiveness of AI cybersecurity lies in the surrounding system architecture and deep security expertise, rather than solely on frontier model scale, impacting the economics of the defensive pipeline.
2026-04-10 2026Anthropic announces Claude Mythos for cybersecurity research news 1 min readLibrary for AI-driven cybersecurity research, Claude Mythos Preview autonomously identifies zero-day vulnerabilities and develops exploits. It has discovered critical issues in OpenBSD, FFmpeg, and the Linux kernel. Access is offered to select partners via Claude API, Amazon Bedrock, Google Cloud’s Vertex AI, and Microsoft Foundry, with an application process for open-source maintainers. Anthropic provides usage credits and donations to security foundations, highlighting significant advances in autonomous vulnerability discovery over prior models.
2026-04-10 2026MCP Security Vulnerabilities: Prompt Injection and Tool Poisoning intermediate 9 min readLibrary for securing Model Context Protocol (MCP) deployments against prompt injection and tool poisoning. It details vulnerabilities like metadata poisoning, over-permissioned tools, supply chain risks, and indirect prompt injection, referencing incidents such as the Supabase MCP Lethal Trifecta Attack. The library emphasizes prevention strategies including strict input validation, sanitization, and the principle of least privilege for tools.
2026-04-10 2026How Agentic Tool Chain Attacks Threaten AI Agent Security intermediate 5 min readLibrary for securing AI agents against agentic tool chain attacks, detailing threats like tool poisoning, tool shadowing, and rugpull attacks that exploit the agent's reasoning layer and natural language-based decision-making. It covers how these attacks can lead to data exfiltration, unauthorized actions, and supply chain risks by manipulating tool descriptions, metadata, and server behavior, and recommends mitigation strategies including tool governance, version control, server identity controls, pre-execution guardrails, and observability.
2026-04-10 20268,000+ MCP Servers Exposed: The Agentic AI Security Crisis of 2026 news8,000+ MCP Servers Exposed: The Agentic AI Security Crisis of 2026
2026-04-10 2026Offensive Security for MCP Servers: How to Prevent AI Agent Exploits intermediate 10 min readLibrary for securing MCP (Multi-Cloud Platform) servers against AI agent exploits, addressing vulnerabilities like command injection, SSRF, and path traversal frequently found in modern deployments. It highlights how AI's autonomous execution and dynamic capability discovery, unlike traditional REST APIs, create new risk classes by enabling agents to chain tool calls and reason across APIs. The library emphasizes adapting security from syntax to intent validation, guarding against prompt injection and tool poisoning where manipulated metadata or input can lead to unintended, privileged operations, ultimately leveraging foundational API security principles.
2026-04-10 2026The New AI Attack Surface: 3 AI Security Predictions for 2026 beginner 6 min readLibrary for confronting three AI attack vectors manifesting in production by 2026: indirect injection via data poisoning, supply chain infiltration through AI development toolchains like MCP servers, and agent-to-agent attack propagation through "toxic combinations" in autonomous agent ecosystems. These vectors exploit how AI agents interpret instructions, trust data sources, and execute permitted actions, moving beyond traditional code vulnerabilities to exploit data as executable commands and the inherent trust in interconnected AI architectures.
2026-04-10 2026Introduction to Data Poisoning: A 2026 Perspective beginner 11 min readLibrary introducing data poisoning, an adversarial attack corrupting AI/LLM training data to cause backdoors or biased outputs. It details real-world incidents like Basilisk Venom poisoning GitHub code, Qwen 2.5's search tool manipulation, Grok 4's "!Pliny" backdoor triggered by X prompts, and hidden instructions in MCP tools like "joke_teller." The library also covers poisoning in retrieval (RAG), synthetic data pipelines (VIA), and diffusion models, highlighting how even small, hidden manipulations can undermine AI safety and trust across the entire LLM lifecycle.
2026-04-10 2026AI Security Research — December 2025 newsAI Security Research — December 2025
2026-04-10 2026From Prompt Injections to Protocol Exploits in LLM Agent Workflows advancedFrom Prompt Injections to Protocol Exploits in LLM Agent Workflows
2026-04-10 2026LLM Security Guide: OWASP GenAI Top-10 Risks beginner 28 min readLibrary detailing offensive and defensive security for Large Language Models and Agentic AI Systems, updated with the OWASP Top 10 for LLMs 2025 and the OWASP Top 10 for Agentic Applications 2026. It covers Agentic AI Security, RAG Vulnerabilities, System Prompt Leakage, Vector/Embedding Weaknesses, and AI Compliance, incorporating tools like DeepTeam, Promptfoo, ARTKIT, and frameworks such as Meta LlamaFirewall and Amazon Bedrock Guardrails.
2026-04-10 2026Prompt Injection Attacks in LLMs: A Comprehensive Review intermediatePrompt Injection Attacks in LLMs: A Comprehensive Review
2026-04-10 2026Prompt Injection Attacks: Examples, Techniques, and Defence intermediate 23 min readLibrary for understanding and defending against prompt injection, a critical LLM security vulnerability. It details direct and indirect injection techniques, including examples like DAN jailbreaks, EchoLeak (CVE-2025-32711), and webpage poisoning attacks, as reported by OWASP, NCSC, and Anthropic. This resource provides practical defense strategies and highlights the inherent challenges in distinguishing trusted instructions from untrusted data within LLM architectures.
2026-04-10 2026Indirect Prompt Injection: The Hidden Threat intermediate 16 min readLibrary for understanding and defending against indirect prompt injection, a vulnerability where hidden instructions within ingested data (webpages, PDFs, emails, code) can hijack AI reasoning or tool actions. It details real-world incidents like the Perplexity Comet leak and CVE-2025-59944, highlighting how agentic AI amplifies risk. Mitigation requires architectural changes, not prompt tuning, focusing on trust boundaries, context isolation, and output verification.
2026-04-10 2026AI Agent Security in 2026: Prompt Injection and Memory Poisoning intermediate 12 min readLibrary for understanding AI agent security risks, focusing on prompt injection and memory poisoning attacks. It details indirect prompt injection's impact via emails and documents, exemplified by CVE-2025-32711, and memory poisoning attacks like MemoryGraft, where agents develop false beliefs. The library also covers tool misuse through hidden instructions in metadata, misleading examples, and permissive schemas, observed in frameworks like CrewAI and AutoGen, and discusses supply chain vulnerabilities where agents fetch runtime dependencies without human review.
2026-04-10 2026Prompt Injection: The Most Common AI Exploit in 2025 beginner 7 min readLibrary detailing prompt injection, the most common AI exploit in 2025, which manipulates AI instructions rather than code. It categorizes attacks into direct, indirect, jailbreak, and cross-plugin poisoning, highlighting risks to enterprise RAG systems and SaaS security operations. The resource emphasizes robust AI agent identity, authorization, continuous monitoring with anomaly detection, and integrating AI security telemetry into existing SIEM infrastructure, aligning with frameworks like NIST AI RMF and ISO/IEC 42001.
2026-04-10 2026AI Prompt Injection Attacks: How They Work (2026) beginner 12 min readLibrary for defending against AI prompt injection attacks, detailing their evolution from academic curiosities to operational threats with documented cases affecting OpenAI's GPT models and Anthropic's Claude. It covers attack mechanisms like "instruction confusion," evolving vectors such as encoding-based and multi-turn conversation attacks, and real-world incidents like the OpenClaw vulnerability, demonstrating data exfiltration and financial losses totaling $2.3 billion globally in 2025. The library addresses insufficient input sanitization, overprivileged AI agents, and a lack of output validation, highlighting detection gaps where current methods catch only 23% of sophisticated attempts.
2026-04-10 2026LLM Security Risks in 2026: Prompt Injection, RAG, and Shadow AI beginner 30 min readLibrary for mitigating LLM security risks, including prompt injection, RAG data poisoning, and autonomous exploits like EchoLeak demonstrated against Microsoft 365 Copilot. It addresses the blurred line between data and instructions, AI outputs triggering actions, and the human element in vulnerabilities, emphasizing containment strategies like limiting AI privileges and validating outputs.
2026-04-09 2026Claude Code security settings nobody told you about beginnerClaude Code security settings nobody told you about
2026-04-08 2026theNET | De-risking the AI rollout intermediate 6 min readLibrary for de-risking AI rollouts, providing probabilistic security to address novel threats like prompt injection, data poisoning, and denial-of-wallet attacks. It emphasizes model-agnostic, inline protection, input/output monitoring, observability, and integration with traditional application security to safeguard AI-powered applications against deterministic and unpredictable attack paths.
2026-04-06 2026Best AI Security Tools in 2026 beginner 11 min readPlatforms for AI security are ranked by their coverage of three critical phases: discovering AI assets and mapping threat graphs (Phase 1), conducting adversarial testing against live applications and RAG pipelines (Phase 2), and deploying runtime guardrails calibrated from red teaming results (Phase 3). Repello AI offers full-lifecycle coverage with its Inventory, ARTEMIS, and ARGUS products. HiddenLayer focuses on model artifact scanning and runtime model anomaly detection. Mindgard provides automated multimodal AI security testing, primarily for Phase 2. Lakera, now part of Check Point, specialized in runtime guardrails for LLM applications.
2026-04-06 2026OWASP Top 10 for Agents 2026 beginner 10 min readFramework for assessing OWASP Agentic AI (ASI) Top 10 2026 risks, including Agent Goal Hijack (ASI01), Tool Misuse & Exploitation (ASI02), and Agent Identity & Privilege Abuse (ASI03). It addresses vulnerabilities introduced by autonomous agents' reasoning, memory, tool integration, and multi-step execution, detecting issues like unexpected code execution (ASI05) and insecure inter-agent communication (ASI07). The framework integrates with DeepTeam's red teaming capabilities for programmatic risk assessment.
2026-04-06 2026Google Workspace's Continuous Approach to Mitigating Prompt Injection intermediate 5 min readLibrary detailing Google Workspace's continuous approach to mitigating Indirect Prompt Injection (IPI) attacks against Gemini. It outlines proactive strategies including human and automated red-teaming, the AI Vulnerability Rewards Program, and public attack monitoring to discover and catalog new vulnerabilities. The library emphasizes ongoing defense refinement through deterministic and ML-based defenses, LLM prompt engineering, and Gemini model hardening, utilizing synthetic data generation via Simula for robust attack variant expansion and defense model retraining.
2026-04-06 2026Prompt Injection Attacks in LLMs: What Developers Need to Know in 2026 beginner 6 min readGuide on prompt injection attacks in LLMs, detailing how attackers manipulate models using natural language to override system instructions. It covers direct (jailbreaking) and indirect injection, citing examples like the Chevrolet dealership GPT and Perplexity Comet credential theft incidents. Developers are advised to implement architectural separation of instructions, conversation token limits, input filtering, AI guardrails, and developer training to mitigate these risks.
2026-04-05 2026LangChain LangGraph Flaws Expose Files Secrets Databases in Widely Used AI Frameworks intermediate 2 min readLibrary vulnerabilities in LangChain and LangGraph, specifically CVE-2026-34070 (path traversal), CVE-2025-68664 (deserialization of untrusted data), and CVE-2025-67644 (SQL injection), allow attackers to access arbitrary files, steal API keys and environment secrets, and manipulate SQL queries. These flaws, impacting widely used LLM application frameworks, have been patched in recent versions of langchain-core and langgraph-checkpoint-sqlite. → thehackernews.com
2026-04-04 2026Detecting and analyzing prompt abuse in AI tools intermediate 5 min readPlaybook detailing detection, investigation, and response to AI prompt abuse. It covers direct prompt overrides, extractive prompt abuse against sensitive inputs, and indirect prompt injection, including the HashJack technique affecting AI summarization tools via URL fragments. This guide leverages Microsoft security tools like Defender for Cloud Apps, Purview DLP, Microsoft Entra ID conditional access, and Microsoft Sentinel to monitor AI interactions and protect against manipulation. → microsoft.com
2026-04-03 2026Training an AI agent to attack LLM applications like a real adversary advanced 3 min readTool that simulates adversarial attacks against LLM-powered applications. This AI pentesting agent autonomously chains techniques like prompt injection, indirect prompt injection, and tool abuse to uncover vulnerabilities missed by traditional scanners. It gathers application context, probes role-based access control, and supports models from OpenAI, Anthropic, and open-source providers, integrating into CI/CD pipelines for continuous testing. Novee Security's agent is trained on real-world vulnerability research, including findings like arbitrary code execution in the Cursor coding assistant. → helpnetsecurity.com
2026-04-03 2026Prompt Injection Attacks in LLMs: Vulnerabilities, Exploitation & Defense intermediatePrompt Injection Attacks in LLMs: Vulnerabilities, Exploitation & Defense
2026-04-03 2026How AI Red Teaming Fixes Vulnerabilities in Your AI Systems intermediate 9 min readLibrary for AI Red Teaming provides a practical playbook for CISOs and AI leaders to test AI systems, including LLMs and chatbots, for vulnerabilities before deployment. It simulates attacks and misuse to identify weaknesses across prompts, data, and agent interactions, addressing risks like prompt injection, data leakage, and abuse of model autonomy. This method moves beyond isolated model testing to system-wide evaluation in operational settings, aligning with frameworks like MITRE ATLAS, EU AI Act, and NIST's AI Risk Management Framework to ensure safe and compliant AI use.
2026-04-03 2026What Is Prompt Injection in AI? Examples & Prevention | EC-Council beginner 7 min readLibrary for defending against prompt injection attacks, a technique where attackers manipulate AI systems through malicious instructions embedded in prompts. This resource details direct and indirect injection methods, citing real-world vulnerabilities like CVE-2025-53773 affecting GitHub Copilot and ChatGPT's Azure backdoor. It also highlights attacks against Google Jules and Devin AI, emphasizing the enterprise-wide compromise risks due to AI access to sensitive data and infrastructure. Mitigation strategies include zero-trust AI architecture, strict privilege separation, real-time threat detection, human-in-the-loop approvals, and continuous red teaming.
2026-04-03 2026Prompt Injection Attacks in 2025: Risks, Defenses & Testing intermediate 3 min readLibrary for detecting and mitigating prompt injection attacks in LLM-powered applications. It focuses on adversarial input testing, prompt isolation analysis, output validation, and workflow abuse simulation to uncover risks missed by traditional security tools. The library addresses how malicious instructions can manipulate model behavior, spread through trusted content, and create business-level impact, emphasizing that prompt injection is a trust problem at the intersection of application logic, content ingestion, and workflow design.
2026-04-03 2026Red Teaming the Mind of the Machine: Evaluation of Prompt Injection and Jailbreak Vulnerabilities intermediate 15 min readSurvey of prompt injection and jailbreak vulnerabilities against state-of-the-art LLMs including GPT-4, Claude 2, Mistral 7B, and Vicuna. This research categorizes over 1,400 adversarial prompts and analyzes their success rates, generalizability, and construction logic, drawing from public repositories and forums. The study also proposes layered mitigation strategies and recommends a hybrid red-teaming and sandboxing approach for robust AI security, noting prompt injection as a critical vulnerability identified by OWASP. → arxiv.org
2026-04-03 2026Practical LLM Security Advice from the NVIDIA AI Red Team intermediate 5 min readLibrary summarizing NVIDIA AI Red Team findings, detailing common LLM application vulnerabilities. It addresses risks like remote code execution (RCE) from executing LLM-generated code (e.g., via `exec` or `eval`), insecure permissions in Retrieval-Augmented Generation (RAG) data stores leading to data leakage and prompt injection, and data exfiltration through active content rendering of Markdown or hyperlinks. Mitigation strategies include sandboxing dynamic code, rigorously managing RAG permissions, and sanitizing LLM output.
2026-04-03 2026OWASP Top 10 for LLMs 2025 | DeepTeam Red Teaming Framework beginner 7 min readFramework integrating OWASP Top 10 for LLMs 2025 risks, including Prompt Injection (LLM01), Sensitive Information Disclosure (LLM02), Supply Chain (LLM03), Data and Model Poisoning (LLM04), Improper Output Handling (LLM05), Excessive Agency (LLM06), System Prompt Leakage (LLM07), and Vector and Embedding Weaknesses (LLM08). It facilitates detection of vulnerabilities in RAG systems and autonomous agents through programmatic assessment or the Confident AI platform.
2026-04-03 2026Continuously Hardening ChatGPT Against Prompt Injection | OpenAI intermediateContinuously Hardening ChatGPT Against Prompt Injection | OpenAI
2026-04-03 2026LLM01:2025 Prompt Injection | OWASP Gen AI Security beginner 6 min readReference detailing LLM01:2025 Prompt Injection, a vulnerability where user prompts unintendedly alter Large Language Model behavior. The OWASP Gen AI Security resource covers direct and indirect injections, including scenarios like CVE-2024-5184 exploitation in email assistants and multimodal attacks. It outlines mitigation strategies such as constraining model behavior, input/output filtering, and adversarial testing, emphasizing that while prevention is challenging, impact reduction is achievable. → genai.owasp.org
2026-04-02 2026Guarding LLMs With a Layered Prompt Injection Representation intermediate 11 min readLibrary for LLM security that learns a low-dimensional latent representation of prompt injection attacks. This approach complements perplexity-based filtering and achieves high precision and recall by training a classifier on features derived from this learned representation, distinguishing benign prompts from adversarial ones. → trendmicro.com
2026-04-02 2026Auditing the Gatekeepers: Fuzzing "AI Judges" to Bypass Security Controls intermediate 6 min readTool for fuzzing AI judges, called AdvJudge-Zero, exploits prompt injection vulnerabilities in LLM-based security gatekeepers. This fuzzer identifies stealthy control tokens, such as formatting symbols and structural phrases, that manipulate the AI's decision-making logic to bypass safety policies and allow prohibited content, or corrupt training data by awarding high scores to incorrect responses. The research demonstrates a 99% success rate in bypassing controls across various LLM architectures, highlighting the need for adversarial training to harden these systems. → unit42.paloaltonetworks.com
2026-04-02 2026AI Security for Apps is now generally available news 6 min readLibrary for securing AI-powered applications, generally available, offering discovery of AI endpoints, detection of prompt injection and PII exposure, and mitigation via WAF rules. New features include custom topic detection and free AI endpoint discovery for all Cloudflare customers, with expanded integrations with IBM and Wiz for unified security posture management. It addresses risks cataloged in the OWASP Top 10 for LLM Applications, such as prompt injection and sensitive data leakage, by analyzing prompt and output behavior rather than fixed operations. → blog.cloudflare.com