AI
A route through the library rather than a dump of it. Items are drawn from the full AI collection, filtered to teaching material (news items are excluded) and ordered within each stage by depth signals — whether the piece carries code, how substantial it is, and what readers actually open.
1
Start here
5 resourcesOrientation and first principles — what the bug class is and how it behaves.
- GPT-5.5-Cyber built a zlib fuzzing lab in a dayLibrary built by GPT-5.5-Cyber for fuzzing zlib; it automatically generated harnesses across numerous entrypoints (including inflate, uncompress2, and MiniZip), configured ASan and UBSan builds, repurposed edge-case tests as seeds, and utilized compile-time variants like INFLATE_STRICT. This autonomous fuzzing campaign, a process that previously took weeks for skilled researchers, was completed in a single day, demonstrating a significant shift in vulnerability discovery capabilities.
- [tl;dr sec] #345 - Bug Rumors → Exploits, Version Control DFIR, Agentic WormsAnalysis details how AI models struggle to review their own code, often missing bugs they introduce, and explores the diminishing effectiveness of traditional vulnerability disclosure timelines. A cheat sheet for version control digital forensics and incident response (DFIR) across GitHub, GitLab, Bitbucket, and Azure DevOps is provided, highlighting visibility gaps and configuration needs. The entry also discusses the challenge of false positives in security tools when benign traffic mimics attack patterns, and mentions a paper on self-replicating agentic worms.
- Armur-Ai/Pentest-Swarm-AI: Autonomous penetration testing using a swarm of AI agents. Orchestrates recon…Tooling for AI-driven pentesting, Pentest Swarm AI utilizes swarm intelligence primitives—stigmergy, emergence, and decentralization—to coordinate multiple independent agents on a shared blackboard. Unlike sequential pipelines, this approach allows attack chains to emerge organically as agents communicate and influence each other through findings and "pheromones." It integrates tools like nmap, sqlmap, Burp, and Metasploit, supporting various LLMs and aiming for emergent, emergent, and decentralized offensive security testing.
- AI fuzzing: where AI helps and what the evidence showsTool demonstrating AI's role in fuzzing; it uses LLMs to automate human-centric tasks like harness generation and crash triage, while coverage-guided fuzzers like AFL++ or libFuzzer remain the core bug-finding engines. Evidence from OSS-Fuzz, an ISSTA 2024 study, and DARPA's AIxCC indicates AI fuzzing can extend code coverage and find vulnerabilities, though challenges persist with complex APIs and distinguishing actual bugs from expected input-validation errors. The provided demo showcases this workflow on a toy parser.
- osint-d2 for agentic identity investigations across public sourcesTool for agentic identity investigations across public sources, osint-d2 utilizes a Python CLI and local dashboard to chain username, email, and phone scans into verifiable identity dossiers. It offers both deterministic scanning and LLM-driven investigation modes, with the latter supporting functions like `scan_username`, `breach_check`, and `compare_avatars` using perceptual hashing (dHash). The application is provider-agnostic for AI integration, supporting options like deepseek, OpenAI, and ollama for local inference. Reporting includes PDF, JSON, and graph exports, with speculative inferences gated behind an opt-in flag.
2
Build depth
5 resourcesReal testing methodology, tooling, and writeups that show the work.
- Practical LLM Security Advice from the NVIDIA AI Red TeamLibrary summarizing NVIDIA AI Red Team findings, detailing common LLM application vulnerabilities. It addresses risks like remote code execution (RCE) from executing LLM-generated code (e.g., via `exec` or `eval`), insecure permissions in Retrieval-Augmented Generation (RAG) data stores leading to data leakage and prompt injection, and data exfiltration through active content rendering of Markdown or hyperlinks. Mitigation strategies include sandboxing dynamic code, rigorously managing RAG permissions, and sanitizing LLM output.
- [tl;dr sec] #333 - Perplexity's Bumblebee, Evading Cloud Logging, AI Vuln Hunting SpecLibrary for detecting malware in packages, agent configurations, and browser extensions, alongside techniques for evading cloud logging, and a specification for building custom AI security scanning systems. It details how formal methods are becoming more practical for AI-generated code, and how Microsoft's Agentic Secret Finder reduced false positives in GitHub's AI secret scanning by 75% through context extraction. The entry also covers the discovery of HTTP/2 Bomb, a DoS vulnerability affecting multiple web servers, and methods for disrupting AWS CloudTrail logging and abusing cloud logging services for defense evasion and visibility.
- Leaking Secrets in the Age of AILibrary for identifying AI-related secret leakage in public code repositories. The analysis found that `.ipynb` notebook files and configuration files like `mcp.json` and `.env` are particularly prone to exposing secrets from emerging AI vendors, impacting numerous companies. This research highlights critical gaps in current secrets scanning tools and practices, especially concerning the handling of AI development artifacts.
- Detecting AI-specific threats in Claude Enterprise from the Compliance API: a prefilter + LLM-as-judge pipeline with Sigma rulesLibrary for detecting AI-specific threats within Claude Enterprise, leveraging the Compliance API through a prefilter and LLM-as-judge pipeline with Sigma rules. It focuses on detecting indirect prompt injection via ingested content, jailbreaks, system prompt extraction, exfiltration preparation, and sensitive output disclosure, differentiating these threats from standard SaaS security concerns by analyzing conversation content rather than just perimeter events.
- Would You Click ‘Accept’? Automatically detecting malicious Azure OAuth applications using LLMsTool for detecting malicious Azure OAuth applications, leveraging insights from homoglyph attacks and analysis of real-world campaigns. This pipeline identifies suspicious applications by comparing their characteristics against a baseline of legitimate integrations, flagging deviations in publisher verification, homepage URLs, and application owner domains. The research highlights common gaps in Azure service principal management and the risk of consent fatigue, leading to potential privilege escalation and persistent access for attackers.
3
Go deep
8 resourcesNovel research, edge cases, and the techniques that push the class forward.
- Prompt Injection Attacks on Agentic Coding Assistants: A Systematic AnalysisAnalysis of prompt injection vulnerabilities affecting agentic AI coding assistants like Claude Code, GitHub Copilot, and Cursor, which integrate LLMs with external tools and protocols such as MCP. This work synthesizes findings from 78 studies, detailing 42 attack techniques including input manipulation, tool poisoning, and protocol exploitation. It identifies that over 85% of attacks succeed against current defenses, often enabling arbitrary code execution and system compromise through vulnerabilities in skill-based architectures and protocol ecosystems.
- Prompt Injection 2.0: Hybrid AI ThreatsLibrary for analyzing Prompt Injection 2.0, which combines LLM manipulation with traditional exploits like XSS and CSRF. It builds upon Preamble's research and mitigation technologies, evaluating them against contemporary threats such as AI worms and multi-agent infections. The library analyzes how these hybrid attacks bypass security controls, referencing CVE-2024-5565 and DeepSeek XSS exploits, and proposes architectural solutions involving prompt isolation and runtime security.
- s1ngularity's Aftermath: AI, TTPs, and Impact in the Nx Supply Chain AttackAnalysis of the s1ngularity Nx supply chain attack details novel TTPs used by AI-powered malware to exfiltrate secrets, including GitHub and npm tokens. The attack progressed through three phases: initial public leakage of thousands of corporate secrets, abuse of leaked GitHub tokens to expose private repositories, and a final phase publishing more repositories. The malware leveraged AI CLIs like Claude and Gemini to identify and exfiltrate sensitive files, impacting over 1,700 users with public secret leakage and hundreds more through the exposure of private repositories.
- EchoLeak: First Real-World Zero-Click Prompt Injection ExploitWriteup of EchoLeak (CVE-2025-32711), the first zero-click prompt injection exploit targeting Microsoft 365 Copilot. This vulnerability allowed unauthenticated data exfiltration via a crafted email by chaining multiple bypasses, including evading XPIA classifiers, using reference-style Markdown, exploiting auto-fetched images, and abusing a Microsoft Teams proxy within the content security policy. The paper analyzes defense failures and proposes mitigations such as prompt partitioning and enhanced filtering, providing generalizable lessons for secure AI copilots.
- Building an Open-Source AI-Powered Auto-Exploiter with a 1.7B Parameter Model: No Paid APIs RequiredLibrary for building an open-source, AI-powered autonomous penetration testing agent. This system utilizes a 1.7 billion parameter qwen3:1.7b model, LangChain, and LangGraph for local execution, eliminating API costs and data exfiltration. It functions as a ReAct agent, independently scanning networks with Nmap, searching for exploits using searchsploit, mirroring them, analyzing code with `inspect_exploit_code`, setting up listeners with `start_listener`, and executing commands via `execute_shell_command` to achieve autonomous exploitation.
- Microsoft Copilot: From Prompt Injection to Exfiltration of Personal Information · Embrace The RedWriteup detailing a Microsoft 365 Copilot vulnerability where prompt injection, automatic tool invocation, and ASCII smuggling were combined to exfiltrate personal information. The exploit chain leveraged malicious emails or shared documents to trigger Copilot's processing, enabling it to access and send sensitive data like emails and MFA codes to attacker-controlled domains via disguised hyperlinks.
- AI broke the deserialization threat modelLibrary for dynamically protecting against Java deserialization threats. This library counters AI-driven discovery of novel gadget chains, which has rendered traditional blocklists and vendor denylists ineffective by automating exploit generation. It focuses on constraining deserialization operations at runtime within micro-compartments, blocking malicious behavior without relying on signatures or prior knowledge of specific attacks, and addresses vulnerabilities like CVE-2025-24813.
- Microsoft FORGE Lab uses agentic tools to find memory safety issues in Rust stdLibrary identifying memory safety issues within Rust's standard library, including undefined behavior triggered by safe Rust code. The research employed Hyenas, an AI-powered vulnerability scanner, to uncover patterns of misplaced trust in unsafe code, specifically regarding panics, user-provided implementations, and underlying system interactions. Notable findings include a panic safety issue in `Iterator::map_windows` and discussions on the challenges of writing correct unsafe Rust, even when leveraging Rust's rich type system and ownership model.