How to use AI agents to find real security vulnerabilities

15 Nov 2026 04:37 6,488 views
Security teams are starting to use AI agents to scan code and systems for vulnerabilities, but doing it well takes more than dropping a repo into a model. This guide breaks down Cisco’s Foundry Security Spec and CodeGuard approach, and shows how to start small with trustworthy, agent-driven security evaluations.

AI models are getting surprisingly good at reading code, spotting bugs, and even suggesting exploits. But if you just throw a code repository at a powerful model and tell it to “find vulnerabilities,” you’ll quickly run into a familiar problem: lots of noisy findings and not enough signal.

To make AI genuinely useful for security, you need structure: clear roles for agents, a way to validate findings, and a feedback loop that gets smarter over time. That’s exactly what Cisco’s Foundry Security Spec and CodeGuard are designed to enable.

Why “just ask the model” doesn’t work for security

It’s tempting to hand an AI model your source code and let it rip. In practice, this approach produces a mix of real issues, false positives, and vague warnings that are hard for engineers to act on.

The core problems are:

• High false positive rates: Large models can confidently describe theoretical vulnerabilities that don’t actually occur in your running system.
• No connection to live systems: Purely static analysis can’t prove exploitability, so teams waste time chasing non-issues.
• No workflow: A list of model-generated findings isn’t enough. You need triage, validation, exploit creation, and remediation guidance.

To fix this, you need an agentic “harness” around the models: a set of coordinated agents that analyze code, interact with running systems, attempt exploits, and feed back high-quality evidence to engineering teams.

What the Foundry Security Spec actually is

The Foundry Security Spec is an open blueprint for building a trustworthy, agent-based AI system that evaluates software for security issues. It’s not a turnkey scanner or a Cisco-specific product. Instead, it captures the design patterns that worked in practice when using frontier models to assess real-world codebases.

At a high level, the spec defines:

• An agent architecture: Eight core agent roles plus optional extensions, and how they coordinate.
• A finding lifecycle: How potential vulnerabilities are discovered, investigated, validated, and reported.
• Operational guidance: How to think about token budgets, model selection, infrastructure choices, and self-improvement loops.

Rather than shipping a one-size-fits-all implementation, the spec includes around 130 functional requirements. These help teams adapt the architecture to their own environments—cloud or on-prem, different LLM gateways, internal sandboxes, and existing security tooling.

The eight core AI agent roles in Foundry

The Foundry spec describes eight primary agent roles, each focused on a specific part of the evaluation process:

• Orchestrator: Coordinates all the other agents, manages tasks, and oversees the overall evaluation flow.
• Indexer: Ingests and organizes the codebase or target system so other agents can navigate it efficiently.
• Cartographer: Maps out the architecture, components, and relationships between services and modules.
• Detector: Performs the core vulnerability detection, using both rules and exploratory analysis.
• Triager: Filters and prioritizes findings, reducing noise before anything is surfaced to humans.
• Validator: Attempts to independently reproduce and exploit findings to prove they’re real and exploitable.
• Coverage Guide: Tracks what parts of the system have been analyzed to avoid blind spots.
• Reporter: Packages validated findings into clear, actionable reports for engineering and security teams.

This separation of responsibilities is key. Instead of a single model doing everything, each agent specializes in a step of the process, making the overall system more reliable and easier to reason about.

Optional extensions for deeper security testing

Beyond the core agents, Foundry defines five optional extension roles that organizations can add as their maturity grows:

• Deeper testing: Agents that go beyond basic checks to perform intensive probing of complex components.
• Variant hunting: Looks for related weaknesses and similar bugs once a vulnerability pattern is found.
• Attack path mapping: Chains multiple issues into realistic attack paths and post-exploitation scenarios.
• Remediation: Generates concrete fix guidance and code changes that align with your coding standards.
• Self-improvement: Continuously refines agent skills and rules based on past runs and new discoveries.

These extensions turn Foundry from a static scanner into a living system that learns from each engagement and gets better over time.

spec.md vs constitution.md: the two main artifacts

The Foundry Security Spec is delivered through two main documents: spec.md and constitution.md.

spec.md: your operational blueprint

spec.md defines the operational architecture of your agentic system. It describes:

• The core and extension agent roles you’ll use.
• How agents communicate and coordinate.
• How models, infrastructure, and policies plug into the system.
• How to adapt the architecture to your organization’s environment.

Teams aren’t expected to copy it blindly. Instead, you use it as a seed specification and refine it based on your own models, infrastructure, security policies, and integration points (for example, CI/CD, ticketing, or monitoring tools).

constitution.md: the principles that must never break

constitution.md defines the core invariants—principles that should hold regardless of how you implement the system. These include ideas like:

• Evidence over model assertion: Don’t trust a model just because it “sounds confident.” Findings should be backed by concrete evidence.
• Only surface triaged findings: No raw, unfiltered model output should be pushed to engineering teams.
• Independent reproduction: A vulnerability should only be labeled “exploited” after independent reproduction, not just a single model’s claim.

Each principle comes from real failures observed in practice and is designed to prevent them from recurring. This is where the “trustworthy” part of a trustworthy agentic system really lives.

Why Cisco shared a spec instead of a finished product

Cisco could have open-sourced its internal implementation, but that would have been tightly coupled to Cisco’s own infrastructure: internal sandboxes, identity systems, LLM gateways, and more. Most organizations wouldn’t be able to drop that into their environment without heavy rework.

Instead, by publishing a specification and blueprint, Cisco gives teams a way to:

• Plug in their own models (local or via API).
• Respect their own policies and standards.
• Integrate with existing ticketing systems (Jira, ServiceNow, or homegrown tools).
• Adapt to their own deployment stacks and repository layouts.

Using tools like GitHub Spec Kit, implementers can instantiate the spec and be guided through key choices: where the system will run, which models to use, how to handle budgets and token costs, what kind of self-improvement loop to enable, and more.

How Foundry and CodeGuard work together

Foundry doesn’t exist in a vacuum. It works hand-in-hand with Project CodeGuard, which focuses on secure coding rules for AI assistants.

CodeGuard is a portable, model-agnostic collection of secure coding skills and rules that AI coding assistants can follow before, during, or after code generation. It encodes best practices so that AI-generated code is less likely to introduce new vulnerabilities.

Foundry is the evaluation and detection system. Its Detector agent operates in two main modes:

• Rule-based sweep: Applies reusable CodeGuard-style rules systematically across the target codebase.
• Exploratory analysis: Uses agents to look for product-specific or design-level weaknesses that aren’t covered by existing rules.

When exploratory agents uncover a real vulnerability that no current rule would catch, Foundry records that as a rule gap. The self-improvement loop then uses this to extend or refine the CodeGuard rule set.

Over time, this creates a powerful virtuous cycle: CodeGuard prevents known classes of bugs in new code, and Foundry discovers new patterns and feeds them back as additional rules. This is similar in spirit to how AI is being used in other large-scale security efforts, such as in AI-driven improvements to Linux kernel security and projects like Project Glasswing.

The self-improvement loop: turning exploits into tests

One of the most powerful ideas in this approach is how it treats successful exploitation attempts.

When the agent system finds a vulnerability and proves it can be exploited against a live or test system, that exploit doesn’t just become a one-off artifact. Instead:

• The exploit code can be turned into a unit test or regression test.
• The finding itself becomes a new specification for how the system should behave securely.
• A coding agent—or a human engineer—implements the fix guided by that spec and test.
• The system then runs regression and variant testing to ensure similar issues don’t exist elsewhere.

This closes the loop from detection to durable prevention, and it’s where agentic systems start to feel less like scanners and more like continuously learning security teammates.

How to start: a practical first use case

If you’re a security or engineering leader interested in experimenting with agent-based security evaluation, it’s important not to start by pointing it at your entire organization’s codebase.

Instead, choose a bounded, source-available application with:

• A clear single owner.
• A known architecture.
• A scope that’s easy to understand and measure.

Good starting targets include:

• A single microservice with an API and admin portal.
• One small but representative repository that’s actively maintained.
• A non-critical internal tool where you can safely experiment.

Run Foundry (or a Foundry-inspired setup) against this bounded target and focus on answering a few key questions before you expand.

What evidence to collect before scaling up

To decide whether to roll out agentic evaluation more widely, you should look closely at both the findings and the evidence behind them.

Key dimensions to assess include:

• Finding quality: Are the vulnerabilities real, exploitable, and relevant to your environment? Are they just re-discovering known issues, or also finding new ones?
• Evidence quality: Does each finding come with clear, specific evidence—such as exploit steps, logs, or requests—that engineers can understand and reproduce?
• Rule coverage and gaps: Did the system uncover gaps in your existing secure coding rules or policies? Were the new or updated rules that emerged from the run precise and actionable, or still too generic?

Use this initial pilot to define your “definition of good” and “definition of done” for AI-assisted security evaluations. Once you have that baseline, you can expand to more repositories and services with much more confidence.

Bringing it all together

Frontier AI models can transform how we find and fix security vulnerabilities—but only if they’re wrapped in the right structure. The Foundry Security Spec provides a blueprint for that structure: clear agent roles, principled constraints, and a lifecycle that favors evidence over guesswork.

Paired with CodeGuard’s secure coding rules and a self-improving loop, this approach doesn’t just find bugs. It helps teams evolve their entire secure development lifecycle for an era where AI is both writing and reviewing code.

Start small, measure carefully, and treat your agentic system as a product that learns over time. Done right, AI agents can become a core part of your detection and prevention strategy—not just another noisy tool in the stack.

Share:

Comments

No comments yet. Be the first to share your thoughts!

More in Threat Detection