Case Study · Structured Multi-Agent Security Research

From scattered CVE intel to a defensible fix verdict.

A container-security company remediates vulnerabilities automatically - AI agents that research, patch, test, and ship a fix for every vulnerable image. We built the stage that comes first: a structured, event-driven multi-agent workflow on the Claude Agent SDK that, for each CVE, finds the upstream fix, judges applicability and backport feasibility, and captures exploits and PoCs - an evidenced verdict the downstream patching agents can act on.

Container Security · Vulnerability Remediation
Event-driven · ECS
Claude Agent SDK
Feeds an agentic patch pipeline

Published with the engagement anonymised at the client's request - a confidentiality bar we treat as part of the deliverable.

As-built · verifiedby the delivery team
0
Specialized agents in a structured pipeline
0
MCP research tools across sources
0
Platform services & integrations
0
Research UI surfaces for reviewers
The client

A container-security company whose platform automatically remediates vulnerabilities in open-source images - AI agents that research, patch, test, and ship fixes across thousands of images and 40+ ecosystems. Identity withheld at the client's request.

The challenge

Automated patching only works if the research beneath it is right. For every CVE the platform had to answer - does a fix exist, does it apply, can it be backported - fast enough to keep up with disclosures, and defensibly enough to patch on.

The build

A structured seven-agent workflow on the Claude Agent SDK that gathers evidence, identifies the fix commit, judges applicability and backport feasibility, and captures exploits/PoCs - all traced.

How it runs

Event-driven on ECS. CVE-feed events or a manual UI trigger start a job; the report lands in PostgreSQL, artifacts in S3, and a review ticket in Jira - reviewers work in a dedicated dashboard.

01The client

A security company whose product fixes vulnerabilities itself.

The client is a container-security company whose platform automatically remediates vulnerabilities in open-source container images and packages. Rather than telling teams to upgrade, its product ships a fixed image: a fleet of specialized AI agents that research, patch, test, and deliver a remedy for each vulnerable package - the "shift out of the CVE grind" the company is built to sell. Identity withheld at the client's request.

That promise runs at serious scale - thousands of container images across dozens of language ecosystems (Python, Node, Java, Go, Ruby, PHP, Rust, .NET, and many more), each with its own upstream repositories, advisories, and release cadence. Every disclosed CVE is a question the platform has to answer, automatically and defensibly, before any patch can be written.

The first stage of that agentic loop is research: for a given CVE, does an upstream fix exist, where is it, does it apply, and can it be backported cleanly. Get that wrong and every downstream patch is wrong too. This engagement built that research stage - the foundation the patch, test, and ship agents stand on.

Container image remediation
Open-source packages
40+ language ecosystems
Agentic patch · test · ship
CVE research at scale
Backport without forced upgrades

One research stage, two consumers

The verdict feeds an automated pipeline and the people who oversee it.
The downstream remediation agents
  • Patch, test, and ship the fix
  • Need the exact upstream fix commit
  • Need PoCs to validate the patch
  • Consume the verdict programmatically
The security engineers & customers
  • Oversee and trust the automation
  • Need evidence, not just a score
  • Must defend every determination
  • Clarity beats volume
One machine consumer, one human one - every verdict must carry the exact fix reference and the evidence behind it, or neither can act on it.
02The challenge

The information existed.
Stitching it together, at scale, was the job.

Nothing about CVE research is one lookup. A defensible verdict means reconciling a dozen fragmented sources - consistently, for every CVE, without missing the one advisory that changes the answer. The traditional way is manual, expert-bound, and impossible to scale with the disclosure rate - which is exactly why it had to become reliable automation feeding the patch pipeline, not a human bottleneck in front of it.

Fragmented sources

CVE databases, upstream repos, vendor advisories, mailing lists, forums, and exploit databases each hold a piece. The answer only appears once they're correlated - by hand, every time.

Finding the real fix commit

A CVE rarely points cleanly at its patch. Researchers trace references across commit history, pull requests, and advisories to find the commit that actually remediates the issue - and confirm it does.

Applicability is not obvious

Whether vulnerable code is even present in the shipped versions requires reading the fix and comparing it against the codebase - not just trusting a version range in a feed.

Backport feasibility unknown

Even when a fix exists, whether it can be cleanly carried back to older supported versions depends on how far the code has diverged - a judgement that took real digging.

Inconsistent, unrepeatable output

Two researchers, two write-ups. Without a fixed process, coverage and depth varied CVE to CVE - and there was no guarantee the same sources were checked each time.

It didn't scale

The disclosure rate outpaces any human team. Research became a bottleneck - the backlog grew, and expert time was spent gathering information instead of judging it.

Why it mattered

The company's whole promise is that vulnerabilities get fixed automatically, in minutes - but a patch is only as trustworthy as the research that says a fix exists and applies. Weak research doesn't just slow the pipeline; it ships the wrong patch. Making the research stage fast, thorough, and defensible was the foundation the entire agentic remediation loop depends on.

03The mandate

Three non-negotiables, fixed before a line of code.

Focaloid engaged as the client's AI engineering partner to automate CVE research without sacrificing the rigour a security team lives by. The constraints below were set as hard requirements, not aspirations.

Non-negotiable · 01

Evidence over assertion

Every determination grounded in a real source - a commit, an advisory, a thread - never the model's unsupported recollection. If it isn't cited, it isn't in the report.

Non-negotiable · 02

Structure & auditability

A fixed, repeatable pipeline that researches every CVE the same way, and traces every step - so a verdict can be replayed and defended, not just trusted.

Non-negotiable · 03

Automate, but escalate

A confident, evidenced verdict flows straight to the patch pipeline; anything short of that is escalated to a human, never guessed. The system knows the difference - and the report makes both paths auditable.

The decision that shaped everything: reproduce the researcher's method, don't replace their judgement. We modelled the exact sequence a senior researcher follows - details, evidence, fix, applicability, backport, exploits, report - as a structured chain of specialized agents. The output is a case, not a verdict handed down from a black box.

A gated path from manual research to an automated pipeline
P1

Method capture

Shadowed the researchers to formalise the exact steps, sources, and decision points of a manual CVE investigation.

P2

Agent & tool design

Split the method into seven single-responsibility agents and designed the MCP tool surface each needs - with evidence and a fixed, repeatable sequence as constraints.

P3

Pipeline build

Built the structured workflow on the Claude Agent SDK, the event-driven runtime on ECS, and the reviewer UI over the report store.

P4

Tracing & hand-off

Wired Langfuse tracing, the Jira review hand-off, and evaluation of tool selection and verdict quality end to end.

Scope discipline as a safety feature. This system researches and reports - it does not apply patches. Backport feasibility is surfaced here; the actual patching and fix validation are carried out downstream by a dedicated patcher agent in a separate project. And exploit/PoC work is deliberately bounded to identification and capture - no fix testing happens in this pipeline.

04The solution

A week of manual research became a traceable, repeatable job.

A CVE-feed event or a manual trigger starts a research job. Seven specialized agents run in a fixed order - each gathering, analysing, and handing off - and the job ends with a structured report: does a fix exist, where is the commit, does it apply, can it be backported, and what exploits or PoCs are known. Every step is traced; a human reviews and signs off.

What a single research job answers
Determined & reported
Captured for downstream use
Is this CVE relevant to our versions?
Does an upstream fix exist?
Which commit is the fix?
Is the vulnerable code present here?
Can the fix be backported?
Which distros are fixed / unfixed?
Are there known exploits or PoCs?
Where are the PoC references?

Watch a CVE become an evidenced verdict.

The simulation on the right replays a research job: the pipeline works through its agents - gathering details and evidence, pinning the fix commit, judging applicability and backport, capturing PoCs - and then narrates a verdict grounded in what it found, never asserted.

  • The research is visible. Each agent's tool calls stream as they run - you can see which source produced which finding.
  • Every claim cites its evidence. The fix commit, the applicability call, the distro status - each links back to the source it came from.
  • Undetermined is a valid answer. When evidence is insufficient, the agent says so and flags for review - it never fills the gap with a guess.
  • It ends in a hand-off. The report persists, artifacts are stored, and a Jira ticket is raised for a researcher to validate.
Research job · CVE-2025-XXXXX · structured pipeline
↻ replay

Illustrative research job with a representative CVE - in the delivered system every finding is produced live from real sources and fully traced.

05Architecture

Not one clever agent. Seven, in a fixed line, over a shared tool surface.

The workflow is a structured chain of seven specialized agents on the Claude Agent SDK, running as an event-driven service on ECS. An MCP server exposes the research tools; a REST API and UI give reviewers control and visibility; PostgreSQL, S3, and Langfuse hold state, artifacts, and traces. Each agent has one job and hands off to the next.

Structured hand-offs · full trace on every hop
Reviewer surface
Research UI
Dashboard · CVE detail · research history · manual trigger
Control plane
REST API
Job control & queries - backs the UI and external callers
Vulnerability Research Workflow
Claude Agent SDK · Python · ECS
Structured 7-agent pipeline
fixed order · single-responsibility agents · every step traced
no free-form routing
↓ each agent hands its output to the next
1 · Details
CVE baseline facts
2 · Evidence
multi-source sweep
3 · Fix commit
identify & analyze
4 · Applicability
impact & distros
5 · Backport
feasibility surfaced
6 · Exploit/PoC
capture, no testing
7 · Report
assemble & hand off
Tool surface
MCP Tool Server - research tooling
Exposes 16 structured tools with a consistent response envelope; each agent calls only the tools its stage needs
CVE intelligence
source & patch
web research
exploits / PoC
Sources of truth
Research sources
NVD · MITRE · GitLab commit history · vendor advisories · mailing lists · forums · exploit databases · open web
State
PostgreSQL
Research jobs, reports, and per-CVE history
Artifacts
S3
Captured evidence, PoC references, report artifacts
Observability
Langfuse
LLM tracing & evals - every agent step, tool call, and token cost

The hand-off is event-driven. When the pipeline finishes, the report is written, artifacts are stored, and a confident verdict is dispatched to the patch agents - job_queued → agent_run ×7 → report_written → dispatched / escalated - so the remediation loop continues automatically, with a Jira ticket for oversight.

Evidence provenance recorded end to end
Synchronous request
Event-driven hand-off
External sources & systems - GitLab, Jira, CVE feeds

The as-delivered system - redrawn from the engagement's architecture documentation, anonymised.

The life of a research job, end to end

STEP 01

A job is triggered

A CVE-feed event or a manual UI action enqueues a research job on ECS. The pipeline picks it up asynchronously.

STEP 02

Details are gathered

The first agent establishes the CVE's baseline facts - affected components, severity, references - from NVD and MITRE.

STEP 03

Evidence is swept

The evidence agent pulls candidate fixes and discussion from repos, advisories, mailing lists, forums, and the open web.

STEP 04

The fix commit is pinned

The fix-commit agent locates the remediating commit in GitLab history and analyses what it actually changes.

STEP 05

Applicability is judged

The applicability agent checks whether the vulnerable code is present in the shipped versions and which distros are fixed.

STEP 06

Backport & PoCs

Backport feasibility is surfaced; known exploits and PoCs are identified and captured - no fix testing here.

STEP 07

Report & hand-off

The report agent writes findings to PostgreSQL, stores artifacts in S3, and dispatches a confident verdict to the patch pipeline (or escalates via Jira).

THE RESULT

Evidenced, not guessed

Every step is traced in Langfuse - a verdict a researcher can replay, defend, and sign off in minutes.

06Inside the build

The decisions that made it trustworthy.

Wiring a model to a search box is easy. The engineering that makes a security-research agent defensible lives in a handful of deliberate choices - each one below is as-built.

Decision A - Orchestration

A structured pipeline, not one open-ended agent.

A single agent told to "research this CVE however you like" is unpredictable - it takes different paths on identical inputs, skips sources, and produces write-ups you can't compare or replay. For a security decision, that's disqualifying.

So the researcher's method is encoded as a fixed chain of seven single-responsibility agents. Each has a narrow prompt, its own slice of the tool surface, and a defined hand-off. The same CVE takes the same path every time - which is what makes a verdict auditable and a regression detectable.

Structure is the feature. Because the pipeline is fixed, tool selection at each stage can be evaluated on its own - an automated question bank checks that each agent reaches for the right source - and run as a first-class test discipline alongside unit and integration tests, so a change that degrades research quality is caught before it ships.
Decision B - Grounding

The research surface, exposed as a curated MCP tool set.

This is why verdicts are evidenced rather than recalled. Instead of letting the model answer from general knowledge, the sources a researcher uses are exposed through a dedicated MCP tool server, grouped by capability. The agents don't recall a fix commit - they fetch it, diff it, and cite it.

CVE intelligence

Query the primary vulnerability databases and normalise severity and affected-product identifiers into comparable form.

Source & patch analysis

Fetch upstream repository state, inspect the exact changes in a candidate fix commit, read merge requests, and compare code across releases.

Open-source research

Broad web research plus retrieval and parsing of advisories, mailing-list archives, changelogs, and forum threads.

Exploits & PoCs

Locate published exploits and proof-of-concept code, and capture their references into the report.

{ "status": "ok", "capability": "source & patch analysis", "source": "…/commit/9f3a1c", "evidence": { …fetched from the real source } }

Why MCP over ad-hoc scripts? A clean, standardised contract between reasoning and sources. Every tool returns the same envelope - status, what ran, the source reference, and the evidence - which makes tool selection reliable, keeps provenance attached to every finding, and lets new sources ship without changing how the agents call them.

Decision C - Coverage

Evidence from every corner a researcher would check.

A verdict is only as good as the sources behind it. The evidence agent is built to sweep the same breadth a senior researcher would - and the report records which corners were checked, so a thin evidence base is visible rather than hidden.

PRIMARY

Databases & repos

NVD, MITRE, and upstream GitLab history - the canonical record of the vulnerability and its fix.

HUMAN SIGNAL

Advisories, lists & forums

Vendor and distro advisories, project mailing lists, and forums - where fixes are discussed before they're formalised.

EXPLOITABILITY

Exploits & PoCs

Exploit databases and PoC references - captured to inform severity and downstream fix validation.

Decision D - Anti-hallucination

Every citation is cross-checked against a real tool call.

A language model will happily invent a plausible commit hash or a CVE advisory URL that doesn't exist. In a pipeline that feeds automated patching, a fabricated source isn't a cosmetic error - it ships the wrong fix. So the system treats every claim as guilty until a real retrieval proves it: nothing enters the report unless it traces back to a source the agent actually fetched.

CITATION-BACKED

No claim without a source

Every determination in the report carries a citation - a commit URL, an advisory, a thread. A claim with no reference is not written as fact, full stop.

GROUNDED QUOTES

Claims verified against fetched content

A quoted commit message, advisory line, or version string is checked against the payload that was actually returned - not paraphrased from the model's memory of it.

NO-GUESS

Evidence-or-undetermined

When retrieval doesn't support a confident call, the result is an explicit "undetermined" and an escalation - never a plausible-sounding fill-in.

BOUNDED

Tool-call caps per stage

Each agent runs under a call budget, so a stuck stage can't spiral into runaway cost or latency - the job stays bounded, always.

READ-ONLY

No writes to source systems

Every research tool is read-only. The pipeline observes and records; it never mutates a repository, advisory, or upstream system.

A verification pass runs before any report is finalized. It walks every determination and confirms two things against the Langfuse trace: the cited source appears in the run's actual tool-call log, and the claimed fact actually appears in the content that call returned. Anything that fails either check is dropped or downgraded to undetermined - so the report can only ever say what the tools genuinely retrieved. Fabricated commits, invented advisory URLs, and "remembered" version ranges cannot survive it.
Decision E - Scope boundary

Exploits and PoCs: identify and capture, never test.

Knowing an exploit exists matters for severity and for the team that later validates a fix. But running exploits or testing patches is a different discipline with different risks - and it belongs to the downstream patcher project, not here. So this pipeline draws a hard line: identify, capture, reference. Nothing is executed.

1

Identify

Locate known exploits and PoC code for the CVE across exploit databases and the open web.

2

Capture

Record the PoC references and artifacts into the report and S3 - with their provenance.

3

Hand off

The captured PoCs travel with the report to the patcher agent, which uses them to validate a backported fix - in its own project.

The boundary is deliberate. This system produces the intelligence to act on a CVE; a separate, dedicated patcher agent does the acting. Keeping research and remediation apart keeps each one auditable.

Decision F - Report & hand-off

A verdict that assembles itself, with a contract.

The final agent turns seven stages of findings into one structured report. Every field carries its evidence, and the whole thing is written once, keyed by CVE and job, then handed to a human - the same six-beat shape every time.

01

Collect

Gathers each prior agent's findings and their cited sources.

02

Verdict

States fix existence, commit, applicability, and backport feasibility.

03

Attach evidence

Every claim links to the source that produced it.

04

Persist

Report to PostgreSQL, artifacts and PoCs to S3.

05

Raise ticket

A Jira review ticket is opened for a researcher.

06

Trace

The full run lands in Langfuse, replayable end to end.

07Auditability & trust

A verdict you can replay - not one you have to believe.

In security work, an answer no one can defend is worse than no answer. Every finding the system produces is bound to the source that produced it, every run is traced end to end, and a human signs off before anything is acted on. Trust is structural here, not a matter of reputation.

Link 1

Source

A tool fetches from a real source - a commit, an advisory, a thread - and returns it with its origin reference.

Link 2

Extraction

The agent extracts a finding and keeps it attached to the source reference it came from.

Link 3

Correlation

Findings across sources are reconciled into a determination - agreements and conflicts both recorded.

Link 4

Verdict

The report states the verdict with its evidence - and a Langfuse trace that replays how it was reached.

CVE
Job run
Evidence

Every claim is bound to this triple. A determination without a cited source cannot enter the report; a report is namespaced by CVE and job; every finding keeps its evidence reference. An unsupported "not affected" isn't caught in review - it's structurally impossible.

DeterminationProduced byEvidence-backedIn the reportNotes
Fix existsFix-commit agent
Yes
Yes
Backed by the identified upstream commit
Upstream fix commitFix-commit agent
Yes
Yes
Exact commit reference + diff analysis
Applicability to our versionsApplicability agent
Yes
Yes
Vulnerable-code presence, not just a version range
Backport feasibilityBackport agent
Surfaced
Yes
Assessment only - patching is a downstream project
Known exploits / PoCsExploit/PoC agent
Yes
Captured
Identified & referenced - not tested here
Insufficient evidenceAny agent
Undetermined
Flagged
Escalated for human review, never guessed

"Undetermined" is part of the product.

When the evidence doesn't support a confident call - conflicting advisories, no traceable fix, an ambiguous version range - the system doesn't manufacture certainty. It records what it found, marks the determination undetermined, and routes the CVE to a researcher with the open questions spelled out. In security research, a well-scoped "we don't know yet" is a feature with a UX.

Evidence bound to every claim
Citations cross-checked vs tool-call log
Full Langfuse trace per run
Per-CVE research history
Human sign-off via Jira
Read-only source access
08Infrastructure

Event-driven, containerised, observable from day one.

The pipeline runs as an event-driven service on ECS: jobs arrive from CVE-feed events or the UI, flow through the seven agents, and land as an evidenced report. State, artifacts, and traces are separated by concern, and the whole stack runs locally for development.

The trigger & runtime

How a job starts and runs

  • CVE-feed events and manual UI actions enqueue research jobs
  • The agent pipeline runs as a containerised service on ECS
  • Jobs are processed asynchronously - the disclosure stream never blocks a reviewer
  • An MCP tool server runs alongside, exposing the research surface
  • A REST API backs the UI and any external callers

Separation of concerns - reasoning, tools, state, and traces each scale and fail on their own.

The hand-off & integrations

Where a job ends up

  • Reports and per-CVE history persist in PostgreSQL
  • Evidence, PoC references, and artifacts stored in S3
  • Review tickets raised in Jira for human sign-off
  • Source access to GitLab commit history and the open web
  • Every run traced in Langfuse - quality and cost, visible from day one
PostgreSQL

Research jobs, structured reports, and a timeline of research attempts per CVE.

Why: relational and durable - reports, verdicts, and history are exactly the structured, queryable shape the reviewer dashboard needs.

S3

Captured evidence, PoC references, commit diffs, and report artifacts.

Why: cheap, durable object storage for the raw material behind every verdict - kept alongside the report so provenance travels with the decision.

Langfuse

LLM tracing and evaluation across every agent step, tool call, and token cost.

Why: a quality lens and a cost lens from day one - and the mechanism that lets any verdict be replayed and any regression in tool selection be caught.

09Outcomes

What shipped, and what it changes.

The build facts below are as-delivered and verified by the engineering team. We deliberately don't publish performance percentages for an anonymised engagement - so the impact is described in terms of what changed in the work, not invented figures.

✓ As-built · verified

The delivered system, in countable facts

0
Specialized agents in a fixed-sequence pipeline
0
MCP research tools across CVE, source, web & exploit sources
0
Platform services & integrations, event-driven on ECS
0
Reviewer UI surfaces - dashboard, detail, history, trigger
Before - manual researchAfter - the research agent
Hours per CVE spent gathering scattered evidence by handA triggered job sweeps every source automatically and returns an evidenced report
Coverage and depth varied by researcher and by dayA fixed pipeline researches every CVE the same way - repeatable and comparable
Write-ups hard to audit or reproduceEvery claim cites its source; every run replayable in Langfuse
Fix commit and backport call buried in tribal knowledgeFix commit, applicability, and backport feasibility stated with evidence
Experts spent time gathering, not judgingExperts review evidenced verdicts and sign off - judgement, not legwork
Backlog grew with the disclosure rateResearch runs continuously and event-driven, at machine throughput
Faster triage

CVEs are researched as they surface - evidence-gathering that took hours runs unattended, so reviewers start from a finished case.

Consistent rigour

Every CVE, the same method. The structured pipeline removes the variance between researchers and the risk of a skipped source.

Higher leverage

Expert time moves to judgement. People validate and act on evidenced verdicts instead of assembling them from scratch.

10What's next

Built to feed what comes after research.

The pipeline was designed as one half of a bigger loop: research produces the evidenced case; a separate patcher agent acts on it. The choices above - structured stages, an MCP tool contract, captured PoCs - were made so that hand-off is clean.

The roadmap

Closing the loop

  • Full agentic loop - the evidenced verdict, fix commit, and captured PoCs flow straight into the platform's patch, test and ship agents that complete the remediation.
  • Broader sources - additional advisory feeds, distro trackers, and language-specific ecosystems added as new MCP tools.
  • Measured outcomes - once live, review-time and coverage metrics turn today's qualitative gains into verified numbers.
  • Confidence scoring - richer signals to help reviewers prioritise which evidenced verdicts to check first.
The reusable pattern

This generalises.

A structured multi-agent pipeline over an MCP tool surface, evidence bound to every claim, an explicit "undetermined" path, and a clean hand-off to a human - the pattern transfers to any research task that has to be right, repeatable, and auditable: threat intelligence, compliance evidence-gathering, due diligence, anywhere a defensible verdict matters more than a fast one.

Product Security
Threat Intel
Compliance
Due Diligence
Any evidence-based verdict

The stack, end to end

CA
Claude Agent SDK
MCP
MCP Tool Server
API
REST API
UI
Reviewer UI
ECS
AWS ECS
PG
PostgreSQL
S3
Amazon S3
LF
Langfuse
GL
GitLab
JR
Jira
Let's build

Have research your experts do by hand that should run itself?

We build structured, evidenced, auditable agent pipelines - the kind a security or research team can actually stand behind. Governed at every gate, shipped at AI speed.