Threat intel4 min read
AI pilot sends 99 checked vulnerability reports to Express, Caddy, ImageMagick and three other projects
An AI scanner flagged 9,791 possible bugs in six open-source projects. After triage and hand review, maintainers got 99 private reports with patches, and 55 bugs were fixed by 15 November.
4Tify Team

GitHub and Microsoft have published the first results of a pilot in which an AI scanner looked for security bugs in six widely used open-source projects. The scanner flagged 9,791 possible issues. After automated triage and manual review, maintainers received 99 private reports, each with a proof of concept and a suggested patch. By 15 November, 55 of those bugs had been fixed and 14 of the fixes had shipped in a release.
What happened
The results were announced on 17 November 2026. The scanning was done by Codename MDASH, Microsoft's multi-model agentic scanning system, running an earlier version of Microsoft's MAI Cyber model. Microsoft announced MAI Cyber 2 the same day, but the pilot did not use it.
Raw output went through three filters before any maintainer saw it. An AI triage agent from GitHub Security Lab removed duplicates, checked whether each finding held up and pushed higher-impact bugs to the top. Seven researchers from Microsoft FORGE Lab then checked what was left by hand, spending about two days per project. That left roughly one report for every 100 raw findings.
Reports went out through GitHub private vulnerability reporting, so nothing was made public before the maintainers were ready. The maintainers decided what to fix, how, and when.
Who is affected
All six projects are alumni of the GitHub Secure Open Source Fund and agreed to the scan in advance:
- AutoGPT, an AI agent platform
- Caddy, a web server
- Express, the Node.js web framework
- FastMCP, a framework for building MCP servers
- ImageMagick, the image processing suite
- Netty, a Java networking framework
Some project-level numbers are public. Caddy received 22 reports and fixed all of them within a week. ImageMagick, which is run by two people, also received 22. One of those looked like a known class of bug, but the report showed where it differed and pointed at the exact lines, and the fix went in 15 minutes later. AutoGPT had shipped a new security check that turned out to be broken, and the report pointing at it arrived three days later. The team wrote its own fix within a day and had it in production in under a week.
No per-project breakdown has been given for Express, FastMCP or Netty. The announcement does not map the 99 reports to individual CVE or GHSA identifiers, and there is no report of any of these bugs being exploited.
Why it matters
Maintainers of popular projects are already buried in AI-generated bug reports, many of them duplicates or simply wrong. The pilot tested a different approach: check every finding before sending it, report it privately and include a patch. The maintainers' feedback was positive. Express maintainers said the reports cut through the noise, and a Netty maintainer said finding bugs is no longer the bottleneck, fixing them is.
The limits are clear too. Six projects is a small sample, and the manual review took seven researchers about two days per project. GitHub says it will offer the same checked scans to every alumni project of its fund, which so far covers 216 projects.
What to do now
A fixed bug only protects you once the release containing it is running in your environment. Of the 55 fixes, only 14 had reached a release by 15 November, so more updates are coming.
- Find out where you use these projects. Check direct and transitive dependencies for express, fastmcp, netty and ImageMagick bindings, and inventory any Caddy or AutoGPT deployments.
- Turn on Dependabot alerts or an equivalent dependency scanner. Once an advisory is reviewed into the GitHub Advisory Database, repositories that depend on the affected package get an alert.
- Read each new advisory against the version you actually run, then confirm that your next update includes the fix. Don't assume a patch merged upstream is in the package you install.
- If you maintain a project, enable private vulnerability reporting on your repository so researchers can reach you without disclosing in public.
The same lesson applies to in-house code. Our team uses AI-assisted code review in the way this pilot did: findings are checked by an analyst before they reach you, and each one comes with a concrete fix, so developers spend their time patching instead of triaging.