Skip to main content

One post tagged with "mythos"

View All Tags

· 7 min read
Arkin Eliezer Julijanto

Meta Description: Anthropic's Mythos found thousands of zero-days — but small open models recovered the same analysis at a fraction of the cost. The moat in AI security is the system, not the model.

The Mythos Moment​

On April 7, 2026, Anthropic announced Claude Mythos Preview and Project Glasswing — a $100M+ initiative to use a frontier AI model to autonomously find and patch zero-day vulnerabilities in critical software. The results were staggering: thousands of vulnerabilities across every major OS and web browser, including a 27-year-old bug in OpenBSD and a fully autonomous remote code execution exploit against FreeBSD.

It was a watershed moment for AI cybersecurity. The narrative was clear: you need the biggest, most expensive model to do real security research.

Then AISLE ran the same vulnerabilities through small, cheap, open-weights models.

The results? 8 out of 8 small models detected Mythos's flagship FreeBSD exploit. One of them had only 3.6 billion active parameters and costs $0.11 per million tokens. A 5.1B-active parameter model recovered the core exploitation chain of the 27-year-old OpenBSD bug.

This changes the conversation entirely.

What AISLE Actually Tested​

AISLE took the specific vulnerabilities Anthropic showcased in their Mythos announcement, isolated the relevant code, and ran them through a range of small and open-weights models. Three tests stood out:

Test 1 — FreeBSD NFS Remote Code Execution (CVE-2026-4747): Mythos's crown jewel — a 17-year-old bug giving unauthenticated root access via NFS. Every single model they tested, from Kimi K2 to Qwen3 32B to a tiny GPT-OSS-20b (3.6B active params), correctly identified the stack buffer overflow and assessed it as critical. Detection was commoditized.

Test 2 — OpenBSD SACK Analysis (27-year-old bug): This one required mathematical reasoning about signed integer overflow. Much harder. Most models struggled — but GPT-OSS-120b (5.1B active) recovered the full public chain. Kimi K2 got close with an A- grade.

Test 3 — OWASP False Positive Discrimination: A Java servlet that looks like SQL injection but isn't. This is the test that flipped the script entirely — small, cheap models outperformed frontier models. Claude Sonnet 4.5 confidently mistraced the data flow. GPT-OSS-20b at $0.11/M tokens got it right. DeepSeek R1 nailed it across four trials. OpenAI o3 gave the most nuanced answer of all.

The pattern is clear: there is no stable "best model for cybersecurity." The capability frontier is jagged. Rankings reshuffle completely depending on the task.

Why This Matters for Developers and Security Teams​

The Mythos narrative suggests you need a frontier model — expensive, proprietary, limited access — to do meaningful AI security work. The evidence says otherwise.

Here's what the AISLE research actually demonstrates:

  • Detection is cheap. A $0.11/M token model can spot a critical buffer overflow just as well as a model that costs 50-100x more. The FreeBSD exploit was detected by every model tested, regardless of size.
  • The system is the moat, not the model. Anthropic's own scaffold — launch containers, scan files, use ASan as a crash oracle, rank by attack surface, run validation — is the same kind of system others in the field have built. The value is in the orchestration, targeting, and validation pipeline, not the specific model.
  • Coverage beats precision. Because small models are sufficient for most detection work, you can deploy them broadly across entire codebases instead of carefully rationing an expensive model. A thousand adequate detectives searching everywhere will find more bugs than one brilliant detective guessing where to look.
  • False positive management is the real differentiator. The OWASP test showed that bigger models often worse at distinguishing real vulnerabilities from false alarms. In production security systems, drowning reviewers in noise is worse than missing some bugs — it's what killed curl's bug bounty program.

The Bigger Picture: AI Security Economics​

This research has implications beyond vulnerability research:

For startups and indie security teams: You don't need to wait for Mythos-level access or budget. Open-weights models wrapped in good engineering can produce production-grade results. AISLE has been doing exactly this since mid-2025 — 180+ externally validated CVEs across 30+ projects, including 15 CVEs in OpenSSL.

For the open-source ecosystem: The security of critical infrastructure shouldn't depend on a single company's proprietary model. When small open models can detect the same bugs, it democratizes security research and makes the ecosystem more resilient.

For AI strategy: The "jagged frontier" phenomenon isn't limited to cybersecurity. It suggests that across many specialized domains, the relationship between model size and capability is non-linear. Smart system design can compensate for — or even outperform — raw model scale.

What Good AI Security Systems Actually Need​

Based on AISLE's production experience, here's what matters more than model size:

  • Targeting intelligence: Knowing which files and functions to examine in a codebase with hundreds of thousands of files
  • Iterative validation: Hypothesis → test → refine loops that separate real bugs from noise
  • Maintainer trust: The metric that actually matters is whether project maintainers accept your findings and patches
  • Full-loop remediation: Discovery is only the first step. Verification, severity assessment, patch generation, and coordination close the loop

The OpenSSL CTO praised AISLE's work for "high quality reports and constructive collaboration throughout remediation." That trust is earned through the system, not the model.

Conclusion​

Anthropic's Mythos proved that AI cybersecurity is real and here. AISLE's research proved it doesn't require a monopoly.

The frontier is jagged. Small models can do big things when wrapped in the right system. For developers, security teams, and anyone building AI-powered tools, the takeaway is empowering: you don't need the biggest model. You need the best system.

The economics of AI security just got a lot more interesting — and a lot more accessible.


FAQ​

Q: Can small AI models really replace frontier models for security research? A: Not replace — complement. AISLE's research shows small models can detect many of the same vulnerabilities as frontier models at a fraction of the cost. But the full pipeline (discovery, triage, validation, remediation) requires orchestration. The system matters more than any individual model.

Q: What is the "jagged frontier" in AI cybersecurity? A: It means AI capability doesn't scale smoothly with model size. A tiny 3.6B-parameter model might outperform a frontier model on one task and fail completely on another. There's no single "best" model across all cybersecurity tasks — rankings reshuffle depending on what you're testing.

Q: How are companies using small AI models for vulnerability detection? A: AISLE has been running a production system since mid-2025 using model-agnostic pipelines. They've found 180+ externally validated CVEs across 30+ projects including OpenSSL and curl. Their approach deploys cheap models broadly for coverage rather than rationing expensive models selectively.

Q: Should I use open-weights models instead of proprietary ones for security tooling? A: It depends on your use case. For detection and triage, open-weights models are often sufficient and far more cost-effective. For complex exploit construction requiring multi-step reasoning, larger models may still have an edge. The best approach is model-agnostic — use whatever works best for each specific task.


About the Author: Arkin is a software engineer and startup founder building Orbits Studio. He writes about AI, developer tools, and the intersection of technology and security. Find him at arkin-dev.com.