Measuring LLMs' ability to develop exploits
Mythos Preview has demonstrated a significant ability to develop exploits, surpassing previous models. Recent benchmarks, including ExploitBench and ExploitGym, have shown that Mythos Preview consistently outperforms other evaluated models in exploit development. This suggests that the expertise required to create exploits may decrease as such advanced capabilities become more accessible.
- ▪Mythos Preview can find complex vulnerabilities and create complete end-to-end attack chains.
- ▪ExploitBench measures the ability of language models to write complete exploits rather than just proofs of concept.
- ▪Mythos Preview has been tested against new benchmarks and has shown superior performance compared to other models.
Opening excerpt (first ~120 words) tap to expand
May 22, 2026 Newton Cheng, Keane Lucas, Winnie Xiao, Nicholas Carlini, and Milad Nasr Introduction Claude Mythos Preview’s ability to develop exploits is a step-change over previous frontier models. This was one of our primary motivations for rolling out the model carefully through Project Glasswing rather than through a general release. Mythos Preview is capable of finding complex vulnerabilities, but what concerned us most in our internal testing was that Mythos Preview could both turn vulnerabilities into exploit primitives, and combine those primitives together into complete end-to-end attack chains. When we published our Mythos Preview results, we measured its capabilities by having it search for novel zero-days and then build exploits for them.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Anthropic.
Discussion
0 commentsMore from Anthropic
