GoKawiilThe AI Security Institute (AISI) conducted evaluations of Anthropic’s Claude Mythos Preview (announced on 7th April) to assess its cybersecurity capabilities. Our results show that Mythos Preview represents a step up over previous frontier models in a landscape where cyber performance was already rapidly improving.
We have tracked AI cyber capabilities since 2023, building progressively harder evaluations to keep pace with AI progress — from chat-based probing, to capture-the-flag challenges, to the multi-step cyber-attack simulations described below. Two years ago, the best available models could barely complete beginner-level cyber tasks. Now, in controlled evaluations where Mythos Preview was explicitly directed and given network access to do so, we observed that it could execute multi-stage attacks on vulnerable networks and discover and exploit vulnerabilities autonomously – tasks that would take human professionals days of work.
In this blog post, we summarise results of cyber evaluations we ran on Mythos Preview. These include both capture-the-flag (CTF) challenges and more complex ranges designed to simulate multi-step attack scenarios.
Capture-the-flag results
In CTF challenges, AI models must identify and exploit weaknesses in target systems to retrieve hidden “flags”. The chart below shows Mythos Preview’s performance on our cyber CTF suite compared to other models. Each point represents a model's average success rate at a given difficulty level.
Figure 1: Performance on technical non-expert and apprentice level Capture the Flag tasks (CTFs) for models since November 2022. GPT-3.5 Turbo through to Claude 4 Opus average 10 runs up to 2.5M tokens. GPT-5 through to Mythos Preview average 5 runs up to 2.5M tokens.
Figure 2: Performance on practitioner and expert level Capture the Flag tasks (CTFs) for models since August 2025. All models average 5 runs up to 50M tokens.
On expert-level tasks — which no model could complete before April 2025 — Mythos Preview succeeds 73% of the time.
Cyber range results
Even expert-level CTFs only test specific skills in isolation. Real-world cyber-attacks require chaining dozens of steps together across multiple hosts and network segments — sustained operations that take human experts many hours, days, or weeks to complete.
... continue reading