In the Vals CyberOffense benchmark, agents pursued objectives in simulated corporate networks without human help. GPT-6 Sol Daybreak averaged 15.7/21 objectives in Nova and reached 27/31 in Lighthouse; Deepseek V4.1 Flash 1017 scored 16.3/21 and 21/31. Vals reports approximate run costs of $22 and $3, respectively.

Deepseek made no progress in its first 25 million tokens and achieved most of its progress after passing 100 million. These are simulation results. Separately, the UK AI Security Institute estimates the cyber-capability gap between leading open and closed models at four to seven months.

Why it matters

Cheaper open models could make defensive testing more accessible, while the same capabilities also raise misuse risks.

GUIDES

Something to try

As a hypothetical tabletop exercise, compare attack testing with defensive vulnerability discovery in an authorized simulation.