Across 19 models released between 2024-05-13 and 2026-04-07 in METR’s Time Horizon 1.1 data, the task length a model completes with 80% predicted success is a median 5.3x shorter (range 4.0x to 10.3x) than the length it completes with 50% success. The most recent point, an early Claude Mythos Preview, reaches 17.4 human-expert hours at 50% but only 3.1 hours at 80%, and METR cautions that estimates above 16 hours are unreliable. The gap matters because attackers can tolerate frequent failure: in the AI-agent hacking campaign covered in today’s SI News brief, Reuters reports that the agents attempted more than 1,200 sites and compromised more than 100, while businesses deploying agents generally need the reliable end of the curve.
Sources: METR, Task-Completion Time Horizons of Frontier AI Models (Time Horizon 1.1 data file; p50 and p80 horizon estimates and release dates) · Reuters (republished by RTÉ) · NorForge, Inside Cairn
Original story: SI News: Agents that hack, Anthropic’s public-benefit fund, Disney’s Character.AI hire
Note: METR’s horizon measures how long the tasks take human experts, not how long an agent runs. Points are point estimates; METR’s confidence intervals are wide (Claude Mythos Preview at 50%: 8.5 to 55 hours).