GoKawiilNew 'livenerf' benchmark tracks whether Anthropic's Claude Opus 5.5 degrades after launch
An open-source project called livenerf has begun a 30-day daily benchmark of Claude Opus 5.5, which launched September 22, 2026, to test long-standing claims that Anthropic quietly degrades models after release. As of September 29, the project has collected 6 of 10 baseline days using a fixed harness, pinned CLI version, and a panel of 2,336 GPQA Diamond and MMLU-Pro questions, with no missed days so far. The first statistical comparison window won't be possible until around October 24, 2026.
GoKawiil's interpretation of the reporting above, not reported fact.
The effort attempts to replace anecdotal 'model nerfing' complaints with a reproducible, statistically grounded record, using methodology drawn from Anthropic's own published error-bar framework so results are harder to dismiss as bias. If it finds measurable drift, it could pressure AI labs toward more transparency about post-launch changes to model weights, routing, or compute allocation; if it finds none, it may curb a recurring source of user distrust. The findings, however, will only cover this specific model and benchmark panel, not necessarily generalize to other models or use cases.
- livenerf is a 30-day daily benchmark tracking possible post-launch degradation of Claude Opus 5.5.
- It uses a frozen, deterministic testing setup (pinned CLI, fixed prompts, exact graders) and Inspect, the UK AI Security Institute's eval framework.
- The first meaningful comparison of performance drift is expected around October 24, 2026, after a 20-day baseline and testing window.
Anthropic Claude Max Subscription — This article centers on tracking Claude Opus performance over time using a Claude Max subscription, which is exactly what powers the livenerf benchmark. If you're curious about testing Claude's capabilities yourself or running headless Claude Code, a Max subscription gives you the access needed to dig into these questions firsthand.”
See Anthropic Claude Max Subscription on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: github.com, 2026-09-29
Published there as: “Livenerf: Has Opus 5.5 been nerfed yet?”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.