Skip to content
Tech News
← Back to articles

METR expands AI benchmark to test research judgment, not just coding speed

read original more articles
GoKawiil Brief

METR, the AI research organization known for its time-horizon benchmark, has added new tasks measuring AI judgment on fuzzier work like designing evaluations and scoring philosophical arguments. In both new tests, top AI models performed worse than humans—none of the AI-generated evaluations were deemed useful by METR staff, and models also lagged on judging philosophical arguments.

Why It Matters

GoKawiil's interpretation of the reporting above, not reported fact.

The update comes amid heightened attention to AI's role in accelerating its own development, following an earlier METR report suggesting AI may already be doubling research speed at Anthropic. By including tasks that resist easy scoring, METR appears to be acknowledging that its original benchmark undersold the difficulty of automating open-ended research judgment, which could temper fears of imminent full recursive self-improvement.

Key Takeaways

Source: technologyreview.com — Grace Huckins, 2026-11-20

Published there as: “The most misunderstood graph in AI just got a major update”

Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.
Keep reading
anthropic 1029 briefs → reinforcement learning 30 briefs → recursive self-improvement 27 briefs → metr 24 briefs →