METR expands AI benchmark to test research judgment, not just coding speed
METR, the AI research organization known for its time-horizon benchmark, has added new tasks measuring AI judgment on fuzzier work like designing evaluations and scoring philosophical arguments. In both new tests, top AI models performed worse than humans—none of the AI-generated evaluations were deemed useful by METR staff, and models also lagged on judging philosophical arguments.