GoKawiilStudy finds 90% of US work hours untested by leading AI benchmarks
A new analysis mapped the US Department of Labor's O*NET taxonomy of 923 occupations and nearly 18,800 workplace tasks against 29 widely used AI benchmarks, including Mercor's APEX suite. The researchers found that only a small fraction of real-world work tasks, by weighted hours, are represented in these benchmarks, leaving roughly 90% of US working hours without benchmark coverage.
GoKawiil's interpretation of the reporting above, not reported fact.
The finding suggests that claims about AI capability based on benchmark performance may not translate into readiness for the bulk of actual paid labor, since most benchmarks skew toward narrow, easily graded tasks rather than the messy, context-heavy work that fills most jobs. This could mean policymakers and companies assessing AI's economic impact are relying on an incomplete picture of what models can actually do in real workplaces.
- Researchers mapped 18,797 O*NET tasks across 923 occupations against 29 AI benchmarks.
- Only a small share of real work tasks, by hours worked, are represented in current benchmarks.
- The gap raises questions about using benchmark scores to judge AI's readiness for real economic tasks.
Source: mercor.com, 2026-10-16
Published there as: “90% of US working hours are spent on tasks not covered well by AI benchmarks”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.