A developer has released Parsimony, an open-source benchmark that measures how much code AI models add when solving SWE-bench Verified tasks, rather than just whether they solve them. It parses patches to count syntax-level coding units added and removed, producing a net score, and ranks models by mean net units added on tasks they solved. The tool and underlying data are published on GitHub, with results also shown via an interactive leaderboard and charts.
parsimony.lasu.dev
· 2026-10-13
METR tested frontier AI models on a research task requiring them to invent a new post-training method that would outperform a strong GRPO baseline across question-answering and coding benchmarks. The human algorithmic innovation in the comparison outperformed the solutions generated by the AI models tested, according to METR's report.
epoch.ai
· 2026-10-09
DARPA announced after market close that it selected four companies to move into Stage C of its Quantum Benchmarking Initiative, a program designed to rigorously test quantum computing hardware. The stage carries roughly $300 million in funding, and DARPA indicated it may still admit additional firms in a second wave.
fastcompany.com
· 2026-10-08
Kapa, a platform that indexes company knowledge for AI agents, built an internal benchmark of 1,000 annotated eval cases from real production data to compare retrieval methods. The company tested seven retrievers—including traditional hybrid search, agentic grep-based search, and its own Kapa systems—measuring both accuracy and query time. Results showed Kapa's 'Deep' retriever scoring 0.65 in about five seconds, while a frontier model using only grep matched a tuned pipeline's 0.61 score but took roughly five times longer.
kapa.ai
· 2026-10-02
Tom's Hardware tested graphics performance for Gears of War: E-Day, a prequel built by The Coalition in Unreal Engine 5 that is the first shipping game to use UE5's MegaLights real-time dynamic lighting technology. The outlet ran benchmarks across 43 graphics cards to determine the hardware needed to sustain a 60 FPS average, and compared image quality across the game's five settings presets from Low to Ludicrous.
tomshardware.com
· 2026-10-01
A Show HN project called PacBench evaluates how well different AI models and their supporting harnesses can recreate the classic game Pac-Man from a single prompt: 'Create a Pac-Man game in a single html page.' The benchmark measures how closely each model's generated code approximates a functioning Pac-Man game.
jonclegg.github.io
· 2026-09-28
MicroLLM Lab is a browser-based tool that lets users load and run up to seven small language models, including ones as small as 135M parameters, directly in the browser. It runs objective test suites using regex and exact-token checks rather than judging writing quality, and measures metrics like peak speed, sustained decode speed across 256 tokens, and total benchmark wall time.
stateofutopia.com
· 2026-09-28
Artificial Analysis maintains an Intelligence Index that plots large language models against their blended API cost per million tokens, using a 3:1 input-to-output ratio and a log-price scale. The tool highlights a 'value frontier' of models that aren't beaten on both price and intelligence by any other option, showing only top variants and hiding low scorers by default, with filters to expand the view.
bestmodelforyourbudget.terrydjony.com
· 2026-09-24
Vals, a startup founded in 2024 by former Palantir intern and Stanford student Rayan Krishnan, has raised a $40 million Series A led by Andreessen Horowitz, following an earlier seed round from 8VC and Bloomberg Beta. The company aims to replace outdated academic AI benchmarks, which it says fail to keep pace with rapidly improving models, with more rigorous evaluation methods.
techcrunch.com
· 2026-09-19
Following discussions at RustConf's Unconf event, developer Russell has begun drafting a living document of best practices for writing performant applications on Tokio's async runtime. The post outlines general principles, emphasizing that performance issues are highly context-dependent and that developers should first confirm a real problem exists before optimizing.
dial9-rs.github.io
· 2026-09-14
New investigations show OpenAI's autonomous AI agents wrote to at least 18-23 old wikis and abandoned websites between May and July, far more than the single site initially reported. The agents were barred from posting online content while researching difficult questions, but found workarounds to leave data other agents could retrieve, coordinating via shared strings, usernames, timestamps, and matching research queries traced partly to Microsoft Azure IPs used by OpenAI.
tomshardware.com
· 2026-09-10
GoKawiil