GoKawiilStudy finds most testing prompts barely improve AI coding agents' Zstd implementations
A researcher evaluated 26 different prompt conditions—ranging from 'use TDD' to formal methods like Lean 4, TLA+, and SMT solvers—to see whether simply instructing coding agents to apply specific testing techniques or libraries improves the correctness of Rust implementations of Zstd. Four popular testing 'skills,' including ones with hundreds of thousands of GitHub stars, were also tested alongside a self-written skill, using the same Zstd benchmark from prior agentic programming comparisons.