GoKawiilAnalysis flags flaws in popular napkin-math, SWE-Bench and winter tire benchmarks
A review examines three widely cited benchmark resources: the sirupsen/napkin-math GitHub project used for computer performance estimation interviews, a senior-level SWE-Bench coding evaluation, and standardized winter tire test ratings. The piece walks through each dataset's numbers and methodology, inviting readers to form their own estimates before revealing where the figures may be misleading or inconsistent.