GoKawiilNew Real-SWE benchmark tests AI coding agents on licensed enterprise codebases
A new benchmark called Real-SWE evaluates frontier AI coding models against tasks drawn from real, private production codebases licensed from actual companies, including billing, tax and customer-migration work. Top performer Fable 5.1 running on Claude Code resolved 38.8% of tasks, followed by GPT-6 Astra and Gemini 3.8 Flash, with several other models trailing well below that mark.