Microsoft research shows AI coding tools fall short in key debugging tasks

In context: Some industry experts boldly claim that generative AI will soon replace human software developers. With tools like GitHub Copilot and AI-driven "vibe" coding startups, it may seem that AI has already significantly impacted software engineering. However, a new study suggests that AI still has a long way to go before replacing human programmers. The Microsoft Research study acknowledges that while today's AI coding tools can boost productivity by suggesting examples, they are limited in actively seeking new information or interacting with code execution when these solutions fail. However, human developers routinely perform these tasks when debugging, highlighting a significant gap in AI's capabilities. Microsoft introduced a new environment called debug-gym to explore and address these challenges. This platform allows AI models to debug real-world codebases using tools similar to those developers use, enabling the information-seeking behavior essential for effective debuggi ... Read full article.