The AI industry has made major promises about its tech boosting the productivity of developers, allowing them to generate copious amounts of code with simple text prompts.
But those claims appear to be massively overblown, as The Register reports, with researchers finding that productivity gains are modest at best — and at worst, that AI can actually slow down human developers.
In a new report, management consultants Bain & Company found that despite being “one of the first areas to deploy generative AI,” the “savings have been unremarkable” in programming.
“Generative AI arrived on the scene with sky-high expectations, and many companies rushed into pilot projects,” the report reads. “Yet the results haven’t lived up to the hype.”
First off, “developer adoption is low” even among the companies that rolled out AI tools, the management consultancy found.
Worse yet, while some assistants saw “ten to 15 percent productivity boosts,” the savings most of the time “don’t translate into positive returns.”
It’s yet another warning shot, highlighting concerns that even in one of the most promising areas, the AI industry is struggling to live up to its enormous hype. That’s despite companies pouring untold billions of dollars into its development, with analysts openly fretting about an enormous AI bubble getting closer to popping.
In July, a damning study by nonprofit Model Evaluation & Threat Research found that AI coding tools may not just realize the expected productivity gains, they could slow software developers down.
“Surprisingly, we find that when developers use AI tools, they take 19 percent longer than without — AI makes them slower,” the researchers wrote. That’s because hallucinations forced the developers to spend extra time cleaning up and reviewing code. The tools also performed worse when dealing with large and complex code repositories, often misunderstanding the context.
Programming website Stack Overflow also found in a survey earlier this year that developers’ trust in AI tools had cratered — despite a rise in the percentage of developers using them. Many of them pointed to solutions that are “almost right, but not quite.”
... continue reading