That number also does not include the use of LLMs by employees outside of official pilots. The MIT researchers found that around 90% of the companies they surveyed had a kind of AI shadow economy where workers were using personal chatbot accounts. But the value of that shadow economy was not measured.
When the Upwork study looked at how well agents completed tasks together with people who knew what they were doing, success rates shot up. The takeaway seems to be that a lot of people are figuring out for themselves how AI might help them with their jobs.
That fits with something the AI researcher and influencer (and coiner of the term “vibe coding”) Andrej Karpathy has noted: Chatbots are better than the average human at a lot of different things (think of giving legal advice, fixing bugs, doing high school math), but they are not better than an expert human. Karpathy suggests this may be why chatbots have proved popular with individual consumers, helping non-experts with everyday questions and tasks, but they have not upended the economy, which would require outperforming skilled employees at their jobs.
That may change. For now, don’t be surprised that AI has not (yet) had the impact on jobs that boosters said it would. AI is not a quick fix, and it cannot replace humans. But there’s a lot to play for. The ways in which AI could be integrated into everyday workflows and business pipelines are still being tried out.
03: Are we in a bubble? (If so, what kind of bubble?)
If AI is a bubble, is it like the subprime mortgage bubble of 2008 or the internet bubble of 2000? Because there’s a big difference.
The subprime bubble wiped out a big part of the economy, because when it burst it left nothing behind except debt and overvalued real estate. The dot-com bubble wiped out a lot of companies, which sent ripples across the world, but it left behind the infant internet—an international network of cables and a handful of startups, like Google and Amazon, that became the tech giants of today.
Then again, maybe we’re in a bubble unlike either of those. After all, there’s no real business model for LLMs right now. We don’t yet know what the killer app will be, or if there will even be one.
And many economists are concerned about the unprecedented amounts of money being sunk into the infrastructure required to build capacity and serve the projected demand. But what if that demand doesn’t materialize? Add to that the weird circularity of many of those deals—with Nvidia paying OpenAI to pay Nvidia, and so on—and it’s no surprise everybody’s got a different take on what’s coming.