GoKawiilIf you liked this piece, you should subscribe to my premium newsletter. It’s $70 a year, or $7 a month, and in return you get a weekly newsletter that’s usually anywhere from 5,000 to 18,000 words, including vast, detailed analyses of NVIDIA , Anthropic and OpenAI’s finances , and the AI bubble writ large . My Hater's Guides To the SaaSpocalypse, Private Credit and Private Equity are essential to understanding our current financial system, and my guide to how OpenAI Kills Oracle pairs nicely with my Hater's Guide To Oracle .
Over the last three weeks , I’ve published an exhaustive three-part guide to how the AI bubble might collapse, the events that might trigger it, and the consequences.
Subscribing to premium is both great value and makes it possible to write these large, deeply-researched free pieces every week.
Something changed in the last week.
Shortly after Uber COO Andrew Macdonald said that it was “getting harder to justify” spending money on AI as it was “very hard to draw a line” from that spend to useful consumer features ( after its CTO said Uber burned its entire annual token budget in four months ), Axios’ Madison Mills reported that one company had accidentally spent $500 million in the space of a month on Anthropic’s models after failing to set spend limits. A few days later, Mills would report that other companies were now looking for ways to reduce their AI spend .
That’s because, as I’ve said before , nobody can actually measure the ROI of AI, or even create a standard measurement of the cost of a task thanks to the inevitable hallucination-prone nature of LLMs and the ever-growing list of different harnesses and “agentic” (sigh) interfaces. Every different prompt and project and interaction can go wrong in a way that is hard to predict or plan for other than having an eternal vigilance that the supposed “intelligence” doesn’t do something catastrophically stupid, because LLMs have no thoughts, consciousness or ability to learn outside of pre and post-training.
If you can’t measure how good something is, how much it might cost, or what your return on investment might be, it’s fair to ask why you’re even paying for it in the first place.
People are (reasonably!) harping on about the ROI problem, but I think the “can’t really measure the cost” part is an even bigger problem.
Yesterday, Microsoft’s GitHub Copilot moved all customers to token-based billing from a premium request model ( as I reported a week before everyone ) as users had been allowed to burn thousands of dollars of tokens on a $39-a-month subscription .
Customers are irate. One burned through 50% of their monthly credits in a single prompt , another burned 60% in the space of a few hours , another 31% in a single prompt , another estimated that they’d burn their monthly credits in the space of a single five hour session , another burned nearly half of their credits in eight prompts , another around 14% of their credits in two prompts , and another lamented that GitHub Copilot had gone from their favorite subscription to their most-stressful overnight after burning 33% of their monthly balance in a few hours .
... continue reading