GoKawiilThe main demand signal for artificial intelligence looks explosive on paper, but it may be significantly overstated. Anthropic, by pricing its tools for that reality, might be the best-positioned AI company if a correction comes.
Tokens are the basic unit of AI usage: words and characters that make up both the queries users send and the output models generate.
Chatting with an AI consumes a couple of hundred tokens per paragraph. Agentic AI, where models write code, browse the web, and execute multi-step workflows, burns through thousands more per session.
Using the rates of Anthropic's latest model, one million tokens of input (prompts) costs $5, and one million tokens of output (the model's responses) costs $25.
AI companies cite the boom in token consumption to justify the hundreds of billions of dollars being spent on infrastructure to serve it.
But token consumption is becoming a distorted metric.
Meta and Shopify say they have created internal leaderboards that track how many tokens employees use. Nvidia CEO Jensen Huang has said he'd be "deeply alarmed" if an engineer earning $500,000 a year wasn't using at least $250,000 worth of compute — measuring what an engineer spends on AI instead of what they produce with it.
Once companies start measuring AI adoption by volume, employees optimize for the metric instead of the outcome.
"If your goal is to just burn a lot of money, there are easy ways to do that," said Ali Ghodsi, CEO of Databricks, which processes AI workloads for thousands of enterprises. "Resubmit the query to ten places. Put up a loop that just does it again and again. It's going to cost a lot of money and not lead to anything."
Jen Stave, executive director of the Harvard Business School AI Institute, hears the same from enterprise leaders.
... continue reading