GoKawiil01 Why your token bill
blew up this year
The warning signs were everywhere the first half of this year: GitHub Copilot switched to token-based billing [1], Uber burned through its annual AI tools budget in just four months [2][3], and developers reported projected cost jumps from $29 to $750 a month [4]. We even invented words for it: "tokenmaxxing", "token panic", "token budgets".
The median employee [5] currently spends ~$140 per year on AI API usage. Sounds harmless, until you look at the tail: the 90th percentile nears ~$7,300 per year, and the 99th approaches ~$90,000. For many organisations agentic AI adoption is only just beginning, so expect these numbers to change in the year ahead.
Why is the bill so volatile? Because of how we use AI agents: Currently, over 70% of ARR across major model providers [4] comes from coding use cases. The more of your workflow you hand to agents, the higher your token bill becomes.
Many organizations start their AI coding agent adoption using one of the major frontier model providers like OpenAI or Anthropic through API access. A few months in, more and more organizations are at least considering the alternatives for a number of reasons: token pricing models become more costly, users are upgrading to newer and more expensive AI models and agent adoption in the organization starts to boom. There are sensible alternatives out there that are certainly worth a closer look: switching to API routers; using cheaper model tiers for routine work while only escalating the hard 20% to a frontier model and others.
However, if your token consumption becomes significant or if you're working with sensitive data, you might want to abandon the API approach altogether and setup your own inference stack. When ownership of hardware is on the table, you'll need to understand what renting or owning a GPU actually gives you in return. When does it make sense to make that switch? Read on as we give you the tools to decide for yourself.
02 You pay for the pool 24/7 ,
your team uses it 8 hours a day
So you've decided to investigate whether buying or leasing your own GPUs makes sense. Once configured correctly, it will give you an API endpoint you can plug into your coding tools, much like Anthropic, OpenAI or other model providers offer.
... continue reading