GoKawiilProgress in large language models is usually reported as what the best model can now do that no model could do before. That is the direction that produces headlines, and it has indeed been truly incredible. Each step up at the top of the range lets a model handle a kind of task that was previously out of reach, whether that is fixing a bug that spans a whole codebase or, lately, making progress on outstanding mathematical problems that had not been solved by anyone.
There is a second direction of progress that gets less attention, which is how cheaply a given level of capability can be bought. A great deal of useful work does not require the smartest model available, but a model that is good enough, applied many thousands of times. Reading every scientific paper on a topic, checking every contract in an archive for a particular clause, or summarizing every thread in a large discussion forum are tasks of this kind. For these, the question is not whether a model exists that can do the job, but whether it can do the job ten thousand times within a budget. The ceiling unlocks new kinds of tasks; the floor unlocks volume.
When you pick a model for an application you are trading off how capable it is against how much each call costs. For agentic coding I have focused almost entirely on capability, with the general sense that the improved quality of the work is worth the money, even when far cheaper models exist that are reasonably capable. My attention was recently drawn to the cost of the floor. We are measuring how often datasets shared on the DANDI Archive are reused in later publications, which means reading on the order of ten thousand candidate papers with a model and asking of each one whether it actually reused the data. At today’s prices a full pass over the corpus costs a little over a hundred dollars with a model whose capability was at the frontier in the spring. At the prices of this past March, the same pass with the same level of capability would have cost several thousand dollars, and a year ago that capability was not available at any price. That change in the floor is what turned the analysis from a thing we could do on a sample into a viable project. I have been surprised by the progress across the cost spectrum, particularly how intelligent cheap models have become.
Artificial Analysis has been benchmarking intelligence and price across hundreds of models for a couple of years, and enough of that data is accessible to reconstruct the tradeoff. In particular, this plot shows the intelligence index vs. the cost per task, providing a realistic cost estimate for different levels of model capability. The top line is what they define as the “Pareto line,” the most capable models at a given price point. This line describes the true frontier of LLMs. I pulled data from artificialanalysis.ai and looked at how the Pareto frontier has moved as new models have been released. I think it is worthwhile to take a beat to review this progress and make some predictions for the next few months.
The short version: the level of intelligence that cost $1.22 per task in February costs $0.022 today, a 56x drop in under six months, and the rate of decline is accelerating. At the measured pace, a 100x drop for a given capability level takes about a year, and the question worth asking is not whether that happens but what it changes.
The Artificial Analysis Intelligence Index
The capability axis throughout this post is the Artificial Analysis Intelligence Index, so it is worth being clear about what that number is. The current version, v4.1.1, is a weighted average over nine evaluations grouped into four categories: agentic tasks at 34%, coding at 24%, scientific reasoning at 24%, and general capability at 18%. The weighting reflects where the field’s attention is: a third of the score comes from a model’s ability to complete multi-step agentic work, not from answering exam questions. The component evaluations, their weights, and the scoring details are documented in Artificial Analysis’s intelligence benchmarking methodology.
What you end up with is a single number that represents model capability, sort of like an IQ for LLMs. It isn’t perfect, and two models with the same score may have different strengths, but I have found that this score does a reasonably good job of indicating a model’s capability.
As a reference point, Anthropic’s “Claude 4.5 Sonnet (Reasoning)” was for me and many others the first time a model felt capable enough to use in an agentic harness for writing code. At the time I was using Cline, and this model provided substantial productivity gains over auto-complete and copy/paste workflows. That model had an intelligence score of 37.4 (based on today’s intelligence scoring system). The top current model is Claude Opus 5 max effort, at 63.1.
To give a more visceral sense of what the different index levels mean, I borrowed Simon Willison’s pelican benchmark: prompt a model with “Generate an SVG of a pelican riding a bicycle” and look at what comes back. It is not what the index measures, but it is a task anyone can judge by eye. The panels below use the GPT-5.6 family at four points on the index: Luna at low, high, and xhigh effort, and Sol at max effort. I generated three samples per model and show the first one; all of them are in the site repository.
... continue reading