Most companies are blowing their AI budgets. In McKinsey's latest research, 93% of organizations surveyed said they have exceeded their AI budget. Spend grows nearly fourfold when AI moves from pilots to company-wide use, and most firms expect it to rise at least another 25% in the next year.
When a budget blows up, teams reach for the most obvious lever: make the AI use fewer tokens. A whole market of tools now promises exactly that, with headline numbers of 60 to 90%.
So we tested it. 2,908 real coding sessions, every one of them run in Claude Code: three compression tools tested inside the agent against it unmodified, every cost read off our own provider bill rather than a token counter. We were looking for one thing: does cutting tokens cut cost?
Spoiler. It did the reverse.
The harder a tool compressed, the more it cost
Cost per completed task against unmodified Claude Code, 2,908 sessions (2,848 analyzed):
| Tool | Tokens cut | Cost vs baseline |
|---|---|---|
| RTK, the open-source compressor, unmodified | -1.3% | -2.9% |
| RTK-ML, our experimental build (our nine gates on top of RTK) | -38.4% | +6.8% |
| Headroom v0.27.0, third party, not ours | not measurable | +46.4% |
Read that top row again. The only tool that came out ahead is the one that barely compressed anything: RTK, the open-source original, exactly as anyone can download it. It cut 1.3% of tokens and saved 2.9%, and even that saving is not statistically significant.
RTK-ML, our own experimental build on top of RTK, cut tokens by 38.4% and ended up costing 6.8% more.
Headroom works at the API boundary, so we cannot measure how much it removed. We can measure what it cost: 46.4% more than doing nothing at all. Not what the marketing says.
Compressing harder did not produce smaller savings. It produced losses.
The full finding, in 54 seconds: watch the summary video.
Disclosure: this study was written at PointFive and is not independent research. We ran RTK, the open-source compressor; RTK-ML, our own experimental build that cuts tokens more aggressively; and Headroom, a third-party tool. This research informs the design of PointFive's token optimization product, built to improve visibility, compliance, and governance. Everything is checkable: arxiv.org/abs/2607.12161, a free download. Read it skeptically. That is what it is for.
So where does the money actually go?
The answers coming out of the model are only 10.4% of the bill. The rest is dominated by text going into the model, and the split is not what you would guess.
About 75% of the cost is the agent framework's own baggage: its system prompt and its tool definitions, re-sent on every turn before your work even starts. Another 19% is the model's own hidden reasoning. Neither is reachable from the outside.
Everything a compression tool can even touch, the tool outputs, file contents and conversation text, adds up to 6.0% of the cost. The surface the tested tools actually attack is 3.3%.
You are not paying for answers. You are paying, over and over, for the machinery's own memory and thinking.
That is the whole ceiling argument. Even perfect compression of everything reachable tops out around 5%. The market is advertising 60 to 90%.
Why cutting context makes it worse
A compressor is not editing a document. It is editing what a working agent can see, in the middle of the job.
RTK-ML was not cutting blindly. It was built to keep only the context relevant to the task, and it worked: it removed 38.4% of the text flowing to the model. It still lost money. The agent noticed things were missing. It searched again, re-read files it had already read, and took more turns to reach the same place. Every one of those extra turns adds context, reasoning and output cost; cached history is cheaper than fresh input, but it is not free. The savings came back as a bill.
Across everything we tested, how much text a tool removed turned out to be a weak and unstable predictor of what it cost: the correlation is close to zero and its confidence interval crosses zero. A token count in a lab says nothing about the bill, because the bill depends on how many turns the session takes, and that responds to what you delete.
Where to optimize instead
Token reduction is a local metric. Agent efficiency is a system property. Cost is driven by how a session unfolds: the retries, the re-reading, the extra loops nobody asked for. Those are habits and workflow choices, not payload problems, and nobody changes a habit they cannot see. See it first, govern it with budgets and policy, and then measure the right thing: cost per completed task, taken from the provider's bill. Not tokens removed. Tokens removed is the number that hid all of this in the first place.
Read the research
- The research hub, results first, with the 54-second video
- The full paper on arXiv, free, no form
- The research summary and industry perspective, ten pages
- The open-source AI Efficiency Benchmark, so any savings claim, including PointFive's own, can be run through it
Further research in the series will follow, published the same way.
The comparisons above are for informational purposes only and are based on publicly available information and subjective opinions at the time of publication. While we strive to ensure accuracy and fairness, we are unable to guarantee that all information is complete, current, or free from errors. Comparisons may not reflect all features, performance metrics, or variations of the referenced services, and individual results may vary. We encourage visitors to independently verify any information and conduct their own research before making purchasing decisions.