Most companies are blowing their AI budgets. In McKinsey's latest research, 93% of organizations surveyed said they have exceeded their AI budget. Spend grows nearly fourfold when AI moves from pilots to company-wide use, and most firms expect it to rise at least another 25% in the next year.
When a budget blows up, teams reach for the most obvious lever: make the AI use fewer tokens. A whole market of tools now promises exactly that, with headline numbers of 60 to 90%.
So we tested it. 2,908 real coding sessions, every one of them run in Claude Code: three compression tools tested inside the agent against it unmodified, every cost read off our own provider bill rather than a token counter. We were looking for one thing: does cutting tokens cut cost?
Spoiler. It did the reverse.
The harder a tool compressed, the more it cost
Cost per completed task against unmodified Claude Code, 2,908 sessions (2,848 analyzed)
Read that top row again. The only tool that came out ahead is the one that barely compressed anything: RTK, the open-source original, exactly as anyone can download it. It cut 1.3% of tokens and saved 2.9%, and even that saving is not statistically significant.
RTK-ML, our own experimental build on top of RTK, cut tokens by 38.4% and ended up costing 6.8% more.
Headroom works at the API boundary, so we cannot measure how much it removed. We can measure what it cost: 46.4% more than doing nothing at all. Not what the marketing says.
Compressing harder did not produce smaller savings. It produced losses.
The study in 54 seconds.
Disclosure: this study was written at PointFive and is not independent research. We ran RTK, the open-source compressor; RTK-ML, our own experimental build that cuts tokens more aggressively; and Headroom, a third-party tool. This research informs the design of PointFive's token optimization product, built to improve visibility, compliance, and governance. Everything is checkable: arxiv.org/abs/2607.12161, a free download. Read it skeptically. That is what it is for.
So where does the money actually go?
The answers coming out of the model are only 10.4% of the bill. The rest is dominated by text going into the model, and the split is not what you would guess.
About 75% of the cost is the agent framework's own baggage: its system prompt and its tool definitions, re-sent on every turn before your work even starts. Another 19% is the model's own hidden reasoning. Neither is reachable from the outside.
Everything a compression tool can even touch, the tool outputs, file contents and conversation text, adds up to 6.0% of the cost. The surface the tested tools actually attack is 3.3%.
You are not paying for answers. You are paying, over and over, for the machinery's own memory and thinking.
Where the cost actually sits
Share of cost by component, per the updated study
That is the whole ceiling argument. Even perfect compression of everything reachable tops out around 5%. The market is advertising 60 to 90%.
Why cutting context makes it worse
A compressor is not editing a document. It is editing what a working agent can see, in the middle of the job.
RTK-ML was not cutting blindly. It was built to keep only the context relevant to the task, and it worked: it removed 38.4% of the text flowing to the model. It still lost money. The agent noticed things were missing. It searched again, re-read files it had already read, and took more turns to reach the same place. Every one of those extra turns adds context, reasoning and output cost; cached history is cheaper than fresh input, but it is not free. The savings came back as a bill.
That last number is the one to sit with. We removed 38.4% of the text and the memory traffic did not fall at all. It rose. The agent's response to a thinner context was to do more work, and more work is the expensive thing.
Push compression further and it stops being an economic question. To change a file, an agent has to quote the existing code back exactly, character for character. Compress that quote and the edit simply will not apply. In one test, successful edits fell from 27 out of 40 to 15. At that point you are not overpaying, you are failing.
So what actually works?
McKinsey says the quiet part plainly: "Organizations cannot optimize what they cannot see." Their research finds 20 to 30% of AI spend is unaccounted for entirely, and only about a quarter of companies have mature AI cost practices. The companies that manage AI spend well save 20 to 30%, and they do it with visibility and accountability, not with a compression trick.
Our data points the same way, and it says something more specific about where the money actually goes. Cost is driven by how a session unfolds: the retries, the re-reading, the extra loops. Those are not payload problems that a tool can quietly optimize away in the background. They are habits and workflow choices, and nobody changes a habit they cannot see.
- See it first. Who is spending, on which model, for what kind of work, at what cost. You cannot fix a bill you cannot read.
- Govern it. Budgets per person, per project and per team. A decision about which agents, models and tools are allowed in the first place, policy you can enforce rather than just report on, and an audit trail when someone asks who approved what. Governance and compliance matter here every bit as much as visibility does.
- Then measure the right thing. What it cost to actually get the job done, taken from the provider's bill and divided by the sessions that finished. Not tokens removed. Tokens removed is the number that hid all of this in the first place.
Compression may still earn a small place. The best any tested tool managed was 2 to 3%, it was not statistically significant, and it came from the tool that compressed least. Our research continues. It is simply not the thing to do first. Even done carefully, keeping only what the task needs, it cost money.
Check everything yourself
We open-sourced the AI Efficiency Benchmark: the real tasks, the judging, and the retained data behind every number above. Run any compression tool through it, including ours: github.com/PointFiveLabs/ai-efficiency-benchmark.
The paper is free and there is no form: arxiv.org/abs/2607.12161. This is the first paper in an ongoing PointFive research program on agent efficiency. More is coming, published the same way.
The comparisons above are for informational purposes only and are based on publicly available information and subjective opinions at the time of publication. While we strive to ensure accuracy and fairness, we are unable to guarantee that all information is complete, current, or free from errors. Comparisons may not reflect all features, performance metrics, or variations of the referenced services, and individual results may vary. We encourage visitors to independently verify any information and conduct their own research before making purchasing decisions.