What changes the cost of a coding task
A coding agent's cost depends on more than the model's token price. Prompt wording, reasoning effort, and the harness that manages tools and context shape how the agent approaches the task.
What the research found
Controlled experiments found that wording could change reasoning and verification behavior without changing the underlying task. Extra effort helped some difficult tasks but also added cost without improving outcomes. An intervention's value could change with the harness.
These factors need to be evaluated together. Token and cache counts describe the agent's activity; they do not establish that the work became cheaper or more successful.
Read the paper and inspect the data
This PointFive Labs paper is by Sarel Weinberger and Amir Hozez. Read the versioned paper below and explore the benchmark and data.
Read the version 6 paper on this page
Source: arXiv, version 6. Open the PDF.