Skills Do Not Save You Tokens
The strongest matched comparison published this month has the skill arm beating workflow memory by 4.8 points for 95,300 more tokens per task on an 83-task matched intersection, and those token figures are per-task averages rather than totals.
That is the trade in its cleanest form. Demystifying agent skills ran three arms over the same benchmark set with token accounting attached, and the arm that scored highest is not the arm that cost least. Worth saying immediately, because it cuts the other way and belongs in the same breath: the skill arm in that same table sits +5.5 points against the raw baseline at 34,200 fewer tokens per task on the same 83-task matched intersection, again as per-task averages rather than totals, so a skill is not uniformly more expensive than no skill at all. What it is not is free, and the progressive-disclosure argument that a skill pays for itself by loading only what is needed does not survive contact with a matched comparison against a cheaper technique.
Serving overhead shows up again in a different lab with a different method. The SkillCorpus authors report 158K against 122K tokens per task, roughly 30%, on the 87-task Opus 4.7 frontier check only, estimated from released eval-cache counters on a single run. That is one run on one harness, and an estimate read off counters rather than a billed figure, so treat the shape rather than the value: turning skills on made each task cost about a third more to serve. Their own counter-example sits a page earlier, where one task that the no-skill run failed was rescued by a skill and finished with less wall-clock time and fewer tokens than the failing run had burned. Both things are true. Skills add overhead per task and occasionally remove a failure loop that was costing far more than the overhead.
The cost is in the procedure, not the prompt
Most reasoning about skill cost treats it as a context-length problem, which is why most reasoning about skill cost is wrong.
Agent Skills Can Be Harmful separated skill-induced efficiency regressions, meaning paired runs where both the target and the reference passed the verifier but the skill run cost substantially more, and found that 62.6% of efficiency regressions are excessive procedure. Skills turn optional validation checklists and construction recipes into mandatory work: the agent runs the verification pass the skill describes and then the construction pipeline it lays out, on a task where neither was needed. Excessive verification at 36.8% sits nested inside that 62.6%, so the two never add together.
Where cost does come from context, the split is lopsided and useful. Skill-body bloat accounts for 43 of the 46 context-overhead cases against 3 for supplementary material, which says the always-loaded mandatory body is the expensive part and lazy-loaded references are close to free. That is an authoring instruction rather than a runtime one, and it is the cheapest fix in this entire literature: shorten the part that always loads.
Filtering shaped that dataset, and the shape matters. Candidate skills were kept only above a cosine similarity threshold to the task, and the agent was allowed to decline skills it judged irrelevant, so these are regressions caused by skills that were topically right for the job.
None of the figures above price the library itself. Building a curated skill set means judge calls, deduplication passes and revision cycles, and maintaining one means repeating a slice of that work every time a tool or an API changes underneath it. The Demystifying authors state explicitly that they did not measure the cost of maintaining or updating a skill library over time, and nobody else in this corpus did either, so every published cost comparison is a serving-cost comparison with the amortised build cost set to zero. The nearest thing to a price anyone prints sits in the SkillCorpus accounting, where one curation pass over a 96,401-skill corpus runs to roughly 269,000 judge calls, which is the line item nobody carries into a per-task cost comparison.
Your token counts are probably ranking your options backwards
The most useful result in the cost literature is a measurement warning rather than a finding about skills.
Skill Blocks compared four ways of getting the same skill content into context under caching-correct accounting, and reports cache reads running at 74% to 94% of raw input in multi-turn runs specifically, with the caveat that the cache attribution model is unverified and the authors concede it, which matters because that attribution is the load-bearing assumption underneath the paper's headline numbers. On single-turn benchmarks raw input remains the primary metric and none of this applies.
What the cache does to a comparison is not a rounding error. In one multi-turn arm the raw counts read 70,528 against 65,380, and yet the arm with more raw input had effective input 14.8% lower. A team that ranks its loading strategies on raw token counters, which is what almost every dashboard shows by default, will pick the wrong one and have a number to justify it with. The instruction the author gives is to record raw, cache-read, new and output tokens at turn level before making any cost claim at all, and it is the correct instruction.
No universal winner, and one uncomfortable result
Across five benchmarks the paper finds no configuration that wins everywhere, and the regime map is the deliverable. On a small single-turn skill, a hybrid arrangement of short stubs with an optional fetch came in at −27.4% against full inlining, while pure on-demand tool loading of the same skill ran to +48.4%, costing more than inlining because the round trip and the schema overhead exceed the mass being avoided, and that arm used a deliberately aggressive loading prompt the authors call an overhead diagnostic rather than a recommended configuration. On a large compressible skill in a multi-turn setting, conditional block loading reached −62.47% in cache-adjusted effective tokens, at 62 of 84 successes against 57 of 84 for full inlining, with no outcome comparison reaching significance. Between those poles the effects shrink toward parity and several confidence intervals include zero.
Then there is the arm that beat everything on cost. On the large single-turn benchmark hybrid loading came in at −39.8% against −55.5% for static deletion of unused sections, a treatment that permanently removes content from the skill and registers no loader at all, and the static arm was cheaper than hybrid on 236 of 276 cases while being explicitly non-content-parity and exploratory. Both words are load-bearing. Non-content-parity means it deleted coverage rather than deferring it, so it cannot demonstrate anything about behaviour under a distribution shift that needs the deleted material; exploratory is the authors' own label for the entire primary outcome family, since final cases were reused during development, and no outcome comparison in that family reached significance. A non-significant outcome test is not a finding of equivalence.
Read at its true strength, the result is still the most interesting thing in the paper. Deleting content beat routing to content, on the benchmark where routing had the most to gain, which is exactly what you would expect if the overhead of conditional loading is comparable to the mass it defers.
Measuring this on your own workload
The experiment that settles it for a given stack takes an afternoon and one instrumentation change.
Pick fifty real tasks and run them twice, once with the skill available and once without, holding the model, the harness and the repository state fixed. Record raw input, cache reads, new input and output at turn level rather than at run level, because the run-level counter is the one that inverts. Then divide by completed tasks rather than attempted ones. That single step decides the answer: a skill that adds overhead to every task and removes a retry loop often enough has paid for itself, while one that adds the same overhead and changes nothing else has not. Look separately at the runs where the skill arm cost more and the verifier still passed, since that is where excessive procedure hides, and it will show up as verification and exploration steps that the task never required.
What actually saves tokens
Loading fewer things saves tokens. Loading the same things more cleverly mostly does not.
The one very large saving in this corpus comes from not putting a catalogue in context at all: 46,915 tokens per task for the whole catalogue against roughly 560 for on-demand retrieval, priced over an 875-entry catalogue, and the authors never tested sufficiency, so that comparison counts catalogue rows rather than completed tasks. A retrieval step that fails to surface the needed skill costs a failed task, and nothing in that arithmetic prices a failed task. Their own summary of where the saving comes from is the sentence to keep, because it generalises past their setup: it comes "from loading less rather than from loading smarter". The retrieval side of that trade is where the risk sits, and it is measured badly almost everywhere.
So the honest position on cost is that skills move it rather than remove it. The prompt gets shorter and the procedure gets longer. The catalogue leaves context and a retrieval failure mode arrives. The per-task serving bill rises by roughly a third in one frontier check and falls on the individual tasks where a skill converts a failure loop into a completion. None of that adds up to a savings argument, and the unit that reflects both halves is effective input per completed task rather than tokens per run.
The savings claim was never the good reason to use a skill anyway. What skills are demonstrably good for is anchoring a procedure the model already knows how to execute, and that case stands on its own without a token argument propping it up.
Every figure here is drawn from the thirty-five papers listed in The Agent Skills Evidence Base, where the full claim register, including the eight claims that did not survive verification, is published in the open. We build Synap, which is a memory layer for agents, so weigh the argument accordingly and check the register.