Published 28 September 2026. Updated 31 October 2026 with three same-model studies from October and a one-task cost anecdote with a 13x price gap between two harnesses.
An agent harness is the software around a language model that turns it into an agent: it calls the model in a loop, runs the tools the model asks for, feeds the results back, decides what goes into the context on every call, and enforces what the agent is allowed to touch. Every LLM call is stateless, and a model on its own cannot act; it turns text into more text, then stops. Anything we call an agent is that model placed inside a harness, and most of the confusion about agents comes from using the two words as if they meant the same thing.
October 2026 added a second reason to care about the difference, and it is money. A Shandong University study ran the same models through pinned releases of Claude Code (2.1), OpenCode (1.18) and mini-SWE-agent (2.4.6) on SWE-bench Verified and found the harnesses statistically equivalent on pass rate but up to 3x apart on cost per task. A one-task anecdote points the same way: Ishaan Jaff of LiteLLM posted that one coding task on one model cost $126 in one harness and $9.59 in another, a gap he puts almost entirely down to prompt caching (0% cache hits against 96%). The model decides what to do next; the harness decides, among other things, how much each of those decisions costs.
The agent is the overlap of model and harness
Picture two circles. One is the model, which thinks and decides what to do next. The other is the harness: the loop that keeps calling the model, the tools it can reach (built-in ones as well as anything connected through MCP), the skills and memory it loads into context, compaction when that context fills up, the permissions that fence it in, and the system prompt that shapes its behaviour.
An agent is only the overlap. A model with no harness answers one question and stops; a harness with no model is a car with nobody in the driver's seat. Put them together, give them a task, and the model asks to read a file, the harness reads it, the model picks the next step, and the cycle repeats until the job is done.
What is inside an agent harness
Production harnesses such as Claude Code, Codex CLI, OpenCode and Hermes Agent are built by different teams and still end up with most of the same parts, and each part leaves its own mark on what the agent costs and how well it does.
| Component | What it does | Where it shows up on the bill or in the result |
|---|---|---|
| The loop | Calls the model, runs what it asks for, calls it again, and decides when to stop | Step count, which multiplies everything else |
| Tools | Built-in tools (read, edit, shell, search) plus tools from MCP servers, each described to the model by a schema | Tool schemas ride along on every call |
| Context assembly | Builds each request: system prompt, instruction files, tool list, history, tool output | Decides whether the prompt prefix stays identical, which decides cache hits |
| Compaction | Summarises or drops old history when the context fills | Rewrites the prefix, so the next call starts a fresh cache |
| Skills and memory | Loads playbooks and remembered facts into context when a task needs them | More tokens per call, fewer wasted steps if chosen well |
| Permissions and hooks | Approves, denies or edits tool calls before they run; hooks let a team add its own rules | Safety, and fewer destructive mistakes |
| Session persistence | Writes the transcript to disk as it goes, so a crashed or paused run can resume | Recovery inside one task; it is not memory across users or sessions |
| Subagents | Starts separate loops with their own prompt and clean context, and returns their result | Keeps the main context small, adds calls of its own |
The model sits outside the harness and plugs into it, which is why one harness can swap one model for another and still be the same harness. Skills are not tools: a skill is a playbook the harness loads into context when a task calls for it, while a tool is something the model calls, and MCP is one more source of those tools. The main agent is the whole harness running with a model; only subagents live inside it, as separate loops. Behaviour is only partly static, because the config and the system prompt are fixed while how proactive an agent actually is comes from the model following them.
Harness vs framework vs SDK
A framework gives you the parts to build a harness, a harness SDK gives you a working loop you configure, and a finished harness is a product you run. LangChain and LangGraph are frameworks: you assemble the loop, the state and the tool calls yourself. The Claude Agent SDK and the OpenAI Agents SDK hand you a loop with tool handling, context management and permissions already wired, and you supply the tools and the prompt. Claude Code, Codex CLI, OpenCode, mini-SWE-agent and Hermes Agent are finished harnesses that already have opinions about every row in the table above.
The line matters when you compare results. A score reported for "model X" is a score for model X inside some harness, and the harness a benchmark submitter picked is part of the number whether or not the leaderboard says so.
Nothing about the harness is new
Every agent has had a harness since the first ReAct-style loop, because somebody had to write the code that calls the model again after a tool returns. We called it scaffolding or the orchestration layer; AutoGPT was a harness long before anyone used the word, and LangChain gave people the parts to write their own.
"Harness" came over from testing, where a test harness is the rig you strap something into so you can run it. It caught on for agents once the harness became the product (Claude Code is the obvious example), and once it became clear how much the harness moves results. When researchers at Queen's University held one model fixed (Qwen3-Next-80B-A3B-Instruct) and ran 35 successive releases of the Qwen Code harness (v0.0.10 to v0.10.3) against the same 50 SWE-bench Verified tasks, twice each and graded by the benchmark's test suites, the share of tasks resolved moved between 23% and 39%, and the latest releases used about 668,000 tokens per task against roughly 391,000 for the first nine, more than 70% more for no gain in resolve rate (Ben Sghaier et al., arXiv 2607.03691).
The harness barely moves the score on short tasks, and a lot on long ones
On short, well-specified tasks, the harness moves the score much less than the field assumed. Yangze Liu and Zhongyi Han at Shandong University ran Claude Code 2.1 (releases 2.1.259 to 2.1.273), mini-SWE-agent 2.4.6 and OpenCode 1.18 (1.18.30 and 1.18.31) on SWE-bench Verified under the Harbor 0.22.0 runner, with a 300-step budget per trial (arXiv 2610.04433, "What Does a Harness Buy? Tokens, Mostly.", 3 October 2026). Five models ran through all three harnesses (Qwen3.6-35B-A3B, Qwen3.8-27B, DeepSeek-V4-Flash, GLM-5.3-Flash and HY4-Preview), with Claude Opus 5 added inside Claude Code as an anchor. On the 447-task pool, run with the two Qwen models, Claude Code and mini-SWE-agent, the heaviest and the lightest harness, came out equivalent within five points: the gap was 1.8 points on Qwen3.6-35B-A3B and 1.4 on Qwen3.8-27B. On a 45-task hard subset, swapping the harness flipped 13% of tasks, exactly as many as rerunning the same harness did. SWE-bench's own test suites decide pass or fail, so there is no LLM judge in these numbers. The one harness effect that cleared the noise was a loss: OpenCode trailed by up to 9.2 points on the large pool, and on one model half of that gap came from runs its output cap cut short.
On long tasks that do not fit in one context, the harness can move the score a long way. Haibo Jin and colleagues at UIUC built HERMES, a harness that pairs each repository component with a resident LLM that knows its own code and dependencies, plus a step that picks which components to wake and a step that maps test failures back to them (arXiv 2610.07832). On all 20 whole-repository migration tasks in SWE Refactor Bench, GPT-5.6 Sol at medium effort went from 6.5% under Codex (the paper does not state which Codex release it ran) to 31.0% under HERMES, measured by the benchmark grader's composite score (migration audit plus behavioural checks, no LLM judge named), with three repeats of the HERMES configuration landing at 31.0 ± 1.3. That gain was not free: in the same table the cost per task rose from $6.0 to $24.8. Across four benchmarks (SWE-bench Verified, SWE Refactor Bench, Terminal-Bench 4.0 and DevOps-Gym, each scored by its own grader), HERMES beat matched baseline harnesses by 12.4 points on average. (This HERMES is a research harness and has nothing to do with Nous Research's Hermes Agent below.)
The two results fit together. On a task one context can hold, any competent harness is enough and the model does the work. On a task too big for one context, what decides the result is who holds which part of the state, and that is a harness decision.
The harness sets the bill
Holding the model fixed, three October results put the harness in charge of cost, each through a different mechanism.
Preamble size times step count: up to 3x
Liu and Han found that cost per task differed by up to 3x across the three harnesses with the same model, the same tasks and one price list. The gap is set at the first call. Claude Code 2.1 sends a preamble of system prompt, tool schemas and message wrapper of 16,581 tokens on every step, OpenCode 1.18 sends 7,025 and mini-SWE-agent 2.4.6 sends 829, and the number of steps multiplies whatever the preamble is. Growth per step differed by less than a factor of 1.5 between harnesses. On the two vendor-API models, DeepSeek-V4-Flash and GLM-5.3-Flash, between 96% and 99% of input tokens in every arm were cache hits, so the provider's cached-input price scaled each bill without changing which harness was cheapest.
Cache hit rate: a one-task anecdote with a 13x gap
Jaff posted on 7 October that for LiteLLM's internal coding agent the harness made a 13x difference on the same task. His full table: Hermes Agent with gpt-6-astra cost $126 and took 24 minutes; the Claude Agent SDK with the same model cost $9.59 and took 19 minutes; the Claude Agent SDK with Claude Opus 5.5 cost $9.02 and took 47 minutes. He attributes almost all of the first gap to prompt caching, 0% hits against 96%. Treat this as an anecdote, not a benchmark: it is one task with one run per harness, posted on X on 7 October 2026, and he did not publish the harness versions, the task or a pass or fail grade for each run, so nobody can reproduce it and it shows how large the effect can be, nothing more. A separate bug report on the Hermes Agent repository shows how a harness can land at zero without anyone noticing: when LiteLLM was reached through its OpenAI-compatible endpoint, the harness sent the system prompt without cache markers, the provider served zero cache hits, and the whole prompt was billed at full price on every turn with no error or warning. The reporter's own billing-weighted measurement had the cache-read share of input-side tokens rising from 14.3% to 95.9% once the markers went out. We do not know whether that bug was the cause of Jaff's result; it shows the mechanism.
Decisions the model is asked to make twice: 96.28k to 11.62k tokens
HarnessMM, from Tsinghua University and Microsoft Research Asia (arXiv 2610.32813), starts from the observation that harnesses call the model again after tool feedback even when an earlier call already settled the next action, such as running the tests after a clean compile. It stores those settled decisions with their exact arguments and a guard, and runs them without a model call when the guard holds. The paper's industrial-harness runs use Codex 0.142.5 and DeepSeek Harness dsh-v0.2.0-rc.1, DeepSeek's own agent harness. On HumanEval's 98 held-out test tasks, one attempt each, scored by HumanEval's released functional tests with no LLM judge and the same fixed backbone in both arms, DeepSeek Harness went from 96.28k tokens per task to 11.62k while success rose from 71.43% to 85.71%, and Codex held 90.82% success while its tokens fell from 66.72k to 47.02k (29.54%). The same figure shows the limit: on the 18 training-split tasks of Terminal-Bench 2.0 with DeepSeek-flash as the model, scored by the benchmark's binary verifier, Codex with HarnessMM used far fewer tokens but its success fell from 77.78% to 55.56%. Caching a decision is only safe while the decision is still true.
How a harness decides your cache hit rate
Prompt caching pays only for an exact prefix match, so a harness gets cache hits by keeping the start of every request byte-identical and loses them by changing anything early. Anthropic's prompt caching docs build the cache prefix in the order tools, then system, then messages, and "changes at each level invalidate that level and all subsequent levels"; editing a tool definition throws away the entire cache. A cache read costs 0.1x the base input price on most Claude models and 0.05x on Claude Opus 5.5 and Sonnet 5.5, a write costs 1.25x with the default five-minute lifetime or 2x for one hour, and every hit refreshes the five minutes at no charge.
Read against that rule, the harness choices that decide your hit rate are concrete. Putting the date, the working directory or a git status line in the system prompt changes the prefix every time those values change, which is why Gemini CLI moves them into the first user message. Loading or unloading MCP tools mid-session, or sorting them differently, changes the tool block at the very top. Compaction rewrites history, so the call after a compaction pays to write a new cache. Editing or trimming earlier messages, switching models, routing requests across deployments that do not share a cache, and pausing longer than the TTL all start the prefix over. Where each of Claude Code, Codex CLI, OpenCode, Pi, Aider, Cline and Goose puts its cache breakpoints, read from their source code, is in our prompt caching in coding agents post.
Read your harness's bill from your own logs
Four numbers describe a harness's bill, and you can read all of them from your own logs before you pay for a bigger model:
- The input tokens on the first call are your preamble, and you pay for them again on every step; if most of a 15,000-token preamble is schemas for tools the agent never calls, trim the tool list.
- Steps per task multiply both the preamble and the growing history.
- The cache-read share of input tokens tells you whether the prefix is holding, and Claude's usage block reports cache writes and cache reads separately on every response. The Shandong arms ran at 96% to 99%; a number near zero means the harness is not marking the prefix or is changing it every turn.
- Calls that repeat a decision are model calls whose output was fully predictable from the previous step, such as running the tests after a successful build, and they are the waste HarnessMM goes after.
The model decides to start a subagent, and the harness runs it
Starting a harness with a model and a task gives you the main agent; there is no separate step where the harness decides to spin one up. Subagents work differently, and the decision there sits with the model. The harness offers a tool for delegating work, the model decides mid-task that a search is big enough to hand off, and the harness checks the request, starts a fresh loop with its own prompt and a clean context, and passes the result back.
A useful way to hold it: the model is a manager deciding to bring in a contractor, and the harness is the operations team that sets the rules and hands over the finished work. Some harnesses do start loops on their own, for example to summarise a conversation when context runs out, or a fixed reviewer step in a pipeline, and in those cases the harness really is the one deciding.
Memory across sessions is a harness job
Memory is where this stops being a vocabulary debate for me. Since every model call starts from zero, whatever continuity an agent has across sessions or users was engineered into the harness by somebody, and that is the layer we build Maximem Synap for. A harness's session file lets one run resume; it does not tell the next session, or the next user's agent, what this one learned, which is the gap between agent state and agent memory. The cost numbers point the same way: the cheapest way to carry history is not to replay all of it on every call, and once a session ends even a perfect cache is gone. Compaction and retrieval cover one session; what Claude Code, Codex and Cursor remember between sessions is a small file per project.
Swapping in a stronger model while keeping a weak harness tends to buy less than people expect, because the model can only act on what the harness remembers and allows, and it pays for whatever the harness sends. So the one-line version I now give people: the model decides what to do, the harness decides what it is allowed to do, what it gets to see and what that costs, and the agent is the two of them getting a job done.
Frequently asked questions
What is an agent harness in simple terms?
An agent harness is the code wrapped around a language model that lets it act: the loop that keeps calling the model, the tools it can use, the context it sees on each call, compaction when that context fills, and the permissions that limit what it may touch. The model chooses the next step; the harness carries it out and calls the model again.
Is LangChain an agent harness?
LangChain and LangGraph are frameworks: they give you the parts to build a harness, and what you build with them is your harness. The Claude Agent SDK and the OpenAI Agents SDK sit one level up, giving you a working loop to configure, and Claude Code, Codex CLI and OpenCode are finished harnesses.
Is Claude Code a harness or an agent?
Claude Code is a harness. Running it with a Claude model and a task gives you an agent, and the same harness can run a different model, which is how the Shandong study tested five models inside it.
Does a better harness beat a better model?
On short, well-specified coding tasks, usually not: the Shandong study found three production harnesses within noise of each other on pass rate with the model held fixed. On long tasks that do not fit in one context, the harness can matter more than the model; HERMES took GPT-5.6 Sol from 6.5% to 31.0% on the 20 whole-repository migration tasks of SWE Refactor Bench, at roughly four times the cost per task.
Why does the same model cost more in one harness than another?
Because the harness decides what is sent on every call and how many calls there are. A bigger preamble of system prompt and tool schemas, more steps, a prefix that changes so the prompt cache misses, and model calls that re-decide something already settled all raise the bill with no change to the model.
What is harness engineering?
Harness engineering is the work of designing, measuring and tuning the harness: the loop, tools, context assembly, permissions and verification around a fixed model. Our glossary entry has the short definition.
Sources
- Liu and Han, "What Does a Harness Buy? Tokens, Mostly.", Shandong University, arXiv 2610.04433, 3 October 2026
- Jin, Li, Kuang and Wang, "Harness Engineering for Software Engineering via Modular Executable Dev-Primitives" (HERMES), arXiv 2610.07832, 6 October 2026
- Jiang, Xu, Gao and Zhao, "HarnessMM: Recursive Self-Improvement for Harness Token Efficiency", arXiv 2610.32813, 23 October 2026
- Ben Sghaier, Li, Adams and Hassan, Queen's University, longitudinal study of 35 Qwen Code releases, arXiv 2607.03691, July 2026
- Ishaan Jaff, post and results table, 7 October 2026
- Hermes Agent issue 84506, "Prompt caching never engages for LiteLLM on the OpenAI-compatible wire"
- Anthropic, prompt caching documentation, read 31 October 2026
- Yao et al., ReAct
