There's a clear progression in where the leverage sits when building on LLMs. In the early days it was prompt engineering, because models were fragile enough that phrasing genuinely moved outcomes. Then it was retrieval plumbing, because the models were smart but ignorant of your data. Now that models are strong instruction-followers with large context windows, the marginal gain has moved again, to what the model can see and do on each turn. That's harness engineering: treating the loop, the tools, the context assembly, and the verification step as engineered software with versions, tests, and measurements, rather than as glue code around the "real" work of prompting.
The practices are recognizably normal engineering, which is the point. Version the harness so you can attribute a regression to a change. Build end-to-end evals that exercise the whole loop, not just single model responses, because a harness bug looks fine at the level of an individual completion. Ablate components deliberately: turn off the reranker, halve the retrieval budget, remove the verification step, and see what the eval does. Half the time a component you assumed was carrying the system turns out to contribute nothing, and occasionally something you added as an afterthought is doing all the work.
Budget the context window explicitly instead of letting it fill opportunistically. Give the system prompt, memory, retrieved documents, and tool output each a share, decide in advance what gets compacted when the budget is exceeded, and log when compaction fires. Keep the tool surface small and orthogonal, because twelve overlapping tools produce more wrong selections than four clean ones. Write tool descriptions as if they're API documentation for a competent engineer who cannot ask you questions, which is precisely what they are.
The organizing insight is that most agent failures are harness failures. The model didn't hallucinate a file path; the harness gave it a stale directory listing. The agent didn't lose the plot; the harness compacted away the requirement on turn twelve. The agent didn't loop forever because it lacked judgment; the harness had no stopping condition. When a run goes wrong, the trace almost always shows a moment where the model made a reasonable decision given bad or missing information, and supplying that information was the harness's job.
This is why harness engineering is inseparable from evaluation and observability. You cannot tune what you cannot see, and per-turn traces are the raw material. It's also where cost gets decided: the harness, not the model, determines how many tokens a task consumes, because it controls how many turns run and how much context each one carries. Teams that treat this as a first-class discipline ship agents that get better with each iteration. Teams that treat it as glue code rewrite their agent every time a new model ships.
Why It Matters
Model capability is roughly the same for everyone, since your competitors call the same APIs. The harness is where differentiation actually lives, and it's the part you own end to end. Harness engineering is what converts a demo that works on a good day into a system with a measured failure rate you can drive down deliberately, which is the difference between an agent you show investors and one you put in front of customers.
Example
An agent handling support escalations passes 60% of eval cases and nobody can say why the other 40% fail. Rather than rewriting prompts, the team instruments each turn and ablates one component at a time. Two findings: tool output was truncated at 1,000 characters, so the agent never saw the tail of the customer history where the actual complaint lived; and memory retrieval only ran on turn one, so context established mid-conversation was lost. Two harness fixes, no prompt changes, no model change. Pass rate goes to 85%.