News · SI

Google researchers open-source Harness², which tunes an agent's harness instead of its model

Google Cloud AI Research has released Harness² under Apache 2.0. It keeps model weights frozen and rewrites the scaffolding around them, from system prompts to skills and subagents, by testing candidate edits on the task at hand and keeping lessons for the next one. The authors report gains of up to 11.8 points on professional benchmarks, at the price of extra inference.

GoogleHarness²AgentsOpen sourceAgent harness

Researchers at Google Cloud AI Research have open-sourced Harness², a framework that improves an AI agent by rewriting the system around the model rather than retraining the model itself[1]. The code is on GitHub under Apache 2.0, with a paper posted to arXiv on 9 October[2].

01 / 05

The harness is the product

A harness is everything that turns a model into an agent: prompts, tools, memory, skills and orchestration[1]. Harness² exposes eight editable parts of it, namely the system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins[1].

The authors' starting point is that the same model can perform very differently depending on that scaffolding, and that hand-tuning it for each new kind of task doesn't scale[2].

02 / 05

How it works

Harness² runs two loops. The first improves the harness for the task in front of it with a Propose–Probe–Compose step: propose edits, run the same task with each, and compose a refined harness from the differences between those runs, without needing a known correct answer[1]. The second loop saves the adopted harnesses and evidence-backed lessons into a library that shapes how later tasks get refined[1]. It can scale by trying several candidates in parallel or by refining one after another[1].

03 / 05

The results

The team tested it on WorkBuddy-Bench (office, web and code work), JobBench and Harvey's Legal Agent Benchmark, using frozen Gemini 3.5 Flash and GLM-5 models inside the off-the-shelf OpenCode and Codex harnesses[2]. A single improvement step raised scores by up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench and 11.8 percentage points on the legal benchmark's partial pass rate[2].

Starting from OpenCode with five parallel candidates, Harness² beat the best unmodified production harness on each WorkBuddy-Bench domain by 4.6 to 5.8 points[2]. Claude Code was the strongest fixed harness on code and office tasks in that comparison, and Codex on web tasks[2]. Notably, the default harness editor in the experiments was Anthropic's Claude Opus 4.8[2].

04 / 05

The catches

The extra performance isn't free: proposing and probing changes means more executions and more inference cost[2]. The authors also say comparing runs doesn't guarantee an edit is correct or will carry over to a different task, and their failure analysis includes a case where the system picked an output scoring 55 over one scoring 95[2]. The repository says it is not an officially supported Google product[1], and the setup guide warns that the Codex executor runs with its sandbox disabled[3].

05 / 05

What it means for builders

If you run agents on repeatable professional work, this is a concrete argument that tuning your harness can buy as much as a model upgrade. You don't have to adopt Harness² wholesale to use the idea: test a prompt, skill or tool change against the same task, keep what the runs support, and write down why. If you do try the code, run it in an isolated environment with network limits, as the setup guide suggests.

FAQ

Questions, answered.

01

What is Harness²?

An open-source framework from Google Cloud AI Research for improving an AI agent's harness, meaning the system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins around a model. It edits those pieces automatically while keeping the model's weights fixed.

02

How does it improve a harness without a right answer to check against?

It uses a Propose–Probe–Compose step: propose edits to the harness, run the same task with those edits, and combine the changes that the differences between runs support. A second loop stores the adopted harnesses and lessons in a library that guides refinement on later tasks.

03

How much does it help?

In the paper, one improvement step lifted Gemini 3.5 Flash and GLM-5 by up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench and 11.8 percentage points on the partial pass rate of Harvey's Legal Agent Benchmark. Running five candidates in parallel added up to 3.9, 4.2 and 6.7 points more. These are best-case gains measured by the authors.

04

Is it an official Google product?

No. The repository is Apache 2.0 licensed and its disclaimer says it is not an officially supported Google product.

05

What does it cost to run?

More than running the agent alone. The authors say proposing, probing and composing changes needs extra executions and editor sessions, so a bigger recursion budget means more inference cost. Their experiments used Claude Opus 4.8 as the default harness editor.

06

Can I try it?

Yes. It needs Linux, Python 3.12 and uv, and a local demo runs without model credentials. Benchmark runs need Node.js 20+, the Google Cloud CLI and Vertex AI model access, and the setup guide warns that the Codex executor runs with its sandbox disabled.