acceptodds
Under review as a conference paper at ICLR 2027

Harness-Zero: Harness Distillation via Agent-as-Harness

18 views

Abstract

Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment. Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. We therefore study agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so that its gains survive under a single fixed target harness. The challenge is that the two harnesses differ in action space and available information, so guidance from the optimized harness cannot serve directly as supervision for the target one. We introduce Harness-Zero, which resolves this through agent-as-harness. During data collection, a harnessing agent wraps the student: guided by the optimized harness, it reviews each proposed response before execution and, when a correction is needed, rewrites it in the student's own action space. Every correction is therefore expressed as behavior the student could have produced itself. Fine-tuning on the resulting trajectories internalizes that behavior into the model, so the harness can be removed at deployment. Our experiments spanning knowledge work, tool use, and science domains show that: (1) For frontier LLMs using the same evolved harness, agent-as-harness outperforms code-as-harness. (2) With the specialized harness removed at deployment, Harness-Zero improves the base model's macro-average task success from 23.3% to 44.3%, even exceeding the 41.7% it reaches with that harness still attached. (3) Harness-Zero recovers harness-induced behaviors absent from the base model, with 82.3% macro-average recovery across 28 patterns in the three domains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agent harness, harness optimization, harness distillation, agent-as-harness, post-training

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_B4r4OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose HARNESS-ZERO, a method for making a small model behave as if it had read a very long framework manual. This method evolves an elaborate framework full of tools, middleware, skills, and memory; then Adapt converts all of that machinery into advice. Qwen3.5-9B attempts the task, GPT inspects every move, and whenever Qwen says “done” too early, GPT replaces it with a Bash script. The resulting trajectories are used to fine-tune Qwen until it can independently reproduce the framework’s rituals, without GPT, h*, or K at deployment time.

Critical strengths
  1. Presentation is good, Figure 1 is clear

  2. The idea of adapt is relatively novel; it can update the harness.

  3. The result in table 3 is good.

Critical weaknesses
  1. This is more likely to distill a big model in critical steps.

  2. The cost seems high since the model needs to check the steps of the student model. (even higher than just using a big model to rollout.)

  3. I'm a little bit concerned about the reward hacking / overfitting problem.

  4. It seems hard to generalize to scenarios that need long thinking in a trajectory.

I think this idea has potential, but it still has some flaws.

Questions for authors

How to generate reasonable thinking via this method? (Is it natural? Will it distill something strange in? For example, some premise in skills)

10 comments
AuthorsOpenReview ↗

Response to W1

Thank you for your review and for recognizing our clear presentation, the novelty of harness adaptation, our ablation results, and the potential of this direction. Below, we address each of your concerns.

W1: This is more likely to distill a big model in critical steps.

Distilling a stronger model alone is insufficient to improve the student. Below, we first present controlled ablations across all three benchmarks showing that distilling the teacher model without the reference harness KK, whether from standalone teacher rollouts or via step-level interventions, falls far short of Harness-Zero. Then we discuss two reasons why KK-guided, step-level correction is more effective.

Controlled ablations across all three benchmarks show that teacher intervention at critical steps without the reference harness KK fails to match Harness-Zero. Across SpreadsheetBench, AppWorld, and USPTO, we keep the base student (Qwen3.5-9B), the strong teacher (GPT-5.6 Sol), the minimal target harness hh, the number of retained training trajectories, and the SFT recipe fixed while varying the supervision source:

SettingSpreadsheetBenchAppWorld (SGC)USPTOAvg.
Base model (untrained, under hh)31.026.812.023.3
Standalone GPT-5.6 Sol rollouts under hh (SFT)20.0 (487 traj.)32.1 (282 traj.)12.0 (500 traj.)21.4
Review with empty KK under hh (SFT)28.0 (487 traj.)48.2 (282 traj.)11.0 (500 traj.)29.1
Harness-Zero (Review with KK under hh, Ours)44.0 (487 traj.)58.9 (282 traj.)30.0 (500 traj.)44.3

Letting the exact same teacher intervene at critical steps without the reference harness (Review with empty K) reaches only 29.1% average accuracy (vs.\ 44.3% for Harness-Zero). Direct SFT on the teacher's standalone rollouts performs even worse (21.4% average). Moreover, on USPTO (Table 3), even giving the teacher the ground-truth answer during review (Review with oracle answer), the distilled student reaches only 15.0% (vs.\ 30.0% with KK). Finally, behavioral analysis (Table 4) confirms that the student trained with KK recovers 82.3% of 28 h⋆h^\star-exclusive behavior patterns that the base model never exhibits on its own.


We discuss two reasons why KK-guided, step-level correction is more effective.

First, structured harness procedures are much easier for a small model to internalize than a frontier teacher's unconstrained reasoning. Recent work on shared-trajectory collaboration shows that smaller models struggle to build upon or internalize guiding steps from a stronger collaborator when those steps rely on reasoning beyond the student's inherent capability (Li & Goyal, 2026). When GPT-5.6 Sol intervenes without KK or with only the oracle answer, its edits rely on implicit model knowledge and ad-hoc shortcuts that the 9B student cannot generalize. By contrast, KK translates the evolved harness into standardized, tool-grounded procedures (such as explicit inspection, candidate enumeration, and mechanical verification scripts) that focus supervision on learnable harness behaviors (Choi et al., 2026).

Second, minimal step-level rewrites keep the training trajectories on-policy and teach the student how to recover from its own intermediate errors. Because a strong teacher solves tasks along its own trajectory, standalone teacher rollouts rarely enter the intermediate failure states that a 9B student encounters at test time, leaving the student with no supervision on how to backtrack or recover once its own rollout makes a mistake (Li & Goyal, 2026). In practice, we find that a student trained on standalone teacher rollouts frequently gets stuck in repetitive error loops, whereas in Harness-Zero the student drives the trajectory under hh, enters its natural failure states, and learns error recovery from minimal corrections at those exact states:

Distillation Source (evaluated under hh)SpreadsheetBench Acc.SpreadsheetBench Repeated Command (≥5×\ge 5\times)AppWorld (SGC)AppWorld Repeated Command (≥5×\ge 5\times)
Standalone GPT-5.6 Sol rollouts20.0%13.0%32.1%21.0%
Harness-Zero (Ours)44.0%0.0%58.9%1.2%
AuthorsOpenReview ↗

Response to W2 and W3

W2: The cost seems high since the model needs to check the steps of the student model.

Step-level review (checking) is a one-time offline data-collection investment that permanently cuts deployment token usage by 27%–87% while outperforming simpler collection alternatives. Below, we first show the substantial inference-time savings unlocked after this one-time collection pass, and then compare against two cheaper collection alternatives (direct strong-model distillation and student rejection sampling).

First, step-level review is paid only once during data collection and substantially reduces deployment inference costs. The harnessing agent checks student steps only during offline trajectory collection (441–700 training trials per domain); after a lightweight 2-epoch LoRA SFT run (~1–2 GPU hours), the harnessing agent, K\mathcal{K}, and the external harness h⋆h^\star are all removed at deployment. Retaining h⋆h^\star at deployment forces every query to repeatedly pay for long skill/memory prompts, custom tool schemas, and middleware guard retries across turns, whereas our distilled student runs under the minimal bash harness hh, cutting average per-task input and output tokens by 26.6%–87.0% while improving average test accuracy:

BenchmarkAvg. Input Tokens / Task (Base + h* → Ours under h)Input ReductionAvg. Output Tokens / Task (Base + h* → Ours under h)Output ReductionTest Performance (Base + h* → Ours under h)
SpreadsheetBench515,964 → 222,143-56.9%17,412 → 9,804-43.7%39.0% → 44.0%
AppWorld339,755 → 249,514-26.6%4,981 → 3,545-28.8%48.2% → 58.9%
USPTO118,099 → 15,364-87.0%9,548 → 2,310-75.8%38.0% → 30.0%

Second, distilling the big model directly does not work. As shown in our response to W1 (Table 3), distilling standalone strong-model rollouts does not improve the student, so its per-rollout cost yields less training gain.

Third, another alternative—rejection sampling—wastes most of the rollout budget and still yields little improvement. On USPTO, we ran the base student under hh with doubled rollout budget (1,000 rollouts vs.\ 500 for Harness-Zero) and trained on the passing trajectories with the same SFT recipe. Only 8.0% of rollouts passed, and test accuracy reached only 14.0% (vs.\ 12.0% base and 30.0% for Harness-Zero). Without step-level review, 92% of the budget is discarded, and the remaining 8% only repeats behaviors the student already has. Paying for step-level review injects the missing harness patterns into the trajectory (which is beneficial whenever the student lacks those patterns) and is especially cost-effective when student successes are rare.

W3: I'm a little bit concerned about the reward hacking / overfitting problem.

Regarding reward hacking, neither the evolved harnesses (h⋆,Kh^\star, K) nor the teacher interventions introduce reward hacking. For the harnesses themselves, all components in h⋆h^\star and KK are human-readable files that we carefully reviewed to ensure they contain no task-specific answers or shortcuts, and recent work further demonstrates that harness optimization can yield generalizable gains across unseen tasks (Xia et al., 2026). During trajectory collection, teacher interventions do not introduce reward hacking either: the harnessing agent only has read-only access to KK and the student-visible history, with no access to test cases, reference answers, verifier feedback, or the student sandbox.

Regarding overfitting, distilling a domain harness brings general improvements within that domain, and we also add experiments on multi-domain joint training. Evaluating on held-out test splits shows that distilling a domain harness yields strong generalization to unseen tasks within that domain (Table 2), which is already valuable on its own. To further examine whether Harness-Zero can improve general agent capabilities across domains, we trained a single LoRA model by directly merging the training trajectories from all three domains (487 SpreadsheetBench, 282 AppWorld, and 500 USPTO). Under the minimal harness hh, this jointly trained model improves over the base model on every benchmark:

Setting (under hh)SpreadsheetBenchAppWorldUSPTOAvg.
Base model (untrained)31.026.812.023.3
Harness-Zero (jointly trained, single LoRA)47.039.321.035.8
Harness-Zero (per-domain specialists)44.058.930.044.3

This suggests that Harness-Zero can build general-purpose agent capabilities by distilling multiple domain harnesses into one model. Closing the remaining gap to single-domain specialists will benefit from scaling data collection and tuning the data mixture, as in standard general-purpose post-training.

AuthorsOpenReview ↗

Response to W4 and Q1

W4: It seems hard to generalize to scenarios that need long thinking in a trajectory.

First, scenarios requiring long thinking and long-horizon execution are a strength of our paradigm, as already reflected in our three evaluated benchmarks. Our evaluation covers both reasoning-intensive scientific derivation (USPTO retrosynthesis) and long-horizon multi-step interaction (SpreadsheetBench and AppWorld, where trajectories reach up to 36–48 turns and over 44k tokens). On such tasks, an unassisted small model often loses track of constraints or skips critical verification across turns, whereas the reference harness KK explicitly encodes the domain's multi-step workflow, state checks, and verification rules. Guided by KK, the harnessing agent translates these structured workflows into deeper step-level reasoning and verification. For example, the harnessing agent expands median reasoning length by +40.4% on AppWorld and replaces superficial commands with KK's comprehensive verification scripts (2.3$\times$ and 5.9$\times$ longer in median command length on SpreadsheetBench and USPTO).

Second, recent follow-up and concurrent studies further confirm that this paradigm scales effectively to challenging long-horizon agent benchmarks and reasoning-intensive mathematical domains. Specifically, Recursive Self-Rewrite (Li et al., 2026) extends harness-guided trajectory rewriting to demanding, long-horizon terminal benchmarks—including Terminal-Bench 2/3/4, Terminal-Bench Hard, and Long-Horizon Terminal-Bench (where trajectories span dozens of turns and up to 180 minutes). Meanwhile, On-Policy Harness Self-Distillation (OPHSD; Zhao et al., 2026) applies temporary-harness distillation to reasoning-intensive competition mathematics (AIME 2024/2025, OlympiadBench, and HMMT 2025), improving pass@8 by +10.8% on HMMT 2025 after the scaffold is removed. Together with our results, these studies confirm that harness distillation generalizes well to both long-horizon execution and deep reasoning tasks.

Q1: How to generate reasonable thinking via this method? (Is it natural? Will it distill something strange in? For example, some premise in skills)

Unnatural or ungrounded thinking is rare and can be systematically audited and excluded. Our harnessing prompt (Appendix C.3) explicitly instructs the teacher to write every replacement's reasoning in the student's first-person voice, grounded strictly in student-visible observations without citing external skills or unverified premises. Across all 4,082 REPLACE steps in our training sets, programmatic checks found 0% leakage of private paths or component names and flagged only 8.2% of steps for reviewer-perspective phrasing. On a random sample of 100 remaining REPLACE steps, a combined human and LLM-judge audit found 0% third-person reviewer voice or ungrounded skill premises, with only 1.5% of the reasoning text briefly mentioning the student's draft command.

These occasional artifacts are filtered out before training via deterministic scripts and LLM-as-a-judge screening. As detailed in Section 3.3 and Appendix D, our pipeline automatically drops any trajectory containing private paths or review tool names, and masks the SFT loss on the 8.2% of REPLACE reasoning turns that slip into a reviewer's voice (while keeping their valid tool calls trainable). Combined with LLM-judge screening, this prevents unnatural phrasing from entering the training objective.

Recent advances in foundation models have made trajectory and reasoning rewriting a reliable, emerging post-training paradigm. As frontier models have become capable of rewriting reasoning traces while respecting strict perspective and information constraints, several concurrent works have successfully adopted similar data-shaping paradigms. For instance, Recursive Self-Rewrite (Li et al., 2026) uses a model-based planner–critic–executor loop to rewrite multi-harness trajectories into clean, leakage-free demonstrations under a general harness. Finetuning with Sampling (Karan et al., 2026) iteratively rewrites off-policy expert reasoning traces into on-policy trajectories tailored to the student's distribution, enabling standard SFT to match or exceed RL across chemistry and mathematical reasoning.

Reviewer_NGqbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies agent harness distillation: moving the gains of an optimized, domain-specific harness into model weights, so that a model deployed under one fixed, minimal harness keeps them. HARNESS-ZERO has three stages (Fig. 1):

  1. A three-round, skill-guided evolution loop builds a student-side harness h* of tools, middleware, skills and memory (App. A). h* is adapted into a private reference harness K of action recipes, review middleware, review guidance and failure patterns (Table 6).
  2. The student runs under a target harness h with one bash execute tool. A harnessing agent reads K, reviews every proposed response, and either passes it or replaces it with a complete response valid under h (Eq. 3), within a per-trial replacement budget. It cannot see the sandbox, hidden solutions or verifier feedback.
  3. LoRA SFT on the accepted responses (Eq. 4).

The benchmarks are SpreadsheetBench Verified (300 train / 100 test), AppWorld (147 train / 168 test_normal tasks in 56 scenarios) and USPTO retrosynthesis (500 / 100). Reported results:

  • Without training, with the same frontier model as student and reviewer, agent-as-harness averages 81.1%, against 78.1% for the student run under h* (six settings; Table 1).
  • Distilling into Qwen3.5-9B, with GPT-5.6 Sol as the harnessing agent, raises the macro average under h from 23.3% to 44.3%. The base model with h* attached reaches 41.7% (Table 2).
  • On USPTO, five alternative supervision sources yield 3–15% after SFT, against 30% (Table 3).
  • The distilled model recovers 82.3% of 28 harness-exclusive behaviors on average (Table 4). On two weaker models, the training-free advantage reverses (Table 8).
Critical strengths
  • A precise formulation and a careful, general mechanism. Harness distillation is defined formally (Eqs. 1–5). Each correction is a complete response valid under h. The reviewer cannot read the sandbox, hidden solutions or verifier feedback, so any extra evidence must come from an action the student executes (#3.2). Table 6, the pre-filled-cell example (Codes 1–3) and the full prompts make the design reusable.
  • Large gains under a minimal harness, with informative and candid controls. The distilled 9B model beats its base under h in every domain: 31.0 -> 44.0, 26.8 -> 58.9 and 12.0 -> 30.0 (Table 2). On USPTO, five other sources yield 3–15%, against 30% for K-guided review: a stronger model's rollouts, two sets collected under h*, empty-K review and oracle-answer review (Table 3). The paper also reports unfavorable results. HARNESS-ZERO trails h on USPTO (30.0 vs 38.0). The training-free advantage turns negative on weaker models (−1.0 and −12.0 points; Table 8). Collection is about 2.4× slower on USPTO (App. F).
Critical weaknesses

The source of the distillation gains is not isolated in the domains that carry the headline. The claim that distillation "surpasses h*" rests on SpreadsheetBench and AppWorld. In those domains, "we retain a trial only if its verifier reward is 1.0" (App. D). The replacements, reasoning included, are written by GPT-5.6 Sol, which scores 83.0 and 96.4 there under h alone (Table 1). The training sets are therefore rejection-sampled and partly written by a frontier model. All Table 3 controls are on USPTO, where every trial is kept and where HARNESS-ZERO trails h*. Three success-filtered baselines on SpreadsheetBench and AppWorld are missing: rejection-sampling fine-tuning of the base student, filtered GPT-5.6 Sol rollouts, and filtered empty-K review. Without them, internalized h* behavior cannot be told apart from verifier-filtered self-training plus teacher corrections. The composition of the training data (replaced turns, teacher-written tokens, distinct tasks) is not reported.

Questions for authors
  • On SpreadsheetBench and AppWorld, please add three success-filtered baselines with the same recipe and checkpoint, matched in collection trials and retained trajectories (487 and 282):

    • (a) rejection-sampling fine-tuning of the base student under h, e.g., three attempts per training task, keeping reward 1.0;
    • (b) GPT-5.6 Sol under h, filtered and subsampled;
    • (c) empty-K review under budgets 5, 3 and 1, filtered.
  • Please report three training seeds for HARNESS-ZERO and three evaluation runs of the base model under h and under h*. Please give paired task-level bootstrap CIs for 44.3 vs 41.7 and for each setting behind 81.1 vs 78.1. Please also evaluate base Qwen3.5-9B under h and h* on the same A100 serving stack, chat template and decoding settings as the fine-tuned checkpoints.

  • What does each training set contain? Please report:

    • collection trials per task (one trial per budget would give up to 900 SpreadsheetBench and 441 AppWorld trials);
    • collection success rates and the number of distinct tasks covered;
    • the fraction of accepted turns from REPLACE, and the fraction of trained tokens written by the harnessing agent;
    • the number of masked reasoning spans, and the trajectories dropped by the privacy filter or the 65,536-token limit.
  • How was K produced? Was it written by an agent or by hand, in how many iterations, and checked against which tasks? Which K components lack an h* counterpart? Please rerun Table 1's (h,K) column without them, or with equivalent middleware added to h*.

  • Does one LoRA adapter trained on all 1,269 trajectories match the per-domain adapters on all three test sets? How does each per-domain adapter perform on the other two domains?

6 comments
AuthorsOpenReview ↗

Response to W1 and Q1 (Part 1)

Thank you for your review and for recognizing our precise formulation, the careful and reusable mechanism design, the large gains under a minimal harness, and our informative and candid controls. Below, we address each of your concerns.

W1: The source of the distillation gains is not isolated in the domains that carry the headline. The claim that distillation "surpasses h⋆h^\star" rests on SpreadsheetBench and AppWorld, where trials are kept only if reward = 1.0 and replacements are written by GPT-5.6 Sol. All Table 3 controls are on USPTO. Three success-filtered baselines on SpreadsheetBench and AppWorld are missing: rejection-sampling fine-tuning of the base student, filtered GPT-5.6 Sol rollouts, and filtered empty-KK review. The composition of the training data is not reported.

Q1: On SpreadsheetBench and AppWorld, please add three success-filtered baselines with the same recipe and checkpoint, matched in collection trials and retained trajectories (487 and 282): (a) rejection-sampling fine-tuning of the base student under hh; (b) GPT-5.6 Sol under hh, filtered and subsampled; (c) empty-KK review under budgets 5, 3 and 1, filtered.

Following your suggestion, we ran all three requested success-filtered baselines on SpreadsheetBench and AppWorld:

  • (a) rejection-sampling fine-tuning (RFT) of the base student under hh;
  • (b) filtered standalone GPT-5.6 Sol rollouts under hh;
  • (c) filtered empty-KK review under budgets 5, 3, and 1.

We additionally ran an unfiltered Harness-Zero control on SpreadsheetBench, i.e.:

  • training on all 691 collected KK-reviewed trajectories (both reward = 1 and reward = 0) without verifier filtering.

All models are trained and evaluated with the same SFT recipe. The results are summarized in the table below, followed by a step-by-step analysis and two mechanistic reasons why KK-guided step-level correction is essential.

Setting (evaluated under minimal harness hh)SpreadsheetBenchAppWorld (SGC)USPTO (Table 3, unfiltered)Avg.
Base model (untrained, under hh)31.026.812.023.3
Base model with specialized harness h⋆h^\star attached39.048.238.041.7
(a) Rejection-sampling SFT (RFT of base student under hh, reward = 1.0)31.0 (713 traj.)50.0 (497 traj.)14.0 (80 filtered traj.)31.7
(b) Standalone GPT-5.6 Sol rollouts under hh (SFT)20.0 (487 traj.)32.1 (282 traj.)12.0 (500 unfiltered traj.)21.4
(c) Review with empty KK under hh (SFT)28.0 (487 traj.)48.2 (282 traj.)11.0 (500 unfiltered traj.)29.1
Harness-Zero (unfiltered, all reviewed trajectories)43.0 (691 traj.)—30.0 (500 unfiltered traj.)—
Harness-Zero (main Table 2)44.0 (487 traj.)58.9 (282 traj.)30.0 (500 unfiltered traj.)44.3

First, removing reward = 1.0 filtering on SpreadsheetBench still reaches 43.0% and surpasses h⋆h^\star (39.0%), proving that verifier filtering is not the driver of the headline gain. The advantage over h⋆h^\star comes from the KK-guided interventions rather than post-hoc verifier filtering.

Second, across all three domains, all three matched baselines (RFT, standalone GPT-5.6 Sol rollouts, and empty-KK review) fall short of Harness-Zero:

  • For (a) RFT, even when we give the base student 2×2\times the rollout budget (1,800 rollouts on SpreadsheetBench, 882 on AppWorld, and 1,000 on USPTO) and train on 46%–76% more successful trajectories (713 vs.\ 487 on SpreadsheetBench; 497 vs.\ 282 on AppWorld), RFT trails Harness-Zero by 13.0 points on SpreadsheetBench (31.0% vs.\ 44.0%), 8.9 points on AppWorld (50.0% vs.\ 58.9%), and 16.0 points on USPTO (14.0% vs.\ 30.0%). Without KK-guided review, RFT only reinforces easier tasks the student can already solve, failing to introduce the missing h⋆h^\star behaviors.
  • For (b) standalone GPT-5.6 Sol rollouts and (c) empty-KK review, standalone teacher distillation (b) reaches only 20.0% on SpreadsheetBench, 32.1% on AppWorld, and 12.0% on USPTO (21.4% avg.\ vs.\ 44.3% for Harness-Zero), while empty-KK review (c) reaches 28.0%, 48.2%, and 11.0% (29.1% avg.). Moreover, our training-set audit (see our response to Q4) confirms that the gains do not come from having GPT-5.6 Sol write most of the trajectory: on AppWorld, where Harness-Zero beats h⋆h^\star by the largest margin (+10.7 points), 85.1% of accepted turns are PASS turns written by the 9B student itself (only 13.4% teacher-written tokens), whereas on USPTO, where GPT-5.6 Sol writes 88.4% of turns, Harness-Zero does not beat h⋆h^\star.
AuthorsOpenReview ↗

Response to W1 and Q1 (Part 2)

We discuss two mechanistic reasons why standalone teacher rollouts (b) and unguided/oracle teacher reviews (c) fail while KK-guided review succeeds:

  • First, structured harness procedures are much easier for a small model to internalize than a frontier teacher's unconstrained reasoning. Recent work on shared-trajectory collaboration shows that smaller models struggle to build upon or internalize guiding steps from a stronger collaborator when those steps rely on reasoning beyond the student's inherent capability (Li & Goyal, 2026). When GPT-5.6 Sol intervenes without KK or with only the oracle answer, its edits rely on implicit model knowledge and ad-hoc shortcuts that the 9B student cannot generalize. By contrast, KK translates the evolved harness into standardized, tool-grounded procedures (such as explicit inspection, candidate enumeration, and mechanical verification scripts) that focus supervision on learnable harness behaviors (Choi et al., 2026).

  • Second, minimal step-level rewrites keep the training trajectories on-policy and teach the student how to recover from its own intermediate errors. Inspecting the trajectories in our ablations confirms a failure mode (also noted by Li & Goyal (2026)): when trained on the teacher's standalone rollouts, the student only sees clean, first-try solutions and never learns how to backtrack or recover once its own rollout deviates into an erroneous intermediate state. In Harness-Zero, the student drives the trajectory under hh, enters its natural failure states, and receives the smallest coherent correction at that exact state before regaining control. Because the rewrites demonstrate how to diagnose and repair a state the student actually reached, the resulting supervision stays close to the student's policy and directly teaches error recovery.

References

  • Aochong Li and Tanya Goyal. Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectories? ICLR, 2026.
  • Moonseok Choi, Taehong Moon, Giung Nam, and Juho Lee. Harness-Aware Distillation for Small Language Model Agents. arXiv preprint arXiv:2610.02858, 2026.
AuthorsOpenReview ↗

Response to Q2, Q3, and Q4

Q2: Three training seeds for Harness-Zero; three eval runs of the base model under hh and h⋆h^\star; paired task-level bootstrap CIs for 44.3 vs.\ 41.7 and for 81.1 vs.\ 78.1.

Multi-seed training. Across three training seeds (42, 43, and 44) on SpreadsheetBench, Harness-Zero achieves 44.0%, 43.0%, and 49.0%, consistently outperforming both the untrained base model (31.0%) and the base model with h⋆h^\star attached at runtime (39.0%).

Repeated evaluations. Because all test evaluations use greedy decoding (temperature = 0), repeated evaluation runs of base Qwen3.5-9B under hh and h⋆h^\star are near-deterministic.

Paired task-level bootstrap CIs.

SettingOursRuntime Harness h⋆h^\starPaired Δ\Delta (pp)95% CI of Δ\Delta
Distillation: Harness-Zero under hh vs.\ Base with h⋆h^\star
SpreadsheetBench (N=100N=100)44.0%39.0%+5.0[−7.0,+17.0][-7.0, +17.0]
AppWorld (N=56N=56 scenarios)58.9%48.2%+10.7[−5.4,+26.8][-5.4, +26.8]
USPTO (N=100N=100)30.0%38.0%-8.0[−19.0,+3.0][-19.0, +3.0]
3-benchmark macro-average44.3%41.7%+2.6[−4.9,+10.1][-4.9, +10.1]
Training-free: Agent-as-harness (h,K)(h, \mathcal{K}) vs.\ Runtime Harness h⋆h^\star
SpreadsheetBench · DeepSeek-V4-Pro86.0%77.0%+9.0[0.0,+18.0][0.0, +18.0]
SpreadsheetBench · GPT-5.6 Sol88.0%84.0%+4.0[−2.0,+10.0][-2.0, +10.0]
AppWorld · DeepSeek-V4-Pro89.3%91.1%-1.8[−8.9,+3.6][-8.9, +3.6]
AppWorld · GPT-5.6 Sol96.4%94.6%+1.8[−3.6,+8.9][-3.6, +8.9]
USPTO · DeepSeek-V4-Pro60.0%56.0%+4.0[−5.0,+13.0][-5.0, +13.0]
USPTO · GPT-5.6 Sol67.0%66.0%+1.0[−5.0,+7.0][-5.0, +7.0]
6-setting macro-average81.1%78.1%+3.0[−0.2,+6.3][-0.2, +6.3]

Notably, Harness-Zero under h (44.3%) matches or outperforms Base with h* (41.7%) after completely removing h⋆h^\star, cutting per-task inference input/output tokens by 56.9% / 43.7% on SpreadsheetBench, 26.6% / 28.8% on AppWorld, and 87.0% / 75.8% on USPTO (see our response to Reviewer sZMT Q3 for the full details).

Q3: Evaluate base Qwen3.5-9B under hh and h⋆h^\star on the same A100 stack, chat template, and decoding settings as the fine-tuned checkpoints.

Re-evaluating base Qwen3.5-9B on the exact same A100 vLLM serving stack, chat template, and decoding settings yields virtually identical results (matching exactly or within 1–2% across repeated runs). Specifically, USPTO under hh (12.0%, varying within 1–2% across repeated A100 evaluations) and h⋆h^\star (38.0%) as well as SpreadsheetBench under h⋆h^\star (39.0%) were already evaluated on our self-hosted A100 vLLM stack, and re-evaluating SpreadsheetBench under hh on the identical A100 stack reproduces the exact same 31.0% (31/100) score. Thus, the serving stack and decoding configuration do not account for the distillation gains.

Q4: Please report, for each training set: trials per task, collection success and task coverage, REPLACE turn fraction and teacher-written token fraction, masked reasoning spans, and trajectories dropped by the privacy filter or the 65,536-token limit.

Below we report the requested training-set statistics across all three domains.

MetricSpreadsheetBenchAppWorldUSPTO
Collection trials per task3 / 2 (700 total)3 (441 total)1 (500 total)
Collection success rate (reward=1.0\text{reward} = 1.0)81.3% (569 / 700)66.0% (291 / 441)59.4% (297 / 500)
Retained SFT trajectories487282500
Distinct training tasks covered264 / 300 (88.0%)133 / 147 (90.5%)500 / 500 (100.0%)
REPLACE turn fraction53.6% (1,539 / 2,869)14.9% (794 / 5,336)88.4% (1,749 / 1,978)
Teacher-written token fraction49.5%13.4%94.1%
Masked reasoning spans171 (11.1% of REPLACE)21 (2.6% of REPLACE)143 (8.2% of REPLACE)
Dropped by privacy filter / 65,536-token limit0 / 10 / 00 / 0
Reviewer_sZMTOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper aims to distill the capability gains provided by optimized, domain-specific harnesses into model parameters, enabling models to rely solely on a fixed, minimal harness at deployment. Two challenges motivate this approach: differences in action spaces and available information between specialized harnesses and the deployment environment make their behavioral trajectories difficult to imitate directly, while a general-purpose harness may sacrifice domain-specific behavioral advantages. To address these challenges, the authors propose HARNESS-ZERO, which uses an agent-as-harness mechanism to construct deployment-compatible training data. First, a domain-specific harness is optimized on training tasks, and its tools, workflow rules, and accumulated experience are adapted into private reference guidance for a teacher agent. The student then performs tasks under the target harness, while the teacher reviews each proposed response and makes minimal corrections when necessary. All corrections are executed through the student’s native action interface. The resulting actions and environmental feedback enter the student-visible trajectory, whereas the teacher’s private review information remains excluded. Finally, the student learns from these trajectories through supervised fine-tuning, allowing the specialized harness and teacher agent to be removed at deployment.

The authors fine-tune Qwen3.5-9B separately for the three domains represented by SpreadsheetBench Verified, AppWorld, and USPTO Retrosynthesis. Experiments show that the macro-average performance of the domain-specific distilled models improves from 23.3% to 44.3%, exceeding the 41.7% achieved by the base model with the specialized harnesses retained. However, the distilled model does not fully retain the specialized harness’s benefits on USPTO. Ablation studies show that harness-guided correction trajectories provide more effective distillation supervision than direct imitation of stronger-teacher trajectories, review without domain-specific guidance, or review with access to oracle answers. Behavioral analysis further reports an average recovery rate of 82.3% across 28 selected harness-exclusive behavior patterns.

Critical strengths

The paper distinguishes between system-level improvements enabled by external harnesses and improvements in the model’s own capabilities. It points out that the benefits of specialized harnesses often depend on retaining them at deployment and focuses on how to internalize these benefits in model parameters. Its analysis of differences in action spaces and available information clearly identifies the obstacles to directly learning from trajectories generated under specialized harnesses.

HARNESS-ZERO uses private reference guidance KK to guide a teacher agent in reviewing the student’s proposed actions, translating the tool-use procedures, workflow rules, and accumulated experience encoded in an optimized harness into corrective actions executable under the target harness. By requiring corrections to be grounded in student-visible information and executed through the student’s native interface, the method converts external guidance into deployment-compatible training trajectories. This establishes a coherent pipeline from harness optimization to capability internalization, directly addressing the paper’s central challenges.

Beyond reporting final task performance, the paper systematically compares alternative sources of supervision, including direct imitation of stronger-teacher trajectories, review without domain-specific guidance, and review with access to oracle answers. These comparisons help clarify how different training signals affect distillation performance. The models trained separately for each domain all achieve substantial gains, while the behavioral recovery analysis provides finer-grained evidence that the models learn behavior patterns introduced by specialized harnesses.

Critical weaknesses
  1. The ablation study in Section 4.4 does not fully isolate the contribution of the step-by-step correction mechanism. The direct-distillation baselines either lack access to the same domain-specific guidance KK or generate trajectories under h⋆h^\star, which includes additional tools and therefore introduces an action-space mismatch. Consequently, it remains unclear whether the gains primarily arise from domain-specific guidance and action-space alignment, or from the proposed mechanism of reviewing and minimally correcting responses along the student’s trajectory.

  2. The models are trained separately for each domain, leaving unverified whether behaviors from multiple specialized harnesses can be jointly internalized in a single model. The current results support within-domain harness distillation, but deployment in the general-purpose agent setting emphasized in the introduction may still require selecting among domain-specific models or adapters. This does not invalidate the domain-specific results, but it limits the evidence supporting the paper’s motivation of reducing maintenance and routing burdens in multi-domain systems.

  3. In the training-free setting, the performance gains of agent-as-harness come with additional review calls and inference overhead. The paper reports approximately 2.4×2.4\times the average latency of the basic harness on USPTO, so its performance advantage should be interpreted alongside its computational cost. Although the teacher review mechanism can be removed after distillation, the paper provides limited discussion of the upfront costs of harness evolution, trajectory collection, and fine-tuning, and how these costs trade off against deployment benefits.

Questions for authors
  1. Could a strong teacher with access to the same KK, but restricted to the target harness hh, generate complete trajectories that yield comparable distillation gains? Could the authors provide an additional comparison or trajectory analysis to clarify the distinct value of step-by-step correction along the student’s trajectory?

  2. Is training separately for each domain primarily a simplification of the experimental setup, or does it reflect potential interference between domain-specific behavior patterns? Have the authors explored training a single model on multi-domain trajectories, or could they discuss the applicability and potential limitations of the method for integrating capabilities across domains?

  3. Could the authors further clarify the cost components at training and deployment time and discuss which application scenarios would justify the upfront investment in harness optimization and distillation? In particular, compared with retaining specialized harnesses at deployment, what deployment scale or frequency of repeated use might make distillation more practically advantageous?

5 comments
AuthorsOpenReview ↗

Response to W1 and Q1

Thank you for your review and for recognizing our problem formulation, coherent distillation pipeline, systematic ablations, and behavioral recovery analysis. Below, we address each of your concerns.

W1: Table 3 does not fully isolate step-by-step correction: the direct-distillation baselines either lack the same KK or run under h⋆h^\star (action-space mismatch). The gains may come from domain guidance and action-space alignment rather than wrapping the student's trajectory.

Q1: Could a strong teacher with the same KK, restricted to hh, generate complete trajectories with comparable distillation gains? What is the distinct value of step-by-step correction along the student's trajectory?

Both domain-specific guidance (K\mathcal{K}) and step-by-step review along the student's trajectory are essential; letting a teacher with access to K\mathcal{K} directly generate standalone trajectories under hh is fundamentally ill-posed for distillation because it leaks privileged K\mathcal{K} references into native reasoning tokens (inducing hallucination) and produces off-policy trajectories. So even if it could overfit to a narrow domain, it cannot train a reliable general-purpose model. Below, we explain these two fundamental limitations in detail.

First, a standalone teacher rollout with K\mathcal{K} under hh leaks privileged K\mathcal{K} references into its native reasoning tokens, training the student to hallucinate. When a teacher directly executes a task with access to K\mathcal{K}, the <think> blocks recorded in the trajectory are its native reasoning tokens, which naturally cite the K\mathcal{K} rules, skills, and files it is consulting. Even if K\mathcal{K} is stripped from the prompt before SFT, the trajectory's reasoning and actions remain conditioned on invisible context absent from ctc_t, teaching the student to hallucinate ungrounded premises and nonexistent files (Choi et al., 2026). By contrast, in Harness-Zero, the teacher's native reasoning over K\mathcal{K} stays inside the reviewing agent and never enters the student trajectory; the replacement <think> and bash command are produced as controlled output text (via REPLACE), allowing the teacher to synthesize a self-contained, first-person trace grounded strictly in ctc_t.

Second, even when action-space aligned, standalone teacher trajectories are off-policy and rarely teach the student how to recover from its own intermediate errors. Because a strong teacher solves tasks along its own trajectory, it rarely enters the intermediate failure states that a student encounters at test time, leaving the student with no supervision on how to diagnose or recover once its own rollout makes a mistake (Li & Goyal, 2026). In practice, we find that a student trained on standalone teacher rollouts frequently gets stuck in repetitive error loops (repeating an identical command ≥5\ge 5 times in 13.0%–21.0% of trajectories), whereas in Harness-Zero the student drives the trajectory under hh, reaches its natural failure states, and learns error recovery from minimal corrections at those exact states:

Distillation Source (evaluated under hh)SpreadsheetBench Acc.SpreadsheetBench Repeated Command (≥5×\ge 5\times)AppWorld (SGC)AppWorld Repeated Command (≥5×\ge 5\times)
Standalone GPT-5.6 Sol rollouts20.0%13.0%32.1%21.0%
Harness-Zero (Ours)44.0%0.0%58.9%1.2%

References

  • Aochong Li and Tanya Goyal. Off-Trajectory Reasoning: Can LLMs Collaborate on Reasoning Trajectories? ICLR, 2026.
  • Moonseok Choi, Taehong Moon, Giung Nam, and Juho Lee. Harness-Aware Distillation for Small Language Model Agents. arXiv preprint arXiv:2610.02858, 2026.
AuthorsOpenReview ↗

Response to W2 and Q2

W2: Models are trained per domain, so it is unverified whether multiple specialized harnesses can be jointly internalized. A general-purpose agent may still need to route among domain-specific models or adapters.

Q2: Is per-domain training just an experimental simplification, or does it reflect interference between domain behaviors? Have you trained a single model on multi-domain trajectories?

Great question! We have added an experiment on multi-domain joint training. A single LoRA adapter trained on all three domains improves substantially over the base model across every benchmark without any domain routing. Specifically, we naively trained one LoRA adapter on the union of all three training sets (487 SpreadsheetBench + 282 AppWorld + 500 USPTO = 1,269 trajectories) with the same SFT recipe and evaluated it under the minimal harness hh:

Setting (evaluated under minimal harness hh)SpreadsheetBenchAppWorld (SGC)USPTOAvg.
Base model (untrained, under hh)31.026.812.023.3
Harness-Zero (jointly trained, single LoRA)47.039.321.035.8
Harness-Zero (per-domain specialists)44.058.930.044.3

The single jointly trained model improves over the base model on every domain and even surpasses the specialist on SpreadsheetBench, confirming that behaviors from distinct specialized harnesses can be consolidated into one set of model weights without inference-time routing. The remaining gap to per-domain specialists on AppWorld and USPTO reflects standard multi-task mixture weighting under a fixed rank-32 LoRA budget (e.g., AppWorld accounts for only 22% of the 1,269 trajectories). We think closing that gap requires scaling data collection and better balancing the domain mixture, as in standard general-purpose post-training.

AuthorsOpenReview ↗

Response to W3 and Q3

W3: Training-free agent-as-harness is ~2.4× slower on USPTO. After distillation the teacher can be removed, but the paper says little about the upfront cost of evolution, collection, and SFT versus the deployment benefit.

Q3: Please break down training vs.\ deployment costs, and say at what scale distillation is more advantageous than keeping specialized harnesses.

Thank you for this constructive suggestion. We show that distilling h⋆h^\star into hh cuts per-task inference token usage by 27%–87%, improves macro-average accuracy, and eliminates runtime harness maintenance; whenever a strong harness is worth deploying, these permanent deployment benefits easily justify the one-time upfront training cost.

1. Harness-Zero saves substantial inference-time cost. Retaining a specialized harness h⋆h^\star at deployment forces every query to repeatedly pay for long skill/memory prompts, custom tool schemas, and middleware guard retries across turns. Internalizing h⋆h^\star into the model under the minimal harness hh removes both the teacher agent and all h⋆h^\star runtime overhead. Across our three evaluated test sets, Harness-Zero substantially reduces average per-task input and output token consumption compared to running the base model with h⋆h^\star attached:

BenchmarkAvg. Input Tokens / Task (Base + h* →\to Ours under h)Input ReductionAvg. Output Tokens / Task (Base + h* →\to Ours under h)Output ReductionTest Performance (Base + h* →\to Ours under h)
SpreadsheetBench (100 tasks)515,964 →\to 222,143-56.9%17,412 →\to 9,804-43.7%39.0% →\to 44.0%
AppWorld (168 tasks)339,755 →\to 249,514-26.6%4,981 →\to 3,545-28.8%48.2% →\to 58.9%
USPTO (100 tasks)118,099 →\to 15,364-87.0%9,548 →\to 2,310-75.8%38.0% →\to 30.0%

Beyond reducing inference tokens and improving average accuracy (44.3% vs.\ 41.7%), distillation eliminates the engineering burden of shipping, maintaining, and routing among domain-specific tools and middleware guards at deployment.

2. Harness-Zero takes relatively modest, one-time upfront training cost. Deploying a specialized harness h⋆h^\star already requires the cost of harness evolution. The marginal upfront cost of Harness-Zero is small in our setting:

  • Adaptation (K=Adapt⁡(h⋆;h)\mathcal{K} = \operatorname{Adapt}(h^\star; h)): A single automated agent run translating h⋆h^\star into review guidance K\mathcal{K}.
  • Trajectory collection: A one-time pass of 441–700 training trials per domain (where the ~2.4$\times$ teacher review overhead is incurred only once offline).
  • Supervised fine-tuning: A lightweight 2-epoch LoRA SFT run over 282–500 retained trajectories (~1–2 GPU hours per domain).

In short, once a valuable harness h⋆h^\star has been developed, distilling it requires only a few hundred offline review rollouts and ~1–2 GPU hours of LoRA SFT, while permanently unlocking 27%–87% lower inference token costs, higher task accuracy, and a zero-maintenance minimal deployment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Area Chair and All Reviewers

We sincerely thank the Area Chair and all reviewers for their time, effort, and constructive feedback. We are encouraged by your recognition of our work's precise problem formulation and motivation (sZMT, NGqb), the novelty of harness adaptation (B4r4) and coherent, reusable agent-as-harness mechanism (sZMT, NGqb), our large gains under a minimal harness (sZMT, NGqb) with informative ablations (sZMT, NGqb, B4r4), behavioral recovery analysis (sZMT), and our clear presentation, examples, and figures (B4r4, NGqb).

We would like to briefly recap our core contributions: our work pioneers agent harness distillation by (1) formalizing how to internalize specialized runtime harnesses h⋆h^\star into model weights under a minimal bash harness hh; (2) proposing the agent-as-harness, which adapts static harness machinery into private reference guidance to bridge the action-space and information gap via on-policy, step-level trajectory synthesis; and (3) achieving strong empirical gains and deployment savings—lifting Qwen3.5-9B under hh from 23.3% to 44.3% (surpassing 41.7% with h⋆h^\star attached and recovering 82.3% of h⋆h^\star-exclusive behaviors), while cutting per-task inference input/output tokens by 26.6%–87.0% / 28.8%–75.8% after removing h⋆h^\star.

Summary of New Experiments & Responses to Core Concerns

Below, we summarize core concerns from the reviewers and how we address each of them through new experiments and analyses (please see our individual responses for full details):

1. Further ablating the source of distillation gains (NGqb W1/Q1, sZMT W1/Q1, B4r4 W1). To further ablate the contributions of verifier filtering, student self-training, and bare teacher distillation across all domains, we ran matched control baselines on SpreadsheetBench and AppWorld (complementing USPTO in Table 3):

  • Verifier filtering is not the driver: Training on all 691 reviewed trajectories on SpreadsheetBench without reward = 1.0 filtering still reaches 43.0% (matching 44.0% filtered and surpassing h⋆h^\star's 39.0%).
  • Student self-training and bare teacher distillation fall short: Across the three domains, rejection-sampling fine-tuning (RFT) of the base student under hh (even with 2×2\times rollout budget) averages only 31.7%, standalone GPT-5.6 Sol rollouts under hh average 21.4%, and teacher review without K\mathcal{K} (empty-K) averages 29.1%—all trailing Harness-Zero (44.3%).
  • Necessity of both K\mathcal{K} and on-policy review: Standalone teacher rollouts with K\mathcal{K} leak privileged K\mathcal{K} references into native reasoning tokens (inducing hallucination) and are off-policy, whereas reviewing the student's own trajectory with K\mathcal{K} injects learnable, tool-grounded procedures and teaches on-policy error recovery.

2. Multi-domain joint training (sZMT W2/Q2, NGqb Q6, B4r4 W3). A single LoRA adapter trained across the three domains improves over the base model on every benchmark (31.0% →\to 47.0% on SpreadsheetBench, 26.8% →\to 39.3% on AppWorld, 12.0% →\to 21.0% on USPTO) without any domain routing.

3. Upfront training cost vs. deployment savings (sZMT W3/Q3, B4r4 W2). Beyond h⋆h^\star evolution, Harness-Zero requires only one-time offline collection and ~1–2 GPU hours of LoRA SFT, while permanently cutting deployment input/output tokens by 56.9%/43.7% (SpreadsheetBench), 26.6%/28.8% (AppWorld), and 87.0%/75.8% (USPTO) and eliminating runtime harness maintenance.

We have incorporated the new results and analyses into the revised paper. Thank you again for your valuable efforts, and we look forward to your feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.