- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper propose HARNESS-ZERO, a method for making a small model behave as if it had read a very long framework manual. This method evolves an elaborate framework full of tools, middleware, skills, and memory; then Adapt converts all of that machinery into advice. Qwen3.5-9B attempts the task, GPT inspects every move, and whenever Qwen says “done” too early, GPT replaces it with a Bash script. The resulting trajectories are used to fine-tune Qwen until it can independently reproduce the framework’s rituals, without GPT, h*, or K at deployment time.
Critical strengths
-
Presentation is good, Figure 1 is clear
-
The idea of adapt is relatively novel; it can update the harness.
-
The result in table 3 is good.
Critical weaknesses
-
This is more likely to distill a big model in critical steps.
-
The cost seems high since the model needs to check the steps of the student model. (even higher than just using a big model to rollout.)
-
I'm a little bit concerned about the reward hacking / overfitting problem.
-
It seems hard to generalize to scenarios that need long thinking in a trajectory.
I think this idea has potential, but it still has some flaws.
Questions for authors
How to generate reasonable thinking via this method? (Is it natural? Will it distill something strange in? For example, some premise in skills)
10 comments
Response to W2 and W3
W2: The cost seems high since the model needs to check the steps of the student model.
Step-level review (checking) is a one-time offline data-collection investment that permanently cuts deployment token usage by 27%–87% while outperforming simpler collection alternatives. Below, we first show the substantial inference-time savings unlocked after this one-time collection pass, and then compare against two cheaper collection alternatives (direct strong-model distillation and student rejection sampling).
First, step-level review is paid only once during data collection and substantially reduces deployment inference costs. The harnessing agent checks student steps only during offline trajectory collection (441–700 training trials per domain); after a lightweight 2-epoch LoRA SFT run (~1–2 GPU hours), the harnessing agent, , and the external harness are all removed at deployment. Retaining at deployment forces every query to repeatedly pay for long skill/memory prompts, custom tool schemas, and middleware guard retries across turns, whereas our distilled student runs under the minimal bash harness , cutting average per-task input and output tokens by 26.6%–87.0% while improving average test accuracy:
| Benchmark | Avg. Input Tokens / Task (Base + h* → Ours under h) | Input Reduction | Avg. Output Tokens / Task (Base + h* → Ours under h) | Output Reduction | Test Performance (Base + h* → Ours under h) |
|---|---|---|---|---|---|
| SpreadsheetBench | 515,964 → 222,143 | -56.9% | 17,412 → 9,804 | -43.7% | 39.0% → 44.0% |
| AppWorld | 339,755 → 249,514 | -26.6% | 4,981 → 3,545 | -28.8% | 48.2% → 58.9% |
| USPTO | 118,099 → 15,364 | -87.0% | 9,548 → 2,310 | -75.8% | 38.0% → 30.0% |
Second, distilling the big model directly does not work. As shown in our response to W1 (Table 3), distilling standalone strong-model rollouts does not improve the student, so its per-rollout cost yields less training gain.
Third, another alternative—rejection sampling—wastes most of the rollout budget and still yields little improvement. On USPTO, we ran the base student under with doubled rollout budget (1,000 rollouts vs.\ 500 for Harness-Zero) and trained on the passing trajectories with the same SFT recipe. Only 8.0% of rollouts passed, and test accuracy reached only 14.0% (vs.\ 12.0% base and 30.0% for Harness-Zero). Without step-level review, 92% of the budget is discarded, and the remaining 8% only repeats behaviors the student already has. Paying for step-level review injects the missing harness patterns into the trajectory (which is beneficial whenever the student lacks those patterns) and is especially cost-effective when student successes are rare.
W3: I'm a little bit concerned about the reward hacking / overfitting problem.
Regarding reward hacking, neither the evolved harnesses () nor the teacher interventions introduce reward hacking. For the harnesses themselves, all components in and are human-readable files that we carefully reviewed to ensure they contain no task-specific answers or shortcuts, and recent work further demonstrates that harness optimization can yield generalizable gains across unseen tasks (Xia et al., 2026). During trajectory collection, teacher interventions do not introduce reward hacking either: the harnessing agent only has read-only access to and the student-visible history, with no access to test cases, reference answers, verifier feedback, or the student sandbox.
Regarding overfitting, distilling a domain harness brings general improvements within that domain, and we also add experiments on multi-domain joint training. Evaluating on held-out test splits shows that distilling a domain harness yields strong generalization to unseen tasks within that domain (Table 2), which is already valuable on its own. To further examine whether Harness-Zero can improve general agent capabilities across domains, we trained a single LoRA model by directly merging the training trajectories from all three domains (487 SpreadsheetBench, 282 AppWorld, and 500 USPTO). Under the minimal harness , this jointly trained model improves over the base model on every benchmark:
| Setting (under ) | SpreadsheetBench | AppWorld | USPTO | Avg. |
|---|---|---|---|---|
| Base model (untrained) | 31.0 | 26.8 | 12.0 | 23.3 |
| Harness-Zero (jointly trained, single LoRA) | 47.0 | 39.3 | 21.0 | 35.8 |
| Harness-Zero (per-domain specialists) | 44.0 | 58.9 | 30.0 | 44.3 |
This suggests that Harness-Zero can build general-purpose agent capabilities by distilling multiple domain harnesses into one model. Closing the remaining gap to single-domain specialists will benefit from scaling data collection and tuning the data mixture, as in standard general-purpose post-training.
Response to W4 and Q1
W4: It seems hard to generalize to scenarios that need long thinking in a trajectory.
First, scenarios requiring long thinking and long-horizon execution are a strength of our paradigm, as already reflected in our three evaluated benchmarks. Our evaluation covers both reasoning-intensive scientific derivation (USPTO retrosynthesis) and long-horizon multi-step interaction (SpreadsheetBench and AppWorld, where trajectories reach up to 36–48 turns and over 44k tokens). On such tasks, an unassisted small model often loses track of constraints or skips critical verification across turns, whereas the reference harness explicitly encodes the domain's multi-step workflow, state checks, and verification rules. Guided by , the harnessing agent translates these structured workflows into deeper step-level reasoning and verification. For example, the harnessing agent expands median reasoning length by +40.4% on AppWorld and replaces superficial commands with 's comprehensive verification scripts (2.3$\times$ and 5.9$\times$ longer in median command length on SpreadsheetBench and USPTO).
Second, recent follow-up and concurrent studies further confirm that this paradigm scales effectively to challenging long-horizon agent benchmarks and reasoning-intensive mathematical domains. Specifically, Recursive Self-Rewrite (Li et al., 2026) extends harness-guided trajectory rewriting to demanding, long-horizon terminal benchmarks—including Terminal-Bench 2/3/4, Terminal-Bench Hard, and Long-Horizon Terminal-Bench (where trajectories span dozens of turns and up to 180 minutes). Meanwhile, On-Policy Harness Self-Distillation (OPHSD; Zhao et al., 2026) applies temporary-harness distillation to reasoning-intensive competition mathematics (AIME 2024/2025, OlympiadBench, and HMMT 2025), improving pass@8 by +10.8% on HMMT 2025 after the scaffold is removed. Together with our results, these studies confirm that harness distillation generalizes well to both long-horizon execution and deep reasoning tasks.
Q1: How to generate reasonable thinking via this method? (Is it natural? Will it distill something strange in? For example, some premise in skills)
Unnatural or ungrounded thinking is rare and can be systematically audited and excluded. Our harnessing prompt (Appendix C.3) explicitly instructs the teacher to write every replacement's reasoning in the student's first-person voice, grounded strictly in student-visible observations without citing external skills or unverified premises. Across all 4,082 REPLACE steps in our training sets, programmatic checks found 0% leakage of private paths or component names and flagged only 8.2% of steps for reviewer-perspective phrasing. On a random sample of 100 remaining REPLACE steps, a combined human and LLM-judge audit found 0% third-person reviewer voice or ungrounded skill premises, with only 1.5% of the reasoning text briefly mentioning the student's draft command.
These occasional artifacts are filtered out before training via deterministic scripts and LLM-as-a-judge screening. As detailed in Section 3.3 and Appendix D, our pipeline automatically drops any trajectory containing private paths or review tool names, and masks the SFT loss on the 8.2% of REPLACE reasoning turns that slip into a reviewer's voice (while keeping their valid tool calls trainable). Combined with LLM-judge screening, this prevents unnatural phrasing from entering the training objective.
Recent advances in foundation models have made trajectory and reasoning rewriting a reliable, emerging post-training paradigm. As frontier models have become capable of rewriting reasoning traces while respecting strict perspective and information constraints, several concurrent works have successfully adopted similar data-shaping paradigms. For instance, Recursive Self-Rewrite (Li et al., 2026) uses a model-based planner–critic–executor loop to rewrite multi-harness trajectories into clean, leakage-free demonstrations under a general harness. Finetuning with Sampling (Karan et al., 2026) iteratively rewrites off-policy expert reasoning traces into on-policy trajectories tailored to the student's distribution, enabling standard SFT to match or exceed RL across chemistry and mathematical reasoning.
Response to W1
Thank you for your review and for recognizing our clear presentation, the novelty of harness adaptation, our ablation results, and the potential of this direction. Below, we address each of your concerns.
Distilling a stronger model alone is insufficient to improve the student. Below, we first present controlled ablations across all three benchmarks showing that distilling the teacher model without the reference harness K, whether from standalone teacher rollouts or via step-level interventions, falls far short of Harness-Zero. Then we discuss two reasons why K-guided, step-level correction is more effective.
Controlled ablations across all three benchmarks show that teacher intervention at critical steps without the reference harness K fails to match Harness-Zero. Across SpreadsheetBench, AppWorld, and USPTO, we keep the base student (Qwen3.5-9B), the strong teacher (GPT-5.6 Sol), the minimal target harness h, the number of retained training trajectories, and the SFT recipe fixed while varying the supervision source:
Letting the exact same teacher intervene at critical steps without the reference harness (
Review with empty K) reaches only 29.1% average accuracy (vs.\ 44.3% for Harness-Zero). Direct SFT on the teacher's standalone rollouts performs even worse (21.4% average). Moreover, on USPTO (Table 3), even giving the teacher the ground-truth answer during review (Review with oracle answer), the distilled student reaches only 15.0% (vs.\ 30.0% with K). Finally, behavioral analysis (Table 4) confirms that the student trained with K recovers 82.3% of 28 h⋆-exclusive behavior patterns that the base model never exhibits on its own.We discuss two reasons why K-guided, step-level correction is more effective.
First, structured harness procedures are much easier for a small model to internalize than a frontier teacher's unconstrained reasoning. Recent work on shared-trajectory collaboration shows that smaller models struggle to build upon or internalize guiding steps from a stronger collaborator when those steps rely on reasoning beyond the student's inherent capability (Li & Goyal, 2026). When GPT-5.6 Sol intervenes without K or with only the oracle answer, its edits rely on implicit model knowledge and ad-hoc shortcuts that the 9B student cannot generalize. By contrast, K translates the evolved harness into standardized, tool-grounded procedures (such as explicit inspection, candidate enumeration, and mechanical verification scripts) that focus supervision on learnable harness behaviors (Choi et al., 2026).
Second, minimal step-level rewrites keep the training trajectories on-policy and teach the student how to recover from its own intermediate errors. Because a strong teacher solves tasks along its own trajectory, standalone teacher rollouts rarely enter the intermediate failure states that a 9B student encounters at test time, leaving the student with no supervision on how to backtrack or recover once its own rollout makes a mistake (Li & Goyal, 2026). In practice, we find that a student trained on standalone teacher rollouts frequently gets stuck in repetitive error loops, whereas in Harness-Zero the student drives the trajectory under h, enters its natural failure states, and learns error recovery from minimal corrections at those exact states: