- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper identifies a gap in on-policy distillation (OPD) of language-model agents: agents are deployed inside a harness (software that manages context, tools, and feedback. The student therefore doesn't need to learn what the harness already provides; it needs the teacher's ability to act on what the harness reports. Standard OPD imitates the teacher's full outputs and treats the harness as part of the input, leaving the teacher's use of the harness implicit and failing to separate harness-provided information from teacher-added ability.
Critical strengths
The observation that distilling a harness-equipped agent is a problem between two systems sharing the same harness, so the student needs the teacher's ability to act on harness information, not the harness information itself is, to my knowledge, original and directly relevant to real deployments (where harnesses like OpenHands, SWE-agent, etc. are standard).
Strong gains on ALFWorld, cross-family transfer, and meaningful behavioral analysis. The ALFWorld gains are large (+16.4 SR over best baseline on unseen, +11.4 on seen) and the student surpasses the 8B teacher (63.4 vs 51.5).
Critical weaknesses
The core assumption that the harness-equipped teacher action is the better action is neither tested nor guaranteed, and the validity filter covers only a narrow class of failures. The paper states this as an assumption ("Assuming that harness information generally helps the teacher choose better actions") but never measures the rate at which it holds.
No variance reporting in any main or ablation table, and the smaller-benchmark gains may be noise. Tables all report point estimates with no standard deviations, no seed count, and no confidence intervals. The WebShop SR gain is +0.6 (19.0→19.6) and the ScienceWorld SR gain is +1.4 (8.05→9.40) are small enough that they could easily be within single-run noise for agentic benchmarks where episode-level variance is high.
Questions for authors
At states where the two teacher actions differ, what fraction of the time does the harness-equipped action lead to better task outcomes (e.g., higher eventual success or progress) than the stripped action?
How many seeds were used for the main results?
What fraction of preference pairs are filtered out by the validity checker (m_t = 0)? Among the unfiltered pairs where the two teacher actions differ, what fraction have the harness-equipped action actually correct (by some task-progress measure) vs. wrong-but-valid?
The students are 0.6B–1.7B. Have you tested whether HAD helps when the student is larger (e.g., 4B or 7B), where the student may already use harness information effectively without explicit contrast supervision? Does the gain shrink with student scale?
HAD is evaluated with one hand-designed harness. How sensitive are the gains to the harness design? If the harness were much richer (e.g., also suggesting next actions) or much sparser (only budget tracking), would the contrast mechanism still help?
5 comments
Official Response (2/2)
[Q4] (Larger scale experiments): Following the reviewer's suggestion, we trained HAD and OPD with a 4B student on ALFWorld, with the same recipe. HAD still helps a larger student.
- 4B student. Table R.8 uses the 8B teacher of the paper and a larger 32B teacher. The zero-shot 4B student is already close to the 8B teacher, and OPD from this teacher does not improve it. On seen tasks, HAD also stays near the zero-shot student. On unseen tasks, HAD is far above OPD and above the teacher with both teachers, so the gain does not vanish with a larger student.
- Stronger teacher. HAD also helps when OPD is already strong. With a Qwen3-4B teacher trained with RL (GiGPO) on the ALFWorld training tasks with our harness, OPD reaches a high success rate, and HAD still improves it (Table R.9).
Table R.8. HAD with a 4B student | ALFWorld, with the harness
| Unseen SR | Seen SR | |
|---|---|---|
| Teacher 8B (Table 2) | 51.5 | 50.0 |
| Teacher 32B | 61.2 | 56.4 |
| Student 4B (zero-shot) | 48.5 | 50.0 |
| OPD, 8B teacher | 47.8 | 47.1 |
| HAD, 8B teacher | 61.9 | 50.0 |
| OPD, 32B teacher | 55.2 | 48.6 |
| HAD, 32B teacher | 64.2 | 48.6 |
Table R.9. HAD with an RL-trained teacher | ALFWorld, 1.7B student (three seeds)
| Unseen SR | Seen SR | |
|---|---|---|
| Teacher RL-4B | 97.0 | 95.7 |
| OPD | 87.1±1.7 | 87.6±0.8 |
| HAD | 93.0±2.3 | 93.1±0.4 |
[Q5] (Sensitivity to the harness design): Following the reviewer's suggestion, we tested HAD with a richer and a sparser harness on ALFWorld. HAD helps in both cases.
- Richer harness. Table R.10 uses a harness designed by others (Life-Harness; Xu et al., 2026), which also gives procedural guidance for the task. We replaced our harness lines in the prompt with its text and changed nothing else. With this harness, OPD only reaches the level of the teacher, and HAD goes above the teacher.
- Sparser harness. Table R.11 removes parts of our harness step by step. State tracking only removes behavior monitoring (Table 1). Budget only keeps just the step budget of state tracking, as the reviewer suggested. Training and evaluation use the same reduced harness. HAD is above OPD in every row, by a wide margin on unseen tasks.
Table R.10. HAD with a richer external harness | ALFWorld, 1.7B student (three seeds)
| Unseen SR | Seen SR | |
|---|---|---|
| Student 1.7B (zero-shot) | 63.4 | 69.3 |
| Teacher 8B (zero-shot) | 77.6 | 80.0 |
| OPD | 76.9±2.0 | 80.5±2.2 |
| HAD | 84.8±0.4 | 85.2±2.3 |
Table R.11. HAD with a sparser harness | ALFWorld (unseen SR / seen SR)
| OPD | HAD | |
|---|---|---|
| Budget only | 32.1 / 27.9 | 43.3 / 42.9 |
| State tracking only (includes the budget) | 47.0 / 40.7 | 56.7 / 41.4 |
| Full harness (Table 2) | 47.0 / 40.0 | 63.4 / 51.4 |
Dear Reviewer RZ2a,
Thank you for your valuable comments and suggestions. We hope that our responses and the revised PDF have addressed your concerns, and we would be happy to answer any remaining questions. We would be grateful if you could take them into account in your final evaluation.
Thank you for your time and consideration.
Official Response (1/2)
[W1-1 & Q1] (The harness-contrast assumption): We thank the reviewer for this question. The harness-equipped action is the better one, both over whole episodes and at single states.
Table R.5. Eventual success after each teacher action | ALFWorld (SR)
Table R.6. Teacher with and without the harness | ALFWorld (%)
[W1-2 & Q3] (The validity check): The validity check is narrow by design, and the harness-equipped action is rarely wrong in the pairs that remain.
Table R.7. Fractions requested in Q3 | ALFWorld, 8 continuations per pair
[W2 & Q2] (Variance across random seeds):
Table R.2. Four random seeds (mean ± std) | with the harness