- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Harness Policy Distillation (HPD), a framework designed to align LLM agent policies with the runtime semantics of their execution harnesses. HPD replays reference trajectories to extract runtime control states and constructs control documents that guide a teacher model to generate harness-verified actions. It then jointly trains a shared model through three supervision views: native-context action prediction, control-document-conditioned action prediction, and runtime-state reconstruction. Experiments on TRAJECT-Bench demonstrate that HPD significantly improves task success and runtime compliance over existing SFT, RL, and distillation baselines. The results also show promising generalization across tasks, model architectures, and agent harnesses.
Critical strengths
Strengths
-
Novel Perspective: The paper introduces an interesting perspective by explicitly incorporating harness runtime semantics into agent policy training, rather than relying solely on action-level supervision.
-
Effective Method: The proposed three-view joint training framework is well-designed. The ablation studies demonstrate the effectiveness of combining action supervision with runtime-state supervision.
-
Comprehensive Experiments: The paper provides extensive comparisons with SFT, RL, and distillation baselines. It also evaluates generalization across different benchmarks, model architectures, and agent harnesses, demonstrating the effectiveness and robustness of the proposed method.
Critical weaknesses
I do not have many concerns about this work.
The only weakness is that the author does not demonstrate their method by using more backbone models
Questions for authors
None
2 comments
Many thanks.
It addresses my concerns.
Thank you for the positive assessment. Regarding backbone coverage, the submission evaluates HPD on three mainstream model families and nine backbones in total, each trained and evaluated under the same procedure and compared with its own Base model:
HPD improves task success and final-answer F1 for every backbone, including dense and mixture-of-experts architectures and model sizes from 0.6B to 32B parameters (Section 4.6; Appendix D.4).