- Rating
- 1(−1)
- Confidence
- 4(+1)
- Informativeness
- 1(−1)
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces SWE-OPD, applying on-policy distillation (OPD) to software engineering tasks. It compares different KL-based objectives and proposes both online and offline training variants. The experiments demonstrate consistent improvements across multiple SWE benchmarks with favorable computational efficiency. In addition, the paper analyzes several phenomena and failure modes observed in OPD, providing useful empirical insights into the behavior of OPD for agentic coding.
Critical strengths
The paper studies both online and offline variants of SWE-OPD, with relatively comprehensive experiments and ablations, including comparisons between the RKL and ST objectives.
Applying OPD to software engineering tasks is meaningful and provides a useful empirical baseline for future research on OPD in long-horizon agentic settings.
Critical weaknesses
Lack of baselines. The baseline comparisons in the paper are not comprehensive enough. It doesn't compare against strong existing methods, such as rejection-sampling SFT, RL, or other recent OPD variants (e.g., agentic OPD methods like EOPD/AdaOPD, or hybrid OPD-RL approaches).
Limited performance improvements. The overall gains are modest. For instance, some results only improve by 0.1 percentage points, which is not very convincing.
Questions for authors
Is OPD truly a necessary or promising training paradigm for agentic coding? Given that we already have access to a strong teacher model and an executable environment with verifiable test outcomes, wouldn't standard SFT on filtered/successful teacher trajectories (e.g., rejection-sampling SFT) be a much simpler and potentially more effective approach? Considering the high engineering complexity of OPD and the modest gains shown in the paper, I am not fully convinced that formulating agentic distillation as an OPD problem is actually necessary. Could the authors justify why OPD should be preferred over a well-curated SFT pipeline in practice?
Response to AC and Reviewers
Dear Reviewers and AC,
Thank you for your time and efforts in reviewing our paper.
We believe that some aspects of our work may not have been sufficiently clarified in the original submission, potentially leading to misunderstandings. Therefore, we provide further explanations below to better clarify our methodology and contributions, along with additional experimental results.
In this work, we present a simple and controlled study of On-Policy Distillation (OPD) for software engineering (SWE) agents. Specifically, a smaller student model (GLM-4.5-Air, 100B) interacts directly with executable code repositories, while a larger teacher model (GLM-5.2 / DeepSeek-V4-Flash, 800B / 300B) provides supervision over the sequence prefixes generated by the student throughout its interactions.
Importantly, our approach requires no reinforcement learning (RL), no task-success-based trajectory filtering, and no curated trajectories. We deliberately retain the original student-generated trajectories and apply vanilla OPD, keeping the training pipeline as simple and controlled as possible.
Additional Experiment: Industrial-Scale Cross-Tokenizer Capability Transfer
To further examine whether OPD can generalize beyond models within the same family, we extend our experiments to a cross-tokenizer setting, pairing GLM-4.5-Air as the student with DeepSeek-V4-Flash-0731 as the teacher.
Unlike conventional same-family distillation, the two models use different tokenizers, meaning their vocabularies and token boundaries do not match. We address this challenge using a naive GOLD-style span alignment strategy, merging tokens when their character spans differ across tokenizations. Notably, we introduce neither explicit cross-vocabulary mapping nor modifications to the original distillation objective.
Despite this minimal alignment strategy, offline ST alone achieves substantial improvements, yielding the largest gains in our study on SWE-Pro (+3.9) and SWE-Multilingual (+4.6).
Industrial-Scale Cross-Tokenizer Capability Transfer (GLM-4.5-Air ← DeepSeek-V4-Flash-0731)
Δ denotes the improvement over the GLM-4.5-Air baseline.
These results demonstrate that OPD is not restricted to same-family teachers or shared tokenizers: a stronger teacher from a different model family can effectively transfer coding capabilities to the student, even with a simple token-span alignment strategy in industrial-scale scenarios.
Three Key Highlights of Our Work.
To the best of our knowledge, our work is the first to successfully demonstrate industrial-scale large-to-small OPD for long-horizon agentic coding, with three key highlights:
Highlight 1: OPD for Industrial-Scale, Long-Horizon Agentic Coding.
We move beyond conventional distillation on short-form text and static datasets, bringing OPD to real-world SWE agent tasks involving multi-turn interactions, tool use, and long-horizon reasoning.
Highlight 2: Vanilla OPD — No RL, No Trajectory Filtering, No Additional Curation, Keep original chain-of-thought (CoT).
We preserve the student's original generation trajectories and chain-of-thought (CoT), without filtering trajectories based on task success, performing additional data curation, or introducing sophisticated training tricks. Our principle is simple: the student explores and generates; the teacher provides supervision.
Highlight 3: Industrial-Scale Models, Cross-Tokenizer Transfer, and Large-Scale Compute.
We conduct experiments across industrial-scale models, including DeepSeek-V4-Flash and GLM-5.2, and further demonstrate cross-family, cross-tokenizer capability transfer. Training and validation are powered by 32 NVIDIA B300 GPUs.
We sincerely hope that these additional clarifications and experimental results help address the reviewers' concerns and provide a more complete picture of our contributions. We respectfully ask the AC and reviewers to reconsider their evaluations in light of these points.