- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Localized Context Distillation, or LoCD, to address limitations of on-policy and off-policy distillation. LoCD identifies critical spans in student trajectories and regenerates only those spans using privileged teacher context. The method is evaluated on mathematical reasoning, coding, and agent tasks, with improvements over several distillation baselines.
Critical strengths
-
The method is simple and well motivated. Localizing teacher intervention to critical spans is a reasonable way to combine on-policy student trajectories with off-policy correction.
-
The reported improvements are consistent across mathematical reasoning, coding, and agent tasks. Some gains are substantial, particularly on LiveCodeBench and ALFWorld.
-
The initial-accuracy analysis provides a useful explanation for why on-policy distillation struggles when the student rarely generates the desired behavior.
Critical weaknesses
-
The main comparisons lack sufficient training details, uncertainty reporting, and compute matching. It is difficult to determine whether the reported gains would hold under comparable budgets and independent runs.
-
Important baselines are missing from the main evaluation, particularly reference-solution SFT and a simple mixture of on-policy and off-policy distillation (e.g., GKD; Agarwal et al., 2024). Figure 10 includes SFT results but does not explain them adequately.
-
The paper does not sufficiently isolate the contribution of localization. LoCD is absent from the diagnostic experiments in Figures 1–2, and localization accuracy, exposure bias, and forgetting are not directly evaluated.
-
Some results are difficult to reconcile. Table 1 is inconsistent about the model checkpoint, Table 2 contains identical entries for two baselines, and the WebShop results in Figure 6 appear different from Table 3.
Questions for authors
-
Can the authors compare LoCD against reference-solution SFT and an on-policy/off-policy mixture under matched compute budgets, and report results across independent seeds? This would help establish whether the improvements are robust.
-
Can the authors evaluate LoCD in the initial-accuracy experiments of Figures 1–2, including held-out problems, and provide ablations that isolate localization from other changes in supervision? This would directly test the paper's central motivation.
-
Can the authors provide the full localization procedure and clarify the model checkpoint, duplicated baseline results in Table 2, and the discrepancy between Figure 6 and Table 3? These details are important for interpreting and reproducing the results.
8 comments
Rebuttal to Reviewer BjXK [2/5]
[W2 & Q1] Important baselines are missing from the main evaluation, particularly reference-solution SFT and a simple mixture of on-policy and off-policy distillation (e.g., GKD; Agarwal et al., 2024). Figure 10 includes SFT results but does not explain them adequately.
Can the authors compare LoCD against reference-solution SFT and an on-policy/off-policy mixture under matched compute budgets, and report results across independent seeds? This would help establish whether the improvements are robust.
[Response] We thank the reviewer for the helpful suggestion. Following it, we add two baselines on Qwen3-8B for math, code, and WebShop:
- Reference-solution SFT [1]: supervised fine-tuning directly on reference solutions. Since WebShop has no reference solutions, we use trajectories generated by Claude Opus 5.5.
- GKD [2] (on/off-policy mixture): we follow the original setting and official implementation of GKD, which mixes student-generated and teacher-generated trajectories, both supervised by the same context-conditioned teacher.
Both baselines share the same data, feedback sources, batch size, and optimization steps as LoCD, with matched training tokens.
Table R2: Comparison with reference-solution SFT and GKD on Qwen3-8B (%).
| Method | AIME24 | AIME25 | LiveCodeBench | WebShop |
|---|---|---|---|---|
| Reference-solution SFT | 52.78 | 36.94 | 18.71 | 45.80 |
| GKD | 45.83 | 37.50 | 18.05 | 26.60 |
| LoCD | 61.30 | 47.87 | 24.14 | 52.40 |
LoCD consistently outperforms both baselines by large margins, improving over reference-solution SFT by 5.43–10.93 points and over GKD by 6.09–25.80 points. Reference-solution SFT trains the student entirely on text it would not generate itself; GKD still trains on whole teacher-generated trajectories in its off-policy batches, while its on-policy batches rarely contain the behaviors the student is missing. In contrast, LoCD keeps the student's own trajectory and introduces teacher supervision only on short spans that need revision, which provides the missing behaviors while keeping training close to the student's own distribution.
Regarding Figure 10, we apologize for the confusion. "SFT" in this figure denotes TRD [3], the off-policy context-distillation baseline in our main experiments, rather than reference-solution SFT. In the revised manuscript (Figure 14), we relabel it as "Off-policy" and explain in the caption that Off-policy and On-policy denote TRD and OPCD, respectively.
[1] Ouyang, Long, et al. "Training Language Models to Follow Instructions with Human Feedback." NeurIPS, 2022.
[2] Agarwal, Rishabh, et al. "On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes." ICLR, 2024.
[3] Jiang, Li, et al. "Trajectory-Refined Distillation." arXiv:2606.08432, 2026.
Rebuttal to Reviewer BjXK [3/5]
[W3 & Q2] The paper does not sufficiently isolate the contribution of localization. LoCD is absent from the diagnostic experiments in Figures 1–2, and localization accuracy, exposure bias, and forgetting are not directly evaluated.
Can the authors evaluate LoCD in the initial-accuracy experiments of Figures 1–2, including held-out problems, and provide ablations that isolate localization from other changes in supervision? This would directly test the paper's central motivation.
[Response] We thank the reviewer for the suggestions on directly testing our central motivation. In summary:
- LoCD in Figures 1–2: LoCD substantially outperforms on-policy context distillation on low- and zero-initial-accuracy problems, including held-out problems.
- Ablations: each of LoCD's three components contributes to its performance, and the best performance requires all three.
- Localization accuracy, exposure bias, and forgetting: we evaluate each of them with controlled variants, span statistics, and parameter-update analysis.
The details are as follows.
(1) LoCD overcomes the low-initial-accuracy failure of on-policy context distillation. We add LoCD to Figures 1–2 of the revised manuscript, under the same settings as the original diagnostic experiments.
Low-initial-accuracy problems (Figure 1). Problems are binned by each model's own initial avg@128. Table R3 aggregates the bins with [0, 40]% initial accuracy; full per-bin results are shown in Figure 1 of the revised manuscript.
Table R3: avg@128 (%) on problems with initial accuracy in [0, 40]%.
| Model | On-policy (OPCD [4]) | Off-policy (TRD [3]) | LoCD |
|---|---|---|---|
| Qwen3-4B | 21.08 | 38.29 | 43.94 |
| Qwen3-8B | 29.09 | 62.65 | 67.13 |
| Falcon3-10B-Instruct | 13.55 | 33.33 | 34.24 |
Zero-initial-accuracy problems (Figure 2). We collect problems on which the initial student fails in all 128 rollouts, and split them into training and held-out problems. All methods are trained on the training problems, and Table R4 reports the results on the held-out problems, which are never seen during training.
Table R4: avg@128 (%) on held-out problems with zero initial accuracy, after 100 training steps (Figure 2 in the revised manuscript).
| Model | On-policy | Off-policy | LoCD |
|---|---|---|---|
| Qwen3-4B | 2.47 | 28.12 | 32.44 |
| Qwen3-8B | 3.00 | 43.23 | 56.23 |
On these problems, On-policy barely improves, whereas LoCD learns to solve them, and the gains carry over to problems never seen in training. Moreover, when OlympiadBench from Table 1 is stratified by initial accuracy, LoCD outperforms On-policy in every bin up to 60% initial accuracy, by up to 19.06 points, and achieves the best overall avg@128 (66.01 vs. 62.08 for Off-policy and 61.75 for On-policy).
[3] Jiang, Li, et al. "Trajectory-Refined Distillation." arXiv:2606.08432, 2026.
[4] Ye, Tianzhu, et al. "On-Policy Context Distillation for Language Models." arXiv:2602.12275, 2026.
Rebuttal to Reviewer BjXK [1/5]
We appreciate the reviewer's careful reading and constructive suggestions. We group related Weaknesses and Questions and respond to each below. The revisions made in response to these comments are highlighted in blue in the revised manuscript.
[Response] We thank the reviewer for raising these points on experimental rigor. In summary:
The details are as follows.
(1) The revised manuscript adds a complete appendix of experimental details (Appendix C). It covers datasets and splits (C.1), feedback construction for each task (C.2), training configurations (C.3 and Table 4), span statistics (C.3 and Table 5), and evaluation and checkpoint-selection protocols (C.4 and Table 6). All methods use full-parameter fine-tuning with AdamW and a constant learning rate of 1e-6, with task-specific batch sizes and numbers of training steps listed in Table 4.
(2) Results are reported over three independent runs, and the main gains are statistically significant. Math and code results come from three independent training runs, each selecting its own checkpoint, and are reported as mean ± std (Tables 1–2). LoCD achieves the best result on all 20 math and code benchmark columns. The improvements over the strongest baseline are statistically significant (one-sided Welch's t-test, p < 0.05):
Table R1: Significance of LoCD's gains over the strongest baseline on Qwen3-8B (three runs each).
For agent tasks, LoCD achieves the best results on all four tasks (WebShop, ALFWorld, SWE-Gym, and SWE-bench Verified) and stays above the baselines throughout training (Figure 6). Since agent training requires extensive environment interaction, we will include multiple independent runs in a future version.
(3) All methods are compared under the same budget. To ensure a fair comparison, LoCD and all baselines, including the reference-solution SFT and GKD baselines added in this rebuttal, share the same model, task data, feedback sources, batch size, and number of optimization steps within each task (Section 5.1 and Appendix C.3 in the revised manuscript). The newly added baselines are further matched with LoCD in training tokens.