acceptodds
Under review as a conference paper at ICLR 2027

iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs

Abstract

On-policy self-distillation fine-tuning (SDFT) learns new skills from demonstrations while reducing forgetting, but it always distils toward the full demonstration-conditioned teacher. This fixes teacher influence at the full-teacher endpoint, providing no control over how much demonstration information should be transferred at each prediction state. We introduce Information-Proximal SDFT (iSDFT), which instead treats the teacher as a budgeted source of information. At each token, iSDFT selects the distribution closest to the current student that satisfies a prescribed teacher-information constraint, yielding a closed-form exponential target with a locally determined tilt. To control cumulative drift, we further anchor the student to its frozen base policy. Across four heterogeneous LLM backbones and two specialisation tasks, iSDFT improves vanilla SDFT in 7 of 8 model–task settings and matches it in the remaining one. It also provides tighter retention on the original SDFT benchmark suite, with 73% of evaluations remaining within 0.5 points of the base model versus 52% for the strongest baseline, while achieving the largest mean improvement on all ten additional mathematics, coding, and competition-mathematics benchmarks. These results show that controlling how much and when teacher information is introduced improves specialisation while preserving broader capability.

Primary area:
generative models
Keywords:
self-distillation, on-policy distillation, continual learning, catastrophic forgetting, large language models, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_aSp9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

iSDFT moderates a demonstration-conditioned teacher through an information-constrained projection toward the current student, while separately anchoring the student to its frozen base model. Across four backbones and two specialisation tasks, it improves over SDFT in seven of eight settings, matches the remaining setting, and reports better aggregate capability retention.

Critical strengths
  1. The motivation distinguishes local teacher influence from cumulative drift. The projection has a clear mathematical interpretation, and separating its role from base anchoring provides a coherent approach to the stability–plasticity trade-off.
  2. The evaluation covers diverse backbones and capability benchmarks. Tables 11–12 provide useful component evidence: scheduled targets improve acquisition with or without anchoring, while anchoring generally improves retention. The paper also acknowledges limitations in sequential forgetting and reasoning evaluation.
Critical weaknesses
  1. Attribution to the information constraint. The ablations establish benefits from the scheduled target and anchor, but do not isolate tokenwise information control from simpler teacher warmup. Fixed information budgets still use the same adaptive projection, and Table 12 omits retention results for SDFT+anchor. Thus, the necessity of the local constraint beyond scheduling and anchoring remains uncertain.
  2. Interpretation of reasoning gains. Performance under a common 4096-token budget has practical value. However, Appendix D.6 documents substantial truncation, while the 32768-token follow-up reports lengths without correctness or an SDFT comparator. Improvements therefore do not yet distinguish retained reasoning ability from changes in response length or answer presentation.
  3. Reliability of modest differences. Table 2 specifies one training seed. Several acquisition and ablation improvements are small, and sign tests across related benchmark evaluations do not establish stability across training runs. The cross-model consistency is encouraging, but uncertainty in these differences remains insufficiently quantified.
Questions for authors
  1. In one representative setting, how does iSDFT compare with a simple time-scheduled geometric target under the same teacher, loss, and anchor? Reporting retention for the existing SDFT+anchor configuration would further clarify the projection's contribution.
  2. Could one representative mathematical evaluation compare Base/SDFT/iSDFT correctness at a longer budget with verified answer extraction? Otherwise, please consistently delimit the claims to performance under the evaluated generation budget.
  3. Could a small paired repeat of one representative SDFT/iSDFT setting quantify training variability? Existing repetitions would suffice; repeating the entire evaluation matrix is unnecessary.
3 comments

We thank the reviewer for the constructive review. Each question asked for a specific experiment, and we have run all; the results are reported below.

Attribution to the information constraint. The ablations establish benefits from the scheduled target and anchor, but do not isolate tokenwise information control from simpler teacher warmup. ... In one representative setting, how does iSDFT compare with a simple time-scheduled geometric target under the same teacher, loss, and anchor?

We added this comparison on Qwen3-1.7B. It keeps iSDFT as is, with the same teacher, loss, anchor and hyperparameters, but uses one coefficient for all tokens instead of the per-token λtλ_t: fixed λλ = 0.5, fixed λλ = 0.75, a linear warm-up λλ: 0.25 →\rightarrow 1 over the first 150 steps (the same timing as our budget schedule), and a pooled budget. The pooled budget keeps iSDFT's ρρ schedule and solves one λλ per response, shared by all its tokens, so that each response receives the same total teacher information as under iSDFT. Only the distribution across tokens differs.

Selection ruleTool UseScience
Shared λλ = 0.551.552.4
Shared λλ = 0.7550.552.0
Shared λλ, linear 0.25 →\rightarrow 148.453.0
λλ = 1 (SDFT + anchor)50.554.2
Pooled budget (one λλ per response, total matched to iSDFT)52.554.8
Per-token budget (iSDFT)52.556.8

On Science, no single coefficient matches iSDFT. The pooled budget isolates tokenwise information control: it has the same budget schedule and the same total teacher information as iSDFT, and differs only in sharing λλ across the tokens of a response. It is the strongest single-coefficient target, yet it trails iSDFT by 2.0 points on Science. The linear warm-up, the simple time-scheduled target, trails by 3.8 points, and the fixed coefficients by more. Each row is a single run, and Tool Use differences are at most four of 97 questions.

Fixed information budgets still use the same adaptive projection, and Table 12 omits retention results for SDFT+anchor. Thus, the necessity of the local constraint beyond scheduling and anchoring remains uncertain. ..... Reporting retention for the existing SDFT+anchor configuration would further clarify the projection's contribution.

We evaluated SDFT + anchor on the same 32 retention evaluations as Table 12 (16 benchmarks after each of the two tasks; retention table below):

  • With the full-teacher target (SDFT), the anchor helps slightly less. SDFT + anchor beats SDFT on 14 evaluations and trails on 11 (mean +0.5). Adding the anchor to the information-proximal target (iSDFT) improves 26 (mean +2.1).
  • With the anchor held fixed, full iSDFT is ahead of SDFT + anchor on 18 evaluations, tied on 4 and behind on 10 (mean +0.1), while improving both target tasks (Tool Use 50.5 →\rightarrow 52.5, Science 54.2 →\rightarrow 56.8).
  • Without the anchor, the information-proximal target retains less than SDFT (better on 10, worse on 20).

So the projection's contribution is better acquisition at equal retention, not better retention on its own, while the anchor helps with retention.

Retention table: Qwen3-1.7B with the anchor, under the full-teacher target (SDFT + anchor) and the information-proximal target (iSDFT); bold marks the better configuration for each task (Tool Use or Science).

BenchmarkBaseTool Use (SDFT + anchor)Tool Use (iSDFT)Science (SDFT + anchor)Science (iSDFT)
HellaSwag42.843.042.343.143.2
MMLU55.555.755.656.556.5
TruthfulQA33.734.034.233.934.0
WinoGrande55.557.157.356.957.4
HumanEval67.064.264.667.767.4
IFEval49.449.150.850.750.3
GSM8K66.649.949.967.964.3
MATH-50021.660.160.665.166.2
HumanEval+65.965.663.464.964.6
MBPP+49.748.048.449.849.5
LiveCodeBench9.47.88.38.29.2
MultiPL-E24.031.232.331.932.6
AIME 202416.620.016.630.026.6
AIME 202516.616.616.623.326.6
AMC 202355.057.560.057.560.0
BeyondAIME5.06.06.09.010.0

Interpretation of reasoning gains. Performance under a common 4096-token budget has practical value. However, Appendix D.6 documents substantial truncation, while the 32768-token follow-up reports lengths without correctness or an SDFT comparator. .... Could one representative mathematical evaluation compare Base/SDFT/iSDFT correctness at a longer budget with verified answer extraction? Otherwise, please consistently delimit the claims to performance under the evaluated generation budget.

We ran this evaluation on MATH-500, which showed the largest improvement among the additional mathematics and coding benchmarks: Qwen3-1.7B before and after Tool Use specialisation with SDFT and iSDFT, at 32,768 tokens instead of 4,096.

Model4,096 tokens32,768 tokens
Base21.627.2
SDFT59.059.8
iSDFT60.661.2

With the increased budget, the base model achieves 27.2, 5.6 points higher than under the original budget, while SDFT achieves 59.8 and iSDFT achieves 61.2, outperforming the base model by 34.0 points and SDFT by 1.4. These results show that the improvements on MATH500 persist under a larger generation budget, suggesting that the gains are not just due to response truncation.

Reliability of modest differences. Table 2 specifies one training seed. Several acquisition and ablation improvements are small, and sign tests across related benchmark evaluations do not establish stability across training runs.

Could a small paired repeat of one representative SDFT/iSDFT setting quantify training variability? Existing repetitions would suffice; repeating the entire evaluation matrix is unnecessary.

We ran additional experiments across two backbone models, Qwen3-1.7B and OLMo3-7B-Think, and two tasks (Tool Use and Science) using three seeds 0, 1 and 42 under identical configurations for SDFT and iSDFT. The two methods share each seed, so the comparison is paired. iSDFT achieves higher mean accuracy in all four model–task settings. Across all configurations, the standard deviations are at most 1.3 percentage points. These results provide additional evidence of robustness to training stochasticity and show that the improvements of iSDFT are consistent across independent training runs.

Mean ± standard deviation over the three seeds:

BackboneTaskSDFTiSDFT
Qwen3-1.7BTool Use52.6 ± 1.352.9 ± 1.1
Qwen3-1.7BScience54.4 ± 0.856.5 ± 0.7
OLMo3-7B-ThinkTool Use51.5 ± 1.254.0 ± 0.9
OLMo3-7B-ThinkScience14.4 ± 1.017.9 ± 0.6

The new experiments isolate the contribution of each component. With the same schedule and the same total teacher information, sharing one coefficient across tokens loses most of iSDFT's Science gain over SDFT + anchor, so most of that gain comes from the per-token information constraint rather than the warm-up.

And with the anchor held fixed, the projection adds this acquisition without a net loss in retention, and the anchor provides the retention. The MATH-500 gains persist at 32,768 tokens, and the seed repeats show that the comparison with SDFT is robust to training stochasticity. We hope these results address the reviewer's questions.

Thank you for the additional experiments. The matched controls, longer-budget evaluation, and seed repeats substantially address my concerns. The retention results also clarify that the projection improves acquisition, while anchoring supports retention. I will maintain my rating of 3 in support of acceptance.

Reviewer_qj1jOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses an important problem in large language model (LLM) adaptation: how a model can learn a new skill without degrading capabilities it already has.

The authors propose Information-Proximal Self-Distillation Fine-Tuning (iSDFT), a modification of on-policy self-distillation fine-tuning (SDFT). The central idea is that a model should not necessarily imitate a demonstration-conditioned teacher completely. Instead, it should transfer a controlled amount of information from the teacher, while limiting how far its behaviour moves from the original model.

Critical strengths
  1. The paper presents a clear and well motivated research problem. The problem is important, the motivation is clear, and the method is designed around a specific limitation of an existing approach.

  2. The authors formulate a constrained optimisation problem and derive a closed-form solution using a KL I-projection, providing a solid mathematical foundation.

  3. The experiments include four different backbones, two specialisation tasks and a broad set of competing approaches. The comparison with vanilla SDFT is particularly relevant because it isolates the proposed changes against the method being extended.

Critical weaknesses
  1. The novelty relative to existing adaptive geometric distillation methods requires further clarification. The proposed target belongs to an established family of geometric student–teacher distributions, and the main distinction is the criterion used to select the interpolation coefficient. The paper should establish more convincingly that information-constrained target selection provides a substantive advantage over carefully tuned alternatives, rather than primarily offering a different parameterisation of adaptive interpolation.

  2. The evidence supporting the continual-learning claims is limited. Most experiments involve independent specialisation on one of two tasks. The sequential experiment considers only one model and a two-task sequence, and the reported forgetting after the task switch is similar for iSDFT and SDFT. The results support improved final combined task performance, but provide more limited evidence that the method substantially reduces forgetting during sequential adaptation.

  3. The reported improvements on mathematics and coding benchmarks are promising, but their robustness requires further investigation. The paper notes that the generation budget truncates many responses from reasoning-oriented models. Also, evaluation under suitable generation budgets would help establish whether the gains reflect robust improvements in reasoning capability.

  4. The information budget and frozen-base anchoring strength are manually specified hyperparameters. Although the paper includes useful ablations, a more systematic sensitivity analysis would clarify how robust the results are to these choices across models and tasks.

  5. The evaluation would benefit from clearer evidence of variability across independent training runs for the main comparisons. Consistency across model–task–benchmark combinations is useful, but it does not fully capture variation arising from training stochasticity.

Questions for authors

Please address points raised in Weaknesses.

4 comments

We thank the reviewer for the detailed review, and have run further experiments to answer the questions.

The paper should establish more convincingly that information-constrained target selection provides a substantive advantage over carefully tuned alternatives, rather than primarily offering a different parameterisation of adaptive interpolation.

iSDFT is not a reparameterisation of adaptive interpolation. Existing methods choose a coefficient, and the information each token receives from the teacher is left uncontrolled. iSDFT prescribes that information at every token and derives the coefficient λtλ_t as a Lagrange multiplier. The geometric form is the solution of this projection (Proposition 1), not an interpolation we adopt. A single shared λλ coefficient solves a different problem, with the constraint pooled over tokens, so in general no constant or schedule reproduces iSDFT's targets.

To test whether this matters, we added a pooled-budget control: at every step, one λλ for all tokens, solved so that the batch receives the same total teacher information as iSDFT. Only the distribution across tokens differs. Teacher, loss, anchor, data and seed are identical (Qwen3-1.7B):

Selection ruleTool UseScience
Shared λλ = 0.551.552.4
Shared λλ = 0.7550.552.0
Shared λλ, linear 0.25 →\rightarrow 148.453.0
λλ = 1 (SDFT + anchor)50.554.2
Pooled budget (one λλ per step, total matched to iSDFT)52.554.8
Per-token budget (iSDFT)52.556.8

Even with the total matched at every step, a shared coefficient is 2.0 Science points below iSDFT, and every hand-set coefficient is lower still. Each row is a single run, and Tool Use differences are at most four of 97 questions. iSDFT was not tuned per setting: one schedule and one anchor weight across four backbones and both tasks.

To our knowledge, no prior method prescribes the information content of the target: TAID and ββ-OPSD schedule one coefficient for all tokens. ββ-OPSD (arXiv, 30 July 2026) falls within ICLR's window for contemporaneous work; we nevertheless compare with it in Table 1 and through the shared-λλ rows above.

The evidence supporting the continual-learning claims is limited. Most experiments involve independent specialisation on one of two tasks. The sequential experiment considers only one model and a two-task sequence, and the reported forgetting after the task switch is similar for iSDFT and SDFT. The results support improved final combined task performance, but provide more limited evidence that the method substantially reduces forgetting during sequential adaptation.

SDFT demonstrates that on-policy self-distillation mitigates catastrophic forgetting during sequential learning compared with SFT. Our original sequential experiment aimed to verify that iSDFT maintains comparable forgetting to SDFT while achieving better skill acquisition, rather than claiming a further reduction in forgetting compared with SDFT. In our original two-task experiment, iSDFT and SDFT showed nearly identical forgetting after the task switch, while iSDFT achieved higher final accuracy on both tasks. We further extended the experiment to three sequential tasks (Tool Use →\rightarrow Science →\rightarrow Medical) across two backbones (Qwen3-1.7B and OLMo3-7B-Think), as shown in Table below and in Figure 3 of the revised manuscript in Section 4.3. iSDFT achieves higher final accuracy in all six model-task evaluations. Figure 3 further shows that these gains mainly come from better skill acquisition and transfer, rather than consistently lower forgetting. On Qwen3-1.7B, iSDFT achieves greater transfer to Science during Tool Use training (+9.9 vs. +7.7 points), with comparable subsequent forgetting. On OLMo3-7B-Think, iSDFT achieves a larger Science gain by step 600 (+15.5 vs. +12.4 points over the base model). These results show that iSDFT achieves better combined skill performance during sequential adaptation while maintaining forgetting performance to SDFT.

BackboneTaskSDFTiSDFT
Qwen3-1.7BTool Use46.347.4
Science54.054.2
Medical23.424.3
OLMo3-7B-ThinkTool Use47.449.4
Science11.414.9
Medical18.418.7

The reported improvements on mathematics and coding benchmarks are promising, but their robustness requires further investigation. The paper notes that the generation budget truncates many responses from reasoning-oriented models. Also, evaluation under suitable generation budgets would help establish whether the gains reflect robust improvements in reasoning capability.

We conducted an additional evaluation of the base reasoning-oriented model, SDFT and iSDFT, with an increased generation budget of 32,768 tokens on MATH500, which showed the largest improvement among the additional mathematics and coding benchmarks. With the increased budget, the base model achieves 27.2, 5.6 points higher than under the original budget, while SDFT achieves 59.8 and iSDFT achieves 61.2, outperforming the base model by 34.0 points. These results show that the improvements on MATH500 persist under a larger generation budget, suggesting that the gains are not just due to response truncation.

The information budget and frozen-base anchoring strength are manually specified hyperparameters. Although the paper includes useful ablations, a more systematic sensitivity analysis would clarify how robust the results are to these choices across models and tasks.

We conducted sensitivity analyses of the information budget ρ\rho and anchoring strength μ\mu. Our results show that the benefits of information budgeting persist across multiple parameter choices, while a small anchor improves capability retention with minimal impact on skill acquisition.

As shown in the table below, several intermediate values of ρ\rho outperform the full-teacher target (ρ=1\rho=1). For example, fixed ρ=0.5\rho=0.5, fixed ρ=0.75\rho=0.75, and the schedule (0.25→10.25\rightarrow1) improve Science accuracy from 54.2 to 56.0, 55.6, and 56.8, respectively. A smaller budget (ρ=0.25\rho=0.25) performs the same as full-teacher target on Tool use and 0.8 points worse on Science, suggesting that insufficient teacher information can limit learning. For μ\mu, the results show a trade-off between acquisition and retention. μ=2×10−3\mu=2\times10^{-3} maintains the same Science accuracy as no anchoring (56.8), while improving retention in 26 of 32 evaluations (retention table below). Larger values (10−210^{-2} and 5×10−25\times10^{-2}) reduce Science accuracy to 53.4 and 52.6, respectively. This suggests that a small anchor can improve retention without harming Science acquisition, whereas excessive anchoring limits learning.

We also note that the ρ\rho schedule and μ\mu are used across all four backbones and both tasks without any model-specific tuning. iSDFT outperformed SDFT in seven of eight settings and matched it in the remaining one, this also indicates our method can be used across different models and tasks.

ConfigurationTool UseScience
Information budget ρ\rho
Fixed 0.2550.553.4
Fixed 0.551.556.0
Fixed 0.7552.555.6
Fixed 1.0 (full teacher + anchor)50.554.2
Scheduled 0.25 →\rightarrow 1 (default)52.556.8
Anchor weight μ\mu
0 (no anchor)53.656.8
2×10−32\times10^{-3} (default)52.556.8
10−210^{-2}52.553.4
5×10−25\times10^{-2}49.452.6

Retention table: Qwen3-1.7B after iSDFT (ρ:0.25→1\rho: 0.25\rightarrow1) without and with the anchor; bold marks the better setting for each task (Tool Use or Science).

BenchmarkBaseTool Use, μ=0\mu=0Tool Use, μ=2×10−3\mu=2\times10^{-3}Science, μ=0\mu=0Science, μ=2×10−3\mu=2\times10^{-3}
HellaSwag42.841.742.342.143.2
MMLU55.555.355.656.256.5
TruthfulQA33.735.134.234.734.0
WinoGrande55.556.357.355.957.4
HumanEval67.062.164.663.867.4
IFEval49.451.250.849.850.3
GSM8K66.649.949.964.164.3
MATH-50021.661.260.667.166.2
HumanEval+65.962.863.464.064.6
MBPP+49.747.848.449.149.5
LiveCodeBench9.48.18.38.99.2
MultiPL-E24.031.632.331.932.6
AIME 202416.613.316.616.626.6
AIME 202516.613.316.616.626.6
AMC 202355.050.060.047.560.0
BeyondAIME5.04.06.06.010.0

The evaluation would benefit from clearer evidence of variability across independent training runs for the main comparisons. Consistency across model–task–benchmark combinations is useful, but it does not fully capture variation arising from training stochasticity.

We ran additional experiments across two backbone models, Qwen3-1.7B and OLMo3-7B-Think and two tasks (Tool Use and Science) using three seeds 0, 1 and 42 under identical configurations for SDFT and iSDFT. iSDFT achieves higher mean accuracy in all four model-task settings. Across all configurations, the standard deviations are at most 1.3 percentage points. These results provide additional evidence of robustness to training stochasticity and show that the improvements of iSDFT are consistent across independent training runs.

BackboneTaskSDFTiSDFT
Qwen3-1.7BTool Use52.6 ± 1.352.9 ± 1.1
Qwen3-1.7BScience54.4 ± 0.856.5 ± 0.7
OLMo3-7B-ThinkTool Use51.5 ± 1.254.0 ± 0.9
OLMo3-7B-ThinkScience14.4 ± 1.017.9 ± 0.6

Thank you for your clarification and detailed description.

It has helped clarify my understanding and improved the paper. I have adjusted my score accordingly.

Reviewer_KXUkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents iSDFT, an information proximal version of SDFT that attempts to find a target distribution that mixes between the teacher and student’s distributions that keeps the target distribution close to the student’s distribution, under the constraint that it retains sufficient information from the demo-conditioned teacher. Compared to a comprehensive set of baselines, iSDFT improves skill acquisition, while leaving general capability mostly unchanged.

Synthesis of review: This paper presents an interesting method to help improve self distillation from an information theoretic framework. The paper presents comprehensive experiments on skill acquisition results compared to a large set of baselines, and demonstrates that the method is promising. However, the results on mitigating forgetting are less convincing due to a single comparison to SDFT (and no other baselines), and a single domain with only 2 steps of learning. I would have liked to see multiple experiments of this kind, and perhaps ones that involved more than 2 tasks in sequence.

Critical strengths
  • Results show that iSDFT outperforms strong baselines on learning from privileged info, while preserving previously learned skills in a continual learning setting.
  • This paper includes theoretical results showing a connection between the teacher-infromation budget, and a local tilt parameter. Instead of including the local tilt parameter as another hyperparam, they introduce a simple scalar equation that can be solved with one-dimensional root finding.
  • The authors provide a comprehensive set of limitations, and also include wall clock timings showing their method is not much more expensive than SDFT, and cheaper than the other baselines.
Critical weaknesses
  • While the framing for this paper surrounds continual learning for LLMs, the results on continual learning are relatively sparse. There is a single (two-domain) study for sequential skill learning, and it only has two tasks. I think that this paper would have been significantly stronger if it had more experiments on longer (3+) tasks in a row in order to test “true” continual learning. I think that this part of the paper is not really comprehensively evaluated.

Nit pick:

  • Results discussion is a bit hard to follow due to writing issues such as an overuse of numbers in the text (e.g. Figure 1 caption and text explanation). I suggest explaining the results a bit more intuitively in each section.
Questions for authors
  • I wonder if this might result in qualitatively better self distillation targets. For example, one common issue with SDFT is that the teacher might “leak” answers and the student learns to speak about hallucinated privileged info that doesn’t exist. If you are regularizing towards the student’s base distribution, does this become less likely? I would love to see an analysis behaviorally of the student to see if this kind of thing is mitigated.
  • How do you think this method would be affected by different kinds of privileged info (i.e. hints vs full gold solutions)?
2 comments

We thank the reviewer for the careful review and the concrete suggestions on the continual-learning evaluation and the presentation. We have run further experiments to answer them; the results are reported below.

While the framing for this paper surrounds continual learning for LLMs, the results on continual learning are relatively sparse. There is a single (two-domain) study for sequential skill learning, and it only has two tasks. I think that this paper would have been significantly stronger if it had more experiments on longer (3+) tasks in a row in order to test “true” continual learning.

We extended the sequential experiment to three tasks, Tool Use →\rightarrow Science →\rightarrow Medical, on two backbones, Qwen3-1.7B and OLMo3-7B-Think. Each task is trained for 300 optimiser steps, and all three tasks are evaluated every ten steps on Qwen3-1.7B and every twenty on OLMo3-7B-Think. The Medical stage uses open-ended clinical questions from the public HuatuoGPT-o1 data, graded by an LLM judge on 1,000 held-out questions. Accuracy after the whole sequence:

BackboneTaskSDFTiSDFT
Qwen3-1.7BTool Use46.347.4
Science54.054.2
Medical23.424.3
OLMo3-7B-ThinkTool Use47.449.4
Science11.414.9
Medical18.418.7

iSDFT finishes ahead by small margins on Qwen3-1.7B:

  • Qwen3-1.7B: iSDFT gains slightly more on each task while that task is trained, and transfers more to Science before Science is trained (+9.9 vs +7.7 points during the Tool Use stage). Measured from each task's peak, both methods lose a similar share of accuracy (Tool Use 11.6% vs 11.8%, Science 4.9% for both).
  • OLMo3-7B-Think: iSDFT gains more while Tool Use and Science are trained (Science +15.5 vs +12.4 points over the base model by the end of the Science stage).

Forgetting is therefore comparable and modest: on Qwen3-1.7B both methods keep about 88% of their peak Tool Use accuracy and 95% of their peak Science accuracy, with iSDFT losing slightly less on Tool Use. iSDFT ends with higher accuracy on every task, mainly because it learns more while each task is trained: in every stage on Qwen3-1.7B, and in the Tool Use and Science stages on OLMo3-7B-Think. This is the claim the submission makes for the sequential setting, better acquisition with comparable forgetting, now supported on a longer sequence and a second backbone.

However, the results on mitigating forgetting are less convincing due to a single comparison to SDFT (and no other baselines), and a single domain with only 2 steps of learning.

The three-task sequence above covers three domains and two backbones. We kept SDFT as the comparison in the sequential runs for two reasons. It is the method iSDFT modifies, so the difference isolates the information budget and the anchor. And SDFT's own sequential experiment already shows that it forgets less than SFT, which makes it the natural reference for forgetting.

Results discussion is a bit hard to follow due to writing issues such as an overuse of numbers in the text (e.g. Figure 1 caption and text explanation). I suggest explaining the results a bit more intuitively in each section.

In the final version, we will rewrite the results discussion to explain the pattern first reduce the use of numbers.

I wonder if this might result in qualitatively better self distillation targets. For example, one common issue with SDFT is that the teacher might “leak” answers and the student learns to speak about hallucinated privileged info that doesn’t exist. If you are regularizing towards the student’s base distribution, does this become less likely? I would love to see an analysis behaviorally of the student to see if this kind of thing is mitigated.

We measured this behaviourally: we searched every Tool Use and Science test response for explicit references to a provided example or solution, which the student never sees at test time (for example "the example response", "the user provided an example", "the reference solution"). SFT, which never trains against a demonstration-conditioned teacher, almost never does this (0-5% of responses across backbones). To isolate the regularisation towards the base model, we compare the four configurations of our ablation on Qwen3-1.7B, with and without the anchor (% of test responses with such a reference; lower is better):

TargetAnchorTool UseScience
SDFT (full teacher)none88.770.8
SDFT (full teacher)= 2×10−32\times10^{-3}81.467.1
iSDFT (information-proximal)none82.570.4
iSDFT (information-proximal)= 2×10−32\times10^{-3}77.370.0

Regularising towards the base model makes these references less likely in all four comparisons: the anchor lowers the rate by 7.3 and 5.2 points on Tool Use and by 3.7 and 0.4 points on Science. Combined with the information budget, full iSDFT produces them in 11.4 points fewer Tool Use responses than SDFT. The anchor reduces the behaviour but does not remove it, because it originates in the demonstration-conditioned teacher. Across backbones, SDFT and iSDFT show it at similar rates, and Qwen2.5-7B-Instruct, the only backbone that answers without long reasoning, shows none (Tool Use / Science, % of responses):

BackboneSDFTiSDFT
Qwen3-1.7B88.7 / 70.877.3 / 70.0
Gemma-4-E2B-it14.4 / 66.316.5 / 67.7
OLMo3-7B-Think94.8 / 35.794.8 / 37.1
Qwen2.5-7B-Instruct0.0 / 0.00.0 / 0.0

Because the references appear inside the reasoning rather than at its start, excluding the first completion tokens from the loss, as SDFT does, does not remove them. Removing them altogether remains an open problem for self-distillation from demonstrations in general, rather than a property of either method.

How do you think this method would be affected by different kinds of privileged info (i.e. hints vs full gold solutions)?

This is where we expect the design of the budget to matter. The budget is set relative to the teacher's own information gain, γt=ρ KL(Tt∥Pt)γ_t = ρ\,\mathrm{KL}(T_t \| P_t). It therefore scales with how informative the privileged context is: a hint that barely moves the teacher gives a small budget, and a full solution gives a large one. Throughout, ρρ keeps the same meaning, the share of that information each target takes. We therefore expect the same ρρ schedule to carry over from hints to full solutions without retuning, as it already does across our two tasks and four backbones.

With a weak hint, the teacher stays close to the student, the iSDFT and SDFT targets nearly coincide, and the budget costs little. With a full solution, as in our tasks, full imitation is riskier, as the leakage analysis above shows. Here our results already favour partial transfer: with the anchor fixed, transferring half or three quarters of the teacher's information per token beats transferring all of it on Science (56.0 and 55.6 vs 54.2; Appendix E.2).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.