acceptodds
Withdrawn from ICLR 2027

SPECTRUM: Proximal Spectral Modulation for Looped Self-Distillation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A model that learns from its own outputs inherits more than their correctness: it inherits which solutions it produces. We formulate Looped Self-Distillation, a self-evolution framework in which a model repeatedly generates and learns from its own raw outputs, under a fixed information budget, without ongoing external assessment or test-based selection of the generated samples. We identify a consequential separation: correctness can improve while the breadth of correct implementations contracts. We introduce SPECTRUM, which re-estimates loss-sensitive key/value geometry from a fixed reference anchor at each round and converts it into full-rank proximal spectral modulation. All generated completions train a single student, whose subsequent inference requires no intervention. After five rounds of experiments on MBPP, SPECTRUM retains 89.9% of the initial model's 64-sample correct AST richness, compared with 66.4% for plain self-distillation and 65.5% for a subspace-projection control. The advantage persists at matched correct-sample counts. Without further training or recalibration, the resulting student also achieves higher matched-correct richness than Plain on HumanEval+ and APPS Intro, demonstrating transfer of the diversity benefit. These findings establish correct-solution retention as a complementary objective of recursive self-improvement and show that generation-time intervention can improve the solution repertoire retained by subsequent students.

Primary area:
interpretability and explainable AI
Keywords:
Self-distillation, Iterative self-improvement, Synthetic data generation, Large language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ygkVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies looped self-distillation, where a code model repeatedly trains on all of its own unfiltered samples and then generates the next round's data. On MBPP it shows that pass@1 rises while the diversity of correct programs (normalized-AST richness) shrinks. SPECTRUM re-estimates, each round, the gradient second moment of the loss on 50 fixed reference solutions with respect to the K/V outputs of two layers, and folds the resulting full-rank shrinkage operator into those weights only while generating training data. With Qwen2.5-Coder-1.5B over five rounds, it retains 89.9% of the initial 64-sample correct-AST richness versus 66.4% for vanilla self-distillation, gaining 2.2 pp pass@64 but losing 1.1 pp pass@1.

Critical strengths
  1. Separating success from diversity among correct outputs (pass@k versus correct-AST richness and correct-count-matched richness, with rarefaction estimators) is a clean measurement. The five-round tracking shows that rising pass@1 can coexist with a one-third drop in correct-AST richness.
  2. The intervention is cheap and absent at inference. A closed-form operator is folded into K/V weights only during data generation, the final student is a plain model, and its richness advantage survives matching the number of correct samples (D4 +0.30, D16 +1.82 classes, paired intervals excluding zero).
Critical weaknesses
  1. At τ=1 all gains lie in [0.5, 1]. Unless the normalized spectrum is unusually flat, the operator is roughly 0.5·I plus a low-rank term, i.e., a near-uniform halving of K/V in two layers. No isotropic or random-basis control, τ/layer sweep, or spectrum is reported, so the gain is not shown to come from loss-sensitive geometry. The half-rank projection control is also indistinguishable from Vanilla SD (C16 3.066 vs. 3.090).

  2. SPECTRUM trails Vanilla SD by 1.1 pp pass@1, and SSD retains more C16 and marginal D4. The paper has no temperature/top-p sweeps, no weight interpolation or KL toward the initial model, and no baseline that also uses the 50 reference solutions (Vanilla SD and SSD use none). It is therefore unclear that SPECTRUM beats simple knobs on the correctness–diversity frontier.

  3. The evidence is narrow: one 1.5B model, MBPP-only training, five rounds, and one training run per design, with intervals covering task resampling only. The four-model matrix in Appendix E reports only initial-model pass@16. The one-round pilot is copied from an archived report after a post-hoc change to code extraction.

  4. The transfer and breadth claims are overstated. Transfer rests on D4 over 111 and 42 eligible tasks (APPS lower bound 0.007), while pass@1 is lower on both benchmarks and C16 is lower on APPS. The AST fingerprint keeps literals and function names, and the control-flow gain over projection is not significant, so much of the retained breadth may be surface variation.

Questions for authors

Please refer to Weaknesses.

3 comments
AuthorsOpenReview ↗

spectral orientation versus calibrated attenuation

1.We thank the reviewer for the careful and constructive technical analysis, particularly the proposed isotropic, random-basis, and simple-control comparisons. Following these suggestions, we ran a broad set of additional experiments. The results have clarified the mechanism and changed several interpretations we will make in the revised paper. We agree that the original manuscript did not isolate gradient-aligned orientation from attenuation strength.

For τ=1\tau=1, let μi=λi/λmax⁡\mu_i=\lambda_i/\lambda_{\max}. The operator admits the exact decomposition

T=12I+Udiag⁡ ⁣(μi2(2−μi))U⊤.T=\tfrac12 I+U\operatorname{diag}\!\left(\frac{\mu_i}{2(2-\mu_i)}\right)U^\top.

It is therefore isotropic attenuation plus a spectrum-dependent correction. The size and effective rank of that correction depend on the measured spectrum; the gain bound [0.5,1][0.5,1] alone does not establish that the correction is negligible.

We added the following five-round controls under the original Qwen2.5-Coder-1.5B / MBPP500 protocol (64 native evaluation samples per task):

Generator interventionPass@1 (%)Pass@64 (%)Correct AST C64
Plain self-distillation41.4771.28.158
Legacy hard-projection reconstruction41.6570.68.128
Fixed SPECTRUM (τ=1\tau=1)40.2472.410.870
Random-basis soft gains39.2573.012.768
Isotropic soft attenuation39.6072.611.888
Frobenius-matched residual blend42.0671.28.004

Random-basis soft gains retain the gradient-derived gain spectrum and replace the eigenbasis with an independently sampled orthogonal basis. Isotropic soft attenuation uses one scalar gain matched to the soft operator's Frobenius distance from identity. Neither is a gradient-free control. Their higher C64 and slightly lower Pass@1 show that gradient-aligned eigenvectors are not necessary for the richness gain, and that attenuation strength and orientation lead to different accuracy–richness trade-offs. Matched Frobenius displacement does not imply matched output-distribution KL.

We will revise the paper accordingly: the proximal map is a classical mathematical construction, and our supported empirical contribution is generation-time, gradient-calibrated attenuation in recursive self-distillation and its measured effect on correct implementation-structure richness. We will not claim that the specific gradient eigenvectors are required. The hard-projection entry is a reconstructed control, not an independently verified upstream SPD reproduction.


2.We agree that the behavior of the operator must be established empirically rather than inferred from its formula. We ran the following five-round sensitivity experiments under the same 1.5B / MBPP500 / 64-sample evaluation protocol:

SettingPass@1 (%)Pass@64 (%)Correct AST C64
Plain41.4771.28.158
τ=0.5\tau=0.541.3671.89.458
τ=1\tau=140.2472.410.870
τ=2\tau=238.9672.812.694
Last layer only40.5372.49.970
Middle layer only41.5071.68.870
Reuse complete first-round operator40.0672.811.496

Increasing τ\tau from 0.5 to 2 increases C64 from 9.458 to 12.694 while reducing Pass@1 from 41.36% to 38.96%. The middle-only and last-only experiments show that placement affects the trade-off. Reusing the complete round-one operator also retains high C64; this experiment does not establish that re-estimating geometry at every round is necessary.

The implementation records each targeted K/V operator's eigenvalues, mean gain, and departure from its scalar component. We distinguish (i) the exact algebraic decomposition, (ii) operator-level eigenvalue diagnostics, and (iii) performance-level controls. Full per-layer/per-round spectral histograms and numerical anisotropy ratios are not present in the current shareable result package, so we do not report inferred values as measurements. We will include measured spectral plots only with the corresponding verified operator artifacts in the revised manuscript.


3.The requested simple controls are important, and we have now separated changes to the synthesis policy from changes to the evaluation decoder. The following are trained five-round endpoints under the common native evaluation decoder:

Training-data generation policyPass@1 (%)Pass@64 (%)Correct AST C64
Plain, default41.4771.28.158
Plain, temperature 1.039.0873.09.932
Plain, top-p 0.842.8770.46.734
SSD generation recipe (T=1.5, p=0.8, k=20)35.9272.012.048
Fixed SPECTRUM40.2472.410.870

SSD achieves higher C64 than Fixed but lower Pass@1. Increased synthesis temperature is also an effective competing trade-off. These observations will be reported as such, without a claim that Fixed dominates the full accuracy–diversity frontier.

Independently, evaluating the same frozen Fixed checkpoint at temperatures 0.6 / 0.8 / 1.0 yields Pass@1 of 42.07% / 40.24% / 37.57% and C64 of 9.288 / 10.870 / 12.240. These are evaluation-time changes, not additional trained models.

We also evaluated full-weight interpolation with the initial model, θa=(1−a)θ0+aθ5\theta_a=(1-a)\theta_0+a\theta_5:

Interpolated studentPass@1 (%)Pass@64 (%)Correct AST C64
Plain, a=0.5039.2773.410.778
Plain, a=0.7540.4571.09.384
Fixed, a=0.5038.5873.612.426
Fixed, a=0.7539.2872.611.388

Interpolation is a meaningful alternative, and the same interpolation can further increase Fixed's richness. This is an inference-time checkpoint-interpolation control; a separate training-time KL-to-initial regularization baseline has not been executed. We will include these distinctions in the revised comparison.


4.We expanded the single-run evidence with three independently trained, paired 1.5B Plain/Fixed trajectories (five rounds, MBPP500, 512-token synthesis cap, 64 evaluation samples per task):

SeedPlain C64Fixed C64DifferencePlain Pass@64 (%)Fixed Pass@64 (%)
438.15810.870+2.71271.272.4
448.48411.498+3.01471.672.8
458.31411.156+2.84270.271.2
Mean ± sample SD8.319 ± 0.16311.175 ± 0.314+34.3% relative71.0072.13

At a matched budget of four correct samples on tasks eligible for both arms, paired AST-D4 differences are +0.262 / +0.289 / +0.300 on 312 / 316 / 315 common tasks. These are distinct training seeds, not bootstrap resamplings of one checkpoint.

We also trained additional backbones for five rounds:

Backbone and protocolPlain Pass@64 (%)Fixed Pass@64 (%)Plain C64Fixed C64
Qwen2.5-Coder-3B, 512-token cap73.272.013.32018.406
DeepSeek-Coder-6.7B, 512-token cap72.672.48.0269.446
Qwen2.5-Coder-7B, separate 4096-token protocol82.484.89.17414.570
Llama-3.1-8B, separate 4096-token protocol62.266.04.8527.066

The C64 increase extends across these trained settings; Pass@64 does not increase for every backbone. The 4096-token runs are a separate protocol, not replacement cells for the original 512-token experiment. Each additional backbone currently has one matched training lineage.

A separate, post-submission learned-gain extension completed ten rounds on Qwen2.5-Coder-7B (MBPP500, 4096-token synthesis cap). At R10, Plain and Learned obtain Pass@1 56.40% vs 56.30%, Pass@64 82.0% vs 82.4%, and correct-program unique fraction 0.621 vs 0.667; mean evaluation output lengths are 166 vs 167 tokens. At R8, Fixed40 and Learned obtain Pass@1 50.37% vs 57.89%, Pass@64 83.8% vs 83.2%, and mean lengths 247 vs 159 tokens. These observations motivate learning intervention gains rather than fixing their strength throughout recursion. Fixed40 was measured through R8 and SSD through R9, not R10. Learned is explicitly an extension, distinct from the submitted Fixed method.


5.The reviewer is correct that Fixed SPECTRUM uses 50 reference answers for calibration, whereas Plain and SSD do not. The generation-stage intervention is removed before standard LoRA SFT; all generated completions are retained without execution-based filtering. To test whether simple access to the same answers explains the result, we added supervised reference replay:

Five-round 1.5B methodPass@1 (%)Pass@64 (%)Correct AST C64
Plain41.4771.28.158
Plain + 50-reference supervised replay41.7970.67.936
Fixed, 50 gradient-calibration references40.2472.410.870
Fixed, 40 calibration references40.2872.611.134

Simple reference replay does not reproduce the Fixed method's correct-structure richness in this control. It does not make the information usage or calibration compute identical: supervised replay and gradient-based modulation use the reference answers differently.

The original main comparison holds the 291 synthesis prompts, 16 raw samples per prompt, five training rounds, LoRA settings, and evaluation protocol constant. It does not establish identical realized supervised-token exposure or equal total FLOPs. We additionally ran a protocol with a common configured minimum of 7.2 million shifted supervised target-token presentations per round, allowing bounded whole-record overshoot:

Five-round 1.5B method, 7.2M target-token minimumPass@1 (%)Pass@64 (%)Correct AST C64
Plain20.9545.45.666
Fixed9.0429.43.072
Plain + replay20.6649.46.132

This high-exposure setting is unfavorable to Fixed. The configured equal target is not evidence of identical realized exposure, wall time, or FLOPs; complete per-round consumption and runtime accounting are not included in the present summary. We will report the main comparison as matched in prompt count, sampling count, training algorithm, and evaluation, not as an equal-compute result. The negative token-exposure outcome will be included as protocol sensitivity in the revision.


6.We added a stricter correctness check using the MBPP+ v0.2.0 Plus tests on existing generated samples. The common evaluated population is 224 tasks shared by the original MBPP500 outputs and the MBPP+ fixture, with 64 pre-existing samples per task. This is a re-verification of an intersecting test set, not a new training dataset or a full 378-task MBPP+ evaluation. C64 below uses normalized-AST implementation fingerprints rather than exact-source hashes.

Model / studentMBPP+ Pass@1 (%)MBPP+ Pass@64 (%)Correct AST C64
1.5B Plain R543.2269.646.571
1.5B Fixed R542.5670.549.188
1.5B Random R541.2070.5411.277
7B Plain R1052.9674.558.594
7B Learned R1055.6476.7910.571

For 1.5B Plain versus Fixed, common-eligible correct-count-matched AST-D4 increases by +0.246 on 142 tasks, while control-flow D4 changes by −0.002. For 7B Plain versus Learned, the common-eligible AST-D4 increase is +0.019 on 154 tasks, whereas full-population C64 increases. We will distinguish full-population richness from diversity conditional on a fixed number of correct samples.

Frozen-checkpoint transfer results are mixed, including adverse results:

Benchmark / evaluationPlainFixed
HumanEval+ (164 tasks), Pass@1686.59%86.59%
HumanEval+ (164 tasks), C165.8486.000
Adapted APPS Intro (200 tasks), Pass@1635.5%34.0%
Adapted APPS Intro (200 tasks), C161.9301.845
Adapted APPS Intro (200 tasks), Pass@64 (separate rerun)48.0%43.5%

AST classes measure implementation structure, not semantic algorithm families. An exploratory, non-blind LLM-only strategy annotation on 30 MBPP+ tasks and 501 correct programs finds 1.200 / 1.233 / 1.233 mean task-local strategies for Plain / Fixed / Random; this is not human validation and does not establish a meaningful increase in algorithm-family breadth. We will remove the stronger semantic interpretation.

We will separate the archived extraction-changing one-round pilot from matched, newly evaluated five-round comparisons, and specify the code-extraction rule, verifier, task set, sample counts, and provenance for each result. Conditional D4 comparisons will always disclose their common eligible-task denominator. The revised manuscript will focus on reproducible correct implementation-structure richness and the measured accuracy–coverage trade-off, explicitly reporting negative transfer and methodological limits.

Reviewer_FUKcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how repeated self distillation can improve correctness while narrowing the variety of correct solutions a model produces. They introduced SPECTRUM, which uses gradients from a fixed reference set to temporarily modulate key and value representations during training data generation, then trains a student on every generated completion without correctness filtering. The resulting student requires no inference intervention. Validation follows several rounds of code generation and training on MBPP, comparing standard self distillation, decoding based SSD, and spectral projection.

Critical strengths

They make a distinction between correctness and the diversity of correct solutions, showing that repeated self distillation can improve pass@1 while reducing implementation breadth, which could be a useful insight. Its evaluation strengthens this observation by measuring diversity after matching correct sample counts with transferability.

Plus, the writing is clear and easy to follow.

Critical weaknesses

The main evidence comes from one five round training trajectory using a single 1.5B model on MBPP. Bootstrap intervals quantify uncertainty across evaluation tasks but do not capture variability from training or synthetic data generation. The authors should show the method works at least across model families, scales, or independent runs.

The comparisons do not fully isolate the benefit of spectral modulation. The method receives a fixed reference anchor unavailable to Vanilla SD and SSD, while training token counts and calibration costs differ across methods. The projection control shares the anchor but uses a different attenuation strength.

also, structurally different programs can implement essentially the same algorithm, and the reported advantage over projection is less conclusive under the coarser control flow metric. SPECTRUM also reduces pass@1 relative to Vanilla SD, and transfer diversity gains do not consistently improve task success. Therefore practical significance of greater AST richness is only partially established.

Questions for authors

See weaknesses.

6 comments

1.Thank you for identifying the limitation of reporting task-bootstrap confidence intervals from a single trained trajectory. We have now completed three independently trained Plain/Fixed pairs for the original Qwen2.5-Coder-1.5B-Instruct protocol: five recursive rounds, original-tests MBPP500, 512-token maximum synthesis generation, 16 synthetic completions per training prompt, and 64 native evaluation completions per test problem. These are training-seed replications, not three samples or bootstrap resamplings from the same student.

Independent training seedPlain Pass@1 (%)Fixed Pass@1 (%)Plain Pass@64 (%)Fixed Pass@64 (%)Plain AST C64Fixed AST C64
4341.4740.2471.272.48.15810.870
4441.3340.0671.672.88.48411.498
4541.5940.0970.271.28.31411.156
Mean41.4640.1371.0072.138.31911.175
Across-seed sample SD of C64————0.1630.314

C64 is the expected count of distinct correct normalized-AST implementations among 64 draws, averaged over all 500 evaluation tasks. Across the three paired training seeds, mean C64 increases by 34.3% and mean Pass@64 by 1.13 percentage points; mean Pass@1 decreases by 1.33 points. All three seeds show a positive C64 difference and a modest Pass@64 difference. Three runs support repeatability in this setting, but do not estimate the full distribution over possible training trajectories.

We will distinguish training-seed sample SD from task-bootstrap confidence intervals and present each trained pair rather than one point estimate.


2.The reviewer also noted that a method with more correct samples can have more observed correct implementations without a corresponding change in diversity conditional on correctness. We repeated the correct-count-matched comparison on identical eligible tasks for each independently trained pair. D4 is the expected number of distinct normalized-AST classes among four correct completions drawn without replacement. A task enters a comparison only if both students have at least four correct samples.

SeedCommon eligible tasksPlain AST D4Fixed AST D4Fixed − PlainControl-flow D4 difference
433122.8273.089+0.262+0.0029
443162.8463.135+0.289+0.0234
453152.8203.120+0.300+0.0526

The AST-D4 advantage is positive for all three training seeds after matching both the number of correct draws and the eligible task population. The smaller control-flow differences are reported in the same table; AST fingerprints and control-flow skeletons capture different structural notions and neither is a human-validated algorithm taxonomy.

We will always present full-benchmark Pass@k and C64 alongside common-eligible D4 and its denominator.

3.We extended the five-round study to two additional trained backbones under the same 512-token synthesis-cap / MBPP500 / 64-sample evaluation protocol. These are trained student comparisons, not evaluations of initial pretrained models.

Backbone (five trained rounds)Plain Pass@1 (%)Fixed Pass@1 (%)Plain Pass@64 (%)Fixed Pass@64 (%)Plain AST C64Fixed AST C64C64 relative change
Qwen2.5-Coder-3B-Instruct50.6249.8973.272.013.32018.406+38.2%
DeepSeek-Coder-6.7B-Instruct47.4446.7972.672.48.0269.446+17.7%

Both trained-model pairs show higher correct-AST richness for Fixed, while neither provides a Pass@1 or Pass@64 improvement. The model-family evidence therefore supports an implementation-richness effect in these settings, not uniform accuracy or coverage improvement. Each additional backbone has one trained pair, and we will not present it as a multi-seed replication.

The revision will add an explicit trained-checkpoint table, separate from the initial-model-only appendix matrix in the submission.


4.We also evaluated larger-model recursive training using a different synthesis length: MBPP500, five rounds, 4096 maximum new tokens, 64 native evaluation samples per task. We report this protocol separately from the 512-token study above, since its length limit and student training exposures differ.

Backbone / five-round methodPass@1 (%)Pass@64 (%)Correct AST C64
Qwen2.5-Coder-7B — Plain59.9082.49.174
Qwen2.5-Coder-7B — SSD56.7184.213.730
Qwen2.5-Coder-7B — Fixed56.8584.814.570
Qwen2.5-Coder-7B — Learned gains (post-submission extension)58.9382.810.186
Llama-3.1-8B — Plain45.4562.24.852
Llama-3.1-8B — SSD44.2667.07.258
Llama-3.1-8B — Fixed44.1566.07.066

Fixed improves C64 and Pass@64 relative to Plain on these two trained backbones. On Llama-8B, SSD achieves higher C64 and Pass@64 than Fixed. On Qwen-7B, the learned-gain extension has better Pass@1 than Fixed, while Fixed retains higher Pass@64 and C64. These are distinct points on the accuracy–coverage–richness frontier, not a single all-metric ranking.

The revised paper will include model family, model size, synthesis length, round number and number of independent training trajectories with every comparison. It will not combine the 512- and 4096-token blocks into one matched-protocol statistic.


5.We agree that five rounds are only one view of recursive self-distillation. A further Qwen2.5-Coder-7B run tracked the same model over as many as ten rounds under a 4096-token synthesis cap (MBPP500, n=64, one training seed). Plain and the post-submission Learned method have measured R10 endpoints; Fixed40/Fixed50 have measured endpoints through R8 and SSD through R9. No extrapolated round is used below.

Measured model / roundPass@1 (%)Pass@64 (%)Correct-program unique fractionMean evaluation output tokens
Plain R559.6281.60.541110
SSD R556.6683.00.687144
Fixed40 R556.6584.00.737192
Learned R558.9782.60.620139
Plain R858.2182.20.588147
SSD R838.0783.60.981196
Fixed40 R850.3783.80.955247
Learned R857.8983.20.627159
SSD R9 (last measured)28.7583.20.996228
Plain R1056.4082.00.621166
Learned R1056.3082.40.667167

The Fixed and SSD trajectories obtain high coverage and correct-program variation but undergo substantial later Pass@1 declines. At R8, Learned has higher Pass@1 than Fixed40 (57.89% versus 50.37%) with shorter mean evaluation outputs (159 versus 247 tokens), while Fixed retains a 0.6-point Pass@64 advantage. At R10, Learned and Plain have nearly identical Pass@1 and mean length, with Learned obtaining 0.4-point higher Pass@64 and a higher correct-program unique fraction.

These longitudinal results are from one 7B seed, and the unique fraction here is an exact-source-program diversity statistic, not correct-AST C64 or a count of distinct algorithm families. Learned gains optimize continuous spectral attenuation subject to reference-NLL, held-out reference guard, and output-KL checks. This is a post-submission extension, not a relabeling of the submitted Fixed method; its extra computational and reference-checking costs will be disclosed.


6.We appreciate the reviewer identifying the fixed reference anchor as additional supervised information. The Plain and SSD generation policies do not consume these 50 reference answers. Fixed SPECTRUM uses them to compute a second moment of reference-loss gradients with respect to selected K/V outputs; it does not add these answers as SFT examples. We added direct reference replay as a simple alternative way to use the same reference solutions.

For each round, the intervention is computed from a fixed training-side reference set:

Cr=1N∑j(∇hjLref)(∇hjLref)⊤,Tr=[I+τ(I−Crλmax⁡(Cr))]−1.C_r=\frac1N\sum_j (\nabla_{h_j}\mathcal L_{\mathrm{ref}})(\nabla_{h_j}\mathcal L_{\mathrm{ref}})^\top, \qquad T_r=\left[I+\tau\left(I-\frac{C_r}{\lambda_{\max}(C_r)}\right)\right]^{-1}.

It is applied only while generating the raw synthetic corpus. The original parameters are restored before ordinary LoRA SFT and native student evaluation; the generated completions are not ranked, verified or filtered for correctness during student training.

Five-round 1.5B comparisonReference usePass@1 (%)Pass@64 (%)Correct AST C64
PlainNo reference answers41.4771.28.158
Plain + reference replaySame 50 answers appended to SFT41.7970.67.936
Fixed SPECTRUM (50 references)Gradient calibration for synthesis40.2472.410.870
Fixed SPECTRUM (40 references)Gradient calibration for synthesis40.2872.611.134

Directly replaying the same reference answers does not reproduce the observed Fixed richness gain in this five-round test: C64 is 7.936 versus 10.870. The 40-reference Fixed variant obtains similar results to the 50-reference variant. This supports the value of the tested generation-time use of the reference information relative to this replay alternative. It does not establish the necessity of the reference gradients, since additional matched random/isotropic controls still use gradient-calibrated strength; nor does it establish equal compute or identical ways of using the reference solutions.

We will explicitly identify the 50 reference answers as additional training-side supervision and retain Plain-plus-reference replay as a full baseline.


7.The following table specifies what each method receives and what the student actually learns from, so that the reference information is not conflated with synthetic-corpus filtering or direct student supervision.

MethodUses fixed reference answers?How reference information is usedGeneration-time interventionStudent SFT samples
PlainNo—NoneEvery raw self-generated completion
SSDNo—Changed synthesis decoding policyEvery raw self-generated completion
Plain + reference replayYesAdds reference answers to supervised corpusNoneGenerated completions plus replayed references
Fixed SPECTRUMYes, 50 by defaultReference-gradient second moment, then a closed-form shrinkage operatorSelected K/V outputsEvery raw self-generated completion
Random-basis / isotropic controlsYesSame gradient spectral calibration, randomized basis or matched scalarSelected K/V outputsEvery raw self-generated completion
Learned gain extensionYes, 40 fit + 10 guardReference-NLL fit and final guard, with native probe-KL constraintLearned spectral attenuationEvery raw self-generated completion

The reference-replay experiment equalizes access to the reference answers as a resource, but does not equalize the mechanism of using the answers, the number of supervised target-token presentations, or gradient-calibration computation. In particular, matching reference identities is not the same as matching total training FLOPs.

The revised method section will provide this information ledger before the main tables. Our claim will concern the empirically observed effect of a reference-calibrated generation policy rather than label-free improvement.


Reviewer_w3G2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates the Looped Self-Distillation method and proposes the SPECTRUM approach to mitigate the decline in diversity.

Critical strengths
  1. The paper addresses a research question of practical significance. It identifies a notable limitation in iterative self-distillation: an increase in accuracy does not necessarily imply a better preservation of diverse correct solutions.
  2. It demonstrates good experimental results in terms of maintaining diversity.
Critical weaknesses
  1. Existing experiments have failed to adequately disentangle the respective contributions of the reference-gradient geometry and the spectral operator.
  2. While current results largely support the potential advantages of this method in terms of conditional structural diversity, they are insufficient to demonstrate that it can consistently enhance the preservation of correct solutions. The experimental framework requires substantial improvement.
Questions for authors

See Weaknesses.

7 comments

1.We thank the reviewer for recognizing the distinction between increasing correctness and retaining a repertoire of correct solutions, and for identifying two key weaknesses in the submitted study: the contribution of reference-gradient geometry was not separated from attenuation strength, and the evidence did not adequately characterize variability across independently trained students. We have therefore extended the study with direction- and norm-matched controls, multiple independent training runs, additional trained backbones, long-horizon self-distillation, stricter correctness re-verification, and controls for reference information and training exposure. These additional results have changed how we interpret our method. In particular, randomized directions and isotropic attenuation perform strongly: random/isotropic controls surpass Fixed in C64, so directional necessity is not established; soft interventions improve C64 under the studied protocol but with a Pass@1 trade-off; three independent Plain/Fixed seed pairs give mean C64 8.319±0.163→11.175±0.3148.319 \pm 0.163 \rightarrow 11.175 \pm 0.314; correct-count-matched D4 on common tasks yields AST-D4 gains of +0.262/+0.289/+0.300+0.262/+0.289/+0.300 across the three seed pairs; trained Qwen 3B and DeepSeek 6.7B pairs, together with a separate 7B/8B study, show C64 gains across tested pairs while task-success effects vary; Qwen 7B R1–R10 trajectories show that Fixed gains richness but loses later Pass@1, whereas Learned extends the accuracy–richness trade-off; and re-scoring existing generations with MBPP+ Plus tests shows that 1.5B Fixed improves AST-C64 and common-task AST-D4 on the 224-task intersection. We will therefore no longer claim that gradient-aligned eigenvectors are necessary for correct structural-richness retention. Instead, we will revise the manuscript to present correct implementation-structure richness as an empirical objective of looped self-distillation and to distinguish attenuation strength, reference calibration, orientation, and their computational costs. We also state explicitly what is not established: no equal-compute superiority, no proof of algorithm-family diversity, and no necessity of gradient directions.

2.We agree that the submitted method can be stated without appealing to a novel proximal theorem. At recursive round rr, reference-answer loss gradients at selected pre-RoPE K/V projection outputs give an uncentered gradient second moment Cr,m=1Nm∑e,tge,t,mge,t,m⊤=Ur,mdiag⁡(λi)Ur,m⊤C_{r,m}=\frac{1}{N_m}\sum_{e,t}g_{e,t,m}g_{e,t,m}^{\top}=U_{r,m}\operatorname{diag}(\lambda_i)U_{r,m}^{\top}; for nonzero Cr,mC_{r,m} and finite τ≥0\tau\ge 0, the fixed modulation is Tr,m=[I+τ(I−Cr,mλmax⁡(Cr,m))]−1=Ur,mdiag⁡(11+τ(1−λi/λmax⁡))Ur,m⊤T_{r,m}=\left[I+\tau\left(I-\frac{C_{r,m}}{\lambda_{\max}(C_{r,m})}\right)\right]^{-1}=U_{r,m}\operatorname{diag}\left(\frac{1}{1+\tau(1-\lambda_i/\lambda_{\max})}\right)U_{r,m}^{\top}. At τ=1\tau=1, all fixed gains lie in [0.5,1][0.5,1], so no eigen-direction is deleted and the operator is not a hard projection. We will describe the inverse form as a standard positive-definite attenuation construction, and locate the empirical contribution in the generation-time intervention and repeated raw-output student learning. In implementation terms, the starting checkpoint at round rr is the native student θr\theta_r; Fixed SPECTRUM uses 50 training-side labeled references for K/V gradient calibration, applies default middle and final decoder-layer K/V modules at synthesis time, temporarily folds the operator during generation and then exactly restores it, trains on all raw synthetic completions without correctness or AST filtering, updates the student with ordinary LoRA SFT merged into the next student, and evaluates the native checkpoint without modulation. We will revise the manuscript accordingly and clarify that our contribution is empirical rather than a claim of theoretical novelty or equal-compute superiority.

3.We conducted a five-round mechanism study on Qwen2.5-Coder-1.5B-Instruct, using original-tests MBPP500, a 512-token synthesis cap and 64 native evaluation samples per test question. C64 is mean expected distinct correct normalized-AST classes in 64 draws over all 500 tasks.

Synthesis interventionPass@1 (%) ↑Pass@64 (%) ↑AST C64 ↑
Plain41.4771.28.158
Legacy half-rank hard-projection reconstruction41.6570.68.128
Fixed SPECTRUM, τ=1\tau=140.2472.410.870
Random-basis soft gains39.2573.012.768
Isotropic soft attenuation39.6072.611.888
Frobenius-matched residual blend42.0671.28.004

The Random and Isotropic controls achieve greater C64 than Fixed, with lower Pass@1. Their performance rules out the strong interpretation that alignment with the reference-gradient eigenvectors is necessary for the observed structural-richness benefit. Fixed remains a different accuracy–richness trade-off point; it is not the best method on every metric. The hard-projection row is a reconstruction of the legacy half-rank control in this experimental framework; we do not present it as an independent, fully reproduced upstream SPD result. We will use this table to distinguish (a) the effect of an intervention from Plain, (b) hard deletion versus continuous attenuation, and (c) the contribution of the particular gradient-derived orientation.


4.The controls are intentionally gradient calibrated, even though the synthesis transform does not use the original reference-gradient eigenvectors in the same way.

Let si=λi/λmax⁡s_i=\lambda_i/\lambda_{\max} and gi=[1+τ(1−si)]−1g_i=[1+\tau(1-s_i)]^{-1}. The comparisons are:

Tfixed=Udiag⁡(gi)U⊤,Trandom=Rdiag⁡(gi)R⊤,Tiso=αI,T_{\rm fixed}=U\operatorname{diag}(g_i)U^\top,\qquad T_{\rm random}=R\operatorname{diag}(g_i)R^\top,\qquad T_{\rm iso}=\alpha I,

where RR is a seeded random orthogonal basis and

α=1−(∑i(1−gi)2)1/2d.\alpha=1-\frac{\left(\sum_i (1-g_i)^2\right)^{1/2}}{\sqrt d}.
ControlGradient eigenvectors used as orientation?Gradient eigenvalues used to set strength?Matching property
Fixed SPECTRUMYesYesOriginal target operator
Random-basis softNoYesEntire gain-eigenvalue multiset
Isotropic softNoYes∥T−I∥F\|T-I\|_F
Matched residual blendYes, selected subspaceYes, through matched scalar∥T−I∥F\|T-I\|_F, when feasible
Hard projectionYes, selected subspaceDetermines rank/subspaceNo soft-strength matching

These controls do not imply equal activation displacement, equal output-distribution KL, equal wall time, or equal computation. Because the random and isotropic arms still derive their strength from the gradient spectrum, they do not test a calibration-free generator.

The revised discussion will distinguish reference-gradient calibration from gradient-aligned orientation rather than treat them as one inseparable contribution.


5.We agree that the fixed operator can be close to an isotropic shrinkage plus a small effective-rank correction. For τ=1\tau=1, with si=λi/λmax⁡s_i=\lambda_i/\lambda_{\max},

T=12I+Udiag⁡ ⁣(si2(2−si))U⊤.T=\frac12 I+U\operatorname{diag}\!\left(\frac{s_i}{2(2-s_i)}\right)U^\top.

This is an exact decomposition. Whether the second term is small in a particular module depends on its measured spectral distribution. Neither the formula alone nor the Random/Isotropic performance ordering determines its empirical size. We will add the realized gain distributions and corresponding diagnostics to the paper if the original per-round operator artifacts are recovered. We will not assign numerical anisotropy values from theoretical bounds or from downstream C64 alone.


6.Higher C64 can arise because a model produces more correct samples, even without changing the diversity of its correct samples. We therefore additionally compare D4 on the same tasks eligible in both arms. D4 is the expected number of distinct normalized-AST classes among four correct samples drawn without replacement, and eligibility requires at least four correct samples for each student.

Comparison, five-round 1.5BCommon eligible tasksLeft AST-D4Right AST-D4DifferenceControl-flow D4 difference
Fixed → Random-basis soft3133.0863.277+0.191+0.044
Fixed → Isotropic soft3133.0863.194+0.108+0.037
Plain → Fixed, seed 433122.8273.089+0.262+0.003

The Random/Isotropic advantage is not restricted to all-draw C64; their AST-D4 estimates are also higher on these common eligible populations. The control-flow differences are much smaller, so we distinguish AST structure from higher-level algorithm identity.

The comparisons are from one mechanism-study training seed; they are not evidence that the random or isotropic controls universally dominate Fixed over independent seeds.


7.We varied the strength of Fixed SPECTRUM under the original Qwen 1.5B five-round protocol, holding the remaining evaluation settings fixed.

InterventionPass@1 (%) ↑Pass@64 (%) ↑Correct AST C64 ↑
Plain41.4771.28.158
Fixed, τ=0.5\tau=0.541.3671.89.458
Fixed, τ=1.0\tau=1.040.2472.410.870
Fixed, τ=2.0\tau=2.038.9672.812.694

Increasing τ\tau is associated with higher C64 and Pass@64 but lower Pass@1 in this tested range. This provides evidence that modulation strength matters and makes a universal “more diversity without a quality cost” claim inappropriate.

We will report the complete accuracy–coverage–richness frontier rather than select one attenuation strength after seeing held-out test scores.


8.We separately tested which default target layers are necessary, and whether the full first-round operator must be recomputed every round.

Five-round 1.5B settingPass@1 (%) ↑Pass@64 (%) ↑Correct AST C64 ↑
Plain41.4771.28.158
Default middle + final layers, K/V, τ=1\tau=140.2472.410.870
Final layer only40.5372.49.970
Middle layer only41.5071.68.870
Reuse entire round-one operator40.0672.811.496

The location matters: the final-layer-only intervention retains more structural richness than the middle-layer-only intervention in this setting. Reusing the entire first-round operator also achieves high C64. Therefore, the current results do not establish that recomputing the geometry every round is necessary.

We will revise the method text accordingly and separate the design choice of refreshing the operator from the empirically supported benefit of the broader generation-time intervention.


9.The original study reported task-bootstrap intervals from one model lineage. We conducted three independent Plain/Fixed training pairs using Qwen2.5-Coder-1.5B-Instruct, original MBPP500, five rounds, 512 maximum synthesis tokens and 64 evaluation draws per test problem.

Training seedPlain Pass@1 (%)Fixed Pass@1 (%)Plain Pass@64 (%)Fixed Pass@64 (%)Plain AST C64Fixed AST C64
4341.4740.2471.272.48.15810.870
4441.3340.0671.672.88.48411.498
4541.5940.0970.271.28.31411.156
Mean41.4640.1371.0072.138.31911.175
Sample SD across seeds0.1270.0980.7210.8330.1630.314

Across these three independent lineages, mean C64 rises by 34.3%, Pass@64 rises by 1.13 percentage points, and Pass@1 decreases by 1.33 points. The increase in C64 is observed in each seed. These numbers support repeatability under this protocol; three seeds cannot prove a universal advantage across all random initializations or budgets.

The paper will report the seed-specific values and sample standard deviations, rather than presenting task-bootstrap intervals as an estimate of training stochasticity.


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.