acceptodds
Under review as a conference paper at ICLR 2027

Attune: State-Adaptive Supervision Allocation for On-Policy Distillation

Abstract

On-policy distillation trains compact language models with teacher feedback on their own responses. Deciding which feedback to emphasize remains challenging: teacher–student disagreement signals a prediction gap, while the benefit of correcting it depends on the student's capacity and current training state. We introduce a controlled paired-intervention protocol using supervised fine-tuning on compositional arithmetic to measure the returns of fixed supervision under matched token exposure. The relative gains vary across student capacities and training states, supporting a learner-conditioned view of supervision utility. We operationalize this view through Attune, a coupled token–group allocation framework for on-policy distillation. Applying the same token calibration to candidate and reference losses couples the learning signal to the gradient comparisons used for allocation. Across eight benchmarks and two model families, Attune achieves higher aggregate scores than the compared distillation methods, and these gains persist across three training seeds. Independent-probe interventions further show that both components reduce distillation loss relative to uniform and shuffled weighting. Attune requires no difficulty labels and adds no inference-time computation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, knowledge distillation, on-policy distillation, adaptive supervision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_xUjKOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a combined approach to address an important question of in an on policy distillation regime, which supervision is useful for a student at its current training stage. More specifically, it leverages two main components i.e. group alignment and token calibration to achieve this. The main gains are supported via evaluation across three training seeds , while single seed ablation studies analyze the effect of each component and their combination.

Critical strengths

1- The paper addresses an important question of which supervision is useful for a student at its current training state. 2- Useful ablations have been provided to better understand the role of different components i.e. group alignment and token calibration

Critical weaknesses

1- First of all, the paper in its current shape is really hard to follow and transition from the diagnosis to the method section requires substantial improvement. Better clarifying the problem. intuition and assumptions before explaining the components and formulation make the paper easier to follow. 2-My main concern is that the paper has not established significant technical contribution over existing approaches and is more of an effective combination of existing approaches. 3-There are missing comparisons with the closely related works such as reference gradient reweighing family of methods which may help to better justify the novelty of the proposed method.

Questions for authors
  1. Improve the transition from the diagnosis to the method section Section 2 shows that the benefit of fixed supervision changes with model capacity and training state. However, it is not clear how these observations lead to Attune's specific design. The diagnosis uses SFT on fixed arithmetic reasoning trajectories, while Attune uses token probability gaps and controller-gradient alignment for on-policy distillation. Why should the token gate in eq. (2) and the alignment score in eq. (3) capture the changes observed in the diagnosis? Explaining the problem, intuition, and assumptions behind each component before introducing the formulation and methodology would make the paper easier to follow.

  2. Expand experiments to show whether the controlled findings extend beyond arithmetic and Qwen. The main paired-intervention study evaluates four Qwen2.5-Instruct sizes on a 500 example arithmetic probe. Although the appendix adds three domain experiments and Attuneis evaluated on both Qwen and Llama, the motivating study that is the main motivation for the proposed method is not as broad as the overall evaluation. So, it would be useful to repeat the controlled interventions on another model family and a natural reasoning task. Evaluating the full 5,000-example held-out probe would also help check the stability of the patterns. Please clearly differentiate between observations specific to this setup and conclusions that apply more generally.

  3. Explain the controller choices and test their sensitivity. The controller construction has been well-described, but it is not clear why you use a fixed pool of 512 prompts and a minibatch of 32? Since its gradient determines which training groups receive more weight, the choice of controller prompts could be significantly important. I would like to see how the results change with the pool size, minibatch size, source mixture, and random choice of prompts. A few of such experiments would establish a better understanding of the stability of the gains across reasonable controller configs.

  4. Justify the choice of gradient-probe layers. It is not clear why you compare gradients only in the LM head and final two Transformer blocks? Table 3 supports combining the head and top-block, but does not explain why these two blocks were chosen. Since the final update trains all student parameters, other layers may provide useful information about update quality. Comparing a few layer subsets and, in a smaller setting, full-model alignment would help justify the choice. Please report both the quality of the allocation scores and the computational cost.

  5. Include a more closely relevant allocation baseline. The paper covers a useful range of on-policy distillation methods, but the closest alternatives also include reference-gradient reweighting and learner-adaptive allocation. An adapted baseline based on ideas from Ren et al., LESS, or GradAlign would help show what Attune adds over existing related work. Since some of these methods are directly applicable, the comparison should preserve the same distillation objectives, with matched compute if possible. If a meaningful adaptation is not feasible, please explain the difference in setting. DMC and PACED are also relevant to the broader comparison with student-adaptive distillation.

Related references

a- Ren et al. Learning to Reweight Examples for Robust Deep Learning. ICML 2018. b-Xia et al. LESS: Selecting Influential Data for Targeted Instruction Tuning. ICML 2024. c-Yang et al. GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning. 2026. d-Huang et al. Tailoring the Curriculum: Student-Centered Reasoning Distillation via Dynamic Data-Model Compatibility. 2026. e-Xu et al. PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence. 2026.

  1. Separate local loss improvement from capability improvement Table 4 measures one-step reductions in unweighted Generalized Jensen-Shannon divergence loss. This is a useful evidence of improved local teacher-student agreement, but it does not directly imply better reasoning or instruction following. Moreover, it is not clear whether the allocation scores predict capability gains beyond a single update. Testing longer update horizons and generation-based outcomes would strengthen that paper. Reporting the starting probe loss and relative reduction would also make the interpretation of the size of the local gains much easier.

  2. State the methodological novelty more specifically Reference-gradient reweighting is already established. Applying token calibration to both candidate and controller gradients appears to be Attune's key contribution, and table 3 provides useful support for that. However, it is not clear how this differs from prior methods and what the rank-fusion step adds. The closer baseline reported above would help understand whether the contribution is a new allocation mechanism or an effective combination of existing ideas for on-policy distillation.

  3. Establish the added benefit of the coupled design across seeds. Table 3 suggests that group alignment accounts for much of the aggregate improvement i.e. TaskAvg rises from 42.24 for GKD to 43.45 with group alignment alone, compared with 44.30 for Full Attune. Adding token calibration still improves TaskAvg by 0.85 and CatAvg by 0.76 but does not help every task. On TruthfulQA-MC2, group alignment alone scores 50.94 vs 50.60 for Full Attune (probably wrongly bolded in the table). Since these components comparisons use only seed 10, please repeat the ablations across training seeds and report uncertainty for the paired gains. This would help establish whether the coupled design reliably improves over group alignment alone. Also, please clarify whether the bold Full Attune row highlights the method or the best result in every column.

12 comments

We sincerely thank Reviewer xUjk for your insightful feedback and for recognizing the importance of studying state-dependent supervision utility and the value of our component ablations. As you rightly noted, controller-aligned group allocation drives a major share of Attune's empirical gains and serves as the core mechanism of our framework. Because establishing the stability and boundary conditions of the controller is central to our claims, we prioritized this comprehensive investigation first and present the results below as Part 1 of our response.

Part 1: Controller Design Choices and Sensitivity Analysis — Addressing Q3

Keeping the 96,000-prompt training stream fixed in Qwen2.5 7B-to-1.5B distillation, we evaluated 10 additional configurations varying the per-step controller minibatch from 8 to 64 with the pool size fixed at 512, varying the reference pool size from 128 to 1024 with the per-step minibatch fixed at 32, excluding individual capability domains from the 512-prompt pool, and folding the cyclic controller trajectories directly into the GKD loss instead of using them for group allocation.

SettingIFEvalMMLU-ProTQA-MC2DROPMuSRBBHGSM8KMATH-500TaskAvgdTaskCatAvgdCat
Default Mix 512, MB 3256.3833.7450.607.3237.8346.7170.4351.4044.30-41.81-
Mini-batch 8, Pool 51253.0531.7149.426.9935.3243.9666.1149.2041.97-2.3339.58-2.23
Mini-batch 16, Pool 51254.3431.8149.467.2635.4544.0266.2649.4042.25-2.0539.96-1.85
Mini-batch 64, Pool 51256.0134.5051.377.2738.6247.7571.8052.2044.94+0.6442.28+0.47
Mix 128, MB 3252.5032.2749.437.1435.9844.5366.1149.8042.22-2.0839.74-2.07
Mix 256, MB 3253.6032.9849.977.1636.9045.3768.4650.6043.13-1.1740.58-1.23
Mix 1024, MB 3256.5634.0750.927.2638.2347.1570.8951.8044.61+0.3142.07+0.26
w/o Math 51255.6433.4950.077.1837.5746.3669.3750.4043.76-0.5441.29-0.52
w/o Reasoning 51256.1933.4050.227.2937.0445.6570.1351.2043.89-0.4141.46-0.35
w/o Knowledge 51257.3033.4350.297.2637.8346.7070.4351.4044.33+0.0341.92+0.11
GKD + Direct Ctrl52.8732.0450.187.1536.3845.0366.0348.8042.31-1.9939.85-1.96
Ref: Vanilla GKD53.0531.9850.127.1436.2444.9265.8848.6042.24-2.0639.81-2.00
  • Why we chose a 512-prompt pool and a 32-prompt minibatch: Our results confirm that overly small controller budgets introduce severe estimation bias and instability across the five training domains. In particular, shrinking the per-step minibatch to 8 drops TaskAvg to 41.97, while a small 128-prompt pool similarly underperforms due to rapid cyclic overfitting. Conversely, scaling beyond our default setting to a 64 minibatch or a 1024-prompt pool brings diminishing marginal returns, reaching 44.94 and 44.61 at the cost of higher runtime overhead or pool size. Moreover, across our three seeded runs in Table 2, the random draw of the 512-prompt pool varies with the seed while maintaining a tight TaskAvg standard deviation of 0.14.
  • Source mixture and task-level specificity: Removing a domain selectively impacts its closely related benchmarks while maintaining strong global performance. Dropping math primarily affects GSM8K and MATH-500, whereas dropping reasoning mainly affects MuSR and BBH. Even under complete omission of math or reasoning, bounded group weights and token calibration keep TaskAvg at 43.76 and 43.89, remaining well above baselines in Table 1.
  • An unexpected gain from removing knowledge prompts: An intriguing anomaly is that removing knowledge prompts slightly surpasses the default mixture, reaching 44.33 TaskAvg as a notable gain on IFEval outweighs minor dips on MMLU-Pro and TQA-MC2. Candidly, this indicates that simply mirroring the training distribution in the controller is not strictly optimal, and we plan to explore adaptive controller curation strategies in future work. We tentatively attribute this to post-training dynamics: since factual recall is already largely anchored by pre-trained weights and token calibration, shifting controller capacity toward procedural tasks yields a cleaner trajectory-level gradient direction.
  • Direct supervision control: Also, we evaluated a control that folds the same cyclic controller trajectories directly into the GKD training loss, which reaches only 42.31 TaskAvg and confirms that our gains stem from adaptive gradient allocation rather than extra data exposure.

We sincerely thank Reviewer xUjk for the insightful comments on establishing multi-seed support for our coupled design and clarifying both our methodological novelty and Table 3's formatting. Following your suggestions, we repeated all Table 3 ablations across three training seeds and added a Controller-only calibration variant to complete the full factorial comparison.

Part 2: Multi-Seed Verification of the Coupled Design and Methodological Novelty — Addressing W2, Q7 & Q8

To test whether our coupled design reliably improves over Group-only and partial calibration variants, we evaluated all Table 3 configurations across Seeds 10, 20, and 30 in Qwen2.5 7B-to-1.5B distillation. Each per-seed column below reports TaskAvg / CatAvg.

SettingSeed 10Seed 20Seed 30TaskAvgdTaskCatAvgdCat
Full Attune44.30 / 41.8144.02 / 41.5444.18 / 41.7544.17±0.14-41.70±0.14-
Controller-only43.85 / 41.2343.67 / 41.0743.76 / 41.1543.76±0.09-0.4141.15±0.08-0.55
Candidate-only43.23 / 40.8443.12 / 40.6943.46 / 41.0543.27±0.17-0.9040.86±0.18-0.84
Uncalibrated43.01 / 40.6442.65 / 40.3243.10 / 40.7242.92±0.24-1.2540.56±0.21-1.14
Group-only43.45 / 41.0543.36 / 40.9843.60 / 41.2143.47±0.12-0.7041.08±0.12-0.62
Token-only42.68 / 40.2842.33 / 40.0342.58 / 40.3242.53±0.18-1.6440.21±0.16-1.49
Vanilla GKD42.24 / 39.8141.91 / 39.4942.23 / 39.8942.13±0.19-2.0439.73±0.21-1.97
Ref: TA-OPD42.68 / 40.3742.61 / 40.3142.58 / 40.3542.62±0.05-1.5440.34±0.03-1.35
  • Reliable multi-seed gain over group alignment alone and Table 3 formatting: Across all three seeds, Full Attune consistently improves over Group-only with low variance, confirming that the coupled design provides a stable advantage beyond group alignment alone. Regarding Table 3's formatting, the original table bolded the Full Attune row to highlight the proposed method; we will revise it to bold strictly the column-best values so that TruthfulQA-MC2 is unambiguous.
  • Why Attune is not a naive combination of existing components: A straightforward uncoupled combination of group alignment and token gating corresponds to Uncalibrated alignment, which applies the token gate to the update loss while scoring uncalibrated gradients. Across all three seeds, this naive combination degrades performance below Group-only and doubles variance, because candidate groups are selected on large-gap outlier tokens that are subsequently downweighted during the actual update.
  • Super-additive synergy of coupled calibration: Relative to Vanilla GKD, the three-seed gain of Full Attune exceeds the linear sum of Token-only and Group-only. Calibrating both controller and candidate scoring gradients aligns the selection geometry with the token-gated update step, nearly doubling the marginal benefit of token calibration compared to using it in isolation.

We sincerely thank Reviewer xUjk for highlighting the need for a clearer transition from our Section 2 diagnosis to the Section 3 methodology. We outline below the problem intuition, underlying assumptions, and analytical formulation connecting our diagnostic observations to Equations (2)–(5).

Part 3: Problem Intuition and Analytical Link from Diagnosis to Method — Addressing W1 & Q1

Our Section 2 diagnosis (and Appendices A.2–A.4) establishes two empirical properties of supervision utility as a student θt\theta_t evolves: (i) Local fit vs. global generation divergence: forcing a capacity-constrained student to fit complex trajectories can improve intermediate-step accuracy or teacher-forced token NLL while degrading end-to-end free-generation accuracy (Section 2.4; Table 11); and (ii) State-conditioned utility: the return R(t)R(t) of fixed supervision varies across student capacities and training states (Figure 1), and cannot be reliably predicted by raw token-level teacher–student disagreement alone (Table 4A, where candidate JSD and positive CE gap achieve Spearman ρ\rho of only 0.14 and 0.19). To operationalize these findings in on-policy distillation without costly counterfactual branch rollouts at every step, Attune couples two state-dependent mechanisms around the current student πS\pi_S:

LCcal(θt+Δθt)−LCcal(θt)≈−ηB∑bwb⟨gCcal,gbcal⟩,⟨gCuncal,gbuncal⟩=⟨gCcal,gbcal⟩+⟨δC,gbcal⟩+⟨gCcal,δb⟩+⟨δC,δb⟩L_C^{\text{cal}}(\theta_t + \Delta\theta_t) - L_C^{\text{cal}}(\theta_t) \approx -\frac{\eta}{B}\sum_b w_b \langle g_C^{\text{cal}}, g_b^{\text{cal}} \rangle, \qquad \langle g_C^{\text{uncal}}, g_b^{\text{uncal}} \rangle = \langle g_C^{\text{cal}}, g_b^{\text{cal}} \rangle + \langle \delta_C, g_b^{\text{cal}} \rangle + \langle g_C^{\text{cal}}, \delta_b \rangle + \langle \delta_C, \delta_b \rangle

  • Why the token gate in Eq. (2) provides state-adaptive local pacing: In on-policy rollouts y∼πS(⋅∣x)y \sim \pi_S(\cdot|x), tokens with a large positive log-probability gap du=[log⁡(puT(yu)/puS(yu))]+d_u = [\log(p_u^T(y_u) / p_u^S(y_u))]_+ mark local positions where the teacher strongly favors a transition that the current student state θt\theta_t assigns low probability. Motivated by Observation (i), Eq. (2) uses dud_u as a student-conditioned discrepancy signal, mapping ru=max⁡(exp⁡(−γdu),ϵ)r_u = \max(\exp(-\gamma d_u), \epsilon) to bounded within-group weights au(θt)∈(0.75,1.25)a_u(\theta_t) \in (0.75, 1.25). Rather than relying on static difficulty labels, au(θt)a_u(\theta_t) softly tempers the gradient influence of large positive-gap tokens early on and automatically relaxes toward 1 as θt\theta_t masters those transitions (puS→puT⇒du→0+p_u^S \to p_u^T \Rightarrow d_u \to 0^+), while retaining full corrective weight whenever the student is over-confident (du=0d_u = 0).
  • Why controller alignment in Eq. (3) serves as a local proxy for state-dependent utility: Motivated by Observation (ii), we need an online surrogate to evaluate which candidate groups GbG_b (b=1,…,Bb = 1, \dots, B) benefit the student at its current state θt\theta_t. Under a first-order Taylor approximation around θt\theta_t with fixed trajectories and detached token weights (left equation above), the local reduction in the calibrated controller loss LCcalL_C^{\text{cal}} induced by candidate update Δθt=−ηB∑bwbgbcal\Delta\theta_t = -\frac{\eta}{B}\sum_b w_b g_b^{\text{cal}} is governed by the inner product ⟨gCcal,gbcal⟩\langle g_C^{\text{cal}}, g_b^{\text{cal}} \rangle. Normalizing by subspace norms within the LM head and top Transformer blocks (Eq. 3) and fusing percentile ranks (Eq. 4) provides a scale-invariant proxy for state-conditioned utility, refreshed on current student rollouts at every step.
  • Analytical decomposition of why Eq. (2) and Eq. (3) should be coupled: Decomposing any uncalibrated gradient into its calibrated counterpart plus the residual from token reweighting, gXuncal=gXcal+δXg_X^{\text{uncal}} = g_X^{\text{cal}} + \delta_X where δX=1NX∑u(1−au)∇θℓuKD(θt)\delta_X = \frac{1}{N_X}\sum_u (1 - a_u)\nabla_\theta \ell_u^{\text{KD}}(\theta_t), reveals why scoring uncalibrated gradients conflicts with the calibrated parameter update in Eq. (5) (right equation above). Because Eq. (5) updates θt\theta_t along gbcalg_b^{\text{cal}}, scoring groups via ⟨gCuncal,gbuncal⟩\langle g_C^{\text{uncal}}, g_b^{\text{uncal}} \rangle introduces three discrepancy terms driven by δC\delta_C and δb\delta_b—allowing tokens that are downweighted during the actual update to distort the allocation scores.
  • Direct consistency with our three-seed 2×22 \times 2 ablation in Part 2: This decomposition directly accounts for the empirical ordering reported in Part 2: leaving both sides uncalibrated retains all three discrepancy terms and degrades TaskAvg to 42.92 ± 0.24 (below even uniform Group-only at 43.47 ± 0.12); calibrating only the candidate side (δb=0\delta_b = 0, 43.27 ± 0.17) or only the shared controller side (δC=0\delta_C = 0, 43.76 ± 0.09) removes a subset of these mismatch terms; and calibrating both sides in Full Attune (δC=δb=0\delta_C = \delta_b = 0, 44.17 ± 0.14) aligns the scoring geometry with the actual parameter update.
Reviewer_DNnbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to allocate teacher supervision during on-policy language-model distillation. A controlled compositional-arithmetic study compares fixed basic and difficult supervision blocks across four student sizes and five training states, matching operation count, target-trace length, and response-token exposure. The relative benefits change with the configured student and its training state.

The proposed method, Attune, combines two mechanisms. A detached token gate softly downweights positive teacher-over-student log-probability gaps, followed by groupwise normalization and shrinkage toward uniform weights. Candidate-group and controller losses use the same calibration rule. Their gradient cosines in the LM head and final two Transformer blocks are converted into fused percentile ranks and group weights in [0.8, 1.2]. The resulting loss updates all student parameters. Across eight benchmarks, three-seed comparisons report TaskAvg improvements over vanilla GKD of 2.04 points for Qwen2.5 and 2.08 points for Llama-3.2. Component ablations, independent-probe one-step interventions, and approximate compute-budget comparisons provide additional evidence.

Critical strengths
  1. The controlled diagnosis is thoughtfully designed. The main arithmetic experiment changes dependency depth while keeping 15 operations and a 15-line target trace fixed. Independent anchor/hard branches start from the same parent state, and response-token mismatch is at most 0.03% (Section 2; Appendix A.2). Reusing the same intervention pair across states and capacities makes the comparison more informative than simply comparing easy-first and hard-first curricula with different exposure histories. The paper also appropriately reports that the 0.5B endpoint confidence interval includes zero.

  2. The empirical improvement is reasonably broad and repeatable. The main comparison includes several distillation objectives, trajectory-construction methods, and student-aware allocation baselines. In Table 2, Attune improves mean TaskAvg by 1.54 points over the selected strongest aggregate baseline in each family, with positive paired confidence intervals. Three-seed results and multiplicity-adjusted tests are a meaningful improvement over relying only on the single-seed ranking in Table 1. The paper also acknowledges the lack of a clear advantage on Llama MuSR.

  3. The coupling claim has targeted ablations. Table 3 keeps the final token-gated update loss while varying whether candidate and controller scoring gradients are calibrated. Full calibration improves TaskAvg by 1.29 points over uncalibrated alignment and by 1.07 points over candidate-only calibration. These comparisons address the specific proposal more directly than removing each component independently. The one-step tests additionally use a separate outcome probe and shuffled-weight controls, reducing the risk of merely evaluating the allocation rule on its own controller objective.

  4. The paper addresses additional training cost rather than relying solely on equal-step comparisons. Appendix A.11 reports 1.242 times the GKD model FLOPs and compares both extended GKD and shortened Attune runs. The 1,200-step Attune run still exceeds 1,900-step GKD by 1.16 TaskAvg points. Although approximate, this is useful evidence that the gains are not explained simply by more training computation. The method also preserves the student architecture and inference cost.

Critical weaknesses
  1. The central coupling ablation is incomplete. Table 3 is informative, but it evaluates no scoring-gradient calibration, candidate-only calibration, and calibration on both sides without including controller-only calibration. All three variants retain the token gate in the final update loss. Consequently, the reported full-versus-candidate-only improvement shows the value of controller calibration, but does not isolate whether calibrating the candidate scoring gradient additionally helps once the controller is calibrated. The missing factorial cell matters for interpreting the specific contribution of coordinating the two sides, rather than improving the reference direction alone.

  2. The local evidence does not yet separate state-dependent utility prediction from persistent group preferences. Table 4 reports a macro correlation between current alignment scores and one-step probe-loss gains. Appendix A.9 holds prompt identities and group assignments fixed across four checkpoints, but does not compare current scores against rankings carried over from another checkpoint. A group could remain consistently useful across states, producing positive contemporaneous correlations without showing that score changes track changes in utility. It leaves open how much of the predictive value specifically comes from tracking the evolving learner. The distinction is especially relevant because the motivating SFT study measures capability gains, whereas the local Attune analysis measures one-step GJS reduction.

  3. The value of using controller supervision indirectly is not compared with a direct-use alternative. Attune obtains fresh student rollouts and teacher feedback on 32 controller prompts per step, drawn cyclically from a fixed pool of 512 prompts. Appendix A.11 usefully compares against GKD with additional unique training prompts under an approximate FLOP budget, but does not compare against directly training on the same recurring controller supervision. A matched direct-use control would establish whether routing that feedback through allocation offers an advantage over incorporating it into the training loss, beyond the already demonstrated advantage over ordinary GKD.

  4. Practical training overhead is not established by the model-FLOP analysis alone. Appendix A.11 reports a 1.242-fold model-FLOP cost and provides useful approximate compute-budget comparisons. However, the supplied analysis does not report measured end-to-end training time or peak GPU memory. Computing and comparing group and controller gradients may introduce overhead from memory use and inter-worker communication that is not captured by matrix-multiplication and attention FLOPs. Reporting these measurements under the same hardware and training configuration would help assess the practical cost of the performance gains.

Questions for authors
  1. Does calibration on both scoring sides outperform controller-only calibration? Please complete the Table 3 comparison with a controller-only variant: retain the final token-gated update loss, calibrated controller scoring gradients, the same Head/Top rank fusion, and the same weight range, but use uncalibrated candidate scoring gradients. Does full Attune still improve performance under this control? This would test whether calibrating the candidate side adds value once the reference side is calibrated.

  2. Do score changes track changes in utility, beyond a persistent preference for particular prompt groups? Appendix A.9 already pairs prompt identities and group assignments across checkpoints. Using those measurements, could you compare contemporaneous score-utility correlations with correlations obtained by carrying a group's scores/ranking over from another checkpoint? In particular, when a group's measured relative utility changes across states, does its current score predict that change better than its earlier score? This would directly test whether the scores track changing utility. A loss-based result should still be distinguished from a capability-gain result.

  3. Is indirect allocation preferable to directly learning from the same controller feedback? Could you compare Attune with a control that incorporates the same cyclic controller prompts, fresh rollouts, and teacher supervision directly into the KD training loss, using an approximately matched total FLOP budget? Retaining the token gate in both variants would focus the comparison on controller-based allocation. The Long GKD result addresses extra computation and additional unique training prompts, but not this alternative use of the same recurring supervision.

  4. What is the measured training overhead relative to vanilla GKD? Appendix A.11 reports 1.242 times the model FLOPs. Under the same hardware, global batch size, numerical precision, and comparable generated-token counts, could you report end-to-end training time and peak GPU memory for Attune and GKD? Please include rollout generation, teacher scoring, probe-gradient computation, and inter-worker communication in the accounting, and explain whether the measured runtime ratio is consistent with the FLOP estimate.

5 comments

We sincerely thank Reviewer DNnb for the positive evaluation and for recognizing our controlled arithmetic diagnosis, repeatable multi-seed benchmark gains, targeted coupling ablations, and compute-budget comparisons. Below, we first address your question regarding indirect controller allocation versus direct supervision reuse, accompanied by a broader controller sensitivity suite.

Part 1: Indirect Controller Allocation vs. Direct KD Supervision — Addressing W3 & Q3

To test whether routing recurring controller feedback through group allocation is preferable to folding that exact same supervision directly into the KD training loss, we evaluated a matched direct-supervision control in Qwen2.5 7B-to-1.5B distillation. At every step, the student generates fresh rollouts on the same cyclic 32-prompt controller minibatch from the 512-prompt pool with teacher scoring, but incorporates them directly into the training loss instead of computing group weights. We present this control alongside nine additional configurations varying the controller's domain composition, pool size, and minibatch size.

SettingIFEvalMMLU-ProTQA-MC2DROPMuSRBBHGSM8KMATH-500TaskAvgdTaskCatAvgdCat
Default Mix 512, MB 3256.3833.7450.607.3237.8346.7170.4351.4044.30-41.81-
GKD + Direct Ctrl52.8732.0450.187.1536.3845.0366.0348.8042.31-1.9939.85-1.96
w/o Math 51255.6433.4950.077.1837.5746.3669.3750.4043.76-0.5441.29-0.52
w/o Reasoning 51256.1933.4050.227.2937.0445.6570.1351.2043.89-0.4141.46-0.35
w/o Knowledge 51257.3033.4350.297.2637.8346.7070.4351.4044.33+0.0341.92+0.11
Mix 128, MB 3252.5032.2749.437.1435.9844.5366.1149.8042.22-2.0839.74-2.07
Mix 256, MB 3253.6032.9849.977.1636.9045.3768.4650.6043.13-1.1740.58-1.23
Mix 1024, MB 3256.5634.0750.927.2638.2347.1570.8951.8044.61+0.3142.07+0.26
Mini-batch 8, Pool 51253.0531.7149.426.9935.3243.9666.1149.2041.97-2.3339.58-2.23
Mini-batch 16, Pool 51254.3431.8149.467.2635.4544.0266.2649.4042.25-2.0539.96-1.85
Mini-batch 64, Pool 51256.0134.5051.377.2738.6247.7571.8052.2044.94+0.6442.28+0.47
Ref: Group Align Only55.6432.9350.947.2837.4345.6967.7050.0043.45-0.8541.05-0.76
  • Indirect allocation vs. direct training on controller feedback: Directly folding the cyclic controller trajectories into the KD loss reaches only 42.31 TaskAvg, trailing Default Attune by 1.99 points and barely improving over Vanilla GKD at 42.24. Even compared against Group Alignment Only at 43.45, which uses uniform token weights, indirect allocation outperforms direct training on the same recurring prompts by 1.14 points. Because the 512 controller prompts cycle every 16 steps, direct gradient updates on them quickly suffer from diminishing returns and local overfitting, whereas using them as a reference compass effectively guides allocation across the training prompts.
  • Domain exclusion and an unexpected gain from removing knowledge: Dropping math or reasoning from the controller selectively lowers their corresponding benchmarks by around 1 point, yet bounded weights keep TaskAvg at 43.76 and 43.89. Removing knowledge prompts slightly surpasses the default mixture, as a notable gain on IFEval outweighs minor dips on MMLU-Pro. Candidly, this shows that simply mirroring the training distribution in the controller is not strictly optimal, and we plan to explore adaptive controller curation in future work. We tentatively attribute this to post-training dynamics: since factual recall is already largely anchored by pre-trained weights and token calibration, shifting controller capacity toward procedural tasks yields a cleaner trajectory-level gradient direction.
  • Pool size and minibatch trade-offs: Overly small controller budgets introduce severe estimation bias and instability. Shrinking the per-step minibatch to 8 drops TaskAvg to 41.97, while a 128-prompt pool similarly underperforms at 42.22 due to rapid cyclic overfitting. Conversely, scaling beyond our default setting to a 64-prompt minibatch or a 1024-prompt pool brings diminishing marginal returns, reaching 44.94 and 44.61 at the cost of substantially higher runtime overhead or pool size. This confirms that our default configuration sits at a practical sweet spot between gradient stability and computational efficiency.

We sincerely thank Reviewer DNnb for identifying the missing Controller-only calibration cell in Table 3 and for suggesting this clean factorial test. Following your advice, we implemented the exact Controller-only control and repeated the entire calibration and component suite across all three training seeds to rigorously evaluate the contribution of each side.

Part 2: Completing the 2x2 Scoring-Gradient Calibration Matrix Across Three Seeds — Addressing W1 & Q1

To test whether calibrating the candidate side adds value once the reference side is calibrated, we evaluated the Controller-only calibration variant across Seeds 10, 20, and 30 in Qwen2.5 7B-to-1.5B distillation. All four alignment variants in the top block retain the exact same token-gated update loss, Head/Top rank fusion, and weight range, varying only whether the controller and candidate scoring gradients are calibrated. Each per-seed column reports TaskAvg / CatAvg.

SettingSeed 10Seed 20Seed 30TaskAvgdTaskCatAvgdCat
Full Attune44.30 / 41.8144.02 / 41.5444.18 / 41.7544.17±0.14-41.70±0.14-
Controller-only43.85 / 41.2343.67 / 41.0743.76 / 41.1543.76±0.09-0.4141.15±0.08-0.55
Candidate-only43.23 / 40.8443.12 / 40.6943.46 / 41.0543.27±0.17-0.9040.86±0.18-0.84
Uncalibrated43.01 / 40.6442.65 / 40.3243.10 / 40.7242.92±0.24-1.2540.56±0.21-1.14
Group-only43.45 / 41.0543.36 / 40.9843.60 / 41.2143.47±0.12-0.7041.08±0.12-0.62
Token-only42.68 / 40.2842.33 / 40.0342.58 / 40.3242.53±0.18-1.6440.21±0.16-1.49
Vanilla GKD42.24 / 39.8141.91 / 39.4942.23 / 39.8942.13±0.19-2.0439.73±0.21-1.97
Ref: TA-OPD42.68 / 40.3742.61 / 40.3142.58 / 40.3542.62±0.05-1.5440.34±0.03-1.35
  • Added value of candidate calibration beyond controller-only calibration: Directly answering Question 1, Full Attune consistently outperforms Controller-only calibration in every single seed with low variance. Even after the reference direction is calibrated, calibrating the candidate scoring gradients provides a reliable additional gain by preventing outlier tokens within individual candidate groups from distorting their similarity scores.
  • Synergy across the 2x2 calibration grid: Comparing marginal effects across the grid reveals a positive interaction between the two sides. The marginal gain of calibrating the candidate side is larger when the controller is already calibrated than when the controller is uncalibrated. Once noise in the shared reference vector is removed, aligning candidate scoring gradients with the actual token-gated update realizes its full benefit.
  • Shared anchor stabilization and scoring-update mismatch: Controller calibration contributes the larger main effect and cuts cross-seed variance because all eight candidate groups are scored against the same controller gradient at each step. Conversely, leaving both scoring sides uncalibrated falls below Group-only, confirming that applying the token gate to the update loss without calibrating scoring gradients creates a detrimental mismatch between group selection and parameter updating.

We sincerely thank Reviewer DNnb for requesting end-to-end runtime and peak GPU memory measurements alongside a functional stage accounting and comparison with our model-FLOP ratio. Following your suggestion, we benchmarked Vanilla GKD and Attune under our exact main training configuration and audited every execution stage.

Part 3: End-to-End Runtime, Peak Memory, Stage Breakdown, and FLOP Comparison — Addressing W4 & Q4

We measured Vanilla GKD and Attune in Qwen2.5 7B-to-1.5B distillation across the 1,500-step schedule under identical settings: eight NVIDIA RTX6000 Blackwell GPUs, global batch size 64, 32 controller prompts per step, BF16 training with FP32 probe accumulation, maximum response length 8,192, and matched candidate token counts. The table below reports end-to-end runtime, peak single-GPU memory, and the breakdown across rollout generation, forward scoring, probe computation, communication, and parameter updates.

Metric or Functional StageVanilla GKDAttuneRatio / Diff
End-to-End Training Time (1,500 Steps)6.11 h8.84 h1.446x (+2.73 h)
Shortened Attune (1,200) vs. Long GKD (1,900)7.74 h7.07 h0.913x (-0.67 h)
Peak PyTorch Allocated Memory38.08 GB44.92 GB+6.84 GB (+18.0%)
Peak PyTorch Reserved Memory43.32 GB51.66 GB+8.33 GB (+19.2%)
Peak Board-Level NVML Memory44.93 GB53.27 GB+8.34 GB (+18.6%)
Stage 1: Total Rollout Generation2.64 h4.05 h+1.41 h (2.19h + 1.86h)
Stage 2: Forward & Teacher Scoring0.71 h2.83 h+2.12 h (0.86h + 1.97h)
Stage 3: Probe-Gradient Backward0.00 h0.45 h+0.45 h (0.30h + 0.15h)
Stage 4: Probe All-Reduce & Group Rank Fusion0.00 h0.59 h+0.59 h
Stage 5: Final Optimizer Backward, Step & Sync2.76 h0.92 h-1.84 h (barrier shift)
  • Why measured runtime ratio exceeds the 1.242x model-FLOP ratio: While the 1.242x ratio in Appendix A.11 counts pure model matrix multiplications and attention FLOPs, the measured wall-clock ratio is 1.446x, or about 1.45x. This gap stems from three non-MatMul factors: memory-bound autoregressive decoding tail latency across the additional 32 controller rollouts per step, probe backpropagation through Qwen2.5-1.5B's tied output-head and input-embedding parameters which triggers activation-checkpoint recomputations, and inter-worker all-reduce communication of FP32 controller probe gradients at 1.31 GB per rank per step.
  • Cross-worker synchronization shift across stages: Note that Stage 5 drops from 2.76 hours in GKD to 0.92 hours in Attune because of where cross-GPU straggler synchronization occurs. In GKD, workers run rollout generation and forward scoring without a barrier, so faster GPUs wait for long-response straggler GPUs inside DDP's optimizer backward all-reduce. In Attune, workers synchronize earlier during controller scoring and probe all-reduce, shifting cross-rank straggler wait time into Stages 2 and 4 and leaving only pure backward compute in Stage 5.
  • Table 19 already validates Attune under the 1.45x wall-clock ratio: In Table 19, Long GKD at 1,900 steps and shortened Attune at 1,200 steps differ by a 1.58x step and data ratio, exceeding our 1.446x runtime ratio. Thus, 1,200-step Attune takes less actual wall-clock time than 1,900-step Long GKD at 7.07 hours versus 7.74 hours while still outperforming Long GKD by 1.16 TaskAvg and 1.25 CatAvg points.
Reviewer_BKG3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Attune adjusts token and group weights during on-policy distillation. It applies the same token-weighting rule to training and reference losses, then compares their gradients to decide which groups receive more training weight. The paper also uses controlled arithmetic experiments to show that the same supervision can have different benefits across students and training stages. Experiments cover eight benchmarks and two model families. Three-seed runs report TaskAvg gains of 2.04 and 2.08 points over vanilla GKD.

Critical strengths

The main gains repeat across three training seeds in both model families. Comparisons with longer GKD training and shorter Attune training also help address whether the improvement comes mainly from a larger training budget. The controlled arithmetic study matches supervision exposure carefully. The one-step update experiments use shared model states and independent evaluation samples, with uniform and shuffled controls. These are useful checks beyond the final benchmark scores.

Critical weaknesses

The group weights depend on how well the reference pool covers the training tasks. Attune cycles through a fixed pool of 512 prompts, using 32 per step (Appendix A.1). If a task is poorly represented, useful training groups from that task may receive lower weights because their gradients differ from the reference direction. Regenerating responses follows the student’s progress, but does not expand prompt coverage. The bounded weights limit the impact of a poor reference, but do not remove this dependence. The paper does not isolate sensitivity to reference-pool size, composition, or selection. Task-level results with different reference pools would help establish robustness. The key calibration comparison needs repeated runs. Table 3 shows that full Attune improves TaskAvg by 1.29 points over uncalibrated alignment and 1.07 points over candidate-only calibration. However, these comparisons use only seed 10. The three-seed results repeat the full method against GKD and selected baselines, rather than these variants. Repeating the key comparisons would show whether the gain from the paper’s main design is stable. The gains are moderate, and the evaluation is limited to small students. The three-seed TaskAvg gain over the selected strong baseline is 1.54 points in each model family (Table 2). The main students are 1B and 1.5B, with an additional 0.5B experiment. In the Qwen setting, Attune uses an estimated 24.2% more model FLOPs than vanilla GKD (Appendix A.11). The approximate budget comparisons in Table 19 are useful, but actual training time, peak memory, and larger-student results would help establish whether the gains justify the overhead and whether the method scales well.

Questions for authors

How sensitive are the results to the reference pool’s size and task composition? With the training data unchanged, does underrepresenting or excluding a task from the reference pool hurt that task’s performance? Please report task-level results as well as aggregate scores. Do the full-versus-uncalibrated and full-versus-candidate-only gains in Table 3 persist across the three training seeds used in Table 15? What are the actual training time and peak memory compared with GKD for the reported runs? Are results available for a larger student to show whether the gains and overhead persist as model size increases?

5 comments

We sincerely thank Reviewer BKG3 for the constructive comments and for recognizing our multi-seed improvements across both model families, our extended and shortened training-budget controls, our exposure-matched arithmetic study, and our one-step intervention checks. Below, we first address your primary questions regarding the controller reference pool.

Part 1: Sensitivity and Robustness of the Controller Reference Pool — Addressing W1 & Q1

Keeping the 96,000-prompt training stream fixed in Qwen2.5 7B-to-1.5B distillation, we evaluated 10 additional configurations across four dimensions: excluding an entire capability domain from the 512-prompt pool, varying the reference pool size from 128 to 1024 with the per-step minibatch fixed at 32, varying the per-step controller minibatch from 8 to 64 with the pool size fixed at 512, and folding the cyclic controller trajectories directly into the GKD loss instead of using them for group allocation.

SettingIFEvalMMLU-ProTQA-MC2DROPMuSRBBHGSM8KMATH-500TaskAvgdTaskCatAvgdCat
Default Mix 512, MB 3256.3833.7450.607.3237.8346.7170.4351.4044.30-41.81-
w/o Math 51255.6433.4950.077.1837.5746.3669.3750.4043.76-0.5441.29-0.52
w/o Reasoning 51256.1933.4050.227.2937.0445.6570.1351.2043.89-0.4141.46-0.35
w/o Knowledge 51257.3033.4350.297.2637.8346.7070.4351.4044.33+0.0341.92+0.11
Mix 128, MB 3252.5032.2749.437.1435.9844.5366.1149.8042.22-2.0839.74-2.07
Mix 256, MB 3253.6032.9849.977.1636.9045.3768.4650.6043.13-1.1740.58-1.23
Mix 1024, MB 3256.5634.0750.927.2638.2347.1570.8951.8044.61+0.3142.07+0.26
Mini-batch 8, Pool 51253.0531.7149.426.9935.3243.9666.1149.2041.97-2.3339.58-2.23
Mini-batch 16, Pool 51254.3431.8149.467.2635.4544.0266.2649.4042.25-2.0539.96-1.85
Mini-batch 64, Pool 51256.0134.5051.377.2738.6247.7571.8052.2044.94+0.6442.28+0.47
GKD + Direct Ctrl52.8732.0450.187.1536.3845.0366.0348.8042.31-1.9939.85-1.96
Ref: Vanilla GKD53.0531.9850.127.1436.2444.9265.8848.6042.24-2.0639.81-2.00
  • Domain exclusion and task-level specificity: Removing a domain selectively impacts its closely related benchmarks while maintaining strong global performance. Dropping math primarily affects GSM8K and MATH-500 by around 1 point, whereas dropping reasoning mainly affects MuSR and BBH. Even under complete omission of math or reasoning, bounded group weights and token calibration keep TaskAvg at 43.76 and 43.89, remaining well above baselines in Table 1.

  • An unexpected gain from removing knowledge prompts: An intriguing anomaly is that removing knowledge prompts slightly surpasses the default mixture, reaching 44.33 TaskAvg as a notable gain on IFEval outweighs minor dips on MMLU-Pro and TQA-MC2. Candidly, this indicates that simply mirroring the training distribution in the controller is not strictly optimal, and we plan to explore adaptive controller curation strategies in future work. We tentatively attribute this to post-training dynamics: since factual recall is already largely anchored by pre-trained weights and token calibration, shifting controller capacity toward procedural tasks yields a cleaner trajectory-level gradient direction.

  • Pool size and minibatch trade-offs: Our results confirm that overly small controller budgets introduce severe estimation bias and instability. In particular, shrinking the per-step minibatch to 8 drops TaskAvg to 41.97, falling even below Vanilla GKD at 42.24, while a small 128-prompt pool similarly underperforms at 42.22 due to rapid cyclic overfitting. Conversely, scaling beyond our default setting to a 64-prompt minibatch or a 1024-prompt pool brings diminishing marginal returns, reaching 44.94 and 44.61 at the cost of substantially higher runtime overhead or pool size. This may demonstrate that our configuration sits at a practical sweet spot between gradient stability and computational efficiency.

  • Direct supervision control: Also, we evaluated a control that folds the same cyclic controller trajectories directly into the GKD training loss, which reaches only 42.31 TaskAvg and confirms that our gains stem from adaptive gradient allocation rather than extra data exposure.

We sincerely thank Reviewer BKG3 for pointing out the importance of verifying the key calibration and component comparisons in Table 3 across multiple training runs. Following your suggestion, we have completed the full three-seed evaluation for all ablation variants and additionally included a Controller-only calibration variant to provide a complete factorial comparison.

Part 2: Three-Seed Stability of Scoring-Gradient Calibration and Component Gains — Addressing W2 & Q2

To verify whether the calibration and component gains in Table 3 hold across multiple runs, we repeated all ablation variants across Seeds 10, 20, and 30 in Qwen2.5 7B-to-1.5B distillation. In the additional Controller-only variant, only the controller scoring gradient is calibrated while candidate scoring gradients remain uncalibrated. Each per-seed column below reports TaskAvg / CatAvg.

SettingSeed 10Seed 20Seed 30TaskAvgdTaskCatAvgdCat
Full Attune44.30 / 41.8144.02 / 41.5444.18 / 41.7544.17±0.14-41.70±0.14-
Controller-only43.85 / 41.2343.67 / 41.0743.76 / 41.1543.76±0.09-0.4141.15±0.08-0.55
Candidate-only43.23 / 40.8443.12 / 40.6943.46 / 41.0543.27±0.17-0.9040.86±0.18-0.84
Uncalibrated43.01 / 40.6442.65 / 40.3243.10 / 40.7242.92±0.24-1.2540.56±0.21-1.14
Group-only43.45 / 41.0543.36 / 40.9843.60 / 41.2143.47±0.12-0.7041.08±0.12-0.62
Token-only42.68 / 40.2842.33 / 40.0342.58 / 40.3242.53±0.18-1.6440.21±0.16-1.49
Vanilla GKD42.24 / 39.8141.91 / 39.4942.23 / 39.8942.13±0.19-2.0439.73±0.21-1.97
Ref: TA-OPD42.68 / 40.3742.61 / 40.3142.58 / 40.3542.62±0.05-1.5440.34±0.03-1.35
  • Multi-seed stability: Across all three seeds, Full Attune consistently outperforms both Uncalibrated alignment and Candidate-only calibration with low variance. Every individual seed preserves the exact same relative ranking across all variants, confirming that the gains from our main design are robust to training randomness.
  • Scoring-update mismatch: Notably, Uncalibrated alignment underperforms Group-only and doubles the cross-seed variance. Applying the token gate to the final update loss without calibrating the scoring gradients introduces a directional mismatch: candidate groups are selected based on large-gap outlier tokens that are subsequently downweighted during the actual parameter update. Coupling token calibration across both scoring and updating resolves this conflict.
  • Complementary roles of controller and candidate calibration: In the completed factorial grid, calibrating the controller side accounts for a larger share of the gain and substantially reduces cross-seed variance. Because all eight candidate groups are scored against the same shared controller gradient at each step, calibrating the controller stabilizes the common reference direction for the entire batch. Calibrating the candidate side then provides a further consistent gain across all seeds by aligning each group's scored gradient with its actual token-gated update.

We sincerely thank Reviewer BKG3 for asking about the actual training time and peak GPU memory overhead of Attune relative to Vanilla GKD. Following your suggestion, we conducted end-to-end runtime and memory benchmarking under the exact training configuration of our main experiments to provide a transparent system accounting.

Part 3: End-to-End Training Time and Peak GPU Memory — Addressing W3 & Q3

We measured end-to-end training time and peak single-GPU memory for Vanilla GKD and Attune in Qwen2.5 7B-to-1.5B distillation under identical settings: eight NVIDIA RTX6000 Blackwell GPUs, global batch size 64, 32 controller prompts per step, BF16 training with FP32 probe accumulation, and maximum response length 8,192. Below we report the 1,500-step runs alongside the 1,200-step shortened Attune and 1,900-step Long GKD budget comparison runs from Table 19.

Run or Memory MetricStepsWall-Clock TimeTaskAvg / CatAvgRatio vs. GKD
Vanilla GKD1,5006.11 h42.13±0.19 / 39.73±0.211.000x
Long GKD1,9007.74 h42.55±0.18 / 39.98±0.111.267x
Attune (Shortened Run)1,2007.07 h43.71±0.14 / 41.23±0.221.157x
Attune (Full Run)1,5008.84 h44.17±0.14 / 41.70±0.141.446x
Peak PyTorch Allocated Memory-38.08 vs. 44.92 GB-+6.84 GB / +18.0%
Peak PyTorch Reserved Memory-43.32 vs. 51.66 GB-+8.33 GB / +19.2%
Peak Board-Level NVML Memory-44.93 vs. 53.27 GB-+8.34 GB / +18.6%
  • Why wall-clock overhead exceeds model FLOPs: While our measured model-FLOP ratio for matrix multiplications and attention is 1.242x, the measured end-to-end wall-clock ratio for 1,500 steps is 1.446x, or about 1.45x, rising from 6.11 hours to 8.84 hours. This gap comes from non-MatMul system costs: autoregressive decoding tail latency for the 32 controller trajectories per step, probe backpropagation through Qwen2.5-1.5B's tied output-head and input-embedding weights which triggers activation-checkpoint recomputations, and inter-worker all-reduce communication of FP32 controller probe gradients across GPUs.
  • Table 19 already covers the 1.45x runtime gap: In Table 19 of the paper, our budget comparison between Long GKD at 1,900 steps and shortened Attune at 1,200 steps spans a 1.58x step and data ratio, which exceeds the 1.446x wall-clock ratio. Consequently, 1,200-step Attune requires less actual wall-clock training time than 1,900-step Long GKD at 7.07 hours versus 7.74 hours while still improving TaskAvg by 1.16 points and CatAvg by 1.25 points, confirming that the performance gains justify the training overhead.
  • Modest peak memory footprint: Peak single-GPU PyTorch allocated memory rises by 6.84 GB from 38.08 GB to 44.92 GB, and board-level NVML peak memory rises from 44.93 GB to 53.27 GB. Because token and group weights are detached before the final parameter update, both methods run comfortably within standard GPU memory limits, and Attune adds zero overhead at inference time.

Comments on the paper

1 comment

Author Final Remarks

Dear Area Chairs and Reviewers,

We sincerely thank the Area Chairs and reviewers for their thoughtful evaluation and constructive discussion. All reviewers recognized the importance of studying state- and capacity-conditioned supervision utility in on-policy distillation, along with our controlled diagnosis, repeatable cross-family gains, and targeted ablations.

During the rebuttal, we directly addressed the initial concerns regarding validation, design choices, overhead, and presentation. We extended our controlled diagnosis to a second model family and the full held-out probe, and completed multi-seed evaluations for the calibration factorial, adapted reference-gradient baselines (LESS-inspired, GradAlign-style, Ren-style), direct controller supervision, and probe-layer subsets including full-model alignment. We also verified larger-student scaling, controller sensitivity, measured end-to-end runtime and peak memory, dynamic utility tracking over stale rankings and extended update horizons, and the analytical link from diagnosis to method.

We are grateful that Reviewer xUjK, following thorough multi-round technical discussion, found that the added evidence addressed most of their concerns and raised their rating from Weak Reject to Weak Accept. The concerns raised by Reviewers DNnb and BKG3 have likewise been addressed through targeted experiments and analyses. We will incorporate the improved motivation, all new results, and clear limitation statements (single-seed controller sweeps, runtime versus exact FLOP matching, and distinguishing multi-step loss reduction from direct capability prediction) into the revised manuscript.

We respectfully hope that the strengthened evidence and the reviewers' feedback during the discussion will be considered in the final decision.

Best, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.