- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes TCLD, which replaces arithmetic averaging of response-token cross-entropies with a power-mean objective. A detached controller estimates upper-tail statistics from a no-gradient observation pass, smooths them, and sets a power below one to reduce the relative influence of high-loss tokens. Experiments compare TCLD with shuffled SFT and sample-ordering curricula on a five-source reasoning mixture, with three matched seeds and two additional model configurations. Fixed-power, observation-window, and ordered-stream diagnostics examine calibration. TCLD improves aggregate accuracy over standard SFT, while its advantage over a competitive fixed power remains statistically unresolved.
Critical strengths
-
The main empirical improvement is supported by controlled, repeated comparisons. Table 1 reports gains of 2.74 Macro and 2.02 Micro percentage points over shuffled SFT on Qwen3-1.7B, using three matched seeds and a common endpoint. Table 4 extends the improvement to Qwen3-0.6B and SmolLM2-1.7B. The resulting evidence supports the practical usefulness of the complete objective and shows that the gain is not confined to one initialization or model configuration.
-
The paper is unusually precise about what its analysis establishes. Equation (3) characterizes relative gradient coefficients, while Sections 3.2–3.3 acknowledge that tail severity neither identifies semantic usefulness nor proves optimal weighting. The fixed-power confidence intervals and computational overhead are also disclosed. This transparency makes the contribution easier to assess and provides a useful starting point for further work on adaptive loss aggregation.
Critical weaknesses
-
The distinctive contribution of dynamic tail calibration is not isolated convincingly. Table 2 improves over fixed p = 0.7 by only 0.79 Macro and 0.34 Micro points, with paired intervals including zero. Appendix L reports a mean controller power near 0.688, making the fixed comparator particularly relevant. Ordered-stream experiments compare against p = 1 rather than a competitive fixed or predetermined schedule. Thus, the evidence establishes useful loss shaping more clearly than a benefit from repeatedly observing and tracking the loss tail.
-
The experiments leave the source of the objective's benefit unresolved. Equation (2) changes both relative token allocation and a batch-dependent common gradient scale, but Section 4.2 explicitly states that these effects are not separated experimentally. The principal baselines mostly change sample order rather than token weights. Consequently, the study does not establish whether the gains require the full controller–power-mean construction or arise from a simpler weighting/scaling intervention. This matters when assessing the method's additional complexity and observation cost.
Questions for authors
-
Clarification: Are existing results available for a fixed power near the realized controller mean, a controller frozen after its first observation, or a replayed power schedule? If not, should the contribution be framed primarily as competitive automatic calibration rather than evidence that online feedback improves accuracy?
-
Clarification: Were any controls separating Eq. (2)'s common scale from its relative token coefficients already run? Such results would materially affect my assessment of which part of the objective provides the improvement.
-
Clarification: Were the tail fractions, smoothing coefficient, and index-to-power mapping fixed before downstream evaluations? Please describe their selection, since avoiding a search over p does not by itself establish independence from downstream tuning of the controller.
Thank you for recognizing the empirical results and the scope of our analysis. We have added controls separating relative token weighting from the common gradient scale, a first-window-frozen comparison, and smoothing-sensitivity experiments. Revisions addressing your comments are highlighted in green in the revised manuscript.
1. Automatic calibration versus continued refresh.
The existing fixed p = 0.7 comparison is close to the realized controller mean of approximately 0.688. We have now added First-window Frozen TCLD, which retains each seed’s first calibrated power throughout training. Across three seeds, Macro/Micro maj@3 accuracy (mean ± sample standard deviation) is:
Online TCLD’s mean gains over fixed p = 0.7 are 0.79 Macro / 0.34 Micro percentage points, with paired confidence intervals including zero. Relative to Frozen, the differences are +0.15 / −0.25 points, with mixed directions across seeds.
These results support competitive automatic calibration of the power parameter from training-loss statistics, without a downstream-validation-accuracy search over p. TCLD uses window-level feedback to recalibrate this parameter as training progresses. The Frozen comparison shows that initial calibration captures most of the observed mixed-pool improvement over RS, with Online and Frozen achieving similar aggregate performance. We have revised the contribution statement and discussion to emphasize automatic calibration and clarify the role of continued refresh. Section 4.3 and Appendix G report these comparisons.
2. Relative token weighting and common gradient scale.
The new three-seed controls give:
Relative-only retains normalized token weights, while Scale-only applies the common gradient factor to uniformly weighted floored CE. Both replay the corresponding seed’s Online power schedule and recompute detached coefficients from their own current batch losses. Appendix B provides the objective definitions.
Relative weighting recovers a substantial part of the improvement, whereas Scale-only remains close to Floor-only CE. The complete objective achieves 1.12 points higher Macro, while Relative-only achieves 0.27 points higher Micro. These findings support an important role for relative token weighting and clarify the trade-off between the two aggregate metrics. Detailed results appear in Appendix G.
3. Controller choices and smoothing sensitivity.
The tail fractions, smoothing coefficient, and index-to-power mapping were used as shared design choices across datasets, without dataset-specific adjustment.
We use fixed tail fractions of 2.5%, 5%, and 7.5% and take the median of the usable Hill estimates to reduce dependence on a single threshold. Median aggregation across thresholds has a related precedent in the weighted log-excesses analysis of Gomes, de Haan, and Henriques; our implementation uses three ordinary Hill estimates. When all three are usable and two agree within an interval, their median remains within that interval despite an atypical third estimate. Exponential smoothing then moderates window-level fluctuations, following the established use of moving averages for noisy statistics, as exemplified by Adam (Kingma and Ba, 2015). Finally, the mapping pₖ = σ(zₖ) converts the smoothed log-tail state monotonically into (0, 1), so a smaller effective tail exponent produces stronger relative downweighting of high-loss tokens. Section 3.2 describes the mechanisms and their motivations.
We evaluated β = 0.5 and β = 0.8 alongside the shared default β = 0.4. The three-seed Macro/Micro summaries are:
All three summaries exceed RS in mean Macro and Micro accuracy. Across the tested settings, mean Macro varies by 0.82 percentage points and mean Micro by 0.13 points, suggesting limited aggregate sensitivity within this mixed pool. The β = 0.5 summary includes a seed42 aggregate combining source evaluations from separate same-configuration runs, as detailed in Appendix L. The sensitivity experiments retain the original default; Appendix L provides the results and evaluation conditions.