- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies the logits-based confidence calibration of LLMs across the post-training pipeline, in which a model is first supervised fine-tuned (SFT) and then further trained with DPO and RLVR. Label smoothing (LS) during SFT is a well-known, cheap remedy for the overconfidence that SFT introduces. However, the authors observe that LS-tuned models fail to improve during subsequent DPO and RLVR training. They attribute this to the LS regularizer lowering the ground-truth logit even for tokens the model mispredicts. To address this, they propose Holistic Smoothing Hinges (HoSHi), an SFT objective that is identical to LS when the target token is ranked first, and otherwise shifts the penalty from the target logit to the logits ranked above it, spread uniformly among them. They motivate the uniform penalty with a lower bound on the LM head’s output entropy under a bounded logit norm, arguing that probability mass among the wrongly top-ranked tokens is close to flat. Across three base models, two SFT datasets and three multiple-choice benchmarks, HoSHi matches LS calibration after SFT. On Llama3-8B further trained with DPO and RLVR, it remains trainable, stays better calibrated than the no-smoothing baseline, and preserves downstream performance on 12 benchmarks better than MaxSup.
Critical strengths
- Well-motivated and practically relevant question. The paper studies the logits-based confidence calibration across the full post-training pipeline (SFT → DPO → RLVR) and asks whether an SFT-time calibration fix interferes with later alignment. Since label smoothing is well known to be cheap and is recommended for calibrating instruction-tuned LLMs, the observation that it may impair subsequent DPO and RLVR training (Fig. 3) is surprising and directly relevant to practitioners.
- A simple, drop-in method with clear positioning. HoSHi uses the same single hyperparameter as LS, adds only O(K) overhead, and reduces exactly to LS whenever the target token is ranked first.
Critical weaknesses
- Most of the theoretical analysis restates prior results without attribution. Prop. 3.2 and Lemma 4.1 have already been proposed in Huang et al. (ICML 2025), and Thm 4.2 is exactly the same as Thm 4.2 in Huang et al. (ICML 2025); the appendix credits only Zhang & Sabuncu (2020) and Zhai et al. (2023) for proof techniques, which suggests the results themselves are new. Please explicitly attribute these results and indicate which parts of the analysis are new; as written, the theoretical contribution claimed in C1 and C2 largely reduces to the HoSHi objective itself.
- Several reported results coincide exactly with previously published numbers. For example, in Table 1, every ECE/RMS value reported for HoSHi is identical to the value reported for label smoothing (β = 0.1) in Huang et al. (ICML 2025, Table 1) under the same models, SFT datasets, and β, while the “Smoothing” row in this paper reports different and worse values. Separately from prior work, the HoSHi SFT results for the same Alpaca/Llama3-8B setting are inconsistent: Table 9 reports ECE 0.093, while Table 11 reports ECE 0.073, which matches the LS row in Table 9. Please clarify the source of these numbers and provide per-run results (ideally with seeds and training logs). Until then, Table 1 cannot be taken as evidence that HoSHi matches LS in calibration.
Questions for authors
- Provenance of Table 1. Could you clarify whether the HoSHi results in Table 1 were obtained from new training runs? Their ECE/RMS values coincide exactly with the label-smoothing (β = 0.1) values reported in Huang et al. (ICML 2025, Table 1), while the “Smoothing” row in this paper reports different values.
- Inconsistent HoSHi results for the same setting. For SFT on Alpaca with Llama3-8B, which HoSHi result is correct: Table 9 (ECE 0.093, RMS 0.092) or Table 11 (ECE 0.073, RMS 0.094)? If Table 9 is correct, does the claim in Appendix G.3 that HoSHi without tuning matches Reference + DACA still hold at SFT stage?
- Novelty of the theory. Which of Def. 3.1, Prop. 3.2, Prop. 3.3, Lemma 4.1, and Thm 4.2 do you regard as new relative to Huang et al. (2025), Liu et al. (2022), and Zhai et al. (2023)? How do Lemma 3.4, Cor. 3.5, and the error-amplification argument differ from the decomposition in MaxSup (Zhou et al., 2025)?