- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
NormCaST addresses norm shrinkage in soft token embeddings by rescaling them to the probability-weighted average norm of their candidates. It preserves direction and requires no training. The paper reports average accuracy gains across four models, with additional coding and SwiReasoning evaluations.
Critical strengths
-
The paper connects a valid norm bound to the shrinkage observed in the examined Qwen3 trajectories and tests a simple correction (Eq. 8 and Figs. 1-2). Calibration preserves direction and adds no model forward passes (Eq. 9 and Appendix B2). All three tested norm targets improve reported average accuracy (Table 2).
-
I lean toward weak acceptance for this useful correction, but uncertainty about the reliability of the gains make me keep the recommendation borderline. All eight model/variant pairs improve in average accuracy over their uncalibrated counterparts, with broadly comparable generation lengths (Table 1). Calibrated Gumbel exceeds CoT’s average on 3 of 4 models, although calibrated Vanilla does so only on Qwen3-8B. SwiReasoning integration also improves average accuracy, though its calibrated variants remain longer than CoT (Appendix D2 and Table 9).
Critical weaknesses
- The paper does not quantify uncertainty in the gains, which limits my confidence in the practical benefit and target ranking. Each problem has three samples, and each AIME set contains only 30 problems. For Qwen3-8B Gumbel, AIME accounts for about 91% of the 2.95-point average accuracy gain (Table 1). The small differences between targets and small coding gains are also uncertain with both calibrated variants decline on LiveCodeBench (Tables 2-3).
- The claim that calibration recovers semantic expressiveness goes beyond the evidence and should be narrowed. Norm analysis and rescaling experiments support the relevance of input scale, but they do not measure semantic recovery.
Questions for authors
Could you use the existing outcomes to report uncertainty for the paired calibration gains and target comparisons, accounting for repeated samples of each problem? If the main gains are unstable, the case for acceptance would be weaker. Uncertainty about the differences between targets would mainly weaken the claim that the weighted target is better.
Thanks, you completely addressed my concerns. I have updated my rating