- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes NuAlign, a two-stage method for aligning LLMs used as evaluators. Stage I performs task-aware supervised fine-tuning. Stage II introduces Numerical Distance Perception (NDP), where the full-score deviation is used to modulate the likelihood gradient of gold score tokens. The method is evaluated across three languages, five LLM backbones, and several scoring datasets.
Critical strengths
- The method is simple and practical, and the evaluation covers three datasets in different languages and five diverse LLM backbones.
- The adaptive cross-weighting rule avoids expensive grid searches for the mixing coefficient.
Critical weaknesses
The theoretical analysis does not sufficiently establish how the proposed reweighting addresses the limitations motivating the paper. NDP introduces numerical distance through a scalar weight on gold-token likelihood gradients, but does not directly impose distance-dependent corrections among alternative score tokens. Moreover, distinguishing equal-mean distributions may already follow from ordinary likelihood supervision. The paper therefore needs clearer evidence that its gains arise specifically from the proposed numerical-distance mechanism rather than generic error-based reweighting.
Questions for authors
-
How does NuAlign compare with simple numerical-error-weighted CE under matched backbones and training budgets?
-
Could you clarify its relationship to REAL (Zhang et al., ICML 2026), and provide a comparison or explain why the settings are not directly comparable?
Zhang et al., "REAL: Regression-Aware Reinforcement Learning for LLM-as-a-Judge," ICML 2026.
Thank you very much for your valuable feedback. Next, I will first address the issues you raised in the “Critical Defects” section; we have broken down your feedback into two main issues and will respond to them one by one.
W1: NDP introduces numerical distance by applying a scalar weight to the likelihood gradient of the gold label, but does not directly apply distance-based corrections between candidate score labels.
Thank you for pointing out the distinction between NDP and distance-based corrections for candidate score labels. We agree that NDP does not apply numerical distance penalties separately to each candidate score label at the same prediction position. According to Equation (9), for valid supervision positions, the NDP gradient is $d_i \nabla_\theta \ell_{i,t}^{\text{CE}} $, so it preserves the local direction of the original likelihood gradient and modulates the gradient contribution for that sample using the bias between the reconstructed full score and the human-assigned score. Our design goal is to incorporate the severity of the full-score error into likelihood optimization, rather than establishing new distance constraints in the candidate score space. For multi-token scoring, the full-score deviation can distinguish between predictions with similar token-level losses but differing degrees of numerical error severity. Therefore, “numerical distance-aware” in this paper specifically refers to the dependence of the sample-level optimization signal on the full-score error; it does not imply that corrections for each error candidate are ranked according to their numerical distance. We will further revise the method’s motivation and related descriptions to clarify this scope of application.
W2: Distinguishing between distributions with equal means may already be achieved by ordinary likelihood supervision.
We are very grateful to the reviewers for this point. We agree that when two distributions with equal means assign different probabilities to the true score sequence, standard likelihood supervision may already be capable of distinguishing between them. In fact, §3.4 of the original manuscript (page 4, lines 194–198) explicitly states that TRACT’s additional likelihood supervision can distinguish between distributions that assign different probabilities to the true sequence. We have also further demonstrated in Appendix A.3.1 that NuAlign possesses a unique ability to distinguish between distributions with equal means. NuAlign’s contribution lies in utilizing the bias in reconstructing whole scores to modulate the likelihood updates for the true scoring sequence, rather than treating the discriminative capability already present in standard likelihood supervision as a new contribution. We will revise the relevant description to avoid ambiguity.