- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper shows that correctness probes can exploit the association between low response entropy and correctness, causing poor discrimination between confidently wrong and uncertain-but-correct answers. It diagnoses this behavior through entropy-conditioned evaluation, matched training, and probe geometry, then proposes group-robust and routed probes to improve performance on the conflicting groups.
Critical strengths
Clear scientific claim: probes trained directly on correctness labels can inherit the same entropy dependence.
Broad empirical coverage: Text, visual QA, and medical-image tasks support the relevance of the phenomenon
Simple remedies.
Critical weaknesses
-
The novelty of the finding needs to be clearer. The existence of low-entropy incorrect answers and high-entropy correct answers is already recognized in prior work. The potentially new contribution is showing that correctness probes learn to rely on this association, rather than simply documenting these cases again. The paper should distinguish these two claims more clearly.
-
The remedies have limited novelty and incomplete theoretical justification. Group reweighting, group DRO, and routing are established techniques. For the routed method, removing band-specific means and rescaling scores does not generally guarantee that predictions become independent of entropy given correctness. The paper should clarify the assumptions behind this interpretation and distinguish empirical improvements in subgroup metrics from a rigorous guarantee of shortcut removal.
-
Suppressing entropy information is not clearly justified as a general detection objective. Entropy can provide useful information about correctness, even though it is imperfect. The reported reduction in pooled AUROC suggests a trade-off between subgroup robustness and overall discrimination. Calibration, selective prediction, or distribution-shift experiments are needed to show when reducing entropy dependence improves practical detection.
Questions for authors
See weaknesses
5 comments
Continuation
- The remedies have limited novelty and incomplete theoretical justification. Group reweighting, group DRO, and routing are established techniques. For the routed method, removing band-specific means and rescaling scores does not guarantee predictions become independent of entropy given correctness. The paper should clarify the assumptions and distinguish empirical improvements: Our draft submission explicitly acknowledges this: Section 6.1 attributes the worst-group optimization procedure to Sagawa et al. (2020), and Section 8 connects the approach to established shortcut-learning methods. We employ these techniques as targeted interventions on the four correctness-entropy groups, rather than claiming a new general-purpose reweighting or DRO algorithm. Their improvements on entropy-misaligned comparisons, together with the controlled entropy-matching and probe-direction analyses, provide converging evidence supporting our diagnosis of learned entropy shortcut.
We also acknowledge that expert routing is an established technique for various applications, for example, mixture-of-experts approaches for spurious correlation [4] and robustness [5]. Our contribution is the entropy-specific post-hoc construction for correctness detection for LLM probes: routing examples to high- or low-sampling-entropy correctness probes and normalizing their score streams before pooling. This directly targets the cross-band ranking asymmetry identified by our analysis. We did not find any paper that uses a routing method for the same. We will clarify that the methodological contribution lies in this problem-specific design and its evaluation, while the central contribution is the systematic diagnosis of reliance on entropy in correctness-supervised probes.
The theoretical result relates to the cancellation of an additive band-dependent term under the assumptions of Proposition A.3, rather than a general guarantee of conditional independence. Proposition A.3 (Appendix A.6.1) assumes that the class-conditional correctness-evidence likelihood ratio is linear and shared across entropy bands. Under this assumption, band-specific correctness prevalence appears as an additive logit offset, which cancels under the centering operation in Eq. 45.
- Suppressing entropy information is not clearly justified. Entropy can provide useful information about correctness, even though it is imperfect. The reported reduction in pooled AUROC suggests a trade-off. Calibration, selective prediction, or distribution-shift experiments to show reducing entropy dependence improves practical detection: We agree that entropy can provide useful information about correctness in certain scenarios, and suppressing it indiscriminately is not a desirable general objective. However, when an LLM probe is trained to predict correctness, it does not serve the purpose when the usual entropy-correctness association reverses. We highlight that this creates a failure mode for the probe due to entropy-shortcut. Our paper unveils this phenomenon, which gets concealed in overall AUC. We argue the seriousness of the shortcut and attempt to mitigate it. Ideally a probe should be devoid of entropy-shortcut and perform uniformly across all subgroup.
Moreover, to understand in what situation the shortcut leads to degradation in the overall CvI AUC, we conducted an additional test-time entropy-composition-shift experiment. We rearrange test samples while preserving overall correctness prevalence and within-group score distributions. Specifically, and control the fractions of high-entropy correct and low-entropy incorrect predictions, respectively.
Table 7: Correctness-detection performance under natural and shifted distributions.
| Dataset | Method | Natural CvI | ES-xAUC | Shifted CvI (80% conflict; ) |
|---|---|---|---|---|
| Text QA | Standard probe | 0.865 | 0.622 | 0.697 |
| Routed Learning | 0.766 | 0.737 | 0.802 | |
| Image QA | Standard probe | 0.819 | 0.552 | 0.639 |
| Routed Learning | 0.749 | 0.715 | 0.742 |
The standard probe's pooled AUROC declines substantially as the entropy–correctness association changes, while Routed Learning remains comparatively stable. In many practical applications, entropy-correctness correlation may not be controllable at deployment scenarios, and hence this may make the probes vulnerable.
[4] Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks. ICML 2023.
[5] Not Eliminate but Aggregate: Post-Hoc Control over Mixture-of-Experts to Address Shortcut Shifts in Natural Language Understanding. TACL 2024.
Gentle reminder
Dear reviewer NzWS,
We have attempted to address the valuable suggestions you pointed out. This is a gentle reminder in case you have any further questions, and also to reconsider your scores.
Thanks,
The authors
Addressing the concerns
We thank the reviewer NzWS for reviewing the draft and providing valuable comments and suggestions.
Clear scientific claim: probes trained directly on correctness can inherit entropy dependence, Broad empirical coverage: Text, visual QA, and medical-image tasks, Simple remedies: Thank you for appreciating our problem setup and the findings.
The novelty needs to be clearer. The existence of low-entropy incorrect answers and high-entropy correct answers is already recognized. New contribution is showing that correctness probes learn to rely on this association, rather than simply documenting these cases again: We emphasize the existence of low entropy mispredictions and benchmarking, while showing that the LLM correctness probes use entropy shortcuts. The additional contributions are in ways that allow mitigation of this problem.
Title (Page 1): NOVEL Contributions - ENTROPY SHORTCUT IN LLM CORRECTNESS PROBES
Abstract (Page 1): In contrast to prior works, we show that a sizable fraction of mispredictions exhibit low sampling entropy. More importantly, standard linear correctness probes fail to distinguish these low-entropy mispredictions from high-entropy correct predictions.
Prior Works (Page 9): .. Recent works point out the existence of low-entropy mispredictions and that consistency is not equivalent to correctness. They show that multiple detector families deteriorate on such errors Simhi et al. (2025); Gekhman et al. (2025); Tan et al. (2025). These studies establish this as a failure mode, but neither formalize it nor examine whether correctness probes themselves exploit entropy as a shortcut.
Regarding the novelty and importance of benchmarking the entropy-correctness subgroups (Section 4): Benchmarking (Section 4) in our paper adds additional findings. The two papers we find closest to this are:
We did not find any prior work on the four entropy-correctness association subgroups which are benchmarked on both text-only and multimodal tasks. Our work benchmarks both text-only (Trivia-QA, NQ, SciQ) and multimodal datasets (OKVQA, VQAv2, AdVQA, RSNA, Camelyon) totaling ~165k samples across four LLMs and five multimodal LLMs.
In terms of novelty, the paper's central contribution is the finding that LLM probes use entropy hacking and take a shortcut to predict correctness, which is undesirable. The paper uses (a) Joint learning using DRO and (b) routed learning as shortcut mitigation approaches. The paper clearly mentions DRO as a prior work developed for shortcut mitigation, and uses it as is for our problem. Routing-based networks are common; however, they have not been used for entropy-shortcut mitigation, to the best of our knowledge.