- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors of this paper investigate whether using both model outputs and internal routing information improves a probe's ability to predict whether a model is correct. They argue that although routing information can improve performance, this does not necessarily mean it provides any new information beyond what is already available from the outputs. Instead, they say that these improvements might come from how the probes are trained or how easily they can learn from the additional features. The authors investigate this using a benchmark where routing is known to provide no additional information, and evaluate how checkpoint selection and different testing approaches affect the results.
The authors find that the choice of checkpoint selection can produce misleading improvements, which disappear when selecting checkpoints using validation log loss rather than accuracy. They also compare this against conditional permutation testing, which is better at detecting small additional signals in their experiments. Finally, they evaluate real correctness labels across several vision transformer models, finding some evidence of conditional dependence in DeiT models, although none passes their additional noise criterion.
Critical strengths
-
The paper raised a useful point about how improvements in probe performance can be incorrectly interpreted as evidence of additional information. Although this is not a new observation, investigating how it affects routing-based uncertainty probes is worthwhile.
-
The experimental setup is reasonable, particularly the benchmark where routing is known to provide no additional information.
Critical weaknesses
-
First, I find the contribution somewhat limited. The distinction between improved probe performance and additional information has already been talked about in previous work, and the authors reuse conditional permutation testing. The empirical audit, particularly the controlled checkpoint selection experiments, is useful, but I am not convinced that the resulting insights are anything more .
-
The conditional testing results are not particularly conclusive. The tests depend on an estimated conditional distribution of routing given outputs, and four of the twelve positive R1 comparisons don't remain positive under the alternative models. Although the authors investigate the power of individual tests, they do not establish the sensitivity of the complete R1/R2 decision rules. This makes it difficult to interpret the real-label findings, particularly the lack of R2 detections, without knowing how sensitive these rules actually are.
-
I also found the presentation of the paper unnecessarily difficult to follow in places, with several important experimental results placed in the appendices rather than the main paper, and the subsections titles are often overly long and awkward.
Questions for authors
-
(Relating to W1): Given that the distinction between probe performance and additional information is already established, what do the authors consider the main novel insight of this work?
-
(Relating to W2): Could the authors evaluate the power of the complete R1/R2 decision rules at confident can we be in the reported findings without establishing the sensitivity of these rules?