- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper tests methods to measure LLM uncertainty using ambiguous and misleading cues. They test verbal reports, answer probabilities, opt-out behavior, reasoning traces, and hidden-state measures in GPT-OSS-20B and Qwen3.5-27B. Several indicators distinguish ambiguous from concentrated evidence; their effectiveness depends on the model and reasoning setting. However, the work is mainly a measurement study; it does not establish a new method or directly contribute to internal belief-state interpretability.
Critical strengths
A key strength is the controlled contrast between ambiguous and confidently misleading evidence, which addresses the common conflation of uncertainty with correctness. The multi-method comparison on shared histories also provides a useful map of which observable indicators converge and which depend on the model or reasoning setting.
Critical weaknesses
-
The paper doesn't have clear definitions of the problem discussed. Notions are used without clear definitions. It is very difficult to follow when reading the text. For example, do and both indicate the cue? It is very confusing to mix two types of notions. I think the paper needs to clearly discuss the problem setup and motivate the experimental design.
-
The contribution is limited; What I understand of the paper is that the strongest idea is the misleading-cue-controlled experiments. The model that can be confident but wrong is well known. But the paper's contribution didn't provide a novel method to mitigate the problem.
-
The claim of an internal uncertainty measure is quite misleading; the line of work to study internal uncertainty aims to decode the model's internal representation to estimate uncertainty. Although the paper uses a trained probe and Jacobian lens, the contribution to internal uncertainty interpretability is limited.
Questions for authors
It seems that you sampled a cue from the 15 trials and observed the probability of the outcome as RED, and the probability is then used to measure M1 and M2. How do the numbers of observations per cue differ across the veridical, ambiguous, and misleading conditions? Could the authors report these distributions and repeat the main indicator comparisons with balanced cue sampling, or with a minimum number of observations per cue?
2 comments
Author response (2/2)
Weakness 3: "no internal uncertainty interpretability"
As stated in Section 2, "internal uncertainty" refers to the model’s assessment, based on the information acquired so far, of the extent to which that information supports a stable representation of an unknown state of the world. "Internal" emphasises that this assessment is generated within the model, that it cannot be directly observed, and that we therefore treat it as a latent construct. This does not mean that it must be measured by decoding internal model representations (just like measuring a person’s uncertainty doesn’t require decoding their brain activity).
That said, decoding uncertainty from internal representations is one of the measurement approaches considered in our paper, alongside verbal reports, abstention behaviour, output distributions, and reasoning traces. Some of the specific probe-based, Jacobian lens-based, and SAE-based methods used to measure internal uncertainty are, to our knowledge, novel, and (though this is irrelevant to the objective assessment of the contribution) we had high hopes for them!
Our negative empirical results on using these interpretability methods are also new and, as discussed in the previous answer, they do inform work on internal representations. To recap, applying our measurement validation criteria to probe-, Jacobian-lens and SAE readouts, we found that some fail validation altogether (Jacobian lens; the Qwen3.5-27B SAE score on held-out data) and that those that pass do not converge with output-based indicators within conditions (r ≤ .16) and do not predict subsequent sampling (see 1.2). We believe this is a valuable finding for researchers working on "internal uncertainty interpretability". We will mention these contributions more explicitly in the introduction.
Question: distribution of observations
Cue sampling in our design is balanced. The number of observations per cue is nearly identical across the three conditions. Each cue is observed about 15/4 ≈ 3.75 times on average across conditions:
| Condition | A | B | C | D |
|---|---|---|---|---|
| Veridical | 3.78 | 3.70 | 3.78 | 3.74 |
| Ambiguous | 3.86 | 3.67 | 3.65 | 3.82 |
| Misleading | 3.84 | 3.68 | 3.74 | 3.74 |
Author response (1/2)
We thank the reviewer for their comments and questions, which we address below. We believe that some of these may arise from a slight misunderstanding of our research objectives and research contributions. We hope that our responses clarify these points and we will also address them in the revised version of the paper. Given these clarifications, we feel that the negative ratings may be harsher than warranted, and hope the reviewer will reconsider them.
Weakness 1: "lack of clarity"
Weakness 2: "no new method"
We thank the reviewer for the comments on this, but we are concerned they may arise from a slight misunderstanding of the research objectives of our paper and perhaps also from how explicitly we presented our findings. We will try to clarify.
Our core question is a measurement question: which observable signals can serve as valid indicators of internal uncertainty; do different measurement approaches give consistent estimates; and how do estimates of internal uncertainty relate to behaviour? Demonstrating or mitigating confident errors are related research avenues, both interesting and valuable, but they are not what our paper is about. Similarly, while our study is motivated by the (known) fact that models can be confident but wrong, that is not, nor do we want to claim this known fact as our discovery.
We do want to underline that, although we do not propose a mitigation method, the measurement validation protocol is itself a methodological contribution. We propose a procedure grounded in psychometric principles (1) to validate uncertainty indicators (some of which were never used in this way before) without using answer accuracy as the validation criterion—a procedure that applies to both behavioural, output-level and representation-level signals—and (2) to aggregate the validated indicators into composite indices.
As for the comment on misleading-cue-controlled experiments: this is one of the empirical findings we obtain by applying the measurement instrument validation procedure described above—other findings being, e.g., that only a subset of existing candidate measures satisfy the validation criteria, and that some indicators are sensitive to the specific language model and reasoning setting. These empirical findings are a contribution of the paper as well.
In particular, we would like to highlight the finding that, after removing condition-level mean differences, the representation-based indicators we tested do not show reliable convergence with output-based indicators and exhibit weak associations with subsequent sampling behaviour. We believe this has direct implications for interpretability research and method development.
Overall, we take the criticism that the objectives and contributions of the paper, including the impact on interpretability research, could be stated more explicitly; we will do so in the introduction of the revised paper.