- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether LLMs notice, carry, and report errors in multi-step arithmetic derivations. Using a controlled setup with one planted inconsistent step, it finds that next-token surprisal reliably detects the error, and in several models the hidden state at the answer position still encodes it. However, the one-word Yes/No verdict uses little of this evidence, revealing a detection–report dissociation. Consistent-twin and surface controls rule out simple number artifacts, and prompting step-by-step recomputation improves rejection. On ProcessBench, a training-free surprisal localizer from a 1.5B model is competitive with early process reward models, while verbal localization remains weak. The main bottleneck is often the readout/report stage, not internal error detection.
Critical strengths
1.The Notice-Carry-Report three-stage analysis framework proposed in this paper breaks down the language model error detection process into three separate stages: noticing, retaining, and reporting. This provides a clear layered research approach to explore why error detection fails and offers a useful methodological reference for studies in this area. 2. The paper uses paired samples and a consistent twin-control experimental design, strictly controlling confounding variables, allowing for accurate separation of the effects of 'arithmetic logic inconsistency' and 'rarity of numbers' on surprise in an artificial setting, with high internal validity in the controlled experiment.
Critical weaknesses
1.The research motivation and experimental scenario are mismatched, and the narrative loop is incomplete: The introduction starts from the research point of model self-verification failure in real scenarios and the field consensus that 'verification is easier than generation', but the core mechanism experiments are all based on highly controlled environments with artificially implanted single-step arithmetic errors, which are fundamentally different in type and distribution from naturally generated model errors, limiting external validity. The main scientific question of whether verification failure comes from perception failure or output reporting failure is not explicitly presented in the introduction, and the grand hypothesis raised at the start is neither verified nor addressed, causing a narrative disconnect and unclear research positioning. 2.The experimental design is biased, and the evidence for generalising the core conclusions is insufficient: Artificially constructed local numerical errors naturally align with the detection characteristics of next-token surprisal, so the positive results obtained in this scenario are not very convincing and cannot fully prove the model's general internal error perception ability. The most radical 'perception-report dissociation' mechanism conclusion only holds on artificial datasets; Chapter 8 only verifies the error localisation effect of surprisal and does not reproduce the core dissociation phenomenon in natural error scenarios, seriously lacking generalisation support. 3. The boundary between the model's inherent ability and the contribution of the reading method is unclear: The ProcessBench experiments in Chapter 8 fail to clearly separate the model's maths reasoning ability from the independent contribution of the surprisal reading method, making it unclear whether the performance comes from the surprisal method itself or is just another way of reading the internal error signal of the model, leaving the incremental contribution of the method undefined.
Questions for authors
Please add ablation control experiments to clarify the independent value of digit-token surprisal. On the same model and test set, provide a horizontal comparison of error localization performance across the following approaches: 1.Explicit verbal output of the error step number by the model 2.Average surprisal across all tokens 3.Average surprisal across digit tokens only 4.Other implicit uncertainty metrics (e.g., token entropy, local perplexity) This will demonstrate the gain and necessity of digit-token surprisal relative to other readout methods. Please explicitly define the positioning of the Section 8 experiments in the main text. Clarify that this section serves as an engineering application demonstration of the controlled mechanism findings, rather than a generalizability validation of the core perception-report dissociation phenomenon. Clearly delineate the scope of applicability of the conclusions to avoid readers misinterpreting the scope of contribution.