- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates whether language models can identify which claims in their input context deserve reconsideration before producing an answer. The central observation is that language models often rely on false premises even when their own internal representations or explicit truth-probability estimates can distinguish those premises from correct claims. To address this discrepancy, the authors propose a claim-level re-checking framework that separates monitoring from control. Monitoring is implemented through either a learned probe over hidden representations or the model's P(True) estimate for individual claims. Control is implemented by appending a targeted note that explicitly identifies a suspicious claim and instructs the model to verify it before answering. A threshold-based policy determines whether to trigger this intervention. The authors evaluate the approach on synthetic false-premise tasks, position-balanced and context-size-controlled datasets, outdated-figure tickets with tool access, and several public benchmarks involving retrieval conflicts and sycophancy. The results show substantial improvements in correcting false-premise answers in controlled synthetic settings. In particular, on the 24-claim benchmark, the proposed gates achieve 58–87% repair rates, compared with 28–50% for the single-prompt verify-all baseline, while requiring fewer additional generated tokens. However, the improvements are considerably less pronounced on public benchmarks, particularly when misleading claims have already been adopted by the model.
Critical strengths
Experimental rigor and validation: Several controls support the naming effect. The paper balances error positions, compares against random and neighboring true claims, and holds gate firing fixed while randomizing the selected knowledge claim. The latter control loses 17–20 percentage points on two models (p. 49). These results provide evidence that choosing the claim matters. The no-type-filter and factored CoVe experiments are also valuable.
Technical novelty and innovation: Connecting input-claim scores to a targeted prompt is a useful extension of work on error detection and correction. The paper tests whether the signal can change the final answer, rather than stopping at probe accuracy.
Critical weaknesses
W1. The methodological contribution is limited. The method combines existing truth scores with a threshold and a targeted prompt. Equation (1) fixes most decision terms, while estimating them performs worse (p. 43). A standard logistic probe also achieves higher localization accuracy among all six claims on two models, but is not tested in the complete pipeline (p. 36), leaving the value of the proposed probe unclear.
W2. Selective checking does not show a consistent accuracy–cost advantage. With 10% false-premise inputs, Qwen's two gates achieve 74%/70% overall accuracy, versus 88%/91% for the unnamed note/verify-all (Table 24). At 24 claims, the gates also add 585/1,055 total tokens versus 344 for verify-all, despite generating fewer tokens (Table 33). These results establish neither consistently better overall accuracy nor a measured runtime advantage.
W3. Incorrect factual labels weaken the evaluation of unnecessary checking. Appendix C.3 acknowledges that some supposedly true claims are false or contestable. Removing nine answer-dependent pairs preserves much of the repair gain, but leaves disputed distractors unaudited. Consequently, gate firing on these nominally correct contexts cannot reliably measure unnecessary checking; the factual labels need independent validation.
W4. The main policy's transfer remains unverified. Algorithm 1 is tested only on the synthetic context-size set; public experiments change the gating and selection rules and show limited gains over the unnamed note with more false alarms (p. 8, Table 26). The headline tool-use comparison also supplies the outdated claim's location manually (p. 5). These results do not establish the same automatic policy beyond the synthetic setting.
Questions for authors
-
Does unchanged Algorithm 1, calibrated on a separate development set, outperform the unnamed note on public text at comparable false-alarm rates?
-
Under identical calculation instructions, how does gating compare with always naming the highest-scoring claim in overall accuracy and latency, including scoring costs?
-
Do the results hold after independently validating and correcting all factual labels, including distractors?
-
How does a standard logistic probe perform in the complete checking pipeline under the same calibration procedure?
-
Can you release the model-running and probe-training code during review and independently audit the current public-benchmark false-alarm labels?