acceptodds
Withdrawn from ICLR 2027

CHAIN-OF-THOUGHT RELOCATES THE FACTUALITY SIGNAL: A LOCKED-PROTOCOL AUDIT AND CLAIM-PAIRED RECOVERY OF INTERNAL PROBES

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Internal-state factuality probes do not lose their signal under chain-of-thought:the signal moves. Across four model families, under a single locked protocol that varies labeling budget, training style, and feature pipeline orthogonally, we make three measurements. First, the direct-to-CoT transfer loss is small relative to protocol choices: the feature pipeline alone shifts CoT-side scores by an order of magnitude more than the style change itself, and target-style labels close most of the residual. Second, the residual is structured internal geometry: a claim-paired affine map recovers it, shuffle-paired controls collapse to chance, generic unsupervised alignment fails on every family, and label-free logit distillation from the same pairs nearly matches the full map – recovery is a label-transfer phenomenon that runs through the claim-paired internal geometry rather than through surface form. Third,natural false claims are measurable without synthetic corruption, and the labels themselves are auditable: adjudicating every generation shows that automatic labels conflate asserted errors with unasserted truncations, and a strict audit retracts two of our own label regimes as artifacts – yet the detection signal survives label cleanup.Protocol and labels determine what a factuality probe measures; the audit itself is the transferable finding, with claim localization and corpus-specific generalization the open frontiers.

Primary area:
foundation or frontier models, including LLMs
Keywords:
hallucination detection, internal probes, chain-of-thought, linear alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RqoWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies the transfer of internal factuality probing from direct answers to chain-of-thought responses. Under one locked protocol that crosses budget, training style, and feature pipeline orthogonally, it finds that the measured transfer gap depends strongly on the evaluation protocol. It thereby studies target-style supervision and claim-paired affine alignment as recovery methods, and audits automatic factuality labels to distinguish asserted errors from truncations and parser artifacts.

Critical strengths
  1. This paper separates the pipeline main effect from the cross-style effect. It shows that the original 0.804 collapse was manufactured by changing the budget, span rule, and pipeline at once, which is a useful correction method for hallucination detection.
  2. The six-feature surface screen is label-free and catches a parser artifact that cross-validation, held-out agreement, and probe AUROC all missed.
Critical weaknesses
  1. The paper's main contribution is a correction of its own prior claim. It is unclear how much of the contribution stands independently of the original incorrect headline.
  2. Localization becomes a bottleneck. Automated claim localization costs −0.217 AUROC, and the paper explicitly states that localization dominates end-to-end performance.
  3. The paper leans heavily on appendix blocks and cross-references rather than presenting a clean narrative. It is difficult to identify the single main result.
Questions for authors
  1. Gemma's failure is reported but not explained. Do you have a mechanistic hypothesis to explain this phenomenon?
Reviewer_vyGWOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary
Critical strengths
Critical weaknesses

The paper is very hard to read. The text reads as largely LLM generated. Almost every sentence is overloaded with numbers. It reads like a lab notebook rather than a paper: it constantly refers to internal experiment blocks, earlier drafts and the authors own retracted results, and keeps them in the paper as history. I could not follow which results are current and which are outdated.

In its current form I cannot assess the correctness or the significance of the results. I believe the paper needs a full rewrite around one clear story before it can be properly evaluated.

Questions for authors
Reviewer_2qxuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Authors challenge the reports that COT prevents the internal LLM states from being used for factuality prediction--which is commonly used by internal probes. They show that much of the difference between using COT and without it, can be eliminated by making sure that the comparison between the two setups is identical, e.g., textual format, data split, compression model. They also show that the signal when COT is used does not vanish, but needs a bit more work to recover. They specifically show that it can be either recovered by training a probe on a set of examples similar (in style) to those used for probing, or by training a model to transfer the internal signals when COT is used compared to the case that COT is not used. They also manually inspect the probing data and report that in substantial number of cases the outputs of COT was incomplete, rather than incorrect.

Critical strengths
  • the claim is very bold, and is backed by "small" synthetic experiments
  • the experiments are very detailed
  • authors reports accurate results even when the outcome is not what they expected--in case of Gemma or llama
Critical weaknesses
  • The paper is very difficult to read, is not written in everyday English, the language used in the paper is not common in the computer science community. This is perhaps due to the heavy usage of AI tools. Unfortunately I don't think that in the current format can attract many people. Interestingly, this is the first time that I am facing such a problem.
  • The experimental setup is not realistic, in many cases templates that are explained in Section 5.1 are used
  • The main claims of the paper are made based on solid conclusions in small datasets, whereas the results on the larger datasets (e.g., TruthfulQA) are not very convincing enough
Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.