acceptodds
Under review as a conference paper at ICLR 2027

Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancy

Abstract

Chain-of-thought (CoT) reasoning improves the problem-solving ability of large language models (LLMs), but generated reasoning traces may not faithfully reflect the model's actual decision process. Existing CoT unfaithfulness detectors mainly rely on external signals from generated rationales, such as textual plausibility or answer consistency, while overlooking evidence from the model's internal computation. Although recent circuit tracing methods provide a way to obtain model-internal evidence by tracing how information flows through model components during reasoning, constructing full reasoning circuits for long CoTs is costly and difficult to scale. To address these challenges, we propose Circuit-guided Internal-External Discrepancy Scorer (CIE-Scorer), a framework for instance-level CoT unfaithfulness detection. The key idea is that faithful reasoning traces should align with the model's computational process, whereas unfaithful traces may diverge from it. CIE-Scorer efficiently traces compact sentence-level circuits from informative reasoning tokens, constructs internal and external reasoning graphs, and measures their discrepancy using Fused Gromov–Wasserstein distance. Experiments on four datasets from FaithCoT-Bench show that CIE-Scorer achieves state-of-the-art performance while reducing the cost of circuit construction, demonstrating the effectiveness of combining mechanistic interpretability signals with external reasoning traces for CoT unfaithfulness detection.

Primary area:
interpretability and explainable AI
Keywords:
large language models, interpretability, unfaithful detection, circuit tracing

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_m5fSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CIE-SCORER, a framework for detecting unfaithful Chain-of-Thought (CoT) reasoning by examining discrepancies between generated reasoning traces and the model's internal computations. Specifically, the method efficiently constructs compact sentence-level circuits from informative reasoning tokens, builds internal and external reasoning graphs, and measures their discrepancies using Fused Gromov–Wasserstein distance. Experiments on four datasets from FaithCoT-Bench demonstrate that CIE-SCORER achieves state-of-the-art performance while reducing the computational cost of circuit construction.

Critical strengths
  1. Existing CoT unfaithfulness detection methods often rely on generated text or output behavior. This paper introduces information from the model's internal computations, offering a new perspective on the relationship between generated reasoning traces and the underlying computational mechanisms of language models.
  2. The paper combines circuit tracing, token selection, graph neural networks, and Fused Gromov–Wasserstein (FGW) distance to represent internal computations and external reasoning traces within a unified graph-based framework, providing a new approach to CoT unfaithfulness detection.
Critical weaknesses
  1. The authors assume that internal computational circuits can provide evidence of the model's actual reasoning process and use them to assess the faithfulness of external CoT traces. Although the authors demonstrate the classification capability of internal circuit features, the ability to predict unfaithfulness labels does not necessarily indicate that these circuits capture the model's actual decision-making mechanisms.
  2. When constructing sentence-level circuits, the authors use only the current sentence as the activation context, without tracing the original question or the preceding reasoning steps. Although this design reduces computational cost, it raises an important concern: whether these sentence-local circuits remain sufficiently consistent with the model's actual internal computations during reasoning.
  3. The proposed method assumes that faithful reasoning traces should align with the model's internal computational process, whereas unfaithful traces should exhibit discrepancies. Although this assumption is intuitively reasonable, more thorough theoretical or empirical justification is needed.
  4. The authors combine predictive entropy and counterfactual necessity scores to select informative tokens. However, high predictive entropy does not necessarily indicate that a token makes a significant contribution to the model's reasoning decisions.
  5. Additional experiments should be conducted to evaluate the effects of FGW hyperparameters and different sentence segmentation strategies on detection performance.
Questions for authors

See weakness

Reviewer_bPctOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes CIE-Scorer, a tool that checks whether an AI model's step-by-step reasoning honestly reflects how it reached its answer. The idea is to compare two views of each reasoning step: what the written text says, and what the model's internals were doing (circuits). If the two views disagree a lot, the reasoning is flagged as unfaithful. Full circuit tracing is very slow, so the authors trace only important words per sentence, which makes it faster. They then measure the mismatch between the two views using a graph-comparison method. On four benchmark datasets and two models, it beats existing methods, though often by small margins on small test sets.

Critical strengths
  • The work makes circuit-based detection practical. Selecting only important tokens per sentence cuts tracing time and lets the method run on LogiQA and HLE-Bio, where the circuit baseline CRV could not run. Scalability is the main barrier to using interpretability tools for auditing, so this is a meaningful practical contribution.
  • Empirical results are consistent across settings. Best F1 on all four datasets and both backbones (Llama-3.1-8B and Qwen2.5-7B), against a broad set of baselines spanning six method families.
Critical weaknesses
  • The “discrepancy” is a learned metric with no anchor. Both the GNN and the adaptor are trained end-to-end on labels, so the FGW score can act as an ordinary classifier rather than a measure of internal–external alignment. A swap test (pairing internal and external graphs from different traces) or a concatenation-classifier baseline should be tested to show the discrepancy framing adds anything.
  • The statistical support is weak. Test sets appear small, no seeds or variance are given, and several gains are within noise. Without confidence intervals, the state-of-the-art claim is not established.
Questions for authors
  1. In appendix B, when you split train-test for response-step pairs, would step-level split leak steps from the same response across train and test?
  2. For the main experiments (Table 1), the test set size appears quite small. What are the test set sizes?
  3. How does performance compare against cheaper internal signals, such as a linear probe on residual-stream activations or SAE features of the same sentences? If these match CIE-Scorer, the circuit-tracing step may not be necessary.
Reviewer_WW2cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CIE-Scorer, a trained detector for instance-level CoT unfaithfulness. For each reasoning sentence, it selects a handful of tokens, traces a compact transcoder-based attribution circuit for the sentence in isolation, and compresses that circuit with a GIN into an “internal” embedding; an “external” embedding comes from a frozen BERT encoder through an MLP adaptor. Both embedding sequences are turned into trace-level graphs by the same rule, and the Fused Gromov–Wasserstein distance between the two graphs is the unfaithfulness score. The GIN and adaptor are trained with a margin loss, with a validation-chosen threshold at inference. On four FaithCoT-Bench datasets with LLaMA-3.1-8B and Qwen2.5-7B, the paper reports the best F1 against counterfactual, logit-based, LLM-judge, CC-SHAP and CRV baselines, above-chance diagnosticity on a causal testbed, lower tracing cost than CRV, and an analysis linking post-hoc vs. spurious reasoning to feature- vs. structure-level discrepancy.

Critical strengths

Token-selected, sentence-local tracing makes circuit-level featurization of multi-step CoTs tractable (≈3× cheaper per step than full-token tracing; runs where CRV OOMs), and the pipeline was replicated on a second backbone with transcoders the authors trained themselves.

The causal-diagnosticity evaluation, which evaluates a frozen detector on held-out faithful/unfaithful explanation pairs built by knowledge editing (Table 2) is a label-free test of construct validity that most detection papers skip, and the method is above chance on all four tasks. Consequential because it is the one result not explained by fitting annotator labels.

Critical weaknesses

The internal branch, in which each sentence is traced in isolation (with no question, no preceeding steps and no answer), does not observe the computation the paper claims to compare against. The circuit reflects how the model does a pass on the decontextualized sentence, not inside the actual generated trace. Thus, the two label categories depend on exactly what is discarded: post-hoc reasoning on when the answer was fixed, spurious chains on logical relations between steps. Moreover, both components are trained on the labels, so the alignment of faithful traces is enforced by the loss, not observed. This is consequential because the paper's contribution is the internal-external comparison; without it, the method is a supervised classifier over an expensive featurization of sentence text and a hand designed outer graph, and the cheaper alternative, a probe on in-context hidden states, is never compared.

FINE-CoT has just over 1000 trajectories across four generators and four domains, about 60-70 per model-domain cell, split three ways and tuned per dataset, and the results are single-seed with no confidence intervals and no split sizes, while the reported margins are within noise at this scale. Moreover baseline accuracies within a column imply different denominators, and Faithful-Judge and CRV on the same compact circuits are omitted. This makes the SOTA claim questionable.

The claim to be “the first approach to explicitly combine the model’s internal computational process with its externally displayed reasoning trace” is overstated. Activation-probing work already detects motivated/post-hoc reasoning from internal representations and CoT content in counterfactual settings, and reports that internal signals beat CoT monitoring (Mirtaheri et al., 2026; arXiv:2605.09502). The attribution-graph analyses the tracing method comes from were themselves first applied to faithful vs. motivated-reasoning CoTs (Lindsey et al., 2025), and Arcuschin et al. (2025), cited here only as a judge method, also probes internal representations. These works are the missing baseline: a residual-stream probe on in-context activations, trained on the same splits, would show whether circuit tracing adds anything over a single forward pass. The novelty claim should be dropped and the related work repositioned accordingly.

Questions for authors
  1. Exact train/val/test counts per backbone × domain, number of seeds, and how the all-integer accuracies in Table 3 arise (what is n?). Please add CIs.

  2. Were the Table 1 baselines re-run on the same test instances as CIE-Scorer, or taken from FaithCoT-Bench? The within-column accuracies imply different denominators. Why is the benchmark’s strongest method (Faithful-Judge) absent, and which judge model backs the BiGGen row?

  3. What is the activation context for tracing, in the main experiments and in the causal-diagnosticity testbed? Does the internal branch ever see the question, prior steps, or the in-context edit?

  4. How does CIE-Scorer compare to an in-context hidden-state probe? Does performance drop when the LLaMA pipeline is applied to the GPT-4o-mini/Gemini traces already in FINE-CoT, or when tracing uses mismatched transcoders?

  5. How much do random-K token selection, FGW with untrained encoders, and CRV run on the same compact circuits (rather than at full budget until it fails) change the results?

Reviewer_VNdBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes CIE-Scorer for detecting unfaithful CoT reasoning by comparing internal computational evidence with generated reasoning traces. It selects high-entropy filtered tokens to construct compact sentence-level circuits, builds internal and external sentence graphs, and learns an unfaithfulness score based on the FGW distance. Experiments across two backbones and four FaithCoT-Bench domains report improved detection over the included baselines and reduced tracing costs.

Critical strengths
  1. Combining internal computational evidence with external reasoning traces offers a promising approach to CoT faithfulness detection.
  2. The evaluation covers two backbones and four domains, with ablations, cross-domain transfer, and controlled causal diagnosticity tests.
  3. Selective circuit tracing addresses computational cost, improving the practicality of circuit-based detection.
Critical weaknesses

Please refer to the "Questions" section for details.

Questions for authors
  1. Could you compare isolated-sentence tracing with original-context tracing to validate the sentence-local approximation?
  2. Could you include additional LLM-as-judge baselines reported in FaithCoT-Bench?
  3. Could you compare FGW against a simpler distance, i.e., cosine or Euclidean distance, to clarify the benefit of graph matching?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.