acceptodds
Under review as a conference paper at ICLR 2027

Local Causal Attribution of Chain-of-Thought Reasoning

Abstract

Understanding the structure of a language model's thought process is an important problem for both transparency and safety. In this work, we take a local and causal approach toward this goal by analyzing the causal relationships among components, termed units, of a given, specific chain-of-thought trace. We construct a structural causal model on these units and relate each unit to the log probability of generating subsequent output units. Our AttriCoT algorithm performs attribution by estimating importance parameters in this structural causal model, using forward passes to compute log probability effects, where is the number of units. Evaluation of perturbation curves across 5 datasets and 4 reasoning models shows that AttriCoT produces attributions that are a more faithful local explanation of the model's behavior than alternative methods. The attribution results also point to differences in thought structure between models and domains.

Primary area:
interpretability and explainable AI
Keywords:
chain-of-thought, local explanation, attribution

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qBCMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AttriCoT to study causal dependencies within a given chain-of-thought (CoT). It removes preceding reasoning units, measures changes in the log probability of subsequent units, and fits a linear model to construct an attribution matrix. The method requires O(U) forward passes. Experiments on five datasets and four models show improvements over Thought Anchors and prompting baselines under the proposed perturbation-based evaluation.

Critical strengths
  1. The method efficiently computes pairwise attribution scores across prompt, reasoning, and answer units.

  2. The experiments cover multiple models and reasoning benchmarks, with additional sensitivity analyses in the appendix.

Critical weaknesses
  1. Limited novelty.

The main approach closely resembles existing perturbation-based attribution methods. In particular, AttriCoT-LOO is mathematically equivalent to measuring leave-one-out changes in the target unit's log probability. MExGen/C-LIME and ContextCite already employ similar attribution techniques and linear surrogate models. Thought Anchors and CAGE have also studied pairwise dependencies between generated reasoning units. Related work on counterfactual reasoning-step importance (e.g., Paul et al., 2024; Yu et al., 2025; Macar et al., 2026) further motivates a clearer distinction between local attribution and reasoning-level causal effects.

The paper does not clearly establish what is fundamentally new beyond combining these ideas into an efficient framework.

  1. Unclear causal interpretation and insufficient validation.

The method measures how removing a preceding unit changes the probability of an already generated subsequent unit. However, it is unclear whether this reflects meaningful reasoning dependencies or simply textual predictability.

For example, a step that repeats the answer may receive high attribution without contributing meaningful reasoning. Conversely, an important reasoning step may receive low attribution if its information is available elsewhere.

The main evaluation also uses essentially the same likelihood-based criterion as the attribution method itself. Therefore, stronger performance does not necessarily establish better causal explanations of reasoning. Notably, the improvements from additional interventions do not transfer to the regeneration-based evaluation.

Furthermore, the input ratio metric in Eq. (16) contains a position-dependent normalization factor, which may confound the interpretation that models increasingly revisit the input during later reasoning.

Questions for authors
  1. What is the key methodological novelty beyond existing LOO attribution, C-LIME, ContextCite, and CAGE? In particular, what does fitting the linear model add in the basic LOO setting?
  2. Why should changes in the likelihood of fixed generated text be interpreted as meaningful causal dependencies in reasoning? Can the authors provide independent validation beyond likelihood-based perturbation evaluation?
  3. Does the observed increase in input importance persist when using the fraction of total attribution assigned to input units, without the position-dependent normalization in Eq. (16)?
5 comments
AuthorsOpenReview ↗

Response to Reviewer qBCM

Thank you very much for your review, and for recognizing the method’s computational efficiency and the coverage of the experiments and sensitivity analyses. Below we respond to your concerns, including a strengthened independent validation (estimating re-generative dependencies) that shows AttriCoT’s competitiveness with a much more expensive method.

Question 2

Why should changes in the likelihood of fixed generated text be interpreted as meaningful causal dependencies in reasoning?

We would like to reiterate that our paper focuses on local causality, referring to causal effects on the given, generated units (as discussed in paragraph 3 of the introduction). These effects are measured by changes in the likelihood of the given units. Removing prior units is an intervention on the model’s context, and the likelihoods of later units causally depend on these prior units. We believe therefore that this is a valid notion of causality and that the main evaluation is appropriate for assessing estimation of local causal effects.

Can the authors provide independent validation beyond likelihood-based perturbation evaluation?

As your review notes, Appendix D.2.2 provides such independent validation by measuring effects on re-generation of output units rather than their likelihoods. We think that this evaluation is closer to what you mean by “meaningful causal dependencies in reasoning.” We have now strengthened this evaluation; please see the common response titled “Strengthened Re-generative Evaluation.” The new results reinforce AttriCoT’s competitiveness in estimating re-generative dependencies: AttriCoT-LOO is never more than 5% below a much more expensive re-generative method, and can even significantly outperform it (on GPQA).

the improvements from additional interventions do not transfer to the regeneration-based evaluation

We provide a possible explanation of why AttriCoT-2x + Ber(0.3)’s improvements do not transfer in our response to Reviewer MgUU.

Question 3

the interpretation that models increasingly revisit the input during later reasoning

The paper does not claim that models “increasingly revisit” the input as reasoning progresses. Section 6.2 states only that the input ratio grows to a peak, which may occur during mid to late reasoning as in Figures 3(a)(b), after which it decreases again. The paragraph goes on to say that “this pattern is suggestive of the model referring back to the input as it reaches one or more answers,” i.e., a peak that may correspond to relating answers to the input, not a steady increase.

Does the observed increase in input importance persist when using the fraction of total attribution assigned to input units, without the position-dependent normalization in Eq. (16)?

We actually computed what you suggest, the fraction of total attribution assigned to input units (now eq. (17)), in an earlier version of this work. We have now restored these plots as Figure 18 in Appendix D.6. The issue with the unnormalized fraction is that the number of input units is small, sometimes only 1, and becomes dominated by intermediate output units as the output position tt increases. The fraction therefore decreases, but largely because of the rapidly decreasing proportion of input units. We thought therefore that it would be better to compare the average importances of input units to all preceding units. The position-dependent factor 1/(S+t−1)1/(S+t-1) serves only to average over a position-dependent number of units. Note that even with input fraction’s downward bias in Figure 18, some curves (particularly DS-Llama-8B and DS-Qwen-14B in Figure 18(a)) still show a mid-reasoning bump, similar to what input ratio shows more clearly in Figure 17. The mid-reasoning bump persists when we segment the model output into sentences rather than reasoning steps, as seen in the new Figure 19(a).

AuthorsOpenReview ↗

Response to Reviewer qBCM (continued)

Question 1

What is the key methodological novelty beyond existing LOO attribution, C-LIME, ContextCite, and CAGE?

  • With respect to MExGen/C-LIME and ContextCite, the difference is that they attribute only one output, to input/context units only. AttriCoT attributes multiple output units to both input units as well as previous output units. Furthermore, AttriCoT attributes all TT output units using a linear O(S+T)O(S+T) number of forward passes, whereas straightforwardly applying MExGen or ContextCite separately to each output unit would be roughly TT times as costly.
  • With respect to CAGE, a key difference is that CAGE focuses on attribution of the final output to prompt sentences. The pairwise dependencies between reasoning units serve only as a means to that end.
  • With respect to both CAGE and TA-KL, our main contributions are:
    1. A linear structural equation model (SEM) that can both exploit multiple-unit interventions as well as make predictions on held-out interventions (as in the random removal experiment), which CAGE and TA-KL cannot. Furthermore, with respect to MExGen C-LIME and ContextCite, this SEM has multiple equations (one per output unit), the equations depend also on previous output units, and they are all fit together from a shared, linear number of interventions.
    2. Faithfulness evaluation of all unit-to-unit attributions using perturbation curves, as opposed to only final-output-to-prompt attributions.
    3. Quantitative analysis of unit-to-unit attribution matrices (Section 6).

Overall, AttriCoT offers a combination of elements that goes beyond what any one of these prior works has, implemented in an efficient manner.

Lastly, we neglected to cite and discuss CAGE in the paper, so we thank you for bringing this to our attention. We have now done so in Section 2.

what does fitting the linear model add in the basic LOO setting?

You are right that for the basic LOO variant, linear regression is not needed to fit the linear model. However, the resulting linear model does turn LOO effects into a predictive model of the effects of multiple-unit removals. This predictive ability is precisely what enables the new random removal experiment.

Dear Reviewer qBCM,

Thank you again for your review. As the author-reviewer discussion will end on November 18 at 11:59pm AoE (a little more than two days from now), we would be grateful if you could read our rebuttal and post any follow-up questions and comments within the next day or so to give us time to respond.

Our responses to your concerns include a strengthened independent validation on estimating re-generative dependencies and an answer to your question about the input ratio metric. Changes to the paper are marked in red.

Thank you,

Authors of submission 34895

Reviewer_ffJfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AttriCoT, an attribution method that quantifies the effect of one earlier unit (e.g., a group of tokens) on another within a fixed CoT trajectory. AttriCoT first feeds the LLM a query and lets it generate a CoT trajectory. Then AttriCoT performs interventions on the fixed trajectory, namely removing individual units within the CoT sequence. To estimate the causal effect, the authors perform leave-one-out intervention (removing one unit in each intervention), run a forward pass to compute the effect on the target outcome, and run linear regression to identify the importance of each intervened unit. The method is evaluated on five reasoning datasets and four open-weight reasoning models, using perturbation-curve ("faithfulness") evaluation against Thought Anchors' KL-masking method (TA-KL) and LLM-prompting baselines, where AttriCoT consistently achieves the best or tied-best area under the perturbation curve among causal methods of comparable cost. The paper emphasizes that AttriCoT is an \O(U)\O(U) procedure, where U is the number of units.

Critical strengths

S1 A clear definition of what the attribution measures. The paper specifies both the intervention and the outcome: remove earlier units and measure how the log probability of a later unit changes while the remaining text stays fixed. This makes the attribution scores straightforward to interpret. The linear model also provides a way to combine measurements from multiple-unit interventions. Although the contribution is incremental, I find this a useful approach to studying dependencies within a reasoning trace.

S2 Thorough evaluation beyond the main results. The appendix tests alternative perturbations and evaluates changes in regenerated text, providing evidence beyond the main log-probability metric. The authors also document their changes to the TA-KL baseline and report a white-box approximation that did not perform well. These experiments help establish where the method works and where its gains are limited. The regeneration results are particularly useful given the close alignment between AttriCoT’s objective and the main evaluation metric.

Critical weaknesses

W1 The main evaluation closely matches what AttriCoT measures.

AttriCoT assigns importance based on changes in the target text’s log probability, and the main evaluation rewards rankings that cause large decreases in that same quantity. This is a reasonable test of the stated objective, but it favors AttriCoT over TA-KL, which measures changes in the full token distribution. The results therefore support a narrower claim than general superiority in causal attribution. The regeneration-based evaluation helps address this concern: AttriCoT-LOO still performs well, but the gains from additional interventions do not carry over.

W2 The structural findings may depend on how the text is split.

The authors split the reasoning and final-answer text at double newlines and merge units when the traces are too long. These choices change what gets removed in each intervention and how quantities such as step distance and attribution concentration are measured. It is unclear to me whether the patterns in Section 6 reflect stable properties of model behavior or depend substantially on the chosen segmentation. Checking an alternative, such as sentence-level segmentation, on a representative model–dataset pair would strengthen these interpretations.

Questions for authors
  1. Clarification: The linear model assumes that the contributions of preceding units are additive, but reasoning steps may interact or repeat the same information. Beyond interpretability, what supports this assumption? Have the authors checked how well the fitted model predicts target log probabilities under held-out combinations of unit removals? This would help distinguish the usefulness of its attribution rankings from how accurately it captures the effects of interventions.

  2. Clarification: The additional-intervention variants improve attribution under the fixed-text log-probability evaluation, but those gains do not carry over to the regeneration-based evaluation in Appendix D.2.2. Should we understand their benefit as better attribution of the original text’s likelihood, rather than better attribution of changes in generated text?

  3. Suggestion for future work: Connecting to the first question, how do the attribution scores behave when multiple steps repeat the same information? In particular, can the additional-intervention variants identify their importance when removing either step alone has little effect, but removing both substantially changes the target score?

2 comments
AuthorsOpenReview ↗

Response to Reviewer ffJf

Thank you very much for your positive assessment, particularly of the clear interpretation of the attribution scores, the ability to exploit multiple-unit interventions, and the thorough evaluation beyond the main results. Below we report on a strengthened re-generation-based evaluation and a different segmentation, and address your questions about the linear models, where we followed your suggestion of a held-out prediction experiment.

Weakness 1

The regeneration-based evaluation helps address this concern: AttriCoT-LOO still performs well

Thanks for acknowledging that the re-generation-based evaluation helps address the concern. We have further strengthened this evaluation; please see the common response titled “Strengthened Re-generative Evaluation.” The new finding is that AttriCoT-LOO is never more than 5% below a much more expensive re-generative attribution method, and can even significantly outperform it (on GPQA).

Weakness 2

The structural findings [in Section 6] may depend on how the text is split… Checking an alternative, such as sentence-level segmentation, on a representative model–dataset pair would strengthen these interpretations.

We re-ran AttriCoT-LOO on the GSM8K DS-Qwen-14B pair with sentence-level units (segmented by spaCy), with no merging of units this time. We then recomputed the metrics of Section 6 and added corresponding plots as Figure 19 in the paper (on the last page). The curves for sentence-level and step-level units agree closely with each other (Pearson correlation 0.94–0.99 across metrics), including the mid-reasoning peak in input ratio seen in Figures 3(a) and 19(a).

Questions

Beyond interpretability, what supports [the linearity] assumption? Have the authors checked how well the fitted model predicts target log probabilities under held-out combinations of unit removals? This would help distinguish the usefulness of its attribution rankings from how accurately it captures the effects of interventions.

Thank you for recognizing the motivation of interpretability, and especially for the suggestion of testing the linear models on held-out predictions. We have now performed a random removal experiment (please see the corresponding common response). The linear models are intended as interpretable local approximations, and the results confirm that linearity approximately holds at high locality and then gradually degrades. Moreover, the usefulness of rankings (correlation) degrades more slowly than the accuracy of magnitude predictions (R2R^2).

The additional-intervention variants improve attribution under the fixed-text log-probability evaluation, but those gains do not carry over to the regeneration-based evaluation in Appendix D.2.2. Should we understand their benefit as better attribution of the original text’s likelihood, rather than better attribution of changes in generated text?

In a word, yes, particularly for AttriCoT-2x + Ber(0.3). Its performance is consistent with our conclusion from the random removal experiment and its AUPC results, namely that Ber(0.3) is better fit to large log probability effects compared to AttriCoT-LOO, on larger removed subsets. Because of this more tailored fit to log probability responses, it is not unexpected that AttriCoT-2x + Ber(0.3) transfers less well to predicting re-generative effects, a different kind of response.

how do the attribution scores behave when multiple steps repeat the same information? In particular, can the additional-intervention variants identify their importance when removing either step alone has little effect, but removing both substantially changes the target score?

Yes, partially, and more likely under AttriCoT-2x + Ber(0.3). Suppose that two steps repeat the same information, and that this pair of steps is covered by a sampled multi-unit intervention. Recall that the AttriCoT-2x variants sample these randomly, and that Ber(0.3) is more likely to cover the pair because of the Bernoulli augmentation. In this case, the LOO interventions on the two steps would show little effect, but the multi-unit intervention that includes both would show a large effect. The linear regression would then likely yield coefficients for the two steps that are intermediate between no effect and the full effect.

Dear Reviewer ffJf,

Thank you again for your review. As the author-reviewer discussion will end on November 18 at 11:59pm AoE (a little more than two days from now), we would be grateful if you could read our rebuttal and post any follow-up questions and comments within the next day or so to give us time to respond.

We have addressed all of your concerns and questions with a strengthened re-generation-based evaluation, a different segmentation, and a held-out prediction experiment on the linear models, following your suggestion. Changes to the paper are marked in red.

Thank you,

Authors of submission 34895

Reviewer_MgUUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AttriCoT, a framework for identifying local causal relationships among units in a given LLM chain-of-thought (CoT) trace. By removing preceding units as interventions, it measures changes in the conditional log probability of generating the original target unit. These measurements are then used to fit a linear structural causal model to estimate unit-to-unit attribution scores, forming attribution matrices. AttriCoT is evaluated against several baseline methods across five datasets and four reasoning models. The authors further analyze the attribution matrices to investigate patterns in CoT reasoning structure.

Critical strengths
  1. This paper investigates unit-to-unit causal dependencies within a given CoT trace, which may provide useful insights into the structure of CoT reasoning in LRMs.
  2. The proposed method is conceptually simple and computationally efficient, requiring only a linear number of non-generative forward passes with respect to the number of units.
  3. The empirical evaluation covers five datasets and four reasoning models. The paper also provides extensive additional analyses of the attribution results.
Critical weaknesses
  1. The main evaluation criterion is coupled with the attribution metric used by AttriCoT to some extent, which may favor AttriCoT over other baselines. Although Appendix D.2.2 provides a re-generative evaluation that does not directly rely on log probability, the experiments are only conducted on GSM8K. Extending this evaluation to additional datasets would help provide stronger evidence for the effectiveness of AttriCoT.
  2. Relatedly, the empirical adequacy of the additive linear formulation, particularly when incorporating multi-unit interventions, deserves further investigation. AttriCoT-2x + Ber(0.3) outperforms AttriCoT-LOO in the main evaluation. However, this advantage does not carry over to the re-generative evaluation in Appendix D.2.2, where it performs worse than AttriCoT-LOO, whose attribution scores can be directly computed without requiring linear regression. Moreover, the paper focuses on local direct attribution between units, while simultaneously deleting multiple units may introduce interaction effects that cannot be captured by the additive linear model. The validity of this linearity assumption is not sufficiently examined.
Questions for authors

N/A

2 comments
AuthorsOpenReview ↗

Response to Reviewer MgUU

Thank you very much for your review. We are glad that you appreciated the potentially useful insights into CoT structure, the computational efficiency of the method, and the extensive additional analyses. Below we address both of your weaknesses with a strengthened re-generative evaluation and a new experiment that further validates the linear models.

Weakness 1

Extending [the re-generative] evaluation to additional datasets would help provide stronger evidence for the effectiveness of AttriCoT.

We have extended the re-generative evaluation to a second dataset, as you requested, and also added a re-generative attribution method as a reference. Please see the common response titled “Strengthened Re-generative Evaluation.” The new evidence of effectiveness is that AttriCoT-LOO is never more than 5% below the much more expensive re-generative method, and can even significantly outperform it (on GPQA).

Weakness 2

simultaneously deleting multiple units may introduce interaction effects that cannot be captured by the additive linear model. The validity of this linearity assumption is not sufficiently examined.

First, we note that evaluating perturbation curves involves simultaneously deleting multiple units (up to 20% of the units preceding an output unit), so the linear models’ rankings of units have already been tested under such multi-unit removals. Second, the linear models are intended only as local approximations to the log probability response function, and thus are expected to be faithful only locally, i.e., close to the given sequence of units. This follows an approach to obtaining local explanations dating back to LIME. We have now clarified the local approximation expectation in Section 3.

With the above in mind, to further establish the validity of the linear models, we performed a random removal experiment. Please see the common response titled “Random Removal Experiment.” The results confirm that additivity approximately holds at high locality (still multi-unit removals) and then gradually degrades. Moreover, correlation with true responses degrades more slowly than the accuracy of magnitude predictions (R2R^2).

AttriCoT-2x + Ber(0.3) outperforms AttriCoT-LOO in the main evaluation. However, this advantage does not carry over to the re-generative evaluation in Appendix D.2.2, where it performs worse than AttriCoT-LOO

This is consistent with our conclusion regarding AttriCoT-2x + Ber(0.3) from the random removal experiment and its AUPC results, namely that it is better fit to large log probability effects compared to AttriCoT-LOO, on larger removed subsets. Because of this more tailored fit to log probability responses, it is not unexpected that AttriCoT-2x + Ber(0.3) transfers less well to predicting re-generative effects, a different kind of response.

Dear Reviewer MgUU,

Thank you again for your review. As the author-reviewer discussion will end on November 18 at 11:59pm AoE (a little more than two days from now), we would be grateful if you could read our rebuttal and post any follow-up questions and comments within the next day or so to give us time to respond.

We have addressed both of your concerns with a strengthened re-generative evaluation and a new experiment further validating the linear models. Changes to the paper are marked in red.

Thank you,

Authors of submission 34895

Comments on the paper

2 comments
AuthorsOpenReview ↗

Random Removal Experiment

We performed a random removal experiment to further validate the linear models, following Reviewer ffJf’s suggestion. We randomly sample subsets of units for removal (with a different seed than used for AttriCoT-2x + Ber(0.3)) and measure the resulting decreases in log probabilities of output units. The size of the subsets is controlled to vary locality (smaller removed subsets are more local). 10 subsets are sampled for each instance and subset size. For each removed subset, the log probability decrease predicted by the linear model is the sum of its coefficients for the removed units. We evaluate these linear model predictions against the actual decreases using both R2R^2 and Pearson correlation (within each instance, then averaged over instances). The tables below show the results for two dataset-model pairs, different levels of removal, and the three AttriCoT variants that fit linear models (R2R^2 / correlation in each cell). Tables for all models on GSM8K and GPQA have been added to Appendix D.5.

Overall, the results of this random removal experiment are as expected for a local linear approximation: At high locality (fewer units removed), the linear model predicts the effects of removal well. As locality decreases, the accuracy gradually degrades. Even so, correlation degrades more slowly than R2R^2 and is still 0.640.64-0.770.77 at 20% removal in these two tables. Regressing actual decreases on predicted ones shows that the linear model overestimates effects as the removal fraction increases, resulting in a decrease in R2R^2, while still maintaining the ability to rank effects. Furthermore, a “count baseline” that predicts based only on the number of removed units has R2R^2 near zero, so AttriCoT’s R2R^2 in the 0.30.3-0.40.4 range at 20% removal is still well above this baseline.

GSM8K Qwen3-8B2 units5%10%20%
AttriCoT-LOO0.89 / 0.950.85 / 0.940.76 / 0.900.37 / 0.77
AttriCoT-2x0.88 / 0.950.84 / 0.940.76 / 0.890.39 / 0.77
AttriCoT-2x + Ber(0.3)0.66 / 0.840.65 / 0.840.62 / 0.820.42 / 0.73
Count baseline0.07 / 0.310.09 / 0.340.10 / 0.360.00 / 0.36
GPQA DS-Llama-8B2 units5%10%20%
AttriCoT-LOO0.78 / 0.890.65 / 0.820.54 / 0.760.30 / 0.66
AttriCoT-2x0.78 / 0.890.66 / 0.820.55 / 0.770.32 / 0.67
AttriCoT-2x + Ber(0.3)0.59 / 0.800.52 / 0.750.44 / 0.720.27 / 0.64
Count baseline0.03 / 0.200.03 / 0.240.03 / 0.27-0.02 / 0.29

For AttriCoT-2x + Ber(0.3), its performance in this random removal experiment is lower than AttriCoT-2x at 10% removal and below. This does not contradict its superior AUPC performance in Tables 1 and 2 because the removals in the latter are top-kk subsets, i.e., high-importance units rather than random. Moreover, at 20% random removal, AttriCoT-2x + Ber(0.3)’s R2R^2 becomes better than the other two variants for half of the dataset-model pairs (e.g. GSM8K Qwen3-8B). Together, the random removal and AUPC results indicate that AttriCoT-2x + Ber(0.3) is better at predicting the log probability effects of removals when the effects are large and the removed subsets are larger.

AuthorsOpenReview ↗

Strengthened Re-generative Evaluation

We have strengthened the re-generative evaluation of Appendix D.2.2 in two ways: 1) We extended it to a second, non-math dataset, GPQA. 2) We added an attribution method that is matched to the re-generative evaluation, to serve as a reference: it estimates the importance of each preceding unit to the re-generation of an output unit, in a LOO fashion. To mitigate randomness, for each LOO prefix, we re-generate an output unit 5 times and average the similarities of the re-generated units to the original unit. Due to the quadratic, O(T(S+T))O(T(S+T)) number of re-generations, this method is much more expensive than AttriCoT. With 5 re-generations per LOO prefix, the number of LLM calls is about 50-300x that of AttriCoT-LOO (and these are generative calls, not just forward passes).

Based on the new results, we believe that the main finding of the re-generative evaluation is now the following: AttriCoT-LOO is never more than 5% below the re-generative attribution method’s AUPC with 5 re-generations (on GSM8K), and both AttriCoT and TA-KL (no log) can even significantly outperform re-generative (on GPQA and GSM8K DS-Qwen3-8B), while being orders of magnitude less expensive. If the re-generative method can use only 1 re-generation, then AttriCoT-LOO and TA-KL (no log) are always better, by up to 45%. The following tables show the AUPCs with BERTScore as the similarity metric (bold = tied for best excluding re-generative, italic = tied for best overall); for GSM8K, the table augments Table 11 in Appendix D.2.2, while the table for GPQA is new. Tables for all similarity metrics have been similarly augmented/added to Appendix D.2.2.

GSM8KDS-Llama-8BDS-Qwen-14BDS-Qwen3-8BQwen3-8B
AttriCoT-LOO32.42 ± 0.2535.53 ± 0.2426.10 ± 0.2339.77 ± 0.24
AttriCoT-2x32.21 ± 0.2535.52 ± 0.2426.23 ± 0.2339.73 ± 0.24
AttriCoT-2x + Ber(0.3) (min size 2)31.00 ± 0.2734.49 ± 0.2425.88 ± 0.2439.21 ± 0.25
TA-KL (no log)31.75 ± 0.2634.73 ± 0.2525.77 ± 0.2439.95 ± 0.25
re-generative (1 sample)30.98 ± 0.2534.78 ± 0.2520.08 ± 0.2037.27 ± 0.26
re-generative (5 samples)33.60 ± 0.2537.11 ± 0.2425.68 ± 0.2440.58 ± 0.24
GPQADS-Llama-8BDS-Qwen-14BDS-Qwen3-8BQwen3-8B
AttriCoT-LOO26.08 ± 0.6530.28 ± 0.7913.08 ± 0.4631.79 ± 0.68
AttriCoT-2x26.28 ± 0.6030.40 ± 0.7813.17 ± 0.4531.85 ± 0.67
AttriCoT-2x + Ber(0.3)26.01 ± 0.5930.07 ± 0.7913.19 ± 0.4531.17 ± 0.67
TA-KL (no log)26.39 ± 0.6330.75 ± 0.8512.99 ± 0.4631.94 ± 0.67
re-generative (1 sample)21.32 ± 0.5926.63 ± 0.789.36 ± 0.4127.48 ± 0.65
re-generative (5 samples)25.00 ± 0.6030.15 ± 0.7712.25 ± 0.4531.06 ± 0.67

On GPQA, we find that TA-KL (no log) closes the gap with AttriCoT-LOO. We do not see this as a negative but rather a finding in itself. The Thought Anchors paper did not systematically evaluate TA-KL using perturbation curves, neither in terms of log probability effects nor re-generative effects. We do so, and thanks in part to the improvements to TA-KL that we made in Appendix C.3.2, we find that it performs well especially under re-generative evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.