- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper uses Transcoders and Lorsa to build feature-level attribution graphs for the chess model LC0 BT4. It examines several tactical decisions through case studies and analyzes how the computation is organized across candidate moves, board squares, and layers.
Critical strengths
-
The framework traces through both attention and MLP features, allowing a more connected view of how the model reaches a decision.
-
The intervention results give some confidence that the framework identifies features that matter for the model's behavior. In the g2-pawn case, ablating its detection feature raises the mating move's probability from 28.75% to 90.6%.
Critical weaknesses
W1. The framework largely combines existing Transcoder, Lorsa, and circuit-tracing methods. The chess-specific interpretation and validation pipeline is useful, but its added value over existing approaches is not clearly established. This makes the broader takeaway especially important, yet I am still left wondering what we learn beyond these particular examples. Look-ahead in LC0 has already been demonstrated, and some of the reported organizational patterns may follow from the architecture or graph construction. The conservative-error case is interesting, though its implications beyond that position remain unclear.
W2. I also have concerns about how firmly the proposed mechanisms are established. It is unclear whether the main pathways and organizational findings are robust to the attribution approximations. In particular, tracing Q and K separately does not fully capture their joint interactions. The paper does not show whether accounting for these interactions would change which features or paths appear important. The causal validation also leaves a gap. Ablating the feature detecting the g2 pawn affects the decision, but does not establish that the effect passes through the proposed pathways involving queen protection and king safety. Likewise, influential features associated with future moves do not fully validate the look-ahead chain. The general zero-patching and completeness checks do not directly verify these specific mechanisms.
Questions for authors
Q1. What is the main technical advance over prior complete replacement and circuit-tracing frameworks? Which analytical capability specifically depends on the proposed extensions?
Q2. Can the identified mechanisms generate predictions that hold on unseen positions? For example, predicting when the same conservative failure will occur? Such a test would help show that the explanations provide reusable understanding.
Q3. How sensitive are the key pathway rankings and separation/branching findings to these approximations, especially the separate treatment of Q and K?
Q4. Can path-specific interventions, with matched controls, establish whether the g2 feature's effect on move preference is actually mediated by the proposed queen-protection/king-safety pathway?
6 comments
Additional Response to Reviewer FaiD Part 2
Q1. Main Technical Advance
The main advance is an architecture-specific extension of Lorsa and circuit tracing that enables us to trace square-to-square information flow through attention and connect feature-level pathways to individual move logits through the bilinear policy head. Combined with our agentic rule-based validation framework, this provides an end-to-end analytical capability unavailable from MLP-only decomposition or conventional circuit tracing.
Q2. Generalization to Unseen Positions
We agree that the current results do not establish that the identified defensive pathway can reliably predict similar failures on unseen positions. Our case study demonstrates a mechanistic explanation for one specific decision, rather than a general predictive mechanism. We will clarify this limitation and leave systematic cross-position prediction to future work.
Q3. Sensitivity to Attribution Approximations
If by approximations you mean the sparse decomposition of the original MLP/attention modules, their faithfulness is evaluated in Appendix D.
If you refer to approximations in attribution computation, we do not approximate the QK operation itself. The current Q-side (K-side) attribution already aggregates a feature's pairwise interactions with all K-side (Q-side) features, respectively. EThis would only reveal individual contributions without changing aggregate attributions or their rankings. Enabling gradients through Q and K simultaneously would, however, break the linearity of the attribution graph, such that summed feature attributions would no longer recover the target logit or feature activation. Following Anthropic circuit tracing paper[1], we choose linear direct effects over nonlinear attributions or ablations to enable clearer mechanistic analysis, as credit assignment under nonlinear interactions is difficult. To validate our attribution rankings, we zero-ablate each feature, measure its effect on the target logit, and compare the rankings; Figure 8(c) shows a top-10% overlap of 50.2% ± 7.25%. We then construct intervention-based graphs, assigning each feature a weight based on its relative effect on the target logit:
We retain the same average number of features and edges as in the attribution graphs and compare their source/target-square ratios with the MAR reported in the paper.
| Layer | Lorsa attribution graph | Lorsa intervention | Transcoder attribution graph | Transcoder intervention |
|---|---|---|---|---|
| L0 | 28.90±9.32 | 24.96±28.28 | 23.48±8.35 | 25.81±15.14 |
| L3 | 63.77±15.29 | 48.87±22.4 | 40.38±13.58 | 33.17±16.28 |
| L6 | 73.95±34.61 | 65.69±38.52 | 48.26±11.57 | 36.30±13.81 |
| L9 | 66.78±29.82 | 54.69±36.69 | 62.24±10.94 | 32.29±16.13 |
| L12 | 89.35±9.04 | 84.03±23.5 | 78.80±10.68 | 52.89±19.71 |
| L14 | 100.00±0.00 | 100.00±0.00 | 100.00±0.00 | 100.00±0.00 |
We also construct intervention-based graphs by establishing directed edges between features and measure their branching factors using feature-to-feature intervention effects, quantified by the relative change in a target feature's activation after ablating a source feature.
We define the edge weight as:
where denotes the original activation of feature , and denotes its activation after zero-patching feature . We then prune the intervention-based graphs to match the average number of edges in the original attribution graphs.
| Metric | Original graphs | Intervention effect graphs |
|---|---|---|
| Branching factor | 75.7% ± 7.58% | 89.01% ± 5.50% |
Although this is a crude estimate, the branching factor remains high, supporting the existence of branching structures captured by our attribution graphs.
Separating Q and K is a simplification of attribution: it sums up the feature interactions between the Q and K sides, treating them as the interaction between one feature on one side and all features on the other side combined. We discuss this in Ans W2.1. We acknowledge that this loses information about which feature pairs are important. Since the attention-like policy head is nonlinear, existing QK tracing methods[2] cannot be directly applied. We also discuss this limitation in Sec. 6.1 of the paper. We will study this issue further in future work.
[1] E. Ameisen et al., “Circuit Tracing: Revealing Computational Graphs in Language Models,” Transformer Circuits Thread, Mar. 27, 2025. [Online]. Available: https://transformer-circuits.pub/2025/attribution-graphs/methods.html
[2] Kamath, H., et al. (2025). Tracing Attention Computation Through Feature Interactions. Transformer Circuits Thread (2025).
Additional Response to Reviewer FaiD Part 3
Q4. Path-specific Causal Interventions
After ablating the g2-pawn detection feature, the downstream protected queen exchange feature Tc.10.1958@h3 decreases by 37%, while the rook-threat target feature Lorsa.8.16275@g2 decreases by 100%. Directly ablating Tc.10.1958@h3 increases the probabilities of Qxh7+ and Qxd7 from 28.7% to 34.9% and from 26.1% to 29.3%, respectively, indicating that the model is more inclined to move the queen when it is no longer protected against exchanges. Randomly selected active features reduce them by only 0.024% and 0.124% on average, respectively. We further intervene directly on the rook-threat feature: applying a steering coefficient of -3 increases the fxg7+ logit from 2.26 to 2.42 and its probability from 35.8% to 38.5%. Conversely, transplanting its activation from g2 to g7 reduces the probability of fxg7+ from 35.78% to 32.58%. Together, these interventions provide causal evidence that the downstream defensive signals influence move preferences, with a spatially specific effect consistent with the proposed mechanism. These results support a causal influence of the g2-related defensive signal on downstream computations and move preferences.
Thanks again for the helpful comments. Should you require any further information, please do not hesitate to let us know. We would greatly appreciate your feedback.
Additional Response to Reviewer FaiD Part 1
We sincerely thank the reviewer for their insightful comments and suggestions regarding our methodology, particularly the separate treatment of Q and K.
The revised part is highlighted in blue in the uploaded revised manuscript. We would like to firstly respond to the issues raised in the "Weakness" section and then "Questions" section, organized by each point.
Ans W1. Technical Contributions and Broader Findings
Our contribution is not simply combining existing methods, but extending them to LC0 BT4's architecture in three aspects:
Lorsa for Smolgen-biased bidirectional attention: Using token-wise z-patterns, we can identify which square sends information to another square and what information is transmitted.
Circuit tracing for the bilinear policy head: We extend attribution tracing through the Q/K policy head to connect feature-level computations to individual move logits.
Chess-specific feature interpretation: We develop an agentic pipeline that translates feature interpretations into executable chess rules and validates them using precision and recall.
Together, these extensions enable end-to-end, feature-level analysis of how information flows across board squares and contributes to move decisions.
Beyond individual case studies, our statistical analysis reveals three organizational patterns: (1) move-specific separation, where different candidate moves rely on largely distinct feature sets; (2) within-move branching, where multiple computational pathways jointly support a move; and (3) spatial concentration, where attribution progressively concentrates on the move's source and target squares.
We do not claim to be the first to demonstrate look-ahead in LC0. Rather, we provide a feature-level account of how future tactical information is represented and routed to influence current decisions. The conservative-error case is intended as a mechanistic diagnosis, not a claim of a general failure mechanism.
Ans W2.1. Q/K Separation and Causal Validation
For the bilinear policy head, jointly tracing Q and K introduces multiplicative interactions between Q-side and K-side features. Explicitly representing these interactions would require feature-pair interaction nodes, changing the interpretation of our first-order feature-level attribution graph.
We therefore trace Q and K separately, fixing the opposite side during each trace. This preserves the linear feature-to-feature pathways within each branch. During backpropagation, feature-to-feature attributions remain identical in both Q-side and K-side traces, as they represent direct interactions between features. A feature's logit attribution in the Q-side trace aggregates its interactions with all K-side features, and vice versa. Thus, the attribution scores remain valid without explicitly decomposing them into individual feature-pair interactions. This approach preserves a linear attribution graph of direct effects, enabling clearer analysis of computational mechanisms, since credit assignment under nonlinear interactions is inherently challenging.
Ans W2.2. Validate the look-ahead chain.
We suppress the h4-related rook-attack supernode to block the h4-to-g6 information pathway.
Downstream feature effects:
Target-move effects:
We also jointly zero-ablate two early-layer rook-detection features (Lorsa.0.10273@d4 and Transcoder.0.13486@d4). This reduces downstream rook-to-king threat feature activations by 63.5% on average and look-ahead feature activations by 93.12%. The target-move probability drops from 78.57% to 0.36%.