- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Circuit discovery is framed as a search problem: if the optimization or attribution is improved, we recover better circuits. The selection objective used is intervention-defined faithfulness, in which the non-circuit activations are replaced with donor activations, mean or zero, and the circuit is then expected to produce an output faithful to the intact model. This paper tests whether intervention-defined faithfulness, measured by (negative KL between the circuit and intact model output distributions on validation prompts), ranks equal-sized circuits the same way as a behavioral metric (the circuit's top-1 task agreement with the intact model on held-out prompts). is computed under resampling, mean and zero interventions, while is always computed under resampling and therefore stays fixed across interventions. Authors show that there is an objective-level recovery gap: can favor the lower- circuit. They show this on four tasks (IOI, Greater-Than, Docstring, Acronym) and ten InterpBench tasks. Authors show that on equal-size edits of the reference circuits, misranking depends heavily on task and intervention, and that the same misranking appears among equal-size outputs of the discovery algorithms (ACDC, EAP, EAP-IG and Edge-SP). Authors attribute this gap to context distortion: the replacement signals alter the inputs to the retained circuit components, so they compute on inputs that differ from those in the intact model. To support this, they restore intact-model activations at selected excluded connections, which repairs 96 of 100 persistent misrankings on both validation and test prompts, while the circuits and their values remain fixed.
Critical strengths
- S1: the experimental design is clear such that all comparisons are size-matched, candidates, prompts and are fixed and the reported measures appear reliable. However, this doesn't outweigh my concerns in W1 and W2 about how the results are interpreted and how well they support the paper's central claim.
Critical weaknesses
-
W1: most discovery algorithms do not actually use F precisely to select better circuits, so the relevance of the - disagreement to improving the selection objective for circuit discovery is unclear, which is one of the central claims of the paper. ACDC and EAP/EAP-IG score individual edges by activation and attribution patching from task-specific contrastive prompts, which is different from the whole-circuit evaluated here. Based on this, I find the setup to be ill-motivated.
-
W2: I don't see how under resampling is a valid ground truth. I appreciate that authors listed this as a limitation, but in my opinion, it is central to the paper's main argument of faithfulness-vs-behavior mismatch. (i) is computed under resampling, so it carries the same context distortion attributed to , and I do not see why is used as a reference, not . (ii) Fixing to resampling also biases the comparison: the largest misrankings occur when and use different interventions, whereas under matched resampling, misrankings to controlled edits are low (Table 3). These results only suggest that different interventions rank circuits differently, which is already known from Miller et al. (2024), "Transformer Circuit Faithfulness Metrics Are Not Robust", rather than failing to capture . Measuring under each intervention would separate these explanations, but the paper does not do this. (iii) Even in the matched resampling setting, the discovery misrankings involve small deficits (median is 1.0 and 0.11 pp on human and InterpBench tasks, Table 19). Based on (i--iii), the central claim is not adequately supported.
Questions for authors
- Q1: How does - disagreement precisely improve circuit discovery, if addressed?
- Q2: What kind of prompts are being used as donors for the resampling intervention?
- Q3: Does the misranking persist when is measured under the same intervention as ? (e.g., under mean and zero ablations)
- Q4: How do misranking rates and deficits change with circuit size within a task, for example, when discovery outputs are extracted at several sparsity levels?
- Q5: why is the right criterion for a better circuit, rather than , and how much misranking remains among pairs whose output distributions differ only off the top-1 token?