- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies three sources of variability in circuit discovery: resampling the data, changing prompt templates, and differences in unfaithfulness across individual samples. It introduces CEAP, which uses conductance to score edges and improves resampling stability over EAP-IG. It also reports better cross-template circuit transfer in weight-sparse models and shows that extreme unfaithfulness can arise from a small normalization denominator.
Critical strengths
The distinction between the three sources of variability is useful. CEAP has a clear mathematical motivation, and the cross-template experiments test behavioral transfer rather than relying only on circuit overlap. The normalization analysis also provides a concrete reason to be cautious when comparing unfaithfulness across samples.
Critical weaknesses
-
The mechanism proposed in Section 6 is not sufficiently established. The correlations support a relationship between the normalization denominator, attribution concentration, and unfaithfulness. However, I am less convinced that they establish selective contribution scaling as the underlying explanation. The conclusion that sample-wise variance is “largely benign” also seems too broad without quantifying how often high unfaithfulness still corresponds to meaningful behavioral errors.
-
The sparsity results need an additional check. Increased circuit overlap provides supporting evidence, but the behavioral transfer comparison in Section 5.2 uses normalized unfaithfulness across models with different sparsity levels. Given the findings in Section 6, reporting unnormalized discrepancies and full-model task performance would help establish the magnitude and practical significance of this improvement.
Questions for authors
-
Clarification: What evidence supports interpreting selective contribution scaling as an underlying mechanism rather than a description of the correlations in Figure 6? In Section 6.2, could you distinguish what follows from partition consistency from what remains an empirical hypothesis?
-
Requested analysis: Beyond the examples and correlations in Section 6, could you quantify how often very high unfaithfulness coincides with a large unnormalized discrepancy or a changed task prediction? This would help determine how broadly the conclusion that sample-wise variance is “largely benign” applies.
-
Requested comparison: Does the cross-template improvement in Section 5.2 remain when evaluated using unnormalized discrepancies? Please also report how full-model task performance and the magnitude of the normalization denominator vary with sparsity.
-
Resampling robustness: Section 4 uses four resampled datasets. Do the stability gains persist with additional independent resamples? An uncertainty estimate that accounts for the shared circuits in the six pairwise comparisons would help assess the reliability of these gains.
4 comments
rebuttal 2
We now measure how often samples with extreme unfaithfulness also have near-maximum unnormalized unfaithfulness , compared with other samples for the same model, template, and circuit size. Here, near-maximum means at least 90% of the largest in that group. Full experimental details and results are provided in Appendix Y, Table 13 (lines 2579–2627).
Pooling together all the setups, only 22.4% (84/375) of cases with and 26.7% (24/90) of cases with have near-maximum . Notably, none of the 17 cases with has near-maximum .
These results support the normalization-based explanation in the main text (lines 449–452): extreme unfaithfulness values need not indicate correspondingly extreme absolute discrepancies.
W2 & Q3: Weight Sparsity and the Effect of Normalization
(W2) The sparsity results need an additional check. Increased circuit overlap provides supporting evidence, but the behavioral transfer comparison in Section 5.2 uses normalized unfaithfulness across models with different sparsity levels. Given the findings in Section 6, reporting unnormalized discrepancies and full-model task performance would help establish the magnitude and practical significance of this improvement.
(Q3) Requested comparison: Does the cross-template improvement in Section 5.2 remain when evaluated using unnormalized discrepancies? Please also report how full-model task performance and the magnitude of the normalization denominator vary with sparsity.
We report the normalization denominator in unfaithfulness (Appendix R, Figure 38a, lines 2325–2335). The medians and central percentile bands remain relatively stable and well away from zero even at lower sparsity (more nonzero parameters). This suggests that the larger cross-template deterioration in denser models is not primarily due to near-zero denominators inflating unfaithfulness.
We separately report task-choice accuracy as task performance: the fraction of clean prompts where the correct answer token or set receives greater probability mass than the competing token or set (Appendix R, Figure 38b, lines 2336–2340). On the same 1,000 samples per task, this accuracy is 100% for single-double-quote and else-elif at every sparsity level. For with-open, it ranges from 72.0% to 80.5%, without a monotonic trend across sparsity.
We also repeat the cross-circuit evaluation without normalization (Appendix S, Figure 39, lines 2379–2393). The unnormalized cross-template deterioration decreases with increasing weight sparsity, following the same overall trend as its normalized counterpart in Figure 3a. This provides a qualitative check that the observed generalizability improvement is not merely a normalization artifact.
Q4: Resampling Robustness and Uncertainty
Resampling robustness: Section 4 uses four resampled datasets. Do the stability gains persist with additional independent resamples? An uncertainty estimate that accounts for the shared circuits in the six pairwise comparisons would help assess the reliability of these gains.
Yes. We have expanded the experiment from four to twelve independently resampled datasets for every model–task setup. Instead of comparing all seed pairs, we now use six disjoint pairs: . Each circuit appears in only one pair, removing the shared-circuit dependence raised in your question (Appendix I, lines 1299–1309).
For the uncertainty estimate, we average the PJI differences over circuit sizes, templates, and model–task setups separately for each seed pair. We then compute a 95% confidence interval and perform a two-sided one-sample -test using these six independent averages. We give each model–task setup equal weight so that tasks with more templates do not dominate the result.
CEAP's mean PJI improvement over EAP-IG is , with a 95% confidence interval of and a Holm-adjusted . The improvement is also significant when each model–task–template combination receives equal weight. Thus, the average stability gain persists with additional independent resamples. Appendix I, Table 7 (lines 1350–1380), provides both analyses and corresponding results for unfaithfulness and RelP.
Thank you again for your thoughtful feedback. Your questions prompted us to clarify the limits of our interpretation and strengthen the empirical evidence with additional experiments and uncertainty estimates. We hope these revisions address your concerns. If so, we would be grateful if you would consider increasing your score. We would be happy to address any remaining questions.
Thank you for the detailed analysis and experiments. I find the additional experiments solve most of my concerns. And I think this paper propose some interesting improvement of some existing theoretical tools and apply them to circuit discovery. Since I have given a score of 3 and I will keep my score of weak acceptance.
Thank you
Thank you so much for your support! We are especially grateful that you challenged our selective contribution scaling story. Your questions pushed us to think it through more carefully and develop a much clearer picture.
rebuttal 1
Thank you for your careful review. We appreciate that you found our contribution useful and clear. You raised a few concerns, which we will address below.
Materials relevant to our responses below are marked in the manuscript with your reviewer ID, CGww, in blue. Newly added text appears in green.
W1 & Q1: Evidence and Limitations of Selective Contribution Scaling
We agree that selective contribution scaling should be presented as a tentative explanation rather than an established mechanism. We have revised the reasoning in Section 6.2 and explicitly call it a "possible explanation" in the main text (lines 480–481).
Selective contribution scaling refers to preferential growth of leading absolute scores relative to lower-ranked scores as ∣QG∣ increases. We have now conducted experiments to examine this directly. We compare samples from the same template with ∣QGn1∣>∣QGn2∣, independently rank their absolute scores, and compute the componentwise ratios rk=skn1/skn2. If leading scores grow proportionally more than lower-ranked scores, these ratios should tend to decrease with rank k. We measure this trend using Spearman rank correlation (SRC).
We restrict the analysis to the top 1000 scores to avoid the extreme tail, where dividing by tiny scores can amplify numerical errors. Within this range, the mean SRC is significantly negative in all six model–task setups, supporting this preferential growth (Appendix AC.2, Table 14, lines 3112–3149). Thus, selective contribution scaling remains a correlation-based explanation, not an established causal mechanism.
Both selective contribution scaling and the sparse coding interpretation are empirical hypotheses.
Selective contribution scaling: The limitations of the above SRC-based evidence are discussed in Appendix AC.4 (lines 3327–3337). We prove that if rk is non-increasing across the full score curve, then μn1≤μn2, matching the direction of the first arrow in Equation (7) (Proposition AC.1, lines 3089–3096). However, a negative SRC is weaker than monotonicity and does not guarantee this ordering. Moreover, our analysis includes only the top 1000 scores, so the observed trend may not extend to the full curve.
Sparse coding: The connection to sparse coding is also a hypothesis. To support it, we examine edge-rank fluctuation (displacement) across samples and find that leading edges generally have smaller rank fluctuations than lower-ranked edges (Appendix AC.3, lines 3153–3166). Together with the preferential scaling at leading ranks, this motivates a tentative sparse-coding interpretation: differences in ∣QG∣ may be attributed primarily to changes in the contributions of a relatively small, recurring set of edges.
In summary, selective contribution scaling is an empirical hypothesis supported by SRC, and sparse coding is a possible interpretation of this phenomenon. Neither follows from partition consistency. They provide a mental image for understanding the first arrow in Equation (7), rather than an established mechanism. We have also removed the discussion of partition consistency from Section 6.2, as its discussion is no longer necessary.
Thank you for this question. It prompted us to refine our reasoning and conduct the SRC and rank-displacement experiments, enriching the discussion.
W1 & Q2: Extreme Unfaithfulness and Unnormalized Discrepancies
We agree that "largely benign" was too broad and hard to quantify. We have replaced it in the conclusion with: "Sample-wise variance in unfaithfulness need not reflect comparable variation in absolute error, because normalization can amplify differences across samples." (Section 7, lines 516–517.)