- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Concept Attribution Analysis. It is a method for decomposing attribute changes in LLM representation into split concept contributions. The authors study each concept can be identified when multiple concepts change together. They build the identifiability results under three assumptions: Local Directness, Conditional Richness, and Support Intersection. Based on the analysis, they develop a method that combining graph-based subspace initialization with block-sparse reconstruction to recover concept-specific contribution from paired representations. The experiments include synthetic Gaussian concept mixtures and real LLM evaluation. The results show improved contribution recover, semantic readout, reconstruction and intervention.
Critical strengths
- this paper studies an important problem in LLM interpretability. Understanding how indivisual concept contributes to attribute changes in representation space is meaningful.
- The theoretical analysis is detailed. This paper explicit conditions/assumptions when individual concept can be identified.
- The synthetic experiment has useful robustness analysis. And the real-world LLM experiment is meaningful on evaluating whether recovered contributions can actually influence model predictions
Critical weaknesses
- Insufficient ablation studies to isolate the main contributions. This paper introduce several components: graph-based initialization, subspace clustering, and multi-dimensional block-sparse reconstruction. However, there is no experiment and ablation study to isolate the contribution of each component. In synthetic experiment, there is no experiment to compare the proposed method with SSAE.
- Limited real-LLM evaluation. This paper studies Pythia-70M and Qwen3-1.7B. But it is unclear whether the results generalize to larger LLMs.
Questions for authors
- The proposed method combines graph-based initialization, subspace clustering, and multidimensional block-sparse reconstruction. Can the authors provide an ablation study to isolate the contribution of each component?
- Since the synthetic experiments provide ground-truth concept contributions, can authors include SSAE as a baseline in these experiments? A direct comparison would help demonstrate whether ConAn offers an advantage over existing methods in recovering individual concept contributions.
- In the synthetic experiments, graph-based and random initialization use different decoder-freezing schedules. can the authors compare them under the same training schedule?
- The real-LLM experiments are limited to Pythia-70M and Qwen3-1.7B. Do the authors have plan to test on larger LLMs to show the proposed method remains effective?
2 comments
3. Interventions on a larger model
"Do the authors have plan to test on larger LLMs to show the proposed method remains effective?"
On Qwen3-4B, graph-initialized ConAn achieves higher contribution-transfer success than SSAE and Top-K SAE both at the predicted magnitude and after norm matching. While we initially used smaller backbones, as is common in SAE research, to facilitate controlled semantic evaluation, we followed your suggestion and extended this experiment to Qwen3-4B to test whether contribution transfer remains effective on a larger model.
Using the original texts and data splits, we extract Qwen3-4B representations and transfer a donor contribution to another context on the same test samples, either applying the predicted contribution directly or rescaling it to match the norm of the donor's full representation difference.
| Method | Transfer success at original magnitude | Transfer success with norm matching |
|---|---|---|
| Graph-initialized ConAn | 63.41% | 94.52% |
| Randomly initialized ConAn | 51.75% | 94.61% |
| SSAE | 29.64% | 84.56% |
| Top-K SAE | 51.45% | 79.06% |
Graph initialization improves transfer when the predicted magnitude is retained. Although Top-K SAE has lower reconstruction error than graph-initialized ConAn (0.0329 versus 0.0447), its transfer success is lower under both evaluations.
The tables report means, with full statistics and protocols provided in Appendix H.2 (ablations), H.3 (SSAE), and H.5 (Qwen3-4B).
Thank you for your positive assessment of our theoretical analysis and experiments, and for suggesting the additional comparisons.
1. Component ablations and a shared training schedule
Following your suggestion, we compared all four initialization variants under a shared training schedule of 5,000 steps with the dictionary frozen and only the encoder trained, followed by 5,000 steps of joint training. Full initialization retains its advantage in contribution recovery under this schedule, achieving a contribution FVE of 0.9914 compared with 0.0877 for a random dictionary.
We originally compared full and random initialization to evaluate the procedure as a whole, because the graph's sample-membership estimates and the subspace clustering jointly determine the initial concept spaces. We used the initial frozen-dictionary phase so the encoder could learn to allocate contributions within those estimated spaces. Your suggested comparison lets us examine the two stages more closely by adding one variant that constructs a dictionary from graph-derived directions without subspace clustering or subsequent refinement, and another that replaces the graph-based memberships with random sample groups while retaining clustering. We train all four variants with the same multidimensional block-sparse objective at d=256,m=32,r=6, with 2–4 concepts changing in each pair.
In this table, contribution FVE measures recovery of individual concept vectors, whereas reconstruction FVE measures recovery of their sum. Full initialization already captures the concept spaces well before training (0.9622 subspace overlap) and reaches 1.0000 as block-sparse training learns to allocate contributions. While both stage-replacement variants also improve during training, their lower contribution FVE confirms the benefit of retaining both initialization stages. The random dictionary's high reconstruction FVE alongside low contribution FVE shows that the advantage of full initialization lies in recovering the individual contributions even when the total change is reconstructed well.
2. Synthetic SSAE comparisons
In the added synthetic comparison, SSAE recovers contribution vectors very accurately when the true dictionary has full column rank, while ConAn has a substantial advantage in the overcomplete setting. Although we originally compared the methods through semantic tasks on real models because SSAE's scalar atoms and ConAn's multidimensional blocks represent different units, expressing both methods' outputs as concept vectors provides the direct comparison with ground-truth contributions that you suggested.
Both methods use the same training and validation data and the same initial dictionary of 32 groups of six directions, with two to four concepts changing in each pair (the true dictionary has full column rank at d=256 and is 1.5× overcomplete at d=128). Because SSAE treats the 192 columns as scalar atoms under its native signed affine model and constrained objective, we evaluate its outputs under the same concept-vector metric as ConAn by summing the six atom contributions within each group fixed before training, preserving the learned directions and coefficients without rotation or regrouping.
The table reports the best tested SSAE contribution recovery at each dimension.
At d=256, SSAE achieves slightly higher contribution FVE than ConAn, though it predicts small nonzero contributions for many concepts that did not change. Under the shared contribution-norm detection threshold, these false positives contribute little to the vector error but lower support precision (the fraction of detected concepts that actually changed) to 9.57%. At d=128, by contrast, the best tested SSAE setting reaches a contribution FVE of only 0.3333 compared with ConAn's 0.9825, showing a clear advantage for ConAn when the dictionary is overcomplete.