- Rating
- 3(+2)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DUNE, a post hoc unsupervised method for disentangling polysemantic units into more monosemantic subunits. For each neuron, DUNE clusters the top-k contributions of the previous (l-1) layer’s units and hypothesizes that each cluster corresponds to a concept. The authors create a subunit for each cluster and compute its incoming weights using the margin-based assignment rule. The model's original function is preserved by summing the subunits’ activations back into the original neuron. The method is compared against PCA, semi-NMF, various SAEs, and a transcoder across several models and achieves competitive results.
Critical strengths
- The empirical baseline is strong: the method was tested across 5+ models against 10+ methods with good results. This could be a nice addition to an XAI toolbox.
- The method is simple to use and implement which would make it easy for practitioner to adopt it.
Critical weaknesses
- There is little novelty to the paper. It is a realization of PURE (neurons are no longer virtual, and HDBSCAN replaces k-means) with ARV unit-normalization from Hesse et al., adapted to the unsupervised setting. As far as I can tell, the method is not compared with its two closest competitors, so it is hard to gauge the improvement PURE provides. The HDBSCAN vs k-means results in Table 14, which are closest to the PURE vs DUNE comparison, are mixed.
- Some parts are misleading or overstated. For faithfulness, the R^2 score is always 100% by design and is what Fig. 1 reports, which artificially inflates the gain over the Pareto front. While Hesse et al. are cited in related work, the ARV unit-normalization link is never mentioned, even though it is explicitly stated as the measurement used in D8.
-> The updates of the authors mostly adress my concerns
Questions for authors
- It would be great to have a comparison with PURE and Hesse et al. or unsupervised surrogates to really be able to measure the improvement of DUNE. This is partly done with the knn version of DUNE in the appendix, but having these 2 models or surrogates adapted to the task in Fig. 1 would help to easily see DUNE's contribution.
- In Fig. 1, the y-axis should be changed to explicitly state that it is R^2 and not faithfulness, or use another, less misleading measure. More generally, the perfect R^2 is presented as a key result, but it is always 100% by design, regardless of whether the split between subunits is relevant. As far as I understand, the random model in Table 14 should also have an R^2 of 100%. While this is discussed in the appendix, it would be great to make it clear in the main paper.
- The link with Hesse et al. ARV should be clearly stated.
4 comments
Official comment to Reviewer 5URp (2/2)
W2/Q2. R$^2$ as a misleading measure for Faithfulness and overstating perfect reconstruction by design.
To clarify the axis label of Fig. 1, we changed it to "Reconstruction faithfulness (R$^2$)"; we will update the wording throughout the paper accordingly. Reconstruction faithfulness is an established measure in related work; a non-exhaustive list of prior work confirming this statement includes [C1, C2, C3].
Furthermore, while DUNE achieves perfect R$^2$ by design, advancing the frontier on both monosemanticity and faithfulness is nontrivial, as proven by random disentanglement and the broad selection of our other baselines. To emphasize this point more directly, we have moved the comparison between DUNE and random disentanglement from the appendix to the main table (Tab. 2) and added a discussion in Sec. 4.1, as suggested by the reviewer.
Q3. Link to ARV attribution in Hesse et al.
Thank you for this suggestion. We already acknowledge Hesse et al. as prior work (ll. 141-144 in the initial submission) and made the connection to "ARV unit-normalization" clearer in Section 3.2 of the revision.
We hope the added comparisons and clarifications address your concerns, especially regarding novelty, and are happy to provide further clarifications if requested.
References
[C1] Rajamanoharan et al.: Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders. arXiv:2407.14435 [cs.LG]
[C2] Dunefsky et al., Transcoders Find Interpretable LLM Feature Circuits. In NeurIPS 2024.
[C3] Dhimoïla et al., Cross-Modal Redundancy and the Geometry of Vision–Language Embeddings. In ICLR 2026.
I thank the authors for their detailed explanation, I have updated my score accordingly.
Dear Reviewer 5URp, we thank you once again for your thoughtful feedback and positive assessment of our work.
Official comment to Reviewer 5URp (1/2)
Dear Reviewer 5URp, thank you for noting that "the empirical baseline is strong" and that DUNE is "simple to use and implement". We appreciate your request for comparisons with PURE and Hesse et al. and your suggestion to label reconstruction R² more precisely.
W1/Q1. There is little novelty to the paper. It is a realization of PURE (neurons are no longer virtual, and HDBSCAN replaces k-means) with ARV unit-normalization from Hesse et al., adapted to the unsupervised setting.
We agree that the relationship to PURE and Hesse et al.\ should be made clearer to emphasize our novelty. Our method introduces several design choices that address limitations of these approaches. PURE produces ``virtual neurons'' that are not part of the model's forward pass (l. 139 of the initial submission), which means it can not be used in applications that require explicit neurons, such as model steering (Sec. 4.3). Hesse et al. requires supervision, which makes extension to commonly used unsupervised probing datasets and non-classification models challenging, and was evaluated only on a single ImageNet-trained CNN (ll. 142-144 of the initial submission). Our approach provides an explicit, unsupervised decomposition (see Tab. 1). Its main methodological differences extend beyond the choice of clustering algorithm. DUNE removes the unexplained residual neuron retained by Hesse et al., assignes edges by groups rather than using their all-or-nothing assignments, and substitutes their supervised signal with an unsupervised one (see Sec. 3).
Following the reviewer's suggestion, in the revision, we have added direct comparisons with PURE and Hesse et al. (DPCiNN) to Table 2 as well as a discussion of results in Sec. 4.1 (paragraph on quantitative results). We initially excluded these methods because their virtual or supervised decompositions differ from our explicit, unsupervised setting, making a comparison to DUNE and the considered SAEs not entirely fair. Nevertheless, we agree that comparisons on shared evaluation criteria help to establish their relative strengths and limitations. To make Hesse et al. a suitable comparison, we extended their algorithm from a supervised ResNet-50 to an unsupervised DINOv2 ViT, using the official linear classifier head that was trained through supervised ImageNet-1k classification. Their evaluated construction achieves at most an expansion factor of 3, comprising two class subunits and an unexplained residual unit per parent neuron.
Compared with PURE, DUNE achieves a much higher C-Score (54.94 versus 64.85), indicating higher monosemanticity, without relying on the k parameter deciding how many clusters to build. In addition, DUNE's explicit decomposition supports interventions in the model's forward pass, as demonstrated by our steering experiment (Sec. 4.3). In contrast, PURE's virtual decomposition cannot directly support this application because its virtual units are not part of the forward pass. DUNE achieves similar semantic coherence to Hesse et al. without requiring supervision (65.17 versus 64.85). Furthermore, DUNE outperforms Hesse et al. by 12.09 percentage points on SIM@1 (22.00 versus 34.09). The SIM@1 score of Hesse et al. is decreased by the residual subunits, which score 9.32 compared with 28.33 for the class-conditioned subunits.
These comparisons show that DUNE improves on PURE in C-Score and supports direct feature interventions, while achieving comparable or improved semantic coherence to Hesse et al. in an unsupervised setting, without restricting each split to two classes or relying on an unexplained residual neuron.