- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a unified optimality framework to theoretically explain feature splitting and feature absorption for SAEs. It derives necessary local-optimality conditions on learned SAE features and residuals. It also analyzes the wide-dictionary limit, showing that increasingly wide SAEs tend to refine representations toward ray-based partitions of the data. Empirical examples are provided to aid the understanding of the theoretical conclusions.
Critical strengths
-
This paper investigates an important problem of theoretically explaining feature splitting and feature absorption for SAEs.
-
This paper proposes a unified optimality framework. Particularly, the feature–residual constraints shown in Section 5 seem novel enough.
-
The theoretical conclusions are justified through empirical examples.
Critical weaknesses
-
The presentations of the theoretical results are a bit unclear. It would be necessary to formulate them clearly as assumptions, theorems, and proofs.
-
The assumption of no optimization gap is not clearly listed as the other assumptions. And empirical justifications on whether this optimization gap is small enough are lacking.
-
Although "relating and generalising the findings", some of the theoretical insights are already discussed in prior related works, which somewhat limits the novelty of the proposed theoretical framework. E.g. Chanin et al. (2024) "show mathematically that if the low level feature activates only when the high level feature is active, feature absorption decreases the loss" and Cui et al., (2026) discuss that their theoretical results "can partly explain the feature absorption phenomenon because non-overlapping sparse features are more recoverable". (The relation with the latter one should also be discussed in Section 4.)
Questions for authors
-
How large is the optimization gap empirically?
-
Can the theoretical analysis explain what kind of parent features are more likely to split than others under a given number of latent dimensions? E.g. is a parent feature with more diverse child features more likely to split? Such a result could further strengthen the paper, as some of the theoretical insights are already known from prior related works.
5 comments
Response 2/2
Q2: Which parents split first?
This is a good question. Our theory is a necessary set of conditions that every optimum, local or global, must satisfy. This is what says that a parent and child feature simply cannot co-exist at an optimum. However, when there are fewer SAE neurons than features, different clusterings of child features can form local minima and our theoretical results merely state that they are allowed, rather than ranking them. Enumerating which clustering is globally optimal is a separate problem that is non-trivial.
A simple extension is to then calculate the value of the loss at each of these local minima, and check what that says about which features will split. In short, subdividing a parent into two is a gain when it contains children that point in different directions and occur often, matching your intuition that diverse children lead a hierarchy to split in narrower SAEs than more homogenous ones. We tried a little set of toy experiments that demonstrate this point which we include in the anonymised git repo and are happy to include if interesting enough. In short, they show that adding latents to the dictionary often but not always splits according to a greedy heuristic such as ‘split the latent that can be subdivided to maximal gain’. That said, the full problem is a combinatorial clustering search, matching the observation that at small SAE widths the problem is non-convex and making statements about the global optimum is challenging.
Thank you for the detailed rebuttal. It partly addressed my concerns. However, I have two concerns remain:
W1: I appreciate the proposed formalization of the theoretical results, but would prefer to see it incorporated into the manuscript during rebuttal rather than deferred to camera-ready.
W2/Q1: Clarifying the optimal-code assumption is helpful, but satisfying necessary optimality conditions does not demonstrate a small optimization gap. Direct empirical quantification would be more convincing.
I maintain my rating of 3 for now
Response 1/2
We thank the reviewer for their review and constructive suggestions.
W1: Formal statements
We agree. All results are derived in full in the appendices, but they are not formatted as theorem–proof statements. We will restate the main results as formal propositions with explicit assumptions for a camera-ready version.
W2: The optimal-code assumption
Thank you; we agree this should be stated alongside the other assumptions. We now list it explicitly in Section 3: codes are optimal for the dictionary, so there is no optimisation (amortisation) gap. We also refer back to it wherever we discuss the theory's limitations.
Q1: How large is the optimisation gap?
There is a measurable gap between the codes produced by a standard SAE encoder and optimal sparse codes. O'Neill et al. (2024) study it in detail, and show that more sophisticated inference, such as iterative sparse coding or more expressive encoders, outperforms standard SAE encoders at modest extra compute. For our purposes the relevant question is whether the structure we predict survives the gap, and the evidence suggests it does. The GPT-2 SAEs we analyse have standard ReLU encoders, yet show no violations of our conditions on their training distribution. In our new toy experiments, ReLU SAEs with jointly trained encoders reach the same absorbed solutions as idealised dictionary learning, while independent features are left in place.
W3: Relation to prior work (Chanin et al., Cui et al.)
Thank you; we agree these relations deserve a fuller discussion, and we have added a comparison with both works, including Cui et al., to Section 4. In short, we believe our results are a significant advance on existing SAE theory, permitting much more flexible application, as our range of applications shows.
As acknowledged in the paper, the core interaction between the data’s co-activation structure and L1 optimisation that produces hierarchical absorption and splitting was previously explored by Chanin et al. However, their result comes from a specific toy model built to explain exactly this phenomenon. Even for absorption, eq. (6) goes further. Chanin et al. analyse one constructed path towards absorption in a toy with four orthogonal features of fixed magnitude. Eq. (6) shows that a dictionary in which one latent fires only when another does cannot be optimal, for any feature geometry and magnitude distribution, and it can be checked on trained SAEs. More broadly, our conditions apply to any dataset, and we use them to study not just hierarchical phenomena but range-independence, dense antipodal feature pairs, the structure of the residuals, and unbounded splitting in the wide limit. This is a significant extension of Chanin et al.’s results.
The price of this generality is that we idealise the encoder away. Tang et al. and Cui et al. (2026) model the encoder explicitly, but their results assume independently sampled sparse features, whereas ours make no assumptions about the data. This is what allows us to study hierarchical features directly. As discussed in our response to Reviewer dmvY and in new text in Section 4 and Appendix G, Tang et al. offer an explanation based on spurious (partial) minima that we do not think is sufficient.
Cui et al. note in their App. B that their recovery theorems hold only for features that are extremely sparse and do not overlap, and suggest this may partly explain absorption. In their example, however, a parent feature is defined as a positive combination of its children, so the two overlap in two ways at once: they point in similar directions, and the parent is always active with the child. Their analysis cannot say which of these causes absorption, and their theorems assume that features are active independently, which rules out a child that only appears with its parent. Our eq. (6) isolates the second effect: if one feature is only ever active when another is, a dictionary with a separate latent for each cannot be optimal, whatever directions the features point in (unless the two latents point the same way). Our new toy experiments (App. G) confirm this. We keep the features’ directions and frequencies fixed and change only whether the child appears independently or only with its parent; L1 dictionary learning and L1 SAEs absorb only in the second case. Cui et al.’s intuition therefore agrees with ours, and eq. (6) makes it precise and identifies the cause, something that lies outside their theoretical setting.
To our knowledge, no previous theory accounts for this range of SAE phenomena in a single framework, or provides a diagnostic that can be checked on trained SAEs. Hindupur et al. (2025) study the complementary question of what different encoders can represent; combining the two approaches is a natural next step.