acceptodds
Under review as a conference paper at ICLR 2027

A Unified Theoretical Framework of Sparse Dictionary Learning in Mechanistic Interpretability

Abstract

As AI models achieve remarkable capabilities across diverse domains, understanding what representations they learn and how they encode concepts has become increasingly important for both scientific progress and trustworthy deployment. Recent works in mechanistic interpretability have widely reported that neural networks represent meaningful concepts as linear directions in their representation spaces and often encode diverse concepts in superposition. Various sparse dictionary learning (SDL) methods, including sparse autoencoders, transcoders, and crosscoders, are utilized to address this by training auxiliary models with sparsity constraints to disentangle these superposed concepts into monosemantic features. These methods are the backbone of modern mechanistic interpretability, yet in practice they consistently produce polysemantic features, feature absorption, and dead neurons, with very limited theoretical understanding of why these phenomena occur. Existing theoretical work is limited to tied-weight sparse autoencoders, leaving the broader family of SDL methods without formal grounding. We develop the first unified theoretical framework that formulates major SDL variants within a common optimization problem and show that, under sparse linear representation assumptions, its feature-wise approximation exhibits piecewise biconvex structure. We characterize the zero-loss conditions and spurious partial minima of this objective, exposing the underdetermined nature of reconstruction-based SDL and providing principled explanations for feature absorption and dead neurons. Furthermore, we demonstrate how to reduce this underdetermination with a novel technique, feature anchoring, as an extension of our theory.

Primary area:
interpretability and explainable AI
Keywords:
Sparse Dictionary Learning, Mechanistic Interpretability, Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_XUh1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper unifies several concept-extraction methods used in mechanistic interpretability, including SAEs, transcoders, and crosscoders. By approximating the objectives optimized by these methods as biconvex problems, the authors theoretically demonstrate the existence of spurious partial minima, providing a theoretical explanation for the phenomenon of feature absorption.

Critical strengths
  • The paper tackles an important and timely question. Although recent empirical studies have highlighted limitations of sparse autoencoders, their theoretical analysis remains relatively under explored. By addressing this gap, the paper may contribute to the development of more principled approaches in a field that is still largely driven by empirical findings.

  • The paper is clearly organized and very easy to follow. The figures are well designed and helpful for illustrating the different scenarios and clarifying the paper’s main ideas.

Critical weaknesses
  • A very important conceptual issue concerns the paper’s definition of Sparse Dictionary Learning (SDL). The method described by the authors is an amortized version of SDL [1], rather than SDL itself. Definition 3.3 is currently inaccurate and imprecise. Standard sparse dictionary learning is formulated as a bilevel optimization problem, whose inner problem is constrained in the case of SAEs [2]. Since this paper aims to provide a theoretical analysis, the authors should refine this formulation and clearly distinguish between standard SDL and its amortized version.

  • Theorem 4.1 appears to be derived by dropping all terms corresponding to k≥2k\geq 2 and putting them in the approximation bound. Thus, the analysis implicitly focuses on the case k=1k=1. This assumption should be stated explicitly rather than inferred from the proof. Related to the question raised below, I am also unsure whether the theoretical conclusions remain meaningful for larger sparsity levels. Even if a corresponding theory is not possible for larger kk, the authors should empirically evaluate whether L~SDL\widetilde{\mathcal{L}}_{\mathrm{SDL}} remains a good approximation for higher sparsity.

  • The final contribution presented in the appendix (including the linear representation benchmark, feature anchoring, and the analysis of dead neurons) is not discussed in the main paper. These analyses and results should be brought into the main text as they are part of the contributions. I did not consider Appendices G and H in my evaluation because they are currently never referenced in the main paper and appear to introduce new analyses rather than supplementing the main results. Without these empirical results, the paper is currently weak, as it remains unclear whether the theoretical findings are applicable and scalable.

All together, these current issues prevent me from recommending the paper for acceptance in its current form. However, I consider this direction of developing a theoretical understanding of concept-extraction methods to be important. If the authors address the points above, I would be happy to increase my score.

[1] Charles O’Neill and David Klindt, “Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders”, ICML 2025.

[2] Sai Sumedh R. Hindupur, Ekdeep Singh Lubana, Thomas Fel, and Demba E. Ba, “Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry”, NeurIPS 2025.

Questions for authors
  • How large is the bound in Theorem 4.1 empirically? Can the resulting approximation be considered accurate when sparsity is typically set to at least approximately 20 in practice, which is far from the sparsity level of 1 implicitly assumed by the theory?

  • Assumption F.1 states that the concept activations are mutually independent. However, in a hierarchical structure, concept activations are generally dependent. Moreover, the implicit sparsity-1 assumption in Theorem 4.1 seems incompatible with hierarchical representations, where the sparsity is typically greater than or equal to 2. How do the authors justify this mismatch?

  • The bias term is omitted for mathematical simplicity. Could the authors provide intuition for how including a bias term would affect their theoretical findings?

  • How do different regularization choices fit into Equation (4) of Definition 3.3?

  • In Definition 4.2, the usual assumption is ck(x)>0⇒p(x)>0c_k(x)>0 \Rightarrow p(x)>0 [3, 4]. Why should one assume that an active parent must always have an active child? This assumption does not appear to hold in the toy models considered in Matryoshka SAEs [3].

Minor Comments

  • The work appears to be related to instability in sparse autoencoders. If spurious minima exist, training with different random initializations might be expected to produce different solutions, but this connection is not discussed.

  • The notation is somewhat difficult to follow, with many variables of the form x...x_{...} and W...W_{...}. The authors might consider using z...z_{...} for sparse codes and D...D_{...} for the ground-truth dictionary.

  • Where is the factor 1/n1/n used in Theorem 4.1?

  • There is an unresolved “??” in Appendix G.

[3] Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda, “Learning Multi-Level Features with Matryoshka Sparse Autoencoders”, ICML 2025.

[4] Valérie Costa, Thomas Fel, Ekdeep Singh Lubana, Bahareh Tolooshams, and Demba Ba, “From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit”, NeurIPS 2025.

Reviewer_K7VvOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a mathematical framework for analyzing sparse dictionary learning (SDL) as a piecewise biconvex learning problem and uses it to characterize several issues observed in empirical SDL training. In the appendix they introduce feature anchoring, which is an approach that uses known ground truth features to improve ground truth feature recovery.

Critical strengths

The paper is well written and gives a sound background to modern applications of sparse dictionary learning. The writing is almost always clear and the diagrams are well made, which together make the paper pleasant to read. It has the admirable aim of giving a mostly empirical domain of work (SDL) a more formal foundation.

Critical weaknesses

The results and framework are of unclear significance. A few observations go into this assessment: The paper’s central contribution (characterizing SDL as a biconvex optimization problem) is a pre-existing classic framing of the core problem. See, for instance, ‘Efficient sparse coding algorithms’ by Lee et al. (2006) (https://papers.nips.cc/paper_files/paper/2006/file/2d71b2ae158c7c5912cc0bbde2bb9d95-Paper.pdf) (“Assuming the use of either L1 penalty or epsilonL1 penalty as the sparsity function, the optimization problem is convex in B (while holding S fixed) and convex in S (while holding B fixed), but not convex in both simultaneously”). See also ‘Neural Networks and Principal Component Analysis: Learning from Examples Without Local Minima’ (Baldi and Hornik, 1989) which made similar observations. There are only minor differences between these classic settings and the setting discussed by the authors. The upper bound on the error seems quite high, especially the n^2 term to the point where it might make sense to assess whether it is of practical relevance. It is not atypical for SDL to assume ns in the range of 2^10 to 2^16. It would be valuable to state typical values of this bound in realistic settings. The work needs to make several assumptions that significantly reduce how closely the formalism applied to real world settings. SDL has not lived up to its expectations for mechanistic interpretability, and its use for this purpose seems to be built on conceptually shaky foundations. For instance, it is not obvious that the data generating process of neural network activations is well modelled as a sparse linear combination of feature directions. Despite substantial effort by the field to salvage value from the approach, the approach does not appear to be an important priority for driving progress in the field. The extension (feature anchoring) aims to solve issues with this unsupervised learning method by introducing supervision. This partly defeats the purpose. That said, it uses some ground truth features to help recover of the remaining unknown ground truth features. But even then it provides only modest gains on recovering the remaining unknown ground truth features.

Questions for authors

What are typical values of the bounds in eq 12 and eq 16 in realistic settings? These would provide some indication of the practical significance of the results.

Reviewer_wLfVOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents a unified sparse dictionary learning (SDL) framework covering SAEs, transcoders, and crosscoders. Under strong assumptions about ground-truth features, the authors analyze the SDL objective and argue that it is underdetermined and can admit spurious solutions, potentially explaining polysemanticity and feature absorption.

The paper also introduces feature anchoring, which encourages selected learned features to align with known directions. Experiments on synthetic data, a Llama-3.1-8B SAE, and CLIP suggest that anchoring can improve feature recovery. Most experimental results are presented in the appendix.

Critical strengths
  1. The paper addresses an important problem by studying dead latents, polysemanticity, and feature absorption through the SDL objective. Its unified framework for SAEs, transcoders, and crosscoders is useful. It also highlights that low reconstruction loss does not guarantee correct feature recovery.

  2. Feature anchoring is a simple and potentially useful approach. The synthetic experiments show promising improvements across several SAE architectures, particularly for TopK and JumpReLU. Evaluating recovery only on non-anchored features is also a reasonable design.

Critical weaknesses

The main concern is that several key theoretical claims are either incorrect or not supported by the stated assumptions.

  1. Theorem 4.1: The sparsity condition 1−S≤1/n1-S \leq 1/n effectively excludes feature co-activation, which is central to superposition. At the boundary, the bound is no better than the trivial bound. Moreover, the constant CC can be infinite under the paper's assumed feature distribution.

  2. Theorem 4.4: The theorem only establishes that a sum of non-negative losses is zero when every term is zero. It does not prove non-uniqueness or underdetermination. The conclusion also relies on a singleton approximation rather than the full reconstruction objective.

  3. Theorem 4.6: The theorem assumes the existence of a blockwise minimizer and does not establish that it is a local minimum of the full objective. Therefore, it does not justify the claims about spurious minima or their prevalence.

  4. Theorem 4.8: The assumptions appear inconsistent. Definition 4.2 contradicts the independence assumption in F.1 and implies that some singleton events have zero probability, violating the Md>0M_d>0 condition required by earlier theorems. Furthermore, the constructed pattern describes feature splitting rather than the claimed feature absorption.

Questions for authors
  1. Theorem 4.1 and 4.4: Can you demonstrate non-uniqueness under the full LSDLL_{SDL}, including feature co-activation, rather than only under the simplified loss? How do you address the restrictive sparsity assumption in Theorem 4.1?

  2. Theorem 4.6: Can you prove that the constructed solutions are local minima of the full optimization problem rather than restricted optima or saddle points?

  3. Theorem 4.8: How do you reconcile the hierarchical activation rule in Definition 4.2 with the independence assumption and Md>0M_d>0? Does the construction demonstrate feature absorption rather than feature splitting?

  4. Experimental validity: What threshold was used in Table 4, and how do you explain the apparent inconsistency between MGT and MIP? Could you also provide baseline hyperparameter sweeps and additional CLIP controls (mean-centering and random anchors)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.