acceptodds
Under review as a conference paper at ICLR 2027

High-probability guarantees for linear accessibility in feature superposition

Abstract

Neural networks can leverage feature superposition to encode more concepts than dimensions, but cross-feature interference constrains the linear accessibility of simultaneously active features. By framing linear accessibility as a compressed sensing problem, we derive high-probability recovery bounds for fixed supports under subgaussian noise, proving that dimensions suffice for the per-input regime in which networks typically operate in, in contrast to the quadratic scaling required by uniform worst-case guarantees. We characterize the asymmetry between active and inactive interference and the trade-off between interference and observation-noise budgets. We then validate these bounds across system parameters through numerical experiments based on Gaussian-tail approximations. We also introduce IHT-SAE, which uses learned iterative refinement to improve feature recovery beyond the limits of linear accessibility. These results quantify the geometric constraints of the linear representation hypothesis, providing a framework for evaluating sparse autoencoders, compositional generalization, and neural interpretability. Code to reproduce this work is available at: https://anonymous.4open.science/r/Linear-Availability-07CD

Primary area:
learning theory
Keywords:
linear representation hypothesis, feature superposition, linear accessibility, compressed sensing, sparse autoencoders, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2M9ROpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the theoretical limits of affine linear + activation SAE encoders (or, in compressed sensing parlance: "decoding") through the lens of sufficient dimension. In particular, it is shown theoretically that the d=O(k2)d = O(k^2) scaling dependence on sparsity kk for the embedding dimension dd is pessimistic, and under certain distributional and noise level assumptions it can be reduced to O(k)O(k). The authors note that an affine+activation mapping from embedding to sparse code may not be sufficient to attain these bounds, so they propose an IHT-SAE, which is an appealingly simple iterated version of TopK SAEs which re-encode the residual.

Critical strengths
  • The logical flow is clear. It seems a priori intuitively clear that the O(k2)O(k^2) bound on linearly accessible features is pessimistic, and it is nice to see a proof. The noise tolerance dependence on mm (the dictionary size) is perhaps new, but seems correct.
  • The simplicity of the proposed IHT-SAE is intriguing.
Critical weaknesses
  • The most substantial weakness I see is conceptual. The paper proves some bounds on linear decoding (in the sense of Definition 2: this means sparse recovery with a standard affine+activation-type SAE encoder). Then, they propose a non-linear decoding scheme (or "iterated linear"), and acknowledge that these bounds apply only per-iteration to the decoding scheme. "In what way do the bounds motivate this algorithm?" is a question left to the reader, and it's not clear to me even as someone who (I think) understands the work reasonably well what they are.
  • The proposed data-generating process/distributional assumptions do not seem especially relevant to the use cases of SAEs; nor are they measured empirically for any practical data distributions. For example, concepts are not in fact uniformly distributed on the sphere; they tend to have structure [1].
  • The paper cites and compares to MP-SAE [2], which seems appropriate, and compares favourably in sparse support recovery on synthetic data but is outperformed by MP-SAE on explanatory power/error, a fact which seems unsurprising given the greedy nature of MP-SAE, and not a point against the proposed IHT-SAE in my opinion. However, at least two other papers also consider iterated approaches to sparse recovery in conjunction with SAEs [3, 4] which may be worth citing and comparing to.
  • In general, I find the experiments somewhat unconvincing. For example, compare to [3] which has substantially more experiments (though admittedly less theory) or [4] which has comparable theory but much more extensive experiments, including benchmarks on real-world LLM activations through SAEBench [5].
  • The main benefit for IHT-SAE over MP-SAE is its ability to do a "full" sparse coding step each iteration rather than greedy selection of a single atom, at least on synthetic data. However, this is not shown to help on real-world activations.
  • SAEs are known to be difficult to train, with convergence particularly difficult to assess. Some mention of how this was confirmed across very different encoder types would be appreciated.

Some more minor issues, that are meant as helpful feedback:

  • The introduction did very little to orient me, and I think it needs to be reworked. How does stochastic separation relate to superposition (paragraph 1)? What does uniformity or concept subspace mean in this context (paragraph 2)? What do “strongly decaying” and “flatter” mean in this context (lines 52-53)?
  • What is RAM (lines 52-53)?
  • Section 2 and Definition 1 uses “encoding” to refer to going from sparse signal to dense measurements and “decoding” to mean recovery of the sparse signal, due to the use of these terms in the compressed sensing literature, presumably. While Figure 1 makes this clear, the authors might like to note that this is the opposite of “encoding” and “decoding” in an SAE, for readers more familiar with SAEs.

[1] Li et al. The Geometry of Concepts: Sparse Autoencoder Feature Structure. Entropy (2025)

[2] Costa et al. From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit. NeurIPS 2025

[3] Barin-Pacela et al. Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisation. UAI 2026

[4] Nelson et al. Toward Identifiable Sparse Autoencoders. ICML 2026

[5] Karvonen et al. SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability. ICML 2025

Questions for authors
  1. Can you explain the relationship between the theory and the experiments with IHT-SAE? Does the theory motivate IHT-SAE? If so, where can I find this in the paper?
  2. Are the assumptions in the theory measurable on real-world activations from e.g. LMs or VMs? If so, are they met? If not, could they explain the discrepancy in performance between synthetic and real data?
  3. If it is correct to say that the main benefit of IHT-SAE is its ability to do a "full" (i.e. multi-coefficient) sparse inference step each iteration? If so, is it appropriate to compare to iSAE [4] or other models in the style of Gregor and Lecun (2010) -- which are becoming increasingly popular alternatives to affine+activation SAE encoders?
2 comments

I thank the authors for taking the time to do additional comparisons.

My position on the novelty of the theory and its use to motivate the experiments has not changed. [1] has done exactly what this paper has done, for example, but arguably with more immediate application to SAEs. What I mean to say is: they have (a) proven a result showing that affine encoders cannot possibly be sufficient for SAEs to achieve their stated goals in practice and (b) used it to motivate bringing sparse coding/dictionary learning results to the realm of SAEs for improved performance. This paper's theory is of a different type, of course, but the takeaway is the same.

This work’s main differentiation is that it proposes an entirely new type of SAE, which is exciting: but it does mean, for me, the paper’s empirical contributions need to be absolutely on point. At least, they need to be on par with its competitors MP-SAE and iSAE-ME, and maybe other recent baselines like ConSDL [2], LocA-SAE [3] or Spatial-SAE [4] if using VLM/ViT representations rather than just LLMs. I am aware that some of these methods are outside the eligibility window for comparison, and am not asking for comparisons, but I reference them to provide evidence of the level of empirical quality in this area.

Given that IHT-SAE performs worse than MP-SAE in real world data, its use would need to be justified somehow: is it more computationally efficient? Is there hope for improvement with e.g. more careful tuning?

Other experimental issues:

  • In general, the experiments are somewhat underspecified, including details on hyperparameters. Convergence of SAEs is challenging to assess and it's not clear that's been done properly.
  • Standardized training regimes are helpful for cross-method comparisons. This means apples-to-apples model fitting procedures and also measuring not just e.g. reconstruction versus sparsity, but also other “domain-specific” metrics that people who use SAEs care about, like those in SAEBench [cited in my original review as 5] or similar. My sense is that there is some limited evidence that there is some tradeoff between SAE encoder expressivity and sparse code quality according to some domain-specific metrics.and is not at all discussed. I would urge the authors to review the papers above and consider how they reported their empirical results.
  • It is not clear whether iSAE or iSAE-ME was used. The new results would be extremely surprising given iSAE-ME’s documented superior reconstruction performance on LLM activations over TopK SAEs.

As discussed, this is now an increasingly popular research area. Many of these methods are recent, so the authors may not be formally obliged to compare to them. But certainly at a minimum the reader needs to be convinced that best empirical practices have been followed (the referenced papers generally do this by reporting training details, etc. -- some of which have been added by the authors in the latest revision, but many are still missing), and in my opinion the use of standardized SAE benchmarks of some kind is likely necessary.

[1] O’Neill et al. Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders (2024).

[2] Choi et al. Beyond SAEs: Consistent Concept Discovery with Sparse Coding (2026).

[3] Sun et al. The Price of Amortized Inference in Sparse Autoencoders (2026).

[4] Mencattini et al. The Independence Prior of SAEs Fragments Visual Concepts (2026).

We thank the reviewer for a fast reply and feedback.

[1] proves that standard SAEs are effectively incapable of robust nonlinear decoding. We explain why that is the case by characterizing linear decoding that makes up the standard SAE performance. These are not equal claims.

We understand that from the perspective of developing the best possible SAE, the outcome may seem the same -- just another SAE which hasn't been empirically shown to be superior to SoTA on benchmarks.

We do not seek to compete with newly developed SAEs because the best SAE likely depends on the model being probed. But when studying the history of nonlinear decoding, then it falls into 2 main categories. Either feature by feature, or all features at once. The first approach, feature by feature, like MP-SAE does, here we need to know the correct sparsity beforehand or somehow dynamically determine a stopping point, which can be difficult but not impossible, so great results can still be achieved when not knowing the sparsity of vector being analyzed. The all at once approach decodes all k sparse features at the same time, like iSAE and IHT-SAE. Here, we show superior performance to iSAE by tying the unexplained residual to the decoder instead of learning it. But that is a secondary contribution. These claims are summarized on Figure 6 with exact supporting evidence. For clarification, we compare against iSAE-ME, the iterative variant but without the aRIP regularization to keep the comparison fair -- it's possible the additional regularization would help, but that is a limitation of iSAE and not our work.

Again, our experiments are there to support the theoretical claims made in our paper instead of showing that it's the best SAE, so introducing SAEBench doesn't serve a purpose with respect to our claims. We hope that you see our perspective.

(We checked up to 100,000 steps of iSAE-ME and on synthetic tests and VLM activations, and 40,000, which they also used, was enough for convergence. It may be insufficient for LLMs, but that is a limitation of iSAE.)

We are happy to discuss the best SAE architecture, but we hope that we've made clear the main point of our paper -- showing the behavior and limits of linear decoding to an exact constant -- the content up to section 4.

Please let us know if this clarifies things?

Best, Authors

Reviewer_N13nOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies linear accessibility under neural‑network feature superposition, distinguishing between global worst‑case and fixed‑support high‑probability conditions. Under sub‑Gaussian noise, it derives an Oε(klog⁡m)O_\varepsilon(k\log m) dimension upper bound for linear decoding, which applies only to fixed‑support settings and does not overturn the original worst‑case O(k2log⁡m)O(k^2\log m) bound. It further proposes the iterative sparse autoencoder IHT‑SAE to improve feature recovery. However, theoretical assumptions cannot be perfectly satisfied by real‑world model dictionaries.

Critical strengths

It clarifies two distinct guarantee regimes for linear accessibility and resolves the mismatch between prior worst‑case theory and practical empirical observations on real models.

Critical weaknesses
  1. There exists a notable gap between theoretical assumptions and real‑world models. The theorem assumes feature dictionaries are uniformly sampled from the unit sphere, while dictionaries learned by large models violate this condition. Hence the high‑probability bounds are only suggestive and cannot be directly applied to real‑world SAE and LLM systems.
  2. The proposed IHT‑SAE delivers incremental improvement rather than architectural novelty. Its core iterative building blocks are adapted from existing IHT and LISTA methods. No novel sparse‑recovery module is introduced, and it lacks theoretical convergence guarantees.
Questions for authors

See the above weakness.

Reviewer_y1jXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper shows that previous results demonstrating a gap between linear decoding and non-linear decoding under the linear representation hypothesis is closed when considering average-case instead of worst-case error.

The paper then proposes a variation of SAE methods where iterative hard thresholding (a standard sparse coding method) is used in the encoder.

Critical strengths

The paper's theoretical result illustrates a meaningful point: that past results illustrating a gap between linear and non-linear decoding may not be as severe when looking at average-case error. This improves our understanding of the linear representation hypothesis and superposition.

Critical weaknesses

The paper's methods contribution is less clearly significant. The paper proposes an SAE variant by using an IHT-style (iterative hard thresholding) encoder. Past work has used a matching pursuit based encoder. My main concern is that this choice isn't clearly motivated by the theoretical results, which seem to suggest that ordinary SAEs may be sufficient. The empirical results would be more convincing if they demonstrated that IHT-SAEs improved performance on a downstream task.

Questions for authors

Can you explain the connection between the theoretical results and the proposed method? What does the theoretical result say about when one method should improve over the other?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.