- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The authors take a deep dive into the limitations of an affine encoder (linear + bias + sparsification activation) in SAEs. In particular, they show that for overcomplete dictionaries, where it is known that the cosine similarity between at least one pair of concepts will exceed zero, it is not possible to represent arbitrary sparse code distributions with such an encoder. This has some important consequences for downstream applications of SAEs. In particular, activation edits are not self-contained to a single concept. The authors propose a new encoder architecture combined with a regularization term to resolve this, and try to show that it has the desired effect in real language model activations.
Critical strengths
The authors’ core claims are correct and important. The limitations of an affine encoder are known, and yet their use seems to persist in the SAE community. I think a direct and clear exposition of these limitations like in this paper has real value, particularly with a new technique that helps address them.
The majority of the writing is extremely clear. I understood the core of the issue from the start, and each conclusion logically follows from the previous ones. It was, mostly, a pleasure to read.
The theoretical propositions are fairly mechanical and not too difficult; the proofs seem correct to me.
Critical weaknesses
The issues I have with recommending acceptance outright is (a) the originality of the idea, and more importantly (b) the limited experiments, particularly relative to these other works.
(a) The difficulties of affine + sparsifier encoders for amortized sparse coding were discussed as early as Gregor & LeCun (2010), which the authors appropriately cite. There are at least two additional recent papers which also address this problem specifically in SAEs but are not mentioned by the authors [1, 2] which I strongly recommend they cite. With that said, these papers are very recent, and touch on other topics as well, so I’m not sure that their existence alone should count against acceptance. I would recommend the authors review them and consider citing them.
(b) In an experiment on toy-sized SAEs, the authors show that their alternative encoder parameterization better recovers the ground-truth coordinate which would otherwise be mixed, thereby achieving lower reconstruction error as well. In LLM activations, they show that their models can achieve better reconstruction performance (EV) and better pre-post edit re-encoding error (RMS) a good fraction of the time, as opposed to techniques not using their encoder + regularizer.
In my opinion, these experiments are insufficient. For example, consider that [1] did much more extensive experiments, investigating both what they call the “amortization gap” using per-sample methods (which encompasses this affine limitation, but also others, which they also investigate experimentally). [2] also did extensive experiments, in both synthetic regimes of a more practical size than this submission’s and real LLM activation regimes. Importantly, they evaluated using a standard benchmark suite for SAEs, which I think is necessary here too [3], if only to show that the proposed models are reaching the concept-learning regimes relevant to real-world practice.
I think these experiments are necessary, because notably Gregor & Lecun (2010) settle on a fairly domain-specific architecture (namely, an iterated soft thresholding approach), so it would be somewhat surprising and very notable if the authors’ MLP + regularizer works for amortized sparse coding as a simpler architecture. On the other hand, these experiments would likely need to be somewhat more careful than those in the current text: SAEs have convergence which is notoriously difficult to assess, and there’s no evidence the authors did so (no loss curves, and the EV metrics feel surprisingly low for me on the real LM experiments). Furthermore, the main evaluation metric besides EV is the thing being regularized, which I think further supports that SAEBench or similar qualitative metrics would be useful here. Head-to-head comparisons with e.g. MP-SAE (cited in the submission but not compared to) or iSAE-ME [2] would be of substantial interest here as they both aim to tackle the same issue, at least in part.
[1] Barin-Pacela et al. Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalization. UAI 2026
[2] Nelson et al. Toward Identifiable Sparse Autoencoders. ICML 2026
[3] Karvonen et al. SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability. ICML 2025
Questions for authors
- Can you compare your work to [1] and [2] mentioned above to better position it in the literature, which discuss the affine encoder issue to varying degrees?
- Can you add more in-depth synthetic experiments of the kind in [1] or [2] or of your own design? If not, why?
- Can you add experiments which interrogate the “quality” of the SAE representations in the context of LLMs, such as with SAEBench [3]? If not, why?
- Can you add head-to-head experiments which compare to other methods aimed at tackling the same challenge?
- Can you add informal statements of your theoretical results to the main text in context so that the readers can better understand them?
3 comments
Response to Reviewer x3zM (Part 2/2)
Q4. Head-to-head comparison. MP-SAE and iSAE-ME address the encoder's limitation by improving sparse inference itself. Our paper studies how the coefficient error affects editing and attribution (Q1, point 2). The comparison therefore tests whether better sparse inference also gives higher edit consistency. Reconstruction and edit consistency are different requirements. Reconstruction requires that decoding the code returns the activation. Edit consistency requires that re-encoding the edited activation returns the edited code. Since an overcomplete decoder maps different codes to the same or nearly the same activation, an encoder can reconstruct well and still not return the edited code.
We trained an iSAE-ME-style multistep encoder and an MP-SAE (Costa et al., 2025) in the Section 7 TopK settings for 30K steps. The multistep encoder is with and the aRIP regularizer of Nelson et al. This encoder applies TopK at every step and MP-SAE runs matching-pursuit iterations per token, so both are compared in the four TopK settings. Appendix I.2 reports this table and lists the iSAE-ME-style encoder as Multistep+aRIP. Each cell gives EV ↑ / ↓:
| Encoder | GPT-2, K = 50 | GPT-2, K = 100 | OPT-125M, K = 50 | OPT-125M, K = 100 |
|---|---|---|---|---|
| Affine | 0.643 / 0.0324 | 0.747 / 0.0173 | 0.799 / 0.0355 | 0.871 / 0.0220 |
| Cross+LC (λ_LC = 0.1) | 0.643 / 0.0299 | 0.746 / 0.0171 | 0.815 / 0.0326 | 0.884 / 0.0208 |
| iSAE-ME-style | 0.700 / 0.1827 | 0.806 / 0.3835 | 0.865 / 0.1386 | 0.945 / 0.0983 |
| MP-SAE† | 0.719 / 2.398 | 0.831 / 3.678 | 0.863 / 1.435 | 0.935 / 2.277 |
†MP-SAE has no candidate values separate from its code, so its error is computed on the encoded coefficients after selection (, Appendix H.2). On that quantity, in the four settings in table order, Affine has 0.231, 0.127, 0.181 and 0.110. The iSAE-ME-style encoder has 0.795, 1.060, 0.450 and 0.241.
Cross+LC is the most consistent encoder in all four settings (bold). In our runs, unlike in the synthetic setting of Q2, the iSAE-ME-style encoder and MP-SAE reconstruct best but are the least consistent. This is in line with Nelson et al.'s observation that iSAE-ME reconstructs best on LLM activations while identifiability worsens.
SAEBench results in the same four settings are in Appendix I.1, with the number of settings in which the encoder is better than Affine in parentheses:
| Encoder | Sparse probing, top-1 accuracy ↑ | CE increase ↓ | SCR ↑ | TPP ↑ |
|---|---|---|---|---|
| Affine | 0.689 | 0.013 | 0.095 | 0.119 |
| Cross+LC (λ_LC = 0.1) | 0.692 (4/4) | 0.012 (2/4) | 0.111 (4/4) | 0.122 (2/4) |
| iSAE-ME-style | 0.713 (4/4) | 0.009 (4/4) | 0.068 (2/4) | 0.099 (1/4) |
| MP-SAE | 0.706 (4/4) | 0.008 (4/4) | 0.094 (2/4) | 0.071 (0/4) |
The iSAE-ME-style encoder and MP-SAE are the two best on sparse probing and CE, in line with their higher EV. Both are below Cross+LC on SCR and TPP, which measure the effect of ablating selected latents.
Q5. We thank the reviewer for this suggestion and have followed it. In the revised submission, each of Sections 3–6 opens with a Summary paragraph that states its result informally, before the formal statement.
Convergence and EV (weakness b). We evaluated held-out EV at all saved checkpoints of 84 Section 7 SAEs, one per setting and encoder (Appendix I.3). Mean EV over the 12 settings:
| Step | 1K | 2K | 5K | 10K | 20K | 30K |
|---|---|---|---|---|---|---|
| Affine | 0.674 | 0.690 | 0.695 | 0.699 | 0.699 | 0.707 |
| Scalar | 0.673 | 0.688 | 0.694 | 0.699 | 0.700 | 0.706 |
| Cross | 0.682 | 0.694 | 0.700 | 0.711 | 0.719 | 0.727 |
| Cross+LC (λ_LC = 0.1) | 0.679 | 0.690 | 0.699 | 0.708 | 0.714 | 0.724 |
| Settings with Cross above Affine | 8/12 | 9/12 | 9/12 | 12/12 | 12/12 | 12/12 |
EV still rises slowly at 30K steps, so we do not call the runs converged. However, from 10K steps on, the cross-coordinate encoder has higher EV than Affine in all 12 settings. The lowest EVs are in the sparsest GPT-2 settings (width 4096, about 123M training tokens).
I appreciate the authors’ efforts to address my concerns. Due to the length of the authors’ response, I will summarize their answers to my questions and ask them to please correct me if I am wrong.
Q1. Philosophical comparison
The main contributions over prior work is (1) an exact analysis of the per-coefficient affine encoder gap, (2) the round-trip edit error idea, and (3) the impact of the affine encoder gap on the decoder.
Q2. Synthetic experiments
The authors have added synthetic experiments which demonstrate that their proposed “cross” encoder exhibits better coefficient recovery than an affine encoder when the dictionary is fixed, but worse than iterative encoders (and presumably per-sample encoders like MP-SAE, based on the response to Q4). The benefit over an affine encoder declines somewhat when concepts have more coherence.
Q3/Q4. LLM activations
The authors have added experiments on real-world LLM activations which show that their method does not substantially improve explained variance (i.e. reconstruction MSE) in real-world applications over an affine encoder. One or two SAEbench metrics may improve despite this, which is potentially interesting.
Q5. Convergence
None of the SAEs in the study as presented are converged.
My takeaway
The authors have improved the positioning of their work relative to early literature, but the results are not particularly impactful, and so I cannot recommend acceptance. On soundness, I am also still not convinced that round-trip edit error is a meaningful metric in practice, which is the primary focus of the paper, nor am I confident that presenting results from unconverged SAEs is reasonable.
While the proposed encoder may slightly edge out the typical affine encoders in certain synthetic settings, its performance gains are extremely marginal and of unknown significance in real-world activations. In general the approach is outperformed by iterative and per-sample encoders, which is to be expected from the literature the paper cites. The one exception are two unlearning metrics where there is some evidence of marginal gain; based on my prior knowledge I would anticipate this is due to the MLP’s ability to learn a different feature scale.
I would focus on validating the meaningfulness of the round-trip edit metric (what consequences does a poor round-trip editor have on metrics that people care about when using SAEs?) and standardizing their experiments with the broader literature (non-convergence is not good, for example; 30k iterations seems awfully low). In my view, the theoretical claims (that affine encoders are insufficient) are perhaps unique in their presentation/precise results but not pedagogically novel in light of the extensive literature on this issue, nor do they motivate a particular solution compared to earlier results about the failures of affine encoding.
Response to Reviewer x3zM (Part 1/2)
We thank the reviewer for finding the core claims correct and important and the exposition clear. We answer the reviewer's questions and address the weaknesses below. The additions described here are in the revised submission, where changes are marked in red.
Q1. Compare with Barin-Pacela et al. and Nelson et al. to position the work.
Our paper makes three original contributions, none of which appears in Barin-Pacela et al. or Nelson et al.
Where the coefficient error comes from (Section 3). Barin-Pacela et al. show that SAE encoders recover the latent factors less accurately than per-sample sparse inference. Nelson et al. state that a near-linear encoder is not expressive enough even with a fixed dictionary. O'Neill et al. (2025, Theorem 3.1) prove this limitation by a rank argument.
We are instead the first to derive the error coefficient by coefficient. Each encoded coefficient contains, besides its own term, the contributions of the features active with it, and these contributions are the source of the coefficient error. We also give an exact condition for when a coefficient is recovered and a lower bound on its error.
How the coefficient error affects editing and attribution (Sections 4, 6 and 7). To our knowledge, we are the first to show that the coefficient error compromises the attribution of edits to individual features. Feature labels and the effect attributed to an edited coordinate are derived from encoded coefficients, which contain the contributions of the other active features. Based on the SAE's encoded coefficients, the effect of an edit therefore cannot be attributed to the edited feature alone. We also introduce a check for this, edit consistency (RRMS), which can be computed for any trained SAE, and an encoder that improves it, Cross+LC.
How the coefficient error affects the decoder (Section 5, Appendix G). We are the first to identify decoder absorption, which is caused by the coefficient error of point 1. During training, the decoder directions move away from the reference directions to compensate for the coefficient error, even when the active support is correct. This may explain part of the finding of Barin-Pacela et al. that SAE-learned dictionaries point in wrong directions. In addition, feature absorption, a well-known SAE phenomenon named by Chanin et al. (2025), is a special case of decoder absorption, so our analysis also explains one way in which it arises.
The revised submission discusses these works in Section 1 and Appendix A.
Q2. Larger synthetic experiments. We ran synthetic experiments in the setting of Nelson et al. (d = 768, m = 4096, 40 active coordinates per input; their i.i.d. and mixture dictionaries), now reported in Appendix I.4. With the decoder fixed to the true dictionary, the cross-coordinate encoder has a lower coefficient error than Affine. The iSAE-ME-style encoder of Q4, here without the aRIP regularizer and listed as Multistep in Appendix I.4, is nearly exact:
In these experiments, the data are generated from a known sparse dictionary. On LLM activations the true features are unknown and the dictionary must be learned. However, as Barin-Pacela et al. find, SAE-learned dictionaries point in wrong directions, and per-sample inference on such a dictionary does not close the gap they measure. From the synthetic results, we therefore cannot tell whether an encoder with better inference also has higher edit consistency on LLM activations. Q4 tests this directly.
Q3. SAEBench. As the reviewer notes, edit consistency is the quantity that LC regularizes, so metrics not used in training are needed. SAEBench results for the Section 7 SAEs are added as Appendix I.1, with the number of settings in which the encoder is better than Affine in parentheses:
Both cross-coordinate encoders stay within 0.006 of Affine on each of these four metrics, so the gain in edit consistency does not come at a cost on metrics not used in training.