- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces LISAE, an autoencoder that selects sparse latent activity through recurrent lateral inhibition. Encoder and decoder weights learn through reconstruction gradients, while inhibitory connections and thresholds follow local anti-Hebbian and homeostatic updates. The implemented reconstruction uses an amplitude-preserving gate: the equilibrium activity selects which feedforward activations reach the decoder. Experiments on MNIST, superposed digits, bar stimuli, and one layer of Gemma-2-9b-it show recognizable decoder features, input-dependent numbers of active latents, and structure in the learned inhibitory matrix. The language-model validation includes qualitative steering examples and semantic-group visualizations. The paper also proposes a theorem about realizable equilibrium supports.
Critical strengths
Interesting integration of local competition with reconstruction learning. The combination of anti-Hebbian inhibition, homeostatic thresholds, and gradient-trained encoder/decoder weights provides a concrete mechanism for sparse feature selection.
Critical weaknesses
-
It is unclear what LISAE adds over existing sparse autoencoders. Many SAE architectures (Vanilla [1], JumpReLU [2], BatchTopK [3], etc.) already allow the number of active features to vary with the input. The paper does not show that LISAE uses this flexibility better at similar sparsity and reconstruction quality. It also does not compare the semantic relationships in the inhibitory matrix with simpler measures such as activation co-occurrence or decoder similarity. These comparisons are needed to establish what the inhibition adds.
-
The language-model evidence is too limited to justify the cost. The paper trains one 7,168-latent SAE on one layer using 500,000 tokens, reused for 20 epochs, which is unusually small scale. Two steering examples do not establish monosemanticity or show how reliably steering works while preserving fluency. There is no quantitative comparison of feature quality or downstream fidelity with baseline SAEs. The dense inhibitory matrix also grows quadratically: at width 65,536 it would have about 4.3 billion entries, plus the cost of recurrent settling. The paper acknowledges this limitation, but reports no runtime or memory measurements and shows no advantage that would justify the cost.
[1] Bricken et al. (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. Transformer Circuits Thread, Anthropic.
[2] Rajamanoharan et al. (2024). Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders. arXiv:2407.14435.
[3] Bussmann, Leask, and Nanda (2024). BatchTopK Sparse Autoencoders. arXiv:2412.06410.
Questions for authors
- Does the inhibitory matrix reveal semantic relationships better than activation co-occurrence or decoder similarity?
- How many settling steps are used, and what is the runtime and memory cost compared with a feedforward SAE?
- Could you provide a quantitative comparison of feature quality or steering with a baseline SAE?
3 comments
Response to Reviewer 7d3b - Part 2
Part 2
-
Issue on "The language-model evidence is too limited to justify the cost." (continue): 3. "reports no runtime or memory measurements and shows no advantage that would justify the cost." We acknowledge the computational complexity limitations of LISAE and have added a new paragraph in the Discussion detailing the runtime and memory requirements, noting that the current memory footprint is not ideal for industrial scaling. However, this complexity is what allows for qualitatively different characteristics.
First, as shown in Section 3.2 and Appendix F, lateral inhibition serves as an adaptive L0 regularization that functions fundamentally differently from L1 penalty or fixed Top-K L0 penalties. Furthermore, this complexity allows the network to co-learn relational representations simultanously. All in all, this architecture does not merely add computational overhead; it introduces genuinely new information processing capabilities to the network. If one would really want to scale it up, since the $W_{LI}$ does not go through the backpropagation, its anti-hebbian training can be potentially moved to CPU to save GPU resources; but yes, that will be even slower. So a better engineering solution is definitely needed for industrial implementation.
Regarding specific questions
-
Does the inhibitory matrix reveal semantic relationships better than activation co-occurrence or decoder similarity? Please refer to our response in Issue 3. The short answer is no—it will not quantitatively outperform direct decoder similarity. Because the learned decoder features already clearly represent the distinct digits, directly calculating their similarity naturally yields an optimal result.
However, our primary purpose here is not to beat a direct mathematical calculation, but rather to demonstrate a novel mechanistic possibility: that this relational structure can be obtained dynamically through simultaneous co-learning within a single neural network. Our goal is to show that a network can autonomously learn both the individual features and the semantic relationships between them as a map at the same time. This capability may have broader implications. For example, computational neuroscientists may be highly interested in this architecture for learning internal "cognitive maps," which could potentially serve as a model for abstract planning in the hippocampus [2].
-
How many settling steps are used, and what is the runtime and memory cost compared with a feedforward SAE? Please refer to our response in Issue 4.3. We have added a thorough analysis to the Discussion section. As noted in the paper, the recurrent dynamics take approximately 20 settling steps to converge.
Currently, LISAE requires about 4 times more memory than a standard feedforward SAE. The runtime overhead is also significant and can be calculated as follows: while a standard feedforward SAE scales at (where is the input dimension and is the hidden neuron number), LISAE requires for inference,where represents the settling steps multiplying the lateral matrix. During training, the anti-Hebbian update for the lateral connections adds another outer-product calculation, meaning the total training runtime scales proportionally to .
Because is typically much larger than , this quadratic scaling is computationally heavy, and we agree it is not yet suitable for industrial scaling. However, as emphasized in Issue 4.3, this complexity is what fundamentally enables the qualitatively different computational capabilities of the network.
-
Could you provide a quantitative comparison of feature quality or steering with a baseline SAE? Please refer to our detailed response for Issue 4.2. We have added the corresponding comparison in Appendix G.
Lastly, I hope the impact of our work could also extend beyond conventional mechanistic interpretability community. For example, leveraging this L0 adaptivity, future work may achieve compositional and continual learning by inhibiting the right groups of neurons in real time via recurrent dynamics. It also highlights an interesting paradigm for algorithm design: allowing different learning and homeostatic rules to work synergistically to achieve complementary representational capabilities in one neural network.
[1]: Hindupur, Sai Sumedh R., et al. "Projecting assumptions: The duality between sparse autoencoders and concept geometry." Advances in Neural Information Processing Systems 38 (2026): 11649-11699.
[2]: Whittington, James CR, et al. "How to build a cognitive map." Nature neuroscience 25.10 (2022): 1257-1272.
Thanks for running the additional experiments and reporting the costs. LISAE underperforms Top-K on the reported Gemma quality metrics, and I don’t see enough evidence of an interpretability gain to justify the extra cost, therefore I decided to keep my rating.
Response to Reviewer 7d3b - Part 1
We thank Reviewer 7d3b for your constructive feedback. As there were shared concerns regarding scalability and quantitative benchmark reporting across the reviews, we kindly refer the reviewer to our "General response to common concerns raised by reviewers" for an overarching discussion of our motivation. Here, we will focus directly on addressing the specific issues raised in your review.
Part 1
Regarding specific weakness:
Issue on Adaptive Sparsity: The reviewer points out that existing methods (Vanilla, JumpReLU, BatchTopK, etc.) already allow the number of active features to vary with the input. To address this, we have added new experiments in Section 3.2 and Appendix F, training Vanilla, JumpReLU, and BatchTopK SAEs on our controlled, spatially superposed MNIST dataset.
Indeed, soft L1-penalty methods (Vanilla and JumpReLU) exhibit seemingly adaptive capabilities, as their activation counts do grow proportionally with input complexity. However, when inspecting the underlying learned decoders, their features are often uninterpretable, even though their reconstruction loss remains similarly low. Conversely, hard L0-penalty methods like BatchTopK (and standard Top-K in Figure 1a) lack this adaptive scaling, yet the learned features seem more interpretable. This stark contrast indicates that L0 regularization plays a fundamentally different role in shaping the network's inductive bias. LISAE distinguishes itself from all these baselines because it successfully imposes an adaptive L0 regularization. It achieves markedly better results when the dataset underlying structural complexity varies greatly, imposing a distinct inductive bias on what can be learned [1], the theoretical implications of which require further investigation.
Issue on "uses this flexibility better at similar sparsity and reconstruction quality": Again, our theoretical analysis and experiments in Section 3.2 and Appendices E, F could serve as an intial evidence that lateral inhibition imposes adaptive L0 penalty, achieving markedly better results when the dataset underlying structural complexity varies greatly, which may affect what utimately can be learned. Besides, we have shown LISAE has significantly lower number of inactive neuron in Section 3.1 due to the homeostatic regulation of firing thresholds, which is also an extra benefits of our model.
Issue on "no comparison of the semantic relationships in the inhibitory matrix with simpler measures such as activation co-occurrence or decoder similarity":
Issue on "The language-model evidence is too limited to justify the cost.":
First, as we mentioned to reviewer qxUE, we believe that demonstrating the ability to learn the relational structure in Figure 3b with limited data is actually a strength of our approach. If the decoder were not already largely monosemantic, the inhibitory connections would not be able to capture the semantic distances, and vice versa. I would argue that demonstrating the emergence of such a map under limited data conditions is arguably more persuasive than doing so after potentially overfitting the decoder on a massive dataset.
"no quantitative comparison" We have added an initial SAEBench comparison in the Discussion and Appendix G to provide a quantitative baseline, training a 16k-width LISAE for 250M tokens on Gemma-2-2b representations. While this evaluation shows that our current implementation does not achieve state-of-the-art performance compared to standard SAEs, we emphasize once again the initial evidence presented in Section 3.2 and Appendices E, F: because of its adaptive L0 characteristics, LISAE excels when the dataset contains a large variation in the number of underlying features. This structural adaptivity falls outside the scope of current benchmarks, yet it may ultimately have a much deeper impact on representation learning.