acceptodds
Under review as a conference paper at ICLR 2027

Inference and learning in sparse autoencoders as natural gradient flow

Abstract

Sparse autoencoders are widely used to uncover interpretable features in neural networks, yet reliable recovery remains difficult when features overlap or activate infrequently. These challenges involve both inferring which features explain an input and learning the dictionary that represents them. Here, we unify inference and dictionary learning as natural-gradient flows on a shared variational free energy. We instantiate this framework as BeFOND, an encoder-free sparse coding model with closed-form inference and learning dynamics. We show how recurrent explaining away reduces interference between overlapping features, while Fisher preconditioning can compensate for the slow learning of rare features. On synthetic data, BeFOND improves dictionary recovery and rare-feature detection, with a growing advantage over amortized baselines as superposition increases. On language-model activations, it improves single-feature concept detection and selective intervention, outperforming pretrained reference SAEs with substantially less training data. Its feature quality continues to improve with dictionary width, whereas the evaluated baselines largely plateau. Together, these results show how improving inference and learning within a unified probabilistic framework can make better use of data and dictionary capacity to interpret and intervene on neural representations.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
sae, sparse autoencoder, mechinterp, variational inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_PXacOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper extends FOND, which treats inference as natural-gradient descent on the variational free energy, to dictionary learning, so that inference and learning follow the same objective. The resulting model, BeFOND, is an encoder-free sparse coding model with Bernoulli latents and a linear Gaussian decoder. Inference includes a recurrent term that implements explaining away between overlapping dictionary atoms, and the decoder is learned with a closed-form second-order update that does not backpropagate through inference. On the theory side, the paper shows that encoders computing each latent from a single projection cannot recover the exact posterior, and that, when the inferred and prior second moments match, the natural-gradient decoder update learns all features at the same local rate regardless of their frequency. On SynthSAEBench and on Gemma-2-2B at several dictionary widths, BeFOND recovers dictionaries and rare features better than amortised SAEs and achieves higher RAVEL scores at every width and higher single-feature probing accuracy at the larger widths, at a higher training cost.

Critical strengths

S1. Encoder-free framework with a sound theoretical motivation. Extending FOND from inference to learning is an interesting idea. Standard SAEs rely on a trained encoder for inference, and BeFOND removes it entirely. I like Proposition 4.1 in particular, which complements the amortisation-gap result of O'Neill et al. [1] by showing that under superposition no single-projection encoder can compute the exact posterior, so iterative inference such as BeFOND's is well motivated. Fig. 6 supports this, since recovery drops sharply under superposition once the lateral coupling is removed.

S2. Large gains in selective intervention and rare-feature recovery. BeFOND outperforms every baseline in RAVEL at every width, including Gemma Scope, by a large margin (Fig. 1d), and it also improves sparse probing at the larger widths. On SynthSAEBench, it recovers rare features much better than all baselines, even after the amortised baselines are retuned, which addresses dictionary learning, recently identified as the key open problem for SAEs under superposition [2].

S3. Well-written and educative manuscript. The paper is very well written, and its educative tone makes the variational and information-geometric background easy to follow. The appendix is thorough and open about the scope of each result.

References

[1] C. O'Neill, A. Gumran, D. Klindt. Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders. ICML 2025, PMLR 267. arXiv:2411.13117.

[2] V. Barin-Pacela, S. Joshi, I. Camacho, S. Lacoste-Julien, D. Klindt. Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisation. UAI 2026, PMLR 337. arXiv:2603.28744.

Critical weaknesses

W1. Unclear whether BeFOND models the actual signal. BeFOND is motivated as a generative model, but the paper does not show how well it reconstructs the activations. SAEs face a trade-off between capturing semantics and modelling the signal, because a model can capture semantics without reconstructing the activations well, or reconstruct them well without capturing semantics [1]. Probing and RAVEL measure only the first, so they cannot show where BeFOND stands on this trade-off. On Gemma, the paper reports neither of the typical in the literature explained variance and cross-entropy recovery, and not even BeFOND's L0, although Table 6 lists a reconstruction and sparsity evaluation. The question is more pressing for BeFOND than for standard SAEs, because its latents are binary and its readout is thresholded, so active features carry little information about magnitude. Table 5 points in the same direction, since BeFOND's single-feature AUC is only slightly above its accuracy, by a clearly smaller gap than for BatchTopK.

W2. Unclear whether the features match the model's own semantics. Sparse probing is useful for quantifying semantic quality and comparing models, but it remains an indirect measure. With this quantification in place, the next question is whether the learned features correspond to the semantics the LLM actually uses. This is usually tested with steering experiments, assessed qualitatively and/or with an LLM judge [2], and/or with automated interpretability [3]. RAVEL interventions partly address this but cover only a small set of entity attributes, and the paper reports no steering or interpretability results.

W3. Results on a single language model. All language-model results are on a single layer of Gemma-2-2B. Without experiments on other architectures and training data, even on small models such as GPT-2 [4] or Pythia [5], it is not possible to tell whether the gains come from BeFOND or from this particular model and layer.

W4. Inconsistent empirical claims. Several headline claims in the abstract and introduction are not supported, or are contradicted, by the paper's own results. First, much of the reduction of about 1,000 times in training tokens with respect to Gemma Scope (lines 91 to 94) is not specific to BeFOND, since the authors’ own baselines, trained with the same number of tokens as BeFOND, also match Gemma’s single-feature probing accuracy at the largest width with a small fraction of Gemma's tokens (Tables 5 and 8, Fig. 1a). The comparison that isolates BeFOND is therefore against these baselines, where its probing gain is modest and its RAVEL gain is large. The reduction factor and the number of tokens needed are also reported differently in the introduction and in Section 5. Second, counting tokens flatters BeFOND, since each token costs it much more compute than it costs BatchTopK (W5). Third, line 89 states that BeFOND outperforms all evaluated baselines on sparse probing, but Gemma Scope is better at the two smallest widths (Table 8), and BatchTopK is better in several AUC and five-feature comparisons (Table 5), which Section 5 partly acknowledges.

W5. Unclear cost of iterative inference. Removing the encoder is a real advantage, but it is replaced by an iterative process whose cost the main text mentions in only one sentence of the conclusion. According to Table 9 in the appendix, training is more than ten times slower per token than BatchTopK. Evaluation also needs ten recurrent steps per token, and RAVEL still improves beyond ten steps at the larger widths (Fig. 12), yet inference wall-clock time is not reported.

W6. Synthetic setting that favours BeFOND. In the main comparison on SynthSAEBench (Fig. 4, Table 4), BatchTopK, JumpReLU and Matryoshka each leave more than a thousand latents dead, and JumpReLU and Matryoshka are trained at sparsity levels different from that of the generator [1], which is known to damage the learned features [7]. With the improved settings (Fig. 9), the margin of BeFOND over the best amortised baseline shrinks substantially. The paper also disables parent scaling, which the benchmark uses by default so that child magnitudes vary with their parents, and thus removes a source of graded magnitudes that BeFOND's binary latents cannot represent. For the rare-feature comparison it also uses a dictionary as wide as the generator's, whereas the benchmark recommends a much narrower one [1], which the paper uses only for the MCC sweep of Fig. 3.

References

[1] D. Chanin, A. Garriga-Alonso. SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data. ICML 2026 Workshop on Mechanistic Interpretability. arXiv:2602.14687.

[2] Z. Wu et al. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. ICML 2025, PMLR 267. arXiv:2501.17148.

[3] G. Paulo, A. Mallen, C. Juang, N. Belrose. Automatically Interpreting Millions of Features in Large Language Models. ICML 2025, PMLR 267. arXiv:2410.13928.

[4] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever. Language Models are Unsupervised Multitask Learners. OpenAI technical report, 2019.

[5] S. Biderman et al. Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling. ICML 2023, PMLR 202. arXiv:2304.01373.

[6] V. Costa, T. Fel, E. S. Lubana, B. Tolooshams, D. Ba. From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit. NeurIPS 2025. arXiv:2506.03093.

[7] D. Chanin, A. Garriga-Alonso. Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders. ICML 2026, PMLR 306. arXiv:2508.16560.

Questions for authors

Q1 (W1). I would ask the authors to report explained variance, cross-entropy recovery and L0 for BeFOND and the baselines at every Gemma width, and to compare probing and RAVEL at matched L0, for example by varying T. If BeFOND reconstructs the signal about as well as the baselines, this would show that it improves semantics without sacrificing signal modelling. If its reconstruction is much worse, I would see it as a concept detector rather than an SAE alternative.

Q2 (W2). Steering examples evaluated by an LLM judge, for instance following AxBench [1], or automated interpretability scores [2] would show whether the features match the model's own semantics. Steering results at least as good as the baselines' would confirm that the probing gains reflect features the model actually uses.

Q3 (W3). Repeating the main comparison on at least one smaller model, such as GPT-2 or Pythia, would show whether the results hold across architectures and data.

Q4 (W4, W5). I would ask the authors to align the claims in lines 89 and 91 to 94 with Tables 5 and 8, and to state the token-efficiency claim against the in-house baselines rather than against Gemma Scope. It would also strengthen the paper to report training and inference wall-clock times in the main text, and to compare with baselines trained for the same wall-clock time, for example on the larger token set that is already extracted (Appendix D.1).

Q5. SAEs trained on the same data with different seeds are known to learn substantially different features [3], which makes interpretations based on a single run unreliable [4]. Since BeFOND captures more rare concepts, it may also learn more consistent features across runs. On SynthSAEBench the seeds reach almost the same MCC against the ground truth (Table 4), but on Gemma this is not measured, although several seeds are trained at every width. Measuring cross-seed consistency on Gemma, for example with PW-MCC [4], and discussing when the Bernoulli-Gaussian dictionary is identifiable [5] would strengthen the paper, and higher consistency than the baselines would be a strong further argument for BeFOND.

Q6. BeFOND keeps improving as the dictionary grows, but it is not clear what the additional latents capture. I would suggest reporting how many more rare features are found at larger widths, whether the common features of the smallest dictionary are kept in the largest one, and the absorption metric [6], since learning rarer features may come at the cost of common ones.

Q7 (clarification). The sparse-probing tasks correspond to the suite of Kantamneni et al. [7] rather than to the standard SAEBench suite [8]. Listing them and showing whether BeFOND's gains are spread across tasks or come from a few of them would make the probing results easier to interpret.

Q8. A two-by-two ablation that crosses {TopK encoder, BeFOND inference} with {Adam, BeFOND decoder update}, together with MP-SAE [9] on Gemma, would show where the gains come from. The TopK encoder with the second-order decoder is close to the variant the paper attributes to Picozzi [10]. If full BeFOND clearly outperforms both mixed configurations, the claim that the joint natural-gradient treatment drives the gains would be supported.

Q9 (W6). It would strengthen the synthetic results to repeat the rare-feature comparison with parent scaling enabled, at the narrower width recommended by the benchmark and against the improved baselines.

References

[1] Z. Wu et al. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. ICML 2025, PMLR 267. arXiv:2501.17148.

[2] G. Paulo, A. Mallen, C. Juang, N. Belrose. Automatically Interpreting Millions of Features in Large Language Models. ICML 2025, PMLR 267. arXiv:2410.13928.

[3] G. Paulo, N. Belrose. Sparse Autoencoders Trained on the Same Data Learn Different Features. ICLR 2026. arXiv:2501.16615.

[4] X. Song, A. Muhamed, Y. Zheng, L. Kong, Z. Tang, M. T. Diab, V. Smith, K. Zhang. Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs. arXiv:2505.20254, 2025.

[5] R. Gribonval, K. Schnass. Dictionary Identification: Sparse Matrix-Factorisation via l1-Minimisation. IEEE Transactions on Information Theory, 2010. arXiv:0904.4774.

[6] D. Chanin, J. Wilken-Smith, T. Dulka, H. Bhatnagar, S. Golechha, J. Bloom. A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders. NeurIPS 2025 (oral). arXiv:2409.14507.

[7] S. Kantamneni, J. Engels, S. Rajamanoharan, M. Tegmark, N. Nanda. Are Sparse Autoencoders Useful? A Case Study in Sparse Probing. ICML 2025, PMLR 267. arXiv:2502.16681.

[8] A. Karvonen, C. Rager, J. Lin, C. Tigges, J. Bloom, D. Chanin, et al. SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability. ICML 2025, PMLR 267. arXiv:2503.09532.

[9] V. Costa, T. Fel, E. S. Lubana, B. Tolooshams, D. Ba. From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit. NeurIPS 2025. arXiv:2506.03093.

[10] D. Picozzi. The Information Geometry of Large Language Models Is Shared, Learned, and Controllable. arXiv:2609.11063, 2026.

2 comments
AuthorsOpenReview ↗

(1/2): Reconstruction and comparisons across L0, AxBench, second language model, and revised claims

Thank you for a detailed and careful review, and for the specific suggestions.

We ran most of them, and we agree that some claims in the intro were not supported by our tables; we have revised the text accordingly.

Reconstruction, cross-entropy recovery and L0 (W1, Q1)

“the paper does not show how well it reconstructs the activations” (W1); “report explained variance, cross-entropy recovery and L0L_0 … at every Gemma width, and to compare probing and RAVEL at matched L0L_0” (Q1)

We now report explained variance, CE recovery and L0L_0 for BeFOND, BatchTopK, Matryoshka and Gemma Scope at all four widths (new Table 10). BeFOND activates more latents per token than the baselines, with comparable or better explained variance and cross-entropy recovery.

We also trained the baselines at target L0L_0 from 200 to 3,200 (new Fig. 16). Against the in-house baselines, BeFOND leads on single-feature probing and RAVEL both at the L0L_0 closest to its own and at each baseline's best L0L_0, with a probing lead that grows with width; its RAVEL score also exceeds every released Gemma Scope checkpoint. Fig. 15 (formerly Fig. 12) already varies TT for BeFOND: probing peaks within a few steps while RAVEL keeps improving. For the final paper, we plan a BeFOND L0L_0 sweep through its decoder-norm target.

That said, the goal of an SAE is to recover monosemantic features: sparsity and reconstruction matter insofar as they help learn a better dictionary, and on quality metrics BeFOND outperforms standard SAEs.

On the AUC–accuracy gap in Table 5: BeFOND's posterior probabilities cluster near 11, which coarsens the ranking among active examples; probing on the posterior logits would test this.

Whether the features match the model's own semantics (W2, Q2)

“the paper reports no steering or interpretability results” (W2); “Steering examples evaluated by an LLM judge, for instance following AxBench, or automated interpretability scores” (Q2)

We ran AxBench (new Table 11). On concept detection, BeFOND matches Matryoshka and exceeds BatchTopK. On steering, every method stays near the unsteered model with overlapping confidence intervals, which we read as uninformative here rather than as a ranking: AxBench requires an instruction-tuned model, all dictionaries here were trained on base Gemma-2-2B, and AxBench itself found that SAE steering underperforms simple baselines. RAVEL asks the same question, whether intervening on a feature changes the attribute the model represents, on the base model, and it separates the methods clearly: BeFOND leads at every width. RAVEL is also the industry standard SAE steering benchmarking in SAEBench. We have added a note about the set of entity attributes RAVEL uses for steering in Appendix D.6.

We consider LLM-judged automated-interpretability scores weak evidence here: a recent study of SAE benchmark reliability (Chanin, 2026) found them noisy and weakly discriminative between architectures, with sparse probing and RAVEL the most reliable signals.

Results on a single language model (W3, Q3)

“All language-model results are on a single layer of Gemma-2-2B” (W3); “Repeating the main comparison on at least one smaller model, such as GPT-2 or Pythia” (Q3)

We agree that a second model would strengthen the paper, and a replication is in progress for the final version. We chose a larger model with released reference SAEs (e.g., Llama-3.1-8B with Llama Scope) over GPT-2 small or Pythia: GPT-2 small is below the scale at which several SAEBench evaluations are reliable (in our GPT-2 runs, the absorption evaluation could not run at all), and Pythia's large outlier activations complicate SAE training.

Consistency of the empirical claims (W4, Q4)

“Several headline claims in the abstract and introduction are not supported, or are contradicted, by the paper's own results” (W4); “align the claims in lines 89 and 91 to 94 with Tables 5 and 8, and to state the token-efficiency claim against the in-house baselines” (Q4)

We agree with each point and have revised the text. (1) Line 89 now says that BeFOND outperforms the in-house baselines on RAVEL and single-feature accuracy at every width, that Gemma Scope is higher on single-feature probing at the two smallest widths, and that BatchTopK leads several AUC and five-feature comparisons. (2) Lines 91–94 now state token efficiency against our baselines trained on the same 50M tokens (at 524K, +7.9 points RAVEL and +1.5 points single-feature probing over the strongest of them), keeping Gemma Scope as context and noting that our baselines also match it at the largest width. (3) The introduction and Section 5 now use the same numbers (8.2M tokens, a ratio of about 976). (4) Training cost and wall-clock times now appear in the main text (Fig. 5); equal-wall-clock baseline runs are planned for the final version.

AuthorsOpenReview ↗

(2/2): Inference cost, parent scaling, cross-seed consistency, added latents, probing tasks, and the inference/learning ablation

Cost of iterative inference (W5)

“Evaluation also needs ten recurrent steps per token … yet inference wall-clock time is not reported” (W5)

Each inference step costs two dense products with the dictionary and no K×KK \times K matrix (Appendix B.8), so T=10T=10 steps cost about 20 dense products, against about one for an amortized encoder. Two things offset this. First, TT is flexible: single-feature probing peaks within the first few steps while RAVEL keeps improving (Fig. 15, formerly Fig. 12), so users can choose the depth their task needs. Second, no gradient flows through the inference trajectory, so training memory does not grow with TT.

Synthetic setting (W6, Q9)

“repeat the rare-feature comparison with parent scaling enabled, at the narrower width recommended by the benchmark and against the improved baselines” (Q9)

We ran this comparison (new Fig. 14): parent scaling enabled, full width, three seeds, against the improved baseline recipes with target L0L_0 close to the generator's. BeFOND still leads on every recovery metric, with Rare F1 of 87.6% against 81.4% for the best baseline (Matryoshka). We had disabled parent scaling because it is off by default in the released SynthSAEBench code. The improved baselines from former Fig. 9 are now also shown in the main Fig. 4 (dashed bars).

On width: with 4,096 latents for 16,384 generating features, the dictionary cannot represent the rare tail, and every method's Rare F1 is at most 3%, so this setting cannot measure rare-feature recovery. We use the narrower width where it is informative, in the superposition sweep (Fig. 3).

Cross-seed consistency and identifiability (Q5)

“Measuring cross-seed consistency on Gemma … and discussing when the Bernoulli-Gaussian dictionary is identifiable” (Q5)

We added this comparison (global response, result 2; new Fig. 17). At 65K and 131K, BeFOND's dictionaries are the most consistent across seeds, about twice as consistent as BatchTopK and Matryoshka at the L0L_0 closest to BeFOND's. At 16K, the baselines at their lowest L0L_0 are more consistent. Our response to Suwt gives the table.

On identifiability: BeFOND's generative model is a sparse dictionary model of the kind studied by Gribonval and Schnass (2010), and on SynthSAEBench it recovers the ground-truth dictionary with MCC of about 0.9, consistent with an identifiable regime. We will add a short discussion of the conditions in the final version.

What the additional latents capture at larger widths (Q6)

“it is not clear what the additional latents capture” (Q6)

BeFOND uses its whole dictionary (at 65K and 131K, every latent fires on 65K held-out tokens), and the added latents are reproducible (cross-seed MCC rises from 16K to 65K and holds at 131K). We will also report how many 16K atoms have a close match in the 524K dictionary, and how the firing-rate distribution changes with width.

Which sparse-probing tasks (Q7)

“Listing them and showing whether BeFOND's gains are spread across tasks or come from a few of them” (Q7)

We use the 113 binary tasks of Kantamneni et al. (2025), called "Sparse probing (SAE probes version)" in SAEBench. This version of the probing benchmark is generally considered superior because more tasks reduce the variance of the average and a recent study of benchmark reliability found this suite the more reliable signal (Chanin, 2026). We emphasize that this is part of the SAEBench suite, available as of SAEBench v0.5.1, and we run it via the official SAEBench codebase. We will add a per-task plot of BeFOND's gain over each baseline across all 113 tasks.

Separating inference from learning (Q8)

“A two-by-two ablation that crosses {TopK encoder, BeFOND inference} with {Adam, BeFOND decoder update}, together with MP-SAE on Gemma” (Q8)

Two parts of this ablation are already in the paper. The mean-field baseline on Gemma pairs iterative inference with an Adam-trained decoder and is below BeFOND on both probing and RAVEL at every width (Fig. 1, Table 5). On SynthSAEBench, the geometry ablation (Fig. 8, formerly Fig. 7) crosses natural and Euclidean updates for inference and for learning, and natural inference with natural learning reaches higher recovery in fewer updates. MP-SAE is in our synthetic comparison (Table 4), where BeFOND recovers more; on Gemma, a prior evaluation found that MP-SAEs underperform standard SAEs on quality metrics while training much longer (Chanin, 2025). We will add the remaining configuration, a TopK encoder with BeFOND's decoder update (close to Picozzi, 2026), in the final version.

Reviewer_SuwtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

BeFOND extends FOND, which applied natural gradient flow to the variational parameters of an encoder-free iterative variational model, by applying natural gradient flow to both variational and generative parameters. The authors discuss, theoretically and empirically, how the proposed model can learn rare features and explain away under superposition, and show empirically that it achieves strong feature quality with substantially fewer training tokens than other models.

Critical strengths
  • BeFOND’s ability to explain away and to learn rare features is very interesting and is justified both theoretically and empirically.
  • The paper provides a thorough experimental and theoretical analysis.
Critical weaknesses
  • Limited applicability. The proposed method has high computational cost (12× to 14× slower per token than BatchTopK, according to Table 9), which could strongly limit its practical adoption. This cost is not clearly stated in the main paper, which can be misleading given the claimed ~1,000× token reduction. Also, while metric variance across the three seeds is small, it is unclear to me whether, and to what degree, the learned dictionaries vary from run to run.
  • Is using a Bernoulli a good choice? I understand the math works well with this assumption, but binary latents do not encode feature magnitude. I am not sure what the impact would be in practice (e.g., how steering strength would be set when implementing model steering).
Questions for authors
  • The authors should clearly state the computational cost in the main paper. It would be helpful to state at least the per-token slowdown from Table 9 next to the token-efficiency claim. Additionally, reporting the wall-clock time needed to reach the Gemma Scope single-feature probing accuracy would increase the paper's transparency. This could further be discussed in the limitations of the current work.
  • Could you explain how model steering would work in the current setting? Since the latents are binary, how would you choose and calibrate the steering strength?
  • Could you report the consistency of the learned dictionaries across seeds (e.g., cross-seed MCC) to show how much the features vary from run to run in BeFOND vs other methods?
1 comment
AuthorsOpenReview ↗

Training cost now in the main text, cross-seed consistency, and how magnitude and steering work with Bernoulli latents

Thank you for the clear and concrete requests.

Training cost and its place in the main paper (W1, Q1)

“Limited applicability. The proposed method has high computational cost (12×12\times to 14×14\times slower per token than BatchTopK, according to Table 9), which could strongly limit its practical adoption. This cost is not clearly stated in the main paper, which can be misleading given the claimed ∼1,000×\sim 1{,}000\times token reduction.” (W1)

“The authors should clearly state the computational cost in the main paper. It would be helpful to state at least the per-token slowdown from Table 9 next to the token-efficiency claim. Additionally, reporting the wall-clock time needed to reach the Gemma Scope single-feature probing accuracy would increase the paper's transparency. This could further be discussed in the limitations of the current work.” (Q1)

In the submitted implementation, we did not optimize for speed, and BeFOND cost up to 14×14\times as much per token as BatchTopK (Table 9). With BF16 matrix products and compilation, BeFOND now trains about 7×7\times faster than the submitted implementation at 16K and 32K latents (global response, result 1). We have included this as a new Fig. 5, and now report the training cost and speedup alongside the token-efficiency results in the intro.

Cross-seed consistency of the dictionaries (W1, Q3)

“Also, while metric variance across the three seeds is small, it is unclear to me whether, and to what degree, the learned dictionaries vary from run to run.” (W1)

“Could you report the consistency of the learned dictionaries across seeds (e.g., cross-seed MCC) to show how much the features vary from run to run in BeFOND vs other methods?” (Q3)

We added this comparison on Gemma-2-2B (global response, result 2). For each pair of seeds, we match decoder atoms one to one by absolute cosine similarity and average the matched similarities. We included these results in the revised manuscript (new Fig. 17); we also summarize them here for your convenience:

Mean cross-seed MCC by dictionary width. Each baseline uses the tested L0L_0 with the highest mean MCC at that width. BeFOND was evaluated at one L0L_0 per width. Fig. 17 in the revised paper shows all tested L0L_0 values and conditions.

Method16K65K131K
BeFOND0.4480.5560.547
BatchTopK0.5560.4580.455
Matryoshka0.5460.4770.430

We see that BeFOND's consistency increases with width while the baselines' drops.

Binary latents, feature magnitude and steering (W2, Q2)

“Is using a Bernoulli a good choice? I understand the math works well with this assumption, but binary latents do not encode feature magnitude. I am not sure what the impact would be in practice (e.g., how steering strength would be set when implementing model steering).” (W2)

“Could you explain how model steering would work in the current setting? Since the latents are binary, how would you choose and calibrate the steering strength?” (Q2)

BeFOND represents magnitude in two places. First, its readout is graded rather than binary: inference returns posterior means in (0,1)(0, 1), and the evaluation readout keeps those above 0.50.5. Second, BeFOND only applies a soft regularization toward a target norm, so each atom's norm can adapt to the typical magnitude of its feature in the residual stream.

We’d also like to mention that, across the four Gemma widths, BeFOND explains 95.45–99.45% of activation variance and achieves 99.73–99.84% cross-entropy recovery, which is comparable to baseline SAEs (global response, result 4).

For steering, the natural unit is the atom itself. Setting a latent to 11 adds its decoder vector, which is the feature at its learned typical magnitude, and a scalar multiplier α\alpha scales from there. This is the same one-parameter calibration used when steering with any SAE latent or steering vector, where α\alpha is chosen on held-out prompts. This choice of how strong to steer is a common hyperparameter in steering in general, typically set by experimentation. Steering using BeFOND is no different in this regard.

RAVEL uses activation patching to test whether an intervention changes a target entity attribute while preserving others. BeFOND achieves the highest RAVEL disentanglement score when each baseline is evaluated at its best L0L_0 (global response, result 3).

Reviewer_QhQmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a sparse autoencoder (SAE) that combines iterative variational inference with natural gradient dictionary learning. Theoretically, the authors derive update formulas for the inference and learning processes, analyze how iterative inference mitigates interference between overlapping features, and explain how Fisher preconditioning enhances the learning of rare features. Experimentally, the authors conduct a series of tests using both synthetic data and the Gemma-2-2B model; the results demonstrate that the proposed method achieves better performance on metrics such as feature recovery.

Critical strengths

This paper unifies inference and dictionary learning within a single framework and provides explicit update formulas.

The experimental results are notable. On synthetic data, the method outperforms several improved baseline approaches in the recovery of rare features. And the method demonstrates performance gains over baselines in the Gemma-2-2B model.

Critical weaknesses

Theoretically, the assumption in Theorem 4.2 that between the prior and posterior second moments requires further explanation and justification. Experimentally, the computational cost is relatively high (exceeding 10x in some experiments).

Questions for authors

Could the authors provide experimental results to justify the validity of the second-order moment condition?

How does BeFOND perform compared to other baseline methods when the computational budget is held constant?

1 comment
AuthorsOpenReview ↗

Moment matching relaxed to proportionality (new Corollary C.5, Fig. 10) and 7× faster training

Thank you for the positive assessment and the two concrete questions, both of which we answer with new material in the revision.

Matched second moments in Theorem 4.2 (Weakness, Q1)

“Theoretically, the assumption in Theorem 4.2 that between the prior and posterior second moments requires further explanation and justification.” (Critical Weaknesses)

“Could the authors provide experimental results to justify the validity of the second-order moment condition?” (Q1)

Thank you for pointing this out. Theorem 4.2 assumes the posterior and prior second moments are equal, and our training does not enforce this. Exact equality is, however, stronger than needed: rarity compensation only requires the two moments to be proportional. This already follows from Appendix C.4. When the posterior moment is a constant multiple cc of the prior moment, the general contraction equation (Eq. 76) gives the same contraction rate, η\eta times cc, for every atom, regardless of how often its feature fires. When proportionality holds only approximately, Proposition C.4 (Eq. 77) implies that every atom still contracts at least at a common rate, ηc(1−ε)\eta c (1 - \varepsilon), with only a frequency-dependent prefactor differing between atoms. We now state this explicitly as Corollary C.5 and refer to it in the main text next to Theorem 4.2.

We measured both moments during training on Gemma-2-2B (16K16\mathrm{K} latents, one seed; new Fig. 10): their relative discrepancy falls from 1.31.3 after 0.5M0.5\mathrm{M} training tokens to 0.0170.017 after 5M5\mathrm{M} tokens and 0.0050.005 at the end of training, so the two moments entering the decoder update become close.

On SynthSAEBench, we also computed both moment matrices exactly at checkpoints spanning training, from initialization to the final update, for the 16K16\mathrm{K} parent-ON model (three seeds, 262,144262{,}144 held-out examples per checkpoint). At initialization the prior is uniform and, as expected, there is no compensation: per-atom rates scale with firing frequency. By the first saved checkpoint, at 5%5\% of training, the learned prior already tracks feature frequencies. From then on, the posterior moments are about 77–88 times the prior moments, matching the ratio of realized latent activity to the prior's mean budget. Relative to this common scale, the per-atom (diagonal) learning rates are nearly independent of firing frequency throughout training. Their median differs by at most 11%11\% between the rarest and most common frequency quintiles, and by 4%4\% at the end of training. Their spread across atoms (9090th vs 1010th percentile) is 1.61.6–1.9×1.9\times during training and 1.2×1.2\times at the end, compared with 7.27.2–7.9×7.9\times without preconditioning. The only exception is a small set of inactive latents held at the prior's floor (under 4%4\% of atoms at 5%5\% of training, 0.3%0.3\% at 30%30\%, none at the end). At the final checkpoint, 91%91\% of modes lie within a factor of 1010 of the common scale, and the few slow modes involve co-active common latents, not rare features. All three seeds agree. We will add these SynthSAEBench measurements to the appendix of the final version.

Cost at a fixed compute budget (Weakness, Q2)

“Experimentally, the computational cost is relatively high (exceeding 10×10\times in some experiments).” (Critical Weaknesses)

“How does BeFOND perform compared to other baseline methods when the computational budget is held constant?” (Q2)

The submitted implementation was not optimized for speed, and a BF16, compiled version trains 7×7\times faster at 16K16\mathrm{K} and 32K32\mathrm{K} latents with unchanged explained variance, probing and RAVEL (global response, result 1; new Fig. 5 in the revised manuscript). This brings BeFOND's per-token cost to about twice that of BatchTopK. In the final paper, we plan to add a direct comparison in which each baseline receives the same GPU time as fast BeFOND.

Comments on the paper

1 comment
AuthorsOpenReview ↗

New results: 7x faster training, cross-seed consistency, L0 sweeps, reconstruction, parent scaling, and AxBench

We thank the reviewers for their careful reading and constructive feedback.

Reviewers found our unified treatment of inference and learning well motivated, and our results on explaining away and rare features well supported. All three reviewers raised the computational cost of training. They also asked for cross-seed consistency, reconstruction and sparsity results on Gemma, a stricter synthetic comparison, and steering. We ran new experiments to address each of these concerns. All new text in the revised PDF is in green. Below, we summarize the main new results.

Result 1: BeFOND trains 7x faster with unchanged feature quality (all reviewers)

In the submitted manuscript, we did not optimize for speed: our implementation used FP32 with FP64 intermediates, without compilation. In the revised implementation, we used BF16 matrix products and compilation. These simple changes yielded ∼7×\sim 7\times faster training speed on one H200, with near-identical performance on both standard training metrics like reconstruction, as well as downstream metrics like sparse probing and RAVEL (new Fig. 5).

This lowers BeFOND's per-token computational cost from ∼14×\sim 14\times that of BatchTopK (Table 9) to ∼2×\sim 2\times, without losing any of the performance.

Result 2: BeFOND's dictionaries are more consistent across seeds (Suwt, PXac)

SAEs trained with different random seeds often learn different features, which makes any single run hard to trust (Paulo & Belrose, 2026). Therefore, we measured how consistently each method recovers the same dictionary across three seeds on Gemma. BeFOND's consistency grows with dictionary width, while the baselines' drops. At the larger widths, BeFOND's dictionaries are about twice as consistent across seeds as those of BatchTopK and Matryoshka when each baseline uses the tested L0L_0 closest to BeFOND's (new Fig. 17).

Result 3: The gains hold across sparsity levels (PXac, Suwt)

Reviewers asked whether our gains depend on sparsity levels. To test this, we retrained BatchTopK and Matryoshka across a wide range of L0L_0 values at every width. On RAVEL, BeFOND outperforms every baseline at that baseline's best L0L_0, including Gemma Scope. On sparse probing, it outperforms both in-house baselines at their best L0L_0, and its lead grows with dictionary width (new Fig. 16).

Result 4: BeFOND reconstructs activations as well as the baselines (PXac)

Reviewers asked whether BeFOND's gains in feature quality come at the expense of reconstruction. They do not. At comparable L0L_0, BeFOND explains as much of the activation variance as the baselines and preserves the language model's next-token loss just as well (new Table 10). Therefore, BeFOND is a full SAE alternative, not only a concept detector.

Result 5: The synthetic results hold under the suggested settings (PXac)

We reran the synthetic comparison with parent scaling enabled and against the improved baseline recipes. BeFOND still leads on every recovery metric, reaching 88% rare-feature F1 against 81% for the best baseline (new Fig. 14). We keep the full-width setting for this test because the narrower dictionary recommended by the benchmark has far fewer latents than ground-truth features, so the baselines miss the rare tail almost entirely and the setting cannot measure rare-feature recovery. We also added the improved baselines from former Fig. 9 to the main Fig. 4.

Result 6: Concept detection on AxBench (PXac, Suwt)

On AxBench concept detection, BeFOND matches Matryoshka and outperforms BatchTopK (new Table 11). On steering, all methods stay close to the unsteered model; we discuss why in our responses to Suwt and PXac.

Revisions to the text

We also revised the abstract, intro and results section. We now state token efficiency against our own baselines trained on the same data, report training cost alongside it, and no longer claim that BeFOND wins every sparse-probing comparison. The main text and the limitations section now discuss training cost and how to reduce it.

For the final version

We are planning to replicate the main comparison on a second language model (aiming for a bigger model than Gemma-2b), extending the cross-seed analysis to 524K latents with more seeds, and adding equal-wall-clock baseline runs, Gemma inference timings and a per-task breakdown of sparse probing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.