- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper extends FOND, which treats inference as natural-gradient descent on the variational free energy, to dictionary learning, so that inference and learning follow the same objective. The resulting model, BeFOND, is an encoder-free sparse coding model with Bernoulli latents and a linear Gaussian decoder. Inference includes a recurrent term that implements explaining away between overlapping dictionary atoms, and the decoder is learned with a closed-form second-order update that does not backpropagate through inference. On the theory side, the paper shows that encoders computing each latent from a single projection cannot recover the exact posterior, and that, when the inferred and prior second moments match, the natural-gradient decoder update learns all features at the same local rate regardless of their frequency. On SynthSAEBench and on Gemma-2-2B at several dictionary widths, BeFOND recovers dictionaries and rare features better than amortised SAEs and achieves higher RAVEL scores at every width and higher single-feature probing accuracy at the larger widths, at a higher training cost.
Critical strengths
S1. Encoder-free framework with a sound theoretical motivation. Extending FOND from inference to learning is an interesting idea. Standard SAEs rely on a trained encoder for inference, and BeFOND removes it entirely. I like Proposition 4.1 in particular, which complements the amortisation-gap result of O'Neill et al. [1] by showing that under superposition no single-projection encoder can compute the exact posterior, so iterative inference such as BeFOND's is well motivated. Fig. 6 supports this, since recovery drops sharply under superposition once the lateral coupling is removed.
S2. Large gains in selective intervention and rare-feature recovery. BeFOND outperforms every baseline in RAVEL at every width, including Gemma Scope, by a large margin (Fig. 1d), and it also improves sparse probing at the larger widths. On SynthSAEBench, it recovers rare features much better than all baselines, even after the amortised baselines are retuned, which addresses dictionary learning, recently identified as the key open problem for SAEs under superposition [2].
S3. Well-written and educative manuscript. The paper is very well written, and its educative tone makes the variational and information-geometric background easy to follow. The appendix is thorough and open about the scope of each result.
References
[1] C. O'Neill, A. Gumran, D. Klindt. Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders. ICML 2025, PMLR 267. arXiv:2411.13117.
[2] V. Barin-Pacela, S. Joshi, I. Camacho, S. Lacoste-Julien, D. Klindt. Stop Probing, Start Coding: Why Linear Probes and Sparse Autoencoders Fail at Compositional Generalisation. UAI 2026, PMLR 337. arXiv:2603.28744.
Critical weaknesses
W1. Unclear whether BeFOND models the actual signal. BeFOND is motivated as a generative model, but the paper does not show how well it reconstructs the activations. SAEs face a trade-off between capturing semantics and modelling the signal, because a model can capture semantics without reconstructing the activations well, or reconstruct them well without capturing semantics [1]. Probing and RAVEL measure only the first, so they cannot show where BeFOND stands on this trade-off. On Gemma, the paper reports neither of the typical in the literature explained variance and cross-entropy recovery, and not even BeFOND's L0, although Table 6 lists a reconstruction and sparsity evaluation. The question is more pressing for BeFOND than for standard SAEs, because its latents are binary and its readout is thresholded, so active features carry little information about magnitude. Table 5 points in the same direction, since BeFOND's single-feature AUC is only slightly above its accuracy, by a clearly smaller gap than for BatchTopK.
W2. Unclear whether the features match the model's own semantics. Sparse probing is useful for quantifying semantic quality and comparing models, but it remains an indirect measure. With this quantification in place, the next question is whether the learned features correspond to the semantics the LLM actually uses. This is usually tested with steering experiments, assessed qualitatively and/or with an LLM judge [2], and/or with automated interpretability [3]. RAVEL interventions partly address this but cover only a small set of entity attributes, and the paper reports no steering or interpretability results.
W3. Results on a single language model. All language-model results are on a single layer of Gemma-2-2B. Without experiments on other architectures and training data, even on small models such as GPT-2 [4] or Pythia [5], it is not possible to tell whether the gains come from BeFOND or from this particular model and layer.
W4. Inconsistent empirical claims. Several headline claims in the abstract and introduction are not supported, or are contradicted, by the paper's own results. First, much of the reduction of about 1,000 times in training tokens with respect to Gemma Scope (lines 91 to 94) is not specific to BeFOND, since the authors’ own baselines, trained with the same number of tokens as BeFOND, also match Gemma’s single-feature probing accuracy at the largest width with a small fraction of Gemma's tokens (Tables 5 and 8, Fig. 1a). The comparison that isolates BeFOND is therefore against these baselines, where its probing gain is modest and its RAVEL gain is large. The reduction factor and the number of tokens needed are also reported differently in the introduction and in Section 5. Second, counting tokens flatters BeFOND, since each token costs it much more compute than it costs BatchTopK (W5). Third, line 89 states that BeFOND outperforms all evaluated baselines on sparse probing, but Gemma Scope is better at the two smallest widths (Table 8), and BatchTopK is better in several AUC and five-feature comparisons (Table 5), which Section 5 partly acknowledges.
W5. Unclear cost of iterative inference. Removing the encoder is a real advantage, but it is replaced by an iterative process whose cost the main text mentions in only one sentence of the conclusion. According to Table 9 in the appendix, training is more than ten times slower per token than BatchTopK. Evaluation also needs ten recurrent steps per token, and RAVEL still improves beyond ten steps at the larger widths (Fig. 12), yet inference wall-clock time is not reported.
W6. Synthetic setting that favours BeFOND. In the main comparison on SynthSAEBench (Fig. 4, Table 4), BatchTopK, JumpReLU and Matryoshka each leave more than a thousand latents dead, and JumpReLU and Matryoshka are trained at sparsity levels different from that of the generator [1], which is known to damage the learned features [7]. With the improved settings (Fig. 9), the margin of BeFOND over the best amortised baseline shrinks substantially. The paper also disables parent scaling, which the benchmark uses by default so that child magnitudes vary with their parents, and thus removes a source of graded magnitudes that BeFOND's binary latents cannot represent. For the rare-feature comparison it also uses a dictionary as wide as the generator's, whereas the benchmark recommends a much narrower one [1], which the paper uses only for the MCC sweep of Fig. 3.
References
[1] D. Chanin, A. Garriga-Alonso. SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Data. ICML 2026 Workshop on Mechanistic Interpretability. arXiv:2602.14687.
[2] Z. Wu et al. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. ICML 2025, PMLR 267. arXiv:2501.17148.
[3] G. Paulo, A. Mallen, C. Juang, N. Belrose. Automatically Interpreting Millions of Features in Large Language Models. ICML 2025, PMLR 267. arXiv:2410.13928.
[4] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever. Language Models are Unsupervised Multitask Learners. OpenAI technical report, 2019.
[5] S. Biderman et al. Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling. ICML 2023, PMLR 202. arXiv:2304.01373.
[6] V. Costa, T. Fel, E. S. Lubana, B. Tolooshams, D. Ba. From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit. NeurIPS 2025. arXiv:2506.03093.
[7] D. Chanin, A. Garriga-Alonso. Sparse but Wrong: Incorrect L0 Leads to Incorrect Features in Sparse Autoencoders. ICML 2026, PMLR 306. arXiv:2508.16560.
Questions for authors
Q1 (W1). I would ask the authors to report explained variance, cross-entropy recovery and L0 for BeFOND and the baselines at every Gemma width, and to compare probing and RAVEL at matched L0, for example by varying T. If BeFOND reconstructs the signal about as well as the baselines, this would show that it improves semantics without sacrificing signal modelling. If its reconstruction is much worse, I would see it as a concept detector rather than an SAE alternative.
Q2 (W2). Steering examples evaluated by an LLM judge, for instance following AxBench [1], or automated interpretability scores [2] would show whether the features match the model's own semantics. Steering results at least as good as the baselines' would confirm that the probing gains reflect features the model actually uses.
Q3 (W3). Repeating the main comparison on at least one smaller model, such as GPT-2 or Pythia, would show whether the results hold across architectures and data.
Q4 (W4, W5). I would ask the authors to align the claims in lines 89 and 91 to 94 with Tables 5 and 8, and to state the token-efficiency claim against the in-house baselines rather than against Gemma Scope. It would also strengthen the paper to report training and inference wall-clock times in the main text, and to compare with baselines trained for the same wall-clock time, for example on the larger token set that is already extracted (Appendix D.1).
Q5. SAEs trained on the same data with different seeds are known to learn substantially different features [3], which makes interpretations based on a single run unreliable [4]. Since BeFOND captures more rare concepts, it may also learn more consistent features across runs. On SynthSAEBench the seeds reach almost the same MCC against the ground truth (Table 4), but on Gemma this is not measured, although several seeds are trained at every width. Measuring cross-seed consistency on Gemma, for example with PW-MCC [4], and discussing when the Bernoulli-Gaussian dictionary is identifiable [5] would strengthen the paper, and higher consistency than the baselines would be a strong further argument for BeFOND.
Q6. BeFOND keeps improving as the dictionary grows, but it is not clear what the additional latents capture. I would suggest reporting how many more rare features are found at larger widths, whether the common features of the smallest dictionary are kept in the largest one, and the absorption metric [6], since learning rarer features may come at the cost of common ones.
Q7 (clarification). The sparse-probing tasks correspond to the suite of Kantamneni et al. [7] rather than to the standard SAEBench suite [8]. Listing them and showing whether BeFOND's gains are spread across tasks or come from a few of them would make the probing results easier to interpret.
Q8. A two-by-two ablation that crosses {TopK encoder, BeFOND inference} with {Adam, BeFOND decoder update}, together with MP-SAE [9] on Gemma, would show where the gains come from. The TopK encoder with the second-order decoder is close to the variant the paper attributes to Picozzi [10]. If full BeFOND clearly outperforms both mixed configurations, the claim that the joint natural-gradient treatment drives the gains would be supported.
Q9 (W6). It would strengthen the synthetic results to repeat the rare-feature comparison with parent scaling enabled, at the narrower width recommended by the benchmark and against the improved baselines.
References
[1] Z. Wu et al. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. ICML 2025, PMLR 267. arXiv:2501.17148.
[2] G. Paulo, A. Mallen, C. Juang, N. Belrose. Automatically Interpreting Millions of Features in Large Language Models. ICML 2025, PMLR 267. arXiv:2410.13928.
[3] G. Paulo, N. Belrose. Sparse Autoencoders Trained on the Same Data Learn Different Features. ICLR 2026. arXiv:2501.16615.
[4] X. Song, A. Muhamed, Y. Zheng, L. Kong, Z. Tang, M. T. Diab, V. Smith, K. Zhang. Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs. arXiv:2505.20254, 2025.
[5] R. Gribonval, K. Schnass. Dictionary Identification: Sparse Matrix-Factorisation via l1-Minimisation. IEEE Transactions on Information Theory, 2010. arXiv:0904.4774.
[6] D. Chanin, J. Wilken-Smith, T. Dulka, H. Bhatnagar, S. Golechha, J. Bloom. A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders. NeurIPS 2025 (oral). arXiv:2409.14507.
[7] S. Kantamneni, J. Engels, S. Rajamanoharan, M. Tegmark, N. Nanda. Are Sparse Autoencoders Useful? A Case Study in Sparse Probing. ICML 2025, PMLR 267. arXiv:2502.16681.
[8] A. Karvonen, C. Rager, J. Lin, C. Tigges, J. Bloom, D. Chanin, et al. SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability. ICML 2025, PMLR 267. arXiv:2503.09532.
[9] V. Costa, T. Fel, E. S. Lubana, B. Tolooshams, D. Ba. From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit. NeurIPS 2025. arXiv:2506.03093.
[10] D. Picozzi. The Information Geometry of Large Language Models Is Shared, Learned, and Controllable. arXiv:2609.11063, 2026.
2 comments
(2/2): Inference cost, parent scaling, cross-seed consistency, added latents, probing tasks, and the inference/learning ablation
Cost of iterative inference (W5)
“Evaluation also needs ten recurrent steps per token … yet inference wall-clock time is not reported” (W5)
Each inference step costs two dense products with the dictionary and no matrix (Appendix B.8), so steps cost about 20 dense products, against about one for an amortized encoder. Two things offset this. First, is flexible: single-feature probing peaks within the first few steps while RAVEL keeps improving (Fig. 15, formerly Fig. 12), so users can choose the depth their task needs. Second, no gradient flows through the inference trajectory, so training memory does not grow with .
Synthetic setting (W6, Q9)
“repeat the rare-feature comparison with parent scaling enabled, at the narrower width recommended by the benchmark and against the improved baselines” (Q9)
We ran this comparison (new Fig. 14): parent scaling enabled, full width, three seeds, against the improved baseline recipes with target close to the generator's. BeFOND still leads on every recovery metric, with Rare F1 of 87.6% against 81.4% for the best baseline (Matryoshka). We had disabled parent scaling because it is off by default in the released SynthSAEBench code. The improved baselines from former Fig. 9 are now also shown in the main Fig. 4 (dashed bars).
On width: with 4,096 latents for 16,384 generating features, the dictionary cannot represent the rare tail, and every method's Rare F1 is at most 3%, so this setting cannot measure rare-feature recovery. We use the narrower width where it is informative, in the superposition sweep (Fig. 3).
Cross-seed consistency and identifiability (Q5)
“Measuring cross-seed consistency on Gemma … and discussing when the Bernoulli-Gaussian dictionary is identifiable” (Q5)
We added this comparison (global response, result 2; new Fig. 17). At 65K and 131K, BeFOND's dictionaries are the most consistent across seeds, about twice as consistent as BatchTopK and Matryoshka at the closest to BeFOND's. At 16K, the baselines at their lowest are more consistent. Our response to Suwt gives the table.
On identifiability: BeFOND's generative model is a sparse dictionary model of the kind studied by Gribonval and Schnass (2010), and on SynthSAEBench it recovers the ground-truth dictionary with MCC of about 0.9, consistent with an identifiable regime. We will add a short discussion of the conditions in the final version.
What the additional latents capture at larger widths (Q6)
“it is not clear what the additional latents capture” (Q6)
BeFOND uses its whole dictionary (at 65K and 131K, every latent fires on 65K held-out tokens), and the added latents are reproducible (cross-seed MCC rises from 16K to 65K and holds at 131K). We will also report how many 16K atoms have a close match in the 524K dictionary, and how the firing-rate distribution changes with width.
Which sparse-probing tasks (Q7)
“Listing them and showing whether BeFOND's gains are spread across tasks or come from a few of them” (Q7)
We use the 113 binary tasks of Kantamneni et al. (2025), called "Sparse probing (SAE probes version)" in SAEBench. This version of the probing benchmark is generally considered superior because more tasks reduce the variance of the average and a recent study of benchmark reliability found this suite the more reliable signal (Chanin, 2026). We emphasize that this is part of the SAEBench suite, available as of SAEBench v0.5.1, and we run it via the official SAEBench codebase. We will add a per-task plot of BeFOND's gain over each baseline across all 113 tasks.
Separating inference from learning (Q8)
“A two-by-two ablation that crosses {TopK encoder, BeFOND inference} with {Adam, BeFOND decoder update}, together with MP-SAE on Gemma” (Q8)
Two parts of this ablation are already in the paper. The mean-field baseline on Gemma pairs iterative inference with an Adam-trained decoder and is below BeFOND on both probing and RAVEL at every width (Fig. 1, Table 5). On SynthSAEBench, the geometry ablation (Fig. 8, formerly Fig. 7) crosses natural and Euclidean updates for inference and for learning, and natural inference with natural learning reaches higher recovery in fewer updates. MP-SAE is in our synthetic comparison (Table 4), where BeFOND recovers more; on Gemma, a prior evaluation found that MP-SAEs underperform standard SAEs on quality metrics while training much longer (Chanin, 2025). We will add the remaining configuration, a TopK encoder with BeFOND's decoder update (close to Picozzi, 2026), in the final version.
(1/2): Reconstruction and comparisons across L0, AxBench, second language model, and revised claims
Thank you for a detailed and careful review, and for the specific suggestions.
We ran most of them, and we agree that some claims in the intro were not supported by our tables; we have revised the text accordingly.
Reconstruction, cross-entropy recovery and L0 (
W1,Q1)We now report explained variance, CE recovery and L0 for BeFOND, BatchTopK, Matryoshka and Gemma Scope at all four widths (new Table 10). BeFOND activates more latents per token than the baselines, with comparable or better explained variance and cross-entropy recovery.
We also trained the baselines at target L0 from 200 to 3,200 (new Fig. 16). Against the in-house baselines, BeFOND leads on single-feature probing and RAVEL both at the L0 closest to its own and at each baseline's best L0, with a probing lead that grows with width; its RAVEL score also exceeds every released Gemma Scope checkpoint. Fig. 15 (formerly Fig. 12) already varies T for BeFOND: probing peaks within a few steps while RAVEL keeps improving. For the final paper, we plan a BeFOND L0 sweep through its decoder-norm target.
That said, the goal of an SAE is to recover monosemantic features: sparsity and reconstruction matter insofar as they help learn a better dictionary, and on quality metrics BeFOND outperforms standard SAEs.
On the AUC–accuracy gap in Table 5: BeFOND's posterior probabilities cluster near 1, which coarsens the ranking among active examples; probing on the posterior logits would test this.
Whether the features match the model's own semantics (
W2,Q2)We ran AxBench (new Table 11). On concept detection, BeFOND matches Matryoshka and exceeds BatchTopK. On steering, every method stays near the unsteered model with overlapping confidence intervals, which we read as uninformative here rather than as a ranking: AxBench requires an instruction-tuned model, all dictionaries here were trained on base Gemma-2-2B, and AxBench itself found that SAE steering underperforms simple baselines. RAVEL asks the same question, whether intervening on a feature changes the attribute the model represents, on the base model, and it separates the methods clearly: BeFOND leads at every width. RAVEL is also the industry standard SAE steering benchmarking in SAEBench. We have added a note about the set of entity attributes RAVEL uses for steering in Appendix D.6.
We consider LLM-judged automated-interpretability scores weak evidence here: a recent study of SAE benchmark reliability (Chanin, 2026) found them noisy and weakly discriminative between architectures, with sparse probing and RAVEL the most reliable signals.
Results on a single language model (
W3,Q3)We agree that a second model would strengthen the paper, and a replication is in progress for the final version. We chose a larger model with released reference SAEs (e.g., Llama-3.1-8B with Llama Scope) over GPT-2 small or Pythia: GPT-2 small is below the scale at which several SAEBench evaluations are reliable (in our GPT-2 runs, the absorption evaluation could not run at all), and Pythia's large outlier activations complicate SAE training.
Consistency of the empirical claims (
W4,Q4)We agree with each point and have revised the text. (1) Line 89 now says that BeFOND outperforms the in-house baselines on RAVEL and single-feature accuracy at every width, that Gemma Scope is higher on single-feature probing at the two smallest widths, and that BatchTopK leads several AUC and five-feature comparisons. (2) Lines 91–94 now state token efficiency against our baselines trained on the same 50M tokens (at 524K, +7.9 points RAVEL and +1.5 points single-feature probing over the strongest of them), keeping Gemma Scope as context and noting that our baselines also match it at the largest width. (3) The introduction and Section 5 now use the same numbers (8.2M tokens, a ratio of about 976). (4) Training cost and wall-clock times now appear in the main text (Fig. 5); equal-wall-clock baseline runs are planned for the final version.