- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In their paper, the authors propose FlowSAE, in which the decoder is the exact inverse of the encoder. The activation is lifted into a higher-dimensional space, passed through an invertible flow, sparsified via TopK, and then inverted back, so at k = n the reconstruction is exact for any parameters. The motivation of the paper is that the reconstruction error of classical SAEs contaminates the effects of interventions. However, both the exactness guarantee and the main positive results apply only to the k = n case, which is not interesting in practice, since the whole point of SAEs is k << n. At the realistic sparsity level (k = 32), the reconstruction offset of FlowSAE is the same as that of a classical SAE, the feature effects become non-additive, and there is a lot of overhead from computational and architectural complexity (unlike vanilla SAEs).
Critical strengths
- The paper is very honest, and the claims are well calibrated. The authors added a linear control themselves and openly report negative results.
- Good reproducibility. the code is attached, and the experimental details are described thoroughly.
Critical weaknesses
-
The motivation of the paper only works for the k = n case, which is not interesting in practice, since the whole point of SAEs is k << n. But even at k = n, the problem is solved trivially. Exact reconstruction can be obtained without any flow, simply by taking an encoder with orthonormal columns and a transposed decoder. This is essentially the authors own linear control.
-
At the realistic k = 32, FlowSAE does not solve the stated problem. Its reconstruction error is the same as that of a classical SAE. At the same time, the feature effects become non-additive, so even after correction the attributions are worse than for any SAE with a linear decoder. The same linear control at k = 32 is both additive and more stable. In the end, the nonlinear flow only gives a gain in FVU, and even then only up to the level of a regular SAE.
-
In addition, the paper does not show at all what changes from an interpretability perspective. Interpretability is the main goal of SAEs, while reconstruction is essentially just a proxy and a loss. The paper has neither autointerp nor other standard metrics (SAEBench, SynthSAEBench). Perhaps the method would actually show improvements on them, but the paper only mentions that sparse probing performs roughly on par with existing SAE baselines.
I am open to discussion. If the authors can show that I have misunderstood something or that my concerns are wrong, I will be happy to respond and reconsider my evaluation.
Questions for authors
Is there a practical interpretability scenario at realistic sparsity (k << n) where FlowSAE outperforms standard SAEs? In particular, do you have results on standard interpretability metrics (e.g., autointerp, SAEBench, SynthSAEBench) at realistic budgets (e.g., k = 32 to 128), compared to classical SAE baselines? Positive results here would substantially change my assessment.
4 comments
Response to Reviewer gPSp (part 2): concept detection and autointerp across budgets
Following up on W3 (interpretability metrics at k = 32 to 128), we have extended SAEBench sparse probing for the multi-k FlowSAE from L0 = 32 to the published budgets L0 = 38, 76 and 155, with three seeds each. We compare against SAEBench's published SAEs on the same model, layer and width. The metric is the accuracy of a probe on the single best latent, averaged over SAEBench's eight datasets.
| L0 | FlowSAE (one model, 3 seeds) | Published TopK | Published BatchTopK |
|---|---|---|---|
| 32 | 0.764 ± 0.013 | – | – |
| 38 | 0.766 ± 0.015 | 0.734 | 0.739 |
| 76 | 0.783 ± 0.011 | 0.746 | 0.748 |
| 155 | 0.787 ± 0.013 | 0.792 | 0.773 |
- At L0 = 38 and 76, FlowSAE features detect concepts better than published TopK and BatchTopK, by 2.7 to 3.7 points, which is 2 to 3.5 seed standard deviations.
- At L0 = 155 it is level with them, 0.5 points below TopK and 1.4 points above BatchTopK.
- Against all seven published architectures at the nearest budget, FlowSAE is above five at L0 = 38 and 155 and above six at L0 = 76. Matryoshka-BatchTopK is the only architecture ahead of it at both larger budgets.
- Accuracy rises monotonically with the budget, and all four points come from one model per seed. Each published point is a separate model trained at that budget.
So in the range your W3 asks about, FlowSAE's features are not only on par with classical SAEs. At the sparser budgets they are more concept-aligned than the most widely used architectures. The published points are single runs, so we report our seed spread beside every number. They are now Table 6b of the revised PDF (App. L, marked in red), next to the loss-recovered comparison in Table 6a.
Autointerp, as an additional experiment. Our first response noted that SAEBench autointerp relies on a paid API. We have since run it with a free open-weight judge (Qwen2.5-32B-Instruct). A language model writes a one-sentence explanation of each latent and then predicts where it fires. The judge reproduces the published gpt-4o-mini ranking of the TopK SAEs, so the comparison is reliable.
| k | FlowSAE (one model, 3 seeds) | Published TopK (same judge) |
|---|---|---|
| 20 | 0.843 ± 0.003 | 0.850 |
| 40 | 0.816 ± 0.004 | 0.841 |
| 80 | 0.792 ± 0.006 | 0.827 |
| 160 | 0.778 ± 0.007 | 0.814 |
FlowSAE's latents score well above chance (0.714) at every budget, are stable across seeds, and all four points come from one model. They are close to TopK at the sparsest budget and a few points lower at larger ones.
The two experiments are best read together. Probing tests whether a latent detects a concept. Autointerp tests whether everything a latent responds to fits into one sentence. FlowSAE latents pass the first test better than TopK, and the second slightly less well as the budget grows. This is consistent with latents that track a concept reliably while also responding to a wider range of contexts. At the small budgets where features are read, FlowSAE is therefore at least as good a concept detector as TopK, and one model covers every budget. The autointerp results are Table 6c of the revised PDF.
Together with reconstruction within about 2 points of models trained for each budget, and the class-semantics evaluation of App. M, this answers the interpretability metrics your review asked for, with a clear positive result on concept detection below k = n. We would be grateful if you would take it into account in your assessment.
Thank you for the detailed responses and additional experiments. I agree that my original statement about positive results being limited to k = n was too broad. The new sparse-probing and ImageNet results address part of my concerns, and I am raising my score to 2.
My main remaining concern is the balance between complexity and demonstrated benefit. FlowSAE introduces a substantially more involved architecture, while at realistic sparsity its reconstruction is comparable to standard SAEs and the interpretability results are mixed: sparse probing improves at some budgets, but autointerp scores are lower than TopK. I appreciate the exact reconstruction guarantee and the careful analysis of its limitations, but the results do not yet establish a sufficiently clear practical advantage to justify this added complexity.
Response to Reviewer gPSp
Thank you for engaging with our responses and for raising your score. We agree the case should rest on concrete use, so we state it precisely.
1. Interventions without a residual pathway. Circuit and attribution analyses with classical SAEs must keep the reconstruction error as a separate node or add it back after intervening. Without it, single-feature attributions can be inverted (Spearman ρ = −0.83 and −0.62 for two of the classical SAEs we test). FlowSAE removes this by construction: features are selected at k = 32 and interventions are made in the exact k = n representation, so every effect is carried by features. The linear control is also exact, but its features at k = 32 reconstruct 1.8× worse (FVU 0.397 vs 0.216). FlowSAE is the only model in our comparison that offers both.
2. One training run for every budget. SAEBench trains a separate SAE for each L0. One FlowSAE model stays within about 2 points of loss recovered of these per-budget models from L0 = 38 to 155, detects concepts better at L0 = 38 and 76, and is exact at k = n. When the budget an analysis needs is not known in advance, this replaces a sweep of trainings with one.
On autointerp, we agree, and we report it as lower than TopK at larger budgets. At the sparsest budget, where features are read, the two are close (0.843 vs 0.850 at k = 20).
On complexity, the cost is a constant factor per layer (App. C, D). Training uses a standard SAE objective and pipeline, and the base model is not modified.
FlowSAE is not a replacement for classical SAEs in every use, and the revised paper says so. Its advantage is specific: exact, residual-free interventions and one model for all budgets, with reconstruction and concept detection comparable to the best per-budget SAEs. We hope this clarifies the practical case.
Response to Reviewer gPSp: results below k = n and new interpretability evidence
Thank you for the detailed review and for your openness to discussion. We would like to correct one central point, and then answer each weakness.
"The exactness guarantee and the main positive results apply only to k = n." Respectfully, the positive results are not confined to k = n. Three are at k < n:
The k = n result also says something new about classical SAEs. Their uncorrected attribution rankings can be inverted (ρ = −0.83), not just noisy.
W1: exact reconstruction is achievable with a linear map. Agreed, and we built that control ourselves to isolate exactly this question. It is exact but reconstructs poorly (FVU 0.397 at k = 32). Classical SAEs reconstruct well but are not exact. FlowSAE is the only model in the comparison with both properties. That combination, not exactness alone, is the contribution, and the revised introduction says so up front.
W2: at k = 32 the offset matches a classical SAE, and effects are non-additive. No SAE is exact at k < n, and FlowSAE is not worse there: its offset is 0.585 vs 0.577. Non-additivity is inherent to any nonlinear decoder, and we quantify it directly (ρ = 0.56 after correction, Table 2). We also give a protocol that sidesteps it: select features at the deployed budget, then intervene in the exact k = n representation, where effects carry no reconstruction error. Non-additivity can also be reduced in training. The revision adds an explicit additivity penalty (App. H), which at λ = 0.1 lowers the additivity error 7× (1.41 to 0.19) at FVU 0.220, a trade-off the linear control (FVU 0.397) does not offer, while the k = n identity stays exact.
Overhead. A forward pass costs O(LNnd), plus an O(nd² + d³) setup per layer that does not grow with the number of samples (App. D). Per sample this is the order of a classical SAE times the depth L, so the overhead is a constant factor. We measure it openly (App. C, D) and do not hide it. The method trains and evaluates at d = 768 on Pythia-160M with the same pipeline as at d = 384.
W3: interpretability metrics at k = 32 to 128. The paper reports more than sparse probing. Table 5 of the submission (Table 6 in the revision) gives SAEBench
coreacross L0 = 19 to 630. The revision adds SAEBench's published TopK SAEs at matched L0, on the same model, layer and width:FlowSAE stays within about 2 points of models that were each trained at that budget. At large budgets, most of that gap is the price of covering every budget with one model: our FlowSAE specialist trained at k = 458 reaches 0.997, against 0.984 for the multi-k model and 0.999 for the best published SAE. On concept detection, sparse probing at L0 = 32 (0.764 ± 0.013, three seeds) is on par with standard SAEs and above full-rank PCA.
To test directly whether individual features correspond to concepts, the revision adds a class-semantics evaluation on ImageNet (new App. M, Fig. 4 and Table 8). For 64 random latents per model at k = 32, FlowSAE latents are as class-selective as those of a classical SAE trained identically (purity of the top-50 images 0.31 vs 0.34, chance 0.03). The five classes a FlowSAE latent promotes in the frozen classifier agree with the labels of the images that activate it almost twice as often (0.43 vs 0.24, p ≈ 1e-5, chance 0.005), and they are more semantically coherent (WordNet similarity 0.65 vs 0.54, random sets 0.45). For example, a latent that fires on stoves promotes stove, wok, frying pan, coffeepot and microwave.
We do not report SAEBench autointerp. In its released form it scores feature explanations with gpt-4o-mini through a paid API, which is not an option for this research.
Practical scenario at k ≪ n.
We hope this shows that the contribution extends beyond k = n, and we would be grateful if you would reconsider your score.