- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a novel SAE architecture for finding stable concepts across runs. The introduce a sparse coding iterative objective, where they switch from optimizing refitting the coefficients of the dictionary and the encoder weights, and also add a covariance regularization. The demonstrate their new architecture on stability and performance measures on a large set of toy as well as language and vision models against several other SAE baselines.
Critical strengths
The paper is well written and generally easy to follow. I appreciate the domain of research aiming to ensure that interpretability methods are stable. The analysis on toy + real models is great and gives a good flow of evidence to the paper. Analysis was done on language and vision models which strengthens the paper significantly.
I also thought the related work, particularly to older sparse coding and dictionary literature, was well done and interesting to read about!
Critical weaknesses
Both excluded baselines DL-FISTA (https://github.com/shrutij01/compositional-sparse-ood) and Archtetypal SAEs have public code releases (https://github.com/KempnerInstitute/overcomplete) and in my knowledge would be very string baselines. I don't think there is any need to 'reproduce' them as stated in Line 1039; the authors should either use the publicly released code on their own datasets, or also run their method (ConSDL) on the datasets/benchmarks released in the above repositories. Additionally, a publicly available Archetypal SAE for DINOv2 can be found at https://huggingface.co/matybohacek/RA-SAE-DINOv2-32k (to at least compare concept quality at inference time). If the baselines perform poorly during training, please simply report what you tried with the hyperparameters and put it in the paper, rather than exclude the evidence and reproducibility details altogether. Overall, it is a bit strange to me that the best performing baselines I was expecting to see (and I personally have experimented with) were omitted from comparison and this needs to be resolved in my opinion with any of the above options.
Questions for authors
What precisely did you try w.r.t DL-FISTA and Archetypal SAE baselines?
In Table 1, the results show that cross-seed consistency does not necessarily correlate with 'quality' of the learned dictionary; for example, when adding similarity reg., the number of duplicates lowers but consistency does as well. How can we know whether the concistency we are getting with ConSDL is 'good' consistency and improving the concepts learned, or actually hurting the dictionary in terms of the concepts learned?
7 comments
W1, Q1 - Missing baselines
Summary: We thank the reviewer for their constructive suggestions on how to compare ConSDL with DL-FISTA and Archetypal SAE (RA-SAE). Following these suggestions, we conducted additional experiments and added the results and implementation details to Appendix E.3. ConSDL outperforms DL-FISTA across all 16 synthetic settings (Table 7). For RA-SAE, we investigated preprocessing and relaxation bounds and extended training to 10 epochs for both methods (Table 8). Although RA-SAE improves with longer training, ConSDL achieves higher DCS at matched epochs on all four backbones, while one-epoch ConSDL outperforms 10-epoch RA-SAE on three of four backbones.
DL-FISTA. Our initial submission included a variant of DL-FISTA as the sparse-coding-only ablation (Table 1), retaining its FISTA-based coefficient inference while replacing closed-form dictionary updates with mini-batch gradient-based optimization. This adaptation allowed us to evaluate sparse coding at the scale of our foundation-model activations under the same training protocol as other SAE baselines. However, this variant exhibited substantial dictionary duplication on CLIP (9,270 atoms) and latent inactivity on Pythia (9,767 dead atoms).
Following the reviewer's suggestion, we directly evaluated DL-FISTA on our synthetic benchmark using the same 5M-sample training budget. ConSDL achieved higher reconstruction quality across all 16 settings, improved ground-truth dictionary and coefficient recovery, and achieved higher cross-seed consistency. It also exhibited substantially fewer duplicate atoms (0 vs. 391) and dead atoms (18 vs. 2,125) on average. We have added the full results in Table 7 and implementation details to Appendix E.3.
| Method | R² ↑ | GT DCS ↑ | GT MCC ↑ | Cross-seed DCS ↑ | Cross-seed MCC ↑ | # Dup ↓ | # Dead ↓ |
|---|---|---|---|---|---|---|---|
| ConSDL | 0.948 | 0.974 | 0.967 | 0.969 | 0.965 | 0 | 18 |
| DL-FISTA | 0.646 | 0.751 | 0.704 | 0.723 | 0.690 | 391 | 2,125 |
Archetypal SAE. Our initial experiments with our reimplementation of Archetypal SAE yielded poor reconstruction or substantial latent inactivity and duplication. Following the reviewer's feedback, we revisited the official implementation and evaluated several preprocessing strategies and relaxation bound (δ), as the original paper specified element-wise standardization but did not fully detail the preprocessing procedure. We found that mean centering followed by per-token L2 normalization gave the best overall results. The table below reports these results on CLIP with 4× dictionary expansion and ℓ₀ ≈ 32.
| Preprocessing | δ | R² ↑ | #Dead ↓ | #Dup ↓ |
|---|---|---|---|---|
| Z-score | 1 | 0.318 | 0 | 1,949 |
| Mean centering | 1 | 0.575 | 0 | 1,462 |
| Mean centering | 5 | 0.661 | 0 | 374 |
| Mean centering | 10 | 0.819 | 2 | 300 |
| Global scaling | 1 | 0.435 | 2,617 | 7 |
| Centering + token L2 | 0.1 | 0.839 | 175 | 1,311 |
| Centering + token L2 | 0.5 | 0.889 | 105 | 339 |
| Centering + token L2 | 1 | 0.900 | 0 | 429 |
Since the publicly available DINOv2 checkpoint has only one seed, we instead trained independent models using the above setup to evaluate cross-seed consistency.
In our original experiments, ConSDL and all SAE baselines were trained for one epoch. Since the original RA-SAE study used substantially longer training (50 epochs), we extended both RA-SAE and ConSDL training to 10 epochs across four backbones (4× expansion, ℓ₀ ≈ 32). Although RA-SAE improved with longer training, one-epoch ConSDL still achieved higher DCS and MCC than 10-epoch RA-SAE on three of four backbones. At matched 10 epochs, ConSDL also achieved higher DCS on all backbones, with slightly lower MCC on DINOv2. Full results are provided in Appendix E.3 (Table 8).
| Backbone | Method | 1 ep. (DCS / MCC) | 10 ep. (DCS / MCC) |
|---|---|---|---|
| CLIP | RA-SAE | 0.644 / 0.354 | 0.806 / 0.792 |
| ConSDL | 0.852 / 0.814 | 0.908 / 0.875 | |
| DINOv2 | RA-SAE | 0.730 / 0.701 | 0.922 / 0.905 |
| ConSDL | 0.905 / 0.866 | 0.935 / 0.896 | |
| Pythia | RA-SAE | 0.763 / 0.712 | 0.821 / 0.791 |
| ConSDL | 0.881 / 0.813 | 0.887 / 0.821 | |
| GPT-2 | RA-SAE | 0.807 / 0.758 | 0.850 / 0.807 |
| ConSDL | 0.887 / 0.835 | 0.898 / 0.850 |
Q2 Cross-seed consistency and concept quality
Summary: We agree that higher consistency alone does not guarantee better concepts. To address this concern, we provide three complementary analyses:
- Quantitative evaluations show that ConSDL maintains competitive concept detection and atom interpretability (Sec. 5.3, Appendix E.5).
- Qualitative analyses suggest that highly consistent atoms tend to capture clearer concepts, while ConSDL discovers diverse, fine-grained concepts (Sec. 6, Appendix E.6).
- Similarity regularization analysis shows that redundant atoms can inflate consistency scores, explaining why removing duplicates reduces measured consistency (Sec. 5.4, Appendix E.4).
Quantitative evaluation. We evaluate concept quality using two complementary SAEBench benchmarks [1]: sparse probing [2], which measures concept detection using a small number of latents, and AutoInterp [3], which evaluates individual atom interpretability. ConSDL achieves performance comparable to the strongest SAE baselines on both evaluations. At 16× expansion, Top-1 probing scores are 0.52/0.44 on CLIP/DINOv2 and 0.74/0.73 on GPT-2/Pythia (best baselines: 0.49/0.45 and 0.75/0.73). AutoInterp scores are 0.88/0.87 on GPT-2/Pythia (best baselines: 0.90/0.87). These results suggest that improved consistency does not compromise concept detection or atom interpretability (Tables 11–13; Sec. 5.3, Appendix E.5).
| Evaluation | Backbone | Baseline Avg. | Best Baseline | ConSDL |
|---|---|---|---|---|
| Sparse Probing (Top-1) | CLIP | 0.42 | 0.49 | 0.52 |
| Sparse Probing (Top-1) | DINOv2 | 0.40 | 0.45 | 0.44 |
| Sparse Probing (Top-1) | GPT-2 | 0.73 | 0.75 | 0.74 |
| Sparse Probing (Top-1) | Pythia | 0.71 | 0.73 | 0.73 |
| AutoInterp | GPT-2 | 0.88 | 0.90 | 0.88 |
| AutoInterp | Pythia | 0.86 | 0.87 | 0.87 |
Qualitative analysis. Our qualitative inspection suggests that low-consistency atoms often exhibit noisy or difficult-to-interpret activation patterns, while highly consistent atoms tend to capture clearer concepts (Fig. 9). We also observe that ConSDL learns distinct, fine-grained concepts, such as bottle necks, bottle shapes, and beer in glasses (Fig. 4), or baseball team names, pitching, and field positions (Fig. 10). To examine whether this pattern extends beyond individual examples, we analyze cross-method atom matching: 38% and 21% of ReLU atoms on CLIP and Pythia match multiple ConSDL atoms, compared with only 4.2% and 11% in the reverse direction (Figs. 11–12). Additional examples and analyses are provided in Sec. 6 and Appendix E.6.
Similarity regularization. Table 1 shows that similarity regularization substantially reduces duplicate atoms but also lowers cross-seed consistency. To investigate this, we analyzed the activation patterns of highly similar atoms in +Refitting (without regularization) and found substantial redundancy (Table 9). For +Refitting, removing these duplicates reduces DCS from 0.85 to 0.78 on CLIP and from 0.84 to 0.79 on Pythia, while Full ConSDL remains largely unaffected (Fig. 8). This suggests that redundant atoms can inflate consistency scores, motivating our joint evaluation of consistency and dictionary utilization rather than consistency alone (Appendix E.4).
[1] Karvonen et al. (2025). SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability. ICML 2025.
[2] Kantamneni et al. (2025). Are Sparse Autoencoders Useful? A Case Study in Sparse Probing. ICML 2025.
[3] Paulo et al. (2025). Automatically Interpreting Millions of Features in Large Language Models. ICML 2025.
Summary
Thank you for the constructive feedback. We hope our responses address the reviewer's concerns.
(W1, Q1) Missing baselines: We add direct comparisons with DL-FISTA and Archetypal SAE (RA-SAE). ConSDL outperforms DL-FISTA across 16 synthetic settings and achieves higher consistency than RA-SAE at matched training epochs across all four backbones.
(Q2) Consistency and concept quality: We provide additional interpretability evaluations and qualitative analyses, alongside existing sparse probing results, showing that ConSDL maintains competitive concept quality while improving consistency. We further show that redundant atoms can inflate consistency scores, explaining why removing redundancy can reduce measured consistency without degrading dictionary quality.
In the revised manuscript, blue text indicates newly added content, while green text indicates revisions to existing content.
We would be happy to provide further clarification if needed.