- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies how linearly shaping the dense activations before learning an SAE affects the semantic selectivity of the learnt dictionary atoms. It provides two approaches: (1) J-space shaping: project activation before SAE onto a "live" space and discard some low-gain quiet directions; (2) The Matched Filter SAE (MF-SAE): a TopK SAE without per-latent scales or biases, whose encoder and decoder share the same unit-norm atoms, wrapped between a learned affine encoder and decoder; encoder and decoder are thus tied up to a single shared linear map. These methods on Gemma-2-2B and GPT-2 Small are evaluated using only sparse-probing accuracy on SAEBench and SAE Probes, compared against dense activations and TopK SAE. The results show that MF-SAE outperforms the baselines on SAEBench when only a few latents are probed, while almost no advantage on SAE Probes. And J-space shaping adds little on top of MF-SAE cross two benchmarks.
Critical strengths
MF-SAE does show some real gains: when only probing a few latents, it consistently outperforms the available Gemma Scope and SAEBench TopK SAEs across both layers.
Critical weaknesses
- The motivation is relatively unclear: there are several changes to the original SAE, but there is no clear motivation for what specific problem these changes aim to solve. There is also no explanation of why these changes work, either intuitively or through theoretical proof.
- “Semantic selectivity” is undefined throughout the paper: the research question mentions “whether linearly shaping can enhance semantic selectivity,” but there is no definition of “semantic selectivity” nor a defined metric to measure it. Instead, the paper uses probe accuracy, which mainly measures whether a few latents can linearly separate predefined labeled concepts, rather than whether a latent responds selectively to a single concept. A more comprehensive set of metrics (e.g., aggregate autointerp detection scores) is needed to support the claim of improved semantic selectivity.
- The effectiveness of J-space shaping is unclear: adding a J-space projection on top of MF-SAE adds little gains. What is the relationship between the components? There is no dedicated analysis of this.
- Relevant work is missing: preprocessing activations before applying SAE has been explored in prior work, such as PCA whitening (Saraswatula, Ashwin, and David Klindt. "Data Whitening Improves Sparse Autoencoder Learning."). The paper neither cites these works nor compares against them.
- The paper is incomplete: some experiments are claimed but not included (e.g., PVE, alive-atom fraction and CE recovered in Sec. 5.3), and some references or links in the paper are missing.
Questions for authors
- Could you please propose a more clearly defined and more comprehensive metric to quantify “semantic selectivity”?
- Could you please improve the motivation and clarify what problem the proposed methods aims to solve?
- Could you please complete the related work and comparisons?