- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes DualSAE, a modular framework that augments standard SAEs with a plug-in residual branch. Traditional SAEs use a single sparse representation responsible for both reconstruction fidelity and interpretability. Since reconstruction favors any useful information, including non-semantic information that may not correspond to interpretable concepts, it can force latents to mix multiple factors. To separate these objectives, a standard SAE serves as the Sparse Branch, which learns a sparse code containing the dominant semantics of the input and reconstructs it as x_M. The added Residual Branch takes the residual r = x − x_M as input and models the non-semantic information omitted by the Sparse Branch. The final reconstruction is x̂ = x_M + r̂. Across diverse encoder-SAE configurations on two vision datasets and two language corpora, DualSAE improves both reconstruction fidelity and interpretability.
Critical strengths
The method is simple and modular, since the residual branch can be plugged into many existing SAE variants without changing their sparsity settings. The experiments are broad, covering both vision and language models.
Critical weaknesses
- The paper's main idea is to keep interpretable information in the Sparse Branch and to move non-semantic information that only helps reconstruction into the Residual Branch. But in Figure 10, after adding r̂, the retrieved images change from “any dog” to “dogs with coat color and texture very similar to the query image”. The caption says “finer visual attributes of the query, including coat color and texture”, and the label in the figure even directly says “Meaningful features to complement x^M”. But coat color and texture are not noise. They are typical interpretable concepts. Standard SAEs often learn features like “brown fur” and “striped texture”. So at least part of what the Residual Branch holds is semantic, but the authors call it “non-semantic” in the same paragraph, which is very contradictory. And the authors never define what “non-semantic” means. Attributes like coat color and texture are naturally shared across classes. A “brown fur” feature can appear in dogs, bears, and horses at the same time, so retrieving with it will of course return images from different classes. So “retrieving different classes” cannot distinguish between two cases: “it is really noise” and “it is a cross-class attribute feature”. The authors also count the latter as non-semantic. Based on this, if part of what goes into the Residual Branch is actually made up of interpretable features, then the good results do not come from a better decomposition, but from attribute features being moved out.
- The comparison with the baselines does not seem fair. The baselines use only 64 sparse latents (TopK, BatchTopK) to reconstruct each input, while DualSAE uses 64 sparse latents + r̂, which contains information from the real input. Obviously, DualSAE has much richer information. The reconstruction metrics of SAEs are meant to measure “how much of the model’s behavior the interpretable features can explain”. But part of DualSAE’s x̂ comes from a black box, so the metric now measures “how much the interpretable features and the black box can explain together”.
- Purity is computed based on ImageNet class labels. This gives an advantage to class-level features such as “dog”, but is less fair to cross-class attribute features such as “brown fur”. DualSAE happens to keep only the dominant class-level semantics, so it benefits from this metric.
- In Table 4 of the appendix, the row for JumpReLU SAE says “BatchTopK activation”. Maybe there are other writing errors, please check carefully.
Questions for authors
- For Weakness 2, maybe the authors could throw r̂ away after training DualSAE and only use x_M. Compute the R² between x and x_M, as well as ΔCE and KL. Also, for a standard SAE, authors could first train it and freeze it, then attach the same Residual Branch and train only the Residual Branch. The only difference between this and DualSAE is whether the Sparse Branch is trained jointly with the Residual Branch. This would show how much of the gain comes from the Residual Branch itself and how much comes from joint training.
- Could the authors evaluate with attribute-level labels or use auto-interp for testing?
- Could the authors report the L0 of JumpReLU and Gated in the baselines?
- Could the authors provide a quantitative evaluation of steering?