- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose a new method, DAR, that addresses the problem of feature absorption in already trained SAEs (i.e., it is the first such post-hoc method). Given a specified feature hierarchy (one parent and several children), the method calibrates the internal activations and the decoder matrix so that the output remains the same, but the parent feature fires more strongly. The experiments were conducted on 3 different models across text and visual modalities, on 3 different datasets with a given concept hierarchy. All experiments confirm that the level of feature absorption drops noticeably, which is further supported by a small causal experiment.
Critical strengths
- The authors propose a very simple, clear, cheap, and effective (absorption almost disappears across all metrics) method for addressing feature absorption in already trained SAEs.
- The authors evaluated their method across different modalities, model families, SAE types, and hierarchical concepts, which inspires confidence in all the conclusions of the paper.
Critical weaknesses
The main weakness of the method is its limited practical applicability. DAR requires a ready-made feature hierarchy as input, and in practice this involves two non-trivial tasks. First, the hierarchy has to come from somewhere: the authors use either the benchmark of Chanin et al., where the absorbing latents have already been identified, or external taxonomies such as ImageNet and WordNet, which simply do not exist for most domains. Second, each concept in the hierarchy has to be mapped to a specific SAE latent, which is hard in itself: a suitable feature may not exist at all, or the concept may be spread across several latents. The paper does not describe how this mapping is done for ImageNet and WordNet, even though all results depend on it directly. In essence, the method solves only the second half of the problem and assumes that the first and hardest half has already been solved. The work would be much stronger if the authors also proposed a way to discover such hierarchies directly in a trained SAE, for example from the feature co-activation matrix. This would give a complete pipeline: first automatically find the feature hierarchy, then remove absorption within it using DAR. In its current form, the contribution feels incomplete, and the method is applicable only to a small number of manually specified groups.
I am open to discussion. If the authors can show that I have misunderstood something or that my concerns are wrong, I will be happy to respond and reconsider my evaluation.
Questions for authors
How exactly are ImageNet and WordNet concepts mapped to SAE latents? What criterion is used, can one concept map to several latents, and what happens when no suitable latent exists? How many concepts could not be mapped, and how many groups were used in each setting?