- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Cross-Layer TopK for training cross-layer transcoders. Instead of selecting a fixed number of features within each layer, it selects features jointly across all layers under a fixed per-token budget. This allows the model to allocate more active features to layers that are harder to reconstruct. Experiments on Qwen3-8B-Base show improved reconstruction and higher circuit replacement and completeness scores compared with JumpReLU and other TopK variants. The paper also combines the method with sparse decoding and a different tensor-parallel layout, reporting a 2.9× reduction in training step time, and studies reconstruction scaling with dictionary size and sparsity.
Critical strengths
-
The idea of sharing a fixed sparsity budget across layers is simple and well motivated. It combines precise control over the total number of active features with flexibility in where those features are allocated, addressing a practical limitation of per-layer TopK.
-
The improvement extends beyond reconstruction to circuit-level metrics. At the same budget of 360 active features, Cross-Layer TopK improves the replacement score from 0.577 to 0.617 over Per-Layer TopK.
Critical weaknesses
-
The conceptual contribution appears relatively incremental. Dynamic budget allocation already exists in BatchTopK, and the paper transfers this idea to the layer axis. The conclusion also acknowledges that TopK variants for CLTs have appeared in earlier implementations, but does not clearly identify what was already available. The most direct comparison isolating cross-layer allocation improves FVU from 27.64% to 27.07%. This is useful, but I am less convinced that it represents a substantial advance beyond a better allocation rule. The 2.9× speedup mainly supports the accompanying implementation improvements rather than the novelty of cross-layer selection itself.
-
The impact on interpretability is less clear than the impact on reconstruction. Replacement and completeness scores measure how much attribution is carried by features rather than error nodes, but do not directly establish that the features are more understandable or that the resulting circuits better reflect the original model’s causal computation. Since improved circuit interpretation is a central motivation, I would like a clearer account of what these results establish beyond a more accurate sparse approximation. This distinction is important to my overall assessment.
Questions for authors
How should we interpret the learned layer allocation: does it mainly reflect reconstruction difficulty, or the fact that earlier-layer features can contribute to more downstream layers? In particular, how does the decoder-norm-based scoring affect this allocation?
3 comments
Response to Reviewer vYyd #2
Q.1. How should we interpret the learned layer allocation: does it mainly reflect reconstruction difficulty, or the fact that earlier-layer features can contribute to more downstream layers?
This is a good question! We believe that such allocation primarily reflects reconstruction difficulty. As shown in Table 2, the intermediate layers (8–24), which are the most challenging layers, receive a higher quota under Cross-Layer TopK than under Per-Layer TopK, accompanied by a clear reduction in FVU. We do not attribute this allocation to the fact that shallow layers serve more downstream layers. This is due to the fact that layers 1–5 in fact feed into the largest number of downstream layers, yet they are assigned a budget below k.
Q.2. In particular, how does the decoder-norm-based scoring affect this allocation?
This is a good question! Decoder-norm-based scoring when selecting activations follows common practice, e.g., https://transformer-circuits.pub/2025/january-update/index.html#DL. Multiplying the activation by the decoder norm yields the actual length of the feature decomposed within the model’s hidden space, consistent with the definition of features introduced in https://www.anthropic.com/research/toy-models-of-superposition.
This step is uncommon for SAEs because the decoder is renormalized to unit norm after training, with the encoder scaled by the original decoder norm (Anthropic, https://transformer-circuits.pub/2025/january-update/index.html). Alternatively, the decoder may be normalized at every training step (OpenAI, Appendix C.4, https://arxiv.org/abs/2406.04093). In these cases, the decoder norm equals one at inference time and can therefore be ignored.
By contrast, CLTs cannot renormalize decoders after training. Since each encoder feeds into multiple decoders, there exists no unique scaling factor. For example, suppose a feature from layer 1 writes to the current layer with a decoder norm of 5 and writes to the next layer with a decoder norm of 10. For a single-layer transcoder, we can scale the corresponding encoder entry by 5 and renormalize the decoder to 1 while preserving reconstruction. For a CLT, however, after scaling the encoder by 5, the decoder to the next layer still needs to retain a norm of 2 so that the vector written to the next layer remains unchanged. It is therefore impossible to normalize all decoders to unit norm, and we must rely on decoder norms to score activations.
Thank you for the response. I agree that Cross-Layer TopK improves reconstruction in several layers, particularly the hardest-to-reconstruct middle layers. Given the incremental nature of the methodological contribution, I find the overall FVU improvement still too modest to change my assessment. I therefore maintain my original rating of 2.
Response to Reviewer vYyd
We appreciate that the reviewer recognized our strengths that: (1) Our idea of sharing a fixed sparsity budget across layer is simple and well motivated. (b) The improvement extends beyond reconstruction to circuit-level metrics. We have summarized the main contributions and core work of our paper in the global response, and we kindly refer you to the global response for further elaboration. We address the reviewer's concerns here:
W.1. The conceptual contribution appears relatively incremental, and the 2.9× speedup mainly supports the accompanying implementation improvements rather than the novelty of cross-layer selection itself.
Thank you for your comment. We would like to point out that although our improvement is structurally simple, we believe that it addresses an important problem. The intermediate layers of the model encode more abstract and rich semantics, so better reconstruction and interpretation of these layers carry greater significance. We provide a more detailed discussion on the importance of reconstructing intermediate layers in our global response. We kindly refer you to our global response for more details. Existing CLT architectures typically achieve relatively good reconstruction on shallow and deep layers yet perform poorly on intermediate layers. The motivation of this work is that, under a fixed sparsity budget, the model can adaptively allocate more budget to the harder-to-reconstruct intermediate layers to achieve improved reconstruction performance for them.
We agree that the overall gains over Per-Layer TopK are small. However, the main advantage of Cross-Layer TopK is that it gained better performance on the intermediate layers which are the hardest to reconstruct. For example, on all layers whose FVU is above 40% on Per-Layer TopK, i.e. Layer 15~19, the differences of FVU between Per-Layer and Corss-Layer TopK are shown below:
The average decrease of these layers is 1.402%. The overall FVU is averaged across all layers so the improvement looks slightly. However, The FVUs of the remaining layers are already relatively small compared with these layers (24 out of 36 layers are below 30%). Further improving the reconstruction of these layers yields marginal gains, which are out of scope of our work.
W.2.The impact on interpretability is less clear than the impact on reconstruction.
Thank you for your comment! It is a good idea to evaluate the feature interpretability directly, such as using auto interp score.
We sampled 100 features on each intermediate layer, and used GPT-5.5 to evaluate the auto-interp score. The results are shown below.
The result shows that the features interpreted from Cross-Layer TopK gained the highest auto-interp score, meaning a better interpretability.