- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates hierarchical concept representations in Gemma-2-9B and their relationship to activation steering. The authors analyze representation geometry across layers, perform reference-derived interventions, and use activation patching to trace how these interventions influence final predictions.
Critical strengths
The paper presents an interesting observation that improving geometric representations does not necessarily lead to correct decisions. However, the authors do not develop this observation into a sufficiently novel mechanistic explanation. The work mainly combines established techniques, relies on ground-truth information for its central interventions, and provides evidence from only one model.
Critical weaknesses
The authors show that early interventions are more effective than late interventions and that transferring the final prediction-position residual state can restore some correct predictions. However, this is not particularly surprising in an autoregressive Transformer, where the prediction position aggregates information from preceding tokens. The paper does not adequately distinguish a hierarchy-specific computational mechanism from general information propagation across layers. Furthermore, the evaluation is restricted to Gemma-2-9B, limiting the generality of the conclusions.
A major concern is that the steering direction is explicitly determined using the known correct category, as stated in Section 2. Therefore, the intervention does not demonstrate autonomous error correction or discovery of the correct semantic category. Instead, it injects ground-truth information into the model. The reported improvements of 2.85 and 3.40 percentage points at B10 are insufficient to establish a novel hierarchical decision mechanism.
Questions for authors
Where is your contribution in Section 1?