- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies how to keep the effect of activation steering consistent across different hidden states in VLMs. Instead of using a fixed steering vector, the authors propose to build local coordinate systems for each state and learn transitions to transfer the intervention between them. They also introduce a denoising-based step to keep the edited activations valid. The experiments show that the method works better for longer multi-hop paths, while the improvements on standard benchmarks are relatively small.
Critical strengths
The paper presents an interesting idea of representing activation interventions using local coordinates tailored to each hidden state, rather than relying on a fixed global steering direction
Critical weaknesses
There are three primary weaknesses: (1) lacking justification for semantic persistence, (2) large inference-time complexity and (3) lack of justification for each component of the proposed method.
For the first point, it remains unclear why semantic persistence is necessary in practical VLM applications. In particular, under what realistic inference scenarios must an intervention be repeatedly transported across multiple anchors, rather than applying an independently constructed intervention at each state? The experiments demonstrate improvements on predefined multi-hop paths, but the connection between these paths and real-world inference scenarios is insufficiently established. Consequently, it is difficult to assess whether the proposed framework addresses a significant limitation of existing activation-steering methods.
The next point is a concern about additional computation during inference. For unseen anchors, it requires constructing local chart bases through activation retrieval and behavioral probe evaluations, computing state-dependent transitions through a neural network and a matrix solve, and performing iterative optimization of edited hidden states using a denoising-based validity objective. Moreover, lthough the paper reports inference latency, the efficiency comparison is primarily against independent local charts rather than popular activation-steering methods. Given the considerable computational overhead and relatively modest improvements on single-step benchmarks, it remains unclear whether the performance gains justify the additional inference cost.
Finally, the method combines several components whose necessity and interactions are not sufficiently motivated from a unified perspective. Although ablation studies are provided, a clearer explanation on how the way each components are designed that way is appreciated.
Questions for authors
- It would be helpful if the authors could provide a clearer step-by-step walkthrough of the extraction and inference phases, explaining the role of each component, why it is needed, and how much computational overhead it introduces, especially during inference.
- It would be nice to see comparisons with some widely used activation-steering methods to better understand the effectiveness and efficiency of Atlas. Also, could the authors elaborate a bit more on why semantic persistence is important in practice, perhaps with a concrete example where existing steering methods struggle, or mechanistic interpretation for it?