- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper examines the limitations of activation steering for abstract reasoning on ARC-AGI and Bongard-LOGO. Using SAE feature analysis, gradient alignment, and support-consensus interventions, the authors study whether steering directions derived from support examples transfer to unseen instances. They find that steering can improve output validity and prediction margins, but rarely corrects errors and sometimes reinforces incorrect predictions. These results suggest that improvements in these metrics do not necessarily translate into more reliable reasoning.
Critical strengths
- The experiments cover ARC-AGI-1, ARC-AGI-2, and Bongard-LOGO, with useful controls such as norm-matched random interventions and donor-concept controls. Together with the gradient-alignment analysis, these help clarify when steering works and where it falls short.
- The distinction between increasing prediction margins and correcting errors is an important point of the paper. I also appreciate the validation on 115 fresh concept combinations and the authors’ willingness to report that an initial hypothesis did not replicate.
Critical weaknesses
- The main gradient-alignment and support-consensus analyses are limited to Gemma-2-9B-it on Bongard-LOGO. Without broader validation, it is hard to tell whether the observed failure modes generalize to other models and tasks.
- The relationship between gradient alignment and correction remains unclear. Misalignment may help explain why interventions fail, but it could also reflect whether the model’s initial prediction was correct. The emphasis on pairwise ranking margins also makes it difficult to assess how much the interventions improve instance-level classification accuracy or reasoning.
Questions for authors
- Could the authors test whether the main gradient-alignment and support-consensus findings hold on other models, such as Qwen3 or DeepSeek-R1?
- Does gradient alignment still predict correction outcomes after controlling for baseline margins, task difficulty, and gradient magnitudes?
- Do the conclusions about error correction hold when evaluated using instance-level classification accuracy rather than pairwise ranking margins?
- How sensitive are these findings to the intervention layer, optimization objective, and intervention strength?