- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents Viscot-steer, a training-free approach for transferring visual chain-of-thought (Visual-CoT) capabilities between large vision-language models. Building on activation steering, the authors propose to disentangle Visual-CoT into two capability-specific directions corresponding to visual grounding and reasoning. These directions are extracted through contrastive prompting, aligned with the target model's hidden space, and injected into different transformer layers during the corresponding generation phases.
The method is evaluated on several LVLM backbones and visual reasoning benchmarks. The results show consistent improvements over the base models and generally outperform existing capability-transfer baselines. The authors also provide ablation studies on steering directions, routing strategies, injection layers, and steering strengths.
Critical strengths
1. Clear motivation and a practical framework. The paper addresses a meaningful problem: transferring visual reasoning capabilities across models without repeating expensive post-training. The proposed method is straightforward, computationally lightweight, and does not require updating the target model's parameters. Separating grounding and reasoning interventions is also a reasonable design choice for Visual-CoT, where these two functions operate at different stages.
2. Strong empirical evaluation and presentation. The paper is clearly written and easy to follow. The experimental evaluation is fairly comprehensive, including several target models from different families, multiple benchmarks, comparisons with relevant baselines, and useful ablation studies. The results demonstrate that the proposed framework can improve downstream performance, sometimes by a considerable margin. Overall, the paper is well executed from an empirical perspective.
Critical weaknesses
1. Limited methodological novelty. The framework builds heavily on existing activation steering methods, particularly L2V-CoT. Its main contribution is to separate the original steering direction into grounding and reasoning directions and apply them at different layers and generation phases. While reasonable, this appears to be a relatively incremental extension of prior work.
2. The benefit of disentanglement is not fully convincing. In Table 4, reasoning-only steering already achieves 92.15% on V*, compared with 92.67% for the full method. This raises questions about the additional contribution of the grounding direction in this setting. Moreover, the prompt-based activation differences do not necessarily establish that the extracted vectors represent independent capabilities. The improvements might partly reflect eliciting existing capabilities of the target model rather than transferring newly learned capabilities from the source.
Questions for authors
1. Contribution of disentanglement. Could the authors compare the proposed two-vector approach against a unified steering vector under comparable phase-aware routing and layer-selection settings? This would help clarify whether the improvements come primarily from disentangling the capability vectors or from applying more appropriate intervention strategies.
2. Nature of capability transfer. Have the authors considered extracting steering vectors from the original backbone of DeepEyes, before Visual-CoT post-training, or directly from the target model using the same contrastive prompts? Such comparisons would help determine whether the method transfers source-specific learned capabilities or primarily elicits behaviors already present in the target model.
Reply: Thank you for raising this point. While both methods build on activation steering, they address different transfer problems. L2V-CoT transfers reasoning from a text-only reasoning model to a vision-language model. In contrast, our goal is to transfer visual-CoT capabilities from a post-trained visual-CoT model to other LVLMs without additional training.
Visual-CoT involves distinct capabilities, including grounding and reasoning. This raises a key question: which capability should be steered at which layer and generation phase? This question has received little attention in L2V-CoT and other activation steering methods. Our key contribution is to address it by steering grounding and reasoning at different layers and their corresponding phases.
Empirically, on Qwen3-VL-4B, L2V-CoT decreases accuracy from 49.66% to 48.20% on MME-RealWorld-Lite and from 72.75% to 72.50% on HR-Bench 8K, whereas our method reaches 50.70% and 74.75%, respectively. Overall, our method outperforms L2V-CoT in 22 of 25 model-benchmark settings.
Reply: Thank you for this observation. Our ablations support effective capability disentanglement, with the grounding and reasoning vectors serving their respective functions. V* answer accuracy provides only an indirect measure of grounding capability, so we additionally examine bounding-box IoU and response length. These ablations show their complementary effects on localization and reasoning.
Table R1. Accuracy, response length, and bounding-box IoU on V* with Qwen2.5-VL-7B (original paper Table 16, latest Table 18).
Grounding-only increases IoU from 0.3102 to 0.3445 with little change in response length. Reasoning-only improves accuracy and generates longer trajectories, but does not improve IoU. Combining both achieves the highest accuracy and IoU.
Our matched unified-vector and source-model comparisons further address the nature of disentanglement and transfer in Q1 and Q2.
Reply: Thank you for this suggestion. In our original unified-vector ablation (original Table 4, latest Table 4), the vector is applied at every decoding step, with its layer selected on the validation set. Following your suggestion, we additionally evaluate unified vectors at the same capability-specific layers and with the same phase-aware routing as our method.
Table R2. Unified-vector comparison on V* with Qwen3-VL-4B. The matched comparison uses the same capability-specific layers as Ours.
Under comparable phase-aware routing and layer-selection settings, our two-vector approach improves accuracy by 4.19 pp, supporting the benefit of capability disentanglement beyond the effect of the intervention strategy. We have clarified the original configuration in the captions and added the matched comparison to revised Tables 4 and 15.
Reply: Thank you for this suggestion. We extracted vectors from Qwen2.5-VL-7B, which is both the target model and the original DeepEyes backbone, using the same extraction data and contrastive prompts. We compare this target-self setting with DeepEyes-derived vectors under the same steering settings.
Table R3. Effect of vector source on V* with Qwen2.5-VL-7B.
DeepEyes-derived vectors outperform target-self by 9.95 pp, supporting the contribution of source-specific learned capabilities from visual-CoT post-training, rather than solely eliciting existing capabilities of the target model. We have added this comparison to revised Table 13 and described its settings in Appendix B, under “Effect of vector source.”