- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DecSteer, a lightweight residual operator trained with behavior cloning. It reports performance comparable to the RL-trained KV-Skill at substantially lower parameter and inference cost.
Critical strengths
The reported improvements in parameter efficiency and inference costs are of practical value, and the paper includes a number of diagnostic analyses.
Critical weaknesses
-
First, supervised learning for parameter-efficient adaptation and skill compression has already been well researched (e.g., ReFT and SoftSkill). The original KV-Skill paper itself reports that BC significantly outperforms RL on ALFWorld based on successful trajectories (69.4 vs. 42.3), so the fact that BC can work as a residual operator is not new (I am referring to the original Version 1 of KV-Skill, which was published prior to this submission). Therefore, a central claimed contribution is the explanation that the V=0 anchoring prevented training with BC. However, this causal explanation is not supported by KV-Skill’s own BC control experiments. V=0 operators can be trained well under BC on DocVQA using both the same 4B backbone and a larger backbone. LoRA, on the other hand, which does not use V=0 anchoring, fails under BC on exactly the same 4B tasks where KV-Skill also fails. The authors’ own new implementation (Table 4) also trains successfully with V=0. These results suggest that failure depends more on the task, the backbone, or the training recipe than on initialization alone, thereby providing insufficient support for the central mechanistic explanation.
-
All five benchmarks are QA-style tasks with explicit answer supervision. KV-Skill, by contrast, also evaluates many interactive and long-horizon settings (such as ALFWorld). A purely response-based BC is not directly applicable when no reference answers or expert demonstrations are available—and this is precisely the scenario that motivates RL. Nevertheless, the paper concludes that “reinforcement learning is unnecessary” and presents its operator as a general System 1 decision-making mechanism. Also, the paper does not clarify whether its efficiency advantage persists beyond answer-supervised tasks or whether the simplification sacrifices capabilities supported by the original design. These broader claims are therefore not justified by the evidence presented.
Questions for authors
-
Since KV-Skill’s own BC control experiments show that LoRA fails on the same 4B tasks, while V=0 operators succeed on other tasks and backbones, what evidence suggests that the failure of BC is due more to the initialization than to the task, the backbone, or the recipe? Could the authors conduct an “initialization × architecture” ablation analysis for Qwen3.5-4B using an identical recipe?
-
Can DecSteer learn a skill from reward alone, without gold answers or demonstrations, on the long-horizon tasks evaluated in KV-Skill (e.g., ALFWorld, OfficeQA)? If not, how should the claim that RL is unnecessary be qualified?
-
Under matched supervision and parameter budgets, how does DecSteer compare with ReFT and with a simple linear readout on the base model's late-layer states, given that the logit-lens analysis indicates the answer is already present there?
Follow-up on Our Response: We Would Greatly Appreciate Your Feedback
Dear Reviewer K8Pk,
Thank you again for your careful review and constructive suggestions, which has been very helpful in strengthening our work.
We have posted our responses, including the clarification on the contribution and additional experiments including more ablation studies about initialization × architecture, the effect of initialization on the training (both BC and RL), extra long-horizon tasks , RL with our methods and extra baselines (ReFT and with a simple linear readout).
When you have a chance, could you please let us know whether our responses adequately address your concerns? If any questions or concerns remain, we would be more than happy to address them. We would be happy to clarify any remaining points.