- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper concentrates on unifying activation steering methods for improving mathematical reasoning. Plenty of experiments are conducted on DeepSeek-R1-Distill-Qwen-1.5B and DeepSeek-R1-Distill-Qwen-7B. The eval sets are MATH500 and GSM8K.
This work first show that three steering methods--SEAL, REBALANCE and CPS--generate correct responses already in sampled results. Then decoder changes are compared to show that removing steering cache can keep most of the gains. Finally, the improvement from stochastic decoding can improve accuracy to the results of steering.
Critical strengths
The results is quite interesting. A quite clean explanation is given to show that steering methods mainly relax greedy decoding to improve the accuracy. The clean-cache experiment shows that the change of decoding gives most of the improvement. I think this experiment gives steering methods some insight.
Critical weaknesses
My main concern for this work is that the conclusion is not solid enough.
- The experiments are limited to 2 models that are only SFT trained. The results on models that are trained with RL are not clear. Since RL is known to turn pass@k to pass@1, I think results on RL models might be necessary.
- Only results on math are shown. Math-500 and GSM-8k are famous math test sets and might not be representative enough. Harder problems like AIME, new problems, and problems from other domains might give us more observations.
- Further, I am willing to see more formal explanation for the reasoning mechanism. I am not sure whether the greedy-relaxation is the only way for explaining steering methods. The greedy-relaxation itself is not quite clear. Are all tokens equally important? Or only some tokens work? I think a deeper analysis might provide more insights.
Besides, the presentation is hard for me. The references for tables are not clear and made me confusing. Tables and figures are not well organized and many results are directly given in text.
Questions for authors
See weaknesses above.