- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper shows that activation steering can be used to improve the model's ability to choose whether to call a tool or do something else (e.g. ask for clarification, refuse, etc.). It compares it to other steering baselines and finds it works better than those. It attempts to make broader claims about these results showing it is possible to "repair' some mistakes that a model makes using these targeted interventions.
Critical strengths
Potentially novel approach to evaluating some claims about model internals: The paper might be attempting to understand internal representations of tool choice. Though it is hard for me to understand what exactly is shown, see critical weaknesses.
Critical weaknesses
Soundness of the approach: The approach lacks comparison with the relevant baselines. My understanding is that the goal is to evaluate whether activation steering can be used to improve how models handle tool choice, and does so using supervised data. To justify progress along this goal, it is crucial to compare the proposed approach to simple supervised fine-tuning approaches such as simply fine-tuning a LoRA. The evaluations are also very narrow, mostly relying on What2Call, without making other measurements of the impact of such steering on simple benchmarks to evaluate how much performance degrades (like agent capabilities benchmarks). The paper claims that prior work might miss degradation due to only evaluating binary outcomes, but the more standard way of measuring such degradations is simply to run capability evaluations, which this paper does not do.
Clarity of the question being investigated and its motivation: I am not sure that my understanding of the problem this paper attempts to solve is correct. If it is correct, it has the soundness issues described above. If this is not the problem the paper attempts to solve, it does a poor job at explaining what problem it attempts to solve. A discussion or conclusion section would help the reader understand what the key takeaways are and how this represents progress towards a goal that could matter to AI practitioners or evaluators.
Questions for authors
- Are you making claims beyond the direct usefulness of these steering interventions and about how mechanistically this task works?
- If so, what are these claims, and why do they matter?
- If not, how does the proposed method compare to simple baselines?
- Why not rely on capability evaluations to measure model degradation?
Minor:
- Could you make the figures less dense and make the font size match the font size of the text? Figure 2 in particular is very hard to read.