- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework, TMA, addressing when an agent should update its policy and how. For the "when" question, TMA builds a linear probe that detects whether the agent hesitates during reasoning, and uses this signal as the cue for subsequent optimization. For the "how" question, TMA adopts online self-distillation to improve the model's task execution and reflection capabilities. Experimental results show that TMA outperforms related baselines on three evaluation sets.
Critical strengths
- Reasonable novelty. The study specifically targets the sub-problem of when to refresh the policy, and introduces a simple linear probe to determine whether the model is hesitating, thereby identifying the optimization signal.
Critical weaknesses
- The study still has related limitations that could be improved:
- The evaluation does not include multiple independent runs, so the statistical robustness of the method remains unclear.
- Failure-case analysis appears in the appendix, but I suggest the main text explicitly highlight in which situations the framework fails, and whether the failing parts can be improved.
- The paper's position is that whenever thinking (hesitation) occurs, it should be trained into the parameters. The paper only demonstrates that the training is effective; the evidence that the training is necessary remains insufficient.
- The paper relies mainly on Qwen models; supporting more base models would be needed to demonstrate that the framework's capability is loosely coupled with the underlying base model.
Questions for authors
Please see the points raised in the Weaknesses section.
2 comments
Official Comment by Authors
Dear Reviewer KZ69,
We would like to express our gratitude once again for the time and effort you have dedicated in reviewing our paper, including the rebuttal and discussion phases. Your insightful comments are invaluable for enhancing our work. In the final version of our manuscript, we will include the results of three independent runs, a clearer discussion of TMA's limitations and the role of parameter internalization, and additional cross-model generalization experiments.
Thank you for your response. After considering the rebuttal, I have decided to raise my score from 2 to 3.