- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper propose PhaseCoT, that reduces the length iterative reasoning based on where to pause and how to resume. PhaseCoT segments reasoning trajectories at semantic milestones and rewrites segment heads to introduce context-aware continuation from prior summaries.
Critical strengths
None
Critical weaknesses
- The main idea of the paper is to make CoT "effective"; however, Table 1 shows that the model trained on the proposed dataset generates more tokens.
- The "semantic milestone" is not defined, and the LLM is asked to identify a milestone. There is no way to evaluate if the LLM has identified a milestone correctly. includes the summaries of all the previous milestones, which might contain the incorrect explored path. Including that in the training dataset does not make sense, and summarizing milestones removes the self-reflection that the models do to update their reasoning.
Questions for authors
- How does the model perform if the model is instructed to generate small CoTs and be more effective?
- What is the accuracy on the datasets without the no-think mode? This will help to identify whether CoT is helping the model or not.
Revised Manuscript: Changes Addressing Your Review (in Orange)
Dear Reviewer MH66,
We have uploaded a revised version of the paper. The change made in response to your comments is shown in orange and tagged [MH66]; changes for the other reviewers use different colors.
"Effective" versus "efficient" (W1). The Introduction now states explicitly that "effective" refers to reasoning accuracy: PhaseCoT does not aim to shorten iterative CoT, but to let it pause and resume in a more human-like way so that it reasons more accurately.
Two additions made for other reviewers also bear on your comments:
Thank you again for your time. We hope the revision clarifies the goal of our work, and we would be grateful if you could take it into account in your final assessment.