- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ARC, a trajectory-aware adaptive reasoning framework that uses hidden-state histories to dynamically control the reasoning process of frozen LLMs. A lightweight controller selects among four actions and is trained through PRM-based supervised initialization followed by GRPO. Experiments across multiple models and reasoning benchmarks demonstrate substantial token savings while maintaining comparable overall accuracy.
Critical strengths
- The motivation is clear and well presented. Using hidden-state histories to capture reasoning dynamics provides an intuitive motivation for trajectory-aware computation control.
- The method is relatively lightweight. The controller requires fewer than 2.8M trainable parameters and does not require fine-tuning the underlying reasoning model.
Critical weaknesses
- The experimental evaluation is largely limited to mathematical reasoning. Most benchmarks focus on mathematics (AIME24, AIME25, and AMC23). This limits the evidence for the generalization of ARC to other reasoning domains, such as coding and instruction following.
- The presentation could be further improved. For example, Figure 3 contains excessive whitespace and conveys relatively limited information, while some mathematical formulations could be presented more clearly.
- The accuracy–efficiency trade-off is not consistently favorable. Although ARC reduces token consumption across all evaluated settings, it still incurs noticeable accuracy degradation on some benchmarks (e.g., AMC23 on DeepSeek-R1-Distill-Qwen-1.5B).
Questions for authors
- How important is the HMM-based pseudo-label construction? Would simpler PRM thresholds or alternative initialization strategies achieve comparable results after GRPO training?
- How reliably do the four control actions affect the actual reasoning behavior? For example, does selecting Deep consistently lead to more extensive reasoning, while Compress and Conclude reduce reasoning length as intended? A brief quantitative analysis would be helpful.
Thank you for this suggestion.
HMM-based pseudo-label construction.
Our motivation is that similar PRM scores may correspond to different reasoning states depending on their sequence context. HMM incorporates score observations and state transitions, whereas independent thresholding does not explicitly use these temporal dependencies. However, without a controlled ablation, we cannot establish HMM’s independent contribution or rule out comparable performance from simpler initialization strategies after GRPO. We will consider comparing these alternatives under identical training settings, evaluating performance both before and after GRPO.
Effects and reliability of the control actions.
Appendices D.1 and D.2 provide qualitative evidence of behavior consistent with the action tags. In D.2, after
[Deep-Step], the model further develops a previously proposed time-dilation correction and completes the calculation correctly. In D.1,[Normal-Step]is followed by further verification, while[Conclude-Step]is followed by a concise final answer.These cases support alignment between tag semantics and reasoning behavior, but do not establish overall reliability, directly demonstrate Compress, or quantify average length reductions. We agree that quantitative analysis would strengthen the evidence and will consider comparing actions under identical reasoning prefixes, measuring subsequent token counts, steps to completion, and accuracy.