- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes Flash Point, a small LSTM probe that uses attention entropy and FFN activations from a frozen reasoning model to decide when to stop reasoning and generate an answer. Experiments on three models and three benchmarks show that the method can reduce reasoning tokens while maintaining or sometimes improving accuracy.
Critical strengths
- The problem is useful and the motivation is clear. Detecting when further reasoning is unnecessary could reduce inference costs and help avoid incorrect changes to an already correct answer.
- The method does not require updating the base model. Only the external probe is trained, which makes the approach useful when changing the base model’s weights is costly.
- The experiments include cross-task tests and layer ablations. These tests help show where the method works and which internal signals matter.
Critical weaknesses
- The structure of the paper is unclear. The contributions are not clearly explained in the introduction, and the discussion of latent reasoning is not clearly connected to the rest of the paper.
- The novelty and improvement over closely related methods is unclear. Prior work, such as [1], already uses hidden-state probes for early stopping, but this paper does not directly compare against such methods. Comparisons with simple entropy thresholds and fixed token budgets are also missing.
- The need for several design choices is not well supported. Removing the FFN features reportedly has little effect on token savings. The paper also lacks clear comparisons showing the value of the LSTM and the additional loss terms over simpler choices.
- The evidence for broad generalization is limited. Experiments use three models, three benchmarks, and a reasoning budget of 2,048 tokens. Some cross-task settings show accuracy drops or very small token savings, so the benefits are not consistent across tasks.
[1] Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. "Reasoning Models Know When They’re Right: Probing Hidden States for Self-Verification". arXiv:2504.05419, 2025.
Questions for authors
- How does Flash Point compare with [1], simple entropy thresholds, and fixed-budget stopping? Can these methods be compared at similar token or compute costs?
- How much do the FFN features, LSTM, and each additional loss term improve the results? Can an entropy-only probe trained with binary cross-entropy achieve similar accuracy and token savings?
- How does the method perform with longer reasoning budgets and harder math tasks? Are the 50 calibration examples and all threshold-selection data kept separate from the final test set?