- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The author of this paper studies the problem that in long-chain reasoning with LLMs, when an issue occurs at a certain step, it cannot be corrected in time, and the error propagates further and further, leading to an incorrect result. Therefore, in order to truly supervise in real time during reasoning whether erroneous reasoning occurs, the author proposes their own method, namely, when the model generates reasoning, there is a monitoring module (a 2-layer MLP with about 0.33M parameters) that supervises in real time. The monitoring module is divided into three parts: a monitor used to diagnose whether the current reasoning will lead to an incorrect result; a diagnostician (the same base model) that opens a separate context to try to repair the current step, avoiding the original model's preconceptions; and a commit gate (a conservative rule-based mechanism) used to determine whether the diagnostician's repair is reasonable. In terms of experiments, the method is mainly validated in two domains: mathematical reasoning and code generation. The mathematical tasks include GSM8K, MATH-500, and AIME 2025, and the code tasks include MBPP, with experiments conducted on models of different scales such as Qwen3-4B/8B/14B/32B. The author compares Self-Agent with methods such as Vanilla, Self-Check, and Self-Consistency, and further validates the effectiveness of the method through budget-matched Self-Consistency, Rescue/Harm analysis, and ablation experiments on the Monitor, independent Diagnostic Context, and Commit Gate. The experimental results show that Self-Agent can improve the model's final accuracy under limited additional inference overhead, and successfully rescue some results from originally incorrect reasoning trajectories, while keeping the damage to originally correct trajectories at a relatively low level; however, its benefits diminish as the base model scale increases, and without considering computational cost, the final accuracy of Self-Consistency is still higher.
Critical strengths
The method proposed in this paper for real-time monitoring of long-chain LLM reasoning is implemented in a lightweight manner. Apart from training a lightweight MLP, it requires no modification to the base model, and at the same time it improves the performance of the base model. The idea of lightweight design is highly valuable.
Critical weaknesses
The innovation is limited, and the performance improvement is insufficient. Previous work, such as Reflective Confidence: Correcting Reasoning Flaws via Online Self-Correction (ICASSP 2026), has already explored online monitoring during the reasoning process, using the model's own generated probabilities as confidence signals to trigger self-correction. In contrast, this work introduces an additional 0.33M-parameter MLP monitor, but it does not demonstrate sufficiently significant improvements in either methodology or lightweight design. In addition, most of the experimental baselines are based on relatively outdated methods from 2022 to 2023, and the proposed method does not consistently achieve state-of-the-art performance. Considering the additional monitoring and diagnostic components, the relatively limited performance improvement makes it difficult to assess whether the added complexity is sufficiently justified.
Questions for authors
- Can comparisons be provided with more recent and up-to-date real-time monitoring self-correction methods?
- Can generalization experiments on zero-shot transfer be provided to demonstrate the generalization ability of the lightweight method?