- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies overthinking in large vision-language models from a layer-wise perspective. The key idea is to detect entropy troughs near the final layers and identify cases where the preferred token changes between the trough layer and the final layer. An Evidence Verifier is then used to decide whether the final-layer token shift should be retained or replaced by the trough-layer token. During training, the method further incorporates changes in answer correctness and reasoning length into a GRPO-based objective.
Critical strengths
The paper studies overthinking from a layer-wise perspective rather than only controlling sequence length, which is an interesting and potentially useful direction. The Trigger–Verify design is intuitive and explicitly accounts for the fact that intermediate-layer predictions are not always better than final-layer predictions. The empirical results and ablations generally support the usefulness of the proposed components, including the verifier and the correctness- and length-aware rewards. The case studies are also informative, as they analyze concrete token substitutions made by EviGRPO rather than only reporting aggregate performance.
Critical weaknesses
The current experiments do not fully establish that the entropy trough itself is the key factor. It remains unclear whether the gains come from entropy-based layer selection or from a more general late-intermediate-layer intervention. The intermediate-layer distributions are obtained through the shared final LM head, so the cross-layer entropy may be affected by layer-dependent calibration rather than directly reflecting uncertainty. The Evidence Verifier is supervised by trajectory-level group-relative advantage instead of direct token-level outcomes. This may introduce credit-assignment issues when multiple token shifts occur in one trajectory, and the supervision target also changes as the policy evolves during RL training. The paper introduces several method-specific hyperparameters. Although the appendix reports sensitivity analyses, it is not fully clear whether these hyperparameters were selected on independent validation data.
Questions for authors
- The intermediate-layer distributions are obtained using the shared final LM head. How robust is the observed entropy-trough phenomenon to layer-wise calibration or alternative readout mechanisms? Does the same behavior hold consistently across different model families?
- The Evidence Verifier is trained from rollout-level relative advantages while the underlying policy is continuously changing. How stable is the verifier under this non-stationary supervision? Is there any evaluation of verifier accuracy or calibration during training, especially when multiple token shifts occur within one trajectory?
- What is the practical inference overhead of the additional intermediate-layer LM-head projections and verifier calls? It would be useful to report the trigger frequency, verifier activation frequency, and actual latency overhead.
8 comments
4. Hyperparameter Selection Details. As clarified in Appendix A.1, we reserve 10% of ViRL39K as an independent validation set and use the remaining 90% for policy optimization. Hyperparameters are selected based on validation accuracy and reasoning efficiency.
We have added these analyses and clarifications to the revised manuscript on Page 23, Lines 1206–1220 and 1227–1241 (Appendix D.4 and Table 15), Pages 23–24, Lines 1222–1225 and 1243–1261 (Appendix D.5 and Table 16), Page 19, Lines 1000–1024 (Appendix C.3 and Table 10), Page 22, Lines 1164–1169 (Appendix D.2 and Fig. 10), and Page 14, Lines 744–747 (Appendix A.1).
Q1. Robustness to Layer-Wise Calibration
Thank you for this question. As discussed in our response to Weakness 2, we conduct additional experiments in Appendix D.5 to examine the robustness of entropy troughs to layer-wise calibration. Specifically, we introduce learnable temperature parameters for the last ten layers and optimize them on LLaVA-CoT while keeping the LLM and verifier parameters frozen. As shown in Table 16, the entropy-trough matching rates reach 93.2% and 93.6% for Qwen2.5-VL-3B and 7B, respectively. These results indicate that most entropy troughs remain stable after calibration across both model scales.
Q2. Evidence Verifier Stability and Token-Level Evaluation
1. Evidence Verifier Stability Analysis. To examine the verifier's stability, we analyze its training dynamics in Appendix D.2. As shown in Fig. 10, the evidential loss decreases during training for both Qwen2.5-VL-3B and 7B. It then gradually stabilizes with moderate fluctuations. We attribute this stability to the supervision mechanism, which uses only the sign of the group-relative advantage and excludes zero-advantage samples, which may reduce the sensitivity to variations in reward magnitude.
2. Evidence Verifier Evaluation. We acknowledge that verifier accuracy and calibration are not separately evaluated during training. As defined in Eq. (11), the supervision signal is derived from the sign of the group-relative advantage and the corresponding token-selection action. When multiple token shifts occur within the same trajectory, their supervision shares the same rollout-level outcome, making it difficult to isolate the contribution of each token decision. We recognize this limitation and believe that introducing more precise token-level supervision could further improve the reliability of the verifier.
Q3. Inference Overhead of the Entropy Trigger and Evidence Verifier
Thank you for this question. We provide a detailed computational overhead analysis in Appendix D.1 (Tables 12 and 13).
As shown in Table 13, during inference, the entropy trigger is activated at 45.03% and 39.42% of reasoning positions for Qwen2.5-VL-3B and 7B, respectively, while the evidence verifier is activated at 32.44% and 30.74%. The corresponding trigger and verifier latencies are 0.21/0.10 hours for 3B and 0.27/0.12 hours for 7B. Despite these additional computations, EviGRPO reduces total inference cost by 35.58% and 34.62% compared with standard GRPO. Peak GPU memory increases by only 2.22% and 3.20%.
Table 12: Computational overhead analysis of GRPO and EviGRPO during training on Qwen2.5-VL-3B and 7B using 8 NVIDIA H100 GPUs.
| Model | Entropy Trigger Frequency (%) | Entropy Trigger Latency (h) | Evidence Verifier Frequency (%) | Evidence Verifier Latency (h) | Total Training Cost (GPU-Hours) | Peak GPU Memory (GiB) |
|---|---|---|---|---|---|---|
| Qwen2.5-VL-3B + GRPO | -- | -- | -- | -- | 83.75 | 53.62 |
| Qwen2.5-VL-3B + EviGRPO | 53.85 | 3.32 | 42.70 | 1.94 | 97.68 (+16.63%) | 55.74 (+3.95%) |
| Qwen2.5-VL-7B + GRPO | -- | -- | -- | -- | 113.55 | 70.90 |
| Qwen2.5-VL-7B + EviGRPO | 46.57 | 4.82 | 37.14 | 2.35 | 136.73 (+20.41%) | 73.65 (+3.88%) |
Table 13: Computational overhead analysis of GRPO and EviGRPO on the MathVision benchmark using Qwen2.5-VL-3B and 7B with 8 NVIDIA H100 GPUs.
| Model | Entropy Trigger Frequency (%) | Entropy Trigger Latency (h) | Evidence Verifier Frequency (%) | Evidence Verifier Latency (h) | Total Inference Cost (GPU-Hours) | Peak GPU Memory (GiB) |
|---|---|---|---|---|---|---|
| Qwen2.5-VL-3B + GRPO | -- | -- | -- | -- | 9.64 | 4.05 |
| Qwen2.5-VL-3B + EviGRPO | 45.03 | 0.21 | 32.44 | 0.10 | 6.21 (-35.58%) | 4.14 (+2.22%) |
| Qwen2.5-VL-7B + GRPO | -- | -- | -- | -- | 12.48 | 4.37 |
| Qwen2.5-VL-7B + EviGRPO | 39.42 | 0.27 | 30.74 | 0.12 | 8.16 (-34.62%) | 4.51 (+3.20%) |
Table 12 additionally reports the training overhead, where EviGRPO increases total training cost by 16.63% and 20.41% for the 3B and 7B models, respectively. The additional training cost mainly arises from the entropy trigger and evidence verifier, as well as the generation of both standard and entropy-trough-substituted responses to calculate the answer correctness and reasoning length rewards. However, inference requires only a shorter entropy-trough-substituted reasoning trajectory, reducing the overall inference GPU-hours despite the additional trigger and verifier computations.
We have added these computational overhead analyses to the revised manuscript on Pages 20–21, Lines 1051–1095 (Appendices D.1.2–D.1.3 and Tables 12–13).
We hope these responses address your concerns comprehensively. Feel free to let us know if you have any additional questions, and we are more than happy to provide further clarification on any aspect of our work.
Respectfully and sincerely,
Your Authors
Thank you for recognizing our layer-wise perspective on overthinking as an interesting and potentially useful direction, and for finding the Trigger–Verify design intuitive. We address your concerns and questions below.
W1. Entropy-Based Layer Selection, Layer-Wise Calibration, Verifier Supervision, and Hyperparameter Selection
Thank you for these comments. We address these concerns below.
1. Entropy-Based Layer Selection Analysis. To examine whether the improvements come from entropy-based layer selection or general late-layer intervention, we conduct additional comparisons in Appendix D.4. Specifically, Table 15 compares entropy-trough selection with ten fixed intermediate layers (L−1 to L−10) on Qwen2.5-VL-7B. The best-performing fixed layer differs across benchmarks, indicating that a fixed layer cannot consistently capture the optimal token predictions across different reasoning tasks. Compared with Layer L−3, which achieves the highest average accuracy among fixed-layers, entropy-trough selection improves average accuracy by 2.7 points while reducing reasoning length by 32.5 tokens. Furthermore, it outperforms the best fixed-layer accuracy on each of the four benchmarks by 1.2–3.2 points. These comparisons demonstrate the advantages of adaptive entropy-based layer selection over fixed late-layer intervention.
Table 15: Comparison of late-layer token substitution and entropy-trough token substitution on Qwen2.5-VL-7B. Acc. denotes accuracy (%), and Len. denotes the average number of reasoning tokens.
2. Layer-Wise Calibration Analysis. We further examine the effect of intermediate-layer calibration in Appendix D.5. We introduce learnable temperature parameters for the last ten layers and optimize them on LLaVA-CoT while keeping the LLM and verifier parameters frozen. As shown in Table 16, entropy-trough matching rates reach 93.2% and 93.6% for Qwen2.5-VL-3B and 7B, respectively. Most trough locations remain unchanged after temperature calibration.
Table 16: Robustness of the entropy-trough phenomenon to layer-wise temperature calibration on Qwen2.5-VL-3B and Qwen2.5-VL-7B. Entropy-trough matching rate measures the percentage of calibrated entropy troughs that match those obtained using the shared final LM head at the same token positions and layers.
3. Evidence Verifier Supervision. We acknowledge that trajectory-level advantages provide coarse supervision when multiple token shifts occur within the same rollout. Eq. (11) combines the group-relative advantage with each token-selection action, but cannot fully isolate individual token contributions. As the policy evolves, the verifier is updated using the latest rollout outcomes to adapt to changing supervision. Fig. 10 shows that the verifier loss gradually stabilizes during training. Furthermore, Table 10 reports 86.2% token-level verification accuracy using independently evaluated token choices. These results suggest that the verifier learns useful token preferences despite coarse supervision. However, we recognize that coarse supervision is not optimal and acknowledge this limitation. Introducing more precise token-level supervision for entropy-trough token selection may further improve EviGRPO.
Table 10: Token-level verification accuracy of the evidence verifier on Qwen2.5-VL-7B.