acceptodds
Withdrawn from ICLR 2027

VERGE: Verified Early-exit Reasoning with Grounded Evidence in Medical Multimodal Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long medical reasoning traces can contain a correct intermediate prediction followed by redundant or harmful continuation. We introduce VERGE (Verified Early-exit Reasoning with Grounded Evidence), a GRPO-based framework that uses intermediate verification to supervise credit assignment and reasoning termination. A reasoning continuation probe forces candidate prefixes to transition to answering and checks the resulting predictions against task-specific verification criteria. Accepted boundaries guide advantage redistribution over the original rollouts and supervise the reasoning-termination token; probe completions are used only for verification. VERGE handles discrete answer correctness and continuous localization quality, preserving each grounding rollout's mean GRPO advantage. Starting from Qwen3-VL-4B-Thinking, VERGE improves average accuracy across four medical QA benchmarks from 54.17% to 57.54%. Under a matched 4,096-token protocol, it outperforms Vanilla GRPO by 4.95 percentage points in MedMCQA accuracy and 5.61 points in grounding mIoU on the evaluated subset, while reducing mean output tokens by 77.2% and 69.6%, respectively. These results support learning to terminate reasoning from verified intermediate predictions.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Medical Multimodal Large Language Models, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XJe1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses overthinking in medical multimodal reasoning, where models reach a correct intermediate prediction but continue reasoning, wasting tokens or overturning the answer. The proposed VERGE framework extends GRPO by locating points where a rollout commits to an answer or bounding box, forcing a transition to answering via </think>, and verifying the resulting prediction. The earliest verified point serves as an early-exit boundary. It is used both to redistribute token-level advantages within the original rollout (with separate rules for discrete QA and continuous grounding rewards) and to supervise </think> generation through an auxiliary loss. Probe outputs are used only for verification, not as training targets. On Qwen3-VL-4B-Thinking, VERGE improves average medical QA accuracy from 54.17% to 57.54% and grounding mIoU across all eight modality groups. Against Vanilla GRPO, it achieves higher accuracy and mIoU with roughly 70–77% shorter outputs, with most gains attributed to the early-exit loss.

Critical strengths
  1. Well-targeted problem in medical reasoning. The paper clearly identifies overthinking as a practical issue in medical MLLMs, where correct intermediate predictions are often followed by redundant or answer reversing reasoning. The motivation is well illustrated (Figures 1 and 3), and the approach improves both accuracy and efficiency rather than trading one for the other.
  2. Strong grounding improvements with substantially shorter outputs. VERGE improves mIoU across all eight modality groups, with large gains on challenging modalities such as fundus (+27.1), ultrasound (+23.5), and MRI (+22.5), while reducing output length by roughly 70% compared to Vanilla GRPO.
  3. Principled verification-only design. Probe completions are used only to verify intermediate predictions and never enter the rollout group or training targets. This keeps the original GRPO reward statistics intact and clearly distinguishes the method from approaches that augment rollouts with edited trajectories.
Critical weaknesses
  1. Limited backbone coverage. All experiments use only Qwen3-VL-4B-Thinking, a thinking model that inherently produces long reasoning. Since VERGE also relies on the model explicitly committing to intermediate predictions, it is unclear whether similar overthinking patterns and gains hold for other models, such as instruct models trained with a <think> format or medical-specific MLLMs.
  2. Unclear contribution of credit redistribution. Although CR is presented as a core contribution, adding it on top of the early-exit loss yields only +0.41 (MedMCQA) and +0.70 (grounding mIoU) from a single run, both on in-distribution data. A CR-only variant (Vanilla GRPO + CR) and ablations on OOD benchmarks (e.g., MMLU-Med, SGPQA) are needed to isolate its effect.
  3. Insufficient related work on credit assignment. Although task-aware credit redistribution is presented as a core contribution, the related work focuses only on efficient reasoning and early exit. In the medical domain, several methods also address fine-grained credit assignment beyond outcome-level rewards (e.g. Med-PRM[1], MRPO[2], ChestX-Reasoner[3]). Since these works share the same motivation as VERGE, discussing them and clarifying VERGE’s distinction would better position its contribution.
  4. Limited analysis of premature exit. Aggregate gains may mask cases where VERGE stops reasoning too early, which is critical in the medical domain. The paper does not report cases where the base model is correct but VERGE is wrong, particularly on difficult benchmarks such as SGPQA where shorter reasoning may be harmful. Beyond the two examples in Figure 3, a quantitative analysis of the quality of the shortened reasoning would strengthen the claim that VERGE preserves task-relevant evidence.

[1] Yun et al. Med-PRM: Medical Reasoning Models with Stepwise, Guideline-verified Process Rewards

[2] Jung et al. Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning

[3] Fan et al. ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verification

Questions for authors

Q1 : It would be helpful to evaluate VERGE on an instruct model from the same family (e.g., Qwen3-VL-4B-Instruct) and on larger backbones of 8B or above (e.g., Qwen3-VL-8B), reporting accuracy, output length, and the proportion of rollouts with a verified boundary.

Q2 : I would suggest adding a GRPO + CR variant (without the early-exit loss) to Table 5 and extending all ablation variants to OOD benchmarks (MMLU-Med, SGPQA, MMMU-Med) and held-out grounding datasets not used in training, reporting both task scores and output length.

Q3 : It would strengthen the paper to expand the related work to cover medical credit assignment methods (see W3) and to clarify how VERGE differs from them.

Q4 : Adding a qualitative analysis of premature-exit failures, such as cases where the base model is correct but VERGE is wrong (especially on SGPQA), would make the evaluation more convincing. Further qualitative examples of the shortened reasoning, beyond the two in Figure 3, showing whether key clinical evidence is retained or omitted, would also be valuable.

  • I would be willing to raise my score if these concerns are adequately addressed.
Reviewer_jifiOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VERGE, a reinforcement learning framework designed to improve the accuracy and efficiency of medical multimodal reasoning by reducing unnecessary reasoning steps. The method uses a Reasoning Continuation Probe to identify and verify intermediate predictions, then applies task-aware credit redistribution and explicit early-exit supervision to encourage the model to stop reasoning when a reliable prediction is available. The framework supports both medical question answering and visual grounding. Experiments based on Qwen3-VL-4B-Thinking show improvements across four medical QA benchmarks and multiple medical imaging modalities. Compared with Vanilla GRPO under a matched evaluation protocol, VERGE improves MedMCQA accuracy by 4.95 percentage points and grounding mIoU by 5.61 points, while reducing generated tokens by 77.2% and 69.6%, respectively. The paper aims to demonstrate that verified early termination can improve both reasoning efficiency and prediction quality.

Critical strengths
  1. The paper addresses a useful problem and reports substantial gains in a controlled comparison. Under the matched checkpoint and 4,096 token protocol in Table 5, VERGE improves over Vanilla GRPO by 4.95 percentage points on MedMCQA and 5.61 mIoU points on the grounding subset, while reducing output tokens by 77.2% and 69.6%, respectively. Improving prediction quality while substantially shortening generation is practically valuable.
  2. The treatment of termination and credit assignment is coherent. Using probe completions only for verification, while training on the original trajectories, provides a useful design distinction. Extending this approach to both discrete QA rewards and continuous grounding rewards is also worthwhile. The centering operation correctly preserves each grounding trajectory's mean advantage, although this property alone does not establish better optimization.
Critical weaknesses
  1. Essential methodological details are missing. Section 4.2 does not explicitly define the region-wise grounding contrast z_{i,t}, including its scale and bounds. Appendix C leaves the verification thresholds and tolerances symbolic and does not provide sufficiently concrete candidate-extraction rules or probe budgets. These choices directly determine the training signal and stopping targets, making the central method difficult to reproduce or assess independently.
  2. The evidence does not yet establish the significance and robustness of the methodological contribution. DRE, Step-GRPO, and S-GRPO already study closely related approaches to learning early exit or preferring verified shorter trajectories [1,2,3], but the paper does not compare against them. VERGE’s distinction is meaningful, yet its incremental advantage remains unclear. Most of the improvement in Table 5 comes from exit supervision; adding credit redistribution contributes only 0.41 accuracy points and 0.70 mIoU, without repeated training runs or uncertainty estimates. Furthermore, the other models' QA scores in Table 3 are imported from MedMO, so the cross-model ranking is not established under demonstrably matched evaluation conditions.

[1] Wei et al. Dynamic Rollout Editing for Reducing Overthinking in RL-Trained Reasoning Models. 2026.

[2] Chen et al. Step-GRPO: Internalizing Dynamic Early Exit for Efficient Reasoning. ACL 2026.

[3] Dai, Yang, and Si. S-GRPO: Early Exit via Reinforcement Learning in Reasoning Models. 2025.

Questions for authors
  1. Reproducibility: Could you provide the exact definition of z_{i,t}, all verification thresholds, candidate-extraction rules, and probe budgets, together with code or detailed pseudocode? How sensitive are the results to these choices?
  2. Uncertainty and ablations: Do the gains, particularly those from credit redistribution, persist across independent training seeds? Please report confidence intervals for paired performance differences and variability in output length, and include a CR-only ablation. Appendix B.2 specifies greedy evaluation, so repeated stochastic training runs and uncertainty over test examples are more informative than simply changing the evaluation seed.
  3. Current baselines and fairness: Could you justify the backbone choice and include Qwen3.5-4B (and -9B) [4] and representative open-weight medical models [5–9]. Please use common evaluation settings and compare the closest early-exit methods with matched backbones and training data. Reporting inference latency and additional training cost from verification probes would clarify the practical efficiency benefit.
  4. Evaluation integrity: How are training, validation, and grounding evaluation examples separated at the patient, volume, and image levels? Also, how much of the improvement comes from fewer truncated or unparseable answers, and does the advantage persist with a more generous output budget?
  5. Reasoning faithfulness: Verification establishes answer or box correctness and consistency, but does not establish that the reasoning faithfully uses visual evidence. Do image masking or counterfactual-image tests support the qualitative claims about attention to task-relevant evidence in Section 5.3?

[4] Qwen Team. Qwen3.5-4B. 2026.

[5] Ossowski et al. OctoMed: Data Recipes for State-of-the-Art Multimodal Medical Reasoning. 2025.

[6] Dai et al. QoQ-Med: Building Multimodal Clinical Foundation Models with Domain-Aware GRPO Training. NeurIPS 2025.

[7] Dai et al. QoQ-Med3: A Multimodal Reasoning Foundation Model for Clinical Analysis. NPJ Digital Medicine 2026.

[8] Huang et al. MedVLThinker: Simple Baselines for Multimodal Medical Reasoning. ML4H 2025.

[9] Huang et al. Synthesizing High-Quality Visual Question Answering from Medical Documents with Generator-Verifier LMMs. ICLR 2026. Paper.

[10] Chen et al. HuatuoGPT-3: RL-Only Domain Adaptation from Base Models. 2026.

Reviewer_zu6hOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposed VERGE, a methods that mainly studies how to enable medical multimodal LLMs to reduce redundant reasoning while ensuring accuracy. The proposed method identifies positions where reasoning can be terminated early by verifying intermediate answers during the model's reasoning process. THe method then assigns different token-level advantages to different reasoning segments through Credit Redistribution, and trains the model via Early-exit Learning to stop reasoning at appropriate positions.

Critical strengths
  • Experiments show that the proposed method improves prediction accuracy while significantly reducing the token consumption.
  • The motivation of the task is really meaningful. I think most current medical multimodal recognition tasks do not require excessive complex reasoning, so being able to reduce the token cost is a good thing.
Critical weaknesses
  1. Can the proposed method distinguish between "harmful redundant reasoning" and "further evidence collection that is still valuable"? If not, does it sacrifice the reliability of medical reasoning in order to stop early? Reflection is the model's evidence query regarding the existing answer, which helps improve interpretability and accuracy in complex cases. I support reducing reasoning costs, but the proposed method may suppress the model's reflection and evidence collection, which may not align with the original intent of medical LLMs.
  2. From the experimental results (Table 5), early-exit learning brings the largest improvement. This may simply reflect some issues of medical MLLMs, for example, most medical image recognition tasks do not require complex reasoning and only need a one-pass response from the model. However, some particularly difficult datasets should still require the model to think repeatedly before obtaining an answer. Does this method merely enhance the model's ability to handle simple problems, and does it have a suppressive effect on relatively difficult medical image recognition tasks?
Questions for authors
  1. Can you provide experimental evidence that the method does not excessively suppress the model's active reflection and evidence collection, and that it will not, due to methodological issues, suppress the search for more evidence when the first round of reasoning yields an incorrect answer?
  2. Can you provide the changes in the model's performance across quesitons of different difficulty levels?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.