- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
STAC augments GRPO for medical multimodal RLVR by probing an early reasoning prefix and using the policy’s relative support for its eventual answer as a stability signal. This signal is conditioned on terminal correctness to reward stably correct rollouts and penalize stubbornly incorrect ones. Experiments on six medical VQA benchmarks with Qwen3-VL-4B report small gains over GRPO and CARE.
Critical strengths
The pilot study is carefully designed, and the negative result on entropy-guided localization is useful.
The “same answer, different stability” framing is clinically intuitive and makes the limitation of outcome-only RLVR concrete.
The ablation on Shuffled Support and Direct Prefix Support shows that both trajectory alignment and outcome conditioning matter.
The method adds only one parallel forward pass and avoids external PRMs or continuation branching, which is a practical advantage.
Critical weaknesses
W1. The gains are marginal and lack statistical support.
STAC improves average accuracy by only 0.9 points over GRPO (62.4 vs. 61.5), smaller than several per-dataset standard deviations, and no confidence intervals or significance tests are reported.
W2. The prefix-support signal is noisy and uncalibrated.
The pilot shows only weak within-query correlation with continuation stability (–) and near-chance correctness concordance (–), yet STAC treats the rescaled support as a reliable trajectory-level score without analyzing fallback frequency or ranking stability.
W3. The distinction from confidence-aware RLVR is not experimentally isolated.
CARE also modulates a policy-internal signal by terminal correctness; the paper does not test whether prefix support beats a simple confidence proxy (e.g., rollout average log-probability) under the same reward structure.
W4. Generalization is under-supported.
All main results use Qwen3-VL-4B; Zero-RL gains shrink from 0.6 points at 2B to 0.5 points at 4B, larger scales (7B–8B) are not evaluated, and the relation to MedVR’s consensus-based credit assignment is not discussed.
Questions for authors
- What fraction of training examples trigger the neutral fallback ? When it does not trigger, how large is typically?
- Can the authors provide a controlled variant where prefix support is replaced by a terminal-confidence proxy under the same outcome-conditioned modulation?
- Why were 7B–8B backbones not evaluated, given the shrinking gains from 2B to 4B?
5 comments
Response to Reviewer VxG5 [2/3]
W2: The prefix-support signal is noisy and uncalibrated.
Ans: Thank you for highlighting this concern. We will address your concern from the following perspectives:
-
Correctness concordance: Prefix support reflects commitment to the eventual answer, which can be correct or incorrect, explaining the near-chance correctness concordance of 0.51–0.56. STAC therefore conditions the process reward on final-answer correctness, reinforcing high support for correct answers and discouraging it for incorrect ones. Removing this conditioning in the Direct Prefix Support ablation lowers accuracy in Table 2.
-
Informative stability rankings: Beyond the positive within-query correlations of –, the pairwise concordance reaches 0.674–0.782, compared with 0.5 under random ranking, as reported in Table 4 in Appendix B. Figure 1b further shows that high-support correct trajectories retain the correct answer more often than low-support ones. The Shuffled Support ablation in Table 2 also supports the value of this trajectory-level ranking signal.
-
Fallback frequency: Neutral fallback occurs in 10.6%, 8.2%, and 5.5% of groups entering rescaling at early, middle, and late training stages, mainly due to singleton candidate sets. STAC assigns these groups neutral support because their trajectories cannot be distinguished by this score. Details are provided in our response to Q1 below.
Q1: What fraction of training examples trigger the neutral fallback ? When it does not trigger, how large is typically?
Ans: Thank you for this insightful question. We analyzed the neutral fallback behavior across early, middle, and late training stages using 512 queries with 8 rollouts per query at each stage. The fallback and singleton rates below use the groups entering rescaling as their denominator.
R3-Table 3: Neutral fallback frequency and within-group support spread (). denotes the proportion of groups with a singleton candidate-answer set.
| Stage | Fallback Rate | Non-Fallback Span (Mean ± SD) | Median | |
|---|---|---|---|---|
| Early | 10.6% | 10.6% | 0.76 ± 0.18 | 0.80 |
| Middle | 8.2% | 8.2% | 0.74 ± 0.19 | 0.77 |
| Late | 5.5% | 4.9% | 0.69 ± 0.20 | 0.72 |
As shown in R3-Table 3, neutral fallback is infrequent among groups entering min-max rescaling and comes almost entirely from singleton candidate sets where . In practice, these singleton sets occur when some rollouts in a group fail format validation while the remaining valid rollouts all produce the same correct answer.
By contrast, fallback caused solely by when is nearly absent across training stages. Moreover, non-fallback groups have a large support spread. Both the mean and median of remain large throughout training, far above the threshold . This confirms that group-wise rescaling acts on well-separated support values rather than amplifying numerical noise.
Response to Reviewer VxG5 [3/3]
W3&Q2: Can the authors provide a controlled variant where prefix support is replaced by a terminal-confidence proxy under the same outcome-conditioned modulation?
Ans: We sincerely appreciate this suggestion for a more controlled comparison. To isolate the effect of early prefix support, we evaluated an Outcome-Conditioned Avg-LogP variant that replaces prefix support with the average token log-probability of the generated rollout, while keeping the same group-wise normalization, outcome-conditioned reward modulation, and process-reward weight as STAC:
R3-Table 4: Controlled comparison of different policy-internal signals under the same reward structure. GRPO, CARE, and STAC results are taken from Table 1 of the submitted manuscript.
| Method | PMC-VQA | MedXQA | OMVQA | VQA-RAD | SLAKE | PathVQA | Avg. |
|---|---|---|---|---|---|---|---|
| GRPO | 56.3 | 26.6 | 80.5 | 58.3 | 82.6 | 64.9 | 61.5 |
| CARE | 56.5 | 26.4 | 80.8 | 58.7 | 82.8 | 65.3 | 61.8 |
| Outcome-Cond. Avg-LogP | 56.7 | 25.7 | 80.7 | 59.2 | 82.6 | 64.4 | 61.6 |
| STAC | 57.0 | 27.5 | 81.2 | 59.9 | 83.2 | 65.6 | 62.4 |
As shown in R3-Table 4, STAC outperforms both CARE and the Avg-LogP variant across all six benchmarks, while the Avg-LogP variant provides little improvement over GRPO and underperforms CARE. While rollout average log-probability mainly reflects token-level generation fluency across the full text, STAC's candidate-relative prefix support measures how strongly an early reasoning state supports the final answer against competing options. This controlled comparison confirms that STAC's gains come from its prefix-support signal rather than outcome conditioning alone. We will add this ablation to Section 5.3 in our latest manuscript.
W4&Q4: Generalization across model scales (7B–8B) and relation to MedVR.
Ans: Many thanks for raising these important questions regarding model scaling and related work. To test how STAC scales to larger models, we extended our Zero-RL experiments to Qwen3-VL-8B-Instruct on the same benchmarks used for the 2B and 4B models:
R3-Table 5: Zero-RL performance across 2B, 4B, and 8B model scales.
| Model | Method | PMC-VQA | MedXQA | OMVQA | Avg. |
|---|---|---|---|---|---|
| Qwen3-VL-2B-Instruct | Zero-shot | 48.9 | 21.3 | 68.8 | 46.3 |
| GRPO | 49.3 | 22.2 | 69.1 | 46.9 | |
| STAC | 49.6 | 23.2 | 69.9 | 47.5 | |
| Qwen3-VL-4B-Instruct | Zero-shot | 53.4 | 25.4 | 77.9 | 52.2 |
| GRPO | 53.6 | 25.8 | 78.1 | 52.5 | |
| STAC | 54.1 | 26.1 | 78.7 | 53.0 | |
| Qwen3-VL-8B-Instruct | Zero-shot | 53.6 | 30.8 | 77.4 | 53.9 |
| GRPO | 53.9 | 31.1 | 77.9 | 54.3 | |
| STAC | 54.3 | 31.7 | 78.6 | 54.9 |
- Generalization across model scales: As shown in R3-Table 5, STAC consistently outperforms GRPO across all three model scales and all evaluated benchmarks. On the newly evaluated 8B backbone, STAC achieves the same average margin over GRPO as on the 2B model, showing that the improvement does not shrink as model size increases. Across all three scales in this challenging Zero-RL setting, which operates without SFT warm-up and includes two out-of-distribution benchmarks, MedXQA and OMVQA, STAC consistently doubles the RL gain over the zero-shot model compared with standard GRPO.
- Relation to MedVR [2]: MedVR and STAC both use group rollout information to refine credit assignment in medical multimodal RLVR without extra annotations, but they focus on different aspects of reasoning. MedVR uses spatial bounding-box consensus among successful rollouts to supervise visual tool-use actions, whereas STAC uses policy-internal prefix support to measure the continuation stability of individual reasoning trajectories, modulated by terminal correctness. The two methods are therefore complementary, addressing visual grounding consistency and reasoning trajectory stability, respectively.
We will add the 8B results to Table 3 and include an additional discussion of MedVR in the Related Work of our revised manuscript.
References.
[1] Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning. TPAMI 2026.
[2] MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning. ICLR 2026.
Thank you for the detailed feedback. We hope the added analyses and scale results, together with the clarified scope of our claims, address your concerns, and we respectfully ask that you reconsider your rating in light of these revisions. We also welcome further questions.
Response to Reviewer VxG5 [1/3]
We sincerely thank the reviewer for the detailed and constructive feedback, and we appreciate your positive comments on our pilot study and the efficiency of our method. Below, we provide additional experiments and clarifications to address each of your questions.
W1: The gains are marginal and lack statistical support.
Ans: Thank you for raising this concern. We added independent training runs for GRPO and CARE under the same training settings as STAC. R3-Table 1 reports the individual results and summary statistics to assess whether the observed gains exceed training variability.
Across these runs, all four STAC runs achieve higher average accuracy than the best observed run of either baseline. Averaging the six benchmark accuracies within each run and applying Welch's test to the four run-level averages per method gives 95% confidence intervals of [+0.59,+1.08] over GRPO and [+0.43,+0.90] over CARE. Both intervals exclude zero, supporting a positive average gain across training runs. We will report the multi-run results in the revised Table 1.
Besides, the magnitude of these gains is comparable to that reported in recent medical multimodal RLVR studies [1, 2]. The improvement is also not limited to average accuracy, as shown in Table 10 of Appendix E; STAC further improves Pass@8 and continuation stability, and it does so at modest cost, about 7% additional training time (R3-Table 2) by obtaining its process signal through a single parallel forward pass over short candidate answers, without external evaluators, step-level annotations, or continuation rollouts.