- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies diagnostic reasoning across multiple medical images. The proposed framework, MedInSight, uses Cross-image Evidence Routing (CER) to select images for further inspection. It also uses Cross-horizon Credit Assignment (CCA) to refine the training signal for individual inspection actions. The authors evaluate the model on six medical benchmarks, with ablations, human assessments, and case studies. The results suggest that targeted evidence selection and finer-grained training feedback can improve diagnostic performance.
Critical strengths
1.The method addresses two practical questions in multi-image diagnosis: what to inspect next, and how to learn useful inspection behavior. CER and CCA address these questions in a coherent way. The appendix also includes a control that gives the model access to all images. That comparison supports the value of targeted evidence selection. 2.The ablation results are consistent across benchmarks. With the same backbone, the full method outperforms both supervised fine-tuning and vanilla GRPO on all three ablation benchmarks. The gains over supervised fine-tuning are about 11 percentage points. Comparisons against random selection and access to all images provide further support. These results are the main reason I lean toward acceptance.
Critical weaknesses
1.The performance gains from CCA are clearer than the evidence for why it works. The long-horizon ablation also changes the strength of the correction, making it hard to isolate the role of action weighting. Shuffled CCA is a useful additional control. However, it is evaluated on only 57 cases, where the full method answers three more cases correctly. Figure 9 also needs a clear account of how the analyzed trajectories were selected. Without that detail, it is hard to assess possible selection effects. 2.Some implementation and evaluation details need to be reconciled. Table 2 and Figure 3 describe the short-horizon variant differently. The loss formula in the main text also differs from the DAPO aggregation described in the appendix. In addition, MMRareBench uses open-ended questions, while the main prompt relies on candidate diagnoses. The authors should provide the prompt used for this evaluation. These details matter for reproducibility and interpretation.
Questions for authors
I would appreciate clarification on the following points. 1.For Figure 9, how was each case’s analyzed trajectory selected from its rollout group? Did it have to produce a correct answer or have a positive advantage? 2.Do Table 2 and Figure 3 use the same short-horizon variant? Please provide the exact definition. Also, was the total correction magnitude matched in the long-horizon ablation? 3.Does the implementation average the loss within each trajectory first, or across all valid tokens? 4.What inference prompt was used for MMRareBench T1? How was the candidate-support readout handled when no candidate diagnoses were provided?
10 comments
Response to Reviewer xo1z (2/2)
Response to W2.1, Q2.1: Definition of the short-horizon variant
Thank you for pointing out this inconsistency. Table 2 and Figure 3 use the same short-horizon variant. In both cases, it uses the local transition sign as its credit direction, replacing the stage-level direction while retaining the same action-level weights. Its exact definition is . Full CCA and this variant therefore share the same action-level magnitude weights and differ only in whether the credit direction is given by the stage-level direction or the local transition sign . For clarification, we have made the following revisions in Section 4.3 of the revised manuscript:
-
Lines 428-429 (ablation study, Table 2): We correct the definition to "The short-horizon-only variant uses , retaining the action-level magnitude but replacing the stage-level direction with the local transition direction."
-
Line 468 (training dynamics, Figure 3): We clarify that this variant is the same as in Table 2: "Here, the short-horizon variant is the same as in Table 2, using each local transition sign as its credit direction."
All revisions are highlighted in blue in the revised manuscript in Section4.3 [Line 428–429].
Response to W2.2, Q3: Loss aggregation
We thank the reviewer for this important question. We double-check our training configuration and confirm that our implementation retains GRPO-style trajectory-level loss aggregation. Specifically, we first compute each token’s policy loss using its assigned advantage, including the CCA credit adjustment where applicable. We then average these token-level losses over all unmasked tokens within each trajectory, followed by averaging the resulting values across all trajectories in the generated group. The phrase “DAPO’s token-level aggregation” in Appendix D.1 is inaccurate. We have revised it to: “The policy loss follows the trajectory-level loss aggregation of GRPO alongside the Clip-Higher mechanism from DAPO …"
This revision is highlighted in blue in Appendix D.1, the main-text formula and appendix description are now consistent with the implementation.
Response to W2.3, Q4: MMRareBench T1 inference prompt
Thank you very much for pointing out this omission. The complete MMRareBench T1 inference prompts are now provided in Figures 16 and 17 in Appendix E.2. Specifically, given the input query and heterogeneous images, MedInSight first proposes four or five differential diagnoses and assigns each a fixed label (A–D or A–E). These labels stay fixed across the subsequent reasoning stages. The candidate-support readout is therefore reported over these self-generated candidates: after each inspection and stage reflection, the model reports an updated confidence score for each differential diagnosis option, and the final answer is the best-supported diagnosis.
To quantify the limitation of a fixed candidate set, we further measure initial candidate coverage on MMRareBench T1. The ground-truth diagnosis appears among the initial candidates in 92.62% of cases. Manual analysis of the remaining 7.38% of cases suggests that these candidate omissions primarily reflect gaps in the model’s knowledge of extremely rare diseases.
Our motivation for including MMRareBench is to evaluate whether our method can generalize beyond multiple-choice VQA to free-text diagnosis without externally provided answer options. We also add this motivation and the corresponding analysis to Section 4.2 [Line 374–377]. All revisions are highlighted in blue in the revised manuscript.
Thank you for your valuable comment again, we hope our responses can address your concern and we look forward to your further reply!
Thanks for the clarifications and the extra experiments. The new results have nudged me toward acceptance. I’d now put my assessment at around 2.5/4, though I’m still on the fence. Two points still give me pause: whether the correction magnitudes are matched, and how trajectories were selected for Figure 9. Clearing those up would help me settle on a final recommendation.
Response to Reviewer xo1z (1/2)
We sincerely thank the reviewer for the detailed and insightful feedback. The comments have been invaluable in helping us clarify our contributions and strengthen our paper. We have conducted several new experiments and revised the manuscript to address all the points raised. Below, we provide a point-by-point response.
Response to W1.1, Q2.2: Correction magnitude in the long-horizon ablation
We thank the reviewer for raising the question about the long-horizon variant. Yes, the total correction magnitude is matched. We acknowledge that the equation in line 426 omits the normalization factor used in our experiments. Actually, in the long-horizon-only ablation, we also normalize the action-level magnitude over the trajectory. We correct its definition to κt=∣T∣dk(t),g(t)L, where ∣T∣ is the number of visual inspection actions in the trajectory defined in Section 3.3. Compared with full Cross-horizon Credit Assignment (CCA), the long-horizon-only ablation replaces the normalized action-specific weights wtS with uniform weights 1/∣T∣, while retaining the same stage-level direction. Since both sets of weights sum to one over the trajectory, the two variants distribute the same total correction budget across actions, so their total correction magnitudes are matched. We correct Line 426 to "The long-horizon-only variant uses κt=dk(t),g(t)L/∣T∣, retaining stage-level direction with a uniform action-level magnitude." The revision is highlighted in blue in the revised manuscript in Section4.3 [Line 427–428].
Response to W1.2: Sample size of the Shuffled CCA
We thank the reviewer for this constructive suggestion. We agree that a larger evaluation set is necessary for a reliable comparison, and we expand the evaluation accordingly. Specifically, we collect all test cases containing exactly four images from four of our evaluation benchmarks, yielding 407 cases in total (72 from MedThinkVQA, 256 from MedFrameQA, 22 from MedXQA-Multi, and 57 from our held-out test set Dtest), denoted as Dcollect. The diagnostic accuracy (%) of the three variants is reported below.
On the 407-case Dcollect, full CCA achieves 55.53% accuracy, outperforming Shuffled CCA (47.42%) and vanilla GRPO (42.75%) by 8.11 and 12.78 percentage points, respectively. Compared with Shuffled CCA, this corresponds to 33 additional correctly answered cases, rather than the 3-case difference on the original 57-case subset. Since Shuffled CCA preserves the total correction magnitude and stage-level direction while permuting its assignment across actions, this larger-scale comparison further supports that the gains of CCA come from assigning credit to the appropriate inspection actions, rather than from the additional shaping signal itself.
We have added these results to Appendix B.4 in the revised manuscript.
Response to W1.3, Q1: Trajectory selection for Figure 9
We thank the reviewer for this thoughtful question. For Figure 9, we analyze trajectories that reach the correct diagnosis and examine how often their visual inspection actions provide evidence that helps distinguish candidate diagnoses. The rationale for restricting this evaluation to correctly answered cases is to directly verify whether MedInSight genuinely localizes diagnostically decisive regions when arriving at a correct diagnosis. We have clarified this selection strategy and its rationale in Appendix B.4 of the revised manuscript.