- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MedChain-R1, a two-stage multimodal framework in which teacher models generate medical explanations conditioned on ground-truth (GT) bounding boxes, and these explanations are then used to supervise vision-language models on sequential medical image localization. The authors construct an explanation dataset spanning eight task types and ten imaging modalities. Experiments across multiple model scales show promising results on MedSG-Bench. The paper also analyzes the context dependence of the explanations and the additional benefits of GRPO.
Critical strengths
- The motivation is reasonable. Generating explanations conditioned on GT bounding boxes and using them to supervise a model could help the model learn interpretable visual grounding.
- The method is evaluated across multiple tasks and imaging modalities, providing evidence of its applicability to a range of settings.
Critical weaknesses
The paper does not provide a sufficiently detailed analysis of the data distribution, bounding-box distribution, or lesion distribution in the test set. Without these analyses, it is difficult to determine whether the reported gains reflect improved disease understanding or are driven by other factors.
The source and construction of the paired images, as well as the origin of the GT annotations, are not clearly described. For example, it is unclear whether two images come from different slices in the same examination of one patient, or from examinations acquired at different time points. The paper also does not explain how the authors ensure that the bounding boxes in paired images correspond to the same lesion. Without these details, the reliability and clinical relevance of the sequential localization task are difficult to assess; in the worst case, the task may not represent a well-defined clinical problem.
Related work, such as TumorChain (ICLR 2026), also constructs chain-of-thought data and uses it for model training. The paper does not sufficiently clarify its technical novelty relative to these approaches. At present, the contribution appears to rely primarily on constructing chain-of-thought data.
The experiments do not disentangle the benefits of explanation-based supervision from those of direct bounding-box optimization. It is therefore unclear whether the gains arise from interpretable supervision or simply from additional localization training. If the explanations do improve performance, further analysis is needed to determine whether the improvement comes from better lesion or object classification, or from more accurate box prediction for structures the model could already detect.
Questions for authors
Is FlemingVL-38B necessary for the proposed pipeline? How would the results change if another model were used? In the second stage, the model already has access to the original image and the GT box, while the first-stage description is only an optional input. Why not have the second-stage model generate the explanation directly?
Please provide the image sources and construction rules for each task. In particular, distinguish among true longitudinal pairs, cross-slice, cross-view, and cross-modality pairs, as well as cropped or synthetically constructed samples. Please also clarify how patient identity and correspondence are established, whether image registration is performed, and how target consistency across paired images is verified. These details are essential for assessing the level of clinical reasoning required by each task and the scope of the method’s generalization.
Please report the category distribution of the target regions, distinguishing pathological lesions, normal anatomical structures, image-change regions, and local correspondence regions. The paper should also report the main disease or lesion categories and their sample counts. This would help determine whether the reported gains reflect improved clinical abnormality recognition or more general visual correspondence and localization
7 comments
W3. Relationship to TumorChain
We agree that TumorChain is relevant and will discuss it explicitly. Both works construct CoT supervision, and TumorChain also includes localization; our distinction is therefore not simply “diagnosis versus grounding.”
TumorChain develops 3D CT tumour analysis across five digestive organs, with interleaved organ-level visual reasoning and report-derived reasoning annotations. Its CoT-construction process uses radiology and pathology text.
In contrast, MedChain generates image-conditioned, box-conditioned explanations of sequential grounding decisions across eight tasks and ten modalities. Its multimodal refiner directly inspects the image sequence rather than constructing explanations from clinical reports.
Our contribution is this task-specific supervision resource and its evaluation across model scales, reasoning interventions, and GRPO, not the general idea of training with CoT or a new optimization algorithm. We will revise the related-work discussion to make these distinctions explicit.
W4. Explanation supervision versus direct box optimization
To address this concern, we conducted an additional matched comparison, training box-only models on the same rows with the rationale removed while retaining the images, questions, boxes, and training settings. This comparison is distinct from the original fixed-checkpoint analysis.
The resulting effects are smaller and task-dependent. At 4B, explanation supervision improves registered difference, non-registered difference, object tracking, and cross-modal grounding by approximately +2.7, +1.4, +1.4, and +1.6 mIoU, respectively. These tasks also improve at 9B, whereas box-only supervision is competitive or better on multi-view, concept, and patch grounding. The overall differences are +0.61 at 4B and −0.48 at 9B.
Thus, the matched results support selective benefits, not a universal advantage of explanation supervision. The full task-wise comparison is provided in our response to Reviewer sfWq.
Target identification versus box precision. Our localization-error analysis distinguishes complete misses from box precision. On difference grounding, zero-overlap predictions remain below **3.5.
These patterns are consistent with changes in localization precision rather than uniformly improved target detection. However, nonzero overlap does not establish correct lesion classification, and conditional means can involve different subsets of examples. We therefore do not interpret this analysis as proof of improved disease understanding or faithful clinical reasoning.
Q1. Is FlemingVL-38B necessary?
Stage I supplies supplementary medical descriptions of modality, anatomy, and visible findings. Stage II re-examines the original images and uses these descriptions, where available, to construct task-specific comparisons. The intended division of labour is therefore medical description followed by multimodal reasoning refinement, rather than compensating for missing visual access.
FlemingVL-38B was selected through a qualitative comparison of six candidates on the same 100 examples under six prompt variants (Appendix A.1). This motivates its selection but does not establish that it is uniquely necessary.
We have not trained downstream models with alternative Stage-I teachers or performed a matched Stage-II-only comparison, so we cannot quantify those differences. The single-expert acceptance rate of **98.25.
We hope these responses address the reviewer's concerns. Please let us know if any questions or concerns remain; we would be happy to address them and provide further clarification during the remaining rebuttal period.
Could you elaborate on how this technically differs from distilling on sampled data, and highlight the key technical innovations?
We thank the reviewer for detailed and constructive comments. We agree that interpreting the gains requires clearer data provenance, target distributions, and controls separating explanation supervision from additional localization training. We address raised points through an additional test-set analysis and matched training comparisons.
W1 / Q3. Data, bounding-box, and target-category distributions
We evaluate on MedSG-Bench, which contains 9,630 questions covering 76 public datasets, ten modalities, and 114 clinical tasks. Additional test-set analysis finds that CT and MRI account for **51.5
The median box width and height are 50 and 49 pixels, and the median box-area fraction is **2.4
Selected annotated lesion categories include brain tumour (1,279 questions), breast findings (269), skin lesions (216), polyps (113), and stroke lesions (59). These are question counts, not counts of unique patients or lesions.
The target-group analysis below reports dCoT-SFT minus box-only SFT for the new matched 4B comparison, in mIoU percentage points:
The clearest improvement is therefore in image-change localization, while the pathological-lesion group shows no supported average improvement. We will narrow our interpretation accordingly: these results support task-specific visual comparison and localization benefits, not a demonstrated improvement in disease recognition. They also do not isolate target size from task type.
W2 / Q2. Image sources, construction, and ground truth
We inherit the images, image sets, and localization annotations from the MedSG resources; MedChain adds rationales rather than constructing new image pairs or ground-truth boxes. The benchmark construction distinguishes the following tasks:
These are the benchmark's construction rules, rather than a claim that every task tracks the same lesion within one patient.
Patient identity and correspondence. Our test-set audit identifies cross-slice separations of 1–21 slices and distinguishes same-case MRI contrasts from cross-case organ correspondence. Cross-case correspondence concerns the same anatomical concept, not the same lesion instance. Filename consistency checks support the released case/slice associations but do not independently verify patient identity or lesion-instance identity.
Ground truth and registration. Ground truth is inherited from source masks or boxes, or from construction coordinates for edited and cropped regions. We perform no additional registration. We will document source-specific alignment and pairing metadata rather than assume universal registration or lesion correspondence.
Clinical scope. Simulated difference pairs are not longitudinal examinations, so their results do not establish disease-progression assessment. We will add a task-to-source manifest specifying the construction, annotation source, and correspondence type, with unavailable metadata explicitly marked.