- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper studies the mismatch between response-level preference optimization and fine-grained clinical correctness in medical LVLMs. It proposes FiRe-MPO, which constructs minimally edited preference pairs differing only in clinically decisive spans, applies token-level preference optimization, pairs them with lesion-corrupted images with reversed preferences, and introduces bidirectional token-wise KL regularization. Experiments on medical VQA and report generation report improvements over DPO and several fine-grained preference baselines.
Critical strengths
-
Well-motivated fine-grained textual supervision. The observation that clinical correctness may depend on only a few decisive spans is compelling, and minimally editing the model's own generations is a sensible way to reduce stylistic confounding between preferred and rejected responses.
-
Useful empirical analysis. The paper goes beyond downstream accuracy by analyzing reward localization and includes reasonably extensive ablations across two LVLMs and multiple medical tasks.
Critical weaknesses
-
The lesion-corruption mechanism confounds clinical evidence removal with strong artificial supervision signals. Corruption is systematically coupled with the reversed-preference branch, so the model can learn corruption present -> switch preference rather than reason about whether clinical evidence is present. More importantly, the corruption location is constructed from the question and ground-truth answer and then localized by MedSAM3. Thus, the corruption position itself provides GT-derived spatial supervision. The whole-image-noise fallback further makes corruption presence a persistent cue even when no localized region is available. The current corruption-type ablations do not control for either shortcut. Consequently, the reported grounding gains cannot be uniquely attributed to improved use of native clinical visual evidence.
-
The corrupted branch is not a valid clinical counterfactual. Adding Gaussian noise over evidence for makes that evidence unobservable; it does not make the opposite answer clinically correct. Nevertheless, FiRe-MPO explicitly reverses the preference to on the corrupted image. This creates a mismatch between the stated goal and the actual supervision . Together with the first issue, this substantially weakens the central visual-grounding claim. The remaining methodological novelty is also relatively incremental with respect to existing fine-grained preference optimization and image-conditioned DPO methods.
Questions for authors
Can the authors rule out corruption-presence and corruption-location shortcuts with artifact-matched controls? In particular, what happens when identical corruption (same type, magnitude, and area) is moved from the GT-relevant region to a random or anatomically matched incorrect/contralateral region? Does the prediction follow the clinical evidence or the corruption location?
Can the authors evaluate a setting where corruption is equally present across preference branches, such that detecting artificial noise cannot reveal which preference regime applies?
What is the justification for preferring after evidence for is obscured? Would an uncertainty/insufficient-evidence target be more semantically appropriate? Results addressing these questions could materially change my assessment.
2 comments
Response to Reviewer JCva (Part 2/2)
Evidence vs noise location (Q1). From the noise table in Part 1: moving identical noise from the relevant region to a random region lowers FiRe-MPO's reward for the correct answer by 0.08 instead of 0.31, and flips 5.0% instead of 12.8% of pairs. Its preferences follow the evidence, not the noise.
Whole-image fallback. FiRe-MPO has none: questions about modality or general knowledge without a localizable region keep the base weight, and no image is ever noised.
Counterfactual validity and abstention (W2, Q3). Our motivation in the first version of FiRe-MPO was to stop rewarding y+ when the model attends to unrelated regions, but we agree that noise makes the evidence unobservable rather than making y− correct, so the reversed target was not valid supervision. We also implemented the uncertainty targets you suggest, under matched data: an abstention response preferred on v′ gives 67.79 average and an abstention loss 68.42, both below no visual term (69.34) and FiRe-MPO (71.60). Both still train on corrupted images, so they keep the presence cue and are out of distribution relative to clean test images. In the revised method, y− is only ever dispreferred, on the clean image.
Grounding claim. Drawing a box around the queried region raises the base model's accuracy on the 257 localizable SLAKE questions from 38.13% to 48.64%. Without any box at test time, FiRe-MPO reaches 46.30%, recovering 78% of that gain, vs 56% with uniform weights and 44% for DPO: the behaviour elicited by the box carries over to test time. We have narrowed the grounding claim in the paper to what these results support.
Main results. With 3 seeds (Avg = mean of SLAKE overall, VQA-RAD overall and IU-Xray GREEN):
| Method | HuatuoGPT-Vision-7B | Qwen3-VL-4B |
|---|---|---|
| DPO | 64.61 | 63.02 |
| mDPO | 64.90 | 63.99 |
| MMedPO | 66.50 | 64.95 |
| RRPO | 68.22 ± 0.38 | 67.33 ± 0.36 |
| FiRe-MPO (old) | 69.75 ± 0.34 | 67.40 ± 0.31 |
| FiRe-MPO | 71.60 ± 0.28 | 69.43 ± 0.28 |
Novelty. Only the visual term changes in the revision; the rest of the method is as submitted. We see three contributions:
• Diagnosis. Response-level DPO places little reward on clinically decisive tokens (Table 1), and reference-as-preferred pairs cause stylistic reward hacking (Section 2.3). This motivates span-level supervision in the medical setting rather than assuming it.
• On-policy minimal-edit pairs. RRPO and Mask-DPO localize the loss, but their pairs are not minimal: Mask-DPO localizes on whether each fact stated by the model is true or false, and RRPO on whatever differs between y+ and y−. Our pairs match the ground truth to the model's own answer and change only the decisive span, whatever the style of the generation, which separates linguistic priors from image-provided evidence. The bidirectional KL also stops the model from collapsing to one behaviour (Figure 4).
• How visual evidence enters. mDPO and MMedPO place a target or margin on corrupted images, which opens the visual shortcut shown in Part 1, and comparing responses under different images means the partition function no longer cancels as in DPO. FiRe-MPO never trains on a corrupted image; visual evidence enters only through the reference-computed weight on a same-image comparison.
We are grateful for these questions; they materially improved the method, and we hope the controls above address the shortcut concerns.
Response to Reviewer JCva (Part 1/2)
Thank you for your thorough review. The presence shortcut in particular changed how we see visual preference objectives in general: we tested it, found that it holds not only for our submitted visual term but also for other methods that train on corrupted images, and sharpened the term so that no corrupted image is used in training. The foundational components of our method (the fine-grained objective, minimal-edit pairs, and bidirectional KL, Eq. 3) remain fully intact.
Presence shortcut (W1, Q2). We tested whether trained models actually use the "noise present → switch preference" cue, and extended the test to mDPO and MMedPO. On 257 SLAKE test questions with ground-truth masks, we apply identical Gaussian noise (same σ and area) either over the lesion or over a random region, and measure the change in each model's reward r = log πθ / πref for the preferred (y+) and dispreferred (y−) responses:
Our old version of visual component reverses its preference in 44.8% of pairs even when the noise lies outside the lesion, compared with 54.6% when the lesion is covered and 4.8% for DPO under random noise. This confirms your concern: reversal-based training learns a cue unrelated to where the clinical evidence is. mDPO and MMedPO show a related one: under random noise their y+ rewards fall by 0.86 and 0.98, versus 0.07 for DPO. The problem is therefore not specific to our original term, and any objective with a target on noised images invites it.
We therefore removed the noised-image branch. Before training, the frozen reference model scores y+ on the clean image v and on vbox, the same image with a box drawn around the queried region:
ρ=∣y+∣1∑t[logπref(yt+∣vbox,q,y<t+)−logπref(yt+∣v,q,y<t+)]
This gives a per-pair weight c~=exp(clamp(ρ/τ,0,1))∈[1,e], normalised to mean 1 (c) with temperature τ > 0, on the clean-image span-level loss:
L=−E[clogσ(∑irθ(v,q,yi+)−∑irθ(v,q,yi−))]+αLKL
where LKL is the bidirectional token-wise KL regularization term defined in Eq. 3.
Since every branch now sees the same clean image, there is no corruption whose presence could reveal which preference applies, which is the setting your Q2 asks for. Under random noise FiRe-MPO behaves like DPO (Δr(y+): −0.08 vs −0.07; flips: 5.0% vs 4.8%).
Location shortcut (W1, Q1). The region is still derived from the question and ground-truth answer, but FiRe-MPO uses it only once, before training, for the frozen reference to compute a scalar weight; the box never enters the policy's input during training or inference, so there is no spatial signal for the policy to copy. To check that the gain comes from clinically meaningful localization rather than reweighting itself, we recomputed the weights from wrong regions, with identical data and hyperparameters (HuatuoGPT):
Only weights from the correct clinical region give a substantial improvement (+2.26 over uniform); shuffled or wrong-region weights match or fall below uniform weighting.