- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents BEAR-Med, a benchmark for medical image forgery detection, localization, and evidence-based reasoning. The authors construct a dataset of one million images covering four medical imaging categories and five forgery types. They train a Qwen3-VL model using four frozen evidence experts, staged supervised fine-tuning, and GRPO-based optimization. While the model achieves strong in-distribution results, its localization performance drops substantially on the external MedForge-90K benchmark.
Critical strengths
1.The paper addresses a useful problem by combining forgery detection, localization, and evidence-based explanations in one framework. 2.BEAR-Med-1M is a substantial dataset covering multiple imaging modalities and forgery mechanisms. The authors also investigate whether editing affects regions outside the annotated boxes. 3.The experiments include comparisons with existing methods, ablations, robustness tests, and external evaluation. The failure analysis is also helpful. 4.The paper clearly states its limitations and does not claim that the results establish reliable deployment in real clinical settings.
Critical weaknesses
I have the following concerns: 1.The near-perfect in-distribution results may partly reflect artifacts specific to the forgery generation pipeline. The paper does not sufficiently establish generalization to unseen generators or editing methods.
2.External localization performance is weak. On MedForge-90K, Box [email protected] is only 0.083, and it remains low on the subset with contained edits. Annotation issues explain part of the gap but do not fully establish the model's generalization ability.
3.The reasoning evaluation relies mainly on LLM judges, without sufficient validation by medical experts. Fluent explanations do not necessarily reflect correct visual evidence, as some failure cases in the paper also demonstrate.
4.The contribution of expert evidence is not fully separated from privileged information used to generate training targets, including ground-truth labels and bounding boxes. More controlled ablations are needed.
5.The GRPO reward mainly encourages appropriate output length, format, correct verdicts, and box overlap. It does not directly measure the medical correctness or visual grounding of the reasoning, making the claims about reasoning improvement less convincing.
Questions for authors
see weakness
5 comments
Response to Reviewer n7fo (2/2)
W3. "The reasoning evaluation relies mainly on LLM judges, without sufficient validation by medical experts. Fluent explanations do not necessarily reflect correct visual evidence, as some failure cases in the paper also demonstrate."
To address the lack of medical-expert validation, we added physician assessment for this rebuttal. Three physicians with medical-image interpretation experience each rated 120 BEAR-Med rationales on selected images, with no overlapping images across physicians and one reader per image. They saw the image, output, and reference labels/boxes. Visual Evidence Grounding (VEG) assesses whether cues and locations match the image, and Forensic Correctness (FC) assesses whether they justify the conclusion. Table R2 summarizes this independent human assessment.
Table R2. Physician assessment of BEAR-Med rationales. Each dimension has 120 ratings on a 1-5 scale.
| Dimension | Mean | Ratings at least 4 |
|---|---|---|
| VEG | 3.68 | 67.5% |
| FC | 3.83 | 69.2% |
Because the reported failures show that correct predictions do not guarantee grounded explanations, we additionally tested whether reasoning context influences localization performance. We varied the rationale prefix while holding the checkpoint, images, instruction, and decoding fixed. Generic text approximately matched the original rationale's length. Table R3 reports F1 on 10,000 test images, [email protected] on all 4,000 local forgeries, and FBR on 5,000 authentic images.
Table R3. Effect of reasoning-prefix interventions.
| Condition | F1 | [email protected] | FBR |
|---|---|---|---|
| Natural generation | .9929 | .84700 | .0016 |
| Original-rationale replay | .9930 | .84600 | .0016 |
| Empty rationale | .9342 | .82275 | .0630 |
| Generic rationale | .9615 | .81550 | .0368 |
Generic substitution lowers [email protected] by 3.15 percentage points (paired-bootstrap 95% CI: 2.50-3.80) and increases false-box rates. By contrast, reusing the model's original rationale as a prefix yields performance close to natural generation, suggesting that the decline with generic text reflects changed rationale content rather than the act of supplying a prefix itself. Together, these evaluations support physician-assessed visual grounding and a functional role for reasoning context in localization.
W4. "The contribution of expert evidence is not fully separated from privileged information used to generate training targets, including ground-truth labels and bounding boxes. More controlled ablations are needed."
Both the full and no-expert CoT conditions retain the same private GT labels and local boxes. The comparison therefore does not give privileged annotations exclusively to the full model. Instead, the ablation removes expert maps and spatial statistics together with expert-dependent guidance, rationale routing, and weighting, as detailed in Table R4.
Table R4. Controls in the no-expert CoT ablation.
| Factor | No-expert condition |
|---|---|
| GT controls | Same labels and local boxes retained. |
| Clinical guidelines | Category-specific guidance retained; expert-dependent rules removed or rewritten. |
| Rationale length | Standard 50-120-word targets requested for all samples, replacing expert-dependent routing; lengths not individually matched. |
| Reasoning weight | Uniform 1.0 after verification, replacing expert-derived weighting. |
Because both variants retain the same GT labels and boxes, the .795-to-.839 [email protected] improvement in Table 6 supports the benefit of expert-guided supervision beyond shared access to privileged annotations.
W5. "The GRPO reward mainly encourages appropriate output length, format, correct verdicts, and box overlap. It does not directly measure the medical correctness or visual grounding of the reasoning, making the claims about reasoning improvement less convincing."
The two training stages have different roles: expert-guided SFT supplies reasoning-content supervision, while G-GRPO optimizes verifiable task and spatial decisions. As specified in Section 4.4, the rationale reward checks length and nonrepetition; it is not a medical-correctness score. Likewise, GT-box IoU rewards spatial alignment, not the truth of every textual claim.
The corresponding ablation in paper Table 6 shows that adding G-GRPO raises [email protected] from .830 to .839 and reduces FPR and FBR from .002 to .001, with F1 unchanged at .995. These results support its role in localization and authentic-image error control.
Reasoning quality is evaluated separately through W3's physician assessment. Accordingly, we attribute the measured localization and false-positive improvements to G-GRPO, while the physician assessment supports the final model's rationale quality rather than an improvement attributable specifically to G-GRPO.
We believe our response has addressed most of your concerns and we respectfully ask if you'recomfortable to re-consider the rating in light of our efforts and dedication
Followup on Rebuttal
Dear Reviewer n7fo,
Thank you for your time and your constructive feedback on our paper. As the rebuttal period will end soon, we would greatly appreciate it if you could revisit our responses and let us know whether any concerns or questions remain. We hope these answers address the reviewer's concerns, and we would be happy to answer any further questions.
Thank you again for your time.
Response to Reviewer n7fo (1/2)
Thank you for recognizing the value of joint detection, localization, and reasoning, the benchmark's coverage, and the spatial audits. We address your five concerns with unseen-generator expert evaluations, physician assessment of rationale quality, and reasoning-prefix interventions, alongside clarifications of the external localization results and the respective roles of expert-guided supervision and G-GRPO.
W1. "The near-perfect in-distribution results may partly reflect artifacts specific to the forgery generation pipeline. The paper does not sufficiently establish generalization to unseen generators or editing methods."
We additionally report leave-one-generator-out evaluation of the three trainable experts. Each fold excludes FLUX, SD3.5, or SDXL from both training and validation, retrains the experts, and selects checkpoints using the remaining generators. Testing covers the held-out generator's INS/REM/SYN images across all four imaging categories, against the same 100,008 authentic test images.
Table R1. Expert detection AUC on unseen generators. Mean is the equal-weight average across the three folds.
Table R1 supports transferable expert detection cues, particularly from image consistency and generation texture, beyond editors encountered during training.
W2. "External localization performance is weak. On MedForge-90K, Box [email protected] is only 0.083, and it remains low on the subset with contained edits. Annotation issues explain part of the gap but do not fully establish the model's generalization ability."
Strict localization remains difficult even with target-domain supervision. In paper Table 3, the official MedForge model's [email protected] increases from .116 on ALL BOX to .177 on CONTAINED. This improvement is consistent with our independent spatial-fidelity audit identifying meaningful edits outside the released boxes. Nevertheless, .177 remains low despite MedForge training supervision. The remaining difficulty can reflect both precise-localization challenges and residual uncertainty in the reference annotations.
Besides, CONTAINED screens for outside-box changes and it does not verify the accuracy of the box itself. Because the audit hides the GT rectangle, it cannot establish whether that rectangle tightly and correctly delineates the realized edit. Moreover, in our existing human assessment, the two reviewers still identified meaningful outside-box changes in an average of 49.0% of the 102 automatically CONTAINED pairs within the 240-pair MedForge sample (Supplementary Table S7). Although containment screening removes cases with detected outside-box changes, these human--automatic disagreements show that it reduces spatial-annotation inconsistencies rather than providing a strict annotation-quality filter.
We therefore place greater emphasis on CONTAINED as a diagnostic under reduced outside-box inconsistency, while interpreting absolute [email protected] as agreement with the released annotations. Per-sample verification of the full edit extent, followed by evaluation on reliably annotated samples, could further improve measured localization where annotation mismatch penalizes otherwise correct predictions. Such verification would help distinguish model errors from reference-box discrepancies. However, expert verification across MedForge-90K would require substantial annotation effort and is not feasible within our available time and resources.
Under paper Table 3's conditional localization protocol, BEAR-Med exceeds all six adapted baselines in both views. In particular, it achieves .150 on CONTAINED without MedForge training supervision, versus .080 for the strongest adapted baseline. This supports a comparative transfer advantage, while the audit explains why the absolute score should not be interpreted independently of annotation reliability.