- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates falsifiable claim–region grounding in 3D CT and introduces the ReX-Verify benchmark and CEG-Verify framework. The benchmark constructs supported and unsupported medical claims through location modifications, finding-type substitutions, and cross-case pairing. The proposed framework combines image–text feature encoding, separate support and counter-evidence prediction heads, relation classification, and false-support suppression to reduce misleading spatial evidence.
Experimental results show that false-support supervision reduces the false-support rate (FSR) from 0.130 to 0.032. However, the model achieves very low support and counter-evidence localization IoUs (0.010 and 0.018, respectively), and most improvements arise from cross-case unsupported claims rather than the core fixed-region claim edits.
Critical strengths
1. Interesting Problem Formulation.
The paper identifies a meaningful limitation of existing medical visual grounding methods: a correctly localized finding may still be incorrectly presented as evidence supporting a false medical claim. By distinguishing supporting and counter-evidence regions, the work introduces an interesting perspective that extends beyond conventional positive-only grounding and global claim verification.
2. Systematic Evaluation and Error Analysis.
The paper introduces the ReX-Verify benchmark and evaluates false-support exposure through controlled claim modifications. The ablation studies and error decomposition provide useful insights into the effect of false-support supervision, particularly the reduction in FSR from 0.130 to 0.032. Although the overall effectiveness remains limited, these analyses help clarify the proposed mechanism and provide a useful foundation for future research.
Critical weaknesses
1. Questionable Dataset Validity and Potential Textual Shortcuts. The automatically constructed negative claims may not necessarily be medically false, as modified findings or locations could still correspond to genuine abnormalities elsewhere in the CT. Moreover, reusing the original lesion masks as counter-evidence does not guarantee that these regions actually falsify the modified claims. The strong text-only performance (AUC = 0.939 for finding edits) further suggests potential linguistic shortcuts. These issues undermine the reliability of the benchmark and raise concerns about whether it genuinely evaluates image-grounded medical claim verification.
2. Limited Technical Contribution and Insufficient Experimental Validation. The proposed framework mainly combines existing image–text encoders, dual prediction heads, and suppression losses, without demonstrating clear advantages over simpler alternatives. Although false-support supervision reduces FSR, it also substantially decreases the exposure of valid support maps (from 0.340 to 0.124). Furthermore, the extremely low support and counter-evidence IoUs (0.010 and 0.018) and weak image-conditioned discrimination (AUC = 0.561) indicate that the model has not convincingly achieved reliable evidence grounding. These limitations significantly weaken the claimed methodological contribution and constitute a major reason for my rejection recommendation.
Questions for authors
How do the authors ensure that automatically generated negative claims are genuinely unsupported by the corresponding CT scans? For example, a modified location or finding type may still correspond to another existing abnormality. Were these samples independently validated by radiologists, and how was potential label noise addressed?
What is the justification for directly reusing the original lesion mask as counter-evidence for modified claims? Since the presence of the original finding does not necessarily falsify the modified claim, how do the authors ensure that these masks represent genuine counter-evidence rather than merely regions inconsistent with the edited description?
What specific advantages does the proposed dual-head architecture provide over simpler single-head alternatives? Given that CEG-Verify does not consistently outperform Single+gate or independent dual-head models, can the authors provide stronger evidence that the proposed design offers meaningful benefits beyond output suppression?
How can the authors demonstrate that the reduction in FSR reflects improved image-grounded verification rather than simply suppressing mask outputs? In particular, can the authors explain the substantial decrease in valid support-map exposure (from 0.340 to 0.124), the extremely low localization IoUs, and the weak image-conditioned discrimination (AUC = 0.561)? Would the improvement remain under matched support-map coverage?