- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies hallucinations in radiology report generation. The authors call the phenomenon where earlier sentences in an autoregressive model contaminate subsequent sentences partial-report drift, and raise a question: should detected errors be corrected during the generation process of the autoregressive model? The paper proposes a framework, Correct Before It Drifts, to repair detected hallucinations online. The framework first uses a Grounded Drift Detection module to parse each sentence into structured clinical assertions, verifies whether the assertions are supported by the image using a frozen BioViL-T encoder, and proposes repair candidates. Next, the CPL (Counterfactual Preference Learning) module offline constructs two matched branches — corrected and unchanged — continues both to completion with the same generator, labels which branch is better using the dominance rule of the GREEN evaluator, and trains a short-lookahead preference model based on Qwen2.5-0.5B. Finally, ORS (Online Repair Selection) applies corrections to the sentences that need repair. Experiments are conducted on two classic datasets and three common radiology report generators; the CREPE clinical error count decreases by about 13%–15%, outperforming four baselines.
Critical strengths
-
The paper's perspective is somewhat novel. It reframes hallucination handling from "post-hoc correction" into a decision problem of "online state correction," and clearly separates "what can be repaired" (a grounding question) from "whether it should be repaired" (a utility question). More importantly, the authors do not let "drift" remain a rhetorical concept: after excluding the repaired sentence itself, the gains on suffix text for early/middle/late repairs are +1.8/+0.9/+0.2 respectively, a result that quantifies the drift problem defined in the paper.
-
The counterfactual pair construction is cleanly designed. The procedure uses the same prefix, the same generator, and the same decoding configuration, with the only variable being whether the current repair is applied; the complete-report outcomes are then used as supervision for the short-lookahead selector. This grounds the idea of rewarding progress[1] in an actionable medical scenario, and samples with ties or error–coverage trade-offs are explicitly discarded, avoiding a hand-designed exchange rate between error count and completeness.
References:
[1] Setlur, Amrith, et al. "Rewarding progress: Scaling automated process verifiers for llm reasoning." International Conference on Learning Representations. Vol. 2025. 2025.
Critical weaknesses
Weaknesses:
-
The experiments lack key baseline comparisons, leaving practicality in doubt. The most related family of methods is diagnosis-prompted generation (e.g., PromptMRG[2]), where a dedicated classifier first predicts findings and the generator then conditions on both the image and the prompts; such methods have been shown to effectively improve clinical accuracy. However, all baselines in this paper are inference-time decoding methods (VCD/OPERA/VISTA/CCD), with no direct comparison against prompt-guided generation or against post-hoc verification-and-rewriting methods (e.g., FactCheXcker[3]). The two families of methods very likely repair the same set of residual errors — negation, location, and unsupported findings — so the actual incremental value of the paper is not demonstrated.
-
The repair coverage is narrow and misses the most clinically dangerous error type. The method can only flip polarity, remove unsupported assertions/modifiers, and modify anatomical locations; pure omissions (real findings never mentioned in the report) are explicitly out of scope, yet omissions are precisely the hallucinations with the highest clinical risk. Moreover, the grounding verifier itself is a bottleneck: location verification recall is only 74.6% and localization mIoU is only 0.48. When GDD abstains, the system degenerates to the original generator, and these "unreachable errors" are not quantified.
-
The applicable scenarios are relatively narrow. Table 3 of the paper shows that when the generator can be updated, the training-time method CURE alone already outperforms Base+Ours, and the residual gain of adding Ours on top is only about 0.1 CREPE points. In other words, the method's real niche is limited to scenarios where "the generator is a frozen black box" (e.g., third-party APIs that cannot be fine-tuned), while offline construction of the preference data requires tens of thousands of complete rollouts, which is not cheap.
References:
[2] Jin, Haibo, et al. "Promptmrg: Diagnosis-driven prompts for medical report generation." Proceedings of the AAAI conference on artificial intelligence. Vol. 38. No. 3. 2024.
[3] Heiman, Alice, et al. "Factchexcker: Mitigating measurement hallucinations in chest x-ray report generation models." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.
Questions for authors
Please see weaknesses.
2 comments
Response to Reviewer 24K5 (Part 2 of 2)
2. Repair coverage, omissions, and grounding limitations (W2)
We agree that pure omissions and localization errors can be clinically important, and that GDD recall is a real bottleneck.
Our contribution addresses a specific question: once an unsupported assertion has entered an autoregressive report, can correcting that state reduce errors in subsequent text? A pure omission has no existing assertion to repair; addressing it requires detecting and adding an unmentioned finding. We do not claim to solve that different task or diminish its clinical importance.
To measure the impact of imperfect GDD recall, we audit 300 randomly sampled MAIRA-2 test reports against their reference reports.
Table R2. Reference-based error coverage (300 reports).
| Error category | Identified errors | GDD proposals |
|---|---|---|
| Unsupported findings | 117 | 64 |
| Incorrect negation | 56 | 31 |
| Incorrect anatomical location | 53 | 24 |
| Pure omissions | 84 | 0 |
| Measurements, severity, and comparisons | 48 | 0 |
| Total | 358 | 119 |
Across the 226 errors in our three primary repair categories, GDD proposes repairs for 119 (52.7%); the other 107 are not reached. The remaining 132 errors receive no valid proposal in this sample. Thus, we explicitly quantify the errors for which our current implementation cannot intervene, rather than claiming complete coverage.
The reported 74.6% location-verification recall and 0.48 localization mIoU evaluate the grounding module, not end-to-end error coverage. Parsing and conservative repair-validity checks further limit proposal generation. When GDD abstains, the original report state is retained: this leaves the missed error unresolved but avoids forcing an unsupported new edit. ORS can reject a proposed repair but cannot retrieve errors GDD misses or independently verify image-level correctness.
Our operator can remove unsupported detachable modifiers but cannot synthesize corrected measurements, severity values, or temporal comparisons. We therefore claim an improvement within a deliberately restricted repair space, not comprehensive clinical error elimination.
3. Complementarity with generator adaptation and cost (W3)
We appreciate the reviewer's concern regarding applicability and computational cost.
Complementarity with generator adaptation. We acknowledge that CURE alone outperforms Base + Ours on MedGemma. This is not unexpected: generator adaptation improves overall reporting capability, whereas our method addresses a distinct failure mode—partial-report drift, where an incorrect intermediate assertion can influence subsequent generation. Our goal is to mitigate this recurring problem, including in generators that have already been adapted, rather than replace generator training.
The key comparison is therefore adaptation versus adaptation + Ours. As shown in Table 3, adding our method further reduces CREPE from 1.38 to 1.29 after CURE and from 1.54 to 1.43 after EditGRPO. These consistent additional gains indicate that generator adaptation does not fully eliminate opportunities for beneficial online state correction. Our approach is thus complementary to generator adaptation, rather than limited to frozen, unadapted generators.
Computational cost. We agree that offline preference construction is not free. CPL requires 50,100 initial reports and 27,284 matched continuations, but this is a one-time offline investment. The generated continuations are reused across preference-model training runs and are not regenerated for individual inference requests.
At inference time, the generator parameters remain unchanged, with measured latency increasing by only 1.22–1.28× (Table 1).
Overall, our contribution is a complementary mechanism for mitigating residual partial-report drift—even after generator adaptation—with a one-time offline training cost and moderate inference overhead.
Response to Reviewer 24K5 (Part 1 of 2)
We sincerely thank the reviewer for the thoughtful assessment and constructive suggestions. We particularly appreciate the concerns about comparisons with other method families, the limitations of GDD, and the practical value of online correction.
1. Incremental value beyond decoding baselines (W1)
We agree that inference-time decoding baselines alone are insufficient. Our existing Table 3 already examines generator-adaptation methods CURE and EditGRPO. To address the reviewer's main concern, we additionally evaluate two other families using the same MAIRA-2 generator and test split:
PromptMRG's original implementation involves a different encoder–decoder architecture, a disease-classification branch, and additional training components. Directly running it would therefore change both the generator and training procedure. Our matched-generator adaptation isolates the contribution of diagnosis guidance without these confounding differences.
We also appreciate the FactCheXcker example (Heiman et al., CVPR 2025). Its original scope primarily concerns quantitative measurement hallucinations, which our current repair operator cannot correct by generating new measurements. We therefore include a more directly matched post-hoc editing control and will clarify these relationships in Related Work.
Table R1. Comparisons across method families (MIMIC-CXR / MAIRA-2).
Both other families improve the Base generator, confirming their relevance. Nevertheless, Ours retains the strongest clinical factuality across all three metrics. The distinction is online intervention according to short-lookahead during generation : diagnosis prompts guide reporting before generation; post-hoc methods edit after completion; our method uses image-grounded proposals and ORS to make online decision according to short-lookahead, ensuring correcting the current autoregressive state will improve the complete report.