acceptodds
Under review as a conference paper at ICLR 2027

Correct Before It Drifts: Counterfactual-Guided Intervention to Mitigate Hallucinations in Radiology Report Generation

Abstract

Factual errors in radiology reports are usually evaluated after generation, yet they arise sequentially: once an autoregressive generator commits to an image-inconsistent statement, that partial report becomes context for all subsequent sentences. We study this intermediate failure mode as partial-report drift and ask a more targeted question: when an image-grounded correction is available, should it be applied now, before the remaining report is generated? We introduce Correct Before It Drifts, a three-stage intervention framework that does not update the report-generator parameters. First, Grounded Error Detection verifies newly generated clinical claims against the radiograph and proposes only minimal image-supported repairs. Second, Counterfactual Preference Learning constructs corrected and unchanged continuations from the same partial report and uses their complete-report outcomes as offline preference supervision for a lightweight shared model. Third, Online Repair Selection predicts this preference from only a short lookahead and selectively applies repairs using a validation-calibrated intervention threshold. Across MIMIC-CXR and IU-Xray, multiple report generators, and strong hallucination-mitigation baselines, the method consistently reduces report-level clinical errors and improves structured factuality while largely preserving report semantics. Process-level analyses further show that short lookahead predicts full-report preference better than local verification alone and that earlier repairs exert a larger effect on subsequent report content. These results support treating radiology hallucination mitigation as a timely decision over evolving reports rather than only a post-hoc correction problem.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
radiology report generation, hallucination mitigation, medical VLMs, counterfactual data collection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_24K5OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies hallucinations in radiology report generation. The authors call the phenomenon where earlier sentences in an autoregressive model contaminate subsequent sentences partial-report drift, and raise a question: should detected errors be corrected during the generation process of the autoregressive model? The paper proposes a framework, Correct Before It Drifts, to repair detected hallucinations online. The framework first uses a Grounded Drift Detection module to parse each sentence into structured clinical assertions, verifies whether the assertions are supported by the image using a frozen BioViL-T encoder, and proposes repair candidates. Next, the CPL (Counterfactual Preference Learning) module offline constructs two matched branches — corrected and unchanged — continues both to completion with the same generator, labels which branch is better using the dominance rule of the GREEN evaluator, and trains a short-lookahead preference model based on Qwen2.5-0.5B. Finally, ORS (Online Repair Selection) applies corrections to the sentences that need repair. Experiments are conducted on two classic datasets and three common radiology report generators; the CREPE clinical error count decreases by about 13%–15%, outperforming four baselines.

Critical strengths
  1. The paper's perspective is somewhat novel. It reframes hallucination handling from "post-hoc correction" into a decision problem of "online state correction," and clearly separates "what can be repaired" (a grounding question) from "whether it should be repaired" (a utility question). More importantly, the authors do not let "drift" remain a rhetorical concept: after excluding the repaired sentence itself, the gains on suffix text for early/middle/late repairs are +1.8/+0.9/+0.2 respectively, a result that quantifies the drift problem defined in the paper.

  2. The counterfactual pair construction is cleanly designed. The procedure uses the same prefix, the same generator, and the same decoding configuration, with the only variable being whether the current repair is applied; the complete-report outcomes are then used as supervision for the short-lookahead selector. This grounds the idea of rewarding progress[1] in an actionable medical scenario, and samples with ties or error–coverage trade-offs are explicitly discarded, avoiding a hand-designed exchange rate between error count and completeness.

References:

[1] Setlur, Amrith, et al. "Rewarding progress: Scaling automated process verifiers for llm reasoning." International Conference on Learning Representations. Vol. 2025. 2025.

Critical weaknesses

Weaknesses:

  1. The experiments lack key baseline comparisons, leaving practicality in doubt. The most related family of methods is diagnosis-prompted generation (e.g., PromptMRG[2]), where a dedicated classifier first predicts findings and the generator then conditions on both the image and the prompts; such methods have been shown to effectively improve clinical accuracy. However, all baselines in this paper are inference-time decoding methods (VCD/OPERA/VISTA/CCD), with no direct comparison against prompt-guided generation or against post-hoc verification-and-rewriting methods (e.g., FactCheXcker[3]). The two families of methods very likely repair the same set of residual errors — negation, location, and unsupported findings — so the actual incremental value of the paper is not demonstrated.

  2. The repair coverage is narrow and misses the most clinically dangerous error type. The method can only flip polarity, remove unsupported assertions/modifiers, and modify anatomical locations; pure omissions (real findings never mentioned in the report) are explicitly out of scope, yet omissions are precisely the hallucinations with the highest clinical risk. Moreover, the grounding verifier itself is a bottleneck: location verification recall is only 74.6% and localization mIoU is only 0.48. When GDD abstains, the system degenerates to the original generator, and these "unreachable errors" are not quantified.

  3. The applicable scenarios are relatively narrow. Table 3 of the paper shows that when the generator can be updated, the training-time method CURE alone already outperforms Base+Ours, and the residual gain of adding Ours on top is only about 0.1 CREPE points. In other words, the method's real niche is limited to scenarios where "the generator is a frozen black box" (e.g., third-party APIs that cannot be fine-tuned), while offline construction of the preference data requires tens of thousands of complete rollouts, which is not cheap.

References:

[2] Jin, Haibo, et al. "Promptmrg: Diagnosis-driven prompts for medical report generation." Proceedings of the AAAI conference on artificial intelligence. Vol. 38. No. 3. 2024.

[3] Heiman, Alice, et al. "Factchexcker: Mitigating measurement hallucinations in chest x-ray report generation models." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

Questions for authors

Please see weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 24K5 (Part 1 of 2)

We sincerely thank the reviewer for the thoughtful assessment and constructive suggestions. We particularly appreciate the concerns about comparisons with other method families, the limitations of GDD, and the practical value of online correction.

1. Incremental value beyond decoding baselines (W1)

We agree that inference-time decoding baselines alone are insufficient. Our existing Table 3 already examines generator-adaptation methods CURE and EditGRPO. To address the reviewer's main concern, we additionally evaluate two other families using the same MAIRA-2 generator and test split:

  • Diagnosis-guided generation: Following PromptMRG's central idea, an independent image classifier predicts clinical findings and supplies them as additional context before report generation. This is a matched-generator diagnosis-guided baseline, not a full reproduction of PromptMRG's architecture.

PromptMRG's original implementation involves a different encoder–decoder architecture, a disease-classification branch, and additional training components. Directly running it would therefore change both the generator and training procedure. Our matched-generator adaptation isolates the contribution of diagnosis guidance without these confounding differences.

  • Post-hoc verification and correction: Inspired by the fact-checking and LLM correction stage in Phrase-grounded APO (Mahmood and Syeda-Mahmood, CVPR 2026), GDD checks the completed report and Llama-3.2-3B performs constrained sentence edits without regenerating the suffix. This tests its static correction strategy, not the full APO optimization procedure.

We also appreciate the FactCheXcker example (Heiman et al., CVPR 2025). Its original scope primarily concerns quantitative measurement hallucinations, which our current repair operator cannot correct by generating new measurements. We therefore include a more directly matched post-hoc editing control and will clarify these relationships in Related Work.

  • We also include Post-hoc GDD, which applies the identical GDD repair operator after completion, without an editing LLM.

Table R1. Comparisons across method families (MIMIC-CXR / MAIRA-2).

MethodCREPE ↓RaTEScore ↑RadGraph F1 ↑
Base1.4658.434.6
Diagnosis-guided generation1.3959.235.1
Post-hoc GDD1.4358.634.9
Post-hoc correction (CVPR'26-inspired)1.4159.035.0
Ours1.2661.436.2

Both other families improve the Base generator, confirming their relevance. Nevertheless, Ours retains the strongest clinical factuality across all three metrics. The distinction is online intervention according to short-lookahead during generation : diagnosis prompts guide reporting before generation; post-hoc methods edit after completion; our method uses image-grounded proposals and ORS to make online decision according to short-lookahead, ensuring correcting the current autoregressive state will improve the complete report.

AuthorsOpenReview ↗

Response to Reviewer 24K5 (Part 2 of 2)

2. Repair coverage, omissions, and grounding limitations (W2)

We agree that pure omissions and localization errors can be clinically important, and that GDD recall is a real bottleneck.

Our contribution addresses a specific question: once an unsupported assertion has entered an autoregressive report, can correcting that state reduce errors in subsequent text? A pure omission has no existing assertion to repair; addressing it requires detecting and adding an unmentioned finding. We do not claim to solve that different task or diminish its clinical importance.

To measure the impact of imperfect GDD recall, we audit 300 randomly sampled MAIRA-2 test reports against their reference reports.

Table R2. Reference-based error coverage (300 reports).

Error categoryIdentified errorsGDD proposals
Unsupported findings11764
Incorrect negation5631
Incorrect anatomical location5324
Pure omissions840
Measurements, severity, and comparisons480
Total358119

Across the 226 errors in our three primary repair categories, GDD proposes repairs for 119 (52.7%); the other 107 are not reached. The remaining 132 errors receive no valid proposal in this sample. Thus, we explicitly quantify the errors for which our current implementation cannot intervene, rather than claiming complete coverage.

The reported 74.6% location-verification recall and 0.48 localization mIoU evaluate the grounding module, not end-to-end error coverage. Parsing and conservative repair-validity checks further limit proposal generation. When GDD abstains, the original report state is retained: this leaves the missed error unresolved but avoids forcing an unsupported new edit. ORS can reject a proposed repair but cannot retrieve errors GDD misses or independently verify image-level correctness.

Our operator can remove unsupported detachable modifiers but cannot synthesize corrected measurements, severity values, or temporal comparisons. We therefore claim an improvement within a deliberately restricted repair space, not comprehensive clinical error elimination.

3. Complementarity with generator adaptation and cost (W3)

We appreciate the reviewer's concern regarding applicability and computational cost.

Complementarity with generator adaptation. We acknowledge that CURE alone outperforms Base + Ours on MedGemma. This is not unexpected: generator adaptation improves overall reporting capability, whereas our method addresses a distinct failure mode—partial-report drift, where an incorrect intermediate assertion can influence subsequent generation. Our goal is to mitigate this recurring problem, including in generators that have already been adapted, rather than replace generator training.

The key comparison is therefore adaptation versus adaptation + Ours. As shown in Table 3, adding our method further reduces CREPE from 1.38 to 1.29 after CURE and from 1.54 to 1.43 after EditGRPO. These consistent additional gains indicate that generator adaptation does not fully eliminate opportunities for beneficial online state correction. Our approach is thus complementary to generator adaptation, rather than limited to frozen, unadapted generators.

Computational cost. We agree that offline preference construction is not free. CPL requires 50,100 initial reports and 27,284 matched continuations, but this is a one-time offline investment. The generated continuations are reused across preference-model training runs and are not regenerated for individual inference requests.

At inference time, the generator parameters remain unchanged, with measured latency increasing by only 1.22–1.28× (Table 1).

Overall, our contribution is a complementary mechanism for mitigating residual partial-report drift—even after generator adaptation—with a one-time offline training cost and moderate inference overhead.

Reviewer_rACvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an online framework to mitigate hallucination propagation in radiology report generation. It combines image-grounded repair proposals, preference learning from corrected versus unchanged complete reports, and selective intervention using short lookahead. The key contribution is separating repair validity from downstream utility, using offline full-report preferences to guide online corrections while keeping the report generator frozen.

Critical strengths
  1. Clear and relevant problem formulation. The distinction between identifying a valid correction and deciding whether to apply it during generation is well motivated. The separation of grounding, preference learning, and online selection makes the framework easy to follow.

  2. Comprehensive evaluation. Comparisons use matched generators and include cross-dataset transfer, leave-one-generator-out evaluation, and integration with generator-adaptation methods. The reported improvements across these settings support the practical usefulness of the approach.

Critical weaknesses
  1. Calibration on the deployment distribution is insufficiently established. Preference construction excludes 4,928 of 13,642 candidate pairs because they exhibit ties or error–coverage trade-offs. Threshold calibration is described on held-out preference pairs, whereas online selection encounters proposals without this filtering. Consequently, precision on the retained subset does not necessarily characterize precision across all deployed interventions. The manuscript should clarify the evaluation population and treatment of ambiguous cases when computing Apply-P.

  2. The benefit of online correction over post-hoc correction is not isolated. The main comparisons evaluate alternative decoding methods, but do not include a matched post-hoc baseline using the same verifier and repair operator. Such a comparison is important because the paper’s principal claim concerns when correction should occur, rather than whether grounded correction improves reports. Without this control, the gains cannot be clearly attributed to intervention timing.

  3. The evidence for partial-report drift remains suggestive. Figure 3(c) shows larger suffix-only improvements for earlier interventions. However, the analysis does not establish whether remaining report length, error type, and intervention difficulty are controlled across positions. The result is consistent with drift mitigation but does not distinguish it from other consequences of modifying the prefix. A matched immediate-versus-delayed correction experiment would strengthen this claim.

Questions for authors
  1. Section 3.2 clearly specifies the GREEN-based dominance rule. Please confirm which error categories contribute to EE and whether the standard GREEN aggregation is used unchanged. More importantly, clarify whether threshold calibration and Apply-P evaluation include ties and error–coverage trade-offs, or are restricted to dominance-labeled pairs.

  2. In Figure 4(b), a supported anatomical correction is rejected because the continuations are nearly identical. How does the selector account for the local factual benefit of a correction when downstream content remains unchanged?

  3. The paper specifies paired bootstrapping and three training seeds, but the principal results are presented without uncertainty estimates. Can the authors provide confidence intervals for the main paired improvements and variability across seeds?

2 comments
AuthorsOpenReview ↗

Response to Reviewer rACv (Part 1 of 2)

We sincerely thank the reviewer for the constructive assessment. We address the concerns about calibration, correction timing, and partial-report drift below.

1. Calibration and GREEN labels (W1 & Q1)

Ambiguous pairs are only included in offline CPL training. validation calibration and test-time evaluation are online modules, therefore they have neither U/C labels nor ambiguous pairs.

Calibration uses the full distribution of valid GDD proposals, including ambiguous outcomes. Apply-P counts all executed repairs: only corrected-dominant outcomes are successes; unchanged-dominant outcomes, ties, and trade-offs remain in the denominator.

Table R1. Full-stream statistics (MIMIC-CXR / MAIRA-2).

StatisticValidationTest
Parser-identified discrepancy states1,0671,999
Valid GDD proposals503923
ORS-executed repairs224430
Corrected-dominant executed repairs202392
Intervention rate21.0%21.5%
Apply-P90.2%91.2%

The intervention rate uses all parser-identified discrepancy states, including GDD abstentions; Apply-P uses executed repairs. Thus, test-time intervention rate is 430/1,999 = 21.5%, whereas Apply-P is 392/430 = 91.2%. The Always apply ablation similarly gives 923/1,999 = 46.2%.

GREEN's EE uses its six original categories: false findings, omitted findings, incorrect locations, incorrect severity, unsupported comparisons, and omitted comparisons. We use GREEN error counts and matched findings unchanged; our dominance rule only determines pairwise preference.

We will clarify these populations and denominators in Sections 3.2–3.3 and Table 4.

2. Online versus post-hoc correction (W2)

We agree that a matched post-hoc baseline is needed. Post-hoc GDD applies grounded repairs after report completion; Online GDD applies the same verifier and repair operator during generation. Neither uses ORS.

Table R2. Correction strategies (MIMIC-CXR / MAIRA-2).

MethodCREPE ↓RaTEScore ↑RadGraph F1 ↑
Base1.4658.434.6
Online GDD (Always apply)1.4059.135.1
Post-hoc GDD1.4358.634.9
Ours1.2661.436.2

Online edits change subsequent generation and can prevent accumulation of errors, so better than post-hoc editing. Our full method additionally uses ORS to select repairs based on predicted complete-report benefit.

The reply of [3. Controlled evidence for partial-report drift (W3)] can also explain the importance of intervention timing

AuthorsOpenReview ↗

Response to Reviewer rACv (Part 2 of 2)

3. Controlled evidence for partial-report drift (W3)

We thank the reviewer for this important observation and apologize that Figure 3(c)'s experimental setup was not sufficiently clear.

Figure 3(c) already conducts the suggested matched immediate-versus-delayed experiment. For each detected error, we use the same image, original prefix, erroneous sentence, grounded repair, and decoding configuration. The only difference is when we repair that same sentence:

  • Immediate: Repair it before generating the next sentence.
  • Delay 1: Repair it after generating one additional sentence.
  • Delay 2: Repair it after generating two additional sentences.

Table R3. Effect of correction timing (95% CI).

TimingΔRaTEScore ↑ΔCREPE ↓ΔRadGraph F1 ↑
Immediate+1.8 [0.9, 2.7]−0.14 [−0.23, −0.05]+0.6 [0.1, 1.1]
Delay 1+0.9 [0.2, 1.6]−0.09 [−0.16, −0.01]+0.3 [−0.1, 0.7]
Delay 2+0.2 [−0.4, 0.8]−0.06 [−0.12, 0.01]+0.1 [−0.2, 0.4]

Since all three conditions ultimately apply the identical repair to the same sentence, their differences arise from suffix rather than the local correction or prefix. The decreasing gains with longer delays indicate that errors can propagate into later sentences before being corrected, and these effects may persist afterward.

We will clarify this protocol and relabel Figure 3(c) as Immediate / Delay 1 / Delay 2.

4. Local benefit when continuations are similar (Q2)

We thank the reviewer for raising this important question.

ORS considers both the local correction and its potential downstream effects. It evaluates the original and corrected sentences in their shared context, together with their H=2H=2 lookaheads. Thus, local factual improvements can influence the predicted preference even when the continuations are similar.

Our ablations help clarify their roles:

  • H=0 uses local information without lookahead. It already reduces CREPE from 1.46 to 1.39, but provides less information for identifying beneficial downstream interventions.
  • Full combines lookahead with a calibrated threshold, further reducing CREPE to 1.26.

In Figure 4(b), the anatomical correction is image-supported, and ORS assigns a small positive margin (m=0.08m=0.08). However, it falls below the calibrated threshold (γ∗=0.37\gamma^*=0.37), meaning only local factual benefit is not enough, so the repair is rejected.

It illustrates the trade-off of conservative selection: some beneficial edits are rejected to limit unreliable interventions, which overall obtains best clinical accuracy .

We will clarify this trade-off and the interpretation of KEEP in Section 4.7.

5. Uncertainty and training variability (Q3)

We use 1,000 paired study-level bootstrap resamples for 95% CIs.

Table R4. Main paired improvements on MIMIC-CXR (95% CI).

GeneratorΔCREPE ↓ΔRaTEScore ↑ΔRadGraph F1 ↑
CheXagent−0.27 [−0.35, −0.19]+3.5 [+2.7, +4.3]+1.8 [+1.2, +2.4]
LLaVA-Rad−0.27 [−0.34, −0.20]+3.8 [+3.1, +4.6]+1.9 [+1.3, +2.6]
MAIRA-2−0.20 [−0.26, −0.14]+3.0 [+2.4, +3.6]+1.6 [+1.1, +2.2]

For seeds 17, 23, and 42, we retrain short-lookahead preference model and complete online inference.

Table R5. Three-seed results (mean ± SD).

GeneratorCREPE ↓RaTEScore ↑RadGraph F1 ↑
CheXagent1.57 ± 0.0256.1 ± 0.2228.4 ± 0.13
LLaVA-Rad1.69 ± 0.0254.6 ± 0.2926.8 ± 0.16
MAIRA-21.261 ± 0.00861.39 ± 0.1636.20 ± 0.08

These results show consistent paired improvements and limited training-seed variability. We sincerely appreciate the suggestions, which have helped us clarify both the evidence and limitations of selective online correction.

Reviewer_eHgjOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an online intervention framework for mitigating hallucinations in RRG. It detects unsupported claims, proposes image-grounded corrections, and uses a learned preference model to decide whether applying a correction improves subsequent generation. The preference model is trained on matched corrected/unchanged report continuations, using complete-report outcomes as supervision.

Experiments on MIMIC-CXR and IU-Xray show improvements across multiple generators, with additional analyses of early intervention, lookahead length, transfer, and inference overhead.

Critical strengths
  1. Interesting problem formulation. Treating hallucination correction as an online decision rather than post-hoc editing. The distinction between a valid local repair and its downstream utility is particularly useful.

  2. The counterfactual preference learning setup is well motivated. Matched corrected/unchanged branches isolate the intervention decision, while complete-report supervision trains a short-lookahead scorer.

  3. Consistent improvements across three generators on MIMIC-CXR.

  4. Strong process-level analysis, which provides evidence for the proposed drift mechanism.

  5. Reasonable practical overhead. The gains also transfer to IU-Xray and remain positive when applied on top of CURE and EditGRPO.

Critical weaknesses
  1. The method depends heavily on the accuracy of the visual grounding module. Incorrect grounding can introduce new errors into the report, and the preference model cannot independently verify proposed edits.

  2. The repair space is restrictive. Pure omissions, unsupported measurements, severity, and temporal comparisons are not handled. These are clinically important hallucination types.

  3. The counterfactual preference labels depend on GREEN, which is itself an imperfect automatic evaluator. Potential supervision noise is not sufficiently investigated.

  4. No independent radiologist evaluation of the final reports or applied interventions. Although CREPE, RaTEScore, and RadGraph improve, it remains unclear whether the selected repairs consistently improve clinical correctness

  5. The drift analysis is suggestive but not fully conclusive. Earlier interventions naturally have longer remaining suffixes, potentially explaining their larger cumulative gains

Questions for authors
  1. How frequently does GDD introduce an incorrect repair, and what proportion of those incorrect proposals are accepted by ORS?

  2. How much of the improvement comes from preventing downstream drift versus simply correcting the current sentence? Could the authors report suffix-only improvements normalized by suffix length or number of subsequent clinical findings?

  3. How sensitive is CPL to errors in GREEN-derived preference labels? Have the authors considered training with alternative supervision from CREPE or RadGraph, or using agreement between multiple metrics?

  4. How does the method compare against a post-hoc correction baseline using the same GDD repair proposals and verification model? This would better isolate the advantage of correcting during generation rather than after report completion.

  5. Can the authors provide a small radiologist-reviewed evaluation of applied and rejected repairs, particularly for anatomical localization and negation errors?

2 comments
AuthorsOpenReview ↗

Response to Reviewer eHgj (Part 1 of 2)

We sincerely thank the reviewer for the constructive assessment and recognition of our problem formulation, counterfactual learning, and process-level analysis. We address grounding reliability and clinical scope here, followed by drift, GREEN labels, and post-hoc comparison in Part 2.

1. Incorrect GDD proposals and ORS acceptance (W1 & Q1)

[W1] We agree that GDD reliability is important: an incorrect grounded proposal may introduce a new error if executed. But importantly, GDD proposed edits (proposals) are not automatically executed: ORS evaluates both the candidate sentence and its subsequent continuation to predict complete-report benefit. Thus, it can reject problematic proposals even after they pass GDD, although it does not independently verify image-level correctness.

[Q1] To quantify this additional filtering, we compare GDD proposals with the existing radiologist-written reference reports and examine their ORS decisions.

Table R1. Reference-discordant GDD proposals and ORS decisions (MIMIC-CXR / MAIRA-2).

StatisticResult
GDD proposals923
Proposals with explicit reference information714
Reference-discordant proposals83/714 (11.6%)
Reference-discordant proposals accepted by ORS15/83 (18.1%)
Reference-discordant / executed repairs15/331 (4.5%)

Among 83 discordant proposals, 44 concern localization, 22 negation, and 17 finding assertions; ORS accepts 8, 4, and 3, respectively. Thus, ORS rejects most proposals identified as reference-discordant.

This is not a radiologist-confirmed image-level error rate. Reference reports may omit relevant details, and 209 proposals lack explicit reference information. Nevertheless, the results demonstrate that ORS provides a meaningful additional safeguard: most identified problematic proposals are rejected rather than propagated into the generated report. Incorrect grounding remains a limitation, but its consequences are mitigated by selective execution.

2. Repair scope and radiologist review (W2, W4 & Q5)

[W2] We agree that omissions, measurements, severity, and temporal comparisons are clinically important. Our focus is on a common failure mode across autoregressive report generators: partial-generation drift, where an incorrect assertion can influence subsequent generation. We formulate this as a testable online state-correction problem and propose selective intervention to mitigate its downstream effects, rather than attempting to exhaustively correct every error type.

Pure omissions lack an existing assertion to repair, while measurements, severity, and temporal comparisons require specialized evidence beyond our current grounding module. Correcting earlier errors may indirectly improve these aspects, but we do not claim comprehensive coverage. We acknowledge this limitation and appreciate the reviewer's suggestion.

[W4 & Q5] We agree that improvements in automatic metrics alone do not fully establish the clinical correctness of individual repairs. We also appreciate the reviewer's suggestion for independent radiologist evaluation, particularly for anatomical localization and negation errors.

Although we have not conducted additional radiologist evaluation, we assess both applied and rejected repairs against existing radiologist-written reference reports to provide more direct evidence of factual consistency.

Localization: APPLYNegation: KEEP
ReferenceLeft basilar linear opacity consistent with mild atelectasisSmall pleural effusion; no associated atelectasis
OriginalLinear opacity at the right lung base.No pleural effusion.
RepairLinear opacity at the left lung base.pleural effusion is present.
Unchanged continuationLikely right basilar atelectasis.No focal consolidation or atelectasis.
Corrected continuationLikely left basilar atelectasis.Moderate pleural effusion with compressive atelectasis.
DecisionAPPLYKEEP

Localization: Applying the image-supported repair corrects the anatomical side and prevents the same laterality error from appearing in the subsequent description of the finding.

Negation: Although the proposed correction accurately identifies an existing pleural effusion, the corrected continuation overstates its severity and introduces unsupported associated atelectasis. In such a case, retaining the original state can avoid greater downstream factual deterioration.

AuthorsOpenReview ↗

Response to Reviewer eHgj (Part 2 of 2)

3. Downstream benefit beyond local correction (W5 & Q2)

[W5] We thank the reviewer for raising the concern about remaining suffix length. We apologize that Figure 3(c)'s experimental design was not sufficiently clear.

Figure 3(c) already compares three correction timings for the exact same intervention position .The only difference is when we repair that same sentence : Immediate, Delay 1, and Delay 2.

  • Immediate: Repair it before generating the next sentence.
  • Delay 1: Repair it after generating one additional sentence.
  • Delay 2: Repair it after generating two additional sentences.

[Q2] Importantly, all three correction timings apply the identical repair to the same erroneous sentence. The local correction is therefore fixed, allowing us to examine how repair timing affects clinical improvement.

Table R2. Clinical improvements under matched correction timings (95% CI).

TimingΔRaTEScore ↑ΔCREPE ↓ΔRadGraph F1 ↑ΔCorrect findings ↑ΔIncorrect findings ↓
Immediate+1.8 [0.9, 2.7]−0.14 [−0.23, −0.05]+0.6 [0.1, 1.1]+0.16 [0.05, 0.27]−0.13 [−0.22, −0.03]
Delay 1+0.9 [0.2, 1.6]−0.09 [−0.16, −0.01]+0.3 [−0.1, 0.7]+0.09 [0.01, 0.17]−0.07 [−0.15, 0.01]
Delay 2+0.2 [−0.4, 0.8]−0.06 [−0.12, 0.01]+0.1 [−0.2, 0.4]+0.03 [−0.03, 0.09]−0.03 [−0.09, 0.03]

All values are paired improvements relative to the corresponding unchanged continuation. Finding counts represent average changes in subsequent clinical findings per intervention state.

Compared with Delay 2, Immediate correction achieves an additional +1.6 RaTEScore, −0.08 CREPE, and +0.5 RadGraph F1, together with 0.13 more correct and 0.10 fewer incorrect subsequent findings per intervention state.

Since the local repair is identical across conditions, these differences demonstrate the additional benefit of preventing downstream drift versus simply correcting the current sentence.

We will clarify the matched timing protocol and suffix evaluation in Figure 3(c).

4. GREEN label noise and alternative supervision (W3 & Q3)

We retrain CPL after randomly flipping different fractions of retained preference labels and evaluate the resulting online systems.

Table R3. Preference-label noise sensitivity (MAIRA-2).

Flipped labelsCREPE ↓RaTEScore ↑RadGraph F1 ↑
0%1.2661.436.2
10%1.3060.735.8
20%1.3559.835.3

Performance degrades gradually but remains better than Base (1.46 / 58.4 / 34.6). As reported in the appendix, GREEN preference orderings agree with CREPE, RaTEScore, and RadGraph on 78.6%, 74.9%, and 72.3% of held-out dominance-labeled pairs.

We retain GREEN's error-and-coverage dominance rule for pairwise training; CREPE and RadGraph provide complementary evaluation signals rather than the same two-dimensional preference criterion. Random flips and cross-metric agreement do not eliminate possible systematic GREEN bias, which we acknowledge.

5. Online versus post-hoc correction (Q4)

We agree that a matched post-hoc baseline is needed. Post-hoc GDD applies grounded repairs after report completion; Online GDD applies the same verifier and repair operator during generation. Neither uses ORS.

Table R2. Correction strategies (MIMIC-CXR / MAIRA-2).

MethodCREPE ↓RaTEScore ↑RadGraph F1 ↑
Base1.4658.434.6
Online GDD (Always apply)1.4059.135.1
Post-hoc GDD1.4358.634.9
Ours1.2661.436.2

Online edits change subsequent generation and can prevent accumulation of errors, so better than post-hoc editing. Our full method additionally uses ORS to select repairs based on predicted complete-report benefit.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.