acceptodds
Under review as a conference paper at ICLR 2027

Reason Before You Generate: Latent Consistency Control for Radiology Report Generation

Abstract

Radiology report generation aims to automatically produce clinically coherent diagnostic reports from medical images. However, most existing RRG methods follow an open-loop autoregressive generation paradigm, where multimodal evidence is primarily used for token prediction, while the evolving report state is rarely re-evaluated against image-grounded semantics during decoding. To address this issue, we propose Latent Consistency Control (LCC), which establishes a shared cross-modal latent space and explicitly regulates semantic consistency throughout autoregressive generation. LCC first condenses visual and textual representations into compact semantic prototypes and performs iterative latent cross-modal reasoning to obtain a unified reasoning representation. Building on this latent space, we introduce a multi-step consistency-aware decoding mechanism that periodically evaluates the generated prefix against image-grounded semantics and conditionally injects a lightweight corrective signal when potential inconsistency is detected. In this way, LCC enables intermediate verification and corrective regulation rather than relying solely on sequence-level generation. Experiments on benchmark RRG datasets demonstrate consistent improvements in both natural language generation quality and clinical factual correctness, outperforming representative state-of-the-art methods on most metrics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Radiology Report Generation, Latent Consistency Control, Cross-modal Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_RjUbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets error accumulation in autoregressive radiology report generation and proposes LCC. Image and report representations are condensed into compact vectors and refined through a few steps of interaction in a shared space. During decoding, a consistency evaluator checks the generated prefix against the image at fixed intervals, and when the score falls below a threshold, a corrective vector is added to the embedding of the latest token. Generation, contrastive alignment, consistency discrimination, and correction losses are trained jointly. On IU X-Ray and MIMIC-CXR, the method achieves the best scores on most text metrics and clinical scores comparable to the strongest methods.

Critical strengths

1 Radiology report generation is highly sensitive to hallucinated findings because generated reports may contain clinically incorrect statements. The paper addresses a meaningful limitation of current autoregressive medical vision-language models.

2 The paper introduces a complete framework combining cross-modal semantic representations with generation-time consistency control.

Critical weaknesses

1 The central claim of detecting and correcting errors during decoding is not supported. Disabling only the corrective injection at inference lowers BLEU-4 on IU X-Ray by about one point, while removing the whole module lowers it by almost four, so most of the gain appears to come from the extra training losses rather than from decoding-time correction.

2 The experimental comparison is neither fair nor sufficient. Most baseline numbers are copied from the original papers, which use different backbones and settings. R2GenGPT, which has nearly the same architecture, is missing, and decoding-time hallucination mitigation and controlled decoding methods are neither compared nor discussed, which makes the novelty hard to judge. Clinical gains over the strongest methods are about half a point with no variance reported, and IU X-Ray has no clinical metrics.

Questions for authors

1 is "w/o MDCC" a retrained model or the same model with control disabled at inference, and how much do MIMIC-CXR clinical metrics change when only the injection is turned off?

2 Under the same backbone, how does LCC compare with R2GenGPT and with decoding-time hallucination mitigation methods such as VCD or OPERA, and can you add clinical metrics on IU X-Ray?

3 Most baseline numbers are taken from the original papers. Do all methods share the same data split, report section (findings only or findings plus impression), preprocessing, and label extractor? If not, can you rerun the strongest baselines (DART, MLRG, SAT-RRG) under your setting, given that LCC uses a 7B language model while many baselines use much smaller decoders?

4 Since the clinical gains over DART and MLRG are around half a point, can you report significance tests and confidence intervals for the main metrics?

5 comments
AuthorsOpenReview ↗

Response to Reviewer RjUb — Part 1 of 3

We thank the reviewer for the constructive feedback and recognition of the clinical importance of our work. In response, we have strengthened the experimental analysis and clarified the evaluation protocols.

Our responses address four concerns: (1) inference-time corrective control (W1, Q1), (2) backbone and decoding comparisons (W2, Q2), (3) baseline fairness (W2, Q3), and (4) statistical reliability (W2, Q4). The first two are addressed below.

1. Concerns Regarding the Contribution of Inference-Time Correction (W1, Q1)

We appreciate the reviewer's concern that the effects of MDCC training and inference-time correction should be distinguished. To clarify their respective contributions, we have distinguished the retrained ablation from inference-only intervention and added a same-model comparison in Section 4.3 (Table 5).

Specifically, w/o MDCC in Table 2 is separately retrained after removing the MDCC module and its associated training objectives. In contrast, No Injection in Table 5 retains the fully trained LCC model and disables corrective injection only during inference. Thus, the two experiments evaluate different contributions rather than an additive decomposition of performance gains.

To isolate inference-time correction, we compare No Injection, matched-rate Random Injection, and LCC Guided Injection under identical trained model weights and decoding settings within each dataset. As shown in Table 5, enabling consistency-guided injection improves clinical F1 from 0.505 to 0.539 and RadGraph F1 from 0.263 to 0.295. On IU X-Ray, BLEU-4 increases from 0.202 to 0.213. Moreover, LCC outperforms random injection at matched rates of 26.4% on MIMIC-CXR and 24.8% on IU X-Ray, demonstrating that consistency-guided intervention timing provides additional benefits beyond corrective injection alone. These results establish the independent contribution of inference-time corrective control, complementing the overall MDCC ablation in Table 2.

Table 5: Comparison of inference-time intervention strategies on MIMIC-CXR.

Inference StrategyInj. RateBLEU-4 ↑Clinical F1 ↑RG-F1 ↑
No Injection0.0%0.1460.5050.263
Random (Matched)26.4%0.1560.5190.274
LCC (Guided)26.4%0.1710.5390.295

2. Concerns Regarding Backbone and Decoding Comparisons (W2, Q2)

We appreciate the reviewer's request for comparisons with R2GenGPT, smaller language models, and decoding-time hallucination mitigation methods. To clarify these comparisons, we have expanded the backbone configurations and additional LCC evaluations in Appendix B.1 (Table S2).

First, R2GenGPT is included in Table 1 and Table S2. On MIMIC-CXR, LCC achieves BLEU-4 of 0.171 versus 0.121 and clinical F1 of 0.539 versus 0.445 for R2GenGPT. On IU X-Ray, BLEU-4 improves from 0.162 to 0.213. These results establish a comparison with an existing LLM-based RRG approach, while Table S2 makes the backbone configurations explicit.

Second, we additionally evaluate LCC with a 3B decoder. On MIMIC-CXR, LCC-3B achieves BLEU-4 of 0.168, clinical F1 of 0.539, and RadGraph F1 of 0.289, compared with 0.171, 0.539, and 0.295 for LCC-7B. These results demonstrate that the proposed framework remains effective with a smaller language model.

Regarding VCD and OPERA, we recognize the value of comparisons with general-purpose hallucination mitigation methods. VCD uses contrastive decoding based on visual perturbations, while OPERA regulates decoding through attention-based penalties and retrospection. In contrast, LCC periodically evaluates intermediate image–report consistency and conditionally injects learned corrective representations. The same-model comparison in Section 4.3 (Table 5) directly evaluates this mechanism, showing improvements over matched-rate random injection in clinical F1 (0.519 to 0.539) and RadGraph F1 (0.274 to 0.295). We consider systematic comparisons with VCD, OPERA, and related methods under unified medical generation settings an important direction for future work.

Regarding clinical evaluation on IU X-Ray, we appreciate the reviewer's suggestion to include clinical correctness metrics. IU X-Ray has a relatively small test set, making clinical metrics derived from automatically extracted findings more sensitive to individual cases and labeling variations. Moreover, the compared baselines in Table 1 do not report clinical metrics on IU X-Ray, but primarily adopt BLEU, METEOR, and ROUGE-L, which also motivates our use of these established metrics for consistent comparison. We therefore conduct comprehensive clinical evaluation on the substantially larger MIMIC-CXR dataset using CheXpert-based precision, recall, F1-score, and RadGraph F1. Furthermore, our additional controlled experiments in Section 4.3 (Tables 5 and 7) provide complementary evidence of improved clinical correctness and reduced error propagation during generation.

AuthorsOpenReview ↗

Response to Reviewer RjUb (Part 2 of 3)

We continue by addressing the experimental protocols and baseline comparisons.

3. Clarification of Experimental Protocols and Baseline Comparisons (W2, Q3)

We appreciate the reviewer's emphasis on transparent and reproducible comparisons across different RRG methods. To clarify the experimental settings and architectural differences, we have detailed the evaluation protocols in Appendix A.3 and expanded the backbone configurations and performance comparisons in Appendix B.1 (Table S2).

Specifically, LCC follows the conventional 7:1:2 split for IU X-Ray and the official MIMIC-CXR split, using the Findings section as the generation target. Clinical evaluation employs micro-averaged precision, recall, and F1 across 14 CheXpert observations, together with the official RadGraph F1 implementation. These settings are consistent with the clinical evaluation protocol reported by MLRG.

Regarding baseline reproduction, R2GenCMN and DCL are reproduced using their official implementations, while the remaining results are obtained from their original publications, as indicated in Table 1. For DART and SAT-RRG, publicly available official implementations suitable for direct reproduction could not be identified. Although MLRG provides an official implementation, it incorporates longitudinal information from historical examinations, whereas LCC uses current-study images. We therefore retain their published results and explicitly distinguish their backbone architectures and input configurations.

To provide a clearer comparison of model capacity, input settings, and performance, we have expanded Appendix B.1 (Table S2). Representative results on MIMIC-CXR are reproduced below.

As shown in Table S2, LCC (7B) and R2GenGPT share the same visual encoder and decoder backbone families, while LCC (3B) and SAT-RRG both employ Swin and LLaMA-3.2 3B. Compared with R2GenGPT, LCC (7B) improves BLEU-4 from 0.121 to 0.171 and clinical F1 from 0.445 to 0.539. Moreover, LCC (3B) achieves BLEU-4 of 0.168, clinical F1 of 0.539, and RadGraph F1 of 0.289, demonstrating competitive performance with a smaller decoder.

Importantly, these comparisons make the differences in model architectures and input settings explicit rather than assuming identical preprocessing or training configurations across published methods. The additional controlled experiment in Section 4.3 (Table 5) further evaluates the contribution of consistency-guided correction using identical trained model weights and decoding settings, thereby separating the effectiveness of the proposed mechanism from cross-model architectural differences.

Table S2 (Excerpt): Backbone configurations and performance comparisons on MIMIC-CXR. Long. indicates whether the model incorporates longitudinal information from prior examinations (Yes) or uses only current-study images (No).

ModelsEncoderDecoderLong.B-1B-4MTRRG-LPRF1RG-F1
R2GenGPTSwinLLaMA-2 7BNo0.3920.1210.1330.2750.4850.4120.445—
MLRGRAD-DINODistilGPT2Yes0.4110.1580.1760.3200.5490.4680.5050.291
DARTResNet-50TransformerNo0.4370.1370.1750.3100.5200.5460.533—
SAT-RRGSwinLLaMA-3.2 3BNo0.4280.1430.1670.303———0.205
LCC (3B)SwinLLaMA-3.2 3BNo0.4250.1680.1910.3290.5510.5280.5390.289
LCC (7B)SwinLLaMA-2 7BNo0.4320.1710.1880.3370.5450.5330.5390.295
AuthorsOpenReview ↗

Response to Reviewer RjUb — Part 3 of 3

4. Statistical Reliability of Clinical Improvements (W2, Q4)

We appreciate the reviewer's concern regarding the statistical reliability of the improvements over strong baselines, particularly DART and MLRG. We would like to clarify that all reported LCC results are averaged over three independent runs, rather than obtained from a single experiment. Due to space constraints, the main paper presents the mean performance without separately listing standard deviations. To provide additional transparency, we summarize the performance variability across three independent runs below.

As reported in Table 1, LCC achieves clinical F1 of 0.539, compared with 0.533 for DART, and RadGraph F1 of 0.295, compared with 0.291 for MLRG, representing absolute improvements of 0.006 and 0.004, respectively. The three-run evaluation is intended to characterize the reproducibility of our reported performance.

Regarding statistical significance and confidence intervals, DART and MLRG are compared using their published results, without access to the corresponding per-study predictions or comparable run-level statistics. Consequently, we report these differences as numerical improvements rather than making unsupported claims of statistical significance. We hope the additional three-run summary clarifies the experimental stability and interpretation of the reported improvements.

Table R1: Performance across three independent runs on MIMIC-CXR.

ModelsB-4 ↑MTR ↑RG-L ↑P ↑R ↑F1 ↑RG-F1 ↑
R2GenCMN0.101 ± 0.0030.131 ± 0.0040.268 ± 0.0030.321 ± 0.0040.262 ± 0.0030.289 ± 0.004—
DCL0.101 ± 0.0020.137 ± 0.0030.274 ± 0.0020.469 ± 0.0030.341 ± 0.0040.395 ± 0.003—
LCC (Ours)0.171 ± 0.0030.188 ± 0.0020.337 ± 0.0020.545 ± 0.0040.533 ± 0.0030.539 ± 0.0020.295 ± 0.002
Reviewer_i3xfOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Latent Consistency Control (LCC) to improve image–text consistency in radiology report generation. The method compresses visual and textual representations into semantic prototypes and performs iterative cross-modal reasoning to construct a shared latent representation. During autoregressive decoding, LCC periodically evaluates the consistency between the generated report prefix and image-grounded semantics. When potential inconsistency is detected, a gated corrective signal is introduced to guide subsequent generation. Experiments on IU X-Ray and MIMIC-CXR demonstrate improvements in language-generation and clinical evaluation metrics, with component ablations examining the proposed reasoning and decoding mechanisms.

Critical strengths
  1. A meaningful approach to consistency-aware decoding.

The paper introduces a concrete mechanism for connecting image–text consistency assessment with intermediate decoding control. By evaluating the current report prefix against visual evidence and cross-modal reasoning representations, LCC can conditionally introduce corrective information during generation. This is a meaningful design because it goes beyond conventional autoregressive decoding, where image information is used for token prediction without explicit intermediate consistency verification. The proposed mechanism provides a promising approach to mitigating error propagation in radiology report generation.

  1. Encouraging experimental results and supporting component ablations.

The reported results demonstrate improvements across both IU X-Ray and MIMIC-CXR. LCC achieves the best reported BLEU-2/3/4, METEOR, and ROUGE-L scores among the compared methods on both datasets, together with the highest clinical F1 and RadGraph F1 on MIMIC-CXR. The component ablations further support the contribution of consistency-aware decoding. In particular, compared with the variant without MDCC, the full model improves clinical F1 from 0.474 to 0.539 and RadGraph F1 from 0.245 to 0.295. Although some improvements over strong baselines are relatively modest, these results provide encouraging evidence for the effectiveness of the proposed framework.

Critical weaknesses
  1. Insufficient validation of the consistency signal and its relationship to actual clinical errors.

The consistency evaluator is trained using prefixes from paired reports as positive examples and prefixes from other reports in the same batch as negative examples. However, image–report pairing does not necessarily correspond to clinical consistency. For example, a common normal description from another report may remain compatible with the current image, while a generated prefix containing a localized clinical error may still exhibit high global semantic similarity. Although the controlled perturbation experiments involving negation, findings, and uncertainty provide useful preliminary evidence, they do not sufficiently establish the evaluator's reliability on actual model-generated prefixes. Since the consistency score directly determines when corrective intervention occurs, this limitation affects confidence in the central control mechanism. An evaluation of false-trigger and missed-error rates on generated prefixes would provide stronger evidence that the evaluator identifies clinically meaningful inconsistencies.

  1. Incomplete isolation of the benefits of consistency-based corrective control.

The selective-control ablation does not adequately distinguish the effects of intervention timing from intervention frequency. The variant without verification applies correction at every scheduled checkpoint, whereas the full model intervenes only when the consistency score falls below a threshold. Consequently, the comparison changes both the number and locations of interventions. Although the results favor selective control, they do not establish whether the gains arise from identifying more appropriate intervention points or simply reducing unnecessary corrections. A random-trigger or uniformly spaced intervention baseline matched for the average number of interventions would provide a more direct test of the proposed consistency-based control mechanism. Furthermore, the construction of inconsistent prefixes for controller training is insufficiently specified in the available description, making it difficult to assess the correspondence between training conditions and errors encountered during generation.

Questions for authors

Q1. Reliability of the consistency evaluator: How reliably does the consistency evaluator identify actual clinical errors in model-generated prefixes? Can the authors report false-trigger and missed-error rates at the selected threshold, particularly for negation errors, incorrect findings, and anatomical inconsistencies? Such results would help establish whether the learned consistency score is sufficiently informative to guide corrective interventions.

Q2. Intervention timing versus frequency: Can the authors compare consistency-based triggering with random or uniformly spaced interventions under a matched average intervention count? Reporting clinical F1 and RadGraph F1 alongside language-generation metrics would help clarify whether the observed gains result specifically from consistency-based intervention timing rather than fewer interventions.

Q3. Construction of inconsistent training prefixes: How are inconsistent prefixes constructed when training the corrective controller? Are they obtained from cross-report negatives, localized perturbations of ground-truth reports, or another corruption procedure? How are these prefixes aligned with their target continuations? A precise description or pseudocode would help establish the reproducibility of the training procedure and clarify the types of errors the controller is expected to correct.

5 comments
AuthorsOpenReview ↗

Response to Reviewer i3xf (Part 1 of 2)

We thank the reviewer for the constructive feedback and recognition of our consistency-aware decoding framework. In response, we have strengthened the experimental evidence and clarified the training procedure. Our responses cover three concerns: (1) consistency evaluation reliability (W1, Q1), (2) intervention timing versus frequency (W2, Q2), and (3) inconsistent prefix construction (Q3). The first is addressed below, followed by the remaining concerns in Part 2.

1. Concerns Regarding the Reliability of Consistency Evaluation (W1, Q1)

We appreciate the reviewer's observation that cross-report negatives do not necessarily represent clinical contradictions and that global semantic agreement may overlook localized errors. To evaluate the reliability of consistency discrimination, we have added a controlled prefix-level evaluation in Section 4.3 (Table 6), complemented by a paired decoding analysis during actual generation (Table 7).

Specifically, we select 400 MIMIC-CXR test cases and construct two paired prefixes for each case: a clinically valid but incomplete prefix extracted from the reference report and a contradictory counterpart created by modifying a supported finding's presence, anatomical location, or status. Each pair shares the same image and has comparable prefix lengths. We retain the trained consistency evaluator and the original threshold of 0.6 without retraining.

As shown in Table 6, correct-but-incomplete prefixes achieve higher consistency scores (0.78 vs. 0.37) and substantially lower trigger rates (11.8% vs. 82.5%) than contradictory prefixes. These results demonstrate that the evaluator distinguishes incomplete descriptions from explicit factual contradictions under controlled evaluation, including discrepancies in finding presence, anatomical location, and clinical status.

Table 6: Consistency evaluation across different prefix categories on MIMIC-CXR.

Prefix CategoryN1N_1Mean sconss_{\mathrm{cons}}Trigger Rate
Correct but Incomplete4000.7811.8%
Contradictory Finding4000.3782.5%

Regarding the requested false-trigger and missed-error rates on naturally generated prefixes, MIMIC-CXR does not provide verified checkpoint-level clinical error annotations for intermediate report states. Consequently, these rates cannot be directly computed from the available annotations. We therefore use the explicitly defined consistency categories in Table 6 and additionally evaluate the effectiveness of corrective intervention during actual generation.

Specifically, we have added a paired decoding experiment in Section 4.3 (Table 7) using 3,276 consistency-triggered checkpoints from MIMIC-CXR. At each checkpoint, we compare continuations with and without corrective injection from identical prefixes under the same trained model and decoding settings. We employ GREEN to identify clinical errors and calculate New Error Rate and Error Persistence, as described in Appendix A.2.

As shown in Table 7, corrective injection reduces New Error Rate from 24.6% to 17.0% and Error Persistence from 35.2% to 24.5%, corresponding to absolute reductions of 7.6 and 10.7 percentage points, respectively. Together, Table 6 evaluates the ability to distinguish controlled clinical inconsistencies, while Table 7 provides process-level evidence that consistency-triggered corrective control reduces clinical error propagation during actual generation.

Table 7: Effectiveness of corrective control on clinical error propagation in MIMIC-CXR.

Inference StrategyN2N_2New Error Rate ↓Error Persistence ↓
Without Injection3,27624.6%35.2%
With Injection (LCC)3,27617.0%24.5%
AuthorsOpenReview ↗

Response to Reviewer i3xf (Part 2 of 2)

We continue by addressing intervention timing versus frequency and the construction of inconsistent prefixes for corrective controller training.

2. Concerns Regarding Intervention Timing versus Frequency (W2, Q2)

We agree with the reviewer that the effect of consistency-guided intervention should be distinguished from intervention frequency. To isolate these factors, we have added a matched-rate inference comparison in Section 4.3 (Table 5), evaluating different intervention strategies using the same trained LCC model and identical decoding configurations within each dataset.

Specifically, we compare three strategies: No Injection, which disables corrective injection during inference; Random (Matched), which retains the learned corrective controller but randomly selects injection checkpoints at a matched rate; and LCC (Guided), which triggers corrective injection according to intermediate cross-modal consistency. Random and guided strategies have identical intervention rates, allowing us to assess the contribution of consistency-based checkpoint selection.

Table 5: Comparison of inference-time intervention strategies on MIMIC-CXR.

Inference StrategyInj. RateBLEU-4 ↑Clinical F1 ↑RG-F1 ↑
No Injection0.0%0.1460.5050.263
Random (Matched)26.4%0.1560.5190.274
LCC (Guided)26.4%0.1710.5390.295

As shown in Table 5, at the matched intervention rate of 26.4%, LCC improves BLEU-4 from 0.156 to 0.171, clinical F1 from 0.519 to 0.539, and RadGraph F1 from 0.274 to 0.295 compared with random injection. Consistent improvements are also observed on IU X-Ray at a matched intervention rate of 24.8%, where BLEU-4 increases from 0.208 to 0.213. These results demonstrate that consistency-guided intervention timing contributes to improved generation quality and clinical correctness beyond intervention frequency alone.

3. Concerns Regarding the Construction of Inconsistent Training Prefixes (Q3)

We appreciate the reviewer's request for a precise description of how inconsistent prefixes are constructed, aligned with target continuations, and used to train the corrective controller. To improve reproducibility, we have clarified the corruption procedure and continuation supervision in Section 3.5 (Eq. 13).

Specifically, at a selected checkpoint tjt_j, we construct inconsistent prefixes through cross-report replacement rather than localized perturbations of ground-truth findings. We cyclically shift prefix token embeddings along the mini-batch dimension, assigning each sample a prefix from another report while retaining its original visual representation. This creates a mismatched image–prefix context without modifying the original target report.

For continuation alignment, each substituted prefix is paired with the original sample's ground-truth continuation after tjt_j, rather than the continuation of the substituted report. Thus, the image and target continuation remain aligned, while the substituted prefix introduces a cross-modal semantic mismatch. The corrective controller is optimized using the continuation prediction objective in Eq. (13), together with control-magnitude regularization.

This training procedure teaches the controller to regulate subsequent generation when the textual context deviates from the corresponding visual evidence. The resulting semantic mismatches may involve clinical findings, their descriptions, or anatomical information, although the corruption procedure does not separately construct each clinical error type. As described in Section 3.4, the controller guides subsequent token predictions rather than rewriting previously generated tokens.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer i3xf,

We sincerely appreciate your time and effort in reviewing our manuscript and for providing valuable suggestions.

We have carefully addressed your comments and posted our detailed responses in the discussion forum. We would be grateful if you could review our replies at your convenience before the discussion period concludes. If you have any remaining questions or concerns, we would be more than happy to provide further clarification.

Thank you again for your time and consideration.

Best regards,

The Authors

Reviewer_kfVAOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to keep a generated radiology report consistent with the input image. The proposed method, LCC, checks the report prefix in a shared image–text space during generation. When the consistency score is low, it adds a learned signal to guide what the model writes next. The authors evaluate the method on IU X-Ray and MIMIC-CXR and report gains in both language quality and clinical metrics.

Critical strengths
  1. The use of representation space is novel. Image–text alignment is usually treated as a training objective or a source of conditioning. Here, the authors use the shared space to assess an evolving report and translate the detected discrepancy back into a control signal. That connection between representation learning and inference-time intervention is a potentially useful contribution. I find this more compelling than the “latent reasoning” terminology itself.

  2. The controller has access to more than a single similarity score. It receives the visual representation, the prefix representation, their interaction through LCSR, and an explicit difference vector. In principle, this lets the system learn how to respond to different discrepancies rather than merely deciding whether a report is aligned. Whether it actually learns such specific responses need further analysis, but the design is sensible.

Critical weaknesses
  1. The report-length analysis does not isolate error accumulation during decoding. Longer reference reports may describe more findings, devices, comparisons with previous examinations, or generally more complex cases. Lower performance in these groups could therefore reflect harder image interpretation and a greater number of facts to report, rather than drift caused by generating more tokens.

  2. BLEU is not a direct measure of the mechanism being claimed. BLEU-2 and BLEU-4 measure reference wording overlap, which also varies with report length, phrasing and the amount of standard template language. An improvement could come from reproducing common wording more closely without preventing a clinically meaningful error. The clinical F1 analysis in the appendix is useful, but it remains a final-report measure. To support the error-propagation claim, I would like to see when incorrect findings first appear, whether they lead to further errors, and what happens immediately after an intervention.

  3. A correct but incomplete prefix may look inconsistent with a complete image representation. Early in generation, a report has only described part of the image. A discrepancy between the pooled image and prefix representations may therefore indicate “not mentioned yet,” rather than “incorrect.” The warm-up period helps avoid extremely short prefixes but does not resolve this distinction.

  4. The evidence used for verification is not independent of the representation used for generation. Both depend on the same visual encoder and condensed image representation. If a subtle finding is lost during encoding or pooling, the generator and verifier could agree with each other while both being wrong. The method may be effective against language-side drift without detecting perception errors.

Questions for authors
  1. How do the length groups differ in abnormality count, finding count and normal/abnormal case proportions? Does LCC’s advantage persist after controlling for these differences?

  2. Can you distinguish correct-but-incomplete prefixes from prefixes containing an actual contradiction? How often does the former trigger intervention?

  3. Using the same trained model, what happens when injection is disabled or triggered randomly at a matched rate? This would help isolate the benefit of consistency-based intervention.

5 comments
AuthorsOpenReview ↗

Response to Reviewer kfVA (Part 1 of 2)

We thank the reviewer for the constructive feedback and recognition of our contributions to cross-modal consistency verification and inference-time corrective control. In response, we have strengthened the manuscript with additional controlled experiments and clarified the corresponding analyses.

Our responses address five concerns in order: (1) report length and clinical complexity (W1, Q1), (2) clinical error propagation (W2), (3) incomplete versus contradictory prefixes (W3, Q2), (4) consistency-guided intervention (Q3), and (5) shared visual representations (W4). The first two are addressed below, followed by the remaining concerns in Part 2.

1. Concerns Regarding Report Length and Clinical Complexity (W1, Q1)

We appreciate the reviewer's observation that report length may reflect underlying clinical complexity. To address this concern, we have clarified the interpretation of Figure 1(b) and provided complementary process-level evidence to evaluate the corrective mechanism independently of report-length grouping.

Specifically, Figure 1(b) evaluates generation performance across groups defined by reference-report length. LCC consistently outperforms the compared methods, with larger improvements observed for longer reports. As longer reports may also contain more abnormalities and clinical findings, we clarify that these results reflect performance differences across report-length groups, rather than establishing a causal relationship between report length and error accumulation. We have refined the corresponding discussion in the revised Introduction.

Furthermore, we introduce a paired decoding analysis that directly examines the effects of corrective intervention on subsequent clinical errors without relying on report length as a proxy. This analysis is presented in Section 4.3 (Table 7) and discussed in the following response.

2. Concerns Regarding Clinical Error Propagation and Corrective Control (W2)

We agree with the reviewer that final-report metrics alone cannot directly characterize clinical error propagation during decoding. To address this concern, we conduct an additional paired decoding experiment that explicitly evaluates how corrective intervention influences the emergence and recurrence of clinical errors.

Specifically, we collect 3,276 consistency-triggered checkpoints from MIMIC-CXR and compare continuations with and without corrective injection from identical generated prefixes. Both conditions use the same trained model and decoding configurations, differing only in the application of the corrective signal. We employ GREEN (Ostmeier et al., 2024) to identify clinical errors and evaluate two process-level metrics: New Error Rate, measuring the proportion of continuations introducing new clinical errors, and Error Persistence, measuring the recurrence of existing errors in subsequent generation. The corresponding evaluation protocol is detailed in Appendix A.2. The quantitative results are presented in Table 7 of the revised Section 4.3.

Table 7: Effectiveness of corrective control on clinical error propagation in MIMIC-CXR.

Inference StrategyN2N_2New Error Rate ↓Error Persistence ↓
Without Injection3,27624.6%35.2%
With Injection (LCC)3,27617.0%24.5%

As shown in Table 7, corrective injection reduces New Error Rate from 24.6% to 17.0% and Error Persistence from 35.2% to 24.5%, corresponding to absolute reductions of 7.6 and 10.7 percentage points, respectively. These results provide direct process-level evidence that LCC mitigates both the introduction of new clinical errors and the recurrence of existing inconsistencies during autoregressive decoding.

To complement the quantitative analysis, we further provide generation trajectories and consistency-score evolution in Appendix C.1 (Figure S4), together with cross-variant report comparisons in Appendix C.2 (Figure S5). These analyses illustrate how intermediate consistency evaluation and corrective intervention influence subsequent report generation.

AuthorsOpenReview ↗

Response to Reviewer kfVA (Part 2 of 2)

We continue by addressing the reliability of prefix consistency evaluation, the effectiveness of consistency-guided intervention, and the role of shared visual representations.

3. Concerns Regarding Incomplete and Contradictory Prefixes (W3, Q2)

We appreciate the reviewer's concern that clinically correct but incomplete prefixes may be mistakenly identified as inconsistent. To investigate this issue, we conduct a controlled paired-prefix evaluation to distinguish incomplete descriptions from explicit factual contradictions.

Specifically, we select 400 MIMIC-CXR test cases and construct two paired prefixes for each case. The correct-but-incomplete prefix is extracted from the reference report without covering all documented findings, while its contradictory counterpart is constructed by modifying a supported clinical attribute, such as finding presence, anatomical location, or status. Each pair shares the same image and clinical context, with comparable prefix lengths. We use the original trained consistency evaluator with the fixed threshold of 0.6, without retraining. The results are presented in Table 6 of the revised Section 4.3.

Table 6: Consistency evaluation across different prefix categories on MIMIC-CXR.

Prefix CategoryN1N_1Mean sconss_{\mathrm{cons}}Trigger Rate
Correct but Incomplete4000.7811.8%
Contradictory Finding4000.3782.5%

As shown in Table 6, correct-but-incomplete prefixes achieve substantially higher consistency scores (0.78 vs. 0.37) and lower intervention rates (11.8% vs. 82.5%) than contradictory prefixes. Notably, 88.2% of clinically valid but incomplete prefixes do not trigger intervention, whereas 82.5% of contradictory prefixes trigger corrective control. These results demonstrate that LCC distinguishes incomplete descriptions from factual contradictions under controlled evaluation, rather than treating unmentioned findings as semantic inconsistencies.

4. Concerns Regarding Consistency-Guided Intervention (Q3)

We agree with the reviewer that the contribution of consistency-guided intervention should be distinguished from the effect of corrective injection alone. To address this concern, we conduct an additional matched-rate inference comparison using the same trained LCC model within each dataset.

Specifically, we compare three strategies: No Injection, which disables corrective injection during inference; Random (Matched), which retains the learned controller but randomly selects injection checkpoints at the same intervention rate; and LCC (Guided), which triggers injection based on intermediate cross-modal consistency. All strategies share identical model weights and decoding configurations, allowing us to isolate the contribution of intervention timing. The results are presented in Table 5 of the revised Section 4.3.

Table 5: Comparison of inference-time intervention strategies on IU X-Ray and MIMIC-CXR.

IU X-Ray:

StrategyInj. RateB-4 ↑MTR ↑RG-L ↑
No Injection0.0%0.2020.2050.421
Random (Matched)24.8%0.2080.2170.430
LCC (Guided)24.8%0.2130.2230.437

MIMIC-CXR:

StrategyInj. RateB-4 ↑F1 ↑RG-F1 ↑
No Injection0.0%0.1460.5050.263
Random (Matched)26.4%0.1560.5190.274
LCC (Guided)26.4%0.1710.5390.295

Compared with matched-rate random injection, LCC improves BLEU-4 by 0.005 on IU X-Ray and 0.015 on MIMIC-CXR. On MIMIC-CXR, clinical F1 and RadGraph F1 further increase by 0.020 and 0.021, respectively. These results demonstrate that consistency-guided intervention timing provides additional benefits beyond applying corrective signals at matched frequencies.

5. Concerns Regarding Shared Visual Representations (W4)

We appreciate the reviewer's observation regarding the shared visual representations. This is an intentional design choice in LCC, establishing a common image-grounded semantic reference for report generation and intermediate consistency verification.

Specifically, the consistency evaluator compares the evolving report prefix with the shared visual representation and latent reasoning state, enabling LCC to detect generation-side semantic deviations and conditionally apply corrective control. As described in Section 3.4, our method focuses on regulating consistency between generated text and extracted visual evidence throughout autoregressive decoding. The controlled intervention results in Table 5 and paired error-propagation analysis in Table 7 further support the effectiveness of this design.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer kfVA,

Thank you again for your thoughtful review and constructive feedback.

We have provided detailed responses to all your comments. As the discussion period is drawing to a close, we would greatly appreciate it if you could take a moment to review our responses at your convenience.

If you have any remaining questions or concerns, we would be more than happy to provide further clarification.

Thank you again for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.