acceptodds
Under review as a conference paper at ICLR 2027

PACER: Progressive Availability-Conditioned Evidence Routing for Radiology Report Generation under Incomplete Clinical Context

Abstract

Radiology report generation (RRG) increasingly incorporates heterogeneous clinical evidence, such as multi-view radiographs and previous reports, whose availability varies across examinations. However, accommodating different input combinations does not ensure effective evidence use: generated reports may still omit or inaccurately describe clinically relevant findings. To address this problem, we propose PACER, a Progressive Availability-Conditioned Evidence Routing framework for structured incomplete-context RRG that follows a Refine–Calibrate–Commit pipeline. It first refines observed visual representations through endpoint-preserving patchwise routing across frozen encoder depths, incorporating complementary cues while retaining the pretrained terminal representation. It then calibrates the language-model prefix according to the observed evidence and availability state, adapting the shared generator’s conditioning as the available source set changes. Finally, it generates polarity-structured clinical commitments before the report in the same autoregressive trajectory, providing structured clinical context for subsequent generation. Experiments demonstrate state-of-the-art clinical efficacy across all four MIMIC-RG4 settings and strong MIMIC-CXR performance, while maintaining competitive language-generation quality.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
radiology report generation, incomplete multimodal learning, multimodal large language models, missing context

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_AnskOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes PACER, which extends LLM-RG4 with three modules. Refine fuses visual features across encoder depths while retaining the final-layer representation through a residual path. Calibrate adjusts the language-model prefix based on observed evidence and source availability. Commit generates positive, negative, and uncertain findings before the report in the same autoregressive sequence. The paper reports consistent CE F1 gains across four MIMIC-RG4 settings, with component and mechanism ablations.

Critical strengths
  1. The application is relevant. Lateral images and previous reports are not always available, so a shared generator should handle these changes.
  2. The results are consistent. CE F1 improves over LLM-RG4 by 0.023–0.028 across all four settings, while language metrics remain broadly competitive. The ablations support gains from combining components. The Calibrate input ablations also match the architecture and parameter count.
Critical weaknesses

W1. The contribution beyond prior work is not clearly established.

The three modules have close precedents. Lin et al. (CVPR 2025) study multi-layer visual fusion. Lee et al. (CVPR 2023) introduce missing-aware prompts, while RAGPT (AAAI 2025) uses content-dependent dynamic prompts. ORGAN (ACL 2023) generates observation plans before reports. PromptMRG (AAAI 2024) encodes positive, negative, uncertain, and unmentioned states as diagnostic prompts. Most relevant works are cited, but the paper does not clearly explain the specific advances of each module over the corresponding precedent.

In particular, LLM-RG4 (AAAI 2025) already addresses factuality through input fusion, clinical token weighting, and evaluation of unsupported descriptions. DiA-gnostic VLVAE (AAAI 2026) already adjusts source contributions through MoE routing when clinical context is missing. Thus, the contrast between “input accommodation” and “evidence utilization” is not sufficiently precise. The claim of the first availability-conditioned evidence-routing formulation also needs a clearer definition and scope.

In summary, the paper fails to clearly establish the unique value of its proposed new research question or mechanisms in relation to existing methods.

Lin et al. (CVPR 2025): multi-layer visual feature selection and fusion.

Lee et al. (CVPR 2023): missing-aware prompting, outside the RRG setting.

W2. The proposed mechanisms are not sufficiently linked to a diagnosed problem.

The motivation that available information may be underused is reasonable but broad. The paper does not identify which evidence the baseline fails to use, under which availability states, or why. It therefore remains unclear why these three changes specifically address the problem and why they should be combined.

Refine mixes encoder depths. Calibrate scales a prefix residual. Commit combines structured generation with mixed training trajectories. Calling all three operations “routing” does not establish a shared evidence-use mechanism. Only Calibrate explicitly uses the availability state.

W3. Essential explanations are deferred to the appendix.

The main text provides the core equations and the two-stage training outline. Yet readers need the appendix to understand what commitments actually contain. Section 4.1 mentions hybrid extraction but omits this important composition. The inherited source compression and fusion interfaces are also not explained clearly enough.

These details affect the interpretation of supervision and novelty. The main text should be able to explain the practical meaning of the method on its own.

Questions for authors
  1. What precisely defines the availability-conditioned evidence-routing problem? What does it add beyond LLM-RG4, DiA-gnostic VLVAE, and existing conditional prompting methods? If the novelty lies in the specific combination, could the “first” claim be narrowed?

  2. Can the authors identify at least one measurable evidence-use failure in the baseline and show how a particular module addresses it?

3 comments

We thank the reviewer for the detailed comments. Below, we clarify PACER’s contributions, present evidence on baseline errors and component effects, and explain training and supervision. Manuscript revisions are highlighted in blue.

PACER (Full) builds on LLM-RG4's established four-context RRG task and improves equal-context CE-F1 over our local Base by .02855 (95% patient-cluster CI [.02048, .03710]; 6,215 test records).

W1/Q1. Problem formulation and specific contributions. The frontal image is always observed, whereas the lateral image and previous report are optional, encoded by a=(al,ap)a=(a_l,a_p). We investigate how a shared generator adapts its use of observed evidence across visual representation, prefix conditioning, and report generation as source availability changes. Sections 1–2 clarify this scope, replace the broad “first” claim, and explicitly credit LLM-RG4's task, fusion, token weighting, and factuality work.

Our technical contributions lie at three complementary interfaces. Building on multilayer-fusion precedents such as Lin et al., Refine learns patch-specific depth corrections while preserving a direct path from the pretrained visual endpoint. Unlike missing-aware or retrieval-conditioned prompting (Lee; RAGPT), Calibrate uses observed evidence and availability to modulate a target-blind residual on valid pre-Transformer prefix embeddings; this also differs from DiA-gnostic VLVAE's latent-posterior routing. While ORGAN and PromptMRG establish planning and diagnostic-state precedents, Commit jointly trains scaffold–report and direct-report trajectories in a shared autoregressive decoder. Only Calibrate explicitly takes the availability code; together, the modules coordinate visual evidence refinement, decoder conditioning, and structured report generation.

W2/Q2. Diagnosed failure and module-level evidence. A measurable baseline failure is denying or omitting reference-positive findings (P→N/M). Across 828 studies matched over all four contexts, Base→Refine reduces these errors from 2,741 to 2,647 among 6,415 opportunities (710 discrepancies resolved, 616 newly introduced): a four-context mean reduction of 1.46 percentage points (95% CI [.09, 2.87]). N→P and M→P assertions also increase, indicating a positive-finding retention gain with an observable trade-off; M means reference-unmentioned, not image-confirmed negative.

Interventions with PACER's weights fixed provide further evidence of Refine's role. On 128 validation patients, removing the visual residual increases the same error rate by 22.70 points [12.81, 32.62], while replacing patch-specific normalized depth weights with their within-image mean increases it by 6.84 [1.07, 14.32]. These interventions link residual refinement and patch-dependent depth allocation to positive-finding retention within the trained model. This endpoint excludes P→U; native/residual-off/image-mean M→P counts are 37/22/32.

The components also exhibit complementary downstream effects: Refine primarily improves recall, while adding Commit increases report CE-F1 from .586185 to .600119 (+.013934 [.005787, .022063]). The controlled RC→PACER test ablation increases CE-F1 from .600119 to .609943 (+.009824 [.003804, .016017]), with the RC parent frozen and only Calibrate trained.

As a complementary Calibrate check, we compare the prefix correction enabled versus disabled while holding all other weights, clinical inputs, and decoding fixed. CE-F1 increases from 60.5625% to 61.0761% across 3,400 validation records from 464 patients (+0.5136 percentage points [−0.0650, 1.0903]). The validation point estimate is positive and directionally consistent with the controlled test ablation.

In the current diagnostic run on 100 studies from the original test set, PACER's initial P/N/U commitments achieve F1 scores of .6175/.6947/.2500 against automatic reference-text states. With weights and clinical inputs fixed, commitment-first versus direct reporting yields CE-F1 of .6212/.6015 and ROUGE-L of .4097/.3929, with BLEU-4 of .2136/.2181. These comparisons show favorable report-level point estimates with a modest lexical-overlap trade-off; this evaluation is separate from the 48-patient validation quality–cost study.

W3. Main-text clarification. Section 3 now specifies the inherited 128-token-per-source query compression, feature-axis fusion, zero-filled missing-source slots, and the 13-finding P/N/U scaffold (unmentioned M is distinct from negative N), with a labeled synthetic example. Section 4.1 explains the training-only 10k LLM-assisted cache and rule-based fallback. Section 3.5 clarifies the optimization stages: RAD-DINO remains frozen, Refine is trained during parent construction, and the resulting parent is frozen while Calibrate is optimized. At inference, PACER generates its own scaffold. Appendix E provides the statistical, dimensionality, and cost analyses with their distinct evaluation protocols.

Dear Reviewer,

Thank you again for your detailed feedback. Your questions helped us make PACER’s contribution and its supporting evidence more explicit. Our response and revised manuscript clarify the distinctions from prior work, quantify the baseline’s negation or omission of reference-positive findings, and provide controlled analyses of Refine’s role in retaining these findings. We have also brought the key supervision and training details into the main text.

We would appreciate your thoughts on whether these additions address your central concerns and would welcome your updated assessment. Please let us know if there are any further points we could clarify.

Thank you for your time and consideration.

Dear Reviewer,

Thank you again for your careful review and constructive feedback. With the discussion nearing its end, we would appreciate your thoughts on the clarified contribution and the additional analyses of baseline errors and Refine’s role in retaining reference-positive findings.

We would be grateful to know whether these additions address your central concerns and would welcome your updated assessment. Please let us know if there are any unresolved points we could clarify before the discussion closes.

Thank you for your time and consideration.

Reviewer_RYiZOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PACER, a radiology report generation framework designed for incomplete clinical contexts. It consists of three modules: Refine, Calibrate, and Commit, which enhance visual representations, adapt language model input embeddings, and introduce structured intermediate clinical supervision, respectively. PACER achieves competitive results across four availability settings on MIMIC-RG4, with ablation studies demonstrating the contributions of individual components.

Critical strengths
  1. Clinically meaningful problem and complementary module design. The paper addresses radiology report generation under incomplete clinical evidence, a practical challenge relevant to real-world clinical workflows. The Refine, Commit, and Calibrate modules are reasonably motivated and complementary: Refine primarily improves recall, Commit improves precision, and Calibrate further enhances overall performance, as supported by the ablation studies.

  2. Comprehensive experimental evaluation. PACER consistently improves CE F1 across all four availability settings while maintaining competitive language generation performance. The component and mechanism ablations also provide useful evidence supporting the effectiveness of the proposed approach.

Critical weaknesses

1. Lack of Direct Evaluation of Clinical Commitment Accuracy

The paper does not directly evaluate the accuracy of the generated clinical commitments. In particular, the current CheXbert evaluation merges positive and uncertain findings into the present category, making it insufficient to assess the model's ability to distinguish different clinical polarities. It's better to evaluate the generated commitments on a verified test subset (e.g., 100–200 samples), report precision, recall, and F1 for positive, negative, and uncertain findings. Additionally, analyzing the consistency between the generated commitments and final reports would provide more direct evidence of the module's effectiveness.

Questions for authors
  1. Clinical Commitment Performance

How accurately does the model predict clinical commitments across different polarities? Are there inconsistencies between the generated commitments and the final reports?

  1. Availability Embedding Design

What motivates the use of 4096-dimensional learnable embeddings for only four availability states? Since these states can be fully represented by two binary indicators, it would be useful to discuss whether lower-dimensional embeddings or direct binary encoding could achieve comparable performance.

3 comments

We appreciate the reviewer’s questions about commitment accuracy and the availability embedding. Below, we examine scaffold accuracy and its relationship to the final report, and compare freshly trained 4096- and 16-dimensional variants under matched training conditions. Changes in the revised manuscript are highlighted in blue.

Q1. Commitment accuracy and report alignment. Our polarity-specific evaluation demonstrates that the Commitment scaffold captures meaningful positive and negative finding information, while subsequent report generation can preserve correct predictions and refine initial discrepancies.

In the current PACER diagnostic run on 100 studies from the original test set, initial P/N/U precision is 60.91/61.89/25.93%, recall is 62.62/79.17/24.14%, and F1 is 61.75/69.47/25.00% against CheXbert-derived reference-report states. Notably, negative findings achieve 79.17% recall. The corresponding TP/predicted/reference supports are 134/220/214 for P, 190/307/240 for N, and 14/54/58 for U. These results directly assess polarity alignment beyond presence-based CE, with unmentioned M distinct from explicit negative N. Uncertainty recognition remains more challenging.

Importantly, the final report not only retains correctly identified findings but also resolves some initial discrepancies. Among 338 initially reference-matching emitted states, 292 (86.4%) remain matched in the report. Of 243 initially discrepant states, 68 (28.0%) become reference-matching, compared with 46 initially matching states that become discrepant. Consequently, reference matches increase from 338 to 360 across the same 581 emitted states, a net gain of 22. This demonstrates that the final report can preserve much of the scaffold's correctly aligned information while refining some initial predictions, rather than simply reproducing the scaffold. Report generation continues to condition on the multimodal prefix throughout decoding.

Commit also provides a measurable downstream benefit to report generation. In the Refine-only→Refine+Commit (RC) component ablation, equal-context CE-F1 increases from .586185 to .600119 (+.013934; 95% patient-cluster CI [.005787, .022063]). The corresponding original-benchmark report-U F1 decreases from .2381 to .2156.

On the same 100 diagnostic test studies, using the same checkpoint and clinical inputs, commitment-first versus direct reporting achieves CE-F1 of .6212 versus .6015 and ROUGE-L of .4097 versus .3929, with BLEU-4 of .2136 versus .2181. Together, these results support the downstream utility of Commit: the component ablation demonstrates a reliable overall CE-F1 improvement, while the commitment-first route achieves higher clinical-label agreement and ROUGE-L point estimates, with a modest BLEU-4 trade-off. This route comparison is distinct from the 48-patient validation quality–cost experiment. Section 3.4 and Appendix A clarify commitment construction and supervision, with the additional diagnostics supplementing the original benchmark.

Q2. Availability embedding. Two binary indicators suffice to identify the four availability states. The learned embedding provides a trainable state representation for the sample gate. We chose 4096 dimensions to match the decoder/prefix/gate width in our implementation, although the concatenation-based architecture also supports lower-dimensional embeddings.

To evaluate this design choice, we trained fresh 4096- and 16-dimensional variants from the same frozen parent, using matched data exposure and fixed 21,576-update endpoints. Test equal-context CE-F1 is .607738 versus .604413, with a 16−4096 difference of −.003324 (95% patient-cluster CI [−.006711, .000228]). The corresponding recall difference is −.00415 [−.00807, −.00014]. The 4096-dimensional variant yields higher F1 and recall point estimates, with the paired recall interval supporting its recall advantage.

Meanwhile, reducing the embedding dimension from 4096 to 16 decreases Calibrate parameters from 50,495,489 to 33,767,489 (−16,728,000; 33.13%), primarily through the gate's input projection. Thus, both embedding width and adapter capacity change. These results characterize the quality–parameter trade-off: the compact embedding substantially reduces adapter size, whereas the wider parameterization retains a recall advantage under the evaluated training protocol. Appendix E.2 provides the full four-context results, paired intervals, and Calibrate parameter counts.

Thanks for the detailed rebuttal and additional experiments. I think the authors have adequately addressed my concerns, and the new results provide stronger evidence for the effectiveness of PACER. However, the relatively limited accuracy of the Commit module, particularly for uncertain findings, and the remaining inconsistencies between commitments and final reports suggest that clinical reliability could be further improved. Additionally, the lack of external OOD evaluation leaves some questions about generalizability. Overall, I appreciate the contributions of this work and remain positive about its acceptance, but I believe my original rating of 3 (Weak Accept) remains appropriate.

We sincerely appreciate the reviewer’s careful engagement throughout this discussion. The questions about commitment accuracy and the availability embedding prompted us to evaluate the generated scaffolds more directly and examine the performance–parameter trade-off more thoroughly. These additions have strengthened the evidence and made our design choices easier to assess. We are particularly grateful that the reviewer took the time to examine the additional results and provide this thoughtful follow-up. The further suggestions on reliability and generalization also offer useful directions for future work. We thank the reviewer again for the constructive exchange and continued positive assessment of our work, and welcome any further questions.

Reviewer_QDsWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Authors have proposed PACER - a framework for radiology report generation under incomplete clinical context, where lateral radiographs and previous reports may be unavailable. The proposed method consists of three components: 1. Refine, which uses patchwise routing across different visual encoder depths to improve image representations. 2. Calibrate, which introduces availability-conditioned low-rank residual adaptation to the language-model prefix 3. Commit, which generates structured positive, negative, and uncertain clinical findings before generating the final report.

Critical strengths
  1. The paper addresses the realistic challenge of radiology report generation when different clinical evidence sources are unavailable.
  2. The experimental results in Table 1 demonstrate improvements in clinical efficacy F1 across all four MIMIC-RG4 settings compared with LLM-RG4.
Critical weaknesses
  1. The evaluation does not sufficiently validate the central claims about clinical evidence utilization. The main clinical efficacy metric relies on CheXbert-extracted labels, where positive and uncertain findings are both treated as present. This evaluation therefore cannot determine whether the proposed polarity-structured commitments improve the distinction between positive and uncertain clinical findings. Similarly, it does not directly evaluate whether the model correctly captures longitudinal changes, anatomical relationships, or clinically important hallucinations.

  2. The experimental improvements need stronger statistical and efficiency analysis. Although the overall gains are consistent, some mechanism-level improvements are small. For example, Table 4. Results across multiple training seeds or confidence intervals would help determine the reliability of these gains. Additionally, the commitment-first decoding introduces additional generated tokens, but the paper does not quantify its inference-time overhead. Since radiology report generation is intended for practical use, the quality-efficiency trade-off should be discussed.

  3. Structured observation-guided radiology report generation has been explored in ORGAN (ACL 2023), OS-RRG (TNNLS 2025) explicitly models positive, negative, and uncertain clinical observation states to guide report generation. But I did not see them discussed or evaluated, additionally patchwise routing is also not completely novel, it seems that particular combination of these three together is presented here which majes novelty somewhat incremental.

Questions for authors

Please see weaknesses.

3 comments

We thank the reviewer for the careful review and the questions on clinical evaluation and statistical reliability. Below, we provide additional analyses, assess the quality–cost trade-off, and clarify the relationship to prior work. Manuscript revisions are highlighted in blue.

W1. Polarity and clinical evaluation. We directly evaluate PACER's 13-finding Commitment scaffold beyond presence-based CE. On 100 original-test studies, the current diagnostic run achieves initial P/N/U F1 scores of 61.75/69.47/25.00% against CheXbert-derived reference-report states, with U precision/recall of 25.93/24.14%. Unmentioned (M) remains distinct from explicitly negative (N). These results demonstrate meaningful P/N alignment. For findings labeled U, we evaluate whether the scaffold identifies the finding while preserving the uncertainty expressed in the reference report.

On the original benchmark, adding Commit to Refine (yielding RC) improves report P-F1 .5944→.6163 and N-F1 .6229→.6349 alongside overall CE-F1, while U-F1 decreases .2381→.2156. This shows improved P/N agreement with an uncertainty trade-off.

Across 6,215 original test records, PACER (Full) improves over local Base by .00533 (95% CI [.00099,.00979]) in RadGraph partial F1; complete F1 changes by .00355 [−.00136,.00862]. This provides complementary graph-based evidence beyond CheXbert. Figure 2 illustrates a reference-supported longitudinal event: PACER describes a newly inserted PICC line when prior-report context is available, whereas the baseline describes it as pulled back. Limited clinician time and resources prevented adjudication of temporal/anatomical correctness and hallucination severity; these are reference-text evaluations.

W2. Statistical reliability. Across 6,215 records/287 patients, PACER improves equal-context CE-F1 over local Base by .02855 [.02048,.03710], with positive difference intervals in all four contexts. Adding Commit to Refine yields +.013934 [.005787,.022063].

The controlled RC→PACER test ablation increases CE-F1 .600119→.609943 (+.009824 [.003804,.016017]): the RC parent is frozen and only Calibrate is trained. An additional on/off check fixes all other weights, clinical inputs and decoding. On 3,400 validation records/464 patients, enabling Calibrate increases CE-F1 60.5625%→61.0761% (+0.5136 percentage points; CI [−0.0650,1.0903]). Its positive point estimate is consistent with the test ablation.

Appendix E.1 reports the original test-set component and mechanism intervals: 10,000 paired patient-cluster resamples, preserving each patient's studies/settings together. These quantify sampling uncertainty for fixed checkpoints.

W2. Quality–cost trade-off. Generating the scaffold adds autoregressive decoding steps before the report. On RTX PRO6000 Blackwell (batch 1, beam 3, BF16), PACER's 32-patient local median/P95 latency is 3.429/4.754 s, with 15.438 GiB peak allocation, excluding loading and warmup.

In a separate 48-patient validation cohort, with weights and clinical inputs fixed, commitment-first/native direct reporting generates 118.67/78.56 token IDs. Mean test-step latency is 3.144/2.061 s (+1.083 s; 1.526× [1.445,1.613]), while CE-F1 improves .5345→.5901 (+.05560; 95% CI [.00663,.10964]; 10.4% relative improvement). Matching total generation-length limits gives mean latencies of 3.144/2.546 s (+.598 s; 1.235× [1.125,1.333]) and CE-F1 +.05392 [−.01289,.12946].

The CE gain motivates commitment-first decoding when prioritizing clinical-label agreement. Appendix E.3 details both timing protocols, generation limits, instructions, histories and report-token allocations.

W3. Contribution and prior work. ORGAN trains a separate autoregressive observation planner and uses a distinct report decoder guided by an observation graph and tree reasoning. OS-RRG predicts observation states through a separate diagnosis branch and supplies them as a report prefix; in its released implementation, these prefix tokens are excluded from the report decoder’s language-model targets. PACER generates the polarity scaffold and report with one shared autoregressive decoder, supervising both spans and mixing complete commitment-first and direct-report trajectories.

On 128 validation patients with PACER's weights fixed, replacing patch-specific normalized depth weights with their within-image mean increases the four-context mean P→N/M error rate by 6.84 percentage points [1.07,14.32]. This supports spatially varying depth selection for reference-positive finding retention within the trained model. The endpoint excludes P→U; native/image-mean M→P counts are 37/32.

Sections 1–2 narrow the broad “first” claim, discuss Lin et al.'s multilayer-fusion precedent and credit LLM-RG4's task, fusion and token weighting. ORGAN's weights load but require matching graph and node-ID/vocabulary mappings; OS-RRG's checkpoint links require access approval in our attempts. We have contacted both author teams for these assets and access.

Dear Reviewer,

Thank you again for your thoughtful comments, which helped us strengthen the evaluation and clarify PACER’s contribution. In our response and revised manuscript, we have added separate polarity analyses, patient-cluster confidence intervals, and a quality–cost evaluation, alongside a more detailed comparison with prior work.

We would be grateful to hear whether these additions address your main concerns and would welcome your updated assessment. If any questions remain, we would be happy to provide further clarification.

Thank you for your time and consideration.

Dear Reviewer,

Thank you again for your time and thoughtful review. As the author–reviewer discussion draws to a close, we would be grateful to hear whether the additional polarity evaluations, statistical analyses, and quality–cost results address your main concerns.

We would welcome your updated assessment in light of our response and revised manuscript. If any questions remain, we would appreciate the opportunity to clarify them while the discussion is still open.

Thank you for your consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.