- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes PACER, which extends LLM-RG4 with three modules. Refine fuses visual features across encoder depths while retaining the final-layer representation through a residual path. Calibrate adjusts the language-model prefix based on observed evidence and source availability. Commit generates positive, negative, and uncertain findings before the report in the same autoregressive sequence. The paper reports consistent CE F1 gains across four MIMIC-RG4 settings, with component and mechanism ablations.
Critical strengths
- The application is relevant. Lateral images and previous reports are not always available, so a shared generator should handle these changes.
- The results are consistent. CE F1 improves over LLM-RG4 by 0.023–0.028 across all four settings, while language metrics remain broadly competitive. The ablations support gains from combining components. The Calibrate input ablations also match the architecture and parameter count.
Critical weaknesses
W1. The contribution beyond prior work is not clearly established.
The three modules have close precedents. Lin et al. (CVPR 2025) study multi-layer visual fusion. Lee et al. (CVPR 2023) introduce missing-aware prompts, while RAGPT (AAAI 2025) uses content-dependent dynamic prompts. ORGAN (ACL 2023) generates observation plans before reports. PromptMRG (AAAI 2024) encodes positive, negative, uncertain, and unmentioned states as diagnostic prompts. Most relevant works are cited, but the paper does not clearly explain the specific advances of each module over the corresponding precedent.
In particular, LLM-RG4 (AAAI 2025) already addresses factuality through input fusion, clinical token weighting, and evaluation of unsupported descriptions. DiA-gnostic VLVAE (AAAI 2026) already adjusts source contributions through MoE routing when clinical context is missing. Thus, the contrast between “input accommodation” and “evidence utilization” is not sufficiently precise. The claim of the first availability-conditioned evidence-routing formulation also needs a clearer definition and scope.
In summary, the paper fails to clearly establish the unique value of its proposed new research question or mechanisms in relation to existing methods.
Lin et al. (CVPR 2025): multi-layer visual feature selection and fusion.
Lee et al. (CVPR 2023): missing-aware prompting, outside the RRG setting.
W2. The proposed mechanisms are not sufficiently linked to a diagnosed problem.
The motivation that available information may be underused is reasonable but broad. The paper does not identify which evidence the baseline fails to use, under which availability states, or why. It therefore remains unclear why these three changes specifically address the problem and why they should be combined.
Refine mixes encoder depths. Calibrate scales a prefix residual. Commit combines structured generation with mixed training trajectories. Calling all three operations “routing” does not establish a shared evidence-use mechanism. Only Calibrate explicitly uses the availability state.
W3. Essential explanations are deferred to the appendix.
The main text provides the core equations and the two-stage training outline. Yet readers need the appendix to understand what commitments actually contain. Section 4.1 mentions hybrid extraction but omits this important composition. The inherited source compression and fusion interfaces are also not explained clearly enough.
These details affect the interpretation of supervision and novelty. The main text should be able to explain the practical meaning of the method on its own.
Questions for authors
-
What precisely defines the availability-conditioned evidence-routing problem? What does it add beyond LLM-RG4, DiA-gnostic VLVAE, and existing conditional prompting methods? If the novelty lies in the specific combination, could the “first” claim be narrowed?
-
Can the authors identify at least one measurable evidence-use failure in the baseline and show how a particular module addresses it?
3 comments
Dear Reviewer,
Thank you again for your detailed feedback. Your questions helped us make PACER’s contribution and its supporting evidence more explicit. Our response and revised manuscript clarify the distinctions from prior work, quantify the baseline’s negation or omission of reference-positive findings, and provide controlled analyses of Refine’s role in retaining these findings. We have also brought the key supervision and training details into the main text.
We would appreciate your thoughts on whether these additions address your central concerns and would welcome your updated assessment. Please let us know if there are any further points we could clarify.
Thank you for your time and consideration.
Dear Reviewer,
Thank you again for your careful review and constructive feedback. With the discussion nearing its end, we would appreciate your thoughts on the clarified contribution and the additional analyses of baseline errors and Refine’s role in retaining reference-positive findings.
We would be grateful to know whether these additions address your central concerns and would welcome your updated assessment. Please let us know if there are any unresolved points we could clarify before the discussion closes.
Thank you for your time and consideration.
We thank the reviewer for the detailed comments. Below, we clarify PACER’s contributions, present evidence on baseline errors and component effects, and explain training and supervision. Manuscript revisions are highlighted in blue.
PACER (Full) builds on LLM-RG4's established four-context RRG task and improves equal-context CE-F1 over our local Base by .02855 (95% patient-cluster CI [.02048, .03710]; 6,215 test records).
W1/Q1. Problem formulation and specific contributions. The frontal image is always observed, whereas the lateral image and previous report are optional, encoded by a=(al,ap). We investigate how a shared generator adapts its use of observed evidence across visual representation, prefix conditioning, and report generation as source availability changes. Sections 1–2 clarify this scope, replace the broad “first” claim, and explicitly credit LLM-RG4's task, fusion, token weighting, and factuality work.
Our technical contributions lie at three complementary interfaces. Building on multilayer-fusion precedents such as Lin et al., Refine learns patch-specific depth corrections while preserving a direct path from the pretrained visual endpoint. Unlike missing-aware or retrieval-conditioned prompting (Lee; RAGPT), Calibrate uses observed evidence and availability to modulate a target-blind residual on valid pre-Transformer prefix embeddings; this also differs from DiA-gnostic VLVAE's latent-posterior routing. While ORGAN and PromptMRG establish planning and diagnostic-state precedents, Commit jointly trains scaffold–report and direct-report trajectories in a shared autoregressive decoder. Only Calibrate explicitly takes the availability code; together, the modules coordinate visual evidence refinement, decoder conditioning, and structured report generation.
W2/Q2. Diagnosed failure and module-level evidence. A measurable baseline failure is denying or omitting reference-positive findings (P→N/M). Across 828 studies matched over all four contexts, Base→Refine reduces these errors from 2,741 to 2,647 among 6,415 opportunities (710 discrepancies resolved, 616 newly introduced): a four-context mean reduction of 1.46 percentage points (95% CI [.09, 2.87]). N→P and M→P assertions also increase, indicating a positive-finding retention gain with an observable trade-off; M means reference-unmentioned, not image-confirmed negative.
Interventions with PACER's weights fixed provide further evidence of Refine's role. On 128 validation patients, removing the visual residual increases the same error rate by 22.70 points [12.81, 32.62], while replacing patch-specific normalized depth weights with their within-image mean increases it by 6.84 [1.07, 14.32]. These interventions link residual refinement and patch-dependent depth allocation to positive-finding retention within the trained model. This endpoint excludes P→U; native/residual-off/image-mean M→P counts are 37/22/32.
The components also exhibit complementary downstream effects: Refine primarily improves recall, while adding Commit increases report CE-F1 from .586185 to .600119 (+.013934 [.005787, .022063]). The controlled RC→PACER test ablation increases CE-F1 from .600119 to .609943 (+.009824 [.003804, .016017]), with the RC parent frozen and only Calibrate trained.
As a complementary Calibrate check, we compare the prefix correction enabled versus disabled while holding all other weights, clinical inputs, and decoding fixed. CE-F1 increases from 60.5625% to 61.0761% across 3,400 validation records from 464 patients (+0.5136 percentage points [−0.0650, 1.0903]). The validation point estimate is positive and directionally consistent with the controlled test ablation.
In the current diagnostic run on 100 studies from the original test set, PACER's initial P/N/U commitments achieve F1 scores of .6175/.6947/.2500 against automatic reference-text states. With weights and clinical inputs fixed, commitment-first versus direct reporting yields CE-F1 of .6212/.6015 and ROUGE-L of .4097/.3929, with BLEU-4 of .2136/.2181. These comparisons show favorable report-level point estimates with a modest lexical-overlap trade-off; this evaluation is separate from the 48-patient validation quality–cost study.
W3. Main-text clarification. Section 3 now specifies the inherited 128-token-per-source query compression, feature-axis fusion, zero-filled missing-source slots, and the 13-finding P/N/U scaffold (unmentioned M is distinct from negative N), with a labeled synthetic example. Section 4.1 explains the training-only 10k LLM-assisted cache and rule-based fallback. Section 3.5 clarifies the optimization stages: RAD-DINO remains frozen, Refine is trained during parent construction, and the resulting parent is frozen while Calibrate is optimized. At inference, PACER generates its own scaffold. Appendix E provides the statistical, dimensionality, and cost analyses with their distinct evaluation protocols.