acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Clinical Risk Allocation for Radiology Report Generation

Abstract

Clinically reliable radiology report generation requires models to identify and control clinically distinct errors. Despite recent progress, existing sequence-level methods typically combine multiple clinical discrepancies with fixed weights, leaving the optimization focus unchanged as the generator's error profile evolves during training. We propose ACRA, an adaptive clinical risk allocation framework that adjusts the optimization focus as the report generator evolves. ACRA maps generated and reference reports into a shared structured finding space and tracks the generator's greedy-decoding risk profile across five reference-conditioned discrepancy categories: omission, unsupported positive finding, polarity inconsistency, laterality error, and severity error. It then applies prior-regularized online mirror ascent to allocate greater optimization mass to persistent risks while preventing excessive concentration on any single category. The resulting allocation defines a risk-aware self-critical objective for comparing sampled and greedy reports. All risk-related components are used only during training and are removed at inference. Experiments on MIMIC-CXR and IU-Xray show that ACRA improves clinical consistency and reduces reference-conditioned clinical risk while maintaining competitive report quality. These results demonstrate that adapting optimization priorities to the evolving clinical error profile enables more effective risk control without adding inference-time modules.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Adaptive Risk Allocation, Radiology Report Generation, Medical Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_3HvSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes ACRA, which categorizes discrepancies between generated and reference reports into five types and dynamically adjusts their penalty weights during self-critical sequence training. Experiments on MIMIC-CXR and IU-Xray demonstrate improved report consistency.

Critical strengths

Accounting for distinct clinical errors and their evolving training dynamics is clinically meaningful.

Critical weaknesses
  1. Limited clinical performance and evaluation. The reported micro-F1 scores are below 0.50, substantially lower than those reported by many recent report-generation models. Please also report macro-F1 and additional clinical efficacy metrics, such as GREEN.
  2. Insufficient evidence for dynamic allocation. The fixed-weight baseline uses only uniform weights. Without comparison against tuned nonuniform weights, it remains unclear whether the gains arise from better weighting or continuous adaptation.
  3. Limited methodological contribution and validation. The method primarily applies an existing online weighting mechanism to report-generation training and is evaluated only on the older R2GenCMN backbone, without comparisons involving strong VLMs or recent optimization methods.
  4. Insufficient independent validation of clinical risk reduction. Training and evaluation rely heavily on the same structured extraction tools, without blinded physician assessment or adequate validation of extraction accuracy.
Questions for authors
  1. Do the gains persist against tuned fixed weights, frozen final ACRA weights, and simpler dynamic weighting schemes?
  2. Does ACRA provide consistent gains on stronger VLMs?
  3. Can physician assessment or independent evaluators confirm reductions in actual clinical errors?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 3HvS (1/2)

We thank the reviewer for highlighting the clinical motivation and for requesting stronger controls and independent evaluation. We address the four critical weaknesses first and then answer the three questions in order.

W1. Clinical performance and micro- and macro-F1

We agree that the absolute CheXbert micro-F1 values in Table 4 are below 0.50. Appendix A.8 explains that cross-study comparisons depend on the test cases, reference text, label inventory, positive-label mapping, and aggregation. For example, DDGIP and REVA-PO use CheXpert labels. Table 4 uses CheXbert micro-F1.

We added a separate CheXbert macro-F1 comparison in Table 5. It uses the same MIMIC-CXR test set, label inventory, and positive-label mapping as the micro-F1 comparison, with equal weight for each finding's positive-class F1 (Appendix A.8).

MIMIC-CXR configurationMicro-F1 (Table 4)Macro-F1 (Table 5)
R2GenCMN (NLL)0.3670.251
Fixed uniform risk weights0.3890.264
ACRA0.3990.272

Relative to fixed uniform weighting, ACRA improves micro-F1 by 0.010 and macro-F1 by 0.008 under the shared protocol. Q3 reports the complementary GREEN evaluation.

W2. Evidence for dynamic allocation

Appendix A.6 adds fixed non-uniform controls, including a run that holds ACRA's final weight vector fixed throughout training. Section 4.4 and Appendix A.4.1 report simpler online variants. Q1 compares these controls with full ACRA.

W3. Methodological contribution and recent comparisons

The contribution is the training formulation for five reference-conditioned clinical discrepancies. ACRA aggregates each risk over eligible report–finding pairs from greedy and reference reports, smooths these risks, and updates the allocation with a prior-regularized KL mirror-ascent step (Sections 3.2–3.3, Eqs. (4)–(7)). It then applies the same allocation to sampled and greedy reports in the self-critical score (Sections 3.4–3.5, Eqs. (11)–(12)). These components operate during training. The generator retains its standard inference procedure. Appendix A.6 characterizes the limiting weights when pairwise risk gaps are constant and prior regularization is positive.

Section 2 distinguishes this risk-driven allocation from MPO (Xiao et al., 2025), which trains a preference-conditioned generator and accepts a preference vector at inference. The revised comparison includes published results for MPO, DDGIP, R2GenGPT + SI + SA, and REVA-PO-7B under their source protocols (Section 4.2, Appendix A.8, Table 12). Q2 gives the controlled result on Qwen2.5-VL-7B.

W4. Independent validation of clinical risk reduction

We agree that the structured extractors used in training overlap with parts of the main clinical evaluation. Appendix A.10 adds GREEN, whose scoring function is separate from the training reward. Q3 reports the result and its scope.

Q1. Tuned fixed, frozen-final, and simpler online allocations

The fixed controls use the same five risk terms on MIMIC-CXR (Appendix A.6, Table 9). The initial-risk control uses the NLL checkpoint's greedy risk profile. The validation-tuned control selects the weights using validation performance. For the frozen-final control, we initialized a separate SCST run from the same NLL checkpoint and held the final ACRA weight vector fixed throughout training.

AllocationMean risk ↓
Fixed uniform0.198
Fixed initial risk0.188
Fixed validation tuned0.182
Frozen final ACRA0.180
ACRA0.177

The tuned weights recover much of the improvement over uniform weights. ACRA lowers mean risk by a further 0.005 relative to validation tuning and 0.003 relative to freezing the final weights. All five category risks are lower than in both controls. The frozen-final comparison supports a modest additional benefit from updating the weights during training in this setting.

Both simpler variants retain online updates. Setting τ = 0 removes the prior regularization, and removing EMA uses current mini-batch risks (Table 6 and Appendix A.6). On MIMIC-CXR, mean risk is 0.185 without prior regularization, 0.183 without smoothing, and 0.177 with the full allocator (Table 6). The corresponding IU-Xray values are 0.170, 0.168, and 0.162 (Table 7, Appendix A.4.1). Removing the prior lowers omission and unsupported-positive risk on both datasets and increases polarity, laterality, and severity risks. The full allocator gives the lowest mean risk among these configurations.

AuthorsOpenReview ↗

Response to Reviewer 3HvS (2/2)

Q2. Does the gain carry over to a stronger VLM?

Appendix A.9 (Table 13) compares fixed-risk SCST and ACRA on the same Qwen2.5-VL-7B backbone and MIMIC-CXR test set, using the same five reference-conditioned risk categories.

MetricFixed-risk SCSTACRA
Omission ↓0.2480.226
Unsupported positive ↓0.2020.191
Polarity ↓0.1500.149
Laterality ↓0.0960.098
Severity ↓0.1950.191
Mean risk ↓0.1780.171
BLEU-4 ↑0.1420.144
ROUGE-L ↑0.2930.292

Mean risk drops by 0.007, with the largest reductions in omission and unsupported positive findings. The laterality rate increases by 0.002. BLEU-4 increases slightly and ROUGE-L decreases slightly. The experiment supports a lower average reference-conditioned risk on this second backbone, with trade-offs at the category and metric level. The controlled evidence covers R2GenCMN and Qwen2.5-VL-7B.

Q3. Independent assessment of clinical errors

We evaluated the R2GenCMN (NLL), fixed uniform risk weighting, and ACRA outputs with GREEN (Ostmeier et al., 2024) on the same MIMIC-CXR test set (Appendix A.10, Table 14). GREEN uses a separately trained language model to score agreement between generated and reference reports, and its scoring function is outside the training reward in Eq. (11).

ConfigurationMean GREEN ↑
R2GenCMN (NLL)0.300
Fixed uniform risk weights0.307
ACRA0.315

ACRA improves mean GREEN by 0.008 over fixed weighting and 0.015 over NLL. This independent automatic evaluation supports stronger clinical agreement with the reference reports, with physician adjudication, image-level verification, and extractor accuracy outside its scope.

Thank you again for your careful review. If the additional results and clarifications address your concerns, we would appreciate your reconsidering the overall rating in light of the revised manuscript.

I thank the authors for their efforts during the rebuttal period. Their responses have addressed some of my concerns, I will therefore keep my original score.

Reviewer_CSVrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper researches radiology report generation. To achieve clinically reliable reports, models need to identify and control clinically distinct errors. This work argues that existing methods typically combine multiple clinical discrepancies with fixed weights, which prevents the optimization from dynamically adjusting its focus on different types of errors during training. To address this issue, this work proposes a method called ACRA that can adaptively perform clinical risk allocation during the training process so as to adjust optimization emphasis according to the evolving clinical risk profile of the generator. This adaptive allocation strategy is incorporated into the self-critical sequence training process to train a generation model with the commonly used negative log-likelihood loss. Experimental study is conducted on benchmark datasets to demonstrate the efficacy of the proposed method.

Critical strengths
  1. For radiology report generation, clinical reliability is a critical issue when it is used to solve real-world problems. This work aims to improve the clinical reliability of a generated report by developing more effective optimisation.

  2. The motivation is clearly presented in this work, and the proposed idea on adaptive allocation is sound. The way of modelling multiple clinical discrepancies is interesting. Experimental study demonstrates the improvements obtained on two commonly used radiology report generation benchmarks.

  3. This paper is overall well-structured and clearly presented.

Critical weaknesses
  1. The clarity of this work can be further improved at several places, especially in Sections 3.3 and 3.4. Detailed comments are provided below.
  2. The experimental study can be further enhanced such as testing the proposed adaptive allocation mechanism with more backbones and discussing the tuning of multiple hyper-parameters.
Questions for authors

Questions whose answers could materially affect the assessment:

  1. For the five reference-conditioned discrepancy categories in Table 1, how are they decided or defined, say, is it based on any existing medical knowledge or standard?

  2. In Eq. (3), please explicitly define \theta here. Also, please describe the role and goal of using this inner maximization over \alpha.

  3. What is the relationship between Eq. (11) and Eq.(3)? Both of them have the term r_task.

  4. As indicated in Line 306, "We set γ = 0.8, λ = 0.5, ρ = 0.9, η = 0.1, τ = 0.5, and β = 0.01." The presence of the multiple hyper-parameters seems to be a problem when the proposed method is applied in practice. Please discuss how to set or tune the hyper-parameters when using the proposed method.

  5. In the experimental results, please report the results for the criteria RadCliQ, GREEN, FineRadScore, and/or RaTEScore. They are recent metrics developed to better assess the quality of a generated report. They are also mentioned in Section of Related Work.

  6. In the experiment, the comparison between R2GenCMN (reprod.) and ACRA is good because it directly shows the efficacy of the proposed risk-aware sequence-level optimization. To further verify its generality, it will be better to test ACRA with another two models in addition to R2GenCMN.

Suggestions for future work:

  1. In Line 046, please provide a reference for the statement that "Existing approaches, however, commonly combine multiple objectives using fixed scalar weights."

  2. In Section of Introduction, this paper could briefly explain what sequence-level optimisation is.

  3. At Line 131, it will be better to explain why two reports need to be generated, for example, as required by the self-critical sequence training approach to be used by this work?

  4. In Eq. (2), please recall the definitions of K and c here.

  5. At Line 229, please define \mu formally, including its components, and this will help understand Eq.(5).

  6. For the prior-regularised KL mirror-ascent step in Eq. (6), please explicitly indicate the derivation is provided in Appendix A.3.

  7. Figure 3 is good to visualise the evolution of the risk and allocation weights. Meanwhile, the result also suggests that the relative importance among the five risks does not change substantially during the training process. Will this result adversely affect the value of the proposed adaptive risk allocation idea? Please comment.

5 comments
AuthorsOpenReview ↗

Response to Reviewer CSVr (1/3)

We thank the reviewer for the constructive assessment and the specific suggestions. We address the two critical weaknesses first, followed by the questions and suggestions in the order of the review.

W1. Clarity of the optimization

Sections 3.3 and 3.4 now define the generator parameters and the role of the inner maximization, and explain how the training score follows from the minimax formulation at a fixed allocation. Q2 and Q3 give the details.

W2. Experimental coverage and practical tuning

Appendix A.5 gives a validation-based tuning procedure, Appendix A.9 adds a controlled comparison on Qwen2.5-VL-7B, and Appendix A.10 reports GREEN. Q4–Q6 summarize these additions and the scope of the evidence.

Q1. How are the five discrepancy categories defined?

Section 3.2 and Table 1 define five task-specific discrepancies between a generated report and its reference. The finding inventory uses CheXbert observation categories (Smit et al., 2020). RadGraph provides observations and assertion states (Jain et al., 2021), and attribute normalization follows Chest ImaGenome (Wu et al., 2021). Appendix A.2 details the extraction and normalization rules.

Omission counts a reference-positive finding that the generated report does not mention. Unsupported positive counts a generated positive finding with no reference mention. Polarity compares assertion states for shared mentions. Laterality and severity require a reference-positive finding with the corresponding attribute specified. A missing or differing generated attribute counts as an error when the generated report also mentions that finding as positive. The distinctions for missing and unsupported findings, location, and severity overlap with error types assessed by GREEN (Ostmeier et al., 2024). Table 1 specifies the eligibility and error indicators for each category. These rates quantify agreement with the reference report.

Q2. Generator parameters and the inner maximization

Section 3.3 now defines θ as the trainable parameters of the report generator GθG_\theta. In Eq. (3), for a fixed generator and λ > 0, the inner maximization chooses category weights on the simplex and assigns more mass to categories with larger mean discrepancies. The KL term pulls the weights toward the prior π when τ > 0. In training, we estimate the current risk profile from eligible greedy–reference pairs and update the weights with one prior-regularized KL mirror-ascent step (Eqs. (4)–(7)). The generator is then updated under this allocation.

Q3. Relationship between Eqs. (3) and (11)

Equation (10) defines the shared task reward, which combines language similarity and positive-finding agreement. At the allocation held fixed for a generator update, the integrand inside the expectation in Eq. (3) equals the negative of the sequence score in Eq. (11). The prior KL term in Eq. (3) is then constant with respect to θ. Equation (12) forms the self-critical advantage by subtracting the greedy report's score from the sampled report's score under the same weights. Section 3.4 now makes this relationship explicit.

AuthorsOpenReview ↗

Response to Reviewer CSVr (2/3)

Q4. How to tune the hyperparameters in practice

Appendix A.5 gives a validation-based tuning procedure. We start from the defaults in Section 4.1 and Appendix A.1, with (γ, λ, ρ, η, τ, β) = (0.8, 0.5, 0.9, 0.1, 0.5, 0.01).

First, vary λ on the validation set to balance language quality and mean risk, and inspect all five category risks for trade-offs. Adjust γ when a different balance between language similarity and positive-finding agreement is needed. Keep β at its default unless the balance between NLL and sequence-level training needs to change. For the allocator, vary one coefficient at a time. Increasing ρ smooths mini-batch fluctuations, decreasing η moderates abrupt weight changes, and increasing τ pulls the allocation toward the prior.

Table 8 reports the one-at-a-time sensitivity to ρ, η, and τ on MIMIC-CXR at fixed γ = 0.8 and λ = 0.5. Mean risk ranges from 0.177 to 0.184 over the tested settings. Table 4 also compares task-only SCST (λ = 0) with fixed uniform weighting and ACRA (both λ = 0.5). Mean risk for these three configurations is 0.224, 0.198, and 0.177 on MIMIC-CXR, and 0.205, 0.181, and 0.162 on IU-Xray. The reported sweep characterizes the allocator coefficients at the stated reward setting.

Q5. Evaluation with a recent clinical metric

We evaluated all three R2GenCMN training configurations with GREEN (Ostmeier et al., 2024) on the same MIMIC-CXR test set. GREEN scores generated reports against their references using a separately trained evaluator. Its scoring function is outside the training reward in Eq. (11).

ConfigurationMean GREEN ↑ (Table 14)
R2GenCMN (NLL)0.300
Fixed uniform risk weights0.307
ACRA0.315

The 0.008 gain over fixed uniform weighting provides complementary evidence of improved agreement with the reference reports (Appendix A.10).

Q6. Evaluation on an additional backbone

Appendix A.9 adds a controlled comparison between fixed-risk SCST and ACRA on Qwen2.5-VL-7B using MIMIC-CXR. Both variants use the same backbone and five discrepancy categories (Table 13).

MetricFixed-risk SCSTACRA
Omission ↓0.2480.226
Unsupported positive ↓0.2020.191
Polarity ↓0.1500.149
Laterality ↓0.0960.098
Severity ↓0.1950.191
Mean risk ↓0.1780.171
BLEU-4 ↑0.1420.144
ROUGE-L ↑0.2930.292

The largest reductions are in omission and unsupported positive findings. Laterality risk increases slightly, and the two generation metrics shown here move in different directions. The lower mean risk supports an added benefit of adaptation on this second backbone. The controlled evaluation in the revision covers two backbones in total, R2GenCMN and Qwen2.5-VL-7B.

AuthorsOpenReview ↗

Response to Reviewer CSVr (3/3)

S1. References for fixed scalar weights

The Introduction (Section 1, lines 44–45) now cites Liu et al. (2019) and Miura et al. (2021) as examples of combining language and clinical objectives with fixed scalar coefficients.

S2. Definition of sequence-level optimization

Section 1 (lines 42–44) now explains that sequence-level optimization updates the generator using scores computed on complete generated reports. Scoring the complete report lets the objective include report-level clinical agreement.

S3. Roles of the sampled and greedy reports

Section 3.1 explains that the multinomially sampled report supplies the trajectory for the policy-gradient update, and the greedy report supplies the self-critical baseline. The greedy report is also paired with the reference to update the online risk state. Equation (12) uses the same allocation to score both reports.

S4. Definitions of K and c in Eq. (2)

Section 3.2 now recalls that k indexes the K = 14 categories in the CheXbert finding inventory and that c ∈ C indexes the five discrepancy types. Appendices A.2 and A.7 clarify that the No Finding category is excluded from the risk computation, leaving 13 findings in the risk inventory.

S5. Definition of the smoothed risk vector

Before Eq. (5), Section 3.3 defines μ(t)=(μc(t))c∈C\boldsymbol{\mu}^{(t)} = (\mu_c^{(t)})_{c \in \mathcal{C}}, the vector of smoothed risks at iteration t with one component for each discrepancy category. Equation (5) updates each component using the mini-batch risk in Eq. (4) when that category has eligible opportunities.

S6. Location of the mirror-ascent derivation

The text immediately after the KL mirror-ascent step in Eq. (6) now points to Appendix A.3 for the derivation of the closed-form update in Eq. (7). The appendix gives the resulting update in Eq. (31).

S7. Stable ordering of risk categories in Figure 3

We agree that the ordering in Figure 3(a) is broadly stable and omission remains the largest risk after warm-up. The update responds to the gaps between the category risks, which can still change with a stable ordering. Section 4.5 discusses these trajectories, and Appendix A.6 shows that constant pairwise gaps with positive prior regularization yield a fixed limiting allocation.

The fixed-weight controls give a direct empirical check (Appendix A.6, Table 9). On MIMIC-CXR, mean risk is 0.198 with fixed uniform weights, 0.188 with fixed initial-risk weights, 0.182 with validation-tuned fixed weights, and 0.180 with ACRA's final weights frozen throughout a separate SCST run initialized from the same NLL checkpoint. Online ACRA achieves 0.177, a further reduction of 0.005 relative to the validation-tuned control and 0.003 relative to the frozen-final control. These comparisons support a modest benefit of updating weights during training in this setting.

Thank you again for your careful review. If the additional results and clarifications address your concerns, we would appreciate your reconsidering the overall rating in light of the revised manuscript.

Reviewer_xBSzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper introduces ACRA (Adaptive Clinical Risk Allocation), a sequence-level reinforcement learning framework for automated radiology report generation (RRG). Noting that prior sequence-level training methods (e.g., SCST) combine clinical discrepancy rewards with fixed static weights throughout training, the authors propose to dynamically adapt the optimization focus as the generator evolves.

Specifically, ACRA maps reference, greedy-decoded, and sampled reports into a structured clinical finding space (leveraging CheXbert observations and RadGraph/Chest ImaGenome attribute extraction) to quantify five reference-conditioned clinical risks: omission, unsupported positive finding, polarity inconsistency, laterality error, and severity error. The framework tracks the generator's greedy-decoding risk profile via exponential moving average (EMA) and uses prior-regularized online mirror ascent to dynamically update the categorical risk distribution. This adaptive allocation is shared when computing the self-critical advantage between sampled and greedy outputs. All risk extractors and calibration modules are active only during training and stripped away during inference. Experiments on MIMIC-CXR and IU-Xray with an R2GenCMN backbone demonstrate reductions in reference-conditioned clinical risks and modest gains in conventional NLG metrics.

Critical strengths
  1. Principled online optimization formulation for multi-objective sequence training: Instead of relying on static scalarization or unconstrained greedy reweighting, ACRA formulates clinical risk weighting as an online minimax problem solved via prior-regularized mirror ascent. The incorporation of both an exponential moving average (EMA) risk tracker and a KL divergence penalty against a uniform prior prevents the objective from collapsing into dominant error modes (such as omission), ensuring stable and distributed weight adaptation across all five discrepancy categories.

  2. Clean zero-overhead inference design and comprehensive ablation: All structured extractors, attribute parsers, and dynamic weight calculators are confined strictly to the RL training stage. At test time, the model executes standard autoregressive beam search without any inference latency or parameter overhead. The ablation studies systematically substantiate the distinct contribution of adaptive allocation over static risk penalization.

Critical weaknesses
  1. Outdated generator backbone and unproven scalability to modern VLMs: The entire empirical evaluation is conducted using R2GenCMN (Chen et al., 2021), which pairs an ImageNet-pretrained ResNet-101 visual encoder with a specialized memory-driven Transformer. In the current state of multimodal learning and medical report generation, standard architectures have overwhelmingly transitioned to modern Vision-Language Models (e.g., LLaVA-Med, Rad-LLaMA, Qwen2.5-VL adaptations) featuring strong autoregressive language priors. Such modern decoders suffer from radically different error distributions (e.g., hallucinations, conversational verbosity, over-hedging) compared to older CNN-RNN pipelines that heavily suffer from simple n-gram omission. Validating ACRA solely on an older 2021 backbone casts serious doubt on whether this complex RL allocation framework provides any meaningful advantage over standard alignment or instruction-tuning on modern medical foundation models.

  2. Evaluation circularity and lack of independent clinical validation: The five reference-conditioned clinical risks in ACRA are derived directly from CheXbert labelers, RadGraph entity/assertion extractors, and Chest ImaGenome lexicons. Crucially, the primary clinical metrics reported in Table 4 are CheXbert micro-F1, RadGraph-F1, and the exact same five reference-conditioned risk formulas. Training an RL agent using rewards directly parameterized by extractors A and B, and then evaluating performance primarily using extractors A and B, constitutes a circular evaluation loop (reward hacking/test-metric alignment). While the authors cite newer, clinically validated metrics in related work—such as GREEN (Ostmeier et al., 2024), FineRadScore (Huang et al., 2024), and RaTEScore (Zhao et al., 2024)—none of these independent benchmarks or double-blind human radiologist evaluations are conducted. Without independent clinical verification, it is impossible to confirm whether ACRA produces genuinely superior clinical reports or merely optimizes the specific biases of the rule-based extractors.

  3. Unreported computational training overhead and extractor noise: At every single sequence-level iteration, ACRA must run multi-stage extraction (RadGraph token parsing, CheXbert fallback, and attribute regex matching) across reference, sampled, and greedy-decoded reports. These parsing steps are computationally expensive and prone to parsing errors on non-fluent intermediate generated text. The manuscript provides no wall-clock training time comparisons, GPU throughput metrics, or sensitivity analyses regarding how parser inaccuracies propagate through the online risk state.

Questions for authors
  1. Scalability to Modern Decoder-Only VLMs: Why was an older architecture (R2GenCMN from 2021) chosen as the sole backbone? Have the authors evaluated ACRA on a modern open-source medical vision-language model (e.g., a fine-tuned Qwen2.5-VL or LLaVA-Med backbone)? Does the dynamic risk allocation remain stable when the generator possesses a much stronger pre-trained linguistic prior?

  2. Independent Clinical Evaluation (GREEN / Human Study): Given the inherent metric circularity between the training reward functions (CheXbert/RadGraph) and the reported clinical test results (Table 4), could the authors provide an evaluation using an independent, LLM-based metric such as GREEN (Ostmeier et al., 2024) or a blind evaluation by board-certified radiologists on a representative test subset?

  3. Wall-Clock Training Latency: What is the exact wall-clock training time per epoch for ACRA compared to standard NLL pre-training and task-only SCST (BLEU/ROUGE) on MIMIC-CXR? How much computational overhead is introduced by executing RadGraph and CheXbert inference on every training mini-batch?

  4. Robustness to Extraction Errors: How robust is the online risk state to false-positive or false-negative extractions caused by RadGraph and lexicon matching on noisy, sampled candidate sentences during early RL stages?

2 comments
AuthorsOpenReview ↗

Response to Reviewer xBSz (1/2)

We thank the reviewer for recognizing the optimization formulation and for raising the questions about backbone choice, independent evaluation, and training cost. We address the critical weaknesses first and then answer the questions in order.

W1. Evaluation on a modern vision-language model

The revised manuscript adds a controlled Qwen2.5-VL-7B comparison on MIMIC-CXR (Appendix A.9, Table 13). Q1 distinguishes this direct test of adaptation from comparisons with published VLM results.

W2. Overlap between training and evaluation

The original clinical evaluation shares some extraction components and discrepancy definitions with the training objective. Appendix A.10 adds GREEN as a separate scoring function applied to outputs on the same test set with the same references. Q2 gives the results and their scope.

W3. Training overhead and extractor noise

Appendix A.11 reports the wall-clock cost of training and the clinical extraction stage (Table 15). Q3 details the timing, and Q4 explains how extraction errors can affect the generator update and the online risk state.

Q1. Backbone choice and evaluation on Qwen2.5-VL-7B

We chose R2GenCMN as a common backbone for MIMIC-CXR and IU-Xray so that the fixed-risk SCST and ACRA variants shared the generator, task reward, and five clinical risk terms. This comparison isolates the effect of how those terms are weighted. ACRA reduces mean reference-conditioned risk from 0.198 to 0.177 on MIMIC-CXR and from 0.181 to 0.162 on IU-Xray (Table 4).

To test the same allocation question on a modern backbone, we compared fixed-risk SCST and ACRA using Qwen2.5-VL-7B on MIMIC-CXR. Both variants use the same five discrepancy categories (Appendix A.9, Table 13).

MetricFixed-risk SCSTACRA
Omission ↓0.2480.226
Unsupported positive ↓0.2020.191
Polarity ↓0.1500.149
Laterality ↓0.0960.098
Severity ↓0.1950.191
Mean risk ↓0.1780.171
BLEU-4 ↑0.1420.144
ROUGE-L ↑0.2930.292

The largest reductions are in omission and unsupported positive findings. Laterality increases slightly and the two generation metrics move in different directions. The lower mean risk supports the benefit of adaptation at the evaluation endpoint on this second backbone. Figures 3 and 4 report allocation trajectories for R2GenCMN. Training-time stability on Qwen2.5-VL-7B remains a separate empirical question. The published REVA-PO-7B results (Appendix A.8, Table 12) provide context under their source protocol.

Q2. Independent clinical evaluation with GREEN

We agree that Table 4 includes metrics that overlap with the clinical signals used during training. To assess agreement with the references using a separate evaluator, we applied GREEN (Ostmeier et al., 2024) to the R2GenCMN (NLL), fixed uniform risk weighting, and ACRA outputs on the same MIMIC-CXR test set. GREEN uses a separately trained language model, and its scoring function is outside the training reward in Eq. (11).

ConfigurationMean GREEN ↑ (Table 14)
R2GenCMN (NLL)0.300
Fixed uniform risk weights0.307
ACRA0.315

ACRA improves mean GREEN by 0.008 over fixed weighting and 0.015 over NLL (Appendix A.10). The gain over fixed weighting provides complementary evidence of stronger agreement with the same reference reports under a separate automatic evaluator. The evidence is reference-based and does not establish image-grounded correctness.

AuthorsOpenReview ↗

Response to Reviewer xBSz (2/2)

Q3. Wall-clock training time and clinical extraction overhead

We timed the three R2GenCMN configurations on MIMIC-CXR using one NVIDIA RTX A6000, a batch size of 16, and 500 batches per timed epoch (Appendix A.11, Table 15). The training window includes data loading, forward and backward passes, generation and reward computation where applicable, and optimizer updates. It excludes validation, test evaluation, report export, and checkpoint writing.

ConfigurationTotal training (s/epoch)Clinical extraction and normalization (s/epoch, included in total)
NLL800
Task-only SCST24040
ACRA30090

Both SCST configurations use the task reward in Eq. (10), which includes positive-finding agreement alongside the BLEU-4 and ROUGE-L components. This accounts for extraction time in task-only SCST even with the structured risk penalty disabled. ACRA adds 60 s/epoch (25%) relative to task-only SCST, with a 50 s/epoch increase in the clinical extraction stage. The extraction timing is for the combined stage, including normalization and any extractor fallback. Dividing 500 batches by the total wall-clock times gives end-to-end training-loop rates of approximately 6.25, 2.08, and 1.67 batches/s, respectively.

Q4. How extraction errors affect the advantage and online risk state

A false-positive or false-negative extraction from the sampled report can change its positive-finding agreement or structured risk terms. This can change the magnitude or sign of the sampled-minus-greedy advantage (Eqs. (9)–(12)) and thus the generator update. The sampled and greedy scores use the same allocation for that update.

The current risk state and allocation are updated from eligible greedy–reference pairs before the advantage is computed (Eqs. (4)–(7), Section 3.5, and Algorithm 1 in Appendix A.3). A sampled-report extraction error therefore has no direct path into the current risk state. It can still have an indirect effect through the generator update and the next iterations' greedy reports. Errors in the greedy or reference extractions can directly affect the risk estimate.

For an eligible category, the EMA with ρ = 0.9 gives the current mini-batch risk a weight of 0.1. If there are no eligible opportunities for that category in the mini-batch, the state retains its previous value (Eq. (5)). This smooths transient batch fluctuations. Persistent extraction errors can still bias the smoothed state and the allocation built from it. These are properties of the update rule, and the revision does not report a controlled extraction-error stress test.

Thank you again for your careful review. If the additional results and clarifications address your concerns, we would appreciate your reconsidering the overall rating in light of the revised manuscript.

Reviewer_b9mhOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Adaptive Clinical Risk Allocation (ACRA) to address the issue of using fixed weights when optimizing for different categories of clinical errors in radiology report generation (RRG). The authors argue that the five defined clinical risk categories exhibit varying frequencies of occurrence and distinct trends during the training process, consequently, fixed weights may fail to adapt to the evolving error distribution of the report generator.

To this end, the authors first employ RadGraph, CheXbert, and rule-based attribute extraction to map both generated and reference reports into a unified structured clinical representation space, within which five types of "reference-conditioned clinical risks" are defined. Subsequently, the authors maintain the current risk state of the generator using an exponential moving average (EMA) and dynamically adjust the weights for these five risk categories via prior-regularized online mirror ascent. Finally, these dynamic risk weights are integrated into Self-Critical Sequence Training (SCST) to optimize the report generation model.

The authors conducted experiments on the MIMIC-CXR and IU-Xray datasets and validated their approach through ablation studies, analysis of dynamic weight variations, and sensitivity analysis of hyperparameters.

Overall, the proposed method is sound, however, I believe the paper falls short in justifying the significance of the research problem, clearly articulating its innovative contributions, and demonstrating the necessity of the core methodology.

Critical strengths
  1. A relatively complete and mathematically formulated optimization framework. The paper presents a coherent framework that integrates structured clinical risk modeling, EMA-based risk estimation, prior-regularized online mirror ascent, and self-critical sequence training. The mathematical formulation and optimization procedure are relatively clear, and the risk-related components are removed during inference, avoiding additional inference-time overhead.

  2. Evaluation on two established RRG benchmarks with multiple analyses. The authors conduct experiments on both MIMIC-CXR and IU-Xray, including comparisons with existing methods, ablation studies, and training-dynamics analyses. These experiments provide an initial evaluation of the proposed framework and offer some insight into the effects of its individual components.

Critical weaknesses
  1. The central motivation of this paper is that different clinical error categories may exhibit different risk trajectories during training, thereby motivating adaptive weight allocation to adjust the optimization priorities of the generator. However, I do not think the authors have sufficiently established the significance of this research problem. In particular, the authors observe that Omission consistently exhibits the highest risk and interpret this as evidence of different learning difficulties across error categories. However, this observation may also be influenced by the prevalence and distribution of different findings in the training dataset, as well as the number of eligible opportunities associated with each risk category. Since the five risks are computed over different reference-conditioned eligible sets, their aggregated magnitudes alone do not necessarily reflect the intrinsic learning difficulty of the model for each category. The paper does not provide sufficient analyses of per-finding omission rates, finding distributions, or category-specific eligible opportunity counts, nor does it adequately distinguish data distribution effects from the generator's actual learning dynamics. Furthermore, the proposed risks essentially measure structured discrepancies between generated reports and reference reports, rather than image-grounded clinical risks. Consequently, the paper does not sufficiently establish a convincing connection between the observed risk differences and a research problem with clear clinical significance.

  2. The central contribution of this work is to dynamically adjust the weights of different risk categories based on the generator's current risk profile using EMA and prior-regularized mirror ascent. However, the current experiments primarily compare adaptive weighting against uniform fixed weighting, without considering a properly tuned fixed non-uniform weighting strategy. In particular, the reported risk trajectories show that Omission remains consistently higher throughout training, while the relative ordering of the risk categories does not change substantially. Under these circumstances, it remains unclear whether dynamic adaptation genuinely provides an advantage over fixed weights determined from initial risk estimates or validation performance, or whether the observed improvements simply result from adopting a more appropriate non-uniform weighting distribution. Therefore, the current experiments do not sufficiently establish the necessity of the proposed dynamic allocation mechanism. Moreover, the experimental comparison covers only a limited number of recent RRG methods, with only one method from 2025 and one from 2026. Such limited coverage makes it difficult to assess the generalizability and robustness of the reported performance improvements across a broader range of recent approaches. Furthermore, the AAAI 2025 paper on Multi-objective Preference Optimization (MPO) is not sufficiently discussed. Although ACRA differs in its weighting mechanism, the two methods share conceptual similarities in multi-objective formulation and reinforcement learning-based optimization. The authors should better clarify the technical distinction and independent novelty of ACRA.

Questions for authors
  1. Can the authors explain why adaptive risk allocation is preferable to a properly tuned fixed non-uniform weighting scheme? Given that Omission consistently exhibits the highest risk throughout training and the relative ordering of the risk categories does not change substantially, could a reasonable non-uniform weighting scheme established at the beginning of training achieve comparable results? Have the authors conducted controlled experiments under identical training configurations to demonstrate that dynamic adaptation itself provides additional benefits, rather than simply identifying a more appropriate weight distribution?

  2. How do the authors distinguish intrinsic learning difficulty across risk categories from statistical differences induced by the training data distribution? In particular, could the consistently higher Omission risk be attributed to the prevalence of different findings in the training set or the number of eligible opportunities associated with each risk category? Could the authors provide the frequency distribution and per-finding omission rates, together with the number of eligible opportunities for each of the five risk categories, to demonstrate that the observed risk differences genuinely reflect different learning dynamics rather than being primarily driven by data distribution?

  3. The experimental comparison includes only one recent method from 2025 and one from 2026. How can the authors establish that the reported performance improvements are representative and competitive across a broader range of recent RRG approaches? Could the authors include additional representative recent methods to provide more comprehensive evidence of the proposed method's effectiveness?

  4. The AAAI 2025 paper on Multi-objective Preference Optimization (MPO) has already investigated multi-objective preference optimization in RRG. What is the central technical contribution of ACRA beyond MPO? Apart from replacing externally provided preference vectors with weights dynamically updated according to the generator's current risk profile, what substantive differences exist between the two approaches in terms of problem formulation, optimization mechanisms, or theoretical contributions? Why was MPO not discussed in the related work or included in the experimental comparisons?

  5. The authors investigate the sensitivity of some hyperparameters associated with EMA and mirror ascent, but do not sufficiently examine γ\gamma and λ\lambda, which directly control the balance between language generation quality and clinical risk. Could the authors clarify how these two parameters affect the final performance and the distribution of the five risk categories? Are the reported improvements robust across different parameter configurations, or do they depend heavily on specific hyperparameter choices?

4 comments
AuthorsOpenReview ↗

Response to Reviewer b9mh (1/3)

We thank the reviewer for recognizing the formulation and for asking us to test when adaptive allocation adds value. We address the critical weaknesses first and then answer the questions in order.

W1. Motivation, data distribution, and clinical scope

Section 3.2 defines the scope as clinically meaningful disagreement with reference reports across five error categories. Sections 4.3 and 4.5 interpret each rate under its own eligibility rule. Aggregate rates alone cannot establish intrinsic learning difficulty or image-grounded clinical correctness. Appendix A.7 examines this distinction with eligibility counts and both pooled and equal-finding omission summaries on a common reference cohort. Q2 gives the results.

W2. Fixed-weight controls, recent comparisons, and MPO

The revision adds fixed non-uniform controls in Appendix A.6, expands the recent comparisons in Section 4.2 and Appendix A.8, and explains the relationship to MPO in Section 2. Q1, Q3, and Q4 address the evidence for adaptation, the evaluation protocols, and the methodological distinction, respectively.

Q1. Does adaptation add value beyond tuned fixed weights?

Appendix A.6 adds three fixed non-uniform controls to the MIMIC-CXR comparison. The initial-risk control derives its weights from the NLL checkpoint's greedy-decoding risk profile. The validation-tuned control selects a fixed vector using validation performance. The frozen-final control takes ACRA's final weight vector and uses it throughout a separate SCST run initialized from the same NLL checkpoint. Each control uses the same five risk categories as ACRA.

MIMIC-CXR test risks (Table 9, lower is better)

AllocationMeanOmissionUnsupported positivePolarityLateralitySeverity
Fixed uniform0.1980.2690.2320.1570.1110.221
Fixed initial risk0.1880.2540.2180.1510.1070.209
Fixed validation tuned0.1820.2460.2110.1470.1040.203
Frozen final ACRA0.1800.2400.2080.1470.1050.200
ACRA0.1770.2380.2040.1440.1020.197

Validation tuning recovers most of the improvement over uniform weights. ACRA lowers mean risk by a further 0.005 relative to the tuned control and 0.003 relative to the frozen-final control, with lower rates in all five categories against both. The frozen-final comparison supports a modest benefit from updating the weights during training in this setting.

Figure 3(a) shows that omission remains the largest risk and that the ranking is broadly stable. The gaps between risks still change, and the update responds to those gaps. Appendix A.6 derives the fixed limit when the gaps are constant and the prior regularization is positive. Sections 4.3 and 4.5 relate this result to the controls and the observed trajectories.

AuthorsOpenReview ↗

Response to Reviewer b9mh (2/3)

Q2. Eligibility counts, prevalence, and finding-level omissions

Table 1 gives the eligibility rules and Eq. (16) gives the pooled estimator. Appendix A.7 reports the training and test counts (Table 10), and report-level frequencies and conditional omission rates (Table 11), for MIMIC-CXR.

SplitReference reportsOmissionUnsupported positivePolarityLateralitySeverity
Train222,758334,8242,336,191559,663179,135102,265
Test3,2695,72333,4769,0213,0631,747

The five risk counts are eligible report–finding pairs. Frequencies below are percentages of all reference reports in the split. Omission rates are percentages of the reference-positive test pairs for that finding.

FindingTrain mention %Train positive %Test mention %Test positive n (%)NLL omission %ACRA omission %
Atelectasis25.120.228.4771 (23.6)27.519.8
Cardiomegaly27.317.630.2668 (20.4)28.422.0
Consolidation9.64.710.4179 (5.5)41.335.2
Edema28.511.930.4452 (13.8)30.521.9
Enlarged cardiomediastinum9.73.110.2120 (3.7)45.038.3
Fracture2.21.72.564 (2.0)60.957.8
Lung lesion3.62.84.0105 (3.2)57.150.5
Lung opacity25.122.928.8870 (26.6)34.826.4
Pleural effusion38.423.942.4909 (27.8)25.319.8
Pleural other1.30.91.434 (1.0)61.861.8
Pneumonia25.97.127.1270 (8.3)45.635.2
Pneumothorax23.64.224.3159 (4.9)40.933.3
Support devices30.929.535.71,122 (34.3)21.116.5

Support devices, pleural effusion, and lung opacity account for 2,901 of 5,723 reference-positive pairs (50.7%). This gives those findings approximately half the weight in the pooled omission rate. On the same references, moving from NLL to ACRA reduces omissions from 1,746 to 1,362 and the pooled rate from 0.305 to 0.238 (Table 4). Giving each finding equal weight, omission decreases from 40.0% to 33.7%, with decreases in 12 of 13 findings. The within-cohort reduction thus holds under both summaries. The ranking in Figure 3 describes risks under their respective eligibility rules. Separating the causal effects of training-set prevalence from intrinsic learning difficulty would require a separate controlled comparison.

Q3. Recent methods and evaluation protocols

Section 4.2 and Appendix A.8 now include four additional 2025–2026 methods. Table 12 gives all six generation metrics on both benchmarks under the source protocols. Selected values are below.

MethodIU-Xray BLEU-4IU-Xray ROUGE-LMIMIC-CXR BLEU-4MIMIC-CXR ROUGE-L
MPO (AAAI 2025)0.2090.4150.1390.309
DDGIP (NAACL Findings 2025)0.2020.4110.1170.288
R2GenGPT + SI + SA (CVPR Findings 2026)0.1910.4090.1450.318
REVA-PO-7B (ECCV 2026)0.1990.4060.1430.293
ACRA (Tables 2 and 3)0.2020.4130.1340.303

The published results span preference-conditioned RL, graph-guided prompting, LLM semantic alignment, and VLM reinforcement learning. They contextualize our scores under different protocols. For instance, MPO uses Findings and Impression on IU-Xray and reports BLEU and ROUGE-L from different preference vectors. The SI + SA row uses global alignment and is reported on 3,858 MIMIC-CXR samples and 590 filtered IU-Xray reports. DDGIP and REVA-PO report CheXpert-based clinical metrics, whereas our Table 4 reports CheXbert micro-F1.

Under the shared scoring pipeline, ACRA improves BLEU-4 over the reproduced R2GenCMN from 0.193 to 0.202 on IU-Xray and 0.127 to 0.134 on MIMIC-CXR. ROUGE-L increases from 0.411 to 0.413 and 0.296 to 0.303, respectively (Tables 2 and 3). As documented in A.8, the archived trainer monitors test metrics and the archive does not identify the exact checkpoints used in these tables. We interpret these as observed differences for the evaluated runs. A second within-backbone comparison with Qwen2.5-VL-7B (Appendix A.9, Table 13) shows a lower mean risk on MIMIC-CXR (0.178 to 0.171), alongside an increase in laterality risk (0.096 to 0.098).

AuthorsOpenReview ↗

Response to Reviewer b9mh (3/3)

Q4. Technical distinction from MPO

We agree that MPO warranted explicit discussion. The revised Related Work (Section 2) describes the distinction, and Table 12 in Appendix A.8 includes the published results from Table 3 of Xiao et al. (AAAI 2025).

MPO trains a preference-conditioned generator to produce reports for a range of user preferences. It samples preference vectors during training, uses them to combine metric rewards, and optimizes the generator with REINFORCE and a greedy baseline. At inference, a supplied preference vector steers report generation.

ACRA addresses a training-time allocation problem over five reference-conditioned clinical discrepancies. Its motivating objective is a prior-regularized minimax problem (Eq. (3)). The allocator estimates risk from eligible greedy/reference pairs, smooths the risks, and applies a closed-form KL mirror-ascent update (Eqs. (4)–(7)). The sampled and greedy reports share the resulting allocation when computing the self-critical advantage (Eq. (12)). Appendix A.6 characterizes the allocator's stationary limit under fixed risk gaps, and Section 3.5 specifies image-conditioned inference without the allocator or a preference input.

Q5. Effects and sensitivity of γ and λ

Section 3.4 clarifies the two roles. Increasing γ shifts the task reward toward language similarity and away from positive-finding agreement (Eq. (10)). At a given allocation, λ scales the combined five-category risk penalty (Eq. (11)). Future allocations can change indirectly through the generator's subsequent greedy reports. The allocator update in Eq. (7) takes the smoothed risks and previous allocation.

Appendix A.5 relates these coefficients to Table 4. At γ = 0.8, task-only SCST uses λ = 0, and both fixed uniform weighting and ACRA use λ = 0.5.

DatasetConfigurationλMeanOmissionUnsupported positivePolarityLateralitySeverity
MIMIC-CXRTask-only SCST00.2240.2970.2650.1770.1240.257
MIMIC-CXRFixed uniform0.50.1980.2690.2320.1570.1110.221
MIMIC-CXRACRA0.50.1770.2380.2040.1440.1020.197
IU-XrayTask-only SCST00.2050.2780.2380.1620.1130.234
IU-XrayFixed uniform0.50.1810.2470.2080.1450.1010.204
IU-XrayACRA0.50.1620.2180.1810.1320.0930.186

Introducing the structured penalty with uniform weights lowers all five risks on both datasets, and online allocation lowers them further at the same λ. For language quality, Tables 2 and 3 report end-to-end configuration comparisons. The one-at-a-time sweep in Table 8 varies ρ, η, and τ at fixed γ = 0.8 and λ = 0.5, with mean risk between 0.177 and 0.184. This characterizes sensitivity to the allocator parameters at the reported reward setting. The available experiments do not establish robustness to changing γ or using other nonzero values of λ. Appendix A.5 recommends tuning λ first on validation data, monitoring language quality and category-specific risks, and adjusting γ if the task requires a different balance between language similarity and positive-finding agreement.

Thank you again for your careful review. If the additional results and clarifications address your concerns, we would appreciate your reconsidering the overall rating in light of the revised manuscript.

Thank you for the authors' very detailed response. I have also read the revised manuscript, particularly the modified sections. Many of the newly added experiments have helped address some of my initial concerns. However, upon further consideration, I believe the following issues are more important:

One of the most serious concerns is that Appendix A.8 of the revised manuscript states: "The archived trainer evaluates validation and test after each epoch. Its checkpoint monitor uses test BLEU-4, ROUGE-L, Fclin, and Ravg." This indicates that the authors evaluated the test set at every epoch during the experiments, which raises serious concerns about the credibility of the experimental conclusions.

Furthermore, as shown in the newly added Table 9, the gains achieved by dynamic allocation over the tuned fixed-weight approach appear to be very limited. Can the authors demonstrate that the additional complexity of their method is truly necessary? More importantly, does dynamic allocation actually provide any meaningful gains in terms of independent clinical evaluation?

In addition, my concerns regarding several important experimental hyperparameters have still not been adequately addressed. Given that this is an optimization-based method, the absence of these experiments leaves me with concerns about the robustness of the overall framework.

Furthermore, the F1-score of the MPO method is not explicitly reported. Does its calculation follow the same protocol as that of ACRA?

Regarding the comparisons of clinical efficacy (CE) metrics, particularly on the MIMIC-CXR dataset, I believe the authors should include comparisons of CE metrics against those reported by other baseline methods.

I believe the results of the above experiments are very important for establishing the significance of the proposed method. Nevertheless, I will slightly adjust my rating to Weak Reject.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.