acceptodds
Withdrawn from ICLR 2027

Self-Editing and Preference Distillation for Chest X-Ray Report Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reinforcement learning has advanced chest X-ray report generation, but assigning a single report-level reward can suppress correct findings in low-reward reports and reinforce local errors in high-reward ones. Self-refinement offers a way to address such local errors, yet existing approaches have limitations: inference-time methods can be unreliable without external feedback, while training-time methods rely on externally derived corrections or refinement supervision rather than explicitly optimizing the generation model itself to perform self-refinement. We propose ED-SR (Editing and Distillation for Self-Refinement), a training-time reinforcement learning framework in which a single vision-language model serves as both generator and editor. The generator produces candidate reports, while the editor re-examines the images and its generated drafts to identify potential errors, explain why they are problematic, and propose sentence-level revisions, deletions, and additions. ED-SR augments report-level policy optimization with two losses: (i) a counterfactual edit-reward loss that assigns fine-grained credit to self-generated editing actions using the reward change from applying each edit alone, and (ii) a correction-distillation preference loss that trains the generation policy to prefer reward-improving revisions over their original drafts. Because the generator and editor share parameters, edit-level supervision directly updates the same model used for report generation, while gradient conflict resolution manages competing report-level and preference updates. At inference time, ED-SR generates reports in a single pass without an additional refinement stage. Experiments demonstrate state-of-the-art performance across natural language generation and clinical efficacy metrics, achieving the best results on 9 of 11 metrics on MIMIC-CXR and all evaluated metrics on CheXpert Plus. Code will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
chest X-ray report generation, vision-language models, reinforcement learning, self-editing, fine-grained credit assignment, preference distillation.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZDeuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ED-SR, a self-correcting framework for generating chest X-ray reports. The authors utilize a single visual language model to handle both report generation and script generation, and train the model through three stages: (1) supervised fine-tuning for report generation, (2) followed by training the model's editing capabilities using a more powerful model, (3) optimizing the model's initial report generation ability through report-level reinforcement learning, counterfactual editing rewards, and preference loss distillation. Furthermore, the authors propose a Gradient Conflict Resolution mechanism to address potential gradient conflicts between report-level strategy optimization and preference loss.

Critical strengths
  1. This study addresses an important and practical problem: the difficulty of accurately assigning credit for specific errors in medical report generation. It proposes a more granular form of supervision for reinforcement learning in this context.
  2. The method framework is comprehensive: incorporating counterfactual editing, preference distillation, and GCR to form a clear training process. The authors also conducted ablation studies on key modules.
Critical weaknesses
  1. The reward function used in this paper consists of BLEU-2, ROUGE-L, CheXbert F1, and RadGraph F1. However, the paper does not clearly explain how the weights for each metric were determined. Furthermore, the authors do not specify the design rationale for using BLEU-2 instead of BLEU-4 or BLEU-1. If the NLG metric is removed and the reward function is constructed solely based on clinical metrics, how would this affect the distribution of the reward, the positive/negative judgments, and the final generated output? Finally, the paper only states that CheXbert F1 is calculated at the sample level using 14 classes of observation labels, but it does not explicitly detail how uncertain (-1) and blank (NaN) labels are handled. These implementation details could directly impact the difference in rewards and, consequently, influence the optimization direction of the editing actions.

  2. Currently, the primary sources of reward signals are CheXbert and RadGraph, along with NLG metrics. However, while CheXbert and RadGraph provide a degree of medical reliability, there is still a gap between them and actual clinical accuracy, which raises concerns about potential error accumulation.

  3. In Experiment A.7, the modifications generated directly by the editor resulted in a decrease in several metrics. However, using the Oracle-filtered scheme, as outlined in the reference report, yielded better results. The modifications proposed by the editor should not be considered reliable clinical corrections, as the final outcome depends on how effectively the edits are identified and selected. Therefore, this editor is less effective when the reasoning process lacks GT (presumably, a specific guideline or method).

  4. The ablation study demonstrates that incorporating edit-level losses on top of report-level reinforcement learning yields additional benefits. However, the current experiments do not fully distinguish whether the performance improvement stems from a meaningful semantic correspondence between edit actions and rewards, or from the additional auxiliary training objective and its optimization process. If the same edit scripts and reward distributions are maintained, but the correspondence between edit actions and rewards is randomized, would the model still achieve similar improvements? Furthermore, this paper utilizes edit scripts generated by the model to construct preferences. Since the edit scripts are only used during the training phase to construct edit-level supervision and preferences, the inference process does not utilize the edit scripts. Therefore, it is necessary to further validate whether the edit-level supervision, constructed using the model's own generated edit scripts, provides independent benefits compared to a simpler reference report-guided preference learning scheme.

Questions for authors

(1) How was the reward function's weights determined? Was hyperparameter search or sensitivity analysis performed? Why was BLEU-2 chosen over BLEU-1, BLEU-4, or not using NLG metrics? (2) What is the specific method used by CheXbert F1? How are "uncertain" and "blank" labels handled? Could different approaches to handling these labels affect the positive/negative judgment of edit rewards or the final performance? (3) Can you provide independent evidence to demonstrate that rewards for editors actually increase the likelihood of correcting clinical errors, rather than simply improving composite automated evaluation metrics? (4) Can further experiments, such as disrupting the relationship between editing and rewards, be used to confirm whether the semantic information contained in editing-level rewards is a key factor in overall performance improvement? (5) What are the primary advantages of the self-editing mechanism compared to the approach of constructing revision preferences directly from reference reports? Where does the author see the mechanism's most significant contribution?

Reviewer_QnmjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ED-SR, which uses a vision-language model to generate reports and propose local edits during training. Counterfactual edit rewards and preference distillation transfer beneficial revisions into improved first-pass generation. The method achieves promising results on MIMIC-CXR and CheXpert Plus.

Critical strengths
  1. Learning to identify and correct errors is an insightful idea with practical value.
Critical weaknesses
  1. What specific correction patterns has the model learned? Please provide concrete examples and evidence of their transferability.
  2. Failure cases may require different treatment. Particularly challenging cases, such as very subtle lesions, may be difficult to resolve through self-editing without additional input information.
  3. Please report performance separately for patients with and without prior information, and isolate the contributions of prior images and prior reports.
  4. Applying all edits at test time degrades performance in the appendix. The results demonstrate the training value of reference-reward-filtered edits, but do not establish that the model can independently identify and apply correct revisions.
Questions for authors
  1. Can correction performance be analyzed by error type and case difficulty, with evidence of transferability?
  2. How can beneficial edits be reliably selected to prevent self-editing from degrading performance?
Reviewer_wCB4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper argues that report-level rewards spread one advantage over all tokens and cannot localise erroneous sentences, while inference-time self-correction is unreliable without external feedback. The authors propose ED-SR, a three-stage scheme in which one vision-language model with LoRA serves as both generator and editor. Stage 1 is supervised generation, stage 2 warms up the editor on 100 studies whose sentence-level edit scripts were produced offline by GPT 5.6, and stage 3 adds two losses to GRPO-style report-level optimisation: one scoring each self-proposed edit by the reward change it yields alone, the other preferring the jointly edited report over the draft with an adaptive margin target and a sign consistency rule for conflicting gradients. Inference is a single pass with no editor. Experiments are conducted on the MIMIC-CXR and CheXpert Plus datasets to demonstrate the effectiveness of the proposed method.

Critical strengths
  1. Edit scripts use explicit sentence identifiers, so any script can be applied deterministically, and every edit can be scored in isolation, which is what makes the counterfactual reward meaningful rather than heuristic. Supervision is injected only during training through shared parameters, so inference stays a single pass.

  2. The study covers two datasets, 23 grouped baselines, a staged ablation isolating each loss and the conflict resolution rule, a comparison of first-pass, self-corrected, and reference-filtered inference, training diagnostics, a cost table, and full hyperparameters.

Critical weaknesses
  1. The claim that a uniform report-level advantage suppresses correct findings is never analysed. The method is motivated by the assertion that a shared report-level advantage suppresses correct sentences in low-reward reports and reinforces local errors in high-reward ones, yet no experiment or statistic tests whether this failure mode actually occurs.

  2. The claimed self-refinement ability is not established by the evidence presented. Edit scores are computed against the reference report, and the editor role is initialised from 100 scripts produced offline by GPT-5.6 with the reference as anchor, so both the supervision and the credit signal are reference-dependent. Table 6 shows that unconditionally applying all self-generated edits at inference degrades CheXbert micro F1 from 0.597 to 0.563, while reference-guided filtering raises it to 0.641. Since selection is only possible with the reference in hand, the evidence does not rule out that the gains stem from reference-guided dense credit assignment rather than from a self-verification ability.

  3. Stage two supervision may itself carry part of the improvement. The 100 sentence-level edit scripts used in stage two are produced offline by GPT-5.6 from the reference report and the stage one draft, so it remains unclear how much of the final improvement stems from inheriting the correction policy of a strong external teacher rather than from self-editing, and this needs clarification.

Questions for authors
  1. Can you show that a uniform report-level advantage actually suppresses correct findings in your baseline, rather than only asserting it?

  2. Since most of the gain comes from the edit loss, have you compared against a sentence- or entity-level reward baseline under the same backbone?

  3. Given that the model’s own edits reduce CheXbert micro F1 at inference while reference-filtered edits raise it, what evidence shows that a self-verification signal has been learned?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.