acceptodds
Under review as a conference paper at ICLR 2027

A Structured Commitment Space for Factual Policy Learning in Chest Radiography

Abstract

Clinical vision-language models can identify the correct finding yet misstate whether it is present or how severe it is. Aggregate reinforcement-learning rewards provide little explicit control over how these different factual errors are penalized. We introduce Fact-Commitment Space, a reward for factual policy learning in chest radiography. It represents free-form clinical text as commitments, each specifying a clinical entity, its assertion state, and relevant attributes. A Clinical Fact Parser (CFP) extracts these commitments, and a deterministic compiler normalizes them. Matching response and reference commitments then assigns separate penalties to omissions, unsupported findings, attribute errors, and contradictions, without an online language-model judge. Under stated conditions on structured commitments, omitting an attribute has lower expected loss than guessing in the zero-confidence limit, reversing an assertion scores below omission, and adding an unsupported commitment lowers the reward. On 600 human-reviewed responses, CFP improves entity–assertion F1 from 13.82% to 75.32% over its Qwen3.5-2B base, and its rewards preserve 72.78% of non-tied reference pair orderings. Training with this reward using Group Relative Policy Optimization (GRPO) yields VeraRad-2B. On 3,000 clinical visual questions, it achieves 75.53% strict accuracy versus 66.70% for its Qwen3.5-2B base, with thinking enabled for both models. On report generation, a task absent from training, it leads the primary quality metric on two of three datasets and produces reports 43.7% shorter than its base in the same mode. Image-swap and no-image interventions assess dependence on study-specific images, complemented by spatial attribution analysis of a cardiomegaly case.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
vision-language models, reinforcement learning, reward design, factuality, structured prediction, chest radiography

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_nrFWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Fact-Commitment Space, a structured reward framework for improving factual correctness in chest radiography. It represents clinical statements using entities, assertion states, and attributes, and assigns different penalties to omissions, unsupported findings, attribute mismatches, and contradictions. The authors develop a Clinical Fact Parser (CFP) and integrate the proposed reward with GRPO to train VeraRad-2B. Experiments show improvements in clinical VQA and several report generation metrics, together with analyses of parsing reliability, reward properties, and visual grounding.

Critical strengths
  1. Clinical factual errors are not equally harmful, and explicitly distinguishing different types of errors is a meaningful direction for medical VLM training.

  2. Representing clinical claims as explicit commitments allows more interpretable and controllable reward computation. The deterministic scoring procedure and theoretical analysis provide useful justification for the design.

  3. The authors validate the parser using human-reviewed data and examine reward consistency through controlled interventions. The VQA results are promising, and the transfer to report generation provides additional evidence that the training affects downstream behavior.

Critical weaknesses
  1. The main experiments demonstrate that VeraRad improves over its base model after GRPO training, but there is no matched-budget comparison with alternative reward functions. Since existing clinical factuality metrics could also be used for RL training, it remains unclear whether the improvements come specifically from Fact-Commitment Space or from RL post-training more generally.

  2. VeraRad achieves a clear improvement in strict accuracy, but the gain is considerably smaller under answer-extraction accuracy. This suggests that improved output formatting may account for part of the reported improvement. A more direct separation of formatting and clinical correctness would make the results more convincing.

  3. The proposed model performs well on the primary report quality metric, but the results are mixed across other metrics. Its reports are also substantially shorter, which may improve some similarity scores while omitting clinically relevant findings. More direct evaluation of unsupported and missing clinical information would strengthen the conclusions.

Questions for authors
  1. Can the authors compare the proposed reward against simpler correctness-based rewards and existing clinical factuality rewards under the same GRPO training budget?

  2. How much of the VQA improvement comes from better format compliance versus improved clinical correctness? Can these two effects be evaluated separately?

  3. Can the authors provide a more direct factuality evaluation of the generated reports, distinguishing hallucinations from clinically important omissions?

  4. How sensitive is the training outcome to the relative penalties for omissions, unsupported findings, and attribute mismatches?

4 comments
AuthorsOpenReview ↗

Reply to Reviewer nrFW (1/2): W1 / Q1 — alternative rewards under the same budget; W2 / Q2 — format versus correctness

Thank you for the clear and constructive review. Each of your four questions is addressed by a new controlled experiment. We answer in two comments, and each contains the numbers it relies on.

W1 / Q1 — alternative rewards under the same budget. We trained two GRPO baselines, a format-tolerant correctness reward (B1) and a RadGraph-F1 reward (B2), together with SFT on the gold answers, a second seed of the FCS reward and two ablations. Every trained model uses the same base (Qwen3.5-2B), the same 3,200 training items in the same order, and the same LoRA, sampling and thinking budgets; the GRPO models share the 800 updates and change only the reward or the seed. Intervals are 95% patient-cluster bootstrap intervals (10,000 resamples) on paired items.

Model (VQA-3000, strict / answer-extracted accuracy, %)Thinking OnThinking Off
Qwen3.5-2B (base)66.70 / 73.2752.93 / 68.20
SFT, same items70.47 / 71.0374.23 / 74.23
GRPO, correctness reward (B1)73.07 / 73.8366.53 / 67.23
GRPO, RadGraph-F1 reward (B2)72.63 / 73.1367.63 / 67.70
VeraRad-2B (FCS reward)75.53 / 75.7369.70 / 69.77
FCS reward, second seed75.97 / 76.2768.13 / 69.40
FCS, entity+assertion only73.20 / 74.1364.37 / 67.27
FCS, κ = 274.70 / 75.1367.80 / 68.53
  • On extracted accuracy, the FCS reward is above B1 by 1.90 points with thinking On (0.39 to 3.40) and 2.53 with thinking Off (1.71 to 3.35), and above B2 by 2.60 (1.03 to 4.15) and 2.07 (1.32 to 2.81).
  • Neither B1 (+0.57 On, −0.97 Off) nor B2 (−0.13 On, −0.50 Off) differs from the base on extracted accuracy (all four intervals include zero): under this budget these rewards improve the answer format only.
  • A second FCS seed reproduces VeraRad within 0.6 points. SFT on the same items is higher with thinking Off (+4.47, 3.21 to 5.72) and lower with thinking On (−4.70, −6.57 to −2.78), where it returns an empty report for 61% to 93% of studies.

W2 / Q2 — format compliance versus clinical correctness. They can be separated, and we now do so with a per-item transition analysis from the base to VeraRad.

  • Of the 265 additional strict-correct items with thinking On, 151 are format repairs (the base answer was correct once extracted) and 300 are content repairs, offset by 185 content regressions. With thinking Off, 377 are format repairs, 166 content repairs and 40 content regressions.
  • Excluding AP/PA projection items, the extracted-accuracy gain over the base is +2.13 points (0.55 to 3.73) On and +1.36 (0.21 to 2.47) Off.
  • By source, the extracted-accuracy gain is +3.58 points on ReXVQA in both modes. On MIMIC-Ext-VQA (On +0.42, Off +1.17) and on Medical-Diff-VQA with thinking Off (−1.67) the intervals include zero.

The strict gain thus combines a format component, which B1 and B2 also obtain, and a content component that only the FCS reward obtains: +2.47 points On (0.98 to 3.99) and +1.57 Off (0.44 to 2.66) on extracted accuracy, and +3.00 On (1.44 to 4.55) for the second seed. The paper now reports strict and extracted accuracy side by side and separates the two (Section 4.2 and Appendix D.2).

AuthorsOpenReview ↗

Reply to Reviewer nrFW (2/2): W3 / Q3 — direct factuality evaluation of reports; Q4 — sensitivity to the penalties

W3 / Q3 — direct factuality evaluation of reports. We now evaluate unsupported and missing findings directly with CheXbert labels (positive classes) and RadGraph precision and recall, and we add GREEN. No model was trained on report generation, so these are transferred effects.

MIMIC-CXR findings, thinking OnFalse-finding rateMissed-finding rateGREEN
Qwen3.5-2B (base)48.9%38.2%0.301
GRPO, correctness reward (B1)47.3%37.8%0.293
GRPO, RadGraph-F1 reward (B2)49.3%34.7%0.294
VeraRad-2B (FCS reward)43.7%45.9%0.280
FCS reward, second seed44.4%40.8%0.288
FCS, entity+assertion only47.2%40.7%0.290
FCS, κ = 245.4%42.6%0.282
  • The base over-reports: 679 characters and 2.76 positive findings per report, against 407 and 2.28 in the reference. VeraRad produces 367 characters and 2.20 findings, lowers the false-finding rate from 48.90% to 43.69% and raises RadGraph precision from 0.221 to 0.271.
  • The cost is more omissions: the missed-finding rate rises from 38.17% to 45.88% and RadGraph recall falls from 0.338 to 0.259.
  • B1, B2 and the entity+assertion-only reward do not lower the false-finding rate; the second FCS seed reproduces the shift.
  • GREEN, which counts omissions and false findings alike, scores VeraRad below its base and below B2.

We therefore report the result as fewer unsupported findings with more omissions, an effect of the full FCS reward, and not as higher scores on every report metric. As a check on the metrics, a fixed three-sentence normal report scores 1/RadCliQ 1.594 on IU X-Ray, above every model, but 0.655 on MIMIC-CXR (VeraRad On: 0.792), so we base this analysis on MIMIC-CXR. Section 4.3 and Appendix D.3 of the revised PDF give all models, both thinking modes and paired intervals.

Q4 — sensitivity to the relative penalties. Beyond the scorer-level ablations in Figure 4, we trained two policies that each change one cost factor of VeraRad (rows 7 and 8 of the table above, and of the VQA table in our first comment):

  • Entity+assertion only (attribute terms removed): extracted VQA accuracy falls by 1.60 points On (−3.06 to −0.13) and 2.50 Off (−3.40 to −1.60), and the reduction in false findings largely disappears. The attribute costs matter for the learned policy.
  • κ = 2 (insertion cost doubled, τ unchanged): extracted accuracy changes by −0.60 (−2.02 to 0.85) On and −1.23 (−1.88 to −0.58) Off and stays above B2 in both modes (+2.00, 0.48 to 3.53 On; +0.83, 0.11 to 1.60 Off). Report metrics with thinking On are within the intervals of κ = 1 on all three datasets.

The learned policy therefore depends on the attribute costs and is robust to κ between 1 and 2. The costs can also be read as confidence thresholds, which Section 3.3 now states. Asserting a finding has lower expected cost than omitting it when its confidence exceeds κ/(1 + κ), which is 0.5 for κ = 1. Guessing an attribute beats omitting it when its confidence exceeds 1 − 1/[(1 + κ)(τ + (1 − τ)d)], where d is the distance of a wrong guess: 0.375 for an adjacent severity grade and 0.5 for nominal attributes at κ = 1, τ = 0.6. Since κ = 2 did not lower the false-finding rate further, we present these thresholds as properties of the scorer and not as a guarantee about the trained policy.

Thank you for the additional experiments and clarifications. I appreciate the effort the authors have put into addressing my concerns. My remaining reservation is mainly about the broader scientific contribution. While I find the proposed reward design useful and the results encouraging, I feel the work is more of a practical improvement than a deeper conceptual advance in clinical factuality. I will therefore maintain my original rating. Thanks again for the thoughtful discussion!

Dear Reviewer nrFW,

Thank you for the careful discussion and for recognizing the additional experiments. We respect your assessment, and we would like to briefly state what we see as the conceptual contribution, since it may be less visible behind the engineering.

First, the work treats clinical factuality as a set of typed commitments (what is asserted, denied, or left unstated, and with which attributes) rather than as a similarity score. This changes what a reward can express. The controlled comparison suggests why this matters: under the same budget, rewards based on answer correctness or on RadGraph overlap changed the answer format but not the clinical content, while the commitment-based reward was the only one that improved content.

Second, the asymmetric costs are not just tuning knobs. They define explicit decision rules for the policy: a finding should be asserted only when its confidence exceeds κ/(1 + κ), and an attribute should be specified only above a corresponding threshold (Section 3.3). The learned behavior matches this, with fewer unsupported findings at the price of more omissions, and that trade-off is now something one can set and reason about rather than an incidental side effect of the reward.

We will make this framing more explicit in the introduction and discussion of the final version. Thank you again for your time and constructive feedback.

Reviewer_uAu5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Fact-Commitment Space, a structured reward framework for improving factual consistency in chest radiography vision-language models through reinforcement learning. The proposed method represents clinical statements as structured commitments containing entities, assertion states, and attributes. A Clinical Fact Parser (CFP) extracts these commitments from free-form responses, followed by deterministic compilation and asymmetric matching to penalize omissions, unsupported findings, contradictions, and attribute errors. The authors also provide theoretical properties of the reward function and use it with GRPO to train VeraRad-2B. Experiments demonstrate improvements in clinical visual question answering and report generation, along with analyses of reward reliability and visual evidence utilization.

Critical strengths
  • The paper identifies an important limitation of existing aggregate rewards for clinical language generation, which do not explicitly distinguish between different types of factual errors. The proposed structured representation offers a reasonable way to address this issue.

  • The separation of fact parsing, deterministic compilation, and asymmetric error penalties makes the reward function relatively transparent. The theoretical analysis provides useful insights into the expected behavior of the proposed scoring mechanism under specified conditions.

  • The paper evaluates fact extraction, reward reliability, visual question answering, and report generation across multiple datasets. The controlled fact interventions and image-swap experiments provide additional evidence beyond conventional benchmark comparisons.

Critical weaknesses
  • The main experimental comparisons evaluate VeraRad-2B against pretrained models, but do not include matched-budget GRPO training with alternative reward functions. So it is unclear how much of the improvement comes from the proposed structured reward rather than reinforcement learning or task-specific post-training itself. This is important because the reward formulation is the central technical contribution of the paper.

  • The theoretical guarantees are limited to structured commitments rather than the complete reward pipeline. The theoretical properties in Section 3.3 assume correctly specified structured commitments. However, the actual reward depends on CFP predictions, which achieve only 61.22% entity-assertion recall and 35.10% bound-attribute F1 (Table 1). Furthermore, the learned rewards preserve only 72.78% of the reference pairwise orderings. These errors can directly affect GRPO optimization, and the paper does not sufficiently establish how robust the policy learning process is to such reward inaccuracies.

  • Although the model improves several report generation metrics, the generated reports are substantially shorter than those of the base model. This raises the question of whether the improvements reflect better factual accuracy or a more conservative generation strategy that avoids unsupported claims at the cost of omitting relevant findings.

Questions for authors
  • How much of the improvement can be attributed specifically to Fact-Commitment Space? Could the authors compare against matched-budget GRPO using simpler rewards or existing structured factuality metrics, while keeping the training data, backbone, and optimization settings fixed?

  • Given the relatively low attribute-level F1 and imperfect reward-ranking preservation, how sensitive is GRPO training to CFP errors? Would using human-verified or higher-quality parsed commitments for a controlled subset substantially change the learned policy or its downstream performance?

  • Does the substantial reduction in report length indicate fewer hallucinations, or does it also lead to more missing clinically relevant findings? How to know whether the proposed reward improves clinical factuality rather than primarily encouraging shorter responses.

4 comments
AuthorsOpenReview ↗

Reply to Reviewer uAu5 (1/2): W1 / Q1 — attribution to the reward; W2 / Q2 — guarantees versus the pipeline

Thank you for the constructive review. We answer your three questions in two comments, and each contains the numbers it relies on.

W1 / Q1 — attribution to the reward. We trained three matched models: GRPO with a format-tolerant correctness reward (B1), GRPO with a RadGraph-F1 reward (B2), and SFT on the gold answers, plus a second seed of the FCS reward and two ablations. Every trained model uses the same base (Qwen3.5-2B), the same 3,200 training items in the same order, and the same LoRA, sampling and thinking budgets; the GRPO models share the 800 updates and change only the reward or the seed. Intervals are 95% patient-cluster bootstrap intervals (10,000 resamples) on paired items.

Model (VQA-3000, strict / answer-extracted accuracy, %)Thinking OnThinking Off
Qwen3.5-2B (base)66.70 / 73.2752.93 / 68.20
SFT, same items70.47 / 71.0374.23 / 74.23
GRPO, correctness reward (B1)73.07 / 73.8366.53 / 67.23
GRPO, RadGraph-F1 reward (B2)72.63 / 73.1367.63 / 67.70
VeraRad-2B (FCS reward)75.53 / 75.7369.70 / 69.77
FCS reward, second seed75.97 / 76.2768.13 / 69.40
FCS, entity+assertion only73.20 / 74.1364.37 / 67.27
FCS, κ = 274.70 / 75.1367.80 / 68.53
  • Against other RL rewards. On extracted accuracy, neither B1 (+0.57 On, −0.97 Off) nor B2 (−0.13 On, −0.50 Off) differs from the base (all four intervals include zero); they improve the answer format only. The FCS reward improves extracted accuracy over the base by 2.47 points On (0.98 to 3.99) and 1.57 Off (0.44 to 2.66). It is above B1 by 1.90 points On (0.39 to 3.40) and 2.53 Off (1.71 to 3.35), and above B2 by 2.60 (1.03 to 4.15) and 2.07 (1.32 to 2.81). The second FCS seed reproduces this. Among RL rewards, the content gain is therefore attributable to the FCS reward.
  • Against SFT. SFT on the same items is stronger with thinking Off (+4.47, 3.21 to 5.72) and weaker with thinking On (−4.70, −6.57 to −2.78), where it also returns an empty report for 61% to 93% of studies, because its targets contain no reasoning text. The two methods are each stronger in one mode. FCS-GRPO keeps both modes usable, and its thinking-On accuracy is the highest of the matched models in either mode.

W2 / Q2 — the guarantees cover structured commitments, not the pipeline. We agree, and the paper now says so wherever the theorems are cited: they are statements about the scorer on structured inputs. Evidence on the full pipeline:

  • Table 1 and Figure 4 separate scorer behaviour from parser behaviour; 77.17% of GRPO advantage signs agree with the reward computed from human-verified commitments.
  • New policy-level test of the noisiest parser component. Attributes have the lowest parser F1 (35.10%). A policy trained with the entity+assertion-only reward is 1.60 points lower on extracted VQA accuracy On (−3.06 to −0.13) and 2.50 lower Off (−3.40 to −1.60), and loses most of the reduction in unsupported report findings (false-finding rate 47.2% vs 43.7%). The noisy attribute signal therefore helps the policy on net.
  • New reference rows (Appendix D.4), on the same 598 responses: entity–assertion F1 is 0.753 for our 2B projector, 0.322 for Qwen3.8-27B with the same prompt and schema, and 0.182 for RadGraph. We state two caveats: the human reference was revised from 27B pre-annotations, and RadGraph sees only the answer text and maps incompletely to our vocabulary.
  • Parser truncation during training. The frozen parser occasionally returns a truncated output for long answers, mostly early in training. We stop and resume from a checkpoint when this happens and never score such an output; Appendix B.2 reports the counts per run.

Training directly on human-verified commitments is not feasible, because the human panel covers 600 responses and not training rollouts; we list it as future work.

AuthorsOpenReview ↗

Reply to Reviewer uAu5 (2/2): W3 / Q3 — shorter reports: fewer hallucinations or more omissions?

Both, and the matched baselines show where the change comes from. Report generation was never trained (the policy saw only closed questions), so this is a transferred effect. We now measure the two error types directly with CheXbert labels (positive classes) and add GREEN.

MIMIC-CXR findings, thinking OnFalse-finding rateMissed-finding rateGREEN
Qwen3.5-2B (base)48.9%38.2%0.301
GRPO, correctness reward (B1)47.3%37.8%0.293
GRPO, RadGraph-F1 reward (B2)49.3%34.7%0.294
VeraRad-2B (FCS reward)43.7%45.9%0.280
FCS reward, second seed44.4%40.8%0.288
FCS, entity+assertion only47.2%40.7%0.290
FCS, κ = 245.4%42.6%0.282
  • Fewer unsupported findings. The false-finding rate falls from 48.90% to 43.69% and RadGraph precision rises from 0.221 to 0.271. The base over-reports: 679 characters and 2.76 positive findings per report, against 407 and 2.28 in the reference. VeraRad produces 367 characters and 2.20 findings.
  • More omissions. The missed-finding rate rises from 38.17% to 45.88% and RadGraph recall falls from 0.338 to 0.259.
  • Specific to the full FCS reward. B1 and B2 leave the base's behaviour largely in place, and B2's reports are longer than the reference (77.5 vs 60.8 words). The second FCS seed reproduces the shift; the entity+assertion-only reward mostly does not. κ = 2 gives a similar false-finding rate to κ = 1.

We describe this as a shift toward precision and not as uniformly better reports. GREEN, which counts omissions and false findings alike, scores VeraRad below its base and below B2 (table above). With thinking Off the shift is the same in direction (false/missed 43.0% / 57.1% against 48.9% / 45.0% for the base), and SFT matches or exceeds VeraRad on the MIMIC-CXR similarity metrics.

Two further checks are in the revised paper. A fixed three-sentence normal report, identical for every study, scores 1/RadCliQ 1.594 on IU X-Ray, above every model, but 0.655 on MIMIC-CXR, where VeraRad (On) scores 0.792; composite scores on IU X-Ray therefore reward normal-sounding text, and we base this analysis on MIMIC-CXR. CheXagent-2-3b shows the extreme of the same trade-off (RadGraph precision 0.345, recall 0.124), which is why we report both error rates and do not use length or precision alone as evidence. Section 4.3 and Appendix D.3 of the revised PDF give all models, both thinking modes and paired intervals.

Reviewer_uAu5OpenReview ↗

Official Comment by Reviewer uAu5

Thank you for your response and the additional experiments, which address some of my concerns. However, several concerns remain. Although the proposed FCS reward improves over the matched-budget GRPO baseline, the improvement is relatively modest. Considering the significant complexity introduced by a separately trained CFP and multiple manually specified penalty parameters, the benefits are not compelling. Also, the proposed reward still relies on imperfect fact extraction. It is still unclear how parsing errors affect reward estimation and policy optimization. For the report generation, the results indicate conservative reporting, which makes the practical benefits of the proposed reward less convincing.

Dear Reviewer uAu5,

Thank you for engaging with our response and for acknowledging the additional experiments. We briefly address the three remaining points.

Size of the gain relative to the added complexity. Under the same training data, backbone and budget, GRPO with a standard reward (answer correctness or RadGraph-F1) improves answer format but not answer content: neither changes extracted accuracy over the base model (all intervals include zero). The FCS reward is the only one of the three that improves content (+2.47 points with thinking On, +1.57 Off), and it is 1.90 to 2.60 points above both standard rewards with intervals excluding zero. A second training seed reproduces this. On complexity, the parser is trained once and used frozen as a fixed scorer during GRPO. The penalty weights were not tuned per experiment: doubling the insertion penalty (κ = 2) changes extracted accuracy by less than 1.3 points and the policy remains above the RadGraph-F1 reward in both modes.

How parsing errors affect optimization. We measured this where it drives the policy update: 77.17% of GRPO advantage signs computed with the parser agree with those computed from human-verified commitments. We also tested the noisiest parser component directly. Attributes have the lowest parser F1 (35.10%), yet removing them from the reward lowers extracted accuracy by 1.60 points (On) and 2.50 (Off) and removes most of the reduction in unsupported report findings. The parser is imperfect, but its signal helps optimization on net. Training on fully human-verified commitments would be the cleanest test; we list it as a limitation and future work.

Conservative reporting. We agree, and the revised paper describes the result in exactly these terms: a shift toward precision, with fewer unsupported findings (48.9% to 43.7%) and more omissions, rather than uniformly better reports. Two observations support its relevance. Report generation was never trained, so the shift is a transferred effect of training on closed questions. It is also specific to the full FCS reward; neither standard reward nor the variant without attribute terms produces it. We present this as evidence that the reward changes what the model commits to, not as a claim of better reports overall.

We hope this clarifies the remaining concerns. If so, we would be grateful if you would consider reflecting it in your assessment, and we are glad to answer further questions before the discussion closes.

Best regards, The Authors

Reviewer_Pws8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes Fact-Commitment Space (FCS), a reward for RL post-training of chest X-ray VLMs.

Method.

  • A 2B parser (CFP) converts free text into commitments: entity, assertion state (present/absent/uncertain), and attributes.
  • A deterministic compiler normalizes the commitments.
  • Response and reference commitments are matched one-to-one, with separate costs for omissions, insertions, and attribute errors.
  • Three properties of the scorer are proved.

Results. GRPO with this reward produces VeraRad-2B:

  • 75.53% vs. 66.70% strict accuracy on 3,000 VQA questions;
  • the best inverse RadCliQ on 2 of 3 report-generation datasets, without training on reports;
  • reports about 44% shorter than its base.

Image-swap probes and a single-case attribution study examine visual grounding.

Critical strengths
  1. Relevant problem, transparent reward. Making the cost of each clinical error type explicit is a good idea. The eligibility rule (same entity and same state) keeps a wrong finding from earning credit through correct attributes. Normalizing by the cost of an empty answer makes the reward easy to interpret.
  2. The reward is validated at the level RL needs. Reporting ordering concordance and GRPO advantage-sign agreement against human-derived rewards, and separating scorer behavior from parser behavior (Fig. 4), is a useful methodology that should be adopted more widely.
  3. Practical. A run takes about 16 GPU-hours, there is no large online judge, and the report metrics are independent of CFP.
  4. Candid reporting. The paper reports answer-extraction accuracy, both thinking modes, and a no-image audit, and it includes a 32-case casebook. The limitations section is honest. With thinking on, missing-image refusals rise from 15/32 to 26/32. That is an improvement over the 2B base, although larger untuned models in the casebook (Qwen3.5-4B, MedGemma) decline in nearly every no-image case.
Critical weaknesses

W1. The central claim is not isolated. Every comparison is VeraRad versus its untrained base or versus off-the-shelf models. On the single-choice and yes/no training items, the reward is close to answer correctness plus a penalty for extra content. GRPO with a simpler reward might give similar gains. Missing:

  • matched-budget GRPO with an exact-match or RadGraph-F1 reward;
  • SFT on the same 8,500 items;
  • policy-level ablations, e.g., an entity+assertion-only reward or a different κ.

The authors acknowledge this gap, but it is the core experiment. Closely related reward work should also be discussed: Miura et al. (NAACL 2021), OraPO (CVPR 2026), and RadVLM-GRPO (MIDL 2026).

W2. The VQA gain is mostly format compliance. Under the paper's own answer-extraction metric, the gain shrinks from +8.83 to +2.46 pp with thinking on, and from +16.77 to +1.57 pp with thinking off. Qwen3.5-4B leads that metric at 79.13%. About a quarter of the headline gain comes from AP/PA view classification, which is unrelated to the reward's mechanism. Some baselines look like protocol failures, e.g., MedGemma-27B at 14.17% on balanced 4-option ReXVQA.

W3. The report results do not show better factuality.

  • Metrics. On MIMIC-CXR, RadGraph F1 is tied (0.179 vs. 0.179) and RaTEScore is lower. With thinking off, both drop. The Q gain comes from BERTScore and SembScore, which are length- and style-sensitive. Reference lengths are not reported.
  • No-image reports. VeraRad (Off) scores higher with no image than with the correct image on IU X-Ray (1.446 vs. 1.342). It emits essentially the same normal template for all 32 no-image inputs.
  • Contradicting outputs. The casebook shows VeraRad (On) contradicting references. It writes "There is no pneumothorax" with a residual pneumothorax present (case 4), and a "top normal" (case 2) or "normal" (case 32) heart size despite cardiomegaly. The base also errs (e.g., both models call the heart normal in case 29), but these assertion reversals are the errors the reward targets.
  • No error-type breakdown. Without it, "fewer hallucinations" cannot be separated from "says less."

W4. The reward is only modestly faithful, and the attribute component is the weakest part.

  • CFP recall is 61% and attribute F1 is 35%.
  • About 23% of advantage signs disagree with the human-derived reward.
  • Removing attributes raises concordance (72.78% → 75.97%).
  • The text-path test covers only verbatim repetition, and even that changes the reward in 15 of 37 cases.
  • The Table 1 baseline is an untrained 2B model; the 27B teacher or a RadGraph extractor would be informative.
  • The human reference comes from a single annotator correcting 27B pre-annotations, with no qualifications or agreement reported.

W5. The cost design works against its goals, and the theory is thin.

  • Hedging. "Possible X" costs the same as contradicting a reference "X present," and more than omitting it.
  • Contradictions. Contradictions with the reference have no separate penalty, despite the abstract.
  • Clinical weighting. b_a defaults to 1.0 and no per-entity values are reported, so missing a pneumothorax may cost the same as missing a surgical clip.
  • Location. Location replacement leaves the reward unchanged (Fig. 4a).
  • Brevity bias. Omission costs only D, while hedges, wrong states, and the prediction-side penalties (U, S, H) can only add cost. Consistent with this, deleting attributes raises the reward by +0.35 on the structured path (Fig. 4a).
  • Theory. Theorem 3 is a special case of Theorem 4, and its κ ≥ 1 condition is unnecessary (the result holds for any κ > 0). Theorem 1 covers only the trivial zero-confidence limit. The implied threshold shows that guessing a severity grade pays at about 38% confidence when a wrong guess is off by one grade (50% for nominal attributes). Framing κ and τ as confidence targets (cf. Kalai et al., 2025) would be more informative.

W6. The grounding evidence is weak. The main case was selected post hoc, and the casebook does not support it in aggregate.

  • On constant or blurred inputs, VeraRad shifts toward the affirmative sentence relative to the base in 32/32 cases.
  • On all 13 target-absent cases, it is less reference-aligned than the base.
  • Over the 11 cardiomegaly cases, the image moves VeraRad less than the base (correct − no image: 1.25 vs. 1.77).
  • Localization is similar for both models (IG peak in box: 5/7 vs. 5/7).
  • Δ_specific is lower than the base on all three datasets with thinking off, and also on IU X-Ray with thinking on (0.027 vs. 0.084).

W7. Rigor and reproducibility.

  • There are no confidence intervals or repeated seeds; CheXpert Plus has 58 studies.
  • The training bank's sources and its patient-level overlap with the evaluation sets are not described, and one casebook case shares a patient with GRPO training.
  • Test-set quotas were "set during development comparisons."
  • The Reproducibility and Ethics statements are empty, the GRPO hyperparameters are missing, and no release is mentioned.
Questions for authors
  1. Can you report GRPO with an exact-match reward and with a RadGraph-F1 reward, plus SFT, all on the same data and at the same budget?
  2. How does a policy trained with an entity+assertion-only reward, or with a different κ, compare?
  3. What are the training bank's sources and task mix, and is it patient-disjoint from all evaluation sets?
  4. What are the false-finding and missed-finding rates for base vs. VeraRad reports (e.g., via GREEN categories), and what is the mean reference length?
  5. Why does location replacement not change the reward? What are O, the w_k values, and any per-entity b_a values?
  6. Why is "uncertain" vs. "present" priced like a contradiction rather than given partial credit?
  7. What are the missing GRPO hyperparameters, how variable are results across seeds, and will the code, registry, and CFP be released?
7 comments
AuthorsOpenReview ↗

Reply to Reviewer Pws8 (1/5): W1 / Q1 / Q2 — isolating the reward

Thank you for the detailed review. Your questions defined the experiments we ran in this period. The matched-budget table and the main paired differences are in the general response; the details you asked for follow.

B1 and B2 on VQA. Strict accuracy, B1 minus VeraRad: −2.47 (−3.98 to −0.93) On, −3.17 (−4.03 to −2.32) Off. Against the base on extracted accuracy, B1 is +0.57 (−0.87 to 2.02) On and −0.97 (−2.01 to 0.07) Off; B2 is −0.13 On and −0.50 Off, both including zero. So both rewards obtain the format gain and neither obtains a content gain. A strict exact-match version of B1 gave a zero reward to 255 of 256 sampled answers in a smoke test (31 of 32 groups had no signal), so we trained B1 with the format-tolerant rule and report the strict version as a diagnostic in the appendix.

B1 and B2 on reports (differences are baseline minus VeraRad on shared studies).

  • Thinking On, MIMIC-CXR: VeraRad is higher on 1/RadCliQ (B1 −0.017, −0.024 to −0.009; B2 −0.016, −0.024 to −0.007). B2 is higher on RadGraph-F1 (+0.007, 0.004 to 0.010) and RaTEScore (+0.012, 0.009 to 0.015); B1 is higher on RaTEScore (+0.009). On IU X-Ray, VeraRad is higher than B1 on all three metrics, and higher than B2 on RadGraph-F1 and RaTEScore.
  • Thinking Off, MIMIC-CXR: B2 is higher on RadGraph-F1 (+0.016, 0.012 to 0.020) and RaTEScore (+0.025, 0.021 to 0.028), with 1/RadCliQ inconclusive (+0.006, −0.002 to 0.015). B1 is higher on RadGraph-F1 (+0.010) and RaTEScore (+0.019). Both are higher than VeraRad on all three IU X-Ray metrics.
  • Mechanism: B2's reports are longer than the reference (77.5 vs 60.8 words on MIMIC-CXR, On), and its false/missed-finding rates with thinking Off (48.0% / 44.8%) match the base (48.9% / 45.0%), against 43.0% / 57.1% for VeraRad. A RadGraph-F1 reward keeps recall and does not reduce unsupported findings.

SFT. Strict accuracy, SFT minus VeraRad: +4.53 (3.28 to 5.78) Off, −5.07 (−6.93 to −3.13) On. On reports with thinking Off, SFT returns a non-empty report for every study and matches or exceeds VeraRad on MIMIC-CXR: 1/RadCliQ 0.804 vs 0.768 (+0.036, 0.027 to 0.046 on 2,181 shared studies), RadGraph-F1 +0.005 (0.001 to 0.009), RaTEScore +0.008 (0.005 to 0.012); false/missed-finding rates 41.6% / 55.5% vs 43.0% / 57.1%. With thinking On, SFT returns an empty report for 61% to 93% of studies; on the 835 MIMIC-CXR studies where it returns one, it is close to VeraRad on 1/RadCliQ (−0.008, −0.022 to 0.006) and lower on RadGraph-F1 (−0.012) and RaTEScore (−0.013). SFT is thus a strong option when thinking is disabled and loses the thinking mode; FCS-GRPO keeps both modes and reaches the highest VQA accuracy of the matched models with thinking On.

Entity+assertion-only reward (your Q2). Same seed, schedule and budget, with attributes removed from the matching and cost terms; this is the policy-level counterpart of the E+A column in Figure 4a. Strict accuracy is 2.33 points lower On (−3.81 to −0.86) and 5.33 lower Off (−6.35 to −4.34). On MIMIC-CXR reports with thinking On, it is lower than the full reward on 1/RadCliQ (−0.012, −0.020 to −0.004) with RadGraph-F1 unchanged, and its false-finding rate returns to 47.2% from 43.7%. With thinking Off it is higher on all three MIMIC-CXR metrics (1/RadCliQ +0.009, RadGraph-F1 +0.009, RaTEScore +0.014; intervals exclude zero), with false/missed-finding rates (45.7% / 50.3%) between the full reward and the base. The attribute terms therefore improve closed-VQA accuracy and account for most of the reduction in unsupported report findings.

κ = 2 (your Q2). Strict accuracy changes by −0.83 (−2.26 to 0.60) On and −1.90 (−2.59 to −1.23) Off. The κ = 2 policy stays above B2 on extracted accuracy in both modes (+2.00, 0.48 to 3.53 On; +0.83, 0.11 to 1.60 Off) and above B1 with thinking Off (+1.30, 0.52 to 2.11). On reports with thinking On, its differences from κ = 1 are within the intervals on all three datasets. Its MIMIC-CXR false/missed-finding rates are 45.4% / 42.6% On and 44.5% / 54.2% Off, against 43.7% / 45.9% and 43.0% / 57.1% for κ = 1.

Related reward work (now in Section 2). Miura et al. (2021) optimize report generation with self-critical RL on an entity F-score and an NLI-filtered variant. RadVLM-GRPO applies GRPO to an 8B model with a −RadCliQ reward and reports that RadGraph-F1 alone as the reward produces very short reports. OraPO extracts atomic facts with GPT-4.1, maps them to the 14 CheXpert labels and rewards a recall-weighted F-score. These rewards score entity- or label-level overlap; none prices assertion state and attributes as separate costs, and OraPO needs a proprietary LLM call per completion. B2 tests this family of rewards directly under our budget.

AuthorsOpenReview ↗

Reply to Reviewer Pws8 (2/5): W2 — format versus content; W3 / Q4 — report errors

W2. We agree that strict accuracy mixes format with clinical judgment, and we now report strict and extracted accuracy side by side. A per-item transition analysis from the base to VeraRad on the 3,000 items separates the two.

  • Thinking On: 151 items are format repairs (the base answer was correct once extracted), 300 are content repairs, and 186 go the other way (1 format, 185 content), for a net gain of 265.
  • Thinking Off: 377 format repairs, 166 content repairs and 40 content regressions.
  • Excluding the AP/PA projection items, the extracted-accuracy gain remains: +2.13 points (0.55 to 3.73) On and +1.36 (0.21 to 2.47) Off.

The gain therefore has a format component and a content component: on all items, extracted accuracy rises by 2.47 points On (0.98 to 3.99) and 1.57 Off (0.44 to 2.66). The matched baselines show that the content component is specific to the FCS reward: B1 and B2 obtain the format gain with no change in extracted accuracy. The paper now reports the two components separately. We agree AP/PA is unrelated to the reward mechanism. Those 270 items are the native view category of Medical-Diff-VQA: that source contributes 600 items with one item per patient, its temporal categories need a verified pair of prior and current reports, and the patient-disjoint pool supported 300 temporal and 30 basic items, so the remaining slots were filled with view items.

MedGemma-27B. In a sample of 20 of its ReXVQA errors, 13 contained the correct answer in an extractable form, 4 a wrong answer and 3 no extractable answer, so its 14.17% mostly reflects not producing the required single label. Table 2 now shows its extracted accuracy (65.03% overall) next to strict accuracy. We also re-ran all 1,200 ReXVQA items through its official chat template and system role. This did not help: strict accuracy fell to 8.92% (−5.25, −7.39 to −3.09) and extracted accuracy from 69.83% to 57.92% (−11.92, −15.09 to −8.78). Both protocols are in the appendix.

W3 / Q4. We agree that a composite score and shorter length do not establish better factuality, and we replaced that argument with a direct error decomposition. Against the base on MIMIC-CXR with thinking On, 1/RadCliQ is +0.053 (0.045 to 0.061), RadGraph-F1 is unchanged (+0.0002, −0.003 to 0.003) and RaTEScore is lower (−0.011, −0.013 to −0.008). With thinking Off, RadGraph-F1 and RaTEScore are lower on MIMIC-CXR and all three metrics are lower on IU X-Ray. CheXpert Plus (N = 58) is inconclusive.

MIMIC-CXRFalse-finding rateMissed-finding ratePositive findings / reportRadGraph P / RLength (chars)
Qwen3.5-2B (On)48.90%38.17%2.760.221 / 0.338679
VeraRad-2B (On)43.69%45.88%2.200.271 / 0.259367
Qwen3.5-2B (Off)48.85%44.95%2.460.231 / 0.343640
VeraRad-2B (Off)43.01%57.13%1.720.267 / 0.248347
Reference——2.28—407

The base over-reports. VeraRad makes about 5 points fewer false findings and, with thinking On, its findings per report and length are close to the reference; it makes 8 (On) to 12 (Off) points more missed findings. This is a shift toward precision. B1, B2 and the entity+assertion-only reward do not produce it (general response), so it is an effect of the full FCS reward. CheXagent-2-3b shows the extreme of the same trade-off (RadGraph precision 0.345, recall 0.124, 180 characters), which is why the paper now reports both rates and does not use length or precision alone as evidence.

GREEN. VeraRad is below its base: MIMIC-CXR 0.280 vs 0.301 (On) and 0.272 vs 0.301 (Off); IU X-Ray 0.702 vs 0.738 (On) and 0.695 vs 0.766 (Off). The matched baselines, which keep higher recall, sit between or above (MIMIC-CXR Off: SFT 0.290, B1 0.289, B2 0.298).

Normal-template floor. A fixed three-sentence normal report, identical for every study and written before scoring, scores 1/RadCliQ 1.594 on IU X-Ray, above every model in Table 4a (highest: 1.529), but only 0.655 on MIMIC-CXR, where VeraRad (On) scores 0.792. Under GREEN the template scores 0.146 on MIMIC-CXR and 0.649 on IU X-Ray, below VeraRad and its base. IU X-Ray composite scores therefore strongly reward normal-sounding text, which explains how VeraRad (Off) without an image can score above its correct-image score (1.446 vs 1.342). This shows limited image dependence of the Off policy on IU X-Ray, and Section 4.3 now says so.

Casebook. We agree that cases 2, 4 and 32 are assertion reversals of the kind the reward targets, and we now discuss them explicitly.

AuthorsOpenReview ↗

Reply to Reviewer Pws8 (3/5): W4 — reward faithfulness; W5 — contradictions and theory

W4. We agree that attributes are the weakest part of the parser (attribute F1 35.10%) and that removing them raises offline ordering concordance (72.78% to 75.97%); the main text now says so. The new entity+assertion-only policy tests what this means for learning. When attributes are removed from the reward during GRPO, extracted VQA accuracy falls by 1.60 points On (−3.06 to −0.13) and 2.50 Off (−3.40 to −1.60), and the reduction in unsupported report findings largely disappears (47.2% vs 43.7%). The attribute terms are therefore a net positive training signal despite their parsing noise.

Appendix D.4 now gives two reference rows next to CFP on the same responses (598 of 600; Qwen3.8-27B reached the length limit on 2, which we keep as failures and did not retry). Entity–assertion F1 is 0.753 for our 2B projector, 0.322 for Qwen3.8-27B with the same prompt and schema, and 0.182 for RadGraph; complete-tuple F1 is 0.317 vs 0.122 (RadGraph outputs no attributes). Two caveats are stated with the table. The human reference was produced by revising pre-annotations from a multi-stage 27B pipeline, so it shares an origin with the 27B row. RadGraph sees only the answer text, without question or options, and 46% of its observation entities could not be mapped unambiguously to our vocabulary and count as false positives (restricted to mapped entities, its F1 is 0.247); its row does not measure its general parsing ability.

The reference was revised by one of the authors, a medical-AI researcher who designed the schema and is not a practising radiologist. The task is a text-to-schema mapping: the reviewer checks that each entity, assertion state and attribute stated in the response text is entered correctly in the FCS schema, and does not read images or make diagnoses. The revision was substantial: 35.2% of the pre-annotated answer facts were not kept verbatim, 45.3% of the final facts are not in the pre-annotation, and 514 of 600 responses changed. Inter-annotator agreement was not measured; we list the single reviewer and the shared 27B origin as limitations. Verbatim repetition changing the reward in 15 of 37 cases is a parser failure mode, and we now describe it as one.

W5 — contradictions. You are right. H counts only contradictions within one response. A contradiction with the reference is priced through the eligibility rule as omission plus insertion, D(g) + I(p) = (1 + κ)b_a, which is strictly worse than omitting the finding (cost b_a). We reworded the abstract and Section 3.

Theorem 3 and κ ≥ 1. We agree on both counts. A reversed state has no eligible edge, so the prediction equals the empty prediction plus one unsupported commitment and Theorem 4 applies for every κ > 0. Theorem 3 is now a corollary without the κ ≥ 1 condition (Corollary 4 in the revised PDF, following the insertion result, which is now Theorem 3).

Theorem 1 and confidence targets. We adopt this framing; it makes the incentive structure clearer. For a matched pair, guessing an attribute with confidence p has lower expected cost than omitting it iff p > t_k(d) = 1 − 1/[(1 + κ)(τ_k + (1 − τ_k)d)]. With κ = 1 and τ_k = 0.6, t = 0.375 for an off-by-one severity grade (d = 0.5) and t = 0.5 for nominal attributes (d = 1). At the entity level, asserting a finding beats omitting it iff p > κ/(1 + κ) = 0.5. Section 3.3 now states these thresholds and relates them to Kalai et al. (2025); Theorem 1 is the p → 0 case. Because the κ = 2 policy did not produce fewer false findings than κ = 1, we present the thresholds as the incentives of the scorer and not as a property guaranteed for the trained policy.

Comments on the paper

1 comment
AuthorsOpenReview ↗

A brief summary for the Area Chair

Dear Area Chair,

We thank the reviewers for a constructive discussion. All three agreed with the starting point of the paper: clinical factual errors are not equally harmful, and a reward should price them explicitly.

The main shared concern was whether the gains come from the reward or from RL post-training itself. We ran the matched-budget comparison all three reviewers asked for. With data, backbone and budget fixed, a correctness reward and a RadGraph-F1 reward improved answer format but not answer content. The Fact-Commitment Space reward improved content, 1.9 to 2.6 points above both, and a second seed reproduced the result. In their latest comment, Pws8 noted that these baselines, the error analysis and the corrected theory address most of their concerns. We agree with Pws8's four remaining points: three are corrected in the revised PDF uploaded today, and the casebook statistics will be added in the final version.

The question behind the paper is general: how should a reward treat each kind of error? We represent outputs as typed commitments, recording what is asserted, denied or left unstated, and give each error type its own cost. These costs define confidence thresholds above which committing to a finding pays off under the reward, which connects reward design to the view of hallucination as an incentive problem (Kalai et al., 2025). The reward is deterministic, auditable and needs no online LLM judge. We tested it in chest radiography; the same approach applies to other domains whose outputs decompose into typed claims, given a validated vocabulary and parser.

We respect nrFW's view that the work is mainly a practical advance, and we agree with uAu5 that the resulting reports are more conservative: fewer unsupported findings, with more omissions. This trade-off follows from the costs the reward states.

We will release the scorer, the parser, the VeraRad-2B checkpoints and the evaluation scripts, together with the checks we used to validate the reward before training, so that others can reuse or audit it.

Thank you for your time.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.