acceptodds
Under review as a conference paper at ICLR 2027

Anchoring the Drift: Gated-Anchor On-Policy Distillation for Faithful Radiology Report Generation

Abstract

Small vision-language models are attractive for deploying clinical report generation on local hardware, yet they need supervision from a stronger model. On-policy distillation supplies dense token-level supervision on the student's own rollouts, and a growing family of methods improves it by reshaping the teacher Kullback-Leibler term. Every repair in that family acts on the teacher term itself, so what it can deliver stays bounded by what the teacher's own distribution contains, and wherever the teacher is unreliable the student inherits that failure; on hard studies a second failure compounds it, since the student drifts and leaves the teacher scoring an already-wrong prefix. We present GA-OPD (Gated-Anchor On-Policy Distillation), which answers both with one construction. A ground-truth anchor, a teacher-forced cross-entropy term on the reference report, never passes through the teacher and therefore stays valid where the teacher's own distribution is misaligned with the reference; a per-token drift gate routes it in by attenuating the on-policy term exactly where the student prefix has diverged from the reference. The mixing rule is what carries the result. In the regime where the teacher fails, the same anchor entering at a constant weight reaches 7.22 RadGraph and the drift-routed anchor reaches 19.36, against 7.15 for plain on-policy distillation; GA-OPD exceeds three state-of-the-art variants of this family there by 1.92 to 12.12 points, and two of them collapse together with plain on-policy distillation. Stratifying by the teacher's own per-sample faithfulness, an axis independent of the compared arms, the advantage grows with teacher failure. On the full three-corpus chest X-ray test set of 4,676 studies the same 4B student attains the highest CheXbert micro-F1 among all arms, 59.34, matching a frozen 30B medical teacher while serving on a single accelerator.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
on-policy distillation, knowledge distillation, radiology report generation, medical vision-language models, chest X-ray, faithfulness, small language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LYKiOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This is a well-scoped and carefully executed study of a practically important problem, and the proposed Gated-Anchor On-Policy Distillation (GA-OPD) is a clean, well-motivated construction whose core mechanism is convincingly isolated by the ablations. The empirical work is extensive and internally consistent, the transparency about limitations is exemplary, and the deployment claims are concrete and directly relevant to the stated motivation. The main themes that need work are the reliance on a single seed, the collapse of the REOPOLD baseline, the controlled teacher-instruction mismatch, and the reverse-direction estimator difference; these are all disclosed honestly, but they deserve deeper analysis or mitigation rather than mere acknowledgment. In particular, the single-seed design limits the strength of the empirical claims, and the instruction mismatch raises questions about whether the gains are partly an artifact of the evaluation setup. The mathematical analysis is correct as far as it goes, but the worst-case O(1/t)O(1/t) bound could be complemented by a discussion of typical-case behavior or empirical drift dynamics.

Critical strengths

I found the following strengths in the paper:

  • Core construction. The core construction is simple, well-motivated, and cleanly isolated by the ablation: the anchor never queries the teacher, the drift gate sets the balance as a function of measured drift, and §4.5 shows the routing rule is what carries the result (7.22 ungated vs 19.36 routed). This directly supports the paper’s central claim that “our delta is not the mixture but the mixing rule” (§2, p3; §3.5, p5; §4.5).

  • Correctness of the Eq. 5 bound. The only proof-like result, the Eq. 5 bound, is correct: for t≥Mt \geq M the multiset intersection is at most MM, so αt≤2M/(t+M)\alpha_t \leq 2M/(t+M), with equality iff the rollout prefix contains every reference token (multiset inclusion). The O(1/t)O(1/t) decay consequence follows, matching the statement that “a rollout that keeps generating therefore loses its teacher-side supervision at rate O(1/t)O(1/t)” (§3.3, p5).

  • Internal consistency of the empirical work. The empirical work is extensive and internally consistent: all margin claims in §4.2–§4.5 check out within rounding against Tables 1–8, and the aggregate RadGraph/RaTEScore values reproduce exactly as weighted means of the split tables (e.g., GA-OPD RadGraph (3852⋅22.78+590⋅28.26+234⋅19.36)/4676=23.30(3852 \cdot 22.78 + 590 \cdot 28.26 + 234 \cdot 19.36)/4676 = 23.30, matching Table 1). The note in B.8 (p18) about rounding is a helpful transparency aid, and the checkpoint-selection rule (“every checkpoint is selected on the held-out validation set by peak RadGraph”, p6) makes the consistency check meaningful.

  • Exemplary transparency about limitations. The single-seed disclosure, the REOPOLD collapse, the controlled teacher-instruction mismatch, the losses on teacher-easy studies, and the reverse-direction estimator difference are all stated explicitly rather than hidden. The statement “each arm corresponds to a single training run with a fixed seed” and “we state this explicitly because the two are often conflated” (B.4, p16) are exactly the kind of candor that makes a paper trustworthy.

  • Thorough application-domain evaluation. The clinical metrics (CheXbert, RadGraph, RaTEScore) are complemented by text statistics (words, ≥500\geq 500w, rep-4, uniq., dist-2) that directly diagnose the length-inflation pathology, and the case analysis in Appendix C grounds the failure modes in verbatim examples such as “the teacher misses the catheter and invents a lung mass” (Figure 1 caption, p2; CheXbert micro-F1 over the 5-label and 14-label sets, p7).

  • Concrete deployment claim. The deployment claim is concrete and directly relevant to the motivation: the student’s weights occupy 8.3 GB and serve on one accelerator at 8.8 GiB, against 58 GB and two accelerators at 29.4 GiB each for the teacher (§4.3, p8). This makes the practical benefit tangible rather than abstract.

  • Captions and cross-references. All 8 table captions and 4 figure captions match their contents (Figure 1 caption values vs teacher/EOPD outputs; Figure 3a values vs Table 2; Table 6 caption verified on all six metrics), and every figure/table/section cross-reference resolves with no “Figure ??” (e.g., “Table 2 gives the split as a whole”, Figure 1 caption, p2).

  • Correct α1\alpha_1 property. The α1\alpha_1 property is correct: for length-1 prefixes, ov=2⋅min⁡(c,c)/(1+1)=1\mathrm{ov} = 2 \cdot \min(c,c)/(1+1) = 1 if the tokens match and 00 otherwise, exactly as stated (“α1\alpha_1 is 1 when y1=r1y_1 = r_1 and 0 otherwise”, §3.2, p4).

  • Accurate harmonic-mean claim. The harmonic-mean claim for Eq. 3 is accurate: with I=∑wmin⁡(cu(w),cv(w))I = \sum_w \min(c_u(w), c_v(w)), the two directional overlap rates are I/∣u∣I/|u| and I/∣v∣I/|v|, whose harmonic mean is 2I/(∣u∣+∣v∣)2I/(|u|+|v|) (§3.2, p4, Eq. 3).

  • Dimensionally consistent equations. All equations are dimensionless and type-consistent; the overlap ov(u,v)∈[0,1]\mathrm{ov}(u,v) \in [0,1] is correctly bounded for non-empty sequences and the clamp in Eq. 7 is well-defined for s>0s > 0 (Eq. 3, §3.2, p4).

  • Pinned metric versions. Metric implementations are pinned to specific versions (f1chexbert 0.0.2, radgraph 0.1.18, RaTEScore 0.6.0), which is a genuine reproducibility asset (B.6, p16).

  • Code release and full specification. A code-release statement is present (“We will release the code upon publication”, Reproducibility Statement, p10), and the teacher-instruction mismatch is fully specified with the exact instruction text in B.9.

  • Novelty of the construction. The construction itself — a reference cross-entropy anchor gated by bag-of-tokens prefix overlap — appears novel relative to the cited literature, reinforcing the claim that “our delta is not the mixture but the mixing rule” (p3). This is a promising direction that we hope to see explored further in future work.

Critical weaknesses

I appreciate the careful experimental design and the honest reporting of the residualized decile analyses. However, I identified the following problems:

  • Qualify the headline teacher-failure result as a controlled manipulation. The striking comparison (19.36 vs 7.15) is obtained under a deliberately constructed regime in which the teacher receives a different task instruction on CheXpert (B.9, p18: “we therefore replace the instruction the teacher receives with a different task specification”). The abstract’s phrase “In the regime where the teacher fails” and §4.2’s first sentence (“the frozen teacher is itself unreliable”) omit this qualifier. We suggest carrying “under a controlled teacher-instruction mismatch” into both the abstract and the first sentence of §4.2, so readers do not infer that the teacher failure arises naturally from the data.

  • Clarify the teacher comparison in the CheXbert claim. The statement that GA-OPD “exceeds the frozen 30B teacher on both” CheXbert sets (§4.3, p8) is partly an artifact of the manipulation: the teacher is evaluated under the same mismatched instruction that depresses its CheXpert scores (30.33/16.16, Table 6). Because the teacher receives this instruction both when it scores rollouts in training and when it is evaluated (B.9, p18), we suggest either explicitly stating that the teacher comparison is made under the mismatched instruction, or reporting the teacher’s unmanipulated CheXpert scores alongside the manipulated ones. This will make the comparison fair and transparent.

  • Reconcile the Conclusion’s “collapse” claim with the REOPOLD exception. The Conclusion states that “methods that reshape the teacher term collapse together with plain OPD” (p9), but the paper’s own REOPOLD result (17.44 RadGraph on CheXpert vs plain OPD 7.15) contradicts this, and §4.2 itself acknowledges that “REOPOLD, the one baseline carrying a reward floor, is the exception and holds” (p7). We suggest adding the reward-floor exception to the Conclusion sentence, e.g., “methods that reshape the teacher term collapse together with plain OPD, with the exception of the reward-floor baseline REOPOLD, which holds.”

  • Qualify “negligible” in the Conclusion’s insurance metaphor. The Conclusion’s “conditional insurance, decisive where the teacher cannot help and negligible where it can” (p9) is contradicted on MIMIC-CXR CheXbert-5, where GA-OPD gains +3.43+3.43 over plain OPD (60.07 vs 56.64, Table 4) even though the teacher is reliable there. We suggest qualifying “negligible” by metric, or clarifying that the phrase refers to the gate’s attenuation (αt\alpha_t near 1) rather than to the anchor’s effect on the final metric. This will prevent an over-generalized reading.

  • Scope the “design rule” to the studied setting. The Conclusion’s “We read this as a design rule” and the accompanying claim that “where a distillation teacher is unreliable on part of the input distribution, the remedy is not a better-shaped divergence but a supervision source the teacher does not provide” (p9) generalize beyond the evidence: one task (chest X-ray reporting), one student (Qwen3-VL-4B), one teacher (Hulu-Med-30A3), one deliberately manipulated failure regime, and one natural stratification where the MIMIC-CXR residualized decile contrast (+1.66†+1.66^\dagger) does not separate from zero. We suggest softening to “a design rule for this setting” or adding a scope paragraph that lists these boundary conditions.

  • Align the abstract’s “advantage grows” claim with the non-significant decile contrast. The abstract’s “the advantage grows with teacher failure” (p1) glosses the non-significant MIMIC-only decile contrast (+1.66†+1.66^\dagger, Table 3). §4.4 is appropriately careful (“from +1.91+1.91 within MIMIC-CXR … to +5.91+5.91 and +9.89+9.89”), but the abstract and Conclusion are not. We suggest adding the qualifier that within MIMIC-CXR the decile contrast does not separate from zero, so the growth claim is restricted to the cross-dataset comparisons where it is supported.

  • Standardize notation and terminology throughout. We noticed several naming inconsistencies that should be unified under the LME Niemann convention (one name per concept). Specifically: (i) Algorithm 1 uses both “is sft” and “is_sft” (p15, line 9); (ii) the anchor is referred to as “ground-truth anchor,” “reference anchor,” and “reference objective” (Introduction, p2); (iii) supervised fine-tuning appears in at least three forms (e.g., “Supervised FT” in Table 2, p7, and other phrasings); (iv) “topk” and “top-k” are both used; and (v) Algorithm 1 conflates rollout and reference token indices under one symbol tt. We suggest standardizing each of these across the whole document, not only at the listed locations, to avoid ambiguity for readers who rely on precise notation.

  1. Literature & Related Work

We appreciate the careful positioning of your work within the on-policy distillation (OPD) literature. The taxonomy of fixed-weight mixtures, the explicit discussion of failure modes, and the honest disclosure of the REOPOLD collapse are valuable. The following suggestions are intended to strengthen the related-work section and sharpen the novelty claims.

Major suggestions

  • Closest prior mechanism missing. Step-Level On-Policy Distillation (Sun et al., 2026, arXiv:2608.16333) explicitly interpolates between OPD and SFT and is the closest prior mechanism to your anchor. Your “mixing rule, not the mixture” framing (p.3) is positioned only against fixed-weight mixtures (Hinton et al.; Kim & Rush; Wen et al.). Please cite and discuss Sun et al. in Related Work, and sharpen the novelty claim by contrasting your mixing rule with their step-level interpolation.

  • External baselines absent. MAIRA-2, CheXagent, LLaVA-Rad, EditGRPO, and HiMed-RL are cited in Related Work but absent from Table 1. The abstract’s “highest CheXbert micro-F1 among all arms” (p.1) is therefore only among your own arms. Please add at least one external baseline, or explicitly scope the claim to “among the evaluated distillation and supervised arms.”

Minor suggestions

  • REOPOLD comparison fragility. The REOPOLD comparison is against a 20-step checkpoint of a collapsed run: the run entered an absorbing state in its second phase (rollout perplexity fell to one, mean response length dropped from 9696 to ∼6\sim 6 tokens), and the selection rule places its best checkpoint at step 20 (B.1, p.15). The headline margin of 1.921.92 is therefore fragile. Keep the disclosure, and consider reporting the first-phase trajectory more prominently.

  • Mis-attribution of Wang et al. 2025. Wang et al. (arXiv:2512.02710) is cited among methods that “rewrit[e] the divergence as a policy gradient with a clipped reward” (p.2), but it is actually HiMed-RL, a hierarchical reward-learning framework for medical report generation — not an OPD divergence rewrite. Please move the citation to the RL-for-reporting sentence (with Zhang et al. 2025a; Miura et al. 2020).

  • Mis-characterization of Armandpour et al. 2026. Armandpour et al. (arXiv:2605.10889) is cited in “surveys collect both the failures and the repairs (Zhang, 2026; Armandpour et al., 2026)” (p.3), but it is a training-free diagnostic framework, not a survey. Zhang 2026 is confirmed a survey; please cite Armandpour et al. as a diagnostic/analysis work.

  • External baselines lack external validation. The three external baselines are re-implementations that cannot be diffed against the original papers’ numbers: the original papers report reasoning-benchmark metrics (Pass@8\mathrm{Pass@8}, Avg@32\mathrm{Avg@32}), none report radiology metrics, so the re-implementations carry no external validation. Please state explicitly that no external cross-check is possible.

  • “State-of-the-art” label unsupported. The “state-of-the-art” label for the three baselines (Revisiting-OPD, EOPD, REOPOLD) is asserted without evidence: they are 2026 preprints evaluated on reasoning benchmarks, and their SOTA status within the OPD family is not established. Please soften the label or provide evidence.

  • CheXbench missing. CheXbench — the standard benchmark for systematic evaluation of chest X-ray interpretation across 8 clinically relevant tasks, introduced with CheXagent (Chen et al. 2024) — is not cited or used, even though CheXagent is cited in Related Work. Please consider adding it or at least citing it.

  1. Mathematical Correctness

The mathematical core of the paper is sound and carefully presented in several places — the clamping discussion in B.6, the acknowledgment of the estimator confound in B.3, and the explicit treatment of the top-k support are all welcome. The suggestions below are intended to tighten the formal presentation and remove ambiguities that could obscure the contributions.

  • Eq. 1 is not a true KL divergence as written. The weights πT(v∣t)\pi_T(v|t) are summed over KtK_t without renormalization, so ∑v∈KtπT(v∣t)≤1\sum_{v \in K_t} \pi_T(v|t) \leq 1. The Introduction itself describes the prior family as "restricting the divergence to the teacher's top-k support and renormalizing it," so the equation omits the renormalization that the paper attributes to that family. Consequently, the "Plain OPD" baseline is a truncated, non-renormalized variant, not the original full-KL OPD of Agarwal et al. (2023). Please either renormalize πT\pi_T over KtK_t in Eq. 1, or state explicitly that the baseline is a truncated variant and justify the name. (§3.1, p4; Introduction, p2)

  • Eq. 1 is undefined when πS(v∣t)=0\pi_S(v|t) = 0 for v∈Ktv \in K_t. The log ratio log⁡(πT/πS)\log(\pi_T/\pi_S) has no stated support assumption. The B.6 clamp ("the per-token loss is clamped at 10.0 and teacher log-probabilities at −10.0-10.0 for numerical stability") addresses numerics, not the mathematical expression. Please state the assumption πS(v∣t)>0\pi_S(v|t) > 0 on KtK_t or define the 0⋅log⁡(0/0):=00 \cdot \log(0/0) := 0 convention. (Eq. 1, §3.1, p4; B.6, p16)

  • Eq. 3 is undefined (0/00/0) for two empty sequences. In the only use (Eq. 4), t≥1t \geq 1 and M≥1M \geq 1, so this cannot occur in practice; a brief note to that effect would make the definition complete. (Eq. 3, §3.2, p4)

  • The "ceiling" argument is a tautology, not a formal theorem. "All information carried by ℓopd\ell^{\mathrm{opd}} is bounded by the teacher" — the loss is a function of the teacher's distribution — and the paper does not prove the anchor is sufficient to overcome the ceiling nor that the drift gate is necessary. The narrative is coherent but informal; please consider stating it as a heuristic or intuition rather than a formal bound. (§3.2, p4)

  • The O(1)O(1)-per-token gate complexity claim cannot be verified from the text. The claim is plausible with an incremental multiset-intersection data structure, but the data structure is not specified. Please provide a brief description or a reference. (§3.2, p4)

  • The drift gate includes the token being predicted. "The gate applied at the position that predicts yty_t is computed from prefixes that include yty_t," so αt\alpha_t is not a pure prefix-drift measure — a high gate can result from sampling a reference token at the current position even after a drifted prefix. Please acknowledge this in the design discussion. (§3.2, p5)

  • "The overlap itself is order-free by design" is only partially true. The overlap is order-free within each prefix, but the prefix boundary y≤ty_{\leq t} vs r≤min⁡(t,M)r_{\leq \min(t,M)} is order-dependent — a reference finding stated early in the reference but late in the rollout is penalized at intermediate positions (the paper's own C.2 examples show this). Please qualify the claim. (§3.3, p5)

  • "As αt→0\alpha_t \to 0 the update at that token is reference supervision alone" is loose. The anchor is a separate sum over reference tokens, so "at that token" conflates the rollout position with the reference position. The statement is true if gtg_t is bounded and the on-policy term vanishes, but the wording should be tightened. Also, please check the Konjunktiv convention (state results, avoid could/should/might) throughout the whole document rather than only at the places listed — the individual occurrences are not repeated here. (§3.3, p5)

  • Argmax tie-breaking in Eq. 6 is unspecified. If multiple tokens tie at the maximum of πT(⋅∣t)\pi_T(\cdot|t), vt∗v_t^* and mtm_t are not well-defined. This is measure-zero in floating point and minor, but a one-line tie-breaking convention would make the definition complete. (Eq. 6, §3.4, p5)

  • The teachability "lead" is measured on the teacher's preferred token only. The teacher could hold a positive margin on its own preferred token while the student is more confident on a different token. The paper acknowledges the component is adopted from prior work, so this is a heuristic, not a hidden assumption — but please make that explicit. (§3.4, p5)

  • The teachability scale ss is named in §3.4 but its value (0.3) is deferred to B.6. Please state the value at first use. (§3.4, p5; B.6, p16)

  • Eq. 8's expectation is not the object actually optimized. αt\alpha_t and gtg_t depend on the sampled yy, and the paper states the gates "enter the objective as a constant, so no gradient flows through it and the rollout expectation is not differentiated through." This is the standard on-policy sample approximation and should be stated as such rather than presenting Eq. 8 as the exact objective. (§3.2–3.3, pp4–5; Eq. 8, §3.5, p5)

  • The batch construction is internally inconsistent. "The trainer consumes the first NN" of 2N2N samples cannot yield "16 rollouts and their 16 references per step" (32 samples) at "the same batch of 32 sequences"; Algorithm 1 line 7 says "keep the first NN of the 2N2N emitted samples." Please state the interleaving explicitly, e.g., "the trainer consumes all 2N2N samples, NN rollouts and NN references." (§3.5, p5; pp5–6)

  • "At β=1\beta = 1 the anchor's share of the objective for one study is exactly M/(M+T)M/(M+T)" holds for token counts only. The on-policy terms are gated (αtgt≤1\alpha_t g_t \leq 1) and differently scaled (truncated log-ratio vs cross-entropy), so the loss-value share is not fixed. Please rephrase as "the anchor's token share." (§3.5, p6)

  • Eq. 8 with an empty rollout (T=0T = 0) or empty reference (M=0M = 0) would contain an empty sum. The paper does not discuss this; not triggered by the data, but a sentence clarifying the convention would be helpful. (Eq. 8, §3.5, p5)

  • The reverse-direction robustness check is confounded. "The plain arm uses a sampled estimator, while the anchored arm uses a normalized top-k form," so the two runs differ in both divergence direction and estimator. The paper acknowledges this (B.3), but the direction-robustness claim is weaker than it appears. Please state this limitation more prominently. (B.3, p16)

  • B.8's claim conflicts with Table 3's printed markers. The claim that "residualizing the axis moves two MIMIC-CXR contrasts from intervals containing zero to intervals excluding it" conflicts with the printed †\dagger markers, which show only one such move (drift+anchor tercile: +0.63†→+1.10+0.63^{\dagger} \to +1.10). Please reconcile the count with the printed markers or report the unrounded intervals. (B.8, p18; Table 3)

  • Suspiciously repeated metric values across tables. For example, REOPOLD and Teacher CheXbert-5 both 40.00 on IU-Xray (Table 5); REOPOLD and budget-matched SFT RaTEScore both 51.67 on CheXpert (Table 6); +teachability gate and +teachability gate + anchor ROUGE-L both 4.89 (Table 6). These are plausibly coincidental at 2-decimal rounding, but please verify against unrounded per-sample scores. (Tables 4–7)

  1. Reproducibility

We appreciate the paper’s transparency in several respects: the explicit statement that bootstrap intervals quantify evaluation-set sampling uncertainty rather than training-seed variance (B.4), the commitment to release code upon publication (Reproducibility Statement, p10), and the patient-level split with the guarantee that no validation patient appears in any test split (§4.1, p6). The following suggestions would further close the reproducibility gap.

  • Anchored vs. unanchored rollouts (§3.5, pp5–6): Anchored arms optimize on 1616 rollouts and their 1616 references per step, while unanchored arms use 3232 rollouts. The paper frames this as “not more data,” but the anchored arm’s on-policy gradient is computed over half as many rollouts. Please acknowledge this asymmetry explicitly; if feasible, add a control with 3232 rollouts for the anchored arm to disentangle the effect of the loss from the number of on-policy samples.

  • External baselines’ internal hyperparameters (§4.1, p7): For the three external baselines reproduced “under our training configuration with only the loss replaced,” please list each baseline’s internal hyperparameters: EOPD’s entropy threshold, REOPOLD’s reward floor and two-phase schedule, and Revisiting-OPD’s top-k configuration. State whether these were taken from the original papers or tuned.

  • CheXpert test split (§4.1, p6): The CheXpert test split of 234234 studies is non-standard; the official test set has 500500 studies (Irvin et al., 2019). Please state how the 234234 studies were selected and whether they are a subset of the official test set, so that cross-paper comparison on CheXpert is possible.

  • Residualization and DiD estimators (§4.1, p6; §4.4, p8; B.8): Please specify both estimators. For residualization, give the regression model (linear? OLS? with intercept? on what scale?). For the difference-in-differences, give the estimating equation. Currently B.8 provides bootstrap intervals but not the estimator itself.

  • Budget-matched supervised control (B.6, p16): The budget-matched supervised control sees 3232 reference sequences per step, twice the anchored arm’s 1616 references. GA-OPD beating this control (19.3619.36 vs 17.8117.81) is therefore achieved despite less reference data per step, which strengthens the result. Please note this asymmetry explicitly.

  • Single-seed caveat (B.4, p16; p8): The single-seed design is disclosed exemplarily: “Each arm corresponds to a single training run with a fixed seed.” Since all significance claims (BH-adjusted p≤0.032p \leq 0.032) rest on paired bootstrap intervals over test studies, which quantify evaluation-set sampling uncertainty rather than training-seed variance, please consider adding a one-line caveat in the main text where significance is claimed (p8).

  • Declared comparison family (B.8, p18): B.8 refers to “the seven contrasts of GA-OPD against every other arm,” but Table 2 lists ten other arms. Please enumerate the seven contrasts explicitly; the phrase “every other arm” is inaccurate as written.

  • Training runtime/cost (B.5, p16): B.5 gives hardware and software versions but no wall-clock time, GPU-hours, or cost. Please add wall-clock time and GPU-hours for training.

  • Prompt paraphrases (B.9, p18): B.9 mentions “a shared pool of eleven paraphrases of one request” while also stating “Prompts are taken verbatim from the source corpora rather than written by us.” These are consistent only if the source corpora’s user turns are exactly these eleven paraphrases, which is not shown. Please list all eleven paraphrases and explain the mapping.

  • IRB/ethics approval (Ethics Statement, p10): The Ethics Statement does not mention IRB/ethics approval. This is likely not required for credentialed public de-identified data, but please say so explicitly.

  • Corpus versions and preprocessing (§4.1, p6; B.6): Please pin corpus versions (e.g., CheXpert Plus release version, MIMIC-CXR version) and fully specify preprocessing. B.6 mentions a “per-image pixel budget taken from the preprocessed inputs” but no filtering/selection criteria for the training pool are given.

  • Supervised checkpoint comparison (§4.3, p8): “GA-OPD improves over the supervised checkpoint on all six axes” compares against the non-budget-matched SFT (batch size 256256, learning rate 2×10−52 \times 10^{-5}). The budget-matched control is the fairer comparison, and GA-OPD also beats it on all six axes (Table 1). This is a framing note, not an error; please consider making the budget-matched comparison the primary one.

  • Validation split (§4.1, p6): The paper creates its own patient-level validation split (3,1563{,}156 studies) rather than using the standard MIMIC-CXR official split. This is common practice, but please note it explicitly for cross-paper comparability. The paper does state “no validation patient appears in any test split.”

Questions for authors
  1. The paper reports results from a single training seed, which limits confidence in the empirical gains. Could you provide additional runs with multiple random seeds (or a variance analysis) to demonstrate that the observed improvements are robust to seed variability?

  2. Table 1 evaluates only internal baselines, yet the abstract claims “highest CheXbert micro‑F1 among all arms.” Please add at least one external baseline (e.g., MAIRA‑2, CheXagent, or LLaVA‑Rad) to the comparison, or explicitly restrict the claim to the evaluated distillation and supervised arms.

  3. The teacher‑failure regime is created by replacing the teacher’s instruction (B.9). Could you report the teacher’s performance under its original instruction and discuss how the GA‑OPD advantage changes when the teacher is evaluated without this controlled mismatch?

  4. The REOPOLD baseline collapses early in training, and the review notes that this failure is only briefly disclosed. Can you analyze the cause of this collapse (e.g., hyper‑parameter sensitivity, reward‑floor settings) and show whether alternative settings prevent it, thereby strengthening the comparison?

  5. The reverse‑direction robustness check confounds the divergence direction with the estimator type (sampled vs. normalized top‑k). Could you run a controlled experiment that isolates the effect of direction while keeping the estimator fixed, and report whether the observed robustness still holds?

Reviewer_7K2FOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper researches the on-policy distillation technique when it is used in radiology report generation. The observation is that the teacher model may not always be reliable to provide useful supervision to a student model. To address this issue, this work proposes a method called GA-OPD (Gated-Anchor On-Policy Distillation). It consists of an anchor-based loss that is directly based on the ground-truth reference report and therefore is not affected by the reliability of the teacher model. Also, it proposes a drift gating mechanism to assess whether the student model still tracks the reference and whether the teacher signal is worth following. The combination of the anchor-based loss and the drift-gated OPD loss is used as the objective function to train the student model. Experimental study is conducted on benchmark datasets to investigate the efficacy of the proposed mechanism.

Critical strengths
  1. The proposed ideas, including the anchor-based loss and the drift-gating mechanism, are technically sound. They are based on the interesting observations on the reliability of a teacher model when performing on-policy distillation.
  2. The idea of drift-gating mechanism is cleverly designed.
  3. The paper actively provides insights on the on-policy distillation process and explain why the proposed ideas make sense. This is appreciated and improves the clarity of this paper.
Critical weaknesses
  1. Compared to the method parts, the experimental part is not clear enough and becomes harder to follow. This affects the quality of experimental support to the proposed ideas. This is a major issue and it needs to be well improved.
  2. In the experimental study, the overall improvements do not seem to be consistent or significant, raising questions on the efficacy of the proposed method in practice.
  3. Although the proposed ideas are sound and the drift-gating mechanism is nicely designed, the overall technical contribution seems to be incremental and the paper does not propose a sufficiently new framework. If understood correctly, the anchor-based loss is just a normal loss based on the ground-truth image-report pair and the drift-gating mechanism is mainly an adaptive weighting process.
Questions for authors

Questions whose answers could materially affect the assessment.

  1. As seen from Table 1 (and Tables 4 and 5 in Appendix), the improvements of proposed method with respect to the competitive methods such as REOPOLD are not consistent or just marginal. This raise questions on whether the proposal method is really useful in practice. This is a major concern. Please comment.
  2. The proposed method shows clear improvement in Table 2 in which "the CheXpert split of 234 studies" is tested. Please explain what the 234 studies are and how they are identified.
  3. Figure 3 can be better explained by its annotation and the text (Line 453 and Line 460). For example, the two numbers 7.56 and 7.22 cannot be found in Figure 3(a). More clear and detailed explanation will be helpful.
  4. For Eq.(8), the dynamic change of αt\alpha_t and gtg_t could make the objective function value fluctuate accordingly. Will this cause any difficulty to the convergence of optimisation during the training process?

Suggestions for future work:

  1. For Eq.(2), could better explain why it is called "teacher-forced."
  2. For Eq.(3), could provide a reference for this overlap criterion. Can word-level edit distance be used to assess the overlap of two sequences in this work?
1 comment

Thank you for recognizing the motivation for the anchor and drift gate. We clarify the practical benefit, methodological contribution, and experimental details below.

1. Practical benefit and the role of each experiment (W1-W2, Q1). Table 1 measures overall report quality, Table 2 tests recovery under misaligned teacher supervision, and Table 3 examines how the gain varies with teacher difficulty.

In the CheXpert stress test, GA-OPD reaches 19.36 RadGraph, versus 7.15 for plain OPD and 17.44 for REOPOLD. The improvement over REOPOLD is 1.92 points, with a 95% paired bootstrap CI of [0.57, 3.36] (Appendix B.8), obtained by paired resampling of evaluation studies.

Across all 4,676 studies, GA-OPD achieves the highest CheXbert micro-F1 on both label sets among the evaluated methods: 59.34 and 51.81. These results demonstrate the practical value of the method: recovering clinical faithfulness under misaligned teacher supervision while achieving competitive report quality on the full test set.

2. What capability does the method add? (W3). We introduce a token-level distillation mechanism that couples reference-prefix-drift-dependent weighting of teacher supervision with an independent reference anchor. Relative to the two classes of approaches discussed in the paper, it adds two complementary elements:

  • Compared with methods that modify only the teacher term: the reference anchor provides a direct path for learning ground-truth reports, with targets that bypass the teacher.
  • Compared with fixed-weight OPD-SFT mixtures: the teacher weight is determined at each token by the overlap between the current rollout and reference prefixes, so the relative influence of the two supervision sources changes with the generation state.

The key is their integration into one update objective: the teacher supplies on-policy feedback on student-generated prefixes, the anchor supplies a stable target on reference prefixes, and the drift gate determines how much teacher feedback to retain at each rollout position.

This design has a direct controlled ablation: holding the anchor coefficient and teachability rule fixed, the drift gate raises CheXpert RadGraph from 7.22 to 19.36. The contribution is thus a concrete supervision structure, supported by an experiment isolating the effect of the drift gate within that structure (Eqs. 2-4 and 8; Table 2; Sec. 4.5).

3. Origin of the 234 CheXpert studies (Q2). These are 234 studies from 200 patients in the public CheXpert validation split with an available Impression section. They serve as the CheXpert test cohort for report generation in our study. The cohort is defined by the source split and report availability, before the teacher-score analysis. We will state its origin and role explicitly in Sec. 4.1.

4. The values 7.56 and 7.22 in Fig. 3(a) (Q3). 7.56 corresponds to the teachability-only row of Table 2, which has no separate bar in Fig. 3(a). 7.22 corresponds to the fixed-weight anchor overlay in the bottom composite bar; its value appears in the caption. The overlays represent absolute scores of different configurations. We will make the corresponding table rows and figure positions explicit in Sec. 4.5.

5. Dynamic gates and training stability (Q4). Within each update, the gates are fixed weights bounded in [0, 1], with no gradient propagation through them. They provide bounded weighting of the teacher term while the reference anchor continuously supplies supervision.

The observed training trajectory also supports stable performance: across all ten stored checkpoints in Appendix B.7, GA-OPD's CheXpert validation RadGraph stays within 20.73-22.07. The reported recovery therefore persists across these checkpoints rather than depending on a single isolated checkpoint.

6. Teacher forcing and overlap (Eqs. 2-3). Teacher forcing in Eq. (2) means that the student predicts the next reference token given the ground-truth prefix. This training-input convention is distinct from querying a teacher model.

Eq. (3) applies the Dice coefficient (Dice, Ecology, 1945) to token multisets, computing the harmonic mean of the two directional overlap rates. Word-level edit distance is an alternative sequence comparison. We choose overlap to permit reordering within the compared prefixes and support incremental updates as the prefixes grow.

Reviewer_cfyKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper looks at radiology report distillation when teacher feedback is unreliable. GA-OPD combines cross-entropy loss on reference prefixes with a distillation loss on student rollouts. A reference token-overlap gate works with an existing teachability gate to reduce the teacher loss.

The largest gain appears on CheXpert when the teacher receives an instruction that doesn’t match report generation. RadGraph increases from 7.15 for plain OPD to 19.36 for GA-OPD (Figure 1). GA-OPD also achieves the highest CheXbert micro-F1 across 4,676 studies. Its relative performance varies across metrics and corpora.

Critical strengths

1.I found the analysis of flawed teacher supervision useful. The paper shows how this supervision can degrade a student model after supervised initialization. It examines report length, repetition, diversity, and checkpoint trajectories. These analyses make the failure mode easier to understand and add information beyond clinical metric scores.

2.The ablation removes the drift gate and shows that reference loss alone doesn’t fix the controlled failure at the tested weight. The authors are also clear about the induced instruction mismatch, differences across corpora, and the single training run. These details help readers put the results in context.

Critical weaknesses

1.My main concern is that the strongest results come from an intentionally induced task mismatch. The authors modify the CheXpert teacher prompt during both training-time scoring and teacher evaluation (Appendix B.9). The modified prompt asks the teacher to list abnormalities and propose diagnoses. The main task asks for report generation. This is a valid stress test. But it mixes instruction mismatch with factual unreliability. The within-MIMIC analysis adds evidence, although its extreme-decile contrasts include zero. I therefore don’t see clear evidence yet of gains for naturally occurring teacher errors under matched task instructions. This limits how broadly I can interpret the main robustness claim.

2.I’m also not sure what explains the recovery. Removing the anchor and the drift gate together doesn’t show whether the anchor itself is necessary. Recovery might already come from suppressing harmful distillation and preserving the supervised initialization. Two controls would help test this possibility. The first should keep both gates and set the anchor weight to zero. The second should use constant attenuation matched to the average reduction in teacher supervision. Without these controls, I can’t tell whether reference-dependent routing is necessary.

Questions for authors

1.Does GA-OPD improve teacher and student performance when teacher errors occur naturally under matched task instructions? This could be tested on CheXpert or existing data with matched instructions. Both models should receive exactly the same task instruction.

2.What happens when both gates stay active and the anchor coefficient is set to zero?

3,Can constant attenuation matched to the average gate strength recover comparable performance under the same validation tuning budget?

1 comment

Thank you for recognizing the value of our analyses of report length, repetition, and training trajectories. Your questions focus on two issues: whether the effect depends on instruction mismatch, and whether recovery comes from simply suppressing the teacher signal or from the proposed combination of supervision sources.

1. Instruction-mismatch stress test and natural teacher difficulty (W1, Q1). The CheXpert experiment asks a specific question: when the teacher's supervision is misaligned with report generation, can the student use reference supervision to maintain clinical faithfulness? Appendix B.9 specifies the teacher instruction. Under the same mismatch, plain OPD reaches 7.15 RadGraph and GA-OPD reaches 19.36. This experiment directly compares how different distillation objectives respond to the same supervision mismatch.

The matched-instruction setting provides complementary evidence. Within MIMIC-CXR, stratifying studies into terciles along the residualized teacher-difficulty axis shows that the GA-OPD advantage over plain OPD is 1.91 RadGraph points larger in the hardest than in the easiest tercile, with a 95% bootstrap interval excluding zero (Table 3). This comparison is entirely within one corpus, with matched report-generation instructions for teacher and student. The CheXpert stress test and MIMIC-CXR stratification thus provide evidence under controlled mismatch and natural variation in difficulty, respectively.

Our focus is how the student uses teacher supervision whose reliability varies across inputs and generated prefixes. The teacher remains frozen; improving the student's use of that supervision is the role of GA-OPD. We will organize these two sources of evidence more explicitly in the paper so that the question addressed by each experiment is clear.

2. How attenuation works with the reference anchor (W2, Q2). Drift attenuation in GA-OPD regulates the relative influence of two supervision sources. The teacher term acts on student-generated rollouts, while the anchor continuously provides reference supervision on ground-truth prefixes. As rollout-reference prefix overlap decreases, the drift gate reduces the teacher term while the anchor coefficient stays fixed, shifting the update toward reference supervision.

This changes both the strength of the update and the relative contributions of the two sources to its direction. Removing the anchor while retaining the gates yields a reference-dependent filter on teacher signals, without the gradient path that directly learns reference tokens. Attenuation alone and the proposed combination therefore serve different roles.

The existing ablation directly tests their interaction: the teachability gate scores 7.56; adding a fixed-weight anchor gives 7.22; keeping the same anchor coefficient and teachability rule, adding the drift gate raises the score to 19.36. The recovery therefore comes from drift-dependent weighting of teacher supervision within the anchored objective, rather than simply adding a reference loss.

3. Why mean-matched constant attenuation differs from the drift gate (Q3). Constant attenuation reduces the overall influence of unreliable teacher supervision, so it could reasonably improve performance under mismatch. Matching the mean gate value, however, matches the average supervision strength, not its allocation across token positions.

Let hth_t denote the teacher gradient including the teachability gate. Dynamic weighting gives E[αtht]\mathbb{E}[\alpha_t h_t], whereas replacing the drift gate by c=E[αt]c=\mathbb{E}[\alpha_t] gives E[αt]E[ht]\mathbb{E}[\alpha_t]\mathbb{E}[h_t]. Their difference is the covariance between the gate and the gradient. GA-OPD uses reference-prefix overlap to make this weight depend on the rollout's deviation.

Our mechanism-based expectation is that constant attenuation weakens useful and misaligned teacher signals together. The drift gate instead retains more teacher supervision at aligned positions and increases the anchor's relative influence at drifted positions. The distinction lies in the relationship between the weights and the generation state, beyond the average attenuation level.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.