acceptodds
Under review as a conference paper at ICLR 2027

SelfCue: Making a 3D CT Report Generator Say What It Already Knows

Abstract

Progress in 3D CT report generation is usually sought in increasingly sophisticated architectures and larger pools of training data. We find instead that a 3D CT report generator already holds what its report leaves out, and loses it when the hidden state becomes tokens. Over the 18 CT-RATE abnormalities, this hidden-to-report surfacing gap is reflected by a drop in macro AUROC from 0.848 in the hidden states to 0.739 in the generated report. We propose SelfCue based on contrastive decoding. It promotes what the hidden state already supports and suppresses what it does not. It raises clinical efficacy F1 to and the LLM-judged GREEN score to . Distilling that behaviour into the weights gives SelfCue-KD, a student that keeps most of the gain, needs nothing extra at inference, and drops into any pipeline already serving the baseline. Code is available at https://anonymous.4open.science/r/SelfCue-anonymous-348F.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
3D CT report generation, radiology report generation, medical vision-language models, contrastive decoding, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wd94OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the discrepancy between abnormality information recoverable from a 3D CT report generator’s representations and information expressed in its generated reports. Supervised probes achieve substantially higher abnormality AUROC than the generated reports across three encoder–language-model combinations. The proposed SelfCue method converts probe predictions into an abnormality condition and applies contrastive decoding between conditioned and unconditioned branches. SelfCue-KD subsequently distills the generated reports into a student that requires neither the condition nor contrastive decoding at inference. Experiments on CT-RATE demonstrate improvements in abnormality-level F1 and AUROC, supported by ablations on conditioning, decoding, and distillation, as well as a held-out hyperparameter-selection experiment across three seeds.

Critical strengths

A useful intervention with a controlled demonstration of its contribution. Recovering findings omitted from generated reports is an important problem in medical vision-language modeling. The comparison between Mcond and SelfCue isolates the effect of contrastive decoding while keeping the model and abnormality condition fixed. On COLIPRI with Llama-3.1-8B, [email protected] improves from 0.430 to 0.481 and F1max from 0.514 to 0.542. The improvements across three model combinations and the hyperparameter sweeps provide credible evidence that the decoding intervention adds value beyond simply supplying predicted abnormalities.

A practical distillation approach with thoughtful treatment of teacher conditions. SelfCue-KD retains much of the teacher’s improvement while restoring ordinary single-branch generation without an abnormality condition. The use of cross-fitted probe predictions for constructing the teacher corpus is a meaningful design choice, and the ablations show that the construction of these conditions matters. The held-out selection experiment and multiple training seeds further strengthen the empirical evidence. Together, these results make the approach potentially useful for existing CT report-generation pipelines.

Critical weaknesses

The interpretation of Figure 1A is stronger than the evidence establishes. It is difficult to understand why substantially different visual encoders and language models produce nearly identical AUROC trajectories and final report AUROC. Are these results consistent with the performance of comparable configurations reported in the literature, or does the common training and alignment recipe constrain all three systems to a similar operating point? In particular, insufficient image–text alignment could leave abnormality information recoverable from the visual representations without enabling the language model to use it effectively for generation.

The probing setup does not fully distinguish these explanations. Hidden states are pooled over both visual and text prompt positions, and the supervised probe learns an additional mapping from representations to abnormality labels. Successful probing therefore establishes recoverability, but does not by itself establish that the generator has integrated this information into its report-generation process. Furthermore, hidden-state AUROC and report AUROC use different readout models. The conclusion that information is lost only when hidden states become tokens should be qualified unless these alternative explanations are addressed. This affects the paper’s central mechanistic contribution, although it does not invalidate the empirical improvements.

Improved abnormality scores do not yet establish a consistent improvement in overall report reliability. Table 4 shows that contrastive decoding strengthens adherence to incorrect conditions as well as correct ones: the mention rate for abnormalities listed in the condition but absent from the scan increases from 0.610 for Mcond to 0.855 for SelfCue. Distillation mitigates this effect, but the teacher remains strongly dependent on probe errors. Meanwhile, the incremental F1 gains over Mcond do not produce consistent GREEN gains; for MedGemma-27B, GREEN decreases from 0.472 to 0.456.

The paper discusses the different coverage of these metrics, which is helpful. Nevertheless, evaluation remains concentrated on 18 abnormality categories in one dataset, with clinical efficacy measured through RadBERT. More direct evidence about whether added findings are clinically correct, appropriately localized, and consistent with the rest of the report would strengthen the practical claim. I view this as a limitation of the demonstrated scope rather than a reason to dismiss the method.

Questions for authors

Clarification of Figure 1A: Why are the AUROC trajectories and final report AUROC so similar across the encoder–backbone combinations? Please contextualize these results against comparable published configurations and clarify how the image–text alignment and fine-tuning procedures were validated for each backbone. Could a shared alignment bottleneck explain the observed similarity?

Evidence for the proposed mechanism: Can you distinguish information retained in visual-token representations from information actively used by the language model for generation? For example, probing text prompt positions after they have attended to the image, separately from visual positions, would help. Given that the visual-embedding probe produces nearly the same downstream results in Table 3, what evidence specifically supports interpreting SelfCue as surfacing the language model’s existing knowledge?

Report reliability: Beyond aggregate RadBERT scores, what evidence shows that the additional abnormality mentions improve report correctness? Please clarify the clinical consequences of the error propagation observed in Table 4 and whether an independently assessed sample supports the claimed benefit, particularly for configurations where GREEN decreases.

2 comments

Thank you for recognizing the controlled decoding gains and the practical value of distillation. We have added diagnostic evidence and clarified the mechanism and scope of the clinical results.

Q1. Why are the trajectories so similar, and could alignment explain this?

We added FlexiCT, ASAP and HLIP with Llama-3.1-8B to Table 1. Across six configurations, visual-probe AUROC ranges from 0.824 to 0.868 and report AUROC from 0.727 to 0.753. Higher visual-probe scores generally accompany higher hidden-state and report scores, while the surfacing gap remains 0.097–0.116. The scores therefore vary with the visual representation. All generators use SFT with frozen visual representations and trained projectors/LoRA adapters; Setup specifies the backbone-specific learning rates. A shared alignment bottleneck may still contribute. We clarify this interpretation in Section 3.2 and provide published configurations for context in Table 2 and Appendix C. Figure 1's caption also explains the rescaling of MedGemma's 62 layers to Llama's 32-layer axis.

Q2. Does probing show information in text positions or its active use by the LLM?

Appendix B.1 and Table 8 compare pooling visual positions, all prompt positions and the last prompt position, which can attend to the image. Their best-layer AUROCs differ by at most 0.010 across the three configurations. This shows that abnormality information is recoverable from text positions as well as visual positions. We agree that probing alone does not establish how each finding is used during generation. Replacing the hidden-state probe with a visual-embedding probe also changes downstream scores by at most 0.008 (Table 5). We now frame SelfCue as expressing information recoverable from the generator's internal representations.

Q3. Could the hidden-to-report gap reflect different measurement tools?

The probe and RadBERT are different readouts. To check reference-label sensitivity, we relabeled the same reference reports with gpt-oss-120b. Keeping all prediction scores fixed changes the COLIPRI–Llama AUROC gap only from 0.1093 to 0.1075. The gap therefore persists under alternative reference annotation. Section 3.2 and Appendix B.2 report this check. We revised the Introduction and Discussion to describe report expression as an important bottleneck.

Q4. How does SelfCue differ from putting probe predictions in the prompt?

The baseline uses predicted labels in the prompt, while SelfCue additionally contrasts conditioned and plain logits from the same model at the same generated prefix. With weights and labels fixed, [email protected] rises from 0.430 to 0.481. To make this distinction easier to follow, we renamed ordinary conditioned generation the “Probe-conditioned baseline.” We explain the distinction after Eq. (1) and in Table 2's caption. SelfCue provides one way to narrow the surfacing gap through decoding.

Q5. What happens when the condition contains errors?

SelfCue helps reports express more abnormality information recoverable from the model. In Table 6, mention rates for reference-positive cues rise from 0.697 to 0.901, and overall [email protected] increases from 0.430 to 0.481. Some incorrect predictions are expressed more often too. Our method promotes expression of the probe's predictions but has no dedicated mechanism to check their correctness against the CT image or filter errors. We added this limitation in “Limitation: error propagation” and identify cue verification and error filtering as future work.

Q6. Why do the F1 gains not consistently improve GREEN?

F1 measures expression of the 18 targeted abnormalities, while GREEN also assesses clinical details beyond these labels. Higher F1 therefore need not yield higher GREEN: NV-Reason-CT (Myronenko et al., 2026) reports F1 of 0.5923 and GREEN of 0.4072. Relative to the probe-conditioned baseline, our GREEN is unchanged on COLIPRI–Llama and falls by 0.008 on VISD–Llama and 0.016 on COLIPRI–MedGemma. Table 2 reports these results. We revised “GREEN” and Discussion's “Complementary metrics” to explain why both metrics are needed.

Q7. What is the scope of the clinical evidence?

Under automated evaluation, SelfCue improves expression of the 18 targeted categories in SFT-trained generators. The same-split template control in Table 3 also shows higher report F1 and GREEN for SelfCue than fixed label templates. Benefits for other findings and finer attributes, including location, size and laterality, remain future work, as stated in “Limitation: scope of the correction.”

Thanks for the response. The author answered most of my question. However, there is still limitations on how useful this work could be, especially considering that this work only shows better F1 score and has insufficient evaluation on quality of report. I think it is fair to keep current rating.

Reviewer_3EQQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This is a thoughtful and well-scoped paper: the authors are refreshingly honest about the readout limit, the central equations are fully specified and type-consistent, and the ablation design cleanly isolates the decoding rule from the condition source, making the reported gains (e.g., [email protected] from 0.430 to 0.481) credible and attributable. The SelfCue-KD variant is a genuinely practical contribution, removing both inference-time costs while retaining most of the gain, and the cross-fitted teacher corpus is a careful design choice. The main work before submission lies in the reproducibility section, which currently leaves too many implementation details implicit (data splits, hyperparameter search ranges, and exact evaluation pipelines), and in a few places where the mathematical notation could be tightened for clarity—particularly around the prompt-mixture gate and the temperature parameter. A small number of prose-to-table inconsistencies and minor writing issues also deserve a pass. However, the method is not applied on the newest state-of-the-art methods on the same data which raises doubts about the paper.

Critical strengths
  • The paper is refreshingly honest about the readout limit, explicitly stating that "The limit is the readout. SelfCue closes about 60% of the distance" and acknowledging that "the probe's errors pass into the teacher's reports." This framing correctly scopes what the method can and cannot fix, and we encourage the authors to keep this nuance prominent in the final version (Section 6, p. 9).

  • Equation (1) is fully specified and type-consistent: ℓcond\ell_{\mathrm{cond}}, ℓplain\ell_{\mathrm{plain}}, and α\alpha are all defined, the combination is a valid logit-vector operation, and the claimed equivalence "At α=0\alpha = 0, SelfCue reduces to Mcond" verifies exactly against Eq. (1) and the Table 1 caption. This precision makes the method easy to reproduce (Section 5.3.1, p. 7; Eq. (1), p. 4).

  • No heuristic is dressed as a principled condition: the prompt mixture, the empty-condition gate, and the hyperparameter choices (α=1.0\alpha = 1.0, τ=0.75\tau = 0.75) are all presented as empirical choices with stated rationales. This transparency is a model for how to report design decisions (Section 4.1, p. 4).

  • Primary sources are consistently cited for datasets, encoders, metrics, and methods (CT-RATE, COLIPRI, ViSD, GREEN, RadBERT, contrastive decoding, sequence-level KD); no survey-as-theorem or method-paper-as-benchmark misuse was found. This strengthens the paper's scholarly foundation (Sections 2.1–2.3, pp. 2–3; References, pp. 11–17).

  • Every stated framework governs the implemented method: contrastive decoding (Eq. (1) matches the two-branch description), sequence-level distillation (Eq. (3) matches Kim & Rush 2016), and context distillation is used only as a distinction. No decorative preliminaries were found, which keeps the paper focused and rigorous (Section 2.3, p. 3).

  • Most prose-to-table numeric claims in Sections 5.2–5.4 were verified consistent with Tables 1–5, e.g., the [email protected] progression 0.395→0.430→0.4810.395 \to 0.430 \to 0.481, the Table 3 swap deltas, and the Table 4 mention-rate comparisons. This consistency gives readers confidence in the reported results (Section 5.2, p. 6; Table 1, p. 5).

  • Every verifiable citation supports (or plausibly supports) the specific claim it is attached to; the Smith-Bindman "93 million examinations" figure and the Kurmukov "about 15 minutes" reading-time figure were both verified against their abstracts. This careful citation practice is commendable (Section 1, p. 1).

  • No mis-attribution was found in the contrastive-decoding lineage: Li et al. 2023b (original form), Sanchez et al. 2024 (prompt-vs-absence form), Burns et al. 2023 (knowing-saying gap), and Kim & Rush 2016 / Hinton et al. 2015 (distillation) are all correctly attributed. This is exactly the kind of historical accuracy that makes a related-work section trustworthy (Section 2.2, p. 2).

  • The internal comparison is clean and well-controlled: McondM_{\mathrm{cond}} and SelfCue "use the same model and the same condition and differ only in how they decode," so the gain from McondM_{\mathrm{cond}} to SelfCue ([email protected] 0.430→0.4810.430 \to 0.481 on COLIPRI+Llama) is attributable to the decoding rule. The ablation structure (Tables 2–4) isolates the gate, the condition source, and the distillation loss, making the contribution of each component unambiguous (Section 5.2, p. 6).

  • SelfCue-KD is a genuinely useful deployment contribution: it removes both inference-time costs (two forward passes; per-volume condition computation) while retaining most of the gain, with the runtime contrast "6.3 hours for the 1564 validation volumes, while SelfCue-KD under vLLM (Kwon et al., 2023) writes all of them in 71 seconds." The 5-fold cross-fitted teacher corpus (Eq. (2)) is a thoughtful design choice that avoids train/test leakage and deserves to be highlighted as a best practice (Section 5.2, p. 6).

Critical weaknesses

We appreciate the careful framing of the teacher–probe distinction and the honest reporting of per-configuration results. The two points below are offered as tightening suggestions rather than fundamental concerns.

  • The Dao et al. (2021) citation is analogical and should be sharpened. In Section 4.2 (p. 4), you write that “the conditions used to write the teacher corpus must be honest out of sample” and cite Dao et al. for this claim. However, Dao et al. analyze in-sample bias of the teacher’s predictions, whereas your concern is that the probe’s conditions (the teacher corpus) are in-sample. This is a meaningful distinction: the cited work does not directly address the probe’s training conditions. I suggest tightening the citation, for example by writing “cf. the in-sample bias analysed by Dao et al.” and then explicitly stating that your concern is the probe’s conditions, not the teacher’s predictive bias. This will prevent a reader from inferring that Dao et al. support a claim they do not make.

  • The abstract’s GREEN claim is contradicted by your own Table 1 on two of three pairs. The abstract (p. 1) states: “It raises clinical efficacy F1 to 0.4810.481 and the LLM-judged GREEN score to 0.5100.510.” However, Table 1 (p. 5) shows that, relative to the Mcond baseline, SelfCue does not raise GREEN on two of the three configurations:

  • COLIPRI+Llama: 0.510=0.5100.510 = 0.510 (no change, not a raise),

  • VISD+CT-CLIP: 0.504<0.5120.504 < 0.512 (a drop),

  • MedGemma: 0.456<0.4720.456 < 0.472 (a drop).

The body text in Section 6 (p. 9) says “SelfCue holds GREEN at the level of Mcond on all three pairs,” which is also inaccurate for the two pairs where GREEN decreases. We recommend qualifying the GREEN claim by configuration (e.g., “raises GREEN for COLIPRI+Llama and holds it within 0.0080.008 for the other two”) or, more conservatively, reporting the GREEN result only relative to the M_base baseline, where the improvement is consistent. This will align the abstract and Section 6 with the tabulated evidence and avoid a reviewer or reader flagging the discrepancy.

  1. Literature & Related Work Thank you for contributing a timely and well-scoped study on self-cueing for 3D CT report generation. The related-work section is clearly organized, and the probing-based framing in Section 2.2 is a useful lens. The following suggestions are intended to strengthen the transparency, completeness, and reproducibility of the literature positioning.
  • Enumerate the candidate set for the Table 1 eligibility screen and justify exclusions. At least four same-benchmark (CT-RATE) 2026 systems report clinical numbers at or above SelfCue’s headline figures ([email protected]$ = 0.481$, AUROC =0.775= 0.775), yet none are cited or screened: NV-Reason-CT (report-derived macro-F1 0.5920.592, macro-AUROC 0.8710.871), Resolution Meets Reduction (clinical macro-F1 0.4950.495), Sparse Concept Channels (F1=0.549F1 = 0.549), and Disease-Centric VLP (CT-RATE AUC 84.4%84.4\%). The claim that “SelfCue is on par with the strongest eligible works on [email protected]” is therefore incomplete at best. Please add a paragraph (or a supplementary table) that lists every same-benchmark system you considered, states the inclusion/exclusion criteria (e.g., “we include systems that report [email protected] on CT-RATE test split and provide a public checkpoint”), and then explicitly explains why each of the four above is or is not included. If they are excluded because they report different metrics (e.g., macro-F1 or AUC), say so and consider adding a secondary comparison on those metrics.

  • Add Merlin (and ideally RadGenome-ChestCT) to the evaluation or justify the single-dataset scope. Merlin (Blankemeier et al., 2026, Nature) is the other major public 3D CT vision-language corpus, and the authors’ own ORCA paper evaluates on it (“We evaluate it across two datasets (CT-RATE and Merlin)”). The current manuscript never mentions Merlin, which weakens the generalizability claim. Please either (a) add Merlin (and RadGenome-ChestCT, if feasible) to the experimental setup, or (b) add an explicit justification for the single-dataset scope in Section 5.1, e.g., “we focus on CT-RATE because it is the only benchmark with a standardized [email protected] evaluation protocol; Merlin lacks this metric.” If the latter, also note that ORCA’s Merlin results do not transfer automatically to SelfCue.

  • Verify the RadAgent Table 1 row or remove it. The RadAgent row ([email protected] = 0.220) cannot be reconciled with its own abstract, which reports a 5.8-point (35.4% relative) macro-F1 improvement over CT-Chat: 0.184+0.058=0.2420.184 + 0.058 = 0.242 (or 0.184×1.354=0.2490.184 \times 1.354 = 0.249), neither of which equals 0.2200.220. Please check the full RadAgent paper (Roschewitz et al., 2026, arXiv:2604.15231) for the correct [email protected] value, or remove the row if the metric is not reported there. If you keep it, add a footnote explaining the derivation.

  • Expand Section 2.1 to cover the broader 3D CT report-generation literature. Section 2.1 currently cites only 8 works, while at least 15 same-domain systems exist and are uncited, including 3D-CT-GPT (Chen et al., 2024), CT-GLIP (Lin et al., 2024), M3 (Hosseini et al., 2025), MvKeTR (Deng et al., 2025), MedScribe (Orlando et al., 2026), CPR-RAG (Yang et al., 2026), MARCH (Lin et al., 2026), ALTER (Li et al., 2026), CORTEX (Malik et al., 2026), CTInstruct (Lei et al., 2026), Jolia/ConQuer (Khlaut et al., 2026), and Rad-JEPA 3D (Trinh et al., 2026). Please add the directly relevant ones to the related-work section, grouping them by approach (e.g., end-to-end generation, retrieval-augmented, agentic, and probing-based). Also consider citing R2Gen (Chen et al., 2020) as the foundational 2D radiology report-generation work, and Belinkov & Glass (2019) for the probing methodology in Section 2.2. This will position SelfCue more accurately within the rapidly growing field.

  • Positive note on reference integrity. We found no predatory or retracted venues among the verified references; all verified venues are legitimate (Nature Biomedical Engineering, Nature, ACL, EMNLP, AAAI, CVPR, ICLR, MICCAI, ISBI, Radiology: AI, JAMA Internal Medicine, PMLR). This is a strength of the manuscript and should be maintained as you add the new citations above.

We hope these suggestions help you sharpen the literature positioning and make the empirical claims more robust. The core idea of self-cueing is promising, and with a more complete and transparent related-work section, the paper will be a stronger contribution to the 3D CT report-generation community.

  1. Mathematical Correctness

The mathematical core of the paper is clear and the probing setup is thoughtfully designed. The suggestions below are about precision rather than correctness; addressing them will make the formal claims airtight.

  • Make the fold partition explicit in Eq. (2). The union notation ⋃kSk\bigcup_k S_k and the sentence “every training volume is paired with a report” imply that the folds cover the training set, but this covering property is never stated. Since Eq. (2) depends on the folds partitioning the training set, please state this explicitly, e.g., the folds form a partition of the training set Dtrain\mathcal{D}_{\mathrm{train}}: Si∩Sj=∅(i≠j),⋃k=15Sk=Dtrain.S_i \cap S_j = \emptyset \quad (i \neq j), \qquad \bigcup_{k=1}^5 S_k = \mathcal{D}_{\mathrm{train}}. This removes any ambiguity about whether some training volumes could be excluded from all folds. (Section 4.2, p. 4)

  • Condition Eq. (3) on the prompt, or state that it is implicit. Eq. (3) writes pstudent(yi∣xi)p_{\mathrm{student}}(y_i \mid x_i), but the text says the student “receives only the plain prompt.” Strictly, the probability should be pstudent(yi∣xi,promptplain)p_{\mathrm{student}}(y_i \mid x_i, \mathrm{prompt}_{\mathrm{plain}}). Since the prompt is fixed throughout, this is a reasonable notational simplification, but the equation as written is incomplete. Please either add the prompt explicitly in Eq. (3), or add a sentence such as “the prompt is fixed and suppressed in the notation.” (Section 4.2, p. 4; Eq. (3), p. 4)

  • State the zero-denominator convention for macro-F1max⁡F_1^{\max}. In Section 5.1, when you define macro-F1max⁡F_1^{\max} as the best F1F_1 over all thresholds and introduce the threshold τ\tau at which the readout becomes a condition, the case of an abnormality that is never mentioned and never present is not addressed. For such a class, precision is 0/00/0 and F1F_1 is undefined. Please state the convention explicitly; the standard choice is to define F1=0F_1 = 0 in this case. This will make the macro-average well-defined for all classes. (Section 5.1, p. 5–6)

  • Qualify the abstract’s “0.848 in the hidden states.” The abstract says “from 0.848 in the hidden states to 0.739 in the generated report,” but the evidence in Section 3.2 shows the hidden-state scores range from 0.8390.839 to 0.8480.848, and the 0.8480.848 value is the pre-LM embedding score quoted from ORCA. Please adjust the abstract wording to match the evidence, e.g., “from up to 0.8480.848 in the hidden states” or “from 0.8390.839 to 0.8480.848 in the hidden states.” This is a small change that will prevent overclaiming. (Abstract, p. 1; cf. Section 3.2, p. 3)

  • Add a control experiment for the “surfacing gap,” or soften the claim. The headline gap of 0.848→0.7390.848 \rightarrow 0.739 (a drop of 0.1090.109) is measured with the trained CheapCT probe on hidden states and frozen RadBERT on reports. As written, the gap may partly reflect probe-versus-RadBERT differences rather than pure information loss at tokenization, because the paper never scores the same text with both rulers. I strongly recommend adding a control experiment in which the same generated reports are scored with both the CheapCT probe and RadBERT. If that is not feasible, please soften the claim “lost only when the hidden state becomes text” to something like “lost as measured by our two scoring instruments.” This is the most important point, because the paper’s central conclusion depends on comparing two different measurement tools. (Section 3.1, p. 3)

  1. Reproducibility

Thank you for releasing the code and for the detailed experimental description. The paper is largely reproducible in spirit, but several gaps currently prevent a competent peer from re-running the central result from the paper alone. The following suggestions are intended to close those gaps; none require new experiments, only additional reporting.

Data and comparability

  • Please clarify the relationship between the two evaluation sets. You report COLIPRI on n=1564n = 1564 volumes and VISD+CT-CLIP on n=2987n = 2987 (Section 5.1, p. 5). Are the 2987 volumes independent scans, or do they include multiple reconstructions of the same scan? If the latter, do multiple reconstructions enter the macro averages? The comparability claim is currently asserted rather than demonstrated; a sentence specifying the scan-to-volume mapping and the averaging unit would resolve this.

  • The claim that the effect is “not a quirk of one pipeline” (Section 1, p. 1) is supported by only three of the four possible encoder–backbone combinations (COLIPRI+Llama, VISD+CT-CLIP+Llama, COLIPRI+MedGemma), all within a single LLaVA-style architecture family and a single dataset (CT-RATE). Please either add the missing combination (VISD+CT-CLIP+MedGemma) or qualify the breadth claim, e.g., “not a quirk of one encoder–backbone pair within the LLaVA-style family on CT-RATE.” The generalization statement in Section 6 (p. 9) and the backbone-dependence discussion in Section 5.2 (p. 6) would also benefit from this qualification.

  • No reader study is reported; the paper explicitly defers this (“reader studies are the next step”, Section 6, p. 9). This is an acknowledged limitation, but please make sure the clinical-efficacy claims are read with this scope in mind—e.g., by adding a sentence in the abstract or conclusion that the reported metrics are automated, not clinician-validated.

  • The paper contains no ethics, informed-consent, IRB, or data-governance statement, despite using the clinical chest-CT dataset CT-RATE. Please add a statement on data usage and governance (e.g., in a dedicated Ethics Statement or in the Data Availability section). The only ethics-adjacent text is the AI Use Statement (p. 10); a data-governance sentence would complete the picture.

Training and hyperparameters

  • Student (SelfCue-KD) training details are incomplete. Please report epochs, learning rate, batch size, LoRA rank, and number of steps for the student. Section 5.3.4 (p. 8) states that the two students “start from the same checkpoint and train for the same number of steps” but gives neither value. The projector/LoRA training in Section 5.1 (p. 6) mentions a cosine schedule and effective batch size of 16, but the student-specific hyperparameters are missing. Please add a table or paragraph with the full student training configuration.

  • No cross-method hyperparameter-tuning protocol is described. The paper states its own tuning (e.g., the MedGemma baseline uses 5×10−65 \times 10^{-6}, “which a sweep selected for it”, Section 5.1, p. 6), but nothing is said about how the quoted baselines were tuned in their source papers, and no re-tuning under a common protocol is reported. Please state the tuning protocol for all compared systems, or explicitly note that no common protocol was applied.

  • The sentence “All three seeds land on that setting” (Section 5.4, p. 9) is ambiguous. If the recipe is unchanged across seeds, the setting is fixed, not re-selected; if the setting was re-selected per seed, that is a different experiment. Please clarify which procedure was used.

Evaluation and claims

  • Selection on the validation split is only partially mitigated. You state that α=1.0\alpha = 1.0 and τ=0.75\tau = 0.75 were “chosen on the validation split” (Section 5.1, p. 6), Table 1 is scored on that same split, and the held-out experiment (Section 5.4, p. 9; Table 5, p. 9) covers only COLIPRI + Llama-3.1-8B. The headline numbers for VISD+CT-CLIP and MedGemma-27B therefore remain selection-on-validation. Please either extend the held-out experiment to those configurations or qualify the claims accordingly.

  • CLarGen’s Table 1 row ([email protected] = 0.486) lacks backbone annotation. Section 5.2 (p. 6) identifies 0.486 as the MedGemma-27B number and 0.472 as the Llama-3.1-8B number. Please annotate the row (e.g., “CLarGen (MedGemma-27B)”) so the reader can compare like-for-like.

  • The claim “SelfCue is on par with the strongest eligible works on [email protected] and GREEN and above them on F1max⁡F1_{\max}” (Section 5.2, p. 6) is only partially supported. The table’s eligible-works rows are not backbone-annotated, and CLarGen’s table value (0.486) exceeds SelfCue’s 0.481. Please qualify the claim (e.g., “on par with the strongest eligible works on [email protected] within the same backbone family”) or add the missing annotations.

  • Several metric values repeat suspiciously across conditions; please check them against unrounded numbers. Specifically: Table 1 (p. 5) shows GREEN = 0.510 for McondM_{\text{cond}}, SelfCue, and SelfCue-KD on COLIPRI+Llama (three identical in a row); SelfCue [email protected] = 0.481 on both COLIPRI+Llama and COLIPRI+MedGemma; Table 5 (p. 9) shows reported vs. held-out means identical for McondM_{\text{cond}} [email protected] (0.430/0.430), McondM_{\text{cond}} GREEN (0.510/0.510), and SelfCue-KD GREEN (0.510/0.510). Please report unrounded values or add a note explaining the coincidences.

  • The Table 4 mention-rate construction for condition-free systems is not stated. For MbaseM_{\text{base}} and “McondM_{\text{cond}}, plain prompt” no condition is in the prompt, so the split must use the probe’s condition for all systems. Please state this explicitly in the caption or text.

  • McondM_{\text{cond}} under the plain prompt degrades substantially [email protected] 0.299 vs. MbaseM_{\text{base}} 0.395, Table 4, p. 8), even though the plain branch is the reference in Eq. (1) and the 30% plain-prompt mixture was intended to “keep the plain branch in distribution.” This unexplained degradation deserves discussion; please add a paragraph interpreting this result.

  • The abstract’s claim that SelfCue “drops into any pipeline already serving the baseline” is broad: the student requires the same visual encoder/compressor and prompt format. Please qualify the drop-in claim (e.g., “drops into any pipeline already serving the baseline with the same visual encoder and prompt format”).

  1. Spelling, Grammar, Acronyms

Thank you for preparing the manuscript with careful attention to cross-referencing. I have two points to raise, one positive and one that will improve consistency for readers.

  • Please expand all acronyms and symbols at first use. I noticed at least one instance of an abbreviation or symbol used without a definition at first use (e.g., Abstract, p. 1), and the same convention should be checked throughout the whole document rather than only at the listed locations. Specifically, the following acronyms are used before being defined or are never expanded:
  • LLM (Abstract) — never defined.
  • GREEN (Abstract) — first defined in Section 5.1 but never expanded there.
  • AUROC (Abstract) — never expanded.
  • KD (Abstract, in “SelfCue-KD”) — never defined.
  • [email protected] (Contributions, p. 2) — defined only later in Section 5.1.
  • F1max (Table 1) — defined only in Section 5.1.
  • τ\tau (Table 1 caption) — defined only in Section 5.1.
  • LoRA (Section 5.1) — never expanded.

For each of these, please provide the full expansion at the first occurrence in the abstract, main text, and table/figure captions, and then use the abbreviation consistently thereafter. This will help readers who encounter the terms early on and will align the manuscript with standard scholarly practice.

Minor editorial findings (typos and wording)

1 minor points, listed here rather than in the sections above so they do not obscure the substantive feedback. None of these affects the scientific content.

  • Typos and copy-editing artifacts: p. 4 'optimized' → 'optimised' (UK consistency); p. 6 'a report mentions 2.4 of the 3.4 abnormalities present' is ambiguous (average?); references: duplicated 'Shaun Lindsay, Shaun Lindsay' and 'Shuqiang Zhang, Shuqiang Zhang'; comma-mangled 'Guangyi, Zhang' and 'Yu, Wang'; 'Jean bastien Grill' → 'Jean-Bastien Grill';
Questions for authors
  1. Could you provide a complete list of all CT‑RATE systems you considered for Table 1, specify the inclusion/exclusion criteria (e.g., reporting [email protected] on the test split and providing a public checkpoint), and explain why the four systems you mentioned (NV‑Reason‑CT, Resolution Meets Reduction, Sparse Concept Channels, Disease‑Centric VLP) were omitted or how they could be incorporated in a secondary comparison?

  2. The manuscript evaluates only on CT‑RATE while Merlin (and optionally RadGenome‑ChestCT) are publicly available 3D CT vision‑language corpora. Please either add experiments on Merlin (and RadGenome‑ChestCT if feasible) or include a clear justification for restricting the study to a single dataset, explicitly addressing the impact on the claimed generalizability.

  3. To address the “surfacing gap” claim, could you run a control experiment that scores the same generated reports with both the CheapCT probe and RadBERT (or otherwise use a single scoring instrument for both hidden states and text) and report the resulting scores, or otherwise soften the claim to reflect the use of two different measurement tools?

  4. It is unclear how hyper‑parameter tuning was performed for the baseline systems you compare against. Could you describe the tuning protocol applied to all compared methods (e.g., search ranges, number of trials, validation criteria), or explicitly state that no common tuning was performed and justify this choice?

  5. Table 1 shows that SelfCue does not raise the GREEN score for two of the three encoder‑backbone pairs, contrary to the abstract and Section 6. Please clarify the GREEN results per configuration (e.g., “raises GREEN for COLIPRI+Llama and holds it within X for the others”) or adjust the claim accordingly.

1 comment

Thank you for the detailed review. We have added comparisons and clarified the measurement, evaluation protocols and scope of our claims.

Q1. Why were recent methods missing from the comparison?

Our original screen closed on 11 September 2026, before NV-Reason-CT's arXiv v1 on 23 September. We now include this concurrent work, RMR and Sparse Concept Channels in Table 2. Sparse Concept Channels reports macro-F1 of 0.492; 0.549 is micro-F1. Disease-Centric VLP uses a different 16-class report labeler, and its headline AUC measures classification. Appendix C records the candidates, backbones, metrics and screening decisions, including unresolved protocol details. All external scores come from the original papers. We also identify CLarGen's backbone as MedGemma-27B, mark RadAgent's score as approximately 0.220 from Fig. 2B, and limit the claims in “Published comparisons.”

Q2. How broadly do the findings apply?

We added FlexiCT, ASAP and HLIP to Table 1, extending the surfacing-gap evidence to six configurations with five visual representations and two LLMs. The gap remains 0.097–0.116. SelfCue itself is evaluated on the original three configurations. CT-RATE provides paired full reports and a shared 18-label evaluation; we keep our conclusions within this setting. Discussion now separates the gap finding from its correction and limits the evidence to SFT-trained generators and 18 categories. Other datasets, richer attributes and reinforcement learning remain directions for further study.

Q3. Could the gap come from using different readouts?

Probing measures recoverability; the probe and RadBERT are different readouts. To check reference-label sensitivity, we relabeled the same reference reports with gpt-oss-120b. Keeping prediction scores fixed changes the COLIPRI–Llama AUROC gap only from 0.1093 to 0.1075. Appendix B.1 also compares three pooling choices, including the last prompt position; their best-layer AUROCs differ by at most 0.010. These checks support the gap's persistence, while leaving the readout distinction explicit. We revised the Introduction and Section 3.2 to identify report expression as an important bottleneck without attributing the entire gap to token generation.

Q4. How were parameters and checkpoints selected?

Section 5.4 now makes independent selection explicit: each seed selects its checkpoint, α and τ on 3,000 training volumes held out by patient, maximizing F1max within 0.010 of its best GREEN. All three seeds select α=1 and τ=0.75. This check covers COLIPRI–Llama; the other configurations have validation-grid ablations. We will release full code and configurations, including search grids, checkpoint candidates and selection records. External methods use their published settings and scores.

Q5. Does GREEN improve consistently?

SelfCue improves expression of the 18 targeted abnormalities, while GREEN also assesses clinical details beyond these labels. Higher F1 therefore need not yield higher GREEN: NV-Reason-CT (Myronenko et al., 2026) reports F1 of 0.5923 and GREEN of 0.4072. Relative to the probe-conditioned baseline, GREEN is unchanged on COLIPRI–Llama and falls by 0.008 on VISD–Llama and 0.016 on COLIPRI–MedGemma. Table 2 reports these results. We revised “GREEN” and Discussion's “Complementary metrics” to explain this distinction.

Q6. How do reconstruction counts affect evaluation?

The Dataset paragraph explains that COLIPRI's 1,564 volumes retain one reconstruction per scan, while VISD's 2,987 include reconstructions sharing a report. Each configuration compares systems on the same volumes. For the three new encoders, using 1,564 instead of 3,039 volumes changes report AUROC by at most 0.00465 and [email protected] by 0.00607.

Q7. Are the KD training budgets matched?

Both KD students use the same initialization, CT training set, trainable modules, optimization settings and training budget. We evaluate their final checkpoints with the same decoding settings. This provides a controlled comparison of the two distillation approaches. Appendix B.4 gives the details; full configurations will be released with the code.

Q8. What mathematical and presentation details changed?

Section 4.2 defines disjoint, exhaustive folds and includes the plain prompt in Eq. (3); Dao et al. is an analogy for cross-fitting, applied here to probe conditions. Appendix B.4 defines zero F1 denominators as zero. Table 6 uses the same probe condition to group all systems, including plain-prompt models. The plain branch's [email protected] is 0.299, but SelfCue uses its logits at the same generated prefix, with weights and cues fixed. We clarified that 2.4 predicted and 3.4 reference categories are separate averages. GREEN and F1 scores coincide after rounding to three decimal places; their unrounded values differ. Table 1's new encoders also have different hidden-state and report AUROCs. We added dataset ethics information and clarified deployment scope, terminology and spelling.

Reviewer_xXTUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper is specific to the medical domain, proposing a slight modification to naive textual radiology report generation from volumetric chest CT data. The report is evaluated on 18 abnormality labels extracted by RadBERT, a separate publicly available model, looking at the only the validation split of one dataset, CT-RATE. This work applies a standard LLaVA-style recipe, namely, a frozen encoder (COLIPRI, VISD CT-CLIP), a trained projector, and LoRA tuning of an LLM, in this case Llama-3.1-8B and MedGemma-27B. The motivation lies in the gap between the ~0.85 AUROC of a linear probe trained on the LLM's hidden states and the ~0.74 AUROC obtained by running RadBERT over the generated report. The authors describe this as a "surfacing gap" and argue that the information is present in the model but lost when the hidden state is decoded into text. As a solution they propose a 3 step method. 1) the probe is run on a new sample and its predicted abnormalities are written as short sentences to append to the prompt. 2) the model is run with and without the helper sentence. 3) the two logits are combined so the output is pushed further in the direction the sentence moves it. Although this seems to be classifier free guidance, the authors call this contrastive decoding. They fine-tune the model on ~30% without sentence, 35% true and corrupted labels. They add distillation to help with 2x forward passes. The guided model writes a report per sample and a copy of the model is fine-tuned on those reports with the plain prompt. As such, the probe can be discarded at test time. Validation is single dataset and ranges from ~0.4 - 0.48 with the contrastive method and the authors show that it is primarily recall driven improvement. What the experiments show is that feeding a classifier's prediction into the generator and amplifying increases recall on that classifier's label set. It is not shown that the generator knows more than the classifier since, SelfCue is ever only able to recover about 60% of the gap, implying the probe alone scores higher on the paper's main metric and there may be more straightforward methods to generate an accurate report. There are also limited comparisons that the new decoding rule surpasses other prompt preparation approaches.

Critical strengths

The most useful result is the cross-fitting finding for distillation. When the teacher’s reports are written under randomly corrupted labels rather than real probe predictions, the student loses a large amount of F1 and mentions far fewer findings per volume, which shows that a student can learn from errors that correlate with the image but not from random ones. This is a transferable lesson for anyone distilling a noisy teacher, and is the one part of the paper that stands regardless of how the main claim is resolved.

The experimental work is careful within its scope. The gap appears with the same shape on two encoders and two LLMs, and the ablations show the gain comes from the contrast step rather than from the helper sentence alone on all three combinations.

Critical weaknesses

The evaluation seems circular. The input to the method is the probe’s prediction for the 18 labels and the output is scored by extracting the same 18 labels from the report, so the experiments measure how well the generator passes the probe’s prediction through to the text. The probe’s own F1 is never reported, but the paper states that SelfCue recovers ~60% of the gap between the base report and probe. Simply printing the probe’s predictions would be expected to match or score higher than any of the generated reports. Consistent with this, the contrast step changes nothing on the metrics that are not derived from the 18 labels and all three encoder and LLM combinations converge to nearly the same F1 despite hailing from very different base models. Crucially, this paper never tests whether the report contains anything novel or interesting that the probe does not. Thus the central claim that the generator knows more than it says is not strongly supported beyond a classifier-to-text bridge.

Comparisons are also lacking. Table 1 only lists published numbers from different setups, and Dia-LLaMA (MICCAI 2025), which does this for CT, is not cited. The one in-house prompt baseline is weak by construction since it is trained on true or corrputed labels but tested with real probe predictions (and output under plain prompt is below base model). The simplest control, a fixed template that writes one sentence per predicted label and therefore scores exactly what the probe scores, is missing. CT-AGRG attempts this but is a published number with a different backbone. A better decoding rule over classifier-in-prompt methods is also untested.

Questions for authors

Does the report improve on anything the probe was not trained on, for example labels held out from the probe, or location, size and laterality, and how does SelfCue compare with a fixed template that writes one sentence per predicted label on the same split?

2 comments

Thank you for the review. Our main contribution is identifying a surfacing gap: CT report generators do not fully express abnormality information recoverable from their representations. We hope this finding encourages further research into closing the gap, with SelfCue providing one approach. The baseline uses predicted labels in the prompt, while SelfCue additionally contrasts conditioned and plain outputs. We recognize that our original presentation made this distinction unclear. We have renamed direct generation with predicted labels in the prompt the “Probe-conditioned baseline” and clarified how SelfCue adds a decoding intervention. We have also narrowed our claims in Discussion and Limitations: the demonstrated correction covers 18 categories, while similar gaps in finer clinical details remain a hypothesis and their recovery requires further exploration.

Q1. Does the report improve on findings or attributes the probe was not trained on?

SelfCue improves the expression of the 18 targeted categories. Extending these gains to other findings and finer clinical details remains future work, as clarified in Discussion and Limitations.

Q2. How does SelfCue compare with a fixed template on the same split?

We used the fixed sentence templates from TG-CTRG (Kyung et al., 2026), the second-place solution for CT report generation in the VLM3D Challenge at MICCAI 2026. For each scan, we threshold our probe predictions, write one positive or negative sentence for each of the 18 categories, and join these sentences into a report. We added this comparison, including probe-only scores, to Table 3 of the revised paper. Using the same probe predictions and validation split, SelfCue achieves higher report F1 and much higher GREEN across all three thresholds. The templates express category labels but leave out patient-specific details beyond them. These results support the value of combining more complete abnormality expression with full report generation, as discussed in the new “Probe and template controls” paragraph.

Q3. What does SelfCue add beyond classifier-in-prompt methods and CFG?

We agree that classifier-free guidance (CFG) and putting predicted labels in a prompt are established ideas. Our contribution is finding the surfacing gap and showing how decoding and distillation can help narrow it. The baseline uses predicted labels in the prompt, while SelfCue additionally contrasts conditioned and plain logits from the same model at the same generated prefix. With weights and labels fixed, [email protected] improves from 0.430 to 0.481 while GREEN remains similar. SelfCue-KD then retains much of this gain without the condition at inference. We now state the CFG connection after Eq. (1) and rename ordinary conditioned generation the “Probe-conditioned baseline.”

Q4. Is the prompt baseline a fair comparison?

We tested whether using actual probe predictions during training would strengthen the prompt baseline. Five-fold cross-fitting gives each training volume a condition from a probe that had not seen it. In Table 5 of the revision, the two training choices give broadly similar results: cross-fitted predictions give [email protected] lower by 0.026 and F1max lower by 0.021. Our mixture performs slightly better in this comparison. The plain branch scores 0.299 [email protected] on its own, but SelfCue uses it to get reference logits at the same generated prefix. With model weights and predicted labels fixed, SelfCue reaches 0.481 [email protected], compared with 0.430 for the probe-conditioned baseline and 0.395 for the baseline generator. This shows a decoding gain for this fixed conditional model.

Q5. What does the surfacing-gap measurement establish beyond a classifier-to-text bridge?

The key finding is that the same abnormalities are more recoverable from the generator's internal representations than from its reports. Probing measures recoverability, while SelfCue tests whether decoding can help express more of this information. We added FlexiCT, ASAP and HLIP to Table 1, and the hidden-to-report AUROC gap remains 0.097–0.116 across all six configurations. We also checked reference-label sensitivity: replacing the released labels with gpt-oss-120b annotations changes the COLIPRI–Llama gap only from 0.1093 to 0.1075, with prediction scores fixed. We now describe report expression as an important bottleneck in Section 3.2, rather than assigning the whole gap to token generation.

Q6. Are the published comparisons sufficiently controlled and complete?

We screened published results for CT-RATE full-report evaluation with the official RadBERT/GREEN weights; all external scores come from the original papers. We added NV-Reason-CT, RMR and CCP-10 to Table 2, and Dia-LLaMA to Related work. Appendix C lists the candidates, configurations and inclusion/exclusion reasons, including Dia-LLaMA's different dataset and evaluator. These comparisons provide context; SelfCue's decoding gain is tested against the probe-conditioned baseline with model weights and predicted labels fixed. We revised “Published comparisons” to clarify this scope. SelfCue achieves competitive results with a simple pipeline and few added components.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.