acceptodds
Under review as a conference paper at ICLR 2027

Seeing Is Not Saying: Closing the Verbalization Gap in 3D CT Report Generation

Abstract

Recent work on 3D Computed Tomography (CT) radiology report generation has largely focused on improving visual representations and the evidence supplied to language models. We identify a substantial gap between what is recoverable from visual representations and what is ultimately expressed in generated reports. On the example of thorax abnormalities, lightweight probes across five frozen encoders recover substantially stronger abnormality signals than report generators express, including for encoders not trained with CT reports. We term this discrepancy the verbalization gap and address it by improving both the evidence available for generation and how that evidence is expressed. Our framework consolidates complementary CT representations by slot-wise convex weights learned over independently resampled tokens and introduces graded verbalization, which guides report generation with diagnostic predictions whose linguistic certainty reflects their class-level reliability. With a 3B language backbone, our system achieves 0.597 macro F1 and 0.526 CRG on CT-RATE, a relative improvement of 44% over AdaRAG-CT and 180% over BTB3D. Under zero-shot cross-dataset evaluation on RAD-ChestCT, it achieves 0.498 macro F1, 123% above BTB3D. These results indicate that 3D CT report generation is limited not only by what can be recovered from visual representations, but also by how effectively recoverable clinical findings are translated into language.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
radiology report generation, 3D computed tomography, large language model

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_W1kDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores the verbalization gap between the information contained in visual representations and what is stated in text reports, for 3D computed tomography (CT) scans. The proposed framework then addressed said gap with explicitly graded verbalization as learnt from resampled tokens, to include a measure of certainty (or confidence) into the linguistic representation reflected in derived radiology reports. The effect is represented by significant gains in clinical efficacy (Table 2) on CT-RATE, and does address a pertinent shortcoming with CT radiology reports.

Critical strengths
  • Articulates the existence of a verbalization gap and attempts to mitigate it through the inclusion of graded verbalization conditioning
Critical weaknesses
  • Further attribution of verbalization gap (to model or conventional limitations) not made
  • Lack of some natural comparisons, such as against raw classifier predictions
Questions for authors
  1. The central issue identified in this study is the presence of a verbalization gap between clinically-relevant information derivable from input CT images (as measured by abnormality label classification), and that which is derivable from the CT (text) report generated from the same CT image(s).

While this is indeed a relevant shortcoming of typical (automatically generated) CT reports, it is less clear as to whether the verbalization gap arises more from an actual inability of existing CT report generation models to determine (and include) such clinically-relevant information, or more because typical CT report formats and practices do not encourage the inclusion of such information (perhaps to aid conciseness for human readers), despite the model(s) being able to provide it.

For example, the Introduction performs linear probing on 18 abnormality labels to quantify the retrieval of relevant clinical information, from both modalities (CT image and generated text report). In principle, it is likely possible for the (multimodal) LLM that generated the text report to have included estimated probability scores for all 18 labels (as indeed shown under Graded Verbalization in Figure 2), if so prompted. Such a structured report would certainly provide more fine-grained information for each abnormality, than a typical human-style CT report (e.g. Figures 2, 5 & 6) that tend to produce binary (absent/present) judgments, or even the proposed method (which tends to use more fine-grained descriptors such as "minimal" and "more clearly", from Figure 5).

The distinction between the ability of existing models to derive clinically-relevant information, and the choice as to whether and how to express such information in the report, might therefore be emphasized and discussed if relevant.

  1. Related to the above, an involved analysis for a structured report generator that (simply) directly expresses/translates raw probabilities for all abnormalities in terms of CRG for clinical efficacy - and also computes F1F_1, precision, recall and accuracy based on the explicitly-stated probabilities - may be appropriate. This would further quantify the verbalization gap in terms of model ability (upper limit) and loss due to stylistic conventions for text report formatting.

  2. In Sections 3.2 and 3.3, it is stated that graded verbalization includes evidence/information from a separate image-based classifier, by converting the raw classifier probability predictions into an additional prompt for further conditioning the final LLM decoder on the visual tokens (Figure 2).

This does imply that the quality of the separate classifier may significantly impact the accuracy and utility of the generated prompt. Given this, details on the training methodology of the classifier should be fully presented in the Appendix (Section B), especially for the training data used and (internal) validation metrics.

  1. Related to the above, the probabilities from the separate classifier can in principle be used to generate a structured text report using the various decision thresholds, without requiring an additional LLM decoder (e.g. by asserting "high probability for <abnormality>" for meeting the high precision threshold, and so on). If possible, a direct assessment of the binarized classifier output might be also presented in Table 2.

  2. In Section 3.2, the minimal LOW assertion level contains all πc<0.65\pi_c < 0.65, and corresponds to "cannot be excluded". While some ablation was attempted (Table 9), it might be discussed as to whether an abnormality might in practice be excluded, and whether additional threshold(s) on that possibility would have been appropriate.

  3. In Section 4, it is stated that for CE, generated reported are converted into 18 binary abnormality labels using a RadBERT-based classifier. The application and validation of this conversion might be described in supplementary material, and it might further be clarified as to whether the lack of mention of an abnormality in report text would be translated as the absence of that abnormality.

2 comments

Thank you for the thorough review and the concrete suggestions. They let us better quantify where the verbalization gap comes from.

Q1. Model ability vs. reporting convention. Three observations separate the two.

  1. The 18 CT-RATE labels are derived from the reference reports, and the evaluator recovers them from those reports with macro-F1 0.998. Every positive label is therefore written in the human report, so the convention includes these findings and a generator that imitates it should write them too.
  2. With the visual pathway, decoder and training reports unchanged, giving the predicted findings in the prompt raises F1 from 0.421 to 0.611 (plain list) and 0.622 (binary list). The report style stays the same, only the availability of the findings as text changes.
  3. The decomposition below locates the loss (Table 2).
Step (CT-RATE validation)F1Loss vs. classifier
Classifier on frozen features0.636n/a
Fixed template of its 18 decisions0.6220.014
Free-text report, findings in prompt0.597 to 0.6220.039 to 0.014
Free-text report, no guidance0.4210.215

Stating the information as text costs 0.014 to 0.039, while deriving it from the visual tokens costs 0.215. Most of the gap is the model's inability to turn visual evidence into stated findings, and style accounts for a small part. A probe on the 128-token prefix the decoder receives still recovers the labels at macro-F1 0.607 (Appendix H.4), so the information reaches the decoder and is lost in decoding.

Q2. Structured probability report as upper limit. Writing all 18 probabilities into a template leaves the predictions unchanged, so the classifier itself (F1 0.636, CRG 0.565) is this upper limit, and the template row above shows that fixed-form text loses 0.014. Both rows are now in Tables 2 and 3.

Q3. Classifier details. Same patient-level training partition as the generator, FlexiCT frozen. Gated attention-MIL pooling (Ilse et al., ICML 2018; hidden 256, dropout 0.25), LayerNorm, 512-d GELU MLP, 18 outputs. AdamW (lr 1e-3, weight decay 1e-4, 600 warm-up steps, cosine decay, batch 128) with the class-weighted BCE of Eq. 2. Model selection and per-class thresholds (grid search maximizing F1) use the early-stopping split, with no further calibration.

SplitF1PrecisionRecallAUROCCRG
Early-stopping (thresholds)0.6450.5890.7260.8950.576
CT-RATE validation0.6360.5920.6970.8810.565

Appendix B now contains these (Table 10) and a per-class table with threshold, precision, tier, AUROC, F1 and support (Table 11).

Q4. Classifier output in the main table.

MethodCRGF1PrecisionRecallBLEU-4ROUGE-L
Classifier0.5650.6360.5920.697n/an/a
Classifier + template0.5390.6220.5890.6810.0000.029
Ours, graded0.5260.5970.5940.6160.2420.343
Ours, all-18 binary0.5460.6220.5970.6580.2470.355

Our binary variant matches the template in F1 and exceeds it in CRG and precision, while writing a full report with localization, measurements and findings beyond the 18 labels (Figs. 5 and 6). The template carries only the 18 decisions, which shows in its NLG scores. All rows are now in Table 3.

Q5. Exclusion threshold. LOW applies only to findings above their class threshold, so it means "selected, from a class with lower precision". Below-threshold findings are left unspecified in the graded prompt. The all-18 binary variant tests explicit exclusion, since it states absent for every class below threshold, and it gives the highest CE (F1 0.622, recall 0.658). A separate high-NPV threshold for reliable absence is a natural refinement, which we discuss in Sec. 5.

Q6. Evaluator. We use the RadBERT-based evaluator of prior CT report generation work (Hamamci et al., 2026b). Negated and unmentioned findings are both scored as absent. It reproduces the official labels from the reference reports with macro-F1 0.998 (lowest class 0.991, Table 16) and loses 0.014 F1 on the template text above. Hedged statements are partly scored as absent, e.g., writing the same decisions with graded phrases lowers recall from 0.681 to 0.499 (Appendix H.1).

We hope this addresses your questions and would be grateful if you considered raising your score.

We thank the authors for the comprehensive clarifications.

Reviewer_UmVAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the verbalization gap in 3D CT radiology report generation, where clinically relevant abnormalities remain recoverable from visual representations but are not adequately expressed in generated reports. The authors first analyze five frozen visual encoders using lightweight diagnostic probes to demonstrate this discrepancy. They then propose a three-stage framework consisting of slot-wise multi-encoder consolidation, reliability-aware graded verbalization, and LoRA adaptation of the language model. Experiments on CT-RATE and RAD-ChestCT demonstrate substantial improvements in clinical efficacy, achieving 0.597 macro F1 and 0.526 CRG on CT-RATE, together with 0.498 macro F1 under external evaluation on RAD-ChestCT.

Critical strengths
  1. Well-motivated research problem. The distinction between information recoverable from visual representations and information expressed in generated reports is meaningful and clinically relevant. The analysis across five encoders provides useful empirical evidence that limitations in report generation cannot be attributed solely to insufficient visual representations.
  2. Effective and computationally compact multi-encoder fusion. The proposed slot-wise consolidation combines complementary visual representations while maintaining a fixed 128-token prefix. The comparison against single encoders and alternative fusion strategies provides reasonable empirical support for this design.
  3. Strong experimental results and comprehensive evaluation. The proposed method substantially outperforms existing baselines on CT-RATE and demonstrates promising cross-dataset generalization on RAD-ChestCT. The authors also provide component ablations, fusion comparisons, language-backbone and token-budget analyses, and sensitivity studies. These experiments strengthen the empirical contribution of the paper.
Critical weaknesses
  1. The contribution of graded verbalization is not sufficiently isolated. The current ablation compares generation with and without graded diagnostic prompts. However, this simultaneously changes the availability of explicit diagnostic predictions and the linguistic certainty assigned to them. Consequently, the improvement cannot be attributed specifically to graded verbalization. A comparison against ungraded diagnostic prompts containing identical predicted findings is necessary to establish the contribution of the proposed reliability-aware formulation.
  2. Class-level precision provides a limited representation of diagnostic uncertainty. The proposed assertion tiers are determined by precision measured for each abnormality class and remain fixed across individual cases. Therefore, predictions with substantially different confidence scores receive the same linguistic certainty as long as they belong to the same class. Moreover, the paper does not directly evaluate whether the generated uncertainty expressions are appropriately calibrated or clinically meaningful. This limits the evidence supporting the reliability-aware interpretation of the method.
  3. The training and inference prompting strategies are inconsistent. During training, diagnostic prompts are constructed using ground-truth positive labels, whereas inference relies on predictions from an auxiliary classifier. This introduces a distribution shift between training and deployment. The paper would benefit from evaluating predicted-label conditioning during training and examining how errors in the auxiliary classifier propagate into the generated reports.
Questions for authors
  1. Can the authors provide an ablation comparing graded verbalization against binary diagnostic prompts using exactly the same predicted abnormality labels? This would help determine whether the improvement originates from the certainty-aware wording or primarily from the additional diagnostic information.
  2. Why is class-level precision preferred over calibrated instance-level prediction confidence for determining linguistic certainty? Have the authors considered evaluating the calibration or clinical appropriateness of the generated uncertainty expressions?
  3. What is the motivation for using ground-truth labels during training but predicted labels during inference? Would training with predicted diagnostic prompts improve robustness to false-positive and false-negative classifier predictions?
1 comment

Thank you for the detailed review. The new experiments confirm the main finding of the paper, and we answer the three points in order.

Q1. Graded vs. binary prompts with the same labels. All variants share the classifier, the selected findings per volume, the visual prefix and the three training stages. Plain lists the selected findings without certainty phrases. All-18 binary states present or absent for every class.

Prompt (Stage 3)CRGF1PrecisionRecall
Graded (paper)0.5260.5970.5940.616
Plain0.5390.6110.5930.639
All-18 binary0.5460.6220.5970.658

All three formats support the main finding, since with the visual pathway and decoder unchanged, adding the classifier's findings as text closes the gap to the auxiliary classifier (F1 0.636) from 0.215 to between 0.039 and 0.014. Every variant exceeds the strongest prior method (AdaRAG-CT, 0.414) by 44% to 50% in macro-F1, and the graded model also transfers zero-shot to RAD-ChestCT (0.498, 123% above BTB3D). The diagnostic information accounts for the CE gain, and the wording of the prompt is a secondary design choice. Graded wording makes the generator more conservative with prompted findings (Q3), while the binary format gives the highest CE. Table 5 now reports all three, and Sec. 4.1 describes the prompt format accordingly.

Q2. Class-level vs. instance-level confidence. Using class-level precision was a deliberate choice, for three reasons.

  • The classifier is trained with class-weighted BCE (w_c = n_c^- / n_c^+), which inflates positive scores, so raw probabilities are not calibrated. Precision at the threshold is an empirical frequency on 3,039 held-out volumes.
  • Training prompts use reference labels, which have no instance-level score. Instance-level tiers would need cross-fitted predictions on the training set.
  • The tiers hold out of sample. Set on the early-stopping split, they give 77.4% (HIGH, n = 2,574), 71.8% (MED, n = 1,516) and 52.6% (LOW, n = 8,816) correct prompted findings on the validation split (Appendix B).

A class-level tier tells the generator how far a finding of that class can be trusted in general. Graded wording moves the share of hedged sentences toward the reference reports (4.71% graded, 3.97% plain, 5.76% reference), and before LoRA adaptation hedged findings are less often correct than definite ones, with the largest separation for graded guidance (6.1 pp vs. 4.2 and 2.7 pp, Appendix H.2). The framework is agnostic to how confidence is expressed, so calibrated instance-level tiers can be used without other changes. We discuss them in Sec. 5.

Q3. Reference labels in training, predictions at inference. Training on reference labels is a deliberate choice, since it teaches the generator to write correct findings in the radiologist's language without absorbing the error pattern of one particular classifier, and it avoids the overconfident scores a classifier gives on its own training data. It also makes the generator classifier-agnostic. With reference findings in the prompt, the same generator reaches 0.892 macro-F1 and 0.778 CRG without retraining (Appendix H.4), so better classifiers translate directly into better reports.

Wrong suggestions keptGradedPlainAll-18 binary
Stage 261.8%68.6%74.7%
Stage 386.1%90.7%91.3%

Graded guidance keeps the fewest wrong suggestions at both stages. LoRA raises retention of correct and wrong suggestions alike (Appendix H.3), and cross-fitted predicted prompts are the natural next step (Sec. 5). Graded reports also follow the patient's own CT most closely. When visual tokens are swapped between patients with the prompt fixed, graded reports move toward the donor's labels by +0.081 F1, against +0.027 for plain and +0.023 for binary prompts (Appendix H.4).

We hope this answers your concerns and would be grateful if you considered raising your score.

Reviewer_CGq5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a verbalization gap in 3D CT report generation, which means abnormality labels are more accurately predicted from forzen visual representations than recovered from generated reports. The proposed framework combines three pretrained encoders through learned slot-wise fusion, supplies diagnostic predictions as prompts with class-level reliability-dependent wording, and adapts the language model using LoRA. The system improves abnormality-label performance on both CT-RATE and RAD-ChestCT.

Critical strengths
  1. The motivation is clear and relevant. The probe experiments suggest that usedul diagnostic information is already available in the visual features, but is not fully reflected in the generated reports.
  2. The improvement on CT-RATE is substantial, with gains in both precision and recall. The external evaluation on RAG-ChestCT further supports the usefulness of the approach.
  3. The paper includes a fairly extensive experimental analysis, covering fusion alternatives, training stages, language backbones, and token budgets.
Critical weaknesses
  1. The motivation figure does not fully isolate the verbalization gap. Figure 1 compares a supervised classifier with report generation followed by label extraction. The observed difference may reflect several factors, including decoder training, visual–language alignment, and label-extraction errors. Their contributions are not clearly separated, which limits the interpretation of the gap.
  2. The evaluation provides limited evidence for the claimed improvement in uncertainty expression. The substantial gains in abnormality precision and recall support better finding-level agreement. However, binary label metrics do not directly assess whether graded verbalization produces appropriately qualified statements. A targeted evaluation of uncertainty wording would strengthen this claim.
  3. The contribution of graded wording is not isolated. Table 4 adds predicted findings together with their graded descriptions. A baseline using the same predictions without graded wording is needed to determine whether the benefit comes from the proposed verbalization scheme or simply from supplying diagnostic predictions.
Questions for authors
  1. Regarding Figure 1, can you estimate how much report-label extraction errors contribute to the measured gap, and clarify how decoder training and visual–language alignment affect its interpretation?
  2. Does graded verbalization improve the appropriateness of uncertainty expressions in the generated reports? Can you provide a targeted evaluation beyond binary abnormality-label agreement?
  3. How does the method perform when given the same predicted findings without graded wording, under the same training setup? This would help separate the benefit of diagnostic guidance from that of graded verbalization.
2 comments

Thank you for the constructive review. Each question led to a new analysis, and together they sharpen the central claim of the paper.

Q1. What Fig. 1 isolates. Label extraction. On the CT-RATE reference reports the evaluator reproduces the official labels with macro-F1 0.998 (lowest class 0.991). As these labels were derived from the reports, we also tested newly written text. Writing the auxiliary classifier's 18 decisions as "There is X / There is no X" and extracting them again changes F1 from 0.636 to 0.622, a cost of 0.014, small against the gaps of 0.22 to 0.26 in Fig. 1 (Appendix H.1). Decoder training. A larger or different decoder (Llama-3.1-8B, Qwen2.5-3B) and 64 to 1024 visual tokens do not close the gap (Table 7). When the findings are given as text, the same decoder writes them, and F1 rises from 0.421 to 0.611 (plain list) and 0.622 (binary list). So the decoder can write the findings, it just fails to get them out of the visual tokens. Alignment. A two-layer MLP probe on the mean-pooled 128-token prefix that enters the LLM reaches macro-F1 0.607, while the report generated from the same prefix reaches 0.410 (Appendix H.4). The information survives fusion and alignment and is lost in decoding. Fig. 1 measures the whole pipeline by design, and the revised Sec. 1 adds this decomposition (Table 2).

Q2. Uncertainty wording. We classified every generated sentence that mentions one of the 18 findings as definite, possible or cannot-be-excluded with fixed regular expressions (Appendix H.2).

ReportsDefinitePossibleCannot be excluded
Reference94.24%5.50%0.264%
Graded95.29%4.58%0.128%
Plain96.03%3.91%0.059%

Graded guidance moves all three proportions toward the reference reports. The tiers hold out of sample, with 77.4%, 71.8% and 52.6% of prompted HIGH, MED and LOW findings correct. Before LoRA adaptation, hedged findings are less often correct than definite ones, and the separation is largest with graded guidance (55.3% vs. 61.4%, a 6.1 pp gap against 4.2 pp for plain and 2.7 pp for binary prompts). After LoRA this separation becomes small for all formats. Graded guidance also keeps the fewest wrong suggestions at both stages (86.1% at Stage 3 vs. 90.7% plain and 91.3% binary).

Binary label metrics cannot assess this, and in fact they penalize it. The evaluator scores part of the hedged statements as absent. Writing the same classifier decisions with graded phrases instead of "There is X" lowers recall from 0.681 to 0.499, so some of the difference between graded and plain prompts in Q3 is likely a measurement effect. A clinical judgement of each hedge needs radiologist annotation, which we list as a limitation in Sec. 5.

Q3. Same predictions without graded wording. All variants share the classifier, the selected findings, the visual prefix and the training stages.

Prompt (Stage 3)CRGF1PrecisionRecall
Graded (paper)0.5260.5970.5940.616
Plain0.5390.6110.5930.639
All-18 binary0.5460.6220.5970.658

All three formats support the main finding. With the same visual pathway and decoder, the gap to the auxiliary classifier (F1 0.636) shrinks from 0.215 to between 0.039 and 0.014, and the binary variant is a new best result on CT-RATE, 50% above AdaRAG-CT in macro-F1. The diagnostic information accounts for the CE gain, and the wording of the prompt is a secondary design choice. Compared with plain prompts, graded wording costs about 0.014 F1 and in return makes the generator more conservative, its hedging closer to the reference reports and its reports three times more responsive to the patient's own visual tokens (donor lift +0.081 vs. +0.027 when visual tokens are swapped between patients, Appendix H.4). Table 5 now reports all variants.

We hope this answers your questions and would be grateful if you considered raising your score.

The results strengthen the empirical findings, although the specific benefit of graded wording and the appropriateness of uncertainty expressions reamin less established. I will maintain my current rating.

Reviewer_8njHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies why abnormalities detectable in CT features are often missing from generated reports. It analyzes this gap, fuses three frozen CT encoders into 128 visual tokens, and uses diagnostic prompts with certainty levels based on class-level precision, followed by LoRA fine-tuning. The authors evaluate the approach through probes on five encoders, fusion comparisons, ablations, and cross-dataset testing.

Critical strengths

The method combines information from different CT encoders and gives the language model explicit diagnostic prompts, helping it include abnormal findings that might otherwise be left out of the report. The gains also carry over to another dataset, which supports the usefulness of this approach.

The comparisons across encoders and model settings also help show how different design choices affect reporting quality, providing useful guidance for building CT report generation systems.

Critical weaknesses

The main idea overlaps with PromptMRG, which already showed a gap between classification and report generation and used diagnostic prompts to reduce it. The paper does not clearly establish what its new analysis and prompting scheme add beyond that work.

The experiments add diagnostic labels and certainty-based wording together, so the gains may simply come from telling the model which abnormalities to mention. Without a comparison against plain diagnostic prompts, the value of the certainty levels remains unclear. The paper also does not check whether the final reports express certainty that matches prediction reliability.

Questions for authors

With the same predicted abnormalities and training setup, how does graded wording compare with simply listing the abnormality names without certainty phrases?

Can the generator change the certainty levels given in the prompts? In Figure 2, some “cannot be excluded” findings become definite statements. What evidence shows that these changes are justified?

How does the full model compare with a report generator trained on the same diagnostic prompts but without visual tokens? This would show how much the visual tokens add at the report-generation stage.

1 comment

Thank you for the careful review. The experiments you asked for produced a new best result on CT-RATE (0.622 macro-F1, 50% above AdaRAG-CT) and close the gap to the auxiliary classifier to 0.014.

Q1. Graded vs. plain prompts with the same predictions. All variants share the classifier, the selected findings per volume, the visual prefix and the three training stages. The second row lists the selected findings without certainty phrases. The third row states present or absent for every class, i.e., a PromptMRG-style text prompt.

Prompt (Stage 3)CRGF1PrecisionRecall
Graded (paper)0.5260.5970.5940.616
Plain0.5390.6110.5930.639
All-18 binary0.5460.6220.5970.658

The comparison shows that the CE gain comes from supplying the findings, the mechanism PromptMRG introduced, and that it holds across prompt formats. How large the gain is depends on our framework (see below). Table 5 and Sec. 4.1 now report all three formats.

This directly supports the central claim of the paper. With the visual pathway and decoder unchanged, the gap to the auxiliary classifier (F1 0.636) shrinks from 0.215 to 0.025 (plain) and 0.014 (binary). The generator writes these findings once they are in the prompt, but does not get them out of the visual tokens on its own, although a probe on its 128-token prefix recovers them at macro-F1 0.607 (Appendix H.4). A class-level tier tells the generator how far a finding of that class can be trusted, which makes it more conservative. Graded guidance keeps the fewest wrong suggestions at both stages (86.1% at Stage 3 vs. 90.7% plain, 91.3% binary) for about 0.014 F1 against plain prompts.

Q2. Changing the certainty. Yes. The prompt conditions generation without fixing the wording, and the generator also sees the CT tokens. In Fig. 2 all four LOW-tier findings are present in the reference report, so the firmer statements are correct there. The tiers set on the early-stopping split stay ordered on validation (77.4%, 71.8% and 52.6% of prompted HIGH, MED and LOW findings correct). To test whether departures from the prompt follow the image, we swapped each patient's visual tokens for another patient's with the prompt fixed. Graded reports move toward the donor's labels by +0.081 F1 over the unswapped reports, three times more than with plain (+0.027) or binary prompts (+0.023), and more than with randomly reassigned tiers (+0.034) (Appendix H.4).

Q3. Prompt-only generator. We trained it with the same graded prompts and LoRA schedule, without the visual prefix (Appendix H.4). On the 18 labels it reaches F1 0.617 vs. 0.597, while BLEU-4 (0.223 vs. 0.242), ROUGE-L (0.318 vs. 0.343) and METEOR (0.238 vs. 0.249) drop. This is what the verbalization gap predicts. The 18-label content reaches the report through the text even though the prefix contains it (probe 0.607), and the visual prefix supplies the rest of the report, e.g., the aortic diameters in Fig. 5. With binary guidance our full model is above the prompt-only generator on both (0.622 F1, 0.247 BLEU-4).

PromptMRG and related work. PromptMRG is the closest work, and Dia-LLaMA (Chen et al., 2025) used diagnostic text prompts with an LLM for chest CT. Both are now discussed in Sec. 2. Three differences explain why guidance is far more effective in our framework.

  • Both train the classification branch jointly with the generator, on the generator's own features. We use a separate attention-MIL classifier on frozen 3D foundation features with calibrated per-class thresholds, and train the generator on reference labels. A better classifier can therefore be plugged in without retraining. With reference findings in the prompt, the same generator reaches 0.892 F1 (Appendix H.4).
  • Text prompts added 0.076 F1 in PromptMRG (MIMIC-CXR) and 0.027 in Dia-LLaMA (CTRG-Chest), against 0.19 to 0.20 here (0.421 to 0.611/0.622). The benchmarks differ, but a difference this large suggests that the decoupled, calibrated classifier is what makes guidance work in 3D.
  • Fig. 1 explains why guidance helps. We compare probe and generator on the same frozen features for 5 encoders, including DINOv3 and an ImageNet ResNet-18 that never saw a report, and the gap persists for 3B/8B and Llama/Qwen decoders and 64 to 1024 tokens (Table 7). PromptMRG compared a separate classifier with other published systems, which cannot tell missing information from missing verbalization.

Slot-wise consolidation adds 12% over the best single encoder and 9.3% over the best fusion baseline, independent of the prompt. We hope this resolves the overlap concern and would be grateful if you reconsidered your score.

Comments on the paper

1 comment

Summary of new results and revisions (all changes in blue in the revised PDF)

  • New best result on CT-RATE: 0.622 macro-F1 with all-18 binary guidance, 50% above AdaRAG-CT. All prompt formats exceed prior work by 44% to 50% (Tables 3 and 5).
  • With the visual pathway and decoder unchanged, guidance closes the gap to the auxiliary classifier from 0.215 to 0.014 (Table 2). A probe on the 128-token prefix the decoder receives recovers the labels at 0.607 macro-F1 while the unguided report reaches 0.410, which locates the verbalization gap in decoding.
  • Slot-wise consolidation of three frozen 3D encoders improves macro-F1 by 12% over the best single encoder and by 9.3% over the strongest fusion baseline (Table 6).
  • Plug-in guidance: with reference findings in the prompt, the same generator reaches 0.892 macro-F1 without retraining (Appendix H.4).
  • A guided free-text report matches a fixed template of the classifier output on F1 (0.622) and exceeds it on CRG, while the template has BLEU-4 0.000.
  • Graded wording keeps the fewest wrong suggestions and makes reports three times more responsive to the patient's own visual tokens (Appendix H.4).
  • Evaluator validation (0.998 macro-F1 on reference reports), full classifier details, PromptMRG and Dia-LLaMA discussion, and limitations added (Sec. 2, Sec. 5, Appendices B and H).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.