- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates the “diagnosis-to-report gap” in medical vision–language models: diagnostic VQA achieves higher macro-F1 than free-form report generation on the same CT studies and disease categories. To narrow this gap, the authors introduce DeCAD, which progressively decomposes CT report generation at diagnostic, regional textual, and regional visual levels. The model generates an explicit diagnostic assessment before report composition, organizes diagnostic assessment and report description by anatomical region, and restricts each regional task to the corresponding visual tokens. Diagnostic and regional textual decomposition consistently improve report-level diagnostic performance across five medical vision–language models, whereas regional visual decomposition provides limited additional gains. In a standard VLM without medical VQA pretraining, the first two levels remain effective, and regional visual decomposition provides further gains with a region-level contrastively pretrained visual encoder. Full DeCAD achieves diagnostic macro-F1 scores of 52.1% on CT-RATE and 39.4% on Merlin, demonstrating that explicitly organizing diagnosis and description improves diagnostic reporting. However, diagnostic evaluation remains restricted to labeled disease categories and relies on automated report labeling, motivating broader clinical evaluation and expert assessment.
Critical strengths
-
Consistent improvements across multiple models and datasets. Diagnostic and regional textual decomposition improve report-level diagnostic macro-F1 over Report Adaptation across all evaluated models on CT-RATE and Merlin. This consistency provides meaningful evidence that the approach has practical value across different model architectures and both chest and abdominal CT reporting.
-
A useful empirical characterization of the diagnosis-to-report gap. The paper compares diagnostic VQA and free-form reporting on the same CT studies using aligned disease categories and reference labels. Although this comparison does not fully isolate the causes of the gap, it highlights an important limitation of evaluating diagnostic capability through VQA alone and motivates evaluating whether identified findings are expressed in generated reports.
Critical weaknesses
-
The incremental methodological contribution over prior work is not sufficiently established. CT-AGRG: Automated Abnormality-Guided Report Generation from 3D Chest CT Volumes (ISBI 2025) already predicts abnormalities before generating corresponding descriptions. Dia-LLaMA: Towards Large Language Model-driven CT Report Generation (MICCAI 2025) uses diagnostic text prompts to guide report generation, while Large Language Model With Region-Guided Referring and Grounding for CT Report Generation (IEEE TMI 2025), referred to as Reg2RG, introduces regional visual features and region-report alignment. DeCAD differs by autoregressively generating the diagnostic assessment and report within the same model and systematically examining successive levels of decomposition. Nevertheless, its central methodological ideas substantially overlap with these established directions. Although the paper includes a comparison with Reg2RG, it does not provide direct quantitative comparisons with CT-AGRG or Dia-LLaMA, or controlled comparisons of alternative diagnostic guidance strategies using a shared backbone and equivalent supervision. Consequently, the experiments demonstrate effectiveness more clearly than they establish a substantive advance over prior approaches, which limits my assessment of the contribution's significance.
-
The interpretation of the central empirical findings is limited by insufficient controls. Although the diagnosis-to-report comparison uses the same CT studies and disease labels, diagnostic VQA explicitly names each disease and queries it separately, whereas free-form reporting requires the model to identify and organize findings without these disease-specific cues. The observed gap therefore does not fully distinguish differences in task prompting from failures to express diagnostic information during report generation. Furthermore, the paper links the benefits of regional visual decomposition in its standard VLM to region-level contrastive pretraining, but all settings in Table 3 use a regionally aligned encoder. A comparison within the same architecture, varying both regional alignment and regional visual decomposition, is needed to isolate this explanation. The current results support the reported performance differences but do not establish their causes. Because systematic empirical insight is a central contribution beyond the existing methodological components, these unresolved explanations weaken the case for acceptance.
Questions for authors
-
The incremental value of diagnostic decomposition over existing diagnostic guidance strategies. The paper positions its main distinction as a systematic analysis of different levels of decomposition. To clarify the contribution of diagnostic decomposition, do the authors have a comparison between autoregressively generated assessments and independently predicted diagnostic text prompts, using the same backbone and diagnostic supervision? Which existing results distinguish the benefit of assessment-before-report generation from that of adding explicit diagnostic supervision? This would help establish the substantive advance over strategies such as CT-AGRG and Dia-LLaMA.
-
The sensitivity of the diagnosis-to-report gap to prompting. The paper defines this gap as a task-dependent discrepancy and examines the role of report-specific fine-tuning through Report Adaptation. Have the authors also evaluated a reporting prompt that provides the same disease checklist as diagnostic VQA while keeping the model weights fixed? This control would help clarify how much of the gap is attributable to explicit disease cues and how far a simple prompting change can narrow it without additional training.
-
Evidence for the role of regional image–text alignment. Table 3 demonstrates that regional visual decomposition is effective with a fixed, regionally aligned encoder. Do the authors have further evidence attributing this benefit specifically to region-level image–text alignment, such as a comparison of the gains from visual decomposition under region-level versus global image–text contrastive pretraining within the same architecture? If no such control is available, should regional alignment be presented as a possible explanation rather than an established cause? This distinction would affect the assessment of the paper’s conclusions about when regional visual decomposition is beneficial.
2 comments
Response to Reviewer z2Kx (Part 2 of 2)
2. Concerns regarding the influence of disease-specific prompting on the diagnosis-to-report gap (W2, Q2)
We thank the reviewer for this insightful comment. We added a disease-specific prompting control that provides the same disease checklist as diagnostic VQA while keeping model weights and CT inputs fixed. As shown in Table R10, disease-list prompting improves F1 for all models, for example, from 17.8 to 27.2 for CT-CHAT and from 21.3 to 36.1 for HuluMed-7B, and outperforms Report Adaptation without additional training for three of five models. However, Diagnostic Decomposition still achieves higher F1 for every model, including 30.7 for CT-CHAT and 45.0 for HuluMed-7B. Therefore, disease-specific cues can contribute to the diagnosis-to-report gap, but explicit diagnostic supervision further improves the model’s ability to generate diagnostically complete reports beyond disease-specific prompting alone. We have updated the result of disease-specific prompting in Section E.2.2 and Table 11 in the revised Appendix.
| Setting | Fine-tuning? | CT-CHAT | MedGemma-1.5-4B | Lingshu-7B | HuluMed-7B | RadSight-8B |
|---|---|---|---|---|---|---|
| Direct | ✗ | 17.8 / 37.4 / 18.1 | 13.7 / 39.4 / 12.4 | 7.1 / 28.6 / 5.4 | 21.3 / 52.5 / 15.4 | 34.6 / 43.9 / 30.9 |
| Disease-specific prompting | ✗ | 27.2 / 24.4 / 40.7 | 20.6 / 23.7 / 38.2 | 10.8 / 15.0 / 17.6 | 36.1 / 31.0 / 59.5 | 38.0 / 37.1 / 40.7 |
| Report Adapt. | ✓ | 17.7 / 39.7 / 15.1 | 19.8 / 46.0 / 19.5 | 21.5 / 56.2 / 17.8 | 37.7 / 53.8 / 33.2 | 37.2 / 43.8 / 33.7 |
| Diagnostic Decomp. | ✓ | 30.7 / 43.9 / 30.1 | 32.3 / 50.7 / 28.9 | 30.8 / 47.2 / 27.8 | 45.0 / 57.4 / 42.9 | 42.2 / 45.3 / 40.6 |
Table R10. Disease-list prompting and diagnostic decomposition on CT-RATE. F1/P/R are macro-averaged diagnostic metrics (%).
3. Concerns regarding ablation study of regional image–text alignment (W2, Q3)
Thank you for highlighting the need to isolate the role of regional image-text alignment. We therefore repeated the decomposition ablation using the same architecture initialized with a whole-image contrastively pretrained encoder, rather than the regionally aligned encoder used in Table 3. As shown in Table R11, under whole-image alignment, diagnostic decomposition improves F1 from 36.2 to 43.8 on CT-RATE (25.3 to 30.1 on Merlin), and adding text decomposition further improves it to 46.9 (33.6). However, adding visual decomposition provides no further gain, decreasing F1 from 46.9 to 46.0 on CT-RATE and from 33.6 to 30.9 on Merlin. In contrast, with region-level alignment, visual decomposition improves F1 from 41.6 to 52.1 on CT-RATE and from 34.1 to 39.4 on Merlin when diagnostic and text decomposition are enabled. This interaction shows that the benefit of visual decomposition depends on region-level image-text alignment, rather than arising merely from adding another decomposition module. We have updated ablation of regional image–text alignment in Table 3 in the revised manuscript.
| Encoder pretraining | Diag. Decomp. | Text Decomp. | Visual Decomp. | CT-RATE F1 / P / R | Merlin F1 / P / R |
|---|---|---|---|---|---|
| Whole-image align. | ✗ | ✗ | ✗ | 36.2 / 43.2 / 32.0 | 25.3 / 29.2 / 23.9 |
| Whole-image align. | ✓ | ✗ | ✗ | 43.8 / 48.5 / 42.5 | 30.1 / 29.2 / 33.6 |
| Whole-image align. | ✓ | ✓ | ✗ | 46.9 / 51.5 / 45.8 | 33.6 / 30.7 / 39.0 |
| Whole-image align. | ✓ | ✓ | ✓ | 46.0 / 54.0 / 44.1 | 30.9 / 29.4 / 38.9 |
| Region-level align. | ✓ | ✓ | ✗ | 41.6 / 47.2 / 42.8 | 34.1 / 37.1 / 38.3 |
| Region-level align. | ✓ | ✓ | ✓ | 52.1 / 55.6 / 54.2 | 39.4 / 43.1 / 42.3 |
Table R11. Decomposition ablation with whole-image contrastive pretraining on CT-RATE and Merlin. The architecture and decomposition settings are identical to Table 3; only encoder pretraining granularity differs.
Response to Reviewer z2Kx (Part 1 of 2)
Thank you for recognizing the effectiveness and practical value of our work, as well as for your constructive suggestions to further strengthen the manuscript. We address your comments point by point below.
1. Concerns regarding comparison with existing diagnostic guidance strategies (W1, Q1)
Thank you for raising this point. First, for clarity in the following discussion, we distinguish the two diagnostic-guidance settings. The “independently predicted diagnostic text prompts” mentioned in the review correspond to external-classifier guidance, where a separate classifier predicts diagnostic labels that are then supplied to the report-generation model as text prompts. In contrast, our method uses autoregressive self-diagnosis guidance, in which DeCAD first generates a diagnostic assessment from the CT input and subsequently generates the report within the same model.
Second, we agree that comparisons with alternative diagnostic-guidance strategies are necessary to clarify our contribution. Since CT-AGRG does not provide public code, and Dia-LLaMA was originally developed on a dataset different from CT-RATE, we reimplemented Dia-LLaMA following its original training protocol on CT-RATE to ensure a matched data and evaluation setting. We further constructed an external-classifier-guided DeCAD variant, in which a linear classifier was trained by linear probing on DeCAD’s frozen visual encoder, and its predicted diagnostic labels were formatted as Dia-LLaMA-style prompts and provided to the LLM for report generation. The linear-probed classifier achieves an F1/precision/recall of 53.1/43.5/74.9. Both DeCAD variants use the same visual encoder, LLM, CT-report training data, and diagnostic supervision; they differ only in whether diagnostic information is supplied by an external classifier or generated autoregressively within the model. As shown in Table R9, our DeCAD achieves the highest F1 for report generation, indicating the advantage of its autoregressive self-diagnosis design.
Third, we acknowledge that external-classifier-guided methods may achieve strong report-generation performance when equipped with a strong classifier. However, their performance may depend on the externally supplied diagnostic information and the LLM’s instruction-following ability. Consequently, improved report-generation scores alone do not establish that the report-generating model itself grounds the generated findings in the visual input, rather than primarily following the diagnostic prompts. We therefore do not claim that self-diagnosis guidance universally outperforms a strong external classifier.
Importantly, we would like to emphasize that our contribution is not the general use of diagnostic guidance or regional representations, which have been explored by CT-AGRG, Dia-LLaMA, and Reg2RG. Rather, we rethink CT report generation by identifying the diagnosis-to-report gap in existing MedVLMs and systematically examining which forms of visual and textual decomposition help bridge it. Starting from our empirical analysis of existing MedVLMs, we identify practical principles for better translating visual diagnostic capability into free-form reports. DeCAD distills these insights into a lightweight design that can be implemented with standard SFT, without relying on large-scale medical pretraining or external diagnostic guidance at inference time. We believe this perspective is valuable because it shifts attention from adding increasingly powerful external components to understanding and better leveraging the diagnostic capability already present in MedVLMs.
We have added this discussion to the revised Appendix E.2.1 and Table 10 to further distinguish our approach from existing diagnostic guidance strategies. And we have also compared our approach with the reimplemented Dia-LLaMA in revised Table 4. Thank you again for your insightful suggestion.
Table R9. Comparison of external-classifier guidance and autoregressive self-diagnosis guidance on CT-RATE.