acceptodds
Under review as a conference paper at ICLR 2027

VEC-CT: Visual Cue-Guided Error-Corrective Clinical Reasoning for Lung CT Report Generation

Abstract

Multimodal large language models (MLLMs) provide strong capabilities in visual perception and reasoning, offering a promising approach to medical image understanding and diagnostic report generation. However, accurate interpretation of lung CT scans remains challenging due to subtle lesion characteristics, complex anatomical structures, and strong dependencies among clinical findings. Moreover, inaccurate visual observations in early clinical sub-tasks may propagate and accumulate through subsequent reasoning steps, ultimately reducing the accuracy and clinical consistency of the generated reports. To address these challenges, we construct the LungCT-CoT dataset, which decomposes lung CT report generation into six clinically ordered and interconnected sub-tasks to reflect the clinical diagnostic workflow. Based on this dataset, we propose VEC-CT, a visual cue-guided error-corrective clinical reasoning framework. The model first performs structured reasoning over the predefined clinical sub-tasks and generates an initial diagnostic report. The Error Tracer then identifies the earliest erroneous sub-task in the reasoning process, after which the Visual Cue Extractor extracts task-specific visual cues relevant to the identified sub-task from the CT image. Guided by these visual cues, the model revisits the erroneous sub-task and corrects the subsequent reasoning process to refine the final diagnostic report. Extensive experiments show that VEC-CT consistently outperforms representative methods on both conventional text generation metrics and clinically oriented semantic evaluation metrics.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multimodal Large Language Models, Chain-of-Thought Reasoning, Radiology Report Generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_hojpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VEC-CT, a visual cue-guided error-corrective clinical reasoning framework for lung CT report generation. The authors construct LungCT-CoT, a dataset that decomposes the diagnostic process into six clinically ordered sub-tasks to support structured reasoning. VEC-CT employs an Error Tracer to identify the earliest erroneous sub-task and a Visual Cue Extractor to provide task-specific visual information for targeted reasoning refinement. During inference, only the optimized Reasoning Model is retained, without explicit error tracing or correction. Experimental results demonstrate improvements over representative baseline models on report generation metrics and clinical classification tasks.

Critical strengths

This paper proposes VEC-CT to address the problem of error propagation across intermediate clinical reasoning steps in lung CT report generation. The proposed framework integrates structured clinical reasoning, error tracing, and task-specific visual cues to improve the reliability of diagnostic report generation. The authors construct LungCT-CoT, which provides task-level supervision across six clinically relevant sub-tasks. Extensive experiments are conducted to compare the proposed method with various general-purpose and medical MLLMs, along with ablation studies to evaluate the contributions of different components. Additionally, qualitative comparisons and failure-case analyses provide further insights into the model's performance and limitations.

Critical weaknesses
  1. The paper claims that LungCT-CoT contains over 100,000 clinical QA pairs, whereas the task-level statistics in Appendix A.4 sum to only 92,796 pairs.
  2. The baseline models are evaluated without fine-tuning on LungCT-CoT, whereas VEC-CT is specifically fine-tuned on this dataset. Therefore, the reported performance improvements may primarily result from dataset-specific fine-tuning rather than the proposed method.
  3. The paper lacks sufficient details regarding the human verification of the generated QA pairs, including the number and qualifications of clinical reviewers, the proportion of manually reviewed samples, and inter-rater agreement.
  4. The Error Tracer and Visual Cue Extractor are used only during training, while inference relies solely on the Reasoning Model.
  5. The paper does not directly evaluate the reliability of the proposed error-correction mechanism, such as the proportion of incorrect predictions successfully corrected or correct predictions degraded after correction. Thus, it is unclear whether the improvements genuinely result from error correction.
  6. The framework uses a single 2D CT slice to generate patient-level diagnostic reports, although some reference findings require information unavailable from a single slice. This potential mismatch between input information and reference labels may affect the validity of training and evaluation.
  7. Although several failure cases are presented, the paper lacks a systematic analysis of different error types, their frequencies, and their potential clinical consequences.
  8. The private clinical dataset is insufficiently described, with limited information about the contributing institutions, patient population, data distribution, and annotation procedures. Its contribution to the reported performance also remains unclear.
  9. The clinical criteria underlying the five malignancy risk categories (A–E) are not clearly defined, raising concerns about the clinical validity and interpretability of the classification results.
Questions for authors
  1. The authors should clarify why the reported task-level statistics sum to 92,796 QA pairs, rather than the claimed total of over 100,000.

  2. The authors should provide comparisons with baseline models fine-tuned on LungCT-CoT under comparable training settings to isolate the contribution of the proposed method.

  3. The authors should clarify how many clinical experts participated in the QA verification process, their qualifications, and the proportion of samples that underwent manual review. Inter-rater agreement should also be reported.

  4. The authors should clarify how the proposed framework achieves error-corrective reasoning at inference time when the Error Tracer and Visual Cue Extractor are removed.

  5. The authors should quantify the correction success rate, the rate at which correction introduces new errors, and the extent to which performance improvements can be attributed to successful error correction.

  6. The authors should explain how reference findings that cannot be inferred from a single CT slice, such as those requiring volumetric imaging, prior examinations, or additional clinical information, are handled.

  7. The authors should clarify the clinical guidelines or annotation criteria used to define the five malignancy risk categories.

  8. The authors should provide a quantitative analysis of different error categories, their frequencies, and their potential clinical impact.

  9. The authors should provide more details about the private clinical dataset, including the number of institutions and patients, data distribution across splits, and annotation procedures. An additional ablation study should be conducted to evaluate the contribution of the private dataset.

6 comments
AuthorsOpenReview ↗

Response to Reviewer hojp (1)

Q1. The paper claims that LungCT-CoT contains over 100,000 clinical QA pairs, whereas the task-level statistics in Appendix A.4 sum to only 92,796 pairs.

Thank you for pointing out this discrepancy. The figure of “over 100,000” refers to the 105,315 QA pairs we initially constructed across seven tasks, including Cancer Subtype. During quality verification, we excluded 5,399 QA pairs, including 3,082 from Diagnostic Finding, leaving 99,916 QA pairs.

Of the 7,120 remaining QA pairs for Cancer Subtype, 5,964 (83.76%) were labeled as adenocarcinoma, and the other three subtypes accounted for 1,156 pairs (16.24%) combined. We sought additional cases for the underrepresented subtypes, but the limited number of suitable cases left the distribution highly imbalanced. We therefore removed the Cancer Subtype task from the dataset and experiments. After these two filtering steps, the final LungCT-CoT dataset contains 92,796 QA pairs across six tasks. We have clarified these counts in Appendix A.4 (Dataset Analysis) of the revised paper.

Q2. The authors should provide comparisons with baseline models fine-tuned on LungCT-CoT under comparable training settings to isolate the contribution of the proposed method.

Thank you for this suggestion. Our experiments include two baselines fine-tuned on LungCT-CoT using the same backbone: VEC-CT (w/o CoT), trained with final diagnostic reports as supervision, and VEC-CT (w/o Correction), trained with Clinical CoT supervision but without corrective training.

The full method builds on Clinical CoT fine-tuning by adding error localization and task-specific visual cues during corrective training. Its comparison with VEC-CT (w/o Correction) assesses the additional contribution of this training procedure. We also include the pretrained backbone without fine-tuning and the Pre-Cue variant, which provides visual cues before reasoning, to examine the effects of fine-tuning and cue timing.

Tables 4 and 5 in Section 4.3 show that the full method achieves higher mean scores than both fine-tuned baselines across the text-generation and automatic clinical metrics. Appendix D.5.2 (Ablation Variant Analysis) presents generated reports and corresponding attention maps for the same input side by side, illustrating differences in the attended lesion regions and report content.

Combined ablation results from Tables 4 and 5. Values are mean ± standard deviation or mean (95% CI). Best mean scores are bold.

MethodBLEU-4ROUGE-LMETEORBERT-F1GREENRaTEScoreRadGraph Partial
VEC-CT (w/o FT)0.05±0.060.20±0.080.24±0.090.69±0.040.347 (0.325, 0.369)0.507 (0.499, 0.514)0.187 (0.179, 0.196)
VEC-CT (w/o CoT)0.12±0.210.31±0.190.34±0.220.71±0.080.440 (0.417, 0.462)0.616 (0.606, 0.627)0.303 (0.287, 0.319)
VEC-CT (w/o Correction)0.20±0.190.40±0.190.42±0.200.78±0.080.404 (0.387, 0.421)0.628 (0.619, 0.638)0.368 (0.354, 0.383)
VEC-CT (Pre-Cue)0.23±0.250.42±0.230.41±0.260.78±0.090.439 (0.415, 0.462)0.620 (0.608, 0.632)0.381 (0.361, 0.401)
VEC-CT (Ours)0.28±0.240.47±0.230.47±0.240.81±0.090.504 (0.482, 0.525)0.665 (0.654, 0.676)0.446 (0.429, 0.463)

Q3. The authors should clarify how many clinical experts participated in the QA verification process, their qualifications, and the proportion of samples that underwent manual review. Inter-rater agreement should also be reported.

Thank you for this suggestion. We clarified the annotator background and manual review procedure in Appendix A.3 (Data Verification). Six annotators with several years of experience in radiology reporting conducted two rounds of review.

Across the seven initially constructed tasks, 5,399 QA pairs (5.13% of the initial 105,315 pairs) remained ambiguous or difficult to assess after model-based verification and were referred for manual review, including 3,082 pairs from Diagnostic Finding.

In the first round, all six annotators independently scored the referred pairs from 1 to 5. They assessed whether each question could be answered from the selected CT slice alone, whether the slice supported the key findings in the reference answer, and whether answers to related subtasks were consistent. Available source annotations were also checked.

In the second round, the annotators reviewed one another’s anonymized scores and comments, revisited the supporting evidence, and rescored each pair. We retained a pair if at least three annotators scored it above 3 and all substantive disagreements about the key content of its reference answer had been resolved.

AuthorsOpenReview ↗

Response to Reviewer hojp (2)

Q4. The authors should clarify how the proposed framework achieves error-corrective reasoning at inference time when the Error Tracer and Visual Cue Extractor are removed.

Thank you for raising this question. The Error Tracer and Visual Cue Extractor are used only during training to guide explicit correction. In the second reasoning round, the earliest-error index specifies the subtask from which the model regenerates the reasoning trace, while task-specific visual cues direct its attention to the corresponding lesion region. We supervise the model with corrected reasoning traces to help it learn to observe lesions accurately and use relevant imaging features for clinical reasoning.

At test time, the model performs single-pass inference without a separate, explicit correction stage. The lesion observation and reasoning capabilities learned through correction training are implicitly reflected in this process. Using the original CT slice as its sole visual input, the model follows the predefined clinical task sequence from the first task to the final report. No correction starting point is therefore required.

Tables 4 and 5 in Section 4.3 compare the full method with VEC-CT (w/o Correction), which uses only Clinical CoT supervision, and VEC-CT (Pre-Cue), which receives visual cues before reasoning and generates the report in a single pass. These comparisons examine the contributions of correction training and the timing of visual cue provision, respectively. The full method outperforms both variants on the text-generation and automatic clinical metrics, supporting the benefit of visual cue-guided correction during training for single-pass report generation at test time. We also present generated reports and corresponding attention maps in Appendix D.5.2 (Ablation Variant Analysis).

Combined ablation results from Tables 4 and 5. Values are mean ± standard deviation or mean (95% CI). Best mean scores are bold.

MethodBLEU-4ROUGE-LMETEORBERT-F1GREENRaTEScoreRadGraph Partial
VEC-CT (w/o FT)0.05±0.060.20±0.080.24±0.090.69±0.040.347 (0.325, 0.369)0.507 (0.499, 0.514)0.187 (0.179, 0.196)
VEC-CT (w/o CoT)0.12±0.210.31±0.190.34±0.220.71±0.080.440 (0.417, 0.462)0.616 (0.606, 0.627)0.303 (0.287, 0.319)
VEC-CT (w/o Correction)0.20±0.190.40±0.190.42±0.200.78±0.080.404 (0.387, 0.421)0.628 (0.619, 0.638)0.368 (0.354, 0.383)
VEC-CT (Pre-Cue)0.23±0.250.42±0.230.41±0.260.78±0.090.439 (0.415, 0.462)0.620 (0.608, 0.632)0.381 (0.361, 0.401)
VEC-CT (Ours)0.28±0.240.47±0.230.47±0.240.81±0.090.504 (0.482, 0.525)0.665 (0.654, 0.676)0.446 (0.429, 0.463)

Q5. The authors should quantify the correction success rate, the rate at which correction introduces new errors, and the extent to which performance improvements can be attributed to successful error correction.

Thank you for this suggestion. Table 19 in Appendix D.4 (Training-Stage Error Analysis and Correction Dynamics) presents a paired evaluation of first- and second-round outputs. CSR measures the fraction of initially unsuccessful responses that pass after correction; EIR measures the fraction of initially successful responses that fail after correction; and C/N measures successful corrections as a fraction of all evaluated responses.

We divide the more than 5,000 training steps into five stages and randomly sample 200 examples per stage. Overall accuracy is the proportion of responses judged correct across all applicable subtasks in these examples. Classification and judgment outputs are checked against reference labels, while generative outputs are evaluated by an LLM for agreement with references in anatomical location, key findings, medical details, and negation. Both rounds use the same criteria.

Second-round accuracy is higher at every stage, with the gain increasing from 2.0 to 12.7 percentage points. Early improvements are limited, suggesting that the model may not yet have learned to use visual cues effectively for correction. As training progresses, CSR and C/N increase while EIR decreases, suggesting more effective correction and better preservation of initially correct responses.

Table 19. Correction evaluation across five training stages. First- and second-round values denote overall accuracy across all evaluated subtasks in each stage’s sampled examples.

StageFirst roundSecond roundCSREIRC/N
Stage 145.2%47.2%22.2%22.5%12.2%
Stage 248.2%54.3%28.4%17.9%14.7%
Stage 353.8%60.9%33.0%15.1%15.2%
Stage 456.3%66.0%38.4%12.6%16.8%
Stage 558.9%71.6%44.4%9.5%18.3%
AuthorsOpenReview ↗

Response to Reviewer hojp (3)

Q6. The authors should explain how reference findings that cannot be inferred from a single CT slice, such as those requiring volumetric imaging, prior examinations, or additional clinical information, are handled.

Thank you for raising this point. All six VQA tasks in LungCT-CoT use a single CT slice as the imaging input. Processing complete three-dimensional CT volumes requires greater computational resources, GPU memory, and training time. Given the available computational resources and the time required for data construction and training, we adopted a single-slice setting, constructing QA pairs from each selected slice and its corresponding metadata rather than using the complete CT volume.

We organize the metadata at the slice level and link the reference annotations to the input slice and the target lesion shown on it. During verification, we check whether the findings in each reference answer are supported by that slice and its corresponding metadata. This criterion also applies to answers extracted directly from the original clinical reports. We exclude QA pairs whose reference answers require adjacent slices, the complete CT volume, prior examinations, or additional clinical information, so that the evaluation targets remain consistent with the information available to the model.

Q7. The authors should clarify the clinical guidelines or annotation criteria used to define the five malignancy risk categories.

Thank you for this suggestion. For LIDC-IDRI cases in LungCT-CoT, the five grades in the Malignancy task follow the original LIDC-IDRI annotation protocol described by Armato et al. (2011). Each CT scan was reviewed by four experienced thoracic radiologists in two phases. In the first phase, the radiologists independently marked lesions. In the second phase, each radiologist reviewed their own markings alongside the anonymized markings of the other readers before independently providing a final assessment. We use the resulting nodule-level malignancy ratings recorded in the preprocessed metadata. Ratings of 1 (highly unlikely), 2 (moderately unlikely), 3 (indeterminate), 4 (moderately suspicious), and 5 (highly suspicious) correspond to grades A–E, respectively.

The reference labels for Malignancy and CancerDiagnosis were already recorded in the preprocessed metadata before QA generation. For the same nodule, Malignancy provides a fine-grained, five-level assessment of malignancy suspicion, while CancerDiagnosis provides a coarse-grained, three-category assessment by grouping grades A–B as No, grade C as Indeterminate, and grades D–E as Yes.

During verification, we use these existing reference labels to check whether the generated textual answers are consistent with the CT findings and the answers to related tasks. We have clarified the annotation basis and verification procedure in Appendix A.3 (Data Verification).

Reference

Armato, S. G., III, McLennan, G., Bidaut, L., et al. (2011). The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans. Medical Physics, 38(2), 915–931. https://doi.org/10.1118/1.3528204

Reviewer_w69dOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VEC-CT, a framework that decomposes lung CT report generation into six clinical sub-tasks and uses error tracing and task-specific visual cues for targeted corrective training. Experiments show improvements over the evaluated baselines in report generation and clinical prediction tasks.

Critical strengths
  1. The motivation is reasonable, addressing error propagation in clinical reasoning through targeted correction.

  2. The experiments and ablation studies are relatively comprehensive, and the dataset may be useful to the community.

Critical weaknesses
  1. Correction is only applied during training. The Error Tracer and Visual Cue Extractor are removed at inference time. Thus, the results demonstrate the benefit of additional corrective supervision, rather than the model's ability to autonomously identify and correct errors during inference.

  2. The clinical validity of the data and evaluation remains uncertain. Some QA pairs are LLM-generated (8B), and the model primarily uses single 2D CT slices to generate diagnostic reports and treatment recommendations. The extent of independent validation of the reference answers and the representativeness of these tasks for real-world clinical reporting remain unclear.

Questions for authors
  1. What proportion of the reference answers for the Diagnostic Finding task comes from actual clinical reports versus LLM-generated content? What are the coverage and inter-annotator agreement of the human review?

  2. How do the authors validate the reliability of the extracted visual cues, particularly the segmentation masks and derived lesion measurements?

  3. Can the authors isolate the contribution of visual cues by comparing the full method with a variant that retains corrective supervision but removes visual cues?

3 comments
AuthorsOpenReview ↗

Response to Reviewer w69d (1)

Q1. What proportion of the reference answers for the Diagnostic Finding task comes from actual clinical reports versus LLM-generated content? What are the coverage and inter-annotator agreement of the human review?

Thank you for this question. The final Diagnostic Finding task contains 19,639 QA pairs: 284 (1.45%) use reference answers extracted directly from physician-authored CT reports in the private dataset, while 19,355 (98.55%) use LLM-generated references for cases from the two public datasets.

For public-dataset cases, physician-authored reports served as few-shot examples to guide reporting structure, terminology, and phrasing. Reference answers were generated from the current case’s CT slice and subsequently checked against available source annotations.

Across the seven initially constructed tasks, 5,399 QA pairs (5.13% of the initial 105,315 pairs) could not be confidently verified through LLM-based verification and were referred for manual review, including 3,082 pairs from Diagnostic Finding. Six annotators with several years of experience in radiology reporting conducted two rounds of review.

In the first round, all six annotators independently scored the referred pairs on a five-point scale, assessing single-slice answerability, support for key findings, and consistency across related subtasks. In the second round, they reviewed one another’s anonymized scores and comments before reassessing and rescoring each pair. We retained a pair if at least three annotators scored it above 3 and all substantive disagreements about its reference answer had been resolved.

Q2. How do the authors validate the reliability of the extracted visual cues, particularly the segmentation masks and derived lesion measurements?

Thank you for this question. To assess whether the extracted visual cues correspond to the target lesions, we compare their associated lesion bounding boxes with available source annotations. Using an IoU-based overlap criterion, the localization agreement rate is 0.769; the proportion of predicted box centers falling within the matched annotation boxes is 0.828; and the mean box-level Dice score is 0.612.

For derived lesion measurements, we use physical information from DICOM metadata to convert pixel-based measurements into physically meaningful values. Since these measurements guide correction, we indirectly assess their usefulness through downstream report-generation performance. Specifically, we compare the full method with VEC-CT (w/o Metadata), which removes DICOM-derived information during cue construction and extracts cues from image pixels alone.

All configurations use the same 5,000 training samples and 500 test samples, with the same training settings and evaluation procedure. Table 16 of Appendix D.3 (Complete Experimental Results) summarizes both text-generation and automatic clinical metrics. The full method achieves higher mean scores than VEC-CT (w/o Metadata) across all reported metrics, supporting the contribution of DICOM-derived measurement information to corrective training. Cue removal and box perturbation variants provide complementary comparisons of cue availability and localization accuracy.

Table 16. Visual cue ablation and robustness results. Values are mean ± standard deviation. Best mean scores are bold.

MethodBLEU-4ROUGE-LMETEORBERT-F1GREENRaTEScoreRadGraph Partial
VEC-CT (w/o Cues)0.28±0.120.51±0.130.50±0.120.83±0.050.4743±0.14850.6612±0.10900.4411±0.1315
VEC-CT (w/ Box Perturbation)0.29±0.130.51±0.130.51±0.130.83±0.050.5059±0.16320.6631±0.10830.4566±0.1360
VEC-CT (w/o Metadata)0.28±0.120.51±0.120.51±0.120.83±0.050.5114±0.17620.6644±0.09080.4483±0.1240
VEC-CT (Ours)0.35±0.150.55±0.140.55±0.140.84±0.060.6464±0.18640.7034±0.11570.5293±0.1619
AuthorsOpenReview ↗

Response to Reviewer w69d (2)

Q3. Can the authors isolate the contribution of visual cues by comparing the full method with a variant that retains corrective supervision but removes visual cues?

Thank you for this suggestion. We added a comparison between the full method and VEC-CT (w/o Cues) in Table 16 of Appendix D.3 (Complete Experimental Results), covering both text-generation and automatic clinical metrics.

VEC-CT (w/o Cues) retains two-round corrective training, the indication of the erroneous subtask, and the original CT slice as input. It removes only the task-specific visual cues provided during correction, preserving corrective supervision. Table 16 also includes variants with perturbed lesion boxes and without DICOM metadata to examine the influence of cue construction.

All configurations use the same 5,000 training samples and 500 test samples, with the same training settings and evaluation procedure. The full method achieves higher mean scores than VEC-CT (w/o Cues) across all reported metrics, supporting the additional benefit of visual cues beyond error localization and corrective supervision.

Table 16. Visual cue ablation and robustness results. Values are mean ± standard deviation. Best mean scores are bold.

MethodBLEU-4ROUGE-LMETEORBERT-F1GREENRaTEScoreRadGraph Partial
VEC-CT (w/o Cues)0.28±0.120.51±0.130.50±0.120.83±0.050.4743±0.14850.6612±0.10900.4411±0.1315
VEC-CT (w/ Box Perturbation)0.29±0.130.51±0.130.51±0.130.83±0.050.5059±0.16320.6631±0.10830.4566±0.1360
VEC-CT (w/o Metadata)0.28±0.120.51±0.120.51±0.120.83±0.050.5114±0.17620.6644±0.09080.4483±0.1240
VEC-CT (Ours)0.35±0.150.55±0.140.55±0.140.84±0.060.6464±0.18640.7034±0.11570.5293±0.1619

Apologies for the delayed response. I have been quite busy recently. Thank you for the rebuttal. The additional experiments have addressed some of my concerns. However, I still have concerns about the quality and reliability of the data. In addition, W1 remains unaddressed. So I will maintain my rating.

Reviewer_gMh9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces VEC-CT, a visual cue-guided error-corrective framework for lung CT report generation. It builds the LungCT-CoT dataset around six ordered clinical subtasks, traces the earliest erroneous subtask during training, extracts task-specific visual cues from CT images and DICOM metadata, and trains a reasoning model to revise the trajectory from that point. The idea addresses error propagation in medical chain-of-thought reasoning and the paper reports improvements on report-generation and clinical semantic metrics. The main concern is that the correction machinery is used only during training, so the paper needs stronger evidence that the learned behavior rather than privileged training supervision produces reliable inference.

Critical strengths
  1. The task decomposition follows a plausible clinical workflow: lesion location, impression, cancer judgment, risk assessment, targeted therapy, and final finding. This makes the intermediate supervision more interpretable than report-only fine-tuning.
  2. The earliest-error tracing idea is a useful way to avoid revising every downstream step. The visual cue extractor also makes the correction signal more task-specific than generic self-refinement.
  3. The paper evaluates both language-generation quality and task-level clinical decisions, and the qualitative examples clearly illustrate how a corrected lesion location or size can change the downstream report.
  4. The work documents patient-level splits, data construction, model configuration, and reproducibility plans, which is important for a clinical MLLM paper.
Critical weaknesses
  1. The inference-time mechanism is a central weakness. Error Tracer and Visual Cue Extractor provide corrective supervision during training but are not used at test time. The paper should compare with training schemes that use the same task labels but no visual cues, and should measure whether the model can identify and correct errors without an oracle earliest-error index.
  2. The LungCT-CoT construction and annotation quality need more detail. Please report the number of patients and slices, labeler expertise, inter-rater agreement, adjudication rules, and the distribution of missing or uncertain answers for each of the six tasks. Patient-level splitting alone does not rule out repeated acquisition conditions or near-duplicate slices.
  3. The visual cues use DICOM metadata and bounding boxes, which may not be available or reliable in a deployment setting. The paper should report results with image-only inputs, with noisy boxes, and with metadata removed. Otherwise it is difficult to know whether the gain comes from the proposed reasoning correction or from privileged localization information.
  4. The reported improvements should be accompanied by statistical significance or bootstrap confidence intervals and by clinically oriented error analyses. Automatic BLEU, ROUGE, or similar metrics can reward fluent but unsafe reports, so the paper should report factuality, hallucination, critical-lesion sensitivity, and contradiction rates judged by blinded clinicians or a validated evaluator.
  5. The paper uses 2D CT slices and acknowledges limited cross-slice continuity. This limitation is especially important because lesion characterization and disease burden are volumetric. Please quantify performance by lesion size and multi-region disease, and clarify whether multiple slices from one study can appear in train and test through any indirect route.
Questions for authors
  1. At inference time, how does the model know where to start correction if the earliest-error index and visual cue extractor are unavailable?
  2. Please provide label counts, annotator expertise, inter-rater agreement, and uncertainty statistics for all six LungCT-CoT tasks.
  3. What are the results with image-only inputs, noisy or perturbed boxes, and no DICOM metadata?
  4. Can the authors report clinician-judged factuality, critical-lesion sensitivity, hallucination, and contradiction rates with confidence intervals?
  5. How are slices from the same study grouped, and are lesion-size and multi-region performance reported separately?
4 comments
AuthorsOpenReview ↗

Response to Reviewer gMh9 (1)

Q1. At inference time, how does the model know where to start correction if the earliest-error index and visual cue extractor are unavailable?

Thank you for raising this question. The Error Tracer and Visual Cue Extractor are used only during training to guide explicit correction. In the second reasoning round, the earliest-error index specifies the subtask from which the model regenerates the reasoning trace, while task-specific visual cues direct its attention to the corresponding lesion region. We supervise the model with corrected reasoning traces to help it learn to observe lesions accurately and use relevant imaging features for clinical reasoning.

At test time, the model performs single-pass inference without a separate, explicit correction stage. The lesion observation and reasoning capabilities learned through correction training are implicitly reflected in this process. Using the original CT slice as its sole visual input, the model follows the predefined clinical task sequence from the first task to the final report. No correction starting point is therefore required.

Tables 4 and 5 in Section 4.3 compare the full method with VEC-CT (w/o Correction), which uses only Clinical CoT supervision, and VEC-CT (Pre-Cue), which receives visual cues before reasoning and generates the report in a single pass. These comparisons examine the contributions of correction training and the timing of visual cue provision, respectively. The full method outperforms both variants on the text-generation and automatic clinical metrics, supporting the benefit of visual cue-guided correction during training for single-pass report generation at test time. We also present generated reports and corresponding attention maps in Appendix D.5.2 (Ablation Variant Analysis).

Combined ablation results from Tables 4 and 5. Values are mean ± standard deviation or mean (95% CI). Best mean scores are bold.

MethodBLEU-4ROUGE-LMETEORBERT-F1GREENRaTEScoreRadGraph Partial
VEC-CT (w/o FT)0.05±0.060.20±0.080.24±0.090.69±0.040.347 (0.325, 0.369)0.507 (0.499, 0.514)0.187 (0.179, 0.196)
VEC-CT (w/o CoT)0.12±0.210.31±0.190.34±0.220.71±0.080.440 (0.417, 0.462)0.616 (0.606, 0.627)0.303 (0.287, 0.319)
VEC-CT (w/o Correction)0.20±0.190.40±0.190.42±0.200.78±0.080.404 (0.387, 0.421)0.628 (0.619, 0.638)0.368 (0.354, 0.383)
VEC-CT (Pre-Cue)0.23±0.250.42±0.230.41±0.260.78±0.090.439 (0.415, 0.462)0.620 (0.608, 0.632)0.381 (0.361, 0.401)
VEC-CT (Ours)0.28±0.240.47±0.230.47±0.240.81±0.090.504 (0.482, 0.525)0.665 (0.654, 0.676)0.446 (0.429, 0.463)
AuthorsOpenReview ↗

Response to Reviewer gMh9 (2)

Q2. Please provide label counts, annotator expertise, inter-rater agreement, and uncertainty statistics for all six LungCT-CoT tasks.

Thank you for asking us to clarify the label counts. The final LungCT-CoT dataset contains 92,796 QA pairs across six tasks. Each of the four generative tasks (Lesion Location, Lesion Impression, Targeted Therapy, and Diagnostic Finding) contains 19,639 QA pairs with free-text reference answers.

Risk Assessment contains 7,120 QA pairs across five ordinal grades of malignancy suspicion. From very low to very high, grades A–E contain 622, 840, 2,420, 1,776, and 1,462 QA pairs, respectively. Cancer Judgment also contains 7,120 QA pairs and groups the same ratings into three categories. Grades A–B map to No (1,462 pairs), grade C maps to Indeterminate (2,420 pairs), and grades D–E map to Yes (3,238 pairs). These counts are reported in Appendix A.4 (Dataset Analysis) and in the data analysis panel of Figure 6(b).

We first constructed candidate QA pairs and used a model to verify them against the selected CT slice and any available source annotations. We then referred QA pairs that remained ambiguous or difficult to verify to six annotators with several years of experience in radiology reporting for two rounds of manual review.

In the first round, all six annotators independently scored the same set of QA pairs on a five-point scale. They assessed whether each question could be answered from the selected CT slice alone, whether the slice supported the key findings in the reference answer, and whether the answers to related subtasks were consistent. They also cross-checked the answers against source annotations when available.

In the second round, the annotators reviewed one another’s anonymized scores and comments, then reassessed and rescored each QA pair. We retained a pair if at least three annotators scored it above 3 and all substantive disagreements about the key content of the reference answer had been resolved.

We initially constructed 105,315 QA pairs across seven tasks, including Cancer Subtype. Quality verification excluded 5,399 QA pairs, including 3,082 from Diagnostic Finding, leaving 99,916 QA pairs. We then removed the Cancer Subtype task due to severe class imbalance. This excluded a further 7,120 QA pairs and left 92,796 QA pairs across the six retained tasks.

Q3. What are the results with image-only inputs, noisy or perturbed boxes, and no DICOM metadata?

Thank you for this suggestion. We added a visual cue ablation and robustness analysis in Table 16 of Appendix D.3 (Complete Experimental Results). All four configurations use the same 5,000 training samples and 500 test samples, with the same training settings and evaluation procedure. The table below combines text-generation and automatic clinical metrics.

We compare the full method with three variants that modify visual guidance during corrective training. At inference, all configurations use only the original CT slice as visual input.

  • VEC-CT (w/o Cues) retains two-round corrective training and the indication of the erroneous subtask, but uses only the original CT slice without task-specific visual cues.
  • VEC-CT (w/ Box Perturbation) perturbs the lesion bounding boxes used for cue extraction while keeping the remaining procedure unchanged.
  • VEC-CT (w/o Metadata) removes DICOM-derived information during cue construction and extracts cues from image pixels alone.

The full VEC-CT achieves the highest mean scores across all reported metrics. Removing visual cues lowers performance despite retaining corrective supervision, supporting the additional benefit of visual guidance. Perturbed boxes and metadata removal also reduce performance, suggesting that localization accuracy and DICOM-derived information affect how effectively the cues guide corrective training.

Table 16. Visual cue ablation and robustness results. Values are mean ± standard deviation. Best mean scores are bold.

MethodBLEU-4ROUGE-LMETEORBERT-F1GREENRaTEScoreRadGraph Partial
VEC-CT (w/o Cues)0.28±0.120.51±0.130.50±0.120.83±0.050.4743±0.14850.6612±0.10900.4411±0.1315
VEC-CT (w/ Box Perturbation)0.29±0.130.51±0.130.51±0.130.83±0.050.5059±0.16320.6631±0.10830.4566±0.1360
VEC-CT (w/o Metadata)0.28±0.120.51±0.120.51±0.120.83±0.050.5114±0.17620.6644±0.09080.4483±0.1240
VEC-CT (Ours)0.35±0.150.55±0.140.55±0.140.84±0.060.6464±0.18640.7034±0.11570.5293±0.1619
AuthorsOpenReview ↗

Response to Reviewer gMh9 (3)

Q4. Can the authors report clinician-judged factuality, critical-lesion sensitivity, hallucination, and contradiction rates with confidence intervals?

Thank you for this suggestion. We added automatic clinical evaluation using GREEN, RaTEScore, and RadGraph Simple, Partial, and Complete in Table 5, alongside targeted human evaluation in Table 15 of Appendix D.3 (Complete Experimental Results).

The human evaluation covers all 751 test samples and compares five model configurations on the same cases. Each report was independently reviewed by six annotators with several years of experience in radiology reporting, using the selected CT slice, reference report, and available source annotations. At least four agreeing annotators determined the final label; three-to-three ties underwent further review. We evaluated:

  • False finding rate (FFR): the proportion of reports containing at least one unsupported positive finding.
  • Critical-lesion sensitivity: the proportion of reference critical lesions correctly described, including their anatomical location and key imaging features.
  • Contradiction rate: the proportion of reports containing mutually incompatible statements about the same lesion or clinical fact, or directly conflicting with a human-confirmed reference finding supported by the selected slice and available annotations.

We calculated 95% confidence intervals using 1,000 bootstrap resamples, with the 2.5th and 97.5th percentiles as interval bounds. The full VEC-CT achieves an FFR of 8.4% (95% CI: 6.5%–10.3%), critical-lesion sensitivity of 79.5% (76.6%–82.4%), and a contradiction rate of 4.7% (3.2%–6.2%). These are the best point estimates among the evaluated configurations, and the full model also achieves the highest mean scores on all automatic clinical metrics.

Table 5. Ablation results for VEC-CT using clinical report evaluation metrics. Values are mean scores with 95% confidence intervals in parentheses, estimated using 1,000 report-level bootstrap resamples on 751 test reports. The highest score for each metric is shown in bold.

MethodGREENRaTEScoreRadGraph SimpleRadGraph PartialRadGraph Complete
VEC-CT (w/o FT)0.347 (0.325, 0.369)0.507 (0.499, 0.514)0.208 (0.198, 0.216)0.187 (0.179, 0.196)0.164 (0.156, 0.173)
VEC-CT (w/o CoT)0.440 (0.417, 0.462)0.616 (0.606, 0.627)0.336 (0.321, 0.353)0.303 (0.287, 0.319)0.292 (0.279, 0.307)
VEC-CT (w/o Correction)0.404 (0.387, 0.421)0.628 (0.619, 0.638)0.416 (0.402, 0.433)0.368 (0.354, 0.383)0.342 (0.327, 0.357)
VEC-CT (Pre-Cue)0.439 (0.415, 0.462)0.620 (0.608, 0.632)0.415 (0.396, 0.435)0.381 (0.361, 0.401)0.356 (0.338, 0.375)
VEC-CT (Ours)0.504 (0.482, 0.525)0.665 (0.654, 0.676)0.486 (0.468, 0.503)0.446 (0.429, 0.463)0.417 (0.399, 0.434)

Table 15. Additional error and lesion-level measures for five VEC-CT variants. Values are shown with 95% confidence intervals. Bold indicates the best point estimate: lower is better for FFR and contradiction rate, whereas higher is better for sensitivity.

MethodFFRCritical-lesion sensitivityContradiction rate
VEC-CT (w/o FT)45.2% [42.1%, 48.3%]38.5% [35.2%, 41.8%]25.4% [22.6%, 28.2%]
VEC-CT (w/o CoT)42.3% [38.8%, 45.8%]41.2% [37.5%, 44.9%]28.6% [25.4%, 31.8%]
VEC-CT (w/o Correction)18.5% [15.8%, 21.2%]58.7% [55.1%, 62.3%]15.4% [12.8%, 18.0%]
VEC-CT (Pre-Cue)16.2% [13.6%, 18.8%]64.3% [60.8%, 67.8%]12.1% [9.8%, 14.4%]
VEC-CT (Ours)8.4% [6.5%, 10.3%]79.5% [76.6%, 82.4%]4.7% [3.2%, 6.2%]
AuthorsOpenReview ↗

Response to Reviewer gMh9 (4)

Q5. How are slices from the same study grouped, and are lesion-size and multi-region performance reported separately?

Thank you for raising this question. We use patient-level training, validation, and test splits, keeping all selected CT slices from the same patient within a single split to prevent data leakage.

Using available lesion-size annotations, we stratified test samples into nodules ≥3 mm, nodules <3 mm, and non-nodules ≥3 mm. Tables 12–14 of Appendix D.3 summarize text-generation and automatic clinical metrics for each group. The full VEC-CT achieves the highest mean scores across the reported metrics in both nodule groups. In the non-nodule group, it leads on the text-generation metrics and RaTEScore, while Pre-Cue obtains slightly higher GREEN and RadGraph Partial scores.

Regarding multi-region performance, the metadata available to us pairs each CT slice with a single annotated lesion region. We followed this structure when constructing LungCT-CoT, with each QA example based on one slice–lesion pair. Consequently, the current dataset does not support a separate evaluation of multi-region performance.

Table 12. Report-generation ablation results in the nodule ≥3 mm group (n = 80). For Tables 12–14, values are mean ± standard deviation, and the highest unrounded mean for each metric is shown in bold.

MethodCountBLEU-1BLEU-2BLEU-3BLEU-4ROUGE-1ROUGE-2ROUGE-LMETEORBERT-F1
VEC-CT (w/o FT)800.21±0.100.08±0.070.04±0.050.02±0.040.25±0.090.04±0.060.16±0.060.23±0.070.67±0.03
VEC-CT (w/o CoT)800.27±0.360.23±0.380.22±0.380.21±0.380.37±0.320.25±0.370.35±0.330.30±0.350.73±0.14
VEC-CT (w/o Correction)800.27±0.220.19±0.230.15±0.230.12±0.240.35±0.190.19±0.230.30±0.200.32±0.210.72±0.08
VEC-CT (Pre-Cue)800.36±0.380.31±0.400.29±0.400.28±0.400.47±0.310.34±0.380.43±0.340.40±0.350.79±0.13
VEC-CT (Ours)800.38±0.370.32±0.380.30±0.390.29±0.390.49±0.310.35±0.370.45±0.330.42±0.350.79±0.12

Table 13. Report-generation ablation results in the nodule <3 mm group (n = 50).

MethodCountBLEU-1BLEU-2BLEU-3BLEU-4ROUGE-1ROUGE-2ROUGE-LMETEORBERT-F1
VEC-CT (w/o FT)500.21±0.100.08±0.070.04±0.050.02±0.030.25±0.090.04±0.060.16±0.060.24±0.080.66±0.04
VEC-CT (w/o CoT)500.27±0.380.24±0.390.22±0.400.22±0.400.38±0.330.25±0.390.35±0.350.31±0.360.74±0.14
VEC-CT (w/o Correction)500.40±0.320.33±0.350.29±0.370.26±0.380.46±0.290.32±0.350.41±0.310.42±0.310.76±0.13
VEC-CT (Pre-Cue)500.37±0.380.32±0.400.31±0.410.30±0.410.49±0.320.36±0.380.45±0.340.42±0.360.79±0.13
VEC-CT (Ours)500.42±0.390.37±0.400.35±0.400.34±0.400.53±0.310.40±0.380.49±0.340.47±0.370.81±0.12

Table 14. Report-generation ablation results in the non-nodule ≥3 mm group (n = 70).

MethodCountBLEU-1BLEU-2BLEU-3BLEU-4ROUGE-1ROUGE-2ROUGE-LMETEORBERT-F1
VEC-CT (w/o FT)700.21±0.100.08±0.070.04±0.050.02±0.030.25±0.090.05±0.050.15±0.060.24±0.080.67±0.03
VEC-CT (w/o CoT)700.26±0.350.21±0.370.20±0.370.19±0.370.35±0.320.22±0.370.32±0.330.29±0.340.72±0.14
VEC-CT (w/o Correction)700.31±0.280.24±0.300.21±0.310.18±0.320.39±0.250.24±0.300.34±0.270.37±0.270.73±0.11
VEC-CT (Pre-Cue)700.35±0.360.29±0.380.27±0.390.26±0.390.46±0.300.32±0.370.42±0.330.39±0.340.78±0.12
VEC-CT (Ours)700.44±0.380.39±0.410.37±0.410.36±0.410.55±0.310.42±0.390.51±0.340.49±0.360.82±0.12

Comments on the paper

1 comment
Area_Chair_YoH2OpenReview ↗

Author-Reviewer Discussion

Dear Reviewers:

Thank you for your thoughtful reviews. The authors have responded to your reviews. Please check their response and reach out to them if you have any further questions. Please act as soon as possible, as the discussion period with the authors closes on Nov 18.

Best regards,

  • AC.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.