- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a new CT pre-training approach and provides evidence for its pre-training performance (using linear probing) and downstream tasks (dense segmentation, multi-organ segmentation, and zero-shot recognition). The results are desent, and the proposed method is interesting. However, the method details are fairly insufficent for ICLR publication. Also the results were from single-run experiments.
Critical strengths
- The proposed method is useful for solving CT pre-training via a geometric view with RoPE.
- Figure 2 is clear and easy to understand.
- The scores are competitive on several tasks, like CT classification and segmentation.
Critical weaknesses
- Only classification results are provided for comparing foundation model baselines with the proposed method.
- Scores provided in Sections 4.2-4.5 are just single-run results. Please consider using at least 3 different random seeds and report average scores with standard deviations.
- Details of the optimization step () is missing. Also, how to update with scheduling is not clear.
- According to Figure 2, Qwen is used for candidate extraction. However, nothing relevant to the approach was described in main text.
- The method is greatly inconsistent with the overview figure (Figure 2). For example, how to pre-process the reports was not described; how to get the atomic sentence pairs was not described; how to dilute the negative-dominant phrases was not described.
- The GPU usage is tremendous. Please compare the GPU hours with the baselines to further show the transparency of the proposed method.
Questions for authors
- Will the pre-training corpus be released after acceptance?
- Since main experiments used ViT-B, is it possible to provide controlled positional-encoding analysis results with ViT-B instead of ViT-S?
4 comments
Response to Reviewer 3haY (part2)
Report preprocessing and pair construction. We extracted candidates from the Findings section, using the Impression when Findings was unavailable. We checked the training manifests against the held-out split, grouped repeated reconstructions by patient, report text and processing mode, and assigned source IDs and character offsets before extraction. We validated each candidate against its cited source unit. We retained definite current findings, set aside uncertain, historical, resolved or comparison-only statements, and did not infer a specific absence from silence or a general statement of normality. We preserved laterality and anatomical scope when forming opposites. During reconstruction we shortened a location or modifier tail only when a single finding could still be identified safely. Ambiguous negation and unresolved multi-finding statements were set aside.
For a de-identified example from our report-recovery records, the source reads “Pleural effusion-thickening was not detected in both hemithorax.” The two supported pleural findings share the same source span. For effusion, the validated candidate is “No pleural effusion in either hemithorax.” Our local rule forms the pair (“Pleural effusion in either hemithorax.”, “No pleural effusion in either hemithorax.”) and assigns because the report supports the second statement. The same procedure yields a separate pair for pleural thickening, also with . This preserves the bilateral scope and makes explicit which member of each pair the report supports.
W5. Dilution and supplementation. We built the counts and donor bank from the CT-RATE alignment training split (24,115 retained report groups covering 47,144 CT reconstructions). Frequencies count distinct patients. We sample slots during training with a pseudorandom generator keyed by the run seed, volume identifier and training microstep. The following numbers come from a replay with seed 17.
- Dilution (Eq. 4). We retain an explicit negative for phrase with probability . For thoracic aortic dilatation, the positive lookup returned 0 and 804 patients had an explicit negative. Its retention probability was (0.124%). The draw was 0.5147, so this negative was skipped. The positive lookup uses counts from section-specific phrase entries; when a phrase occurs in multiple sections, the stored entry supplies the count.
- Supplementation (Eq. 5). We fill remaining negative slots from eligible section–phrase entries, with weight equal to the number of distinct positive donor patients. In the same replay, “Atelectasis.” had weight 184 and was sampled as (“Atelectasis.”, “No atelectasis.”) with . We require a known section status, exclude entries whose donor set includes the current patient, and reject exact normalized phrases already positive in the record or already selected. Synonymous wording can escape this check, so these labels carry weak supervision. Each CT contributes up to four positive and four negative slots, with retained explicit negatives taking priority and empty slots ignored.
We will expand Appendix A.5 with the extraction prompts, output schema, a worked report example and sampling pseudocode. We will also link the corresponding steps in Figure 2(b) directly to Section 3.4 and Eqs. (4)–(5).
W6: Training cost in GPU-hours and comparison with baselines
A: Across both stages of visual pretraining, we used approximately 14,066 NVIDIA L20Z GPU-hours. We summed the allocated GPU count multiplied by each job’s elapsed time across all recorded attempts, including initialization, restarts, and interrupted runs.
| Visual pretraining stage | GPUs | Cumulative job duration | GPU-hours |
|---|---|---|---|
| Stage 1 | 48 L20Z | 217.35 h | 10,432.87 |
| Stage 2 | 88 L20Z | 41.28 h | 3,633.05 |
| Total | 14,065.92 |
Stage 1 uses complete monitoring logs. We estimated Stage 2 from distributed-training timestamps, with an uncertainty of several minutes in elapsed time. Counting only time inside the training loop gives 13,948.62 GPU-hours. These totals cover visual pretraining. We separately report the CT–report alignment configuration in Table 7 (8 GPUs, 6,000 updates). A measured GPU-hour total for alignment is unavailable here.
We did not re-pretrain the baselines. For the external models in our main comparisons, we therefore give the original authors’ figures or calculate GPU-hours where they provide both GPU count and elapsed time. “Not reported” means we found no pretraining total for the indicated model or checkpoint in the cited material.
Response to Reviewer 3haY (part3)
| Baseline | Pretraining GPU-hours | Scope and source |
|---|---|---|
| VISTA3D | ≈20,000 | The authors report total training on 64 V100 (32 GB) GPUs. He et al., Training Requirements |
| CT-FM | ≈2,304 | Calculated from 4 Quadro RTX 8000 GPUs × 24 days × 24 h/day. Pai et al., training methods |
| Merlin | ≈160 | The authors report approximately 160 hours on one A6000. Blankemeier et al. |
| CT-CLIP | ≈288 | Zero-shot model, calculated from 4 A100 GPUs × 3 days × 24 h/day. Hamamci et al., Supplementary Table 3 |
| COLIPRI | 288 for the released model | The authors’ model card reports 4 A100 GPUs × 72 h. Separate totals for the C, CM and CRM variants are not reported. Microsoft model card, Environmental impact |
| Swin UNETR | ≈6 GPU-days as reported (≈144 GPU-hours by unit conversion) | Tang et al. give this figure for a 45k-step setting. Their main paper lists 450k iterations, and Li et al. separately cite 1,152 GPU-hours for the 5,050-scan comparator. The cost attributable to the evaluated checkpoint is therefore uncertain. Tang et al., supplement · Main paper, §5.2 · Li et al., Fig. 2 |
| SuPreM | Not reported for the 2,100-scan checkpoint | The 40 GPU-hours in Fig. 2 refer to a supervised experiment on 21 CT volumes. Li et al. |
| VoCo (Swin-B and nnU-Net/VoComni) | Not reported per checkpoint | Wu et al. report 8 H800 GPUs for pretraining. Their figure of over 10,000 GPU-hours covers downstream evaluation. Wu et al., Experiment settings |
| SPECTRE | Not reported | The paper specifies 24 B200 GPUs for the two pretraining phases. Appendix B.2 |
| FlexiCT | Not reported | Table S1 specifies 16 B200 GPUs per phase. Supplementary Table S1 |
| DINO-Med3D | Not reported | The method starts from DINOv3 and describes two stages of medical adaptation. Hu et al. |
The GPU types, training objectives, and treatment of failed runs differ across studies, which limits direct comparisons of training efficiency.
Q1: Availability of the pretraining corpus after acceptance
A: We cannot publicly release the clinical pretraining corpus after acceptance. As stated on page 10, the participating centers approved retrospective research use of the de-identified data in secure computing environments. Their data-governance and privacy restrictions preclude public redistribution. Upon publication, we will release the code, configuration files, and evaluation prompts listed in our Reproducibility Statement. We will also publicly release the pretrained TriPoD weights so that other researchers can evaluate and adapt the model.
Response to Reviewer 3haY (part4)
Q2: Controlled positional encoding ablation on ViT-B
A: We do not have a controlled ViT-B positional encoding ablation to report. To isolate the contribution of physical coordinates in the final ViT-B model, we would need to repeat both pretraining stages with normalized voxel-grid coordinates, holding the training data, objectives, and optimization schedule fixed, and then evaluate the resulting checkpoint under the same downstream protocols. Our original two-stage ViT-B visual pretraining used approximately 14,066 NVIDIA L20Z GPU-hours (see W6). We did not have the resources for a matched full-scale run within this study.
Within our available budget, we trained two ViT-S encoders for 30,000 updates on the same fixed 10% subset, varying only the coordinate parameterization (Section 4.4, Table 2). Physical coordinates improved macro Dice on TotalSegmentator v2 by 1.90 percentage points (paired 95% CI [+1.16, +2.73]), while the difference on AbdomenAtlas1.0Mini was −0.020 points (−0.087 to +0.047). These results establish a dataset-dependent effect at ViT-S scale. The independent contribution of positional encoding to the final ViT-B results remains unquantified, and we will make this limitation explicit alongside those results.
Response to Reviewer 3haY (part1)
Thank you for your constructive suggestions.
W1: Foundation model comparisons beyond classification
A: We compare TriPoD with other foundation models on segmentation across 11 benchmarks (Section 4.3, Table 1) and on zero-shot recognition on CT-RATE and RAD-ChestCT (Section 4.5, Table 4), in addition to the classification comparison in Section 4.2 (Figure 3).
Each protocol tests a different aspect of transfer. For classification, we freeze the visual encoder and train a linear classifier on its features. For segmentation, we fine-tune each encoder end to end with its corresponding decoder. Table 1 therefore measures the performance of complete segmentation systems and does not isolate dense prediction from frozen features. For zero-shot recognition, we keep the visual encoder fixed during CT–text alignment and use text prompts to recognize findings without fitting a classifier to the evaluation labels. Together, these results show that TriPoD’s pretrained representation supports global classification, dense anatomical and lesion segmentation after fine-tuning, and language-aligned zero-shot recognition.
W2: Mean and standard deviation across at least three random seeds for Sections 4.2–4.5
A: For each downstream task, we compared all methods on the same dataset split so that they were evaluated on the same cases. Repeating every method on every dataset in Sections 4.2–4.5 for at least three seeds would exceed our available compute budget. We therefore repeated TriPoD fine-tuning on two representative segmentation tasks: SegTHOR for thoracic anatomy and MSD Hepatic Vessel for vessels and tumors.
We used three seeds to generate three data splits and kept all other training and evaluation settings fixed. We report Dice (%) for each split and the mean and sample standard deviation across the three runs.
These standard deviations quantify variation across data splits for TriPoD on the two tasks.
W3: Definition of the optimization step r and the loss-weight schedule λ(r)
A: After Eq. (8), we describe r as the optimization step and refer readers to Appendix A.4, but we omit the explicit schedule and the direct reference to Table 7, where we report the target weight and warm-up length.
During CT–report alignment, we count optimizer updates starting at r=1. At update r, we multiply the PD-OSL loss in Eq. (8) by λ(r)=0.5min(r/300,1). The weight is 0.5/300 on the first update, reaches 0.5 on update 300, and stays at 0.5 through update 6,000. We will revise the text immediately after Eq. (8) to define the update count, state the formula, explain the warm-up in words, and cite Table 7 directly.
W4&W5: Qwen candidate extraction and construction of PD-OSL training examples from reports
A: W4. Qwen’s role. We used
qwen3.8-maxoffline to organize report passages and extract candidate findings. During CT–report alignment, we used the resulting sentence pairs to train CXR-BERT. The two extraction prompts are summarized below.W5. Correspondence between Figure 2(b) and Section 3.4. We respectfully disagree that the figure and the method are inconsistent. The operations labeled dilute and supplement are both defined in Section 3.4. Equation (4) gives the probability of retaining an explicit negative for a negative-dominant phrase. Equation (5) specifies how we sample weak counterfactual negatives for positive-dominant phrases. Section 3.4 also describes how we anchor atomic findings to report passages and construct their opposites. We agree that we gave too little detail on how we preprocess reports to obtain those findings.