- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DART, a text-conditioned diffusion model for correcting medical segmentation predictions. DART takes a single-class probability map and an anatomical description, adds noise to an intermediate timestep, and then applies reverse diffusion to obtain a corrected mask. The model is trained on predicted probability maps from CT-family datasets, with a vocabulary of 377 anatomical classes. The experiments cover different upstream segmentation models, transfer to MRI and unseen classes, and downstream training with corrected labels. The paper also presents DART-45K, an annotation resource produced through a pipeline combining specialist models, DART correction, and DART-Seg.
Critical strengths
(1)The method addresses a useful problem. A correction model that works on segmentation probabilities can be applied after different upstream models without requiring the original medical image. The automatic CT experiments and MRI results provide useful evidence for this design. For example, Appendix C reports an improvement from 74.1 to 82.3 Dice for MRI pancreas segmentation.(2)The broad anatomical coverage and the use of multiple controls strengthen the evaluation. In the 90-class TotalSegmentator comparison, mean Dice increases from 73.6 to 75.6, and DART performs best on 77 classes. The identity, volume-matched identity, pure-noise, and traditional postprocessing controls help check whether the improvements can be explained by preprocessing or simple shape operations. The inclusion of connectivity and boundary metrics is also helpful.(3)The corrected labels improve downstream segmentation models. All fifteen configurations in Table 8 improve when trained with DART-corrected labels. The gain is therefore not limited to one student architecture. The annotation pipeline and the audit of 10,000 scans also represent a substantial effort and could be useful to the community.
Critical weaknesses
Major concerns (1)DART and the single-pass U-Net baseline are not given equivalent input information. Section 4 and Appendix M state that DART receives soft probability maps, while the single-pass U-Net receives thresholded masks. Thresholding removes uncertainty information that remains available to DART. Using the same backbone and text conditioning does not remove this difference. The comparison therefore cannot show how much of the improvement comes from diffusion or partial noising, rather than from retaining the original probabilities. A strong single-pass model trained with probability inputs is an important missing baseline. (2)The text-versus-index ablation mixes the effect of conditioning with class coverage. Table 6 reports 74.20 Dice for text conditioning and 69.10 for index conditioning. However, Appendix J explicitly states that the evaluation includes classes outside the index model’s vocabulary. This shows an advantage of text for representing unseen classes, but it does not isolate the benefit of anatomical semantics. The mismatch and paraphrase experiments do not fully answer this question either. A model that mainly uses the text embedding as a class identifier could also perform worse with a wrong description and remain stable under similar paraphrases. (3)I could not trace how the training probability maps were generated or how parameters were chosen for unseen classes. Appendices A and B describe the datasets, but it is difficult to reconstruct which upstream model and checkpoint generated the training probabilities for each anatomical class. The specialist models listed in Appendix N describe the 45K annotation pipeline and do not directly resolve this question about DART training. There is also an unclear example in Section 3.3: the hippocampus provides the noise level for a hippocampal tumor, while Figure 4a lists the hippocampus itself as an unseen task. The source of this parameter should be explained. (4)The vocabulary-scaling control only evaluates the original 50 classes. In Table 5, the 50-class and 377-class models are both tested on the same original 50 classes, obtaining 73.85 and 74.20 Dice. This supports the claim that expanding the vocabulary does not reduce the old-class mean. It does not show how the added classes perform, whether low-frequency classes benefit, or whether some old classes become worse despite the stable average. Other experiments do evaluate more classes, but the controlled vocabulary experiment still gives limited evidence about what is gained by sharing one model across 377 classes. (5)Preservation of abnormal anatomy needs more direct evaluation. The KiTS23 and lesion-transfer results are useful, but an improvement in whole-organ Dice does not necessarily mean that a small abnormal region is preserved. For example, the reported kidney Dice improves from 91.8 to 95.1, without a separate measure of how much of the tumor region remains covered by the corrected kidney mask. Since DART learns a distribution of predicted shapes, it may favor common shapes during correction. The central uncertainty is when correction removes a genuine segmentation error and when it might suppress uncommon but valid anatomy.
(6)The overview in Figure 2 looks clearly AI-generated. It should be redrawn. (7)The baseline comparisons are less complete in the automatic and MRI settings. The most complete 90-class comparison uses MedSAM predictions obtained with reference-derived bounding boxes. The paper also includes automatic-input experiments, but Figure 3 aggregates selected structures, and the external tables do not provide the same complete comparison against learned correction methods. Appendix M indicates that additional external baselines were evaluated. Reporting these results would help determine whether DART remains better than other correction methods under the transfer settings, beyond simply improving over the original input.
Questions for authors
(1)Could the authors train the single-pass U-Net with soft probability inputs, using the same backbone, text conditioning, subject splits, and a comparable training budget? Does DART still show a consistent advantage? A DART variant trained and evaluated with binary inputs would further help separate the effect of input information from the correction procedure.
(2)What is the text-versus-index difference when evaluation is restricted to classes covered by both models, with matched training examples? Please report unseen classes separately. Could the authors also compare class-name conditioning with full anatomical descriptions under matched training conditions, to test whether the shape and location descriptions provide additional benefit?
(3)Please provide the actual training, validation, and test counts, together with the upstream model and checkpoint used to generate the training probabilities for each class. Please also specify the crop margin and the source of (t^*) for every unseen task. In particular, how is the hippocampus noise level obtained in the example in Section 3.3?
(4)For the classes with reference annotations, could the authors report results separately for the original and added classes, and for frequent and rare classes? Which original classes become worse after expanding the vocabulary? For the positive-transfer claim, a small comparison with separately trained class groups would be more informative.
(5)On the existing abnormal-anatomy cases, what fraction of cases becomes worse after correction, and how large are these decreases? Could the authors report coverage of the annotated abnormal region by the corrected organ mask, together with representative failure cases? Repeated sampling on a small subset would also help establish whether these findings are stable.
(6)Please replace Figure 2 with a redrawn technical overview of the method.
(7)Please provide the external baseline results by dataset and upstream model, including paired differences against the strongest correction baseline and uncertainty on those differences. On representative automatic CT and MRI tasks, does DART retain its advantage over the probability-input baseline requested in Question 1?
10 comments
Response to Reviewer vGk9 (2/6): Vocabulary Coverage and Anatomical Text Semantics
Q2. Text conditioning, class coverage, and anatomical semantics
Concern: Table 6 mixes text conditioning with class coverage, because the index-conditioned model cannot represent classes outside its training vocabulary. Prompt mismatch and paraphrase tests do not fully isolate descriptive semantics.
Question 2: What is the text-versus-index difference when evaluation is restricted to classes covered by both models, with matched training examples? Please report unseen classes separately. Could the authors also compare class-name conditioning with full anatomical descriptions under matched training conditions, to test whether the shape and location descriptions provide additional benefit?
Thank you for this important methodological distinction. We agree that Table 6 measures a combination of vocabulary extensibility and anatomical conditioning, and that its overall difference cannot be attributed solely to the semantic content of descriptions.
Table 6 evaluates TotalSegmentator, AMOS22, and WORD cases, including classes outside the learned index vocabulary:
| Model | Mean Dice (%) | Gain (points) |
|---|---|---|
| Original predictions | 68.71 | — |
| Index-conditioned | 69.10 | +0.39 |
| Text-conditioned DART | 74.20 | +5.49 |
The 5.10-point gap is not a pure estimate of semantic understanding. Instead, it demonstrates a practical advantage of an extensible conditioning interface: a frozen text-conditioned model can receive a description of a new class, whereas an index embedding has no learned entry for it. We do not claim that detailed text necessarily improves Dice on familiar classes.
We have also conducted a preliminary class-name-only versus full-description comparison. On the tested subset, full anatomical descriptions did not provide a substantial improvement over class names alone. We will report this observation transparently; it suggests that representations of anatomical identity may account for much of the observed conditioning benefit.
At the same time, Figure 4(a) documents correction on seven transfer tasks excluded from DART training, including four lesion classes and six MRI tasks. For example, hippocampus improves from 78.0% to 81.4%, hippocampal tumor from 73.0% to 78.1%, and pancreatic cyst from 55.2% to 59.1%. These findings demonstrate zero-shot transfer, not a causal decomposition of the text embedding.
We additionally tested an incorrect full anatomical description on the held-out hippocampus task. Dice fell from approximately 81.4% with the correct description to 76.3% with the incorrect one (a 5.1-point decrease). Appendix E similarly finds near-invariance to correct paraphrases (−0.02 points on average) but a 2.07-point decrease under mismatched descriptions. These tests establish sensitivity to conditioning, including outside the training vocabulary, but changing a whole description also changes class identity.
We agree that the specific question about shape and location semantics remains unresolved by the current experiments. In the revision we will add (i) a text-versus-index evaluation restricted to classes seen by both models with matched training examples, (ii) separate seen- and unseen-class results, and (iii) an expanded class-name-only versus full-description analysis. We will additionally preserve the class name while perturbing individual words, shape descriptions, and location cues, on both seen and unseen classes.
We appreciate the suggestion and will clarify that vocabulary coverage, prompt sensitivity, and independently measured semantic contribution are distinct claims.
Response to Reviewer vGk9 (3/6): Training Provenance, Data Splits, and Noise Levels
Q3. Training probability maps, reproducibility, and noise-level transfer
Concern: The manuscript does not make it possible to trace the upstream models/checkpoints used to generate training probability maps, the dataset splits, or the source of noise levels on unseen tasks. Section 3.3 appears inconsistent with the unseen hippocampus task in Figure 4(a).
Question 3: Please provide the actual training, validation, and test counts, together with the upstream model and checkpoint used to generate the training probabilities for each class. Please also specify the crop margin and the source of t* for every unseen task. In particular, how is the hippocampus noise level obtained in the example in Section 3.3?
Thank you for identifying these reproducibility gaps. We agree that listing the datasets is not sufficient to reconstruct which upstream prediction model and checkpoint generated each class's training probabilities.
Training-probability provenance. We initially considered approximately 25–30 segmentation models, drawing on reported performance and independent evaluations including Touchstone (Bassi et al., NeurIPS 2024), as well as published results for TotalSegmentator, AtlasNet, MOOSE, CADS, and other systems. We prioritized strong class-specific upstream performance rather than relying on one universal segmentation model. As Appendix A reports, DART was trained on predicted probability maps from 3,066 volumes across eight public datasets.
We will add a reproducibility table giving, for every class, the source dataset, upstream model and exact checkpoint, and will distinguish those training-prediction sources from the specialist models employed later in the separate DART-45K annotation pipeline (Appendix N). We will also report exact subject-level training/validation/test counts and numerical preprocessing crop margins, replacing the current approximate description of dataset-specific partitions.
How t* was chosen. For classes with validation calibration, t* was selected once from the specified candidate noise levels on validation data, then fixed across test cases and upstream models. For an unseen class, we reused the calibrated noise level of the closest anatomically similar class. The anatomical analogue was selected based on structural similarity; this was a nearest-anatomy matching decision, not an additional optimization on target-task test labels. We will provide a mapping of each unseen task to its source class and fixed transferred t* so that this procedure is fully reproducible. We will not describe the matching step as an automatically learned parameter predictor.
Hippocampus clarification. We agree that Section 3.3 is inaccurate as written. Figure 4(a) states that the hippocampus was not in the DART training vocabulary, so the manuscript's example should not imply that a trained hippocampus class supplied a calibrated t* for hippocampal tumor. For these held-out tasks, the levels were instead transferred from anatomically similar classes with existing calibrations, without choosing t* using their test annotations. We will correct the example and explicitly identify the actual source class and numerical t* for both hippocampus and hippocampal tumor in the revision.
These details are not yet fully specified in the current manuscript; we do not want to invent checkpoint IDs, split counts, crop margins, or the particular source class in this reply. We appreciate the request and will supply the exact provenance and parameter mapping in the revised appendix.
Response to Reviewer vGk9 (1/6): Matched Probability Inputs and Binary-Input DART
Thank you for your detailed and constructive review. We appreciate your recognition of DART's usefulness as an image-independent correction framework, its broad anatomical coverage and controls, and the improvements obtained when downstream models train on DART-corrected labels. Your questions have helped us identify important comparisons and reporting details to strengthen.
We respond to each of your seven concerns and corresponding questions below. In particular, we have completed new probability-matched and binary-input experiments, an additional 327-class evaluation, and case-level tumor analysis. We distinguish completed evidence from analyses that will be added in the revision.
Q1. Probability inputs versus binary-mask baselines
Thank you for identifying this legitimate confound. We agree that the original U-Net comparison did not isolate the advantage of retaining soft probabilities from that of DART's correction procedure. Following your suggestion, we conducted two complementary experiments.
Probability-matched U-Net. We trained a single-pass U-Net with the same backbone, soft probability-map inputs, anatomical text conditioning, training data, subject splits, and training settings as DART. We evaluated on approximately 500 cases from five datasets—TotalSegmentator, AMOS22, FLARE22, PDDCA, and MSD Pancreas—with upstream predictions generated by TotalSegmentator and MedSAM.
Table R1. Mean Dice across the five-dataset matched comparison.
DART exceeds the probability-matched U-Net by 3.5 Dice points. We also tested the same TotalSegmentator cases highlighted in the manuscript: DART outperformed the matched U-Net on 88 of 90 organs. We will include the full per-organ comparison in the revision.
Binary-input DART. We also trained and evaluated a DART variant using thresholded binary inputs rather than the full probability maps. Mean Dice remained within approximately 1–2% between the two DART variants. However, retaining probabilities improved normalized surface Dice (NSD) by up to 20% on selected organs, showing that the additional uncertainty information is particularly useful for boundary refinement.
Together, these experiments establish that probability information helps boundary quality but does not explain DART's entire advantage over the single-pass model. We do not claim that they separately isolate partial noising, iterative denoising, and the diffusion training objective; rather, they isolate the probability-input confound and support the advantage of DART's overall diffusion-based correction procedure. We will add both experiments, with per-organ Dice and NSD results, to the revision.