- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents TUG, a BrLP-based conditional 3D latent diffusion model designed to align whole-image and tumor-specific objectives. TUG incorporates three key designs: (1) an Evolution-Aware Residual Modulator (ERM) conditioning ControlNet residuals on baseline tumor masks, intervals, and treatments; (2) two auxiliary losses (ECL and ERA) that supervise future lesions using frozen segmentation networks; and (3) a test-time trajectory pruning and blending strategy via a learned ranker. On 228 validation pairs across three cohorts (LUMIERE, MU-Glioma-Post, UCSF-ALPTDG), TUG achieves the highest Dice scores while matching BrLP in PSNR and SSIM.
Critical strengths
- The problem is well motivated: Fig. 1 and the intro make a convincing case that PSNR/SSIM alone hide clinically relevant lesion errors, and reporting image and lesion fidelity jointly (Table 1, Fig. 4) is a useful practice for this field.
- The method is well designed: ERM is initialized to preserve the pretrained pathway, the future mask is used only as supervision, and the trajectory scorers are trained with patient-grouped out-of-fold predictions so that no future information is used at inference.
- The comparison is compute-matched (Eq. 14), and the ablations cleanly separate the contribution of training-time guidance (Table 2a) from inference-time refinement (Table 2b). The reported numbers are internally consistent.
Critical weaknesses
-
The tumor-fidelity metric is the same quantity that TUG optimizes.
- The nnU-Net segmenter used for evaluation (L356, L705-706) is the one used in the ECL loss, in the prospective scorer's inputs (L305-306), and to build the scorer's training targets (L724-725). TUG is therefore trained and selected against the exact function used to score it, while the baselines are not. Part of the Dice gain may come from fitting this specific segmenter rather than from better tumor evolution.
- The limitations section acknowledges this (L483-484), but it is central to the main claim. At least one independent segmenter (different architecture or training data), ideally complemented by expert reading on a subset, should be used for evaluation.
-
The evaluation measures the final lesion state rather than tumor evolution.
- Dice against the future mask can be high even if nothing changes. A copy-current-scan baseline (X̂f = Xs) is missing and is essential in longitudinal forecasting, especially since MU and UCSF masks include the resection cavity (L672-677), which is largely static.
- Change-specific metrics would directly support the paper's claim: e.g. error in lesion volume change, Dice restricted to the growth/regression regions, or agreement in response category (progression/stable/regression).
-
The inference-time gain is not isolated from the effect of averaging fewer samples.
- Table 2b compares Mean20 with "Pruning + Mean8". Averaging fewer samples by itself produces a sharper lesion (the paper itself argues that averaging suppresses spatially varying structure, L50-53), so Dice can increase without a good scorer. A random-pruning + Mean8 control is needed, together with a simple heuristic selector (e.g. consensus/medoid) and an oracle best-of-N upper bound to show how much of the available gap the learned scorer closes.
- The final output is still an (weighted) average image, and Dice is computed on that average. Results on individual samples (e.g. mean Dice of the top-1 trajectory) would clarify whether the method improves forecasts or mainly improves aggregation.
-
The evaluation protocol needs clarification and statistical support.
- Results are reported on a "held-out validation set" (L325), and the TaDiff-3D checkpoint is selected "from validation performance" (L805). It is unclear which split was used to choose TUG's hyperparameters (loss weights, timestep ranges, the 24→20→8 schedule, τ). If the same 228 pairs were used, the results are not test results.
- No standard deviations, confidence intervals or significance tests are reported. Several key gaps are small (e.g. 0.5217 vs 0.5155 case-avg. Dice against TaDiff-3D). A paired test over the 228 cases and results over multiple sampling seeds are needed.
- Results are pooled across cohorts with different lesion definitions (L672-677). Per-cohort results should be reported.
-
Baselines and the role of ERM.
- It is not stated whether BrLP and LDM-GBM were trained on the same 949 pairs, nor what the "pretrained forecasting model" used for initialization (L215-216) was trained on.
- SADM (PSNR 15.51, SSIM 0.196, Dice 0.06) and TaDiff-3D (SSIM 0.52) are author adaptations whose results suggest they may be undertrained or poorly suited to this setting. This weakens the comparison, as acknowledged in L484-485.
- ERM alone does not appear to help much: "No extra loss" (ERM + Mean20, Table 2a) vs. BrLP (Table 1) gives +0.012 case-avg. Dice but −0.011 SSIM. ERM is never ablated against ECL+ERA without ERM.
-
Novelty is mainly in the combination. Each component closely follows an existing technique: FiLM-style residual modulation (ERM), reward feedback on one-step clean estimates as in ControlNet++ (ECL), representation readout/alignment as in Readout Guidance/REPA (ERA), and progressive seed pruning/TTSnap (trajectory pruning). The application to longitudinal forecasting is reasonable, but the paper should state more explicitly what is new beyond adapting these techniques.
Questions for authors
- Please report Dice with an independent segmenter, a copy-current-scan baseline, and at least one change-specific metric.
- Please add the random-pruning + Mean8 control and an oracle best-of-N upper bound for Table 2b.
- Please clarify which data were used for hyperparameter selection, and report per-cohort results with confidence intervals or paired significance tests.
9 comments
W2: Future Lesion-State Fidelity vs. Evolution-Specific Accuracy
The evaluation measures the final lesion state rather than tumor evolution.
Dice against the future mask can be high even if nothing changes. A copy-current-scan baseline () is missing and is essential in longitudinal forecasting, especially since MU and UCSF masks include the resection cavity (L672-677), which is largely static.
Change-specific metrics would directly support the paper's claim: e.g. error in lesion volume change, Dice restricted to the growth/regression regions, or agreement in response category (progression/stable/regression).
We thank the reviewer for this insightful comment. We agree that future-tumor Dice evaluates agreement with the observed future lesion state rather than explicitly measuring the spatial localization of growth or regression. However, we would like to clarify that future lesion-state fidelity remains a meaningful and relevant objective in longitudinal MRI forecasting.
In our pairwise forecasting setting, the initial state is fixed and known, making the observed future lesion geometry a clinically relevant prediction target. This endpoint-based evaluation is also consistent with established longitudinal glioma forecasting studies, such as TaDiff (Liu et al., IEEE TMI 2025). Accordingly, our objective is to generate future MRI volumes whose expressed lesion structures spatially agree with the observed future state, rather than to predict lesion masks independently. Future-tumor Dice therefore measures an important aspect of image-based forecasting fidelity, although it does not fully characterize the underlying evolution.
We therefore regard change-specific metrics as important complementary evaluations, rather than replacements for future-state fidelity. Our contribution is to introduce lesion-aware supervision and trajectory selection into full-volume MRI forecasting to improve the agreement between generated and observed future lesion states.
W3: Effectiveness of Trajectory Pruning and Aggregation
The inference-time gain is not isolated from the effect of averaging fewer samples.
Table 2b compares Mean20 with "Pruning + Mean8". Averaging fewer samples by itself produces a sharper lesion (the paper itself argues that averaging suppresses spatially varying structure, L50-53), so Dice can increase without a good scorer. A random-pruning + Mean8 control is needed, together with a simple heuristic selector (e.g. consensus/medoid) and an oracle best-of-N upper bound to show how much of the available gap the learned scorer closes.
The final output is still an (weighted) average image, and Dice is computed on that average. Results on individual samples (e.g. mean Dice of the top-1 trajectory) would clarify whether the method improves forecasts or mainly improves aggregation.
We thank the reviewer for this valuable suggestion. We agree that reducing the number of averaged trajectories may itself affect lesion fidelity, independently of the learned scorer. To isolate these effects, we conducted additional ablations involving random pruning, oracle selection, and different aggregation strategies.
(1) Learned pruning vs. random pruning. We introduced a Random Pruning + Mean8 baseline, which retains the same number of trajectories as our learned pruning strategy but selects them randomly.
| Selection / Aggregation | Future Dice |
|---|---|
| Mean20 (without pruning) | 0.4871 |
| Random Pruning + Mean8 | 0.4847 |
| Learned Pruning + Mean8 | 0.5089 |
| Learned Pruning + Soft Weighting (τ=1) | 0.5217 |
| Oracle Top-8 + Mean8 | 0.5288 |
| Oracle Top-1 | 0.5505 |
With the number of averaged trajectories held constant, learned pruning improves Dice over random pruning by +0.0242 (95% CI [0.0168, 0.0325]). Learned pruning also outperforms the unpruned Mean20 baseline, whereas random pruning does not. This indicates that the improvement cannot be attributed solely to reducing the number of averaged samples.
(2) Oracle selection. We further evaluated ground-truth-guided oracle selection to estimate the remaining potential of candidate ranking.
| Selection Strategy | Future Dice |
|---|---|
| Learned Pruning + Mean8 | 0.5089 |
| Oracle Top-8 + Mean8 | 0.5288 |
| Oracle Top-1 | 0.5505 |
Oracle Top-8 + Mean8 achieves a Dice of 0.5288, compared with 0.5089 for learned pruning under the same eight-candidate averaging protocol. This leaves a gap of 0.0199, suggesting that the learned scorer captures a substantial portion of the available improvement while leaving room for better candidate selection. Oracle Top-1 reaches 0.5505, additionally indicating the existence of high-quality individual trajectories within the candidate pool.
(3) Selection vs. aggregation. The comparison between learned pruning with uniform averaging (0.5089) and subsequent soft weighting (0.5217) suggests that both candidate selection and aggregation contribute to the final prediction. Importantly, our random-pruning control establishes that the benefit of learned selection persists even when the aggregation operation and number of retained candidates are identical.
Overall, these ablations support the effectiveness of the learned trajectory scorer beyond the confounding effect of averaging fewer samples, while highlighting the complementary roles of candidate selection and aggregation.
W1: Independent Segmentation Evaluation and Evaluator Bias
We thank the reviewer for this important observation. To address potential evaluator bias, we conducted additional experiments using an independently trained 3D SegResNet segmenter (S2), which differs architecturally from the original nnU-Net evaluator and was not involved in ECL supervision, prospective scorer training, or trajectory selection. To account for random initialization variability, S2 was trained with three independent random initializations, and the reported results were averaged across the three runs.
TUG achieves a paired improvement of +0.0513 (95% CI [0.0187, 0.0791]) over TaDiff-3D. These results demonstrate that TUG's improvement in future lesion-state agreement is not confined to the original evaluation segmenter, mitigating the concern that the observed gains arise solely from fitting the segmentation function used during training.