- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
UniCTPET is a shared 3D latent flow-matching framework for single-modality, structurally conditioned, CT-to-PET, and joint CT/PET generation. It introduces TTSE for time- and feature-dependent cross-modal exchange and SLG for skeletal and lesion guidance. Experiments on three datasets demonstrate improved generation quality and downstream lesion segmentation over competing methods.
Critical strengths
-
Useful task unification. The framework accommodates different input-output configurations within one model, with evaluation on three large-scale datasets comprising 1,599 paired scans.
-
Comprehensive experimental evaluation. The paper covers multiple generation tasks and downstream lesion segmentation, providing extensive empirical evidence.
Critical weaknesses
-
Limited Novelty and Insight. The motivation for unifying different CT/PET generation tasks is reasonable. However, the proposed methods mainly build on existing techniques such as cross-attention, feature gating, and multi-scale conditioning. The paper lacks a sufficiently compelling new insight beyond combining these components, making the methodological novelty somewhat incremental for ICLR.
-
Insufficient Validation of the Core Contribution. TTSE is the central contribution, but its adaptive structural exchange mechanism is not thoroughly validated or understood. The current ablations mainly compare TTSE with standard cross-attention, without isolating its key design choices. Although Fig. 1 provides some analysis of exchange strength and gradient alignment, the paper lacks deeper diagnostics explaining how the adaptive exchange works, why it improves performance, and whether it actually reduces interference between generation tasks.
Questions for authors
-
What exactly is conditioned on during evaluation? Please specify the skeletal and lesion inputs available to every method in Tables 2–3 and explain how reference images are assigned for PSNR/SSIM evaluation.
-
More detailed ablations are needed to validate TTSE, including fixed, time-only, and feature-only gating, as well as removing shared or differential structural features. The authors should also analyze the learned gating behavior across generation tasks and flow timesteps.
-
As I understand it, the paired conditional generation tasks are closely related to medical image translation. The authors should include more representative medical image translation baselines, rather than focusing mainly on generative models, to provide a more comprehensive comparison.
-
More details on the VQGAN tokenizer training are needed, including its architecture, loss functions, and training configurations. It would also be helpful to report reconstruction performance (e.g., PSNR and SSIM) for CT and PET.
-
The motivation and novelty of SLG are less convincing than those of TTSE. Its differences from existing conditioning approaches should be better explained.
-
The paper claims that SLG is lightweight, but provides no computational evidence. A comparison with ControlNet in terms of parameters, GPU memory, and computational cost is needed.
-
The motivation for the two-stage training strategy is unclear. Why is TTSE trained separately rather than jointly with the backbone? The choice of 200k and 5k training steps also needs justification, ideally through comparisons with alternative training schedules.
6 comments
R3. Additional medical image translation baseline. Following our previous response, we have completed the Uni-Brain (MICCAI 2026) comparison. We evaluated 321 cases using 50-step DDIM sampling. Uni-Brain achieves an overall PSNR of 36.885±2.380 dB and SSIM of 0.93736±0.00707 (mean±SD), both slightly lower than our model’s 36.99 dB PSNR and 0.943 SSIM. Dataset-specific results are reported below.
| Dataset | Cases | PSNR (dB) | SSIM |
|---|---|---|---|
| DeepPSMA | 20 | 34.442±2.411 | 0.93820±0.00611 |
| FDG | 181 | 37.834±2.037 | 0.93456±0.00672 |
| PSMA | 120 | 35.862±2.146 | 0.94144±0.00559 |
| Overall | 321 | 36.885±2.380 | 0.93736±0.00707 |
Thank you for your comprehensive rebuttal, which has addressed most of my concerns. However, the methodological novelty of the paper remains relatively modest. Therefore, I decide to maintain my original rating.
Thank you for your detailed comments. We clarify the methodological and implementation details below and outline our planned additional analyses and experiments.
R1. Conditioning inputs and evaluation. For structurally conditioned generation in Tables 2–3, the generative baselines receive skeletal information and lesion masks through ControlNet, while our method uses SLG to incorporate the same conditions. The cross-modal methods, IL-CLDM and CoCoLIT, instead use VQGAN-encoded CT latents as conditioning. PSNR/SSIM are computed against the corresponding real CT or PET image associated with the conditioning inputs, over the full volume, including background and brain regions.
R2. TTSE ablations and gating behavior. The original submission compares TTSE with standard cross-attention. We are conducting additional ablations with fixed, time-only, and feature-only gating, as well as removing shared or differential structural features. We are also analyzing gating behavior across CT-to-PET and joint generation at different flow timesteps, and will report the results before the rebuttal/discussion period ends.
R3. Medical image translation baselines. The original submission includes IL-CLDM (IEEE JBHI, 2025; published online in 2024) and CoCoLIT (AAAI 2026) as CT-to-PET translation baselines. Table 3 reports PSNR/SSIM of 16.38 dB/0.173 for IL-CLDM and 17.33 dB/0.621 for CoCoLIT, compared with 36.99 dB/0.943 for our method. We also plan to add Uni-Brain (MICCAI 2026) as an additional baseline and report the results before the rebuttal/discussion period ends.
R4. VQGAN configuration and reconstruction quality. CT and PET use separate 3D VQGANs with residual blocks, four-head bottleneck self-attention, and 4× downsampling along each spatial axis. Their hidden widths are 96/48, respectively; both use 24 normalization groups, an 8,192-entry codebook, and four latent channels. Parameter counts excluding discriminators are 30.87M/7.78M. A 64³ patch maps to a 4×16×16×16 latent tensor. The flow model uses continuous pre-quantization latents, with quantization applied during decoding.
Each VQGAN was trained for 200 epochs on random 64³ crops with batch size 1, using Adam (initial learning rate 1e-4, betas 0.5/0.9). The learning rate decreased to 5e-5 and 1e-5 after epochs 20 and 40. The generator objective is:
LG=L1+0.3Lperc+0.25MSE(ze,sg(zq))+0.1Ladv.
The perceptual term uses VGG-based LPIPS on slices sampled along three axes; the adversarial term uses least squares. The codebook is updated by EMA with decay 0.99.
Across 321 volumes per modality, reconstruction PSNR/3D SSIM are 37.06±1.28 dB / 0.975486±0.010472 for CT and 54.70±2.46 dB / 0.995977±0.000601 for PET (mean±SD). Metrics use full volumes normalized to [0,1], including background, with PSNR data range 1 and a 7³ SSIM window.
R5. SLG motivation. Our motivation is to unify conditional and unconditional generation within one model, allowing structural guidance to be introduced as needed while preserving a reusable generative backbone. ControlNet is commonly used for such guidance, but requires duplicating the full U-Net encoder and middle block. We designed SLG to provide sufficient skeletal and lesion guidance through independent, lightweight condition encoders that inject features directly into the backbone.
R6. SLG efficiency. We measured SLG and ControlNet as standalone conditioning modules, excluding the backbone and other model components.
SLG reduces parameters by 95.6%, peak inference memory by 57.7%, and peak training memory by 40.5%.
R7. Training schedule. The backbone was trained for 200k steps, and TTSE and the baseline ControlNet modules were each trained for 50k steps. The “5k” reported in the original manuscript is a typographical error, which we will correct.
Training TTSE with the backbone fixed preserves the established generation capability while learning cross-modal interactions. We also completed joint training from the outset: CT-to-PET reaches 40.18 dB PSNR and 0.97496 SSIM, but unconditional joint generation collapses. Images illustrating this collapse are available in our anonymous GitHub repository.