acceptodds
Under review as a conference paper at ICLR 2027

A High-Compression Transformer VAE for Efficient 3D Medical Image Generation

Abstract

Diffusion Transformers have enabled large-scale image and video generation, but their extension to three-dimensional medical images is constrained by long spatial token sequences and the resulting attention cost. High spatial compression offers a practical route to volumetric generation, yet preserving fine anatomical and pathological detail at such compression remains challenging. We introduce a high-capacity Transformer variational autoencoder trained on BrainTumor-256K, a large multi-sequence brain tumor MRI collection. Our model compresses each spatial dimension by a factor of , yielding a Gaussian latent representation with fewer spatial locations. The model combines high reconstruction fidelity with efficient computation on this compact grid. Across four public brain MRI datasets, our model achieves the highest mean PSNR among the evaluated autoencoders, with consistent advantages in structural and perceptual metrics. Despite its large parameter count, it delivers the lowest encoding-decoding latency at among the evaluated autoencoders and requires substantially less training memory than comparably sized convolutional backbones. Trained exclusively on brain MRI, the model also reconstructs non-brain MRI and CT without target-domain fine-tuning, exceeding all evaluated medical autoencoders in PSNR on every non-brain dataset, and accommodates full volumes of varying sizes and shapes. Modality-controlled and tumor-mask-guided diffusion generation further demonstrate the utility of its compressed representation. These results establish high-capacity Transformer autoencoding as an effective approach to high-compression medical image representation and a practical foundation for volumetric generation. Code is available at https://anonymous.4open.science/r/transformer-vae3d-2D31

Primary area:
generative models
Keywords:
Variational autoencoder, Vision Transformer, 3D medical imaging, High-compression latent representation, Latent diffusion, Efficient autoencoding

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NFrSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a high-capacity transformer VAE for volumetric medical images. The model applies isotropic 8×8×88\times8\times8 spatial compression, reducing the number of spatial locations by 512×512\times, and produces a 16-channel Gaussian latent representation. In addition, it's trained on BrainTumour-256K, which is a quite large number and private multi-sequence brain MRI collection. The paper evaluates reconstruction fidelity, encoding-decoding latency, training memory, cross-domain reconstruction, tumor-region preservation, and the utility of the learned representation for modality-controlled and tumor-mask-guided generation with a conditioned DiT.

The proposed model achieves strong reconstruction results across multiple public brain MRI datasets and favorable latency and memory measurements despite its large parameter count.

Critical strengths
  1. This paper addresses an important practical bottleneck in volumetric generation. Reducing each spatial dimension by a factor of 8 produces a (512×512 \times) reduction in spatial locations, which can substantially reduce the token count and computational cost of the generative models. Sepcially, at 2−8×2-8\times higher spatial compression than most baselines, the C16 has the highest PSNR on all four public brain datasets, at Table 1. The distinction between spatial compression and total element compression is clearly formulated, and the proposed representation achieves strong reconstruction fidelity despite the compact spatial grid.

  2. The empirial evalution is broad and carefully designed. The paper evaluates reconstruction on multiple public brain MRI datasets, cross-domain reconstruction on non-brain MRI and CT data, tumor-region presevation, encoding-decoding latency, training memory and downstream DiT resource usage. The proposed model achieves the highest mean PSNR across four public datasets and favorable encoding-decoding latency at 2563256^3 resolution, while the controlled backbone comparison provides useful evidence that the Transformer design can reduce training memory at comparable parameter counts.

Critical weaknesses
  1. It's unclear how much of the gain comes from the architecture. The paper's main takeaway is that high-capacity Transformer autoencoding works well, but the comparisons can't really show this. The proposed model has 526M parameters and is trained on 256K brain volumes, while the baselines differ in size, training data, objectives and target domains. So the results tell us the full system is strong, but not whether the Transformer backbone is what makes it strong, as opposed to model capacity or the training data. The efficiency claim has the same gap, the source of the reported efficiency advantage is unclear beacuse the comparison does not isolate the effects of the backbone architecture and early spatial compression, and no FLOPs are reported. Although Table 3 matches model capacity and training objective, it reports only memory consumption and therefore does not establish whether the Transformer architecture provides a better fidelity efficiency trade-off than a matched CNN VAE.

  2. The generation results not show this latent space is a good basis for generation. This paper shows the f8/C16 latent maskes DiT training cheaper, but never compares sample quality against other latent spaces under the same DiT and training budget. This matters because Section 5.6 shows f4/C4 with a larger DiT patch size costs about the same, so the question is whether this latent actually gives better samples. The existing evidence is also limited to answer this.

Questions for authors
  1. Could the authors include a small-scale controlled comparison between a Transformer VAE and a CNN VAE of similar size, trained on the same data subset with the same objective? Reporting both reconstruction quality and latency, along with FLOPs, would show whether the gains come from the architecture or from model and data scale.

  2. Section 5.6 shows that an f4/C4 latent with a larger DiT patch size has a similar training cost. Given this, why should we expect the f8/C16 latent to give better generation quality, rather than just a different way of reaching the same token count? Is there evidence or reasoning that higher reconstruction fidelity in this latent translates into better samples?

  3. How was patient-level overlap between BrainTumor-256K and the public evaluation sets (especially BraTS) ruled out?

2 comments

Thank you for asking for a same-data, same-size comparison with latency and FLOPs. It turned out to be the most informative experiment in the revision, and the paper's main argument now rests on it.

1. Comparable-capacity fidelity and computational cost

Question: can similarly sized models be trained on the same data and objective, with quality, latency and FLOPs reported?

Yes. We trained a 249.2M Transformer VAE and a 246.5M CNN VAE from scratch on the same collection, both with f8/C16 Gaussian latents, 128³ crops and the same loss terms. The CNN uses the CONFLUX architecture because it already has this interface; the released CONFLUX checkpoint is evaluated separately. Batch sizes differ because of the CNN's memory footprint.

ModelBraTSOASISIXIInternal
CNN VAE, 246.5M32.4630.0633.8232.18
Transformer VAE, 249.2M32.5531.1334.0832.95
Transformer VAE, 499.3M33.5332.1635.4633.80
Resource measureCNN, 246.5MTransformer, 249.2MTransformer, 499.3M
256³ encode–decode latency (ms)2195.7288.5574.0
256³ forward TFLOPs387.49104.29208.64
256³ encoder latency (ms)690.757.456.7
128³ training step (ms)508.265.9129.6
128³ training peak memory, batch 1 (GiB)47.313.415.3

At equal size the Transformer leads on all four sets, by 0.09 dB on BraTS to 1.07 dB on OASIS. The larger difference is cost, which shapes the reconstruction–cost tradeoff:

  • Capacity. The 499.3M Transformer improves every brain set and is still 3.8× faster than the 246.5M CNN. At 384³, both Transformers process the whole volume, whereas this CNN runs out of memory.
  • Input extent. On the same 128³ scoring region, the two 250M models are tied with crop-only input (−0.005 dB [−0.057, 0.058]) and the Transformer leads by +0.749 dB [0.677, 0.838] with full-volume input, so crop-only evaluation misses an advantage observed with full-volume processing (Section 5.2, Appendix E).

Latency is whole-volume, batch one, on the same GPU type. The training step uses a shared MSE-only objective and measures per-step cost, not time to convergence. FLOPs count executed operations, including fused attention, with one MAC as two FLOPs (Table 2, Appendix D).

2. Compression, DiT patch size and generation quality

Question: why prefer f8/C16 over f4/C4 with a larger DiT patch, and does better reconstruction yield better samples?

Our resource benchmark shows that a larger patch makes f4/C4 about as cheap as f8/C16 at the same token count, so cost alone does not favor either. The case for f8/C16 is where spatial reduction is learned. DC-AE (Table 1) fixes the generator's token count and finds better FID when the autoencoder, trained for reconstruction, does more of the compression. These results motivate our design choice, but do not establish the corresponding generation-quality advantage for 3D MRI. Our study does not demonstrate superior generation quality at a matched DiT token or training budget, and the revised manuscript states this limitation explicitly.

Compression matters for conditioning too. Spatial references are concatenated with the target tokens, so compression shortens both streams, and every image condition must be encoded. Random holes and degradations are constructed in image space and encoded online during training, allowing one DiT checkpoint to support inpainting, deblurring, de-aliasing and motion correction alongside synthesis (Figure 5, Appendix J). The frozen VAE used for these tasks is the benchmarked 499.3M model, whose encoder takes 56.7 ms at 256³ compared with 690.7 ms for the CNN.

3. Patient overlap with public benchmarks

Question: how was overlap between BrainTumor-256K and public datasets, especially BraTS, ruled out?

We agree this matters, and we changed the evaluation accordingly. Within our collection, the training, validation and test splits are patient-disjoint. OASIS and IXI are non-tumor cohorts and do not overlap with our tumor collection.

Because BraTS pools cases from many institutions, the BraTS column of Table 1 now uses 200 BraTS-GLI subjects (800 volumes) sampled after excluding cases that may overlap with our training collection (Section 4, Appendix D). We also removed MSD Brain, which is a subset of earlier BraTS releases.

On the filtered BraTS set the ordering is unchanged: 33.53 dB for the 500M Transformer, 32.55 dB for the 250M Transformer and 32.46 dB for the CNN. The Transformer–CNN comparison also uses the same training data for both models, and the context experiment uses held-out internal patients, so neither depends on public-set overlap.

Thank you for your detailed rebuttal. And the additional experiments to compare the transformer VAE and CNN VAE under similar scale and the explination of compression and patient overlap address my previous concerns. Overall, I am inclined to increase my score from weak reject to weak accept.

Reviewer_yvdTOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a high-capacity Transformer VAE model for efficient 3D medical image reconstruction and generation. The model combines isotropic patch projection, Transformer encoder-decoder modules, and 3D rotational position encoding. In its baseline configuration, the number of spatial positions is reduced by a factor of 512, and the number of scalar elements is reduced by a factor of 32. The model was trained on a dataset of approximately 256,000 private brain MRI voxels. Experiments evaluated reconstruction fidelity, computational efficiency, tumor preservation, and cross-domain reconstruction capabilities. A Diffusion Transformer with 2B parameters further demonstrates how to leverage learned latent representations to achieve MRI-sequence-controlled and tumor-mask-guided generation.

Critical strengths
  1. This paper addresses a real-world bottleneck in volume reconstruction: reducing the number of latent tokens while preserving anatomical and pathological information. The paper clearly distinguishes between spatial compression and scalar element reduction, which helps explain its computational advantages. Efficiency metrics provide strong evidence that large Transformer autoencoders remain computationally feasible.
  2. Reconstruction evaluations were extensive and yielded encouraging results across four publicly available brain MRI datasets. Matching volume comparisons and patient-specific bootstrap confidence intervals further strengthen the evidence, rather than merely reporting average reconstruction scores.
Critical weaknesses
  1. The methodological innovation of this approach appears limited, and the experiments fail to fully elucidate why the proposed design is effective. The architectures described in Sections 3.1 through 3.4 largely follow existing Transformer VAE designs (such as ViTok) and have been adapted for medical volumetric data and compression settings. While these adaptations may be beneficial, the paper does not explicitly explain which modifications address specific medical challenges or propose new design principles. The paper’s contribution would be more significant if it provided rigorously controlled empirical results on the interplay between compression, model capacity, and anatomical fidelity—even without relying on formal theoretical derivations. As currently presented, while the paper demonstrates a successful system configuration, it offers only limited new insights into representation learning under high compression rates.
  2. Table 1 compares a model trained on the BrainTumor-256K dataset—with approximately 526 million parameters—to published models that use different training datasets, model capacities, and training objectives. Meanwhile, the comparison of models with similar parameter counts in Section 5.3 primarily measures computational resource consumption rather than reconstruction quality. Although Section 6 acknowledges these differences, the results do not distinguish the contribution of the Transformer backbone from the effects of model capacity and training scale. Conducting controlled reconstruction comparisons would make the arguments regarding architecture more persuasive.
  3. Table 5 reports slice-level FID, which does not directly assess consistency across a generated volume. Tumor-mask agreement supports spatial control but does not establish the anatomical continuity of surrounding tissue. A 3D feature-based Fréchet distance or blinded multiplanar assessment would strengthen the evidence for volumetric fidelity during generation.
  4. Section 3.2 achieves stronger VAE compression by reducing the number of DiT tokens, but Section 5.6 notes that larger DiT patches can also achieve similar resource consumption. Since the architectures and training methods of the generative baselines vary, these experiments do not demonstrate whether VAE compression achieved through learning can provide better generation quality than DiT patching under the same computational budget.
Questions for authors

What specific design choices or empirical insights distinguish this study from existing Transformer VAE models such as ViTok?

Given comparable training data, model capacity, and training objectives, is it possible to evaluate the advantages of the Transformer backbone network in reconstruction on its own?

Given a matched DiT token budget, does stronger VAE compression improve generation quality compared to less compressed representations and larger DiT patches?

Can volume-based evaluation methods (e.g., Fréchet distance based on 3D features or blind multi-plane evaluations) be provided to assess anatomical consistency in the generation process?

3 comments

Thank you for pushing us to state the contribution clearly and to keep reconstruction, cost and generation evidence apart. The paper presents a complete pipeline for controllable volumetric generation: a Transformer VAE with a favorable reconstruction–cost tradeoff, and a 2B multimodal DiT that builds on it to perform synthesis, completion and restoration from text and image conditions.

1. What distinguishes this work from Transformer VAEs such as ViTok?

ViTok was developed for natural images and video; at the time of submission, we were not aware of a Transformer VAE trained and evaluated on 3D medical volumes. We build on ViTok, ViTok-v2 and DC-AE and do not claim a new block or bottleneck. The contribution is what this design enables in the volumetric setting and what we observe there:

  • Full-volume context. Against a same-size, same-data CNN, our Transformer is tied on 128³ crops but leads by 0.75 dB when given the full volume, so crop-only evaluation misses this advantage (question 2).
  • Cost and capacity. At 256³ the Transformer is 7.6× faster than the CNN, and a larger decoder improves reconstruction by 0.85–1.38 dB while remaining faster than the CNN.
  • One conditional generator. Spatial references double the visual sequence, and every image condition must be encoded. Strong compression keeps both streams short, and fast encoding lets holes and degradations be constructed online. One 2B DiT therefore performs sequence- and mask-controlled synthesis, inpainting, lesion-region completion, deblurring, de-aliasing and motion correction, with tasks differing only in how the condition is constructed (Sections 3.4 and 5.5, Figure 5, Appendix J).
  • Variable-shape, isotropic latents. Equal-axis compression and 3D RoPE support inference beyond the 128³ training crop, across varied volume sizes and aspect ratios and zero-shot on non-brain MRI and CT.

ViTok and ViTok-v2 are now discussed in Related Work, and we treat reconstruction and generation separately.

2. Reconstruction with comparable data, capacity and objective

Yes. We trained a 249.2M Transformer VAE and a 246.5M CNN VAE (CONFLUX architecture) from scratch on the same collection, with the same f8/C16 interface, 128³ crops and loss terms.

MeasureCNN, 246.5MTransformer, 249.2MTransformer, 499.3M
BraTS PSNR (dB)32.4632.5533.53
OASIS PSNR (dB)30.0631.1332.16
IXI PSNR (dB)33.8234.0835.46
Internal PSNR (dB)32.1832.9533.80
256³ encode–decode latency (ms)2195.7288.5574.0
256³ forward TFLOPs387.49104.29208.64
128³ training step (ms)508.265.9129.6
128³ training peak memory, batch 1 (GiB)47.313.415.3

Latency is whole-volume at batch one; the training step uses a shared MSE-only objective and measures per-step cost (Appendix D).

The paired context study shows how the fidelity comparison changes with input extent. On 400 internal volumes, the two 250M models are tied on 128³ crops (−0.005 dB [−0.057, 0.058]). Given the full volume and scored on the same 128³ region, the Transformer leads by +0.749 dB [0.677, 0.838] (Section 5.2, Appendix E).

3. At a matched DiT token budget, is VAE compression better than a larger DiT patch?

Our resource benchmark confirms your premise: with a larger patch, f4/C4 reaches the same 1,296 tokens as f8/C16 at nearly the same cost (2.01B DiT: 118.3 vs. 113.2 ms per step, 30.24 vs. 30.23 GiB; Appendix I). Token count alone does not decide between them.

The difference is where spatial reduction is learned: in an autoencoder trained for reconstruction, or in the generator's patch projection. DC-AE (Chen et al., ICLR 2025, Table 1) tests this at a fixed token count and finds better FID when the autoencoder does more of the compression, which motivates our design. We have not repeated this comparison for 3D MRI at a matched DiT budget, and the Limitations state this.

4. Can volumetric evaluation establish anatomical consistency?

We added 3D Fréchet radiomic distance (FRD), which compares volumetric radiomic feature distributions rather than independent slices. In healthy-region inpainting on 100 cases, hole-region FRD (log d²) is 5.32 for our model and 7.54 for Zhang25, the BraTS 2025 inpainting winner, while Zhang25 has higher paired PSNR and SSIM; we report both (Appendix J). To address surrounding tissue, we also report FRD outside the generated region for inpainting and lesion-region completion, as a check that full-volume decoding preserves the observed anatomy (Appendix J).

FRD measures regional volumetric statistics rather than whole-brain anatomical continuity, and we have not run a reader study; the Limitations note this.

Thank you for your response and clarifications. The more experimental results support the paper to some extent. However, I remain unconvinced by its novelty and contributions to the field. I still view this work primarily as an engineering improvement rather than a substantial methodological contribution. Nevertheless, I am willing to raise my score to 2.

Thank you for the follow-up. We would like to note that the revised manuscript shifts its focus: the multimodal DiT, a small part of the original submission, now performs synthesis, inpainting, deblurring, de-aliasing and motion correction in a single model (Sections 3.4 and 5.5, Appendix J). The contribution is therefore twofold: an efficient volumetric VAE, and its use with a multimodal DiT to address medical image processing tasks that are usually handled by separate models. We would be grateful if the reviewer could take the revised version into account.

Reviewer_jLgPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a 500M Transformer VAE that compresses 3D medical images by 8× per spatial axis with 16 latent channels, and evaluates a smaller 8-channel variant. It trains on a private 256K-volume brain-MRI corpus using reconstruction, perceptual, KL, and adversarial/feature-matching losses. Across public brain MRI, cross-domain MRI/CT, tumor segmentation, and diffusion-generation tests, the C16 model reports strong PSNR, competitive latency, preserved segmentation performance, and favorable slice-FID, while acknowledging training-data and evaluation limitations.

Critical strengths
  1. The proposed C16 model consistently achieves strong reconstruction performance across four public brain MRI benchmarks. Its paired PSNR improvements over HunyuanVideo are approximately 0.75–1.13 dB while using half as many latent spatial locations. The reported encoding–decoding latency and capacity-matched memory measurements further indicate useful computational properties under the evaluated implementations and settings.
  2. The cross-domain experiments, patient-grouped bootstrap analysis, evaluation-coverage reporting, and tumor segmentation preservation study strengthen the empirical contribution. In particular, assessing reconstructed images with a fixed downstream segmenter provides useful evidence about whether the compressed representation retains task-relevant information.
Critical weaknesses
  1. The methodological contribution is insufficiently distinguished from established designs and confounded with training scale. The architecture primarily combines established components: volumetric patch projection, Transformer blocks, rotary position encoding, a Gaussian bottleneck, and standard reconstruction, perceptual, and adversarial objectives. The paper therefore needs stronger evidence identifying which design choices account for its improvements.
  2. The downstream generative benefit is not established through a quality-matched comparison.The generation comparisons also use models trained on different distributions. Finally, the mask-adherence analysis combines four independently generated sequences in a segmenter designed for co-registered inputs. Its lower precision and volume overestimation cannot be attributed specifically to anatomical inconsistency without additional controls.
  3. The related-work discussion also omits “Foundation VAEs for 3D CT Reconstruction, Augmentation, and Generation”, which studies frozen pretrained VAEs as interfaces for medical reconstruction and controllable generation.
Questions for authors

Can you provide at least one reconstruction comparison trained on the same data subset, with matched latent interface and objectives? Which benefits remain after controlling for data scale and model capacity?

1 comment

Thank you for the careful review. Your point that the original comparisons mixed architecture with data and capacity was the right one, and the new same-data control changed how we understand our own result. The paper is now built around it.

1. Contribution and a controlled reconstruction comparison

Concern: the architecture uses established components, and the comparisons do not separate architecture from data scale and capacity.

We agree that the building blocks are established; we do not claim a new Transformer block. The contribution is the measured reconstruction–cost tradeoff of a volumetric autoencoder whose Transformer blocks operate on a grid downsampled eightfold along each spatial axis, together with its use in a shared conditional-generation framework.

To separate architecture from data, we trained a 249.2M Transformer VAE and a 246.5M CNN VAE from scratch on the same collection, with the same f8/C16 interface, 128³ crops and loss terms. The CNN uses the CONFLUX architecture; the released CONFLUX checkpoint stays a separate baseline. Batch sizes differ because the CNN needs far more activation memory: at 128³ and batch size one, its peak training memory is 47.3 GiB, compared with 13.4 GiB for the 250M Transformer (Appendix D).

ModelBraTSOASISIXIInternal
CNN VAE, 246.5M32.4630.0633.8232.18
Transformer VAE, 249.2M32.5531.1334.0832.95
Transformer VAE, 499.3M33.5332.1635.4633.80

At equal size the Transformer leads on all four sets, by 0.09 dB on BraTS to 1.07 dB on OASIS. The larger difference is cost: at 256³, encoding plus decoding takes 288.5 ms for the Transformer and 2195.7 ms for the CNN (7.6×).

  • Input extent. On the same 400 internal volumes, the two models are tied on 128³ crops (−0.005 dB, 95% CI [−0.057, 0.058]). Given the full volume and scored on the same central 128³ region, the Transformer leads by +0.749 dB [0.677, 0.838].
  • Capacity. The 499.3M Transformer improves all four brain sets and takes 574.0 ms, still 3.8× faster than the smaller CNN.

The central finding is therefore a favorable reconstruction–cost tradeoff: the Transformer supports full-volume processing at substantially lower measured latency, and increasing its decoder capacity improves reconstruction while retaining a latency advantage over the trained CNN (Section 5.2, Appendix E).

2. Generation comparisons across training distributions

Concern: the generation results do not isolate the benefit of the latent space, and generators were trained on different data.

Agreed. The generation experiments show that the VAE supports a capable conditional generator; they do not isolate the VAE's causal effect, and the paper no longer presents them that way. What we do measure directly is cost: at a fixed patch size, f8/C16 makes a 2B DiT step 1.4× faster than (4,8,8)/C16 and 16× faster than f4/C4. The lack of a matched-token generation comparison in 3D is stated in the Limitations.

For cross-generator FID, Table 4 now compares models only within the same reference cohort (internal or BraTS), since generators were trained on different data.

The revision also extends the generator to image-conditioned tasks: inpainting, lesion-region completion, deblurring, de-aliasing and motion correction (Figure 5, Appendix J). Because encoding is fast, these conditions are built online during training, which shows another way the representation's efficiency is used.

3. Interpretation of mask adherence

Concern: independently generated sequences may violate the segmenter's co-registration assumption.

You are right. A shared lesion mask provides a common spatial condition, but does not guarantee identical lesion boundaries or surrounding anatomy across independently generated sequences. NS-HGlio, however, expects co-registered inputs. We no longer attribute the adherence errors to any single cause. The paper reports the result plainly (Med-DDPM, which also receives a brain-region label, has higher Dice and precision; ours has higher recall) and notes the co-registration issue in Appendix J and the Limitations.

4. Foundation VAEs for 3D CT

Thank you for the pointer. We now discuss Foundation VAEs for 3D CT Reconstruction, Augmentation, and Generation in Related Work. It reuses frozen natural-image and video VAEs for CT, whereas we train a medical autoencoder and study its fidelity–compute trade-off; the two approaches are complementary.

Comments on the paper

1 comment

We thank the reviewers for their feedback. The main changes in the revision are as follows.

  1. Controlled reconstruction comparison. We trained a 249.2M Transformer VAE and a 246.5M CNN VAE from scratch on the same collection and latent interface, and report fidelity, latency, FLOPs, training-step cost and memory (Sections 5.1–5.2; Appendices D and E).
  2. Evaluation data. The BraTS evaluation now uses BraTS-GLI subjects sampled after excluding cases that may overlap with our training collection, and MSD Brain has been removed (Section 4).
  3. Image-conditioned generation. The original submission presented the DiT mainly for sequence- and mask-controlled synthesis. The revision applies the same generator to inpainting, lesion-region completion, deblurring, de-aliasing and motion correction, and gives this part more space (Sections 3.4 and 5.5; Appendix J). We extended this part because it is where encoder efficiency enters generative training: conditions are constructed in image space and encoded online, so a single checkpoint learns all tasks without precomputed latents. This depends on fast encoding; at 256³, our encoder takes 56.7 ms, compared with 690.7 ms for the same-data CNN. The abstract, introduction and discussion now describe the generator as one conditional model for several medical imaging tasks; the reconstruction experiments and their conclusions are unaffected by this change.
  4. Generation evaluation. We added 3D Fréchet radiomic distance, restricted cross-generator FID comparisons to matched reference cohorts, and added mask-adherence and DiT resource benchmarks (Appendices I and J).
  5. Scope. The Limitations now state that we do not compare latent spaces at a matched DiT token budget for generation, that mask adherence relies on a segmenter expecting co-registered inputs, and that restoration uses simulated degradations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.