acceptodds
Under review as a conference paper at ICLR 2027

Contrast-X: A Multi-Modal Contrast Image Synthesis Benchmark and Universal Modality Flow Matching

Abstract

Contrast-enhanced imaging is central to oncologic diagnosis, but contrast agents can be contraindicated for many of the patients who need them most. Synthesizing contrast scans from non-contrast inputs is the natural response. Two obstacles stand in the way: no benchmark provides paired contrast data with lesion-level evaluation, and no single model handles the arbitrary missing patterns seen in practice. We introduce Contrast-X, a benchmark of paired contrast-enhanced and non-contrast imaging spanning 10 organs in CT (1526 patients) and multi-phase breast DCE-MRI (1116 patients). Every case carries radiologist-verified phase labels and tumor masks. We further propose FlowMI, a single model that handles arbitrary subsets of available modalities through a unified multi-modal latent space and flow matching. We benchmark a range of missing-modality configurations, reporting standard image-quality metrics, radiologist reader studies, and downstream lesion analysis on the synthesized scans. We further evaluate cross-organ generalization to test whether the model has learned a transferable contrast-enhancement operation. Dataset, code, and leaderboard will be released.

Primary area:
datasets and benchmarks
Keywords:
Paired, Multi-Modal, Pan-Cancer, Medical Image Synthesis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YNWGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Contrast-X, a curated benchmark for paired CT contrast synthesis and breast DCE phase imputation, with phase annotations and tumor masks. It also proposes FlowMI, which encodes observed modalities through a product-of-experts autoencoder and transports the partial-observation latent toward a full-observation latent using mask-conditioned flow matching. A trajectory reconstruction loss links interpolated latents to interpolated images. Evaluation includes image quality, downstream segmentation, held-out organs, and a reader study.

Critical strengths
  1. A common latent endpoint gives missing-modality learning a coherent target. Each observation subset represents partial evidence about the same patient, so transporting its latent toward the complete-data latent is a sensible way to share parameters across configurations. The reported shared-model results and trajectory-loss ablation support investigating this formulation beyond separate source-target models.
  2. Lesion-aware benchmarking addresses a meaningful evaluation gap. Global pixel similarity alone cannot determine whether a synthesized scan preserves the anatomy needed for analysis. Tumor masks, downstream segmentation, and the reader study provide complementary checks, making the resource potentially useful even independently of FlowMI.
Critical weaknesses
  1. The trajectory constraint does not establish the claimed protection against flow drift. Equation (5) trains the decoder on straight lines between encoded endpoints. An integrated learned vector field need not remain on those lines, so accurate decoding of the prescribed interpolation does not ensure reliable decoding of off-path states. Moreover, an absent modality must be embedded into a full image tensor before the image-space interpolation is defined; the paper does not specify that representation. If missing channels are filled with zero, their interpolation is an intensity ramp rather than a demonstrated physiological enhancement trajectory. The ablation supports the loss as a useful regularizer, but the stronger geometric explanation remains unestablished. Clarifying the construction and measuring decoding errors along actual inference paths would directly improve this argument.
  2. Patient-specific information recovery is not separated sufficiently from plausible image generation. Enhancement can contain information not uniquely determined by the available scan. Deterministic inference from the posterior mean can produce a useful estimate, but cannot by itself establish that a particular lesion's enhancement is recovered faithfully. Realism and segmentation overlap only partially address this issue. The available-input boundary is also unclear: preprocessing uses organ-specific cropping and registration, while the split description does not explicitly group all studies from a patient. These details matter because target-derived preprocessing or overlapping patients could make the apparent prediction problem easier. The CT counts in Table 1 additionally total 1,506 rather than 1,526, so the evaluation cohort needs reconciliation.
Questions for authors
  1. Clarification: How is a missing-modality observation represented in the full image tensor in Eq. (5), and what is the complete autoencoder loss? Does any training term constrain decoded states along the learned inference trajectory, beyond the straight endpoint interpolation?
  2. Evidence affecting the recommendation: Does the trajectory loss improve lesion-region reconstruction along actual inference paths, or primarily improve endpoint image metrics? A comparison of intermediate decoding and endpoint enhancement errors with and without this loss would distinguish these explanations; the current ablation is already a useful starting point.
  3. Clarification: Can registration and input cropping be performed using only modalities available at inference, and are all examinations of a patient kept in one partition? Please reconcile the Table 1 cohort totals and identify the timing and pre-/post-contrast status of DCE1-3.
  4. Evidence affecting the recommendation: What existing lesion-level results show preservation of enhancement magnitude or phase ordering, and what failures occur when that information is weak in the input? This would materially strengthen the fidelity claim beyond realism and segmentation overlap. It need not establish equivalence to clinical contrast administration.
2 comments
AuthorsOpenReview ↗

Response to Reviewer YNWG(1)

Thank you for your careful review.

Critical Weakness 1

In Eq. (5), missing channels in the image tensor x_0 are zero-filled. Consequently, observed channels remain unchanged in x_t, whereas missing channels equal t times their corresponding complete-image channels. This intensity ramp is a computational training target for the decoder, not a model of physiological enhancement over time. Missing modalities are excluded from PoE fusion in the encoder.

To distinguish the benefit of interpolation-based supervision from that of an additional reconstruction term, we added an endpoint-target control: the decoder receives the same interpolated latent z_t, but its target is the complete image x_1 instead of x_t. Average PSNR across the four tasks is 25.21 dB without the trajectory loss, 26.22 dB with the endpoint target, and 26.70 dB with the interpolated target. Thus, the interpolated target provides an additional 0.48 dB over the endpoint-target control, with improvements on all four tasks.

These results support the usefulness of the proposed regularization, but do not establish a guarantee against drift. We replaced “has no opportunity to drift” with a more precise statement: the loss provides reconstruction supervision along the prescribed latent interpolation and improves endpoint reconstruction in our experiments.

Critical Weakness 2

FlowMI produces an estimate conditioned on the available observations; deterministic inference does not establish unique recovery of information that those observations do not determine. We will qualify the fidelity claim and distinguish reconstruction accuracy, downstream segmentation performance, and visual realism from direct validation of enhancement magnitude or temporal behavior.

We also clarify the preprocessing and partitioning protocol. The full registered images were retained without additional cropping, and the manuscript’s description of organ-specific bounding-box cropping will be corrected. All examinations and phases belonging to the same patient were assigned to one partition, with no patient overlap between training, validation, and testing.

Registration used CTC and DCE2 as fixed references for CT and DCE, respectively. This defines an aligned-pair benchmark, and we acknowledge that target-referenced registration is not an input-only preprocessing pipeline for the evaluated synthesis tasks.

Finally, we rechecked the original Excel records and underlying data. The correct CT cohort count is 1,526; the discrepancy resulted from a transcription error in the pancreas entry of Table 1. This has been corrected and did not affect the experimental data or results.

Question 1

Missing modalities are represented by zero-filled channels in the image tensor x_0 used in Eq. (5). With m denoting the observation mask, the construction is:

x_0 = m ⊙ x_1

x_t = (1 − t)x_0 + tx_1

Observed channels therefore remain equal to their values in x_1, while missing channels equal t times their values in x_1. This construction is a decoder-training target, not a physiological enhancement trajectory. In the encoder, missing experts are excluded from PoE fusion.

The complete autoencoder objective is:

L_AE = L_rec + 0.1 × L_perc + 10⁻⁶ × L_KL + 0.1 × L_Trajectory

Here:

  • L_rec combines L1 and L2 reconstruction losses over all modalities.
  • L_perc is the perceptual loss, with weight 0.1.
  • L_KL regularizes the posterior toward the standard normal prior, with weight 10⁻⁶.
  • L_Trajectory is the trajectory reconstruction loss defined in Eq. (5), with weight 0.1.

Beyond the prescribed straight interpolation, no training term directly supervises decoded states along the integrated inference trajectory. The integrated learned field can deviate from the interpolation used during training.

Question 2

Thank you for distinguishing these explanations. We added an endpoint-target control and evaluated decoded states along actual inference paths.

The endpoint-target control retains reconstruction supervision on the same interpolated latent z_t but replaces x_t with x_1 as the target. The average results across the four tasks are:

VariantAverage PSNR (dB)Average SSIM (%)
Without trajectory loss25.2172.3
Endpoint target (x_1)26.2274.9
Interpolated target (x_t)26.7075.7

Thus, an additional reconstruction term explains a substantial part of the improvement, while the interpolated target provides a further benefit across all four tasks.

The reported actual-path measurements do not show uniformly higher lesion PSNR at every intermediate time: the full model has lower values at t = 0.25 and 0.5, and higher values at t = 0.75 and 1.0. We therefore interpret the current evidence primarily as supporting improved endpoint reconstruction, rather than uniformly improved intermediate decoding or guaranteed protection against drift.

AuthorsOpenReview ↗

Response to Reviewer YNWG(2)

Question 3

Registration used CTC as the fixed image for CT and DCE2 as the fixed image for DCE, followed by visual inspection. The full registered images were retained without additional cropping. Neither organ nor tumor masks were used to define input crop regions, and we will correct the manuscript’s description of organ-specific cropping.

Because CTC or DCE2 is unavailable in the corresponding synthesis tasks, the original target-referenced registration cannot be reproduced using only the available inputs. We will explicitly describe the experiments as evaluation on registered image pairs and distinguish this setting from an input-only deployment pipeline.

All examinations, acquisition dates, and phases belonging to the same patient were assigned to a single partition. There is no patient overlap between training, validation, and testing.

The correct CT cohort count is 1,526. Rechecking the original Excel records and underlying data identified a transcription error in the pancreas entry of Table 1. This has been corrected, and the experimental data and results were unaffected.

DCE1 is the pre-contrast acquisition, whereas DCE2 and DCE3 are post-contrast acquisitions.

Question 4

The reader study reported in Table 7 of the original supplementary material provides indirect support for the clinical plausibility of the synthesized enhancement. Radiologists’ assessments indicate that the generated images approximate the appearance of real clinical scans, providing expert-based evidence complementary to pixel-level reconstruction and segmentation metrics. We interpret this as indirect support for plausible enhancement appearance.

When the available inputs provide insufficient information about lesion enhancement, failure cases include insufficient enhancement within the lesion and poorly delineated lesion boundaries. These cases illustrate the limitations of estimating patient-specific enhancement from incomplete observations.

Reviewer_QuinOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Contrast-X, a benchmark for contrast-enhanced medical image synthesis, and FlowMI, a unified framework for missing-modality imputation.

Contrast-X integrates publicly available CT and breast DCE-MRI datasets, covering 11 organs and different contrast synthesis tasks. FlowMI combines modality-specific VAE encoders, Product-of-Experts fusion, and latent flow matching to transport representations of partially observed modalities toward a complete latent representation. An additional trajectory reconstruction loss is introduced to regularize intermediate latent states.

The authors evaluate the framework using image quality metrics, downstream tumor segmentation, reader studies, and cross-organ generalization experiments, reporting improvements over several existing image translation methods.

Critical strengths

1. An interesting and potentially valuable benchmark direction.

A standardized benchmark for contrast-enhanced image synthesis across different organs would be valuable to the community. The effort to consolidate existing datasets and provide tumor annotations is meaningful, particularly given the fragmented evaluation protocols currently used in this field.

2. A practical unified framework for missing-modality configurations.

The attempt to handle multiple missing-modality configurations using a shared model is reasonable and potentially useful. The experiments also cover several evaluation perspectives, including reconstruction quality, segmentation, and generalization.

Critical weaknesses

1. Insufficient understanding and consideration of the clinical characteristics of contrast-enhanced medical imaging.

This is my primary concern and the main reason for my negative assessment.

The paper largely treats contrast enhancement as a general image translation or missing-modality reconstruction problem. However, contrast-enhanced medical imaging is fundamentally different from conventional image-to-image translation.

First, contrast enhancement depends on specific physiological processes, acquisition protocols, and contrast phases. For CT, arterial, portal venous, and delayed phases contain different diagnostic information and should not be treated as interchangeable contrast-enhanced targets. Nevertheless, the manuscript does not adequately explain how these phases are identified and standardized across the different source datasets.

A similar concern applies to breast DCE-MRI. The manuscript does not clearly establish whether DCE1 represents a genuine pre-contrast acquisition or an already enhanced phase. This distinction is essential because predicting a later enhanced phase from an earlier enhanced phase is fundamentally different from synthesizing contrast enhancement from a non-contrast scan. Furthermore, the use of per-scan z-score normalization raises questions about how the relative enhancement information across DCE phases is preserved.

Second, and more fundamentally, patient-specific contrast enhancement is not necessarily recoverable from non-contrast anatomy alone. Enhancement patterns reflect vascularity, perfusion, tissue characteristics, and pathological processes that may not be directly observable in the input. The proposed framework does not explicitly address this fundamental uncertainty or explain how clinically meaningful patient-specific enhancement can be recovered.

Despite this, the experiments rely predominantly on generic image reconstruction metrics and downstream segmentation Dice. These metrics do not demonstrate whether the synthesized enhancement intensity or lesion-specific enhancement patterns are clinically accurate. A visually plausible image or a high segmentation Dice score does not necessarily indicate faithful contrast enhancement.

In my opinion, these are not minor experimental omissions. They concern the fundamental definition and validity of the proposed medical imaging task. Without addressing them, the reported quantitative improvements are difficult to interpret in a clinically meaningful way.

2. The methodological and experimental contributions are insufficient to support the paper's ambitious claims.

Although the paper proposes a universal modality flow matching framework, the core methodology is largely built upon established techniques, including multimodal PoE-VAE representations and latent flow matching. Transporting a partially observed latent representation toward a target representation is not fundamentally new. The trajectory reconstruction constraint provides an additional training objective, but its conceptual novelty appears limited.

Moreover, the claimed universality is not convincingly established. The manuscript demonstrates shared modeling across several breast DCE-MRI configurations, but it remains unclear whether CT and MRI are genuinely modeled using the same parameters. The improvement over ConcatFM on the DCE-MRI tasks is also relatively modest.

The benchmark itself raises further concerns. The overall dataset size is limited relative to the claimed multi-organ generalization scope, and several individual organs contain very few cases. There are also inconsistencies between some reported dataset sizes and the original public collections, as well as an internal discrepancy in the CT cohort total. These issues require clarification, particularly regarding unique patient counts and patient-level data splitting.

Overall, the paper attempts to simultaneously establish a comprehensive medical imaging benchmark, a universal generation framework, and clinically meaningful contrast synthesis. However, the current methodological development and experimental evidence are not sufficient to support these broad contributions.

Questions for authors

1. Clinical definition of contrast synthesis.

How were the CT contrast phases identified and standardized across the different datasets? Were the non-contrast and contrast-enhanced images acquired within the same examination? For breast MRI, does DCE1 represent the true pre-contrast phase? How is contrast enhancement information preserved under the adopted normalization strategy?

2. Patient-specific enhancement.

How does FlowMI account for enhancement information that is not observable in the non-contrast input? Can the authors provide quantitative evaluation of lesion-specific enhancement intensity or enhancement patterns, rather than relying mainly on PSNR, SSIM, and segmentation Dice?

3. Methodological novelty and universality.

What is the fundamental methodological difference between FlowMI and existing latent flow matching or bridge-based image translation methods? Are CT and MRI trained using the same model parameters? How does the proposed framework compare against a direct PoE-VAE missing-modality reconstruction baseline without flow matching?

4. Dataset reliability.

Could the authors clarify the number of unique patients versus imaging samples in the benchmark, explain the discrepancies with the original public datasets, and confirm that all training and testing splits are strictly patient-independent?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Quin(1)

Thank you for your careful review. We address your comments below.

Critical Weakness 1

Clinical acquisition characteristics are incorporated into the dataset construction through metadata-based pairing, expert review of contrast-state and phase labels, exclusion of ambiguous cases, and registration quality control.

Supplementary Section B.1 describes the CT target-phase selection protocol, and Table 15 provides the phase distributions by organ and dataset. Each case retains one post-contrast target according to a consistent selection rule. The supplementary dataset taxonomy also documents the breast MRI acquisition protocols as pre-, early post-, and late post-contrast DCE-MRI.

The evaluation includes image reconstruction, downstream tumor segmentation, and reader assessment. The revised supplementary material additionally reports patient-level 3D lesion-ROI reconstruction metrics, directly examining performance within annotated tumor regions.

Critical Weakness 2

FlowMI’s contribution lies in the formulation and training scheme for unified missing-modality imputation.

The same multimodal autoencoder encodes partial and complete observations within a shared latent space. Different observed subsets define different starting representations with a common full-observation target. PoE fusion accommodates these subsets within a fixed-dimensional representation, while mask conditioning enables one velocity field to handle different missing-modality configurations.

The trajectory reconstruction objective supervises decoded latent interpolations using corresponding image-space interpolations. The supplementary ablations examine the contributions of trajectory reconstruction, mask conditioning, and PoE fusion.

One VAE and one flow module jointly support all three evaluated DCE tasks. This shared-model setting is relevant when interpreting the comparison with ConcatFM, whose reported results use separately trained models for each configuration. FlowMI combines improved reconstruction performance with shared modeling across the evaluated configurations.

The benchmark contribution includes paired imaging data, tumor annotations, and a common evaluation framework across multiple organs. The leave-one-organ-out experiments further evaluate transfer to the specific held-out organs reported in the manuscript.

Question 1

The manuscript describes initial pairing using SeriesDescription, ContrastBolusAgent, and AcquisitionTime, followed by manual inspection and expert confirmation.

Supplementary Section B.1 specifies the CT target-phase selection rule. Each case retains one post-contrast target. When multiple phases are available, the portal venous phase is prioritized, followed by the arterial and delayed phases. Cases with only one available post-contrast phase retain that phase.

Table 15 reports the retained phases by organ and dataset. Overall, the targets comprise 77.3% portal venous, 16.4% arterial, and 6.3% delayed acquisitions. These statistics explicitly document the phase composition of the benchmark.

For breast MRI, the supplementary dataset taxonomy lists the acquisition protocols for TCGA-BRCA, UCSF, and I-SPY1 as “pre-, early post-, late post-contrast DCE-MRI.”

Question 2

FlowMI learns predictive relationships from paired training observations. At inference, the available modalities are encoded into a partial-observation latent representation, which the learned flow transports toward a complete-observation representation for decoding.

Our evaluation extends beyond whole-image reconstruction metrics and segmentation Dice. Supplementary Section A.4, “Downstream Clinical Analysis.” and Table 13 report downstream clinical experiments, including tumor grading and overall survival prediction evaluated by AUC. The downstream models are trained on real contrast-enhanced images and applied to synthesized images, assessing whether the generated images retain information useful for these clinical tasks.

The original supplementary material also includes a reader study in Section A.1, “Reader Study.” It evaluates image authenticity, perceived clinical realism, and paired preferences through randomized presentation with method identities concealed.

In addition, the revised supplementary material provides patient-level 3D lesion-ROI evaluation. ROI PSNR is computed within the annotated tumor masks, and ROI SSIM is obtained by averaging a local 3D SSIM map at voxel centers within those masks. FlowMI achieves the highest ROI PSNR and SSIM among the methods included in these comparisons across all four evaluated synthesis tasks.

Together, these experiments assess reconstruction within lesions, the utility of synthesized images for downstream clinical prediction, and reader-perceived image quality.

AuthorsOpenReview ↗

Response to Reviewer Quin(2)

Question 3

FlowMI formulates missing-modality imputation as transport between partial- and complete-observation representations produced by the same multimodal autoencoder.

PoE fusion produces a fixed-dimensional latent representation for different observed subsets. The observation mask conditions the velocity field, enabling a shared flow module to handle different missing configurations. The trajectory reconstruction objective additionally supervises decoding along the prescribed latent interpolation.

The demonstrated parameter-sharing result is that one VAE and one flow module jointly support the three evaluated breast DCE configurations.

The supplementary ablations separately examine trajectory reconstruction, mask conditioning, and PoE fusion. Removing the trajectory reconstruction objective produces the largest reported decrease in average PSNR, while mask conditioning and PoE fusion are particularly relevant to the multi-modality DCE configurations.

Question 4

A single patient may contribute one or more imaging samples because some patients underwent imaging examinations at multiple time points. Repeated acquisitions from the same patient therefore contribute additional imaging samples, but do not increase the unique-patient count.

Our benchmark comprises curated subsets of the original public collections. Some cases in those collections lack either the non-contrast or the contrast-enhanced acquisition and therefore cannot form the complete pairs required by our benchmark. These cases are excluded, which explains why the retained cohorts differ in size from the original collections.

All data splits are strictly patient-independent. Every imaging sample from the same patient, including acquisitions at different time points, is assigned to the same split. No patient appears in both the training and test sets.

Reviewer_ART3OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on medical image synthesis and generation methodologies. ContrastX, targets differentiating the generation of contrast-enhanced CT in different phases. A collection of 2642 paired cases 1526 patients 10 organs, containing non contrasst CT, to enhanced CTs. Also MR images, with 1116 breast DCE MRI patient scans. The method contains FlowMI, a single model for arbitrary missing modality configurations. Modality-specific encoders that produce Gaussian experts. Diverse datasets are used for evaluation and validation, compared with GANs, diffusion, flow matching, etc.

Critical strengths
  1. This is a comprehensive paper that studies medical image generation; the benchmarks, datasets, and methods used and evaluated are convincing and promising.
  2. The experimental designs are technically sound and solid, with clinical relevance. Condition controls are validated, such as CT contrasts.
  3. FlowMi's unified formulation is well motivated, as the technical community for image generation is moving towards unification.
Critical weaknesses
  1. To provide more convincing proof of effectiveness of the paper, the evaluation and baseline setup can be more clear, like settings, fairness.
  2. The authors can highlight more on the methodological innovations that differentiate from existing methods. This is to highlight the novelty in the image synthesis literature.
Questions for authors
  1. In medical image settings, image generation shall separate 2D and 3D. The implementation and baseline studies can be separate. How do the 2D and 3D designs differ?
  2. How much scales and computes run impact the performance.
  3. How many training runs are behind each study? Like how much training impacts the marginal gains for FlowMI.
  4. More ablation studies are needed to prove the trajectory loss, such as linear image-space interpolation.
2 comments
AuthorsOpenReview ↗

Response to Reviewer ART3(1)

Thank you for your careful review .

Critical Weakness 1

All compared methods use the same patient-level data splits, preprocessing, input resolution, and evaluation pipeline. We use the publicly available baseline implementations and follow their original training protocols, training them to convergence.

The 200-epoch schedule applies specifically to FlowMI: 200 epochs for the autoencoder and 200 epochs for the latent flow model. It was not imposed on the baselines. We will correct the wording “all models are trained for 200 epochs” in Section 4.1.

We retain each baseline’s original 2D or 3D implementation. For reconstruction evaluation, slice-level predictions are assembled into patient volumes, and both 2D and 3D methods use the same volumetric evaluation pipeline. The comparison therefore shares data and evaluation conditions while retaining method-specific training configurations. We will clarify these details in Section 4.1.

Critical Weakness 2

FlowMI builds on established PoE and flow-matching techniques. Its contribution lies in the formulation and training scheme for unified missing-modality imputation.

First, the same multimodal autoencoder encodes a patient under partial and complete observation masks. Different observed subsets therefore define different starting points with a common full-observation latent target. Second, PoE fusion accommodates different input subsets within a fixed-dimensional latent space, while mask conditioning enables one velocity field to handle different missing-modality configurations. One VAE and flow module jointly support all three evaluated DCE tasks.

Third, the trajectory loss supervises decoded latent interpolations using corresponding image-space interpolations. The endpoint-target ablation reported below separates the benefit of this interpolation target from that of adding reconstruction supervision alone: the interpolation target improves average PSNR by a further 0.48 dB.

Question 1

FlowMI is a 3D model: its autoencoder and flow network process volumetric inputs. We retain each baseline’s original dimensionality:

DimensionalityMethods
2DHiNet, ResViT, I2IMamba, CycleGAN, Pix2Pix, Palette, SelfRDB, PMRF
3DConcatFM, DirectFM, FlowMI

The 2D methods generate slice-level predictions, which are stacked in their original slice order to reconstruct patient volumes. The 3D methods process volumes or volumetric patches and can exploit spatial context across slices. Both are evaluated using the same volumetric reconstruction metrics.

We use 3D modeling to capture the cross-slice context of vessels, organs, and lesions. We additionally evaluated a 2D implementation of FlowMI:

MethodDimensionalityCT → CTCDCE1 → DCE2DCE1 → DCE2,3DCE1,3 → DCE2
FlowMI2D24.62 / 77.126.38 / 73.326.69 / 73.529.04 / 76.0
FlowMI3D24.47 / 78.526.52 / 74.226.63 / 73.729.17 / 76.2

Values are PSNR (dB) / SSIM (%). The two implementations achieve similar PSNR, while the 3D implementation has higher SSIM on all four tasks. We plan to release both implementations.

We will label model dimensionality and clarify volumetric evaluation in Section 4.1, and add the 2D-versus-3D comparison to the appendix’s “Ablation Study.”

Question 2

Our standard protocol uses the 70% training partition of the CT and DCE-MR cohorts, with 200 epochs for each of FlowMI’s two training stages and an effective batch size of 8. We conducted additional analyses of model size, autoencoder batch size, and inference steps:

FactorSettings comparedObserved effect
Flow model size148M → 330M parametersLesion-region R² changes by at most 0.02; time per epoch increases to 1.83×
Autoencoder batch size, CT4 / 8 / 16Autoencoder reconstruction PSNR: 28.21 / 27.74 / 27.40 dB
ODE steps, DCE1 → DCE25 / 20 / 50 / 100Lesion-region PSNR gain over copying the input: +1.67 / +1.83 / +1.80 / +1.78 dB
ODE steps, CT → CTC5 / 20 / 50 / 100Lesion-region PSNR gain over copying the input: +1.71 / +1.76 / +1.83 / +1.73 dB

Within these tested settings, increasing model size substantially increases training time without a correspondingly large change in lesion-region R². Larger autoencoder batches do not improve reconstruction PSNR. The highest observed lesion-region PSNR gains occur at 20 or 50 ODE steps; increasing to 100 steps does not improve either task.

These are targeted sensitivity analyses within FlowMI, rather than a general scaling law or a compute-matched comparison against every baseline. We will add these results and their respective metric definitions to “Additional Experiments” in the appendix and clarify the standard configuration in Section 4.1.

AuthorsOpenReview ↗

Response to Reviewer ART3(2)

Question 3

The results in Tables 3, 4, and 8 are based on one primary training run for each evaluated model or variant, rather than averages over multiple independent seeds. For FlowMI, the three DCE task results come from the same jointly trained model; they do not represent three independent training runs.

We additionally retrained the flow stage with a different random seed as a preliminary robustness check. This is not a full multi-seed study of the complete two-stage pipeline. Repeated stochastic evaluations of the same model varied by less than 0.08 dB, which measures inference variability rather than training-seed variability.

The observed PSNR margin over ConcatFM is 1.37 dB on CT and 0.11–0.38 dB on the DCE tasks. The smaller DCE margins require a fuller multi-seed comparison to establish their stability. A separate practical advantage is that FlowMI covers all three DCE tasks with one shared model, whereas the compared ConcatFM models are trained separately.

Extending flow training to 400 epochs did not improve the best validation performance in the tested runs: the best validation checkpoints occurred at epoch 134 for CT and epoch 189 for DCE. This supports the adequacy of the 200-epoch schedule in these experiments.

Question 4

FlowMI already uses linear image-space interpolation as the target for decoding intermediate latent states:

zt=(1−t)z0+tz1,xt=(1−t)x0+tx1.z_t = (1-t)z_0 + tz_1,\qquad x_t = (1-t)x_0 + tx_1.

Here, x0x_0 contains the observed modalities and zero-filled missing channels, while x1x_1 is the complete modality tensor. This image-space interpolation is an auxiliary decoder-training target, not a physiological enhancement trajectory.

The original removal ablation supports the usefulness of the loss, but does not distinguish the interpolation target’s contribution from that of additional reconstruction supervision. We therefore evaluated an endpoint-target variant that supervises the same intermediate latent states using x1x_1 instead of xtx_t:

VariantTarget for decoded intermediate statesCT → CTCDCE1 → DCE2DCE1 → DCE2,3DCE1,3 → DCE2Average
Without trajectory lossNone22.81 / 75.224.97 / 70.425.18 / 70.127.89 / 73.425.21 / 72.3
Endpoint targetx1x_123.95 / 77.426.12 / 73.626.03 / 73.228.78 / 75.426.22 / 74.9
Linear image-space interpolationxtx_t24.47 / 78.526.52 / 74.226.63 / 73.729.17 / 76.226.70 / 75.7

Values are PSNR (dB) / SSIM (%); averages are unweighted across the four tasks.

The endpoint target recovers approximately two thirds of the average PSNR improvement over removing the loss. The linear interpolation target provides a further 0.48 dB PSNR and approximately 0.8 SSIM percentage points on average, with improvements on all four tasks.

These results support the benefit of the interpolation target beyond additional reconstruction supervision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.