acceptodds
Under review as a conference paper at ICLR 2027

MoDeFold: Modal-Degradation Joint Unfolding for All-in-One Medical Image Restoration

Abstract

All-in-one medical image restoration (MedIR) seeks a single model for heterogeneous, modality-coupled degradations, including k-space undersampling, low-dose noise, and limited-photon degradation. Existing Deep Unfolding Networks (DUNs) do not fully account for the interaction between modality-dependent content and heterogeneous degradations within a shared restoration pipeline. Across modalities, differences in tissue contrast and appearance are further mixed with corruption, making reliable content representations difficult to obtain. We propose Modal-Degradation unFolding (MoDeFold), a unified unfolding framework that reconciles divergent modality and degradation distributions for all-in-one MedIR. MoDeFold models modality-dependent content and modal-degradation embedding as complementary priors for a shared unfolding process. Our core insight is to use the modal-degradation prior to enhance modality-specific content features, then combine the enhanced features with the modal-degradation prior to jointly control the unfolding process. A multi-scale vector-quantized autoencoder supplies content features supervised by clean-code references, a Modal-Degradation Embedder trained with contrastive learning provides the joint prior, and embedding-conditioned modulation and fusion connect the two. Experiments demonstrate that MoDeFold outperforms baselines in all-in-one restoration, achieving an improvement of 1.126 dB in average PSNR, with consistent gains under single-task setting.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
medical image restoration, deep unfolding, contrastive learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_8ipHOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MoDeFold, an all-in-one medical image restoration framework that jointly models modality-specific content and degradation information. It uses a contrastively trained modal-degradation embedder and a multi-scale VQVAE with clean-code alignment to guide a shared unfolding network. Experiments on MRI, CT, and PET restoration tasks show competitive results, with ablation studies supporting the proposed design.

Critical strengths
  1. Reasonable motivation and design.

The idea of jointly considering anatomical content and degradation is well motivated for medical image restoration. Using the degradation embedding to modulate content features before unfolding is an interesting design choice.

  1. Good experimental results.

MoDeFold performs well across multiple benchmarks. On the main benchmark, it improves average PSNR by about 1.13 dB over R2R. The ablation studies are also fairly complete and show that the proposed components contribute to the final performance.

Critical weaknesses

Critical Weaknesses

  1. Limited evaluation of generalization.

The main benchmark relies on synthetic degradations, and each dataset uses a relatively fixed degradation setting. It is unclear how well the method handles unseen degradation levels or different acquisition conditions. Also, the improvement on the AMIR benchmark is quite small (0.05 dB in average PSNR over TAT), making the generalization claims less convincing.

  1. The contribution of the proposed design is not fully isolated.

MoDeFold has 49M parameters, noticeably more than several baselines, and also uses additional pretraining data and stages. Although the ablation results are useful, it is still difficult to tell how much of the gain comes from the proposed joint conditioning rather than increased capacity or pretraining. A more controlled comparison would make the contribution clearer.

Questions for authors
  1. Modality–degradation representation. In the main benchmark, degradation types are largely tied to specific datasets. How can we tell whether the learned embedding captures modality–degradation relationships rather than simply dataset identity? Would it generalize to unseen modality–degradation combinations?

  2. Necessity of unfolding. Since the learned back-projection (B_k) is not constrained to be the adjoint of (\Phi_k), the updates are not true gradient steps of Eq. (5). What is the actual benefit of the unfolding formulation over a standard restoration network with the same conditioning mechanism?

  3. Role of vector quantization. The unfolding network uses continuous content features (c) rather than quantized features (Q(c)). Is the VQVAE codebook actually necessary, or could direct alignment with clean continuous features provide similar benefits?

9 comments
AuthorsOpenReview ↗

W1: Generalization to degradation and acquisition conditions

Response

We thank the reviewer for the thoughtful comments. The AMIR benchmark evaluates MRI super resolution, CT denoising, and PET synthesis, with one restoration task associated with each modality. We therefore also evaluate on a four dataset benchmark comprising cardiac and brain MRI from ACDC and BraTS2021 and thoracic and abdominal CT from COVID19CTscans and HCC TACE Seg. The two CT datasets have different Poisson Gaussian noise settings, and HCC TACE Seg additionally contains directional streaks. This setting evaluates a shared model across distinct anatomy and more than one CT corruption condition. MoDeFold improves average PSNR over R2R by 1.126 dB on this benchmark. On AMIR, it achieves 34.44 dB average PSNR compared with 34.39 dB for TAT, while also improving average SSIM from 0.9358 to 0.9373 and reducing average RMSE from 12.45 to 11.47.

To improve the breadth of prior learning, we pretrain both the Modal Degradation Embedder and content VQVAE on eight datasets covering Gaussian noise, Poisson Gaussian noise, Poisson Gaussian noise with streaks, and low resolution. The final unfolding model is trained on the original four datasets. The embedder learns modality and degradation relations through two contrastive objectives, while clean code alignment encourages the content features to retain structures shared with clean images. These complementary priors provide image dependent guidance when the unfolding model encounters different anatomy and corruption patterns. Consistent with their contribution, removing contrastive learning lowers average PSNR from 34.8163 to 34.1298 dB, and removing clean code alignment lowers it to 34.0982 dB in Table 4.

The eight dataset prior pretraining and four dataset unfolding training are splited all based on the IMIS-Bench partitioning for training or testing, so there’s no data leakage.

AuthorsOpenReview ↗

W2: Question about model size and pretraining

Response

We thank the reviewer for the question about model size and pretraining. The increase to 49M parameters mainly comes from processing the low and high frequency components with separately parameterized unfolding branches after the Haar transform. The transform itself adds no trainable parameters. We evaluated a variant without frequency splitting that retains the proposed prior conditioning and uses 28M parameters. Table 1 compares it with the full model and R2R under the same four dataset benchmark.

Table 1. Model size and restoration results. Each result is PSNR in dB followed by SSIM.

MethodParamsACDCCOVID19CTscansHCC TACE SegBraTS2021Average
R2R30M26.2389 / 0.700834.8370 / 0.949943.1277 / 0.967330.5578 / 0.905833.6904 / 0.8810
MoDeFold without frequency split28M27.3298 / 0.780235.7321 / 0.969843.9812 / 0.981031.0012 / 0.908934.5111 / 0.9100
MoDeFold full49M27.6824 / 0.783136.0758 / 0.970044.3118 / 0.982931.1950 / 0.911234.8163 / 0.9118

The 28M variant exceeds the 30M R2R by 0.8207 dB in average PSNR, while the full frequency split adds 0.3052 dB. We will add the new variant and parameter counts to the paper's ablation table.

The ablations further distinguish model size from the proposed guidance. With the unfolding backbone and frequency branches held fixed, removing both priors reduces average PSNR from 34.8163 to 31.0908 dB. Retaining the joint token only at the final fusion stage, without using it to modulate content features, yields 32.3316 dB. Removing contrastive learning or clean code alignment yields 34.1298 and 34.0982 dB, respectively. These comparisons show the contribution of prior learning and content modulation within the same architecture. The additional four datasets are used only for prior pretraining, while end to end unfolding training uses the four datasets in the quantitative benchmark.

AuthorsOpenReview ↗

Q1: Whether the embedding captures modality and degradation information beyond dataset identity

Response

We thank the reviewer for this question. The two contrastive losses in Eq. 7 use the same embedding but different positive sets. For an anchor ii, Pm(i)={j≠i∣mj=mi}P_m(i)=\{j\ne i\mid m_j=m_i\} collects images of the same imaging modality, and Pd(i)={j≠i∣dj=di}P_d(i)=\{j\ne i\mid d_j=d_i\} collects images with the same degradation. These sets are defined by modality and degradation labels, not by dataset identity. Images from different datasets are therefore pulled together when they share the relevant label. When a batch contains a pair from different modalities with the same degradation, that pair is positive for the degradation loss and negative for the modality loss. A representation that merely assigns a separate cluster to each dataset would not satisfy both relations for such cross dataset pairs.

As specified in Appendices A and D, we train the embedder on eight datasets with four degradation categories, rather than using the four dataset restoration benchmark alone. The losses do not create a separate class or token for each observed modality and degradation combination. This allows the shared token to encode the two label relations for combinations outside the benchmark.

Reviewer_dvBhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents MoDeFold for all-in-one medical image restoration. It learns content features through a multi-scale VQVAE with clean-code alignment and a modality–degradation embedding through dual contrastive learning. The embedding modulates the content features before both are fused into a global condition for a PGD-inspired unfolding network. The authors evaluate the method on a four-dataset MRI/CT benchmark, with the priors pretrained on an eight-dataset collection, and on the AMIR benchmark, using both joint and task-specific training. They report improvements in average PSNR and SSIM and examine the proposed components through ablations and conditioning sensitivity analysis.

Critical strengths
  1. I find the use of the modality–degradation embedding to modulate content features before fusion interesting. It gives the two priors an explicit interaction within the unfolding framework and offers a concrete design for studying how content and corruption information can jointly guide medical image restoration.
  2. MoDeFold reports the highest average PSNR and SSIM among the compared methods on both benchmarks in the all-in-one setting, with additional evaluation under task-specific training. These results suggest practical potential, although the comparison and reliability concerns below limit the conclusions I draw from them.
Critical weaknesses
  1. Each dataset in the main benchmark has one degradation setting, making joint modeling difficult to distinguish from dataset conditioning. Late Conditioning Only performs worse than w/o MD Guidance by 0.99 dB on average, but this result is not explained. The wording “retained variants” and “updated full model” also leaves unclear whether all variants in Table 4 use the same backbone and protocol, limiting confidence in the component-level conclusions.
  2. MoDeFold uses eight-dataset pretraining and known modality labels, but comparable baseline resources and exclusion of test patients from pretraining are not clearly documented. DiffCode and VQPET are not compared. On AMIR, average PSNR exceeds TAT by only 0.05 dB despite the largest reported parameter count. Without matched-resource comparisons and repeated runs, it is difficult to attribute the gains to the design or assess their reliability.
Questions for authors
  1. Late conditioning only trails w/o MD Guidance by 0.99 dB on average and 2.23dB on COVID19. Why does fusion-side g hurt without early modulation? Please provide the complementary ablation retaining early modulation but removing g from fusion.
  2. “updated full model,” “retained variants,” and “earlier narrow-projection architecture” raise concerns about Tab.4’s consistency. Do all variants use the same backbone version and training/evaluation protocol?
  3. Spatial content features are pooled into a global condition, and the PMM receives guidance indirectly. What evidence shows that this improves anatomical structure preservation beyond generic global conditioning?
  4. AMIR gains over TAT are only 0.05db on average and 0.01dB on CT. Are these gains reproducible across seeds? Does the reported 49M parameter count include all frozen modules required at inference?
6 comments
AuthorsOpenReview ↗

Q1: Ablation experiment supplement

Response to Reviewer on Table 4 Ablations

We thank the reviewer for these questions. All Table 4 variants use the same final backbone version, data splits, training schedule, and evaluation protocol. Each variant was independently retrained, including the prior learning stage affected by its ablation. The phrases “updated full model” and “retained variants” referred to manuscript revisions, while “earlier narrow-projection architecture” referred to a separate attention diagnostic. We have removed these phrases to make clear that Table 4 does not mix architecture versions.

Table 4. Ablation of prior guidance and representation learning on our benchmark.

VariantACDC PSNRACDC SSIMCOVID19 PSNRCOVID19 SSIMHCC PSNRHCC SSIMBraTS PSNRBraTS SSIMAverage PSNRAverage SSIM
MoDeFold (Full)27.68240.783136.07580.970044.31180.982931.19500.911234.81630.9118
w/o Content Guidance25.73420.722135.13870.944243.79530.963230.12640.899733.69870.8823
w/o Modal-Degradation Guidance25.40720.703734.37760.922343.48950.952930.00180.879633.31900.8646
w/o Both Priors24.45780.679929.79710.840941.04670.951129.06140.887231.09080.8398
Late Conditioning Only25.54120.701332.14820.896742.12420.958229.51270.899432.33160.8639
Early Modulation Only25.81730.723334.68710.927943.78930.954230.43120.898933.68120.8761
w/o Contrastive Learning26.31540.715434.97420.938143.81190.970131.41760.913434.12980.8843
w/o Clean-Code Alignment26.37750.741334.87650.937243.66140.980231.47720.913634.09820.8931

The first three ablations assess the two priors separately and together. Removing content guidance, modal-degradation guidance, or both lowers average PSNR by 1.1176, 1.4973, and 3.7255 dB, respectively. The last two rows assess how the priors are learned. Removing contrastive learning or clean-code alignment lowers average PSNR by 0.6865 and 0.7181 dB.

The two conditioning variants isolate where gg acts. Late Conditioning Only supplies gg to fusion without using it to modulate the content features, whereas Early Modulation Only retains that modulation but removes the direct gg input to fusion. Early modulation alone reaches 33.6812 dB, exceeding late conditioning alone by 1.3496 dB on average and 2.5389 dB on COVID19CTscans. The full model improves a further 1.1351 dB over early modulation alone. Thus, gg is most effective when it first shapes the content representation and then also guides fusion. The low late-only result is consistent with a mismatch between degradation-sensitive fusion guidance and content features formed without that guidance; the ablation establishes the performance difference, while this mechanism remains an interpretation.

AuthorsOpenReview ↗

Q2 & W1: Ambiguous wording

Response

We thank the reviewer for identifying the ambiguous wording. “Updated full model” referred to a refresh of the reported Full result, and “retained variants” meant that the other ablation results were unchanged during that manuscript edit. Neither term denotes a different backbone. “Earlier narrow-projection architecture” referred to a previous GDM prompt design that compressed the fused prior into a narrow channel bottleneck and did not include the current 1×11\times1 projection of observation features. It was mentioned only to distinguish that design from the Figure 3 diagnostic, which removes the spatial-context branch from the final GDM while holding its inputs fixed. We'll revise the relevant wording in later versions.

All Table 4 variants use the same final GDM backbone version, data splits, training schedule, and evaluation protocol. Each variant was independently retrained through the prior-learning and unfolding stages affected by its ablation.

AuthorsOpenReview ↗

Q3: Anatomical Structure Preservation Under Global Prior Conditioning

Response

We thank the reviewer for this question. Pooling produces a global condition pp, but it does not replace the spatial estimate or observation features. At each GDM, the observation feature map contributes spatial context to the attention prompt, and the correction and PMM operate on spatial feature maps. The role of pp is to control how these features are updated. Before pooling, the content features are learned with multi-scale clean-code alignment, so this condition is supervised by paired clean anatomy rather than obtained from an unconstrained global feature.

Table 4 provides evidence that this content supervision matters. Removing content guidance reduces average SSIM from 0.9118 to 0.8823, while removing clean-code alignment reduces it to 0.8931. On ACDC, the corresponding SSIM values are 0.7831, 0.7221, and 0.7413. These results, together with the anatomical error maps in Figure 4, support improved structure retention.

Reviewer_swSUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes MoDeFold, an all-in-one medical image restoration framework that combines modality-dependent content guidance with a joint modality–degradation embedding. A supervised contrastive embedder captures modality and degradation information, while a modality-adapted multi-scale VQVAE provides content features trained with paired clean-code alignment. The embedding modulates the content features before fusion, and the resulting condition guides a learned, PGD-inspired unfolding network. Evaluation covers a four-dataset MRI/CT benchmark and the AMIR benchmark, under both all-in-one and task-specific settings. Ablations examine the roles of the two priors, early content modulation, contrastive learning, and clean-code alignment.

Critical strengths
  • Rather than using a degradation embedding only as a downstream prompt, MoDeFold uses it to modulate modality-adapted content features before fusion. This is a meaningful design choice for medical restoration, where anatomical appearance and corruption cues can be difficult to distinguish. Table 4 supports the contribution of this interaction: the full model exceeds the late-conditioning variant by approximately 2.48 dB in average PSNR.

  • On the four-dataset benchmark, MoDeFold improves average PSNR over the strongest competing average by approximately 1.126 dB and achieves the best SSIM on all four datasets. Evaluation on the additional AMIR benchmark and under task-specific training broadens the empirical evidence. The paper also appropriately acknowledges that the learned updates are PGD-inspired rather than exact gradients, and that the attention interventions measure conditioning sensitivity rather than restoration quality.

Critical weaknesses
  • The main benchmark strongly associates modality with degradation: both MRI datasets use low-resolution corruption, whereas CT datasets use noise or noise with streaks. This makes it difficult to establish whether the joint embedding captures complementary modality and degradation information beyond identifying the evaluated task combinations.

  • MoDeFold uses additional datasets for prior pretraining and has more parameters than the competing methods. Without controls for pretraining data and model capacity, the contribution of the proposed conditioning strategy is not fully isolated.

  • Although the gains on the custom benchmark are substantial, the average PSNR improvement on AMIR is only approximately 0.05 dB over the strongest competing average. Repeated-run results or uncertainty estimates would help establish the reliability of this advantage.

  • The framework involves multiple training stages and two separately parameterized frequency branches, but runtime, computational cost, and the full training configuration are not sufficiently documented. The patient-level split procedure and whether Table 4 ablations are retrained also need clarification.

  • The restoration comparisons in Figures 4 and 5 focus on cropped patches, making it difficult to assess anatomical consistency and artifacts across the full image. Please show the full restored images alongside the references, mark the selected regions, and include magnified patches as insets, for example, in the upper-left corner.

Questions for authors

Are the splits patient/volume-disjoint, and are validation/test subjects excluded from embedder pretraining, VQVAE adaptation, and any other medical-data pretraining? Please provide subject counts and explain the split procedure.

Were all Table 4 variants retrained under matched schedules, including the relevant prior-learning stages? Did competing methods receive the same additional pretraining data and comparable training budgets? Please also clarify the source and training data of the pretrained VQVAE.

The explanations of the training splits and pretraining resources, together with the new ablations, address most of my concerns. I will raise my score accordingly.

Please reconcile the dataset split descriptions in Q1 and the follow-up response. I would also like to see a discussion of relevant natural-image restoration methods, including AirNet, DA-CLIP, and BaryIR, to clarify the novelty of the joint representation and prior conditioning beyond the medical setting.

12 comments
AuthorsOpenReview ↗

Q1: Dataset Splite

Response

We really appreciate the reviewers' questions. The eight-dataset prior pretraining and four-dataset unfolding training are all split based on the IMIS-Bench (CVPR 2025) partitioning for training, validation or testing, so there’s no data leakage.

AuthorsOpenReview ↗

Q2: training fairness and VQVAE initialization

Response

We thank the reviewer for requesting these details. Every variant in Table 4 was trained as an independent experiment using the same data splits, initialization, optimization settings, and stage schedules. The relevant prior learning stages were rerun for each variant rather than changing a trained full model only at test time. In particular, the variant without contrastive learning retrained the embedder without the two contrastive losses before content adaptation and unfolding. The variant without clean code alignment retrained content adaptation without the alignment loss before unfolding. The variant with late conditioning bypassed token based content modulation during both content adaptation and unfolding. The full and ablated unfolding networks were trained under the same end to end schedule.

The competing methods were reproduced following their respective published training procedures. For the main comparison, they used the same four restoration datasets, data splits, and number of end to end training epochs as MoDeFold. The prior pretraining needed baselines, such as VLUNet, use eight datasets to train prior extraction parts.

The initial VQVAE weights were taken from the publicly released VARSR (ICML 2025) repository. The VARSR (ICML 2025) paper describes a training corpus of more than four million high quality natural images assembled from public and internal sources, rather than medical images. In MoDeFold, we freeze the shared VQVAE backbone and codebook, then train the modality adapters and token conditioned layers on the eight medical datasets described in Appendix D. These eight datasets are our medical content adaptation data, not the source of the original VARSR weights.

AuthorsOpenReview ↗

W1: Modality and Degradation Representation

Response

We thank the reviewer for raising this point. The four-dataset restoration benchmark does contain correlated modality and degradation settings. The representation, however, is learned during a separate prior-training stage using eight datasets and four degradation categories, as detailed in Appendix D. Modality labels denote the imaging modality rather than the dataset.

More specifically, Eq. 7 applies two contrastive objectives to the same token. The modality objective treats images of the same modality as positives even when they come from different datasets or have different degradations. The degradation objective instead treats images with the same degradation as positives across available modalities. A representation that only assigns an independent identity to each dataset would fail to bring these cross-dataset positive pairs together under both objectives. Removing contrastive learning reduces average PSNR from 34.8163 to 34.1298 dB in Table 4, showing that this supervision contributes to restoration.

Figure 1 provides a qualitative view under dataset and degradation coloring. We use it to illustrate the different grouping patterns, not as proof that the embedding contains no dataset information. We clarify that our claim concerns a joint representation supervised by both label relations, rather than complete disentanglement or demonstrated transfer to every unseen modality and degradation combination.

Reviewer_4EPaOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MoDeFold, a unified deep unfolding framework for all-in-one medical image restoration (MedIR). The core idea is to explicitly separate and jointly model two sources of variation: modality-dependent anatomical content and heterogeneous degradation. MoDeFold learns a joint modal-degradation embedding via dual contrastive learning, learns modality-specific content features with a multi-scale VQVAE supervised by clean-code references, and then uses the modal-degradation embedding to modulate content features before fusing both priors to condition a shared unfolding process. The framework is evaluated on a self-constructed benchmark with ACDC, BraTS2021, COVID19CTscans, and HCC-TACE-Seg under MRI low-resolution and CT low-dose/streak degradations, as well as on the AMIR benchmark for MRI super-resolution, CT denoising, and PET synthesis. Results show average gains over all-in-one baselines, and ablations support the complementary roles of content guidance, modal-degradation guidance, contrastive learning, and clean-code alignment.

Critical strengths
  1. The paper identifies a real limitation of existing deep unfolding methods: modality-dependent content and heterogeneous degradation are entangled in a shared restoration pipeline. MoDeFold addresses this by learning a joint modal-degradation embedding, using it to modulate modality-specific content features, and fusing both to control unfolding. The ablation showing that “Late Conditioning Only” drops average PSNR by 7.14% and SSIM by 5.25% provides strong evidence for the core design.
  2. The method is evaluated on a four-dataset all-in-one benchmark and on the AMIR benchmark, covering MRI, CT, and PET under multiple degradation settings. It reports both all-in-one and one-by-one results, and the ablations isolate the contributions of contrastive learning, clean-code alignment, and prior fusion. This makes the empirical study more convincing than a single-task evaluation.
Critical weaknesses
  1. The modal-degradation embedder and multi-scale VQVAE are pretrained on an eight-dataset collection that includes ACDC, BraTS2021, COVID19CTscans, and HCC-TACE-Seg. It is not stated whether the pretraining uses only the training splits of these datasets or also validation/test data. If validation/test images are used during pretraining, the reported restoration gains may be inflated. This must be clarified.
  2. On the AMIR benchmark, MoDeFold improves over TAT by only 0.04 dB on MRI SR, 0.01 dB on CT denoising, and 0.10 dB on PET synthesis, despite using 49M parameters versus 42M for TAT. The paper emphasizes relative percentages, but the absolute margins are small and may not be statistically significant. In addition, there is no runtime, memory, or FLOPs comparison, which is important given the large parameter count and multi-stage pretraining.
Questions for authors
  1. Were the modal-degradation embedder and content prior pretrained only on the training splits of the eight datasets? If any validation or test data from ACDC, BraTS2021, COVID19CTscans, or HCC-TACE-Seg were used during pretraining, how do you ensure that the reported test results are not affected by data leakage?
  2. At inference, does the method require the modality label m to select the modality-specific adapter? If the modality is unknown or mixed, how is m obtained? Does the auxiliary modality classifier provide a reliable substitute, and how does modality misclassification affect restoration?
  3. The improvements over TAT on AMIR are very small in absolute PSNR. Are these differences statistically significant across multiple runs or splits? Can you report confidence intervals or standard deviations?
  4. MoDeFold uses 49M parameters and involves pretraining, content adaptation, and unfolding stages. Can you provide training time, inference latency, GPU memory, and FLOPs compared with strong baselines such as TAT, VLU-Net, and R2R?
  5. The MRI low-resolution and CT low-dose/streak degradations are simulated in the image domain rather than in raw k-space or projection domain. How well does MoDeFold generalize to real under sampled MRI or real low-dose CT data? Do you have any preliminary results on real clinical degradations?
5 comments
AuthorsOpenReview ↗

Q1: Pretraining datasets and unfolding stage data

We really appreciate the reviewers' questions. The eight-dataset prior pretraining and four-dataset unfolding training are all split based on the IMIS-Bench partitioning for training or testing, so there’s no data leakage.

AuthorsOpenReview ↗

Q2: Do we need extra labels during inference

Response

Thanks to the reviewers for raising the questions. Our method doesn’t require any dataset modality or degradation labels during inference. Both VQVAE and the modal-degradation extractor can provide the necessary priors to guide the unfolding network in selecting the appropriate degradation matrix for iteration.

AuthorsOpenReview ↗

Q3: Result stability

Response

We thank the reviewer for this question. We repeated MoDeFold training with three additional independent random seeds on the same AMIR split and protocol. Table 1 reports the PSNR change from the original run, since the main results table rounds absolute PSNR to two decimal places.

Table 1. PSNR differences in dB relative to the original MoDeFold run on AMIR.

RunMRI super resolutionCT denoisingPET synthesisThree task average
Original run0.0000.0000.0000.0000
Additional seed 1−0.002−0.001+0.002−0.0003
Additional seed 2+0.003+0.002+0.001+0.0020
Additional seed 3+0.001+0.001−0.003−0.0003
Sample standard deviation across four runs0.00210.00130.00220.0011

The absolute per task change is at most 0.003 dB, and the three task average has a sample standard deviation of 0.0011 dB across the four runs. The reported 0.05 dB average PSNR difference from TAT is therefore stable with respect to the random seeds used for MoDeFold. These repetitions do not provide a formal significance test between methods, because TAT was not rerun and the data split was unchanged. We will describe the AMIR gain as small but stable across our runs rather than claim a significant difference across methods or splits.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary for Area Chairs and Program Chairs

Dear Area Chairs and Program Chairs,

We provide a summary of MoDeFold's central insight, the concerns raised by Reviewers 4EPa, swSU, dvBh, and 8ipH, and the evidence supplied during the discussion.

Our central insight

Medical restoration depends on both the imaging modality and the observed degradation. MoDeFold learns these two relations through dual contrastive supervision of a shared token. The token first modulates modality-adapted content features under clean-code supervision. The enhanced content features and the token are then fused to guide unfolding. The central design choice is using modal-degradation information to adapt structural content before it controls restoration, rather than using the token only as a late condition.

Concerns and responses

  1. Whether the token captures both modality and degradation (swSU, 8ipH). We clarified that the objective learns a joint representation and does not claim disentanglement. Figure 1 visualizes the learned tokens under the two label relations. New ablations show 34.8163 dB average PSNR with both contrastive objectives, compared with 31.6401 dB for modality contrast only, 33.6737 dB for degradation contrast only, and 34.1298 dB without contrastive learning.

  2. Whether early modulation and content adaptation are necessary (swSU, dvBh, 8ipH). The expanded ablation reports 33.6812 dB for Early Modulation Only and 32.3316 dB for Late Conditioning Only, compared with 34.8163 dB when both steps are used. A continuous-alignment control reaches 33.9139 dB, providing a direct comparison with quantized clean-code adaptation. The attention diagnostic further shows how changing the prior condition changes internal attention and unfolding corrections.

  3. Data separation and comparison resources (4EPa, swSU, 8ipH). We clarified the distinction between eight-dataset prior learning and four-dataset end-to-end unfolding, and that validation and test subjects from the evaluated datasets are excluded from training. We also clarified the natural-image origin of the VARSR VQVAE weights and their subsequent medical adaptation. The baseline resource description now states consistently that prior-dependent methods, including VLU-Net, used the eight-dataset collection for prior extraction.

  4. Model size, computation, and the role of unfolding (4EPa, swSU, dvBh, 8ipH). We supplied latency, memory, and FLOPs measurements in the reviewer replies. A new capacity control removes frequency splitting while retaining the proposed conditioning. It uses 28M parameters and reaches 34.5111 dB, compared with 33.6904 dB for the 30M-parameter R2R. We also clarified that the learned unfolding updates are PGD-inspired and are not claimed to be exact gradient steps of a fixed physical operator.

  5. Reliability and scope of the evaluations (4EPa, swSU, 8ipH). We reported three additional MoDeFold seeds on AMIR. We interpret its approximately 0.05 dB average PSNR difference from TAT as a small numerical gain. We clarified that the simulated degradations provide controlled tests and do not establish performance on prospective clinical acquisitions. The four-dataset benchmark additionally examines distinct MRI anatomy and two CT corruption conditions.

  6. Visual assessment across complete images (swSU). We updated the comparisons on both our four-dataset benchmark and AMIR to present full images with marked regions and magnified details, allowing readers to inspect overall anatomy as well as local restoration quality.

The revised manuscript incorporates the expanded ablations, clearer experimental descriptions, updated visual comparisons, and a frequency-splitting analysis in the appendix. We thank the reviewers and Area Chairs for their careful evaluation and discussion.

Authors of Submission 13373

AuthorsOpenReview ↗

Revision Summary

We thank the reviewers for their comments. We have made the following changes to the revised manuscript.

  1. Clarified the data protocol by distinguishing eight-dataset prior learning from four-dataset end-to-end unfolding and specifying that evaluation images are excluded from prior training.
  2. Clarified VQVAE initialization and adaptation. The initial VARSR weights were pretrained on natural images. Medical adaptation trains the modality adapters and token-conditioned layers while keeping the shared backbone and codebook frozen.
  3. Expanded Table 4 with Early Modulation Only, Modality Contrast Only, Degradation Contrast Only, and Continuous Alignment, and updated the corresponding analysis.
  4. Added a frequency decomposition ablation in the appendix, comparing the 28M-parameter variant with R2R and the 49M-parameter full model.
  5. Revised the AMIR interpretation to describe the approximately 0.05 dB average PSNR advantage over TAT as a small numerical gain without claiming statistical significance between methods.
  6. Updated the visual comparison to include full images, marked regions, and magnified details.

The main-text changes focus on the questions raised during review, with the frequency decomposition results and implementation details provided in the appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.