acceptodds
Under review as a conference paper at ICLR 2027

Reconstruct What the Task Needs: Progressive Task-Prior Information Bottlenecks for Accelerated MRI

Abstract

Magnetic resonance imaging (MRI) reconstruction is increasingly integrated with downstream tasks to preserve task-relevant structures. However, in multi-stage unrolled networks, stage-wise task feedback can entangle newly emerging task information with redundant and task-irrelevant content. We propose Purified Task Prior Guided MRI (PTP-MRI), which represents each stage update as a history-conditioned stochastic task increment. A variational lower bound encourages the retention of newly emerging task information, while a conditional variational upper bound limits task-irrelevant residual information. The resulting constrained information objective is optimized through Lagrangian duality with an adaptive dual variable. The purified task states are progressively accumulated and decoded into spatial priors that guide reconstruction according to task relevance. Experiments on SKM-TEA at acceleration and AHEAD at acceleration show that PTP-MRI improves downstream segmentation while maintaining reasonable reconstruction fidelity. Compared with the competing baseline for each metric, PTP-MRI achieves Dice/mIoU gains of 0.78/1.28 percentage points on SKM-TEA and 0.87/1.71 percentage points on AHEAD.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Magnetic Resonance Imaging, Accelerated MRI Reconstruction, Task-Driven Reconstruction, Unrolled Networks, Variational Information Bottleneck, Task Prior, Downstream Segmentation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_isfhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is about task-aware MRI reconstruction with unrolled networks. At every stage, a segmentation network predicts at the current image, and its label is fed back to guide the next stage. The authors argue that the full label at every time is spurious and may contain redundant and task-irrelevant content that should be purified. PTP-MRI instead keeps a running task state. At each stage, TPIB produces task increments to update task state. This task state then guide the reconstruction. The reconstruction network, TPIB, and the task network are trained in alternation. PTP-MRI gets the best Dice/mIoU in all benchmarks.

Critical strengths
  1. The motivation is clear, and the method is clearly written. The paper starts from a concrete observation in fig1. The method follows this observation: keep only new at each stage, accumulate it, and use the state to decide where the denoiser should be trusted. The formulation is carefully written.
  2. Clear improve on segmentation. Dice/mIOU improve.
Critical weaknesses
  1. The method is heavy for what it buys. 1st, training starts with a segmentation teacher pre trained on fully sampled images and labels. 2nd, each iteration runs 3 Training stages(reconstruction, TPIB, segmentation), each update runs the full six-stage(6 recon stages) pipeline. Too much hyper parameters to tune. At inference, segmentation network runs 7 times(6 recon and 1 final). These costs are not reported: no training time, inference time, memory usage. 3rd, It is also not clear that this gain comes from TPIB. The baselines seem not to get the knowledge distillation loss & task feature consistency loss. neither loss seems to be ablated.

  2. It is hard to see how this would be used in practice. 1st, everything is limit to one task. The reconstruction is trained to serve one segmentation task on one anatomy. A new organ means retraining all modules. 2nd, It needs a lot supervision. Requires pixel-level labels and fully sampled images. Many of the clinical settings where acceleration matters have neither. This limits where the method can be used. 3rd, most of the image is left largely unrestored. when t_n approx 0, the denoiser has no effect. In practice, radiologists still read the whole image and look for other findings, so an image optimized for one segmentation mask isn't that practical.

Questions for authors

Questions:

  1. It seems that the code contains only model, no losses, training loop?
  2. What are the training time, memory usage and inference time, compared with the baselines?
  3. Is TPIB needed? If available, please show (a)The model with t_n set to the segmentation network's foreground probability, without TPIB; (b)PTP-MRI without knowledge distillation loss & task feature consistency loss

small suggestions:

  1. Fig. 3 would be easier to read as table
  2. Please report the standard deviation across seeds
  3. typos: "With TPIB and the task network fixed" seems repeat; "stagesin" ?
6 comments
AuthorsOpenReview ↗

Response to Reviewer isfh (Question1-3 and Weaknesses 1–2)

Thank you for your careful review and constructive feedback. Below, we address your questions and concerns point by point and describe the corresponding revisions to the manuscript.

Weaknesses1 & Question2 & Question3: We acknowledge that teacher pretraining and the three alternating training stages introduce additional training complexity, and that the computational costs were insufficiently documented in the original manuscript. To quantify model size, computational complexity, and inference efficiency, we have added comparisons of parameter counts, FLOPs, and per-slice inference time using the same hardware, input size, and batch size (Table 1, Page 8). These comparisons show that PTP-MRI achieves the highest Dice score among the evaluated methods while maintaining competitive computational efficiency. Compared with A2MC-fix, the strongest baseline in terms of Dice, PTP-MRI improves Dice by 0.78 percentage points while reducing the parameter count by 44.0%, FLOPs by 32.8%, and inference time by 8.2%. Compared with IDSLR, which has the fewest parameters and the shortest inference time among the evaluated methods, PTP-MRI improves Dice by 13.76 percentage points. These results demonstrate a favorable balance between segmentation performance, model complexity, and inference efficiency. Regarding performance attribution, we have added ablation experiments within the PTP-MRI framework to examine the contributions of TPIB, the knowledge distillation (KD) loss, and the task feature consistency (TFC) loss. As reported in Table 3 of page 8, removing either KD or TFC reduces Dice, while removing TPIB leads to the largest decrease among the evaluated ablations. These findings support the contribution of both loss terms and highlight the importance of TPIB within the proposed framework. Table R2. Ablation study

MethodDice (%) ↑mIoU (%) ↑PSNR (dB) ↑SSIM (%) ↑
Unroll then Seg78.1566.3327.3963.47
Only foreground79.4766.5727.4866.71
w/o TFC80.7368.2728.5167.81
w/o KD80.3867.9928.6967.94
w/o KD and TFC80.1667.4428.4267.49
PTP-MRI (ours)81.0568.8228.7368.16
*Table R3. Computational efficiency comparison *
MethodParams (M) ↓FLOPs (G) ↓Time (ms) ↓Dice (%) ↑
:------:---:---:---:
Medl31.39144.01180.8679.56
MTrans785.222354.55191.8279.33
IDSLR2.59373.9496.4967.29
A2MC-fix8.64478.53131.8480.27
NO143.73697.82234.9780.06
PTP-MRI (ours)4.84321.62121.0481.05

Weaknesses2: Our intended application is accelerated MRI for a predefined anatomy and segmentation task, where reconstruction is optimized to preserve the structural information needed for subsequent segmentation and quantitative analysis. We agree that the current model is task-specific. Extending it to a new anatomy or task requires appropriate adaptation, training, and validation; direct transfer across anatomies or tasks has not been established in this study.

Regarding supervision, our method requires fully sampled reference images and pixel-level segmentation labels during training, but neither is required at inference. Once trained, the model processes undersampled measurements for the corresponding task. Nevertheless, we acknowledge that acquiring these training resources can be costly and limits applicability in settings where paired data or annotations are unavailable. Our experiments evaluate the method within this supervised setting and do not resolve the problem of insufficient training data.

To examine robustness to changes in sampling conditions, we additionally evaluated reconstruction and segmentation under different sampling masks (Table 2 of Page 8). Our method continued to outperform the comparison methods under the evaluated conditions. These results support generalization to the tested sampling patterns, but should not be interpreted as evidence of generalization across anatomies or tasks.

Regarding regions where (t_n) is close to zero, we would like to distinguish attenuation of the task-weighted denoising update from exclusion of those regions from reconstruction. When (t_n\approx0), the corresponding denoising contribution is attenuated. However, these regions remain subject to reconstruction loss in Eq (23). Thus, a near-zero denoising weight alone does not establish that the corresponding region is excluded from reconstruction.

Question1: We have added the loss functions and training loop to the appendix. The trained model weights will be publicly released upon acceptance of the paper.

AuthorsOpenReview ↗

Response to Reviewer isfh (Suggestion1-3)

Thank you for your thoughtful suggestions. Below, we explain how we addressed each one in the revised manuscript.

Suggestion1: The quantitative results previously shown in Fig. 3 are now presented in Table 1 of the revised manuscript, alongside the computational efficiency results, to improve readability.

Suggestion2: We have added the standard deviations across three random seeds (42, 88, and 100) to Tables 1, A1, and A2 in the revised manuscript.

Suggestion3: Thank you for pointing out these errors. We have removed the duplicated sentence containing “With TPIB and the task network fixed” and corrected “stagesin” to “stages in” in the revised manuscript.

AuthorsOpenReview ↗

Follow-up on Our Response to Reviewer isfh

We hope that our clarifications and additional experiments have addressed your main concerns. We would greatly appreciate your further feedback and welcome any remaining questions during the discussion period. We also hope that the new evidence and discussions will be taken into account in your final assessment.

Reviewer_Pa6qOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a method, PTP-MRI, to both reconstruct MRIs and identify structures in the MRI by filtering out irrelevant information at each stage of reconstruction, and emphasizing the relevant information for more accurate segmentation. This is done through using the reconstruction of MRI images to segment and identify structures within the MRI. The method is evaluated on datasets to measure the accuracy of segmentation and the quality of reconstruction.

Critical strengths

The idea to reconstruct using a history-conditioned method is interesting and it is a logical method for preventing noise during MRI reconstruction. Information-based reconstructions may be more accurate than previous methods that either take in unnecessary information or no information at all.

Critical weaknesses
  1. In the Introduction section, the figure (Figure 1) accompanying it is too small to read without zooming in and does not clearly indicate what the paper is trying to do. It is also confusing in showing the relationship between previous methods and the current method.

  2. In the Methods section, there are too many equations and not enough intuitive explanations for how the method is actually being executed. The figure (Figure 2) is not very well-explained as well.

  3. The Experiments section figure (Figure 3) did not showcase clear improvement visually in many cases, and only in a difference of numbers. Some of the visual results do not seem very different from other methods used and again, since the figure is small, it is hard to differentiate between the methods and their differences.

  4. The theory behind the entire method is not very sound, where assuming a reconstructed image could be trusted to give accurate segmentations. If all segmentation and structure identification is based purely on reconstructed images, there is risk that the segmentations become unusable if reconstruction is inaccurate.

Questions for authors

If segmentation accuracy is based on information from reconstructed images, how is reconstruction (PSNR/SSIM scores) accuracy ensured? If it is not, how useful is the segmentation from these MRIs?

5 comments
AuthorsOpenReview ↗

Response to the Weaknesses Identified by Reviewer Pa6q

We are grateful for your thoughtful feedback and explain below how we have addressed each of your comments and revised the manuscript.

Weaknesses1: We have revised Figure 1 to improve readability and clarify both our motivation and the relationship between existing approaches and PTP-MRI. Our goal is to determine what information reconstruction should preserve according to the downstream segmentation task. We are motivated by the observation that improved reconstruction metrics do not necessarily translate into better segmentation. In unrolled reconstruction, inter-stage information may contain both redundancy relative to previously exploited information and content irrelevant to segmentation. We therefore introduce a task-guided information bottleneck to regulate this information flow while preserving information needed for reconstruction and segmentation. The revised Fig.1 highlights how our approach builds on task-guided reconstruction through explicit inter-stage information constraints.

Weaknesses2: The central idea of our method is to encourage each stage of joint unrolled reconstruction and segmentation to contribute additional segmentation-relevant information while suppressing redundant and task-irrelevant information. The derivations translate this principle into explicit information constraints, tractable loss functions, and a corresponding optimization procedure.

Specifically, Eqs. (1)–(4) formulate this objective by defining the information that each stage should preserve or suppress. Equations (5)–(10) derive the task innovation loss, which encourages newly extracted information to be informative for segmentation. Equations (11)–(17) derive the task compression loss through a variational formulation, enabling the suppression of redundant and task-irrelevant information. Finally, Eqs. (18)–(20) incorporate these losses into the optimization problem in Eq. (3), yielding the Lagrangian objective in Eq. (19) and the multiplier update in Eq. (20). TPIB implements this formulation by learning the distributions required to compute and optimize the two losses in Eqs. (10) and (17). Figure 2(a) shows its architecture and the components used for loss computation, while Figure 2(b) illustrates the alternating optimization procedure. We have expanded the caption of Figure 2 to clarify the principle of our method.

Weaknesses3: We have revised Figure 3 in three ways. First, for the SKE-TEA dataset, we now present 3D segmentation results to facilitate comparison of the spatial morphology of the segmented structures (video is also given in the appendix). Second, for the AHEAD dataset, we use boundary-based visualizations to make differences in the segmentation contours easier to inspect. Third, all image panels have been enlarged to improve the visibility of structural details. These revisions enable a clearer qualitative comparison across methods and complement the quantitative results in Table1.

Weaknesses4: We apologize for the lack of clarity in our original explanation. We agree that reconstruction errors can propagate to segmentation and compromise its accuracy. PTP-MRI is designed to mitigate this problem and does not assume that reconstructed images or intermediate task predictions are accurate.

During training, reference segmentation labels provide supervision for learning task-guided information constraints. Specifically, the task innovation loss encourages the incorporation of additional segmentation-relevant information, while the task compression loss suppresses task-irrelevant information. These constraints guide reconstruction toward preserving information needed for the downstream segmentation task, addressing the fact that improved PSNR/SSIM alone does not necessarily lead to improved segmentation. Intermediate task predictions serve as guidance within this supervised learning framework and are not treated as ground-truth segmentation labels. At inference, segmentation remains dependent on the information recovered during reconstruction, and inaccurate reconstruction can still degrade segmentation performance. Our objective is therefore to reduce the impact of imperfect reconstruction on segmentation; we do not claim to eliminate this risk or guarantee accurate segmentation under arbitrary reconstruction errors.

AuthorsOpenReview ↗

Response to the Question Raised by Reviewer Pa6q

We appreciate your thoughtful question and address it below.

Question1: We agree that incomplete recovery of anatomical structures can compromise segmentation. This is a limitation shared by segmentation pipelines that rely on reconstructed undersampled MRI, and it is precisely the problem that motivates our work. PTP-MRI does not assume that reconstruction is already accurate. It incorporates downstream task supervision into reconstruction to improve the recovery of segmentation-relevant information under incomplete acquisition. Our objective is therefore to improve segmentation under a given undersampling condition, rather than to guarantee accurate segmentation despite arbitrary reconstruction failures. PSNR/SSIM alone cannot establish downstream segmentation accuracy; the benefit of task-guided reconstruction must be evaluated directly against reference segmentation annotations under matched acquisition conditions.

AuthorsOpenReview ↗

Follow-up on Our Response to Reviewer Pa6q

We hope that our clarifications and additional experiments have addressed your main concerns. We would greatly appreciate your further feedback and welcome any remaining questions during the discussion period. We also hope that the new evidence and discussions will be taken into account in your final assessment.

Reviewer_b2twOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes PTP-MRI for task-aware accelerated MRI reconstruction. A conditional information bottleneck extracts task increments at successive reconstruction stages, and the accumulated state produces a spatial prior that guides reconstruction. The reconstruction, bottleneck, and segmentation modules are trained alternately. Experiments on SKM-TEA and AHEAD show improved segmentation accuracy, with a trade-off in reconstruction quality.

Critical strengths
  1. This work is motivated-well: better global image-quality metrics do not necessarily translate into better downstream segmentation.
  2. The experiments include knee and brain MRI at two acceleration factors, subject-level splits, and averages over three random seeds.
  3. The ablation studies investigate history conditioning, state accumulation, compression, spatial guidance, and the information budget. This work also explains the limitations of the local variational objective.
Critical weaknesses
  1. The methodological novelty needs stronger justification. Task-guided unrolling and conditional information bottlenecks are established ideas. However, the main contribution is accumulated stage-wise states.
  2. The main comparisons confound the bottleneck with differences in task supervision. Most baselines freeze reconstruction before training segmentation, whereas PTP-MRI alternates task-aware updates and uses feature consistency and distillation losses. Comparisons using the same reconstruction backbone, task losses, and training protocol are needed.
  3. The statistical improvements are relatively limited. Dice gains over the strongest baselines are approximately 0.59–1.02 percentage points across four settings.
  4. Training and test use the same sampling mask for each configuration, leaving robustness to unseen masks untested. AHEAD has only ten test subjects and automatically generated labels, with no validation split or model-selection procedure specified. These limitations weaken the reliability of empirical results.
Questions for authors

See strengths and weaknesses, please.

5 comments
AuthorsOpenReview ↗

Response to Reviewer b2tw

We sincerely appreciate your valuable comments and address each point below, outlining the revisions we have made to the manuscript.

Weaknesses1: We agree that task-guided unrolling and conditional information bottlenecks are established concepts. Our methodological contribution lies in developing a theory-guided implementation of the task-guided information bottleneck specifically for unrolled MRI reconstruction. The accumulation of stage-wise states serves as a supporting design choice rather than the central methodological contribution. The specific challenge we address is how to translate the information-theoretic constraints into tractable training objectives within the unrolled reconstruction process. To this end, we formulate two complementary objectives, the task innovation loss and the task compression loss, based on the variational formulation of the information bottleneck. Together, these objectives encourage the preservation of task-relevant information and the suppression of task-irrelevant information during reconstruction. TPIB architecture is specifically designed to enable the implementation and optimization of these objectives, making its design integral to the proposed information bottleneck implementation. We have clarified the derivation and information-theoretic role of each loss and revised the contribution statements to emphasize the novelty.

Weaknesses2: To distinguish the contribution of TPIB from those of the auxiliary knowledge distillation (KD) and task feature consistency (TFC) losses, we added ablation experiments within PTP-MRI using a shared unrolled reconstruction backbone (Table 3, Page 8, can also refer to W1 of Reviewer isfh). Replacing TPIB with foreground guidance while retaining both auxiliary losses leads to a larger decrease in segmentation performance than removing KD, TFC, or both. Moreover, the variant retaining TPIB without either auxiliary loss outperforms the foreground-guidance variant that uses both losses. These comparisons support the contribution of TPIB. Moreover, TPIB, the proposed innovation/compression losses, and alternating optimization jointly constitute our bottleneck learning scheme within unrolled reconstruction. Imposing the same backbone and training protocol on all methods would require redesigning some baselines. We therefore preserve each method’s architecture and training strategy in cross-method comparisons. Regarding auxiliary supervision, TFC requires method-specific integration into reconstruction training, so we assessed its contribution through ablations within PTP-MRI. We added KD to the baseline segmentation networks while retaining their original reconstruction objectives (Table R1). KD improves MedI and MTrans but has mixed effects on the other methods. Table R1. Comparison with KD-augmented baselines on SKM-TEA at 8× acceleration.

MethodDice (%) ↑mIoU (%) ↑
MedI + KD80.1667.61
MTrans + KD79.6867.09
NO + KD79.5666.73
A2MC-fix + KD80.0366.58
PTP-MRI (ours)81.05|68.82

Weaknesses3: The Dice improvements should be considered jointly with computational efficiency. PTP-MRI achieves consistent gains of 0.59–1.02 percentage points over the strongest baselines across the evaluated settings. In the efficiency comparison, PTP-MRI achieves 81.05% Dice versus 80.27% for A2MC-fix, the strongest segmentation baseline in this setting, while using 44.0% fewer parameters (4.84M vs. 8.64M). Thus, the 0.78-point Dice improvement in this setting is accompanied by lower model size, computational cost, and inference latency. Together with the consistent improvements across settings, these results support a favorable accuracy–efficiency trade-off. We have included this efficiency analysis in the Table1 of Page 8.

Weaknesses4: Thank you for highlighting these limitations. We have added direct evaluation on SKM-TEA using two previously unseen 8× sampling masks: Gaussian random and Poisson-disc, without retraining or fine-tuning PTP-MRI (Table 2, Page 8). Our method achieves Dice/mIoU of 81.02%/68.78% and 81.43%/69.30%, respectively, outperforming all compared methods on both segmentation metrics. These results provide additional evidence of segmentation generalization to the evaluated unseen masks. For AHEAD, we did not reserve a separate validation set because of the limited number of available subjects. We evaluated the final checkpoint after training convergence, rather than performing validation-based checkpoint selection. We will explicitly document the actual stopping criterion and checkpoint policy to make this procedure reproducible. Moreover, on AHEAD, Dice evaluates segmentation performance against automatically generated reference labels, it does not establish anatomical accuracy against expert manual annotations. We will clarify this evaluation scope and acknowledge potential annotation bias.

AuthorsOpenReview ↗

Follow-up on Our Response to Reviewer b2tw

We hope that our clarifications and additional experiments have addressed your main concerns. We would greatly appreciate your further feedback and welcome any remaining questions during the discussion period. We also hope that the new evidence and discussions will be taken into account in your final assessment.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal to Reviewer b2tw

Dear Reviewer,

Thank you for your initial review and constructive feedback. We would like to briefly summarize the substantial revisions and additional experiments we have conducted to directly address the concerns raised during the rebuttal:

  1. Methodological clarification: We clarified the proposed information bottleneck, incremental state-update mechanism, and the distinction between learning task-relevant innovations and simply accumulating historical information.

  2. Additional controlled experiments: We introduced comparisons with Unroll then Seg and Foreground Guidance, together with ablations of TPIB, the task-innovation and compression losses, KD, and TFC, to better isolate the contributions of the proposed components.

  3. Fairness and computational efficiency: We supplemented comparisons with KD-enhanced baselines and reported training costs, memory consumption, inference efficiency, and model complexity.

  4. Additional validation and discussion: We expanded the experimental analysis and clarified the motivation, practical applicability, and limitations of the proposed framework.

These additions required considerable experimental and analytical effort and were specifically designed to address the concerns raised by the reviewers.

We sincerely hope that these new results and clarifications will receive careful consideration. As emphasized in the ICLR reviewing guidelines, active engagement in the discussion phase is an important part of the review process. We believe that meaningful discussion of the additional evidence is essential to a thorough and fair assessment of the paper.

We fully respect that reviewers may maintain their original assessments. However, we would greatly appreciate your feedback on whether the new evidence addresses your concerns and, if not, which specific questions remain unresolved.

Thank you again for your time and consideration. We sincerely look forward to your feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.