acceptodds
Withdrawn from ICLR 2027

Recurrent Amortized Solvers for Deformable Image Registration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

15 views

Abstract

Deformable image registration aligns anatomical structures across medical images, enabling comparison across subjects, modalities, and time points. It is commonly formulated as a regularized optimization problem balancing image alignment against deformation regularity. A central difficulty, shared by many regularized optimization problems, is choosing the regularization strength \(\lambda\) in advance: different choices induce different accuracy–regularity trade-offs. Current learning-based registration methods amortize this optimization across image pairs, but typically fix \(\lambda\) during training or use a single value throughout each prediction. We introduce the Recurrent Amortized Solver (RAS), a meta solver that can condition unrolled optimization with flexible \(\lambda\) scheduling. RAS enables \(\lambda\) to be controlled throughout the unrolling process without retraining the model. On each of three public 3D image registration benchmarks, we show that our RAS solver reaches competitive accuracy–regularity fronts with a single trained model. Furthermore, we introduce a per-case Pareto framework that evaluates the trade-offs available to each image pair, extending registration evaluation beyond cohort averages. RAS therefore provides a general formulation for learning iterative solvers with inference-time regularization control, applicable to regularized optimization problems beyond image registration.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Deformable Image Registration, Amortized Optimization, Algorithm Unrolling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HNpmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Paper Summary: The paper proposes Recurrent Amortized Solver (RAS) designed to enable step-wise regularization control during unrolled, iterative medical image registration. The network (LapIRN) is conditioned on both a step index and the regularization weight allowing users to vary the regularization schedule at inference time without retraining the model. The evaluation is performed across 3 benchmarks, well known in the registration community. Moreover, the authors report improved accuracy-regularity Pareto fronts compared to single-pass baselines (VoxelMorph and single-pass LapIRN) and they also propose a label-free per-case selection rule that enforces a per-pair folding limit, together with a “per-case Pareto” view of evaluation.

Critical strengths
  1. The paper introduces a conditioning mechanism and unrolled optimization allowing learning based registration to get best of both worlds, namely iterative optimization and controlling the regularization weight at run-time.
  2. Although per-case Pareto selection is not novel in itself, I appreciate that the paper reintroduces it: it is relevant to clinical practice, yet the registration community has largely neglected it.
  3. I appreciate the length of the experiments and results and the further results in the appendix.
Critical weaknesses
  1. Although I know that it is very common to compare single and multi-resolution models, I find this slightly unfair. I would appreciate very much to compare, e.g., single resolution models Voxelmorph, vs Hypermorph, vs RAS on Voxelmorph. This would allow us to dissentangle whether the benefit comes from the multi-resolution, the reg weight conditioning or the proposed method.
  2. In the same direction, I believe that the paper should perform statistical testing on the results of Table 1 since some of the reported differences might not be significant. Going beyond the performance in saturated benchmarks and discussing honestly the results and the methods would only benefit the field. I do not believe that we always need to have this bold line at the end of the table for a paper to have significant contribution.
  3. It would be nice to have a table that shows training time, test time, encoder parameters, and network parameters in total. Iterative optimization methods should also be included. This would help assess when and how each method can be used. Larger encoders (more parameters), see for example the comparison between Transmorph and Voxelmorph, help improve the performance, making the comparison a bit unfair again.
  4. Although the paper has extensive baselines for the brain datasets, the best performing methods on NLST on Learn2Reg are missing, e.g., Convex Adam[1].
  5. Results on standard Learn2Reg setups (the 128³ resampled OASIS is not comparable to published numbers).
  6. The continuation motivation (large λ early, small λ late) conflicts with the results: the best NLST schedule increases λ, and decreasing schedules hurt on OASIS. The claim that one-step models generalize poorly is also not supported by any experiments.

[1] Siebert H, Großbröhmer C, Hansen L, Heinrich MP. Convexadam: Self-configuring dual-optimization-based 3d multitask medical image registration. IEEE Transactions on Medical Imaging. 2024 Sep 16;44(2):738-48.

Questions for authors
  1. Can the authors run a single-scale unrolled model (e.g. VoxelMorph RAS)? This would cleanly isolate the effect of recurrent λ\lambda-scheduling without the confounding benefits of a multi-resolution pyramid.
  2. Can the authors include a table that shows training time, test time, encoder parameters, and network parameters in total? Iterative optimization methods should also be included.
  3. Given that a simple 2-pass execution of standard LapIRN (Appendix B.5) achieves comparable Dice, what is the practical justification for the added complexity of training unrolled, step-wise lambda conditioned hypernetworks? Can you please revise the limitation sections to report any further limitations that were omitted during the submission?
  4. Can the authors provide paired Wilcoxon signed-rank tests across all entries in Table 1 to confirm whether the minor accuracy gains are statistically significant?
  5. How does RAS compare on NLST against classical Learn2Reg methods like ConvexAdam[1]?
Reviewer_E8XaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes the Recurrent Amortized Solver (RAS), which uses an unrolled recurrent network and conditions the update network on regularization strength and step count. At inference, it generates a family of solutions using different regularization strengths and step counts, aiming to address the problem that fixed regularization parameter inputs in general registration methods lead to non-optimal trade-offs between folding and accuracy.

Critical strengths

The paper is clearly written, though some parts require effort to understand. The paper has the following strengths:

  1. Rigid λ scheduling is a real pain point, and exploring the optimal accuracy–deformation smoothness trade-off and Pareto front is practically meaningful.
  2. The method brings some performance improvement, and the experiments include both performance validation and basic mechanism analysis.
Critical weaknesses

However, I have some concerns about the reliability of the proposed method and the experimental validation, specifically:

  1. The paper focuses on the rigid λ scheduling problem and compares results across several methods, such as VoxelMorph, LapIRN, and CorrMLP, but this is still insufficient. For λ scheduling, comparison with methods such as cLapIRN is missing, and for performance, comparison with SOTA methods such as RDP and CGNet is missing.
  2. RAS has three conditioning ways: Concat, CIN, and HyperConv. Among them, HyperConv has the strongest expressive capacity, while HyperConv also has the largest parameter demand, but the corresponding ablation is missing, so the source of its advantage is questionable.
  3. Section 3.4 contains experiments that isolate scheduling. The gain on NLST is clear, but the hold–release schedule on OASIS remains negative, and there is a lack of corresponding explanation and experiments. More validation experiments at the mechanism level of RAS are needed.
  4. As a solution to rigid λ scheduling, the applicability of the method cannot be guaranteed. LapIRN has a natural advantage for condition injection, and for most registration networks, the cost of combining with RAS may be too large relative to the performance gain.
Questions for authors

Please refer to the questions and concerns raised in the "Critical Weaknesses" section.

Reviewer_EfAfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Recurrent Amortized Solver (RAS), a learning-based framework for deformable image registration that combines recurrent, unrolled refinement with inference-time control of the regularization strength. Specifically, RAS repeatedly applies a shared update network to predict residual deformation fields, while conditioning each update on the current regularization strength and a normalized iteration index. This design enables a single trained model to produce different accuracy–regularity trade-offs by changing the regularization schedule during inference. The paper further introduces a per-case Pareto selection strategy that chooses a registration result for each image pair according to an individual deformation-regularity constraint rather than using one cohort-wide setting. Experiments are conducted on OASIS, NLST, and LPBA40, using Dice and TRE for anatomical alignment and NDV and SDLogJ for deformation regularity, with comparisons to VoxelMorph, LapIRN, and several additional learning-based registration methods. The results suggest that recurrent refinement improves the accuracy–regularity trade-off and that per-update regularization scheduling can further modify the attainable Pareto front.

Critical strengths
  1. A useful unified formulation of iterative refinement and inference-time regularization control This paper integrates recurrent deformation refinement and regularization conditioning into a unified training framework, allowing each update step to use a different λ_t and enabling the regularization strength to be adjusted at inference time without retraining the model. Compared with recurrent models restricted to a fixed regularization strength or conditional models that perform only a single forward prediction, this design provides greater flexibility.
  2. The per-case Pareto perspective addresses an important limitation of cohort-level model selection The paper correctly observes that satisfying a deformation-regularity constraint on average does not guarantee that every individual registration satisfies that constraint. On OASIS, the Shared setting can satisfy the cohort-average NDV threshold while violating the threshold for a substantial number of individual cases, whereas the per-case rule removes these violations when a feasible candidate exists.
Critical weaknesses
  1. The methodological novelty is not sufficiently clear. The core idea of RAS is essentially to combine two existing research directions: regularization-conditioned registration and recurrent registration. Regularization conditioning, residual recurrent updates, and iteration-aware conditioning are not individually new. The current manuscript does not yet clearly demonstrate what fundamentally new optimization behavior or representational capability emerges from this combination that cannot be achieved by existing methods, beyond allowing a different λ_t to be specified at each update step.
  2. Insufficient ablation studies make the independent contribution of per-update regularization control unclear. RAS simultaneously incorporates recurrent refinement, λ_t-conditioning, and iteration-index conditioning \tau_t, but the paper lacks systematic experiments that separately remove or fix these components. As a result, the independent contribution of each component remains unclear. 3.The contributions of scale-dependent and iteration-dependent regularization have not been sufficiently disentangled. The validation-selected schedule on NLST mainly changes λ across different resolution levels, whereas the core motivation of the paper emphasizes dynamically varying λ_t across recurrent updates. Therefore, the current results do not clearly show whether the performance gains mainly come from using different regularization strengths across scales or from the proposed per-update regularization control.
Questions for authors

Q1. Could the authors provide a strictly matched ablation study to disentangle the effects of recurrence, λ_t-conditioning, and iteration-index conditioning τ_t? Q2. On NLST, the validation-selected schedule uses λ=1/16 at the two coarser levels and λ=1/2 at the finest level, which does not appear to align with the continuation motivation in the Introduction of using a largerλ at earlier stages and a smaller λ at later stages. How do the authors explain this behavior? Q3. How should the dynamic λ schedule be determined in practical deployment? The current experiments mainly rely on predefined candidate schedules and validation-set selection. For a new dataset or case, would the schedule need to be searched again, or is there a selection strategy that can generalize directly?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.