acceptodds
Under review as a conference paper at ICLR 2027

Locally Connected Pyramid Representation for Instance Optimization in 3D Image Registration

Abstract

We propose Locally Connected Pyramids for image registration (LCPReg), a light-weight neural displacement representation built specifically for pair-wise optimization. LCPReg explicitly optimizes only a coarse displacement field and carries it to image resolution through a cascade of learnable, locally connected upsampling layers. Each coarse site holds its own kernel, so the refinement can adapt to the local deformation, and each kernel is factorized into one-dimensional components, which keeps the parameter count to manageable numbers and encourages smooth transformations. We further introduce an approximate B-spline weight initialization scheme with warm start optimization that exactly reproduces a prior displacement estimate, allowing LCPReg to serve as a drop-in replacement in existing image registration pipelines. We integrate LCPReg into three frameworks (NODEO, convexAdam, and uniGradICON) and evaluate it on benchmark datasets (OASIS, CANDI and EMPIRE10). LCPReg consistently improves Dice scores over each framework's original representation with comparable regularity. It also substantially lowers optimization time for NODEO and uniGradICON.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Registration, Locally Connected Network, Instance Optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_7VKxOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LCPReg, a displacement-field representation designed for pair-wise and test-time instance optimization in 3D image registration. A coarse displacement field is refined through learnable, locally connected upsampling layers with separably factorized kernels. The method also introduces an approximate B-spline initialization and a Laplacian-pyramid warm start that exactly reproduces an existing displacement estimate before optimization. Integrations into NODEO, convexAdam, and uniGradICON demonstrate consistent Dice improvements on OASIS, CANDI, and EMPIRE10, with substantial reductions in optimization time for the NODEO and uniGradICON settings.

Critical strengths
  1. A clear and practically relevant contribution to instance optimization. The paper addresses whether representations designed for amortized prediction are necessarily suitable for refining a single image pair. Its focus on the displacement parameterization, rather than another large registration backbone, is well motivated.

  2. Consistent empirical benefits across different optimization frameworks. LCP improves Dice in all nine framework–dataset combinations. The integrations retain the corresponding registration objectives and match optimization iteration counts, making these within-framework comparisons particularly informative.

  3. Substantial computational benefits in the most expensive refinement settings. The reported runtime decreases from 66.10 to 23.13 seconds when replacing the NODEO representation, and from 68.08 to 9.67 seconds in the uniGradICON integration. These reductions accompany improved Dice rather than an accuracy sacrifice.

Critical weaknesses

1. The lung evaluation would be stronger with landmark-based correspondence measurements.

EMPIRE10 is evaluated using segmentation Dice, with HD95 provided in the appendix, but no landmark-based target registration error is reported(If I remember correctly, the dataset provides landmarks). Since the method is intended to improve local deformation refinement, landmark-based evaluation would provide a more direct test of this capability in the lung setting and complement the already high Dice scores.

2. Small performance differences require paired statistical analysis.

Some improvements are modest, particularly the EMPIRE10 Dice increase from 0.9820 to 0.9826 for the uniGradICON integration. Appropriate paired tests, such as a paired t-test, would help distinguish consistent gains from case-level variability.

Questions for authors
  1. Can you provide landmark-based evaluation for the lung registration task?

  2. Are the smaller improvements statistically supported in paired comparisons?

  3. What additional advantage does the full Laplacian warm start provide over the finest-level-only alternative?

  4. What are the end-to-end runtime and memory requirements of each integration?

4 comments

We thank the reviewer for the positive assessment and for recognizing the relevant contributions of our work. We address each of your concerns point-by-point below.


W1 / Q1. The lung evaluation would be stronger with landmark-based correspondence measurements.

(Weakness 1.) EMPIRE10 is evaluated using segmentation Dice, with HD95 provided in the appendix, but no landmark-based target registration error is reported(If I remember correctly, the dataset provides landmarks). Since the method is intended to improve local deformation refinement, landmark-based evaluation would provide a more direct test of this capability in the lung setting and complement the already high Dice scores. / (Q1) Can you provide landmark-based evaluation for the lung registration task?

We agree that landmark-based correspondence is a more direct test of local refinement than whole-lung Dice, which is close to saturation on EMPIRE10. However, because we were not able to acquire the landmark annotations originally used in the EMPIRE10 challenge, we instead evaluated all three integrations on two additional lung CT datasets with publicly available landmark annotations: NLST (landmarks and keypoints) and DirLab-4DCT (landmarks). For ease of readability, we have provided these results in the table in our general comment, or they can be viewed in Table 2 (new) in our revised manuscript. LCP reduces landmark TRE in all six framework–dataset combinations.

W2 / Q2. Small performance differences require paired statistical analysis.

(Weakness 2.) Some improvements are modest, particularly the EMPIRE10 Dice increase from 0.9820 to 0.9826 for the uniGradICON integration. Appropriate paired tests, such as a paired t-test, would help distinguish consistent gains from case-level variability. / (Q2) Are the smaller improvements statistically supported in paired comparisons?

As requested, we provide a paired t-test for the EMPIRE10 uniGradICON comparison:

MetricOurs(uGI)-uniGradICONp(raw)p(corrected)d_z
Dice0.00060.0090.0180.514
Neg. JD%-0.0070.4680.468-0.134

"p(corrected)" refers to Holm–Bonferroni corrected p-values and "d_z" is paired-samples Cohen’s d.

Although small, the increase in Dice is statistically significant, while the improvement in Neg. JD is not. For reference, we also provide the paired t-test for the OASIS uniGradICON comparison:

MetricOurs(uGI)-uniGradICONp(raw)p(corrected)d_z
Dice0.011<0.001<0.0012.212
Neg. JD%-0.002<0.001<0.001-0.496

Q3. Advantage of the full Laplacian warm start over the finest-level-only alternative

(Q3) What additional advantage does the full Laplacian warm start provide over the finest-level-only alternative?

We agree that the accuracy difference is negligible; the reason we keep the full warm start is that it is the only variant that initializes the pyramid in a state consistent with its intended optimization dynamics. Seeding only the coarse field u(L)u^{(L)} discards detail below h(L)h^{(L)} and lowers Dice to 0.8007. Seeding only the finest level preserves u^\hat{u}, but leaves every intermediate level u(ℓ)u^{(\ell)} at zero. Since the gradient with respect to each kernel weight is proportional to u(ℓ)(q)u^{(\ell)}(q) (Eq. 15), the kernels then receive no gradient at the start of optimization, and the prior is held entirely in a fixed residual that the kernels cannot adapt. The full warm start sets u(ℓ)=D(ℓ)u^u^{(\ell)} = D^{(\ell)}\hat u at every level, so the adaptive term of the tangent kernel (Eq. 18) is active from the first iteration and concentrates kernel adaptation where the prior model predicts motion. This matters most in short instance optimization schedules such as uniGradICON's 50 iterations. Empirically, it also yields less folding (3.4e-3% vs 4.9e-3%) at no additional cost (9.67 s vs 9.65 s).

Q4. End-to-end runtime and memory

(Q4) What are the end-to-end runtime and memory requirements of each integration?

We added Table 3 (new), which reports the time of the initial prediction (Base), the optimization time (Opt.), and GPU memory requirements during optimization. The times in Table 1 are the end-to-end totals (Base + Opt.).

MethodBase (s)Opt. (s)Total (s)Peak memory
NODEO–66.1066.102.53 GB
LCPReg-opt (Ours)–23.1323.132.69 GB
convexAdam-MIND0.013.033.043.84 GB
LCPReg-conv (Ours)0.013.363.375.58 GB
uniGradICON (w/ IO)0.3567.7368.0812.77 GB
LCPReg-uGI (Ours)0.359.329.677.62 GB

In the uniGradICON integration, LCP reduces end-to-end time by 86% and peak memory by 40%, since backpropagation no longer passes through the U-Net cascade. In the NODEO integration, it reduces time by 65% at a 6% higher memory footprint. In convexAdam, which already optimizes a dense field directly, LCP adds roughly 0.33 seconds of optimization time per pair and requires around 45% more GPU memory.


We thank the reviewer again for the constructive suggestions, which we believe have strengthened the evaluation of our work.

I thank the authors for the additional experiments. My concerns have been addressed, and I will maintain my original score.

We sincerely thank the reviewer for the time investment in assessing our work and rebuttal. We are glad that our response addressed your concerns!

Reviewer_Uz9POpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LCPReg, a displacement representation based on separable, locally connected upsampling layers for 3D image registration. It improves average Dice across three frameworks and reduces optimization time for NODEO and uniGradICON.

Critical strengths
  1. The paper addresses the practical cost of instance optimization. Its integration into three existing frameworks shows its usefulness across different pipelines.

  2. The architecture balances local flexibility and parameter cost. Spatially varying kernels allow local adaptation, while separable factors reduce the number of parameters.

  3. The accuracy and speed improvements are encouraging. For uniGradICON, OASIS Dice increases from 0.7942 to 0.8053, while reported runtime drops from 68.08 to 9.67 seconds.

Critical weaknesses
  1. Comparable deformation regularity is not consistently supported. On CANDI, LCPReg-conv doubles the negative-Jacobian percentage from 2.9e-3% to 5.8e-3% (Table 1), and uniGradICON from 3.5e-5% to 3.5e-4%.

  2. The ablations do not fully isolate the key design choices. Direct comparisons of shared convolutional upsampling, kernel separability, and channel sharing are missing.

  3. The full multilevel warm start needs stronger justification. Table 2 shows nearly identical Dice for full and finest-level-only initialization: 0.8049 versus 0.8047.

  4. Please sweep regularization strength and compare Dice at similar folding rates and severity. Table 3 shows worse Sum JD for LCPReg-uGI, increasing from 3.99 to 9.31 on OASIS and from 0.08 to 0.73 on CANDI.

  5. Please verify the CANDI negative-Jacobian entry 3.5e-5 (7.3e-4) for uniGradICON with IO.

  6. Please specify the evaluated EMPIRE10 structures, add landmark TRE if available, and report Dice changes by brain structure on OASIS. Whole-lung overlap may miss internal misalignment, while average brain Dice may hide losses in individual structures.

  7. Lack of some related works, such as CorrMLP-CVPR2024, SDHNet-TMI2023, RDN-JBHI2022, and more.

Questions for authors

Please refer to "Critical Weaknesses"

3 comments

We thank the reviewer for the detailed and meaningful feedback. We address each point below and have included several of the suggested improvements in our revised manuscript.


W1. Deformation regularity is not consistently comparable

(Weakness 1) 1. Comparable deformation regularity is not consistently supported. On CANDI, LCPReg-conv doubles the negative-Jacobian percentage from 2.9e-3% to 5.8e-3% (Table 1), and uniGradICON from 3.5e-5% to 3.5e-4%.

We acknowledge the two regressions on CANDI pointed out by the reviewer. Regarding uniGradICON, its initial predictions on CANDI are exceptionally smooth (4.9e-6% negative Jacobians before instance optimization), but this comes at the cost of accuracy, as it is among the weakest methods on CANDI in Table 1 (0.7313 Dice), despite performing strongly on OASIS and EMPIRE10. Starting from such a regular field leaves very little room to lower the folding rate further, and any refinement that recovers the missing accuracy is likely to introduce some folding. We also provide an analysis of results at comparable negative Jacobian percentages for uniGradICON in our response to W4.

LCPReg-conv, on the other hand, inherits the regularization of convexAdam, which was tuned for a dense displacement grid and which we did not retune for our representation. The additional local flexibility of the pyramid improves Dice on CANDI by 0.009 (0.7432 vs 0.7338). Even then, across all six datasets, LCPReg-conv lowers the negative Jacobian percentage on four (OASIS, EMPIRE10, NLST, and DirLab-4DCT).

W2. Isolating shared upsampling, separability and channel sharing

(Weakness 2) 2. The ablations do not fully isolate the key design choices. Direct comparisons of shared convolutional upsampling, kernel separability, and channel sharing are missing.

We added a dedicated ablation (new Table 4, LCPReg-uGI setting) that covers exactly these three choices:

WeightsSeparableChannel-wiseOASIS Dice ↑OASIS Neg. JD (%) ↓CANDI Dice ↑CANDI Neg. JD (%) ↓Memory ↓
Frozen upsample--0.79821.9e-30.76175.5e-54.98 GB
Shared✗all0.79821.6e-30.76174.6e-54.98 GB
Shared✓coarse0.79821.4e-30.76173.9e-54.98 GB
Local✗coarse0.78801.5e-20.75995.3e-311.25 GB
Local✓none0.80223.5e-30.76732.4e-47.55 GB
Local (default)✓coarse0.80493.4e-30.77083.5e-47.62 GB
Local✓all0.80503.9e-30.77083.5e-49.23 GB

Local and shared weights comparison. Local weights improve Dice by 0.007 (OASIS) and 0.009 (CANDI). Shared kernels are cheaper but do not significantly improve accuracy compared to using non-learnable trilinear upsampling ("Frozen" settings). Consistent with our discussion in Section 3.5, shared kernel weights add only a few global directions to the tangent kernel, whereas local kernels add a term that acts where the field deforms.

Separability. For locally connected kernels, separable factors are better on every metric: +0.017 / +0.011 Dice, 4× / 15× fewer folds, and 32% less memory than full kernels.

Channel-wise kernels. Using them at the coarse levels adds +0.003 Dice on both datasets; however, extending them to the finest level adds no further accuracy for an additional ~1.6 GB, which motivates our default setting.

W3. Justification of the full multi-level warm start

(Weakness 3) 3. The full multilevel warm start needs stronger justification. Table 2 shows nearly identical Dice for full and finest-level-only initialization: 0.8049 versus 0.8047.

We agree that the accuracy difference is negligible; the reason we keep the full warm start is that it is the only variant that initializes the pyramid in a state consistent with its intended optimization dynamics. Seeding only the coarse field u(L)u^{(L)} discards detail below h(L)h^{(L)} and lowers Dice to 0.8007. Seeding only the finest level preserves u^\hat{u}, but leaves every intermediate level u(ℓ)u^{(\ell)} at zero. Since the gradient with respect to each kernel weight is proportional to u(ℓ)(q)u^{(\ell)}(q) (Eq. 15), the kernels then receive no gradient at the start of optimization, and the prior is held entirely in a fixed residual that the kernels cannot adapt. The full warm start sets u(ℓ)=D(ℓ)u^u^{(\ell)} = D^{(\ell)}\hat u at every level, so the adaptive term of the tangent kernel (Eq. 18) is active from the first iteration and concentrates kernel adaptation where the prior model predicts motion. This matters most in short instance optimization schedules such as uniGradICON's 50 iterations. Empirically, it also yields less folding (3.4e-3% vs 4.9e-3%) at no additional cost (9.67 s vs 9.65 s).

W4. Regularization sweep; Dice at similar folding rates and severity

(Weakness 4) 4. Please sweep regularization strength and compare Dice at similar folding rates and severity. Table 3 shows worse Sum JD for LCPReg-uGI, increasing from 3.99 to 9.31 on OASIS and from 0.08 to 0.73 on CANDI.

In our original experiments, we did not specifically ablate the regularization weight of uniGradICON because we believed it was more valuable to compare models at the same parameter settings. However, we believe that the default configuration used during training (λ=1.5\lambda=1.5) is insufficient for Instance Optimization. This becomes more apparent when Instance Optimization is performed for more steps:

OASIS (uniGradICON IO)initial100 steps200 steps300 steps
Dice0.78710.79730.80130.8026
Neg. JD (%)2.2e-41.1e-22.3e-22.6e-2

We argue that, because our LCPReg-uGI model converges faster than uniGradICON, the weak regularization term becomes more apparent. For this reason, a higher value of λ\lambda benefits our model more than the default uniGradICON. In the table below, we compare the performance of LCPReg-uGI at values of λ\lambda chosen to have comparably low folding to the default uniGradICON setting:

SettingOASIS DiceNeg. JD (%)Sum JDCANDI DiceNeg. JD (%)Sum JD
uniGradICON (default) λ\lambda =1.50.79426.1e-33.990.74813.5e-50.08
LCPReg-uGI λ\lambda=90.79951.3e-34.580.76446.3e-50.11
LCPReg-uGI λ\lambda=120.79747.8e-43.260.76192.9e-50.05

This shows that, even at lower Neg. JD values, LCPReg-uGI still outperforms the baseline uniGradICON in Dice score.

W5. CANDI negative-Jacobian entry for uniGradICON IO

(Weakness 5) 5. Please verify the CANDI negative-Jacobian entry 3.5e-5 (7.3e-4) for uniGradICON with IO.

Thank you for the keen observation; the standard deviation is unnaturally high. We have rerun the experiment and corrected the value to be 3.5e-5 (1.8e-4). We apologize for the mistake as it was likely a transcription issue, and we absolutely appreciate the correction.

W6. EMPIRE10 structures, landmark TRE, per-structure Dice

(Weakness 6) 6. Please specify the evaluated EMPIRE10 structures, add landmark TRE if available, and report Dice changes by brain structure on OASIS. Whole-lung overlap may miss internal misalignment, while average brain Dice may hide losses in individual structures.

Following the evaluation in PIRATE (Hu et al., 2024), we computed Dice scores using the lung masks provided in the EMPIRE10 dataset. We completely agree that this is not sufficient for accurate evaluation of lung registration, but as landmark annotations are, to our knowledge, not publicly available, we decided to conduct additional experiments on the NLST and DirLab-4DCT datasets. In addition, we also included the AbdomenCT dataset from the Learn2Reg challenge. Please refer to our general comment for the results, or Table 2 (new) in the revised manuscript.

Regarding individual brain structure dice scores, we included a box plot in Figure 5 of the revised manuscript. For better readability, we merged the dice scores of the same structures across both hemispheres (e.g. "left hippocampus" and "right hippocampus" are merged as one structure labeled "hippocampus").

W7. Missing related works

(Weakness 7) 7. Lack of some related works, such as CorrMLP-CVPR2024, SDHNet-TMI2023, RDN-JBHI2022, and more.

We appreciate the suggestion of related papers. We have added CorrMLP (CVPR 2024), SDHNet (TMI 2023), and RDN (JBHI 2022) to our related work section in the discussion of learning-based registration networks. While not a direct comparison to our instance optimization-based approach, we believe that these works could also provide the initial prediction u^\hat u that our LCP warm-starts from, in the same way as uniGradICON in LCPReg-uGI.


We hope these changes address the reviewer's concerns, and we would be glad to provide any further details.

Dear Reviewer Uz9P, as the author-reviewer discussion period is about to end, we kindly remind you that we will be unable to communicate after today. Please let us know if we have addressed your concerns and if there is anything else that requires clarification.

Reviewer_PZZvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a displacement-field representation for pairwise and instance-optimized 3D registration. A coarse field is expanded through locally connected upsampling operators whose spatial kernels are factored into three one-dimensional vectors. Fixed Laplacian residuals preserve the fine-scale content of an initial field. The authors analyze the induced optimization dynamics and replace the deformation representations in NODEO, convexAdam and uniGradICON. The replacements improve segmentation overlap on two brain MRI datasets and a lung CT dataset, with substantial time reductions in the NODEO and uniGradICON settings.

Critical strengths

First of all, I think the paper addresses a practical problem in image registration. A network designed to generalize across different image pairs may not be the best choice for optimizing a single pair. The proposed representation is relatively simple, allows spatially adaptive refinement, and can be initialized from an existing deformation field.

Another strength is that the authors test their method in three different registration frameworks while keeping the original optimization objectives and iteration counts. This makes the comparisons more convincing. In particular, for uniGradICON, the Dice score improves from 0.7942 to 0.8053 on OASIS and from 0.7481 to 0.7708 on CANDI, while the optimization time decreases from 68.08 to 9.67 seconds. The HD95 results also support the improvements in brain registration.

Finally, the optimization analysis provides some useful insights into the proposed representation. Equation (11) explains the contributions of the coarse displacement field and the learnable upsampling kernels. The authors also acknowledge the limitations of their analysis in Appendix B, particularly regarding its extension to Adam. The ablation studies on different representations, kernel learning, and initialization strategies further support the effectiveness of the proposed design.

Critical weaknesses

First, I think some of the design choices need stronger justification. Figure 4 shows that learning the upsampling kernels helps, but it is unclear whether locally connected kernels are better than shared kernels. Also, Table 2 shows very similar Dice scores for the full warm-start strategy (0.8049) and finest-level-only initialization (0.8047).

Another concern is deformation regularity. On OASIS, the sum of negative Jacobians increases from 3.99 to 9.31 for LCPReg-uGI (Table 3). Similar trade-offs can be observed on CANDI. More experiments would help clarify the balance between registration accuracy and deformation regularity.

Finally, the improvement on EMPIRE10 is quite small, especially for LCPReg-uGI compared with uniGradICON IO (0.9826 vs. 0.9820 Dice). Landmark-based evaluation and statistical analysis would make the results more convincing.

Questions for authors

First, have the authors tested spatially shared separable kernels? It would be useful to compare their accuracy, runtime, and memory usage with the proposed method.

Second, do the improvements on OASIS and CANDI still hold at similar folding rates? Are the larger negative Jacobian values mainly caused by a few cases?

Finally, I have a question about the initialization of LCPReg-conv. Eqs. (9)–(10) suggest that the initial displacement is reproduced exactly, but Eq. (26) applies additional smoothing. Does this change the actual initial deformation?

4 comments
AuthorsOpenReview ↗

Response to Reviewer PZZv

We thank the reviewer for the positive assessment in recognizing the practical motivations of improving instance optimization in image registration and for their appreciation of our optimization analysis. We address each of your concerns point-by-point below.


W1a / Q1. Locally connected vs. spatially shared separable kernels (accuracy, runtime, memory)

(Weakness 1a) First, I think some of the design choices need stronger justification. Figure 4 shows that learning the upsampling kernels helps, but it is unclear whether locally connected kernels are better than shared kernels. (...) / (Q1) First, have the authors tested spatially shared separable kernels? It would be useful to compare their accuracy, runtime, and memory usage with the proposed method.

We added an ablation over the type of upsampling layer (new Table 4), run in the LCPReg-uGI setting. The comparison the reviewer asked for is:

Upsampling weightsSeparableChannel-wiseOASIS Dice ↑OASIS Neg. JD (%) ↓CANDI Dice ↑CANDI Neg. JD (%) ↓Memory ↓Time (s) ↓
Frozen upsample--0.79821.9e-30.76175.5e-54.98 GB9.63
Shared (convolutional)✓coarse0.79821.4e-30.76173.9e-54.98 GB9.63
Locally connected (default)✓coarse0.80493.4e-30.77083.5e-47.62 GB9.67

Firstly, when compared to simple non-learnable trilinear upsampling layers ("Frozen"), shared weights do not affect accuracy at all, but they are able to slightly smooth the deformation field. Locally connected kernels improve Dice by 0.007 on OASIS and 0.009 on CANDI over shared separable kernels. That local freedom is the source of the accuracy gain, but also of the additional folding (see W2).

Shared kernels require about 35% less GPU memory than LCP, as the majority of optimized parameters remain in the coarse field u(L)u^{(L)}. The locally connected default still uses about 40% less memory than uniGradICON IO (7.62 GB vs 12.77 GB, Table 3 (new)). We did not observe a significant difference in execution time between the different settings.

W1b. Full warm start vs. finest-level-only initialization

(Weakness 1b) (...) Also, Table 2 shows very similar Dice scores for the full warm-start strategy (0.8049) and finest-level-only initialization (0.8047).

We agree that the accuracy difference is negligible; the reason we keep the full warm start is that it is the only variant that initializes the pyramid in a state consistent with its intended optimization dynamics. Seeding only the coarse field u(L)u^{(L)} discards detail below h(L)h^{(L)} and lowers Dice to 0.8007. Seeding only the finest level preserves u^\hat{u}, but leaves every intermediate level u(ℓ)u^{(\ell)} at zero. Since the gradient with respect to each kernel weight is proportional to u(ℓ)(q)u^{(\ell)}(q) (Eq. 15), the kernels then receive no gradient at the start of optimization, and the prior is held entirely in a fixed residual that the kernels cannot adapt. The full warm start sets u(ℓ)=D(ℓ)u^u^{(\ell)} = D^{(\ell)}\hat u at every level, so the adaptive term of the tangent kernel (Eq. 18) is active from the first iteration and concentrates kernel adaptation where the prior model predicts motion. This matters most in short instance optimization schedules such as uniGradICON's 50 iterations. Empirically, it also yields less folding (3.4e-3% vs 4.9e-3%) at no additional cost (9.67 s vs 9.65 s).

AuthorsOpenReview ↗

Response to Reviewer PZZv

W2 / Q2. Regularity trade-off, accuracy at similar folding rates, and concentration in few cases

(Weakness 2) Another concern is deformation regularity. On OASIS, the sum of negative Jacobians increases from 3.99 to 9.31 for LCPReg-uGI (Table 3). Similar trade-offs can be observed on CANDI. More experiments would help clarify the balance between registration accuracy and deformation regularity. (Q2) Second, do the improvements on OASIS and CANDI still hold at similar folding rates? Are the larger negative Jacobian values mainly caused by a few cases?

We have included additional experiments on 3 new datasets to better validate our model's superior performance in deformation regularity (with this we achieve a lower negative-Jacobian percentage in 14 of the 18 baseline comparisons).

Regarding results at similar folding rates, we evaluate higher regularization coefficient values λ\lambda in our LCPReg-uGI setting below:

SettingOASIS DiceNeg. JD (%)Sum JDCANDI DiceNeg. JD (%)Sum JD
uniGradIcon (default) λ\lambda =1.50.79426.1e-33.990.74813.5e-50.08
LCPReg-uGI λ\lambda=90.79951.3e-34.580.76446.3e-50.11
LCPReg-uGI λ\lambda=120.79747.8e-43.260.76192.9e-50.05

As demonstrated, even at similar folding rates, our method maintains superior registration accuracy to the baseline uniGradICON.

Although we did not modify the value of λ\lambda in our original experiments, we have found that uniGradICON is under-regularized for its Instance Optimization setting. Namely, at longer optimization, the result converges to a significantly higher folding rate:

OASIS (uniGradICON IO)initial100 steps200 steps300 steps
Dice0.78710.79730.80130.8026
Neg. JD (%)2.2e-41.1e-22.3e-22.6e-2

Even with 300 iterations and high folding, uniGradICON IO underperforms our method in accuracy.

We believe that our LCPReg-uGI model also suffers from this issue, which is exacerbated due to its faster convergence rate. For this reason, we suggest a higher value for λ\lambda in the future.

(Q2b) Are the larger negative Jacobian values mainly caused by a few cases?

In the case of LCPReg-uGI on OASIS, the median value for Sum JD is 3.69 (over 225 image pairing);the top 5 pairs accounted for 36.5% of the total Sum JD value. In comparison, uniGradICON IO has a median of 1.56 and the share of the top 5 pairs sits lower at 21.5%. This does partially explain the higher folding areas of LCPReg-uGI even though its other regularity metrics are lower than the baseline.

W3. Small EMPIRE10 improvement; landmarks and statistics

(Weakness 3)Finally, the improvement on EMPIRE10 is quite small, especially for LCPReg-uGI compared with uniGradICON IO (0.9826 vs. 0.9820 Dice). Landmark-based evaluation and statistical analysis would make the results more convincing.

We agree that whole-lung Dice on EMPIRE10 is close to saturation and is a weak test of local alignment. However, as mentioned in our General Comment, to our knowledge landmark data is not publicly available for the EMPIRE10 dataset. To compensate, we conducted experiments on two new lung datasets (NLST and DirLab-4DCT) that both contain manually annotated landmarks, and also included experiments on the Learn2Reg AbdomenCT dataset. The results of these experiments are in our General Comment and in the newly added Table 2 of the revised manuscript. Our method shows improved results across all baseline comparisons.

Additionally, we performed a paired t-test statistical analysis for the uniGradICON comparison on EMPIRE10:

MetricOurs(uGI)-uniGradICONp(raw)p(corrected)d_z
Dice0.00060.0090.0180.514
Neg. JD%-0.0070.4680.468-0.134

"p(corrected)" refers to Holm–Bonferroni corrected p-values and "d_z" is paired-samples Cohen’s d.

Q3. Does the smoothing in Eq. (26) change the initial deformation of LCPReg-conv?

(Q3) Finally, I have a question about the initialization of LCPReg-conv. Eqs. (9)–(10) suggest that the initial displacement is reproduced exactly, but Eq. (26) applies additional smoothing. Does this change the actual initial deformation?

The pyramid reproduces u^\hat{u} exactly, and the effective initial field S(u^)S(\hat{u}) is identical to convexAdam’s own starting point. In convexAdam, smoothing is applied to the displacement field at each step of the optimization (this is similar to Eq. 20 in our LCPReg-opt implementation). In LCPReg-conv, we apply the initialization so that it exactly mirrors the behavior of the Adam optimization performed in the original method, which means that the output of the Locally Connected Pyramid mirrors the displacement field before the smoothing is applied. This way, the LCP is a direct replacement of the dense field used by convexAdam so that both methods have the exact same initial deformation at the start of optimization (which is the smoothed field). We have added a sentence to Appendix Section C.2 to clarify this detail.


We hope these additions address the reviewer's concerns, and we are happy to provide further details.

Dear Reviewer PZZv, as the author-reviewer discussion period is about to end, we kindly remind you that we will be unable to communicate after today. Please let us know if we have addressed your concerns and if there is anything else that requires clarification.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their thoughtful and constructive feedback. We are glad that the reviewers found the problem practically relevant, appreciated the integration of LCP into three different registration frameworks with matched objectives and iteration counts, and recognized the reductions in optimization time for NODEO and uniGradICON.

Revised manuscript. We have uploaded a revised manuscript with the additional experiments to openreview. In it, changes are highlighted in yellow.

Note on table numbering. Because of the new tables, some table numbers in the revised manuscript have changed: the component ablation previously referred to as Table 2 is now Table 5, and the additional metrics for OASIS and CANDI (HD95, Sum JD, SDLogJD), previously Table 3, are now Table 6. Table 1 and all figure numbers are unchanged. The new additions are Table 2 (NLST, DirLab-4DCT and AbdomenCT), Table 3 (runtime and memory), Table 4 (upsampling layer ablation), Tables 7 and 8 (additional metrics for the new datasets) and Figure 5 (per-structure Dice on OASIS).

Experiments on three additional datasets

We replied to each reviewer's concerns individually; however, a request shared by all three reviewers concerned the evaluation on the EMPIRE10 dataset. We agree with the reviewers' concerns that Dice score is not a sufficient measure for lung registration tasks. Unfortunately, we were not able to find ground truth Landmarks that correspond to the EMPIRE10 challenge, as those seem to be kept private by the challenge organizers. Due to the limited timeframe of this discussion period, instead of seeking out these additional evaluation metrics for EMPIRE10, we decided to conduct experiments on three additional datasets that have more conveniently available ground truth information. This includes two additional lung CT datasets with landmark annotations, NLST and DirLab-4DCT, and an inter-patient abdominal CT dataset, AbdomenCT.

The results are reported in the new Table 2 of the revised manuscript and reproduced below. Values are mean (std); TRE is in mm, measured on manually identified landmarks TRE(LM) and automatically generated keypoints TRE(KP).

MethodNLST TRE (LM) ↓NLST TRE (KP) ↓NLST Neg. JD (%) ↓DirLab-4DCT TRE (LM) ↓DirLab-4DCT Neg. JD (%) ↓AbdomenCT Dice ↑AbdomenCT Neg. JD (%) ↓
Initial (φ=Id\varphi = \mathrm{Id})10.223 (2.181)9.703 (2.202)0.0 (0.0)8.461 (3.160)0.0 (0.0)0.2631 (7.3e-2)0.0 (0.0)
NODEO4.345 (1.339)3.220 (1.472)7.8e-3 (7.9e-3)2.675 (2.175)6.9e-5 (1.2e-4)0.4642 (8.0e-2)9.1e-2 (5.5e-2)
LCPReg-opt (Ours)4.048 (1.175)2.631 (1.209)4.8e-3 (5.3e-3)2.184 (1.695)2.4e-6 (7.5e-6)0.4646 (8.3e-2)8.8e-2 (5.7e-2)
convexAdam-MIND4.133 (1.926)2.785 (1.848)5.9e-1 (5.7e-1)2.852 (2.786)1.4e-2 (1.6e-2)0.3684 (9.9e-2)4.2e-1 (3.7e-1)
LCPReg-conv (Ours)3.547 (1.726)2.128 (1.586)2.5e-1 (2.8e-1)2.279 (2.144)3.1e-3 (8.9e-3)0.3862 (1.1e-1)4.4e-1 (3.4e-1)
uniGradICON (w/ IO)3.391 (0.660)2.070 (0.452)1.4e-1 (2.3e-1)1.718 (0.572)6.0e-4 (1.0e-3)0.5140 (6.5e-2)9.7e-1 (1.7e-1)
LCPReg-uGI (Ours)3.024 (0.715)1.756 (0.600)8.2e-2 (1.5e-1)1.697 (0.841)4.0e-5 (1.0e-4)0.5153 (7.1e-2)9.2e-1 (1.5e-1)

Bold marks the better value within each baseline/LCP pair.

On NLST and DirLab-4DCT, replacing the original representation with LCP lowers both the landmark TRE and the percentage of negative Jacobian determinants in all six framework–dataset combinations. On AbdomenCT, Dice improves in all three integrations, most clearly for LCPReg-conv (+0.018). As with the original three datasets, additional regularity metrics are reported in Tables 7 and 8 of the appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.