- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes ENF-SSM for anatomical statistical shape modeling. It builds on the SmS/FUSS architecture and trains a shape interpolant with an elastic deformation energy. The model supports template estimation, endpoint inversion, deformation transfer, and path regression. Experiments use pancreas, hippocampus, and heart shapes.
Critical strengths
The paper reports competitive reconstruction results across three datasets.
Critical weaknesses
- Differences between the paper and code, and poor presentation. Algorithm 2 uses a midpoint on the learned path, while the code uses a coordinate average. Appendix A.6 warns that these two midpoints are not always the same and should not be used interchangeably without an error analysis, yet the code still makes this substitution.
For template estimation, Algorithm 1 uses a decreasing weight, 1/(i+1). However, in the code, direct_initial ignores the factor argument passed by ENDE_karcher_mean and instead uses the fixed weight 1/(pose_timestep+1). This does not follow the stated update rule, and it is unclear whether this function was used to produce Table 3.
Additionally, the formulation does not clearly explain how the objectives, assumptions, and update rules are connected. None of the figures in the paper are vector graphics, and their text becomes less clear at high zoom levels.
These issues make the method difficult to evaluate and leave it unclear whether the reported results support the stated algorithms.
- Insufficient evidence for the claimed benefits. The endpoint-inversion test in Table 4 checks consistency within ENF, not accuracy against an independent reference. Table 6 shows improved fitting after correction, but lacks a comparison with directly optimizing both endpoints. The benefit and costs of the proposed operations relative to SmS/FUSS remain unclear. These gaps limit the support for the claimed advantages.
Questions for authors
Which code produced tables? Was Table 3 generated using the function that ignores the argument factor? Are the trend and results the same when the implementation follows the stated midpoint and update rules in the paper?
What are the gains and costs remaining when using SmS/FUSS on the same downstream tasks?
Why was SmS evaluated without CD-based test-time adaptation? Why are FlowSSM and Point2SSM++ included in Table 1 but not Table 2?
Why are so many entries in Table 4 marked with dashes (--)? Do these entries mean that the methods cannot be applied, or that the settings were not tested?
Does your correction rule give better fitting results or run faster than directly optimizing both endpoints? Appendix A.5 gives the gradients for this optimization, so why does the main text say they are too hard to compute?
Were all data from each subject kept in the same split? How many subjects were used for later-visit prediction, and were they excluded from ENF training?
3 comments
Response to Reviewer boGR on W2
Response to Table 6 shows improved fitting after correction, but lacks a comparison with directly optimizing both endpoints.:
To evaluate accuracy against an independent reference, in the revised manuscript we add Appendix A.15 (also required by Reviewer ZMGV and Reviewer VyXc, in red) that compares the learned logarithm of the trained ENF with a numerical logarithm obtained by minimizing the same GESM path energy. Correspondence is taken from the ENF-SSM matching module, and the numerical path is computed with the correspondence-based ESA solver (Dirichlet energy used for consistency).On the pancreas dataset the first shape is fixed as source and the remaining shapes are corresponded to it. For each pair we solvewith fixed endpoints and (matching the setting). The numerical logarithm is the first-step displacement of the optimized path; the learned logarithm is the initial field of the frozen ENF.On 272 pairs the mean error between the learned and numerical logarithms is 2.685. All tested pairs satisfy the error bound of Proposition 1 and the remainder condition . A step-size sweep shows that the remainder decreases monotonically with and stays below the threshold (e.g., at , remainder ).
These results confirm that the trained ENF logarithm approximates the numerical logarithm under the identical GESM energy, providing the independent reference check requested for Table 4.
Response to The endpoint-inversion test in Table 4 checks consistency within ENF, not accuracy against an independent reference:
The SmS architecture used to build the neural interpolant in ENF-SSM fixes the source shape by design, so the current framework does not directly optimize the starting point. Other Riemannian geodesic-regression methods (DCM, FCM, LDDMM, Geomstats) can optimize both endpoints simultaneously; however, as shown in Table 6 they require substantially longer runtimes and still yield higher errors and worse deformation metrics than ENF-SSM.
Directly optimizing Eq. (14) in Euclidean without the neural path primitive reduces to ordinary linear SSM. Following Reviewer VyXc’s request we have added these closed-form Euclidean baselines (both with fixed and free endpoints). They finish in <1 s but produce higher errors and larger deformation measures than ENF-SSM. While fast, they are linear methods and fall outside the Riemannian setting that is the focus of this work.
Thus the rectification step remains a practical improvement within the amortized Rm-SSM pipeline: it improves fit over the non-rectified ENF-SSM while staying far more efficient than iterative Riemannian solvers that optimize both endpoints.
Response to The benefit and costs of the proposed operations relative to SmS/FUSS remain unclear.:
The benefit of the proposed GESM energy relative to the original SmS/FUSS losses is quantified in the new ablation (Section 6.2, Table 8). We train the identical SmS architecture with the original objectives (SmS with ARAP, FUSS with Chamfer, and FUSS without the Chamfer term) versus our GESM energy, then evaluate the resulting networks on endpoint inversion because it is the basic Rm-SSM operation that underlies subsequent transport and regression.
As shown below, the GESM-trained model (ENF-SSM) consistently outperforms the vanilla-loss variants on both self-consistency and later-visit volume error:
Method c=1/7 c=2/7 c=3/7 c=4/7 c=5/7 c=6/7 c=1.0 VD ↓ FUSS (CD) 5.65±0.36 3.84±0.25 2.31±0.19 1.47±0.21 1.31±0.22 1.27±0.18 1.26±0.15 0.232±0.153 SmS (ARAP) 3.25±0.26 0.79±0.14 0.23±0.09 0.16±0.05 0.13±0.03 0.12±0.03 0.11±0.02 0.142±0.080 FUSS w/o CD 2.43±0.14 0.52±0.13 0.16±0.02 0.13±0.02 0.11±0.01 0.09±0.01 0.08±0.01 0.113±0.058 ENF-SSM (GESM) 2.25±0.15 0.41±0.10 0.11±0.01 0.09±0.01 0.07±0.01 0.06±0.01 0.05±0.01 0.095±0.061
Because all variants use the same frozen-network evaluation, the online computational cost remains essentially identical. The accuracy gains therefore come from the improved path primitive produced by the GESM energy. This demonstrates that the elastic training objective yields a measurably better amortized operator for the Rm-SSM pipeline.
Once trained, online costs of the proposed Rm-SSM operators are reported throughout Tables 3–6 and are substantially lower than repeated numerical solves after the one-time training.
Response to Reviewer boGR on Questions
Reponse to Q1: Please refer to Response to current inconsistency of codes and statements part in the response to W1. We claim that ENF-SSM results in the table and figures are produced by the codes in the current anonymous release.
Reponse to Q2: Please refer to Response to The benefit and costs of the proposed operations relative to SmS/FUSS remain unclear part in the reponses to W2.
Reponse to Q3:
Why no test-time adaption in SmS: We evaluated SmS without the Chamfer-distance test-time adaptation because, on anatomical shapes, this adaptation (as explored in FUSS) tends to collapse intermediate frames and degrade path quality. The same observation was first noted in the FUSS paper; we therefore report the plain SmS baseline for a cleaner comparison of the training objectives.
Why are FlowSSM and Point2SSM++ included in Table 1 but not Table 2: Table 1 assesses reconstruction / correspondence quality, where FlowSSM and Point2SSM++ are relevant learning-based baselines. Table 2 evaluates geometric properties of the interpolation paths (incremental strain, curvature, etc.). FlowSSM and Point2SSM++ are designed as shape reconstruction methods, and do not produce an explicit time-parameterized path under the same discrete elastic energy, so the path-wise descriptors of Table 2 are not directly applicable; they are therefore omitted from that table.
Reponse to Q4: The dashes (--) in Table 4 indicate that the corresponding methods cannot be applied to those experimental settings, not that the settings were left untested. The column corresponds to the standard initial-value problem solvers supplied by LDDMM, ESA, DCM and FCM. For the other intermediate values of c, these methods do not offer a corresponding solver, so the entries are left blank. ENF-SSM can be evaluated directly at every by a single forward pass. This is now clarified in the table caption.
Reponse to Q5: According to the Response to The endpoint-inversion test in Table 4 checks consistency within ENF, not accuracy against an independent reference in the response to W2, compared with the Rm-SSM geodesic regression methods that can directly optimize both endpoints (DCM, FCM, LDDMM, and Geomstats), ENF-SSM using correction rule produces better results and runs faster. Compared with Linear SMM regression optimizing both endpoints with a closed-form solution, ENF-SSM produces better results. However, our focus is Rm-SSM and thus linear SSM methods with the same purpose fall outside the setting.
Gradient issue of Q5: Gradients in Eqn (31) is gradient of Eqn (14) (path regression in ENF-SSM without rectification). Its basic components, such as and , do not have explicit formulations and are obtained by differentiation of the frozen network in a smooth local chart when using SmS architecture as a path primitive for Rm-SSM. This is the reason why we call it hard to compute and is used to explain our intuition for better understanding.
Reponse to Q6: The 80/10/10 split is used only for the unsupervised reconstruction experiment that trains the interpolant. The later-visit prediction task (Table 4) is a separate downstream evaluation: the already-trained interpolant is frozen and used to solve an initial-value problem.
Subjects with at least three imaging visits are included (Table 11 lists the counts: 410 subjects with 3 visits, 5 with 4, 7 with 5, and 54 with 6). These subjects are not required to be excluded from interpolant training, because the training objective of neural interpolant only concerns pairwise interpolation, while the prediction task queries the frozen operator to solve an initial-value-type problem. Overlap therefore does not constitute leakage for the initial-value setting.
Response to Reviewer boGR on W1
We thank the reviewer for the careful examination of the code and the detailed comparison with the paper, and for helpful suggestions on the presentations of our work. The comments on the implementation details allowed us to identify and correct several inconsistencies during the revision stage, and to clarify the intended algorithms more precisely. We address each point below.
Response to current inconsistency of codes and statements:
(a) Midpoint issue: In the revised manuscript, we explicitly claim that we adopt the Euclidean (coordinate) midpoint in Schild’s ladder, consistent with the practice in ESA (Hartman et al., 2023)(Line 29, https://github.com/emmanuel-hartman/H2_SurfaceMatch/blob/main/H2_deformation_transfer.py), a common efficient realization of parallel transport. In the revised manuscript, we changed the updating rule of Algorithm 2 into M←(Bj+S)/2 to keep consistent with the code.
The explanation of coordinate midpoint and geodesic midpoint in the Appendix is due to some open-source Rm-SSM toolkits that operate with sparse control points (e.g., Deformetrica) or are designed around specific metrics for efficiency (e.g., Morphomatics, Geomstats) employing the more rigorous manifold midpoint. Although we adopt the coordinate midpoint of ESA for the same elastic energy and the same efficiency purpose, we discuss the intrinsic difference between the two kinds of midpoints in the paper for compactness. In the revised paper, we revised the technical parts of Deformation transfer in Sec 4.2 and Appendix A.6 (in red) for better presentation. The reported deformation transfer results of our methods are all from coordinate midpoint.
(b) Factor issue: The results in Table 3 were generated with the decreasing weight 1/(i+1) stated in the paper, consistent with the discrete geodesic calculus approach (Hartman et al., 2023), and the theoretical analysis of decreasing weight 1/(i+1) is provided in (Ho et al., 2013) . When preparing the anonymous release, we accidentally introduced an incorrect version of this function (fixed weight) due to the tight timeline. This was our mistake in the code dump.
In addition, the fixed-weight version could not have produced the reported results. On the pancreas cohort (N=273), holding the weight fixed at 1/(pose_step+1) with pose_step=6 would keep a large step size throughout the iterations and would essentially fail to converge, which is inconsistent with the stable templates we obtained. The actual runs used the 1/(i+1) schedule.
Response to presentation of the manuscript:
In the revised manuscript, we adopted the suggestions of Reviewer ZMGV to emphasize the connection between our work and the discrete geodesic calculus framework (Rumpf & Writh, 2015), and to position the proposed framework as a neural network-based implementation of operators in discrete geodesic calculus.
Hence, in the revised manuscript, for each Rm-SSM task, we first give a general introduction to each operator with its clinical application at Operator tasks part of Sec. 3 Problem formulations, and then show its mathematical formulation and optimization procedure with additional motivations and its forms in discrete geodesic calculus. These parts are in blue in the main text of the revised manuscript. The assumption for the bound proof of each Rm-SSM operator's property is numerically verified in the Appendix accordingly, as required by Reviewer 1 and Reviewer 2. These parts are in red in the Appendix of the revised manuscript.
We believe the presentation of the manuscript are substantially improved by the above revisions. Also, we used the vector graphics of the involved figures for better visualization in the revised manuscript.
Reference:
Hartman, Emmanuel, et al. "Elastic shape analysis of surfaces with second-order sobolev metrics: a comprehensive numerical framework." International Journal of Computer Vision 131.5 (2023): 1183-1209.
Ho, Jeffrey, et al. "Recursive Karcher expectation estimators and geometric law of large numbers." Artificial intelligence and statistics. PMLR, 2013.
Rumpf, Martin, and Benedikt Wirth. "Variational time discretization of geodesic calculus." IMA Journal of Numerical Analysis 35.3 (2015): 1011-1046.