acceptodds
Under review as a conference paper at ICLR 2027

MeshLoom: Feed-Forward Non-Rigid Registration of Mesh Sequences

Abstract

We present MeshLoom, a feed-forward registration network that directly reconstructs vertex deformations across mesh sequences. Our approach advances non-rigid registration beyond existing models, which are typically constrained by costly per-instance optimization, narrow object categories, pairwise-only inputs, or merely intermediate outputs. The network is simple and efficient, registering multiple meshes within seconds. At its core lies a topology-aware encoder–decoder design. Specifically, we first introduce a topology-aware point representation that encodes the anchor (reference) mesh's topology into its per-vertex features. This representation strengthens the network's understanding of the anchor-mesh geometry and disambiguates points that are Euclidean-close yet geodesically distant. We then propose a multi-modal encoder that fuses this anchor-mesh representation with complementary cues from each frame, such as shape latents and image features. These multi-source signals are compressed into a compact global motion embedding that captures dense inter-frame correspondence. A lightweight decoder then queries this global embedding with the anchor-mesh point representation, retrieving per-vertex deformations at target timestamps. Through extensive experiments across diverse motions and object categories, we show that MeshLoom achieves state-of-the-art results on non-rigid registration. In addition, the decoder can be queried at intermediate timestamps, which extends MeshLoom to motion interpolation and mesh morphing.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Non-rigid mesh registration, dynamic meshes, dense correspondence, mesh deformation, motion interpolation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 5 reviews, above 85% of the others.
Reviewer_vg4tOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MeshLoom, a feed-forward network for non-rigid registration of mesh sequences whose vertex count and connectivity vary across frames. The key idea is to carry the anchor mesh's connectivity into the per-vertex features through multi-hop aggregation, so that the encoder can tell surface regions that are close in space but far apart on the surface. These features are combined with per-frame 3D shape latents from a pretrained model and optional image features, and a time-conditioned decoder regresses per-vertex deformations with separate global-translation and local-residual branches. Intermediate-frame supervision lets the decoder be queried at timestamps the encoder never saw, which the paper uses for motion interpolation and morphing. The model is trained on about 30K TexVerse animations and evaluated on ActionBench and DT4D under 2-, 4- and 8-frame inputs, and on real 4D-DRESS scans at 4 frames, reporting state-of-the-art average correspondence accuracy, a large reduction in near-contact failures, and an inference time of a few seconds.

Critical strengths
  1. The topology-aware representation is a well-motivated idea, and the paper tests it in the setting where it matters most. Vertex entanglement is the failure mode that point-based encoders actually hit in near-contact situations, and propagating the anchor mesh's connectivity into the per-vertex features is a clean way to separate "close in space" from "close on the surface". The dedicated near-contact experiment supports this directly: the error in the affected regions drops by about 40%, the fraction of entangled pairs falls from about a quarter to a few percent, and the error maps show a ubstantial reduction in this failure mode. The experiment is well matched to the claim it is meant to support.

  2. The framework is practical in a way that matters for this problem. Registering variable-topology sequences in a single feed-forward pass, without per-instance optimization, is the right target, and the results on DT4D and on the real 4D-DRESS scans suggest that the prior learned on TexVerse transfers across data sources. The running time is well below that of the optimization-based alternatives, and the image-free fallback is a sensible choice for untextured assets.

Critical weaknesses
  1. I would like to see how much of the gain over ActionMesh comes from the proposed design rather than from the other changes that came with it. MeshLoom differs from ActionMesh in several ways at once: a new deformation branch, a different 3D backbone (TRELLIS.2 rather than TripoSG), and more than twice the training data. The ablations isolate the topology-aware representation very well, and the shape-latent prior is ablated properly, but the global-translation / local-residual split is only illustrated qualitatively in Fig. 4, and nothing is compared under a matched backbone and training set. Since the shape latent turns out to be the single most important ingredient in the authors' own ablation, I am left unsure whether the improvement over ActionMesh reflects the registration design or a different 3D prior and more training data. This matters for the paper's headline claim, which is about the framework as a whole rather than about the near-contact case alone. A quantitative ablation of the split, and a comparison under matched conditions if it is feasible, would clarify how much of this is architectural.
Questions for authors

Q1. Could you add a quantitative ablation of the global-translation / local-residual split, for instance a single-branch decoder under the same 4-frame protocol as Table 3? If a comparison against ActionMesh's deformation stage under a shared backbone and training set is feasible, that would make the attribution unambiguous.

Q2. Appendix G notes that ActionMesh benefits more from denser temporal observation, and DT4D at 8 frames is also where it overtakes MeshLoom (CD-Motion 0.0594 vs. 0.1003; NGE 0.0259 vs. 0.0379, Table 5). As far as I can tell, this is the only place in the paper where the behaviour with respect to input density is discussed, so I would like to understand it a little better: is the difference a property of the two designs, or of how each model was run at each input length - for instance, which ActionMesh Stage II checkpoint was used, and how 2- and 4-frame inputs were provided to it? This would help me judge how much of the averaged advantage in Table 1 reflects the method itself.

Q3. Do the reported inference times cover the full pipeline, including the per-frame latent extraction and full-vertex decoding, or only the registration network? A short note on what is being timed would he

3 comments
AuthorsOpenReview ↗

Response to Reviewer vg4t (1/2)

We sincerely thank the reviewer for the thoughtful review and for recognizing the contributions of our work.


Reviewer’s comment - W1 & Q1: I would like to see how much of the gain over ActionMesh comes from the proposed design rather than from the other changes that came with it. MeshLoom differs from ActionMesh in several ways at once: a new deformation branch, a different 3D backbone (TRELLIS.2 rather than TripoSG), and more than twice the training data. The ablations isolate the topology-aware representation very well, and the shape-latent prior is ablated properly, but the global-translation / local-residual split is only illustrated qualitatively in Fig. 4, and nothing is compared under a matched backbone and training set. Since the shape latent turns out to be the single most important ingredient in the authors' own ablation, I am left unsure whether the improvement over ActionMesh reflects the registration design or a different 3D prior and more training data. This matters for the paper's headline claim, which is about the framework as a whole rather than about the near-contact case alone. A quantitative ablation of the split, and a comparison under matched conditions if it is feasible, would clarify how much of this is architectural. Could you add a quantitative ablation of the global-translation / local-residual split, for instance a single-branch decoder under the same 4-frame protocol as Table 3? If a comparison against ActionMesh's deformation stage under a shared backbone and training set is feasible, that would make the attribution unambiguous.

Thanks for the question.

To verify that our advantage over ActionMesh comes from our architecture, we compare our design with ActionMesh's design under the same 3D backbone (TRELLIS.2), shape-latent prior, and training set. Our design outperforms ActionMesh's design on all six metrics, confirming that the gain comes from our architecture:

DesignCD-3D ↓CD-4D ↓CD-M ↓CLIP ↑LPIPS ↓DreamSim ↓
ActionMesh's design0.06080.08920.16520.95790.06720.0446
Our design0.05730.08530.15390.97530.04830.0231

Table note: ActionBench, 4-frame setting, same protocol as Table 3. Both designs are trained for 100K iterations.

To validate the global-local splitting, we conduct an ablation study on 50 sequences with large global translations, comparing our decoder with a single-branch decoder. With the splitting, the mean dihedral-angle change drops from 21.6° to 8.0°, and the ratio of edges whose dihedral angle changes by more than 30° drops from 21.3% to 6.1%, showing far fewer high-frequency surface artifacts:

DecoderMean dihedral-angle change (°) ↓Edges changed by more than 30° (%) ↓
Single-branch21.621.3
Ours (global-local)8.06.1

Table note: 50 translation-only sequences with translations of 1 to 8 bounding-box diagonals. Under an exact translation, both values are 0.

We have added these experiments to Sec. 4.2 and Appendix H of the revised manuscript.

We hope these clarifications address the reviewer’s concern about the contribution of our architectural design.

(Continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer vg4t (2/2)

Reviewer’s comment - Q2: Appendix G notes that ActionMesh benefits more from denser temporal observation, and DT4D at 8 frames is also where it overtakes MeshLoom (CD-Motion 0.0594 vs. 0.1003; NGE 0.0259 vs. 0.0379, Table 5). As far as I can tell, this is the only place in the paper where the behaviour with respect to input density is discussed, so I would like to understand it a little better: is the difference a property of the two designs, or of how each model was run at each input length - for instance, which ActionMesh Stage II checkpoint was used, and how 2- and 4-frame inputs were provided to it? This would help me judge how much of the averaged advantage in Table 1 reflects the method itself.

We believe the difference comes neither from the two designs nor from how each model was run. For ActionMesh, we use the authors’ released checkpoint and strictly follow their inference setting, with only the number of frames changed. On ActionBench, which was introduced together with ActionMesh, ActionMesh does not perform better with longer and denser inputs: its CD-M changes smoothly with the number of input frames, as ours does. We therefore conjecture that ActionMesh’s sharp drop in CD-M on DT4D, from 0.1925 with 4 frames to 0.0594 with 8 frames (NGE behaves similarly), relates to its training strategy and to the DT4D dataset. To examine the behavior with respect to input density, we additionally evaluate 16 input frames on both benchmarks:

Input framesActionBench: ActionMeshActionBench: OursDT4D: ActionMeshDT4D: Ours
20.13250.12450.14120.0867
40.16150.15240.19250.0951
80.17310.16420.05940.1003
160.17990.16980.03920.1124
Average0.16180.15270.10810.0986

Table note: CD-M, lower is better. The 2-, 4-, and 8-frame results are taken from Tables 6 and 7, and the last row averages all four settings.

Averaged over all four settings, our method remains better on both benchmarks, consistent with Table 1. We have added these results to Appendix G of the revised manuscript.

Reviewer’s comment - Q3: Do the reported inference times cover the full pipeline, including the per-frame latent extraction and full-vertex decoding, or only the registration network? A short note on what is being timed would he

The inference time in Table 1 covers the full pipeline, including TRELLIS.2 shape encoding, DINOv3 image encoding, and our network’s encoding and decoding of all vertices.


We thank the reviewer again for the constructive suggestions. We hope the new results further strengthen the reviewer’s confidence in our work, and we would be happy to discuss any remaining questions.

Thanks for adding the matched comparison with ActionMesh and clarifying the timing. I have no further questions.

Reviewer_jMsaOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposed a feed-forward non-rigid registration of mesh sequence. Experiments have been conducted to evaluate the performance or proposed technique.

Critical strengths

Table 1 and 2 shows that the proposed technique outperforms the others.

Critical weaknesses

Nil

Questions for authors

Nil.

1 comment
AuthorsOpenReview ↗

Response to Reviewer jMsa

We sincerely thank the reviewer for the positive assessment and for recognizing that MeshLoom outperforms the compared methods. We would be happy to answer any further questions.

Reviewer_Vpe2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a feed-forward non-rigid registration framework that directly reconstructs deformable points across mesh sequences. It designs a topologically meaningful point representation, encoding the mesh topology to better handle the problem of adjacent points in Euclidean space but with large geodesic distances. Secondly, it proposes a multimodal encoder, utilizing Trellis 2.0 and DINOv3 as 3D and 2D priors combined with a transformer architecture for better registration.

Critical strengths
  • Utilizing multimodal priors is helpful in establishing potential correspondences, which is reasonable;

  • The experiments are quite comprehensive, and the numerical and visualization results demonstrate that the proposed method outperforms existing methods.

Critical weaknesses
  • The application scope of this problem setting appears more limited compared to other non-rigid registration methods, and the actual difficulty of the problems it can handle is still unclear.

    1. Common problems encountered in non-rigid registration include sensor noise, outliers, and partial overlap in the registration data. However, this paper sets the task as a clean and complete mesh sequence, manually constructing topological inconsistencies, resulting in a narrow application scope. Furthermore, the authors do not discuss whether the proposed method can handle such imperfect and common data. Although the authors also performed 4D-DRESS on real scan data, the inputs are the pre-processed high-quality meshes.

    2. For the complete and clean mesh processed in this paper, another type of shape matching method appears to perform well, such as [A][B]. Although some methods do not perform deformation, they seem to establish relatively good correspondences, and they can handle more cross-species cases. The proposed method does not demonstrate the ability to establish cross-species correspondences. The example in Figure 8(a) is an example of very similar shapes.

    3. The task setting of this paper uses a mesh sequence as input, which may further simplify the problem. If the action difference between two adjacent frames is small, establishing correspondences will be an easier task. Although the authors conducted tests with 2 frames, 4 frames, etc., they didn't specify which frames in the original sequence these frames represented, leaving the differences between shapes unclear, and whether they all behaved as shown in the individual visualizations. The paper should clarify which frames in the original sequence the 2-frame, 4-frame, and 8-frame settings refer to. This is crucial because it reflects the difficulty of the problem. The difficulty is significantly different if the 8-frame setting refers to frames 1 through 8 of the original sequence compared to frames 1, 11, 21, 31, ... 81.

  • The paper emphasizes its topological point representation. However, if the topology of the input frames is incorrect, it will be unprocessable. For example, if the table and drawer in Figure 3 are actually connected. This situation is quite common, whether for realistically captured shapes or 3D-generated shapes.

  • Some methods for reconstructing topologically consistent meshes from point cloud sequences [C, D] were not mentioned. It is recommended to compare the latest method [D], which appears to perform well and can also handle the task described in the paper.

  • For the registration of a continuous sequence, the continuity between frames is very important. The animation in the article does not look continuous enough between adjacent frames, and the frames in the interpolation example are not dense enough. Further sampling in time can provide a better visual experience and better verify the inter-frame continuity of the proposed method. If the paper mainly focuses on improvements to ActionMesh's Stage II, why not compare it with a fair sequence of 16 frames or even more, instead of using so few frames?

[A] Cao et al. Unsupervised Learning of Robust Spectral Shape Matching. TOG, 2023. [B] Niladri Shekhar Dutt et al. Diffusion 3D Features. CVPR 2024. [C] Yao et al. DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface Reconstruction. ECCV 2024. [D] Kaltheuner et al. Neu-PiG: Neural Preconditioned Grids for Fast Dynamic Surface Reconstruction on Long Sequences. CVPR 2026.

Questions for authors
  • Clarify the application scope of the proposed method and whether it can handle more challenging tasks;

  • Explain which frames in the original animation sequence the motion amplitude changes between frames 2, 4, and 8 in the experimental setup correspond to, or quantize the deformation amplitude, to clarify the difficulty of the experimental data;

  • Can the proposed method handle cases where the first frame has an incorrect topology?

  • Discuss and compare the proposed method with other topology-consistent mesh sequence reconstruction methods from point cloud sequences, such as Neu-PiG.

  • How is the inter-frame continuity of the registration results?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Vpe2 (1/2)

We thank the reviewer for the detailed review.


Reviewer’s comment - W1 & Q1 & Q2: The application scope of this problem setting appears more limited compared to other non-rigid registration methods, and the actual difficulty of the problems it can handle is still unclear.

Thanks for the question. We respond to each sub-point below.

W1-1) & Q1: Common problems encountered in non-rigid registration include sensor noise, outliers, and partial overlap in the registration data. … Clarify the application scope of the proposed method and whether it can handle more challenging tasks;

Our paper already evaluates MeshLoom on noisy inputs (ActionBench). Its meshes are generated by a 3D reconstruction model, and most of them contain multiple connected components and floating fragments.

To further test MeshLoom on imperfect inputs, we take 50 DT4D sequences and create variants by damaging the anchor mesh with remeshing, holes, cuts along curves, and non-manifold structures. We report the relative change in CD-M and NGE with respect to the clean anchor:

Anchor damageCD-MNGE
Remeshing0%+2%
Holes+3%+5%
Cuts along curves0%0%
Non-manifold−1%−1%

Table note: DT4D, 50 sequences, 4 frames.

We observe that all changes stay within ±5%, showing that our method is robust to these anchor defects. We have added the details and discussion to Appendix H of the revised manuscript.

W1-2): For the complete and clean mesh processed in this paper, another type of shape matching method appears to perform well, such as [A][B]. … The proposed method does not demonstrate the ability to establish cross-species correspondences. The example in Figure 8(a) is an example of very similar shapes.

We thank the reviewer for raising this point. MeshLoom does not target cross-species cases; it targets the registration of mesh sequences. Compared with shape-matching methods such as [A] and [B], MeshLoom generalizes to diverse object categories and is far more efficient. We have added cross-species correspondence to the limitations in Appendix J of the revised manuscript. We have also cited [A] and [B] in the related work.

W1-3) & Q2: … The paper should clarify which frames in the original sequence the 2-frame, 4-frame, and 8-frame settings refer to. … Explain which frames in the original animation sequence the motion amplitude changes between frames 2, 4, and 8 in the experimental setup correspond to, or quantize the deformation amplitude, to clarify the difficulty of the experimental data;

Thanks for the constructive suggestion. We sample 16 frames evenly from the first to the last frame of each sequence, and the 2-, 4-, and 8-frame settings take every 8th, 4th, and 2nd of them. For an n-frame sequence, adjacent input frames are thus about n/2, n/4, or n/8 frames apart. The input frames are therefore widely spread in time: on our evaluation datasets, they cover up to 84 original frames in the 2-frame setting and nearly 150 in the 4- and 8-frame settings. We have added these details to Appendix E of the revised manuscript.

Reviewer’s comment - W2 & Q3: The paper emphasizes its topological point representation. However, if the topology of the input frames is incorrect, it will be unprocessable. For example, if the table and drawer in Figure 3 are actually connected. This situation is quite common, whether for realistically captured shapes or 3D-generated shapes. Can the proposed method handle cases where the first frame has an incorrect topology?

Our topology-aware representation effectively alleviates vertex entanglement (Table 13). As with other registration methods, MeshLoom cannot handle an anchor mesh whose topology is itself incorrect. We discuss this limitation in Appendix J.

(Continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer Vpe2 (2/2)

Reviewer’s comment - W3 & Q4: Some methods for reconstructing topologically consistent meshes from point cloud sequences [C, D] were not mentioned. It is recommended to compare the latest method [D], which appears to perform well and can also handle the task described in the paper. Discuss and compare the proposed method with other topology-consistent mesh sequence reconstruction methods from point cloud sequences, such as Neu-PiG.

Thanks for the suggestion. We compare with Neu-PiG under the same setting on both ActionBench and DT4D and also report its runtime:

MethodActionBench CD-M ↓DT4D CD-M ↓DT4D NGE ↓Runtime (s) ↓
Neu-PiG0.1539 / 0.1737 / 0.17890.2290 / 0.2321 / 0.15880.0982 / 0.0949 / 0.082147.8
Ours0.1245 / 0.1524 / 0.16420.0867 / 0.0951 / 0.10030.0292 / 0.0356 / 0.03793.1 / 2.9

Table note: Each cell lists the 2-, 4-, and 8-frame settings. Runtime is measured under the protocol of Table 1; ours is given with image input (ActionBench) / without (DT4D).

The results show that MeshLoom also outperforms Neu-PiG in both accuracy and runtime. We have added this comparison to Appendix G and cited [C] and [D] in the related work of the revised manuscript.

Reviewer’s comment - W4 & Q5: For the registration of a continuous sequence, the continuity between frames is very important. The animation in the article does not look continuous enough between adjacent frames, and the frames in the interpolation example are not dense enough. Further sampling in time can provide a better visual experience and better verify the inter-frame continuity of the proposed method. If the paper mainly focuses on improvements to ActionMesh's Stage II, why not compare it with a fair sequence of 16 frames or even more, instead of using so few frames? How is the inter-frame continuity of the registration results?

Number of input frames. We focus on sparse inputs because larger gaps between frames make registration harder, and the 2-frame setting matches the pairwise protocol of most registration methods. Following the reviewer’s suggestion, we additionally evaluate 16 input frames on ActionBench:

Input framesActionMeshOurs
20.13250.1245
40.16150.1524
80.17310.1642
160.17990.1698
Average0.16180.1527

Table note: ActionBench, CD-M, lower is better. The 2-, 4-, and 8-frame results are taken from Table 6, and the last row averages all four settings.

On ActionBench, which was introduced together with ActionMesh, our method remains better with 16 frames, and both methods change smoothly with the number of frames.

Inter-frame continuity. We quantify inter-frame continuity with the acceleration ratio, i.e., the mean vertex acceleration of the output divided by that of the ground truth, and compare with representative methods on DT4D:

Method8 frames16 frames
DPF1.402.17
OAReg1.171.94
ActionMesh0.940.96
Ours0.920.94

Table note: Median over the DT4D sequences. 1 means as smooth as the ground truth, and values above 1 indicate jitter.

As the table shows, our registered sequences stay close to the ground-truth smoothness, whereas DPF and OAReg show clear jitter that grows with 16 frames.

We have added these results to Appendices G and I of the revised manuscript.


We hope these clarifications and experiments address the reviewer’s concerns, and we would be grateful if the reviewer could reconsider the assessment in light of them. We would be happy to discuss any further questions.

Reviewer_Vpe2OpenReview ↗

Official Review by Reviewer Vpe2

Thank you for the detailed responses and extensive additional experiments, which have helped me better understand the proposed method.

The paper appears to focus primarily on mesh sequence registration. However, when more input frames are considered (e.g., 8 and 16 frames), the proposed method does not appear to offer a particularly significant advantage over ActionMesh, a closely related approach. In particular, on DT4D, MeshLoom exhibits substantially higher registration errors than ActionMesh under these settings, as shown in Table 9 of the revised manuscript.

Furthermore, the paper lacks sufficiently comprehensive video comparisons of complete mesh sequences (e.g., 16-frame sequences). Since sequence registration is a central focus of this work, I believe that evaluating and visualizing the temporal consistency of the registered meshes over longer sequences is particularly important, rather than focusing primarily on sparse settings with only 2 or 4 input frames, which are closer to conventional pairwise non-rigid registration tasks.

Given these remaining concerns, I have decided to maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer Vpe2's follow-up

We thank the reviewer for the follow-up.


Reviewer’s comment - Dense inputs on DT4D: When more input frames are considered (e.g., 8 and 16 frames), the proposed method does not appear to offer a particularly significant advantage over ActionMesh … on DT4D, MeshLoom exhibits substantially higher registration errors than ActionMesh under these settings, as shown in Table 9 of the revised manuscript.

ActionMesh’s results with 8 and 16 inputs on DT4D depart from its behavior on its own benchmark. On DT4D with imperfect target meshes, our method remains consistently better than ActionMesh with 2, 4, 8, and 16 inputs on all four metrics.

1. ActionMesh’s dense-input results on DT4D depart from its behavior on its own benchmark. ActionMesh is trained on data drawn from Objaverse, Objaverse-XL, and an undisclosed internal dataset, whose overlap with DT4D cannot be checked, and it reports no results on DT4D. On ActionBench, which was introduced together with ActionMesh, its CD-M changes smoothly with the number of inputs, whereas ours is lower across all settings from 2 to 16 inputs (Table 9). On DT4D, its CD-M instead drops sharply from 4 to 8 inputs, so these results do not reflect how ActionMesh generalizes and instead indicate overfitting. Also, DT4D has several properties that are statistically different from our training data (e.g., larger translation between frames), so we expect scaled-up training of our method (more comparable to the ActionMesh training dataset) to do even better.

2. On DT4D with imperfect target meshes, our method remains better in every setting. We corrupt the DT4D target meshes with holes, low-frequency distortions, vertex noise, and floating fragments, and evaluate against the ground truth. Our method outperforms ActionMesh on all four metrics with 2 to 16 inputs (also in Table 9, Appendix G):

MetricMethod24816
CD-M ↓ActionMesh0.22170.34390.29420.2988
Ours0.17290.25230.26430.2394
NGE ↓ActionMesh0.06810.10240.09640.1001
Ours0.06210.08920.09600.0917
CLIP ↑ActionMesh0.89440.87640.89900.9133
Ours0.95460.94320.93870.9414
DreamSim ↓ActionMesh0.10600.12850.10550.0930
Ours0.05560.07090.07610.0730

Reviewer’s comment - Complete sequences: The paper lacks sufficiently comprehensive video comparisons of complete mesh sequences (e.g., 16-frame sequences) … evaluating and visualizing the temporal consistency of the registered meshes over longer sequences is particularly important, rather than focusing primarily on sparse settings with only 2 or 4 input frames …

Fig. 10 (top, Appendix G) now visualizes complete 16-frame registration on DT4D: our results stay smooth and aligned with the ground truth across all frames. Quantitatively, over 8 and 16 frames, our registrations stay close to the ground-truth smoothness (median acceleration ratio 0.92 and 0.94, where 1 matches the ground truth), whereas DPF and OAReg add jitter (2.17 and 1.94 with 16 frames; Table 16, Appendix I). With imperfect targets (Fig. 10, bottom), ours also stay close to the ground truth, while ActionMesh shows visible artifacts (circled in red). We will add videos of complete 16-frame sequences to the final version.


We hope these results address the remaining concerns and would be grateful if they could be taken into account in the final assessment.

Reviewer_8hKuOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents MeshLoom, a feedforward non-rigid registration framework for heterogeneous mesh sequences, aiming to efficiently generate mesh sequences with consistent topology and vertex correspondence. This method first extracts anchor mesh features through a topology-aware graph convolutional network, then combines pre-trained 3D Shape Latent and optional image features, uses Transformer encoding to capture cross-frame motion information, and finally directly predicts vertex deformation through a temporally conditioned decoder. This framework not only supports mesh sequences of different lengths but also can generate intermediate deformations at unobserved timestamps. Experiments show that MeshLoom achieves good registration accuracy and inference efficiency on ActionBench, DeformingThings4D, and real-scanned dataset 4D-DRESS.

Critical strengths
  1. Uniformly process the grid sequences with different vertex numbers and connection relationships, effectively meeting the requirements of dynamic 3D content generation.
  2. Using single feedforward prediction, without the need for instance-specific optimization, the inference time reported in the paper is 3.1 seconds, superior to the main comparison methods.
  3. Includes multiple synthetic and real datasets, different input frame numbers, module ablation experiments, and motion interpolation evaluation, providing good experimental support.
Critical weaknesses
  1. The absence of key ablation experiments for removing cross-frame attention makes it impossible to fully prove that joint encoding is superior to independent frame-by-frame registration.
  2. Only the test results for the longest 8 frames are reported. The cross-frame global attention has quadratic complexity, and it is not clear whether the 3.1-second inference time includes pre-training feature extraction.
  3. The key improvements were mainly verified on 50 programmatic close-contact sequences, and DT4D uses anchor grids that retain the true topology, lacking tests on complex or damaged grids.
  4. The CD-4D on DT4D is 0.0419, significantly higher than DPF's 0.0103. At the same time, there is no systematic evaluation of geometric distortion indicators such as face flipping and self-intersection.
Questions for authors
  1. Can we conduct an ablation experiment to remove inter-frame self-attention and compare it with independent frame-by-frame registration to quantify the actual contribution of joint temporal modeling to CD-Motion and NGE?
  2. Does the reported 3.1-second inference time include TRELLIS.2, DINOv3 feature extraction, and the decoding process of all vertices? Can we further report the complete inference time and peak memory usage for different sequence lengths (such as 8, 16, 32 frames)?
  3. How does the model performance change when the anchor grid has irregular remeshing, holes, non-manifold structures or topological errors? Can we further verify the effectiveness of the topology-aware module in real close-contact scenarios?
  4. Although MeshLoom performs well in terms of corresponding accuracy, its geometric fitting error is still higher than some optimized methods. Can we further report indicators such as face flipping rate, self-intersection rate, etc., and analyze whether lightweight geometric refinement can improve the surface fitting quality while retaining the existing corresponding accuracy advantage?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 8hKu (1/2)

We sincerely thank the reviewer for the positive assessment and the constructive suggestions. We address each comment below. All revisions in the manuscript are highlighted in blue.


Reviewer’s comment - W1 & Q1: The absence of key ablation experiments for removing cross-frame attention makes it impossible to fully prove that joint encoding is superior to independent frame-by-frame registration. Can we conduct an ablation experiment to remove inter-frame self-attention and compare it with independent frame-by-frame registration to quantify the actual contribution of joint temporal modeling to CD-Motion and NGE?

Thanks for the suggestion. To quantify the contribution of joint temporal modeling, as suggested, we remove inter-frame attention by registering each frame independently. We compare this frame-by-frame variant with joint 4-frame and 8-frame inputs on DT4D and report the relative change in CD-Motion and NGE:

Input framesCD-MotionNGE
4−12.8%−12.3%
8−10.3%−11.8%

Table note: DT4D, all evaluation sequences. Change of joint registration relative to frame-by-frame registration with the same model (negative is better).

As shown in the table, joint temporal modeling brings better performance than frame-by-frame registration.

Reviewer’s comment - W2 & Q2: Only the test results for the longest 8 frames are reported. The cross-frame global attention has quadratic complexity, and it is not clear whether the 3.1-second inference time includes pre-training feature extraction. Does the reported 3.1-second inference time include TRELLIS.2, DINOv3 feature extraction, and the decoding process of all vertices? Can we further report the complete inference time and peak memory usage for different sequence lengths (such as 8, 16, 32 frames)?

The reported 3.1-second inference time includes TRELLIS.2 shape encoding, DINOv3 image encoding, and our network’s encoding and decoding of all vertices. The complete inference time and peak memory usage for different numbers of input frames are as follows:

Input framesTRELLIS.2 shape encodingDINOv3 image encodingNetwork encoding and decodingTotalPeak memory (GB)
20.350.050.360.767.9
40.700.110.671.487.9
81.420.241.483.148.1
162.850.464.137.4410.1
325.780.9113.4120.1016.1

Table note: Measured under the protocol of Table 1 (150K vertices and 300K faces per frame). Times are in seconds. With 2 and 4 frames, the peak memory comes from computing the topology-aware representation of the anchor mesh, which depends on the mesh size rather than the number of frames.

We have added this breakdown to Sec. 4.1 (Table 4) of the revised manuscript.

(Continued in the next comment.)

AuthorsOpenReview ↗

Response to Reviewer 8hKu (2/2)

Reviewer’s comment - W3 & Q3: The key improvements were mainly verified on 50 programmatic close-contact sequences, and DT4D uses anchor grids that retain the true topology, lacking tests on complex or damaged grids. How does the model performance change when the anchor grid has irregular remeshing, holes, non-manifold structures or topological errors? Can we further verify the effectiveness of the topology-aware module in real close-contact scenarios?

We address the two questions in turn.

How does the model performance change when the anchor grid has irregular remeshing, holes, non-manifold structures or topological errors?

As suggested, we take 50 DT4D sequences and create variants by damaging the anchor mesh with remeshing, holes, cuts along curves, and non-manifold structures. We report the relative change in CD-M and NGE with respect to the clean anchor:

Anchor damageCD-MNGE
Remeshing0%+2%
Holes+3%+5%
Cuts along curves0%0%
Non-manifold−1%−1%

Table note: DT4D, 50 sequences, 4 frames.

We observe that all changes stay within ±5%, showing that our method is robust to these anchor defects. We have added the details and discussion to Appendix H of the revised manuscript.

Can we further verify the effectiveness of the topology-aware module in real close-contact scenarios?

We select another 50 DT4D sequences that contain close-contact frames. Following the protocol used for the synthetic sequences in Appendix H, we compare our model with the variant without the topology-aware representation:

VariantVertex error ↓ (all)Vertex error ↓ (near contact)Unseparated pairs (%) ↓NGE ↓
w/o topology-aware representation14.2416.3076.40.1381
Ours6.878.7142.30.0785

The results confirm that the topology-aware representation effectively alleviates vertex entanglement in real close-contact scenarios. We have added this experiment with more visual results to Appendix H of the revised manuscript.

Reviewer’s comment - W4 & Q4: The CD-4D on DT4D is 0.0419, significantly higher than DPF's 0.0103. At the same time, there is no systematic evaluation of geometric distortion indicators such as face flipping and self-intersection. Although MeshLoom performs well in terms of corresponding accuracy, its geometric fitting error is still higher than some optimized methods. Can we further report indicators such as face flipping rate, self-intersection rate, etc., and analyze whether lightweight geometric refinement can improve the surface fitting quality while retaining the existing corresponding accuracy advantage?

We add a lightweight optimization-based refinement step similar to DPF. This refinement improves the surface fitting quality of our method while retaining our existing correspondence accuracy advantage. In particular, it improves our CD-4D from 0.0419 to 0.0080, better than DPF's 0.0103:

MethodCD-4D ↓CD-M ↓NGE ↓
DPF0.01030.23970.1006
Ours + refinement0.00800.07430.0344

Table note: DT4D, averaged over the 2-, 4-, and 8-frame settings as in Table 1.

We have added this experiment to Appendix G of the revised manuscript.

In addition, we further report the face flipping rate, self-intersection rate, edge-length distortion, and area distortion in the table below:

MethodFace flipping ↓Self-intersection ↓Edge-length distortion ↓Area distortion ↓
Ground truth09.500
ActionMesh11.620.1124.1956.9
DPF14.124.343.081.2
Ours4.513.116.431.1

Table note: DT4D, averaged over the 2-, 4-, and 8-frame settings, in %. The ground-truth meshes are listed for reference.

From the results above, our method outperforms ActionMesh and DPF on all these indicators. We have added these metrics to Sec. 4.1 (Table 5) of the revised manuscript.


We thank the reviewer again for the supportive review. We hope the additional experiments further strengthen the paper, and we would be happy to discuss any remaining questions.

Reviewer_XmxsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MeshLoom, a feed-forward method for registering mesh sequences with varying topology by predicting deformations of a reference mesh across frames. It introduces a GCN-based encoder to incorporate mesh connectivity into reference vertex features, which are combined with pretrained 3D shape latents and optional image features through a temporal transformer. A time-conditioned decoder predicts global translation and local vertex deformations, supervised by vertex-level and geometry-aware losses. The method reports improved registration accuracy and efficiency over existing approaches and also supports motion interpolation.

Critical strengths
  1. Strong empirical performance and efficiency.

    The method achieves good registration results across multiple benchmarks, with notable improvements over ActionMesh, particularly on DT4D. It also reports substantially faster inference than ActionMesh, highlighting its practical value for mesh sequence registration.

  2. Effective handling of near-contact regions.

    The topology-aware encoder helps distinguish spatially close but topologically distant surface regions, reducing incorrect vertex movements. The dedicated near-contact ablation shows clear improvements in both vertex accuracy and the separation of nearby surfaces, demonstrating the practical value of incorporating mesh connectivity.

Critical weaknesses
  1. Limited novelty compared with ActionMesh.

    ActionMesh already uses a temporal 3D autoencoder that jointly processes per-frame 3D shape latents and directly predicts deformations of a reference mesh. It also identifies the ambiguity between spatially close but topologically distant vertices and uses surface normals to help distinguish them. MeshLoom builds on this pipeline by adding GCN-based mesh feature aggregation, separating global translation from local deformation, and introducing edge and Jacobian losses. These are natural extensions using established techniques rather than substantially new methodological ideas. While these modifications are effective, I find that they offer limited new insight beyond improving the existing pipeline.

  2. Insufficient evidence attributing the performance gains to the proposed contributions.

    MeshLoom reports clear improvements over ActionMesh, particularly on DT4D. However, the two methods differ in several important aspects beyond the proposed components, including their pretrained 3D representations and training data. Specifically, MeshLoom uses TRELLIS.2 shape latents and a larger training dataset, while ActionMesh uses TripoSG-based representations and a smaller animation dataset. Since these factors change alongside the network design, it is unclear how much of the improvement comes from the proposed components themselves. A more controlled comparison with ActionMesh under matched 3D representations and training conditions would help establish their independent contribution.

Questions for authors

Could the authors provide a more controlled comparison with ActionMesh, ideally using the same pretrained 3D representations and training data? Since the current comparison changes these factors alongside the proposed network components, it is unclear how much of the reported improvement can be attributed to the proposed method itself.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Xmxs

We thank the reviewer for the feedback. We address each comment below.


Reviewer’s comment - W1: Limited novelty compared with ActionMesh.
ActionMesh already uses a temporal 3D autoencoder that jointly processes per-frame 3D shape latents and directly predicts deformations of a reference mesh. It also identifies the ambiguity between spatially close but topologically distant vertices and uses surface normals to help distinguish them. MeshLoom builds on this pipeline by adding GCN-based mesh feature aggregation, separating global translation from local deformation, and introducing edge and Jacobian losses. These are natural extensions using established techniques rather than substantially new methodological ideas. While these modifications are effective, I find that they offer limited new insight beyond improving the existing pipeline.

We respectfully disagree. MeshLoom builds on a 3D representation fundamentally different from that of ActionMesh: the sparse-voxel latents of TRELLIS.2 rather than the vecset latents of TripoSG. The sparse-voxel latents of different frames differ in number and have no correspondence to each other, and how to predict mesh deformations from such latents has, to our knowledge, not been studied. We therefore design a new architecture for this setting, which performs better than ActionMesh’s design (see our response to W2 & Q1). The topology-aware representation, the global-local decoder, and the edge and Jacobian losses each address a concrete challenge that we observed in registration. With these designs, MeshLoom outperforms ActionMesh on all metrics in Table 1 and runs about 2.5 times faster.

Reviewer’s comment - W2 & Q1: Insufficient evidence attributing the performance gains to the proposed contributions.
MeshLoom reports clear improvements over ActionMesh, particularly on DT4D. However, the two methods differ in several important aspects beyond the proposed components, including their pretrained 3D representations and training data. Specifically, MeshLoom uses TRELLIS.2 shape latents and a larger training dataset, while ActionMesh uses TripoSG-based representations and a smaller animation dataset. Since these factors change alongside the network design, it is unclear how much of the improvement comes from the proposed components themselves. A more controlled comparison with ActionMesh under matched 3D representations and training conditions would help establish their independent contribution. Could the authors provide a more controlled comparison with ActionMesh, ideally using the same pretrained 3D representations and training data? Since the current comparison changes these factors alongside the proposed network components, it is unclear how much of the reported improvement can be attributed to the proposed method itself.

To establish the independent contribution of our design, we conduct a controlled comparison with ActionMesh’s design under matched conditions: both use the same pretrained 3D representation (TRELLIS.2 shape latents) and the same training data, and both are trained for 100K iterations. Our design outperforms ActionMesh’s design on all six metrics, so the improvement can be attributed to the proposed method itself:

DesignCD-3D ↓CD-4D ↓CD-M ↓CLIP ↑LPIPS ↓DreamSim ↓
ActionMesh’s design0.06080.08920.16520.95790.06720.0446
Our design0.05730.08530.15390.97530.04830.0231

Table note: ActionBench, 4-frame setting, same protocol as Table 3.


In the revised manuscript, we have incorporated the corresponding context, experiments, and discussion into Sec. 3.2, Sec. 4.2, and Appendix H. All revisions are highlighted in blue.

We hope these clarifications address the reviewer’s concerns, and we would be grateful if the reviewer could take them into account in the final assessment. We would be happy to discuss any further questions.

Thank you for the responses and additional experiments. The controlled comparison provides useful evidence that the proposed design contributes to the performance gains. However, my central concern about the significance of the methodological contribution remains.

ActionMesh already performs feed-forward mesh-sequence registration through reference-mesh deformation and explicitly identifies the ambiguity between spatially close but topologically distant points, using surface normals to alleviate it. MeshLoom’s topology-aware aggregation is useful, but I view it as a relatively straightforward extension. I also acknowledge that using sparse-voxel latents introduces a different architectural setting. Nevertheless, the encoder adaptation, global–local decomposition, and additional geometric losses, considered together, have not convinced me of a sufficiently substantial methodological advance beyond ActionMesh.

In my assessment, the paper’s framing and contribution statements still do not adequately distinguish these incremental improvements from capabilities already established by ActionMesh. The rebuttal strengthens the evidence of effectiveness, but does not sufficiently resolve my concern about the novelty and significance of the contribution.

I therefore maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer Xmxs's follow-up

We thank the reviewer for the follow-up and for recognizing the effectiveness and efficiency of MeshLoom.


MeshLoom advances feed-forward mesh-sequence registration in both accuracy and speed. Averaged over 2, 4, and 8 input frames, it is best on every metric on both benchmarks of Table 1, reducing CD-Motion and NGE by up to 28% and 26% relative to the strongest competitor, ActionMesh. It runs nearly twice as fast as the fastest baseline (3.1 s vs. 5.9 s) and 2.5× faster than ActionMesh. It also leads on real 4D-DRESS scans (Table 2) and, with every number of input frames, on imperfect target meshes (Table 9).

The core methodological advance is a registration architecture built for sparse-voxel latents. Sparse-voxel latents, such as those of TRELLIS.2, give each frame a different number of tokens, whereas ActionMesh is designed for vecset latents with a fixed number of tokens per frame, which it concatenates and processes with self-attention. MeshLoom is instead centered on the anchor. Compact anchor tokens query the latents of each frame through cross-attention, so the cost grows only linearly with the latent size, and a lightweight decoder queries the resulting motion embedding for every vertex at any timestamp. With the same latents and training data, this architecture outperforms ActionMesh’s design on all six metrics and nearly halves DreamSim (Tables 3 and 12).

The other components each solve a failure that we observed in registration. The topology-aware representation encodes the mesh connectivity, which ActionMesh does not use, and reduces the vertex error at near contacts by 42%. The global–local decoder keeps surfaces intact under large translations, reducing the fraction of edges whose dihedral angle changes by more than 30° from 21.3% to 6.1%. The edge and Jacobian losses preserve local structure (Sec. 4.2).

We have revised the abstract and introduction to state these advances explicitly, with the numbers above, and the related work now states that ActionMesh encodes the reference mesh only as points with normals (highlighted in blue).

Comments on the paper

1 comment
Area_Chair_RduFOpenReview ↗

Pending for your comments

Dear Reviewers,

The authors have posted their rebuttal. Please read it and reply in the discussion, covering:

Whether each of your questions and concerns has been addressed. Whether your rating and recommendation change, and why. Any remaining concern that must be settled before the meta-review. Best.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.