- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces MeshLoom, a feed-forward network for non-rigid registration of mesh sequences whose vertex count and connectivity vary across frames. The key idea is to carry the anchor mesh's connectivity into the per-vertex features through multi-hop aggregation, so that the encoder can tell surface regions that are close in space but far apart on the surface. These features are combined with per-frame 3D shape latents from a pretrained model and optional image features, and a time-conditioned decoder regresses per-vertex deformations with separate global-translation and local-residual branches. Intermediate-frame supervision lets the decoder be queried at timestamps the encoder never saw, which the paper uses for motion interpolation and morphing. The model is trained on about 30K TexVerse animations and evaluated on ActionBench and DT4D under 2-, 4- and 8-frame inputs, and on real 4D-DRESS scans at 4 frames, reporting state-of-the-art average correspondence accuracy, a large reduction in near-contact failures, and an inference time of a few seconds.
Critical strengths
-
The topology-aware representation is a well-motivated idea, and the paper tests it in the setting where it matters most. Vertex entanglement is the failure mode that point-based encoders actually hit in near-contact situations, and propagating the anchor mesh's connectivity into the per-vertex features is a clean way to separate "close in space" from "close on the surface". The dedicated near-contact experiment supports this directly: the error in the affected regions drops by about 40%, the fraction of entangled pairs falls from about a quarter to a few percent, and the error maps show a ubstantial reduction in this failure mode. The experiment is well matched to the claim it is meant to support.
-
The framework is practical in a way that matters for this problem. Registering variable-topology sequences in a single feed-forward pass, without per-instance optimization, is the right target, and the results on DT4D and on the real 4D-DRESS scans suggest that the prior learned on TexVerse transfers across data sources. The running time is well below that of the optimization-based alternatives, and the image-free fallback is a sensible choice for untextured assets.
Critical weaknesses
- I would like to see how much of the gain over ActionMesh comes from the proposed design rather than from the other changes that came with it. MeshLoom differs from ActionMesh in several ways at once: a new deformation branch, a different 3D backbone (TRELLIS.2 rather than TripoSG), and more than twice the training data. The ablations isolate the topology-aware representation very well, and the shape-latent prior is ablated properly, but the global-translation / local-residual split is only illustrated qualitatively in Fig. 4, and nothing is compared under a matched backbone and training set. Since the shape latent turns out to be the single most important ingredient in the authors' own ablation, I am left unsure whether the improvement over ActionMesh reflects the registration design or a different 3D prior and more training data. This matters for the paper's headline claim, which is about the framework as a whole rather than about the near-contact case alone. A quantitative ablation of the split, and a comparison under matched conditions if it is feasible, would clarify how much of this is architectural.
Questions for authors
Q1. Could you add a quantitative ablation of the global-translation / local-residual split, for instance a single-branch decoder under the same 4-frame protocol as Table 3? If a comparison against ActionMesh's deformation stage under a shared backbone and training set is feasible, that would make the attribution unambiguous.
Q2. Appendix G notes that ActionMesh benefits more from denser temporal observation, and DT4D at 8 frames is also where it overtakes MeshLoom (CD-Motion 0.0594 vs. 0.1003; NGE 0.0259 vs. 0.0379, Table 5). As far as I can tell, this is the only place in the paper where the behaviour with respect to input density is discussed, so I would like to understand it a little better: is the difference a property of the two designs, or of how each model was run at each input length - for instance, which ActionMesh Stage II checkpoint was used, and how 2- and 4-frame inputs were provided to it? This would help me judge how much of the averaged advantage in Table 1 reflects the method itself.
Q3. Do the reported inference times cover the full pipeline, including the per-frame latent extraction and full-vertex decoding, or only the registration network? A short note on what is being timed would he
3 comments
Response to Reviewer vg4t (2/2)
Reviewer’s comment - Q2: Appendix G notes that ActionMesh benefits more from denser temporal observation, and DT4D at 8 frames is also where it overtakes MeshLoom (CD-Motion 0.0594 vs. 0.1003; NGE 0.0259 vs. 0.0379, Table 5). As far as I can tell, this is the only place in the paper where the behaviour with respect to input density is discussed, so I would like to understand it a little better: is the difference a property of the two designs, or of how each model was run at each input length - for instance, which ActionMesh Stage II checkpoint was used, and how 2- and 4-frame inputs were provided to it? This would help me judge how much of the averaged advantage in Table 1 reflects the method itself.
We believe the difference comes neither from the two designs nor from how each model was run. For ActionMesh, we use the authors’ released checkpoint and strictly follow their inference setting, with only the number of frames changed. On ActionBench, which was introduced together with ActionMesh, ActionMesh does not perform better with longer and denser inputs: its CD-M changes smoothly with the number of input frames, as ours does. We therefore conjecture that ActionMesh’s sharp drop in CD-M on DT4D, from 0.1925 with 4 frames to 0.0594 with 8 frames (NGE behaves similarly), relates to its training strategy and to the DT4D dataset. To examine the behavior with respect to input density, we additionally evaluate 16 input frames on both benchmarks:
| Input frames | ActionBench: ActionMesh | ActionBench: Ours | DT4D: ActionMesh | DT4D: Ours |
|---|---|---|---|---|
| 2 | 0.1325 | 0.1245 | 0.1412 | 0.0867 |
| 4 | 0.1615 | 0.1524 | 0.1925 | 0.0951 |
| 8 | 0.1731 | 0.1642 | 0.0594 | 0.1003 |
| 16 | 0.1799 | 0.1698 | 0.0392 | 0.1124 |
| Average | 0.1618 | 0.1527 | 0.1081 | 0.0986 |
Table note: CD-M, lower is better. The 2-, 4-, and 8-frame results are taken from Tables 6 and 7, and the last row averages all four settings.
Averaged over all four settings, our method remains better on both benchmarks, consistent with Table 1. We have added these results to Appendix G of the revised manuscript.
Reviewer’s comment - Q3: Do the reported inference times cover the full pipeline, including the per-frame latent extraction and full-vertex decoding, or only the registration network? A short note on what is being timed would he
The inference time in Table 1 covers the full pipeline, including TRELLIS.2 shape encoding, DINOv3 image encoding, and our network’s encoding and decoding of all vertices.
We thank the reviewer again for the constructive suggestions. We hope the new results further strengthen the reviewer’s confidence in our work, and we would be happy to discuss any remaining questions.
Thanks for adding the matched comparison with ActionMesh and clarifying the timing. I have no further questions.
Response to Reviewer vg4t (1/2)
We sincerely thank the reviewer for the thoughtful review and for recognizing the contributions of our work.
Thanks for the question.
To verify that our advantage over ActionMesh comes from our architecture, we compare our design with ActionMesh's design under the same 3D backbone (TRELLIS.2), shape-latent prior, and training set. Our design outperforms ActionMesh's design on all six metrics, confirming that the gain comes from our architecture:
Table note: ActionBench, 4-frame setting, same protocol as Table 3. Both designs are trained for 100K iterations.
To validate the global-local splitting, we conduct an ablation study on 50 sequences with large global translations, comparing our decoder with a single-branch decoder. With the splitting, the mean dihedral-angle change drops from 21.6° to 8.0°, and the ratio of edges whose dihedral angle changes by more than 30° drops from 21.3% to 6.1%, showing far fewer high-frequency surface artifacts:
Table note: 50 translation-only sequences with translations of 1 to 8 bounding-box diagonals. Under an exact translation, both values are 0.
We have added these experiments to Sec. 4.2 and Appendix H of the revised manuscript.
We hope these clarifications address the reviewer’s concern about the contribution of our architectural design.
(Continued in the next comment.)