acceptodds
Under review as a conference paper at ICLR 2027

SpectralWeave: A shared spectral framework for Native 4D Mesh Generation

Abstract

Native 4D mesh generation animates an unrigged input mesh from a semantic action condition while preserving its vertex indexing and connectivity. Sharing motion across independently meshed inputs requires common learning coordinates that retain reconstruction on each input topology. We introduce **SpectralWeave**, a framework for native 4D mesh generation that builds its spectral motion representation from the geometric structure inherent in the input mesh. A native Laplacian basis computed from each mesh remains fixed throughout the sequence, representing motion through compact coefficient trajectories that decode directly on the input vertices without a learned mesh decoder. However, native coefficients remain difficult to compare across identities because their bases differ. We align native spectral coordinates through jointly estimated adapters, establishing shared coordinates while preserving each mesh's native reconstruction subspace. We then use partial surface correspondences for alignment and condition motion diffusion on the action and input morphology. We decode the generated shared coefficients and refine the sequence to preserve local details. Experiments in separate animal and human domains demonstrate better motion accuracy and surface quality on new identities than the compared methods. Ablations support shared spectral coordinates, morphology conditioning, and geometric refinement. See the Project page for more demos: [Project page](https://anoymouscell.github.io/spectralweave).

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Diffusion Models, Spectral Geometry Processing, Cross-Identity Motion Generation, Action-Conditioned Generation.

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_etzkOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The proposed method animates a static mesh from an action description while preserving its native topology. It represents motion using a native Laplacian basis for each character and aligns motion coefficients across characters with invertible adapters. A morphology-conditioned diffusion model generates shared coefficient trajectories, which are decoded on the input mesh and refined using DQS and ARAP. For new characters, adapters are fitted using training characters as anchors to generate motions.

Critical strengths
  1. Experiments on human and animal domains of motion generation show improvements on both seen and unseen identities.
  2. The motions are well-constrained, so the generated motions look smooth, even for some specific complex motions.
  3. The overall idea of motion decomposition and pattern generalization is quite interesting.
Critical weaknesses
  1. The proposed method works well on a specific structure (i.e., humanoids, animals), but I suppose it may fail if the structure deviates too much from the established structure patterns. For instance, if we aim to generate an animation of a bird flapping its wings, the bird's topology deviates significantly from that of four-footed animals and humanoids.
  2. Does the initial pose matter for this method? Or is it supposed to work on a specific initial pose? When generalizing to real-world cases, the initial pose may not be exactly a standard, unanimated pose.
  3. Most of the baselines target general motion pattern generalization with different structures. Compared to these methods may be a little bit unfair. The fine-tuned BiMotion indeed alleviates this concern. However, comparing with rigging-based methods (which may share a similar idea to the proposed method about generating motion based on a specific structural template), such as UniMate [A], could be beneficial.

[A] Mou, Linzhan, et al. "UniMate: One Unified Model to Animate Diverse Skeletons." arXiv preprint arXiv:2609.05415 (2026).

Questions for authors

The paper does not separately evaluate the effects of Dirichlet energies and area-weighted norms on morphology conditioning.

Reviewer_e8LcOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a generative method for 4D mesh generation. From a static mesh of any topology, it learns motion displacements in a fixed Laplacian basis (recomputed for each input mesh). A diffusion model generates trajectories conditioned on action (text input) and morphology. These are refined with dual quaternion skinning and ARAP regularization.

Critical strengths

The addressed problem and proposed method are coherent and make sense to me. Learning deformation generation usually assume a common template or a decoder which limit greatly practical applications.

Reported numbers are surprisingly good: the method claims better quantitative metrics, for all metrics, on two datasets and better user preference across the board (Table 13). The same goes for qualitative comparisons with a clear superiority of the proposed approach on the chosen sequences.

The paper is generally well-written and easy to follow.

Critical weaknesses

The method is generative but I do not see any “generative adapted” metrics (CLIP score and others). How much seed variance can you observe ?

I am concerned about the robustness of the method to increasing vertex count: Table 12 (in the appendix) reports degrading performance as K increases. I think it would be critical to look at this degrading effect when changing vertex counts (and maybe other remeshing strategies). Basically, how can you identify if the error comes from spectral truncation and correspondence instability (as you increase the mesh density).

Novelty is quite limited, the proposed approach is a mix of existing techniques and models which is fine as long as justified and clearly showing superior performance.

About the user-study: more participants are needed for a statistically significant result and I suggest putting the table in the main paper because for such a task, we care more about human feedback than geometric metrics in the end I believe.

Questions for authors

How much seed variance can you observe ?

How does the method react to an increase of vertex count ? Are performance unchanged, degrade but the degradation can be resolved with a higher K or worse no matter what ?

The method computes vertex displacements added to an initial static mesh: for long-range motions (walking, running around, basically when the size of the motion is a lot larger than the size of the mesh) do you experiment with the scale of the generated displacements?

Reviewer_fovEOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper approaches the problem of conditional animation generation of unrigged triangle meshes. SpectralWeave tackles this through the use of a diffusion model generating a representation of the full motion. Given mesh ii pointcloud Xi0∈RNi×3X^0_i \in R^{N_i \times 3}, the motion is defined as a series of LL displacements from the initial mesh: ΔXil=Xil−Xi0\Delta X^l_i = X^l_i - X^0_i,

The main contribution of the paper is the particular representation for this displacement being consistent across different meshes with varying number of vertices NiN_i while avoiding techniques such as VAE to construct a common latent representation. This is then utilized as input for a conditional diffusion model. The method combines a series of steps in order to do this.

It represents the motion of mesh ii approximately using a linear combination of the KK first eigenvectors ϕi\phi_i of its Laplacian matrix. This is done through the solution of a weighted MSE. The resulting matrix Bil∈RK×3B^l_i \in R^{K \times 3} represents the motion natively in each mesh eigenbasis ϕi\phi_i.

From these mesh-specific coordinates, SpectralWeave is concerned with constructing an approximate common spectral coordinate for motion across meshes. In order to do this, each mesh eigenbasis is transformed as $\phi_i D_i $ with DiD_i a per-mesh matrix, leading to displacement in this shared representation Zil=Di−1BilZ^l_i = D_i^{-1}B^l_i which is then used for the diffusion model.

The construction of these DiD_i matrices is the main component in SpectralWeave. It is done through two phases. First constructing a pairwise transformation CijC_{ij} and then constructing the global transformation DiD_i from these fixed pairwise transformations. The pairwise transformation CijC_{ij} is created by a soft vertex matching matrix Πij\Pi_{ij} between vertices in meshes ii and jj through the use of codes from a DiffusionNet and then by finding CijC_{ij} such that $\phi_j C_{ij} \approx \Pi_{ij} \phi_i $ using a weighted loss. The global transformation DiD_i is then computed by finding Di,DjD_i , D_j such that $ C_{ij} D_i \approx D_j$ which is done through a regularized MSE optimization through i,ji,j pairs.

The model is trained on animal and human meshes datasets.

Critical strengths
  • The construction allows for a fixed size representation of animations with different numbers of vertices without relying on the training of a latent encoder model, leveraging a common representation through the use of spectral basis.
  • It removes ambiguity in eigenbasis representation (sign flips, rotations) through this shared representation
  • SpectralWeave performs strongly on new heldout meshes, which indicates that the common motion representation is accurate for the new mesh components movement.
Critical weaknesses
  • It relies on constructing a shared representation for the motion across meshes through the use of transforming the eigenbasis. While this feature is key for the construction presented, a serious limitation of this approach is that it relies on all meshes sharing the same approximate general structure. This is the case for the human and animals datasets used but it might not be able to scale to the general motion generation setup where objects may present very diverse structures and so this construction of the global coordinate for displacement might show unreliable as an encoding for motion. This makes this approach more limited in scope compared to prior work such as AnimateAnyMesh, that through encoding the animation in latent space allows for a more general representation that supports a wide variety of mesh structures.
  • The construction requires approximating the displacement through the use of the first KK eigenvectors. By construction this is similar to a low-pass filter on the actual displacements. Animation combining movements of large and small parts are common in practice. Examples of this are animations involving arm and finger movement. However, this approach is likely to remove those finer movements as they are not well represented in the eigenbasis. A refinement on local details is done on the final generated animation, but this process cannot recover the removed details from the source animation.
Questions for authors
  • How does the K=32 approximation remove animation details from the source animation (fingers, etc)? Since post-generation refinement fits an ARAP transformation, it seems unable to recover finer motions removed from the initial K approximation. Is that correct?
  • In this same direction, the metrics are area-weighted which might undermine important but small movements. Could the authors complement the reported metrics on table 1 with an extra metric that is sensitive to such motions? That is, one where big displacements such as torso don’t dominate the metric. Any reasonable choice would do. This is to better analyze how SpectralWeave compares against baseline methods in these finer motions.
  • It might be worth it to include a brief clarification on how the method handles meshes with disconnected components or whether the method assumes meshes have only one. Computing the eigenvectors naively when having multiple components can have undesirable effects on the representation but it is not clear to me how the method approaches this. How does SpectralWeave handle multi-component meshes? Is the method limited to one-component objects?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.