acceptodds
Under review as a conference paper at ICLR 2027

Motion is Structure: ZORA for Zero-Shot 3D Rigging from In-the-Wild 4D Dynamics

Abstract

Automating the creation of articulated 3D characters is a longstanding challenge in computer graphics and embodied AI, with applications that span gaming, visual effects, and physical simulation. Recent advances in monocular 4D reconstruction enable extraction of dynamic 3D surfaces from an input single-view video, yet these outputs remain baked geometry: temporally re-playable but not editable or animatable. Converting them into articulated assets with explicit skeletons and skinning weights is a critical unsolved step. Existing automatic rigging methods rely on rig-label supervision or category-specific priors, making them brittle under severe self-contact and out-of-distribution topologies such as multi-limbed creatures or non-humanoid figures. We introduce **ZORA** (Zero-shot Optimization for Rigging Assets), a training-free framework built on a single principle: motion is structure. Given a reconstructed 4D surface, ZORA extracts a topology-aware kinematic scaffold by routing geodesic paths through regions of high temporal motion variance, suppressing spurious shortcuts at self-contact without any learned prior. It then recovers an explicit linear blend skinning rig via joint optimization of skinning weights and per-frame rigid transformations under four complementary objectives: motion affinity, topological locality, temporal smoothness, and volumetric consistency. Crucially, no template skeleton or labeled training data is required, coherent 4D motion alone serves as a geometric prior, binding vertices with correlated dynamics to the same articulated part. ZORA produces editable, production-ready rigs across a diverse range of characters, including those with complex topologies where prior methods degrade.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Rigging, 3D

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_TaGqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on how to automatically recover editable 3D skeletal structures, skinning weights, and corresponding skeletal motions from dynamic 4D meshes reconstructed from monocular videos. The authors propose ZORA, an optimization-based framework that requires no additional training of a rigging model. The core idea is to leverage temporal motion coherence among vertices as a cue for inferring underlying skeletal structures and skinning relationships. First, the authors construct Motion Affinity from vertex trajectories obtained through 4D reconstruction and motion cues extracted from upstream models to identify surface regions exhibiting similar motion patterns. Next, Stretch-Aware Kinematic Scaffold Extraction adjusts graph traversal costs based on the temporal stretching of mesh edges, reducing erroneous connections caused by self-contact or temporary proximity, and constructs an initial skeletal structure through geodesic path search. Finally, Unified-Coherence Rigging Optimization jointly optimizes skinning weights and per-frame rigid bone transformations using LBS, ultimately producing editable 3D assets. Experiments are conducted on AnimalKingdom, ModelsResource, and examples constructed from generated videos. Quantitative comparisons are conducted with MoRig and RASR, and qualitative comparisons are conducted with RigAnything and Puppeteer. The experiments covers reconstruction quality, structural validity and ablation studies.

Critical strengths

ZORA leverages temporal motion coherence among vertices to provide additional cues for joint discovery and part segmentation. Meanwhile, the Stretch-aware scaffold incorporates temporal edge stretching into graph traversal costs, thereby reducing spurious connections caused by self-contact or shortcuts across distinct articulated parts.

The authors evaluate their method on unusual animals and generated characters, covering diverse morphologies and complex topological structures. The experimental results demonstrate advantages in certain skeletal structural validity metrics.

Critical weaknesses

The paper somewhat overclaims its zero-shot, prior-free, and joint skeleton optimization capabilities, while the effectiveness of its core contributions remains insufficiently validated. Although the rigging optimization itself does not require additional training, Section 3.1 explicitly states that the construction of Motion Affinity leverages motion cues extracted from the pretrained PAGE-4D model.

I have concerns about the claimed joint optimization of the skeletal structure. In the overview of Section 3, the authors explicitly state that they jointly optimize the skeleton S, skinning weights W, and per-frame rigid bone transformations. However, in Section 3.2, the skeleton is primarily initialized through the Stretch-Aware Scaffold, while the optimization described in Section 3.3 mainly focuses on skinning weights and bone transformations.

Furthermore, one of the central contributions of this paper is the use of Motion Affinity as a key cue for articulation discovery and skinning optimization. However, its effectiveness is currently supported only by the qualitative ablation in Appendix Figure 11. I believe this evidence is insufficient to independently quantify the contribution of Motion Affinity to the final rig quality.

The experimental results do not sufficiently support the main claims. First, the quantitative results do not demonstrate a consistent advantage of ZORA in overall rigging quality. In Table 1 on AnimalKingdom, ZORA achieves the best performance only on the two structural validity metrics, E_viol and E_cross, while performing substantially worse than RASR on E_LBS, E_norm, and E_coh. Similarly, in Table 2 on ModelsResource, ZORA outperforms MoRig-standard only on E_cross, while falling behind on all other metrics.

I find the ablation study not convincingly verify the necessity of the proposed components. In Table 3, most metrics show only marginal changes, and some ablated variants even outperform the full model on certain metrics.

Questions for authors

Section 3.1 mentions extracting motion cues from PAGE-4D attention, while Equation (1) defines Motion Affinity directly using differences in per-frame vertex displacements. Could the authors clarify the relationship between the pretrained features and the geometric motion affinity, as well as their respective roles in the framework?

Could the authors clarify whether S is actually updated during optimization, including the rest-pose joint positions and skeletal topology? If the skeleton is only constructed during initialization, should the scope of the claimed joint optimization be revised accordingly?

The paper provides quantitative comparisons only with MoRig and RASR, while more recent methods such as RigAnything and Puppeteer are evaluated only qualitatively. Could they provide quantitative comparisons with these state-of-the-art methods under the same datasets and evaluation protocols to better substantiate these claims?

The quantitative ablation study mainly focuses on regularization terms in the optimization process, without independently quantifying the contributions of two components (Motion Affinity and the Stretch-Aware Scaffold). Could the authors provide additional quantitative ablations to evaluate their individual contributions?

I would like to see more direct experimental evidence for the self-contact scenarios explicitly highlighted in the Introduction, such as crossed arms and wings folded tightly against a bird's torso?

2 comments

We thank Reviewer TaGq for the thorough critique regarding claim precision, optimization scope, ablations, resting self-contacts, and quantitative comparisons with supervised static riggers. All updates are integrated into the revised manuscript.

1. Terminology, PAGE-4D vs. Eq. (1), and Fixed Skeleton S\mathcal{S} (W1, W2, Q1, Q2)

  • Zero-Shot vs. Prior-Free (W1, revised Abstract & §1): Following standard usage (analogous to zero-shot segmentation/classification with frozen DINO/CLIP backbones), zero-shot rigging denotes zero rig annotations, zero rigging model training, and zero category skeletal templates. Because Stage 2 extracts general visual-motion cues from frozen PAGE-4D, we replaced "prior-free" with "rig-supervision-free and template-free."
  • PAGE-4D Cues & Eq. (1) (Q1, revised §3.1, §3.3, Algorithm 1): Revised §3.1 details the exact two-stage link: multi-view renders pass through frozen PAGE-4D attention layers and back-project onto visible 3D vertices (confidence>τconf\text{confidence} > \tau_{\text{conf}}) to form normalized cosine-Gram cues GijG_{ij}, which modulate the geodesically regularized 3D displacement affinity AijA_{ij} (Eq. 1) and gate sparse local pairs PA\mathcal{P}_A (Eq. 7). Meanwhile, scaffold discovery (§3.2) uses only 3D vertex trajectories via temporal edge stretch ses_e (Eq. 2).
  • Fixed Skeleton S\mathcal{S} (W2, Q2, revised §3 & Algorithm 1): Agreed. Rest joints JJ and topology EE are extracted once in §3.2 and held strictly fixed during §3.3, which jointly optimizes W\mathbf{W} and {Gbt}\{G_b^t\}.

2. Ablation Analysis of Table 3, Stretch Scaffold, and Motion Affinity (W3, W5, Q4)

  • Table 3 Behavior (W5): w/o L_loc shifts modestly because Stage 3 already enforces a hard geodesic support mask (0.25D, Table 7 & Algorithm 1) blocking distant weight bleeding. Dropping spatial coherence (w/o L_mesh) lowers 6-frame ELBSE_{\text{LBS}} (0.0291 →\to 0.0135) only via severe pose overfitting that fragments weights, degrading EnormE_{\text{norm}} (0.0966 →\to 0.1293) and EcohE_{\text{coh}} (0.0273 →\to 0.0449).
  • Stretch Scaffold & Motion Affinity Ablations (W3, Q4, Appendix E.6 Fig. 10 & Appendix F Figs. 13–14): Without the Stretch Scaffold (Fig. 13, 14760), joints drift from hinges or collapse—6-frame ELBSE_{\text{LBS}} drops (0.02448 →\to 0.01953), but GT Joint F1 plunges from 0.5000 to 0.0870 (Joint CD: 0.0212 →\to 0.0766). Without Motion Affinity (Fig. 14(d,e)), fragmented per-vertex weights locally collapse under novel articulation, whereas our affinity prior preserves smooth, cohesive part deformation (further validated on unseen motions in Figs. 11–12, Appendix E.7).

3. Direct Self-Contact Validation (Q5, Appendix E.6 Fig. 10 & Appendix G Tables 5–6)

  • Quantitative GT Stress-Tests (Appendix G, Tables 5–6): Under 40% resting self-contact (TN 40%), RASR fuses co-moving parts (Joint F1 0.322 →\to 0.114), whereas ZORA achieves 2.6$\times$ higher Joint F1 (0.301), 34% lower Joint CD (0.062 vs. 0.094), and 34% lower Animation MPVE (0.030 vs. 0.046) (and directionally reduces contact-crossing paths from 0.89 to 0.30 on real AnimalKingdom self-contact clips).
  • Folded-Wing Bird Case Study (§4.3, Appendix E.6, Fig. 10): Directly testing a bird unfolding tightly folded wings shows that ZORA (α=100\alpha=100) cleanly separates and rigs the wings alongside the beak, claw, and tail; w/o Stretch (α=0\alpha=0) misroutes wing bones via wing-torso contact shortcuts and omits both the beak and claw; RASR collapses joints into an unusable dorsal cluster; and RigAnything & Puppeteer fail to rig the folded wings.

4. Fitting vs. Structural Trade-Offs (W4) & RigAnything / Puppeteer Comparison (Q3, Revised Table 1)

Our claim centers on structural reliability and OOD/corruption robustness rather than unconstrained least-squares fitting: in Table 1, RASR lowers ELBSE_{\text{LBS}} (0.0111) only by routing 67.5% of bones across empty space (Ecross=0.6749E_{\text{cross}} = 0.6749 vs. ZORA 0.2040), while in Table 2, ZORA cuts monocular MoRigsgl\text{MoRig}_{\text{sgl}} ELBSE_{\text{LBS}} by 13.6$\times$ (0.0172 vs. 0.2338).

We also added RigAnything (trained on 9,388 rigs) and Puppeteer (>74,000>74,000 rigs) directly into Table 1 & §4.1 on OOD AnimalKingdom (N=42N=42, frozen rest rigs + shared per-frame rigid pose refitting):

  • Training-Prior Projection vs. Discovering Unknown Moving Parts: RigAnything and Puppeteer infer structure solely from memorized training shapes, whereas ZORA discovers unknown moving parts from 4D kinematics. Facing unseen morphologies, Puppeteer omits unfamiliar appendages, collapsing to ≤4\le 4 joints on sequences (bear: 2; snake/fish/squid: 3–4) and ≤8\le 8, while RigAnything projects ungrounded priors that route 49.3% of bones outside the volume (Ecross=0.4926E_{\text{cross}} = 0.4926, +0.29 worse than ZORA, 95% CI [0.21, 0.37] on 33/42 clips).
  • Metric Trade-Offs vs. Motion Understanding: Articulating full, fine-grained moving parts under strict geodesic locality incurs slightly higher ELBSE_{\text{LBS}} (0.0586 native / 0.044 refit vs. unmasked 6-DOF fitting 0.0252 / 0.0365) and slightly higher EcohE_{\text{coh}} / EcrossE_{\text{cross}} than Puppeteer's near-rigid 2–4 joint stubs (ZORA beats RigAnything on both). Crucially, ZORA achieves the best surface normal consistency among all supervised baselines (Enorm=0.3708E_{\text{norm}} = 0.3708, outperforming Puppeteer's 0.4214 on 34/42 clips and RigAnything's 0.4478) alongside low volumetric violation (Eviol=0.0107E_{\text{viol}} = 0.0107, 4.4x below RigAnything's 0.0471), demonstrating faithful understanding of unknown object motion.
Reviewer_aa4ZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ZORA, a zero-shot method for recovering a LBS rig from a dynamic mesh sequence with vertex correspondences. It uses a stretch-aware scaffold for skeleton initialization and motion-affinity constraints to optimize skinning weights and per-frame bone transformations, without rig annotations or a predefined skeleton.

Critical strengths
  • The stretch-aware scaffold is well motivated and targets a concrete failure mode. Two parts may touch in the initial pose but move apart later. Using changes in their distance to penalize false graph connections is a sensible way to resolve ambiguities in static geometry.
  • The topology-noise experiment provides preliminary evidence for more robust joint recovery under the authors' corruption setting. At TN40, Joint F1 is 0.301 for ZORA versus 0.114 for RASR. This supports the method's motivation, but RASR still has higher Bone F1 and lower EcrossE_{cross} under the same condition.
Critical weaknesses
  • Some important details of the input pipeline and motion affinity are unclear. Section 3.1 describes affinity extracted from PAGE-4D attention, while Eq. (1) computes it from vertex displacements. It is unclear how these descriptions relate and which signal is actually used in the experiments. There is also no clear description of how the AnimalKingdom videos are converted into mesh sequences with consistent vertex correspondences.
  • It remains unclear whether the recovered rig works well beyond reproducing the input motion. The reconstruction metrics evaluate fitting to the observed sequence, and Fig. 11 shows skinning weights rather than deformation under new poses. On AnimalKingdom, RASR has lower ELBSE_{LBS}, EnormE_{norm}, and EcohE_{coh}, while ZORA performs better on the two bone-interiority metrics, EviolE_{viol} and EcrossE_{cross}. This is a trade-off, but it does not establish whether ZORA's structural improvements lead to better deformation when editing the rig.
  • The ablations are also difficult to connect to the main results. Section 4.4 states that the skeleton-scale prior is disabled for the ablations, but this prior and its use in other experiments are not clearly explained. It is unclear whether the ablation findings apply to the configuration used in the main comparison.
Questions for authors
  • How are mesh sequences and vertex correspondences obtained for each dataset? How does the PAGE-4D affinity relate to Eq. (1), and which signal is used in the rigging objective?

  • Are rest-joint positions fixed after scaffold extraction? Are per-frame bone transformations optimized independently or coupled through kinematic constraints? Appendix B describes parent-relative conversion for export; is joint attachment also constrained during optimization?

  • Please clarify the data subsets and configurations used in Tables 2, 3, 5, and 6. What is the skeleton-scale prior, and which ZORA configurations use it? For the missing-parts experiments, which reference sequence and vertex subset are used to compute LBS and MPVE?

  • Could the examples in Figs. 10–11 be extended with aggregate quantitative results under the main evaluation setting, separately disabling the temporal stretch penalty and motion affinity while keeping the remaining procedure unchanged?

  • Could a small comparison with RASR on held-out poses or controlled joint edits be provided, with the recovered skeleton and skinning weights fixed? For a held-out evaluation, test poses should be excluded from scaffold extraction, affinity construction, and initial rig recovery, with only pose parameters fitted at test time.

3 comments

We thank Reviewer aa4Z for the precise, actionable review. All clarifications and leak-free unseen-pose evaluations have been integrated into the revised manuscript.

Q1 — Correspondences, PAGE-4D Affinity vs. Eq. (1), and Signal Roles

  • Correspondences (§4): ModelsResource provides native vertex-correspondent animated meshes, while AnimalKingdom clips are reconstructed via V2M4 (Chen et al., 2025) into fixed-topology 25-frame meshes (9 Hunyuan3D-2 keyframes + 16 cross-frame registered/interpolated frames).
  • PAGE-4D Cues & Eq. (1) (§3.1, §3.3, Algorithm 1 (in appendix page 27)): Revised §3.1 explicitly formulates the two-stage computation: multi-view renders of the 4D mesh pass through frozen PAGE-4D attention layers, back-projecting visible patch tokens onto 3D vertices (confidence>τconf\text{confidence} > \tau_{\text{conf}}) to form normalized cosine-Gram descriptors Gij=((fi⊤fj+1)/2)γG_{ij} = ((\mathbf{f}_i^\top \mathbf{f}_j + 1)/2)^\gamma. Because 2D tokens lack 3D metric scale, we compute explicit 3D displacement affinity AijA_{ij} via Eq. (1), regularize it geodesically on the mesh manifold, and modulate/gate it with GijG_{ij} within local neighborhoods (dgeo≤rA,top-kA=32,Aij≥τAd_{\text{geo}} \le r_A, \text{top-}k_A = 32, A_{ij} \ge \tau_A) to form PA\mathcal{P}_A in Eq. (7). Meanwhile, the Stretch-Aware Scaffold (§3.2) uses only 3D vertex trajectories via temporal edge stretch ses_e (Eq. 2).

Q2 — Fixed Rest Joints and Independent Per-Frame Transforms

Revised §3 and Algorithm 1 make explicit that rest joints JJ and tree topology EE remain strictly fixed after §3.2. During Stage 3, bone poses (Rt,b,δt,b)∈SE(3)(R_{t,b}, \delta_{t,b}) \in SE(3) are optimized independently per frame and bone without kinematic-chain attachment constraints (and with temporal smoothness set to 0 on sparse keyframes). Parent-relative conversion (Appendix B) is applied purely at glTF export, preventing monocular scale drift from distorting skinning weights.

Q3 — Subsets, Unified Configuration, and Missing-Parts Protocol

  • Unified Configuration (§4.5, Table 7): The skeleton-scale prior is 0 across all reported experiments in the paper (Tables 1, 2, 3, 5, and 6), which share one unified configuration (λrec=10,λloc=50,λcoh=80  [0.5 mesh+0.5 affinity],λbal=1\lambda_{\text{rec}}=10, \lambda_{\text{loc}}=50, \lambda_{\text{coh}}=80\;[0.5\text{ mesh}+0.5\text{ affinity}], \lambda_{\text{bal}}=1). Thus, all ablations apply directly to our main comparison setting.
  • Subsets & Missing Parts: Tables 3, 5, and 6 use the exact same 100 ModelsResource cohort and 6 source frames [0, 20, 40, 60, 80, 100]. Under Missing Parts (Tables 5–6), ELBSE_{\text{LBS}} evaluates self-reconstruction over surviving corrupted vertices, whereas MPVE\text{MPVE} evaluates against the uncorrupted clean reference over shared surviving vertices.

Q4 — Ablation of Temporal Stretch and Motion Affinity (§4.5, Appendix E.6 & F)

As analyzed in §4.5, Appendix E.6 (Fig. 10), and Appendix F (Figs. 13–14), dropping structural or affinity regularizers artificially lowers in-sample ELBSE_{\text{LBS}} via overfitting while breaking anatomical structure and novel-pose deformation:

  • Stretch Penalty (Fig. 10 & Fig. 13): Removing the scaffold on asset 14760 (Fig. 13) lowers 6-frame ELBSE_{\text{LBS}} (0.02448 →\to 0.01953) via weight overfitting, but collapses Joint F1 from 0.5000 to 0.0870 (Joint CD: 0.0212 →\to 0.0766). Isolating α=0\alpha=0 (w/o Stretch) on the folded-wing bird (Fig. 10) similarly allows wing-torso contact bridges to corrupt geodesic routing, misrouting the wings and completely omitting joints for both the beak and claw.
  • Motion Affinity (Fig. 14 & Table 3): Just as w/o L_mesh in Table 3 lowers ELBSE_{\text{LBS}} (0.0135) while degrading EnormE_{\text{norm}} (0.0966 →\to 0.1293) and EcohE_{\text{coh}} (0.0273 →\to 0.0449), isolating the same bone in Fig. 14(d,e) shows that removing affinity fragments skinning weights across individual vertices, causing local mesh collapse under novel articulation.

Q5 — Leak-Free Held-Out Pose & Cross-Sequence Comparison vs. RASR (§4.4, Appendix E.7, Figs. 11–12)

We implemented your exact leak-free protocol—excluding test sequences from affinity, scaffold, and rig recovery, freezing (J,W^)(J, \widehat{\mathbf{W}}), and fitting only per-frame rigid bone poses at test time:

  1. Cross-Sequence Transfer & Temporal-Split Generalization (§4.4, Appendix E.7, Figs. 11–12): When transferring a rig extracted on a generated elephant clip onto the unseen real AnimalKingdom elephant sequence (Fig. 11), and when recovering a rig on the first half of a fox video (Seen: walking) and testing on the held-out second half (Unseen: sitting down and lifting a front paw, Fig. 12), ZORA maintains clean interior bones and faithfully reproduces unseen deformations, whereas RASR protrudes exterior bones and collapses on the unseen sitting transition.
  2. Quantitative 95-Unseen-Frame Check (active-100): On clean studio ModelsResource animations (synthesized by exact artist LBS, floor 0.0001), unconstrained SSDR (RASR with GT joint budget) naturally fits clean LBS tighter (0.0009 unseen MPVE vs. ZORA 0.0147), yet ZORA achieves a lower unseen-to-source MPVE ratio (1.09x vs. 1.20x)—confirming zero unseen-pose overfitting—while surviving 40% self-contact corruption where RASR collapses (Joint F1 0.301 vs. 0.114, Anim MPVE 0.030 vs. 0.046, Tables 5–6).

Thank you for the clarification. I will take your response, along with the comments from other reviewers, into account for my final assessment.

Reviewer_NHRzOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary
  1. This paper studies how to recover an editable skeleton, skinning weights, and animation transforms from a topology-consistent 4D dynamic mesh, with the goal of avoiding manually rigged data and category-specific templates.

  2. The method first constructs motion affinity and a stretch-aware kinematic scaffold, and then optimizes an LBS rig.

  3. The paper reports improvements on several structural-validity metrics on Animal Kingdom, ModelsResource, and synthetic videos.

Critical strengths
  1. The paper addresses an important problem of recovering editable rigs from dynamic 4D surfaces and proposes a reasonable motion-as-structure perspective for self-contact and non-humanoid topologies.

  2. The method combines motion affinity, stretch-aware graphs, geodesic locality, and LBS optimization in an interpretable pipeline.

  3. In addition to reconstruction error, the paper evaluates motion coherence, boundary violation, cross-space bones, and ablations, and reports advantages on several of its structural-validity metrics.

Critical weaknesses
  1. The method figure and description do not make the actual algorithmic procedure sufficiently clear. Figure 1 mainly presents a conceptual chain of “motion affinity → scaffold → optimization,” but it does not clearly explain how motion affinity is converted into candidate joints and bones, how the bone hierarchy is constructed, how skinning weights are initialized, or which variables are fixed or optimized at each stage. The overall idea is understandable, but the complete method is difficult to reproduce, and the specific novelty relative to existing motion-aware rigging methods is difficult to identify. The authors should provide clearer pseudocode, variable definitions, and intermediate data structures.

  2. The input pipeline and experimental fairness are not sufficiently specified. The paper combines video generation, 4D reconstruction, and rigging, but does not clearly report the number of frames, the 4D reconstruction method, the video-generation model, camera and scale processing, or whether all baselines receive exactly the same 4D input. In particular, generative motion augmentation may change the observable range of joint motion. The authors should clarify which quantitative results use real videos and which use Veo 3.1 or augmented sequences.

  3. The unlabelled experiments rely mainly on metrics proposed by the paper, while the dataset size, sample composition, category split, statistical unit, and evaluation thresholds are not sufficiently specified. Some results in Table 1 have relatively large standard deviations, but the paper does not report the number of samples, per-sample distributions, or confidence intervals. Since these metrics are closely related to the structural constraints used by ZORA, the authors should explain whether they independently reflect rig quality or may favor the proposed method.

Questions for authors
  1. Could the authors provide clearer pseudocode describing the complete data flow from the 4D mesh and motion affinity to the scaffold, skinning weights, and final animated skeleton?

  2. Do all datasets and baselines use exactly the same 4D input? Please specify where video generation, 4D reconstruction, and motion augmentation are used.

  3. How many samples, categories, and sequences are included in the unlabelled Animal Kingdom dataset, and how are the data split? Are the means and standard deviations in Table 1 computed per sequence, per object, or per frame?

2 comments

We thank Reviewer NHRz for requesting an auditable algorithmic specification, input-fairness protocol, and per-sequence statistical reporting. All requested additions are now incorporated into the revised manuscript.

Q1

We have added the complete symbolic pseudocode in Algorithm 1 (Appendix, p. 27) alongside expanded formulations in §3.1–§3.3 and Table 7 (Appendix I):

  • Stage Separation & Fixed vs. Optimized Variables: Motion Affinity construction (Algorithm 1, lines 1–5) and Stretch-Aware Scaffold extraction (lines 6–19) execute independently on the 4D mesh sequence {Xt}t=1T\{X^t\}_{t=1}^T before converging in LBS Optimization (lines 20–32). Rest joints JJ, directed bone tree EE, root rr, and sparse affinity pairs PA\mathcal{P}_A are strictly fixed prior to Stage 3, which optimizes only skinning logits Z∈RV×BZ \in \mathbb{R}^{V \times B} (under a hard geodesic mask dG(i,b)≤rlocd_{\mathcal{G}}(i,b) \le r_{\text{loc}} and annealed softmax temperature τW\tau_W) and independent per-frame rigid bone poses {Rt,b,δt,b}\{R_{t,b}, \delta_{t,b}\} without rig labels or category templates.
  • Distinction from Prior Motion-Aware Riggers: Unlike supervised point-cloud predictors (MoRig) or downstream Euclidean trajectory clustering (RASR), ZORA embeds temporal edge stretch ses_e directly into graph routing to sever resting self-contacts while retaining proximity bridges, and couples PAGE-4D Gram cues with 3D trajectory affinity inside zero-shot weight optimization.

Q2

As documented in §4 (Datasets and Input Protocols), all sequences are centered and normalized at canonical frame t=1t=1 by bounding-box diagonal DD. No quantitative table uses Veo 3.1 generated or motion-augmented clips, which appear strictly in qualitative visualizations (Figs. 8–11). Within every quantitative benchmark, all motion-driven baselines receive the exact same 4D frames:

  • ModelsResource (Tables 2, 3, 5, 6): Native artist-LBS meshes evaluated from the exact same 6 source frames ([0, 20, 40, 60, 80, 100] of 101) for ZORA and RASR, with MoRigstd/sgl\text{MoRig}_{\text{std/sgl}} sampling full-surface or single-view point clouds from those exact frames.
  • AnimalKingdom (Table 1): Real monocular videos reconstructed into fixed-topology 4D meshes via V2M4 (Chen et al., 2025) (25 frames = 9 Hunyuan3D-2 keyframes + 16 registered/interpolated frames for 40 clips; 10 and 24 frames for 2 clips). ZORA and RASR consume the exact same V2M4 meshes; MoRig samples point clouds from the same frames; static riggers (RigAnything, Puppeteer) take canonical frame 1.
  • Generated / Augmented Clips (Figs. 8–11 only): 192 raw frames subsampled to 25 and reconstructed via V2M4 (qualitative only).

Q3

  • Dataset Accounting & Paired CIs (N=42N = 42): Of 49 curated AnimalKingdom clips, 7 contained entire non-finite (NaN/Inf) upstream frames invalid for any method, leaving 42 valid sequences spanning ~40 species across 7 taxonomic groups (§4). As per-instance zero-shot optimization, there is no train/test split; means and SDs in Table 1 are computed per sequence (one animal per clip, seeds averaged first). While a cohort of ~50 wild sequences across diverse species has inherent sample-size limitations for strong asymptotic statistical claims, we report paired bootstrap 95% percentile CIs (10,000 resamples; Δ=ZORA−Baseline\Delta = \text{ZORA} - \text{Baseline} over the exact shared N=42N=42 sequences, where negative Δ\Delta means ZORA is lower/better) for reference below:
MetricΔ\Delta vs. MoRig (95% CI)Δ\Delta vs. RASR (95% CI)
ELBS↓E_{\mathrm{LBS}} \downarrow−0.405    [−0.533,−0.296]-0.405\;\;[-0.533, -0.296]+0.048    [+0.041,+0.055]+0.048\;\;[+0.041, +0.055]
Enorm↓E_{\mathrm{norm}} \downarrow−0.185    [−0.230,−0.148]-0.185\;\;[-0.230, -0.148]+0.141    [+0.113,+0.172]+0.141\;\;[+0.113, +0.172]
Ecoh↓E_{\mathrm{coh}} \downarrow−0.130    [−0.255,−0.033]-0.130\;\;[-0.255, -0.033]+0.041    [+0.035,+0.048]+0.041\;\;[+0.035, +0.048]
Eviol↓E_{\mathrm{viol}} \downarrow−0.282    [−0.399,−0.185]-0.282\;\;[-0.399, -0.185]−0.057    [−0.071,−0.044]-0.057\;\;[-0.071, -0.044]
Ecross↓E_{\mathrm{cross}} \downarrow−0.729    [−0.806,−0.643]-0.729\;\;[-0.806, -0.643]−0.471    [−0.551,−0.390]-0.471\;\;[-0.551, -0.390]

Although RASR attains lower in-sample ELBSE_{\mathrm{LBS}}, EnormE_{\mathrm{norm}}, and EcohE_{\mathrm{coh}} via unconstrained least-squares surface fitting, its volumetric adherence (EviolE_{\mathrm{viol}}) and interiority (EcrossE_{\mathrm{cross}}, routing most of bones (across) empty space) are severely degraded. As visually confirmed in Figs. 10, 11, and 12 of the revised manuscript, RASR produces highly concentrated, degenerate skeletal clusters or protruding exterior bones—failing to recover a usable rig and collapsing when transferred to unseen poses.

  • Metric Constants (§4.1, Appendix D): Both EviolE_{\text{viol}} and EcrossE_{\text{cross}} sample K=16K = 16 points per bone segment. EviolE_{\text{viol}} uses no threshold; EcrossE_{\text{cross}} uses exterior SDF tolerance ϵ=0.01D\epsilon = 0.01D and duration fraction ρ=0.5\rho = 0.5.
  • Metric Objectivity: (1) ModelsResource GT metrics (Joint CD/F1, Bone F1, Skin L1, MPVE in Tables 2, 5, 6) are completely independent of our losses and closely agree with our unsupervised metrics (Appendix G); (2) Stage 3 never optimizes volumetric SDFs (Eviol,EcrossE_{\text{viol}}, E_{\text{cross}}), which objectively test whether animated bones remain interior across frames; and (3) Table 1 openly penalizes ZORA on raw ELBSE_{\text{LBS}} (0.0586 vs. RASR 0.0111), showing no built-in bias.

Comments on the paper

1 comment

Dear Reviewers,

The authors have posted their rebuttal. Please review their responses and the comments from your fellow reviewers, and provide any further feedback. Thank you.

The AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.