acceptodds
Withdrawn from ICLR 2027

ORMA: Optimization-based Monocular 4D Reconstruction of Articulated Animals

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recovering articulated 4D representations of animals from monocular videos remains challenging due to the large diversity of quadruped morphologies and lack of animal 4D supervision data. Existing learning-based reconstruction methods operate on individual images and rely on synthetic or model-fitted 3D supervision, which inherits the constraints of strong parametric priors and limits generalization to out-of-distribution species. When applied to out-of-distribution animals, they often recover a plausible pose while producing inaccurate geometry because the underlying shape model cannot faithfully represent the observed instance. We present ORMA, a training-free reconstruction framework that decouples articulation from shape, using the predicted pose as reference for optimization while leveraging generative 3D priors for accurate shape reconstruction. Given a reference image, we reconstruct the animal geometry and register it to the parametric model SMAL+, yielding an articulated shape adapted to the observed instance. We then combine per-frame articulated pose estimates with globally consistent camera poses to recover animal motion in a shared world coordinate frame, and further refine the reconstruction using self-supervised DINO correspondences and temporal consistency. To enable quantitative evaluation, we introduce PAW4D, a synthetic multi-species benchmark with ground-truth 3D geometry and camera motion. Experiments on PAW4D, PFERD, and challenging in-the-wild videos demonstrate that ORMA improves reconstruction accuracy while recovering globally consistend animal motion across diverse quadruped species.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
4D Reconstruction, Animal Motion, Articulated Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_FrFaOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

ORMA combines generative shape priors, a parametric animal model, semantic correspondence, and video-based optimization to reconstruct animal geometry and world-space motion. The paper also introduces a synthetic benchmark with ground-truth geometry and camera information.

Critical strengths

World-space animal reconstruction is an important and challenging problem. Combining instance-specific geometry with structured articulation is a reasonable way to address the limitations of fixed shape models. The proposed benchmark could support useful evaluation. Ground-truth geometry and camera motion enable more direct assessment of global reconstruction than image-space metrics alone.

Critical weaknesses

The differentiability of the core RSC objective is unclear. Equations 3–4 compare fixed vertex descriptors with fixed image features over a local pixel neighborhood. Under a discrete-pixel interpretation, the loss is locally independent of pose while neighborhood membership remains unchanged. Continuous feature sampling or another differentiable dependence may resolve this, but it is not specified. This is a substantive gap in a central optimization component.

The experiments do not establish the claimed contribution. Table 3 contradicts the statement that the full model is best across all metrics: removing DINO improves Chamfer from 0.005 to 0.004. Results with predicted cameras are mixed, while much stronger results require ground-truth cameras. A controlled comparison against the AniMer+ initializer with the same depth, camera, and optimization losses is missing.

Questions for authors

Through which explicit computational path does the RSC loss produce gradients with respect to pose? Please provide the missing formulation and a gradient verification.

What improvement remains over AniMer+ when camera estimates, depth supervision, and optimization objectives are matched?

How are scale and alignment handled in world-space trajectory evaluation? Is one alignment applied to the entire sequence, or are frames aligned separately?

Reviewer_HmQ2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this work, the authors propose ORMA, a training-free optimization framework for world-grounded 4D animal reconstruction from monocular videos. In addition, they introduce PAW4D, a new synthetic benchmark with diverse ground-truth annotations.

Critical strengths
  • Dataset and benchmark: The authors introduce PAW4D, a new dataset that provides ground-truth 3D geometry, camera parameters, masks, and depth across diverse camera settings (follow, fixed, and orbit) for 4D reconstruction. This benchmark enables quantitative evaluation of both 3D reconstruction accuracy and world-space animal motion, which are difficult to assess using only 2D observations.

  • Training-free integration: ORMA effectively combines powerful pretrained models without requiring additional task-specific training. The integration of generative shape priors, articulated representations, and semantic correspondences provides a practical solution to the challenging problem of monocular 4D animal reconstruction.

  • Temporally coherent reconstruction: ORMA recovers animatable animal shapes and world-space motion from monocular videos. Unlike methods that focus primarily on per-frame reconstruction, the proposed method jointly considers geometry, articulation, and temporal consistency, enabling coherent 4D reconstruction from in-the-wild videos.

Critical weaknesses
  • Missing video results: Although the paper addresses world-grounded 4D animal reconstruction, no video results are provided. Video results are essential for assessing temporal coherence, particularly the consistency of animal motion in world coordinates. Without such results, it is difficult to verify the claimed temporal coherence of the reconstructions.

  • Missing SOTA baselines: Limited comparisons with recent SOTA methods make it difficult to assess the performance gains and advantages of the proposed framework. Comparisons with methods such as 4DEquine and SAM 3D Animal are missing. Although these methods have different objectives, evaluating their shape and pose reconstruction quality would help clarify the advantages of ORMA.

  • Unclear evaluation protocol: It is unclear whether the baseline models were evaluated on PAW4D in a zero-shot setting or retrained/fine-tuned on this dataset. Clarifying this protocol would help ensure fair comparisons and improve the reproducibility of the benchmark results.

  • Camera dependency: Performance improves substantially when ground-truth camera parameters are provided, suggesting that camera estimation is a major bottleneck. Further analysis of reconstruction quality under different camera motions and camera estimation errors would help characterize the robustness of ORMA.

  • Limited gains: The ablation results show relatively small improvements from Hunyuan3D, DINO, and SMAL+. A more detailed analysis of when and how these components improve reconstruction quality would better justify their individual contributions.

Questions for authors
  • Why is 4DEquine not evaluated on PFERD?
  • How does ORMA compare with SAM 3D Animal in shape and pose reconstruction?
  • Were the baseline models evaluated zero-shot on PAW4D, or were they retrained/fine-tuned?
  • How does ORMA perform on animal keypoint estimation benchmarks such as APT-36K?
  • How robust is ORMA to camera estimation errors?
  • What is the inference time per video?
  • How does ORMA perform on other animal species, such as cats, deer, and lions? Could the authors provide qualitative reconstruction results for these species?
Reviewer_xRb7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ORMA reconstructs articulated animals from monocular video by combining instance-specific geometry generated by Hunyuan3D with a SMAL+ articulation model. It initializes pose using AniMer+, estimates camera motion and depth using MegaSaM, and optimizes pose and world-space placement with geometric, semantic, and temporal objectives. A DINO-based robust semantic correspondence term is proposed to guide alignment. The paper also introduces PAW4D, a synthetic four-category benchmark, and evaluates on PAW4D and real PFERD sequences.

Critical strengths
  1. Combining instance-adapted geometry with explicit articulated kinematics is a useful approach to the limited expressivity of parametric animal shape spaces. The real-data geometry results support this motivation: on PFERD, Table 1 reports IoU3D of 0.348 and Chamfer distance of 0.149 with estimated cameras, compared with 0.204 and 0.407 for AniMer. These are meaningful improvements even though not all pose and image metrics improve.
  2. PAW4D provides controlled geometry and camera ground truth for evaluating a difficult problem, and the paper usefully separates estimated-camera and ground-truth-camera results. This exposes a substantial camera-related bottleneck rather than hiding it.
Critical weaknesses
  1. I could not identify the gradient path for the proposed RSC loss. In Eqs. (3)–(4), the baked surface descriptors and image features are fixed. If the neighborhood contains discrete image pixels, the loss is locally independent of pose while membership stays unchanged. Continuous sampling around the projected vertex could resolve this, but neither p.6 nor Appendix A.3 specifies that operation. This matters because RSC is presented as a contribution to pose optimization; it is a clarification of the actual computation, not a claim that the implementation is necessarily wrong.
  2. The world-space evaluation and comparison protocol need more detail (pp.7–9, Tables 1–3). Alignment, scale handling and trajectory units are not specified, and the camera setting for the six-sequence ablation is unclear. On PAW4D, estimated-camera IoU3D is 0.215 versus 0.212 for the best baseline, whereas GT cameras raise it to 0.422. The latter is clearly labeled and useful, but the distinction makes the evaluation protocol important. The paper discusses video optimization methods such as 4D-Animal, yet their absence from the geometry/pose comparison is unexplained; the AniMer baseline also differs in name from the AniMer+ initialization. These gaps limit attribution of the observed gains, without negating the stronger PFERD results.
Questions for authors
  1. What exact continuous operation connects projected vertex coordinates to Eq. (4)? Please specify the sampling coordinates and gradient path used in the experiments.
  2. What alignment, scale convention and units are used for the world-space metrics, and which cameras are used in Table 3? Per-sequence results for the existing ablation would help interpret the small differences.
  3. Is the AniMer baseline the same implementation as the AniMer+ initializer? Could you explain whether a representative video optimizer such as 4D-Animal can be evaluated with comparable inputs on the common geometry/pose metrics, or what prevents that comparison?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.