acceptodds
Under review as a conference paper at ICLR 2027

VideoArtGS: Building Digital Twins of Articulated Objects from Monocular Video

Abstract

Building digital twins of articulated objects from monocular video requires jointly recovering part geometry and articulation from entangled camera and object motion. Although 3D point tracks provide motion cues, their noise and lack of explicit part structure complicate articulation learning. To address this problem, we introduce VideoArtGS, a motion-prior-guided framework that turns these tracks into structured initialization for joint geometry-and-motion reconstruction. Our pipeline analyzes trajectories under articulation constraints, filters unreliable tracks, and groups them to initialize part centers, joint parameters, and an articulation-based deformation field. We also design a hybrid center-grid part-assignment module that represents movable parts with learnable centers and the static base with a flexible spatial grid. We then jointly optimize 3D Gaussians and the deformation field using rendering and tracking supervision. VideoArtGS achieves state-of-the-art performance in articulation estimation and mesh reconstruction on iTACO-S, iTACO-L, and VideoArtGS-20 datasets. On VideoArtGS-20, which contains objects with up to nine movable parts, mean joint-axis error decreases from to , a reduction of over two orders of magnitude. Visualizations are available at: https://videoartgs-2026.github.io.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Articulated Object Reconstruction, Real to Sim, Digital Twins

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dnADOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents VideoArtGS, a framework for reconstructing articulated objects from monocular video. It fits motion patterns to 3D tracks, filters unreliable trajectories, and clusters the remaining tracks to initialize part centers and joint parameters. A hybrid assignment module combines center-based scores for movable parts with a hash-grid score for the static base. The initialized deformation field and canonical Gaussians are then jointly optimized using rendering and tracking supervision. The paper reports joint-estimation and mesh-reconstruction results on iTACO-S, iTACO-L, and the newly curated VideoArtGS-20 benchmark.

Critical strengths
  1. VideoArtGS reports substantial improvements in joint estimation and part-level reconstruction on the evaluated benchmarks, including objects with multiple movable parts. The paper also provides component ablations, threshold sensitivity analysis, tracking evaluation, efficiency comparisons, and failure cases.
  2. Converting noisy 3D tracks into explicit part and articulation initialization is a useful design. The ablations demonstrate the importance of coherent part-center initialization. They also support the effectiveness of the hybrid assignment module: replacing it with center-based assignment substantially degrades movable-part reconstruction.
Critical weaknesses
  1. The Abstract and Introduction frame the task as jointly recovering geometry and articulation from entangled camera and object motion. However, iTACO-S uses ground-truth depth and camera poses, while VideoArtGS-20 begins with 150 static frames from multiple viewpoints before articulation. Thus, the strongest quantitative results are obtained under substantially simplified conditions, whereas the fully RGB-only real-world setting is evaluated only qualitatively.
  2. It remains unclear whether existing reconstruction methods could achieve comparable performance when provided with the proposed track-derived initialization. Evaluating a prior reconstruction framework with the same initialized part centers and articulation parameters would clarify how much of the reported performance gap can be closed by transferring this initialization.
Questions for authors
  1. The paper presents articulated reconstruction from a single monocular video. But the current evidence is insufficient to establish applicability to uncontrolled monocular videos. How does VideoArtGS perform when depth and camera poses are estimated from RGB, and when the initial static segment is shortened or provides less viewpoint coverage? Quantitative results under these conditions would help clarify the method’s practical scope.
  2. Could the authors evaluate a representative reconstruction baseline with the proposed track-derived part-center and articulation initialization, under matched geometry inputs and a comparable optimization budget?
  3. The hybrid center-grid design is motivated by improved representation of complex static geometry. Could the authors report part-assignment accuracy, or analyze the w/o hybrid results across objects with different static-base complexity to directly test this explanation for the observed gains?
4 comments
AuthorsOpenReview ↗

Response to Reviewer dnAD

We thank Reviewer dnAD for the questions on scope, component attribution, and hybrid assignment.

1. Predicted geometry and static initialization

Synthetic experiments use GT depth and poses under the benchmark protocol. Real captures use RGB-estimated geometry (Sec. 4.3, Appendix C.3) and are evaluated qualitatively because GT part meshes and joint parameters are unavailable.

The 150-frame static segment is a shared acquisition condition across the comparisons, not extra input reserved for VideoArtGS. It initializes canonical geometry while the camera moves around the stationary object. Alternatives include first-frame points or later-frame points warped back through the initialized dynamic field. In our experience, these are feasible but give somewhat lower geometric fidelity.

2. Initialization and hybrid assignment

Track-derived initialization is itself a contribution, with center, axis, and deformation-field initialization separately ablated in Table 3. Cross-method transfer is therefore unnecessary for validating these component claims.

Table 3 directly tests whether initialization alone explains the gains: retaining the same initialization but removing hybrid assignment increases CD-m from 0.26 to 10.35 cm, axis error from 0.34° to 1.21°, and position error from 0.10 to 2.51 cm. This establishes the additional contribution of hybrid assignment.

3. Why a spatial grid for the static base?

Movable parts form coherent motion clusters, whereas the static base may be extended, non-convex, or enclose movable parts. Our dedicated static hash-grid branch models spatial membership without relying on a static-center distance prior, while retaining center-based scores for movable parts. This differs from ArtGS's combination of center distances and residual assignment scores.

For enclosing structures, a center-distance prior can favor a nearby movable part for static points, coupling assignment and articulation errors. The spatial branch addresses this limitation. Table 3 supports its reconstruction benefit, although it does not directly measure assignment IoU or performance stratified by base complexity.

These requested extensions do not negate the contributions demonstrated by the controlled ablations and substantial quantitative gains. We respectfully ask that this evidence receive due weight and that you reconsider your assessment and consider raising your score.

Thank you for the response and clarifications. However, my two main concerns remain unresolved.

  1. Shared GT inputs ensure fair comparisons but do not establish effectiveness to predicted depth/poses or shorter static observations. These can be quantitatively evaluated on existing synthetic benchmarks.

  2. Internal ablations do not replace the requested cross-method comparison with shared initialization, which is necessary to isolate its contribution to the reported gains.

  3. The explanation is reasonable, but direct assignment accuracy would better support the claimed mechanism.

Overall, I appreciate the clarifications, but the response does not provide sufficient evidence to resolve my primary concerns. I maintain my current rating for now.

Thank you for the follow-up. We now provide direct assignment measurements in addition to the reconstruction ablations, and respectfully clarify the remaining points.

1. Predicted geometry and static observations

Our additional DA3 camera audit finds a median positional RMSE of 0.764 m on the canonical segments even after optimal Sim(3) alignment.DA3 produces meter-scale camera-pose errors on these background-free synthetic sequences, despite performing reasonably on our real-world captures. Using these unreliable estimates would confound upstream pose-estimation failures with downstream reconstruction performance. This limitation of DA3 should not be attributed to VideoArtGS or negate the reconstruction gains established by our fair comparisons.

The static segment is a shared acquisition condition, rather than extra input reserved for VideoArtGS. We agree that this establishes comparison fairness, not robustness to shorter observations. Additional first-frame-initialization and no-static-image evaluations are running; complete quantitative results are not yet available. This limits the demonstrated operating range, but does not invalidate the gains under the reported protocol.

2. Initialization attribution

We disagree that cross-method initialization transfer is necessary to validate our contributions. Transfer into another adapted algorithm addresses a different question and should not be a prerequisite for recognizing those contributions.

3. Direct assignment accuracy

We evaluate the models on identical GT canonical-surface points: 20,000 samples per part with three seeds, surface-area weighting, and Hungarian matching of moving-part labels while fixing the static label. Means and medians are computed across all 20 objects; assignment metrics below are percentages.

Metric (%)Mean: w/o hybridMean: FullMedian: w/o hybridMedian: Full
Static-part IoU ↑86.0091.6292.0895.19
Moving-part mIoU ↑71.5083.0082.0590.44
All-part mIoU ↑75.2884.6884.4390.70
Assignment accuracy ↑90.1694.8093.6896.31
Static-to-moving misassignment ↓5.782.071.021.22
Moving-to-static misassignment ↓19.3912.778.907.66

All-part mIoU improves on 20/20 objects, with mean and median gains of 9.39 and 6.26 percentage points. Mean static-to-moving misassignment decreases from 5.78% to 2.07%. On the nine-movable-part table (30666), it decreases from 68.73% to 2.62%, directly supporting improved separation of this complex static structure.

These results provide the direct assignment evidence requested and complement the reconstruction gains. We respectfully ask that this additional evidence receive due weight and that you reconsider your assessment and raise your score.

AuthorsOpenReview ↗

Addtional experiments for canonical Gaussian init

We have completed additional experiments on all 20 objects. Directly initializing Gaussians from first-frame points, achieves comparable performance to the original model (axis error 0.35°; CD-w 0.08 cm). In the variant excluding all initial static images, articulation remains accurate (0.38° axis error and 0.13 cm position error), while mesh error increases (CD-w 0.49 cm). These results demonstrate robustness to the alternative initialization and suggest that the initial observations primarily benefit mesh completeness. Adequate surface coverage remains important for high-quality geometry.

Both additional variants share the same motion initialization and use 20,000 joint-optimization steps. Results are mean ± standard deviation over all 20 objects; lower is better.

SettingAxis (°)Position (cm)CD-w (cm)CD-m (cm)CD-s (cm)
Original model0.34 ± 0.800.10 ± 0.100.09 ± 0.090.26 ± 0.610.24 ± 0.58
Direct first-frame initialization; static views retained0.35 ± 0.810.10 ± 0.120.08 ± 0.090.24 ± 0.450.25 ± 0.69
Direct first-frame initialization; static views removed0.38 ± 0.800.13 ± 0.120.49 ± 0.581.08 ± 3.280.57 ± 0.69
Reviewer_sUuzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper reconstructs articulated Gaussian representations from videos. It uses tracked 3D motion to initialize parts and joints, and combines different spatial representations for static and movable regions. Experiments evaluate geometry reconstruction and articulation estimation on existing datasets and a new multi-part dataset.

Critical strengths

Multi-part articulated reconstruction is an important problem. Recovering both geometry and articulation has clear value for digital twins and interactive simulation.

The results show substantial improvements in movable-part geometry and joint estimation. Tracking-based initialization is well motivated, and the ablations indicate that initialization is consequential to performance.

Critical weaknesses

The main quantitative evidence does not establish fully automatic monocular-RGB performance. The iTACO-S evaluation uses ground-truth depth and camera poses, and incorrect structural predictions are manually corrected before reconstruction. These are legitimate controlled settings, but they do not measure the complete system under its intended input conditions.

The breadth and attribution of the improvements require clarification. In Table A3, the large whole-object CD improvement is dominated by one baseline failure assigned 100.00. Across the other 19 objects, the rounded entries average approximately 0.0716 for iTACO and 0.0868 for the proposed method. This does not invalidate the failure reduction or articulation improvements, but it limits claims of uniformly better geometry. A comparison with ArtGS using the same tracking-based initialization is also needed to isolate the new representation’s contribution.

Questions for authors

What are the reconstruction errors and failure rates with predicted depth, predicted cameras, and no manual correction of structural predictions?

Can you separate failure rates from successful-case geometry errors and report medians alongside means?

How much improvement remains over ArtGS when both methods use the same tracking-based initialization and supervision?

3 comments
AuthorsOpenReview ↗

Response to Reviewer sUuz

We thank Reviewer sUuz for the careful review and questions on evaluation and component attribution.

1. Predicted depth and poses

Synthetic experiments use GT depth and poses to isolate articulated reconstruction from upstream estimation errors (Sec. 4.1). Real-world experiments use RGB-estimated depth and poses (Sec. 4.3, Appendix C.3); evaluation is qualitative because these captures lack GT part meshes and joint parameters. Thus, the quantitative results concern the controlled benchmark setting, while real videos demonstrate applicability with predicted inputs.

2. Structural predictions and manual correction

Appendix C.7 reports GPT-4o errors in 2/73 iTACO-S and 3/20 VideoArtGS-20 cases. Corrections concerned only discrete part counts or joint types, never geometry or continuous joint parameters. All compared methods received corrected part counts. GPT-6 Sol correctly resolves all five previously failing cases without manual correction.

The VLM is a replaceable structure-selection front end. Joint types could instead be selected by fitting revolute and prismatic hypotheses, as in iTACO. Given the same discrete structure, the downstream model and objective remain unchanged. We use a VLM to accommodate varied objects and part configurations, including part counts that two-type enumeration alone does not determine.

3. CD-w and medians

We agree that object 103015 strongly affects iTACO's mean CD-w. The following statistics are recomputed from the rounded per-object values in Table A.3:

MetricAll-20 Mean (iTACO / Ours)Excl. 103015 Mean (iTACO / Ours)All-20 Median (iTACO / Ours)
CD-w (cm)5.068 / 0.0950.072 / 0.0870.065 / 0.075
CD-m (cm)30.630 / 0.25926.979 / 0.27223.010 / 0.020
CD-s (cm)10.220 / 0.2405.494 / 0.2383.080 / 0.095
Axis (°)48.879 / 0.33548.570 / 0.34951.245 / 0.135

Excluding 103015, iTACO has a slightly lower mean CD-w; its all-object median CD-w is also lower. However, CD-w does not measure part ownership or articulation. Our part-geometry and axis errors remain substantially lower under both summaries: excluding 103015, mean CD-m, CD-s, and axis errors are approximately 99×, 23×, and 139× lower, respectively.

4. ArtGS and component attribution

ArtGS assumes two discrete states; adapting it to continuous video introduces additional design choices. Transferring our initialization to that adapted system is therefore unnecessary for validating its contribution here.

Table 3 directly tests whether initialization alone explains the gains: retaining the same initialization but removing hybrid assignment increases CD-m from 0.26 to 10.35 cm, axis error from 0.34° to 1.21°, and position error from 0.10 to 2.51 cm. This establishes the additional contribution of hybrid assignment.

We respectfully submit that the concerns about input protocols, structural prediction, and whole-object statistics do not overturn the central contributions. In light of these clarifications and the additional statistical analysis, we would be grateful if you would reconsider your assessment and consider raising your score.

Thank you for the response.

  1. The CD-w concern is resolved. Good job.

  2. The reconstruction errors and failure rates with predicted depth, predicted cameras concern is not resolved, which is shared by other 2 reviewers. Appendices C.7 reports VLM error counts but no reconstruction results without manual correction, and there are no quantitative results with predicted depth and poses, which can be measured on the synthetic benchmarks, where ground truth is available.

  3. The comparative experiment I requested: running existing methods using your initialization scheme has not been performed. Reviewer dnAD also noted this. While the "w/o hybrid" ablation experiment may serve as an alternative provement, it has a CD-m score of 10.35 ± 23.84. The error values ​​are non-negative, a standard deviation more than twice the mean indicates a highly skewed data distribution. Is this becasue the mean primarily driven by a small number of very hard samples? Please provide the median results or list the specific results for each individual sample.

I keep my rating for now.

Thank you for resolving the CD-w concern. We respectfully disagree that the remaining requests negate our contributions.

  1. In our tests, DA3 produces meter-scale camera-pose errors on these background-free synthetic sequences, despite performing reasonably on our real-world captures. Using these unreliable estimates would confound upstream pose-estimation failures with downstream reconstruction performance. This limitation of DA3 should not be attributed to VideoArtGS or negate the reconstruction gains established by our fair comparisons.

  2. GPT-6 Sol correctly resolves all five documented GPT-4o errors, automatically supplying the same structural inputs used in those reconstruction experiments. Running with the earlier incorrect predictions would give no valid information.

  3. ArtGS assumes two discrete states and requires adaptation for continuous video, which is the task addressed by VideoArtGS. Initialization transfer is therefore unnecessary for validating our component claims.

  4. We report all 20 per-object CD-m results below, with no objects excluded. Hybrid assignment reduces the median from 0.05 to 0.016 cm (67.5%) and improves CD-m on 19/20 objects.

Object IDw/o hybrid CD-m (cm)Full model CD-m (cm)
100481100.000.03
1012840.000.00
1012870.000.00
1018080.020.01
1019080.040.01
1030150.010.01
10381123.562.64
104895.410.01
106550.040.01
12800.160.03
16850.540.42
254930.320.18
306669.451.16
312490.440.23
451940.060.01
455030.020.01
456120.030.02
4764816.910.33
89610.030.02
90160.020.02
Mean10.350.26
Median0.050.02

The large mean and variance reflect severe reconstruction errors without hybrid assignment, including on complex objects such as the six-movable-part printer (103811) and storage furniture (47648). Their CD-m errors decrease from 23.56290 to 2.64428 cm and from 16.90878 to 0.33288 cm, respectively. Hybrid assignment therefore improves robustness on these difficult objects, while the lower median and improvements on 19/20 objects show that its benefit extends beyond the high-error cases.

We respectfully ask you to reconsider your assessment and raise your score in light of these clarifications and the additional per-object results.

Reviewer_ecY4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces VideoArtGS, a 3D Gaussian Splatting based pipeline for reconstructing interactable digital twins of articulated objects from monocular input videos. The core insight is to leverage existing 3D point trackers to supply motion priors. The authors clean up noisy tracking trajectories and run clustering to bootstrap part centers and joint kinematics, which serves as initialization for their articulation-aware deformation field. To fix known issues of pure center-based part segmentation for objects with complex static bases, they propose a hybrid center-grid assignment strategy. Movable parts are modelled via learnable centers, while a hash grid handles the static base region. Two tracking losses, canonical-to-observation and observation-to-observation, are used to regularize the deformation field, which is jointly optimized alongside 3D Gaussians. The authors also put forward VideoArtGS-20, a new test set focusing on complex multi-part articulated objects with up to nine movable joints. They benchmark against existing methods iTACO and ArticulateAnything over iTACO-S, iTACO-L and their new dataset and observe clear performance improvements. A full suite of ablations validates each design choice, and they also show qualitative reconstruction outputs from real-world videos captured with a consumer mobile phone.

Critical strengths
  1. Empirical gains are notable across different benchmarks. The method works well for both simple single-joint cases and more complex multi-part objects. Joint-axis error sees very large reduction, on the order of two orders of magnitude. Chamfer distance for both static background bases and movable object parts consistently outperforms compared baselines.

  2. Technical motivations make practical sense. Reconstructing articulated objects from monocular video is an ill-posed problem. Instead of solving everything from scratch, the authors make good use of off-the-shelf 3D tracking outputs to get reasonable starting points for part structure and joint parameters. This substantially eases optimization difficulty. The hybrid center-grid assignment directly targets failure cases for static parts with irregular geometry. The two tracking losses together improve temporal consistency for the learned deformation field. Design rationales are mostly well explained in the main text.

  3. Experiments and ablations are reasonably thorough. The ablation table clearly demonstrates the importance of motion-prior initialization, hybrid part assignment, and tracking-loss terms. The appendix contains plenty of supporting material: tracking quality analysis, hyper-parameter sensitivity tests on fitting thresholds, runtime and GPU memory profiling, and discussions on observed failure cases. Per-object detailed metrics are provided for all datasets. Releasing VideoArtGS-20 is a useful contribution for follow-up research within the community.

  4. Paper is well-organized and easy to follow. Figure 1 gives a high-level but clear overview of the whole pipeline. Related work properly covers both dynamic 3D Gaussian reconstruction and prior work on articulated-object reconstruction. Real-world phone-captured examples show that the approach has promising practical potential.

Critical weaknesses
  1. There are non-trivial upstream dependencies plus hard practical assumptions that limit real-world deployment. The pipeline assumes the input monocular video starts with several static frames, which are required to build the canonical 3D Gaussian representation. This is a big limitation for many real-world online videos, where the object is already moving from the very first frame. In addition, multiple external modules are needed: depth and camera pose estimators, 3D point trackers like TAPIP3D, plus a VLM (GPT-4o) to predict discrete structural inputs including part count and joint types. Severe errors coming from any upstream component will propagate and ruin final reconstruction quality. While the authors acknowledge this limitation in the appendix, I wish to see more quantitative evaluation showing how performance degrades under noisy upstream inputs. Right now we only have qualitative discussion.

  2. Part segmentation depends almost exclusively on motion signals. When different parts have subtle relative motion or share similar movement patterns, motion-based clustering can mistakenly merge parts or produce incorrect segmentation boundaries. Appearance or semantic cues are not incorporated at all. The appendix already presents concrete failure examples stemming from this exact issue.

  3. Real-world evaluation is limited to qualitative visual results. All large-scale quantitative benchmarking is done on synthetic datasets. Real-world mobile examples only show renderings, without accompanying quantitative metrics. Therefore it is hard to judge how robust the method truly is under real capture noise.

Questions for authors
  1. Could you add quantitative robustness tests against upstream noise? For example, inject controlled amounts of noise into camera poses, depth maps and 3D point tracks, then report changes in joint-axis error and Chamfer distance.

  2. From ablation results, removing joint-axis initialization only leads to moderate performance drop, but removing part-center initialization causes catastrophic failure. Can you give an intuitive explanation for this large gap in importance between center initialization and axis initialization?

  3. Your approach relies heavily on initial static frames to set up canonical Gaussians. So, what directions could be explored to relax these static-frame requirements so that this method can be applied to monocular video of any natural scene?

2 comments
AuthorsOpenReview ↗

Response to Reviewer ecY4

We thank Reviewer ecY4 for recognizing our improvements in joint estimation and part reconstruction, as well as our initialization and hybrid assignment designs.

1. Robustness to upstream noise and real-world evaluation

Our pipeline addresses noisy 3D tracks through kinematic fitting, outlier rejection, and joint optimization. Two existing analyses provide relevant evidence:

  • Track refinement (Appendix C.4): Jointly optimizing the articulated deformation field reduces mean 3D tracking EPE from 0.16 m to 0.09 m on VideoArtGS-20.
  • Fitting-threshold sensitivity (Appendix C.5): Increasing the line/circle fitting threshold from 0.01 to 0.1 changes CD-m from 0.26 to 1.17 cm and axis error from 0.34° to 2.32°. Reducing the threshold to 0.005 rejects too many trajectories, increasing these errors to 14.57 cm and 5.13°. This analysis illustrates the trade-off between retaining informative motion cues and rejecting inconsistent tracks.

These analyses evaluate track refinement and fitting sensitivity; they do not constitute independent camera, depth, and track-noise perturbation sweeps. Our real-world captures (Sec. 4.3, Appendix C.3) additionally demonstrate reconstruction using depth, poses, and tracks estimated from RGB video. We evaluate these captures qualitatively because they lack ground-truth part meshes and joint parameters. The synthetic benchmarks provide quantitative evaluation under controlled geometry inputs, while the real captures demonstrate applicability with estimated inputs.

2. Why removing center initialization degrades performance much more than removing axis initialization

VideoArtGS optimizes part segmentation and articulation jointly, so an incorrect initial grouping affects both tasks. Removing center initialization severely degrades part assignment: trajectories from different components can be grouped together, giving the articulation model inconsistent motion constraints. Incorrect motion estimates then make segmentation refinement harder. Removing axis initialization preserves the motion-derived part grouping, allowing trajectory supervision to refine the joint axes within a substantially better-posed optimization problem.

Table 3 supports this distinction: removing center initialization increases axis error from 0.34° to 20.64° and CD-m from 0.26 to 83.32 cm, whereas removing axis initialization gives 0.60° and 0.25 cm, respectively. Thus, the large center-initialization gap reflects the coupling between segmentation and articulation learning.

3. Static initialization fairness and practical alternatives

Here, “static” means that the object remains in one articulation state while the camera moves. The initial static segment is part of the shared acquisition protocol used for the comparisons, rather than additional observations available only to VideoArtGS. Its role is to provide canonical geometry before learning articulation.

This segment can be relaxed by initializing canonical Gaussians from (i) first-frame depth alone or (ii) points from later frames warped back to the first frame using the inverse of the initialized dynamic field. In our experience, both are viable but yield somewhat lower geometric fidelity. Limited initial surface visibility and inaccuracies in the initial deformation field can affect the resulting geometry and subsequent joint optimization. These are practical alternatives, rather than a quantitative static-segment ablation reported in the paper.

4. Weak or indistinguishable part motion

Weak but nonzero relative motion makes joint estimation poorly conditioned and sensitive to noise. In the limiting case where a part never moves, its joint parameters are not constrained by observed motion; if two parts move rigidly together throughout the video, motion cues alone cannot distinguish their independent articulation. These are observability limitations of motion-based reconstruction. Semantic or appearance information could help separate the parts, but recovering an unobserved joint would still require additional priors or observations.

We hope these explanations clarify the evidence for our design choices and the conditions under which motion-based reconstruction is reliable. If these clarifications address your concerns, we would be grateful if you would consider raising your score in support of the work.

AuthorsOpenReview ↗

Addtional experiments for canonical Gaussian init

We have completed additional experiments on all 20 objects. Directly initializing Gaussians from first-frame points, achieves comparable performance to the original model (axis error 0.35°; CD-w 0.08 cm). In the variant excluding all initial static images, articulation remains accurate (0.38° axis error and 0.13 cm position error), while mesh error increases (CD-w 0.49 cm). These results demonstrate robustness to the alternative initialization and suggest that the initial observations primarily benefit mesh completeness. Adequate surface coverage remains important for high-quality geometry.

Both additional variants share the same motion initialization and use 20,000 joint-optimization steps. Results are mean ± standard deviation over all 20 objects; lower is better.

SettingAxis (°)Position (cm)CD-w (cm)CD-m (cm)CD-s (cm)
Original model0.34 ± 0.800.10 ± 0.100.09 ± 0.090.26 ± 0.610.24 ± 0.58
Direct first-frame initialization; static views retained0.35 ± 0.810.10 ± 0.120.08 ± 0.090.24 ± 0.450.25 ± 0.69
Direct first-frame initialization; static views removed0.38 ± 0.800.13 ± 0.120.49 ± 0.581.08 ± 3.280.57 ± 0.69

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.