acceptodds
Withdrawn from ICLR 2027

ArtMesh: Part-Aware Articulated Mesh Fields with Motion-Consistent Dynamics

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We present ArtMesh, a mesh-native method for reconstructing articulated objects explicitly as connected triangle meshes with per-part rigid motion from multi-view images in start and end states. Existing 3D Gaussian Splatting pipelines for articulated reconstruction inherit the unstructured point-based geometry of their splatting base, which provides no surface topology for reasoning about part boundaries or enforcing motion consistency along the object's connectivity. ArtMesh instead builds on a mesh-based differentiable rendering backbone, enabling part-aware dynamics to act directly on the structured topology. To make the topology compatible with articulation, we introduce part-aware restricted Delaunay remeshing, producing connected submeshes whose triangles do not cross semantic part boundaries. The dynamic mesh field then optimizes articulation using bidirectional Vertex-wise Motion Consistency on transported mesh vertices and Pixel-wise Motion Consistency on rendered RGB-D observations. We introduce Articulate-100, a new benchmark of 100 articulated objects spanning 16 PartNet-Mobility categories. On this benchmark, ArtMesh outperforms prior 3DGS-based pipelines in joint-axis estimation and part-level geometric reconstruction, with the largest gains on objects with many movable parts.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Articuated Object, 3D Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vnVdOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ArtMesh reconstructs articulated objects from multi-view RGB-D observations at two articulation states and part segmentation information. Built on MeshSplatting, it performs restricted Delaunay remeshing independently within each part to eliminate cross-part triangles. It then fixes the reconstructed meshes and estimates per-part rigid motions using bidirectional vertex-wise and pixel-wise consistency, with backward transformations defined as analytic inverses of the forward transformations. The paper also introduces Articulate-100, a benchmark of 100 objects, and evaluates on this benchmark, PARIS, and additional real captures. With ground-truth segmentation, the method achieves strong joint-axis estimation and part-level geometry results.

Critical strengths
  1. The mesh-native formulation is a useful contribution to articulated object reconstruction, with part-aware remeshing improving motion estimation and part geometry.

  2. Articulate-100 provides a useful benchmark for evaluating reconstruction across different part counts.

Critical weaknesses
  1. Limited methodological novelty. The technical advance over MeshSplatting and existing articulation frameworks, including ArtGS, GaussianArt, and REArtGS++, is insufficiently established. Part-wise application of existing remeshing, rigid transformations, and consistency losses appears incremental, and the paper does not clearly isolate a substantial contribution beyond their integration.

  2. Unclear attribution of gains to representation versus segmentation. Although Table 1 states that the baselines receive ground-truth masks, how ArtGS incorporates this supervision is unclear. The comparison therefore does not convincingly separate the benefits of mesh connectivity from part assignments. Furthermore, the predicted-mask variant changes the optimization pipeline, confounding the segmentation-robustness evaluation.

  3. Missing relevant baselines. Comparisons with Part²GS [1], ArticulatedGS [2], and SplArt [3] are missing. Furthermore, adapting LARM’s [4] part-mesh extraction and correspondence-based joint estimation pipeline to the same RGB-D observations and part masks would provide a useful baseline for assessing the benefits of ArtMesh’s proposed formulation.

  4. Insufficient evidence of simulation readiness. The Isaac Sim demonstration and self-penetration metric provide limited validation of reliable physical interaction. Table 4 reports higher penetration for the directly exported mesh than for the baselines’ TSDF outputs, leaving the claimed advantage of simulation-ready assets without post-processing insufficiently supported.

  5. Limited real-world validation of multi-part reconstruction. The real-world evaluation centers on single-joint objects: two PARIS objects and two illustrated SplArt examples. This provides limited evidence that the claimed advantages on complex, multi-part objects transfer to real captures.

  6. Insufficient justification of the acquisition requirements. The method requires multi-view RGB-D observations at two articulation states, camera poses, and part information. Although estimated-input experiments are provided, the paper does not sufficiently establish when the reconstruction benefits justify this acquisition burden or which practical applications require this operating regime.

  7. Unclear practical benefit of the mesh-native representation. Although ArtMesh improves Chamfer distance, several qualitative examples exhibit noisier surfaces than ArtGS, and visible artifacts remain in the supplementary video. More importantly, its native mesh has worse self-penetration than all three TSDF-extracted baselines (Table 4), with the best result obtained only after TSDF processing. The simulation video also labels its assets as “postprocessed meshes.” These results leave the claimed advantage in directly usable mesh quality insufficiently established.

References

[1] Yu et al. Part²GS: Part-aware Modeling of Articulated Objects using 3D Gaussian Splatting. CVPR 2026.

[2] Guo et al. ArticulatedGS: Self-supervised Digital Twin Modeling of Articulated Objects using 3D Gaussian Splatting. CVPR 2025.

[3] Lin et al. SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splatting. ICCV 2025.

[4] Yuan et al. LARM: A Large Articulated-Object Reconstruction Model. SIGGRAPH Asia 2025.

Questions for authors
  1. What is the central technical innovation beyond combining MeshSplatting with established part-motion modeling and consistency objectives? Which controlled comparison best demonstrates the benefit of this innovation over a straightforward integration?

  2. How exactly are ground-truth part masks used by ArtGS? Could you evaluate an ArtGS variant with explicitly mask-constrained part assignments? For predicted masks, could you report results with the main optimization pipeline unchanged to isolate the effect of segmentation quality?

  3. Could you compare against Part²GS, ArticulatedGS, and SplArt? Additionally, could you adapt LARM’s part-mesh extraction and joint estimation pipeline to the same RGB-D inputs and part masks, with joint-type assumptions explicitly controlled, to isolate the benefits of your mesh-native optimization?

  4. What collision geometry and preprocessing are used in Isaac Sim? Can you provide quantitative evidence of collision validity or interaction success for the directly exported assets, separately from TSDF-processed outputs?

  5. How many SplArt objects were evaluated beyond the two illustrated examples? Can you provide real-world multi-part results to substantiate the claimed benefits on complex articulated objects?

  6. Which concrete application scenarios justify acquiring posed multi-view RGB-D observations at two articulation states? What evidence shows that the resulting fidelity or functionality compensates for this acquisition cost relative to methods with less demanding inputs?

2 comments
AuthorsOpenReview ↗

Reply to Reviewer

Summary of responses

  1. Central innovation (Q1). Replacing only our mesh backbone with a Gaussian one raises Axis Ang from 3.48° to about 47° and CD-m from 15.01 to 117 to 134 mm (Supp. Table 9). Without per-part restricted Delaunay, Axis Ang is 7.02°; without the analytic-inverse cycle, 10.25°.
  2. Masks (Q2). ArtGS and ReArtGS++ use only the number of parts, not per-pixel masks (Table 1 caption error). With part centers initialized from GT masks, ReArtGS++ reaches 3.60° vs 3.48° for ArtMesh, and ArtMesh has lower CD-m (15.01 vs 54.21 and 84.31 mm) (Table R3). With the same predicted PArtSAM masks, GaussianArt goes from 34.45° to 50.49° and CD-m from 38.43 to 277.73; ArtMesh reaches 10.42° with Part Motion unchanged (3.58). Isolating segmentation quality with an unchanged pipeline was not feasible within the rebuttal period.
  3. Additional baselines (Q3). Unfortunately is not feasible within the rebuttal period; LARM's training data overlaps our test set, and we do not have resources for retraining within the rebuttal period.
  4. Collision validity (Q4). The Isaac Sim video uses TSDF meshes with convex hulls. The direct export with no TSDF step reaches 74.4% pull-open success in SAPIEN vs 67.2% ArtGS, 65.6% ReArtGS++ and 39.3% GaussianArt on their TSDF meshes, each with its own predicted joints (Table R2); its penetration is the same with GT joints (Table R1).
  5. Real captures (Q5). On all 9 SplArt real captures, ArtMesh predicts the correct joint type on 9/9 objects.
  6. Acquisition (Q6). Two-state posed RGB-D is the standard setting; with Depth-Anything-V2 depth, Axis Ang goes from 3.26° to 5.84°, and masks are predicted by PArtSAM.
  7. Mesh quality (Weakness 7). Under the same TSDF pass, ArtMesh has the highest normal consistency (0.865 vs 0.794 and 0.708) and the lowest penetration (0.0043) (Table R7, Table 4).

We thank the reviewer for the detailed review and the specific experimental requests.

A. Central innovation. The backbone swap in Supp. Table 9 is the requested "straightforward integration" comparison: our pipeline is kept and only the mesh backbone is replaced. The two non-trivial components are isolated by their own ablations: per-part restricted Delaunay (a global remesh lets the consistency losses pull vertices across joints) and the analytic-inverse bidirectional cycle. Vertex-wise consistency is analyzed in our response to gh3r Q1 (Tables R4, R5).

B. Masks. We thank the reviewer for catching an error in the Table 1 caption: ArtGS and ReArtGS++ do not consume per-pixel masks; they take only the number of parts and initialize part centers by clustering. To separate the representation from the part assignment, we re-ran both with part centers initialized from GT masks (Table R3). ArtMesh also keeps a lower CD-s (18.98 vs 23.65 and 34.94 mm).

Table R3. Articulate-100, all methods with GT part masks, same protocol as Table 1. Lower is better.

Axis AngPart MotionCD-sCD-m
GaussianArt-GTMask34.4516.4824.6338.43
ArtGS-GTMask8.442.3534.9484.31
ReArtGS++-GTMask3.604.3623.6554.21
ArtMesh-GTMask3.483.6318.9815.01

For predicted masks, we ran GaussianArt with the same PArtSAM masks as Ours-PArtSAM on all 100 objects. ArtMesh's Axis Ang rises from 3.48° to 10.42° with Part Motion unchanged, whereas GaussianArt degrades sharply, so with predicted masks ArtMesh is about 5x more accurate in axis angle and part motion (GaussianArt Part Motion 20.31). Without GT masks, ArtMesh-PArtSAM also has the lowest CD-m (58.52 vs 151.41 ArtGS and 98.94 ReArtGS++, Table 1).

Running the predicted-mask setting with the main optimization pipeline unchanged was not feasible within the time and compute of the rebuttal period. With improved segmentation models this adaptation may become unnecessary, but current models do not yet solve the challenging problem of part segmentation on their own, which is why the predicted-mask variant includes our articulation-aware PArtSAM frontend.

C. Additional baselines. Part²GS, ArticulatedGS and SplArt each require per-object training on all 100 objects and, in several cases, data-pipeline adaptation; with the other requested experiments this exceeded our compute in the rebuttal period, so we prioritized concerns shared across reviews. LARM is trained on PartNet-Mobility data that overlaps with our test set, so its released model would not give a fair comparison, and retraining on a disjoint split is beyond our resources.

AuthorsOpenReview ↗

Reply to Reviewer

D. Collision geometry and interaction success. The Isaac Sim scene uses our TSDF meshes ("postprocessed meshes") with a convex-hull collision shape per part; for the window, self-collision between its thin adjacent panels is disabled. For the directly exported assets, we ran a SAPIEN interaction study on all 100 objects (Table R2). Every direct export loads and simulates stably, and with no TSDF step and our own predicted joints it opens more joints than the baselines' TSDF meshes, each driven by its own predicted joints. It is not guaranteed to be watertight; we recommend the TSDF pass when watertight collision geometry is required.

Table R2. SAPIEN 2.2: fraction of joints a constant force (20 N prismatic, 5 N m revolute) opens over at least 90% of their range. CoACD collision geometry, all collision pairs enabled. Each method is driven by its own predicted joint motion, with the joint range taken from GT; GT: PartNet meshes with GT joints.

GTGaussianArt + TSDFArtGS + TSDFReArtGS++ + TSDFOurs (direct)
Pull-open success (%) ↑91.339.367.265.674.4

Penetration of the direct export is identical with our predicted joints and with GT joints and is already present at the two observed states, so it comes from seam-level surface noise, not joint error; after TSDF it drops from 0.0494 to 0.0043 (Table 4, Table R1).

Table R1. Self-penetration along the joint sweep (21 states), mean over 100 objects, as % of the direct export's maximum over the sweep. Absolute SPD is in Table 4 (direct 0.0494, TSDF 0.0043).

meshmotionmax over s (%)s=0 (%)s=1 (%)intermediate (%)
direct exportGT100798399
direct exportours100798299
+ TSDFours8678

E. Real-world evaluation. We evaluated all 9 SplArt real captures (the paper shows two), all single-joint. Joint type is correct on 9/9 for ArtMesh, 9/9 for ReArtGS++, 8/9 for ArtGS (toolbox predicted prismatic) and 4/9 for GaussianArt (every hinge predicted prismatic). Real multi-part captures with two scanned states and reference joints are scarce, and building such a dataset is not feasible in the rebuttal period.

F. Acquisition requirements. Posed multi-view RGB-D at two states is the standard setting of the compared methods and matches how digital-twin assets are built: an object is scanned closed and opened with a handheld or robot-mounted sensor. With Depth-Anything-V2 depth, Axis Ang rises moderately and COLMAP poses dominate the remaining error (Supp. Table 5). Part masks are predicted by PArtSAM (point B).

G. Mesh quality. The baselines' native output is a set of Gaussian ellipsoids, not a surface (Appendix A, Fig. 8). Their TSDF mesh is fused afterwards from rendered depth, a lossy proxy that loses thin parts and absorbs floaters, while our exported mesh is exactly what ArtMesh optimizes. Under the same TSDF pass, ArtMesh gives the best surface agreement (Table R7) and the lowest self-penetration (Table 4: 0.0043 vs 0.0247 ArtGS, 0.0128 GaussianArt, 0.0100 ReArtGS++).

Table R7. Normal consistency with GT parts at state 0 (symmetric mean absolute cosine to the nearest-point normal; higher is better).

methodall partsmovable parts
GaussianArt + TSDF0.7940.771
ArtGS + TSDF0.7080.658
ArtMesh + TSDF0.8650.870
Reviewer_qwY6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ArtMesh, a framework for reconstructing articulated objects based on explicit triangular meshes. It recovers an object’s 3D geometry, part segmentation, and joint motion parameters from multi-view RGB-D images captured across two different motion states. The method employs part-aware Delaunay remeshing and bidirectional motion consistency constraints to jointly model geometry and motion, ultimately producing triangular meshes ready for use in simulations. The authors also present "Articulate-100," a dataset comprising 100 articulated objects. Experimental results demonstrate that ArtMesh outperforms existing 3D Gaussian Splatting (3DGS)-based methods in joint axis estimation and part-level geometric reconstruction, particularly for complex, multi-part objects.

Critical strengths
  1. The mesh-native representation offers clear practical value. Unlike 3DGS, an explicit mesh provides direct access to vertices and their connectivity, making it better suited for handling part boundaries and eliminating the need for repeated TSDF surface extraction from Gaussian primitives. The proposed part-aware remeshing is a sound design choice, preventing erroneous triangular connections between distinct rigid parts.
  2. There is a significant quantitative improvement for multi-part objects. The authors evaluate performance not only on simple two-part objects but also by categorizing objects based on their number of components, demonstrating that ArtMesh holds a clear advantage for objects with three or more parts—notably in joint axis estimation and the reconstruction of movable parts.
  3. The supervision strategy for motion consistency is comprehensive. Vertex-level 3D consistency complements pixel-level RGB consistency, and the bidirectional computation helps mitigate biases inherent in unidirectional matching.
Critical weaknesses
  1. The core methodology shows limited innovation, appearing primarily as a combination of existing techniques.
  2. The framework relies too heavily on ground-truth (GT) part masks, which may compromise its effectiveness in real-world scenarios.
  3. The claim that the output is "simulation-ready" lacks sufficient validation. To substantiate this, the authors should at least evaluate the meshes for watertightness, manifold properties, and self-intersections, as well as their success rate when loaded into simulators like SAPIEN or MuJoCo.
  4. The paper lacks an analysis of computational efficiency and overhead.
Questions for authors
  1. Performance drops significantly when ground-truth masks are replaced by predicted PArtSAM masks. Given that robustness to complex, multi-part objects is a core contribution, how do the authors demonstrate the method's effectiveness in real-world scenarios where ground-truth masks are unavailable?
  2. A key motivation for ArtMesh is to generate meshes that are ready for immediate use in simulators, without the need for post-processing. However, as shown in Table 4, the self-penetration depth of the raw ArtMesh output (0.0494) is higher than that of all baselines processed via TSDF; the best result (0.0043) is only achieved after applying TSDF. Could the authors clarify which specific properties render the direct output "simulation-ready"?
  3. Does the ArtMesh optimization process require more time compared to the baselines?
2 comments
AuthorsOpenReview ↗

Reply to Reviewer

Summary of responses

  1. Novelty (Weakness 1). Replacing only our mesh backbone with 3DGS or 2DGS raises Axis Ang from 3.48° to about 47° and CD-m from 15.01 to 117 to 134 mm (Supp. Table 9).
  2. Reliance on GT masks (Weakness 2, Q1). With PArtSAM-predicted masks, ArtMesh reaches 10.42° Axis Ang and 3.58 Part Motion (ArtGS: 10.53°, 3.50). GaussianArt with the same masks reaches 50.49° and 20.31. With Depth-Anything-V2 depth, Axis Ang goes from 3.26° to 5.84°.
  3. Simulation readiness (Weakness 3, Q2). All 100 direct exports load and simulate in SAPIEN, with 74.4% pull-open success vs 67.2% ArtGS, 65.6% ReArtGS++ and 39.3% GaussianArt, each on its TSDF mesh and with each method's predicted joints (Table R2). Penetration of the direct export is the same with GT joints (Table R1). After TSDF, ArtMesh has the lowest penetration (0.0043 vs 0.0100 to 0.0247) and the highest normal consistency with GT parts (0.865 vs 0.794 GaussianArt, 0.708 ArtGS).
  4. Runtime (Weakness 4, Q3). About 32 min per object on one A10, vs 18 to 40 min for the baselines.

We thank the reviewer for recognizing the practical value of the mesh-native representation and the multi-part gains.

A. Novelty. In the backbone swap of Supp. Table 9, only the mesh backbone is replaced and everything else is fixed. Together with the part-aware remeshing ablation (7.02° without it) and the analytic-inverse backward pass (10.25° without it), this isolates what the mesh-native, part-aware formulation contributes. ArtMesh is a novel articulated reconstruction pipeline that estimates joints on the same triangle surface it exports, so no mesh extraction step separates the optimized geometry from the final asset.

B. Reliance on ground-truth inputs.

Predicted part masks. Ours-PArtSAM in Table 1 uses masks predicted by our articulation-aware PArtSAM frontend (Appendix D), which separates movable from static surfaces using the motion between the two states. It is on par with ArtGS and far ahead of GaussianArt with GT masks (34.45°, 16.48); on two-part objects it matches the GT-mask variant (1.74° vs 2.02°). We additionally ran GaussianArt with the same predicted masks on all 100 objects: its Axis Ang degrades from 34.45° to 50.49° and its CD-m grows sevenfold (38.43 to 277.73), while ArtMesh's Part Motion is unchanged (3.63 to 3.58) and its CD-m rises from 15.01 to 58.52, still far below GaussianArt's. With predicted masks, ArtMesh is about 5x more accurate in axis angle and part motion.

Real captures. For the real-world captures (Fig. 7 / Fig. 13), part masks were labeled on only a subset of the training views.

Predicted depth and estimated poses. Supp. Table 5 uses Depth-Anything-V2 depth and COLMAP poses on 42 StorageFurniture objects. With predicted depth, Axis Ang rises moderately while Part Motion improves (5.16 to 2.38); estimated poses account for most of the remaining error.

AuthorsOpenReview ↗

Reply to Reviewer

C. What makes the direct output simulation-ready? The direct export (i) is a set of per-part triangle meshes in which no triangle crosses a part boundary, so each part moves as one rigid body; (ii) carries an estimated joint for every movable part, so it is written as a URDF with no manual step; and (iii) loads and simulates stably in SAPIEN for all 100 objects. In a SAPIEN interaction study that includes the baselines' TSDF meshes and the GT PartNet assets under the same protocol, our direct export, with no TSDF step and our own predicted joints, opens more joints under a constant pull than the baselines' TSDF meshes, each driven by its own predicted joints (Table R2). It is not guaranteed to be watertight; we recommend the TSDF pass when watertight collision geometry is needed.

Table R2. SAPIEN 2.2: fraction of joints a constant force (20 N prismatic, 5 N m revolute) opens over at least 90% of their range. CoACD collision geometry, all collision pairs enabled. Each method is driven by its own predicted joint motion, with the joint range taken from GT; GT: PartNet meshes with GT joints.

GTGaussianArt + TSDFArtGS + TSDFReArtGS++ + TSDFOurs (direct)
Pull-open success (%) ↑91.339.367.265.674.4

Self-penetration. Table 4 (0.0494 for the direct export) compares an unprocessed mesh with the baselines' TSDF meshes, which are fused afterwards from rendered depth. Under the same TSDF pass, ArtMesh has the lowest penetration (0.0043 vs 0.0247 ArtGS, 0.0128 GaussianArt, 0.0100 ReArtGS++). Table R1 decomposes the penetration of the direct export: it is identical with our predicted joints and with GT joints, it is already present at the observed states, and it disappears after TSDF, so it comes from surface noise at part seams, not joint-axis error. ArtMesh has no explicit collision term; we state this as a limitation, and a penetration penalty along the joint sweep is a natural extension.

Table R1. Self-penetration along the joint sweep (21 states), mean over 100 objects, as % of the direct export's maximum over the sweep. Absolute SPD is in Table 4 (direct 0.0494, TSDF 0.0043).

meshmotionmax over s (%)s=0 (%)s=1 (%)intermediate (%)
direct exportGT100798399
direct exportours100798299
+ TSDFours8678

Mesh quality. After TSDF, all methods produce manifold meshes. ArtMesh + TSDF has the highest normal consistency with the GT parts at state 0 (0.865 vs 0.794 GaussianArt and 0.708 ArtGS; 0.870 vs 0.771 and 0.658 on movable parts; Table R7 in our response to vnVd).

D. Runtime. On one NVIDIA A10, ArtMesh takes about 32 min per object (about 2 min PArtSAM + 30 min reconstruction and articulation), vs about 18 min for ArtGS, 30 min for GaussianArt and 40 min for ReArtGS++, each including its TSDF mesh extraction. ArtMesh needs no mesh extraction step, because the optimized mesh is the output.

Reviewer_gh3rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The ArtMesh study reconstructs articulated objects from multi-view RGB-D images captured in two joint states. The method first reconstructs meshes for each state separately and performs independent restricted Delaunay remeshing using component labels to prevent triangles from connecting different moving components. Subsequently, the geometry and topology are frozen, and the rigid-body motion of each component is estimated through vertex-level and pixel-level consistency. Inverse motion is obtained by analytically inverting the forward transformation. The final output consists of component meshes and joint parameters. Additionally, the paper introduces Articulate-100, a dataset comprising 100 objects, and validates the method on this dataset, the PARIS dataset, and a selection of real-world objects.

Each component of the model uses a rigid-body transformation, and triangles spanning multiple components are not allowed, thereby preventing non-rigid deformation of connecting surfaces as different components move. The formula for the analytical inverse transformation holds, and the supplementary materials explain how to determine joint types by comparing rotation and translation candidates, as well as how to handle the issue of non-unique rotation axis origins along the axis direction.

Critical strengths
  1. The component topology design has a clear geometric rationale and is supported by corresponding ablation results. Limiting remeshing to the interior of a single component aligns directly with the segmented rigid-body motion assumption. In Table 3 of the main manuscript, after disabling part-aware remeshing, the Axis Ang increased from 3.48° to 7.02°, and the Chamfer Distance of the moving part increased from 15.01 mm to 19.30 mm. This indicates that this constraint plays a practical role in both motion estimation and the reconstruction of moving parts.

  2. Under accurate segmentation conditions, the multi-part reconstruction results are robust, and the experiments also cover several important practical factors. Additionally, in Supplementary Table 7, the PSNR values for Ours-GTMask and ArtGS are close, at 41.66 and 41.64, respectively, while the geometric error of the moving parts is significantly lower, indicating that the improvements go beyond mere enhancements in rendering appearance. In conclusion, this work is novel and meaningful.

Critical weaknesses
  1. Vertex-wise motion consistency (VMC) is a key component of the motion consistency learning mechanism proposed in this paper. However, supplementary Table 9 shows that its effectiveness varies across different metrics. The results indicate that VMC improves motion magnitude and the geometric accuracy of moving parts, but does not demonstrate a consistent gain in axial direction estimation. Pixel consistency, on the other hand, plays a more significant role in optimization. Therefore, the current experiments support the notion that VMC plays a supplementary role, but they do not yet fully explain which specific types of motion estimation challenges it addresses, nor why the various vertex constraints need to be used in combination.

  2. The model ensures rigid-body consistency within components, but constraints regarding the geometric validity of motion between components are still lacking. In Table 4, the Self-Penetration Depth of the original output mesh is 0.0494, which decreases to 0.0043 after TSDF, indicating a significant difference. The paper has not explained the primary causes of mutual penetration among these different components during motion, neither can it guarantee high-quality motion collision handling.

Questions for authors
  1. How can we explain the differences among the various ablations for vertex consistency? In Table 3, removing either the color or opacity constraints individually results in a significant decrease in axial accuracy, but in Table 9, removing the entire VertexLoss does not lead to a decline in this metric. Were the loss weights, activation times, and joint type selection processes consistent across all experiments? What type of error do these results indicate that VMC primarily improves?

  2. Where do mesh penetrations primarily occur in the direct output, and what are the causes? In Table 4, there is a significant difference in SPD between the mesh and the mesh after TSDF. Do these penetrations primarily occur in the two observed endpoint states or in intermediate motion states? Are they primarily caused by component surface errors or by joint axis estimation errors? Once these questions are clarified, I’d be willing to adjust the score.

1 comment
AuthorsOpenReview ↗

Reply to Reviewer

Summary of responses

  1. VMC (Weakness 1, Q1). The vertex loss helps most when GT masks are not available. With PArtSAM-predicted masks, removing the vertex loss raises Axis Ang from 10.42° to 14.41° and the number of objects above 30° from 12 to 22; removing only V-Color or only V-Opacity gives 17.27° and 16.11° (Table R4). All ablations share the same loss weights, schedule and joint-type selection. The full model has the lowest error on revolute, prismatic and textureless parts (Table R5).
  2. Penetration (Weakness 2, Q2). Penetration of the direct export is the same with our predicted joints and with ground-truth joints, and about 80% of it is already present at the two observed states; after TSDF it drops from 0.0494 to 0.0043 (Table 4, Table R1). It comes from seam-level surface noise, not joint-axis error.
  3. Collision handling (Weakness 2). ArtMesh has no inter-part collision term; we state this as a limitation.

We thank the reviewer for the precise questions and for offering to revisit the score once they are clarified.

Q1. Table 3 vs Supp. Table 9, and what VMC fixes. With GT masks, the masks already constrain each part's motion well, so the vertex loss changes Axis Ang little. Its main benefit appears when GT masks are not available: Table R4 ablates it on runs with predicted masks, and Table R5 breaks the result down by joint type and on textureless parts, where the task is harder. All ablation runs use the same loss weights, activation schedule and joint-type selection as the full model; only the named term is removed. Removing only one of the two terms hurts more than removing both: one term alone gives an ambiguous correspondence that can pull a part toward the wrong match.

Table R4. VMC ablation, PArtSAM masks, 100 objects. Last column: objects with Axis Ang above 30°. Lower is better.

Axis AngAxis PosPart MotionCD-sCD-m#obj>30°
Full10.420.153.5826.1458.5212
w/o V-Color17.270.116.7126.0366.1524
w/o V-Opacity16.110.115.9029.0461.9620
w/o VertexLoss14.410.123.7128.2059.4722

Table R5. Breakdown of Table R4 (112 revolute, 175 prismatic movable parts). Textureless: the third of movable parts (95) with the lowest image-gradient energy in their GT mask. Last two columns: revolute / prismatic parts above 30°. Unlike Table R4, which averages per object, these columns average and count per part, so objects with many parts weigh more; this is why the per-type means exceed the overall Axis Ang and why more parts than objects exceed 30°.

Rev AngPris AngTexless AngTexless MotionRev>30°Pris>30°
Full13.8715.7413.144.911329
w/o V-Color20.2723.6217.317.152348
w/o V-Opacity20.2422.4418.699.742247
w/o VertexLoss18.2521.7617.435.682344

VMC supplies a 3D signal on the articulation where the photometric loss is weak. When a moving part is flat or uniformly colored, a small rotation or translation changes few pixels, so the pixel-wise loss constrains the motion only loosely. VMC transports every vertex through its part's motion and compares it with the same part's surface in the other state, penalizing these errors directly in 3D. Color identifies which surface point should land where, and opacity identifies which triangles belong to the moving surface, so either alone leaves the correspondence ambiguous.

Q2. Where do penetrations occur, and are they surface or axis errors? We decomposed self-penetration by mesh source, motion source and articulation state (Table R1). Penetration in the direct export is identical whether our parts move with our predicted joints or with the ground-truth joints, and it is already present at the two observed states, with only a small increase at intermediate states. It therefore comes from surface noise at part seams, not from joint-axis error. After the TSDF pass, penetration with our predicted joints drops from 0.0494 to 0.0043 (Table 4).

Table R1. Self-penetration along the joint sweep (21 states), mean over 100 objects, as % of the direct export's maximum over the sweep. Absolute SPD is in Table 4 (direct 0.0494, TSDF 0.0043).

meshmotionmax over s (%)s=0 (%)s=1 (%)intermediate (%)
direct exportGT100798399
direct exportours100798299
+ TSDFours8678

Collision handling. ArtMesh enforces rigid consistency within each part but has no term that keeps parts from intersecting during motion. A penetration penalty along the joint sweep or contact-aware joint estimation is a natural direction for future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.