acceptodds
Under review as a conference paper at ICLR 2027

ArtModeling: An Instance-Matched Dataset and Benchmark for Articulated-Object Perception and Dynamics

Abstract

Simulation assets are widely used to evaluate articulated-object perception and interaction, but performance measured in simulation does not by itself establish accuracy on physical objects. Controlled comparison requires real observations, detailed digital assets, and recorded motion linked to the same physical instances. We introduce ArtModeling, an instance-matched dataset and benchmark pairing 497 physical objects across 43 categories with articulated digital assets, within a collection of 1,500 simulation-ready models. Each paired instance combines calibrated multi-state RGB-D observations, camera poses, part-level masks, an articulated digital asset, and recorded interaction motion. The assets are manually modeled from independent scans and physical measurements, capturing detailed part geometry and occluded structures. Registration to the metric frame of the real observations enables evaluation on real and rendered inputs with matched viewpoints and joint states against a shared geometric reference. The benchmark evaluates articulation estimation from real observations, performance differences under matched real and rendered inputs, and physical consistency on held-out interactions. Across the evaluated reconstruction methods, geometric rankings remain relatively stable across input domains, while joint-estimation rankings change, with larger discrepancies for methods using RGB-D inputs. Motion-based calibration improves trajectory agreement on held-out interactions. ArtModeling provides a reusable basis for assessing how simulation findings extend to real articulated objects, enabling controlled evaluation of articulated perception and dynamics against shared physical references.

Primary area:
datasets and benchmarks
Keywords:
articulated objects, 3D reconstruction, paired dataset, real-to-digital

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ZkRVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ArtModeling links 497 physical articulated objects from 43 categories to detailed simulation assets, within a collection of 1,500 assets. It provides registered RGB-D captures, matched rendered views, part and joint annotations, and interaction recordings. The paper uses these data to compare articulation methods on real and synthetic inputs and to test whether physical calibration improves agreement with a second recording of the same joint motion.

Critical strengths
  1. The main value of the dataset, in my view, is the correspondence between 497 physical objects, detailed assets, and registered captures. Independently modeled geometry and recorded interactions provide useful references for checking reconstruction and simulated behavior. That combination gives the collection clear value as a benchmark resource.
  2. I found the real-versus-rendered comparison informative. It holds object identity, camera views, joint states, and reference assets fixed, yet the rankings change for joint and part-pose estimation. This adds a concrete finding about evaluation under real sensing conditions and is a main reason I lean toward acceptance.
Critical weaknesses
  1. The optimization benchmark includes 153 of the 497 paired objects, restricted to cases where all four methods produce valid outputs. Without per-method completion rates and a clear account of which objects were attempted and excluded, I cannot judge how representative this subset is. Omitting failures may change the method rankings and the conclusions about the gap between real and synthetic inputs.
  2. The dynamics test covers 36 objects and a second repetition of the same joint motion in the same direction. Because the applied forces are inferred from motion, different force and parameter combinations can produce similar trajectories. I see evidence of improved consistency under this setup, but not enough to judge parameter accuracy or transfer to different speeds, loads, or directions.

Missing references: PD²GS: Part-Level Decoupling and Continuous Deformation of Articulated Objects via Gaussian Splatting (ICLR 2026), Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Dataset (CVPR 2025).

Questions for authors
  1. How many of the 497 paired objects were attempted and completed by each method, and why were the others excluded? Please report failure rates on real and synthetic inputs and whether accounting for these failures changes the main comparisons.
  2. How is the excitation for the second interaction obtained independently of its target trajectory? Does any existing evaluation test a different speed, load, or motion direction and help distinguish parameter accuracy from fitting a particular motion?
  3. What does ArtModeling add beyond earlier collections pairing real observations with digital assets, and which paired captures, complete assets, and evaluation scripts will be released? These details affect my assessment of its value as a reusable benchmark.
4 comments
AuthorsOpenReview ↗

Response to Reviewer ZkRV 1/2

We thank the reviewer for the valuable feedback and for recognizing the significance of the instance-level correspondence between real-world objects, digital assets, and registered observations, as well as the informative findings from real-versus-synthetic evaluations.

Below, we provide detailed responses to the reviewer's specific comments and questions:

W1 & Q1: Evaluation Subset Selection Bias

A1: We thank the reviewer for raising this concern. The 497 real-world object–digital asset pairs represent the full ArtModeling dataset across 43 categories, rather than the number of objects attempted in this baseline evaluation. For the unified comparison of four methods, we selected 170 pairs through category-stratified sampling, averaging approximately four objects per category while covering both revolute and prismatic joints. Of these, 153 formed the original common valid evaluation set, and 17 previously excluded objects were added for the expanded analysis. The remaining 327 pairs belong to the full dataset but were not selected for this baseline comparison; they are not method failures. We will make the full dataset and evaluation protocol available to support broader community benchmarking.

The original common evaluation set contained 153 objects with valid outputs from all four methods (105 revolute and 48 prismatic). We additionally evaluated 17 previously excluded objects (10 revolute and 7 prismatic). The remaining 327 candidate objects were not selected for evaluation and are therefore not counted as method failures. We intend to make the broader paired dataset available to support further benchmarking.

We count a run as a failure if it fails to produce an evaluable mesh after training. The additional objects include scissors, knives, sunglasses, and objects with moving parts made of transparent materials such as glass or acrylic. Across the 170 attempted objects, overall failure rates (Real/Sim) are 4.1%/4.1% for PARIS, 3.5%/2.9% for DigitalTwinArt, 1.2%/2.4% for ArtGS, and 2.4%/3.5% for SplArt.

Accuracy and Real–Sim Gaps Across the 170 Attempted Objects.   We calculate Mean CD = (CD-s + CD-m + CD-w) / 3. Each metric cell reports Real / Sim / (Real − Sim); a positive difference indicates higher error on Real inputs. Axis and revolute-joint Motion are measured in degrees. Pivot and prismatic-joint Motion are measured in meters.

Table 1. Method-wise completion counts, reconstruction errors, and Real–Sim differences in aggregate means across 170 attempted objects.

Joint TypeMethodCompleted (Real; Sim)Mean CD ↓Axis ↓Pivot ↓Motion ↓
RevolutePARIS109/115; 109/11511.56 / 7.86 / +3.7024.480 / 20.839 / +3.6410.156 / 0.066 / +0.09020.330 / 18.047 / +2.283
DigitalTwinArt110/115; 112/1159.72 / 1.48 / +8.2437.440 / 10.669 / +26.7710.684 / 1.831 / −1.14731.000 / 10.457 / +20.543
ArtGS114/115; 112/1153.95 / 1.29 / +2.6625.780 / 16.114 / +9.6660.367 / 0.366 / +0.00124.250 / 16.202 / +8.048
SplArt112/115; 112/11538.02 / 6.17 / +31.8511.790 / 3.738 / +8.0520.440 / 0.142 / +0.29813.460 / 3.456 / +10.004
PrismaticPARIS54/55; 54/5522.66 / 9.10 / +13.5614.020 / 9.071 / +4.949—0.484 / 0.297 / +0.187
DigitalTwinArt54/55; 53/557.57 / 1.16 / +6.4114.400 / 9.931 / +4.469—0.233 / 0.030 / +0.203
ArtGS54/55; 54/555.17 / 0.97 / +4.2016.510 / 7.247 / +9.263—0.264 / 0.016 / +0.248
SplArt54/55; 52/5531.92 / 8.81 / +23.1110.460 / 4.979 / +5.481—1.229 / 0.172 / +1.057

 Expanded Evaluation. Compared with the original 153 common evaluation objects, the ranking for Mean CD on real inputs with prismatic joints changed, with PARIS moving from 4th to 3rd and SplArt moving from 3rd to 4th. For Pivot on revolute joints with synthetic inputs, PARIS moved from 2nd to 1st, SplArt from 3rd to 2nd, and ArtGS from 1st to 3rd. The rankings of all other metrics remained unchanged. On the original 153 common evaluation objects, 27 out of 28 comparable metrics showed higher errors on real inputs than on synthetic inputs. The only exception was PARIS's revolute-joint Motion metric (-0.02%). After expanding the evaluation to 170 objects, 27 out of 28 metrics again showed higher errors on real inputs, with the only exception being DigitalTwinArt's revolute-joint Pivot metric (Real: 0.684, Sim: 1.831).

AuthorsOpenReview ↗

Response to Reviewer ZkRV 2/2

W2 & Q2: Independent Excitation for the Second Interaction and Dynamics Validation Across Interaction Conditions

A2: We thank the reviewer for raising these concerns. We clarify the excitation protocol and provide additional dynamics experiments under varied interaction conditions.

  • We jointly optimize physical parameters and external excitation using the first interaction trajectory, assuming piecewise-constant force or torque between consecutive sampling times. For the second interaction, the inferred excitation is reused unchanged across all four parameter configurations, while the second recorded trajectory serves as an independent evaluation reference.

  • We further conducted nine real-world pulling experiments on three articulated objects. The suspended masses range from 150 to 350 g, and the additional drawer or tray loads range from 0 to 350 g. A pulley system applies excitation along the joint axis, and an optical-inertial motion capture system records the resulting displacement at 90 Hz. Images of the experimental setups are presented in the ArtModeling Dataset Website.

Table 2. Trajectory residuals from shared-parameter calibration under different suspended masses and additional loads.

ObjectSuspended Mass (g)Additional Load (g)RMSE (cm) ↓NRMSE (%) ↓
cabinet163002001.727.71
cabinet1635000.713.18
cabinet163503500.492.18
cabinet1725002.6115.10
cabinet1730001.407.92
cabinet173002001.729.76
letter_tray215000.985.00
letter_tray225001.346.21
letter_tray22501501.969.79
Mean——1.447.43

For each object, one shared physical parameter set is jointly calibrated across its three trials. The suspended masses provide nominal gravitational excitation, and motion-capture trajectories provide independent motion references. Across the nine trials, the shared parameter sets achieve a mean RMSE of 1.44 cm and NRMSE of 7.43%, demonstrating consistent motion reproduction across varied loading conditions. ArtModeling further integrates SolverAdapt, enabling users to inspect and adjust mass, density, contact friction, and joint-drive parameters, preview motion responses, and save adapted configurations to USD assets.

W3 & Q3: Relationship to Existing Datasets and Data Availability.

A3: We thank the reviewer for pointing out these relevant works. We will cite PD$^2$GS [1] (ICLR 2026), which introduces RS-Art, and Digital Twin Catalog [2] (CVPR 2025), and clarify their differences from ArtModeling.

  • Compared with RS-Art, which already provides simulation-ready articulated assets with joint annotations and mass, inertia, and friction properties, ArtModeling provides 497 instance-matched real-world articulated objects and extends its physical annotation and evaluation scope with stiffness and damping annotations for the dynamics subset, estimated joint trajectories from real-world interaction videos, and dedicated protocols for physical parameter calibration and dynamics evaluation.

  • Compared with Digital Twin Catalog, which primarily focuses on large-scale digital twins of general objects through high-fidelity geometry, photorealistic PBR materials, and real-world DSLR/egocentric image observations, ArtModeling specifically targets articulated objects. It provides articulation-aware annotations, including part segmentation and joint structures, together with real-world interaction trajectories and physical-property annotations, enabling systematic evaluation of articulated reconstruction, physical parameter calibration, and dynamics.

Data Availability. We will publicly release approximately 300 real-world–digital asset paired instances upon paper publication, including the corresponding real-world captures, digital assets, and annotations. The remaining paired instances will undergo further preparation and quality checks and will be released incrementally within the following two months. We will also release the associated evaluation protocols and scripts to facilitate reproducible research.

[1] Wang, Haowen, et al. "PD$^2$GS: Part-level decoupling and continuous deformation of articulated objects via Gaussian splatting." International Conference on Learning Representations. 2026: 12354–12379.

[2] Dong, Zhao, et al. "Digital Twin Catalog: A large-scale photorealistic 3D object digital twin dataset." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2025: 753–763.

I appreciate the authors' thorough response, which has resolved my key concerns. I will keep my rating as Weak Accept.

AuthorsOpenReview ↗

Response to Reviewer ZkRV

Thank you for your careful review and positive evaluation of our work. We appreciate your confirmation that our response resolved your key concerns and will incorporate the corresponding revisions and clarifications into the final manuscript.

Reviewer_yz87OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ArtModeling, a dataset and benchmark designed for the perception and physical dynamics evaluation of articulated objects. It comprises 1,500 simulatable digital assets across 43 categories, 497 of which are matched one-to-one with real-world object instances. For each matched instance, the dataset provides multi-view RGB-D images, camera poses, part segmentations, joint structures, and real-world interaction trajectories. The paper evaluates and compares various methods for articulated object reconstruction and pose estimation using both real and synthetic inputs.

Critical strengths
  1. The instance-matched real-synthetic dataset offers significant research value. By establishing instance-level correspondences for 497 real-world objects—along with a unified camera coordinate system, joint states, and digital assets—ArtModeling allows researchers to control object identity, camera viewpoint, and joint state, facilitating a direct analysis of performance discrepancies between real and synthetic inputs.
  2. The dataset is large-scale and richly annotated. It includes not only RGB-D data, multi-view camera poses, part masks, and articulated geometry, but also interaction trajectories, joint parameters, and part-level physical properties such as mass, friction, stiffness, and damping.
  3. The ground-truth (GT) geometry is constructed independently of the RGB-D captures used for evaluation. The authors developed the digital assets using a Lixel L2 Pro scanner, manual measurements, and professional 3D modeling, rather than by fusing meshes directly from the benchmark’s RGB-D input. This helps prevent the ground truth and the input data from sharing the same reconstruction errors.
  4. The real-synthetic benchmark reveals meaningful experimental insights. The study finds that the performance of a method on synthetic data does not necessarily reflect its performance on real-world data, even when the object, viewpoint, and joint state are identical.
Critical weaknesses
  1. Although the dataset features multi-joint structures, the real-world data collection only captures the motion of a single joint at a time—with one selected joint moving while the others remain fixed. Consequently, the real-world data does not fully cover the kinematic space of complex articulated objects.
  2. The physical validation within the dynamics benchmark remains relatively limited. The calibration experiments for physical parameters cover only 36 objects and are restricted to repetitive motions of the same type and orientation. Furthermore, the calibration relies on joint trajectories extracted via VideoArtGS, yet the impact of trajectory estimation error on the final physical parameters has not been sufficiently quantified.
Questions for authors
  1. Ground-truth (GT) geometric accuracy and evaluation reliability. Appendix D.3 reports a median surface distance of 1.06 cm between the real-world objects and the digital assets; however, some reconstruction methods have already achieved Chamfer Distances of a smaller magnitude. Could the authors provide further analysis on how measurement and registration errors in the GT geometry affect the final reconstruction metrics and the ranking of the algorithms?
  2. Accuracy of the actual motion trajectories. The authors use VideoArtGS to extract joint trajectories from interaction videos, which serve as a reference for the calibration and validation of physical parameters. How is the accuracy of these trajectories evaluated?
4 comments
AuthorsOpenReview ↗

Response to Reviewer yz87 1/2

We thank the reviewer for the valuable feedback and for recognizing the contributions of ArtModeling, particularly its instance-matched real-to-sim dataset, rich geometric and articulation annotations, independently constructed digital assets, and insightful findings from real-versus-synthetic evaluations.

Below, we provide detailed responses to the reviewer's specific comments and questions:  

W1: Insufficient Coverage of Multi-Joint Motion

A1: We appreciate the reviewer's insightful comment. Our controlled per-joint acquisition protocol is designed to systematically characterize the individual degrees of freedom of multi-joint articulated objects.  

  • Specifically, we actuate one movable joint at a time while keeping the remaining joints fixed. This isolates individual joint motions, avoids motion ambiguity between different parts, and establishes clear correspondences between articulated parts and their associated joints. It also facilitates accurate annotation of joint types, axes, and motion ranges.

  • In terms of configuration-space coverage, this protocol samples each joint's motion dimension independently while maintaining the other joints at fixed configurations, rather than exhaustively sampling combinations of all joint configurations. Combined with the complete multi-joint structures preserved in the corresponding digital assets, these observations support systematic evaluation of part-level articulation understanding and per-joint kinematic estimation.

We will clarify the acquisition rationale and evaluation scope in the revised manuscript.

Q1: GT Geometry Accuracy and Evaluation Reliability

A2: We appreciate the reviewer’s important concern regarding the geometric accuracy of our GT assets and its impact on benchmark reliability.

  • The reported 1.06 cm is the median surface discrepancy between real-world observations and their corresponding USD assets after registration. It is affected by depth-sensing noise, incomplete visibility, registration uncertainty, and asset discrepancies, rather than solely reflecting asset modeling error. In contrast, the benchmark CD measures reconstruction errors against the USD references. Since they compare different pairs of surfaces using different metrics, the 1.06 cm discrepancy should not be interpreted as a lower bound on the benchmark CD.

  • To further assess GT reliability, we select 12 objects from the common evaluation set and acquire additional scans using the same Lixel L2 Pro scanner employed in the original asset construction. These scans are not used in the original modeling or refinement. We compare the USD assets and all four methods against these scans under consistent joint states, physical scales, and surface regions, with three repeated registrations to quantify alignment sensitivity. Table 1 reports the resulting geometric errors, registration variations, and ranking changes.

Table 1. GT Geometry Sensitivity Analysis. CD–USD and CD–Scan denote the Chamfer Distance against the original USD reference and the independent scan, respectively. Reg. Range reports the minimum–maximum CD across three registration trials. Rank (USD→Scan) indicates the method ranking under the two references, where rank 1 denotes the lowest CD. All CD values use the same evaluation protocol and units.

DomainMethodCD–USD ↓CD–Scan ↓Reg. Range ↓Rank (USD→Scan)
—USD Asset (Ours)—1.1401.10–1.19—
RealArtGS0.4292.7492.68–2.821→1
RealDigitalTwinArt1.1333.0833.00–3.162→2
RealPARIS3.8426.3226.15–6.503→3
RealSplArt12.71815.35814.95–15.784→4
SimArtGS0.2592.3592.30–2.431→2
SimDigitalTwinArt0.2601.9101.86–1.972→1
SimPARIS5.1316.8516.67–7.043→3
SimSplArt11.21013.14012.80–13.504→4

On the audited 12 objects, the ordering is unchanged for real inputs. For synthetic inputs, ArtGS and DigitalTwinArt exchange the top two positions, indicating that close method comparisons can depend on the geometric reference.

AuthorsOpenReview ↗

Response to Reviewer yz87 2/2

W2 & Q2: Motion Trajectory Accuracy and Dynamics Validation Across Interaction Conditions

A3: We thank the reviewer for raising two related concerns. We address them separately below.

Calibration under Different Interaction Conditions (W2). ArtModeling contains 497 real-world–digital asset pairs. We selected 36 objects for dynamics evaluation, covering diverse joint types and a broad range of motion ranges. We further conducted nine real-world pulling experiments on three articulated objects, varying suspended masses from 150 to 350 g and additional drawer or tray loads from 0 to 350 g. A pulley system applies excitation along the joint axis, while an optical-inertial motion capture system records displacement at 90 Hz. Experimental setups are shown on the ArtModeling Dataset Website, and the corresponding trajectory residuals are reported in Table 2.

Table 2. Trajectory residuals from shared-parameter calibration under different suspended masses and additional loads.

ObjectSuspended mass (g)Additional load (g)RMSE (cm) ↓NRMSE (%) ↓
cabinet163002001.727.71
cabinet1635000.713.18
cabinet163503500.492.18
cabinet1725002.6115.10
cabinet1730001.407.92
cabinet173002001.729.76
letter_tray215000.985.00
letter_tray225001.346.21
letter_tray22501501.969.79
Mean——1.447.43

Motion Reproduction and Parameter Interpretation. With one shared parameter set jointly calibrated across the three trials of each object, the nine experiments achieve a mean RMSE of 1.44 cm and a mean NRMSE of 7.43%. We therefore interpret trajectory error as a measure of motion reproduction, rather than evidence of accurate recovery of each physical parameter. To support practical use under this parameter ambiguity, ArtModeling provides SolverAdapt Each digital asset provides a default physical parameter configuration, and SolverAdapt enables users to inspect and adapt it to specific application conditions and simulation platforms. Users can adjust mass, density, contact friction, and joint-drive parameters, assess the resulting behavior through motion response curves, and save the adapted configuration back to the USD asset. This integrated workflow provides both a default configuration and tools for its subsequent evaluation and adaptation.

VideoArtGS Trajectory Accuracy (Q2). We recorded the same real-world opening and closing sequences using a Noitom PN Hybrid optical-inertial motion capture system at 90 fps as independent motion references. One marker was attached to the fixed body and another to the moving part. Their relative positions, together with the joint geometry, were used to calculate the one-degree-of-freedom displacement or rotation angle. The motion-capture setup and corresponding real-world–simulation motion comparison are provided here, with VideoArtGS trajectory errors reported in Table 3.

Table 3: VideoArtGS trajectory errors relative to motion capture references across five scenarios.

Scenario/JointComplete CyclesMAE ↓RMSE ↓
cabinet1 / joint 131.76 cm2.20 cm
cabinet16 / joint 131.58 cm1.99 cm
cabinet17 / joint 121.63 cm2.09 cm
microwave1 / joint 124.52°6.74°
oven4 / joint 125.08°6.55°

For the three prismatic-joint scenarios, the average MAE across scenarios is 1.66 cm, and the average RMSE is 2.09 cm. For the two revolute-joint scenarios, the corresponding averages are 4.80° and 6.65°, respectively.

Trajectory Errors and Parameter Interpretation. Trajectory-estimation errors can influence both the inferred excitation and the fitted friction, stiffness, and damping. We therefore interpret the calibrated values as effective parameters conditioned on the observed trajectories and physical priors, providing a configuration for reproducing the recorded motion in simulation.

Thanks for the response; it has addressed my concerns. Considering the overall contribution of the paper, I maintain my original assessment: weak accept.

AuthorsOpenReview ↗

Response to Reviewer yz87

Thank you for your positive assessment of our work and our response. We appreciate your confirmation that our responses addressed your concerns and will revise the final manuscript accordingly.

Reviewer_8BoAOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

ArtModeling introduces an instance-matched dataset of real articulated objects and detailed digital assets, a pipeline for asset construction and physical-parameter calibration, and a benchmark for evaluating perception and dynamics across real and simulated settings.

Critical strengths

I appreciate the motivation behind this work. Simulated data may appear visually convincing, yet simulated behavior can differ from real-world dynamics. This dataset provides a valuable resource for assessing these discrepancies by pairing digital assets with observations and interaction recordings of the same physical objects.

Critical weaknesses

Physical parameters may not be uniquely identifiable and could be ambigious. The human-applied forces are not directly measured but are jointly optimized with friction, stiffness, and damping. Different combinations of forces and parameters may produce similar trajectories, so reduced trajectory error does not necessarily indicate accurate recovery of the underlying physical parameters.

The evaluation includes only 36 objects and uses a second repetition of the same joint motion in the same direction for testing. Without testing different speeds, loads, motion directions, or coordinated multi-joint interactions, it remains unclear whether the calibrated parameters generalize to new interaction conditions.

There is no video demo showcasing the dataset, making it difficult to assess the quality of the recorded dynamics. The visualizations in Fig. 11 also appear unclear. What is the image resolution of the real-world captures? There also appear to be color differences between the real observations and their instance-matched digital assets. It would be helpful to evaluate Astra’s articulated-object generation as well.

I would consider raising my score if additional comparisons, discussion and video demonstrations provided stronger evidence of the dataset’s visual and dynamic quality, and more diverse evaluation.

Questions for authors

Since applied forces are inferred jointly with physical parameters, how do you distinguish accurate parameter recovery from trajectory fitting through adjustments to the inferred forces?

Is it possible to evaluate the calibrated parameters under different speeds, loads, or motion directions to assess generalization beyond repeated interactions?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 8BoA 1/2

We thank the reviewer for the valuable feedback and for recognizing the motivation and potential value of ArtModeling, particularly its instance-matched real-world articulated objects, corresponding digital assets, and interaction recordings for evaluating real-to-simulation discrepancies.

Below, we provide detailed responses to the reviewer's specific comments and questions:

W1 & Q1: Physical Parameter Identifiability and Adaptation

A1: We thank the reviewer for this insightful observation. With unmeasured external forces, different force–parameter combinations can explain similar trajectories. To constrain calibration, we use human-reviewed VLM estimates as priors, with the optimized parameters typically restricted to 85%–115% of their initial values. We interpret the calibrated values as effective simulation parameters conditioned on the observed motion and physical priors, with trajectory agreement evaluating their ability to reproduce motion. These constraints restrict the parameter search while preserving the ability to refine the initial estimates.  

  • In the repeated-interaction evaluation, the calibrated parameters and excitation are estimated from the first interaction. During evaluation, each parameter configuration is held fixed, and all four configurations receive the same excitation inferred from that first interaction. The second recorded trajectory is used solely for evaluation and does not enter parameter or excitation fitting. Thus, the comparison does not involve separately adjusting the excitation for each configuration to match the second target trajectory.

  • To support practical use under this parameter ambiguity, ArtModeling integrates this prior-constrained calibration with SolverAdapt. Each digital asset provides a default physical parameter configuration, and SolverAdapt enables users to inspect and adapt it to specific application conditions and simulation platforms. Users can adjust mass, density, contact friction, and joint-drive parameters, assess the resulting behavior through motion response curves, and save the adapted configuration back to the USD asset. This integrated workflow provides both a default configuration and tools for its subsequent evaluation and adaptation.

W2 & Q2: Evaluation under Different Interaction Conditions

A2: We thank the reviewer for the valuable suggestion to evaluate beyond repeated interactions under the same conditions. The full ArtModeling dataset contains 497 pairs of real-world objects and their corresponding digital assets. We selected 36 representative objects to construct the dynamics evaluation subset, covering the motion types and the vast majority of motion ranges in the full dataset. We have added nine real-world pulling experiments on three articulated objects, varying the suspended masses from 150 to 350 g and the additional drawer or tray loads from 0 to 350 g. A pulley system applies excitation along the joint axis, and an optical-inertial motion capture system records the resulting displacement at 90 Hz. Images of the experimental setups are presented on the ArtModeling Dataset Website.

For each object, we jointly fit one shared parameter set to its three trials. We compare the resulting one-degree-of-freedom numerical trajectories with the motion capture recordings. The suspended masses specify nominal gravitational excitation; instantaneous rope tension was not directly measured. Table 1 reports the conditions and trajectory residuals for all nine trials.

Table 1. Trajectory residuals from shared-parameter calibration under different suspended masses and additional loads.

ObjectSuspended mass (g)Additional load (g)RMSE (cm) ↓NRMSE (%) ↓
cabinet163002001.727.71
cabinet1635000.713.18
cabinet163503500.492.18
cabinet1725002.6115.10
cabinet1730001.407.92
cabinet173002001.729.76
letter_tray215000.985.00
letter_tray225001.346.21
letter_tray22501501.969.79
Mean——1.447.43

Across the nine trials, the mean RMSE is 1.44 cm, and the mean NRMSE is 7.43%. Joint calibration yields one shared parameter set per object that reproduces the recorded responses across three excitation and loading conditions.

AuthorsOpenReview ↗

Response to Reviewer 8BoA 2/2

W3 & W4: Dataset Quality Presentation and Additional Comparisons

A3: We thank the reviewer for the suggestions regarding dataset presentation quality and additional comparisons. We provide further clarification on four aspects: dynamic dataset demonstrations, image quality, appearance differences, and the comparison with Astra.

  • Dataset Demonstration and Visual Quality. We provide an ArtModeling Dataset Website showcasing real-world captures, corresponding digital assets, interaction sequences, physical-fidelity evaluations, and asset comparisons. Figure 11 will be replaced with its original high-resolution version, as a compressed PDF was mistakenly used. As described in Appendix A.1, RGB-D images are captured using an Intel RealSense D435i at a resolution of 1280 × 720 and 30 Hz, with approximately 3 Hz retained after frame selection. The color differences between real-world observations and digital assets mainly result from different illumination conditions in the real and simulated environments.

  • Additional Comparison with Astra. We evaluate Astra on 10 instances using 30 and 240 RGB images with corresponding masks. The scanned point cloud of each instance serves as GT. We compare semantically matched parts across ArtModeling and Astra-generated assets, independently aligning each part pair using rigid transformations (rotation and translation only), without scaling. We report part-level Chamfer Distance (CD) in millimeters and normalized by the diagonal length of the corresponding scanned static body: Normalized CD (%) = 100 × Part-level CD (mm) / Static-body diagonal (mm). Each instance reports the average CD across its matched parts. Table 2 summarizes the geometric comparison.

Table 2. Part-level CD Comparison with Astra. Each entry reports CD in mm / normalized CD in %. Lower is better.

Instance#PartsArtModeling ↓Astra-30 ↓Astra-240 ↓
basket136.5 / 1.0623.1 / 3.7714.1 / 2.30
cabinet2037.8 / 0.9029.2 / 3.3720.5 / 2.36
first_aid_kit126.4 / 1.9623.0 / 7.0316.4 / 5.02
knife124.8 / 2.585.3 / 2.855.5 / 2.96
laptop126.0 / 1.669.0 / 2.498.2 / 2.27
letter_tray346.3 / 1.1917.0 / 3.2112.5 / 2.36
oven427.1 / 1.5220.0 / 4.2817.0 / 3.64
ricecook426.5 / 1.4216.8 / 3.6815.6 / 3.41
scissors324.8 / 2.615.0 / 2.725.5 / 2.99
support536.1 / 1.9814.4 / 4.6810.4 / 3.38

As shown in Table 2, the average normalized CD across the 25 matched parts decreases from 3.78% to 2.97% when increasing Astra's input images from 30 to 240. ArtModeling achieves lower CD than both Astra settings across all 10 instances under this evaluation protocol.

We further compare joint structures and motion ranges, as summarized in Table 3. R and P denote revolute joints (degrees) and prismatic joints (meters), respectively; R∞ denotes revolute joints without finite angular limits.

Table 3. Joint Structure and Motion Range Comparison with Astra.

InstanceArtModelingAstra 30Astra 240
basket12R; avg. 100 °2R; avg. 350 °2R; avg. 110 °
cabinet201 finite R (90°) + 8R∞ + 1P (0.300 m)1R (120°) + 1P (0.280 m)1 finite R (115°) + 8R∞ + 1P (0.250 m)
first_aid_kit14R; avg. 98.8° + 1P (0.200 m)1R (115°)2R; avg. 105°
knife11P (0.100 m)1P (0.065 m)1P (0.075 m)
laptop11R (135°)1R (140°)1R (135°)
letter_tray33P; avg. 0.300 m3P; avg. 0.190 m3P; avg. 0.150 m
oven4Door: 1R (90°); knobs: 2R∞Door: 1R (100°)Door: 1R (95°); knobs: 2R (270° each)
ricecook41R (100°)1R (110°)1R (108°)
scissors31R (105°)1R (95°)1R (100°)
support52R; avg. 120°2R; avg. 142.5°2R; avg. 95°

As shown in Table 3, ArtModeling contains 29 articulated joints, compared with 15 and 26 recovered by Astra using 30 and 240 images, respectively. More input views improve joint recovery, although discrepancies in joint counts and motion ranges remain.

Dear Reviewer 8BoA,

As the rebuttal deadline is approaching, please review the authors’ response and participate in the discussion with them at your earliest convenience.

Best regards,

The AC

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Reviewers and AC

We sincerely thank all reviewers and the Area Chair for their insightful comments and valuable recommendations.

Reviewer 8BoA appreciated the motivation behind our work and recognized ArtModeling as a valuable resource for assessing real-to-simulation discrepancies by pairing digital assets with observations and interaction recordings of the same physical objects.

Reviewer yz87 highlighted the significant research value of our large-scale, richly annotated instance-matched dataset and its independently constructed ground-truth geometry.

Reviewer ZkRV emphasized the value of the correspondence between 497 physical objects, detailed digital assets, and registered captures, and found the real-versus-rendered comparisons informative.

After discussion, Reviewers yz87 and ZkRV confirmed that their concerns had been addressed and maintained their Weak Accept ratings. Reviewer 8BoA acknowledged that the additional video demonstration and experiments had addressed most of their concerns and raised their rating from 2 to 3.

To facilitate further assessment by the AC, we have summarized all reviewers' primary feedback in the table below:

Reviewer CommentsReviewer 8BoAReviewer yz87Reviewer ZkRV
Strengths
Instance-Matched Real–Digital Asset Pairing✔✔✔
Rich Geometric and Physical Annotations✔
Independently Constructed GT Geometry✔✔
Real-World Interaction Recordings✔✔✔
Controlled Real–Synthetic Comparison✔✔
Concerns & Clarifications
Physical Calibration and Dynamics✔✔✔
Multi-Joint Motion Coverage✔✔
GT Geometry and Trajectory Accuracy✔
Evaluation Subset and Failure Rates✔
Visualization and Astra Comparison✔
Related Datasets and Data Release✔
Initial Rating233
Final Rating333

Regarding the reviewers' concerns, we have provided additional clarifications and experiments, summarized below:

  1. Physical Parameter Calibration and Dynamics Validation (Reviewers 8BoA, yz87, ZkRV): We clarified that the excitation inferred from the first interaction remains fixed across all four parameter configurations, with the second trajectory used only for evaluation. We conducted nine additional pulling experiments on three objects, jointly fitting one shared parameter set to each object's three loading conditions, achieving a mean RMSE of 1.44 cm and NRMSE of 7.43%. Experimental demonstrations are available in Physical Fidelity Evaluation, and SolverAdapt supports parameter adaptation.

  2. Motion Trajectory Accuracy (Reviewer yz87): We added independent motion-capture validation of VideoArtGS, achieving average RMSEs of 2.09 cm and 6.65° for prismatic and revolute joints. Results and demonstrations are available in Physical Fidelity Evaluation.

  3. GT Geometry Reliability (Reviewer yz87): We added an independent scan-based evaluation on 12 objects, confirming stable rankings for real inputs and a top-two ranking swap between ArtGS and DigitalTwinArt for synthetic inputs.

  4. Multi-Joint Coverage and Benchmark Reliability (Reviewers 8BoA, yz87, ZkRV): We clarified the per-joint acquisition protocol and expanded the evaluation subset from 153 to 170 objects, including completion and failure analysis. Real-input errors are higher in 27 of 28 comparable metrics.

  5. Dataset Visualization and Astra Comparison (Reviewer 8BoA): We improved the ArtModeling Dataset Website and dataset visualizations, and clarified the capture specifications. Additional Astra comparisons on 10 instances show lower part-level CD for ArtModeling across both input settings, with 29 joints compared with Astra's 15 and 26. Further visual comparisons are available in Asset Comparison.

  6. Related Datasets and Data Availability (Reviewer ZkRV): We added comparisons with PD²GS/RS-Art and Digital Twin Catalog, and clarified the staged release of approximately 300 paired instances upon publication, followed by the remaining data within two months.

We sincerely thank all reviewers for their positive comments and constructive feedback, and have carefully integrated all suggestions into the revised manuscript, with changes clearly highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.