acceptodds
Under review as a conference paper at ICLR 2027

UniPhysGen: Unified Physical Grounding for Simulation-Ready 3D Assets

Abstract

Physically grounded 3D assets are increasingly important for embodied AI and robotic simulation. However, most existing 3D assets lack unified physical semantics, including articulation semantics and intrinsic physical properties, required for realistic interaction. Current approaches either treat these semantics independently or rely on canonicalized object structures, limiting robustness across heterogeneous 3D assets. We present UniPhys, a scalable framework for automatically transforming raw 3D assets into simulation-ready assets with unified physical semantics. Based on UniPhys, we construct UniPhys-40K, a large-scale physically grounded dataset, together with UniPhys-Bench, a carefully verified benchmark for unified physical grounding evaluation. We further introduce UniPhysGen, a unified physical grounding model that jointly reasons over articulation semantics and intrinsic physical properties. UniPhysGen incorporates geometry-robust articulation grounding to mitigate geometric shortcut bias under heterogeneous part decompositions. Extensive experiments demonstrate state-of-the-art performance across articulation grounding and intrinsic physical property estimation tasks, while the resulting assets can be directly deployed in robotic simulation environments for realistic physical interaction. Code: https://github.com/anonymous-abc123/anonymous-code.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
physical grounding, 3D understanding, articulated object understanding, physical property prediction, 3D representation learning, multimodal learning, simulation-ready assets

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jSjNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to equip existing 3D meshes with the information needed for interaction. The authors propose an automated annotation pipeline, UniPhys, and use it to construct a training dataset of approximately 40,000 objects and UniPhys-Bench, a benchmark with manually verified annotations for motion-relevant parts. Building on these resources, they train UniPhysGen, which uses Sonata to encode point clouds of the full object and a target part, and Qwen3-1.7B to predict part-level physical properties, joint parameters, motion dependencies, and object-level properties.

Critical strengths
  1. The data resources are a notable contribution of this work. UniPhys-40K covers 3D assets from multiple sources with diverse part decompositions, providing both articulation and physical property annotations. UniPhys-Bench has undergone manual inspection and correction. The authors commit to releasing the annotations for both datasets and the pretrained models, which could benefit the research community.

  2. The authors conduct targeted experiments on rotation augmentation, part decomposition granularity, and positional encoding, and use human evaluation to assess annotation quality before and after simulation-based verification. These results provide empirical support for the effectiveness of the pipeline and the robustness of the model.

Critical weaknesses
  1. The paper’s clearest contributions lie in its automated annotation pipeline and data resources, while the novelty of the model itself is relatively limited. PhysX-3D and PhysForge already incorporate material, functional, and kinematic information into physical asset modeling, and Articulate AnyMesh has explored constructing articulated structures from raw meshes. The methodological novelty of this work rests on its treatment of heterogeneous part decompositions. However, the SO(3) data augmentation, spherical parameterization, and shared global positional encoding emphasized in the paper mainly improve rotational generalization, directional representation, and spatial correspondence between parts and the full object. The paper does not sufficiently substantiate how these designs resolve ambiguities in mechanical interpretation arising from variations in part granularity and incomplete structures.

  2. It is unclear what the physical annotations are intended to represent and whether the evaluation reflects this objective. The paper treats numerical values obtained through VLM annotation and human plausibility checks as a single set of reference values and evaluates predictions using attribute-level errors. Closer agreement with these values does not necessarily imply greater physical accuracy. The authors provide human plausibility assessments based on motion videos and demonstrations of robotic interaction, but they do not report part-level interaction success rates or the overall proportion of physically executable assets produced by the model.

Questions for authors

Could the authors share additional interaction videos after the double-blind review process?

4 comments
AuthorsOpenReview ↗

To Reviewer jSjN (Part 1/3)

Response to Weaknesses 1 : Methodological Contributions

Weaknesses 1:

The paper’s clearest contributions lie in its automated annotation pipeline and data resources, while the novelty of the model itself is relatively limited.

Response:

We thank the reviewer for the constructive feedback. We clarify the conceptual and methodological contributions of UniPhys and UniPhysGen, and provide additional experimental evidence on generalization to heterogeneous geometry.

Our work addresses a fundamental challenge in 3D physical understanding: How can we ground existing 3D assets with diverse geometries and part decompositions in a unified physical semantic framework, without assuming canonical orientations, predefined articulation structures, or geometry regeneration?

To address this challenge, we introduce two complementary methodological contributions. UniPhys is a scalable geometry-aware annotation framework integrating physical semantic reasoning, geometry-constrained candidate generation, and simulation-based consistency verification. UniPhysGen directly learns physical semantics from native 3D point clouds, leveraging shared object-part geometric representations to infer articulation semantics and intrinsic physical properties across heterogeneous 3D structures.

Unlike PhysX-3D and PhysForge, which focus on image-conditioned generation of physically grounded 3D assets, UniPhysGen directly grounds physical semantics on existing 3D meshes with heterogeneous part decompositions, without regenerating their geometry. Articulate AnyMesh similarly aims to construct articulated assets from input meshes, but it re-segments movable parts and reconstructs their articulation structures, with optional geometry completion. Consequently, its output is a newly constructed articulated asset rather than a direct physical annotation of the original mesh and its existing part decomposition.

In contrast, UniPhysGen preserves the original geometry and part decomposition while recovering comprehensive physical properties, including articulation parameters, material, density, friction, scale, mass, and affordance. This distinction is important for real-world 3D asset repositories, where the same functional object may have different part granularities, incomplete structures, or manually defined decompositions. Nevertheless, we conducted additional quantitative comparisons with PhysX-3D and PhysX-Anything to evaluate articulation plausibility and simulation interaction success, as detailed in our response to Weakness 2.

Our methodological contribution lies in integrating shared object-part geometric representations with unified physical-semantic learning across heterogeneous 3D structures. Specifically, shared object-part representations provide geometric context for interpreting individual parts relative to the complete object, while SO(3) augmentation and spherical parameterization primarily improve orientation robustness. Table 4 demonstrates robustness to controlled variations in part grouping, while our additional evaluation on TRELLIS 2.0-generated assets provides further evidence of generalization to unfamiliar generated geometry.

Table R17. UniPhysGen performance on TRELLIS 2.0-generated assets.

MetricUniPhysGen
Kinematic Parameters
Joint Type Accuracy ↑91.78%
Axis Angular Error ↓12.05°
Pivot Distance Error ↓0.1043
Limit mIoU ↑92.91%
Articulation Structure
Structure mIoU ↑82.28%
Structure Micro-F1 ↑67.54%
Intrinsic Physical Properties
Material Accuracy ↑74.61%
Density ALDE ↓0.490
Friction MAE ↓0.067
Dimension ALDE ↓0.182
Dimension MnRE ↑84.68%
Mass ALDE ↓0.404
Mass MnRE ↑70.61%
Affordance MAE ↓2.194

We evaluated UniPhysGen on 32 TRELLIS 2.0-generated assets, containing 387 parts, including 73 movable parts (21 prismatic and 52 revolute). Reference annotations were first generated using our UniPhys Pipeline and subsequently inspected and corrected by human annotators before evaluation. Articulation metrics cover all 73 movable parts, while part-level physical metrics are computed over 382 valid predictions out of 387 annotated parts.

The results demonstrate that UniPhysGen maintains strong articulation prediction performance on generated geometry, including 91.78% Joint Type Accuracy and 12.05° Axis Error. Together with Table 4, these findings support its robustness to variations in geometric structure and part decomposition, while also identifying upstream segmentation quality as a remaining limitation.

AuthorsOpenReview ↗

To Reviewer jSjN (Part 2/3)

Response to Weaknesses 2: Physical Annotation Validity and Simulation Evaluation.

Weaknesses 2:

The evaluation relies on attribute-level agreement with VLM-generated, human-verified reference values, which may not fully reflect physical validity. The paper lacks quantitative evidence of part-level interaction success and overall asset executability.

Response:

We thank the reviewer for this important concern. As clarified in Section 3.1 (Lines 188–191), our objective is to establish physically plausible, internally consistent, and simulation-ready physical configurations for existing virtual 3D assets. UniPhys-Bench annotations undergo human verification and correction, including refinement of part decomposition and verification of articulation parameters, materials, densities, friction coefficients, and masses.

To further assess physical validity beyond numerical agreement with reference annotations and the qualitative Isaac Sim demonstrations in Figure 8, we conducted additional quantitative experiments evaluating articulation plausibility and part-level simulation interaction success.

Specifically, we randomly selected 50 UniPhys-Bench assets, comprising 488 parts and 124 movable parts. We rendered front-view images and used them as inputs to PhysX-3D and PhysX-Anything to generate complete 3D assets with physical annotations, while UniPhysGen directly operated on the original meshes.

Since the assets generated by PhysX-3D and PhysX-Anything differ from the original meshes in geometry, scale, and part decomposition, direct numerical comparison of joint axes and pivots is not reliable. We therefore adopted a unified three-level human evaluation protocol for articulation parameters: 2 (Correct) indicates physically reasonable motion; 1 (Partially Correct) indicates generally plausible motion with visible deviations; and 0 (Incorrect) indicates implausible motion or failure to achieve the intended articulation. We additionally evaluated part-level simulation interaction success through standardized physical interaction tests.

Table R18. Articulation plausibility and part-level simulation interaction success.

MethodJoint Type Acc. ↑Axis Score ↑ / 2Pivot Score ↑ / 2Limit Score ↑ / 2Interaction Success Rate ↑
UniPhysGen (Ours)87.90%1.5561.5651.60569.35% (86/124)
PhysX-Anything60.00%1.2271.0670.76037.33% (28/75)
PhysX-3D52.50%0.6250.6580.71626.72% (31/116)

Interaction Success Rate is measured over each method's evaluated movable parts, rather than all 124 ground-truth movable parts.

To further assess structural completeness, we matched the generated movable parts to the corresponding ground-truth parts and measured how many original movable parts were successfully recovered.

Table R19. Ground-truth movable-part matching performance.

MethodScored Motion PartsGT Motion-Part Match Rate ↑
UniPhysGen (Ours)124100.00% (124/124)
PhysX-Anything7522.58% (28/124)
PhysX-3D11659.68% (74/124)

UniPhysGen preserves the original part decomposition by operating directly on the input meshes, whereas image-to-3D generation methods must reconstruct the movable components.

We also evaluated intrinsic physical properties by matching corresponding components between the generated assets and the ground-truth assets.

Table R20. Intrinsic physical property comparison.

MethodMaterial Accuracy ↑Density ALDE ↓Affordance MAE ↓
PhysX-3D—0.888—
PhysX-Anything67.21%0.7411.656
UniPhysGen (Ours)77.05%0.3541.148

Metrics are computed over corresponding evaluable components; unavailable predictions are marked with “—”.

UniPhysGen achieves the highest part-level interaction success rate (69.35%), compared with 37.33% for PhysX-Anything and 26.72% for PhysX-3D. It also achieves stronger intrinsic physical property predictions on the evaluated metrics. These results provide quantitative evidence of simulation executability beyond agreement with reference annotations, while highlighting the importance of preserving the original articulated structure.

Together, these experiments provide quantitative evidence of articulation plausibility and part-level simulation executability beyond attribute-level agreement with reference annotations, while highlighting the practical advantage of grounding physical semantics directly on existing 3D geometry without regenerating the original asset.

AuthorsOpenReview ↗

To Reviewer jSjN (Part 3/3)

Response to Question: Additional Interaction Videos and Open-Source Availability

Question 1:

Could the authors share additional interaction videos after the double-blind review process?

Response:

Yes. We have recorded and released additional simulation interaction videos in NVIDIA Isaac Sim, demonstrating the physical interactions of our annotated 3D assets.

Furthermore, we are committed to fully open-sourcing the UniPhys project. We have already anonymously released the complete codebase (UniPhys Pipeline and UniPhysGen), model weights, and UniPhys-Bench:

The full UniPhys-40K training dataset has also been prepared and will be released publicly.

AuthorsOpenReview ↗

Revision Summary and Updated Manuscript

Revision Summary and Updated Manuscript

We thank the reviewer for the constructive feedback. We have uploaded a revised manuscript incorporating additional experiments and clarifications to address your concerns. The main revisions are summarized below:

  1. Methodological Contributions and Model Architecture (Section 5.2 and Appendix F.4). We have added controlled comparisons with Sonata + Motion Head and UniPhysGen + Regression Head, providing further evidence for the effectiveness of our LLM-based architecture and token-based articulation parameter prediction beyond the shared 3D encoder and training data.

  2. Generalization to Generated 3D Assets (Section 5.3 and Appendix F.6). We have incorporated additional experiments on TRELLIS 2.0-generated meshes, evaluating articulation semantics and intrinsic physical properties under variations in generated geometry and part decomposition.

  3. Physical Validity and Simulation Interaction (Section 5.4 and Appendix F.7). We have added quantitative comparisons with PhysX-3D and PhysX-Anything, including articulation plausibility, movable-part recovery, and intrinsic physical properties. Importantly, we report part-level simulation interaction success rates, providing quantitative evidence of physical executability beyond attribute-level agreement and qualitative demonstrations.

The detailed experimental results and evaluation protocols are provided in Appendix F, with corresponding summaries and references in the main text.

We appreciate the reviewer's constructive feedback.

Reviewer_KW24OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a GPT-5-plus-geometry pipeline (UniPhys) turns raw meshes into assets with part decomposition, joint type/axis/pivot/limits and per-part material/density/friction/mass, which is defined as physical property. It builds a dataset with 40K objects and a human-corrected benchmark with 1927 objects, and distils both into a Qwen3-1.7B point-cloud LLM (UniPhysGen). The paper's own tables do not support the SOTA wording claimed in the abstract.

Critical strengths

There are plenty of prior work focusing on this direction, therefore the novelty is not the strength for this paper. Dataset efforts, extensive experiments and interesting ablation are something interesting. I summarize the core strengths of this paper as follows:

  1. Dataset scale and heterogeneity: 40K objects and 84 categories from five repositories is the core contribution to the field.
  2. Simulation-driven verification is concrete, with explicit thresholds (A.4, Alg. 2), and Fig. 2 shows measurable error reductions.
  3. Rotation robustness is real and practically relevant for y-up/z-up confusion in Objaverse-style data.
  4. Global positional encoding ablation (Table 5) is a clean, well-motivated effect.
  5. Appendix is unusually complete. There are metric definitions (App. D), baseline adaptation (App. E), training configs (Table 8), instruction templates (Listings 1–8).
Critical weaknesses

The majority of the concerns goes into SOTA claims of the method and the evaluation which I summarize as follows:

  1. SOTA claim contradicted by the paper's own tables (Table 2 and 9).The honest claim would be "competitive on canonical data, much more robust to rotation and incomplete parts".
  2. Benchmark GT is the pipeline's own GPT-5 output, human-corrected, and the model is a distillation of the same pipeline. The property GT is checked for plausibility, not for metrics. So Table 2 measures agreement with GPT-5 priors, and the student is scored on its teacher's labels while NeRF2Physics uses a different material-to-property lookup, which is kind of weird as evaluation setting.
  3. No baseline is trained on the proposed dataset, so Table 2 conflates method with training distribution.
  4. Weak property baselines. There are closest prior work not compared. GaussianProperty, PhysXGen, PhysX-Anything and PhysForge's VLM planner are all cited, all predict part-level physical attributes, but none is evaluated.
  5. Simulation deployment is qualitative only. There are three Isaac Sim clips (Fig. 8) with no success rate or downstream metric. And Fig. 2 has no sample size.
Questions for authors

Besides the above concerns, I have the following misc questions for the authors:

  1. How does the UniPhys pipeline itself (GPT-5 + candidate selection) score on UniPhys-Bench under the Table 2 metrics? Does UniPhysGen exceed its teacher anywhere?
  2. Experimental details. Why is the final model's axis error worse than the canonical-Z variant under every condition in Table 3? Please add true SO(3) test-time rotations and a Cartesian-vs-spherical ablation under them.
  3. Is UniPhysGen one set of weights or three checkpoints?
  4. How was GT for density/friction/mass established beyond plausibility? Any subset validated against real measurements?
  5. Will the industry-partner meshes be released to the community?
16 comments
AuthorsOpenReview ↗

To Reviewer KW24 (Part 1/8)

Response to Weakness 1: Clarification of SOTA Claims

Weakness 1:

SOTA claim contradicted by the paper's own tables (Table 2 and 9). The honest claim would be "competitive on canonical data, much more robust to rotation and incomplete parts".

Response:

We thank the reviewer for pointing out this imprecise wording. We agree that the SOTA claim in the abstract is overly broad. Our original intention, as stated in the Introduction (L97-99), was to emphasize that UniPhysGen achieves state-of-the-art performance on most evaluation settings, rather than outperforming every baseline across all metrics and datasets.

As shown in Tables 2 and 9, UniPhysGen achieves competitive performance on standard benchmarks, with advantages in selected physical and articulation metrics, while other methods perform better on certain joint prediction tasks. More importantly, our experiments highlight the model's robustness to rotational variations and incomplete part geometries, alongside its unified prediction of multiple physical attributes.

We will revise the abstract and related statements to avoid an unqualified SOTA claim and more accurately reflect the experimental findings.

AuthorsOpenReview ↗

To Reviewer KW24 (Part 2/8)

Response to Weakness 2: Benchmark Ground Truth and Evaluation Protocol

Weakness 2

Benchmark GT is the pipeline's own GPT-5 output, human-corrected, and the model is a distillation of the same pipeline. The property GT is checked for plausibility, not for metrics. So Table 2 measures agreement with GPT-5 priors, and the student is scored on its teacher's labels while NeRF2Physics uses a different material-to-property lookup, which is kind of weird as evaluation setting.

Response:

We thank the reviewer for raising these concerns. We first clarify the construction of UniPhys-Bench, the distinction between UniPhys Pipeline and UniPhysGen, and the evaluation protocol. Additional quantitative comparisons are provided in our response to Question 1.

1. Benchmark Reliability and Evaluation Protocol. UniPhys-Bench does not directly use raw GPT-5 predictions as its ground-truth annotations. Although the benchmark is initialized through our annotation pipeline, its reference annotations undergo human inspection and correction, covering joint types, axes, pivots, motion limits, materials, densities, friction coefficients, and object masses. The finalized annotations therefore reflect human-reviewed and corrected physical configurations rather than unverified GPT-5 outputs.

Accordingly, Table 2 evaluates predictions against these human-verified benchmark references, rather than directly against the original GPT-5 annotations. To further address the reviewer's concern, we provide a direct quantitative comparison between the initial GPT-5 annotations and UniPhysGen, together with an analysis of the annotation verification process, as detailed below.

2. UniPhys Pipeline is not merely GPT-5 annotation. The pipeline combines LVLM-based semantic reasoning with explicit 3D geometric constraints and physics-based verification (Sec 3 and Appendix A). In particular, articulation axes and pivots are first generated as candidates based on explicit 3D geometric constraints, then rendered into images for LVLM-based semantic selection, followed by simulation-based verification and quality filtering. UniPhys-40K is constructed from these quality-controlled annotations, rather than unverified GPT-5 outputs.

In contrast, UniPhysGen directly processes object-level and part-level 3D point clouds to predict physical properties and articulation parameters, without relying on rendered geometric candidates or LVLM-based semantic selection at inference time.

3. Regarding NeRF2Physics and the evaluation protocol, we would like to clarify a distinction between two related methods. NeRF2Physics uses GPT to propose candidate materials and their associated physical properties, followed by CLIP-based kernel regression, rather than relying solely on a fixed material-to-property lookup. In contrast, GaussianProperty, which the reviewer also mentions later, employs a predefined material-property database for property matching after material recognition.

Importantly, all methods are evaluated using the same metrics and human-verified reference annotations, ensuring a consistent evaluation protocol despite differences in their physical-property estimation strategies.

AuthorsOpenReview ↗

To Reviewer KW24 (Part 3/8)

Response to Question 1: Comparison with the UniPhys pipeline

Question 1

How does the UniPhys pipeline itself (GPT-5 + candidate selection) score on UniPhys-Bench under the Table 2 metrics? Does UniPhysGen exceed its teacher anywhere?

Response:

We thank the reviewer for this question. To directly evaluate the annotation pipeline, we randomly selected 120 assets from UniPhys-Bench, with 20 assets from each of the six data sources (comprising 1,197 parts and 306 movable parts). This re-evaluation is necessary because UniPhys-Bench undergoes extensive human verification and correction, including adjustments to part decompositions and physical annotations. Therefore, the original pipeline outputs are no longer directly aligned with the finalized benchmark. We reran the GPT-5 annotation pipeline on these assets and compared its predictions with UniPhysGen.

Here, GPT-5 Initial refers to the pipeline outputs after geometry-constrained candidate generation and LVLM-based semantic selection, but before the subsequent physics-based verification and quality filtering.

Table R7. Comparison between initial GPT-5 annotations and UniPhysGen on the same 120 UniPhys-Bench assets.

MetricGPT-5 InitialUniPhysGen (Ours)
Kinematic Parameters
Joint Type Acc. ↑85.95%88.24%
Axis Angular Error ↓21.07°11.13°
Pivot Distance Error ↓0.08020.0433
Limit mIoU ↑83.96%83.68%
Articulation Structure
Structure mIoU ↑84.69%86.82%
Structure Micro-F1 ↑83.54%85.80%
Material Properties
Material Accuracy ↑81.79%73.43%
Density ALDE ↓0.3530.475
Friction MAE ↓0.0650.068
Scale & Mass Properties
Dimension ALDE ↓0.0990.261
Dimension MnRE ↑91.91%82.70%
Mass ALDE ↓0.2680.615
Mass MnRE ↑82.37%69.63%
Affordance
Affordance MAE ↓1.2951.905

The results show that UniPhysGen outperforms the initial GPT-5 pipeline annotations on several articulation-related metrics, including Joint Type Accuracy (88.24% vs. 85.95%), Axis Angular Error (11.13° vs. 21.07°), and Pivot Distance Error (0.0433 vs. 0.0802). UniPhysGen also achieves higher Structure mIoU (86.82% vs. 84.69%) and Structure Micro-F1 (85.80% vs. 83.54%). Meanwhile, the initial pipeline achieves better results on most intrinsic physical properties, scale, mass, and affordance metrics.

These results demonstrate that UniPhysGen does not merely reproduce the initial GPT-5 predictions, but can achieve better agreement with the human-verified benchmark on several articulation-related tasks. Unlike the LVLM-based pipeline, which relies on geometry-constrained candidate generation, visual rendering, and semantic selection, UniPhysGen directly infers physical parameters from native 3D geometry.

UniPhys-40K uses physics-verified, quality-filtered annotations rather than raw GPT-5 predictions. We further evaluate this verification process below.

Table R8. Annotation quality before and after physics-based verification.

MetricBefore VerificationAfter Verification
Kinematic Parameters
Joint Type Acc. ↑85.95%93.98%
Axis Angular Error ↓21.07°12.05°
Pivot Distance Error ↓0.08020.0478
Limit mIoU ↑83.96%83.69%
Articulation Structure
Structure mIoU ↑84.69%85.47%
Structure Micro-F1 ↑83.54%86.06%
Intrinsic Physical Properties
Material Accuracy ↑81.79%81.55%
Density ALDE ↓0.3530.347
Friction MAE ↓0.0650.064
Mass ALDE ↓0.2680.265
Affordance MAE ↓1.2951.278

Table R9. Verification pass rates.

Verification StagePass Rate
Intrinsic Physical Property Verification92.98%
Motion Simulation Verification63.11%

Verification serves primarily as a quality-filtering mechanism rather than a correction process applied to every prediction. Accordingly, Table R8 reports the quality of annotations retained after verification, while Table R9 reports the proportion of records with definitive verification outcomes that pass the corresponding filters. Detailed verification procedures are provided in Appendix A.

The results demonstrate the effectiveness of this filtering process. Among retained annotations, Joint Type Accuracy increases from 85.95% to 93.98%, while Axis Angular Error decreases from 21.07° to 12.05°. Moreover, only 63.11% of motion annotations pass simulation verification, highlighting the importance of identifying and filtering unreliable articulation predictions before constructing UniPhys-40K.

Together, these results clarify that UniPhysGen is trained on quality-controlled annotations incorporating geometric constraints and physics-based verification, rather than directly imitating unverified GPT-5 outputs. Its inference process is also fundamentally different from the LVLM-based annotation pipeline.

Reviewer_sadkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes three things.

1 UniPhys, an automated pipeline that turns raw meshes into simulation-ready assets. ◦ Multi-view SAM masks lifted onto the mesh pick among PartField decompositions (Hungarian matching), and Apriori itemset mining merges the result. ◦ GPT-5 labels each part's material, density, Young's modulus, hardness, Poisson's ratio, friction and affordance, plus object scale and mass, from part-centric renderings. ◦ GPT-5 also picks joint axes and pivots from geometry-derived candidates. ◦ Rule-based property checks and MuJoCo contact/penetration checks filter or refine the labels.

2 UniPhys-40K and UniPhys-Bench. UniPhys-40K has about 40K objects and 370K parts from Objaverse-Sketchfab, HSSD, 3D-FUTURE, ABO and PartNet. UniPhys-Bench has 1,927 articulated objects and 5.5K movable parts: 454 held out from the training distribution, plus 1,473 objects from an industry partner with designer-made decompositions. Bench labels are pre-filled by the pipeline, then corrected by humans.

3 UniPhysGen, a SpatialLM-style model (Qwen3-1.7B, Sonata point encoder, MLP projector) that reads object and part point clouds and outputs JSON. It is pretrained on part-level semantic and physical descriptions, then fine-tuned separately for kinematic parameters, groups of parts that move together, and object-level properties. It uses "SO(3)" rotation augmentation, spherical axis angles, and a voxel grid origin shared by part and object.

Critical strengths
  1. A dataset and pipeline at real scale. The lack of articulated, physics-annotated assets is a real bottleneck, and this paper addresses it at scale. UniPhys-40K covers 40K objects from five different sources, with articulation and intrinsic-property labels together. The simulation check measurably reduces label error (Fig. 2: axis 25% to 16%, pivot 22% to 13%). The benchmark also includes 1,473 objects whose decompositions were made by designers, not by the training pipeline.

  2. Code release. The anonymous repository contains both the pipeline and the model code, which makes the work much easier to check and build on.

  3. A model that reproduces the pipeline's labels without GPT-5. The generated data is used to fine-tune a Qwen3-1.7B model (UniPhysGen) that predicts the pipeline's articulation and physical-property annotations directly from point clouds, with no GPT-5 calls or simulation at inference time. It still needs a part decomposition as input, so it replaces the labelling stages rather than the whole pipeline from raw meshes.

Critical weaknesses

1 The labelling pipeline depends on a closed model. GPT-5 assigns every property label and chooses every axis and pivot (L807), and it also pre-fills the benchmark labels. The paper gives no model version or API cost, and no other labelling model is tried. So the released code can't regenerate or extend the dataset without paid access to a model that may change or be retired, and the benchmark can't be rebuilt independently. This cuts into the reproducibility and impact of the pipeline, which is the paper's main contribution. Using an open-weight VLM, or at least comparing against one, would address this.

2 The "raw mesh" claim. Meshes inputs seems curated, good-quality meshes, and generated meshes are never tested. The paper motivates the work with generative 3D models such as TRELLIS and Hunyuan3D (L31-33), and lists "generative 3D assets" among the inputs prior methods handle poorly (L50-51). Yet no test or training asset comes from a generative model.

3 1. Non-LLM baselines. Have you trained a 3D-native model on UniPhys-40K, either the Sonata encoder with regression heads and no LLM, or existing estimators such as Shape2Motion, ANCSH or Pixie? How do they compare with UniPhysGen on UniPhys-Bench?

Questions for authors

Question 1 – Labelling model: Have you tried other models, especially open-weight ones? How much would the labels and the error rates in Fig. 2 change? Which version of GPT-5 did you use, and what was the cost of labelling?

Question 2 – Generated meshes: Have you run the pipeline or UniPhysGen on outputs from TRELLIS or Hunyuan3D? How do the decomposition, the choice of axis and pivot, the mass check and the MuJoCo checks behave on these meshes? How does the model handle shells with no interior?

Question 3 – Non-LLM baselines: Have you trained a 3D-native model on UniPhys-40K, either for part segmentation or for articulation generation?

4 comments
AuthorsOpenReview ↗

To Reviewer sadk (Part 1/3)

Response to Weakness 1 & Question 1: Labelling Model and Reproducibility

Weakness 1 & Question 1:

Labelling model: Have you tried other models, especially open-weight ones? How much would the labels and the error rates in Fig. 2 change? Which version of GPT-5 did you use, and what was the cost of labelling?

Thank you for raising this concern. We used GPT-5 (gpt-5-2025-08-07) with reasoning_effort="medium". The average API cost was approximately $0.20 per asset for the two-stage annotation process, totaling $8,388 for 41,941 assets, including the benchmark.

To evaluate reproducibility with open-weight VLMs, we evaluated Qwen3.5-35B-A3B using the same pipeline. We sampled 120 UniPhys-Bench assets (20 per source), containing 1,197 parts and 306 movable parts, and compared GPT-5, Qwen3.5-35B-A3B, and UniPhysGen on this same subset under the original evaluation protocol..

Table R1. Initial annotation performance compared with UniPhysGen.

MetricGPT-5Qwen3.5-35B-A3BUniPhysGen (Ours)
Kinematic Parameters
Joint Type Acc ↑85.95%61.76%88.24%
Axis Angular Error ↓21.07°37.86°11.13°
Pivot Distance Error ↓0.08020.07620.0433
Limit mIoU ↑83.96%74.39%83.68%
Articulation Structure
Structure mIoU ↑84.69%71.56%86.82%
Structure Micro-F1 ↑83.54%71.92%85.80%
Material Properties
Material Accuracy ↑81.79%72.24%73.43%
Density ALDE ↓0.3530.4860.475
Friction MAE ↓0.0650.1150.068
Scale & Mass Properties
Dimension ALDE ↓0.0990.1900.261
Dimension MnRE ↑91.91%85.34%82.70%
Mass ALDE ↓0.2680.4950.615
Mass MnRE ↑82.37%71.31%69.63%
Affordance
Affordance MAE ↓1.2952.2511.905

All three methods are evaluated on the same 120 assets. Axis Error, Pivot Error, and Limit mIoU are conditioned on correctly predicted joint types.

Table R1 shows that GPT-5 generally outperforms Qwen3.5-35B-A3B in initial physical-property annotation, while UniPhysGen achieves lower Axis Error (11.13° vs. 21.07°) and Pivot Error (0.0433 vs. 0.0802). Unlike the VLM-based pipeline, UniPhysGen directly predicts physical semantics from native 3D geometry.

These initial predictions are not the final UniPhys-40K annotations, which undergo physics-based verification and quality filtering. Table R2 evaluates the effects of this process.

Table R2. Effect of physics-based verification (Before → After).

MetricGPT-5Qwen3.5-35B-A3B
Kinematic Parameters
Joint Type Acc ↑85.95% → 93.98%61.76% → 90.57%
Axis Error ↓21.07° → 12.05°37.86° → 14.81°
Pivot Error ↓0.0802 → 0.04780.0762 → 0.0343
Limit mIoU ↑83.96% → 83.69%74.39% → 81.99%
Articulation Structure
Structure F1 ↑83.54% → 86.06%71.92% → 76.57%
Material Properties
Material Acc ↑81.79% → 81.55%72.24% → 70.94%
Density ALDE ↓0.353 → 0.3470.486 → 0.521
Friction MAE ↓0.065 → 0.0640.115 → 0.106
Scale & Mass Properties
Dimension ALDE ↓0.099 → 0.1090.190 → 0.195
Dimension MnRE ↑91.91% → 91.22%85.34% → 85.17%
Mass ALDE ↓0.268 → 0.2650.495 → 0.507
Mass MnRE ↑82.37% → 82.03%71.31% → 71.07%
Affordance
Affordance MAE ↓1.295 → 1.2782.251 → 2.312
Verification Pass Rates
Intrinsic Verification Pass Rate ↑92.98%63.12%
Motion Simulation Verification Pass Rate ↑63.11%61.02%

Verification filters unreliable annotations rather than correcting every prediction. After-verification metrics use retained annotations, and pass rates use records with definitive outcomes. See Appendix A for details.

Table R2 demonstrates the effectiveness of physics-based filtering. Although Qwen's initial predictions are weaker, its retained annotations approach GPT-5's quality (90.57% vs. 93.98% Joint Type Accuracy; 14.81° vs. 12.05° Axis Error). Its lower intrinsic verification pass rate (63.12% vs. 92.98%) reflects reduced coverage. These results demonstrate that UniPhys supports different underlying VLMs, rather than depending exclusively on GPT-5.

Open-source availability. To further support reproducibility, we have already anonymously released the complete codebase (UniPhys Pipeline and UniPhysGen), UniPhys-Bench, and model weights:

The complete UniPhys-40K training dataset has also been prepared for public release. We will include the model configuration, annotation cost, and additional open-weight VLM experiments in the revised manuscript.

AuthorsOpenReview ↗

To Reviewer sadk (Part 2/3)

Response to Weakness 2 & Question 2: Generalization to Generated 3D Meshes

Weakness 2 & Question 2:

Generated meshes: Have you run the pipeline or UniPhysGen on outputs from TRELLIS or Hunyuan3D? How do the decomposition, the choice of axis and pivot, the mass check and the MuJoCo checks behave on these meshes? How does the model handle shells with no interior?

Response:

We thank the reviewer for raising this important question. We clarify that raw meshes refer to existing 3D geometries without physical annotations, including both manually created and AI-generated meshes. To further evaluate the applicability of our method to generated 3D assets, we conducted additional experiments on TRELLIS 2.0-generated meshes.

1. Evaluation on TRELLIS 2.0-generated Assets.

We randomly selected 100 UniPhys-Bench instances, rendered front-view images, and generated corresponding 3D assets using TRELLIS 2.0. We then applied our part decomposition pipeline.

We observed that the initial segmentation quality was substantially lower than on the original assets, primarily due to the limited robustness of PartField on generated meshes. Our SAM3-guided segmentation refinement remained applicable, but its effectiveness was constrained by severe errors in the initial decomposition. We additionally tested P3-SAM as an alternative initial segmentation method. Although it generally produced better initial partitions than PartField, challenging cases remained, including merged cabinet doors, attached components, and incorrectly grouped structures.

Consequently, 32 assets with sufficiently reliable part decomposition were selected for evaluation, containing 387 parts and 73 movable parts (21 prismatic, 52 revolute). Reference annotations were verified on the generated geometries rather than transferred from the original meshes.

Table R3. UniPhysGen performance on TRELLIS 2.0-generated assets.

MetricUniPhysGen
Kinematic Parameters
Joint Type Acc ↑91.78%
Axis Angular Error ↓12.05°
Pivot Distance Error ↓0.1043
Limit mIoU ↑92.91%
Articulation Structure
Structure mIoU ↑82.28%
Structure Micro-F1 ↑67.54%
Material Properties
Material Accuracy ↑74.61%
Density ALDE ↓0.490
Friction MAE ↓0.067
Scale & Mass Properties
Dimension ALDE ↓0.182
Dimension MnRE ↑84.68%
Mass ALDE ↓0.404
Mass MnRE ↑70.61%
Affordance
Affordance MAE ↓2.194

The lower Structure Micro-F1 is concentrated in a few components with large dependency sets: nine parts account for 82 of 99 missed or spurious dependency links.

Despite substantial geometric differences, UniPhysGen maintains strong articulation prediction performance, achieving 91.78% Joint Type Accuracy, 12.05° Axis Error, and 92.91% Limit mIoU. These results support generalization to generated geometry when usable part decomposition is available.

2. How do the choices of axis and pivot, the mass check, and the MuJoCo checks behave on these meshes?

We evaluated UniPhys's consistency checks and MuJoCo motion verification on generated assets. 6 of 32 assets failed the object-level mass consistency check, demonstrating the detection of inconsistent mass assignments. We also compared articulation annotations before and after motion verification.

Table R4. Effect of MuJoCo-based motion verification on TRELLIS 2.0-generated assets.

MetricBefore → After
Joint Type Accuracy ↑82.72% → 85.71%
Axis Angular Error ↓16.02° → 13.93°
Pivot Distance Error ↓0.070 → 0.080
Motion Limit mIoU ↑96.76% → 96.66%

After-verification metrics are computed on retained annotations. Axis, pivot, and limit metrics are evaluated only on ground-truth movable parts with correctly predicted joint types.

Motion verification improves the overall quality of retained articulation annotations, increasing Joint Type Accuracy by 2.99 percentage points and reducing Axis Error by 2.09°. Although Pivot Error and Limit mIoU slightly deteriorate, the results support simulation-based filtering of unreliable articulation predictions.

3. Handling Shells with No Interior.

Generated meshes may lack internal structures, such as drawers represented only by front panels, challenging part decomposition and geometric candidate generation.

UniPhysGen leverages object-level and part-level geometric representations to infer articulation semantics from incomplete structures. For example, a drawer front can provide geometric and contextual cues for predicting a prismatic joint even without the complete drawer body. The generated-asset evaluation and Sec. 5.3 provide supporting evidence of robustness to geometric variations.

Nevertheless, severe geometric defects and incorrect part merging remain limitations, particularly during initial segmentation. We will clarify these limitations and include the generated-asset experiments in the revised manuscript.

AuthorsOpenReview ↗

To Reviewer sadk (Part 3/3)

Response to Weakness 3 and Question 3: Non-LLM 3D-native Baseline

Weakness 3 & Question 3:

Non-LLM baselines: Have you trained a 3D-native model on UniPhys-40K, either for part segmentation or for articulation generation?

Response:

We thank the reviewer for this suggestion. We clarify that Table 2 already includes PARTICULATE, an existing 3D-native non-LLM model that jointly performs part segmentation and articulation prediction. To further investigate the contribution of UniPhys-40K and isolate the architectural benefits of UniPhysGen, we conducted two additional experiments.

1. Controlled comparison with Sonata + Motion Head. We trained a Sonata encoder with task-specific regression heads, without an LLM backbone, using the same UniPhys-40K training data and evaluation protocol as UniPhysGen.

Table R5. Controlled comparison with a non-LLM 3D-native baseline.

MethodJoint Type Acc. (%) ↑Axis Error (°) ↓Pivot Error ↓Limit mIoU (%) ↑
Sonata + Motion Head89.3232.970.20671.93
UniPhysGen (Ours)89.969.770.09984.95

Despite comparable Joint Type Accuracy, UniPhysGen reduces Axis Error by 70.4% and Pivot Error by 51.9%, while improving Limit mIoU by 13.02 percentage points. These results demonstrate that UniPhysGen's advantages in continuous articulation parameter estimation extend beyond the Sonata encoder and shared training data.

2. Fine-tuning PARTICULATE on UniPhys-40K. We further fine-tuned PARTICULATE on UniPhys-40K for 50K and 100K steps. Since PARTICULATE jointly predicts part decomposition and articulation parameters, we follow its original evaluation protocol by matching predicted parts to ground-truth parts before computing articulation metrics.

Table R6. PARTICULATE fine-tuning and part matching performance on UniPhys-Bench.

MethodPart Matching Rate ↑Joint Type Acc. ↑Axis Error ↓Pivot Error ↓Limit mIoU ↑
PARTICULATE (Original)47.96%77.17%12.80°0.26163.42%
PARTICULATE (FT, 50K)54.96%69.61%8.19°0.24376.88%
PARTICULATE (FT, 100K)54.29%68.30%9.41°0.24576.96%

Part Matching Rate is evaluated over 5,487 ground-truth movable parts. Articulation metrics are evaluated on matched parts. For all methods, axis angular error, pivot distance error, and joint-limit mIoU are computed only on ground-truth movable parts with correctly predicted joint types.

Interestingly, although PARTICULATE's conditional Joint Type Accuracy decreases after fine-tuning, its Part Matching Rate improves substantially, from 47.96% to 54.96% (50K steps), indicating better coverage of ground-truth movable parts. Its Axis Error and Limit mIoU also improve, demonstrating the value of UniPhys-40K for training existing 3D-native models. The decrease in conditional Joint Type Accuracy should therefore be interpreted alongside the increased matching coverage, rather than in isolation.

Together, these experiments show that UniPhys-40K benefits existing non-LLM models, while UniPhysGen provides stronger fine-grained articulation prediction than the controlled Sonata regression baseline. We will include these results and evaluation details in the revised manuscript.

AuthorsOpenReview ↗

Revision Summary and Updated Manuscript

Revision Summary and Updated Manuscript

We thank the reviewer for the constructive feedback. We have uploaded a revised manuscript incorporating additional experiments and clarifications to address your concerns. The main revisions are summarized below:

  1. Labelling Model and Reproducibility (Section 3.1 and Appendix F.3). We have clarified the LVLM annotation setup and added quantitative comparisons with the open-weight Qwen3.5-35B-A3B model, including initial annotation quality and physics-based verification results, to demonstrate the flexibility of UniPhys across different LVLMs.

  2. Generalization to Generated 3D Assets (Section 5.3 and Appendix F.6). We have added experiments on TRELLIS 2.0-generated meshes, covering physical property prediction, articulation parameter estimation, and MuJoCo-based verification. Additional analyses of generated mesh decomposition and incomplete structures are provided in the appendix.

  3. Non-LLM 3D-native Baselines (Section 5.2 and Appendix F.4). We have incorporated controlled comparisons with Sonata + Motion Head and UniPhysGen + Regression Head, together with PARTICULATE fine-tuning experiments on UniPhys-40K, to further evaluate the contributions of the model architecture and training data.

Detailed experimental results and evaluation protocols are provided in Appendix F, with corresponding summaries and references in the main text.

We appreciate the reviewer's constructive feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their constructive feedback. We are encouraged that the reviewers recognized the following key merits:

  • Significant community contribution through large-scale physical grounding resources. The value of UniPhys-40K and UniPhys-Bench, which provide diverse 3D assets with unified articulation semantics and intrinsic physical properties, addressing a key bottleneck in developing simulation-ready assets for embodied AI and robotics (Reviewers sadk, KW24, jSjN).
  • A scalable, geometry-aware annotation and verification pipeline. The practical value of UniPhys's automated annotation pipeline, which integrates geometric constraints, semantic reasoning, and simulation-driven verification to produce quality-controlled physical annotations (Reviewers sadk, KW24).
  • Robust physical grounding across heterogeneous 3D structures. The robustness of UniPhysGen to rotation and part decomposition variations, supported by targeted ablation studies on global positional encoding, rotation augmentation, and part grouping, demonstrating its ability to learn physical semantics from heterogeneous 3D geometry (Reviewers KW24, jSjN).
  • Strong experimental foundations and reproducibility efforts. The extensive experimental evaluation, human-verified benchmark annotations, detailed implementation documentation, and released code, supporting the reliability and broader utility of the proposed framework (Reviewers sadk, KW24).

Conceptual and methodological contributions. Our work addresses a fundamental question in 3D physical understanding: How can we ground existing 3D assets with diverse geometries and part decompositions in a unified physical semantic framework, without assuming canonical orientations, predefined articulation structures, or geometry regeneration?

To address this challenge, we introduce two complementary contributions: UniPhys, a scalable geometry-aware annotation framework combining physical semantic reasoning with simulation-driven verification; and UniPhysGen, a unified physical grounding model that directly learns physical semantics from native 3D point clouds, leveraging shared object-part representations to infer articulation semantics and intrinsic physical properties.

Together, they establish a foundation for embodied AI and Real2Sim, enabling existing or reconstructed 3D geometry to support physically grounded simulation and executable robotic interactions, including dexterous manipulation from egocentric videos.

To address the reviewers' concerns, we conducted the following additional experiments:

  • Open-weight VLM reproducibility. Evaluated Qwen3.5-35B-A3B against GPT-5 on 120 assets (1,197 parts). After physics-based verification, Qwen achieved 90.57% Joint Type Accuracy, versus 93.98% for GPT-5. We also reported the exact GPT-5 version and annotation cost.
  • Effectiveness of UniPhysGen's architecture. We fine-tuned Real2Code, URDFormer, and PARTICULATE on UniPhys-40K and introduced a Sonata + Motion Head baseline using the same training data. Although fine-tuning improved existing baselines, UniPhysGen achieved stronger articulation predictions, reducing Axis Error by 70.4% and Pivot Error by 51.9% compared with Sonata + Motion Head. This demonstrates the benefits of our unified architecture beyond training data alone.
  • Rotation robustness. Evaluated Cartesian and spherical parameterizations on 5,487 kinematic samples under random X/Y/Z rotations. Spherical parameterization reduced average Axis Error by 26.4% (18.92° → 13.92°).
  • Generalization to AI-generated geometry. We evaluated UniPhysGen on 32 TRELLIS 2.0-generated assets, achieving 91.78% Joint Type Accuracy, 12.05° Axis Error, and 92.91% Limit mIoU. These results provide further evidence of generalization to unfamiliar generated geometry.
  • Quantitative evidence of simulation executability. We evaluated part-level physical interaction success on 50 UniPhys-Bench assets, comparing UniPhysGen with two image-conditioned physically grounded 3D asset generation methods, PhysX-Anything and PhysX-3D. UniPhysGen achieved 69.35% success, outperforming PhysX-Anything (37.33%) and PhysX-3D (26.72%), measured over each method's evaluated movable parts.

Manuscript Revision. Revisions are highlighted in blue; new experiments appear in Appendix F.3–F.7.

Open-source commitment. We anonymously released the UniPhys code, model weights, and UniPhys-Bench. The full UniPhys-40K dataset is ready for public release.

Together, these additional results strengthen UniPhys as a unified framework connecting 3D geometry with physical semantics and executable simulation.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.