acceptodds
Under review as a conference paper at ICLR 2027

ArtSplat: Feed-Forward Articulated Object Reconstruction via Gaussian Splatting

Abstract

Articulated object reconstruction requires simultaneous inference of geometry and underlying articulation structure. Existing methods based on NeRF and 3D Gaussian Splatting (3DGS) typically rely on dense views or strong priors (, depth maps, joint types, predefined number of joints), requiring costly per-object optimization. In this paper, we propose ArtSplat, the first feed-forward 3DGS framework that reconstructs both geometry and joint parameters from sparse multi-view images across multiple articulation states in a single forward pass. Our novel per-pixel joint map representation enables integration of joint parameter estimation into the feed-forward pipeline. We further propose a Cross-State Attention (CSA) mechanism with state tokens, which effectively captures discrete motion across input states. Comprehensive experiments demonstrate competitive geometry reconstruction, accurate joint estimation and cross-dataset generalizability of ArtSplat on both single- and multi-joint objects. Compared to the per-object optimization baselines, ArtSplat achieves 389 times faster inference time.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Articulated Object Reconstruction, 3D Gaussian Splatting, Feed-Forward 3D Reconstruction, Sparse-View 3D Reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_hxXPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper develops a fast, feedforward method based on VGGT to acquire 3D Gaussians from sparse views for an articulated object, along with joint information. The model outputs the joint information and geometry in seconds, faster than prior similar methods. The model is trained on the partnet-mobility dataset, and evaluated on a held out set from the same dataset, as well as the AKB-48 dataset. The model results are compared against relevant reconstruction and feedforward methods.

Critical strengths
  • The lightweight input required (sparse view inputs) and inference time that are much faster than comparable methods are a great advantage.
  • The framework is clear and easy to follow, elegant in simplicity.
Critical weaknesses
  • There is no real-world tests or validation that show that the method is applicable to more noisy and challenging real world inputs. Datasets worth considering includes the two real-world objects of PARIS.
  • Segmentation usually is a challenging topic in articulated objects. The paper doesn't discuss the accuracy of the clustering in detail.
Questions for authors
  • How well would ArtSplat work on real world data that may have more noisy and complex input?
  • What are the categories of the objects in the evaluation set; how diverse are they?
  • Often times in a partnet-mobility object with many parts, some parts have very similar motion such as two drawers or two doors that move similarly. How reliable is the segmentation on those type of objects?
  • At inference, how is the background segmented from the foreground? And how reliable is the clustering, how accurate are the clustering results against GT part masks?
  • The reconstruction baselines were tested on sparse inputs as well, not their usual setting of dense view input. How badly did this affect their results?
2 comments

We thank the reviewer for the encouraging review and the concrete suggestions. We followed them with new experiments, discussed point by point below. Edits to the paper are marked in blue in the revised manuscript.


[W1] No real-world tests or validation on noisy and challenging real-world inputs (e.g., the two real-world objects of PARIS)

We agree that real-world results were missing, so we added two sets of real captures: the two real PARIS objects, which have joint ground truth, and seven RealScan objects (Table III and Fig. I in App. D.3; the table is reproduced below).

Table. Real-world evaluation.

MethodTypeViews / statePARIS (real) Ang$_m$↓PARIS (real) PSNR↑PARIS (real) SSIM↑RealScan PSNR↑RealScan SSIM↑
ArtGSOptimization448.7417.090.80721.350.8488
ScrewSplatOptimization411.7810.030.55419.900.8530
LARM†Feed-forward339.2018.350.84114.970.7990
ArtSplat (Ours)Feed-forward415.3018.120.85219.240.8550

On PARIS, LARM is given GT camera poses and intrinsics, the GT joint type for axis estimation, and the articulation value of each input state.

With only uncalibrated RGB images and no real training data, ArtSplat recovers the joints of both PARIS objects, second to ScrewSplat and ahead of ArtGS and LARM, which each miss one joint, with the highest SSIM and a PSNR on par with the best method, and outperforms the feed-forward LARM on RealScan. We acknowledge that the overall performance on real data is still well below that on synthetic data, and that this evaluation is small. We attribute the gap mainly to training on synthetic data only. Closing it is what we need to improve next, and we state this in the limitations.


[W2, Q4] The paper doesn't discuss the accuracy of the clustering in detail; how is the background segmented at inference?

(1) Background. ArtSplat uses an object mask to separate the object from the background. In all experiments, we use the object masks provided with the dataset; i.e., the alpha channel of the synthetic renderings and the released masks of the real captures.

(2) Clustering accuracy. As suggested, we compared the predicted parts with the GT part masks, using ARI and the mean IoU of matched parts (Table V in App. E.1, reproduced below).

Table. Clustering accuracy against GT part masks.

MetricArtSplat
Single-joint (18) ARI↑0.939
Single-joint (18) mIoU↑0.977
Multi-joint (50) ARI↑0.875
Multi-joint (50) mIoU↑0.761

The predicted parts closely match the GT parts on both single-joint and multi-joint objects. Qualitative examples of the clustering and of the predicted parts are provided in Figs. II and III (App. E.1). Adjacent parts that share an axis and move similarly are the hardest case, which we examine in Q3 below. To see how much this affects the articulation, we also replaced the clustering with the GT part labels at inference, and the axis error changes only slightly (Table IV in App. E.1, copied below).

Table. HDBSCAN settings and GT part labels at inference.

min_cluster_sizemin_samplesAng$_m$↓PSNR↑
1056.2526.24
50 (Default)255.7926.30
150506.4226.15
3001008.8525.40
GT Labels–5.2126.85

[Q2] What are the categories of the objects in the evaluation set; how diverse are they?

The 68 test objects (18 single-joint, 50 multi-joint) cover all 15 categories of our PartNet-Mobility subset: StorageFurniture 24, Table 17, Oven 3, and 2 for each of the other 12 (App. A). The most complex object has 14 joints.


[Q3] How reliable is the segmentation on parts with very similar motion, such as two drawers or two doors?

This is an important case, so we looked at it directly. 37 of the test objects have parts that share an axis, such as two drawers or two doors, and we re-rendered them so that these parts move identically (table below).

Table. Same-axis parts moving identically (37 objects).

Motion of same-axis partsARI↑PSNR on pseudo-labels↑PSNR on GT labels↑
Original states0.79526.0627.16
Identical motion0.80226.6726.81

The segmentation is unchanged from the original states, because the clustering also uses the 3D position of each Gaussian. By combining the predicted joint axes with the 3D position of each Gaussian, ArtSplat thus separates same-axis parts even when they move identically, a case in which motion cues alone would merge them.

[Q5] The baselines were tested on sparse inputs, not their usual dense-view setting; how badly did this affect their results?

Yes, considerably. To measure it, we re-ran ArtGS and ScrewSplat with the dense (100 and 50 per state, respectively) views they were originally designed for. We report both settings in Table II in App. D.1, copied below.

Table. Optimization baselines with sparse (4) and their intended dense views vs. ArtSplat with 4 views per state.

Method# Views / StateCD-w↓CD-s↓CD-m↓Ang$_m$↓Pos$_m$↓PSNR↑SSIM↑Time
ArtGS40.0960.1020.78249.311.82525.210.89315m 50s
ArtGS1000.0210.0450.16821.250.4036.660.98918m 59s
ScrewSplat40.0350.0440.40546.783.36823.490.91223m 05s
ScrewSplat500.0180.0260.05524.460.32734.030.98026m 59s
ArtSplat (Ours)40.0340.0240.0305.790.86426.300.921<2 s

With dense views, they render better than ArtSplat and their axis error is roughly halved, so the sparse input does account for a large part of their drop. On multi-joint objects, however, their axis error still remains well above that of ArtSplat with 4 views. The sparse setting therefore explains the gap in appearance, but only part of the gap in motion.

We thank the reviewer again for the valuable suggestions, which made the paper more complete. We are happy to answer any further questions and will respond promptly.

Reviewer_tDcHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ArtSplat, a feed-forward reconstruction framework for articulated objects based on 3D Gaussian Splatting. It recovers 3D geometry, camera poses, and articulation parameters simultaneously from sparse multi-view RGB images captured across two different motion states. Evaluations on the PartNet-Mobility and AKB-48 datasets demonstrate that ArtSplat achieves competitive results in geometric reconstruction, articulation estimation, and novel-view synthesis.

Critical strengths
  1. It extends articulated object reconstruction to a feed-forward setting. ArtSplat requires only 8 RGB images (2 states × 4 views) and does not necessitate prior knowledge of camera parameters or the number of joints, with an inference time of less than 2 seconds.
  2. The "per-pixel joint map" is a well-conceived representation. Instead of predicting a fixed number of joint parameters, the model predicts an 11-dimensional joint map at each pixel—encompassing joint type, axis, pivot, angle, and displacement. The authors employ HDBSCAN to derive part-level articulation from these per-pixel predictions, allowing the framework to handle objects with varying numbers of movable parts.
  3. The "Cross-State Attention" mechanism is specifically designed for the challenges of articulated reconstruction. The paper utilizes state tokens and cross-state attention to capture changes between different states, while an invariant/variant decoder separately predicts the underlying joint structure and the magnitude of motion.
Critical weaknesses
  1. Limited innovation; the approach is primarily a combination of existing techniques (VGGT and AnySplat), adapted for the task of articulated reconstruction.
  2. Lack of real-world experiments, which diminishes the persuasiveness of the results.
  3. Absence of a systematic analysis regarding the number of input views, the degree of state variation, or the amplitude of motion. All primary experiments rely on a fixed configuration of 2 states and 4 views. However, the performance of this approach—which recovers articulation parameters from discrete states—is likely to be significantly influenced by two factors: the number and coverage of camera views at each state, and the magnitude of the articulation motion between states.
Questions for authors
  1. Is the comparison with optimization-based methods fair? Four camera views pose a major challenge for optimization-based approaches, as they lack any prior information. Furthermore, is the reconstruction quality achieved by this feed-forward method using only four views sufficient for simulation purposes? This might be a "pseudo-requirement" scenario—one that offers high speed but fails to deliver the high-quality reconstruction necessary for practical use.
  2. All quantitative experiments in this paper are based on synthetically rendered images, including tests on the AKB-48 dataset. Given that real-world scenes differ significantly from synthetic data in terms of lighting, texture, occlusion, and camera distribution, could the authors provide qualitative or quantitative experiments using real-world captured images to further validate the actual generalization capability of ArtSplat?
4 comments

We thank the reviewer for thoughtful feedback. We added an experiment for each point, and the revised parts of the paper are marked in blue.


[W1] Limited innovation; the approach is primarily a combination of existing techniques (VGGT and AnySplat)

We agree that ArtSplat builds upon VGGT and AnySplat, which provide the static geometry. However, we emphasize that neither of them handles articulation. Our contribution lies in the ideas that realize articulation in a feed-forward 3DGS model for the first time, to the best of our knowledge: a per-pixel joint map that densely represents the joints, and cross-state modules (state tokens and CSA) that estimate the joints from the motion between two states.

To further verify that this is more than a naive combination of existing techniques, we compare against a baseline that directly combines the existing models without our components: AnySplat reconstructs the two states separately, and the joints are then estimated by aligning the two reconstructions, taking the points that moved as the movable parts, and converting the rigid motion of each part into a revolute or prismatic joint (Table 4 in Sec. 5.3, copied below; details in App. D.2).

Table. Direct combination (modular baseline) and ablations on the same backbone.

MethodCD-s↓CD-m↓Ang$_m$↓Pos$_m$↓PSNR↑
AnySplat (per state) + registration0.1230.27739.396.52719.36
w/o CSA and state token0.0410.15631.462.91722.51
w/o CSA0.0290.08218.741.85224.18
w/o Inv/Var split (single branch)0.0260.0417.121.64325.07
Single-stage joint training0.0300.06912.851.42823.74
ArtSplat (Ours)0.0240.0305.790.86426.30

With the combination alone, the articulation is not recovered reliably. It becomes accurate only once our components are added, most of all the cross-state modules. The backbones thus supply the geometry, while the articulation comes from the proposed components. We added this baseline to Table 4.


[W2] Lack of real-world experiments

Following the suggestion, we conducted additional experiments on the two real PARIS objects, which have joint ground truth, and seven RealScan objects (Table III and Fig. I in App. D.3; the table is reproduced below).

Table. Real-world evaluation. LARM uses 3 views per state and GT poses, joint type and articulation values; all other methods use 4 views.

MethodTypePARIS (real) Ang$_m$↓PARIS (real) PSNR↑PARIS (real) SSIM↑RealScan PSNR↑RealScan SSIM↑
ArtGSOptimization48.7417.090.80721.350.8488
ScrewSplatOptimization11.7810.030.55419.900.8530
LARMFeed-forward39.2018.350.84114.970.7990
ArtSplat (Ours)Feed-forward15.3018.120.85219.240.8550

With only uncalibrated RGB images and no real training data, ArtSplat recovers the joints of both PARIS objects, second to ScrewSplat and ahead of ArtGS and LARM, with the highest SSIM and a PSNR on par with the best method, and outperforms the feed-forward LARM on RealScan. We acknowledge that the overall performance on real data is still below that on synthetic data, and that this evaluation is small. We attribute the gap mainly to training on synthetic data only. Closing it is what we need to improve next, and we state this in the limitations.

[W3] Absence of a systematic analysis regarding the number of input views, the degree of state variation, or the amplitude of motion

We added analyses of the motion amplitude between the two states and of the number of input views per state (App. E.2). Even in the hardest settings we tested, the axis error of ArtSplat stays below that of every baseline at the default setting (Table 1).

(1) State variation and motion amplitude: results grouped by the articulation change between the two input states (Table VI in App. E.2, copied below).

Table. Results by the articulation change between the input states.

Range (Revolute)Ang$_m$↓PSNR↑Range (Prismatic)Ang$_m$↓PSNR↑
0°–10°9.5024.100.00–0.10 m10.2024.20
10°–20°5.4226.510.10–0.20 m6.1026.45
20°–30°5.6826.380.20–0.30 m5.9526.30
>30°5.8126.25>0.30 m5.8826.20

When the two states differ only slightly, the accuracy drops somewhat. Above about 10° or 0.1 m, the size of the motion makes no notable difference.

(2) Views per state, selected by farthest-point sampling (Table VII in App. E.2, reproduced below).

Table. Input views per state.

Views / StateAng$_m$↓PSNR↑SSIM↑
28.2721.340.880
4 (default)5.7926.300.921
84.6526.330.923
165.0226.570.924

With more than 4 views, the axis error improves slightly, while PSNR and SSIM change little. With 2 views, the accuracy drops. We attribute this to the model being trained with 4 views per state. Regarding view coverage, ArtSplat reconstructs only what the input views observe, so parts that are not visible in any view cannot be recovered. We acknowledge this limitation and are considering generative priors to complete unobserved regions in future work.


[Q1] Is the comparison with optimization-based methods fair, and is the four-view reconstruction sufficient for simulation?

We agree that four views are a hard setting for optimization-based methods, so we also ran them with the dense views they were designed for, 100 and 50 per state (Table II in App. D.1, copied below).

Table. Optimization baselines with sparse (4) and their intended dense views vs. ArtSplat with 4 views per state.

Method# Views / StateCD-w↓CD-s↓CD-m↓Ang$_m$↓Pos$_m$↓PSNR↑SSIM↑Time
ArtGS40.0960.1020.78249.311.82525.210.89315m 50s
ArtGS1000.0210.0450.16821.250.4036.660.98918m 59s
ScrewSplat40.0350.0440.40546.783.36823.490.91223m 05s
ScrewSplat500.0180.0260.05524.460.32734.030.98026m 59s
ArtSplat (Ours)40.0340.0240.0305.790.86426.300.921<2 s

As expected, dense views help them considerably; the baselines render better than ArtSplat, and their axis error is roughly halved. On multi-joint objects, however, it still remains well above that of ArtSplat with 4 views. The sparse setting, therefore, explains the gap in appearance, but only part of the gap in motion.

On the "pseudo-requirement" concern, we think the sparse setting is a real one, for example, when a robot meets a new object and cannot capture 50-100 calibrated views of each state. What such a use needs first is the kinematics; that is, which parts move and about which axes, and here ArtSplat is more accurate than the dense-view baselines on multi-joint objects (Table II). To see how the output behaves in practice, we also tried it in simulation. Exported as a URDF, the parts move about the predicted axes in SAPIEN (Fig. IV, App. F). There is still room to improve the reconstruction quality, and with that improvement we believe the method can be used in practice.

We hope the new results clarify the raised concerns. If anything remains unclear, please feel free to ask, and we will respond as soon as possible.

Thanks for the response. The results of the author's new submission address some of my concerns. But I still have my doubts about this setup. I checked the newly added visualization results and quantitative results on real-world data. ArtSplat does not achieve high-fidelity rendering, and the geometry looks poor. I still don't think this 4-view setting has practical value for simulation or even robot training. Considering the paper's contribution comprehensively, I give a weak rejection.

We thank the reviewer for examining the new results. We agree that the rendering and geometry, on real data in particular, are not yet at the level needed for simulation or robot training, and we state this in the limitations.

We would still like to note two points. First, four views per state is hard for every method. In this setting, ArtSplat estimates the joints more accurately than the baselines on both synthetic benchmarks and recovers the joints of both real PARIS objects. It is also not limited to this setting: it takes 8 or 16 views per state without retraining, with lower axis error (Table VII).

Second, obtaining an articulated object with existing optimization-based methods requires 50 or more views for each articulation state, with known camera poses. ArtSplat reduces this to a few uncalibrated images and a single forward pass. It is not yet a practical tool, but we believe that removing the need for dense, posed capture is a meaningful step in that direction.

Reviewer_wsYJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a feed-forward 3DGS pipeline to reconstruct articulated objects from sparse views. Basically the network follows AnySplat and predicts pixel-aligned Gaussians. The difference is that this paper additionally predicts articulation parameters for each gaussian, through the joint prediction module. The experiments show that ArtSplat has good performance on PartNet-Mobility and good generalization on AKB-48, and also is much faster in inference speed than those optimize-based methods.

Critical strengths
  1. The idea is simple and straightforward. Using a feed-forward pipeline will be more pratical in the downstream applications.
  2. The performance is inspiring, especially the motion metrics on PartNet-Mobility. And visually ArtSplay looks better than the compared methods.
  3. The pipeline figure is clear and the paper writing is easy to follow.
Critical weaknesses
  1. Current visual results still do not satisfy me enough. The rendering has artifacts, and the exported mesh is broken. Compared with those mesh-based methods such as Particulate, what's the benefit of Gaussian-based methods? If the 3D Gaussian cannot provide good enough rendering, why we still choose it? It's not as that useful as mesh in the simulator.
  2. I didn't find any visual results of how the clusering method works. The authors should provide more visualizations of intermediate results. The ablation studies also lack supporting visualizations.
  3. Why ArtSplat performs much better than other methods in the motion metrics? Can the authors provide some deep analysis?
  4. Lacks the discussion and comparison with the latest methods, e.g. ART, REArtGS++, Artic-O, StructureGS, LARM.
Questions for authors
  1. can you also evaluate the articulated geometry, like particulate does?
  2. How does the clustering method work? Can the authors provide some visual results or evidence?
4 comments

We appreciate the review and candid feedback on visual quality, which we address in W1. We also ran the requested comparisons and analyses. The corresponding revisions appear in blue in the revised manuscript.


[W1] Current visual results are not satisfying; what is the benefit of Gaussian-based methods over mesh-based methods such as Particulate?

Particulate addresses a different setting, taking a complete 3D mesh and predicting its articulation. In contrast, our ArtSplat takes a few uncalibrated images and reconstructs the geometry and joints together, so it applies when no mesh exists. We compared the two by running Particulate on a GT mesh and on a TRELLIS mesh generated from images (table below).

Table. Particulate on a GT mesh and on a mesh generated from images vs. ArtSplat.

MethodInputType acc.↑Ang$_m$↓Pos$_m$↓Time
ParticulateGT mesh1.0000.190.2573.1 s
TRELLIS mesh + Particulate4 images (state 0)0.55617.471.64991.2 s
ArtSplat (Ours)8 images1.0006.650.436<2 s

With a clean mesh, Particulate is nearly exact. From images, its axis error is considerably higher than that of ArtSplat, and it takes far longer per object. The two methods are complementary.

We see two practical advantages of Gaussians for reconstructing an articulated object from a few uncalibrated images in a single forward pass. They can be predicted directly by a pose-free feed-forward backbone, and they render in real time, which makes it inexpensive to supervise the motion through rendering during training and to render the object at any joint state afterwards. We clarify that we do not claim that they are preferable to meshes in general.

We agree that there is room to improve the rendering and mesh quality from 4 views per state. The predicted kinematics are already accurate: ArtSplat has the lowest axis error and movable-part Chamfer distance (Table 1), and when its output is exported as a URDF, the doors and drawers move about the predicted axes in SAPIEN (Fig. IV, App. F). With better surface quality, which we plan to pursue with representations such as 2D Gaussians, we believe the method can become practical for simulation. We thank the reviewer again for this insightful feedback.


[W2] No visual results of how the clustering method works or for the ablation studies

Following the suggestion, we added Fig. II (the clustering stage by stage) and Fig. III (predicted vs. GT parts on eight objects) in App. E.1. The clustering groups the Gaussians by joint type, represents each by its predicted joint parameters (Plücker coordinates for revolute joints; axis, displacement and 3D position for prismatic joints), and clusters them into parts with HDBSCAN.

Against the GT part masks, the predicted parts closely match the GT parts on both single-joint and multi-joint objects (App. Table V).

For the ablations, Fig. II shows the intermediate outputs of the part assignment (the per-pixel joint type in each state and the clustered parts against the GT parts). App. Table IV ablates the clustering itself: varying the HDBSCAN settings or replacing the clustering with the GT part labels changes the axis error only slightly.


[W3] Why does ArtSplat perform much better than other methods in the motion metrics?

We conducted two additional analyses to find where the motion accuracy comes from.

(1) Attention. We measured where the cross-state modules attend. For each state token, we take its attention over the image patches of the other state and compute the share that falls on the moving part, using the GT part masks. Although moving-part patches make up only 4.9% of all patches, they receive 42.5% of the attention, about 9 times more than under uniform attention.

(2) Sanity check. When the same state is given twice, the predicted rotation between the two inputs vanishes (0.16°, against 29.4° for two different states), so the model does not produce a motion unless the two states differ.

These analyses are consistent with the ablation in Table 4. Removing CSA and the state tokens substantially increases the axis error. We therefore attribute the motion accuracy to the cross-state modules, which make the model focus on the motion between the two states.

[W4] Lack of discussion and comparison with the latest methods (ART, REArtGS++, Artic-O, StructureGS, LARM)

We compared four of the five suggested methods on the 18 single-joint objects in the table below; ART could not be run, as explained below. ArtSplat has the lowest axis and pivot errors, also against the optimization methods given GT poses and part counts, and is the only method that needs neither GT camera poses nor joint priors. It is also orders of magnitude faster than the optimization methods.

Table. Recent methods; required priors are given from the ground truth. StructureGS is re-implemented, as its code is not released.

MethodTypeCamera poseOther priors#PartsAng$_m$↓Pos$_m$↓CD-w↓CD-s↓CD-m↓Time
ARTFeed-forwardRequired––––––––
LARMFeed-forwardRequiredJoint type, articulation valuesOne moving part per state pair22.506.890.1100.1080.13764 s
Artic-OFeed-forwardNot requiredJoint typeOne moving part per state pair8.991.660.1360.1300.2680.75 s
REArtGS++OptimizationRequired–Given42.712.160.0670.0700.13421 min
StructureGS (re-impl.)OptimizationRequired–Given39.412.430.0710.1460.34333–37 min
ArtSplat (Ours)Feed-forwardNot requiredNoneVariable6.650.4360.0310.0230.028<2 s

ART has not released code or weights, and it is pre-trained on a licensed set of 130k part-annotated objects, so it cannot be faithfully re-trained on our data.


[Q1] Can you also evaluate the articulated geometry, like Particulate does?

Yes, we ran Particulate's evaluation code on our outputs: part-wise Chamfer distance (PC), whole-object Chamfer distance (OC), and part-wise gIoU, at the two input states and at five unseen states over the joint range (table below).

Table. Articulated geometry with Particulate's protocol.

MethodPriorsInput states PC↓Input states OC↓Input states gIoU↑Unseen states PC↓Unseen states OC↓Unseen states gIoU↑
Naive (GT mesh, no motion)GT mesh0.1816–0.2260.17380.00750.267
Artic-OGT joint type0.03900.01610.1910.05450.01800.195
REArtGS++GT poses, #parts0.01260.00490.4260.02560.00940.393
ArtSplat (Ours)–0.00890.00450.4630.01030.00670.427

ArtSplat performs the best on all six metrics, also against REArtGS++ with GT poses and part counts, and the margin in Chamfer distance is largest at the unseen states. This evaluation covers the single-joint split.

We hope our responses address the reviewer’s concerns. Should any further questions arise, we would be happy to answer them promptly.

Thank you for the detailed responses and additional experiments, which have addressed some of my concerns. I still have reservations about reconstruction quality and practical usability in simulation. I will consider the other reviewers’ and AC’s feedback before deciding whether to change my score.

We thank the reviewer for the reply. Regarding practical usability in simulation, we would like to point to Fig. IV (App. F): the parts and joints predicted by ArtSplat are exported as a URDF and loaded into SAPIEN, where the doors and drawers open and close about the predicted axes.

If the reviewer could let us know which aspect remains a concern, we would be glad to run additional experiments on it and report the results promptly.

Reviewer_Wi9TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the reliance of articulated 3D object reconstruction on dense multi-view inputs and time-consuming per-scene optimization, this paper proposes ArtSplat, a feed-forward articulated 3D Gaussian Splatting framework. Using a pre-trained large visual geometry model (AnySplat/VGGT) as the backbone, the model takes uncalibrated, unordered cross-state sparse images (2 states × 4 views) as input. It prepends state tokens to each frame and captures inter-state motion parallax via a Cross-State Attention (CSA) module. The authors design an 11-channel per-pixel joint map and decouple parameters into state-invariant (joint type, axis, anchor) and state-variant (rotation, translation) components through symmetric pooling and independent conditioning. Under a two-stage training strategy—first fitting geometry and joint parameters, then fine-tuning jointly with rendering loss—the model clusters predicted Plücker line coordinates via HDBSCAN at test time to determine part labels and perform explicit rigid transformations. On the synthetic PartNet-Mobility and AKB-48 datasets, the method achieves hundreds of times faster inference than traditional optimization baselines, alongside significant improvements in joint orientation accuracy.

Critical strengths

Paradigm Innovation and Inference Efficiency Breakthrough: The first work to advance sparse-view articulated 3DGS reconstruction from expensive test-time optimization to an end-to-end single forward pass. Without requiring test-time optimization, ground-truth joint counts, or depth priors, the reconstruction time is compressed to under 2 seconds (389–700× faster than optimization-based methods).

Principled Per-pixel Parameterization and Decoupled Decoding: Formulating joint estimation as an 11-channel per-pixel regression avoids the instability of discrete segmentation and fixed-slot matching. Factoring parameters into state-invariant and state-variant branches—with symmetric aggregation and FiLM conditioning driven by CSA outputs—aligns well with the underlying physical symmetry of articulated kinematics.

Effective Two-Stage Training Scheme: Explicit Gaussian articulated transformations are prone to corrupting photometric gradients during early training when joint predictions are noisy. The proposed two-stage curriculum (constraining geometry/motion manifolds first before enabling differentiable photometric supervision) is technically sound, and its necessity is well-supported by the ablations.

Substantial Gains in Motion Axis Estimation: Compared to optimization baselines that severely degrade under sparse views (axis angular error > 30°–50°), ArtSplat achieves an angular error of 5.79° on the PartNet-Mobility multi-joint split, confirming the efficacy of the CSA module in extracting discrete motion features across states.

Critical weaknesses

Incomplete Comparison with Feed-forward Articulated Baselines (ART / LARM):

Omission of ART: The paper explicitly cites ART (Li et al., 2026) as one of the most closely related feed-forward articulated reconstruction frameworks. While the reviewer acknowledges differences in input assumptions (ART requires calibrated camera poses and Plücker ray embeddings) and representations (SDF volume rendering with fixed slots vs. pose-free 3DGS), both share the core objective of single-pass feed-forward articulated modeling. The complete absence of quantitative comparison or detailed experimental alignment with ART leaves ArtSplat’s exact performance positioning unclear.

Incomplete Evaluation against LARM: In Table 2 (AKB-48 evaluation), only axis angular error (AngmAng_m), PSNR, and SSIM are reported for LARM (Yuan et al., 2025). Core 3D geometric metrics (Chamfer Distance: CD-w, CD-m) and axis position error (PosmPos_m) are omitted. Relying solely on 2D rendering and axis angle fails to sufficiently substantiate the claimed superiority in 3D geometric fidelity.

Discrepancy Between Multi-State Formulation and Binary-State Implementation:

Section 3 formalizes the problem for an arbitrary number of states SS.

However, the concrete formulations in Section 4.2 are strictly hardcoded for binary states: the CSA block (Equation 1: z~s=CSA(Q=zsbb,K=F(1−s),V=F(1−s))\tilde{z}_s = \text{CSA}(Q=z_s^{bb}, K=F^{(1-s)}, V=F^{(1-s)}) where s∈{0,1}s \in \{0, 1\}) and the invariant conditioning vector (cinv=(z~0+z~1)/2c^{inv} = (\tilde{z}_0 + \tilde{z}_1)/2) are explicitly binary. Furthermore, all training and evaluation experiments are limited to S=2S=2. The mechanism for scaling to S≥3S \ge 3 states remains unaddressed both theoretically and empirically.

Lack of Attribution Against Foundation Geometry Baselines:

The model is initialized from the 1B-parameter VGGT/AnySplat backbone. The current ablations do not isolate how much of the performance advantage stems from the foundation model's strong 3D spatial priors versus the proposed CSA and per-pixel joint head.

A natural modular baseline is missing: running pre-trained AnySplat/VGGT independently on each state to yield separate point clouds, followed by standard geometric registration (e.g., ICP or rigid point cloud alignment) to recover articulation parameters.

Geometric Ambiguity in Prismatic Joint Clustering:

At inference time, part pseudo-labels are assigned via HDBSCAN. Section 4.3 specifies that clustering for prismatic joints is performed solely on axis direction aa and displacement dd.

This parameterization admits degenerate failure modes: if an object contains multiple spatially independent prismatic parts that move along the same direction by the same distance (e.g., two parallel drawers opening synchronously), they become indistinguishable in (a,d)(a, d) space and will be erroneously merged without 3D spatial coordinates (e.g., Gaussian centers μn\mu_n) to constrain spatial adjacency.

Train-Test Discrepancy in Part Assignment and Synthetic-to-Real Gap:

Training relies on ground-truth part labels for consistency regularization and Gaussian transformation, whereas inference relies entirely on heuristic HDBSCAN clustering.

Moreover, experiments are confined to synthetic rendered datasets (PartNet-Mobility and AKB-48). The robustness of the feed-forward predictions and Plücker-space clustering under real-world sensor noise, specularity, and clutter remains unverified.

Questions for authors

1 Given that ART also performs single forward-pass articulated reconstruction—notwithstanding differences in pose assumptions and slot mechanisms—could the authors elaborate on the technical positioning between ArtSplat and ART? If feasible, could comparative results or a targeted analysis on a shared subset (e.g., under known camera poses) be provided?

2 In Table 2 (AKB-48 evaluation), could quantitative results for Chamfer Distance (CD-w, CD-m) and joint position error (PosmPos_m) be reported for LARM to offer a complete assessment of 3D geometric fidelity and kinematic accuracy?

3 When the input comprises S≥3S \ge 3 discrete states, how are the CSA Query/Key/Value tensors and the invariant conditioning vector cinvc^{inv} constructed from Equation 1 and Equation cinvc^{inv}? Does the current codebase natively support inference for S≥3S \ge 3, and are there preliminary results available?

4 In Stage 2 training, while ground-truth part labels are used for grouping, part-level motion parameters are derived by averaging per-pixel predictions. Early in training when local axis and anchor predictions are imperfect, does transforming Gaussians via these averaged parameters inject suboptimal gradients via Lrgb\mathcal{L}_{rgb}? How does the two-stage schedule ensure stable convergence in practice?

5 For independent prismatic parts sharing identical motion directions and displacement magnitudes but located at distinct spatial positions (e.g., dual parallel drawers), how does HDBSCAN avoid merging them if clustering relies solely on (a,d)(a, d) features? Are 3D coordinates (such as Gaussian centers μn\mu_n) incorporated into the feature representation?

6 Could the authors provide or discuss a modular baseline (e.g., using pre-trained AnySplat/VGGT to reconstruct per-state geometry independently, followed by standard rigid registration to fit joint parameters) to demonstrate the net benefit of end-to-end joint prediction over a strong geometric model paired with classical post-processing?

3 comments

We thank the reviewer for the detailed review and constructive feedback. We carried out the requested experiments and addressed each point below. All changes to the paper are highlighted in blue in the revised manuscript.


[W1] Incomplete Comparison with Feed-forward Articulated Baselines (ART / LARM)

Omission of ART. ART has not released code or weights, and it is pre-trained on a licensed set of 130k part-annotated 3D objects, so it cannot be faithfully retrained on our data. We therefore compare with LARM and Artic-O, the other state-of-the-art feed-forward methods. Like ART, they take sparse images of two articulation states, each given its required priors from the ground truth. We also evaluate ArtSplat with known camera poses in the table below.

Table. Feed-forward methods with their required priors.

MethodPriors given (GT)Angₘ↓Posₘ↓CD-w↓CD-s↓CD-m↓
LARMCamera poses, joint type, articulation values22.506.890.1100.1080.137
Artic-OJoint type8.991.660.1360.1300.268
ArtSplat (Ours)None6.650.4360.0310.0230.028
ArtSplat + known posesCamera poses6.070.4030.0290.0250.024

Without any prior, ArtSplat is more accurate than both baselines with their GT priors. When GT camera poses are available, its axis and pivot errors decrease further, since the pose-estimation error is removed. Unlike ART, which requires calibrated poses and a fixed number of part slots, ArtSplat needs neither, and it can still benefit from poses when they are given.

Incomplete Evaluation against LARM. Table 2 (Sec. 5.2) now reports CD-w, CD-s, CD-m and Pos$_m$ for all methods, computed with the same script and ground truth (reproduced below).

Table. AKB-48 with all metrics.

MethodCD-w↓CD-s↓CD-m↓Angₘ↓Posₘ↓PSNR↑SSIM↑
ArtGS0.0400.0480.07072.211.63428.610.9179
ScrewSplat0.2030.2190.26641.601.93521.600.8920
LARM0.1290.1200.22265.301.32922.700.9240
ArtSplat (Ours)0.0650.0590.09322.001.10528.700.9384

ArtSplat is superior to the feed-forward LARM on every metric and has the lowest axis and pivot errors overall. ArtGS, which uses GT poses, depth and the number of joints, has the lowest Chamfer distance; we revised the geometry claim in Sec. 5.2 accordingly.


[W2] Discrepancy between Multi-State Formulation and Binary-State Implementation

We thank the reviewer for this clarifying question. We revised the manuscript to clearly state in Sec. 3 that the implementation and all experiments used S = 2. For S > 2, the current code runs without retraining; with a fixed reference state r, the pairwise model is applied to every pair (r,s), so CSA is equivalent to Eq. (1) with F(s)\mathbf{F}^{(s)} as keys and values. The invariant predictions (type, axis, pivot) are averaged over the S − 1 pairs; the variant ones (θs\theta_{s}, dsd_{s}) come from the pair (r,s). We compare the performance with S = 2, 3, 4 below.

Table. Inference with S > 2 states without retraining.

Input#ImagesAng$_m$↓Pos$_m$↓PSNR↑
S = 2 (from Table 1)85.790.86426.30
S = 3, pairwise + invariant averaging124.440.72326.78
S = 4, pairwise + invariant averaging164.290.65326.85

With this result, we confirm that more states lower both the axis and pivot errors (Table VIII in App. E.3, shown above). A native multi-state model, with all other states as keys and values and cinv=1S∑sz~sc_{\mathrm{inv}} = \frac{1}{S}\sum_s \tilde z_s, requires retraining.


[W3] Lack of Attribution against Foundation Geometry Baselines

Following the suggestion, we implemented the recommended modular baseline (AnySplat per state, global registration and ICP, RANSAC + ICP per moving part; see App. D.2 for more details) and added it to Table 4 (Sec. 5.3), next to ablations that keep the backbone and joint head and remove only our modules (copied below).

Table. Modular baseline and ablations on the same backbone.

MethodCD-s↓CD-m↓Ang$_m$↓Pos$_m$↓PSNR↑
AnySplat (per state) + registration0.1230.27739.396.52719.36
w/o CSA and state token0.0410.15631.462.91722.51
w/o CSA0.0290.08218.741.85224.18
ArtSplat (Ours)0.0240.0305.790.86426.30

With classical registration, the backbone alone does not recover the articulation reliably. With the backbone fixed, adding the state tokens and CSA substantially reduces the axis error. The backbone thus supplies the geometry, and the articulation accuracy comes from the cross-state modules. We do not claim that ArtSplat works without a pretrained backbone.

[W4] Geometric Ambiguity in Prismatic Joint Clustering

Yes, each prismatic Gaussian is clustered by (a^,d)(\hat{a}, d) together with its 3D position, so identical motion alone does not merge spatially separate parts. We apologize for omitting this detail from the paper; it is now clarified in Sec. 4.3. We tested the case mentioned by the reviewer by re-rendering the 37 test objects with same-axis parts so that these parts move identically (table below).

Table. Same-axis parts moving identically (37 objects).

Motion of same-axis partsARI↑PSNR with pseudo-labels↑PSNR with GT labels↑
Original states0.79526.0627.16
Identical motion0.80226.6726.81

When same-axis parts move identically in every observed state, the observations alone cannot distinguish them from a single rigid part, since they contain no motion difference between the parts. The remaining cue is geometry, which is why the clustering includes the 3D position of each Gaussian. With it, identical motion leaves the segmentation unchanged. Because its clustering combines the predicted motion with the 3D position of each Gaussian, ArtSplat separates same-axis parts even when they move identically, a case in which motion cues alone would merge them.


[W5] Train-Test Discrepancy in Part Assignment

With all network predictions fixed, replacing the HDBSCAN pseudo-labels with GT part labels marginally changes the axis error and PSNR only, and the results are stable across a wide range of HDBSCAN settings (Table IV in App. E.1, copied below).

Table. HDBSCAN settings and GT part labels at inference.

min_cluster_sizemin_samplesAng$_m$↓PSNR↑
1056.2526.24
50 (Default)255.7926.30
150506.4226.15
3001008.8525.40
GT Labels–5.2126.85

[W5] Synthetic-to-Real Gap

We added the two real PARIS objects, which have joint ground truth, and seven RealScan objects (Table III in App. D.3, copied below); qualitative results are in Fig. I.

Table. Real-world evaluation. LARM uses 3 views per state and GT poses, joint type and articulation values; all other methods use 4 views.

MethodPARIS (real) Ang$_m$↓PARIS (real) PSNR↑RealScan PSNR↑
ArtGS48.7417.0921.35
ScrewSplat11.7810.0319.90
LARM39.2018.3514.97
ArtSplat (Ours)15.3018.1219.24

With only uncalibrated RGB images and no real training data, ArtSplat recovers the joints of both PARIS objects, second to ScrewSplat, with rendering quality on par with the best method, and outperforms the feed-forward LARM on RealScan. We acknowledge, however, that the overall performance on real data is still below that on synthetic data, and that this evaluation is small. We attribute the gap mainly to training on synthetic data only. Closing it is what we need to improve next, and we state this in the limitations.


[Q4] Do averaged part-level parameters inject suboptimal gradients early in Stage 2?

Stage 2 starts after 80k iterations of Stage 1, in which the joint map is supervised directly, so the averaged part-level parameters are already accurate when the rendering loss is applied. Applying it from the first iteration increases the axis error substantially (Table 4).

We hope these results resolve the concerns. Please let us know if any point needs further clarification; we will reply as quickly as we can.

Dear Reviewer Wi9T,

As the discussion period closes in a few hours, we would like to briefly summarize our response to your review.

  • [W1] Comparison with the feed-forward methods LARM and Artic-O, and all metrics on AKB-48 (Table 2). ART could not be run, as its code and weights are not released.
  • [W2] Inference with S = 3 and 4 states without retraining (Table VIII).
  • [W3] The suggested modular baseline, AnySplat per state with registration (Table 4).
  • [W4] Prismatic clustering also uses the 3D position; tested on 37 objects whose same-axis parts move identically.
  • [W5] GT part labels in place of the clustering (Table IV), and real-world results on PARIS and RealScan (Table III).

If anything remains unclear, please feel free to ask, even briefly. We will reply right away. Thank you again for the detailed and constructive review.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

We thank all reviewers for their constructive feedback. All revisions are shown in blue in the revised PDF; below, each is tagged with the reviewers who raised it (Wi9T, wsYJ, tDcH, hxXP).

1. Real-world evaluation [Wi9T, tDcH, hxXP]

  • PARIS (real objects with joint GT) and RealScan: Sec. 5.2, App. D.3, Tab. III, Fig. I

2. New experiments

  • Modular baseline on the same backbone [Wi9T, tDcH]: Sec. 5.3, App. D.2, Tab. 4
  • Optimization baselines with their intended dense views [tDcH, hxXP]: App. D.1, Tab. II
  • Clustering stages and accuracy against GT parts [wsYJ, hxXP]: App. E.1, Figs. II–III, Tab. V
  • Articulation change and number of input views [tDcH]: App. E.2, Tabs. VI–VII
  • Inference with more than two states [Wi9T]: App. E.3, Tab. VIII
  • Use as simulation assets [wsYJ, tDcH]: App. F, Fig. IV

3. Completed metrics and revised claims

  • CD-w, CD-s, CD-m and Posₘ for all methods on AKB-48; geometry claim revised [Wi9T]: Sec. 5.2, Tab. 2

4. Additional results in the individual responses

  • Feed-forward methods given GT priors, and ArtSplat with known camera poses [Wi9T]
  • Same-axis parts moving identically [Wi9T, hxXP]
  • Particulate on GT and generated meshes, Particulate's geometry protocol, and recent methods [wsYJ]
  • Attention to moving parts and a same-state sanity check [wsYJ]

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.