acceptodds
Withdrawn from ICLR 2027

Geometric and Semantic Coupling for Interaction Understanding in 3D Scenes

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Interaction understanding in 3D scenes requires a joint description of movable parts, their motion, and the regions through which they can be operated. We present SEGMENT-SNAP, which connects these outputs through the physical relationship between parts and handles. Learned predictors identify broad part surfaces and small handles. A geometric decoder uses planar and upright priors to constrain motion, then selects hinge lines using predicted handle locations, without training a motion regressor. Conversely, a joint part-and-handle predictor supplies additional handle candidates, whose motion classes are refined using containing parts. Each information transfer is applied once, without iterative feedback. On Articulate3D validation, handle guidance raises motion-gated AP from 13.74% to 40.98% at fixed masks and axes. Additional handle candidates raise handle AP from 24.63% to 29.65%; part-based class correction adds 0.98 points, and full context reaches 30.99%. Repeated training, learned-decoder controls and paired visualizations establish the benefits and limitations of combining geometric and semantic evidence for interaction understanding.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D interaction understanding, part-handle coupling, part segmentation, motion estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xH7hOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Segment–Snap for recovering movable parts, their motion parameters, and handles from a static RGB point cloud. The method combines three independently trained predictors for parts, dense handles, and part-associated handle proposals. Detected handles guide a training-free geometric decoder to resolve hinge-side ambiguity, while part-associated proposals and contextual class correction improve handle prediction. These two information transfers are applied once, without iterative feedback. Experiments on the Articulate3D validation set report improvements in hinge localization and handle detection, with fixed-input ablations, repeated training, and learned-decoder comparisons examining the benefits and limitations of the coupling.

Critical strengths

The paper provides convincing evidence that detected handles are useful for resolving hinge-side ambiguity. In Table 4, replacing centroid origins with handle-guided origins improves motion AP from 13.74 to 40.98 while keeping part masks, classes, scores, and axes fixed. This is a meaningful result because it separates better hinge placement from better segmentation. It shows that a relatively simple geometric decoder can make effective use of information that is already available from scene perception.

The supplementary analysis is thorough and helps explain what the system actually improves. Repeated-training experiments distinguish the relatively consistent proposal gains from the smaller and more variable class-correction gains. The matched learned-decoder comparisons are also useful, and the authors appropriately acknowledge that the confidence intervals do not establish superiority over the strongest learned alternatives. These controls make the empirical findings more credible than an aggregate benchmark improvement alone.

Critical weaknesses

The main concern is the limited contribution beyond existing geometric reasoning and complementary prediction sources. As the paper acknowledges, selecting a hinge opposite a handle already appears in S2O; the distinction here is primarily the use of independently detected scene handles. This is a useful adaptation, but the reverse transfer does not yet provide strong evidence of a more substantial methodological advance. In Table S3, parent conditioning changes handle AP by only +0.34 before correction and −0.52 after correction, with both confidence intervals including zero. Table S2 also shows that dense-only label correction reaches 30.66 AP, close to the full correction’s 30.99. These results support the usefulness of additional proposals and label evidence, but do not clearly establish the added value of the proposed part-conditioned mechanism over a simpler combination of independent predictions. The authors acknowledge these limitations; nevertheless, they leave the central novelty argument unconvincing to me. The evaluation provides limited evidence that the reported advantages generalize beyond the development setting. Model and hyperparameter selection consulted the same 42-scene validation split used for reporting results. This is particularly relevant because the decoder relies on planar surfaces and fixed upright rotational axes. An independent evaluation would help determine whether the gains persist once the configuration is frozen. The external comparisons do not fully resolve this concern: Table 1 uses published results without matched training resources or a verified common origin-distance implementation, while REACT3D is not trained on Articulate3D and the proposed predictors are. These differences are disclosed, but they make the comparisons less informative about the contribution of the coupling itself. My concern is insufficient independent validation, not that using a validation set for model selection is inherently improper.

Questions for authors

Could you clarify the intended contribution of parent conditioning in light of Table S3? Is there evidence that it improves the final handle output over an unconditioned proposal source with matched capacity and training, rather than simply providing additional detections?

Could you provide an input-matched comparison with the S2O-style geometric rule discussed in the paper, holding part predictions, geometric fitting, and evaluation fixed? In particular, how much improvement comes from replacing a geometry-derived handle proxy with independently detected scene handles? This comparison would materially affect my assessment of the methodological contribution.

Which model choices and decoding thresholds were selected using the 42 validation scenes, and are any results available on an untouched scene split or test set with those choices frozen? I am asking for clarification of the existing selection protocol and any independent evidence, rather than suggesting a broader future-work agenda.

3 comments
AuthorsOpenReview ↗

Reply 1/3 — Matched S2O comparison, conditioning and class correction

We organize your review into six concerns and answer them in this order:

  • Concern 1 (Q2): S2O's existing hinge rule; input-matched comparison.
  • Concern 2 (Q1): Parent conditioning's AP intervals include zero; intended contribution and final output versus an unconditioned source.
  • Concern 3: Dense-only label correction is close to full correction.
  • Concern 4: Added value over a simpler combination of independent predictions.
  • Concern 5 (Q3): Generalization beyond the development setting, planar/upright priors, validation-selected choices and evidence with choices frozen.
  • Concern 6: Published results without matched training resources or a verified common origin-distance implementation; REACT3D's different training; contribution of coupling.

>>> Concern 1 (Q2) Contribution and matched S2O comparison

>>> Response Thank you. Our contribution is using independently detected scene handles to improve hinge selection. We credit S2O's opposite-handle rule (L162–163). S2O estimates handles from protrusions within part geometry; our scene detector can find handles outside the predicted part mask. Our additional matched comparison raises motion AP 30.72 → 40.98 (+10.27 pp) by replacing S2O's cue with ours on identical predicted parts.

We conducted the requested comparison with only the handle cue changed. On 42 validation scenes, we evaluate annotated and predicted parts separately, fixing masks, classes, scores, geometric fitting, axes, four hinge candidates, hinge selection and evaluation. Both cues act on rotating parts where our detector finds a handle within 0.5 m of the part support; others retain centroid origins. We use S2O's released estimator and constants, taking the room-facing side as front because object-body associations are unavailable.

Origin cue/ruleAnnotated parts: motion APPredicted parts: motion AP
Part centroid54.1613.74
S2O-style geometric estimate76.9430.72
Detected handles (ours)83.1540.98

Detected cues improve motion AP under the same hinge rule. Gains are 10.27 pp on predicted parts (paired 95% scene interval [6.09, 14.44]) and 6.20 pp on annotated parts ([2.51, 9.89]). Across three additional part-model runs, predicted-part gains are 8.43–11.96 pp. We will add this comparison to the paper.

>>> Concern 2 (Q1) Purpose and evidence for parent conditioning

>>> Response Thank you. The complete per-part design improves recovery over the unconditioned source; final-AP superiority is unresolved. We explain conditioning's intended contribution below.

1. Conditioning alone has no established AP benefit at fixed shared-map readout (Tab. S3a). This holds before and after class correction. The paper attributes the AP gain to complementary joint-network proposals (L321–322). We will state this distinction in the abstract.

2. Parent conditioning lets each part propose its own handle. Training uses part-associated handle targets. Proposals inherit their parent's score and class and survive only if their parent survives filtering. The unconditioned head uses a shared map trained on the union of all handles.

3. The complete design improves recovery; its final-AP advantage is not statistically resolved. Our additional comparison matches model capacity, training data, schedule and seed. Both sources use the same dense detections (24.63 AP), appending and class correction. Conditioning, readout and training targets change together:

Added proposal sourceNew handles before correctionFinal handle AP
Unconditioned joint head, shared-map readout2729.78
Conditioned joint head, per-part readout5330.99

Our design recovers more missed handles in 18 scenes and fewer in 2 (scene-level sign test, p = 0.0004). The final-AP difference has paired 95% scene interval [−1.10, +3.53].

>>> Concern 3 Full correction adds 0.33 pp beyond dense-only correction

>>> Response Thank you. Full correction adds 0.33 pp over dense-only label correction; our complete design also recovers more missed handles than the tested independent source. We separate these two benefits below.

1. Part labels improve correction on identical proposals. Tab. S2 fixes joint-proposal masks and scores: full correction reaches 30.99 handle AP, versus 30.66 with dense-only correction. Parts-only correction reaches 30.63, versus 29.65 without correction. The part and dense-handle sources provide overlapping label evidence.

2. The broader contribution includes recovery before class correction. Our additional comparisons fix dense detections. Before correction, our complete per-part design recovers 53 missed handles, versus 27 for the unconditioned shared-map variant and 8–11 for a second independently trained dense detector across three runs.

AuthorsOpenReview ↗

Reply 2/3 — Independent proposals, hidden-test results and validation selection

>>> Concern 4 Added value over a simpler combination of independent predictions

>>> Response Thank you. Our joint-network proposals provide added value over the tested simpler combination of independent predictions: they recover more missed handles and achieve higher final handle AP. We explain these two results below.

1. Before class correction, joint proposals recover 53 missed handles, versus 8–11 from a second dense detector. We conducted an additional comparison that fixes the reference dense detections (24.63 AP) and replaces joint proposals with each of three independently trained dense repeats from Tab. S5. Both sources use the same appending procedure, score scale, class correction and part predictions:

Appended sourceNew handles before correctionHandle AP before correctionHandle AP after correction
Second dense detector (three repeats)8–1125.34–25.7725.42–25.83
Joint-network proposals (reference)5329.6530.99

2. Joint proposals achieve 30.99 final handle AP, versus 25.42–25.83 under the same class correction. The advantage is already present before correction: joint proposals reach 29.65 AP, versus 25.34–25.77 for the independent dense sources. Thus, the tested independent combination does not reproduce the joint source's recovery or AP benefit. We will add this control to Tab. S2.

>>> Concern 5 (Q3) Generalization beyond the development setting and selection protocol

>>> Response Thank you. Our official hidden-test results support generalization beyond the development setting with the planar/upright priors. We report these results, then clarify which model choices and decoding thresholds were informed by the 42 validation scenes.

1. Our best challenge entries lead both official hidden-test leaderboards. Across multiple submissions, our best scores are 48.28 motion AP and 34.46 handle AP. Test labels remain hidden; the challenge server scores submissions:

Motion (hidden test)AP50AP50 (axis)AP50 (origin)Motion AP
Ours55.3852.3449.7648.28
pico-mr———41
coreMany46.6843.1543.2840.56
teamzb47.2044.9938.5537.11
core44.4441.0539.5536.89
ZZZ47.7244.8137.7735.72
linxii43.5241.3535.4733.60
coreN43.7040.6035.9433.38
astar———29
ZeroR39.5337.2530.1428.61
Uni Stuttgart———27
USDNet (official baseline)43.6637.9831.5426.34
wwwjh43.6637.9831.5426.34
ww———26
ShinNam!35.5233.7329.0424.67
ttt30.5828.3825.9924.00
Handles (hidden test)Handle AP
Ours34.46
coreMany32.86
core32.60
teamzb31.10
ZZZ30.69
coreN28.86
ZeroR25.71
moon24.80

We include current and previous editions, using USDNet's latest official baseline. Dashes mark unreported metrics; USDNet's test handle AP is unreported. We report these test leaderboards as permitted by ICLR 2027.

2. The system achieves strong axis- and origin-gated test performance with the planar/upright priors. The reported test entry achieves 52.34 axis-gated AP and 49.76 origin-gated AP, versus at most 44.99 and 43.28 for other reporting entries. These priors are geometric rules with no learned parameters; the axis-rule validation comparison is disclosed below.

3. Validation informed checkpoint selection, the axis-rule comparison and decoding settings.

ComponentSelection or setting
CheckpointsPart model: best validation motion AP, at 70% of training. Dense model: best validation segmentation score. Joint model: final weights.
Axis priorUpright rotation axis, retained after validation showed no statistically resolved advantage for an alternative axis rule.
Part decoder0.5 m handle gate, 5 cm cleanup radius, connectivity rescoring.
Dense handles2.5 cm grouping radius; 3-point minimum size to retain small handles.
Children and correction0.30 mask threshold, 90% containment, part confidence 0.3.
Child score scale0.05; tested values from 0.05 to 0.5 give identical AP.
Learned-decoder baselinesCheckpoints minimize loss on a held-out training fold; the reported best-of-three head seed is selected by validation AP, as disclosed in the supplement.
AuthorsOpenReview ↗

Reply 3/3 — USDNet and REACT3D comparisons

>>> Concern 6 Contribution of coupling and scope of external comparisons

>>> Response Thank you. Our fixed-input controls establish the contribution of coupling, with a common origin-distance implementation for the new motion comparisons. Below, we address training resources and evaluation, explain REACT3D's training conditions, and summarize the controlled results for both transfers.

1. The additional USDNet comparison fixes part predictions and uses a common origin-distance implementation. Tab. 1 reports published Articulate3D benchmark results. To measure handle guidance on controlled part inputs, we additionally retrained USDNet on the 195-scene Articulate3D training split for 200 epochs using its released recipe, selecting the checkpoint by validation part AP50. All new rows use the same 42 validation scenes and released line-distance evaluator. The retrained model reaches 31.87 part AP50 and 10.26 motion AP; the published 41.8 and 25.0 remain references. Every row uses the same retrained USDNet masks, classes and scores:

Decoder on the same retrained USDNet partsMotion AP
USDNet's motion head10.26
Our geometric decoder, centroid origins9.44
Our geometric decoder, our detected handles27.22

Detected-handle guidance adds 17.78 pp (paired 95% scene interval [12.55, 23.02]) between the two geometric-decoder rows, with masks, classes, scores and axes fixed. This isolates the handle-guidance gain from differences between trained part predictors.

The official hidden-test leaderboard also lists USDNet and our entry under the same reported metrics: 31.54 origin-gated AP and 26.34 motion AP for USDNet's official baseline, versus our 49.76 and 48.28.

2. REACT3D in Tab. 2 is an additional comparison that broadens our evaluation. For our Articulate3D benchmark comparisons, please refer to Tab. 1. REACT3D combines pretrained models without additional Articulate3D training; our predictors use the Articulate3D training split. We evaluate the released method as designed and score both systems on the same 42 validation scenes with the benchmark evaluator. Their inputs and training differ, as disclosed at L417–418.

3. Fixed-input controls show gains in both directions of coupling. For handles to motion, the additional USDNet experiment raises motion AP 9.44 → 27.22 with part predictions and decoder axes fixed. For parts to handles, Tab. S2 fixes dense detections and shows joint proposals raising handle AP 24.63 → 29.65 before class correction. These gains measure the two transfers with their base predictions held fixed.

Reviewer_QBRUOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work introduces SEGMENT-SNAP, which predicts movable parts, their motion (rotation/translation, axis, origin) and their handles from a static colored point cloud. Three independently trained networks, built on the Volt-B backbone [1], predict standalone parts with motion labels, dense per-point handles, and part-associated "child" handles. The networks are coupled only at inference by two rules. For motion, a decoder fits a rectangle to each part, finds a vertical rotation axis, and places the hinge on the edge farthest from the nearest handle (a heuristic proposed in S2O [2]). For handles, it appends the child proposals to the dense handles, which inherit the motion label of the part that contains them. Evaluation is conducted on the 42-scene Articulate3D [3] validation split, which was also used for checkpoint selection.

[1] Yilmaz et al., Volume Transformer: Revisiting Vanilla Transformers for 3D Scene Understanding, ECCV 2026.

[2] Iliash et al., S2O: Static to Openable Enhancement for Articulated 3D Objects, WACV 2026.

[3] Halacheva et al., Articulate3D: Holistic Understanding of 3D Scenes as Universal Scene Description (USDNet), ICCV 2025.

Critical strengths

S1. A simple geometric prior (vertical axis, hinge opposite the handle) beats USDNet's [3] learned motion head on this benchmark at low cost (Tab. 3).

S2. The ablations are controlled. Tab. S7 keeps masks, classes, scores and axes fixed and changes only the origin rule, and Tab. S2 shows the handle-prediction gain vanishes with random or permuted children.

Critical weaknesses

W1. Evaluation on the model-selection split. All results, main and supplementary, are on the validation set (L656). This split was also used to select the checkpoints during training, which is a bad practice. Additionally, REACT3D [4] in Tab. 2 is not trained on Articulate3D (L1194-L1195), which further hinders the fairness of evaluation.

W2. Limited novelty. The method is a set of hand-designed inference rules on top of standard detectors (L143-L144), and the hinge rule has already been introduced in previous work S2O (L162-L163). What is new is using detected handles as the cue and the append-and-relabel rule. These rules only refine already detected parts, yet missed parts are the main source of error (L481-L483). Handles are never used to find parts...

W3. Headline gains are measured against weak baslines.

  • For motion prediction, the baseline puts the hinge at the center of the part (L303). For doors, the center is always far from the hinge, so any rule that moves the hinge to an edge can get the advantage. The real question is how much, and whether, the handle helps in picking the correct edge. Tab. S9 shows that this is gain is much smaller.
  • For handles, the method appends extra candidates from the second detection network, to the end of the list (L318-L319). Candidates added to the bottom of the list cannot hurt the AP metric, so a gain is expected. The paper claims the gain comes from the part conditioning, but a version of the method without it gives the same performance (Tab. S3a).

[4] Huang et al., REACT3D: Recovering Articulations for Interactive Physical 3D Scenes, IEEE RA-L 2026.

Questions for authors
  1. (W1) Can you report the test-server result for SEGMENT-SNAP? The Articulate3D leaderboard already lists USDNet on the hidden test split.
  2. (W1) Articulate3D is annotated on ScanNet++ training scenes, and Volt-B is trained with supervision on the official ScanNet++ training split. Did the Volt-B initialization (L206-L207) see the 42 Articulate3D validation scenes?
  3. (W3) What is the motion AP in Tab. S7 when the origin is placed on a randomly chosen edge of the fitted rectangle?
  4. (W3) What handle AP do you get when a second dense handle detector (Repeat 3 in Tab. S5) is appended instead of the joint network's children?
  5. (W3) The reported handle AP is the best of four dense-model runs (Tab. S5); what is the mean?
Reviewer_XhqDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the joint prediction of movable parts, their motion parameters, and handles from a static RGB point cloud. Three networks, predicting parts, dense handles, and joint part-handle pairs, are trained independently and coupled at inference in two directions. From handles to parts, a training-free geometric decoder sets the origin by selecting the edge opposite the detected handle as the hinge. From parts to handles, additional handle proposals from the joint model are appended to the existing detections, and their labels are corrected using the class of the part that contains them. On the Articulate3D validation set, the method substantially improves motion metrics over prior work while achieving comparable handle performance, and the paper provides a range of controlled experiments and analyses.

Critical strengths
  • Bidirectional coupling. The method exploits related but independently predicted part and handle information in a complementary way. Handles inform part motion estimation, while parts aid handle detection and label correction.
  • Training-free motion decoder. Hinge-side ambiguity is resolved using detected handles, without a dedicated motion regressor. This substantially improves motion prediction at low cost.
  • Disentangled analysis of effects. Fixed-input interventions and controlled experiments measure the contribution of each direction separately. The analysis also shows that missing part masks are the main remaining bottleneck.
Critical weaknesses

[1] The paper’s incremental contribution to its main gains is not clearly established

  • The core handle-to-part rule is not novel. S2O already selects the hinge edge opposite a geometry-derived handle proxy. The distinguishing element here is the use of independently detected scene handles, but the two approaches are not compared under identical part predictions and hinge candidates.
  • The only reported baseline is weak. Under the origin-distance threshold, a centroid origin fails by construction for parts of typical door width. The gain over this baseline therefore mixes the effect of placing the origin on an edge with that of selecting the correct edge. Learned edge selection without any handle information already far exceeds the centroid baseline. Since this is the paper’s largest gain, it does not show the benefit of the proposed approach over the closest prior method.
  • The gain in the parts-to-handles direction is weakly supported. Child proposals are always scored below every dense detection, so they are appended only to the tail of the ranking, which cannot lower AP. The gain thus reflects added recall, and the precision of the appended proposals is not reported. Parent conditioning alone shows no significant effect, and the final handle AP does not exceed USDNet.

[2] Improvements over prior methods are not fully validated under controlled conditions

  • Validation-based selection. The authors acknowledge that validation-informed model selection limits generalization claims. Still, the key results in the abstract and the main comparisons rest on the same validation data used to select checkpoints and hyperparameters. Since the reference configuration also lies at the top of the range across repeated runs, the reported results may be somewhat optimistic.
  • The comparison with USDNet is not controlled. USDNet’s results are quoted from its paper rather than re-evaluated with the same evaluator. Since part segmentation performance already differs, it is difficult to separate the contribution of the perception stage (backbone, pretraining, training setup) from that of the coupling. In particular, without handle guidance the method’s motion performance is below USDNet, so a comparison that isolates these two factors would strengthen the claims.
  • The REACT3D comparison provides useful context but does not isolate the coupling. REACT3D is not trained on Articulate3D and differs in both inputs and objective. As it is not a baseline under matched conditions, the performance gap does not by itself demonstrate the effect of the coupling.

Minor weaknesses:

  • Inference cost. Tab. 3 excludes the joint predictor. The cost of the full system, including handle outputs, should also be reported.
  • Class-correction gain. The reported gain comes from the reference run and varies considerably across runs. Reporting mean ± std would be more representative.
Questions for authors
  1. Under identical part predictions and hinge candidates, how does hinge selection with detected handles compare against (i) an S2O-style geometry-derived handle proxy, (ii) a no-handle edge choice (e.g., a random side), and (iii) GT handles?
  2. How do the main results change if checkpoints and thresholds are selected on a held-out fold of training scenes, and the validation set is used only once for final evaluation? Results from the official test evaluation, if feasible, would also be helpful.
  3. Could you retrain USDNet on the training split, evaluate it with the same evaluator, and apply your decoder to its outputs? This would separate the effect of the perception stage from that of the coupling.
  4. How many child proposals are appended per scene, and what is their precision?
  5. The appended child handles are not fed back into hinge selection. If the handle set including children were passed through the motion decoder again, would rotating parts that currently fall back to centroid origins improve, or would low-precision children degrade hinge selection? The answer would help clarify whether the two directions are truly complementary.
4 comments
AuthorsOpenReview ↗

Reply 1/4: Hinge cues, motion attribution and handle-proposal evidence

>>> W1a Hinge-rule novelty and matched evidence

>>> Response Thank you.

1. Our contribution is independently detected handle evidence that improves hinge selection over S2O's geometric proxy. We credit S2O's opposite-handle principle (L162–163). S2O estimates a handle from protrusions within the part geometry; our separate detector searches the scene and can find handles outside the predicted part mask. In the additional matched experiment in Q1, replacing S2O's cue with ours raises motion AP on predicted parts 30.72 → 40.98 (+10.27 pp).

2. The additional comparison uses identical part predictions and hinge candidates. On 42 validation scenes, we fix predicted parts, geometric fitting, axes, hinge candidates and evaluator. Both cues act on parts covered by our detector, with centroid fallback elsewhere. Please refer to Q1 for the detailed experimental setup and full results.

>>> W1b Centroid baseline and attribution

>>> Response Thank you.

1. The centroid is our pipeline's starting point, not a competing-method baseline. Tab. 4 measures the gain from refining centroid origins; Tab. 1 compares prior methods. For an ideal rectangular door with an edge hinge and correct axis, the centroid fails the 0.25 m gate when its width exceeds 0.5 m. Points 2–4 establish the benefit of handles against stronger alternatives.

2. We separate random edge placement (+8.37 pp) from handle-guided edge selection (+18.88 pp). The table reports our additional random-edge experiment (Q1) with the same detector-covered predicted parts and candidates. Please refer to Q1 for the detailed setup.

ChangeWhat changesMotion AP (gain)
Centroid → random edgeMove to an edge without handle information13.74 → 22.11 (+8.37 pp)
Random edge → handle-selected edgeBoth already use an edge; only the choice changes22.11 → 40.98 (+18.88 pp)

The +18.88 pp gain (paired 95% scene interval [15.05, 22.71]) measures the benefit of using handles to select among the same candidates.

3. Our handle guidance further improves stronger learned edge baselines. Tab. S9's fixed-axis learned-edge heads exceed the centroid (25.36–28.75 vs. 13.74 AP). Our handle cue adds 10.05–12.53 pp across matched seeds, demonstrating its effectiveness beyond the centroid comparison.

4. The matched S2O comparison shows our cue's benefit over the closest prior approach. Our additional Q1 experiment fixes predicted parts, hinge candidates and decoder: 30.72 motion AP with S2O's geometric cue versus 40.98 with ours (+10.27 pp). Please refer to Q1 for the matched setup and full results.

>>> W1c Evidence for the parts-to-handles gain

>>> Response Thank you. Our joint proposals recover missed handles and improve AP across runs. We present the controls, precision results, conditioning analysis and USDNet comparison below.

1. Tail appending cannot lower AP, but proposal content determines the improvement. Tab. S2 keeps dense detections unchanged and compares real children with count-, size- and score-matched random/shuffled proposals:

Handle outputHandle AP
Dense detections24.63
+ random or spatially shuffled children24.63
+ joint-network children29.65
+ contextual class correction30.99

2. Recovering missed handles adds 5.01 pp AP, and the gain persists across runs. Before class correction, joint proposals recover 53 additional handles, increasing coverage 140/388 → 193/388 (Tab. S2). The +5.01 pp gain has paired 95% scene interval [1.47, 8.56] (Tab. S1). It is positive in all three joint-model runs, averaging +4.62 ± 0.59 pp; please refer to Minor 2 for the run summaries.

3. Appended proposals recover 53 missed handles, with 1.3% incremental precision. Our additional analysis counts 53 new true positives among 4,129 proposals before class correction, after matching the unchanged dense detections. Duplicates count as false positives. Please refer to Q4 for the full counts and precision analysis.

4. The AP benefit comes from complementary proposals, not conditioning alone. Tab. S3a finds no significant AP improvement from conditioning at matched readout. Appending joint proposals to fixed dense detections raises AP 24.63 → 29.65 (Tab. S2), a separate effect (L321–322). Before correction, the conditioned per-part design recovers 53 missed handles versus 27 for the unconditioned shared-map variant. This comparison changes conditioning, readout and training targets together. We will state this distinction in the abstract.

5. Our best hidden-test entry achieves the highest reported handle AP: 34.46 versus the next entry's 32.86. On validation, our 30.99 is comparable to USDNet's published 31.1. USDNet's test handle AP is unreported. Please refer to W2a + Q2 for both full official test leaderboards.

AuthorsOpenReview ↗

Reply 2/4: Validation selection, hidden-test results and USDNet controls

>>> W2a + Q2 Generalization and robustness

>>> Response Thank you. We address generalization with official hidden-test results and robustness with repeated runs and sensitivity checks. For Q2, we report this evidence instead of training-fold reselection.

1. Our best entries lead both official hidden-test leaderboards. They reach 48.28 motion AP and 34.46 handle AP, providing evidence beyond the validation split used for selection. These are our best challenge entries across multiple submissions. Test labels remain hidden; only the challenge server scores predictions.

Motion (hidden test)AP50AP50 (axis)AP50 (origin)Motion AP
Ours55.3852.3449.7648.28
pico-mr---41
coreMany46.6843.1543.2840.56
teamzb47.2044.9938.5537.11
core44.4441.0539.5536.89
ZZZ47.7244.8137.7735.72
linxii43.5241.3535.4733.60
coreN43.7040.6035.9433.38
astar---29
ZeroR39.5337.2530.1428.61
Uni Stuttgart---27
USDNet (official baseline)43.6637.9831.5426.34
wwwjh43.6637.9831.5426.34
ww---26
ShinNam!35.5233.7329.0424.67
ttt30.5828.3825.9924.00
Handles (hidden test)Handle AP
Ours34.46
coreMany32.86
core32.60
teamzb31.10
ZZZ30.69
coreN28.86
ZeroR25.71
moon24.80

We include current and previous editions, using USDNet's latest official baseline. Dashes mark unreported prior-edition metrics; USDNet's test handle AP is unreported.

Leaderboard note: We report these test leaderboards as permitted by ICLR 2027.

2. Performance remains strong across training runs and tested selection choices. Across four part-model runs, mean motion AP is 40.36 ± 2.31, with the reference (40.98) ranking second. Across four dense-model runs, mean handle AP is 29.88 ± 1.19, with the reference (30.99) ranking first. Even the weakest part-model run exceeds the compared published baselines: 46.24 / 37.78 part AP50 / motion AP, versus 41.8 / 25.0.

The tested checkpoint and threshold changes have limited effects. In an additional four-run check, replacing validation-selected part checkpoints with final checkpoints lowers motion AP 0.66–2.09 pp, with other choices fixed. Tested changes to handle-association distance and class-correction confidence change AP by at most 0.3 pp.

>>> W2b + Q3 USDNet, evaluator and perception controls

>>> Response Thank you for proposing this control. We agree that published scores alone do not isolate decoding. We retrained USDNet on 195 training scenes, evaluated all 42 validation scenes with the same evaluator, and applied our decoder with its masks, classes and scores fixed.

We evaluate the new comparisons with the same benchmark code. New rows use the benchmark’s released line-distance evaluator on the same 42 validation scenes; USDNet’s published row is for reference. The official hidden-test leaderboard now lists USDNet’s baseline with the same metrics: 26.34 motion AP versus our 48.28 (W2a + Q2).

We retrain USDNet using only the training split. Its released test checkpoint used training and validation scenes, so we retrain on training only. We follow its released recipe for 200 epochs, matching the checkpoint duration, and select by validation part AP50.

Part predictionsMotion decoderPart AP50Motion AP
USDNet publishedUSDNet head41.825.0
USDNet retrainedUSDNet head31.8710.26
USDNet retrainedOurs, centroid origin31.879.44
USDNet retrainedOurs, detected-handle cue31.8727.22
OursOurs, centroid origin47.9313.74
OursOurs, detected-handle cue47.9340.98

Detected handles improve motion AP with both part predictors. They raise motion AP 9.44 to 27.22 on retrained USDNet’s parts and 13.74 to 40.98 on ours. Within each comparison, masks, classes, scores and decoder axes are fixed, so the benefit transfers to USDNet predictions rather than depending on our part predictor. On fixed USDNet parts, the gain is +17.78 pp (paired 95% scene interval [12.55, 23.02]).

We agree that without handles our decoder scores below USDNet’s head; with handles it exceeds that head on the same parts. On retrained USDNet parts, our centroid decoder scores 9.44 versus 10.26 for USDNet’s head; handle guidance raises ours to 27.22, exceeding its head by 16.96 pp (paired 95% scene interval [8.08, 25.85]).

AuthorsOpenReview ↗

Reply 3/4: REACT3D comparison, full-system cost and run variation

>>> W2c Scope of REACT3D

>>> Response Thank you.

1. REACT3D is an additional comparison that broadens our evaluation. For our Articulate3D benchmark comparisons, please refer to Tab. 1. Tab. 2 evaluates shared articulated-part outputs; the fixed-input controls below measure the coupling benefit.

2. REACT3D requires no additional Articulate3D training by design. It combines pretrained models in a zero-shot pipeline, and we evaluate the released method in its intended setting. Its authors also evaluate it using Articulate3D annotations on 30 selected ScanNet++ scenes, with their own detection and articulation metrics.

3. We score both systems on the same 42 scenes and shared articulated-part outputs. We use all official validation scenes and the benchmark evaluator for masks, motion types, axes and origins. REACT3D receives RGB-D keyframes, camera poses and a mesh; ours receives an RGB point cloud. The paper explicitly states that inputs and training conditions differ (L417–418).

4. Fixed-input controls show benefits in both directions of coupling. With masks, classes, scores and axes fixed, detected handles raise motion AP 13.74 → 40.98 (Tab. 4). With dense detections fixed, joint proposals raise handle AP 24.63 → 29.65; contextual class correction reaches 30.99 (Tab. S2). Please refer to Q1 for the additional matched S2O experiment, W2b + Q3 for decoder transfer on fixed USDNet parts, and W1c for proposal controls and handle recovery.

>>> Minor 1 Full-system inference cost

>>> Response Thank you. Including the joint predictor and handle post-processing, the full system takes 54.43 s/scene (median). We conducted the requested additional timing experiment with all three predictors on all 42 validation scenes under the paper's timing protocol. The run reproduces 40.98 motion AP and 30.99 handle AP.

Full-system measurementValue
Median time54.43 s/scene
Peak host memory5.21 GiB
Logged GPU peak10,344 MiB
Predictors / parameters3 / 0.308 B

We will add the full-system measurements to Tab. 3.

>>> Minor 2 Variation across training runs

>>> Response Thank you. Class correction improves handle AP in every tested run, with greater variation across joint-model runs. Using the existing runs in Tabs. S1 and S5, we compute the requested mean ± sample std below. Each series includes the reference run and varies the indicated predictor while keeping the others fixed.

Handle AP gain (pp)ReferenceJoint runs (n=3)Dense runs (n=4)
Class correction+1.34+0.64 ± 0.62+1.21 ± 0.10
Appending joint proposals+5.01+4.62 ± 0.59+4.58 ± 0.48

We will add these mean ± std summaries to Tab. 4.

AuthorsOpenReview ↗

Reply 4/4: Matched handle cues, proposal precision and feedback

>>> Q1 Matched comparison of handle cues

>>> Response Thank you. Under matched part inputs and hinge candidates, our detected handles outperform S2O’s geometric proxy and random hinge selection. We conducted the requested additional experiment, including ground-truth (GT) handles, as follows.

We hold part inputs and hinge candidates fixed. On 42 validation scenes, we evaluate annotated and predicted parts separately, fixing masks, classes, scores, geometric fitting, axes, four hinge candidates and evaluator. Random selection, S2O and our detections act on the same rotating parts where our detector supplies a handle within 0.5 m of the part’s support; others retain centroid origins. We use S2O’s released estimator and constants, taking the room-facing side as front because object-body associations are unavailable.

The second GT control also supplies cues to parts our detector missed. “Same covered parts” restricts GT cues to parts where our detector supplies a cue. “Wherever available” also includes parts our detector missed. Both accept a GT handle centroid only within 0.5 m of the part’s support; otherwise, the part retains its centroid origin.

Origin cue/ruleAnnotated parts: motion APPredicted parts: motion AP
Part centroid54.1613.74
Random candidate (50 draws, mean ± std)62.92 ± 1.2822.11 ± 1.34
S2O-style geometric proxy76.9430.72
Detected handles (ours)83.1540.98
GT handles (same covered parts)82.0339.68
GT handles (wherever available)86.8641.29

Our detected handles outperform S2O’s proxy and random selection. Over S2O’s proxy, gains are 6.20 pp on annotated parts and 10.27 pp on predicted parts (paired 95% scene intervals [2.51, 9.89] and [6.09, 14.44], respectively). Over random selection, the gains are 20.23 pp and 18.88 pp, respectively.

GT handles perform best when their additional coverage is included. GT provides handle locations; nearest-handle association and hinge selection remain automatic, without GT part–handle assignments or hinge choices. GT locations therefore do not guarantee the best fitted hinge candidate. On the same covered parts, GT scores slightly below our detections; including GT handles on detector-missed parts raises AP above ours in both settings.

>>> Q4 Number and precision of appended proposals

>>> Response Thank you. We append 98 proposals per scene on average, with 1.3% incremental precision before class correction. Our additional analysis covers all 4,129 appended proposals across the 42 validation scenes, with 25–161 proposals per scene.

Appended proposalsCountFraction of all appended proposals
All4,129100%
Same-class GT overlap at IoU > 0.593622.7%
New true positives after matching dense detections531.3%

After matching dense detections, 53 appended proposals count as new true positives. The 936 overlapping proposals cover 154 unique GT handles: 101 already found by the dense detector and 53 new. Duplicates count as false positives, giving an incremental precision of 53/4,129 = 1.3%. The 22.7% overlap rate includes duplicates.

>>> Q5 Feeding child handles back

>>> Response Thank you. Feeding appended child handles back improves motion AP from 40.98 to 41.65. We conducted this additional experiment on all 42 validation scenes, passing the combined dense and child handles through the motion decoder again while keeping part masks, classes and scores fixed. We also ran a third pass; its results are identical because handle construction does not use decoded motion.

MeasurementSingle pass (paper)Second pass: dense + child handlesThird pass
Motion AP40.9841.6541.65
Predicted parts with a handle within 0.5 m3,883 / 6,2854,914 / 6,2854,914 / 6,285
Rotating predictions passing all motion gates869899899

1. Child handles improve predictions that previously fell back to centroid origins. Among 512 rotating predictions newly receiving a handle within 0.5 m, those passing all motion gates increase from 23 to 49, accounting for 26 of the net increase of 30.

2. Adding child handles improves motion performance overall despite their low precision. Among the 2,220 rotating predictions that already had a handle, those passing all motion gates also increase in aggregate, 843 → 847. Motion AP improves in all four tested part-model runs (+0.49 to +0.75 pp).

3. The two directions are complementary. The joint part–handle network supplies additional handles, and feeding them back improves part-motion prediction, chiefly on parts that previously lacked handle cues. We will add this feedback experiment to the supplement.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to Reviewer Feedback and Revised Manuscript

Dear Reviewers and Area Chairs,

Thank you for your time and constructive feedback. We have responded to each concern and question point by point, clarified ambiguities, and provided additional experiments and quantitative analyses where relevant.

We have also revised the main paper and supplementary material accordingly, with changes highlighted in blue. We welcome further questions or feedback and would be happy to continue the discussion.

Sincerely,

All Authors of Submission 5543

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.