- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies how to encode object geometry for frozen-policy transfer of a unified humanoid controller to unseen shapes. Its proposed interface, SIGMA, combines Structured Surface Attributes (SSA) with Interaction-Conditioned Geometry Compression (ICGC). SSA samples an object mesh and selects local representatives using a joint position-normal distance; each representative stores a position, an aggregated normal, and an estimated surface-area fraction. The implementation uses 26 valid surface elements selected from 2,048 dense samples. A shared encoder processes these descriptors in a humanoid-relative coordinate frame. ICGC pools them into a 64-dimensional token using a learned task query, distances to task-associated body keypoints, and an area prior. The interface is integrated into a TokenHSI-based Transformer policy trained jointly with PPO and AMP on Trajectory Tracking, Sit, Carry, and Climb.
The main evaluation measures object-macro Carry success, requiring pickup, transport, and stable goal completion, on Alias, Notch, and Furniture suites without target-object adaptation. At 50k iterations, SIGMA reports 74.24%, 72.89%, and 27.85%, respectively, compared with 72.53%, 67.82%, and 22.12% for independently trained SSA + Area Pooling. The equal average across the three suites is 58.33% for SIGMA, versus 22.67% for the implemented PointNet and 6.76% for AABB-shape-token; the corresponding SSA + Area Pooling average is 54.16%, calculated from Table 1. The paper also reports frozen-policy interventions, object-wise and initial-state robustness analyses, and a held-out Sit evaluation with 89.03% average target-entry success. The Sit metric measures whether the root enters the target region at any step, and is distinct from sustained supported sitting.
Critical strengths
-
A useful problem formulation with a well-targeted benchmark and substantial observed representation gains. Alias holds global extent, mass, and inertia fixed while varying local interaction geometry, providing a controlled test of geometry information that a bounding-box token cannot distinguish. The results extend to eight Notch objects and 49 Furniture objects. SSA + Area Pooling improves over the implemented PointNet by 46.07, 34.99, and 13.39 percentage points on these three suites, respectively, under the reported common policy framework and training budget. These are consequential empirical gains, particularly because Carry success requires a complete pickup-transport-placement sequence. Frozen transfer without target-object adaptation is also a useful contribution alongside the shape-adaptation setting in TokenHSI (Pan et al., 2025), where the Carry tokenizer is fine-tuned for new object categories (TokenHSI, Sec. 3.4). The evidence makes compact geometry interfaces a worthwhile subject of study even though the source of the gains needs further isolation.
-
A coherent compact interface with meaningful controls and robustness checks. Orientation-aware anchor selection and explicit position/normal/area descriptors have a clear geometric motivation, and the area-biased, task/body-conditioned pooling rule is simple and precisely specified. The independently trained SSA + Area Pooling arm is a meaningful control for ICGC as a whole. The positive Furniture completion gap persists under a second initial-state sample (+3.90 points), under the additional stable-termination criterion (+5.47 and +3.14 points for the two samples), and after excluding the source model shared with Climb training. The two available continued seeds also favor SIGMA at the evaluated 55k-70k checkpoints, although those curves use episode-micro rather than object-macro scores. These checks strengthen the case that the observed improvement extends beyond one aggregate evaluation. The reported 10.8% additional geometry-tokenization latency is modest relative to static pooling in the measured setup, with the timing scope explicitly limited to the encoder.
Critical weaknesses
-
The specific contribution of the proposed geometry design is insufficiently isolated from richer inputs and alternative pooling. The PointNet comparator uses 64 area-uniform xyz-only samples with max pooling. SSA simultaneously introduces preprocessing from 2,048 samples, position-normal anchor selection, explicit normals and area fractions, centroid-based coordinate organization, and different pooling. This establishes a useful complete-interface comparison, but leaves open whether a comparably compact point encoder with normals and matched preprocessing would recover much of the gain. Seen-object Carry results would also help distinguish optimization from transfer: PointNet seed 1 achieves only 0.13%, 0.00%, and 0.07% on the three held-out suites, but its seen-object Carry performance and continuation curves are not reported.
The independently trained area-pooling control isolates ICGC's combined addition, but its mean benefits are smaller: +1.71, +5.08, and +5.73 points using the authors' unrounded rates, with a negative difference for one of the three training seeds in every suite. The favorable robustness checks deserve credit, while the magnitude and consistency of this incremental effect remain less certain than the headline comparisons. No Task and No Body are frozen-policy interventions, so they establish reliance on those signals rather than the benefit of each component when trained independently. This matters for positioning against CORN (Cho et al., 2024), which already uses hand-conditioned point-cloud representations and task-state cross-attention. A compact learned conditioning baseline and matched-feature controls would clarify the additional value of SIGMA's particular surface construction and explicit distance/area biases. This uncertainty is the main reason for my recommendation.
-
The evidence beyond Carry does not establish stable supported interaction or an ICGC-specific cross-task benefit. The held-out Sit metric is satisfied when the humanoid root comes within 0.20 m of the target at any simulation step; it does not require sustained seated support, a stable posture, or absence of a subsequent fall . The 89.03% result therefore supports Sit target entry, while the conclusion's claim about whole-body supporting balance is stronger than this measurement establishes. Moreover, Table 5 omits SSA + Area Pooling. The authors correctly acknowledge that this is an interface-level comparison, but it leaves the value of ICGC on the second interaction task unresolved. Near-saturated performance on the other jointly trained tasks in Sec. 4.6 establishes compatibility with that evaluation, not the missing held-out stable-Sit result. Because interaction-dependent geometry across tasks is central to the paper's framing, a sustained-support Sit evaluation with the static-pooling control could materially strengthen the acceptance case.
Questions for authors
-
Additional evidence for the representation claim: How does a compact point encoder perform with explicit normals and coordinate preprocessing matched to SSA, using the same mesh source and a comparable representation/compute budget? A controlled comparison using the same samples or anchors would help separate descriptor content, anchor selection, and pooling. Please also report seen-object Carry performance for the existing PointNet seeds, especially seed 1, so that poor transfer can be distinguished from incomplete acquisition of Carry within the training budget. These results would directly address Weakness 1.
-
Additional evidence for ICGC's specific contribution: What happens when No Task and No Body variants are trained from scratch, or when ordinary learned task/body-conditioned pooling is used on the same SSA descriptors without the explicit distance/area biases? Please quantify uncertainty in SIGMA-minus-area-pooling performance across independent training runs and use a common aggregation metric when comparing later checkpoints. I recognize the existing independent area-pooling arm and positive continuation results; the unresolved issue is which conditioning components are responsible for the added benefit.
-
Additional evidence for the cross-task claim: Do the held-out Sit gains persist under a criterion requiring sustained supported sitting and no fall, and how does SSA + Area Pooling perform under that same criterion? Please specify the dwell, support/contact, and termination conditions used. This would determine whether the current target-entry results support the stronger stable-support interpretation and whether ICGC contributes beyond Carry.
-
Clarification of the existing experimental protocol: Please identify the exact observation/reward and benchmark configuration files in the described archive, or provide their specification. In particular, which task observations or rewards contain geometry-derived quantities, what surface-anchor set is used in the pickup evaluator, and is that evaluator common across methods? Please also provide the Alias/Notch construction parameters and the asset IDs, counts, and split definitions for the held-out Sit groups. The PDF provides substantial encoder and PPO detail, but these remaining specifications affect independent reproduction and interpretation of the controlled geometry comparison. This is a request to resolve missing information, not an allegation of unequal inputs or data leakage.
3 comments
Thank you very much for your thorough review and helpful suggestions. With the discussion period closing soon, we wanted to kindly check whether our additional experiments and revisions (highlighted in red) have resolved your concerns. Please let us know if anything further needs clarification in the remaining time. Thank you again for your time and effort!
Thanks for the additional experiments to clarify my concerns. I would like to maintain my score.
We address the four questions through expanded experimental specifications and additional analyses in the revised manuscript (changes highlighted in red).
Q1. Matched PointNet and Seen-Object Performance
To test whether PointNet's lower performance stems from raw point inputs or symmetric max pooling, we use Matched PointNet, which feeds the exact same SSA descriptors and preprocessing into a PointNet-style max-pooling aggregator.
Held-out Carry full-task success (%):
Giving PointNet structured SSA inputs substantially improves its performance over raw point clouds, but SIGMA still outperforms Matched PointNet by +14.32, +10.68, and +6.66 pp across the three benchmarks, isolating the gain from interaction-conditioned aggregation (ICGC).
To separate optimization failure during training from cross-shape transfer failure, we also evaluate the original PointNet policies on training-seen Carry objects across Seeds 0–2, yielding 22.98%, 0.07%, and 58.40% full-task success, respectively. This shows that Seed 1 fails to acquire the task during training itself, whereas Seeds 0 and 2 learn functional policies on seen objects (up to 58.40%) that subsequently degrade on held-out geometries.
Q2. Contribution of ICGC
We assess ICGC's contribution from two angles: comparing against unconstrained learned attention, and examining paired multi-seed deltas over static area pooling.
First, we use Learned Conditioning on the same SSA features—generating attention queries from task and body states, but omitting SIGMA's explicit body-to-surface distance bias and area prior. As shown in the table above, it reaches only 45.77% (Alias), 27.28% (Notch), and 4.51% (Furniture). Diagnostics across 5 checkpoints and 448,327 valid states confirm that its attention weights become increasingly selective without collapse or numerical instability; however, without explicit spatial and area priors, learned attention fails to ground contact surfaces that generalize to unseen geometries.
Second, we report the paired 3-seed improvements of SIGMA over independently trained SSA + Area Pooling:
In addition, across both available continuation seeds from 55k to 70k iterations, SIGMA remains consistently ahead of SSA + Area Pooling at every evaluated checkpoint under the same episode-micro metric.
Q3. Cross-Task Generalization on Held-Out Sit
We apply a stricter held-out Sit evaluation by requiring the downward root ray to intersect the annotated seat surface within 0.10 m for 30 consecutive control steps, and evaluate SSA + Area Pooling under the same 3-seed protocol. (We note in the text that this measures sustained kinematic seat proximity rather than contact forces or dynamic balance.)
Held-out Sit success (%), mean ± SD over 3 seeds:
SIGMA leads across all three categories, improving average sustained sitting success by +4.04 pp over SSA + Area Pooling and +10.06 pp over PointNet, with the clearest margins on Stool (+4.16 pp) and Coffee Table (+6.59 pp).
Q4. Experimental Protocol and Reproducibility
Appendices B and G.1 now provide expanded documentation of the exact training and evaluation protocols in one place: