- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies unified 3D semantic-part and affordance grounding from a single RGB image. The method first reconstructs a complete 3D object with SAM3D, extracts image-derived and geometry-derived features using SAM3D and Utonia, and aligns them through an optimal-transport-based fusion module. It further adapts text features to the reconstructed object through cross-attention and predicts query-conditioned 3D masks using point-text similarity. The paper also constructs a joint semantic-affordance dataset by cross-annotating PartNet and 3DAffordanceNet. Experiments on synthetic and real-world datasets report strong improvements over existing semantic and affordance segmentation baselines.
Critical strengths
- The problem formulation is useful and practically motivated. Predicting both semantic parts and affordances over a complete reconstructed object from a single partial observation is a meaningful setting, and the unified formulation is more useful than treating the two tasks independently.
- The empirical results are generally strong. The method achieves substantial gains on PartNet and 3DAffordanceNet and also transfers reasonably to real DREDS images. The constructed joint semantic-affordance supervision is also useful, and Table 5 suggests that joint training, especially, provides a sizable benefit for affordance prediction.
Critical weaknesses
- The contribution of the proposed OT fusion is not sufficiently supported by the current ablations. In Table 4, OT fusion improves semantic segmentation over additive fusion, but decreases affordance performance from 36.73 to 35.02 on A. and from 33.65 to 31.81 on AO. There is also no full-model ablation without OT or comparison with stronger fusion alternatives. It is therefore unclear how much OT itself contributes to the final performance.
- Some generalization claims are stronger than what the experiments currently demonstrate. SO./AO. are described as out-of-distribution queries, but the training-time text augmentation already contains very similar object-conditioned and compositional templates. The DREDS unseen-category results are also mixed, while only the overall average is emphasized. I would like to see genuinely unseen language evaluations and separate seen/unseen category results.
- The relation to closely related 2D-to-3D affordance learning work is unclear. Unlocking 3D Affordance Segmentation with 2D Semantic Knowledge (Huang et al., CVPR 2026) appears closely related in motivation, as both methods use 2D semantic knowledge to improve 3D representations for affordance segmentation. The specific alignment mechanisms are different, and this paper additionally considers joint semantic-affordance learning, but I did not find this work discussed in the related-work section. The authors should clarify the conceptual and methodological differences.
Questions for authors
- Can the authors provide a full-model ablation without OT and compare against stronger fusion baselines?
- Why are SO./AO. considered OOD given the similar training-time text templates? Can the authors also report seen vs. unseen DREDS results?
- How does this work differ from Unlocking 3D Affordance Segmentation with 2D Semantic Knowledge (Huang et al., CVPR 2026), which appears closely related but is not discussed?
I currently lean toward weak reject, but I may raise my score if the rebuttal addresses these points convincingly.
Thank you for the additional experiments. They partially address my concern, but my main reservations remain.
First, the LASO-style evaluation is still conducted under the authors’ own setting rather than the standard LASO[1] benchmark protocol. To demonstrate algorithmic advancement on affordance grounding, comparison with recent methods should be reported on established benchmarks such as LASO, not mainly on the proposed dataset/setting.
Second, the task significance is still unclear. The method centers on reconstructing 3D from a single image and then grounding concise semantic/affordance labels. The constructed dataset covers 16 object categories and 16 affordance types, which does not substantially broaden existing 3D affordance taxonomies. If reconstruction from 2D is partly motivated by the limited category coverage of existing 3D affordance datasets[1] relative to 2D ones[2], this paper does not expand that taxonomy, which makes the practical benefit of the reconstruction-based setting less clear to me.
Moreover, the proposed Optimal Transport fusion and spatio-functional adaptation appear closely tied to this reconstruct-then-align pipeline. The added 3D+text results show some transfer, but they still do not establish that the method is competitive or necessary under standard 3D+text affordance grounding settings such as LASO / 3D-AffordanceLLM / GREAT.
Neither the necessity of the proposed task nor the method’s effectiveness under standard comparable protocols has been sufficiently demonstrated. I therefore keep my score.
[1] LASO: Language-guided Affordance Segmentation on 3D Object, CVPR2024
[2] Learning Affordance Grounding from Exocentric Images, CVPR2022