acceptodds
Under review as a conference paper at ICLR 2027

BridgeAfford: Bridging Query–Target Gap for Generalizable 3D Affordance Grounding

Abstract

Language-guided 3D affordance grounding aims to localize functional regions on 3D objects according to interaction intents. However, existing methods exhibit poor generalization to unseen objects. One challenge is the query–target gap: a semantic discrepancy between query features encoding interaction intents and target-region features, leading to inaccurate grounding. End-to-end training favors query representations tailored to seen objects, limiting their compatibility with shifted target-region features on unseen objects. To address this issue, we propose BridgeAfford, a framework that reduces the query–target semantic gap from two complementary perspectives. First, Interaction Region Alignment (IRA) promotes cross-object alignment among geometrically similar interaction regions during training. This enables structures on unseen objects that resemble previously observed ones to produce representations similar to those learned during training, facilitating target-region recognition. Second, Visual Evidence Aggregation (VEA) leverages a 2D affordance model to refine the attention distribution and guide the query toward the correct region, yielding more target-aware queries. Extensive experiments on the LASO and PIAD datasets demonstrate that BridgeAfford outperforms existing methods, particularly on unseen objects, highlighting its robust generalization ability.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Affordance, Deep Learning, Generalization

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_3jJrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets limited generalization in language-guided 3D affordance grounding, attributing the failure on unseen objects to the query–target gap: a semantic discrepancy between query features encoding interaction intents and the shifted target-region features on unseen objects. It proposes BridgeAfford, which narrows this gap from two complementary perspectives. First, Interaction Region Alignment (IRA) encourages consistent representations of geometrically similar interaction regions across objects during training, via a Category-Agnostic Interaction Memory (CAIM) with Diversity-Aware Initialization (DAI) and Affinity-Based Contrastive Learning (ABCL). Second, Visual Evidence Aggregation (VEA) renders multi-view images with normal-based coloring, applies a lightweight 2D affordance model to localize target regions in each view, and lifts these predictions into dense point-wise 3D evidence (CAMVA) that biases text-to-point cross-attention. Experiments on LASO and PIAD show leading performance, particularly in unseen settings, along with cross-dataset transfer (23.6 aIoU versus 19.7 for ViSPLA) and data efficiency (best aIoU with 20% of the training data).

Critical strengths

1.The problem diagnosis is clear and actionable: the paper quantitatively attributes unseen-object failure to the query–target gap and designs each module to address it, rather than making ad-hoc architectural changes. The resulting gains are large and consistent across benchmarks — best on all metrics on LASO unseen (aIoU 21.8 versus 18.7 for GLANCE) and a large margin on PIAD unseen (14.50 versus 9.20 for the best language-guided baseline). 2. The evidence is complete. Beyond accuracy tables, the query–target similarity analysis (Table 8, Δ from 0.193 to 0.245) and the attention visualizations (Figure 7) substantiate the claimed mechanism, and the fine-grained ablations (Table 5) justify each design choice (geometry-based over label-based alignment, DAI over random initialization, dense CAMVA over sparse keypoints). 3. The design is practically oriented: unlike ViSPLA and HAMMER, it avoids LLMs; CAIM is discarded at inference, leaving only the lightweight 2D model with 6-view rendering; and the data-efficiency study is directly relevant to the costly point-wise annotation setting.

Critical weaknesses
  1. The improvements are concentrated in the unseen setting, while the seen-setting advantage is marginal and partially behind prior work: on LASO seen, aIoU is only 0.4 above GLANCE (23.7 versus 23.3) with AUC 1.3 lower (86.8 versus 88.1), and on PIAD seen the AUC (87.60) is below HAMMER's 88.43. Readers who weight overall performance rather than unseen-only gains may find the contribution less compelling.
  2. The novelty is mainly integrative, with key components inherited from prior work: the 2D affordance model is GEAL's 2D branch, the rendering pipeline follows GEAL, and the backbone (Uni3D + RoBERTa) matches GLANCE. The paper's own increments (CAIM/DAI/ABCL/CAMVA) are relatively lightweight. Moreover, the computational overhead of multi-view rendering plus per-view 2D inference is not reported — there is no FPS, parameter-count, or memory comparison against baselines that do not use rendering.
Questions for authors

4

3 comments

Thank you for your careful review and for recognizing our problem diagnosis, supporting analyses, and practical design. We will address your suggestions and questions below.

For Weakness 1:

The improvements are concentrated in the unseen setting, while the seen-setting advantage is marginal and partially behind prior work... Readers who weight overall performance rather than unseen-only gains may find the contribution less compelling.

Thank you for your feedback. We acknowledge that the improvements are primarily in the unseen setting, but our method remains competitive in the seen setting. When compared individually with ViSPLA, GLANCE, and PART, our method shows advantages over each on multiple metrics under the seen setting.

  • Compared with ViSPLA: On LASO seen, our method improves two metrics. On PIAD seen, it achieves better results on all four metrics: aIoU increases from 23.10 to 25.40, AUC from 85.80 to 87.60, and SIM from 0.664 to 0.668, while MAE decreases from 0.089 to 0.080.
  • Compared with GLANCE: On LASO seen, our method improves three of the four metrics: aIoU increases from 23.3 to 23.7, SIM from 0.632 to 0.646, and MAE decreases from 0.089 to 0.086.
  • Compared with PART: On LASO seen, our method also achieves better results on three metrics: aIoU increases from 20.8 to 23.7, a gain of 2.9 points; SIM increases from 0.637 to 0.646, and MAE decreases from 0.090 to 0.086.

Although our method does not outperform every baseline on every metric, these results demonstrate its competitiveness under the seen setting.

For Weakness 2:

The paper's own increments (CAIM/DAI/ABCL/CAMVA) are relatively lightweight.

The contribution of our additional modules lies in the representation constraints and spatial guidance mechanisms designed to address the query–target gap.

Specifically, CAIM, DAI, and ABCL jointly form a geometry-driven region alignment mechanism. By constructing geometrically diverse region prototypes and selecting alignment targets according to geometric similarity, they encourage consistent local representations across objects. CAMVA converts multi-view 2D predictions into point-wise 3D evidence and uses attention biases to guide queries toward target-region features. These designs improve performance under the unseen setting.

Moreover, the computational overhead of multi-view rendering plus per-view 2D inference is not reported.

Regarding computational overhead, we have supplemented timing measurements over 30 runs, reporting the mean ± standard deviation: six-view rendering and 2D inference take 0.973 ± 0.067 s, while complete end-to-end inference takes 1.031 ± 0.059 s. BridgeAfford therefore completes a full prediction, including multi-view visual guidance, in approximately one second while achieving the grounding improvements demonstrated in our experiments.

For Question 1:

PART is cited in Related Work but not compared numerically. Could the authors add this comparison to the tables and explain the AUC gap?

Thank you for your suggestion. We have added PART’s results to Table 1 in the paper.

IoU measures the overlap between the predicted region and the ground-truth region, whereas AUC measures how well prediction scores rank target points above background points. Improvements in these two metrics therefore do not necessarily occur together. As illustrated in Figure 4, our method typically provides more complete coverage of the target region and produces high responses within it, consistent with its higher IoU. Meanwhile, some high responses may extend beyond the annotated region, causing certain background points to receive higher scores than some target points. This provides one possible explanation for improved regional overlap without a corresponding improvement in AUC.

For Question 2:

QueryMe is not cited; it evaluates on PIADv2, reporting 21.76 aIoU on unseen objects. Although no directly comparable numbers exist, could the authors discuss it and clarify the relation between PIAD and PIADv2 protocols?

Thank you for recommending QueryMe. We have cited and discussed it in the related work section.

We appreciate the value of enhancing query learning through multimodal evidence, particularly its use of 3D interaction information from human–object interaction images to improve affordance grounding, and its strong performance under the unseen-object and unseen-affordance settings of PIADv2. Both QueryMe and our work aim to improve how queries identify target regions, but they consider different input settings. QueryMe uses target point clouds, additional human–object interaction images, and model-generated textual attributes as multimodal evidence. Our work studies language-guided 3D affordance grounding, taking a target point cloud and an interaction instruction as input. VEA derives its visual evidence from multi-view renderings of the target point cloud itself.

Regarding the datasets, PIADv2 extends PIAD by increasing the number of 3D object instances from 7,012 to 38,889, object categories from 23 to 43, and affordance categories from 17 to 24. It retains seen-object and unseen-object evaluation and introduces a separate unseen-affordance setting, providing a valuable benchmark for studying broader generalization.

We did not adopt PIADv2 primarily because it does not directly provide natural-language interaction instruction annotations matching our task. Moreover, prior language-guided affordance prediction methods have not evaluated on this benchmark, leaving no directly comparable baselines. We therefore follow the benchmarks commonly used in prior language-guided affordance studies and evaluate on LASO and PIAD.

Dear Reviewer 3jJr,

Thank you for your time and thoughtful reviews. Please be reminded that the author-reviewer discussion period ends on November 18.

Since the authors have procvided their feedback, please review the feedback and actively participate in the discussion. If you have any remaining questions or concerns, please raise them before the deadline so the authors have sufficient time to respond. Thanks.

Best regards,

AC

Reviewer_jWtQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BridgeAfford, which aims to reduce the query–target semantic gap through two modules: Interaction Region Alignment and Visual Evidence Aggregation. Extensive experiments on LASO and PIAD demonstrate the superiority of the method.

Critical strengths
  1. The paper identifies the query–target semantic gap as the obstacle to affordance grounding on unseen objects, and proposes BridgeAfford: IRA promotes consistent representations of geometrically similar interaction regions across objects, while VEA converts multi-view 2D predictions into dense spatial evidence to guide target-region feature extraction and produce target-aware queries.
  2. The paper evaluates on both the seen and unseen splits of LASO and PIAD, and further includes LASO-to-PIAD transfer, ablations of IRA and VEA, and sensitivity checks for memory size, loss weight, and the number of views.
Critical weaknesses
  1. The memory size KK is selected only by a validation sweep, with no analysis of whether 16 slots cover the distinct local geometries in unseen data. When the memory is full during initialization, the two most similar entries are merged. The paper reports neither a merge error rate nor a check of whether functionally different regions with similar local shape are placed in the same slot. Later EMA updates cannot split a merged slot.

  2. The unseen setting is defined as held-out object–affordance pairs, while the paper also refers to unseen objects and unseen categories. Geometric alignment across objects is not a new motivation, and the experiments do not show performance when both the object category and the affordance are absent from training.

  3. VEA instantiates its 2D branch with GEAL and trains it on the same data. The unseen gain is therefore not separated from that 2D model. The paper does not show how much remains after removing it and keeping only the proposed multi-view aggregation and attention bias.

Questions for authors
  1. I am curious about how the memory size KK is set. Although the appendix contains an ablation, whether the size of KK depends more on the feature parts of the unseen categories in the dataset, and this memory is updated while training, for example, once it is full, two similar features have to be merged. How to guarantee that this merging is not problematic. The paper does not report a merge error rate, and it also does not check whether the two merged entries are functionally different and only locally similar in shape.

  2. Since using geometric features as the basis for generalization is not a novel motivation, I am concerned about whether the unseen setting described in the paper means unseen objects, unseen affordances, or both unseen, and whether the method can truly generalize.

  3. Is the contribution of VEA mainly the integration of a generalizable 2D model for feature extraction and fusion? The 2D branch appears to be GEAL’s model trained on the same data. What is the actual motivation, and how much of the unseen gain remains after separating that 2D model from the proposed aggregation and attention bias?

If the authors can satisfactorily address my concerns, I would be inclined to raise my rating.

7 comments

We sincerely thank you for your careful review and feedback. We address your concerns point by point below.

For Question 1:

whether the size of K depends more on the feature parts of the unseen categories in the dataset

In our method, the memory size K does not correspond to the number of object or affordance categories; rather, it controls the granularity of the representations stored in the memory slots. CAIM represents a set of local interaction structures in the training data and uses alignment constraints during training to encourage regions with similar structures across different objects to develop consistent representations, summarized by prototypes in the corresponding memory slots. A small K may compress substantially different regions into the same prototype, whereas an excessively large K may distribute similar structures across multiple prototypes, reducing opportunities for cross-object representation sharing. The most suitable value of K is therefore difficult to determine directly from data analysis, motivating our validation-based capacity selection.

How to guarantee that this merging is not problematic. The paper does not report a merge error rate, and it also does not check whether the two merged entries are functionally different and only locally similar in shape.

In our design, regions with different affordance labels sharing the same memory slot do not necessarily constitute an erroneous merge. Different affordances may involve interaction regions with similar semantics. For example, although “grasp a mug” and “lift a bag” correspond to different affordances, their interaction regions—the mug handle and the bag handle—both exhibit curved local geometric structures. We therefore consider it reasonable for the features of these two local structures to share the same slot despite their different affordance labels.

To examine the composition of our memory bank, we visualize the affordance labels associated with all region features merged into each slot, as shown in Figure 6 in the paper.

As shown in the left panel, although an anchor may incorporate regions with different affordance labels, these regions often share similar interaction semantics. For example, in the third anchor, wrap grasp and grasp involve semantically related interaction regions. In the twelfth anchor, stab and cut are both associated with sharp blade structures. In the fifteenth anchor, support, lay, and sit are associated with flat, load-bearing surfaces. The right panel quantifies the affordance composition of each anchor. Most anchors contain a dominant affordance, while the less frequent affordances often share semantic relationships with the dominant one.

Furthermore, we provide a t-SNE visualization of local object representations (fused features) across different affordances in the LASO dataset in Figure 7 in the paper. Regions with the same label do not always form a single, compact cluster. This suggests that the same interaction intent can involve diverse local representations, potentially reflecting different local geometric patterns. Forcing regions to merge solely according to their affordance labels may therefore overlook semantic differences between distinct local structures.

Beyond these visualizations, in table 5, we compare contrastive learning based on affordance labels with contrastive learning based on local structural similarity. Our approach achieves better performance, supporting the effectiveness of the proposed memory bank construction and alignment strategy.

For Question 2:

I am concerned about whether the unseen setting described in the paper means unseen objects, unseen affordances, or both unseen, and whether the method can truly generalize.

We thank the reviewer for pointing out the ambiguity in our use of “unseen.” Following the evaluation protocols of the LASO and PIAD benchmarks, “unseen” refers to object categories absent from training. Our evaluation therefore primarily focuses on generalization to unseen object categories. The relevant wording has been updated in the revised manuscript.

Subsequent works, including GEAL, GLANCE, and ViSPLA, adopt the corresponding benchmark protocols. Therefore, the unseen setting is widely adopted in the research community and provides a relatively fair and reasonable basis for comparing generalization to unseen objects given the currently available data.

Regarding generalization, we first demonstrate the effectiveness of our method under the unseen setting. We further investigate its ability to learn transferable representations from limited training data. As shown in Figure 3, the model maintains competitive performance when trained on only 20%–80% of the original training set, with the test setting held fixed. These results provide additional evidence of its data efficiency and its ability to generalize to unseen objects with reduced training data.

For Question 3:

Is the contribution of VEA mainly the integration of a generalizable 2D model for feature extraction and fusion? What is the actual motivation?

VEA’s contribution lies in converting 2D predictions into effective 3D spatial guidance. Transferring the capabilities of models with stronger generalization to a specific task is a widely adopted approach, and the methodological contribution often lies in how these capabilities are utilized.

In VEA, we do not directly fuse features from the 2D and 3D models. Instead, the 2D model predicts affordance maps on rendered images, and CAMVA aggregates these predictions into 3D space to indicate which regions are more likely to be the target. We incorporate this spatial information as an attention bias to guide the 3D model’s localization.

VEA is motivated as a complement to IRA. IRA encourages local representations to remain close to those of similar regions observed during training, making them easier for queries to recognize. VEA, in turn, uses 2D guidance to directly steer how queries identify and attend to target regions.

how much of the unseen gain remains after separating that 2D model from the proposed aggregation and attention bias?

In the paper, we compare the sparse point sampling strategy introduced by GLANCE with CAMVA. Appendix D.3 and Table 10 evaluate the two guidance strategies using the same 2D model and multi-view predictions: GLANCE-style sparse keypoint guidance achieves an aIoU of 21.2, whereas CAMVA achieves 21.8. This comparison shows that our aggregation and injection strategy provides additional benefits even when the source of 2D evidence is held fixed.

In Table 5, we also evaluate a “2D affordance model only” variant, which directly aggregates the 2D predictions into 3D space as the final affordance predictions, without introducing a 3D model. This variant performs worse than our full method, indicating that directly lifting predictions from the more generalizable 2D model is insufficient to achieve the same quality of 3D affordance grounding. These results support combining the 2D model’s spatial guidance with the 3D model’s prediction capabilities.

Thank you for the clarification. Your response resolves part of my concern. However, since generalization is a central claim of the paper, evaluating only unseen object categories still seems incomplete. Recent works such as HAMMER[1] further examine generalization to unseen affordances on PIADv2. I believe a similar experiment would strengthen the paper’s generalization claims.

Regarding VEA, my main remaining concern is its novelty and contribution separation. Table 10 still uses the same 2D model for both GLANCE-style guidance and CAMVA, with only a +0.6 aIoU gap. This does not show how much of the unseen gain remains after removing or replacing that 2D model while keeping only the proposed aggregation and attention bias. Without that separation, it remains unclear whether the improvement mainly comes from the stronger 2D predictor rather than from the proposed method itself.

[1] HAMMER: Harnessing MLLM via Cross-Modal Integration for Intention-Driven 3D Affordance Grounding

Reviewer_6iG6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BridgeAfford addresses language-guided 3D affordance grounding through Interaction Region Alignment (IRA) and Visual Evidence Aggregation (VEA). IRA aligns regional representations using geometrically retrieved memory prototypes, while VEA converts multi-view 2D predictions into spatial attention biases. Experiments on LASO and PIAD show improved performance, particularly under unseen settings.

Critical strengths

The method achieves meaningful improvements on the evaluated benchmarks. On LASO unseen, aIoU increases from GLANCE’s 18.7 to 21.8; on PIAD unseen, it improves from ViSPLA’s 9.20 to 14.50 among language-guided methods. These results support the practical effectiveness of the framework.

The two components provide complementary benefits under a shared backbone. Table 4 reports unseen aIoU improvements from 19.3 to 20.7 with IRA, to 20.4 with VEA, and to 21.8 with both. The appendix also provides useful analyses of query–region similarity and alternative visual guidance strategies.

Critical weaknesses

The DAI ablation does not isolate the benefit of diversity-aware initialization. In Table 5, DAI achieves 21.8 aIoU compared with 19.8 for random-vector initialization. However, DAI initializes the memory from actual region features, whereas random vectors may lie outside the learned feature distribution. Section 4.3 explicitly reports that the latter can cause training collapse. Consequently, this comparison cannot distinguish the benefit of preserving geometric diversity from simply using meaningful initial prototypes. Comparisons against randomly sampled training-region features or clustering-based initialization are needed to substantiate this component’s contribution. The t-SNE visualization in Appendix C does not resolve this confound.

The evidence does not sufficiently isolate the proposed advantage of dense visual guidance. Table 10 provides a useful controlled comparison, but the GLANCE-style variant and CAMVA differ in both evidence representation and injection mechanism: sparse keypoints are used as query priors, while dense evidence is introduced as an attention bias. The 21.2-to-21.8 aIoU improvement therefore cannot be attributed specifically to preserving dense information. Moreover, neither this 0.6-point gain nor the 0.4-point gain from depth confidence in Table 5 is accompanied by variability across runs. This limits confidence in the claimed benefit of these particular design choices over existing multi-view guidance.

Questions for authors

Can the authors compare DAI with initialization from randomly sampled real region features, using the same memory size, warm-up schedule, and loss? This would help determine whether diversity preservation contributes beyond meaningful initialization.

Can the authors separate evidence density from its injection mechanism, for example by comparing sparse and dense evidence through the same attention-bias interface? Please also report variability across runs for the comparisons in Tables 5 and 10.

For the LASO-to-PIAD experiment in Table 3, which training and test splits are used, and were original shape identities checked to exclude overlapping instances? Please also clarify whether PIAD unseen holds out entire object categories or specific object–affordance combinations, as described in §4.1.

4 comments

Thank you for your thoughtful review. We appreciate your recognition of BridgeAfford’s strengths and your constructive suggestions, which have helped us improve the paper. We address your questions below.

For Question 1:

Can the authors compare DAI with initialization from randomly sampled real region features, using the same memory size, warm-up schedule, and loss? This would help determine whether diversity preservation contributes beyond meaningful initialization.

We have added a comparison with initialization from randomly sampled real training-region features, keeping the memory size, warm-up schedule, and loss identical to those used with DAI. The results are as follows:

Initialization StrategyaIoU ↑AUC ↑SIM ↑MAE ↓
Random real-region feature initialization20.882.10.5890.103
Ours21.883.00.5970.099

Compared with random real-region feature initialization, DAI improves aIoU by 1.0 point and also improves the other metrics. This additional comparison supports that DAI benefits not only from meaningful initial features but also from preserving geometric diversity.

For Question 2:

Can the authors separate evidence density from its injection mechanism, for example by comparing sparse and dense evidence through the same attention-bias interface? Please also report variability across runs for the comparisons in Tables 5 and 10.

To further separate these factors, we have added a comparison that injects sparse evidence through the same attention-bias interface. Specifically, we sample keypoints following the GLANCE-style strategy and map them back into 3D space to construct sparse spatial guidance, which is then injected through the same attention-bias interface as our method. The results are as follows:

Visual GuidanceInjection MechanismaIoU ↑AUC ↑SIM ↑MAE ↓
Sparse keypoint evidenceAttention bias20.681.70.5910.102
CAMVA dense evidenceAttention bias21.883.00.5970.099

With the injection interface held fixed, CAMVA dense evidence achieves better performance, improving aIoU by 1.2 points. This improvement is consistent with our expectation that dense evidence can more comprehensively represent the target region.

Furthermore, we conducted three runs for each of the two strategies in the original Table 10. The following table reports the mean ± sample standard deviation:

aIoU ↑AUC ↑SIM ↑MAE ↓
GLANCE-style20.97 ± 0.2582.00 ± 0.100.5867 ± 0.00380.1030 ± 0.0010
CAMVA21.67 ± 0.1582.87 ± 0.150.6013 ± 0.00380.0980 ± 0.0010

CAMVA achieves an average aIoU improvement of 0.70 points. Its aIoU in each of the three runs exceeds all three results of the GLANCE-style comparison, and its mean performance is also better on the other metrics. These repeated runs provide additional evidence for the benefit of the overall guidance strategy evaluated in Table 10.

Repeated runs for all ablations in Table 5 have not yet been completed due to time constraints and the substantial experimental workload. We will supplement these experiments as soon as possible.

For Question 3:

For the LASO-to-PIAD experiment in Table 3, which training and test splits are used, and were original shape identities checked to exclude overlapping instances?

The original cross-dataset experiment in Table 3 follows the setting used by ViSPLA: training on the LASO seen training split and directly evaluating on the PIAD seen test split.

Following the reviewer’s suggestion, we further examined the point-cloud files and annotations and found that 25.75% of the original PIAD test data overlapped with the training data. Thank you for encouraging us to check this issue. We removed these overlapping data from the test set and reevaluated our model on the remaining test data. The results are as follows:

MethodOverlapping instances removedaIoU ↑AUC ↑SIM ↑MAE ↓
PointReferNo17.783.20.5670.110
GEALNo18.583.80.5820.096
3D-AffLLMNo17.682.40.570–
ViSPLANo19.784.50.610–
Ours*Yes22.986.00.6290.092

These results still show a clear improvement over the baselines. Based on the description in ViSPLA, overlapping instances do not appear to have been removed from their evaluation, yet our method still achieves better performance after removing them. Furthermore, the original comparisons in our paper follow the same evaluation protocol as the results reported in ViSPLA and therefore provide a fair comparison under that protocol.

Please also clarify whether PIAD unseen holds out entire object categories or specific object–affordance combinations, as described in §4.1.

We follow the standard PIAD dataset split throughout our work. PIAD uses an object-category holdout protocol: the six object categories in its unseen test set do not appear in the training set. The evaluation therefore focuses on transferring learned affordance associations to new objects. We have revised the relevant wording in the manuscript to avoid ambiguity.

Dear Reviewer 6iG6,

Thank you for your time and thoughtful reviews. Please be reminded that the author-reviewer discussion period ends on November 18.

Since the authors have procvided their feedback, please review the feedback and actively participate in the discussion. If you have any remaining questions or concerns, please raise them before the deadline so the authors have sufficient time to respond. Thanks.

Best regards,

AC

Thank you for the detailed rebuttal and additional experiments. My main concerns regarding the experiments and analyses have been addressed. I am therefore maintaining my positive score.

Reviewer_NAbTOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses language-guided 3D affordance grounding and attributes poor generalization to a "query–target gap" between decoder queries and target-region features. BridgeAfford builds on a LASO-style query decoder with a frozen Uni3D point encoder and a RoBERTa text encoder, and adds two modules. Interaction Region Alignment (IRA), used only during training, keeps a memory of interaction-region prototypes clustered by frozen geometric features and applies a contrastive loss that pulls each sample's fused region feature toward its nearest prototype. Visual Evidence Aggregation (VEA) renders six views, predicts 2D affordance maps with GEAL's 2D branch, lifts them to per-point scores by depth-weighted multi-view aggregation, and adds these scores as a bias to text-to-point cross-attention. The method reports 21.8 aIoU on the LASO unseen split (GLANCE: 18.7), as well as gains on PIAD and in LASO→PIAD transfer.

Critical strengths
  • IRA is simple, adds no inference cost, and has a clean ablation: adding it to the backbone improves aIoU by 1.4 on both the seen and unseen LASO splits (Table 4).
  • Results are consistent across LASO, PIAD, cross-dataset transfer and reduced training data (Tables 1–3, Fig. 3), and code is provided.
  • The paper includes hyperparameter studies and per-combination results (App. A, Table 11).
Critical weaknesses
  1. The gains are not separated from the backbone and training recipe. The backbone alone already reaches 19.3 unseen aIoU (Table 4), above every baseline in Table 1, including GLANCE (18.7), whose Uni3D and RoBERTa encoders the paper says it follows (p.6). The two proposed modules then add 1.4 (IRA) and 1.1 (VEA). No baseline is retrained with the same backbone and recipe, so the gains on PIAD (9.2 → 14.5 unseen aIoU, Table 2) and in cross-dataset transfer (19.7 → 23.6, Table 3) cannot be attributed to IRA or VEA either. Fig. 3 is consistent with this: the model trained on 20% of the data already achieves the best aIoU among all methods.

  2. The improvements are not specific to generalization and come from a single run. IRA improves seen and unseen aIoU by the same 1.4 points (Table 4), so the evidence does not show that it particularly helps unseen objects, as claimed (pp.1–2). The module gains of 1.1–1.4 aIoU are comparable to the differences between adjacent hyperparameter settings in Tables 6–7 (up to about 1 aIoU), and all experiments appear to use a single seed (seed 42 in both supplied training configurations). The query–target gap itself is quantified only by a small change in cosine similarity (−0.009 to 0.062, Table 8), which is neither related to accuracy nor reported separately for seen and unseen objects.

  3. The supplied VEA implementation does not match the paper.

    • In model/geometry.py (line 21), the camera center is computed as −Rᵀt from camera-to-world poses, so all six rendered views share the same camera center (0, 0, −1.6) in the projection step.
    • model/branch_3d.py applies post-processing that the paper does not describe: KNN smoothing of the lifted point prior, percentile calibration (1st–99th), and zeroing of scores below the 60th percentile.
    • Eq. 7 updates both the geometric key and the fused value of the selected memory entry by EMA, but scripts/train_3d.py updates only the fused value.

    If the reported numbers were produced with this code, the VEA gains (Table 4) and the dense-versus-sparse comparison (Table 10) cannot be attributed to multi-view lifting as described.

  4. The unseen split does not isolate unseen objects. Following the LASO protocol, only specific object–affordance pairs are removed from training (UNSEEN_REMOVE in dataset/data_utils.py). Of the 58 pairs in the unseen test set (Table 11), 33 appear in training. Macro-averaging Table 11, aIoU is about 15.0 on the 25 held-out pairs (about 14.9 on the 17 pairs of the 7 objects whose test pairs are all held out), versus 22.0 on the 33 pairs seen in training. This protocol is shared by all baselines, so the comparison itself is not unfair, but the headline "unseen" number is driven largely by seen pairs and overstates generalization to unseen objects.

  5. Novelty is limited, and closely related work is missing. IRA closely resembles prototypical contrastive learning (PCL; Li et al., ICLR 2021), which clusters embeddings into prototypes and pulls each sample toward its prototype; the main difference is that cluster assignment here is determined by frozen geometric features. VEA reuses GEAL's renderer and 2D branch (p.5) and lifts multi-view 2D predictions to 3D points, as in PartSLIP and OpenScene (both CVPR 2023); its new element is using the lifted scores as an attention bias. Neither PCL nor PartSLIP/OpenScene is cited.

Questions for authors
  1. Could you retrain GEAL and GLANCE with the same Uni3D-g and RoBERTa-large encoders (as in the supplied configuration) and the same training recipe, and report all methods separately on the 25 held-out pairs?
  2. Which split was used to train the 2D affordance model for the unseen experiments? The supplied config/train_2d.yaml uses setting: seen, under which dataset/laso.py does not remove the held-out pairs. If that checkpoint was used, the 2D prior has seen the held-out pairs.
  3. Were the reported results produced with the supplied code? Could you rerun Tables 4, 5 and 10 after correcting the camera-center computation, and add an oracle in which ground-truth 2D maps are lifted with the same aggregation?
  4. In Table 5, do the label-based alignment and random-initialization variants include VEA? If so, both fall below Backbone + VEA (20.4), which would mean these alternatives hurt rather than help.
  5. Could you report mean ± standard deviation over at least three seeds for Tables 1, 4 and 5?
  6. Are the Table 1 baseline numbers rerun under your split or taken from the original papers? Page 6 describes the unseen setting of both datasets as holding out object–affordance combinations; does the PIAD unseen split follow the standard PIAD release?
4 comments

Thank you for your thorough review and insightful suggestions. We address your questions point by point below.

For Question 1 & Weakness 1 & Weakness 4:

Could you retrain GEAL and GLANCE with the same Uni3D-g and RoBERTa-large encoders (as in the supplied configuration) and the same training recipe, and report all methods separately on the 25 held-out pairs?

We replaced the point-cloud and text encoders in GEAL and GLANCE with the Uni3D-g and RoBERTa-large encoders used in our method, retrained both methods, and evaluated them on the 25 object–affordance combinations held out during training. For GLANCE, since its 2D mask generation code has not been fully released, we used our rendered images and 2D affordance model to provide visual predictions while retaining GLANCE’s main architecture and sparse keypoint sampling guidance.

The results are as follows:

MethodaIoU ↑AUC ↑SIM ↑MAE ↓
GEAL (retrained)13.9168.70.5290.121
GLANCE* (retrained)14.5473.30.5160.119
BridgeAfford (ours)17.1076.20.5520.118

The aIoU reported here differs from the value calculated by the reviewer from Table 11 (15.0). The reviewer’s calculation assigns equal weight to each of the 25 combinations, without accounting for differences in their sample counts. All methods are evaluated on exactly the same 25 held-out object–affordance combinations using the same sample-weighted evaluation protocol as our main experiments.
Our method outperforms both comparisons on all four metrics across these 25 held-out combinations, with aIoU improvements of 3.19 and 2.56 points over GEAL and GLANCE*, respectively. These results further support the effectiveness of our method on held-out combinations.

For Question 2:

Which split was used to train the 2D affordance model for the unseen experiments? The supplied config/train_2d.yaml uses setting: seen, under which dataset/laso.py does not remove the held-out pairs. If that checkpoint was used, the 2D prior has seen the held-out pairs.

For the unseen results reported in our paper, both the 2D and 3D models were trained on the training split of the unseen setting. The setting in the configuration file is adjustable, allowing users to select the training split when reproducing the experiments.

For Question 3 & Weakness 3:

Were the reported results produced with the supplied code? Could you rerun Tables 4, 5 and 10 after correcting the camera-center computation, and add an oracle in which ground-truth 2D maps are lifted with the same aggregation?

Thank you for carefully examining the code and identifying this issue. The experimental results in the paper were not produced using the submitted version containing the coordinate transformation error. Our early implementation used the Mitsuba renderer and its corresponding coordinate transformation code. To reduce rendering overhead, we subsequently adopted the normal-based Gaussian renderer described in the paper and updated the coordinate transformation implementation accordingly. However, these updates were not correctly included when preparing the submitted code, resulting in a mismatch between the submitted version and the version used in our experiments. We have corrected and uploaded the relevant code.

Following the reviewer’s suggestion, we added an oracle experiment: we projected the 3D ground-truth annotations into each rendered view to generate 2D target-region masks, replacing the predictions of the 2D model. We then lifted these masks into 3D using the same CAMVA aggregation method and guided 3D affordance grounding through the same attention-bias interface. The results under the unseen setting are as follows:

SettingaIoU ↑AUC ↑SIM ↑MAE ↓
Ground-truth 2D mask guidance (Oracle)26.691.70.7180.078

This result reflects the upper limit of the gains our aggregation method can provide when the 2D predictions are perfectly accurate.

model/branch_3d.py applies post-processing that the paper does not describe: KNN smoothing of the lifted point prior, percentile calibration (1st–99th), and zeroing of scores below the 60th percentile.

KNN smoothing is applied as post-processing to improve the smoothness of the aggregated point-wise 3D scores. Percentile calibration normalizes these scores to the range of 0–1 and spreads their distribution. Setting some scores to zero is intended to prevent priors or noise in irrelevant regions from interfering with localization. Overall, these post-processing operations aim to improve the quality of the masks aggregated into 3D. We have added an experiment without these post-processing operations on LASO unseen:

aIoU ↑AUC ↑SIM ↑MAE ↓
21.482.50.6010.099

Performance decreases slightly relative to the original results, but the difference is small. We therefore consider these operations engineering refinements rather than decisive components, and attribute most of the improvement to VEA itself.

Eq. 7 updates both the geometric key and the fused value of the selected memory entry by EMA, but scripts/train_3d.py updates only the fused value.

The results reported in the paper were obtained by updating both the geometric features and the fused features. Not updating the geometric features slightly reduces performance. We provide the results without geometric-feature updates below:

aIoU ↑AUC ↑SIM ↑MAE ↓
21.382.50.5950.100

For Question 4:

In Table 5, do the label-based alignment and random-initialization variants include VEA? If so, both fall below Backbone + VEA (20.4), which would mean these alternatives hurt rather than help.

Both the label-based alignment and random-initialization variants in Table 5 retain VEA and only replace the corresponding designs within IRA. The reviewer’s interpretation is correct: these alternatives negatively affect performance in the reported experiments.

We believe that alignment based on affordance labels overlooks the fact that the same affordance can correspond to different local geometric structures, making forced alignment of these local representations undesirable. Random initialization of the memory bank leaves each slot without meaningful semantics and may lead to incorrect alignment.

In contrast, the complete IRA design improves performance from 20.4 to 21.8. These results indicate that the benefits of alignment depend on the choice of alignment targets and initialization strategy; introducing an arbitrary alignment constraint does not necessarily improve performance.

For Question 5 & Weakness 2:

Could you report mean ± standard deviation over at least three seeds for Tables 1, 4 and 5?

Thank you for this suggestion. The baseline results in the original Table 1 were taken from their respective papers, which did not provide standard deviations, and we previously followed the same reporting format.

We have supplemented experiments for our method in Table 1 under the seen and unseen settings using three random seeds and report the metrics as mean ± standard deviation below:

SettingaIoU ↑AUC ↑SIM ↑MAE ↓
Seen23.73 ± 0.2587.00 ± 0.260.6453 ± 0.00600.0863 ± 0.0025
Unseen21.67 ± 0.1582.87 ± 0.150.6013 ± 0.00380.0980 ± 0.0010

We also conducted three runs for the unseen-setting ablations in Table 4 and report the results as mean ± standard deviation below:

VariantaIoU ↑AUC ↑SIM ↑MAE ↓
Backbone19.13 ± 0.3880.97 ± 0.250.5700 ± 0.00620.1090 ± 0.0026
Backbone + IRA20.83 ± 0.2382.47 ± 0.060.5877 ± 0.00310.1020 ± 0.0020
Backbone + VEA20.60 ± 0.2081.53 ± 0.450.5923 ± 0.00150.1020 ± 0.0020
Backbone + IRA + VEA21.67 ± 0.1582.87 ± 0.150.6013 ± 0.00380.0980 ± 0.0010

Due to time and computational constraints, the remaining experiments for Table 4 and the repeated experiments for Table 5 have not yet been completed. We will supplement these results as they become available. Nevertheless, the repeated experiments on the unseen setting in Table 4 already demonstrate consistent gains from our method.

For Question 6:

Are the Table 1 baseline numbers rerun under your split or taken from the original papers?

All baseline results in Table 1 were taken from their respective original papers rather than obtained by retraining them ourselves. Both our method and the baselines follow the standard dataset splits without additional modifications.

Page 6 describes the unseen setting of both datasets as holding out object–affordance combinations; does the PIAD unseen split follow the standard PIAD release?

Yes, our PIAD unseen experiments follow the split in the standard PIAD release. Specifically, LASO excludes selected object–affordance combinations from training, leaving some object categories entirely unseen. PIAD directly holds out entire object categories, so all object–affordance combinations associated with these categories are absent from training.

Thank you for identifying this ambiguity. The original manuscript described both splits as holding out object–affordance combinations, which did not accurately convey their distinction. We have revised the wording to describe the two dataset protocols separately and align the description with the actual experimental settings.

For Weakness 5:

Thank you for pointing out these related works. We have added citations to these papers in the revised manuscript.

IRA and PCL both use prototypes for representation learning, but they construct alignment relationships differently. PCL clusters learned embeddings, whereas IRA establishes correspondences through local geometric similarity and applies alignment constraints to fused representations. This design encourages regions with similar interaction structures across different objects to share representations, avoiding the alignment of structurally different regions solely based on their affordance labels. The ablations comparing label-based alignment, random initialization, and the complete IRA design support the role of these choices.

VEA’s contribution lies in how 2D evidence is used for 3D localization. PartSLIP uses 2D detections for 3D part segmentation, while OpenScene supports 3D understanding through multi-view feature fusion and distillation. VEA aggregates instruction-related 2D predictions into point-wise spatial evidence and uses attention biases to guide text queries toward 3D target regions.

Dear Reviewer NAbT,

Thank you for your time and thoughtful reviews. Please be reminded that the author-reviewer discussion period ends on November 18.

Since the authors have procvided their feedback, please review the feedback and actively participate in the discussion. If you have any remaining questions or concerns, please raise them before the deadline so the authors have sufficient time to respond. Thanks.

Best regards,

AC

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

Thank you for the careful reviews and follow-up discussion. We have responded to each concern raised about BridgeAfford. Below, we summarize what each reviewer asked, the comparisons we conducted, and the corresponding revisions.

Reviewer NAbT. With matched encoders, BridgeAfford still achieved 17.10 aIoU on the 25 held-out LASO object–affordance pairs, providing further evidence of its ability to generalize to unseen combinations. We confirmed that both the 2D predictor and the 3D model excluded these pairs during training. We corrected the uploaded camera code to match the experimental implementation, verified both memory updates in Eq. 7, and reported memory-update and visual-prior ablations alongside a ground-truth-mask oracle (26.6 aIoU). Both Table 5 variants retained VEA, isolating the effects of label-based alignment and random initialization. Three-seed results in Tables 1, 4, and 5 showed consistent performance. We identified the sources of the Table 1 baselines, clarified in §4.1 that LASO holds out object–affordance pairs while PIAD holds out object categories, and added the suggested related work. We have responded to each concern the reviewer raised and would welcome their feedback on our clarifications and additional experiments.

Reviewer 6iG6. With memory size, warm-up, and loss fixed, diversity-aware initialization outperformed random initialization from real region features (21.8 vs. 20.8 aIoU). With the attention-bias mechanism fixed, dense guidance outperformed sparse keypoints (21.8 vs. 20.6 aIoU), and three-run results supported the stability of the original Table 10 comparison. We clarified the LASO-seen to PIAD-seen transfer protocol and obtained 22.9 aIoU after excluding the 25.75% of PIAD test instances that overlapped with LASO training. The reviewer confirmed that our responses addressed their main experimental and analytical concerns and maintained their positive rating.

Reviewer jWtQ. We explained how validation guides the choice of memory size, illustrated how affordances share memory slots (Figures 6–7), compared geometry-based and label-based alignment (Table 5), and clarified the original unseen splits. The reviewer indicated that our initial response addressed some of their concerns and raised two follow-up questions. First, to test generalization to unseen affordances, we jointly held out four affordance categories and nine object categories on PIAD. BridgeAfford achieved 11.54 aIoU, compared with 6.44 for our backbone and 7.51 for GEAL with the same backbone. Second, to isolate VEA’s contribution, we held the 2D predictions fixed and found that VEA outperformed direct lifting (Table 5), GLANCE-style sparse guidance in a three-seed comparison (Table 10), and GEAL’s cross-modal alignment with a matched 3D backbone and IRA disabled in both methods. We reported these experiments on November 15 and have yet to receive explicit feedback on whether they resolve the two follow-up questions. We hope the reviewer will consider these results in their final assessment.

Reviewer 3jJr. We added PART to Table 1, showed that BridgeAfford leads on three of four LASO-seen metrics (23.7 vs. 20.8 aIoU), and used Figure 4 to explain how improved target coverage can coexist with lower AUC. Table 4 isolates each module’s contribution on LASO unseen, with aIoU rising from 19.13 to 20.83 with IRA, 20.60 with VEA, and 21.67 with both. Over 30 runs, six-view rendering and 2D inference accounted for 94% of the 1.031 s mean end-to-end inference time. We added QueryMe to the related work and clarified that PIADv2’s unseen-affordance protocol does not directly provide the language instructions required by our task.

We appreciate the questions that led to these revisions and experiments. We respectfully ask the reviewers to consider this evidence when finalizing their ratings and hope it assists the Area Chair in preparing the meta-review.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.