acceptodds
Under review as a conference paper at ICLR 2027

QUERY-CONDITIONED RECONSTRUCTION REFINEMENT FOR 3D AFFORDANCE GROUNDING UNDER OCCLUSION

Abstract

3D affordance grounding aims to localize affordance regions specified by natural-language instructions. However, partial observations and severe occlusion often result in missing functional surfaces, while reconstructed geometry may contain unreliable local structures that adversely affect affordance grounding. To address this issue, we propose a query-conditioned reconstruction refinement framework for 3D affordance grounding under occlusion. Given partial multi-view RGB-D observations, we first perform amodal geometry completion to recover the complete object structure. We then introduce query-conditioned localization reliability estimation to assess the reliability of reconstructed local regions with respect to the target affordance, and use the estimated reliability to guide adaptive local detail refinement. Furthermore, we develop query-guided spatial-functional reasoning to jointly exploit geometric spatial relations and query-conditioned functional relations among local regions. By selectively refining reconstructed geometry according to its reliability for the target affordance, our framework enables robust affordance grounding under occlusion. Experimental results on the public dataset demonstrate the effectiveness of the proposed framework, achieving 24.0 IoU on the 3D affordance grounding task under occlusion.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D local part localization, occlusion, point cloud completion, geometric uncertainty, spatial-functional reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_2mnmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work studies language-guided 3D affordance grounding from partial multi-view RGB-D observations by employing a pretrained PCDreamer backbone for amodal point cloud completion, followed by localization-aware reliability estimation, reliability-guided local detail refinement, and query-guided spatial-functional graph reasoning. The proposed framework is evaluated on the Affogato benchmark through quantitative comparisons and ablation studies under incomplete observations.

Critical strengths
  1. In my view, this work investigates a valuable problem in partial-view 3D affordance grounding. Noting that generative point cloud completion may introduce geometric inaccuracies in occluded areas that affect downstream localization, the motivation to consider local reconstruction reliability during feature processing is reasonable.
  2. The two-stage training strategy is well designed, and the progressive ablation studies clearly demonstrate the contribution of each component.
Critical weaknesses
  1. Insufficient Evidence for Query-Conditioned Reliability: Eq. (2) incorporates language features to predict query-conditioned reliability, whereas the explicit supervision in Eq. (3) depends solely on geometric Chamfer distance. Although the downstream localization loss may indirectly introduce query dependence, the paper does not explain or empirically demonstrate whether the predicted reliability genuinely captures query-specific information rather than query-independent reconstruction quality.
  2. The manuscript omits essential technical definitions, including how completion confidence is extracted in Eq. (2), the specific operators in Eqs. (2), (4), and (6), the message-passing formulations in Sec. 3.4, and the exact loss functions in Sec. 3.5. Furthermore, Sec. 4.1 lacks the view-sampling protocol, point cloud resolution, point-to-GT metric mapping strategy, and the standard Affordance Chamfer Distance (aCD) metric.
  3. The comparison with completion-free methods (OpenAD, PointRefer, and Espresso-3D) may be unfair due to differences in the evaluation protocols. In Table 2, the simple Completion baseline (19.8 aIoU) already outperforms the best baseline in Table 1 (19.1 aIoU), suggesting that much of the improvement may come from the PCDreamer backbone rather than the proposed modules. A fairer comparison with baselines such as PCDreamer + Espresso-3D and PCDreamer + Affostruction is needed.
  4. In Fig. 3, the zoomed-in results of PointRefer, Espresso-3D, and Affostruction look nearly identical across five objects, and the "Depth Prior" column is not explained. The predicted reliability maps are also missing. In addition, Table 4 lacks baseline comparisons, while Table 5 does not distinguish between observed and reconstructed regions.
Questions for authors
  1. How does Eq. (2) learn meaningful query-dependent reliability, and can the authors validate this through an ablation of the language features? Could they also clarify the completion confidence, graph message passing, and region-to-point mapping? 2.Were the baselines evaluated on partial or completed point clouds, and could the authors provide fair comparisons using the same PCDreamer backbone, along with aCD and inference time ? 3.Why do the baseline predictions in Fig. 3 appear nearly identical across several objects, and how do reconstruction errors and affordance localization performance differ between observed and unobserved regions?
2 comments

We thank the reviewer for raising these related questions. We clarify the implementation of query-conditioned reliability estimation, provide a controlled comparison protocol, and further explain the qualitative prediction behavior and the scope of our reliability analysis.

1. Query-conditioned reliability estimation and implementation.

In Eq. (2), the predicted reliability is jointly estimated from the local geometric feature (\alpha_i), PCDreamer completion confidence (c_i^{pcd}), observation state (m_i), and language feature (F_T): ui=σ(ψ(αi,cipcd,mi,FT)).u_i=\sigma\left(\psi(\alpha_i,c_i^{pcd},m_i,F_T)\right). Here, (c_i^{pcd}) represents the completion confidence associated with the local region, whereas (m_i) indicates whether the region is directly observed or reconstructed. Importantly, the reliability target in Eq. (3) is derived from local geometric reconstruction discrepancy: ui∗=exp⁡(−β,CD(Si,Sigt)).u_i^*=\exp\left(-\beta,CD(S_i,S_i^{gt})\right). Thus, the Chamfer Distance provides geometric supervision, while the language feature enters the reliability predictor rather than the target construction. The query dependence comes from conditioning the prediction on (F_T): the same reconstructed region can receive different reliability estimates under different affordance queries. The subsequent localization objective further encourages the learned representation to support query-specific functional localization.

To validate the contribution of language conditioning, we conduct an additional ablation by removing (F_T) from the reliability estimator while keeping the remaining architecture and training protocol unchanged.

VariantaIoU (%)AUC (%)SIMMAE
w/o Language Feature22.874.70.4450.180
Full Model24.075.90.4510.177

The consistent improvements indicate that language conditioning provides useful information beyond local geometry, completion confidence, and observation state.

For spatial-functional reasoning, each local region is represented as a graph node. Geometry-based spatial relations and query-conditioned functional relations are propagated separately and then integrated to update the regional representations. Finally, the localization head predicts a score for each region, and these regional scores are projected onto their constituent points to obtain the dense affordance prediction.We have revised the Ablation Study subsection in the manuscript to include the language-feature ablation, and the Reliability-Guided 3D Affordance Grounding subsection to clarify the reliability estimation and spatial-functional reasoning mechanisms.

2. Fairness and consistency of baseline comparisons.

In the original Table 1, baseline methods follow their respective evaluation protocols, which may involve different input conditions. Therefore, these results provide a reference comparison rather than a strictly controlled ranking. We explicitly acknowledge this distinction and avoid attributing all performance differences solely to the proposed modules.

To provide a more controlled evaluation, we additionally compare the methods using the same PCDreamer completion backbone and identical completed point clouds, data splits, and evaluation settings. In addition to the original affordance metrics, we report average Chamfer Distance (aCD) and inference time to assess localization quality, geometric accuracy, and computational cost.

MethodaIoU (%)AUC (%)SIMMAEaCDTime (ms)
OpenAD4.265.70.3380.1470.09152.4
PointRefer11.876.50.4110.1180.07361.7
Espresso-3D15.279.40.4360.1080.06174.3
Affostruction20.172.80.4310.2080.04888.6
Ours24.075.90.4510.1770.03996.2

Under this controlled setting, our method achieves the best aIoU, SIM, and aCD, while Espresso-3D obtains better AUC and MAE. The additional refinement and spatial-functional reasoning introduce extra inference time. These results therefore demonstrate the benefit of our approach in localization overlap and geometric accuracy, while also making its computational overhead explicit.

3. Prediction behavior and reliability under different observation conditions.

The predictions in Fig. 3 may appear visually similar because the compared methods receive the same partial observations and affordance queries, and therefore identify broadly similar semantically relevant functional regions. Their differences are more evident in the spatial coverage and boundary accuracy of the predicted regions, particularly where the available geometric evidence is incomplete. Our method addresses this challenge by incorporating observation-state information and query-conditioned reliability into local feature refinement, followed by spatial-functional reasoning that exploits contextual relationships among regions.

We further clarify the scope of our quantitative reliability analysis. Table 5 groups local regions by their predicted reliability and reports their corresponding reconstruction errors and localization accuracy:

Predicted ReliabilityLocal Reconstruction ErrorLocal aIoU (%)
0.00–0.250.18215.7
0.25–0.500.14118.9
0.50–0.750.10323.1
0.75–1.000.07128.6

The decreasing reconstruction error and increasing local aIoU indicate that higher predicted reliability is associated with more accurate local geometry and better affordance localization. This supports the use of reliability to regulate local refinement. We emphasize that Table 5 analyzes regions grouped by predicted reliability, rather than directly comparing observed and unobserved regions. Similarly, Table 4 evaluates visibility at the whole-object level. These analyses provide complementary evidence of robustness under incomplete observations, but do not constitute a separate quantitative comparison of reconstruction errors and localization accuracy between observed and unobserved regions.

Overall, the proposed framework does not assume that all reconstructed regions are equally reliable. Instead, it estimates their usefulness for the current affordance query, selectively refines local features, and exploits spatial-functional context to improve final localization.We have revised the Qualitative Results in the manuscript subsection to clarify the comparison among methods, and the Robustness to Observation Completeness and Localization Reliability Analysis subsections to supplement the relevant analyses.

Reviewer_s2yLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a query-conditioned reconstruction refinement framework to improve 3D affordance grounding (localizing functional regions from natural language) in occluded environments. After performing amodal geometric completion, the method estimates the “local reconstruction reliability” conditioned on the target query and uses it to adaptively perform detail refinement as well as spatial and functional reasoning. Evaluation on the Affogato benchmark shows that the method achieves higher aIoU and SIM than existing approaches, demonstrating robust localization of functional regions under occlusion.

Critical strengths
  • Rather than relying solely on geometric reconstruction accuracy, the paper introduces a notion of "reliability" that reflects how useful a region is for a specific functional query and leverages it for refinement.
  • The paper integrates geometric completion (PCDreamer), reliability estimation, refinement, and spatial/functional reasoning into a single pipeline, forming a coherent workflow.
  • On the Affogato benchmark, the method outperforms existing approaches in aIoU (24.0) and SIM (0.451), demonstrating its effectiveness.
  • By incrementally introducing the LGR, RLDR, and QSFR components, the paper clearly quantifies the contribution of each.
Critical weaknesses
  • The supervisory signal for reliability is the discrepancy between the reconstruction and the ground-truth geometry (Chamfer Distance). However, this merely measures "geometric accuracy," and it is unclear how it differs conceptually from the “task-specific localization reliability” claimed in the paper.
  • In the discussion of Table 1, the authors state that the numerical results are only "for reference" because the input settings (observation protocols) differ. This makes the evidence too weak to quantitatively assert a clear advantage over existing methods.
  • The method optimizes the downstream stages with PCDreamer frozen, followed by only limited fine-tuning at the end. The paper does not discuss why it did not fully integrate and jointly optimize the completion and grounding stages, or the implications of this choice.
  • The evaluation is limited to a single benchmark, Affogato, and generalizability to other datasets or more extreme occlusion conditions is not sufficiently verified.
Questions for authors
  • The reliability estimator is supervised with geometric error (CD). Doesn’t this simply measure reconstruction accuracy? Please explain, theoretically or experimentally, how this connects to “query-conditioned functional reliability.”
  • In Table 1, the authors note that a strict ranking is difficult because the observation protocols differ. Under these circumstances, what is the valid basis for concluding that the proposed method outperforms existing methods?
  • Why was only “limited joint fine-tuning” applied to the PCDreamer backbone? What effect on performance would you expect from full end-to-end training?
  • When the occlusion ratio is extremely high, and the amodal completion itself produces a fundamentally incorrect shape, does the proposed reliability estimation mechanism still function properly and suppress false detections?
3 comments

We provide further clarification on the reliability estimation mechanism, the validity of baseline comparisons, the training strategy, and the limitations under severe occlusion.

1、Reliability Supervision and Its Interpretation

The reviewer raises an important distinction between geometric reconstruction accuracy and query-conditioned functional reliability. We clarify that CD-based supervision does not directly measure functional reliability. Instead, it provides a geometric quality prior for estimating whether a reconstructed local region can support reliable affordance localization. Specifically, the target reliability is defined as: ui∗=exp⁡(−β,CD(Si,Sigt)).u_i^*=\exp\left(-\beta,\mathrm{CD}(S_i,S_i^{gt})\right). This supervision encourages regions with lower geometric reconstruction errors to receive higher reliability scores. However, the predicted reliability is not determined by geometric accuracy alone. As defined in Eq. (2), ui=σ(ψ(αi,cipcd,mi,FT)),u_i=\sigma\left(\psi(\alpha_i,c_i^{pcd},m_i,F_T)\right), it jointly depends on the local geometric feature αi\alpha_i, PCDreamer's completion confidence cipcdc_i^{pcd}, the observation state mim_i, and the language representation FTF_T. In particular, conditioning on FTF_T allows the same reconstructed region to receive different reliability estimates for different affordance queries. Moreover, the downstream localization objective further encourages the reliability estimates to support query-specific functional localization. Therefore, CD provides geometric supervision, whereas the language-conditioned estimator and localization objective make the resulting reliability task-oriented rather than a direct measure of reconstruction accuracy.

This distinction is also supported by Table 5 of the original manuscript. As predicted reliability increases from 0.00–0.25 to 0.75–1.00, the average local reconstruction error decreases from 0.182 to 0.071, while local aIoU increases from 15.7 to 28.6. These results indicate that the estimated reliability is associated with both local geometric quality and downstream localization performance.We have revised the Ablation Study subsection in the manuscript to include the language-feature ablation, and the Reliability-Guided 3D Affordance Grounding subsection to clarify the reliability estimation mechanism.

2、Fairness and Validity of Baseline Comparisons

We agree that comparisons under different observation protocols should not be interpreted as a strictly controlled ranking. In the original Table 1, methods are evaluated following their respective input protocols. Therefore, the valid conclusion is that our method achieves the best aIoU (24.0%) and SIM (0.451) among the reported methods, rather than outperforming all baselines on every metric. Espresso-3D achieves higher AUC (79.0%) and lower MAE (0.111).

To provide a more controlled comparison, we additionally evaluate the methods using the same PCDreamer-completed point clouds, data splits, and evaluation settings. The results are reported below:

MethodaIoU (%)AUC (%)SIMMAEaCDTime (ms)
OpenAD4.265.70.3380.1470.09152.4
PointRefer11.876.50.4110.1180.07361.7
Espresso-3D15.279.40.4360.1080.06174.3
Affostruction20.172.80.4310.2080.04888.6
Ours24.075.90.4510.1770.03996.2

Under this controlled setting, our method achieves the highest aIoU, SIM, and the lowest aCD, demonstrating more accurate affordance-region localization and stronger agreement with the ground-truth affordance distribution. Espresso-3D still performs better in AUC and MAE, which we explicitly acknowledge. Although our refinement and spatial-functional reasoning introduce additional inference time, the controlled comparison provides a more consistent basis for evaluating the contribution of our localization framework under shared geometric inputs.

3、Training Strategy and End-to-End Optimization

We adopt a staged training strategy followed by limited joint fine-tuning to balance geometric reconstruction and downstream affordance localization. Initially, PCDreamer is kept frozen while the reliability estimation, local refinement, spatial-functional reasoning, and localization modules are optimized. This allows the downstream modules to adapt to a relatively stable completed geometry. We then perform limited joint fine-tuning, using a smaller learning rate for PCDreamer (1×10−51\times10^{-5}) than for the downstream modules (1×10−41\times10^{-4}).

Fully end-to-end optimization could potentially improve task-specific adaptation by allowing localization gradients to directly influence geometry completion. However, it could also cause the pretrained geometric prior to drift, because the completion objective favors overall geometric fidelity, whereas the localization objective focuses on query-dependent functional regions. In particular, sparse and query-dependent localization supervision may bias the completion backbone toward the current affordance query at the expense of general geometric consistency. Our staged strategy therefore aims to preserve the pretrained geometric prior while allowing moderate task-specific adaptation. We consider full end-to-end optimization a possible alternative, but do not claim that it has been experimentally shown to perform worse.

4、 Robustness to Severe Occlusion and Reconstruction Errors

Our reliability estimation is designed to reduce the influence of unreliable reconstructed regions, rather than guarantee the detection of every reconstruction failure. Under severe occlusion, local geometric features, completion confidence, observation states, and query semantics jointly contribute to the reliability estimate. Regions receiving lower reliability have a weaker contribution to the query-conditioned residual refinement: αˉi=αi+uiR(αi,FT).\bar{\alpha}_i=\alpha_i+u_iR(\alpha_i,F_T). Thus, when an incorrectly reconstructed region is assigned low reliability, its influence during local feature refinement is correspondingly reduced. The spatial-functional reasoning module further provides contextual evidence from neighboring regions, which can help resolve ambiguities when individual local geometries are incomplete.

Nevertheless, we acknowledge an important limitation: if an incorrectly reconstructed region is geometrically plausible and receives an overestimated completion confidence, the current estimator may fail to identify the error, particularly when the region is almost entirely unobserved. In such cases, false detections may not be fully suppressed. Therefore, our method improves robustness to spatially varying reconstruction errors but does not guarantee reliable localization under arbitrarily severe or fundamentally incorrect amodal completion.We have revised the Conclusion section in the manuscript to discuss the limitations under severe occlusion.

The authors addressed my previosus concerns by adding a controlled comparison experiment and providing reasonable explanations for the reliability estimation and training strategy. Based on these responses, I evaluate the contribution of the proposed method and raise my rating to Weak Acceptance.

Reviewer_aXcCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a method for language-guided 3D affordance grounding under occlusion, focusing on how to identify relevant reconstructed regions for refinement according to the current query.

Given partial multi-view observations and a query, the method first uses PCDreamer to complete the object geometry. It then estimates the reliability of local regions based on their geometric features, completion confidence, observation states, and the query. The estimated reliability is used to guide query-conditioned local feature refinement. The method further models spatial relations and query-dependent functional relations among these regions to predict the target affordance region.

Experiments on Affogato report an aIoU of 24.0, compared with 19.8 for the completion-based baseline, with ablation studies evaluating the contributions of reliability estimation, local feature refinement, and spatial-functional reasoning.

Critical strengths

The paper addresses a meaningful problem in 3D affordance grounding under occlusion: fully completed point cloud may contain unreliable structures under severe occlusion. The idea of estimating query-conditioned reliability to guide local refinement is well motivated, as the usefulness of reconstructed regions for affordance grounding depends both on their geometric accuracy and the target query.

Critical weaknesses
  1. The paper targets 3D affordance grounding under occlusion. It means the method should process both incomplete geometry and invisible affordance region. The experiments in Tab.4 only group via 'visible-surface ratio' of the whole object, which cannot illustrate that the method performs well especially when the affordance region is occluded.
  2. It is unclear whether the proposed method actually performs reconstruction refinement. In Eq. (4), the refinement only operates on local features, without explicitly modifying the reconstructed geometry. This seems closer to reliability-guided feature refinement than reconstruction refinement. Since query-conditioned reconstruction refinement is presented as the main contribution of the paper, I think this distinction needs to be clarified.
Questions for authors
  1. Could the authors provide a brief evaluation under different visibility levels of the target affordance region, rather than the whole object as in Table 4? For example, reporting the performance on fully visible, partially occluded, and mostly or fully occluded target regions, would help demonstrate the effectiveness of the method when the queried affordance region itself is occluded.
  2. Could the authors clarify whether the proposed reconstruction refinement modifies the reconstructed geometry or only the local features? From Eq. (4), the refinement appears to operate primarily in feature space.
3 comments

We thank the reviewer for these valuable suggestions. We address the two concerns below.

1. Localization Performance under Different Target-region Visibility Levels

To directly evaluate the impact of occlusion on the queried affordance region, rather than the entire object, we group test samples according to the visible ratio of ground-truth affordance points in the partial point cloud. Specifically, samples are categorized into fully visible (80%–100%), partially occluded (20%–80%), and mostly/fully occluded (0%–20%) groups.

Target-region VisibilitySamplesaIoU (%)AUC (%)SIMMAE
Fully Visible (80%–100%)18629.181.20.4820.148
Partially Occluded (20%–80%)41224.677.10.4580.169
Mostly/Fully Occluded (0%–20%)15317.868.40.4090.207

As shown in Table 3, localization performance gradually decreases as target-region visibility declines, with aIoU dropping from 29.1% to 17.8%. This confirms the increased difficulty caused by target-region occlusion and demonstrates the importance of completed geometry, reliability estimation, and query-guided spatial-functional reasoning for localizing partially or fully occluded affordance regions.We have revised the Robustness to Observation Completeness subsection in the manuscript to include the target-region visibility results and corresponding analysis.

2. Clarification of the Reconstruction Refinement Mechanism

We clarify that RLDR performs feature-level refinement rather than geometric reconstruction. Specifically, it updates local region features according to Eq. (4): αˉi=αi+uiR(αi,FT).\bar{\alpha}_i = \alpha_i + u_i R(\alpha_i,F_T). Here, αi\alpha_i denotes the original local feature, uiu_i represents the estimated localization reliability, and R(⋅)R(\cdot) generates a query-conditioned residual feature. The reliability score adaptively controls the residual contribution, enabling selective refinement of query-relevant features. Importantly, RLDR does not modify the point coordinates or geometric structure produced by PCDreamer; it only refines the corresponding local features. We have clarified this distinction in the manuscript.

Thank you for the detailed response and the additional experiments. The target-region visibility analysis directly addresses my first concern, and the clarification of feature-level refinement is also helpful. Given that my main concerns have been addressed, I am inclined to raise my rating to weak accept.

As a minor suggestion, it would be helpful to include the completion-based baseline ("Completion" in Table 2) in the target-region visibility analysis, especially for heavily occluded regions. This would further demonstrate the benefits of the proposed components when the target affordance region is occluded.

Thank you for this helpful suggestion. We agree that including the completion-based baseline provides a more direct comparison and helps better demonstrate the benefits of the proposed components under target-region occlusion. Following this suggestion, we conducted additional experiments to compare the Completion baseline with the Full Model under different target-region visibility levels.

As shown in the table below, under severe occlusion (0–20% visibility), the Full Model achieves an aIoU of 17.8%, compared with 10.9% for the Completion baseline. It also achieves better AUC (68.4 vs. 61.2), SIM (0.409 vs. 0.356), and MAE (0.207 vs. 0.249). For partially visible target regions (20–80%), the Full Model also outperforms the Completion baseline, achieving an aIoU of 24.2% compared with 20.1%. These results further demonstrate the benefits of the proposed components, particularly when the target affordance region is heavily occluded.

Additional experimental results:

MethodVisibilityaIoU ↑AUC ↑SIM ↑MAE ↓
Completion20–80%20.172.40.4260.190
Full Model20–80%24.276.10.4580.169
Completion0–20%10.961.20.3560.249
Full Model0–20%17.868.40.4090.207

Finally, we have incorporated these additional results into Table 4 of the manuscrip and revised the corresponding discussion in the Robustness to Observation Completeness subsection to reflect the new comparisons and findings.

Reviewer_TJqgOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a framework for language-guided 3D affordance grounding under occlusion. It completes partial multi-view RGB-D point clouds with PCDreamer, estimates local-region reliability using geometry, completion confidence, observation state, and the language query, and uses the estimated reliability to guide feature refinement before spatial-functional reasoning.

Critical strengths

1.This paper proposes query-conditioned localization reliability estimation and reliability-guided adaptive local refinement

  1. The proposed method achieves promising performance on the Affogato dataset.
Critical weaknesses

1.The proposed method improves performance through its relation modeling components. However, do these components increase GPU memory consumption or inference latency?

2.It is recommended that the spatial-functional relational modeling component be compared with cross-attention, which is commonly used to model relationships among tokens.

3.Can the authors specify how partial RGB-D views are generated.

Questions for authors

1.The proposed method improves performance through its relation modeling components. However, do these components increase GPU memory consumption or inference latency?

2.It is recommended that the spatial-functional relational modeling component be compared with cross-attention, which is commonly used to model relationships among tokens.

3.Can the authors specify how partial RGB-D views are generated.

1 comment

We thank the reviewer for the constructive comments. We address the three concerns below.

1. Computational Overhead of QSFR

To evaluate the computational overhead of QSFR, we compare the full model with a variant without QSFR under identical input settings and the same PCDreamer backbone.

MethodGPU Memory (GB)Grounding Latency (ms)Total Latency (ms)
w/o QSFR3.8218.360.9
Full Model (Ours)4.1723.766.3

QSFR introduces only 0.35 GB additional GPU memory consumption and 5.4 ms additional grounding latency, demonstrating a relatively modest computational overhead for spatial-functional relation modeling.

2. Comparison with Conventional Cross-Attention

To assess the effectiveness of QSFR, we replace it with conventional cross-attention, using local region features as queries and language features as keys and values, while keeping all other components unchanged.

MethodaIoU (%)AUC (%)SIMMAE
Cross-Attention22.974.80.4460.180
QSFR (Ours)24.075.90.4510.177

QSFR consistently outperforms conventional cross-attention across all four metrics, improving aIoU and AUC by 1.1 percentage points each. These results demonstrate the benefit of explicitly modeling spatial and query-conditioned functional relations beyond conventional feature interaction.

3. Partial RGB-D Data Generation

We generate partial point clouds by back-projecting valid depth pixels from multi-view RGB-D images using camera intrinsics, transforming them into a common coordinate system using camera poses, and registering and fusing the resulting points while excluding invalid measurements and background points. The fused partial point cloud is then fed into PCDreamer for geometry completion, providing the geometric input for subsequent reliability estimation, feature refinement, and affordance localization.We have revised the Experimental Setup section in the manuscript to clarify the construction of partial RGB-D inputs.

Comments on the paper

1 comment

Dear Reviewers,

The authors have submitted detailed responses to your reviews.

Please review their feedback, engage with the authors if any points remain unclear, and update your evaluation and score if their response has addressed your concerns.

Thank you

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.