- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether reconstruction-pretrained geometry-aware features can improve visual affordance prediction. The proposed approach uses a frozen VGGT-Ω encoder to extract intermediate features from a target image and, when available, auxiliary views. These features are projected through lightweight adapters and fused with existing visual representations used by several affordance predictors. The method is evaluated on 3DOI, HANDAL, and AGD20K, covering scene-level images/videos, object-centric affordance segmentation, and weakly supervised affordance grounding. The paper studies single-view versus multi-view inputs, different VGGT-Ω feature depths, and several fusion strategies. The main findings are that VGGT-Ω features generally improve affordance prediction, additional views do not always monotonically improve performance, and shallow/multi-layer features can be more useful than deeper individual layers.
Critical strengths
The paper is clearly written and evaluates the proposed feature augmentation across several datasets and existing affordance predictors rather than relying on a single benchmark. The experiments cover different input regimes, including single-view and multi-view settings, and include analyses of feature depth and fusion strategy. The paper also makes a reasonable effort to keep the pretrained encoders frozen and isolate the effect of lightweight feature adaptation, which makes the empirical comparisons relatively easy to understand.
Critical weaknesses
- The main research question has already been widely studied, so the conceptual novelty is limited. The paper asks whether 3D geometry-aware features can improve affordance prediction. However, prior work has already shown that geometric/shape cues [1], depth-based geometric priors [2], and geometry encoded in frozen visual foundation models [3] can improve affordance prediction. More broadly, the geometric awareness of foundation-model features has also been systematically studied [4], and VGGT has already demonstrated transferability to downstream tasks [5]. Thus, the main difference here appears to be using VGGT-Ω intermediate features as the geometry source and fusing them into existing affordance predictors. This feels more like a feature/backbone substitution than a new methodology or scientific insight.
- Some key experimental protocols are not sufficiently convincing or clearly specified.
The HANDAL evaluation is particularly concerning. Table 4 reports AffordanceNet at 60.3/60.8 under the original setting, but only 31.0/13.3 under the authors' split, after which VGGT-Ω improves the result to 67.5/69.7. Such a large baseline drop makes it difficult to interpret the magnitude of the reported improvement without a clearer explanation of how the new split differs from the standard evaluation.
Moreover, the HANDAL sampling protocol is described inconsistently: Appendix A.3 uses auxiliary offsets of ±4, ±8, and ±12, while Appendix B.2 describes non-overlapping 11-frame segments with the sixth frame as query and stride-based sampling. The paper also does not report repeated runs or variance, making it hard to judge the robustness of the smaller improvements.
[1] Affordance Detection of Tool Parts from Geometric Features, ICRA 2015.
[2] Learning Precise Affordances from Egocentric Videos for Robotic Manipulation, ICCV 2025.
[3] Probing and Bridging Geometry-Interaction Cues for Affordance Reasoning in Vision Foundation Models, CVPR 2026.
[4] Probing the 3D Awareness of Visual Foundation Models, CVPR 2024.
[5] VGGT: Visual Geometry Grounded Transformer, CVPR 2025.
Questions for authors
What is the main scientific novelty beyond showing that VGGT-Ω intermediate features also transfer to affordance prediction? How does this differ from prior work using geometric/depth cues or geometry encoded in frozen foundation models?
In Table 4, AffordanceNet drops from 60.3/60.8 to 31.0/13.3 under “our split,” while adding VGGT-Ω raises it to 67.5/69.7. Why does the baseline deteriorate so substantially under the new split, and how much of the reported gain depends on this evaluation protocol?
Please clarify the HANDAL auxiliary-view sampling protocol. Appendix A.3 uses offsets of ±4, ±8, and ±12, while Appendix B.2 describes non-overlapping 11-frame segments with the sixth frame as query and stride-based sampling. How do these two descriptions correspond?
Were the reported results obtained from a single run? If so, could the authors report results across multiple seeds or provide variance, especially for the relatively small gains in some of the multi-view and fusion comparisons?
5 comments
Response to Reviewer B9mi’s Question 2
Q2:In Table 4, AffordanceNet drops from 60.3/60.8 to 31.0/13.3 under “our split,” while adding VGGT-Ω raises it to 67.5/69.7. Why does the baseline deteriorate so substantially under the new split, and how much of the reported gain depends on this evaluation protocol?
We identified the cause of the anomalous results. The low reproduced scores of 31.0/13.3 were primarily due to an incompatible inference environment. To accommodate VGGT-Ω, we used a different version of Transformers from the one specified by the official AffordanceNet implementation. This changed CLIP’s forward pass and attention behavior; numerical differences under bf16 further affected the [AFF] features and mask decoding. As a result, the model incorrectly labeled desks, floors, walls, and nearby objects as foreground, producing many false positives. We have replaced Table 4 with the corrected evaluation results.
The performance gains persist under both evaluation protocols. After resolving the environment issue, we reran the experiments using Our Split. We also reproduced the original AffordanceNet sampling procedure as closely as possible for Original Split: within each complete video, we selected auxiliary frames at offsets of ±4, ±8, and ±12 from the target frame . We excluded only target frames near video boundaries for which the full set of auxiliary frames was unavailable, and did not subsample the target frames at fixed intervals. The improvement from adding VGGT-Ω remained under Original Split.
| Methods | View | gIoU (Our Split) ↑ | gIoU (Original Split) ↑ | cIoU (Our Split) ↑ | cIoU (Original Split) ↑ |
|---|---|---|---|---|---|
| AffordanceNet | Single | 60.01 | 59.35 | 59.98 | 58.95 |
| + Geometry Feature | Single | 67.50 | 69.66 | 69.70 | 72.08 |
| + Geometry Feature | 1+2 | 68.10 | 70.67 | 71.00 | 71.91 |
Response to Reviewer B9mi’s Question 3
Q3: Please clarify the HANDAL auxiliary-view sampling protocol. Appendix A.3 uses offsets of ±4, ±8, and ±12, while Appendix B.2 describes non-overlapping 11-frame segments with the sixth frame as query and stride-based sampling. How do these two descriptions correspond?
We thank the reviewer for pointing out that our description was unclear. The previous sampling procedure had two stages. Appendix B.2 describes target-view sampling: we divided each video into non-overlapping 11-frame segments and selected the sixth frame of each segment as the target view. Appendix A.3 describes auxiliary-view sampling after the target view was selected: we chose frames at offsets of ±4, ±8, and ±12 from the target frame within the complete video. This procedure was intended to minimize repeated use of the same video frames.
We will replace the original experimental results and sampling description with those from the corrected protocol. We appreciate the reviewer’s correction.
Response to Reviewer B9mi’s Question 1
Q1: What is the main scientific novelty beyond showing that VGGT-Ω intermediate features also transfer to affordance prediction? How does this differ from prior work using geometric/depth cues or geometry encoded in frozen foundation models?
We thank the reviewer for highlighting the contributions of related work. Our contribution is not a new reconstruction model or fusion operator. Rather, we investigate when geometry-aware features pretrained for reconstruction help affordance prediction and how to use them without explicitly reconstructing a 3D scene. Across diverse datasets and models with substantially different architectures, we examine single-view inputs versus related views from the same scene, feature levels, and fusion strategies to understand the role of 3D geometric information in different types of data.
Whereas [1]–[3] use local shape, estimated depth, or geometric cues from frozen models, our work focuses on the conditions under which reconstruction-pretrained geometric features are useful for affordance prediction. Works [4] and [5] do not examine this behavior in affordance prediction. The table summarizes the distinctions between our work and the studies you mentioned. We will state our contribution more clearly as empirical findings and an analysis of the underlying mechanisms, rather than as a new fusion algorithm.
[1] Affordance Detection of Tool Parts from Geometric Features, ICRA 2015.
[2] Learning Precise Affordances from Egocentric Videos for Robotic Manipulation, ICCV 2025.
[3] Probing and Bridging Geometry-Interaction Cues for Affordance Reasoning in Vision Foundation Models, CVPR 2026.
[4] Probing the 3D Awareness of Visual Foundation Models, CVPR 2024.
[5] VGGT: Visual Geometry Grounded Transformer, CVPR 2025.