- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses affordance grounding: localizing where a human would naturally interact with an object upon seeing it. In particular, it tackles this problem in a weakly supervised setting, learning from only two inputs, an exocentric image showing a human interacting with the object and an egocentric image containing only the object, rather than from dense and costly labels. The authors argue that existing methods either rely solely on visual information or use exocentric images only during training, and therefore cannot exploit that context at inference time. WSMA adds a text branch, but its text is fixed and independent of the current scene. To address these limitations, the authors propose EEMAG. Built on top of WSMA, EEMAG uses a VLM so that exocentric images can also provide information at inference time. It outperforms WSMA on two evaluation datasets, AGD20K and HICO-IFF.
Critical strengths
- Affordance grounding is an important problem for both interpretability and robotics, especially as the era of Physical AI approaches. The weakly supervised setting is a data-efficient and scalable choice, since it does not require fine-grained labels.
- The paper shows that adding a multi-modal branch can benefit the WSMA architecture.
- The authors are candid about the method’s weaker performance, and they transparently disclose the limitation that training is unstable.
Critical weaknesses
- Above all, EEMAG appears to underperform Selective-CL. The gap is not large, but the trend holds on both datasets (AGD20K and HICO-IFF) regardless of the split, which is disappointing given that a large vision-language model is used. Moreover, LoopTrans [1], a published work, outperforms Selective-CL. Taking this into account, there is no metric/dataset pair on which EEMAG achieves the best performance.
- The paper clearly lays out how it is similar to and different from WSMA, and this reviewer appreciates that. Even so, proposing a multi-modal branch falls short on novelty for the acceptance bar of a top-tier venue such as ICLR.
- A paper should be self-contained. This includes experimental hyperparameters, the computational cost of the experiments, how the metrics are computed (it is risky to assume that the reviewer already knows them), and dataset details. These are usually expected in the Appendix, but this submission covers them only thinly, which hurts both reproducibility and the review process.
- Finally, I cannot be confident that the paper compares fairly against the other baselines. Unlike the baselines, EEMAG uses an exocentric image, which (1) gives it additional information at inference time, (2) undermines the zero-shot inference that a text encoder would otherwise offer if an exocentric image is strictly required, and (3) is not explained in the paper: how the exocentric image is selected is never disclosed. Is it chosen from validation data, or at random? If from validation data, which data? If at random, what is the standard deviation? If neither, how robust is the method?
[1] Jiajin Tang, Zhengxuan Wei, Ge Zheng, and Sibei Yang. Closed-loop transfer for weakly-supervised affordance grounding. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9530–9539, 2025.
Questions for authors
- As far as I know, the ICLR guidelines place table captions above the table. Please revise accordingly.
- In Table 2, the SIM metric for EEMAG on the AGD20K-Seen split is listed as 0.419, but shouldn’t it be 0.420, as in the Table 3 ablation?
- It appears that DINO-ViT-S and Qwen2.5 are used as the image feature encoder and the VLM, respectively. DINO has progressed to DINOv3 and comes in a range of model sizes, so I would like to see experiments on these variants. Do newer models yield better performance? If not, the method may not generalize well. A similar experiment on the VLM backbone should support the same kind of conclusion.
- INTRA demonstrates zero-shot inference results by using a text encoder. I am curious whether EEMAG can do the same. If not, there would be no advantage to using a separate text encoder.
- When using quotation marks in LaTeX, please use `` for opening quotes rather than the straight or curly double quote “, as this is the correct LaTeX syntax.