- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces AffiMoE for language-grounded affordance segmentation. It uses GPT to identify the target object, infers the intended action, and generate a bounding box. It then introduces cross-attention between MLLM representations and SAM 3 features, followed by token-wise routing between global and locally aligned representations. It also constructs AffordBench for training and manually revise UMD annotations for evaluation.
Critical strengths
-
The paper addresses a meaningful problem: identifying action-specific functional regions rather than merely localizing objects. The proposed MoE-style routing provides a well-motivated way to selectively combine global semantics with locally aligned visual evidence, supporting token-level fine-grained text–visual alignment.
-
The fine-grained alignment module shows clear benefits. In Table 2, adding FGA without LoRA improves UMD-relabeled performance from 73.82/72.40 to 82.71/80.79 cIoU/gIoU.
-
The human-verified training annotations are a useful contribution. The additional HANDAL results also provide positive evidence: AffiMoE improves over AffordanceVLM using the same training data.
Critical weaknesses
-
The routing mechanism is underspecified. Section 3.3 does not precisely define the response aggregation function, explain how scores from the two branches are calibrated, or clarify which tokens are routed. Larger feature responses are not necessarily more reliable evidence.
-
MoE provides limited and inconsistent gains. In Table 2, adding MoE with LoRA improves UMD-relabeled cIoU by 1.48 points but decreases gIoU by 0.85 points. In Table 4, its gIoU advantage over shared-weight fusion is only 0.10 points, without repeated-run statistics.
-
Architectural gains are not sufficiently isolated. GPT already resolves the object, action, and bounding box before downstream segmentation. Whether baselines share this preprocessing is unclear, and the substantially different SAM 3 baseline results in Tables 2 and 3 require an explicit configuration summary.
-
Fine-grained alignment needs more targeted evaluation. UMD and UMD-relabeled use the same images, while aggregate IoU alone does not establish action-specific alignment. Same-object/different-action tests, boundary metrics, and per-affordance results would strengthen the evidence.
Questions for authors
check the weakness, and I will raise my score if you can solve my questions.