- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
To address the inference bottleneck in Vision Transformers (ViTs) caused by quadratic self-attention complexity, as well as the spatial distortion and loss of small objects in existing merging methods, the paper proposes a training-free Graph-based Token Merging framework (GrabMe). It preserves spatial structure via one-hop constraints on a spatial adjacency graph and constructs a conflict-free, maximum-cardinality candidate pool using dynamic programming for O(N) parallel merging. Additionally, a similarity-based context score regulates merge decisions to protect fine-grained details. It substantially reduces tokens and accelerates inference while maintaining mIoU performance across semantic, instance, and panoptic segmentation benchmarks.
Critical strengths
- The paper is well-motivated and clearly presented. It introduces one-hop spatial graph constraints, a dynamic-programming-based conflict-free candidate pool, and complementary structural context to improve boundary distortion and performance degradation for small objects.
- The paper conducts thorough experiments across diverse semantic, instance, and panoptic segmentation benchmarks, demonstrating its effectiveness.
Critical weaknesses
- Although the motivation is reasonable, GrabMe does not elaborate on its essential differences from existing local/graph-structure-based merging methods (such as G2TM, DHTM, etc.). The authors need to better illustrate its differences at the level of topological modeling or merging decisions, as well as provide solid corresponding quantitative comparisons.
Questions for authors
- Although the motivation is reasonable, GrabMe does not elaborate on its essential differences from existing local/graph-structure-based merging methods (such as G2TM, DHTM, etc.). The authors need to better illustrate its differences at the level of topological modeling or merging decisions, as well as provide solid corresponding quantitative comparisons.
- While Table 4a demonstrates that combining pairwise similarity and context scores outperforms either component alone, it also indicates that the fusion ratio is performance-critical;nevertheless, no sensitivity analysis over the fusion coefficients (½, ½) is reported
- The concluding claim "Taken together, these results demonstrate that GrabMe substantially accelerates inference while maintaining superior performance across a diverse set of segmentation tasks" is not supported by the reported evidence. For semantic segmentation, the performance is on par with baseline (as shown in Table 1), while on instance and panoramic segmentation tasks, the performance exhibits a drop (as shown in Table 3).