- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces SPAM 2, an efficient object-aware superpixel segmentation framework that improves upon the original SPAM model. To eliminate the computational bottleneck of iterative clustering, SPAM 2 formulates superpixel generation as a trainable, single-pass local pixel-to-seed assignment. It leverages foundation segmentation models (like SAM) in two ways: (1) extracting high-level object features that are fused with low-level superpixel features via a FiLM module, and (2) utilizing a prior object map for test-time masked correction to prevent superpixels from leaking across object boundaries. Additionally, the authors propose an oracle superpixel supervision strategy that distills the regular structure of the original SPAM into SPAM 2, improving shape regularity. Extensive evaluations across five datasets (BSD, NYUv2, SBD, KITTI, DRIVE) demonstrate that SPAM 2 achieves state-of-the-art accuracy (ASA) while being up to 15× faster than its predecessor in its lightweight variant (SPAM 2 Fast). The authors also validate the real-world utility of their method on a downstream semantic segmentation task and introduce Wilcoxon signed-rank tests to confirm the statistical significance of their improvements.
Critical strengths
- Exceptional Accuracy-Efficiency Trade-off: By moving from iterative clustering to a feed-forward local assignment mechanism and leveraging lightweight foundation models (EfficientViT + MobileSAMv2), the authors achieve an impressive 15× speedup over SPAM. This effectively bridges the gap between high-level foundation model priors and the strict latency requirements of low-level vision preprocessing.
- Rigorous and Comprehensive Evaluation: The experimental protocol is highly robust. Beyond standard multi-scale curves, the inclusion of a downstream task (semantic segmentation via CAST) and pairwise Wilcoxon signed-rank tests across per-image scores sets a new, much-needed standard for evaluating superpixel algorithms in high-accuracy regimes.
Critical weaknesses
- Marginal Contribution of Joint Features: While "Joint Object Features" is highlighted in the paper's title, the ablation study (Figure 7 and Figure 13) reveals that the feature fusion module provides only a very marginal improvement in Achievable Segmentation Accuracy (ASA) compared to using the test-time masked correction alone. The heavy lifting is largely done by the local assignment network and the hard mask constraint.
- Inherited Bottlenecks from Foundation Models: As accurately noted by the authors in the DRIVE dataset evaluation, the method's performance is strictly bounded by the zero-shot capabilities of the chosen foundation model. When the foundation model fails to capture specific structures (e.g., thin retinal vessels), the object-guided constraint becomes a liability rather than an asset, limiting true domain-agnostic applicability.
Questions for authors
- Given the minimal performance gap between "SPAM 2 w/o joint" and the full "SPAM 2" in Figure 7, do you consider the added architectural complexity and training overhead of the FiLM fusion module to be strictly necessary for practical deployment?
- For highly specialized domains where off-the-shelf SAM fails (like the DRIVE dataset), would it be feasible to fine-tune the lightweight object encoder (e.g., EfficientViT) end-to-end with the superpixel network, or does the SPAM 2 pipeline strictly require a frozen, pre-computed object prior?
We thank the reviewer for their feedback and for their questions on the practical deployment of SPAM 2.
Q1. Joint object features improve accuracy at almost no computational cost. We agree that the practical benefit of joint features should be weighed against the additional training stage. To do so, we now report in Table 4 the statistical significance of ASA gains over the contributions in our method. Using joint object features yields a statistically significant ASA gain, especially at low superpixel counts, for only 0.5ms of additional inference time, since the object-encoder features are reused (Appendix B.4). The deep object features provide richer object-level information than the prior map alone and guide the assignment before masked correction is applied.
W2. Q2. Extending SPAM 2 to specialized domains. The object model is frozen in our experiments, but priors do not need to be precomputed offline. A domain-adapted object model could provide both the features and the masks. In principle, the object encoder could also be trained jointly through the fusion module; however, full end-to-end training would require specific handling of the discrete mask-generation and correction steps. The DRIVE results show that the benefit of object guidance depends on the quality of the prior in the target domain. When the prior fails, objects are mostly missed rather than wrongly segmented: the masked correction is then inactive and SPAM 2 relies on its learned local assignment. On DRIVE, SPAM 2 thus remains close to CDS, whereas SPAM's performance is strongly affected (Appendix C.1).