acceptodds
Under review as a conference paper at ICLR 2027

SPAM 2: Efficient Superpixel Anything using Joint Object Features

Abstract

Recent superpixel methods based on foundation segmentation models have pushed accuracy further by constraining superpixels to lie within high-level objects. However, this gain comes with a high computational cost, which limits their use in practical or large-scale vision pipelines. In this work, we introduce SPAM 2 (SuperPixel Anything Model 2), an efficient object-aware superpixel framework that notably replaces the costly traditional iterative clustering by a fast trainable local pixel-to-seed assignment and leverages lightweight foundation segmentation models to constrain superpixels within objects. To further exploit high-level information, SPAM 2 fuses superpixel and object features through a joint representation before final assignments. We also introduce a new supervision strategy based directly on oracle superpixel maps to recover more regular regions from a denser training signal. Comprehensive experiments on standard benchmarks and a downstream superpixel-based task show that SPAM 2 achieves state-of-the-art superpixel accuracy, while its lightweight variant is an order of magnitude faster than other object-based methods, making it suitable for dedicated computer vision pipelines. Code and pretrained models will be made available.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Superpixels, SAM, Efficiency

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_gCQaOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces SPAM 2, an efficient object-aware superpixel segmentation framework that improves upon the original SPAM model. To eliminate the computational bottleneck of iterative clustering, SPAM 2 formulates superpixel generation as a trainable, single-pass local pixel-to-seed assignment. It leverages foundation segmentation models (like SAM) in two ways: (1) extracting high-level object features that are fused with low-level superpixel features via a FiLM module, and (2) utilizing a prior object map for test-time masked correction to prevent superpixels from leaking across object boundaries. Additionally, the authors propose an oracle superpixel supervision strategy that distills the regular structure of the original SPAM into SPAM 2, improving shape regularity. Extensive evaluations across five datasets (BSD, NYUv2, SBD, KITTI, DRIVE) demonstrate that SPAM 2 achieves state-of-the-art accuracy (ASA) while being up to 15× faster than its predecessor in its lightweight variant (SPAM 2 Fast). The authors also validate the real-world utility of their method on a downstream semantic segmentation task and introduce Wilcoxon signed-rank tests to confirm the statistical significance of their improvements.

Critical strengths
  1. Exceptional Accuracy-Efficiency Trade-off: By moving from iterative clustering to a feed-forward local assignment mechanism and leveraging lightweight foundation models (EfficientViT + MobileSAMv2), the authors achieve an impressive 15× speedup over SPAM. This effectively bridges the gap between high-level foundation model priors and the strict latency requirements of low-level vision preprocessing.
  2. Rigorous and Comprehensive Evaluation: The experimental protocol is highly robust. Beyond standard multi-scale curves, the inclusion of a downstream task (semantic segmentation via CAST) and pairwise Wilcoxon signed-rank tests across per-image scores sets a new, much-needed standard for evaluating superpixel algorithms in high-accuracy regimes.
Critical weaknesses
  1. Marginal Contribution of Joint Features: While "Joint Object Features" is highlighted in the paper's title, the ablation study (Figure 7 and Figure 13) reveals that the feature fusion module provides only a very marginal improvement in Achievable Segmentation Accuracy (ASA) compared to using the test-time masked correction alone. The heavy lifting is largely done by the local assignment network and the hard mask constraint.
  2. Inherited Bottlenecks from Foundation Models: As accurately noted by the authors in the DRIVE dataset evaluation, the method's performance is strictly bounded by the zero-shot capabilities of the chosen foundation model. When the foundation model fails to capture specific structures (e.g., thin retinal vessels), the object-guided constraint becomes a liability rather than an asset, limiting true domain-agnostic applicability.
Questions for authors
  1. Given the minimal performance gap between "SPAM 2 w/o joint" and the full "SPAM 2" in Figure 7, do you consider the added architectural complexity and training overhead of the FiLM fusion module to be strictly necessary for practical deployment?
  2. For highly specialized domains where off-the-shelf SAM fails (like the DRIVE dataset), would it be feasible to fine-tune the lightweight object encoder (e.g., EfficientViT) end-to-end with the superpixel network, or does the SPAM 2 pipeline strictly require a frozen, pre-computed object prior?
1 comment

We thank the reviewer for their feedback and for their questions on the practical deployment of SPAM 2.

Q1. Joint object features improve accuracy at almost no computational cost. We agree that the practical benefit of joint features should be weighed against the additional training stage. To do so, we now report in Table 4 the statistical significance of ASA gains over the contributions in our method. Using joint object features yields a statistically significant ASA gain, especially at low superpixel counts, for only 0.5ms of additional inference time, since the object-encoder features are reused (Appendix B.4). The deep object features provide richer object-level information than the prior map alone and guide the assignment before masked correction is applied.

W2. Q2. Extending SPAM 2 to specialized domains. The object model is frozen in our experiments, but priors do not need to be precomputed offline. A domain-adapted object model could provide both the features and the masks. In principle, the object encoder could also be trained jointly through the fusion module; however, full end-to-end training would require specific handling of the discrete mask-generation and correction steps. The DRIVE results show that the benefit of object guidance depends on the quality of the prior in the target domain. When the prior fails, objects are mostly missed rather than wrongly segmented: the masked correction is then inactive and SPAM 2 relies on its learned local assignment. On DRIVE, SPAM 2 thus remains close to CDS, whereas SPAM's performance is strongly affected (Appendix C.1).

Reviewer_ZVksOpenReview ↗
Rating
3(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SPAM constrains superpixels with SAM object masks so that they do not cross objects. It is accurate but slow: automatic grid mask generation with SAM ViT-H is the main bottleneck, and the object-constrained iterative k-means itself takes about 750 ms at K = 1000. Moreover, object information enters only at the last step as a frozen constraint and plays no part in forming the superpixels.

SPAM 2 consists of four components:

  1. A local assignment decoder reuses the CDS/SFCN architecture and loss and, in a single forward pass, predicts for every pixel a softmax assignment over the seeds in its 3×3 neighbourhood, replacing iterative clustering.
  2. Masked correction uses the SAM prior map at test time to find each superpixel's dominant object, cancels the assignment of inconsistent pixels, and propagates labels only from valid neighbouring superpixels within the same object; an object without a seed receives a new label.
  3. A joint object–superpixel feature module injects intermediate object-encoder features into the superpixel features through FiLM; training is two-stage, with the second stage freezing everything except the fusion module.
  4. Oracle superpixel supervision runs the original SPAM with the BSD training ground truth as the object prior (K ≈ 100) to produce oracle superpixel maps, then fine-tunes the model on them, yielding the more regular Reg variants.

Two configurations are evaluated: the default SPAM 2 uses SAM ViT-L with automatic grid masks; SPAM 2 Fast uses an EfficientViT encoder with a MobileSAMv2 decoder.

Main results:

  1. On BSD, the AUC of ASA over K from 100 to 1000 is 0.9725 for SPAM 2, 0.9717 for SPAM 2 Fast, 0.9713 for SPAM and 0.9679 for CDS, so SPAM 2 is only about 0.001 above SPAM and 0.005 above CDS. SPAM 2 ranks first on NYUv2 and SBD and the Fast variant ranks first on KITTI, but on DRIVE CDS is best (0.9309 against 0.9261). The accuracy advantage exists but is small, and it does not hold on every dataset.
  2. SPAM 2 Fast takes about 290 ms per BSD image, which the authors present as about 15× faster than SPAM and 10× faster than SPAM 2. With the same ViT-H + grid prior, SPAM 2 is only about 1.5× faster than SPAM, and the default SPAM 2 still takes 3.3–3.4 s per image. Excluding the object model, the superpixel stage drops from 751 ms to 19.5 ms. The speedup of the superpixel stage is genuine (about 40×), but the overall 15× comes mainly from the lighter SAM variant rather than from the method itself.
  3. The Reg variants raise GR on BSD from 0.284 to about 0.40, against 0.478 for SPAM. SPAM 2 trades regularity for speed; oracle supervision recovers part of it but does not return to SPAM's level.
  4. Pairwise Wilcoxon tests on per-image ASA interpolated at K = 500 are reported, which the authors describe as a first for superpixel segmentation. The tests show that the small ASA gains are consistent across test images rather than driven by a few.
Critical strengths
  1. The paper identifies the two computational bottlenecks of object-prior superpixels and addresses each separately; the motivation is clear and the engineering value is real. The superpixel stage is timed in isolation (751 → 19.5 ms) and scaling with image size and K is reported, so the evidence is adequate.
  2. The choice of object model is studied systematically. The paper sweeps five encoders and two decoders for accuracy against runtime and reports plainly that, under the same ViT-H prior, the gain is only 1.5×, which is a useful reference for choosing a prior in deployment.
  3. The evaluation is broad and negative results are reported. Full curves and AUCs for three metric families on five datasets, method rankings and per-image distributions are all given; the weaker results on DRIVE are reported and explained.
  4. Oracle supervision is simple and practical. Dense pseudo-labels are generated by the slow but regular SPAM with ground-truth priors, and regularity is distilled into the feed-forward model at no extra inference cost.
Critical weaknesses
  1. The attribution of the speedup is unclear, and "an order of magnitude faster" is overstated

The paper credits the switch to a lightweight SAM to its architecture: the introduction states that "this architecture also enables a lightweight configuration, up to 15x faster than SPAM", and the abstract claims "an order of magnitude faster than other object-based methods". However, both SPAM and SPAM 2 consume only a prior map, so substituting MobileSAMv2 does not depend on the SPAM 2 architecture. The authors' own data support this: Figure 5 shows that, under the same ViT-H prior, SPAM 2 is only about 1.5× faster than SPAM, and the default SPAM 2 still takes 3.3 s per image, the same order as SPAM's 4.4 s; of the 290 ms of SPAM 2 Fast, about 270 ms is the object model, so SPAM with the same lightweight prior would take roughly 0.9 s and the algorithmic speedup is about 3×. The 15× therefore stems mainly from replacing SAM rather than from the method. SPAM with the same lightweight prior should be reported for accuracy, regularity and runtime, and the abstract and introduction revised accordingly.

  1. The description is unclear and several definitions and details are missing

The statement of the second contribution is inconsistent with the ablation. The contribution list claims "a joint object features architecture ... enabling more consistent and structure-aware assignments", and Section 3.3 presents FiLM fusion as the core design; yet Appendix B.1 / Figure 13 shows that the Add, Concat and FiLM curves nearly coincide, and the authors themselves write that the gains "primarily stem from the availability of object-level information rather than from a specific fusion mechanism". A more accurate statement would be that object-encoder features can be injected into a feed-forward superpixel network and that the result is insensitive to the fusion mechanism. In addition, several details that affect understanding and reproduction are not given: which parameters the Reg variants fine-tune, and whether the oracle maps replace or complement the ground-truth supervision; the description of masked correction is inconsistent, as the main text propagates neighbouring superpixel labels iteratively until convergence, whereas Figure 16 in Appendix B.3 describes a reassignment among the 8 neighbouring seeds within the same object, which mentions no iteration and cannot produce the new labels the main text refers to; and whether the baselines are retrained or use official weights, and how the CPU and GPU timings in Figure 1 are aligned, are not stated.

  1. The accuracy advantage is not separated from the effects of regularity and superpixel count

The accuracy advantage of SPAM 2 over SPAM is only about 0.001 AUC ASA, and two factors could each account for a gap of this size; the paper rules out neither. The first is the difference in regularity. ASA and GR trade off against each other: the more closely superpixels follow object contours and the less regular their shapes, the higher the ASA and the lower the GR. SPAM 2 has GR = 0.284 on BSD, far below SPAM's 0.478, ranking 9th of 15 methods and below SLIC and CDS, so its ASA gain may stem mainly from less regular shapes. The paper's Reg experiment supports this interpretation: when GR is raised to about 0.40, still short of SPAM, ASA already falls back to SPAM's 0.9713, and SPAM 2 Fast Reg is even significantly worse than SPAM (Table 1, p = 0.009). This indicates that SPAM 2 occupies a different accuracy–regularity trade-off point rather than being more accurate at comparable regularity. The second is the actual number of output superpixels. Section 4.1 states that metrics are reported against the number of generated superpixels, whereas several captions refer to the requested number; the two are inconsistent. Masked correction creates new labels for objects without a seed, so the actual count exceeds the requested one; since ASA increases monotonically with the count, aligning methods at the requested number systematically favours those that output more superpixels, by a margin large enough to cover a 0.001 gap. The authors should state which axis definition is used and report, for each method, the deviation between actual and requested counts.

Questions for authors
  1. What are the AUC ASA, GR and runtime of SPAM when it uses the same EfficientViT + MobileSAMv2 prior as SPAM 2 Fast? What is the actual speedup of SPAM 2 over SPAM under an identical prior?
  2. Are all curves, AUCs and the K = 500 interpolation computed at the requested K or at the actual number of output superpixels? How large is the deviation between the two for each method?
  3. What is the per-image Wilcoxon result for the full model versus "w/o joint" in Figure 7? Since "w/o joint, w/o mask" reuses the CDS architecture, loss and training recipe, can it be read as a retrained CDS? If so, how far is it from the 0.9679 of CDS in Table 5, and does Table 5 use the official CDS weights or a retrained model?
  4. Can simply increasing w_p reach an ASA–GR trade-off similar to the Reg variants? Which parameters are fine-tuned for the Reg variants?
2 comments

We thank the reviewer for their detailed review, which helped us clarify the attribution of the speedup, the evaluation protocol, and several implementation details.

W1. Q1. Speedup: architecture vs. object prior. We agree that the attribution should be clearer, since both methods can use a lightweight prior. The reported 15×15\times end-to-end speedup results from both a lighter object model and faster superpixel generation, while also achieving a higher ASA. The updated Fig. 5 now includes SPAM with the same EfficientViT + MobileSAMv2 prior as SPAM 2 Fast: in this matched comparison, SPAM 2 Fast is roughly three times faster and achieves a significantly higher ASA. Appendix B.4 additionally isolates the cost of the superpixel stage. We have made this distinction explicit in the abstract and the introduction.

W2. Clarified contributions, definitions and implementation details. As the reviewer notes, the gain comes from the object features themselves, with similar results across fusion mechanisms; FiLM is simply our default implementation. The Reg variants fine-tune only the fusion module, with oracle maps replacing the object annotations. Masked correction first unassigns pixels that are inconsistent with the dominant object of their superpixel, then iteratively propagates the labels of valid neighboring pixels within the same object until no further reassignment is possible. An object without any seed receives a new label, and ambiguous pixels of the prior keep their initial assignment. We have updated Appendix B.3 accordingly. CDS was retrained with the authors' code, and this same model serves as the starting point for every fine-tuning in our experiments. In Figure 1, only SLIC runs on CPU; all other methods use their official repositories and run on the same RTX A4000 GPU.

W3. Q2. Fair evaluation at the actual number of superpixels. We thank the reviewer for pointing out this mistake in the captions. Accuracy curves, AUCs, and per-image interpolation are all computed with respect to the actual number of output superpixels, including the labels introduced by masked correction. The ASA comparison therefore already accounts for differences in the number of generated superpixels. We have harmonized the captions and clarified the scalability plots. Regularity remains a separate factor: these results compare different accuracy-regularity operating points and do not establish superiority at matched GR. We discuss regularity in more detail in our answer to Q4.

Q3. Significance of the ablation. We have added statistical significance tests on ASA between all ablation variants (Table 4). Each component also yields a significant gain when added individually to CDS.

Q4. Oracle supervision versus a larger position weight. To more fairly compare with the baseline CDS architecture, we initially used the same wpw_p parameter. We have added the comparison with an increased wpw_p for SPAM 2 Fast (Figs. 8 and 9, and Appendix Fig. 18). Increasing wpw_p logically improves regularity at the expense of accuracy and contour adherence. At similar ASA, oracle supervision yields a higher GR up to about 650 superpixels, with slightly lower contour adherence. The two strategies also produce different shapes: with wpw_p alone, superpixels surprisingly tend to have similar shapes in object-free homogeneous regions but noisy contours, whereas oracle supervision yields very smooth boundaries and still adapt to the image content in such regions due to the denser training signal (Figs. 9 and 18). Since GR includes a shape-consistency term, it rewards similar shapes within a decomposition, which likely explains the similar GR performance between the noisy superpixels from increased wpw_p and the ones obtained with oracle supervision that qualitatively seem more regular. Finally, note that only the fusion module is fine-tuned for the Reg variants. Since it requires an additional fine-tuning, we will add the corresponding comparison for SPAM 2 Reg in the final version.

Thank you for the detailed response and the additional experiments. The matched comparison with SPAM using the same EfficientViT + MobileSAMv2 prior, together with the revised abstract and introduction, resolves my main concern about the attribution of the speedup. The clarifications on masked correction, the Reg fine-tuning and the evaluation at the actual number of superpixels also address my questions on the method description and the evaluation protocol, and the added significance tests and w_p comparison are helpful. One limitation remains: the results show a different accuracy–regularity trade-off rather than higher accuracy at matched regularity, which the authors also acknowledge. I encourage the authors to state this clearly in the final version and to add the promised w_p comparison for SPAM 2 Reg. Based on the rebuttal, my rating is weak acceptance. Note that my original rating of 4 was selected by mistake.

Reviewer_CEv2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SPAM 2, an efficient object-guided superpixel segmentation framework. The method combines the local pixel-to-seed assignment network from CDS with high-level object features and masks provided by SAM. SPAM 2 directly predicts superpixel assignments through a feed-forward decoder, making it faster than SPAM, which relies on iterative clustering. In addition, the proposed method uses FiLM to fuse features from the superpixel encoder and the foundation model's object encoder, and applies masked correction to reduce superpixels crossing object boundaries. This paper also introduces an oracle superpixel supervision strategy. SPAM is used to generate pseudo-superpixel labels from ground-truth annotations, which are then used to further fine-tune SPAM 2 to improve superpixel regularity. The proposed method is evaluated on five datasets, with comparisons covering segmentation accuracy, boundary quality, regularity, and inference time. The experiments also include ablation studies, statistical significance tests, and a downstream semantic segmentation task. The reported results show improved accuracy on several datasets and reduced inference time.

Critical strengths

S1. Improved segmentation accuracy and efficiency: SPAM 2 replaces the iterative clustering used in SPAM with feed-forward local assignment, reducing the computational cost of object-guided superpixel generation. By combining object model features with masked correction, SPAM 2 achieves higher segmentation accuracy on several datasets. SPAM 2 Fast further reduces end-to-end inference time, improving efficiency while maintaining competitive accuracy.

S2. Comprehensive evaluation across multiple datasets and settings: The paper evaluates the method on five datasets and across different superpixel scales, using metrics including segmentation accuracy, contour detection, regularity, and runtime. It also provides component ablations, statistical significance tests, and downstream semantic segmentation results, which help assess the method under different settings.

Critical weaknesses

W1. Limited methodological novelty: Compared with SPAM, this paper integrates object features into the superpixel pipeline rather than using object information only as an additional constraint. However, the method is mainly built upon existing work, including the local pixel-to-seed assignment and training objective from CDS, object-guided superpixel segmentation from SPAM, and FiLM-based feature fusion. The overall design is relatively straightforward and appears to be an integration of existing methods rather than the introduction of a new and significant method.

W2. Limited segmentation accuracy gains with a substantial reduction in regularity: SPAM 2 improves ASA, but some of the gains over SPAM are marginal. For example, on BSD, ASA AUC increases from 0.9713 to 0.9725, a difference of only 0.0012. On the other hand, the global regularity of SPAM 2 decreases considerably. On BSD, GR AUC drops from 0.4777 for SPAM to 0.2842 for SPAM 2. Even with oracle supervision, SPAM 2 Reg reaches a GR AUC of only 0.3999. Similar issues can be observed on other datasets. For example, on DRIVE, SPAM 2 achieves a GR AUC of 0.3205, ranking only 13th among the 15 methods and variants listed in Appendix Table 6. Although the paper achieves competitive results in segmentation accuracy and some contour detection metrics, these results come with a substantial accuracy-regularity trade-off. Therefore, when considering overall superpixel quality rather than ASA alone, it is not clear whether SPAM 2 achieves comprehensive state-of-the-art performance.

Questions for authors

Q1. Conceptual contribution of oracle superpixel supervision: The authors state in the conclusion that oracle pseudo-supervision provides new insights into how object-level knowledge can be distilled into superpixel formation. However, the method described in Section 3.4 primarily uses SPAM with ground-truth region annotations to generate denser, more regular pseudo-superpixel labels, which are then used to fine-tune SPAM 2.

Although the experiments show that this strategy improves regularity, it remains unclear whether the improvement comes from object-level knowledge transfer or simply from using more regular training targets. Would similar improvements be obtained using other superpixel generators, such as SLIC, SSN, or CDS constrained by the same ground-truth regions, or by increasing the weight of the position regularization loss?

Q2. Choice of SAM over SAM 2: The paper states that SAM 2 produces weaker object priors but provides no corresponding quantitative comparison, offering only a brief textual explanation. Since SAM 2 also supports mask generation for static images and uses an updated image encoder, could the authors compare SAM and SAM 2 under the same protocol? In particular, it would be useful to report their superpixel segmentation accuracy, regularity, and inference time to support the choice of backbone.

Q3. Effects of individual components on contour detection, regularity, and runtime: Figure 7 provides ablation studies on feature fusion and masked correction, but the comparisons mainly focus on segmentation accuracy. Could the authors report the corresponding contour detection (BR/BP), regularity (GR), and runtime results under the same settings? This would help clarify whether these components improve other superpixel quality metrics beyond ASA and whether they introduce additional computational costs.

Q4. Object consistency and robustness to object prior errors: Section 3.2 describes masked correction as a leakage-reduction procedure rather than a strict object constraint over the entire image. Since preventing cross-object assignments is one of the main motivations of the method, could the authors quantify cross-object leakage before and after masked correction to directly evaluate its effectiveness? In addition, the results on DRIVE suggest that the object priors provided by the foundation model may be less reliable in other image domains. How much is SPAM 2 affected when object masks are inaccurate, some objects are not detected, or object proposals overlap substantially?

Q5: Implementation details of the CDS-derived architecture and training procedure: Section 3.1 mentions using Sobel-filtered images as an auxiliary modality and adopting the content-disentanglement objective from CDS, but Figure 2 does not show this auxiliary training branch. Could the authors further clarify the relationship between the superpixel encoder, the auxiliary branch, and the CDS architecture? In particular, are the relevant CDS components fully retained, and how do the two modalities participate in training? In addition, only the feature fusion module is updated during Stage 2. Could the authors clarify which loss terms in the CDS objective contribute gradients to the fusion parameters, particularly the content-alignment loss and mutual-information loss? Finally, which parameters are updated during oracle-supervision fine-tuning, and is the same training objective used?

Q6. GPU memory usage and computational resources: The paper primarily evaluates efficiency in terms of runtime but does not report GPU memory usage or peak VRAM consumption for the compared methods. Since SPAM 2 additionally uses a foundation object model, its memory requirements may be higher than those of superpixel methods that do not rely on foundation models, such as CDS or SLIC. Could the authors report the GPU memory usage of SPAM 2, SPAM 2 Fast, and the main baselines under the same hardware, image resolution, and inference settings? Reporting model size or computational cost, such as FLOPs, would also help provide a more complete evaluation of the practical resource requirements. Given the paper's emphasis on efficiency and practical applications, memory consumption is also an important consideration beyond inference time.

2 comments

We thank the reviewer for their careful analysis and for their suggestions to make the comparisons more explicit.

W1. Novelty: three complementary contributions. SPAM 2 builds on existing ideas but combines carefully designed components: it replaces the iterative clustering of SPAM with a trainable local assignment, fuses low-level and object features, significantly improving accuracy, and introduces the first knowledge-distillation scheme for superpixels. To further improve computational efficiency, we also investigate several encoder/decoder configurations of the object model (Fig. 5), leading to the lightweight SPAM 2 Fast. We also provide an extensive validation on 5 datasets, introducing for the first time, statistical significance tests (see answer W2), as well as a downstream task.

W2. Statistically significant accuracy gains and the accuracy-regularity trade-off. We agree that ASA scores have historically been numerically close across superpixel methods; superiority is therefore typically assessed through the consistency of the accuracy curves across multiple datasets and scales (Figs.8 and 9). This is why we report results on 5 datasets, detailed curves, AUC scores, and a downstream-task experiment. We also provide per-image distribution statistics (variance, box plots) for each dataset in Appendix C.2 (Fig.19). In addition, we provide, for the first time, per-image paired significance tests: whenever SPAM 2 and SPAM 2 Fast achieve a higher ASA than all competing methods, including SPAM, the difference is statistically significant. Regarding regularity, SPAM 2 was primarily designed to reduce computation time while maintaining SPAM-level object accuracy, as measured by ASA, the reference metric. Part of the regularity can also be recovered at no additional inference cost through oracle supervision (SPAM 2 Reg, Sec.3.4).

Q1. Oracle supervision: a new superpixel-level training signal. Superpixel methods have historically been trained on fine-grained manual object annotations (mostly BSD). With oracle supervision, a superpixel method is, for the first time, trained on superpixel-level ground truth. To avoid significant modifications or retrainings, we simply derived this ground-truth from SPAM's object-based regular partitioning, constrained by the annotated object boundaries. These oracle maps replace the BSD annotations during fine-tuning; only the fusion module is updated, with the same objective and position weight as in CDS and SPAM 2. SPAM generates these targets offline and is not used at inference. In the low-to-intermediate superpixel range and at similar ASA, oracle supervision yields a higher GR than increasing the position weight (Fig.8). The resulting shapes also differ: with a larger position weight alone, superpixels have similar shapes but noisy contours, whereas oracle supervision yields smooth boundaries (Fig.9 and Appendix Fig.18). The transferred information therefore goes beyond a stronger compactness constraint.

Q2. Quantitative comparison with SAM 2. We thank the reviewer for suggesting this comparison. The updated Fig. 5 now includes SAM 2-Hiera-L under the same BSD evaluation protocol. Its lower ASA in this setting supports our choice of the SAM configuration.

Q3. Ablation extended to contours, regularity and runtime. Following this suggestion, we have extended Fig.7 with contour adherence (summarized by BR×BP\sqrt{\mathrm{BR}\times\mathrm{BP}}) and regularity (GR). Both components improve contour adherence. Masked correction mainly helps at coarse scales, whereas joint features reduce regularity at finer scales. Feature fusion adds only 0.5ms per image, as it reuses the object-encoder features (Appendix B.4).

Q4. Quantified object consistency and robustness to prior errors. We now quantify cross-object leakage with respect to the prior (Appendix B.3, Table 5). This effect is naturally larger at low superpixel counts, where larger superpixels are more likely to span several objects. This is also where the accuracy gain over CDS, which uses no object guidance, is the largest (Fig. 7). After correction, superpixels are consistent with the prior by construction. When the prior fails, objects are mostly missed rather than wrongly segmented: the masked correction is then inactive and SPAM 2 relies on its learned local assignment. And as presented in Fig.7, performance w/o mask are still good. On DRIVE, SPAM 2 thus remains close to CDS, whereas SPAM's performance is strongly affected (Appendix C.1).

Q5. Clarified architecture and training procedure. We retain the CDS architecture: RGB and Sobel encoders, a shared content/style gate, and a shared decoder with a local assignment head. In both branches, the content features go through the fusion module, conditioned on the same object features. The Sobel branch is only used during training. Since we did not modify the training loss, Fig. 2 only depicts the inference path. During fine-tuning (Stage 2), the superpixel encoders and decoder are frozen. The alignment and mutual-information losses are computed before fusion and therefore provide no gradient to the fusion parameters. Only the superpixel losses of both branches provide gradients, which update the fusion module alone. Oracle fine-tuning follows the same setup, with oracle maps used as ground-truth labels. We modified the paper to integrate these clarifications.

Q6. GPU memory and computational resources. We agree that memory matters in practice, and that the object model makes SPAM 2 heavier than methods such as SLIC or CDS. This cost, however, is what enables the accuracy gain: it allows SPAM 2 to leverage the object knowledge of current segmentation foundation models, as do many other methods. The superpixel model itself remains lightweight, so the memory footprint is dominated by the chosen object model. We added a comparison of the peak GPU memory used in the Appendix B.5.

Comments on the paper

1 comment

We thank the reviewers for their careful reading and constructive comments. Below, we clarify several points and describe what we have added or modified to the paper identified in blue color.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.