- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes PACE, a framework that treats requested slot count as an ensemble axis for object-centric segmentation. Prefix training enables one model to produce decompositions at multiple slot counts. A training-free consensus readout converts soft masks into slot-invariant pairwise patch co-membership, averages this evidence, and applies average-linkage hierarchical clustering to obtain a segmentation at a chosen output region count.
Experiments on COCO, VOC2012, and MOVi-C demonstrate improved foreground grouping over single-cardinality predictions. Comparisons with single-cardinality hierarchies, repeated stochastic inference, separately trained models, and resource-matched controls support the benefit of complementary decompositions. Ablations examine population composition and readout choices. Improvements are strongest in FG-ARI; overlap gains and advantages over published methods are less consistent.
Critical strengths
-
A meaningful perspective on slot cardinality. The paper shows that different slot counts can supply complementary evidence rather than merely represent competing hyperparameter choices. Separating evidence-generation cardinality from output granularity provides a useful framework for object-centric segmentation.
-
Strong controls for the central empirical claim. Comparing against single-cardinality hierarchies at the same output count, repeated stochastic inference, and resource-matched alternatives helps distinguish consensus gains from postprocessing or additional computation. The consistent FG-ARI improvements across three datasets make the central finding credible.
Critical weaknesses
-
Limited methodological novelty beyond established consensus clustering. Pairwise co-association, evidence averaging, and hierarchical consolidation closely follow existing cluster-ensemble principles, including soft clustering ensembles. The contribution therefore rests primarily on applying cardinality diversity within object-centric learning. A sharper account of what the prefix construction contributes beyond general variable-cardinality training would strengthen the novelty claim.
-
The demonstrated benefit has a restricted scope. Most validation uses DINOv2 ViT-S/14 with a DINOSAUR-derived MLP decoder. Results establish improved foreground grouping, but do not establish broadly superior segmentation: overlap gains are inconsistent, and several published methods remain stronger on VOC2012 or overlap metrics. Limited replication of attribution experiments and the absence of qualitative failure analysis further constrain conclusions about generality.
Questions for authors
The following are clarification requests that could affect my assessment:
-
Have you compared ordered-prefix training with variable-cardinality training using a shared Gaussian initializer or randomly selected initializer subsets? Such evidence would clarify whether the specific prefix construction is important, beyond exposure to multiple cardinalities.
-
How stable are the Figure 5 attribution results across training seeds? The reported image-bootstrap intervals do not capture training variability, which matters for the claim that one prefix-trained model approximates separately trained cardinality-specific models.
-
Appendix D.3 lists sampling probabilities summing to 0.70 rather than 1. Were these normalized weights, or is a value or cardinality missing? Please also clarify why COCO evaluation uses 4,946 images and identify the population configuration corresponding to the headline improvements.
-
Are there existing results on another decoder or backbone that support the claimed applicability beyond the primary configuration? If unavailable, please clarify the intended scope of the conclusions.
5 comments
W2. Investigation scope. Thanks for pointing this out. We have expanded the evaluation, to clarify where PACE helps and extend the evidence that complementary decompositions can improve object-centric segmentation across multiple model configurations and evaluation criteria. The main results are reported here.
Backbones and decoders. Additional experiments extend the grouping gains to other encoders and a transformer decoder; see our response to Q4.
Training variability. We repeated the prefix-attribution experiments across three training seeds; see Q2.
Qualitative analysis. We added qualitative comparisons showing individual cardinalities and their consensus (Appendix C.2), alongside failure cases in crowded scenes (Appendix G.3).
Segmentation quality. We agree that the results do not establish broadly superior segmentation over existing methods. Our contribution is to improve a given object-centric model by combining complementary decompositions. We expanded the evaluation with instance- and class-level overlap, foreground mIoU, class-agnostic foreground PQ, boundary metrics, and SO/PO/GO analysis as suggested by FHk9 (Appendix G).
The table below report the relevant results on COCO (bold: best; underline: second best). PACE improves all five metrics over the same checkpoint’s single prediction and repeated inference.
| Method | FG-ARI ↑ | mBO$_i$ ↑ | mBO$_c$ ↑ | fg-mIoU ↑ | PQ ↑ |
|---|---|---|---|---|---|
| SPOT | 36.8 | 34.0 | 44.2 | 31.7 | 38.1 |
| AdaSlot | 38.4 | 26.6 | 30.6 | 25.2 | 38.3 |
| DINOSAUR-prefix, single | 39.6 | 27.4 | 31.7 | 26.0 | 38.0 |
| DINOSAUR-prefix, 3 restarts | 27.4 | 31.9 | 25.9 | ||
| DINOSAUR-prefix, PACE | 44.9 | 44.3 |
The gains also persist in multi-object scenes: excluding single-object COCO images retains a 4.8 point FG-ARI improvement. Overlap gains vary with object count and output granularity, with adjusted hierarchy cuts recovering gains in crowded scenes. The SO/PO/GO analysis finds fewer group-of-objects regions but more part-of-object regions; area weighting shows less area in object parts at matched group-of-objects area. Boundary accuracy does not improve in the default configuration (Appendices G.1–G.2).
Across datasets, FG-ARI and PQ improve over the single-prediction baseline, while overlap improves on COCO and VOC2012 but decreases slightly on MOVi-C. Published methods retain stronger overlap scores on all three datasets (Appendix G.1).
Q3. Manuscript fixes. Thank you for identifying these ambiguities. We have clarified all three in the revised manuscript.
-
Sampling probabilities (Appendix D.3): The original text ambiguously described the training cardinalities. Training samples all ten cardinalities , whereas PACE evaluates a subset at inference, e.g., . The endpoint-weighted probabilities are , , and for each of the eight intermediate cardinalities and sum to 1.
-
COCO evaluation (Appendix B.4): Of the 5,000 validation images, 54 contain no annotated foreground objects, which leaves 4,946 eligible for foreground evaluation.
-
Headline configuration (Appendix B.4): The reported improvements use , without reflections, cut at , compared against the raw prediction from each corresponding checkpoint and averaged over three training seeds. The revised FG-ARI gains of 5.27, 4.33, and 5.61 points use uniform slot-count sampling during training; previous values used weighted sampling.
Q4. Decoder and backbone ablations. We added COCO experiments with larger DINOv2 encoders, the original DINO encoder, and a four-layer non-autoregressive transformer decoder. Each configuration uses three training seeds and 40,000 training steps. The table compares PACE (, ) with each model’s own prediction, reporting means ± standard deviation across seeds (Appendix H).
| Backbone | Decoder | FG-ARI | PACE FG-ARI | mBO$_i$ | PACE mBO$_i$ |
|---|---|---|---|---|---|
| DINOv2 ViT-B/14 | MLP | 37.3 ± 0.1 | 41.5 ± 0.2 | 26.4 ± 0.1 | 27.2 ± 0.1 |
| DINOv2 ViT-L/14 | MLP | 37.2 ± 0.2 | 41.6 ± 0.1 | 26.2 ± 0.1 | 26.8 ± 0.1 |
| DINO ViT-B/16 | MLP | 36.2 ± 0.1 | 39.7 ± 0.2 | 26.4 ± 0.0 | 27.7 ± 0.0 |
| DINOv2 ViT-S/14 | Transformer | 37.3 ± 0.3 | 41.5 ± 0.1 | 26.5 ± 0.2 | 27.9 ± 0.0 |
PACE improves both grouping and overlap in every configuration, supporting applicability beyond the primary backbone and MLP decoder. It also improves FG-ARI by 1.6–4.1 points over the best single-cardinality hierarchy in each configuration, keeping the readout and output count matched (Appendix H).
Dear reviewer cTmD,
As the discussion period is approaching its end, we would appreciate any feedback on whether our responses address your concerns. Please let us know if there are remaining points we could clarify.
Thanks in advance for your time and contribution.
Thanks for your positive feedback on our work! We are grateful for your appreciation for using slot cardinality as an ensemble axis (shared by FHk9 and RcU3), the empirical setup (also highlighted by RcU3), and the positive evaluation on the manuscript in general. We also appreciate the constructive observations and questions raised, which we will address in the following responses.
W1. Methodological novelty. We agree that co-association and hierarchical consolidation are established techniques and explicitly credit them. Our contribution is to identify and experimentally characterize slot cardinality as a source of complementary grouping evidence. We do this in contrast with the main direction that the object-centric community has explored recently, which is unsupervised, dynamic inference of the slot count. We motivate this arguing that even when the slot count is correct, objects can still be split and merged sub-optimally. On the other hand, different cardinalities induce different merge–split errors. These multiple views can provide useful evidence about object relationships that would be discarded otherwise when committing to one decomposition.
PACE combines these alternatives from a single checkpoint through variable-cardinality training and a slot-invariant hierarchical readout that decouples evidence cardinality K from output regions L. The ablation in Q1 shows that ordered prefixes offer a small advantage but are not essential to consensus. On the other hand, training exposure to each cardinality is necessary: truncating a fixed-K=12 model yields 37.0 FG-ARI, versus 42.7 for one prefix-trained checkpoint and 43.3 for three separately trained models, at one-third of the training cost (Q2).
More broadly, PACE makes complementary decompositions an explicit design consideration in object-centric learning: alongside selecting cardinality, models can learn to produce useful alternatives and consolidate their evidence instead of committing to one decomposition.
Q1. Role of prefix training. To isolate the contribution of the ordered prefix training beyond exposure to multiple cardinalities, we added the requested shared-Gaussian and random-initializer-subset comparisons. All variants use the same variable-K schedule and differ only in initialization.
We evaluate six-cardinality PACE on COCO at L=7, using three 40,000-step training runs per variant (means ± standard deviation across training seeds; Appendix D.4).
PACE improves both metrics under every variant, showing that generating useful evidence from one checkpoint extends beyond our initialization choice. The ordered prefix training thus offers a simple, nested construction using the first K position-specific initializers and provides a small advantage in absolute scores, but is not required for the consensus benefit.
Q2. Seed stability. Thanks for your observation; we agree that image-bootstrap intervals do not capture training variability. We therefore repeated the Figure 5 attribution experiments with three training seeds per model, using the same 40,000-step training budget. The table reports means ± standard deviation on COCO at L=7 (Appendix D.5). Every FG-ARI comparison in Fig. 5 (Fig. 6 now) retains its direction across all three seeds. Prefix training outperforms fixed-K=7 training with the same inference cardinalities, and adding K=9,12 further improves both metrics. Simply truncating a model trained at K=12 does not reproduce these gains. One prefix-trained model remains close to the consensus of three separately trained models.