- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
The paper proposes SlotMask, an unsupervised object discovery method leveraging object-centric learning. In their work, the authors propose replacing the autoregressive decoder of DINOSAUR with a bi-directional masked decoder. They propose slot-aware initialization for the mask tokens instead of a shared mask embedding. This is further combined with a randomly sampled masking ratio and an attention-matching loss that aligns decoder cross-attention with the Slot Attention assignments. SlotMask outperforms relevant baselines and improves property prediction from frozen slots.
Critical strengths
- The problem of object-centric learning and unsupervised object discovery, and the authors propose an interesting approach to further progress this field
- The presented results are significant across various benchmarks among truly unsupervised methods (GLASS excluded)
- The authors present interesting and well-controlled comparisons and thorough ablations
- The paper is clearly and well written
Critical weaknesses
-
The actual source of the gains is unclear: The papers story is built around masked reconstruction and this is credited as the main factor that improves performance. However, from Table 3 it appears that the slot-aware initialization shows benefits independent of the decoder. With autoregressive decoding + slot-aware initialization the performance is at 46.4 mBOc, which is only 2.4 below SlotMask and already above SPOTv3. Without the slot-aware initialization, using masked reconstruction only drops below plain autoregressive decoding. So it appears as if masking only helps in combination with the slot-aware initialization. To make this more clear, the authors should provide a full 2x2 ablation (masked/AR x with/without slot-aware init) in the same implementation and potentially reframe their paper around the slot-aware initialization
-
The method has a risk of information leakage, since the assignment weights are computed from the feature of the masked token itself, so masked tokens partially encode their own target. This makes unclear if the gains come from the object-level signal or from leaking the target
-
The MOGENT* baseline is discussed, but never compared against
*MOGENT (Nakano et al., “Efficient Object-Centric Representation Learning Using Masked Generative Modeling”, TMLR 2025)
Questions for authors
-
For the autoregressive version in Table 3, how is the slot-aware initialization applied and trained? How does the autoregressive decoder perform without slot-aware initialization in the same implementation? This bring clarity about whether the gains are specific to masked reconstruction
-
Sec 3.2 details that the gains come from reconstruction gradients flowing into the assignments. What happens when the gradients through the assignments are stopped, experimented with and without the attention matching loss?
-
How does the performance of the model change when the mixture weights of a masked token are computed only from visible neighbouring tokens instead of its own feature?
2 comments
Response to Rebuttal
Dear authors,
Thanks for the thorough and nice-to-read rebuttal. Given the new experiments and presented analyzes, I'm willing to increase my score and support acceptance of the paper. Please make sure to include the presented results in the revised paper.
We thank the reviewer for the thoughtful review. The requested ablations helped us clarify better where the gains of SlotMask come from. All changes in the revised paper are marked in blue.
Q1 and W1: Source of the gains; how slot-aware initialization is applied in the AR variant.
In the AR variant, slot-aware initialization is applied in the same way as in SlotMask, but one position at a time: the input at the step that predicts position i is the previous ground-truth token plus the slot mixture ∑kai,ksk of position i. This is now explained in the "Design ablation" paragraph of Sec. 4.2. We also added an AR variant without slot-aware initialization to Table 3. The requested 2×2 comparison (masked / AR × with / without slot-aware initialization) is:
Slot-aware initialization improves both decoders, but the masked decoder benefits from it the most (+8.9 vs. +3.2 mBOc) and clearly outperforms its AR counterpart with slot-aware initialization, whereas without it the masked decoder performs worse than the AR one. Our contribution is therefore the combination of slot-aware token initialization, masked decoding and attention matching: together, these design choices enable masked decoding to outperform the standard autoregressive approach. This analysis is now in Appendix A.10.
Q2, Q3 and W2: Gradient path through the assignments and potential information leakage.
We ran the requested experiments on MS COCO:
Stopping the gradient through the slot mixture significantly decreases performance in both cases, which shows that it is an important gradient pathway. In contrast, computing the mixture weights from the neighbouring tokens only, excluding the token's own feature, decreases performance only slightly. This addresses the concern about information leakage (W2): when the token itself is excluded from the aggregation, the model still learns to reconstruct it by aggregating its neighbours into meaningful slots. These results are now in Appendix A.11.
W3: Comparison with MOGENT.
We did not initially include a direct comparison with MOGENT because it is evaluated on synthetic data at 64–128 px resolution with encoders trained from scratch, and its code is not publicly available. We re-implemented its decoder-side recipe from the original paper (shared mask token, cosine masking schedule, loss on masked tokens only) and trained it under our experimental protocol on MS COCO. It is now included in the ablations (last row of Table 3). It reaches 32.2 / 37.2 mBOi / mBOc, far below SlotMask (36.1 / 48.8).