acceptodds
Under review as a conference paper at ICLR 2027

SlotMask: Masked Feature Reconstruction for Object-Centric Representation Learning

Abstract

Humans perceive the world as a composition of distinct objects. Inspired by this, object-centric learning represents scenes as sets of object-level vectors called slots. Recent approaches learn slots by reconstructing pretrained visual features with slot-conditioned autoregressive transformers. We introduce SlotMask, which replaces causal reconstruction with a bidirectional transformer trained to predict randomly masked features. Central to our approach is tying masked reconstruction to the object structure discovered by Slot Attention: each masked token is initialized with a mixture of slots weighted by its Slot Attention assignments, and decoder cross-attention is trained to align with these assignments. Together with a randomly sampled masking ratio, these mechanisms make masked reconstruction an effective objective for learning object-centric representations. SlotMask achieves state-of-the-art unsupervised object discovery on PASCAL VOC, MS COCO, and MOVi-C/E, outperforming autoregressive baselines under matched backbones, same training budgets, and with cheaper decoding.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
object-centric learning, slot attention, masked modeling, self-supervised representation learning, unsupervised object discovery, feature reconstruction

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_bJZkOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper proposes SlotMask, an unsupervised object discovery method leveraging object-centric learning. In their work, the authors propose replacing the autoregressive decoder of DINOSAUR with a bi-directional masked decoder. They propose slot-aware initialization for the mask tokens instead of a shared mask embedding. This is further combined with a randomly sampled masking ratio and an attention-matching loss that aligns decoder cross-attention with the Slot Attention assignments. SlotMask outperforms relevant baselines and improves property prediction from frozen slots.

Critical strengths
  • The problem of object-centric learning and unsupervised object discovery, and the authors propose an interesting approach to further progress this field
  • The presented results are significant across various benchmarks among truly unsupervised methods (GLASS excluded)
  • The authors present interesting and well-controlled comparisons and thorough ablations
  • The paper is clearly and well written
Critical weaknesses
  • The actual source of the gains is unclear: The papers story is built around masked reconstruction and this is credited as the main factor that improves performance. However, from Table 3 it appears that the slot-aware initialization shows benefits independent of the decoder. With autoregressive decoding + slot-aware initialization the performance is at 46.4 mBOc, which is only 2.4 below SlotMask and already above SPOTv3. Without the slot-aware initialization, using masked reconstruction only drops below plain autoregressive decoding. So it appears as if masking only helps in combination with the slot-aware initialization. To make this more clear, the authors should provide a full 2x2 ablation (masked/AR x with/without slot-aware init) in the same implementation and potentially reframe their paper around the slot-aware initialization

  • The method has a risk of information leakage, since the assignment weights are computed from the feature of the masked token itself, so masked tokens partially encode their own target. This makes unclear if the gains come from the object-level signal or from leaking the target

  • The MOGENT* baseline is discussed, but never compared against

*MOGENT (Nakano et al., “Efficient Object-Centric Representation Learning Using Masked Generative Modeling”, TMLR 2025)

Questions for authors
  • For the autoregressive version in Table 3, how is the slot-aware initialization applied and trained? How does the autoregressive decoder perform without slot-aware initialization in the same implementation? This bring clarity about whether the gains are specific to masked reconstruction

  • Sec 3.2 details that the gains come from reconstruction gradients flowing into the assignments. What happens when the gradients through the assignments are stopped, experimented with and without the attention matching loss?

  • How does the performance of the model change when the mixture weights of a masked token are computed only from visible neighbouring tokens instead of its own feature?

2 comments

We thank the reviewer for the thoughtful review. The requested ablations helped us clarify better where the gains of SlotMask come from. All changes in the revised paper are marked in blue.

Q1 and W1: Source of the gains; how slot-aware initialization is applied in the AR variant.

In the AR variant, slot-aware initialization is applied in the same way as in SlotMask, but one position at a time: the input at the step that predicts position ii is the previous ground-truth token plus the slot mixture ∑kai,ksk\sum_k a_{i,k} s_k of position ii. This is now explained in the "Design ablation" paragraph of Sec. 4.2. We also added an AR variant without slot-aware initialization to Table 3. The requested 2×2 comparison (masked / AR × with / without slot-aware initialization) is:

DecoderSlot-aware initmBOi↑\text{mBO}_i \uparrowmBOc↑\text{mBO}_c \uparrow
Masked✓36.148.8
Masked✗34.739.9
AR✓35.046.4
AR✗34.643.2

Slot-aware initialization improves both decoders, but the masked decoder benefits from it the most (+8.9 vs. +3.2 mBOc\text{mBO}_c) and clearly outperforms its AR counterpart with slot-aware initialization, whereas without it the masked decoder performs worse than the AR one. Our contribution is therefore the combination of slot-aware token initialization, masked decoding and attention matching: together, these design choices enable masked decoding to outperform the standard autoregressive approach. This analysis is now in Appendix A.10.

Q2, Q3 and W2: Gradient path through the assignments and potential information leakage.

We ran the requested experiments on MS COCO:

Mixture weights of a masked tokenGrad.Lattn\mathcal{L}_{\text{attn}}mBOi↑\text{mBO}_i \uparrowmBOc↑\text{mBO}_c \uparrow
own assignments (SlotMask)✓✓36.148.8
own assignments, stop-gradient✗✓28.734.8
own assignments, stop-gradient✗✗33.738.9
neighbouring tokens only✓✓35.747.7

Stopping the gradient through the slot mixture significantly decreases performance in both cases, which shows that it is an important gradient pathway. In contrast, computing the mixture weights from the neighbouring tokens only, excluding the token's own feature, decreases performance only slightly. This addresses the concern about information leakage (W2): when the token itself is excluded from the aggregation, the model still learns to reconstruct it by aggregating its neighbours into meaningful slots. These results are now in Appendix A.11.

W3: Comparison with MOGENT.

We did not initially include a direct comparison with MOGENT because it is evaluated on synthetic data at 64–128 px resolution with encoders trained from scratch, and its code is not publicly available. We re-implemented its decoder-side recipe from the original paper (shared mask token, cosine masking schedule, loss on masked tokens only) and trained it under our experimental protocol on MS COCO. It is now included in the ablations (last row of Table 3). It reaches 32.2 / 37.2 mBOi\text{mBO}_i / mBOc\text{mBO}_c, far below SlotMask (36.1 / 48.8).

Reviewer_bJZkOpenReview ↗

Response to Rebuttal

Dear authors,

Thanks for the thorough and nice-to-read rebuttal. Given the new experiments and presented analyzes, I'm willing to increase my score and support acceptance of the paper. Please make sure to include the presented results in the revised paper.

Reviewer_v39POpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SlotMask replaces the causal AR decoder of DINOSAUR/SPOT with a bidirectional masked decoder. Masked tokens are initialized as SA-assignment-weighted slot mixtures, and an attention-matching loss aligns ASAA_\text{SA} with ACAA_\text{CA}. The paper reports SOTA mBO/mIoU on COCO, VOC and MOVi-C/E.

Critical strengths
  • Slot-aware initialization is simple, and its ablation effect is large (-8.9 mBOc).
  • Baselines are re-trained under a controlled DINOv3 setting, and code is provided.
Critical weaknesses

W1. Missing in-place citations in Section 3. Several components in Section 3 are not cited where they are introduced, even though the relevant works are cited elsewhere in the paper. This makes it hard to tell prior contributions from the authors' own. Building on prior work is not a weakness, but it should be made explicit:

  • The masked decoding in Sec. 3.2 is closely related to the random autoregressive decoder of DIAS, and also to MaskGIT/MAR. This should be stated in Sec. 3.2.
  • The slot mixture in Eq. (4) is the mixture formulation of the original SA decoder with spatial broadcast decoding (Locatello et al., 2020; Watters et al., 2019). The end of Sec. 3.2 mentions this, but it should be cited where Eq. (4) is introduced.
  • The slot-conditioned transformer decoder is also used by SLATE, STEVE, SPOT and DIAS. Despite technical differences, at the level of this subsection it is the same design, and these works should be cited.
  • By contrast, the citations for Lrec\mathcal{L} _ \text{rec} and Lattn\mathcal{L} _ \text{attn} are appropriate.

W2. Incomplete evaluation metrics. Only mBO (plus mIoU on MOVi) is reported; ARI and FG-ARI are omitted entirely. These metrics are not interchangeable:

  • ARI and FG-ARI are pair-counting measures, so they are effectively weighted by region area. They mainly reflect large background regions (ARI) or large objects (FG-ARI), which is what is most visible to a human observer.
  • mBO averages the best-overlap IoU over ground-truth masks. Every object gets equal weight regardless of size, and redundant slots are not penalized.
  • A method can therefore raise mBO by delineating small and medium objects more finely while fragmenting large objects or the background. Only ARI/FG-ARI would expose this.

In my experience, designs that feed SA assignments directly into the decoder input can easily raise mBO and mIoU, but at a substantial cost in FG-ARI and some cost in ARI. This is especially relevant here: the gains are concentrated in mBOc, and A.6 shows a clear change in slot usage. If FG-ARI is a concern because it can favor over-segmentation, it should be reported alongside mBO, not omitted. Please report ARI, FG-ARI, mBO and mIoU on all datasets. No need to re-train the models.

W3. Missing comparison with recent state of the art. The baselines stop at 2024 - early 2025. The most recent SA-based methods are not compared: RandSF.Q (video OCL; Zhao et al., AAAI 2026) and SmoothSA (image and video OCL; Zhao et al., ICML 2026). Kim et al. (2024b) and Tian et al. (2025), which the paper already cites, are also not compared.

RandSF.Q and SmoothSA provide a strong open-source basis for such comparisons:

  • standardized universal random seeds for both baselines and themselves;
  • a unified fair training protocol for all methods;
  • training logs and checkpoints for every method and seed are released;
  • a modular codebase that should make it straightforward to re-run them with the DINOv3-B backbone used here.

W4. Orthogonality to these methods; testing the combination would strengthen the paper. SlotMask modifies the decoder: how masked tokens are constructed and how decoder attention is aligned with SA. SmoothSA modifies SA query initialization, and RandSF.Q modifies inter-frame query prediction. The changes are orthogonal and could be combined, potentially setting a new state of the art.

  • If gains compound, SlotMask is a general decoder-side improvement.
  • If they do not, an explanation would clarify what problem slot-aware initialization actually addresses.

Either way, including the latest state of the art would make this work a more useful reference.

W5. Outdated evaluation protocol for video. The method applies directly to video OCL: per-frame slot-aware initialization plus temporal slot propagation. Yet MOVi-C/E are evaluated as independent frames, a protocol common in earlier work but no longer standard. The method should be evaluated in the standard video OCL setting, as SmoothSA does for both images and video, and compared with video methods such as RandSF.Q. This would substantially increase the paper's significance.

6. Ablation/framing. There is no λ=0 ablation. Moreover, masked tokens receive their own soft assignment, computed from the unmasked features. Fixed ρ=1 still reaches 42.5 mBOc, and AR + slot init (46.4) already beats SPOTv3, so the gains stem mainly from assignment-routed inputs, rather than from masked modeling as the title suggests.


Questions for authors
  1. How is slot init applied in the AR variant?
  2. At inference, are unmasked positions filled with e^i\hat e_i or with eie_i?
  3. Why does DINOv1 -> v3 give SlotMask +11 mBOc on VOC but SPOT only +2? Are the DINOv1 baseline rows actually matched?
3 comments

We thank the reviewer for the detailed and constructive review. The suggestions led to a substantially stronger paper. All changes in the revised paper are marked in blue.

Q1: How is slot-aware initialization applied in the AR variant?

It is applied in the same way as in SlotMask, but one position at a time: the input at the step that predicts position ii is the previous ground-truth token plus the slot mixture ∑kai,ksk\sum_k a_{i,k} s_k of position ii. This is now explained in the "Design ablation" paragraph of Sec. 4.2.

Q2: At inference, are unmasked positions filled with e^\hat{e} or ee?

With ee (teacher forcing), which is standard for object discovery with slot-conditioned decoders and is also used by DINOSAUR and SPOT. Feeding back the predictions e^\hat{e} over the 64 steps instead gives 35.6 / 48.1 mBOi\text{mBO}_i / mBOc\text{mBO}_c on COCO, against 36.1 / 48.8 with ee, so the masks barely depend on this choice. This is now discussed in Appendix A.12.

Q3: Why does DINOv1 → DINOv3 give SlotMask +11 mBOc\text{mBO}_c on VOC but SPOT only +2?

Yes, the training and evaluation setups are completely matched. Our explanation is that an AR decoder predicts each token from all preceding ground-truth tokens, so better features improve its reconstruction without requiring better slots. A masked decoder sees less context and has to predict many tokens at once, so it can exploit better features mainly through better slots. We added this discussion to Appendix A.2.

W1: Missing in-place citations in Section 3.

We thank the reviewer for this comment; it is fixed. Sec. 3.2 now cites MaskGIT, MAR and DIAS where masked decoding is introduced, including a discussion of random-order decoders such as DIAS, and cites SLATE, STEVE, DINOSAUR, SPOT and DIAS for the slot-conditioned transformer decoder.

We would also like to clarify that slot-aware token initialization differs from the mixture of a spatial broadcast decoder. A spatial broadcast decoder decodes each slot independently over all positions and only then mixes the per-slot outputs. We instead mix the slots at the input of the decoder, at the token level, and only for the masked tokens.

W2: ARI and FG-ARI are missing.

We now report ARI and FG-ARI for COCO and VOC, together with additional baselines (Appendix A.8). Original published numbers are reported where available. Best value per column in bold, second best in italics.

MethodCOCO ARICOCO FG-ARICOCO mBOi\text{mBO}_iCOCO mBOc\text{mBO}_cVOC ARIVOC FG-ARIVOC mBOi\text{mBO}_iVOC mBOc\text{mBO}_c
SA–21.417.219.2–12.324.624.9
DINOSAUR (MLP)–40.527.730.9–24.639.540.9
SLATE–32.529.133.6–15.635.941.5
DINOSAUR (AR)–34.332.338.8–24.844.051.2
Self-modulating SA–37.433.040.3–26.743.951.0
SPOT28.3**37.035.044.730.2**19.948.355.6
SPOT+FS+RC––35.745.3––49.356.5
SlotAdapt–41.435.139.2–29.651.551.9
GLASS*49.3**34.140.848.5–22.558.962.2
DIAS25.641.231.7–30.933.543.4–
SmoothSA29.341.333.4–35.033.645.2–
DINOSAUR$_{v3}$25.937.134.742.334.726.146.053.3
SlotMask (ours)34.635.236.148.857.324.858.669.8

* weakly supervised. ** not reported in the original paper; computed by us from the official checkpoint. "–": not reported, and no checkpoint is available for evaluation.

W3: Missing comparison with recent state of the art.

We extended Table 1 with more recent baselines: Self-modulating SA, SmoothSA, DIAS, and the stronger SPOT variant of Tian et al. (2025) with Fusion Stage and Region Combination (SPOT+FS+RC). Some of the new baselines outperform the older ones, but SlotMask retains its lead.

W4: Combination with SmoothSA / RandSF.Q.

Following the reviewer's suggestion, we trained SlotMask with the query preheating of SmoothSA on MS COCO:

ModelmBOi\text{mBO}_imBOc\text{mBO}_cFG-ARIARI
SlotMask36.148.835.234.6
SlotMask + SmoothSA query preheating36.649.131.438.1

Although our work focuses on the decoder, this shows that SlotMask benefits from orthogonal improvements to Slot Attention, which we agree is important to demonstrate. This result is now included in the revised paper (Appendix A.9). For the combination with RandSF.Q and SmoothSA on video, see W5.

W5: Video evaluation protocol.

We ran video experiments on MOVi-C following the RandSF.Q / SmoothSA protocol and their public code bases, replacing their decoder with the SlotMask decoder:

Method (MOVi-C)ARIFG-ARImBOmIoU
RandSF.Q65.467.429.226.8
RandSF.Q + SlotMask decoder74.264.530.728.2
SmoothSA50.969.031.730.2
SmoothSA + SlotMask decoder71.566.232.030.8

As on images, the SlotMask decoder improves ARI, mBO and mIoU, at the cost of a minor decrease in FG-ARI. We thank the reviewer for suggesting these experiments, which are now included in the main paper (Table 6, Sec. 4.2).

W6: λ=0\lambda = 0 ablation and framing.

The λ=0\lambda = 0 run is now included in the ablation table of the main paper (Table 3). Regarding the roles of slot-aware initialization and masked decoding: slot-aware initialization indeed also improves the AR decoder, but the masked decoder benefits from it much more (+8.9 vs. +3.2 mBOc\text{mBO}_c), and only with slot-aware initialization does masked decoding outperform AR decoding. The contribution is therefore the combination of slot-aware token initialization, masked decoding and attention matching.

Thank the authors. The new video-protocol and combination experiments address most of my concerns. However, the added results reveal a consistent trade-off (ARI up, FG-ARI down) that should be discussed explicitly in the main text rather than described as minor, and the state-of-the-art claim should be qualified accordingly. Therefore, I am raising my rating to weak acceptance.

Reviewer_qKmiOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposed a method for slot attention-based object-centric representation learning. The high-level idea is to use ideas from masked representation learning. The main contribution lies in how to transfer these ideas to slot attention, specifically, on how the masked tokens are formed, especially how they are initialized. Further, the sampling strategy has to be adapted from the fixed one as used in masked autoencoders. The approach is validated on the common datasets Pascal VOC, MS COCO, and MOViC/E, using common metrics such as object discovery and object property prediction. Furthermore, experiments on zero-shot and unsupervised segmentation have been conducted. Also, the architectural design choices have been reasonably extensively validated.

Critical strengths

S1: Interesting methodological combination of using the concept of masked reconstruction for object-centric learning. The paper shows how the masked reconstruction needs to be adapted to be effective for OCL. The results show multiple benefits of the final approach: better performance on common OCL metrics and cheaper decoding than autoregressive decoders.

S2: Fair and comprehensive experiments validating the claims. E.g., baselines have been rerun with the same stronger backbone (DINOv3) and meaningful metrics have been used, providing evidence for the claimed improved performance. The paper also included a comparison to unsupervised semantic segmentation and outperformed methods developed specifically for that task, showing further the strength of the learned representations.

Critical weaknesses

W1: Testing zero-shot capabilities by training on COCO and evaluating on VOC is a weak indicator, as VOC contains classes from COCO. So the small difference there is not surprising. A dataset with a stronger domain gap could be CLEVRTex (synthetic) or Obj365, as done in the GLASS paper.

-> Addressed in the revised manuscript with the suggested additional evaluation confirming the made claims.

Minor L. 304 numbe -> number Sometimes the dot after the paragraph title is missing, e.g. L. 414

Questions for authors

Q: Slot encoding is described as part of the proposed method, however, from the description it is unclear how this differs from encoders of existing slot attention methods. Can this be clarified?

-> Has been done in the revised version.

2 comments

We thank the reviewer for the careful and positive assessment of our work. We address the comments below. All changes in the revised paper are marked in blue.

Q: How does the slot encoding differ from existing Slot Attention methods?

It does not: we do not modify the Slot Attention module itself and use standard Slot Attention exactly as in DINOSAUR and SPOT. All components introduced by SlotMask act on the decoder side. We now state this explicitly at the end of Sec. 3.1.

W1: Zero-shot transfer from COCO to VOC is a weak indicator.

We agree, and we thank the reviewer for the suggestion. Following the GLASS protocol, we evaluated our COCO-trained model zero-shot on CLEVRTex and Objects365, which have a much larger domain gap to COCO. Best value in bold, second best in italics.

Model (trained on COCO)CLEVRTex mIoUi\text{mIoU}_iCLEVRTex mBOi\text{mBO}_iObjects365 mIoUi\text{mIoU}_iObjects365 mBOi\text{mBO}_i
SPOT39.543.718.020.7
GLASS46.150.118.621.4
GLASS†47.252.319.622.4
SlotMask (ours)48.852.618.721.7

† uses ground-truth class labels.

On CLEVRTex, SlotMask achieves the best performance. On Objects365, it outperforms SPOT and GLASS and is slightly below GLASS†, which uses ground-truth class labels. We find these experiments to be a valuable addition to our work and have included them in the revised paper (Table 4 and Appendix A.13).

Minor. We thank the reviewer for pointing out the typos, they are now fixed in the revised paper.

I thank the authors for running the extra evaluations, which further support the method. Given that these confirm the claims of the paper and the manuscript has improved in writing and clarity, I have increased my rating from weak acceptance to acceptance. I still highly recommend checking the added modifications for correctness and readability carefully (there are still some typos, e.g.,")" on the last line on page 8).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.