acceptodds
Under review as a conference paper at ICLR 2027

Learning When Temporal Consistency Should Begin in Video Object-Centric Learning

Abstract

Unsupervised video object-centric learning must both discover new objects and preserve representations of objects already present in the scene. Slot–slot contrastive learning (SSC), a common temporal-consistency objective, favors the latter by encouraging each slot to match its previous-frame state. We show that stronger SSC improves representations of existing objects but progressively degrades representations formed for newly appearing objects, even when idle slot capacity remains available. This reveals an acquisition–maintenance trade-off in how temporal consistency is imposed. We introduce SLOTCOMMIT with learned-onset SSC, which learns a slot-specific onset for temporal consistency to support new-object acquisition without sacrificing the consistency of established representations. This onset is tied to a learned activity gate that determines whether a slot contributes to reconstruction. Upon first activation, the slot becomes committed, and its temporal-consistency constraint begins on the following transition. The first enabled SSC loss also trains the activation decision through commitment, coupling slot recruitment to a representation-dependent consistency cost. On matched MOVi-C models, learned-onset SSC increases acquisition of new objects by idle slots from 27.5% to 50.9% and improves simultaneous entrant–resident representation on separate slots from 25.5% to 33.2%. It also improves overall acquisition and both entrant and resident representation quality. Ablations show that both delaying the onset of SSC and training this onset through commitment contribute to the observed gains. On standard video grouping benchmarks, the complete SLOTCOMMIT model also improves over SlotContrast across MOVi-C, MOVi-E, and YouTube-VIS.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
object-centric learning, video representation learning, temporal consistency, slot-based models, contrastive learning, self-supervised learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_8rfTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper diagnoses a conflict between new-object acquisition and temporal consistency. SlotCommit learns slot activation and begins slot–slot contrastive learning after first activation, allowing new representations to form before enforcing consistency.

Critical strengths
  1. Clear problem diagnosis supported by event-centered acquisition and resident-preservation measurements.

  2. Strong matched-architecture ablations separate delayed onset, commitment gradients, regularization strength, and idle capacity.

Critical weaknesses
  1. Commitment is irreversible within a sequence. The mechanism does not explicitly renew acquisition freedom when a previously committed slot is reused.

  2. Detailed acquisition evidence is concentrated on MOVi-C; equivalent mechanism validation on real videos is missing.

Questions for authors
  1. How does the method handle slot reuse after object departure, prolonged occlusion, or capacity exhaustion?

  2. Can acquisition and resident-preservation gains be demonstrated directly on real videos?

  3. Do onset-only gains persist on matched architectures across MOVi-E and YouTube-VIS?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 8rfT

We thank the reviewer for the careful assessment and for highlighting the problem diagnosis and the matched-architecture ablations. We address each point below.

W1 & Q1: Irreversible commitment, and slot reuse after departure, prolonged occlusion, or capacity exhaustion.

Irreversibility is a deliberate design choice, not an omission. A committed slot keeps its SSC term so that it remains bound to a single object identity. This is the standard notion of temporal consistency in video object-centric learning, where a slot serves as “a stable object-specific identifier over time” (Manasyan et al., 2025). It is also assumed by the standard video-level FG-ARI and mBO, which penalize a slot that represents different objects at different times. “Renewing acquisition freedom” for a committed slot would mean letting one slot carry two identities within the same video, which these metrics, and any downstream use of slot trajectories, treat as an error.

Departure and prolonged occlusion. In both cases the slot stays committed and keeps its SSC term, exactly as under standard SSC, so it remains tied to its object instead of being released to a different one. Identity-sensitive tracking metrics on full MOVi-C videos, which penalize slot switches and reuse, show better identity consistency than prior SSC-based methods:

MethodHOTA ↑IDF1 ↑
SlotContrast23.017.4
SRL27.122.8
SlotCurri24.117.7
SSC (matched)25.2 ± 0.820.2 ± 1.2
SlotCommit31.2 ± 0.228.4 ± 0.5

Reuse for new objects. Under persistent commitment, new objects are meant to be acquired by idle, uncommitted slots rather than by reusing a committed one. On MOVi-C, at least one idle slot is available at t_e−1 in 100% of entry events (5.69 on average).

Capacity exhaustion. When all slots are already committed, a new object must share or reuse a slot. This limitation is inherent to any model with a fixed number of slots (e.g., VideoSAUR, SlotContrast) and is not introduced by learned onset. In this case, the recruited slot incurs exactly the standard SSC term, so SlotCommit reduces to the SSC baseline and is never more restrictive. Addressing exhaustion requires varying the number of slots, which is orthogonal to learning when consistency begins.

W2 & Q2–3: Validation on real videos, and onset-only gains on matched architectures across datasets.

Why the detailed analysis focuses on MOVi-C. The paper establishes the mechanism through controlled experiments on MOVi-C (Tables 1–3, 8; Appendices B–D), and this choice is deliberate. Entering objects make up 20.7% of annotated objects on MOVi-C, compared with 8.5% on MOVi-E. MOVi-C’s exhaustive instance annotations also allow acquisition, pre-entry slot utilization, and resident preservation to be measured jointly. YouTube-VIS annotates only a subset of salient objects, so an apparently unused slot may represent unannotated content, and an object’s first annotated appearance need not correspond to its physical entry. We therefore use MOVi-C for the fine-grained analysis.

Onset-only gains on matched architectures (Q3) and on real videos (Q2). To test whether the gains hold beyond MOVi-C, we ran the matched comparison on MOVi-E and YouTube-VIS: same architecture, gate, and evidence aggregation, with only the SSC onset rule differing.

DatasetObjectiveIdle acquisition ↑Entrant IoU ↑Resident IoU ↑
MOVi-CSSC (matched)27.5 ± 1.627.8 ± 0.840.6 ± 0.5
Learned-onset SSC50.9 ± 0.631.4 ± 0.641.8 ± 0.2
MOVi-ESSC (matched)25.9 ± 1.016.4 ± 0.636.8 ± 0.5
Learned-onset SSC27.5 ± 0.620.5 ± 1.437.4 ± 0.4
YouTube-VIS*SSC (matched)5.2 ± 1.027.1 ± 1.637.9 ± 0.2
Learned-onset SSC17.4 ± 0.929.7 ± 0.739.9 ± 0.2
  • YouTube-VIS provides annotations only for a subset of salient objects.

The onset-only gains persist across datasets. On MOVi-E, learned onset improves entrant IoU from 16.4 to 20.5 and idle-slot acquisition from 25.9 to 27.5, without reducing resident IoU (36.8 → 37.4), the same pattern as on MOVi-C. On real videos (YouTube-VIS), learned onset improves idle-slot acquisition from 5.2 to 17.4 and entrant IoU from 27.1 to 29.7, while resident IoU also improves (37.9 → 39.9). Qualitative real-video examples in Fig. 6 show the same behavior: an idle slot is recruited for the entering object instead of reusing a slot already assigned to another object.

Reference

Manasyan, A., Seitzer, M., Radovic, F., Martius, G., and Zadaianchuk, A. Temporally Consistent Object-Centric Learning by Contrasting Slots. In CVPR, pp. 5401–5411, 2025.

Reviewer_TCWMOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the acquisition and maintenance trade-off in unsupervised video object-centric learning. The authors propose SlotCommit, which uses a learned activity gate to determine when each slot begins to incur its own temporal consistency loss. Experiments on MOVi-C show improved acquisition and joint representation, while the complete model outperforms SlotContrast on three video grouping benchmarks.

Critical strengths
  1. This paper studies the trade-off of acquisition and maintenance, which is a challenging yet important problem in unsupervised video object-centric learning.
  2. The case studies are relatively thorough, with qualitative examples and analysis that help explain the observed behavior.
Critical weaknesses
  1. The main concern is that the detailed acquisition analysis is limited to MOVi-C. Verifying the trade-off and acquisition gains on MOVi-E and YouTube-VIS would make the argument more convincing.
  2. For acquisition and joint representation, the paper mainly compares SlotCommit with a matched SSC baseline. Including existing slot-allocation and object-centric methods in these comparisons can help clarify the advantage of the proposed approach.
  3. I found the gate computation in Sec. 3.2 and the masked straight-through estimator in Sec. 3.3 not easy to follow. A short algorithm description or pseudocode would help clarify how these computations are implemented.
  4. The organization could be improved by moving Related Work after the Introduction. Its current position between the method and experiments interrupts the flow of the paper.
Questions for authors
  1. How does SlotCommit affect training time compared with SlotContrast under the same setup? Does the activity gate also introduce noticeable inference overhead?
6 comments
AuthorsOpenReview ↗

Response to Reviewer TCWM

We thank the reviewer for the constructive feedback and for recognizing the importance of the acquisition–maintenance trade-off and our qualitative analysis. We address each point below.

W1: Trade-off and acquisition gains beyond MOVi-C.

Why the detailed analysis focuses on MOVi-C. We focus the controlled acquisition analysis on MOVi-C because entering objects make up 20.7% of annotated objects, compared with 8.5% on MOVi-E. MOVi-C’s exhaustive instance annotations also support joint analysis of acquisition, pre-entry slot utilization, and resident preservation. YouTube-VIS annotates only a subset of salient objects. Apparently unused slots may therefore represent unannotated content, and an object’s first annotated appearance need not correspond to its physical entry.

The trade-off. On MOVi-C, the trade-off holds both on SlotContrast (Table 8, λ ∈ {0, 0.5, 1}, three seeds each) and on our gated architecture (Table 2). Removing SSC gives the best entrant IoU (33.2) but the worst resident IoU (37.5), whereas SSC at λ = 0.5 reverses this (27.8 / 40.6). Learned onset avoids this choice (31.4 / 41.8). The matched comparison below tests whether this also holds on MOVi-E and YouTube-VIS.

Trade-off and acquisition gains on MOVi-E and YouTube-VIS. Following the reviewer’s suggestion, we ran the matched comparison on both datasets (same architecture, gate, and evidence aggregation; only the SSC onset rule differs):

DatasetObjectiveIdle acquisition ↑Entrant IoU ↑Resident IoU ↑
MOVi-CSSC (matched)27.5 ± 1.627.8 ± 0.840.6 ± 0.5
Learned-onset SSC50.9 ± 0.631.4 ± 0.641.8 ± 0.2
MOVi-ESSC (matched)25.9 ± 1.016.4 ± 0.636.8 ± 0.5
Learned-onset SSC27.5 ± 0.620.5 ± 1.437.4 ± 0.4
YouTube-VIS*SSC (matched)5.2 ± 1.027.1 ± 1.637.9 ± 0.2
Learned-onset SSC17.4 ± 0.929.7 ± 0.739.9 ± 0.2
  • YouTube-VIS provides annotations only for a subset of salient objects.

On MOVi-E, learned onset improves entrant IoU from 16.4 to 20.5 and idle-slot acquisition from 25.9 to 27.5, without reducing resident IoU (36.8 → 37.4). This reproduces the MOVi-C pattern: better acquisition of new objects without sacrificing maintenance of existing ones, so learned onset avoids the trade-off on MOVi-E as well. YouTube-VIS shows the same trend on real videos: idle-slot acquisition rises from 5.2 to 17.4 and entrant IoU from 27.1 to 29.7, while resident IoU also improves (37.9 → 39.9).

W2: Comparison with existing methods on acquisition and joint representation.

We evaluated SlotContrast, SRL, and SlotCurri with the same entry-centric protocol on MOVi-C, using their released checkpoints:

MethodIdle acquisition ↑Entrant IoU ↑Resident IoU ↑Joint score @0.3 ↑
SlotContrast22.128.539.625.3
SRL25.829.341.827.6
SlotCurri35.429.740.231.5
SSC (matched)27.5 ± 1.627.8 ± 0.840.6 ± 0.525.5 ± 1.0
SlotCommit50.9 ± 0.631.4 ± 0.641.8 ± 0.233.2 ± 1.6

SlotCommit achieves the highest idle-slot acquisition, entrant IoU, and joint entrant–resident representation, and ties SRL for the best resident IoU. The compared methods improve grouping (SRL) or adapt slot allocation (SlotCurri), but all retain uniform SSC and reach at most 35.4% idle-slot acquisition. Changing how content is grouped or how slots are allocated therefore does not remove the consistency pressure on newly recruited slots. Learning when that pressure begins addresses it directly, and is complementary to these mechanisms.

AuthorsOpenReview ↗

Response to Reviewer TCWM

W3: Pseudocode for the gate computation and the masked straight-through estimator.

Thank you for highlighting this. App. A.2 (Eqs. 9–12) specifies the evidence aggregation and activity gate, and App. A.3 (Eqs. 13–16) the masked straight-through estimator. We agree a procedural summary makes them easier to follow and will add the two algorithms below to App. A.

Algorithm 1: Activity gate at frame t (training; Sec. 3.2, App. A.2)

# sg = stop-gradient; k indexes slots, n indexes tokens
# inputs, final corrector iteration: logits e_nk = q_k·k_n/√d, token values v_n

a_nk ← softmax_k(e_nk)                            # slots compete per token      (Eq. 9)
r_nk ← a_nk / (Σ_n' a_n'k + ε)                    # normalize over tokens        (Eq. 9)
h~_nk ← softmax_k((log a_nk + Gumbel) / τ_h)        # τ_h = 1 (App. A.1)
h_nk ← one_hot_k(argmax_k h~_nk) + h~_nk − sg(h~_nk) #hard forward, ST backward (Eq. 10)
w_nk ← r_nk · h_nk
u_k ← Σ_n w_nk v_n ;  m_k ← Σ_n w_nk             # evidence content, support    (Eq. 11)

p_k ← σ((G_θ(m_k, u_k) + η_k) / τ_z)                # η: logistic noise, training only
z_k ← 1[p_k > 0.5] + p_k − sg(p_k)               # binary forward, ST backward  (Eq. 12)
ŷ_n ← Σ_k z_k π_nk f_nk        # π: mask softmax over slots; no renormalization (Eq. 5)

Algorithm 2: Learned-onset SSC with the masked estimator, one minibatch (training; Sec. 3.3, App. A.3)

# j indexes all B·K slots of the minibatch, as in Eq. 2

c_j ← 0 for all j;  L ← 0
for t = 1..T:
    S_t ← C(P(S_{t−1}), X_t)  # all slots updated, independent of z (t = 1: learned initial slots)
    z_t ← Algorithm 1 on frame t
    if t > 1:
        for each slot j:                # c_j still holds c_{t−1,j}
            L += where(c_j > 0.5, ℓ_SSC(t, j), 0) · c_j        # Eq. 13; comparison not differentiated
    c ← 1 − (1 − c) ⊙ (1 − z_t)     # equals max(c, z_t) in the forward pass (Eq. 3); product form (Eq. 7) for gradients
L_SSC ← L / (B(T−1)K)                   # Eq. 4

The gate is binary in the forward pass with a straight-through gradient. It controls only decoder participation; every slot still receives the recurrent update. SSC eligibility uses the previous-frame commitment, so the transition into first activation is exempt. In Eq. 13 the two factors coincide in the forward pass: the mask removes a disabled term's loss and its direct commitment gradient, while the multiplication by c lets the first enabled SSC loss reach the activation that established commitment. All slots remain in the contrastive candidate pool of Eq. 2. The matched SSC baseline shares Algorithm 1 and differs only in applying ℓ_SSC to every slot on every transition. Gate noise is disabled at evaluation, and Algorithm 2 is used only during training.

W4: Placement of Related Work.

Thank you for the suggestion. We agree and will move Related Work to directly follow the Introduction in the revised version, so that the method and experiments read without interruption.

Q1: Training time and inference overhead.

The overhead is small. On MOVi-C, our 100k-step runs take 21.73 h for SlotContrast and 22.98 h for SlotCommit, about 5.8% additional training time.

For inference, we measured all models on the same RTX A6000 with identical device-resident inputs (batch 1, 336×336 frames):

ModelAverage time per frame
SlotContrast3.06 ms
Gated SSC baseline (matched)3.53 ms
SlotCommit3.54 ms

Learned onset, our core contribution, adds no inference cost: commitment is used only by the training loss (App. A.1), so SlotCommit and the matched gated baseline run the same computation (3.54 vs. 3.53 ms). The difference from SlotContrast, about 0.5 ms per frame, comes from the gated architecture shared by both matched models; the gate head itself has only 4,289 parameters.

Thank you for the additional experiments and explanations. They address most of my concerns. I am inclined to raise my score and would like to read the revised manuscript first.

Reviewer_a73iOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies an acquisition-maintenance trade-off in video slot learning. Temporal self-matching helps established objects but can prevent idle slots from adapting to new entrants. SlotCommit uses a learned activity gate and delays each slot's own SSC loss until the transition after its first activation. The first enabled loss also trains the onset decision through commitment. Matched MOVi-C experiments show improved idle-slot recruitment and joint entrant-resident representation, and the complete model improves over SlotContrast on three grouping benchmarks.

Critical strengths
  1. The activation gate and commitment mechanism are a useful contribution. They let a slot acquire content before applying its own SSC constraint, while the first enabled SSC loss also trains the activation decision. The experiment verify the effectiveness of proposed method and its motivation, where idle acquisition increase from 27.5% to 50.9%.
  2. Comprehensive ablation studies and components analysis are conducted to support effectiveness of the method and make the main results more convincing.
Critical weaknesses
  1. Does a committed slot’s SSC loss remain enabled during training after its object leaves the frame? If so, how does this affect reusing the slot for a new object?
  2. The mechanism is quantitatively isolated only on MOVi-C. MOVi-E and YouTube-VIS compare complete pipelines, and SlotCommit is not best on every grouping metric. Can you provide ablation results on more datasets and provide more analysis on why proposed method fall behind on some of the metrics.
Questions for authors

Please refer to my questions listed in Weakness sections.

3 comments
AuthorsOpenReview ↗

Response to Reviewer a73i

We thank the reviewer for the positive assessment and for recognizing the activation gate and commitment mechanism and our ablation studies. We address the two questions below.

W1: Does SSC stay enabled for a committed slot after its object leaves, and how does this affect slot reuse?

Yes, and this is intentional. Once committed, a slot keeps its SSC term for the rest of the sequence, so that each slot stays bound to a single object identity.

Why persistence. This follows the standard notion of temporal consistency in video object-centric learning, where a slot serves as “a stable object-specific identifier over time” (Manasyan et al., 2025). It is also assumed by the standard evaluation: FG-ARI and mBO are computed over the full video, so a slot that represents one object and later a different one merges two objects into one cluster and is penalized. Reusing a slot for a new object would mix two identities in one trajectory. Downstream uses that read object histories from slots (e.g., dynamics prediction, tracking) could not tell which part belongs to which object, and in the unsupervised setting there are no identity labels to fix this afterwards. Persistence also keeps a slot tied to its object while that object is occluded or temporarily out of view, instead of handing the slot to another object.

Effect on new objects. New objects are therefore meant to be acquired by uncommitted slots, not by reusing committed ones, and the capacity to do so exists: on MOVi-C, at least one idle slot is available at t_e−1 in 100% of entry events (5.69 on average).

Evidence that identities are preserved. We treat each slot as a track and evaluate identity-sensitive tracking metrics on MOVi-C. IDF1 matches each slot to one ground-truth identity over the whole video, so slot switches or reuse lower the score.

MethodHOTA ↑IDF1 ↑
SlotContrast22.9617.44
SRL27.0922.82
SlotCurri24.1117.69
SSC (matched architecture)25.16 ± 0.7720.15 ± 1.19
SlotCommit (learned-onset SSC)31.23 ± 0.1828.35 ± 0.48

Relative to the matched SSC baseline, which differs only in the onset rule, learned onset improves HOTA by +6.1 and IDF1 by +8.2. Learned onset with persistent commitment therefore improves identity consistency without hindering the acquisition of new objects.

Reference

Manasyan, A., Seitzer, M., Radovic, F., Martius, G., and Zadaianchuk, A. Temporally Consistent Object-Centric Learning by Contrasting Slots. In CVPR, pp. 5401–5411, 2025.

AuthorsOpenReview ↗

Response to Reviewer a73i

W2: Ablations on more datasets, and why SlotCommit is not best on every grouping metric.

Why the mechanism analysis focuses on MOVi-C. We focus the controlled mechanism analysis on MOVi-C because entering objects make up 20.7% of annotated objects, compared with 8.5% on MOVi-E. MOVi-C’s exhaustive instance annotations also support joint analysis of acquisition, pre-entry slot utilization, and resident preservation. YouTube-VIS, in contrast, annotates only a subset of salient objects. Apparently unused slots may therefore represent unannotated content, and an object’s first annotated appearance need not correspond to its physical entry.

Matched ablations on more datasets. Following the reviewer’s suggestion, we ran the matched comparison on MOVi-E and YouTube-VIS: same architecture, gate, and evidence aggregation, with only the SSC onset rule differing.

DatasetObjectiveIdle acquisition ↑Entrant IoU ↑Resident IoU ↑
MOVi-CSSC (matched)27.5 ± 1.627.8 ± 0.840.6 ± 0.5
Learned-onset SSC50.9 ± 0.631.4 ± 0.641.8 ± 0.2
MOVi-ESSC (matched)25.9 ± 1.016.4 ± 0.636.8 ± 0.5
Learned-onset SSC27.5 ± 0.620.5 ± 1.437.4 ± 0.4
YouTube-VIS*SSC (matched)5.2 ± 1.027.1 ± 1.637.9 ± 0.2
Learned-onset SSC17.4 ± 0.929.7 ± 0.739.9 ± 0.2
  • YouTube-VIS provides annotations only for a subset of salient objects.

On MOVi-E, learned onset improves entrant IoU from 16.4 to 20.5 and idle-slot acquisition from 25.9 to 27.5 without reducing resident IoU (36.8 → 37.4), reproducing the MOVi-C pattern: better acquisition of new objects without sacrificing maintenance of existing ones. On YouTube-VIS, idle-slot acquisition rises from 5.2 to 17.4, entrant IoU from 27.1 to 29.7, and resident IoU from 37.9 to 39.9. Learned onset thus consistently increases idle-slot acquisition and entrant IoU while preserving resident IoU across all three datasets.

Why SlotCommit is not best on every grouping metric. FG-ARI and mBO are general benchmarks for video object-centric learning: they measure overall grouping quality, not acquisition at object entry, which is what our method targets. We do not claim the best result on every such metric. Our claim on these benchmarks is that SlotCommit improves over SlotContrast, which holds on all three datasets and both metrics (Table 4); the other methods are reported there for reference.

Methods with different temporal objectives. Among these, SlotCommit has the highest FG-ARI on all three datasets and the highest mBO on YouTube-VIS. The only exception is mBO on MOVi-C and MOVi-E, where temporal-similarity prediction yields sharper masks but much weaker video-level consistency (FG-ARI lower by 11–15 points).

Recent SSC-based methods with added grouping mechanisms. These methods keep SSC as the temporal objective and improve grouping through additional mechanisms such as encoder–decoder synergy or slot curricula. Compared with them, SlotCommit’s overall performance remains competitive: it has the highest FG-ARI on MOVi-C and MOVi-E and the second-highest on YouTube-VIS (43.9 vs. 44.8), and the second-highest mBO on all three datasets. Learned onset is complementary to these mechanisms.

Entry-centric evaluation. On the property our method targets, SlotCommit leads SlotContrast, SRL, and SlotCurri under the same entry-centric evaluation on MOVi-C: idle-slot acquisition 50.9 vs. at most 35.4, entrant IoU 31.4 vs. at most 29.7, IDF1 28.4 vs. at most 22.8, with resident IoU tied for best (table below; baselines use their released checkpoints).

MethodIdle acquisition ↑Entrant IoU ↑Resident IoU ↑
SlotContrast22.128.539.6
SRL25.829.341.8
SlotCurri35.429.740.2
SSC (matched)27.5 ± 1.627.8 ± 0.840.6 ± 0.5
SlotCommit50.9 ± 0.631.4 ± 0.641.8 ± 0.2

Thank authors for the response. I think my concerns all mostly addressed and I will hold my rating at weak acceptance.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Author Responses

We thank all reviewers for their constructive feedback. The reviewers recognized the acquisition–maintenance diagnosis (TCWM, 8rfT), the gate and commitment mechanism (a73i), and the matched-architecture ablations (a73i, 8rfT). Below we summarize the main concerns and how we addressed them.

1. Mechanism evidence beyond MOVi-C (a73i, TCWM, 8rfT). This was the one concern shared by all reviewers. We ran the matched comparison on MOVi-E and YouTube-VIS: same architecture, gate, and evidence aggregation, with only the SSC onset rule differing.

DatasetObjectiveIdle acquisition ↑Entrant IoU ↑Resident IoU ↑
MOVi-CSSC (matched)27.5 ± 1.627.8 ± 0.840.6 ± 0.5
Learned-onset SSC50.9 ± 0.631.4 ± 0.641.8 ± 0.2
MOVi-ESSC (matched)25.9 ± 1.016.4 ± 0.636.8 ± 0.5
Learned-onset SSC27.5 ± 0.620.5 ± 1.437.4 ± 0.4
YouTube-VIS*SSC (matched)5.2 ± 1.027.1 ± 1.637.9 ± 0.2
Learned-onset SSC17.4 ± 0.929.7 ± 0.739.9 ± 0.2
  • YouTube-VIS provides annotations only for a subset of salient objects.

Learned onset consistently increases idle-slot acquisition and entrant IoU while preserving resident IoU across all three datasets, including real videos. We also explain why the detailed analysis focuses on MOVi-C: entering objects make up 20.7% of annotated objects, compared with 8.5% on MOVi-E, and MOVi-C's exhaustive instance annotations support joint analysis of acquisition, pre-entry slot utilization, and resident preservation, whereas YouTube-VIS annotates only a subset of salient objects.

2. Irreversible commitment and slot reuse (a73i, 8rfT). Persistence is a deliberate design choice: a committed slot keeps its SSC term so that it remains bound to a single object identity, as assumed by the standard notion of temporal consistency and by video-level FG-ARI and mBO. New objects are meant to be acquired by idle, uncommitted slots, and on MOVi-C at least one idle slot is available at t_e−1 in 100% of entry events (5.69 on average). Identity-sensitive tracking metrics on MOVi-C confirm that identities are preserved: relative to the matched SSC baseline, learned onset improves HOTA from 25.2 to 31.2 and IDF1 from 20.2 to 28.4, also exceeding SlotContrast, SRL, and SlotCurri. Under capacity exhaustion, the recruited slot incurs exactly the standard SSC term, so SlotCommit reduces to the SSC baseline and is never more restrictive.

3. Comparison with existing methods on acquisition and joint representation (TCWM). We evaluated SlotContrast, SRL, and SlotCurri with the same entry-centric protocol on MOVi-C, using their released checkpoints. SlotCommit achieves the highest idle-slot acquisition (50.9 vs. at most 35.4), entrant IoU (31.4 vs. at most 29.7), and joint score (33.2 vs. at most 31.5), and ties SRL for the best resident IoU (41.8).

4. Why SlotCommit is not best on every grouping metric (a73i). FG-ARI and mBO measure overall grouping quality, not acquisition at object entry, which is what our method targets. Our claim on these benchmarks is that SlotCommit improves over SlotContrast, which holds on all three datasets and both metrics (Table 4). On the entry-centric evaluation, SlotCommit leads SlotContrast, SRL, and SlotCurri (point 3).

5. Clarity and cost (TCWM). We provide pseudocode for the activity gate and the masked straight-through estimator, which we will add to App. A, and we will move Related Work to directly follow the Introduction. Training takes about 5.8% longer than SlotContrast (22.98 h vs. 21.73 h). Learned onset adds no inference cost: SlotCommit and the matched gated baseline run the same computation (3.54 vs. 3.53 ms per frame), since commitment is used only by the training loss.

Dear reviewers,

Thanks for serving as a reviewer. As the discussion period comes to a close and the authors have submitted their rebuttals, I kindly ask you to take a moment to review them and provide any final comments.

If you have already updated your comments, please disregard this message.

Thank you once again for your dedication to the OpenReview process.

Best,

Area Chair

AuthorsOpenReview ↗

Revised manuscript uploaded

We thank all reviewers for their constructive feedback. We have uploaded a revised manuscript; all changes are highlighted in blue. The main revisions are:

  • New Sec. 5.6 (Generalization and Comparison at Object Entry), which brings the new results into the main paper:
    • the matched comparison on MOVi-E and YouTube-VIS (Table 5);
    • the entry-centric comparison with SlotContrast, SRL, and SlotCurri, together with identity-sensitive tracking metrics (HOTA, IDF1) (Table 6);
    • training and inference cost.
  • Pseudocode for the activity gate and the masked straight-through estimator (Algorithms 1 and 2, Appendices A.2 and A.3).
  • Related Work moved to directly follow the Introduction (Sec. 2).

Because of these changes, some numbers in our earlier responses have shifted. Section numbers after the Introduction have increased by one (e.g., Sec. 3.2 and 3.3 are now Sec. 4.2 and 4.3). Tables 5 and 6 are new, so Tables 5–10 of the original submission are now Tables 7–12 (e.g., the SSC weight sweep cited as Table 8 is now Table 10). Equation and figure numbers are unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.