TSA: Temporal Slot Activation for Persistent Object-Centric Video Representation
Abstract
Unsupervised video object-centric learning aims to decompose dynamic scenes into temporally persistent entity representations. Existing recurrent video slot-attention methods propagate a fixed set of slots across frames, but typically assume *unconditional slot propagation*: every slot is updated and decoded at every frame, regardless of whether its corresponding object is visible. We show that this design violates a basic lifecycle requirement for persistent slots: when an object is absent or fully occluded, its slot should preserve its previous state and avoid explaining unrelated visible content. Instead, unconditional propagation creates *update-induced state drift*, where current-frame evidence overwrites the absent object's representation, and *decoder-induced reconstruction interference*, where the inactive slot remains coupled to reconstruction through decoder attention. We propose **Temporal Slot Activation (TSA)**, a lightweight mechanism that learns a per-slot, per-frame activation score without visibility supervision. TSA uses this activation as a shared latent control variable for slot lifecycle modeling. When a slot is inactive, TSA anchors its state to the previous slot through activation-gated updating and suppresses its decoder participation through an activation-dependent additive bias on attention logits before softmax normalization. This jointly reduces state drift and reconstruction-driven interference. To improve decisions under partial occlusion and gradual reappearance, TSA further conditions activation prediction on a per-slot temporal memory produced by a Temporal Context Encoder. We evaluate TSA on MOVi-C, MOVi-E, YouTube-VIS HQ, and the occlusion-heavy OVIS benchmarks, using both standard metrics (FG-ARI, mBO) and tracking-based metrics (IDF1, HOTA). TSA improves object decomposition and temporal identity preservation, with large gains on long, heavily occluded videos. Source code will be released upon publication.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.