- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CoSlot studies video object-centric learning by jointly binding frozen features from multiple visible frames into shared global slots. It combines two reconstruction objectives: shared prototypes reconstruct features from all visible frames through the binding assignments, while a spatial decoder reconstructs only sampled visible frames. This allows the binding context to grow while keeping the number of densely decoded frames fixed. The full model additionally includes masked-frame prediction (MFP), slot coordination, and temporal evidence aggregation. Evaluation on MOVi-C, MOVi-E, and YouTube-VIS measures object grouping and mask overlap. Fixed-checkpoint inference comparisons distinguish the effects of joint binding scope, temporal coverage, and global-slot state passing. Training costs are measured using cached frozen features.
Critical strengths
- A clear implementation and empirical support for separating binding context from dense decoding cost. Shared-prototype reconstruction (ALG) covers all visible frames and can be evaluated through compact matrix statistics. Spatial decoding operates on uniformly sampled frames and provides an unbiased estimator of the visible-frame mean spatial loss for a fixed input and model. The objectives are empirically complementary: Core achieves video FG-ARI of 66.6 under Whole inference, compared with 50.6 for Only-Spatial. Table D.1 also shows relatively small quality changes from decoding more spatial frames at substantially higher computational cost, supporting a small decoding budget. The full model decodes one visible spatial-reconstruction target and one MFP target, rather than only one frame in total. (pp. 4 and 7–8; Appendix A; Table D.1)
- Informative fixed-checkpoint diagnostics distinguish within-frame grouping from cross-frame consistency. Shortening independent inference chunks primarily reduces video-level consistency, while image-level grouping changes relatively little. Global-slot state passing recovers much of this loss. The 48-frame YouTube-VIS experiment reports paired results across three seeds, with positive gains from state passing and whole-sequence joint binding for every seed. Full-reference alignment reindexes slots using the model’s own Whole predictions rather than ground truth, and the manuscript distinguishes offline whole-sequence access from state continuation across chunks. These controls help avoid conflating inconsistent slot identities with failures of single-frame segmentation. (pp. 5–6; Appendices B.2 and C.2)
Critical weaknesses
- Lower per-update cost does not yet establish a competitive quality–total-training-cost trade-off. In Table 2, CoSlot achieves video FG-ARI of 30.2±0.6 on YouTube-VIS, compared with 38.0 for SlotContrast and 42.9 for SRL, with lower mBO as well. Its mBO also trails these two principal baselines on MOVi-C and MOVi-E. Table D.3 genuinely demonstrates lower per-update time and memory: for example, 87.98 ms/update versus 149.18 ms/update for SlotContrast on YouTube-VIS, approximately a 41% reduction. However, these measurements use cached features and a physical batch size of 16, whereas comparison quality is taken from published results. Training schedules, actual batch sizes, convergence, and cumulative resource budgets are not aligned with the per-update measurements. The current evidence therefore does not establish the quality achievable under equal training time or the resources required to reach a shared quality level. The consequential concern is the absence of a direct cross-method quality–cost evaluation for an efficiency-oriented contribution. (Tables 2 and D.3; pp. 8, 15–17, and 21)
- The independent effect of training context remains confounded by feature exposure and randomness. The training-length sweep in Figure 3(c) fixes 100k updates and 128 clips per update while increasing context from four to 24 frames. Consequently, visible-frame feature exposure per update increases by up to sixfold, and the sweep uses only one fixed seed. The interpretation that longer joint context itself improves single-frame Reset consistency is therefore not isolated from greater feature exposure, different gradient estimation, or training-compute effects. Fixed-checkpoint inference experiments support comparisons of inference context but do not remove this training confound. The four-, eight-, and twelve-frame YouTube-VIS training sweep is also nonmonotonic, limiting the scope of the proposed interpretation. (Figure 3(c); Tables B.1 and C.2; pp. 6–7 and 15–18)
Questions for authors
- Using existing training logs, can you align validation quality for CoSlot, SlotContrast, and SRL with cumulative training GPU time or updates, specifying feature-caching costs, actual batch sizes, and training stages? On at least one existing dataset, can you report quality under a matched training budget or time to reach a shared quality level?
- Is there an existing or feasible limited MOVi-C comparison between four-frame and 24-frame training with comparable cumulative visible-feature exposure or training compute, together with seed variability? If such a control is unavailable, please describe the single-frame Reset improvement as an effect of the complete training configuration and explicitly acknowledge that context length itself has not been isolated.
- For chunked inference with global-slot state passing, can you report paired effects of ALG relative to Only-Spatial across matched training seeds and checkpoints? In Table C.1, Core achieves 55.7 under Chunk-1 GS passing, below 60.6 for Only-Spatial, reversing the ordering under Whole inference. Please explain this protocol-dependent benefit and MFP’s role, since it materially affects the assessment of the state-passing use case.
Thanks for the response. I think the authors answers most of my questions. The comparisons with other methods are included now. However, the limited performance on real-world dataset is still not good enough. I will keep my rating as weekly acceptance.