- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Correction: the previous text was pasted from a review of another submission. It is withdrawn and superseded in full by the review below. I apologize for the error.
This paper studies object binding and object permanence in video foundation models. It removes an object's earlier tokens from a frozen predictor's context and measures object-selective prediction degradation using a calibrated leakage ratio. A separate readout compares predicted features at occluded positions with hidden-object and occluder templates. Experiments across several foundation models, synthetic and real videos, and an object-centric VideoSAUR control find object-selective predictive dependencies but limited hidden-object preference under the proposed readout. Occlusion-completion fine-tuning improves feature and identification scores, while an erased-history counterfactual reproduces most of the feature-level gain. The main contribution is an intervention-based diagnostic separating predictive binding, hidden-object preference, and dependence on object history.
Critical strengths
1. A well-chosen research question and useful counterfactual diagnostic.
Distinguishing decodable object information from information used by a model's own predictions is meaningful for evaluating video representations. The erased-history experiment is particularly valuable: comparing fine-tuning gains when the object did and did not previously exist directly challenges a tempting interpretation of improved occlusion scores. The paper deserves credit for testing and qualifying its own apparent positive result.
2. Substantial controls within the investigated setting.
The evidence is considerably more extensive than a single negative probe result. It includes untrained and shuffled-label endpoints, distance and appearance controls, independent clips, several foundation models, real-video tests, readout-floor calibration, and a paired comparison on 62 clips. These support the reported predictive-dependency and readout-level observations. The acknowledgement of residual history traces and limits on cross-model score comparability is also valuable.
Critical weaknesses
1. The permanence readout and cross-model comparison do not fully identify the stronger maintenance claim (Sections 3.2 and 5).
A predictor trained on visible content may reasonably produce occluder-like features at an occluded location while preserving some object information elsewhere. A non-positive POperm therefore establishes failure to favor the hidden object in this particular output-space contrast, rather than absence of persistent object information in general. The visible-patch floor correction addresses a concrete estimator bias, but does not resolve this distinction. Section 6.2 itself finds subdominant object-history traces.
The VideoSAUR comparison makes this issue consequential: its positive score is measured in persistent slot space, whereas the foundation-model score is measured in predicted fill features. VideoSAUR's own mask readout is negative (-0.261). Matching the same clips removes a data-composition explanation but does not make these readouts equivalent. The paper acknowledges this limitation, yet the title and some main-text statements remain stronger than the common measurement supports. This is my principal concern: the empirical diagnostic is useful, but the broader claim about maintenance requires a more precise operational scope or a shared behavioral test.
2. The fine-tuning evidence constrains the mechanism without uniquely establishing it (Section 6.2).
The reproduction of 96-108% of the fill gain in erased-history worlds is persuasive evidence that this continuous feature-level gain does not require the object's history. However, the same section reports that the original-minus-erased identification gap increases from .058 to .156, and that the violation-of-expectation readout changes from -.021 to +.012. These results make it important to distinguish a history-independent feature shift from improved history-dependent behavior. The proposed explanation that a generic prior amplifies an existing weak trace is plausible, but its direct event-level prediction is supported only in direction (Spearman +.18, p=.176). The categorical statement that the gain adds no tracking therefore needs tighter qualification.
This interpretive limitation is compounded by the evaluated scope: all tested foundation models are feedforward predictors without persistent per-object state, the positive object-centric side uses one architecture, and real-video occlusions are pasted. These are acknowledged limitations, but they constrain how broadly the reported dissociation can be generalized. My concern is whether the existing observations sufficiently establish the claimed mechanism, rather than a lack of experimental effort.
Questions for authors
1. Operational meaning of permanence. Can the authors state precisely which maintenance claim is falsified by non-positive POperm, and distinguish it from retention of subdominant object information or prediction of visible content? How should VideoSAUR's positive slot score and negative mask readout be reconciled when interpreting the paired cross-model gap? A common identity- or future-behavior readout would be especially informative if available.
2. Feature-level versus behavioral history dependence. How do the authors reconcile the nearly history-independent fill gain with the increased original-minus-erased identification gap and the positive violation-of-expectation result after fine-tuning? What existing evidence separates amplification of a fixed trace from an improvement in history-dependent processing? Please distinguish results that rule out a history-dependent explanation of the fill gain from those that establish the proposed prior-amplification mechanism.
3. Scope of the conclusion. Would the authors narrow the main claim to the tested feedforward predictors and the specified readouts, as the Limitations section already suggests? Broader tests on persistent-state video models and natural occlusions would be valuable future work; my present assessment primarily depends on clarifying the existing evidence in Questions 1-2.