- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the problem of Object Permanence in video generation. The authors construct a multi-camera synthetic dataset, PermaKube, and an evaluation benchmark, PermaEval, and propose a scene representation based on cross-view and cross-time supervision, which encodes multi-view videos into a unified Scene Latent. Compared with traditional Video Latents, Scene Latents significantly improve object identity preservation after occlusion.
Critical strengths
-
The paper investigates Object Permanence from the perspective of Latent Representation, which is meaningful for addressing long-term consistency issues in video generation.
-
The proposed PermaKube and PermaEval support controlled occlusions and multi-view evaluation, and relatively extensive ablation experiments are conducted with different supervision strategies.
-
The model achieves good performance, with PermaTok reaching an object identity preservation rate of 78.7%, significantly outperforming WF-VAE's 47.7%.
Critical weaknesses
-
The experimental evaluation lacks validation on real-world videos or more complex dynamic scenes. More complex datasets may be needed to demonstrate the effectiveness of the model.
-
The comparison between Scene Latents and Video Latents involves differences in network architecture, latent dimensionality, temporal sampling, and training budgets. The ablation studies do not sufficiently control these variables, and more experimental results are needed.
-
Latent Probing is mainly performed on Scene Tokens before compression, rather than the compressed Latents actually used for generation. This is a clear distinction and may require further validation.
Questions for authors
-
I am curious whether there are more realistic or complex datasets in this field. Could the authors further evaluate the generalization ability of PermaTok on real-world videos, complex occlusions, and nonlinear motion scenarios?
-
If Probing is directly performed on the compressed Latents, or even the Latents predicted by the generator, would similar Hidden-object Recovery capabilities still be observed?
Since there is no borderline option in the current rating scale, I assigned this score.
Thank you for the experiments. They addressed several of my concerns. However, I remain concerned that, the performance gains cannot be fully attributed to the scene latent representation (the tokenizer architectures are different). And in particular, PermaTok-SC achieves 74.7% identity preservation but shows little hidden-object recovery in probing. This raises questions about the relationship between hidden-object representations and generation performance. And, the lack of evaluation on realistic scenes and complex motion remains a limitation (this may need further reviewer-ac discussion).
I appreciate the authors' efforts, but I will keep my rating unchanged for now.