- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes VTok, a unified video tokenization framework for both video understanding and text-to-video generation. Its core idea is to decouple spatial and temporal information in videos by preserving spatial tokens from a single key frame while encoding changes in subsequent frames relative to the key frame as compact residual motion tokens, thereby reducing the token complexity from to . These video tokens are then integrated into a unified autoregressive multimodal language model, enabling the same framework to support both video-to-text understanding and text-to-video generation. The paper evaluates the proposed method on multiple video understanding benchmarks, VBench, and the newly introduced TV-Align benchmark.
Critical strengths
None
Critical weaknesses
The paper contains a substantial amount of content that resembles AI-generated “autoresearch”-style output, particularly in the formatting and presentation of tables, table captions, and figure captions. Moreover, the paper provides no appendix or supplementary material. In my view, the current experimental evaluation is far from sufficient to support the conclusions claimed in the paper.
Questions for authors
-
Technical clarification on visual token generation. VTok represents video tokens as continuous vectors, while the generation objective is formulated as an autoregressive language-modeling loss over visual tokens. Could the authors clarify how these continuous visual tokens are actually predicted during generation, e.g., through quantization, a codebook, or an explicit continuous probability distribution? In addition, when the video decoder is frozen, which parameters receive gradients from ?
-
Clarification on the scope of AI use. The paper’s AI use statement states that generative AI was used only for language polishing and LaTeX formatting. However, a substantial amount of content in the manuscript, including tables, table captions, figure captions, and parts of the experimental descriptions, exhibits strongly templated patterns, raising concerns that broader AI-assisted research or “autoresearch”-style tools may have been used. Could the authors explicitly clarify whether, beyond the disclosed uses, AI tools were involved in research ideation, experimental design, method implementation, data analysis, result interpretation, or the generation of tables and figures?