- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a post-training framework called DTTD (Dense Trajectory Teacher Distillation), designed to address specific limitations in video world models—namely, static imagery, inaccurate camera motion, and significant geometric drift—that arise when these models encounter sparse, large-stride camera movements.
Critical strengths
- This paper proposes a "Self-Teacher" approach, in which a foundation model acts as its own teacher; by leveraging the model's inherent familiarity with dense trajectories—facilitated through interpolation—it eliminates the need for additional data or external models.
- Experimental results demonstrate significant improvements in 3D consistency and trajectory-following capability, particularly in scenarios involving large-stride camera movements.
- A training strategy analogous to curriculum learning has been designed, with its effectiveness validated through experiments.
Critical weaknesses
- The method was only evaluated on ScanNet; its performance on other datasets remains unknown. Given that DL3DV features more unstable camera trajectories with a wider range of motion, why was it not tested on this dataset?
- Some of the examples produced by this method, as shown in Figure 4, are not of particularly high quality; for instance, the cabinet in the fifth row exhibits some deformation and distortion, while the canvas in the sixth row appears somewhat visually blurred.
- During the training phase, it appears that multiple candidate videos must be generated for each student scenario, with several "teacher" videos sampled for each dense trajectory. Furthermore, depth estimation and geometric re-projection evaluations must also be performed. This could lead to excessively high computational overhead and slow training speeds; however, the paper does not seem to report these metrics, nor does it provide comparisons with other methods in this regard.
Questions for authors
See Weaknesses