- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a scale-decoupled camera representation for camera-controlled text-to-video generation. It introduces two branches to encode camera trajectories and magnitudes. It also uses Trajectory Preference Optimization (TPO) (similar to Condition Preference Optimization) for post-training. The method is evaluated using a dataset assembled from multiple public video sources.
Critical strengths
The paper targets a meaningful and practical problem. The scales are ambiguious in camera-controlled video generation, especially for the Text to Video scenarios. Constructing preference pairs seems meaningful for post-training.
Critical weaknesses
- Incomplete manuscripts. It is mentioned in the paper that "Our Wan2.1 version is still training, and its results will be updated." This leaves part of the experimental discussion provisional.
- Insufficient differentiation or discussion, compared to CPO [1]. CPO (condition preference optimization) formulates preference learning over alternative control conditions (without cosntraining what formats of the conditions). Therefore, TPO seems like an application of CPO for the camera-conditioned video generation. However, CPO is very briefly cited but the proposed equations share very similar stuctures to CPO.
- Insufficiently controlled comparisons and unclear attribution of improvements. The comparisons involve different training datasets and backbones, while the training budgets and numbers of trainable parameters are not sufficiently documented to establish matched conditions. Especially, UCPE is a light trainable adapter and does not train on the proposed dataset. Moreover, UCPE achieves a translation error of 0.0104, compared with 0.0131 for the best proposed variant, where the translation measurements are closely realted to the aims of this paper.
Questions for authors
Please see the weakness section.
- More qualitative results are desired.
- Please also report the training configurations, including used datasets and trainable parameters of baselines. Moreover, more recent baselines, such as those from UCPE, could also be compared for better understanding.