- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work presents Geo-Align, which leverages reinforcement learning to enhance camera-controlled video retake. Given an input video and a target camera trajectory, the method generates videos with novel viewpoint movements. Built upon ReDirector/Wan2.1-1.3B, the model undergoes post-training via GRPO. It integrates multiple metrics: 1) Rotation and translation errors estimated by MapAnything. 2) Visual and motion quality scores from VideoAlign. 3) Per-frame aesthetic scores from HPSv3. The training dataset does not require paired multi-view videos: CityWalk provides real conditional videos, and OmniWorld offers independently sampled and rescaled camera trajectories.
Critical strengths
-
Real synchronized multi-view videos are indeed scarce, and existing video-retake methods rely heavily on synthetic training data. Using verifiable geometric rewards to get around ground-truth videos for target viewpoints is a sensible and practically valuable research direction.
-
Combining real video appearances with independently sampled trajectories lowers the barrier for data collection, and also enables access to a richer set of real-world scenes and motion distributions. Compared with continuing to depend on paired data from Unreal Engine, this training paradigm scales more easily.
-
Instead of only optimizing camera errors, the authors add video-level quality, motion quality and per-frame aesthetic rewards at the same time. This mitigates image degradation caused by purely geometric rewards to some extent.
Critical weaknesses
-
The core components adopted in the paper are all existing methods, including ReDirector, MapAnything, MixGRPO, the normalization strategy from LongCat-Video, VideoAlign and HPSv3. The main distinction of this work lies in assembling these techniques for the video retake setting and its unpaired data strategy. While this combination is valuable, the claim of being the “first RL framework” requires stronger evidence.
-
MapAnything is designed for 3D reconstruction, yet generated videos may contain dynamic objects, non-rigid deformations, missing textures and generation artifacts. Under such conditions, the estimated “camera motion” can mix true camera movement, object motion and generation errors. The model may also learn to produce content that MapAnything interprets as matching the target trajectory, rather than preserving the actual scene geometry.
-
Beyond camera error metrics, most gains in Table 1 are marginal. At speed 1.5 in Table 3, RotErr is slightly worse than ReDirector. A blanket claim of overall SOTA cannot be made.
Questions for authors
-
How are the coordinate system, origin and first frame aligned between the target trajectory and the estimated trajectory before error computation?
-
Why does RotErr degrade slightly at speed 1.5, while the main text still claims superior performance over baselines across different speeds?