acceptodds
Under review as a conference paper at ICLR 2027

Geo-Align: Video Generation Alignment via Metric Geometry Reward

Abstract

Camera-controlled video generation has achieved remarkable progress in recent years. However, existing video-to-video re-rendering methods primarily rely on Supervised Fine-Tuning using synthetic datasets. At present, there is an extreme scarcity of synchronized, multi-view real-world video data. Consequently, the prevailing paradigm often exhibits limited generalization when processing out-of-distribution real-world videos, with models struggling to accurately adhere to physical scales and camera trajectories. To bridge this gap, we propose Geo-Align, the first Reinforcement Learning framework specifically designed for camera-controlled video re-rendering. Built upon a pretrained flow matching model, we optimize the model through a scale-aware perceptual reward mechanism. Specifically, we introduce a metric 3D estimator to extract precise camera trajectories from generated videos, explicitly penalizing deviations in rotation and translation. Furthermore, we meticulously designed a data pipeline strategy based on real-world conditioning videos and target camera trajectories derived from synthetic data. Extensive experiments demonstrate that Geo-Align significantly outperforms existing supervised learning baselines in both precise camera controllability and visual fidelity, indicating the effectiveness of our method.

Primary area:
reinforcement learning
Keywords:
RL, Video Generation, Computer Vision

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6FwZOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work presents Geo-Align, which leverages reinforcement learning to enhance camera-controlled video retake. Given an input video and a target camera trajectory, the method generates videos with novel viewpoint movements. Built upon ReDirector/Wan2.1-1.3B, the model undergoes post-training via GRPO. It integrates multiple metrics: 1) Rotation and translation errors estimated by MapAnything. 2) Visual and motion quality scores from VideoAlign. 3) Per-frame aesthetic scores from HPSv3. The training dataset does not require paired multi-view videos: CityWalk provides real conditional videos, and OmniWorld offers independently sampled and rescaled camera trajectories.

Critical strengths
  • Real synchronized multi-view videos are indeed scarce, and existing video-retake methods rely heavily on synthetic training data. Using verifiable geometric rewards to get around ground-truth videos for target viewpoints is a sensible and practically valuable research direction.

  • Combining real video appearances with independently sampled trajectories lowers the barrier for data collection, and also enables access to a richer set of real-world scenes and motion distributions. Compared with continuing to depend on paired data from Unreal Engine, this training paradigm scales more easily.

  • Instead of only optimizing camera errors, the authors add video-level quality, motion quality and per-frame aesthetic rewards at the same time. This mitigates image degradation caused by purely geometric rewards to some extent.

Critical weaknesses
  • The core components adopted in the paper are all existing methods, including ReDirector, MapAnything, MixGRPO, the normalization strategy from LongCat-Video, VideoAlign and HPSv3. The main distinction of this work lies in assembling these techniques for the video retake setting and its unpaired data strategy. While this combination is valuable, the claim of being the “first RL framework” requires stronger evidence.

  • MapAnything is designed for 3D reconstruction, yet generated videos may contain dynamic objects, non-rigid deformations, missing textures and generation artifacts. Under such conditions, the estimated “camera motion” can mix true camera movement, object motion and generation errors. The model may also learn to produce content that MapAnything interprets as matching the target trajectory, rather than preserving the actual scene geometry.

  • Beyond camera error metrics, most gains in Table 1 are marginal. At speed 1.5 in Table 3, RotErr is slightly worse than ReDirector. A blanket claim of overall SOTA cannot be made.

Questions for authors
  • How are the coordinate system, origin and first frame aligned between the target trajectory and the estimated trajectory before error computation?

  • Why does RotErr degrade slightly at speed 1.5, while the main text still claims superior performance over baselines across different speeds?

Reviewer_EzeKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Geo-Align, a reinforcement learning-based framework for camera-controlled video re-rendering, aiming to address the reliance of existing SFT methods on synthetic multi-view data, their limited generalization to real-world scenes, and inaccurate camera motion scales. The core contribution is a Geometry-aware Reward, which uses MapAnything to estimate camera trajectories from generated videos and constructs geometric rewards based on rotation and translation errors. Combined with perceptual quality rewards from VideoAlign and HPSv3, the framework employs GRPO to optimize the video generation model.

Critical strengths
  1. The paper addresses the scarcity of real-world multi-view video data and the inaccuracy of camera trajectory scales, which has practical application value.

  2. By combining geometric constraints with perceptual quality rewards, the method can simultaneously optimize camera trajectory control accuracy and the visual quality of generated videos.

  3. The experimental results and the information presented in the paper are relatively comprehensive.

Critical weaknesses
  1. The performance improvements are relatively limited, while the training cost is high. In Table 1, compared with ReDirector, TransErr decreases from 0.0149 to 0.0129, and RotErr decreases from 1.4635 to 1.3645, while improvements in other visual quality metrics are generally small. The training requires 64 A800 GPUs for approximately 130 hours, but the performance gains are relatively modest.

  2. Ablation experiments for key components are seriously lacking. Table 2 compares the visual quality reward with the full reward, but many important experiments are missing, such as using only Geometry Reward, removing Truncated Gaussian Sampling, and removing Temporal Weighting.

Questions for authors
  1. MapAnything may produce trajectory estimation errors in dynamic scenes or under large camera motions. Could these errors affect the accuracy of the RL rewards? Have the authors validated the reliability of the Geometry Reward?

  2. Compared with CamPilot, an existing reinforcement learning method based on geometric rewards, what are the key differences of Geo-Align in terms of reward design and optimization mechanisms? This is important for evaluating the novelty of the proposed method.

  3. The current quantitative experiments are mainly conducted on the DAVIS dataset. I am curious whether the proposed method can maintain similar performance improvements on other real-world video datasets, more complex dynamic scenes, and longer video sequences.

Since there is no borderline option in the current rating scale, I assigned this score.

Reviewer_yz3rOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Geo-Align, a framework for aligning generated videos with target camera trajectories through metric geometry rewards. It combines real-world conditioning videos with synthetic camera trajectories, rescales trajectories to realistic motion ranges, and uses GRPO to optimize geometric alignment alongside aesthetic quality. Experiments demonstrate improvements over ReDirector on several camera-control metrics.

Critical strengths
  1. Promising experimental results. The proposed method improves translation and rotation errors over ReDirector while maintaining competitive visual quality.
Critical weaknesses
  1. Insufficient hyperparameter analysis for trajectory rescaling. The proposed trajectory rescaling strategy depends on the sampled maximum translation and rotation speeds τtrans\tau_{\mathrm{trans}} and τrot\tau_{\mathrm{rot}}, which directly determine the scaling factors stranss_{\mathrm{trans}} and srots_{\mathrm{rot}}. These parameters affect the magnitude of camera motion, trajectory diversity, and potentially the final camera alignment performance. However, the paper lacks a systematic sensitivity analysis of these parameters. Ablations with different translation and rotation speed ranges would help assess the robustness of the proposed sampling strategy and its impact on model performance.

  2. Lack of convergence analysis. The paper does not provide training curves or convergence analysis for GRPO optimization. Without tracking the evolution of geometry and aesthetic rewards, it is difficult to assess convergence behavior and whether the two objectives remain balanced throughout optimization.

Questions for authors
  1. Could the authors provide a sensitivity analysis of the sampled maximum translation and rotation speeds (τtrans\tau_{\mathrm{trans}} and τrot\tau_{\mathrm{rot}}) to evaluate their impact on trajectory diversity and camera alignment performance?

  2. Could the authors provide training curves showing the evolution of geometry and aesthetic rewards during GRPO optimization to clarify convergence behavior and training stability?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.