acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Geometric Alignment for Camera-Controlled Video Generation

Abstract

Camera controlled video generative models produce visually compelling results yet routinely violate the geometry of the scenes they depict. Some existing remedies inject 3D priors at training time, requiring generator-specific optimization. We instead treat geometric consistency as an inference-time alignment problem: scene observations and generated videos should depict the same underlying geometry, and embeddings from geometric foundation models provide a proxy for this agreement. We introduce GeoReward, a training-free, differentiable reward defined by geometry-embedding similarity between scene references and generated videos. The reward supports candidate search and gradient-based steering of denoising trajectories, enabling generation quality to improve as test-time compute scales. Empirically, GeoReward improves geometric consistency in both image-conditioned and video-conditioned generation across different base generators and diverse static and dynamic scenes. Video comparisons are available at https://georeward-review.github.io.

Primary area:
generative models
Keywords:
3D consistency, generative model, world model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_a9aXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GeoReward uses similarity between frozen geometric-model embeddings of reference and generated frames as an inference-time reward. It supports best-of-N selection, gradient guidance, and efficient search without updating the generator. Experiments on Geometry Forcing and ReCamMaster report improvements in reconstruction fidelity and geometric consistency, with additional analyses of reward features and computational cost.

Critical strengths

The reward is simple and easy to apply to frozen generators. ReCamMaster results are particularly useful because scoring uses the available source video rather than target ground truth. Best-of-8 selection reduces GeCo error on both controlled and natural scenes.

The paper provides substantial implementation and cost analysis, including multiple critics, two seed groups, camera-following measurements, and search variants. These experiments expose both benefits and limitations rather than reporting only favorable quality metrics.

Critical weaknesses

The RealEstate10K comparison does not match the available information. Appendix A.1 states that GeoReward uses held-out ground-truth target frames, whereas the baseline generates from one context image. The fidelity gains therefore combine additional scene observations with inference-time alignment. Without a context-only reward or a baseline given the same references, Table 1 does not establish improvement under the original image-conditioned setting.

The evidence that the reward improves geometry without compromising the requested view is incomplete. Guidance increases rotational ATE from 3.66° to 4.70°, with a paired confidence interval excluding zero (Table 14). Meanwhile, the appearance-feature controls differ in backbone size, preprocessing, and pooling, so they do not isolate the contribution of geometry training. These issues leave unclear how much the reward measures geometric fidelity versus reference appearance similarity.

Questions for authors

How does GeoReward perform on RealEstate10K when the reward uses only the input context image? Alternatively, can you compare methods with access to the same reference frames?

Can you compare the geometry-trained π³ encoder with its original DINOv2 ViT-L/14 counterpart using identical preprocessing and pooling?

Can guidance improve GeCo while maintaining camera adherence, for example by varying guidance strength and reporting the resulting trade-off?

2 comments

The matched DINOv2 comparison helps clarify the benefit of geometry training. However, the context-only gains remain small, and the weaker guidance has yet to be tested with best-of-8 to establish whether it preserves camera accuracy. These concerns still limit my confidence in the method’s applicability, so I’m keeping my score at 2.

AuthorsOpenReview ↗

Response to the remaining concerns

Dear Reviewer a9aX,

We sincerely thank you for your follow-up and for the time you have given to our paper. We are glad that "the matched DINOv2 comparison helps clarify the benefit of geometry training", and we will add it to the paper. We address the two remaining points below.

1. Context-only gains

  • The reason is the information available to the reward. With only the context image, the reward can check each candidate only against the view the generator already conditions on, so it adds little that the generator does not already have. A single additional view changes this: one held-out frame gives significant gains on every per-video metric.
  • The gain on RealEstate10K is also smaller because we chose a strong baseline. The baseline Geometry Forcing is already trained on the exact RealEstate10K dataset with geometry alignment to VGGT, so what we report on RealEstate10K is an additional gain at test time on top of a generator that has already been aligned with geometry.
  • Where GeoReward needs no extra observation, namely source-video conditioned generation, the reference is the generator's own input. There, best-of-8 reduces fused GeCo by 13.2% and 16.4% on controlled standard and OOD trajectories and by 6.2% and 10.3% on natural videos (Tables 2 and 3), without any target ground truth.

2. Best-of-8 with the weaker guidance

  • We acknowledge that η=2\eta = 2 with best-of-8 has not been tested directly. It requires 1,600 guided videos (two seed groups, 100 videos, eight candidates each), about 1,000 GPU hours, and we regret that we could not complete it within the discussion period.
  • We will run η=2\eta = 2 with best-of-8 on both seed groups and add it to Table 14 in the final version, whatever the outcome. If it shows a camera cost, we will recommend selection alone whenever camera accuracy is the priority, and state this in the paper.
  • The applicability of the method does not depend on guidance. Best-of-8 alone, the configuration behind our main results, has no detectable camera change (Table 14). Guidance is an optional addition for larger geometric gains, and it helps most when the camera trajectory is out of the training distribution, where the base generator already fails to follow the camera (Table 13). On the controlled OOD loops, guidance raises the GeCo reduction of best-of-8 from 16.4% to 20.9% (Table 2).

We are grateful for your careful engagement throughout the discussion, which has made the claims of the paper clearer and more precise. We sincerely hope that you will consider our overall contribution when making your final decision.

Reviewer_h3HYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GeoReward, a training-free method for improving geometric consistency in camera-controlled video generation. A frozen geometry model encodes reference frames and generated frames, and their feature similarity serves as a reward. This reward is used to select among candidate videos, guide denoising through gradients, or combine both approaches. The paper also explores early stopping and tree search to allocate inference compute. Experiments use Geometry Forcing on RealEstate10K and ReCamMaster on controlled and natural videos, including two out-of-distribution camera trajectories. The RealEstate10K reward uses additional ground-truth frames, while ReCamMaster uses only its source video. Candidate selection improves the reported geometric metrics across these settings. Guidance gives mixed additional gains, with higher estimated camera error for guidance plus search on natural standard trajectorie

Critical strengths
  1. Candidate selection gives useful improvements without updating the generator or training a reward model. On ReCamMaster, best-of-8 reduces mean per-trajectory fused GeCo error by 13.2% and 16.4% on controlled standard and OOD trajectories, and by 6.2% and 10.3% on natural videos (Tables 2–3). The natural-video results are particularly relevant because the reward uses the source video without target ground truth.

  2. The evaluation goes beyond a single sampling configuration. It includes two seed groups, harder loop trajectories, and several search budgets. The tree-search comparison in Table 5 is also useful: on five controlled scenes, it improves the pooled GeCo reduction from 17.5% to 23.9% at nearly matched denoiser work. Appendix B.5 reports the actual end-to-end cost, including the roughly 15% increase in GPU-seconds.

Critical weaknesses
  1. The RealEstate10K results leave the single-image setting untested. Appendix A.1 (p. 14) states that the reward uses held-out ground-truth frames; the main π³ configuration uses eight target frames, excluding the context image. Table 1 therefore evaluates generation with additional scene observations available to the reward. The reference ablations also retain target frames. This is a valid experimental setting, but the main text should make the extra input requirement explicit and clarify what performance can be expected when those observations are unavailable.

  2. Guidance introduces a trade-off on natural standard trajectories. Table 3 reports a 6.9% mean per-trajectory GeCo reduction for guidance plus search relative to the unguided baseline. However, Table 14 (p. 17) reports an increase in ViPE-estimated rotational ATE from 3.66° to 4.70°, with a paired 95% confidence interval of [+0.52°, +1.57°] for the increase. Search alone has no statistically detectable ATE change. The concern is therefore about the evaluated guidance configuration, and whether its geometric gains justify the loss in camera accuracy. The OOD pose results need further validation as well: Appendix B.4 acknowledges that the large loop-trajectory errors cannot distinguish generation errors from pose-estimation failure

Questions for authors
  1. What reference images are expected to be available when applying GeoReward to image-to-video generation? Could you report a context-image-only result on RealEstate10K? If the intended setting requires additional observations, could you also test references from separate views that are not frames of the target sequence?

  2. Can a different guidance strength retain the GeCo improvement without the ATE increase seen in Table 14? A joint comparison of GeCo and pose error across guidance strengths would clarify this trade-off. For the OOD loops, could you provide trajectory visualizations or an independent pose check to help separate camera-control errors from ViPE failures?

Reviewer_C2jDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GeoReward scores generated videos using feature similarity from a frozen geometry model, then improves results through candidate selection or denoising guidance. It needs no model retraining but relies on reference images: the Geometry Forcing experiments use additional ground-truth target frames, while ReCamMaster uses an existing source video.

Critical strengths
  1. With a source video available, candidate selection improves geometric consistency and several visual-quality measures without target-video ground truth or generator retraining (Tables 2–3).
  2. Comparisons of candidate selection, progressive search, and tree search include runtime and memory costs, making the extra computation easier to assess (Tables 5 and 15).
Critical weaknesses
  1. The information required goes beyond ordinary single-image generation. The Geometry Forcing generator receives one image, but its reward also uses ground-truth target frames. Table 1 therefore supports generation with extra references, without establishing the same benefit in ordinary single-image use. The source-video setting of ReCamMaster is more directly practical (Table 1; Appendices A.1 and B.1).
  2. The role of geometry training is not clearly separated. CLIP and DINOv2 features also improve results, while comparisons with the geometry model change both the backbone and feature processing. The evidence therefore does not clearly distinguish geometry training from generic visual similarity, weakening the central contribution (Table 4; Appendices A.5 and B.2).
  3. Better geometric scores can come with worse camera control. On natural videos with standard trajectories, guidance plus search raises rotational error from 3.66° to 4.70°. Better geometric consistency therefore need not mean better camera control, and the reward design should explain how it balances these goals. This concern mainly applies to guidance and does not negate the value of selection alone (Tables 3 and 14).
Questions for authors
  1. Which references must be available beforehand? How much of the single-image result holds without ground-truth target frames?
  2. What does geometry training add to the reward beyond ordinary visual similarity?
  3. Why can optimizing the reward move the video away from the requested camera path, and how does the method balance these goals?
1 comment
Reviewer_C2jDOpenReview ↗

reply to the rebuttal

Thanks for the additional experiments and explanations. I appreciate the effort. They clarify several points, but my main concerns remain.

The method’s scope is still fairly limited by the need for reference observations. Guidance can also improve geometric consistency while making camera control worse, and this issue has not been fully addressed. The new comparison supports using geometry features, but their added benefit over ordinary visual similarity still seems limited.

I see some practical value in the method, especially candidate selection. However, I am still not convinced that the overall contribution is strong enough. I will keep my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.