acceptodds
Under review as a conference paper at ICLR 2027

ReView: Novel-View Video Generation with Spatiotemporal Consistency

11 views

Abstract

We study novel-view video generation: given an input video and a target camera viewpoint, the goal is to synthesize a high-quality video that faithfully replays the same dynamic event from the target view while preserving spatiotemporal consistency with the source video. We introduce ReView, a two-stage framework built on a pretrained video generation model. First, we adapt the model to novel-view generation through supervised training on a mixture of synthesized monocular-video pairs, simulated multi-view videos, and real-world multi-view images. We then apply reinforcement learning (RL) as a post-training stage to further improve spatiotemporal consistency. Specifically, we introduce a Cross-View Reprojection Reward that promotes cross-view geometric consistency through 3D reprojection and visual matching. To improve the efficiency of video RL, we further propose Progressive-Horizon Reinforcement Learning, which progressively shifts updates from single-frame to joint-video rollouts through single-frame, mixed, and full-video stages. This strategy attains a geometric consistency performance comparable to full-video RL using less than one third as many training compute. Experiments show improved viewpoint accuracy and cross-view consistency, with competitive novel-view reconstruction and video quality. These results suggest that pretrained video priors, together with multi-view supervision and geometry-aware post-training, offer a practical route toward more scalable 4D generation.

Primary area:
generative models
Keywords:
Video Generation, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_bfxVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents ReView for generating videos for novel view synthesis. It introduces a kind of complete pipeline for tuning video diffusion models into novel view synthesis model, including collecting data for finetune, reward training and RL for post training. Specifically, it introduces Cross-View Reprojection Reward and Progressive-Horizon Reinforcement Learning to improve the generation consistency. Experiments demonstrate the proposed model improves camera control ability and outperforms baseline models.

Critical strengths
  • The progressive frame-to-video curriculum provides a practical solution to the high cost of video RL, and comparisons with single-frame, fixed-mixture, direct-switching, and full-video training support its efficiency advantage.
  • The geometry-aware post-training improves both camera control and motion reconstruction, which demonstrates the effectiveness of the proposed method.
Critical weaknesses
  • The novelty relative to closely related work is not sufficiently clarified. IC-World[1] already applies GRPO with geometric and motion consistency rewards to multi-view video generation. I would recommend that the authors discuss the differences in reward design with IC-World, and clarify the contributions of the proposed method.
  • The method is evaluated only on a 5B backbone and underperforms Vista4D (14B) on five of the eight metrics in Table 3, despite its stronger geometric consistency. I would recommend that the authors provide comparisons using the same backbone and model size to clarify whether these quality differences arise from model capacity or the proposed method.

[1] IC-World: In-Context Generation for Shared World Modeling (ECCV2026)

Questions for authors

N.A.

4 comments

We thank the reviewer for the careful assessment and thoughtful feedback. Below, we clarify the differences from IC-World and provide additional ablation comparisons using the same backbone and model size as Vista4D.

W1. Distinction from IC-World: reward design and contributions

IC-World proposes to generate multi-view videos from a set of multi-view input images, using GRPO with geometric and motion consistency rewards. The key differences are the reference and target used for consistency supervision, how correspondences are established, and how the requested camera transformation is enforced.

Different task and reference. IC-World spatially arranges multi-view initial images into a grid along the height and width axes and jointly generates the corresponding videos using a single video diffusion model. Its rewards encourage agreement among these simultaneously generated views. Generation is conditioned on images and text, without an explicitly specified camera trajectory. ReView instead takes a complete observed source video and a prescribed camera transformation, and focuses on reproducing the same event at corresponding timestamps from the requested viewpoint. This provides a source observation at every output timestamp, enabling source-referenced and camera-constrained supervision, rather than only consistency among generated content.

Different geometric and motion rewards. IC-World’s geometry reward computes Chamfer distance between registered point clouds; its released implementation aggregates points across time and applies a single rigid transformation. Its motion reward matches trajectories by nearest-neighbor search over their temporal-mean 3D positions, then compares them at corresponding timestamps.

In contrast, CVRR uses SuperPoint and LightGlue to establish visual correspondences within synchronized source–target frame pairs, then evaluates reprojection residuals using estimated source depths and the requested camera transformation (Sec. 3.4.1 and Appendix B.3). It thus checks whether observed source content appears at the correct time and target-view location, without relying on trajectory centroids or fitting an alignment to generated outputs. Its frame-wise formulation also naturally supports PHRL’s single-frame, mixed, and joint-video updates, reducing the cost of video post-training.

We have added a comparison with IC-World, highlighting the task and reward differences, to Sec. 2 (Related Works) of the revised manuscript (in blue).

W2. Backbone-matched comparison with Vista4D

To address concerns about model capacity and video quality, we trained ReView on Wan2.1-14B, the same backbone and model size used by Vista4D, retaining our existing training settings without 14B-specific hyperparameter tuning.

Under the same evaluation protocol, ReView-14B improves over ReView-5B on all eight metrics and outperforms Vista4D on seven of the eight metrics (the CLIP-T score remains close to Vista4D's, 0.3265 vs. 0.3278). These results show that ReView is competitive in visual quality at a matched backbone and model size, without additional backbone-specific tuning, and support our main paper analysis that the quality differences mainly arise from model capacity.

New results on the reference-free benchmark (extended Table 3). Bold indicates the best result in each column.

MethodBase model sizeFID ↓FVD ↓CLIP-T ↑Aesthetic quality ↑Imaging quality ↑Subject consistency ↑Background consistency ↑Temporal style ↑
Vista4D14B90.4365246.70420.32780.60250.68520.96780.97880.2811
ReView-5B5B88.2768240.97330.32020.58150.68430.96820.97390.2799
ReView-14B (new)14B86.6596238.11770.32650.60880.69100.97920.97910.2835

Geometric consistency. ReView-14B outperforms Vista4D on viewpoint-control and cross-view consistency metrics, reducing RE@SG from 8.51 to 6.23. Note that the smaller ReView-5B already achieves an RE@SG of 6.73 and lower translation and rotation errors than Vista4D. These results show that ReView still maintains its geometric-consistency advantage when matched to Vista4D in backbone and model size, while also achieving improved video quality.

Viewpoint-control accuracy and cross-view 3D consistency. All metrics are lower-is-better. Bold denotes the best result in each column.

MethodTranslation numerical error ↓Translation direction error ↓Rotation error ↓RE@SG ↓
ReCamMaster0.3228.729.1228.64
TrajectoryCrafter0.2255.696.7521.29
EX-4D0.1985.016.1515.86
GEN3C0.1954.836.3613.79
SpaceTimePilot (new)0.2157.135.9216.10
Vista4D0.1874.275.078.51
ReView-5B0.1753.954.566.73
ReView-14B (new)0.1703.824.616.23

We have added the backbone-matched comparisons and analysis to Appendix E (Tables 6–7) of the revised manuscript (in blue).

Dear Reviewer bfxV,

Thank you for your valuable and constructive feedback. We have provided detailed responses and additional experimental analyses to address your concerns.

As the discussion period is nearing its end, we would greatly appreciate it if you could let us know whether our responses have adequately addressed your concerns.

Thank you so much for your time and consideration.

Best regards, The Authors

Dear Reviewer bfxV:

Thank you once again for your insightful feedback and for taking the time to review our work.

We hope that our responses have satisfactorily addressed your concerns. As the rebuttal deadline approaches, we would like to kindly inquire whether our replies have resolved your questions or if any points would benefit from further discussion.

Best Regards,

The Authors

Reviewer_5DteOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

ReView is a two-stage novel-view video generation framework that adapts a pretrained video model using a diverse multi-view dataset, followed by geometry-aware reinforcement learning. It introduces a Cross-View Reprojection Reward to strictly enforce spatiotemporal consistency alongside a Progressive-Horizon RL strategy that reduces training compute by over 66% through staged single-frame to full-video updates.

Critical strengths
  • The writing is clear and the paper is complete.
  • The paper compares with sufficient number of baselines with metrics.
  • The proposed Cross-View Reprojection Reward effectively enforces spatial and temporal consistency by pairing feature-matching with joint two-view geometry, penalizing content distortion and view-copying shortcuts.
  • Progressive-Horizon RL efficiently scales RL optimization by starting with low-cost single-frame rollouts
Critical weaknesses
  • Because this paper presents a video synthesis framework, the absence of video demonstration results makes it impossible for readers and reviewers to assess qualitative performance comparisons against baselines.
  • The manuscript should include a discussion comparing the proposed approach to SpaceTimePilot, which similarly addresses video recapturing when the initial viewpoint differs from the source video. Furthermore, please clarify whether this capability is primarily an architectural innovation or largely a function of the underlying training dataset distribution.
  • To address initial viewpoint mismatches, why not employ a two-stage approach: first, perform camera-controlled image-to-image editing on the initial frame (using existing state-of-the-art image models or fine-tuning on the proposed dataset), and then condition a video generator on this edited first frame for video re-angling? A discussion or baseline comparison against such a pipeline would strengthen the paper.
Questions for authors
  • For the modeling/conditioning part, what exactly is the difference against TrajectoryCrafter?
  • Is 'Self-Constructed Training Pair' the same strategy used in TrajectoryCrafter?
  • How sensitive is the Cross-View Reprojection Reward to inaccurate depth estimates or feature-matching failures in textureless or highly dynamic video regions?
  • What specific criteria or metric threshold determines the transition timing between stages in the three-stage schedule?
12 comments

We thank the reviewer for the helpful suggestions on evaluation and methodological clarification. Below, we address the video demonstrations, related methods, and implementation details.

W1. Video demonstrations and qualitative comparisons

We have updated the supplementary material with side-by-side video demonstrations, showing the source input on the left and ReView’s output on the right. These examples illustrate source-event preservation and spatiotemporal consistency under viewpoint changes. We also provide 3 × 3 comparison grids with simultaneous playback, containing the source input and outputs from ReView, Vista4D, SpaceTimePilot, Reangle-A-Video, EX-4D, TrajectoryCrafter, GEN3C, and ReCamMaster.

Following the comparison protocol of prior methods, we retain each method’s original inference settings, including a camera transition toward the target viewpoint where required by part of baselines. For assembling the comparison videos, outputs shorter than 61 frames are extended by holding their last generated frame, and videos with different aspect ratios are resized to a common display width.

W2. Comparison with SpaceTimePilot and the role of architecture versus data

Discussion and comparison on SpaceTimePilot. We thank the reviewer for highlighting SpaceTimePilot and have added discussions and quantitative comparisons to the revised manuscript. SpaceTimePilot aims to jointly control camera motion and animation time, while ReView focuses on synchronized novel-view replay, preserving the observed source event under a requested camera viewpoint. Both methods support video reshooting but use different conditioning and optimization mechanisms.

SpaceTimePilot adds the respective camera and animation-time embeddings to source and target video tokens, then concatenates the two streams along the frame dimension for joint DiT processing. ReView instead explicitly reprojects source content to its predicted target-view locations, constructs a projected video and visibility mask, and injects these priors through a multi-layer geometric control branch. Source-reference fusion blocks supply complementary appearance and motion context. Beyond these architectural designs, our geometry-aware post-training further optimizes source-content alignment under the requested camera transformation at corresponding timestamps. Together, this explicit target-view conditioning and source-referenced geometric optimization promote both geometric consistency and source-content fidelity.

New quantitative comparison. We additionally evaluate SpaceTimePilot under our evaluation protocol. ReView achieves lower errors on reported metrics, reducing rotation error from 5.92 to 4.56 (23.0%) and RE@SG from 16.10 to 6.73 (58.2%). These results demonstrate better viewpoint-control accuracy and cross-view 3D consistency.

MethodTrans. num. ↓Trans. dir. ↓Rot. ↓RE@SG ↓
ReCamMaster0.3228.729.1228.64
TrajectoryCrafter0.2255.696.7521.29
EX-4D0.1985.016.1515.86
GEN3C0.1954.836.3613.79
SpaceTimePilot (new)0.2157.135.9216.10
Vista4D0.1874.275.078.51
ReView0.1753.954.566.73

We have added this discussion to Sec. 2 and the SpaceTimePilot comparisons to Sec. 4.2 (Tables 1–3) of the revised manuscript (in blue).

Architecture and data make complementary contributions. The architecture determines how projected geometry and source-video context are used; the conditioning ablations in Q1 and Table 4 support these design choices. Moreover, synchronized multi-view data provide additional forms of training evidence (Sec. 3.3 and Appendix B.2):

  1. Independent target-view observations. Synchronized multi-view videos show the same event from different cameras, including surfaces and interactions hidden in the source view. They directly supervise how source appearance and motion should be reproduced from the target viewpoint, beyond what can be recovered by warping the source alone.
  2. Source-to-target visibility relationships. Double-reprojection pseudo-pairs simulate incomplete renderings by projecting a video to another view and back. Our paired videos additionally expose the model to conditions obtained by projecting one independently observed view toward another, together with its complete target observation. This pair-data supervision matches the source-to-target conditioning process used at inference and broadens training beyond round-trip visibility patterns.

We additionally include an ablation without our collected synchronized multi-view videos, retaining the other training settings, to assess their contribution separately (variant "w/o Synch. Data" in the table).

Ablation of conditioning architecture, synchronized multi-view data, and geometry-aware post-training. All variants except ReView-RL use supervised training only; ReView-RL further applies PHRL to ReView-Base. The first four metrics evaluate viewpoint control and cross-view consistency, and the remaining six evaluate novel-view synthesis on N3DV. The prefix "m" denotes covisibility-masked evaluation.

VariantTrans. num. ↓Trans. dir. ↓Rot. ↓RE@SG ↓mPSNR ↑mSSIM ↑mLPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
Camera Param.0.2977.509.2025.8914.150.4650.38614.110.4270.415
w/o Ref. Fusion0.2114.825.8015.2217.690.5890.30314.750.5050.396
TC's Condit.0.2225.586.8317.1917.250.6010.29915.830.5350.355
w/o Synch. Data0.2105.065.9515.1617.870.5920.30114.900.5620.377
Token Concat.0.2054.775.4914.5518.200.5970.31017.010.5950.320
ReView-Base0.2084.935.8914.3618.020.6100.29716.850.5920.324
ReView-RL0.1753.954.566.7319.560.6720.25617.350.5890.338

Adding the synchronized multi-view videos, ReView-Base improves all ten metrics over the “w/o Synch. Data” variant. In particular, full-frame PSNR increases from 14.90 to 16.85 dB (+1.95 dB), LPIPS decreases from 0.377 to 0.324 (14.1%), and RE@SG decreases from 15.16 to 14.36. These gains support the value of independently observed target-view supervision for both reconstruction quality and cross-view consistency.

We have added the conditioning and synchronized-data ablations, together with their settings and analysis, to Sec. 4.3 (Table 4) of the revised manuscript (in blue).

Reviewer_xsjmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce ReView, a framework for novel-view video generation that aims to reproduce the same dynamic event from a different camera viewpoint while maintaining spatiotemporal consistency with the source video. The method finetunes a pretrained Wan2.2-5B model to be capable of geometric control with source-reference attention using a mixture of monocular pseudo-pairs, simulated multi-view videos, and real-world multi-view images. The authors also proposes Cross-View Reprojection Reward (CVRR) to improve geometric consistency through reinforcement learning. To reduce the computational cost of video RL, they propose Progressive-Horizon Reinforcement Learning (PHRL). Experiments shows improved viewpoint accuracy, cross-view consistency, and motion reconstruction compared with existing methods.

Critical strengths
  1. The paper addresses an interesting and challenging problem of novel-view video generation: how to preserve the same dynamic event as the source while being able to perform free-form camera control. The proposed combination of geometric control and source-reference fusion is well motivated.

  2. The quantitative results are encouraging. ReView achieves the best viewpoint-control accuracy and cross-view reprojection consistency in Table 1.

  3. The proposed RL framework improves video RL efficiency through progressive training. Figure 5 demonstrates that PHRL can reach comparable geometric consistency to full-video RL with considerably less training computation.

Critical weaknesses
  1. Novelty concern. The proposed supervised pipeline closely resembles several recent works, particularly TrajectoryCrafter (ICCV 2025) and Vista4D (CVPR 2026), which also combine depth-based geometric reprojection, source-video conditioning, and multi-view supervision. ReView's fundamental design appears largely built upon established techniques.

  2. Ablation concern. Table 4 shows that the proposed architecture does not consistently outperform the simpler token-concatenation baseline before RL. I would suggest the authors to better clarify this issue.

  3. Novelty concern on geometry-aware RL. Recent works such as Geo-Align (2026), VGGRPO (ECCV 2026), VIGOR (ECCV 2026), and World-R1 (ICML 2026) have also explored geometry-aware rewards and RL-based post-training for video generation. Espeically, Geo-Align addresses camera-controlled video re-rendering through geometry-aware RL. I would encourage the authors to provide a more comprehensive comparison with these closely related approaches.

  4. Concern on cross-view reward. From the paper, CVRR mainly relies on feature correspondences and geometric reprojection errors between temporally aligned frames. It seems like that it does not explicitly evaluate whether the generated video preserves the complete motion dynamics of the source, especially in regions with occlusion or large viewpoint changes?

Questions for authors

See the weakness section.

5 comments

We thank the reviewer for the thoughtful evaluation and for recognizing the motivation of our task, the strong quantitative results, and the efficiency of PHRL. The comments have helped us clarify the architectural contributions and strengthen the experimental comparisons.

W1. Architectural novelty and distinction from prior methods

ReView shares geometric reprojection and source-video conditioning with prior methods, but differs significantly in how these signals are represented, processed, and integrated during denoising. Our task-specific design combines target-aligned geometric guidance with source appearance and semantic context for synchronized novel-view video generation. We clarify the distinctions from TrajectoryCrafter and Vista4D below.

Architectural distinction from TrajectoryCrafter. For projection conditioning, TrajectoryCrafter concatenates the projected-video latent, a downsampled visibility mask, and the noisy latent before patch embedding and DiT processing. ReView instead encodes the projection and visibility mask and processes them through a dedicated geometric control branch. Conditioned on the projection features, diffusion timestep, and text context, this branch injects residual control features at multiple backbone depths (Sec. 3.2 and Appendix B.1). This supplies spatiotemporally aligned geometric guidance throughout the network, rather than fusing the conditions only at the input, helping ground the generation to the requested target viewpoint.

For source-video conditioning, TrajectoryCrafter retrieves appearance information from fixed reference tokens through target-to-reference cross-attention. ReView's RefBlocks first apply timestep modulation and self-attention to the reference features, then cross-attention and an FFN to the target features. The updated reference representation is passed to subsequent RefBlocks within each denoising evaluation. Source conditioning thus adapts to both the diffusion timestep and feature depth through an evolving reference representation.

The projection-conditioning difference can be summarized as follows. Let ztz_t be the noisy target latent, PP the projected video, MM the visibility mask, cc the text context, and EE the respective method's VAE encoder. With channel concatenation denoted by [⋅;⋅][\cdot;\cdot], TrajectoryCrafter initializes the target stream as X0TC=ΦTC([zt;Resize⁡(M);E(P)])X_0^{\mathrm{TC}} = \Phi_{\mathrm{TC}}\bigl([z_t;\operatorname{Resize}(M);E(P)]\bigr). ReView separates target-stream initialization, X0=ΦWan(zt)X_0 = \Phi_{\mathrm{Wan}}(z_t), from geometric control, Hk=GGEO(X0,[E(P);E(M)],t,c)\\{H_k\\} = G_{\mathrm{GEO}}(X_0,[E(P);E(M)],t,c), and injects the resulting residuals at selected backbone depths: Xℓ+1=X^ℓ+λHπ(ℓ)X_{\ell+1} = \widehat X_\ell + \lambda H_{\pi(\ell)}.

Here, GGEOG_{\mathrm{GEO}} denotes the geometric control branch, X^ℓ\widehat X_\ell the target features after the backbone block and any RefBlock, and Hπ(ℓ)H_{\pi(\ell)} the residual injected at a selected layer with strength λ\lambda.

Quantitative support for the conditioning design. To further quantify this architecture difference, we added a TrajectoryCrafter-style conditioning variant to the supervised architecture ablation shown below, extending Table 4. Our proposed ReView-Base outperforms this variant (TC-style Cond. in the table) across the reported metrics, including a reduction in RE@SG from 17.19 to 14.36 (16.5% lower). This comparison supports the effectiveness of the proposed geometric control and source-reference fusion design.

VariantTrans. num. ↓Trans. dir. ↓Rot. ↓RE@SG ↓mPSNR ↑mSSIM ↑mLPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
Camera Param.0.2977.509.2025.8914.150.4650.38614.110.4270.415
w/o Ref. Fusion0.2114.825.8015.2217.690.5890.30314.750.5050.396
TC-style Cond. (new)0.2225.586.8317.1917.250.6010.29915.830.5350.355
Token Concat.0.2054.775.4914.5518.200.5970.31017.010.5950.320
ReView-Base0.2084.935.8914.3618.020.6100.29716.850.5920.324
ReView-RL0.1753.954.566.7319.560.6720.25617.350.5890.338

Architectural distinction from Vista4D. Vista4D concatenates source-video, point-cloud-render, and noisy tokens along the frame dimension for conditioning (Vista4D, Sec. 3.3). With three equally sized token streams, this triples the backbone sequence length and increases dense self-attention's pairwise interactions nine-fold. Our ReView instead injects geometric control and source-reference features at selected backbone depths, preserving the target-token sequence length in the backbone (Sec. 4.3 and Appendix B.1). This distinction is particularly beneficial to RL, where candidate rollouts repeatedly incur denoising costs.

The Token Concat. ablation (in Table 4 and the table above) examines this alternative conditioning strategy under our training setup, and we have added an inference-time comparison between this variant and ReView: ReView maintains competitive quality while reducing denoising time from 535 s to 92 s and total inference time from 832 s to 297 s. These measurements support the computational benefit of the design; this trade-off is also discussed in W2.

Contributions beyond architecture. Beyond architecture, ReView contributes synchronized dynamic multi-view data and task-specific geometry-aware post-training. Our Cross-View Reprojection Reward promotes source-event alignment under the requested viewpoint, while Progressive-Horizon RL reduces video-RL cost through progressive frame-to-video optimization.

We have expanded the architectural comparison in Appendix B.1 and added the conditioning ablation and efficiency analysis to Sec. 4.3 (Table 4) of the revised manuscript (in blue).

W2. Interpreting the architecture ablation and the quality–efficiency trade-off

Importantly, our architectural choice is supported by both the competitive supervised quality and substantially lower inference cost, as discussed in Section 4.3 of the main paper. Table 4 shows Token Concat. achieves performance comparable to ReView-Base before RL, but token concatenation expands the sequence processed by the backbone's attention and significantly increases the calculation time cost. We add an inference-time comparison below: Our method reduces denoising time by 82.8% and total inference time by 64.3%, corresponding to 5.82× and 2.80× speedups, respectively, supporting this quality-efficiency trade-off. These savings, together with the competitive supervised results, motivate our conditioning design choice. These efficiency advantages are particularly relevant to RL post-training, where repeated candidate generation makes denoising efficiency especially important.

Conditioning designDenoising time (s) ↓Total inference time (s) ↓
Token Concat.535832
ReView92297
Speedup with ReView5.82×2.80×

We have added the inference-time comparison and the corresponding quality–efficiency discussion to Sec. 4.3 of the revised manuscript (in blue).

W3. Distinction from geometry-aware post-training methods

We thank the reviewer for highlighting these related works. Our CVRR is designed to transfer an observed source event video under a requested novel viewpoint, providing a different reward target from within-video geometric consistency or camera-trajectory agreement.

VGGRPO, VIGOR, and World-R1 primarily assess geometric self-consistency within generated videos, whereas ReView's video-to-video formulation reinforces cross-view consistency between the generated output and the observed source video. Specifically, VGGRPO uses latent-space 4D self-consistency, VIGOR compares projected and tracked points across generated frames, and World-R1 combines agreement between a video and its reconstructed 3DGS renders with requested-camera rewards. ReView instead has an observed source reference at every output timestamp. CVRR therefore evaluates whether the generated novel view preserves the source event, beyond whether the output is internally consistent.

Geo-Align proposes a geometric reward that evaluates agreement between estimated and requested camera trajectories, whereas CVRR combines visual and geometric constraints to reward source-content alignment under the requested viewpoint. CVRR combines visual feature matching with geometric reprojection: visual correspondences identify matching source content in the generated frame at the same timestamp, while reprojection determines where that content should appear under the requested transformation. The discrepancy between the visually matched and geometrically projected locations provides point-level supervision that couples visual correspondence with geometric placement. Thus, CVRR rewards source-content alignment under the requested viewpoint, rather than camera-trajectory agreement alone.

Besides, our PHRL further complements CVRR by addressing the cost of video RL: it retains CVRR while progressively reallocating computation from single-frame to joint-video rollouts. Thus, ReView contributes both source-event-aligned reinforcement and an efficient post-training strategy tailored to this task.

We have added the comparison with these geometry-aware post-training methods to Sec. 2 (Related Works) of the revised manuscript (in blue).

W4. Temporal consistency and occluded regions

Temporal alignment is supported by two complementary mechanisms: source-reference fusion supplies appearance and motion context from the input video, while CVRR compares each generated frame with the source frame at the same timestamp. Frozen or delayed motion can therefore increase the reprojection residual when matched moving content deviates from its expected target-view position. Appendix D and Fig. 6 illustrate how CVRR penalizes these temporal mismatches.

The motion-reconstruction evaluation results and VBench scores in the main paper provide complementary evidence of temporal quality. Table 2 reports EPE between the inter-frame optical flows estimated from generated and synchronized ground-truth target videos. ReView achieves the lowest EPE on both datasets: 2.078 on N3DV and 1.093 on iPhone, compared with Vista4D’s 2.356 and 1.175, respectively. On VBench, ReView is also among the top-performing methods in subject consistency, background consistency, and temporal style (Table 3). Together, these results support improved source-motion fidelity alongside strong temporal consistency and quality.

For disoccluded content, the key distinction is between target-view supervision during training and source-view observations at inference. Our synchronized multi-view data provide training targets for content hidden in the source. At inference, target-view content unobserved throughout the source video must be generated rather than recovered by direct reprojection. ReView guides this completion through multi-view training, source-reference context, and the pretrained video model's generative prior, encouraging plausible appearance and temporally coherent motion. CVRR anchors the co-visible, matchable content, while these complementary mechanisms guide completion beyond its coverage.

We have added a discussion of temporal quality and the two complementary mechanisms to Sec. 4.2 of the revised manuscript (in blue).

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response and Summary of Revisions

Dear Area Chair and Reviewers,

We sincerely thank you for your time and constructive feedback. We appreciate the reviewers’ recognition of the task motivation, the strong viewpoint-control and motion-reconstruction results, and the practical efficiency of PHRL. These comments have helped us clarify our contributions and strengthen the experimental evaluation.

We have provided point-by-point responses, conducted additional experiments, and updated the manuscript and supplementary material with manuscript revisions highlighted in blue. We summarize the discussions and updates below:

  1. Novelty clarification. We clarified ReView’s architectural distinctions from TrajectoryCrafter in its multi-layer geometric control and evolving source-reference features (R1–W1; R2–Q1), and from Vista4D in its conditioning design, which avoids expanding the backbone’s target-token sequence (R1–W1). We also clarified how our cross-view reprojection reward (CVRR) combines visual correspondence and geometric reprojection to optimize source-content alignment, beyond generated-video consistency or camera-trajectory agreement alone. (R1–W3; R3–W1) Additional controlled ablations provide quantitative support for our conditioning design. (R1–W1; R2–Q1) These clarifications appear in Sec. 2 and Appendix B.1, with supporting ablations in Sec. 4.3 (Table 4).

  2. Additional ablations and controlled comparisons. We conducted an additional TrajectoryCrafter-style conditioning ablation to assess our architectural design under the same supervised training setup. (R1–W1; R2–W2; R2–Q1) The synchronized-data ablation separately assesses the contribution of independently observed multi-view targets beyond double-reprojection pseudo-pairs. (R2–W2; R2–Q2) Besides, our method ReView maintains competitive supervised quality while achieving 5.82× faster denoising than the token-concatenation variant. (R1–W2) ReView also achieves better overall video quality and geometric consistency than prior method when both use the same Wan2.1-14B backbone. (R3–W2) The additional results are reported in Sec. 4.3 (Table 4) and Appendix E (Tables 6–7).

  3. Expanded baseline comparisons and evaluation of two-stage alternatives. We added quantitative comparisons with SpaceTimePilot, showing improved viewpoint-control accuracy and cross-view consistency. (R2–W2) We also compared ReView with three first-frame-editing pipeline variants and Reangle-A-Video, demonstrating the advantages of our unified video-to-video pipeline. (R2–W3) These comparisons are included in Sec. 4.2 (Tables 1–3) and Appendix F (Tables 8–9).

  4. Expanded the temporal-consistency analysis and post-training details. We clarified how source-reference fusion and synchronized reward evaluation support temporal fidelity, with complementary evidence from motion-reconstruction and VBench results. (R1–W4) We also documented CVRR’s observed sensitivity to depth and matching errors, together with the corresponding reward safeguards and RL-data selection practices. (R2–Q3) We clarified the empirical criterion for switching between PHRL stages. (R2–Q4) These discussions have been incorporated into Sec. 4.2 and Appendices B.3–B.4.

  5. Video demonstrations and qualitative comparisons. We updated the supplementary material to include side-by-side source/output demonstrations and synchronized comparison grids with seven baseline methods, enabling direct inspection of source-event preservation and novel-view generation quality. (R2–W1)

We would be grateful if you could revisit our responses and revisions and let us know whether the additional evidence and clarifications address your concerns. We welcome further discussion and would be happy to clarify any remaining questions.

Thank you again for your time and consideration.

Best regards,
The Authors

AuthorsOpenReview ↗

Summary of Discussions and Revisions for the Area Chair (1/2)

Dear Area Chair,

As the author–reviewer discussion period comes to a close, we thank you and the reviewers for your time and constructive feedback. As noted, Reviewer 5Dte found our RL-driven optimization interesting and raised score from 2 to 3. While we have not yet received follow-up comments from Reviewers bfxV and xsjm, we summarize our contributions, point-by-point responses, and corresponding manuscript revisions below to support your final assessment.

1. Novelty Clarification and Distinctions from Prior Work

Direct synchronized novel-view replay. In contrast to previous trajectory-controllable video generation methods based on gradual, high-overlap camera transitions, our method, ReView, tackles direct novel-view event replay with an emphasis on source-reference consistency. This requires jointly addressing viewpoint control, source-content fidelity, disocclusion, and motion synchronization. Our evaluations demonstrate leading viewpoint-control accuracy, cross-view 3D consistency, and motion fidelity, including the lowest optical-flow EPE on both evaluated datasets.

Coordinated multi-view supervision, efficient conditioning, and RL post-training. Our ReView is designed as a unified framework in which task-specific data curation, supervised conditioning, and post-training make complementary contributions. Our game-engine and robot-simulation data pipelines provide synchronized multi-view observations, including ground-truth target content hidden in the source view that monocular double-reprojection pseudo-pairs cannot provide. Our multi-layer geometric residual control and evolving source-reference fusion improve supervised quality over TrajectoryCrafter-style conditioning, reducing RE@SG by 16.5%. By preserving the backbone's target-token sequence length, our design also achieves 5.82× faster denoising than the token-concatenation variant in our inference benchmark (92 s vs. 535 s). These results support both the supervised initialization and the computational efficiency needed for repeated RL rollouts.

Task-specific reward design and efficient video RL. Existing geometry-aware post-training methods emphasize internal 3D/4D consistency among generated frames/views (VGGRPO, VIGOR, World-R1, IC-World) or camera-trajectory agreement (Geo-Align, World-R1). Our Cross-View Reprojection Reward (CVRR) instead exploits the observed source video as reference at every timestamp. It couples visual feature matching with 3D geometric reprojection to evaluate whether matched source content appears at the corresponding time and requested target-view location. This provides a source-referenced signal for reducing content distortion, temporal lag, and camera drift. Complementing this reward, Progressive-Horizon RL (PHRL) progressively transitions through single-frame, mixed, and full-video updates. Supported by our theoretical analysis and controlled ablations, this strategy achieves geometric consistency comparable to full-video RL with less than one-third of the training compute. Together, CVRR and PHRL reduce RE@SG by 53.1% relative to our supervised base.

AuthorsOpenReview ↗

Summary of Discussions and Revisions for the Area Chair (2/2)

2. Resolution of Concerns from Reviewers

A. Reviewer bfxV (Rating: 3 — Weak Acceptance)

Reviewer bfxV appreciated our PHRL's practical efficiency and our geometry-aware post-training. We addressed their two concerns as follows:

Distinction from IC-World (W1). In Sec. 2, we clarified that IC-World encourages consistency among jointly generated views, whereas ReView evaluates generated content against an observed source video under a prescribed camera transformation. Our response further distinguishes IC-World's registered-point-cloud and trajectory-based rewards from CVRR's visual matching and geometric reprojection within synchronized source–target frame pairs. CVRR's frame-wise formulation also naturally supports PHRL's frame-to-video curriculum.

Backbone-matched comparison with Vista4D (W2). To address the possible effect of model capacity, we trained ReView on Wan2.1-14B, matching Vista4D's backbone and model size, without 14B-specific hyperparameter tuning. Appendix E (Tables 6–7) reports that ReView-14B outperforms Vista4D on seven of eight video-quality metrics, including FID (86.66 vs. 90.44), FVD (238.12 vs. 246.70), and all five reported VBench dimensions. ReView-14B further improves geometric consistency, achieving an RE@SG of 6.23, compared with 8.51 for Vista4D and 6.73 for ReView-5B. These results demonstrate that our geometric-consistency advantage is retained at matched capacity, alongside improved video quality.

B. Reviewer xsjm (Rating: 2 — Weak Rejection)

Reviewer xsjm recognized our task motivation, strong quantitative results, and PHRL's efficiency. We addressed all four of their concerns with additional ablations and analysis:

Architectural distinctions from TrajectoryCrafter and Vista4D (W1). Appendix B.1 clarifies our multi-layer geometric residual injection and timestep- and depth-adaptive source-reference fusion. The controlled TC-style conditioning ablation in Sec. 4.3 (Table 4) shows that ReView-Base outperforms the TC-style variant on all ten reported metrics, including 16.5% lower RE@SG and 1.02 dB higher PSNR. We also clarify how our conditioning differs from Vista4D-style token concatenation.

Quality–efficiency trade-off (W2). The inference-time comparison in Sec. 4.3 supports our conditioning design by demonstrating lower inference cost while maintaining supervised quality comparable to the token-concatenation variant. ReView reduces denoising time by 82.8% (535 s to 92 s; 5.82× speedup) and total inference time by 64.3% (832 s to 297 s; 2.80× speedup). This efficiency benefits practical inference and is particularly relevant to repeated candidate rollouts during RL.

Comparison with geometry-aware RL methods (W3). In response to Reviewer xsjm, we expanded Sec. 2 to discuss VGGRPO, VIGOR, World-R1, and Geo-Align. These comparisons clarify CVRR's source-referenced visual and geometric supervision, beyond internal consistency of generated outputs or camera-trajectory agreement alone.

Temporal consistency and occluded regions (W4). Sec. 4.2 and Appendix D (Fig. 6) clarify how same-timestamp matching supports motion synchronization, with complementary evidence from the lowest optical-flow EPE on both datasets in Table 2. We also explain how multi-view supervision, source-reference context, and the pretrained video prior guide completion in disoccluded regions.

3. Discussion and Revisions with Reviewer 5Dte (Score: 2 → 3)

We provided video demonstrations in the Supplementary Material (W1), added comparisons with SpaceTimePilot (W2), and evaluated Reangle-A-Video and three first-frame-editing–plus–video-generation variants (W3). The results in Sec. 4.2 and Appendix F (Tables 8–9) demonstrate ReView's advantages in viewpoint control and cross-view consistency, including a 58.2% reduction in RE@SG relative to SpaceTimePilot. We also clarified the conditioning differences from TrajectoryCrafter (Q1), described the double-reprojection strategy in Sec. 3.3 (Q2) while distinguishing our additional synchronized multi-view supervision (Q2), detailed CVRR's sensitivity and robustness safeguards in Appendix B.3 (Q3), and explained PHRL's validation-based stage switching in Appendix B.4 (Q4).

Following the reviewer's follow-up, we highlighted ReView's learned first-frame viewpoint transformation capability and the two-stage ablation findings in Sec. 4.3, and clarified the supporting image/video training details and connection to PHRL in Appendix F.

We have provided a point-by-point response to each review and incorporated the corresponding clarifications and additional evidence into the revised manuscript, with revisions highlighted in blue. We hope this summary helps clarify the scope of ReView's contributions and the evidence supporting them. Thank you again for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.