- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes GAD, a method that can improve the geometry consistency in the generated video for diffusion models. The authors identfy that the current model cannot identify reliable geometry correspondences in attention block. Therefore, they propose to distills the correspondences from a pretrained geometry model (VGGT-omega) into the attention of a video diffusion model. To make it work, this paper introduces a few techniques (e.g. confidence-weighted supervision and noise-dependent layer selection) for better distillation. Experiments are conducted to domenstrate the design choice.
Critical strengths
The paper is well-written and easy to follow. The method seems effective, and the visual results show improvements in geometry consistency. I appreciate the careful design, and the ablations provide support for several design choices.
Critical weaknesses
I still have several concerns:
- The method is a bit heuristic to me, as it explicitly injects correspondences from a geometry model to constrain the video model’s attention, and the way they are injected (e.g., layer selection based on visualization) is also pretty heuristic. I am wondering whether this affects the general generation ability of the original video model, as the injection may destroy attention patterns naturally learned from data. These cases are not sufficiently evaluated.
- There are other ways to introduce geometric information, such as RL with geometry-based rewards. The paper discusses VGGRPO and World-R1 but does not compare against them. They can be viewed as an implicit way of injecting geometry information from a geometry model. I would like to see a comparison under matched settings.
- The improvements are only shown on relatively small models, i.e., Wan2.1-1.3B and Wan2.2-5B. In my experience, larger models often already have much better geometric consistency. Geometry modeling ability can also be largely limited by model size. Would this method still provide clear gains on a stronger 14B model?
Questions for authors
How is the camera trajectory encoded and injected into Wan, and which initial checkpoints are used?
4 comments
Response to The Reviewer (2 / 3)
2. Whether this affects the general generation ability of the original video model?
I am wondering whether this affects the general generation ability of the original video model, as the injection may destroy attention patterns naturally learned from data. These cases are not sufficiently evaluated.
No, our method improves consistency without compromising the original video model’s generation ability. As explained in our response to the previous question, cross-frame attention supports consistent generation by gathering information from correctly associated regions across frames. Our method strengthens this mechanism by using a geometry teacher with more reliable correspondences to guide the video model’s attention. Unlike feature-level alignment approaches such as GeometryForcing and Gen3R, GAD supervises cross-frame relations without requiring the video model to reproduce the teacher’s feature representations. This preserves flexibility in learning representations suited to video generation.
Quantitative and qualitative results under matched training settings support this design. Tables 1–2 show improved FID, FVD, and CLIP over the matched Wan baseline across both backbones and all three datasets. For example, on OmniWorld-Game with Wan2.2 5B, FID improves from 61.29 to 59.52, FVD from 657.6 to 622.3, and CLIP from 27.46 to 28.23; motion smoothness, aesthetic quality, and image quality also improve. The supplementary Wan/Ours comparisons further demonstrate preserved appearance and dynamics in nonrigid flames (Dynamic Scene 1), running with displaced snow (Dynamic Scene 3), running across grass (Dynamic Scene 7), and foliage and grass under viewpoint changes (Static Scenes 1 and 3). Together, these results show that our method preserves diverse appearance and dynamics.
3. Comparison with RL-based methods VGGRPO and World-R1.
There are other ways to introduce geometric information, such as RL with geometry-based rewards. The paper discusses VGGRPO and World-R1 but does not compare against them. They can be viewed as an implicit way of injecting geometry information from a geometry model. I would like to see a comparison under matched settings.
We agree that VGGRPO and World-R1 are valuable baselines. Regarding availability, we could not find an official implementation or pretrained checkpoints linked from the VGGRPO [1]. World-R1 [2] has released training and inference code, but we could not find released model checkpoints or LoRA weights in its official repository. World-R1 reports using 48 and 96 H200 GPUs to train its 1.3B and 14B variants, respectively. These RL-based methods require generating video samples before evaluating their geometry-based rewards, incurring substantial computation and training time. Given these availability and computational constraints, we were unable to reproduce these methods for direct comparison within the rebuttal period.
While these availability constraints limit our ability to conduct direct comparisons, our method offers two key methodological advantages over these RL-based approaches: more direct supervision of cross-frame associations and a simpler training pipeline without online reward evaluation. First, our method provides explicit guidance on where to match across frames. VGGRPO and World-R1 optimize generated videos through aggregated rewards, without explicitly specifying which locations should correspond. Our method instead uses location-specific correspondence targets to supervise the video attention governing cross-frame associations, directly targeting incorrect matches that undermine consistency. Second, our method avoids computationally expensive and time-consuming online geometry-based reward evaluation. These RL-based methods must first generate video samples before evaluating their geometric rewards, adding substantial computation and slowing training. Our teacher correspondences are extracted offline and cached for reuse during flow-matching fine-tuning, avoiding repeated online video sampling and reward computation and simplifying the training pipeline.
Our correspondence supervision is also complementary to RL-based optimization and could be combined with geometry-based rewards to further improve consistency.
[1] VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward. https://arxiv.org/abs/2603.26599
[2] World-R1: Reinforcing 3D Constraints for Text-to-Video Generation. https://arxiv.org/abs/2604.24764
Response to The Reviewer (3 / 3)
4. Would this method still provide clear gains on a stronger 14B model?
The improvements are only shown on relatively small models, i.e., Wan2.1-1.3B and Wan2.2-5B. In my experience, larger models often already have much better geometric consistency. Geometry modeling ability can also be largely limited by model size. Would this method still provide clear gains on a stronger 14B model?
Yes, our additional experiments show that GAD provides consistent gains on the stronger Wan2.1 14B model. Following exactly the same experimental setup as in the original paper, GAD and the Wan baseline use identical initialization, training data, training budget, and evaluation settings, with the baseline trained using flow matching alone.
For long-range generation, GAD improves all seven metrics on both RealEstate10K and DL3DV-10K. Rotation error decreases by 41.7% / 44.7%, and matching hit rate increases by 15.3% / 7.7% relative, respectively, alongside improvements in FID, FVD, and CLIP.
| Static dataset | Method | FID ↓ | FVD ↓ | CLIP ↑ | R_dist ↓ | T_dist ↓ | M_rate ↑ | M_dist ↓ |
|---|---|---|---|---|---|---|---|---|
| RealEstate10K | Wan2.1 14B | 24.80 | 201.8 | 32.00 | 4.724 | 0.078 | 0.413 | 30.09 |
| RealEstate10K | + GAD | 23.15 | 194.8 | 32.14 | 2.754 | 0.058 | 0.476 | 26.01 |
| DL3DV-10K | Wan2.1 14B | 31.02 | 255.8 | 34.05 | 5.497 | 0.083 | 0.365 | 39.68 |
| DL3DV-10K | + GAD | 28.66 | 228.8 | 34.34 | 3.039 | 0.046 | 0.393 | 34.10 |
These benefits also extend to dynamic generation. On OmniWorld-Game, GAD improves subject and background consistency, motion smoothness, and all reported visual-quality metrics:
| Method | FID ↓ | FVD ↓ | CLIP ↑ | Subject consistency ↑ | Background consistency ↑ | Motion smoothness ↑ | Aesthetic quality ↑ | Image quality ↑ |
|---|---|---|---|---|---|---|---|---|
| Wan2.1 14B | 45.94 | 419.7 | 28.18 | 0.850 | 0.909 | 0.971 | 0.546 | 0.612 |
| + GAD | 44.75 | 410.2 | 28.88 | 0.863 | 0.917 | 0.978 | 0.552 | 0.617 |
These results show that correspondence supervision remains beneficial even with increased model capacity, improving both consistency and generation quality.
5. How is the camera trajectory encoded and injected into Wan, and which initial checkpoints are used?
How is the camera trajectory encoded and injected into Wan, and which initial checkpoints are used?
We use the camera-conditioned checkpoints released by VideoX-Fun and retain their camera-conditioning mechanism. The exact initializations are:
| Backbone | Initial checkpoint |
|---|---|
| Wan2.1 1.3B | alibaba-pai/Wan2.1-Fun-V1.1-1.3B-Control-Camera https://huggingface.co/alibaba-pai/Wan2.1-Fun-V1.1-1.3B-Control-Camera |
| Wan2.2 5B | alibaba-pai/Wan2.2-Fun-5B-Control-Camera https://huggingface.co/alibaba-pai/Wan2.2-Fun-5B-Control-Camera |
| Wan2.1 14B | alibaba-pai/Wan2.1-Fun-V1.1-14B-Control-Camera https://huggingface.co/alibaba-pai/Wan2.1-Fun-V1.1-14B-Control-Camera |
Specifically, we express camera poses relative to the first-frame camera and combine them with camera intrinsics to construct per-pixel six-dimensional Plücker embeddings, , where and denote the camera center and unit ray direction in this reference coordinate system. These embeddings are temporally rearranged and passed through the camera adapter, whose output is added elementwise to the video patch embeddings before the Transformer blocks.
Thank you very much for your thoughtful comments and the time you have invested in reviewing our paper. We have provided a detailed response to the concerns you raised earlier. Could you please let us know if our clarification sufficiently addressed your questions? If anything remains unclear, we would be more than happy to continue the discussion.
Response to The Reviewer (1 / 3)
We sincerely thank the reviewer for the constructive comments on our work. We greatly appreciate your description of the paper as "well-written and easy to follow", your recognition that "the visual results show improvements in geometry consistency", and your appreciation of the "careful design" and the evidence that "the ablations provide support for several design choices". To address your concerns, we have added 1) quantitative structure-formation analysis, 2) additional noise-timestep routing comparisons, and 3) experiments on the stronger Wan2.1 14B backbone. These results further support our design rationale and its effectiveness at larger model capacity. We address each concern below.
1. Rationale for correspondence supervision and noise-timestep routing.
Correspondence supervision helps video attention gather information from the correct associated regions for consistent generation. The flow-matching objective supervises denoising but does not explicitly specify which locations should correspond across frames. Our method therefore uses VGGT-Ω, whose attention provides more reliable cross-frame correspondences, as a teacher to guide the video model’s Q/K relations while retaining the original flow-matching objective. We have added a quantitative evaluation on TAPVid-DAVIS to substantiate the quality of this correspondence supervision. On ground-truth correspondences from all 30 TAPVid-DAVIS videos, our teacher targets achieve 90.88% target coverage, compared with 54.48% for video attention, supporting the teacher as a substantially stronger source of correspondence supervision.
For noise-timestep routing, we have added a quantitative analysis across 80 randomly selected RealEstate10K scenes. The scene-averaged curves show that deeper layers recover spatial structure earlier under high noise and gradually plateau, while middle and shallow layers show greater improvements in structure recovery as noise decreases. The estimated structure formation rates peak at t=988, 884, and 680 for the late, middle, and early layer groups, respectively, aligning with our routing intervals of t≥950, 850≤t<950, and t<850. These findings support noise-timestep routing from deeper to shallower layers during denoising. Further details are provided in Appendix E.3, with the quantitative curves shown in Fig. 5.
We have added two controls to further test the effectiveness of our noise-timestep routing under the same experimental settings. Random routing (R9) selects one of L5, L15, and L25 independently of noise level. Inverse routing (R10) uses the same noise intervals but reverses our shallow- and deep-layer assignments. The table below reports all ten configurations on DL3DV-10K with Wan2.2 5B.
Our noise-timestep routing outperforms all these alternatives on all four geometric metrics. Its matching hit rate reaches 0.335, compared with 0.286–0.295 for fixed-layer supervision, 0.306 for random routing, and 0.265 for inverse routing. The gains over R9 support conditioning layer selection on noise timestep, while those over R10 support routing from deeper to shallower layers as noise decreases. Together, these experiments and the structural analysis substantiate the design and effectiveness of our noise-timestep routing.