acceptodds
Withdrawn from ICLR 2027

WorldAlign: Decoupled 4D Reward for World-Consistent Video Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Faithful visual world simulation requires generated videos to maintain 4D world consistency: static environments should preserve coherent 3D structure across viewpoints (static consistency), while dynamic subjects should exhibit plausible motion and consistent appearance over time (dynamic consistency). Existing approaches either modify the generator architecture to incorporate explicit geometry or improve pretrained models through geometry-aware post-training. The former may compromise the broad generalization acquired through large-scale pretraining. The latter preserves this valuable generalization capability but remains unreliable and limited in dynamic scenes. These methods either assume fully static scenes or accommodate dynamic scenes but risk conflating legitimate subject motion with undesired distortions in static regions when evaluating static consistency. Dynamic consistency is either overlooked or evaluated solely through appearance preservation. These limitations prevent effective improvements in world consistency in real-world dynamic scenes. In this paper, we introduce WorldAlign, a decoupled 4D reward framework that semantically separates static regions and dynamic subjects and aligns each with a world prior suited to its assumptions. For static regions, WorldAlign aligns static geometry with a geometric world prior through masked reprojection, excluding dynamic subjects while retaining undesired distortions in static regions for evaluation; an auxiliary camera-motion reward discourages nearly static solutions. For dynamic subjects, WorldAlign uses a strong vision-language model as a dynamic world prior, with sample-specific checklists assessing dynamicity, physical plausibility, shape, and texture consistency. This decoupled design enables online post-training without modifying the generator architecture or requiring human preference annotations. Across two pretrained image-to-video generators, WorldAlign jointly improves static and dynamic consistency over existing methods without suppressing overall or subject motion. These results demonstrate the effectiveness of decoupled world-prior alignment for advancing video generation toward faithful visual world simulation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video generation, world-consistency, geometric consistency, physical faithful, world model, reinforcemenr learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_r8k7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WorldAlign aims to improve world consistency in video generation by decoupling rewards for static backgrounds and dynamic subjects. It uses semantic tracking masks to evaluate background reprojection consistency and introduces a camera-motion reward to mitigate “static degradation.” Subject motion and appearance consistency are assessed through sample-specific VLM checklists. These three rewards are used for GRPO post-training, with experiments conducted on two pretrained video generators.

Critical strengths
  • The project page provides numerous generated examples that illustrate WorldAlign’s advantages over the baselines.
  • The paper addresses an important and well-motivated problem.
  • The paper is well written and clearly explains the existing challenges and the specific issues the method aims to resolve.
Critical weaknesses
  • While the identified problem is valid, the proposed solution relies heavily on checklist-based VLM scoring without establishing its agreement with human judgments or objective measures of consistency. Answers to checklist questions may not reliably reflect the correctness of subject motion and appearance. A comparison with human annotations or independent metrics would help substantiate the reward’s validity.
  • The method uses VGGT to estimate point clouds and camera parameters from the full video before removing points associated with dynamic subjects. However, these subjects may already have affected the estimated camera parameters and background geometry. Filtering out their points afterward does not necessarily correct those errors. The authors should assess the robustness of this procedure in scenes with substantial subject motion.
  • The paper does not report Flow-GRPO training curves, making it difficult to assess convergence, stability, and potential reward exploitation. Additional training details and curves showing reward evolution and independently measured performance would strengthen the experimental evidence.
  • The Checklist “Yes” Rate closely follows the evaluation procedure used to construct the dynamic reward. Improvements on this metric may therefore reflect optimization toward the reward evaluator rather than broader gains in world consistency. Independent evaluations are needed to substantiate the metric’s validity and support claims of improved physical reasoning.
Questions for authors

I hope the authors to solve my weakness above.

6 comments
AuthorsOpenReview ↗

Author Response to Reviewer r8k7 - (1/2)

Weakness 1

While the identified problem is valid, the proposed solution relies heavily on checklist-based VLM scoring without establishing its agreement with human judgments or objective measures of consistency. Answers to checklist questions may not reliably reflect the correctness of subject motion and appearance. A comparison with human annotations or independent metrics would help substantiate the rewardʼs validity.

Thanks for recognizing the identified problem is valid. To address your concern about the rewardʼs reliability, we report the agreement between our VLM judge and human annotations.

We randomly select 100 prompts from the training dataset and generate one video per prompt using a random inference seed. Human annotators provide Yes/No labels for four dimensions: subject motion, physical plausibility, shape preservation, and texture preservation, which are the four dimensions used in our dynamic reward. This gives 400 human labels in total. We compare these labels with Qwen3.6-Plusʼs answers using our sample-specific checklists. The VLM judge achieves 78.0% agreement with human annotations (312/400). For comparison, we use a VisionReward-style generic checklist covering the same four dimensions yields only 64.25% agreement (257/400). These results validate the effectiveness of the proposed vlm reward.

The rewardʼs effectiveness is also reflected in the videos generated after training. Our independent human evaluation shows that WorldAlign outperforms the compared methods in static consistency, dynamic consistency, and overall preference. We have moved the human evaluation to Sec. 4.3 (Tab. 2) of the main paper from supplemetary material. This provides further evidence that the proposed rewards are effective.

Weakness 2

The method uses VGGT to estimate point clouds and camera parameters from the full video before removing points associated with dynamic subjects. However, these subjects may already have affected the estimated camera parameters and background geometry. Filtering out their points afterward does not necessarily correct those errors. The authors should assess the robustness of this procedure in scenes with substantial subject motion.

Thanks for your constructive suggestion. Indeed, VGGT is not the strongest geometry estimation method for dynamic scenes as proven in paper PAGE-4D[1]. However, it is well-established and widely recognized, and still performs well in most cases, as reflected in the improvements shown by our human evaluation and qualitative results. This is the reason we still use it as our geomety foundation model.

We also want to clarity that our method is not limited to specific models. As the field advances, VGGT can be replaced with newer geometry estimators for dynamic scenes, such as PAGE-4D. Gemini-3.1-Pro-Preview could likewise be replaced with a newer model such as Gemini-4-carbon. Even with stronger models, semantic decoupling remains necessary to penalize static inconsistencies without suppressing valid subject motion. Our checklist-based VLM scoring also remains necessary to improve dynamic consistency. The core reward design therefore remains essential as the underlying models improve.

Overall, we sincerely hope you can consider the contribution of our decoupled 4D reward and the improvements demonstrated by our experiments, rather than our current implementation.

[1] PAGE-4D: DISENTANGLED POSE AND GEOMETRY ESTIMATION FOR VGGT-4D PERCEPTION

Weakness 3

The paper does not report Flow-GRPO training curves, making it difficult to assess convergence, stability, and potential reward exploitation. Additional training details and curves showing reward evolution and independently measured performance would strengthen the experimental evidence.

Thank you for the suggestion. We now provide Flow-GRPO learning curves for both backbones, including rewards on the training and test sets and independent metrics on the test set (Figs. B and C). As training progresses, both training and test rewards generally improve, alongside lower M.Epi. and higher Subject DINO and VBench scores. These trends support stable training and show that the improvements extend to metrics not used as training rewards

AuthorsOpenReview ↗

Author Response to Reviewer r8k7 - (2/2)

Weakness 4

The Checklist “Yes” Rate closely follows the evaluation procedure used to construct the dynamic reward. Improvements on this metric may therefore reflect optimization toward the reward evaluator rather than broader gains in world consistency. Independent evaluations are needed to substantiate the metricʼs validity and support claims of improved physical reasoning.

Thanks for your comment. We have reported independent human evaluation in our original supplementary material. We have now moved these results to Sec. 4.3 (Tab. 2) of the main paper. Human raters directly assess static consistency, dynamic consistency, and overall preference, and WorldAlign outperforms the compared methods across these criteria.

Thanks for the authors' response. In Figure B in the Appendix, the Checklist Yes Rate curve shows only a slight increase, suggesting that this reward component makes a relatively limited contribution to the overall results, which somewhat weakens the novelty of the proposed design. Furthermore, the overall experimental design does not introduce substantial innovations over World-R1. Although certain components, such as the decoupled 4D reward may indeed be effective, I do not believe the current evidence is sufficiently compelling to warrant a positive assessment. Therefore, I maintain my original score of Weak Reject.

Reviewer_sSiaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses world consistency in image-to-video generation, meaning coherent 3D structure for static environments and plausible, appearance-preserving motion for dynamic subjects. It proposes WorldAlign, a GRPO-based post-training framework that separates each generated video into static and dynamic regions using tracked subject masks. Static regions are rewarded through masked reprojection with a geometry foundation model together with a camera-motion hinge reward that prevents near-static solutions. Dynamic subjects are rewarded by a VLM that answers a sample-specific checklist on dynamicity, physical plausibility, shape and texture. The method is evaluated on two Wan generators against four post-training baselines, using geometric, subject-consistency and VBench metrics.

Critical strengths

This paper identifies a concrete flaw in prior geometry-aware post-training. Motion-based filtering, as in VGGRPO and GeoFlow, can remove the very background distortions a static-consistency reward should penalize, and Fig. 4 illustrates this convincingly. Assigning static and dynamic criteria by semantic region is a sensible response. This matters because it explains why existing rewards trade static consistency against motion. The method also improves consistency without the motion collapse seen in Epipolar-DPO and VideoGPA.

Critical weaknesses

The evaluation does not yet establish the gains reliably. All results come from 100 held-out prompts with no seeds or confidence intervals. M.Reproj and the Checklist Yes Rate are measured with the same tools used as rewards, and the reward-independent M.Epi appears to rely on the same tracker masks and possibly the same geometry model. Dynamic Degree is directly encouraged by the camera reward. On the truly independent Subject DINO metric, WorldAlign differs from GeoFlow by only 0.003 to 0.004 and stays below Epipolar-DPO. Human evaluation appears only in the supplementary material. These issues are fixable, but they currently leave the size of the improvement uncertain.

The methodological novelty is limited, and the central design choice is not validated quantitatively. Reprojection rewards from predicted geometry already appear in VideoGPA and World-R1, and the online optimization follows Flow-GRPO. The related work should also include ExPose, Reinforcing Video Generation Models for Extreme Pose Estimation(CVPR26). ExPose already fine-tunes a video generator with GRPO using 3D-consistency rewards derived from pose estimation, which is the same post-training paradigm WorldAlign builds on. The main distinguishing claim, that semantic masks are better than motion-based masks, rests only on the qualitative Fig. 4, with no ablation that swaps the mask type inside the same pipeline. The semantic design also assumes a single annotated subject. New objects, multiple moving entities and non-rigid backgrounds such as water or foliage would be treated as static and penalized. That is a partly fundamental limitation, and the paper does not discuss it.

Questions for authors

1.Please report the main results in Table 1 over several random seeds with confidence intervals, given that the test set contains only 100 prompts. Please also state whether M.Epi is computed with the same geometry foundation model and tracker masks as the reward, and if so, report a static-consistency metric computed with an independent geometry model.

2.Report an ablation that replaces the tracked semantic masks with scene-flow or optical-flow masks inside the same WorldAlign pipeline. Please also report an ablation that replaces the checklist reward with a DINO-based appearance reward, so that each of the two claimed advantages over GeoFlow is isolated.

3.Specify which VLMs construct the checklists and judge the videos, report the values of the threshold τ and the weights λ, and report the agreement between the VLM judge and human labels on a subset of videos.

4.Extending the decomposition to videos with multiple or newly appearing dynamic subjects and non-rigid background elements, for example through open-vocabulary segmentation of all movable entities, would make the static reward applicable to more realistic scenes.

6 comments
AuthorsOpenReview ↗

Author Response to Reviewer sSia - (1/4)

Weakness 1

Thanks for your comments and suggestions. Your concerns focus on evaluation reliability, metric independence, and performance gains over other methods. We have addressed these concerns and present our response in four parts below for clarity.

W1-1: Report results in Tab1 over several random seeds with confidence intervals to demonstrate the reliability of our method

In our Tab.1, all methods used the same seed during inference for a fair comparison. Following your suggestion, we further report results averaged over five different inference seeds and 95% confidence intervals to demonstrate the stability and reliability of our method. We have updated these results in Tab.C and provided disscussion and evaluation details in Sec.G of our supplementry material for your kind reference.

In Tab.C, these results show that the gains of our WorldAlign remain after averaging over five inference seeds, and the CIs also show the same overall trend as the average scores. Therefore, these results support the stability and reliability of our improvements.

( Note: Consistency scores should still be read alongside Dynamic Degree, since static collapse can produce misleadingly favorable scores, as highlighted in gray.)

W1-2: Independence of M.Epi and Dynamic Degree

We report M.Reproj. and Checklist Yes Rate to show the training objectives also improve on test set. The remaining reported metrics are not used as training rewards in our WorldAlign.

  1. M.Epi. does not use VGGT or any 3D geometry model used during training. As detailed in Sec. F.1 of our supplementary material, M.Epi. computes Sampson error from SIFT matches and a fundamental matrix estimated using RANSAC. This computation is fundamentally different from that of VGGT-based M.Reproj. The mask tracker is used only to exclude dynamic subjects and does not compromise the independance of M.Epi scores. We further replaced VGGT with DA3 when computing M.Reproj., keeping the masking and reprojection protocol unchanged, to provide further evidence from an independent geometry model. The results below support the same conclusion as M.Epi.: our method retains a clear advantage in static consistency over GeoFlow and World-R1, whereas Epipolar-DPO and VideoGPA suffer from static collapse.

    Table: Static Consistency scores of each method on Wan2.2-TI2V-5B

    MethodBaseEpipolar-DPOVideoGPAWorld-R1GeoFlowOurs
    VGGT-based M.Reproj.0.0400.0220.0240.0380.0310.026
    M.Epi.0.4630.2680.3080.4320.3670.321
    DA3-based M.Reproj.0.0290.0140.0170.0270.0240.019
  2. Dynamic Degree is not directly optimized either. Our camera reward uses VGGT-estimated camera motion, whereas VBench Dynamic Degree uses RAFT optical flow computed from the generated frames. Training improves camera motion, and a separate evaluator detects that improvement in the videos. This is evidence that the gains extend beyond the training reward, not evidence that Dynamic Degree itself was optimized.

Therefore, our improvements are supported by several independent evaluation metrics that are not used as training rewards!

W1-3: Subject DINO Performance Relative to GeoFlow and Epipolar-DPO

  1. GeoFlow includes a DINO-based appearance reward as discussed in Sec.2.2, whereas WorldAlign does not use DINO as a training reward. This is why WorldAlignʼs advantage on Subject DINO is relatively small. In addition, WorldAlign outperforms GeoFlow in human evaluation, static consistency, and all reported VBench metrics. These results show that WorldAlign achieves better overall performance than GeoFlow.

  2. Epipolar-DPOʼs higher Subject DINO scores come with strong motion suppression: its Dynamic Degree is only 0.31 and 0.33, compared with 0.72 and 0.74 for WorldAlign. Nearly static subjects obviously can obtain high feature similarity without demonstrating plausible motion. Since Subject DINO alone does not fully capture dynamic consistency, it should be read alongside Dynamic Degree.We have discussed these in Sec. 4.1 and the caption of Tab. 1. As shown in Tab. 1 of our main paper, WorldAlign maintains strong subject consistency while preserving motion. These results demonstrate WorldAlignʼs overall advantage over Epipolar-DPO.

AuthorsOpenReview ↗

Author Response to Reviewer sSia - (2/4)

W1-4: Independent Human Evaluation

We also conducted an independent human evaluation of static consistency, dynamic consistency, and overall preference. WorldAlign is preferred over the competing methods across all three criteria. These results provide further evidence that our improvements are recognized by human raters, beyond the automated metrics.

The human evaluation results were originally reported in Sec.B of the supplementary material. We have moved the human evaluation results to Sec. 4.3 (Tab. 2) of the revised main paper. The detailed evaluation protocol is provided in Sec. B of the supplementary material.

Weakness 2

This comment raises four points: methodological novelty, the comparison with ExPose, quantitative validation of semantic masking, and applicability beyond a single subject. We have added experiments and revised the manuscript to address these points. Our response is organized into four parts below.

W2-1: Methodological Novelty

We do not claim reprojection rewards or Flow-GRPO as new contributions. Our contribution is a semantically guided, decoupled 4D reward that assigns appropriate consistency constraints to static environments and dynamic subjects.

This design addresses three key problems in previous methods. First, whole-video geometric constraints, as used in Epipolar-DPO and VideoGPA, can incorrectly penalize valid subject motion. Second, motion-based filtering or weighting, as used in VGGRPO and GeoFlow, can exclude or downweight distorted background regions, weakening penalties on errors that should be corrected. Third, methods that ignore dynamic consistency or rely only on DINO-based appearance rewards lack explicit supervision for motion plausibility: maintaining similar appearance does not ensure that subjects move realistically.

WorldAlign implements this design with two complementary rewards under the novel decoupled principle. 1) The masked reprojection reward uses semantic masks to retain penalties on background distortions while excluding dynamic subjects, avoiding both missed errors in static regions and incorrect penalties on valid motion. 2) The checklist-based VLM reward converts Geminiʼs world knowledge into sample-specific criteria for plausible motion and consistent appearance. These criteria provide explicit references for evaluating subject evolution, going beyond generic scoring dimensions. Together, these rewards assign appropriate signals to static environments and dynamic subjects, enabling joint improvements in world consistency.

The original paper already provides evidence for this design. Sec. 3.2 explains how full-scene reprojection incorrectly penalizes subject motion, and Sec. 4.4 (Tab. 3) evaluates the benefits of masked reprojection and the dynamic reward. Sec. 4.5 (Fig. 4) illustrates how motion-based filtering can overlook background distortions.

We sincerely hope you can consider these contributions as our novelty, we also added ablations on semantic masking and checklist reward to further validate our design.

W2-2: Differences from ExPose

Thanks for pointing out ExPose as relevant work. We have included a discussion of ExPose in the Related Work section of the revised manuscript. However, the two methods differ in both their tasks and optimization targets.

  1. ExPose uses video generation model to generate intermediate frames to improve camera pose estimation, as evaluated in Tabs. 3 and 4 of their paper. WorldAlign aims to improve the world consistency of generated videos themselves.

  2. ExPoseʼs main reward targets pose estimation accuracy, whereas WorldAlign jointly rewards static geometric consistency and plausible subject motion and appearance. Accurate camera pose estimation from a generated video does not guarantee plausible subject motion. Our decoupled reward explicitly addresses both static and dynamic consistency.

Both methods use Flow-GRPO, but Flow-GRPO is a general optimization framework. Sharing this framework does not imply that the two methods introduce the same reward mechanism or address the same problem.

AuthorsOpenReview ↗

Author Response to Reviewer sSia - (3/4)

W2-3: Quantitative Comparison of Semantic and Motion-Based Masks

We have added the requested quantitative mask ablation on Wan2.2-TI2V-5B, as shown in the table below, which validates our claim in Sec. 4.5 of our main paper.

We also included this quantitative ablation into our revised main paper for your reference.

In this experiment, we replace the semantic masks in the masked reprojection reward with either scene-flow or optical-flow masks and retrain each variant. Only the mask type changes; the camera-motion reward, dynamic VLM reward, and other training settings remain unchanged.

The expriment shows that both motion-based variants yield worse static consistency than our semantic-mask variant. This supports the explanation illustrated in Fig. 4: motion-based masks can also exclude distorted static regions, weakening static penalties where it is needed.

Although all variants use the same dynamic reward, the scene-flow variant shows subject-motion suppression (higher Subject DINO but lower Dynamic Degree and Checklist “Yes” Rate). This is because scene-flow masks sometimes cover only part of a dynamic subject, leaving the uncovered regions subject to static geometric penalties. These penalties conflict with the dynamic reward and can partially freeze the subject, increasing feature similarity while reducing motion.

Overall, this quantitative ablation and the qualitative results in Fig. 4 demonstrate the effectiveness of semantic decoupling: it improves static consistency while avoiding unnecessary suppression of dynamic subjects.

ExpM.Epi.M.Reproj.Subject DINOChecklist “Yes” RateDynamic DegreeMotion SmoothAesthetic Quality
scene-flow0.4120.03530.9410.723 (0.68)0.690.9770.480
optical-flow0.4350.03710.9330.775 (0.81)0.710.9770.482
semantic mask (Ours)0.3210.02600.9380.781 (0.85)0.720.9790.488

W2-4: Applicability to Multiple Subjects and Dynamic Backgrounds

The use of a single annotated subject is a choice in our current implementation. Our design can seamlessly extend to multiple subjects, newly appearing objects, and non-rigid background regions without changing the reward formulations. This is because SAM3, which we use to produce semantic masks, already supports segmentation for these scenarios. We only need to provide the corresponding class prompts for SAM3, which existing VLMs can readily generate. Therefore, multiple subjects, newly appearing objects, and non-rigid background regions can be excluded from the static reward computation and will not receive static geometric penalties.

We provide segmentation examples for these three scenarios in Fig.A of our supplementary material for your reference. We have also added clarification about these scenarios in our revised manuscript and Sec.H of our supplementary material.

Finally, we emphasize that our contribution is the decoupled 4D reward, which assigns appropriate consistency rewards to different regions, rather than the segmentation method itself.

Reviewer_bBoyOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper employs online reinforcement learning to enhance the world consistency of video generation. The key observation is that distinguishing between motion and stillness based solely on motion cues can lead to the misclassification of distortions in static regions as dynamic content, thereby causing the model to overlook errors that should be penalized. Instead, this paper uses semantic masks to separate static and dynamic components. For static regions, a geometric foundation model is used to calculate the mask-based reprojection error as a reward, supplemented by a camera-motion reward to prevent the video from becoming entirely static. For dynamic subjects, a vision-language model (VLM) provides scores based on a sample-specific checklist covering four dimensions: dynamism, physical plausibility, shape, and texture. Experiments on the Wan2.2 and Wan2.1 base models demonstrate that camera and subject motion are not suppressed, while both static geometric consistency and dynamic consistency are improved, outperforming four geometric-alignment baselines.

Critical strengths
  1. The core motivation is intuitive: using masks to prevent distortions in static regions from being misidentified as dynamic content, which would otherwise lead to incorrect reward signals.
  2. The effectiveness of the method is validated across multiple backbones.
Critical weaknesses

While this paper serves as a competent engineering report, it lacks significant novelty.

  1. Technically, the approach largely combines existing methods: the VGGT reprojection reward is derived from the same principles as VideoGPA. The hinge-based camera-motion reward is a standard technique to prevent "static collapse". VLM-based scoring is a common reward mechanism. The GRPO training follows the Flow-GRPO framework.
  2. The paper should include a discussion regarding multi-dimensional video reward models, such as VisionReward [1].
  3. Results from human evaluations should ideally be included in the main text.
  4. The discussion in the section "Architecture Modifications for Geometry Integration" is incomplete. It fails to account for literature, such as GaC [2], that incorporates geometry as context for video generation.

[1] VisionReward: Fine-grained multi-dimensional human preference learning for image and video generation. AAAI 2026.

[2] Geometry-as-context: Modulating Explicit 3D in Scene-consistent Video Generation to Geometry Context. CVPR 2026.

Questions for authors

Since the pass rate on the checklist serves as a training reward, the substantial improvement (from 0.697 to 0.781 and 0.856) raises the possibility of overfitting to the custom-built checklist. How is the diversity of the checklist ensured?

7 comments
AuthorsOpenReview ↗

Author Response to Reviewer bBoy - (1/3)

Weakness1

Technically, the approach largely combines existing methods: the VGGT reprojection reward is derived from the same principles as VideoGPA. The hinge-based camera-motion reward is a standard technique to prevent "static collapse". VLM-based scoring is a common reward mechanism. The GRPO training follows the Flow-GRPO framework.

Thank you for your comment. We believe that describing WorldAlign as a simple combination of existing methods overlooks its core contribution: assigning appropriate consistency constraints to different regions based on their semantic roles. This addresses missed penalties on errors in static regions, incorrect penalties on valid subject motion, and the lack of effective constrains for motion plausibility.

Specifically, whole-scene VGGT reprojection can mistake valid subject motion for geometric errors and incorrectly penalize dynamic subjects (Sec. 3.2). Simply adding a dynamic reward does not resolve this problem: the same subject motion is encouraged by the dynamic reward but penalized by the static reward. These conflicting signals limit joint improvements in static and dynamic consistency (Sec. 4.4, Tab. 3). Meanwhile, motion cues used by methods such as VGGRPO and GeoFlow can also identify sliding and distortion in static regions as dynamics. Excluding or downweighting these regions weakens geometric penalties on errors that should be corrected (Sec. 4.5, Fig. 4). Simply combining existing rewards does not resolve these problems, because they arise from where the constrains apply and what it evaluates.

WorldAlignʼs decoupled 4D reward addresses these problems through semantic decomposition. It retains geometric constraints on static environments, penalizing background distortions while excluding interference from valid subject motion. For dynamic subjects, it evaluates whether motion and appearance evolve consistently with their properties and scene context. This division of reward signals reduces conflicts between geometric constraints and dynamic evaluation, allowing both to improve world consistency together.

Regarding VLM scoring, our contribution also goes beyond using a VLM as a scorer. Compared with world-consistency methods that overlook dynamic consistency or mainly rely on DINO-based appearance similarity, we explicitly use VLM world knowledge to evaluate plausible subject motion and consistent appearance evolution. Appearance similarity measures whether a subject looks similar across frames, but is insufficient to determine whether its motion is appropriate for its properties and scene context. Furthermore, we convert VLM world knowledge into sample-specific checklists that describe how each subject should evolve under the input conditions. Another VLM then answers each question. This provides concrete references for evaluating each sample, beyond general scoring dimensions. Please see our response to Weakness 2 for the detailed comparison with other VLM scoring and checklist-based methods.

We acknowledge the existing foundations of the camera-motion reward and Flow-GRPO and do not claim them as our innovations. The former mitigates static collapse, while the latter provides the online optimization framework. Our contribution lies in our new decoupled 4D reward for world consistency, and controlled ablations directly validate this contribution. Tab. 3 shows that, with the same camera and dynamic rewards, semantic decoupling outperforms their direct combination with whole-scene reprojection. Tab. 4 shows that semantic masks yield better static consistency than scene-flow and optical-flow masks when all other rewards and training settings remain unchanged. Sec. C of our supplementary material further shows that replacing the checklist reward with GeoFlowʼs DINO-based appearance reward reduces dynamic consistency. These three experiments validate the need for decoupling, the role of semantic masking, and the value of dynamic world priors, respectively. Together with the automatic and human evaluations in Tabs. 1 and 2, they support both our reward design principle and its effectiveness.

We sincerely hope that these clarifications and additional experiments help you reconsider the novelty of our decoupled reward design and its value for improving world consistency.

AuthorsOpenReview ↗

Author Response to Reviewer bBoy - (2/3)

Weakness 2

The paper should include a discussion regarding multi-dimensional video reward models, such as VisionReward.

Thank you for the suggestion. We have added a discussion of VisionReward and other multi-dimensional video reward models in Sec. 2.3 of the revised manuscript. Our method differs from VisionReward in two main aspects:

  1. Checklist construction. VisionReward uses a generic multi-dimensional checklist. We instead generate a sample-specific multi-dimensional checklist from the conditioning image and text. These checklists translate Geminiʼs rich world knowledge into concrete world-consistency criteria.

    The following example pairs VisionReward-like generic questions with checklist questions taken directly from our training dataset.

    Prompt: “Camera pans left across the pasture. A speckled horse walks in a grassy field near a wire fence.”

    DimensionGeneric checklist (VisionReward-like)Sample-specific checklist (WorldAlign)
    Subject motionDoes the subject show noticeable motion in the video?Does the speckled horse exhibit visible, continuous movement, such as articulating its legs or shifting its body, throughout the video?
    Physical plausibilityIs the subject’s motion physically realistic?Does the speckled horse's observed movement look physically realistic, displaying a natural walking gait with proper alternation of legs and firm ground contact?
    Shape preservationDoes the subject maintain a normal shape throughout the video?Does the speckled horse maintain proper anatomical structure in every frame, allowing for natural bending of its legs while avoiding any missing, duplicated, or unnaturally distorted body parts?
    Texture preservationDo the subject’s color and texture remain consistent throughout the video?Does the speckled horse's coat coloration and specific spot pattern remain consistent and stable throughout the video without flickering or shifting?
  2. VLM scoring. VisionReward uses a scoring model trained on human annotations, and the accuracy of its answers depends heavily on the trained modelʼs reasoning ability. WorldAlign provides explicit world-consistency criteria in the checklist, so the judge only needs to determine whether the video satisfies each criterion. These concrete references simplify the judgment task and enable more accurate evaluation.

We also evaluate this benefit by comparing a VisionReward-like generic checklist with our sample-specific checklists. We randomly select 100 prompts from the training dataset and generate one video per prompt using a random inference seed. Human annotators provide Yes/No judgments for four dimensions: subject motion, physical plausibility, shape preservation, and texture preservation. This gives 400 human labels as ground truth. We then use the same Qwen3.6-Plus judge to evaluate the videos under each checklist design and compare its 400 answers with the human labels. The generic checklist achieves 64.25% agreement (257/400), whereas our sample-specific checklists achieve 78.0% agreement (312/400). These results demonstrate that sample-specific checklists provide clearer references for evaluation, leading to more accurate judgments and better feedback for post-training toward world-consistent video generation.

Furthermore, WorldAlign does not rely solely on checklist-based VLM scoring to evaluate all aspects of world consistency. VLM scoring can assess multiple video frames at a perceptual level, but it does not provide explicit pixel-level geometric measurements. Geometric reprojection error provides finer-grained reward signal. We therefore decouple static environments and dynamic subjects and assign each region the most appropriate reward. This division of reward signals is also a core contribution of our method.

Weakness 3

Results from human evaluations should ideally be included in the main text.

Thanks for your suggestion. We have moved the human evaluation to Sec 4.3 of our revised main paper for your reference.

Weakness 4

The discussion in the section "Architecture Modifications for Geometry Integration" is incomplete. It fails to account for literature, such as GaC [2], that incorporates geometry as context for video generation.

Thanks for pointing out this relevant work. We have added GaC to the discussion of “architecture modifications for geometry integration” in Sec. 2.2 of the revised manuscript for your kind reference.

AuthorsOpenReview ↗

Author Response to Reviewer bBoy - (3/3)

Question 1

Since the pass rate on the checklist serves as a training reward, the substantial improvement (from 0.697 to 0.781 and 0.856) raises the possibility of overfitting to the custom-built checklist. How is the diversity of the checklist ensured?

Our method does not overfit to the custom-built checklists.

  1. All test conditions were held out from training. They cover diverse scenes and subjects, including human activities, animal motion, and vehicle movement, such as cycling, dogs, and cars driving through urban intersections.

  2. Our checklists are sample-specific. The questions change with each new test condition, so the test checklists differ from those used during training.

  3. More importantly, we report not only the Checklist “Yes” Rate but also independent human evaluation (Sec. 4.3, Tab. 2). Human evaluation also shows improvements on the test set. This demonstrates that our gains are perceptible to humans and extend beyond higher scores from the same evaluation mechanism used during training, providing independent evidence against merely overfitting to the custom-built checklists.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.