- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors propose WorldRIDE, a framework for distilling world models into efficient interactive streaming world models without additional post-distillation training. The method consists of three main designs: (1) Ordered reward-guided distillation, which applies quality gating before action rewards and enables geometry rewards only on action-valid comparisons. (2) Multi-anchor rollout, which reuses prefix generation to sample local candidates at multiple temporal anchors. (3) Locally self-normalized distillation, which normalizes reward weights within each local candidate group to have unit mean. Experiments on WBench and VBench demonstrate improvements in overall benchmark performance over SANA-WM Streaming.
Critical strengths
- The paper is well structured and easy to follow.
- The qualitative results are clearly presented and allow for easy comparison.
- The proposed solutions are straightforward.
Critical weaknesses
- Missing comparisons with closely related baselines. One main concern is the lack of direct comparison with methods like Reward Forcing (Re-DMD). Re-DMD already incorporates rewards into distribution matching distillation through reward-weighted updates, which is similar to the core idea of WorldRIDE. An adapted Re-DMD baseline using the same SANA-WM backbone and settings would help identify the improvements more clearly.
- Insufficient ablations for ordered reward guidance. The results in Table 3 are rather mixed. In particular, removing both action and geometry rewards only reduces the WBench Overall score from 77.1 to 76.9, although the improvement in Perspective Consistency is more noticeable. More importantly, the current ablations do not clarify whether the proposed ordering of rewards is essential. A comparison against a naive combination of quality, action, and geometry rewards without hierarchical gating would help validate the first contribution. In addition, the paper lacks controlled ablations isolating multi-anchor rollout and locally self-normalized weighting, making it difficult to attribute the observed improvements to the individual proposed components.
- Unclear computational efficiency compared to baselines. Multi-anchor rollout indeed saves computation compared to generating independent full trajectories. However, this does not necessarily imply improved training efficiency over standard distillation. WorldRIDE processes multiple candidates per student update, requiring additional sampling, reward-model inference, and DMD score evaluations. A compute-budget-matched comparison against SANA-WM Streaming and other relevant baselines would be more convincing. The authors should report total training GPU-hours, including rollout generation, reward evaluation, and distillation updates, to better support the efficiency claims.
Questions for authors
See the "Critical Weaknesses section".
Minor weaknesses:
- Source code is not provided in the submission, which may hinder reproducibility.
3 comments
Response to Reviewer MKY7 (Part 2/3): Controlled Ablations of Ordered Gating, Multi-Anchor & Local Normalization (W2)
2. Controlled Ablations of Ordered Gating, Multi-Anchor Rollout & Local Self-Normalization (W2)
To attribute gains cleanly to each individual design and clarify the progression in Table 3, Table 2A isolates each of our three contributions on the identical SANA-WM backbone and training dataset (paired with a 192-group / 384-clip live counterfactual gate audit), and Table 2B tracks the step-by-step effect of adding Action and Geometry rewards on long-horizon and 3D metrics.
Table 2A: Single-factor controlled ablations of Contributions 1–3 & 192-group (384-clip) live counterfactual gate audit.
| Isolated Factor | Controlled 4-Step Variant (Same Backbone & Data) | Anchors / Norm / Gate | Wrong-Action Trap False Upweight ↓ | WBench Overall ↑ | SegCont ↑ | DynDeg ↑ | Setting ↑ | 60s VBench Total (%) ↑ |
|---|---|---|---|---|---|---|---|---|
| Baseline ref. | SANA-WM Streaming (official 4-step) | K=1 / — / — | — | 72.5 | 91.8 | 98.7 | 71.7 | 69.07 |
| K=1 equal-weight | Unweighted DMD (same training data) | K=1 / Equal (w=1) / — | — | 72.8 | 99.4 | 97.5 | 71.9 | 68.80 |
| Contrib. 2 (Multi-anchor) | w/o Action + Geometry (multi-anchor equal) | K=3 / Equal (w=1) / — | — | 73.6 | 98.7 | 99.4 | 71.7 | 69.23 |
| Contrib. 1 (Ungated sum) | Multi-Anchor + Ungated Flat Sum | K=3 / Group Softmax / Flat Sum | 25.0% | 74.0* | 97.5 | 94.9 (-1.9) | 70.4 | 68.58 |
| Contrib. 3 (Global norm) | Multi-Anchor + Global EMA | K=3 / Global EMA / Ordered | 3.1% | 74.1* | 98.7 | 82.9 (-13.9!) | 70.9 | 67.42 |
| Full Method (Ours) | WorldRIDE Full | K=3 / Group Softmax / Ordered | 3.1% | 73.9 | 98.1 | 96.8 | 72.3 | 69.14 |
*Denotes short-clip static consistency inflation from under-motion (Global EMA suffers a -13.9 collapse in Dynamic Degree, dropping NavTraj to 81.1 [lowest overall] and crashing 60s VBench Total to 67.42%).
Table 2B: Clarifying Table 3—step-by-step gains of Action and Geometry rewards (fixed K=3, M=2 multi-anchor setup).
| Stepwise Variant (Fixed K=3, M=2) | Reward Hierarchy | WBench Overall ↑ | Perspective ↑ | 60s Camera RotErr (°) ↓ | 60s Revisit PSNR (dB) ↑ |
|---|---|---|---|---|---|
w/o Action + Geometry (equal-weight multi-anchor) | Equal (w=1) | 73.6 | 62.7 | 15.81 | 15.08 |
w/o Geometry (+ quality-gated Action) | Q → Action | 73.5 | 62.6 | 13.69 (-13.4%) | 14.94 |
WorldRIDE Full (+ conditional Geometry, Ours) | Q → A → Geo | 73.9 | 67.0 (+4.4) | 16.58 | 15.26 (+0.32 dB) |
-
Isolating Contribution 2 (Multi-Anchor Shared-Prefix) & why
"w/o Action + Geometry"already scores high: BecauseHPSv3serves strictly as a binary group feasibility gate in Eq. (2), disabling Action and Geometry yields uniform unit weights (). Thus, comparingUnweighted DMD() tow/o Action + Geometry() strictly isolates multi-anchor rollout: it lifts WBench from72.8to73.6(+0.8), boosts 60s Revisit PSNR from14.31to15.08 dB, and eliminates single-anchor chunk breaks (SegCont 92.4/94.9 → 98.1in Table 1), while sharing the prefix spine speeds up 6-branch sampling6.2×(6,552 → 1,063 ms) and saves2.86 GBVRAM vs. independent rollouts. -
Isolating Contribution 1 (Ordered Gating vs. Naive Ungated Sum): Holding and Group Softmax fixed, combining
Quality + Action + Geometrywithout gating falsely upweights sharp wrong-action/static clips25.0%of the time on 192 live groups; Ordered Gating cuts this 8× to3.1%(0.0%leakage on degraded-quality motion traps). In end-to-end training,Ungated Flat Sumdampens motion (DynDeg -1.9), dropsSetting Adherenceto70.4(-1.9, lowest overall), and degrades 60sVBench Totalbelow un-distilledStreaming(68.58% < 69.07%). -
Isolating Contribution 3 (Local Group Softmax vs. Global EMA): Holding and Ordered Gating fixed,
Global EMAclips/saturates51.0%–72.7%of weights (0.0%for Group Softmax) because low-motion scenes systematically earn higher raw scores—inducing severe motion-freeze hacking (DynDeg: 96.8 → 82.9,NavTraj: 81.1,60s VBench: 67.42%). -
Step-by-step Action & Geometry gains (Table 2B): Once multi-anchor rollout saturates short 5s WBench paths, 60-second (960-frame) closed-loop trajectories (
160videos,800revisit pairs) expose clear orthogonal gains: adding Action cuts 60sRotErrby-13.4%to13.69°(-39.7%vs.Streaming 22.72°), and adding conditional Geometry boosts WBenchPerspective Consistencyby+4.4(62.6 → 67.0) and lifts 60sRevisit PSNRto15.26 dB(surpassing the 60-step teacher's14.98 dB).
Response to Reviewer MKY7 (Part 3/3): Total Training GPU-Hours, Compute-Budget Comparison & Code Release (W3, Minor)
3. Total Training GPU-Hours, Compute-Budget Comparison & Code Open-Sourcing (W3 & Minor)
To support our efficiency claims with complete end-to-end accounting, Table 3A reports CUDA-synchronized per-step latency (including rollout, candidate decoding, reward inference, and distillation updates), peak trainer VRAM, and 1280×704 streaming inference FPS on NVIDIA H200; Table 3B reports formal end-to-end 1280×704 wall-clock time and total active H200 GPU-hours on one 1×8 NVIDIA H200 node.
Table 3A: Per-step latency (including rollout, decode, reward & DMD updates), peak VRAM, and 1280×704 streaming inference speed (NVIDIA H200, CUDA-synchronized).
| Method & Training Paradigm (Same SANA-WM Backbone) | Step Time (s/step) ↓ | Peak Trainer VRAM (GB) ↓ | 720p Infer FPS ↑ (DiT / Stage-1) |
|---|---|---|---|
SANA-WM Chunk-Causal (60-step Teacher) | — | — | 1.56 / 1.41 |
Self-Forcing (unrewarded standard distillation) | 5.84 | 39.17 | 35.06 / 10.99 |
Self-Forcing + WorldCompass RL | 5.84 + 20.80 | 39.17 / 35.24 | 34.90 / 10.97 |
Global Full-Seq Re-DMD (M=2) | 20.06 | 54.07 | 35.01 / 10.91 |
Independent Rollouts (6 separate prefixes) | 16.38 | 56.93 | 35.01 / 10.91 |
WorldRIDE (Shared-Prefix + Detached, Ours) | 15.23 (-26.8% vs. RL) | 54.07 | 35.01 / 10.91 |
Table 3B: Formal 1280×704 end-to-end training wall-clock & total active GPU-hours on a 1×8 NVIDIA H200 node (including rollout generation, CausalVAE decoding, HPSv3 + DA3 evaluation & distillation updates).
Method (1280×704, 1×8 H200 Node) | Stages (Post-ODE) | Wall-Clock Time ↓ | Active H200 GPU-h ↓ | Speedup vs. SF + WorldCompass RL ↑ |
|---|---|---|---|---|
Self-Forcing (unrewarded distillation) | 1 Stage | 12.7 h | 76.2 | (Unrewarded baseline) |
Self-Forcing + WorldCompass RL | 2 Stages | 41.6 h (12.7h + 28.9h) | 307.5 (332.9 node) | 1.00× (2-stage RL baseline) |
WorldRIDE (Ours) | 1 Stage | 29.2 h | 233.6 | 1.42× faster (-30% time, -24% GPU-h) |
-
Clarifying efficiency relative to standard unrewarded distillation & step breakdown (Table 3A): We fully agree with your observation that processing multiple candidates is not cheaper than unrewarded standard distillation (
Self-Forcing / SANA-WM Streaming:5.84 s/step,12.7 h / 76.2 H200-h), and we will revise Section 1 and Section 3.2 to state explicitly that our training efficiency claim is relative to reward-aligned world-model pipelines rather than unrewarded DMD. Amortized over the 5:1 critic-to-generator schedule,WorldRIDE's15.23 s/stepincludes shared-prefix + 6-branch rollout, local CausalVAE decoding (3.31 s/stepamortized), frozenHPSv3 + DA3scoring (0.94 s/stepamortized; depth reprojection<0.2%), and detached DMD updates—making it26.8%faster per step thanWorldCompass RL(20.80 s/step) and24.1%faster thanGlobal Full-Seq Re-DMD(20.06 s/step), while saving6.13 GBVRAM over a 4-step differentiable unroll. -
Superior quality at a lower total GPU-hour budget than two-stage
Streaming + WorldCompass RL(Table 3B & Table 1): Counting all rollout generation, candidate decoding, reward inference, and policy/DMD updates (sharing the one-time Causal ODE initialization of1.3 h / 10.4 H200-h), scalingSelf-Forcing(Streaming) with Stage-2WorldCompass RLconsumes41.6 h(307.5activeH200 GPU-h) yet plateaus at72.8WBench (SegCont 90.5, 60sVBench 68.67%). Single-stageWorldRIDEcompletes in29.2 h(233.6activeH200 GPU-h)—achieving higher performance across WBench (73.9),SegCont(98.1), 60sVBench(69.14%), and 60sRotErr(16.58°) using30%less wall-clock time and24%fewer GPU-hours (1.42×faster), with zero inference overhead (35.01 FPSDiT). -
Source code & reproducibility (
Minor): Due to our internal code-release approval process, we are temporarily unable to upload our code repository at this stage; however, we unequivocally commit to open-sourcing the complete codebase (training, multi-anchor rollout,HPSv3 + DA3reward service, counterfactual gate audit, and WBench / 80-Scene evaluation scripts, configurations, and checkpoints) upon publication.
Thank you again for your valuable feedback, which has substantially improved our paper. Please let us know if you have any further questions during the discussion period.
Response to Reviewer MKY7 (Part 1/3): Same-Backbone Comparisons with Reward Forcing / Re-DMD (W1)
Thank you sincerely for your thoughtful, constructive review and for recognizing the clear structure of our paper, and the clarity of our qualitative visual comparisons.
First, we sincerely apologize to you for the delay in posting our response. Two practical factors required substantial time during the rebuttal window: (1) compute constraints, as we trained multiple full-resolution (
1280×704) controlled baselines—including two same-backboneRe-DMDbaselines and orthogonal ablations—from scratch and evaluated 160 one-minute closed-loop trajectories per model; and (2) the upstream deprecation of WBench's earlier VLM judge (doubao-lite-0215), which required re-evaluating all original baselines alongside our new controlled variants under upstream WBench's current officialqwen3.8-maxprotocol for a strictly fair comparison. Under this unified protocol, all 16 deterministic GPU metrics are bit-identical, and WorldRIDE's Overall gain overSANA-WM Streamingremains invariant at+1.4(75.7 → 77.1previously vs.72.5 → 73.9underqwen3.8-max).We organize our response into three parts: Part 1/3 (
W1: Same-Backbone Comparisons with Reward Forcing /Re-DMD), Part 2/3 (W2: Controlled Ablations of Ordered Gating, Multi-Anchor Rollout & Local Self-Normalization), and Part 3/3 (W3 & Minor: Total Training GPU-Hours, Compute-Budget Comparison & Code Open-Sourcing).1. Direct Same-Backbone Comparisons with Reward Forcing (
Re-DMD) (W1)To directly evaluate WorldRIDE against Reward Forcing (
Re-DMD) under identical conditions, we implemented and trained two controlledRe-DMDbaselines on the exact sameSANA-WM Streamingbackbone, training dataset (112,573windows at1280×704), Causal ODE initialization, and 4-step schedule (Table 1):Table 1: Same-backbone controlled comparisons against two-stage RL and both faithful & 3-reward adapted
Re-DMDbaselines (unified WBench & 60s SANA-WM 80-Scene).SANA-WM Streaming(official 4-step)Unweighted DMD(same training data & init)w=1) / NoneSelf-Forcing + WorldCompass RL(2-stage)G=12, top/bot 3)Single-Anchor Quality Re-DMD(faithful Re-DMD)Single-Anchor 3-Reward Re-DMD(adapted Re-DMD)WorldRIDE Full(Ours)Two fair same-backbone
Re-DMDimplementations: (1)Single-Anchor Quality Re-DMDfaithfully implements original Re-DMD (Lu et al.) using single-anchor rollout (K=1), cross-batch Global EMA weighting, andHPSv3visual quality reward; (2)Single-Anchor 3-Reward Re-DMDdirectly tests your suggested adaptation by equipping Re-DMD's K=1 + Global EMA framework with all three of WorldRIDE's quality, action, and geometry rewards.Comparison with faithful
Quality Re-DMD(73.4vs.73.9): Optimizing single-anchor visual quality alone lacks action and 3D structural supervision—lagging behindWorldRIDE FullinNavTraj(81.3vs.82.6),Perspective Consistency(61.7vs.67.0,-5.3), andSegCont(94.9vs.98.1), while suffering a-1.45%collapse in 60s closed-loopVBench Total(67.69%vs.69.14%) and higher camera rotation error (18.34°vs.16.58°).Comparison with adapted
3-Reward Re-DMD(73.6vs.73.9): Even when given the exact same three rewards, Re-DMD's single-anchor supervision (K=1), Global EMA normalization, and ungated combination cause severe chunk-boundary discontinuity (SegCont 92.4,-5.7belowWorldRIDE Full 98.1), lowerCausal Fidelity(45.7vs.49.6,-3.9), and degrade 60sVBench Totalbelow un-distilledStreaming(68.63% < 69.07%) with19.33°rotation error (25.71°on the hard split).WorldRIDE Fulloutperforms bothRe-DMDvariants and two-stageWorldCompass RLacross every metric above.