- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Evidence-Gated State Anchoring (EGSA) to improve state persistence in video world models. The method uses correspondence and motion evidence to control whether predicted visual states should be updated or anchored to previous estimates. It can be integrated into pretrained diffusion and flow-matching models with a small number of trainable parameters. Experiments on MiniWorld and DFoT show improved reconstruction consistency when the camera moves away from a scene and returns.
Critical strengths
The paper addresses an interesting and practical problem in video world models, particularly the preservation of scene information during recursive generation and camera revisits. The proposed method is lightweight and can be applied to different pretrained backbones without full model retraining.
Critical weaknesses
-
The effectiveness of evidence gating is not sufficiently demonstrated in realistic dynamic scenes. Most of the main experiments are conducted on static-scene camera revisits, where preserving previous observations is naturally beneficial. The authors also acknowledge that evidence-free anchoring can achieve similar improvements in these settings. The advantage of evidence gating over a simple hold mechanism is mainly demonstrated using a synthetic experiment with a known change mask. It remains unclear whether the proposed evidence mechanism can reliably distinguish legitimate scene changes from unwanted drift in realistic videos with moving objects, occlusions, and complex interactions.
-
The main experiments rely on constructed palindromic camera trajectories and rotation-dominant correspondence. In particular, the DFoT evaluation generates only four future frames. More diverse and challenging evaluation scenarios are needed to demonstrate the generality of the proposed approach.
Questions for authors
-
How does EGSA compare with existing memory-based or geometry-guided world models, such as WorldMem, GEN3C, or Closing the Loop?
-
The attention-logit bias contributes almost no additional improvement according to the ablation results. Is this component necessary?
6 comments
Thank you for the additional experiments. The new results address some of my concerns, but two points remain unclear.
1. Necessity of evidence gating. In Table R3, a simple constant hold (loop_w0.5) achieves lower return MSE than EGSA at dose 2 (0.141707 vs. 0.186906), with a comparable view-change ratio and a higher direction proxy. While the paper argues that evidence gating helps preserve legitimate changes, it remains unclear why EGSA performs worse than a simple constant hold in this setting.
2. Contribution of memory versus evidence gating. In Table R2, previous-frame EGSA performs worse than the LoRA baseline, whereas adding last-valid memory substantially improves return MSE. Notably, evidence-free memory achieves even lower return MSE than EGSA with memory (0.007505 vs. 0.010252), although its overall MSE is higher. These results raise questions about the relative contributions of memory retrieval and evidence gating. In particular, it is unclear why evidence-free memory achieves better return accuracy and to what extent the reported gains can be attributed to evidence gating itself.
Thank you for the follow-up. Below we clarify both points directly.
1. Necessity of evidence gating in Table R3
Every clip in Table R3 is a time-reversed RealEstate10K indoor sweep with no moving objects. Because nothing in the room changes state, the true change mask is zero everywhere (). Setting (loop_w0.5) acts as the static-scene oracle mask, whereas any nonzero in a static room can only relax a valid anchor.
On MiniWorld RealEstate10K, EGSA outputs because MiniWorld runs in Wan2.2's 4x temporally compressing causal VAE space, where each token encodes a four-frame chunk including motion direction. As measured in Section 4.1.1, two ground-truth latent frames of the exact same static view under mirrored outbound/return motion still differ by an L2 residual of ~1.0, plus translation parallax under a rotation-only warp. Online residual detection responds to these velocity and parallax residuals during sweeps and turnarounds, lowering EGSA's hold below loop_w0.5 on static rooms (0.186906 vs. 0.141707 return MSE).
That static-world shortcut fails once real scene dynamics or multi-window drift enter. On the exact same MiniWorld backbone and VAE, when real arm and object motion occur in DROID (Tables R1 and R6), a constant hold freezes moving regions: at equal read weight (0.127295), EGSA cuts motion-magnitude MAE by 46.71% across 64 episodes, 3 seeds, and 3 noises (0.018935 vs. 0.035532, 64/64 episodes improved, Holm p = 0.000030, R1), and across 64 253-frame trajectories and 3 seeds (R6) cuts LPIPS by 14.13% (0.083246 vs. 0.096940, Holm p = 0.00018) and changing 2D RAFT flow error by 5.57% (0.801974 vs. 0.849290 px, Holm p = 0.00018). Likewise, once initial frames leave the window on DFoT (Table R7), constant holding without memory anchors to drifted frames and loses to EGSA with memory across all (0.019396 for loop_w0.2 vs. 0.010252 at matched view-change ratio, -47.14%, Holm p = 0.00017).
2. Contribution of memory retrieval versus evidence gating in Table R2
First, previous-frame EGSA (0.017680) has higher return MSE than LoRA (0.016253) in Table R2 because windows 2 and 3 (4-11, 8-15) condition entirely on generated frames, unlike the paper's single-window setup where real context reached past the turnaround. Previous-frame anchoring pulls return frames (12-15) toward warped, drifted turnaround generations rather than evicted observations (0-3). Supplying last-valid memory restores the initial ray bank and cuts return MSE by 42.01% (0.017680 to 0.010252, 56/56 scenes, Holm p = 0.000020).
Second, the trade-off between evidence-free memory and EGSA with memory comes from camera translation under a pure-rotation ray memory. At return (12-15), the palindrome walks back onto the exact camera centers of frames 3-0, so parallax vanishes; setting pulls return frames toward stored ground-truth rays at full weight (approaching Table R5's 0.000000 direct-reuse bound) and overwrote fewer bins during departure by suppressing outbound view change (0.703 vs. 0.755 for EGSA, 0.993 for LoRA), giving 0.007505 vs. 0.010252 return MSE. During outbound (4-7) and middle (8-11), however, camera translation creates parallax and disocclusions where pure-rotation warps fail. Evidence-free memory () forces tokens toward misaligned warps, more than doubling outbound MSE (+115.0%, 0.026927 vs. 0.012524) and middle MSE (+106.6%, 0.026101 vs. 0.012634). EGSA detects these discrepancies () and releases misaligned anchors, cutting 12-frame all-generated MSE by 41.50% vs. evidence-free memory (0.011803 vs. 0.020177, CI [-45.44%, -37.39%], 56/56 scenes, p = 0.000010). Gating reads by camera-center support (Table R5) cuts return MSE to 0.000562 with zero outbound penalty (all MSE 0.007063, LPIPS 0.100472 across 56 scenes).
Holding generator weights fixed within each controlled pair isolates each component:
| Component | Controlled comparison | Isolated effect |
|---|---|---|
| Memory retrieval | EGSA + memory vs. EGSA + prev (R2) | Recovers evicted views: -42.01% return MSE (56/56, p = 0.000020) |
| Geometric support | Supported mem vs. No anchor (R5) | Removes parallax ghosting: -30.80% LPIPS, -44.19% MSE (56/56) |
| Evidence gate (parallax) | EGSA + mem vs. Evidence-free + mem (R2) | Releases misaligned warps: -41.50% overall MSE (56/56, p = 0.000010) |
| Evidence gate (dynamics) | EGSA vs. Const / Shuf, equal budget (R1, R6) | Releases moving objects: -46.71% / -46.26% mag MAE; -14.13% LPIPS, -5.57% / -6.00% flow err |
In the revised paper, we attribute long-range static revisit recovery directly to historical memory retrieval and geometric support, and attribute evidence gating to selective release under object motion and unmodeled correspondence error.
Thank you for the thoughtful review. Below we report our new dynamic manipulation evaluations, rolling-window generation beyond four frames, comparison with memory-based and geometry-guided world models, and long-horizon attention-bias ablations.
1. Evidence gating in realistic dynamic scenes (W1)
We evaluate online latent-change evidence (with no ground-truth mask at inference) on real action-conditioned DROID manipulation sequences containing moving arms, object interactions, and occlusions. Within each training seed, all arms share LoRA weights, initial frame, actions, noise, and sampler:
2. Generation beyond four frames and camera translation (W2)
Alongside the non-palindromic 253-frame DROID rollouts, DFoT observes only frames 0-3 and generates 12 future frames (4-15) across three 8-frame windows (0-7, 4-11, 8-15), evicting frames 0-3 before return (12-15). Across 56 scenes, 3 seeds, and 2 noises (Table R2), last-valid memory lowers return MSE by 36.92% vs LoRA (0.010252 vs 0.016253, CI [-40.85%, -32.15%], Holm p=0.000020) and 42.01% vs previous-frame EGSA (0.017680, CI [-44.42%, -39.39%], Holm p=0.000020; 56/56 scenes improved; 12-frame MSE -11.88%), while beating matched-tracking EMA (-34.15%), warp-and-blend (-48.92%), and in-sampler constant hold (-47.14%) in Table R7 (all Holm p=0.00017). To handle camera translation where pure-rotation warping is invalid, restricting reads to geometry-supported poses across all 56 scenes (Table R5) cuts all-generated LPIPS from 0.145196 to 0.100472 (-30.80%, 56/56 scenes; -30.86% on 50 held-out scenes).
3. Relation to WorldMem, GEN3C, and Closing the Loop (Q1)
WorldMem, GEN3C, and Closing the Loop determine how past observations are stored, projected, and fed into network conditioning. EGSA operates inside the denoising step to control how strongly an aligned state prior constrains the clean-state estimate, releasing that constraint where online change evidence appears. Retrieved memory supplies EGSA's prior in Tables R2 and R5, while Tables R1 and R6 isolate evidence-gated release under object motion.
4. Necessity of the attention-logit bias (Q2)
It is not necessary. Removing attention-logit bias on our 56-scene, 3-seed, 2-noise DFoT memory evaluation (Table R2) shifts return MSE by +0.002441% (0.010252 to 0.010253) and all-generated MSE by -0.007850% (0.011803 to 0.011802), and all new DROID evaluations run without it. We therefore treat clean-state anchoring as the core mechanism and keep the bias optional.