acceptodds
Under review as a conference paper at ICLR 2027

SyncVid: World-Equivariant Diffusion For Synchronized Video Generation

Abstract

Generating multiple videos of a dynamic scene from a single reference video requires synthesizing both observed and unobserved content consistently across viewpoints. Existing camera-control methods for video diffusion typically rely on architectural mechanisms, including camera embeddings, cross-view attention, and geometric conditioning. Although effective for static scene content, these approaches often struggle with the fine-grained, dynamic geometry that must remain coherent across views and over time. We synchronize the diffusion process itself by sampling the Gaussian noise prior in a canonical 3D space derived from the source video and rasterizing it into target cameras, rather than sampling independently in each 2D image plane. Our method, SyncVid, lifts a monocular video into a canonical 4D epresentation, associates noise with scene geometry and unseen space, and rasterizes the resulting structured noise into arbitrary views. Because rendering can introduce local correlations, we further propose a cross-view-consistent whitening procedure that decorrelates neighboring pixels while preserving correspondence across views. The structured noise serves as the diffusion prior for a LoRA-adapted pretrained video diffusion model, requiring no architectural modifications. By enforcing spatiotemporal and cross-view coherence at the source of stochasticity, SyncVid generates synchronized videos and improves both perceptual quality and cross-view consistency in our experiments.

Primary area:
generative models
Keywords:
Video Diffusion, Camera Control, Structured Noise

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_yDkLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new method, namely, SyncVid, to generate synchronized video from a single source video. The main idea is to replace standard Gaussian noise with a structured noise in canonical 3D space. In particular, the proposed method first reconstructs a dynamic 4D Gaussian representation from the source video. Given Gaussian primitives, fixed noise vectors are assigned to the 3D Gaussian primitives and rasterized into target camera views. A view-consistent whitening is further introduced to reduce local correlations while preserving cross-view correspondence. The structured noise feature, source, and rendered video embeddings are concatenated as the input of a video diffusion model, where the video diffusion model is fine-tuned with LoRA. Experiments show that the proposed method outperforms state-of-the-art approaches in overall view Synchronization, temporal consistency, and camera accuracy, as well as in human preference evaluations.

Critical strengths
  1. The paper is well motivated and interesting.

  2. The paper provides a theoretical analysis of how whitening reduces local correlations

  3. Extensive experiments and the demo show that the proposed method outperforms the state-of-the-art methods.

Critical weaknesses
  1. The main contribution is to replace typical standard Gaussian noise with the noise that preserves correspondence across views. However, as shown in Table 4, canonical 3D noise without whitening reduces FVD-V from 518.77 to 468.39. It is unclear whether the improvements mainly come from cross-view noise correspondence. It would be better to provide an additional ablation that removes cross-view noise correspondence. For example, first independently sample Gaussian noise for different target views and combine it with the whitening procedure.

  2. The paper introduces the whitening procedure, which is claimed one of the main contributions. However, the ablation study results show that whitening only reduces FVD-V from 468.39 to 461.44. In addition, the RFF frequency scale plays an important role in the whitening procedure. Yet, it seems that the paper does not provide the details of how this parameter is determined.

Questions for authors

How are the RFF frequency vectors and scene-dependent frequency scale determined? How sensitive is the whitening procedure to these parameters?

3 comments

We thank the reviewer for the positive assessment and encouraging recommendation. We are pleased that the reviewer found the paper well motivated, the theoretical analysis useful, and the experiments convincing. We address the two weaknesses and the two questions below; the new results have been added to the revision (in blue).

W1. Do the gains come from cross-view noise correspondence?

We agree that this should be isolated directly. We removed the correspondence where it acts: the per-primitive noise vectors zi\mathbf z_i are resampled independently at every frame, so the noise keeps the same rasterized structure and is whitened identically, but no longer follows the same scene content, neither over time nor between views:

Variant (Wan2.2-5B, +both)Mat. Pix. ↑FVD-V ↓CLIP-V ↑CLIP-F ↑
standard i.i.d. noise222518.7788.7697.34
independent per-frame noise + whitening205520.3188.6996.98
shared canonical 3D noise, w/o whitening208468.3988.5396.92
shared canonical 3D noise + whitening (SyncVid)267461.4491.0597.61

With the correspondence removed, whitening alone brings no benefit: the variant performs on par with, or slightly below, standard i.i.d. noise on all four metrics, whereas the same whitening applied to shared canonical 3D noise yields the full gain. The synchronization improvement therefore comes from binding the noise to persistent scene content, and whitening is what lets the pretrained model exploit it. We note that the existing Table 4 already gives partial evidence: shared 3D noise without whitening reduces FVD-V from 518.77 to 468.39, i.e. most of the FVD-V improvement is obtained by the correspondence alone, before any whitening is applied.

W2/Q1/Q2. Effect of whitening and how the RFF frequency scale is set.

Effect of whitening. FVD-V alone understates the role of whitening. Its role is not mainly to further improve distribution-level similarity, which the correspondence already provides, but to remove the local spatial correlations introduced by overlapping splat footprints (Eq. 12), which push the noise away from the i.i.d. distribution the pretrained model expects. Without whitening, the structured noise improves FVD-V but degrades Mat. Pix. (222→208), CLIP-V and CLIP-F relative to standard noise. Whitening reverses this: it raises Mat. Pix. by 28% (208→267), CLIP-V by 2.5 points (88.53→91.05) and CLIP-F by 0.7, and the full method is the only variant that improves all four metrics over standard noise. We also measured this decorrelation directly (App. A.8, Fig. 9): the lag-1 correlation between neighbouring noise values drops from 0.180.18 before whitening to 0.090.09 at the reference scale s0s_0 and to 0.020.02 at 16s016s_0.

Q1. How the RFF frequency vectors and the frequency scale ss are set. The fixed frequency vectors of Eq. 14 are random unit directions scaled by dyadic octaves, ωa=2ℓara\boldsymbol\omega_a=2^{\ell_a}\mathbf r_a with ℓa∈{0,…,L−1}\ell_a\in\{0,\dots,L{-}1\} and L=12L{=}12, so the projection spans frequencies from ss to 2L−1s2^{L-1}s. Because the feed-forward reconstruction has an arbitrary metric scale, we tie ss to the correlation length of the rasterized noise, ℓc=(2σˉ)2+dˉ 2\ell_c=\sqrt{(2\bar\sigma)^2+\bar d^{\,2}}, from the median footprint radius σˉ\bar\sigma and the median 3-NN distance dˉ\bar d of the visible Gaussians, and set s0=1/(2⋅2L−1ℓc)s_0=1/(2\cdot2^{L-1}\ell_c), i.e. the highest frequency sits at the Nyquist limit of ℓc\ell_c (Eq. 35, App. A.8). Higher scales alias and become sensitive to small errors in Xp\mathbf X_p; lower scales leave neighbouring bases nearly identical. s0s_0 is computed in closed form, with no tuning.

Q2. Sensitivity. Sweeping s=Ms0s=Ms_0 (Wan2.2-5B, +both; Table 9 of the revision):

MMMat. Pix. ↑FVD-V ↓CLIP-V ↑CLIP-F ↑
1/161/16207505.6688.2996.93
1/41/4211491.1989.7097.25
11267461.4491.0597.61
44230474.0790.0597.20
1616218487.3988.8697.28

The results are sensitive to ss and degrade on both sides of M=1M{=}1, which is best on all four metrics, as predicted by Remark 1 (App. A.3); every swept scale still improves FVD-V over i.i.d. noise. At the noise level (Fig. 9), increasing MM removes more of the neighbour correlation (6%, 50%, 93% at M=116,1,16M{=}\tfrac1{16},1,16) but keeps less of the cross-frame agreement (98%, 64%, 19%); the untuned, closed-form s0s_0 lands at the balance point.

We thank the reviewer again for the constructive suggestions, which we believe strengthen the paper's central claim. If there are any remaining questions, concerns, or aspects that would benefit from further clarification, we would greatly appreciate your feedback. Your insights are invaluable to us, and we would be grateful for any additional suggestions that could help strengthen the paper.

Dear Reviewer yDkL,

Thank you once again for your valuable and positive comments and the time you have invested in reviewing our paper. As the discussion period is nearing its end, we would greatly appreciate hearing whether our responses have helped clarify and address the concerns outlined in your review.

Thank the authors for the detailed response and additional experiments. The new ablation studies show the benefits of noise correspondence and whitening, and the additional analysis of the RFF frequency scale addresses most of my concerns. I will maintain my positive rating.

Reviewer_Bi8JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SyncVid for generating synchronized target-view videos from a monocular source video and a given camera trajectory. It constructs a canonical 4D Gaussian representation, assigns shared noise to scene primitives, and rasterizes the noise into different target views. An approximate whitening step is introduced to reduce local noise correlations while preserving cross-view correspondence. The resulting structured noise is used as the diffusion prior for a LoRA-adapted video diffusion model, and experiments on Multi-ViewBench show improvements in cross-view consistency and perceptual quality.

Critical strengths
  1. The paper provides a clear mechanism for cross-view noise correspondence. Equation 9 describes the shared covariance before whitening, while Equation 18 characterizes the remaining covariance after projection. Under the same-dominant-primitive assumption, corresponding pixels share the same projection and structured noise across views.
  2. The analysis of view-consistent approximate whitening explains how the projection reduces rasterization-induced correlations. It also clarifies the conditions for preserving cross-view correspondence and the module’s limitations.This makes the whitening module easier to understand and more convincing.
  3. The gains on the metrics the method targets are the best among the compared methods: Mat. Pix. 1332 vs. 1268, FVD-V 146.99 vs. 195.55, CLIP-V 91.66 vs. 89.96 and CLIP-F 97.87 in Table 1, with a 78% to 7% preference over ArtiFixer in the blind study.
  4. Multi-ViewBench is selected automatically against six admissibility gates and split deterministically, Table 8 breaks down runtime per component, and Appendix A.7 shows disocclusion and foreground failures rather than only successes, which makes the claims easier to audit than a paper reporting only qualitative wins.
Critical weaknesses
  1. Although the structured-noise design is well motivated, its main methodological novelty is not sufficiently isolated. The framework combines an existing 4D Gaussian representation, primitive-level noise rasterization, RFF/QR-based whitening, and LoRA adaptation. However, Table 4 does not separately evaluate the contribution of binding noise to persistent 4D primitive identities, making it difficult to determine which part of the pipeline drives the improvement.

  2. The evaluation protocol is not fully clear. Section 4.1 states that the videos are standardized to 5 seconds at 15 fps, whereas Table 1 reports generation lengths of 49, 69, and 75 frames. The paper should explain whether all metrics are computed over the same temporal window and how outputs of different lengths are processed.

  3. Although the paper discusses the conditions and limitations of its theoretical analysis, the exact equality in Equation 32 appears stronger than Assumption 2 supports. The assumption requires one primitive to dominate while the remaining weights are only approximately zero, and the proof further relies on the dominant weight being large relative to the stabilization term ε. The authors should clarify whether Equation 32 is an exact result, an approximation, or a limiting statement. Additional analysis under occlusion boundaries, primitive blending, and temporal visibility changes would also help show how the noise-level guarantee translates into consistency in the generated videos.

  4. The main results use Wan2.2-14B, while the whitening, conditioning, and LoRA-rank ablations in Table 4 use Wan2.2-5B. Although the smaller-backbone ablations are informative, they do not directly establish that the observed component gains and trade-offs transfer to the final 14B model. Key ablations should be reproduced on the same backbone as the main results, or the consistency of these gains across backbone sizes should be reported.

Questions for authors

see above weakness

3 comments
AuthorsOpenReview ↗

Responses to W1, W2, and W3.

We thank the reviewer for the detailed review and for highlighting the clarity of the correspondence and whitening analysis, the strength of the results, and the auditability of Multi-ViewBench. We address each weakness below; all new results have been added to the revision (in blue).

W1. Isolating the methodological novelty; binding noise to persistent primitive identities.

Our contribution is not the 4D representation or the LoRA adaptation, both of which are off-the-shelf and interchangeable. It is to synchronize the stochastic source of diffusion itself, through (i) binding the noise to persistent primitive identities and to a finite proxy for unseen space, and (ii) a whitening indexed by canonical 3D position that restores the i.i.d. marginal without breaking that correspondence; screen-space warping or whitening (e.g. Go-with-the-Flow) cannot provide (ii) under out-of-plane motion. To isolate (i) as requested, we added an ablation that removes the binding of noise to persistent primitive identities while keeping whitening, conditioning and the marginal statistics fixed: the per-primitive noise vectors zi\mathbf z_i are resampled independently at every frame, so the noise keeps the same rasterized structure and is whitened identically, but no longer follows the same scene content. We report it alongside the noise rows of Table 4, where it now also appears:

Variant (Wan2.2-5B, +both)Mat. Pix. ↑FVD-V ↓CLIP-V ↑CLIP-F ↑
standard i.i.d. noise222518.7788.7697.34
independent per-frame noise + whitening205520.3188.6996.98
shared canonical 3D noise, w/o whitening208468.3988.5396.92
shared canonical 3D noise + whitening (SyncVid)267461.4491.0597.61

Removing the identity binding eliminates the gain: with whitening kept but zi\mathbf z_i resampled per frame, the model performs on par with, or slightly below, standard i.i.d. noise on all four metrics, whereas the same whitening applied to identity-bound canonical 3D noise yields the full gain. The two components therefore play distinct roles. Binding noise to persistent primitive identities (i) is what drives the FVD-V gain: even without whitening it already lowers FVD-V from 518.77 to 468.39. Whitening (ii) contributes nothing on its own, but it is what allows the pretrained model to exploit the identity-bound noise, raising Mat. Pix. from 208 to 267 and CLIP-V from 88.53 to 91.05.

W2. Evaluation protocol for different generation lengths.

All Multi-ViewBench sources are 75 frames (5 s at 15 fps, 480p). Each baseline is run with its official checkpoint at its native length (DaS and Go-with-the-Flow: 49 frames, Reshoot-Anything: 69, ReCamMaster and ArtiFixer: 75), since forcing other lengths takes these models out of their training regime. Methods that generate shorter clips receive the first source frames and the corresponding prefix of the target trajectory, i.e. the trajectory is cut rather than temporally compressed, so camera speed is identical for all methods. All metrics are computed on the full generated video of each method and are length-normalized, so they do not scale with the number of frames. Longer outputs are, if anything, evaluated on a harder task: the target trajectories diverge progressively from the source pose (App. A.2), so the frames beyond 49 contain the largest viewpoint changes, and SyncVid is scored over all 75 frames. We have stated this protocol explicitly in Sec. 4.2 and in the Table 1 caption (in blue).

W3. Exactness of Eq. (32) under Assumption 2.

We agree: Eq. (32) is a limiting statement. Proposition 2 shows that Xp\mathbf X_p, and hence Qp\mathbf Q_p and ϵp3D\boldsymbol\epsilon^{\mathrm{3D}}_p, depend on the camera only through the rasterization weights wp,jw_{p,j}, never through the pose itself. Equality across views is exact only in the limit where one primitive dominates (δ/w→0\delta/w\to0, with δ=∑j≠iwp,j\delta=\sum_{j\ne i}w_{p,j}) and ε/w→0\varepsilon/w\to0. Even for a single primitive, ε\varepsilon leaves a residual (Xp=ww+εμˉi\mathbf X_p=\tfrac{w}{w+\varepsilon}\bar{\boldsymbol\mu}_i), and since ww depends on the view, the residual differs slightly between views. In practice ε\varepsilon is negligible relative to the weight of a visible dominant primitive, so the gap is governed by blending, i.e. by occlusion boundaries and visibility changes, where the generated videos show the failure modes in App. A.7.

As related empirical evidence, Fig. 9 of the revised appendix (App. A.8) measures noise agreement at exactly matched pixels across frames, which additionally includes motion and visibility changes: before whitening, matched pixels agree with correlation +0.34+0.34, and at the reference scale whitening keeps 64% of this agreement. We have restated Proposition 2 and Remark 2(ii) as limiting statements and added the first-order bound to App. A.4 (in blue).

AuthorsOpenReview ↗

Response to W4.

W4. Ablations on the 5B rather than the 14B backbone.

The 5B model was used to keep the full ablation grid affordable: each Wan2.2-14B experiment actually consists of two separate multi-GPU trainings (it is a mixture-of-experts architecture), so we used the next-largest model available to us, Wan2.2-5B, for the controlled ablations. Regarding the reviewer's suggestion that the consistency of these gains across backbone sizes should be reported, Table 4 already shows that the full method scales consistently with backbone size (Wan2.1-1.3B→Wan2.2-5B→Wan2.2-14B: Mat. Pix. 217→267→313, FVD-V 689.30→461.44→438.69). Nonetheless, to verify the key component of our contribution on the Wan2.2-14B model of Table 1, we trained Wan2.2-14B with exactly the same training data, conditioning (+both), schedule and inference settings as SyncVid, differing only in using standard i.i.d. noise (Table R2):

Same-backbone control: Wan2.2-14B (+both) with i.i.d. vs. structured noise on Multi-ViewBench, under the Table 1 protocol of the paper.

MethodGeneration LengthMat. Pix. ↑ DynamicMat. Pix. ↑ StaticFVD-V ↓CLIP-V ↑CLIP-F ↑RotErr ↓TransErr ↓
SyncVid (i.i.d. noise)7511901705218.5189.2397.710.1640.148
SyncVid (full)7513321834146.9991.6697.870.1510.121

Replacing i.i.d. noise with our whitened structured noise improves every metric (dynamic Mat. Pix. +12%, FVD-V −33%, CLIP-V +2.4). Notably, with i.i.d. noise the 14B model is only on par with the strongest baselines (e.g. its FVD-V of 218.51 is worse than that of ArtiFixer, 195.55, and Reshoot-Anything, 202.22); the structured noise is what makes it the best method, so the gains in Table 1 are not explained by the backbone. This row has been added to Table 1. The same holds at 5B (Table 4: Mat. Pix. 222→267, FVD-V 518.77→461.44); in both cases the gain comes with no architectural change and no additional parameters.

We hope these clarifications and additional analyses address the reviewer's concerns, either fully or in part. If there are any remaining questions, concerns, or aspects that would benefit from further clarification, we would greatly appreciate your feedback. Your insights are invaluable to us, and we would be grateful for any additional suggestions that could help strengthen the paper.

Dear Reviewer Bi8J,

Thank you once again for your valuable comments and the time you have invested in reviewing our paper. As the discussion period is nearing its end, we would greatly appreciate hearing whether our responses have helped clarify and address the concerns outlined in your review.

Reviewer_df9kOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper generates multiple synchronized videos of a dynamic scene from one source video by changing where the diffusion noise comes from. It lifts the source video into a feed-forward 4D Gaussian Splatting reconstruction, attaches a fixed noise vector to each 3D primitive (plus a cubemap for unseen regions), and rasterizes this noise into each target camera. A "view-consistent whitening" step, a per-pixel orthonormal projection built from random Fourier features of the canonical 3D position, removes local correlations while keeping cross-view correspondence. The structured noise replaces i.i.d. noise in a LoRA-finetuned Wan2.2-14B model. The paper also releases Multi-ViewBench, a 200-clip real-video benchmark. Results beat ReCamMaster, ArtiFixer and Reshoot-Anything on cross-view and VBench metrics.

Critical strengths
  • Simple, intuitive idea: noise in 3D naturally handles out-of-plane rotation and disocclusion that 2D noise warping cannot. No architecture change needed.
  • The whitening construction is the real contribution: indexing the projection by 3D position decorrelates locally without breaking cross-view correspondence. The paper is honest that it is approximate.
  • Solid evaluation effort: several recent baselines, user study, ablations, and a reproducible benchmark protocol.
Critical weaknesses
  • Main comparison does not isolate the contribution. Table 1 pits a finetuned Wan2.2-14B against baselines' official checkpoints with different backbones and clip lengths. The key control, the same 14B model with the same conditioning and standard i.i.d. noise, is missing. It only appears in the 5B ablation, where the gain is modest (Mat. Pix. 222 to 267).
  • Ablation undercuts the core claim. Unwhitened 3D noise is worse than standard noise (Table 4). The raw idea hurts, and whitening only recovers a moderate gain.
  • Inconsistent model selection. "+source view" beats "+both" on 3 of 4 metrics, including the synchronization metric, yet "+both" is chosen based on FVD-V.
  • No analysis of key sensitivities. The frequency scale s and the reconstruction quality are both described as critical but never ablated.
  • Missing detail. The paper never says how the noise is mapped to the spatially and temporally compressed VAE latent grid.
Questions for authors
  1. Can you add a "Wan2.2-14B + same conditioning + standard noise" row to Table 1?
  2. How is the noise rasterized onto the compressed latent grid (spatial and temporal)?
  3. Why choose "+both" conditioning when "+source view" wins on most metrics?
  4. How sensitive are results to the frequency scale s?
3 comments
AuthorsOpenReview ↗

Responses to W1/Q1 and W2 and W2/Q3.

We thank the reviewer for the careful review and for recognizing the simplicity of noise-in-3D, the whitening construction as the key contribution, and the evaluation effort. We address each weakness and question below; all new results have been added to the revision.

W1/Q1. Same-backbone control with standard noise.

We agree that this is the most direct control. We therefore trained Wan2.2-14B with exactly the same training data, conditioning (+both), schedule and inference settings as SyncVid, differing only in using standard i.i.d. noise (Table R1):

Same-backbone control: Wan2.2-14B (+both) with i.i.d. vs. structured noise on Multi-ViewBench, under the Table 1 protocol of the paper.

MethodGeneration LengthMat. Pix. ↑ DynamicMat. Pix. ↑ StaticFVD-V ↓CLIP-V ↑CLIP-F ↑RotErr ↓TransErr ↓
SyncVid (i.i.d. noise)7511901705218.5189.2397.710.1640.148
SyncVid (full)7513321834146.9991.6697.870.1510.121

Replacing i.i.d. noise with our whitened structured noise improves every metric (dynamic Mat. Pix. +12%, FVD-V −33%, CLIP-V +2.4). Notably, with i.i.d. noise the 14B model is only on par with the strongest baselines (e.g. its FVD-V of 218.51 is worse than that of ArtiFixer, 195.55, and Reshoot-Anything, 202.22); the structured noise is what makes it the best method, so the gains in Table 1 are not explained by the backbone. This row has been added to Table 1. The same holds at 5B (Table 4: Mat. Pix. 222→267, FVD-V 518.77→461.44); in both cases the gain comes with no architectural change and no additional parameters.

Clip lengths of the baselines. The different generation lengths in Table 1 are inherited from the baselines' official checkpoints, each built on a different DiT backbone. Methods that generate shorter clips receive the first source frames and the corresponding prefix of the same target trajectory (cut, not temporally compressed), and all metrics are length-normalized, so longer outputs are, if anything, scored on the harder part of the trajectory.

W2. "Unwhitened 3D noise is worse than standard noise."

We respectfully note that this result is expected and motivates whitening rather than undermining the idea. As discussed in Sec. 3.3, overlapping splat footprints make neighbouring pixels share primitives (Eq. 12), producing locally correlated noise that departs from the i.i.d. distribution on which the pretrained model was trained; a LoRA adapter cannot fully absorb this shift. This is precisely the motivation for whitening, and the two components should be read together: cross-view noise correspondence provides the synchronization signal, and whitening makes it compatible with the pretrained prior. Relative to unwhitened noise, whitening raises Mat. Pix. by 28% (208→267) and CLIP-V by 2.5 points. Relative to standard noise, the full method improves Mat. Pix. by 20%, CLIP-V by 2.3 points and FVD-V by 57 points, and it is the only noise variant that improves all four metrics.

W3/Q3. Choice of "+both" over "+source view".

Both configurations use the same structured noise, so this choice does not affect the core claim. We selected "+both" because FVD-V (530.42→461.44, a 13% reduction) compares the distributions of generated and reference views and therefore also reflects quality in disoccluded and newly revealed regions. Mat. Pix. counts high-confidence correspondences only in co-visible regions and thus favours outputs that stay conservatively close to the source view; the rendered view adds explicit geometry for the target camera, which mainly helps the novel content.

AuthorsOpenReview ↗

Responses to W4/Q4 and W5/Q2.

W4/Q4. Sensitivity to the frequency scale ss and to reconstruction quality.

We added both analyses to the revised appendix (App. A.8, marked in blue).

Frequency scale. ss is not tuned: it is computed in closed form from the reconstruction (Eq. 35). The fixed frequency vectors are random directions scaled by dyadic octaves, so the projection spans frequencies from ss to 2L−1s2^{L-1}s, and s0s_0 places the highest one at the Nyquist limit of the noise correlation length, which is set by splat size and spacing (median footprint radius and 3-NN distance of the visible Gaussians). Higher scales alias, making the projection sensitive to small errors in Xp\mathbf X_p; lower scales leave neighbouring bases nearly identical. Sweeping s=Ms0s=Ms_0 (Wan2.2-5B, +both; Table 9):

MMMat. Pix. ↑FVD-V ↓CLIP-V ↑CLIP-F ↑
1/161/16207505.6688.2996.93
1/41/4211491.1989.7097.25
11267461.4491.0597.61
44230474.0790.0597.20
1616218487.3988.8697.28

The results are sensitive to ss and degrade on both sides of M=1M{=}1, which is best on all four metrics, as predicted by Remark 1 (App. A.3); at M=116M{=}\tfrac1{16} they fall back to the unwhitened case, and every swept scale still improves FVD-V over i.i.d. noise. At the noise level (Fig. 9), increasing MM removes more of the neighbour correlation (6%, 50%, 93% at M=116,1,16M{=}\tfrac1{16},1,16) but keeps less of the cross-frame agreement (98%, 64%, 19%); the untuned, closed-form s0s_0 lands at the balance point.

Reconstruction quality. Relating the source-view re-rendering PSNR of the reconstruction to the novel-view metrics on Multi-ViewBench clips (Fig. 10), CLIP-V is largely unaffected (its confidence band admits a flat line), whereas Mat. Pix. and the temporal metrics (CLIP-F, VBench) increase with PSNR. Reconstruction quality therefore mainly affects temporal smoothness and the number of matched dynamic pixels, consistent with the 4D representation only synchronizing the noise and rendering the conditioning view; severe tracking failures remain a limitation (App. A.7).

W5/Q2. How noise is mapped to the compressed latent grid.

The noise is rasterized directly at latent resolution. Spatially, we scale the target intrinsics by the VAE stride (×18\times\tfrac18 for Wan2.2-14B, ×116\times\tfrac1{16} for Wan2.2-5B), so each latent cell receives one HH-dimensional rasterized vector, which is then normalized and whitened to CC channels (Eqs. 11, 16). Temporally, the Wan causal VAE encodes the first frame alone and each subsequent group of 4 frames into one latent frame. We rasterize the noise for every video frame, with the Gaussian state G(t)\mathcal G(t) and target pose Πttar\boldsymbol\Pi^{\mathrm{tar}}_{t} of that frame, and assign latent frame 00 the first video frame and latent frame k≥1k{\ge}1 the second frame of its 4-frame group (the one nearest its centre); the same mapping is used in training and inference. We have added this description to Sec. 3.5 (in blue).

We hope these additions address the reviewer's concerns, either fully or in part. If there are any remaining questions, concerns, or aspects that would benefit from further clarification, we would greatly appreciate your feedback. Your insights are invaluable to us, and we would be grateful for any additional suggestions that could help strengthen the paper.

Dear Reviewer df9k,

Thank you once again for your valuable comments and the time you have invested in reviewing our paper. As the discussion period is nearing its end, we would greatly appreciate hearing whether our responses have helped clarify and address the concerns outlined in your review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.