acceptodds
Under review as a conference paper at ICLR 2027

On-Policy Distillation of Video and Multi-View Priors for Multi-View Video Generation

Abstract

Multi-view video generation aims to synthesize synchronized videos of the same dynamic scene from different viewpoints. Direct generation methods face a shortage of synchronized training data, while combining monocular videos and static multi-view images provides only separate supervision for motion and geometry. Geometry-guided methods rely on estimated depth or 3D proxies, whose errors can compromise cross-view consistency. We propose MultiLive, an on-policy dual-teacher distillation framework that combines separately learned video and multi-view priors in a single generator. Our key idea is to concurrently match the temporal and multi-view distributions of the student's own generations to these complementary priors. The video teacher guides motion within each monocular video, while the multi-view teacher guides geometry and appearance across synchronized views. Both distribution-matching objectives update the student using the same generated sample, encouraging the two capabilities to remain compatible as the scene evolves without explicit 3D reconstruction. Extensive experiments demonstrate that MultiLive achieves high visual quality, strong cross-view consistency, and coherent temporal dynamics without multi-view video training data, using only 4 denoising steps at inference.

Primary area:
generative models
Keywords:
Diffusion Models. Multi-View Video Generation. Diffusion Distillation. Knowledge Distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_zfcKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes MultiLive, a framework for generating synchronized multi-view videos from text and camera poses without synchronized multi-view video training data. It distills a video teacher and a camera-conditioned multi-view teacher through distribution matching on complementary slices of the same student-generated sample. A geometry-aware adapter enables interaction across time and views, while a shared fake-score model uses separate forward passes for the two objectives. The multi-view teacher is trained on causal VAE representations of repeated static images to support supervision at different latent-time positions. Experiments demonstrate the effectiveness of the proposed method.

Critical strengths
  • The proposed method applys both teachers to the same generated sample is a useful approach to learning without synchronized multi-view video data, and the comparisons with mixed-data SFT, single-teacher distillation, and sequential distillation support this design. This is interesting and novel to me.
  • Experiments demonstrate the proposed method outperforms other baselines and achieves better results.
  • The paper is well-written and easy to understand.
Critical weaknesses
  • The performance of the multi-view teacher (trained by the authers) is not reported, making it difficult to assess the quality of the geometric prior and how well it is transferred to the student. I would recommend that the authors evaluate the teacher’s visual quality, camera accuracy, and cross-view consistency, and compare these with the corresponding student outputs under matched condition.
  • The proposed method involves two teachers, which also makes the training to cost more time and computational resources. As discussed in the paper, the proposed method is limited to four 81-frame videos currently. I would recommend that the authors clarify the maximum supported sequence length and provide ablations with different frame counts and numbers of views, reporting generation quality, inference time, and peak GPU memory usage.
Questions for authors

N.A.

2 comments
AuthorsOpenReview ↗

Response to Reviewer zfcK (1/2)

Thank you for the helpful suggestions and encouraging feedback!

W1: The performance of the multi-view teacher is not reported, making it difficult to assess the quality of the geometric prior and how well it is transferred to the student.

Thank you for the suggestion. We evaluate the multi-view teacher's visual quality, camera accuracy, and cross-view consistency, and compare them with the corresponding student outputs under matched conditions. When training the multi-view teacher, we update only the adapter and keep the Wan backbone frozen, so the teacher can also generate multi-view videos directly. We therefore compare both images and videos:

MethodFID ↓CLIP-T ↑RotErr ↓TransErr ↓Mat. Pix. (K) ↑CLIP-V ↑NFE ↓Latency (s) ↓
MV teacher (image)128.8530.355.6211.55187.3397.685012.38
MultiLive (first frame)135.0630.1112.5213.64187.7097.304–
MethodFID ↓FVD ↓CLIP-T ↑CLIP-F ↑Dynamic Degree ↑Flow Score ↑RotErr ↓TransErr ↓Mat. Pix. (K) ↑FVD-V ↓CLIP-V ↑NFE ↓Latency (s) ↓
MV teacher (video)116.88137.5429.2399.710.2151.076.8912.05328.1562.5495.1750801.40
MultiLive114.67111.3230.6799.690.5263.2113.1818.21443.2624.7997.91438.12

The multi-view teacher achieves low camera error and high cross-view consistency, providing a strong geometric prior. This prior is well transferred to the student: on images, the student matches the teacher in cross-view consistency, and on videos, it clearly surpasses the teacher. The teacher only learns cross-view consistency within a single time step, so its consistency degrades when it generates whole videos. During distillation, however, it only needs to score the student one time step at a time, which is exactly what it does well.

The multi-view teacher offers more precise camera control, but this comes at the price of more denoising steps and less dynamic motion. Bringing such camera precision to few-step generation is a promising direction for future work.

AuthorsOpenReview ↗

Response to Reviewer zfcK (2/2)

W2: The two teachers make training cost more time and computational resources, and the method is currently limited to four 81-frame videos. Please clarify the maximum supported sequence length and provide ablations with different frame counts and numbers of views, reporting generation quality, inference time, and peak GPU memory usage.

Thank you for raising this practical concern. Below, we report the training and inference costs, clarify the supported sequence length, and provide the requested ablations over frame counts and numbers of views.

Cost of two teachers. Both teachers are frozen and only run forward passes. They need no gradients or optimizer states, so they add little memory. In the default setting (four views × 81 frames), a training step takes 25.37 s with 42.0 GiB peak memory, and distillation converges in only 2,000 iterations. More importantly, the teachers are used only during training. At inference, only the student runs, in 4 steps without CFG. Four views take 89.65 s with 22.8 GiB peak memory, and two views take 38.12 s with 18.6 GiB, over 10× faster than SynCamMaster (400.31 s, Table 1).

Maximum sequence length. The joint attention and both readouts are defined for any number of views VV and latent times TT. In fact, the model trained with four views directly generates eight views without retraining (last row of the table below). In practice, the sequence length is set by the video teacher, whose scores are most reliable within its pretraining length. We therefore use 81 frames to match Wan. This is not a cost bottleneck: peak training memory is only 42.0 GiB, about half of a single 80 GB GPU. With a video teacher pretrained on longer videos, MultiLive can directly distill longer sequences.

Different numbers of views and frames. We train MultiLive with 2 or 4 views and 33 or 81 frames. We also apply the default four-view model to eight views at inference. Latency and peak memory are measured for generating all views at inference:

ViewsFramesFID ↓FVD ↓CLIP-T ↑CLIP-F ↑RotErr ↓TransErr ↓Mat. Pix. (K) ↑FVD-V ↓CLIP-V ↑Latency (s) ↓Peak Mem. (GiB) ↓
233114.64113.5229.8099.7114.2519.23370.7532.0296.3410.6218.4
281112.33109.6130.4798.8716.6716.94387.5436.4997.0238.1218.6
433117.56114.6330.6899.7814.2616.55429.9125.6297.9626.9618.5
481 (default)114.67111.3230.6799.6913.1818.21443.2624.7997.9189.6522.8
8 (trained on 4)81121.93131.6429.3198.8712.3315.71467.0222.2796.98348.1431.5

MultiLive works robustly across the trained settings, with similar visual quality and camera accuracy. Notably, cross-view consistency improves with more views: Mat. Pix. rises from two to four to eight views, and FVD-V drops. Since all views are generated jointly, more views constrain each other more, which shows the benefit of joint multi-view generation. The eight-view result also shows that MultiLive generalizes beyond the number of views seen in training.

Reviewer_YyY8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes MultiLive, an on-policy dual-teacher distribution-matching framework based on DMD for multi-view video generation without synchronized multi-view video training data. The key idea is to distill two complementary generative priors, a pretrained video teacher for temporal dynamics and a multi-view teacher for cross-view geometric and appearance consistency, into a single unified generator. Rather than optimizing temporal and spatial capabilities separately on monocular videos and static multi-view images, the proposed method applies both distribution-matching objectives to two orthogonal readouts of the same student-generated multi-view videos, encouraging compatibility between the two priors.

The method further introduces a geometry-aware adapter with joint spatiotemporal attention and camera conditioning, a causal-latent multi-view teacher, and branch-specific student-score estimation within a shared fake-score model. Experiments demonstrate competitive video quality, cross-view synchronization, and efficient four-step generation without requiring synchronized multi-view video data.

Critical strengths
  1. A simple and effective framework for combining complementary generative priors without multi-view video data.

The proposed on-policy dual-teacher distillation provides a meaningful way to address the training–inference mismatch arising from separately supervising temporal dynamics and cross-view geometry. Instead of merely combining monocular video and multi-view image datasets through supervised fine-tuning, the method jointly optimizes both objectives on readout sliced from the same student's own generated samples. I find this formulation technically well motivated and practically valuable, particularly given the scarcity of synchronized multi-view video data.

  1. Comprehensive and well-designed ablations supporting the main methodological choices.

I particularly appreciate the ablation study, which investigates different training arrangements (mixed-data SFT, single-teacher distillation, sequential distillation, and simultaneous dual-teacher distillation), multi-view teacher construction, student initialization, and branch-specific score estimation. These experiments provide reasonably convincing evidence for the benefit of simultaneous dual-teacher distillation and the importance of matching the student-score distributions to the corresponding teacher readouts. The reported improvements in video quality and cross-view synchronization, together with four-step inference, further support the practical effectiveness of the proposed method.

Critical weaknesses
  1. Limited theoretical and empirical justification for joint spatiotemporal consistency.

While the proposed dual-teacher DMD is technically well motivated, the sum of the two distribution-matching objectives only constrains the per-view temporal distributions and per-timestamp multi-view distributions, rather than directly matching the full joint spatiotemporal distribution. Therefore, matching these two sets of marginals does not necessarily guarantee coherence across the complete space-time-view dimensions. Although the authors acknowledge in Appendix H that some dependencies coupling time and viewpoint remain only indirectly constrained, I believe this important limitation deserves more explicit discussion in the main paper, especially given that joint spatiotemporal consistency is one of the central motivations of the method. The current experiments demonstrate improvements in video quality and cross-view synchronization, but do not fully establish whether the orthogonal readout-based supervision is sufficient to capture more complex joint spatiotemporal dependencies.

  1. Limited validation of the generality of dual-teacher distillation.

In the main experiments, the multi-view teacher is obtained by adapting the same pretrained Wan video backbone used as the video teacher. Consequently, the two teachers share much of their pretrained representations, and the student is initialized from the multi-view teacher. While the ablation studies convincingly demonstrate the effectiveness of this teacher combination, it remains unclear whether the framework can robustly combine more heterogeneous or independently pretrained video and multi-view teachers. The action-conditioned extension provides additional evidence of applicability, but follows a similar teacher construction strategy. This limits the generality of the conclusions supported by the current experiments.

Questions for authors

See weaknesses.

  1. Architectural choices and interpretation of the ablations (clarification).

I appreciate the comprehensive ablation studies, but would like to clarify two points: The branch-aware score ablation demonstrates the advantage of separate score forwards over a single joint forward, while both variants share fake-score parameters. Is the shared fake-score backbone important for optimization, or mainly motivated by computational efficiency?

The MV-only variant achieves slightly better FVD-V and camera accuracy than Full MultiLive, whereas the full method improves per-view FVD and matched-pixel correspondence. Could the authors explain whether the video-teacher objective mainly preserves the pretrained video prior, or whether it additionally improves joint spatiotemporal consistency beyond the reported metrics?

2 comments
AuthorsOpenReview ↗

Response to Reviewer YyY8

We appreciate your insightful questions and thoughtful feedback!

W1: The sum of the two distribution-matching objectives only constrains the per-view temporal distributions and per-timestamp multi-view distributions, rather than directly matching the full joint spatiotemporal distribution. This limitation deserves more explicit discussion in the main paper.

This is a fundamental observation. We fully agree that our objective approximates the full joint spatiotemporal distribution, and that direct supervision on synchronized multi-view video data remains the more principled solution. However, such data are scarce and costly to capture, which severely limits this route in practice.

Compared with SynCamMaster, which is trained on a small set of synthetic multi-view videos, our method achieves better diversity and consistency (Table 1 and Figure 4). This suggests that our approach is a highly competitive solution at the current stage.

Moreover, our approach is complementary to future SFT on large-scale synchronized multi-view video data. It can provide a stronger starting point for such direct joint supervision.

We will discuss this limitation more explicitly in the main paper.

W2: The two teachers share much of their pretrained representations, and the student is initialized from the multi-view teacher. It remains unclear whether the framework can robustly combine more heterogeneous or independently pretrained video and multi-view teachers.

Thank you for raising this valuable question. Our current experiments mainly validate teacher pairs that share a similar pretrained backbone. For more heterogeneous or independently pretrained teachers, with different latent representations and distribution priors, it remains open whether simply summing their DMD gradients can effectively guide a single student, and whether such a combination remains robust.

We position MultiLive as an exploratory step toward dual-teacher distillation in a specific model and task setting, rather than a complete proof of its generality. In future work, we hope to build on MultiLive to study knowledge alignment and distribution fusion between heterogeneous teachers, and to extend the framework to a wider range of teacher combinations.

Q1(a): The branch-aware score ablation shows the advantage of separate score forwards, while both variants share fake-score parameters. Is the shared fake-score backbone important for optimization, or mainly motivated by computational efficiency?

Sharing is not important for optimization. We use it mainly for simplicity and efficiency. To verify this, we train a variant with two separate fake-score models, one per branch:

Fake scoreFID ↓FVD ↓CLIP-T ↑CLIP-F ↑RotErr ↓TransErr ↓Mat. Pix. (K) ↑FVD-V ↓CLIP-V ↑
Separate114.45113.4030.7799.6614.4715.72452.7624.5798.04
Shared (Ours)114.67111.3230.6799.6913.1818.21443.2624.7997.91

The two variants perform on par. So the gains in the branch-aware score ablation come from the separate forwards, not from sharing parameters. Separate models cost slightly more: peak memory rises from 42.0 to 43.3 GiB, and each training step takes 26.13 s instead of 25.37 s. We therefore keep the shared model, which is simpler and slightly cheaper.

Q1(b): The MV-only variant achieves slightly better FVD-V and camera accuracy than Full MultiLive, whereas the full method improves per-view FVD and matched-pixel correspondence. Does the video-teacher objective mainly preserve the pretrained video prior, or does it additionally improve joint spatiotemporal consistency beyond the reported metrics?

The video teacher mainly preserves the pretrained video prior, and it also contributes to joint spatiotemporal consistency.

Preserving the video prior. Compared with MV-only, Full MultiLive reduces FVD from 140.21 to 111.32. It also raises Dynamic Degree from 0.027 to 0.526 and Flow Score from 0.69 to 3.21. MV-only videos are nearly static, and the video teacher restores the motion prior of the pretrained model.

Contributing to joint spatiotemporal consistency. The multi-view teacher constrains the views at each time, but not how each view evolves over time. The video teacher supplies this temporal constraint on the same sample. As a result, Full MultiLive stays on par with MV-only in FVD-V and achieves higher CLIP-V and matched-pixel correspondence, while generating far more motion. Keeping views consistent while the scene moves is what joint spatiotemporal consistency requires, and nearly static videos do not test this.

Thank you for the detailed and constructive rebuttal. I appreciate the additional experiments and clarifications, which satisfactorily address my methodological questions. While the theoretical limitation of orthogonal distribution matching and generalization to heterogeneous teachers remain open, I acknowledge the authors' clarification of these limitations and their commitment to addressing them in future work. Therefore, I will maintain my original score.

Reviewer_caBsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MultiLive, a framework for generating synchronized multi-view videos without using multi-view video training data. It does so by distilling a video teacher and a multi-view image teacher into one generator. The method uses on-policy distribution matching distillation (DMD) for distillation. Experiments show improved visual quality and cross-view synchronization over SynCamMaster, although camera rotation accuracy is worse. MultiLive jointly generates four views in four denoising steps, substantially reducing inference time relative to the iterative baselines.

Critical strengths
  1. The paper studies a valuable problem.
  2. The core idea is simple and well motivated. The method makes good use of separately available video and multi-view supervision.
  3. Joint generation in a few denoising steps is a useful practical outcome.
  4. The ablations cover the major design choices.
Critical weaknesses
  1. The multi-view teacher may penalize motion but the current evaluation does not evaluate motion. Because the teacher model is trained only on static scenes, and the method encode 4 identical frames as latent for the teacher, for moving content, the MV teacher is out of distribution and could reward static content. In terms of evaluation, the dynamic degree metric from VBench is missing from Table 2. Because the reported gains (flicker, smoothness, CLIP-F) naturally improve when motion is suppressed, without measuring motion directly, it's hard to tell if the model is truly more consistent or just moving less. The paper should evaluate dynamic degree, optical-flow magnitude, or motion statistics compared against Wan itself.

  2. The claim of geometric consistency is supported mainly by image matching and estimated camera poses. These do not establish that the generated views admit a consistent 3D reconstruction. It would be good to use a reconstruction or reprojection-based evaluation to provide more direct evidence.

  3. Some ablations do not isolate the claimed contribution. In the image-only teacher ablation, the authors explicitly pass student latents into a teacher using a different encoder without conversion. Its failure therefore could come from an incompatible interface, but does not establish the advantage of their particular causal-latent construction over a properly aligned alternative. Likewise, the same-time attention ablation changes the teacher, student, and fake-score model together, so it cannot isolate the effect of the student’s attention architecture.

Questions for authors
  1. Does the multi-view teacher suppress motion? What is VBench dynamic degree, or mean flow magnitude, for MultiLive versus Wan and SynCamMaster?
  2. Do the gains over SynCamMaster persist under identical camera conditions? Appendix E uses independently sampled camera pairs for SynCamMaster and four-view rigs for MultiLive. I wonder if it's possible to remove differences in viewpoint separation and overlap, and compare the two methods in the same camera setting.
  3. How much does learned distillation improve over directly composing the same teachers? A score-composition baseline using the same video and multi-view models would help distinguish improvements from the training method from those due to stronger underlying priors.
2 comments
AuthorsOpenReview ↗

Response to Reviewer caBs (1/2)

We thank the reviewer for the valuable questions and constructive suggestions!

W1 & Q1: Does the multi-view teacher suppress motion? What is VBench dynamic degree, or mean flow magnitude, for MultiLive versus Wan and SynCamMaster?

No. MultiLive preserves nearly all of Wan's motion. As suggested, we evaluate VBench dynamic degree and mean flow magnitude (Flow Score), compared against Wan itself. Flow Score follows EvalCrafter (Liu et al., 2023): the mean RAFT (Teed & Deng, 2020) optical-flow magnitude over all pixels and adjacent frame pairs.

MethodDynamic Degree ↑Flow Score ↑
SynCamMaster0.2570.52
Wan0.5483.40
MV teacher only (ablation)0.0270.69
MultiLive (Ours)0.5263.21

MultiLive closely matches Wan and is far more dynamic than SynCamMaster. We attribute this to the video teacher, which constrains the temporal distribution of each view and thus preserves the motion prior of the pretrained video model. Without it (MV teacher only), motion drops sharply.

We will add these results to the revised paper.

W2: The claim of geometric consistency is supported mainly by image matching and estimated camera poses, which do not establish that the generated views admit a consistent 3D reconstruction. A reconstruction- or reprojection-based evaluation would provide more direct evidence.

Thank you for the suggestion. We add a reconstruction-based evaluation and compare against SynCamMaster.

Camera setting. For each prompt, two of MultiLive's four views use exactly the same camera pair as SynCamMaster, and the other two are randomly sampled. Reconstruction and all metrics use only these two shared views, so both methods have identical viewpoint separation and overlap. The same setting is used in Q2.

Reconstruction and metrics. Both methods go through the same reconstruction pipeline. We reconstruct the scene from the two generated views and evaluate the reconstruction using PSNR, SSIM, and LPIPS.

MethodPSNR ↑SSIM ↑LPIPS ↓
SynCamMaster19.330.7340.242
MultiLive (Ours)19.940.7600.223

MultiLive achieves better reconstruction quality on all three metrics, indicating that its generated views are more consistent with a single 3D scene.

W3(a): In the image-only teacher ablation, student latents are passed into a teacher using a different encoder without conversion. Its failure could therefore come from an incompatible interface, and does not establish the advantage of the causal-latent construction over a properly aligned alternative.

We apologize that "separate image-native encoder" in the appendix is inaccurate. The image-only teacher uses the same causal Wan VAE, and encodes each image in the same way as the first frame of a video: one frame is mapped to one latent frame, without temporal downsampling. You are right that the original ablation (Image-only, all TT) passes student latents in without conversion. It therefore cannot rule out an interface mismatch.

In fact, the causal-latent teacher is designed precisely to resolve this interface issue. Wan's causal VAE encodes only the first frame without temporal downsampling; every later latent position compresses four frames. The causal-latent teacher works directly in the student's native latent space, so it can supervise every position. A fair comparison should therefore use another interface-aligned alternative.

We implement such an alternative for Image-only, all TT. We decode the student latents to pixels and encode every frame separately, in the same way as the first frame, for the image-only teacher. Its target is then decoded and re-encoded into the student's latent space. This avoids backpropagating through the VAE, which runs out of memory.

MethodFID ↓FVD ↓CLIP-T ↑CLIP-F ↑RotErr ↓TransErr ↓Mat. Pix. (K) ↑FVD-V ↓CLIP-V ↑
Image-only, all TT (decode → re-encode)116.67113.0129.0299.0513.8315.19400.3024.8397.30
Causal-latent, all TT (Ours)114.67111.3230.6799.6913.1818.21443.2624.7997.91

The causal-latent teacher performs on par with or slightly better than the aligned baseline, while being much simpler: it works directly in the student's latent space and needs no decoding and re-encoding at each step, which also makes training 2.4× faster (25.37 s vs. 61.77 s per step). We will correct the appendix and add this baseline in the revision.

AuthorsOpenReview ↗

Response to Reviewer caBs (2/2)

W3(b): The same-time attention ablation changes the teacher, student, and fake-score model together, so it cannot isolate the effect of the student's attention architecture.

We apologize for the misleading wording. The original Appendix F stated that the three models "all use this layout." In fact, the ablation changes only the student's computation: for the multi-view teacher and the fake-score model, the two attention patterns are equivalent.

  • Multi-view teacher. Both settings use the same frozen teacher. During distillation, it only scores the multi-view readout, i.e., the four views at a single latent time. With T=1T=1, joint 4D attention and same-time attention compute exactly the same function.
  • Fake-score model. Its multi-view branch also processes one latent time at a time, so the two patterns are again equivalent. Its video branch uses only the original Wan path and bypasses the adapter. Its initialization is identical in both settings.
  • Student. The student is the only model that generates the full T×VT \times V grid at once, and hence the only one whose computation changes.

The ablation therefore isolates the effect of the student's attention architecture. We have corrected Appendix F accordingly.

Q2: Do the gains over SynCamMaster persist under identical camera conditions? Appendix E uses independently sampled camera pairs for SynCamMaster and four-view rigs for MultiLive. Is it possible to remove differences in viewpoint separation and overlap, and compare the two methods in the same camera setting?

Yes, the gains persist. Using the same setting as in W2, two of MultiLive's four views share SynCamMaster's camera pair, and the other two are randomly sampled. All metrics are computed on the two shared views only.

MethodFID ↓FVD ↓CLIP-T ↑CLIP-F ↑RotErr ↓TransErr ↓Mat. Pix. (K) ↑FVD-V ↓CLIP-V ↑
SynCamMaster131.34129.1826.2399.458.2219.32213.7657.9789.82
MultiLive (same cameras)111.12111.9430.1899.2214.0014.35438.4233.8397.71
MultiLive (original setting)114.67111.3230.6799.6913.1818.21443.2624.7997.91

Under identical cameras, MultiLive keeps its clear advantage over SynCamMaster.

Q3: How much does learned distillation improve over directly composing the same teachers?

Thank you for the suggestion. We build a score-composition baseline using the same video and multi-view models. At every denoising step, both teachers predict a velocity for the current sample: uvid\mathbf u_{\mathrm{vid}} from the video teacher on each view's frames, and umv\mathbf u_{\mathrm{mv}} from the camera-conditioned multi-view teacher on the views at each time. We take the step with their combination:

u=w uvid+(1−w) umv,w=0.5.\mathbf u = w\,\mathbf u_{\mathrm{vid}} + (1-w)\,\mathbf u_{\mathrm{mv}}, \quad w = 0.5.

We use 50 steps and apply CFG to both teachers. Evaluation follows Table 1.

MethodFID ↓FVD ↓CLIP-T ↑CLIP-F ↑RotErr ↓TransErr ↓Mat. Pix. (K) ↑FVD-V ↓CLIP-V ↑NFE ↓Latency (s) ↓
Score composition118.97137.4427.2296.9026.2336.56285.9525.3297.2550554.44
MultiLive (Ours)114.67111.3230.6799.6913.1818.21443.2624.7997.91438.12

Since both methods share the same underlying priors, the gap comes from the training method. Composition falls behind mainly in video quality and camera control. Both teachers write into the sample at every step. The video teacher ignores the cameras and weakens camera control. The multi-view teacher handles each time step independently and breaks temporal coherence.

MultiLive avoids this through dual-teacher distribution matching. The teachers never write into the sample. Instead, each teacher matches the distribution of the student's own samples in its readout space: per-view videos for the video teacher, and same-time view sets for the multi-view teacher. Both gradients then update the same student. The student thus learns one distribution that satisfies both teachers at once, instead of trading them off at every step.

Comments on the paper

2 comments

We thank all reviewers for their valuable comments. We have uploaded a revised PDF, with changes marked in blue:

  1. Conclusion: a more explicit discussion of the joint spatiotemporal distribution.
  2. Appendix F: a corrected description of the image-only teacher, and a clarification of the same-time attention ablation.
  3. Appendix I: additional results on motion dynamics, comparison under identical cameras with 3D reconstruction, additional ablations, the multi-view teacher versus the student, and cost and scalability.

We are deeply grateful to all three reviewers for the time and effort you have devoted to evaluating our work.

Reviewer caBs asked questions that were all directly measurable. We added motion metrics, a reconstruction-based evaluation, an interface-aligned teacher baseline, a same-camera comparison, and a score-composition baseline. Our claims hold under all of them. The empirical support is now much stronger, and every concern is answered by an experiment rather than by an argument. These questions also led us to correct an inaccurate description in Appendix F.

We sincerely appreciate Reviewer YyY8's recognition of our rebuttal. The comments on our objective and on heterogeneous teachers identify real limits of the method. We now state both in the main paper instead of only in the appendix. This makes the scope of our contribution clearer, and it defines what we study next.

Reviewer zfcK asked what the method costs and how far it scales. We now report our multi-view teacher against the student under matched conditions. We also report quality, latency, and peak memory across 2, 4, and 8 views and 33 and 81 frames. A reader can now judge whether the method fits their own budget, which makes it much more practical to adopt. These experiments also brought a pleasant surprise: our four-view model generates eight synchronized views without any retraining, and cross-view consistency improves rather than degrades.

All results are in the revised PDF, marked in blue. We hope our responses address your concerns. If you have any further questions or suggestions, please let us know. We would be very happy to provide additional clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.