- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MVTrack4Gen improves dynamic novel-view video generation through auxiliary multi-view point-tracking supervision. The paper analyzes temporal and cross-view correspondences in diffusion attention features, then attaches a tracking head to a selected intermediate layer and directly supervises attention correspondences. The framework also incorporates richer camera conditioning. Experiments on ReCamMaster and Redirector show improved geometric consistency across DAVIS, iPhone, and Neural3D, supported by component and layer ablations.
Critical strengths
-
The method offers a practical way to improve geometric consistency without requiring explicit reconstruction during generation. On DAVIS, MEt3R decreases from 0.337 to 0.274 for ReCamMaster and from 0.318 to 0.267 for Redirector. Improvements across two backbones and additional real-world benchmarks make the contribution more convincing than a single-model demonstration.
-
The attention analysis leads to a testable design choice. The component ablations in Table 3 support the usefulness of both tracking and correspondence supervision, while Table 4 supports selecting the correspondence-rich intermediate layer. The tracking head can also be omitted during generation, making the auxiliary training approach practically attractive.
Critical weaknesses
-
The specific benefit of multi-view supervision over temporal tracking supervision remains insufficiently isolated. Track4Gen already establishes the usefulness of tracking supervision for video generation, as acknowledged in the related work. Table 3 separates the tracking and correspondence objectives, but neither comparison isolates within-view supervision from cross-view supervision. A matched temporal-only baseline would better establish the incremental contribution.
-
The central claim of preserving reference motion is evaluated mostly indirectly. MEt3Rdynamic measures geometric consistency in dynamic regions, while VBench smoothness and consistency do not directly measure whether the generated subject follows the reference motion. Table 6 evaluates tracking capability rather than motion fidelity in generated videos. Consequently, the evidence for geometric improvement is stronger than the evidence for faithful motion transfer.
Questions for authors
-
Under identical camera conditioning, training data, and optimization, how does temporal-only tracking/correspondence supervision compare with the full multi-view objectives? This would clarify the contribution beyond existing tracking-supervised generation.
-
Can you report reference-motion fidelity using independently evaluated trajectories or synchronized ground-truth target tracks, including occlusions and large viewpoint changes? This would directly test the paper’s main motion-preservation claim.
4 comments
Regarding W2 and Q2 — Directly evaluate motion fidelity between the reference and generated videos.
We agree that geometric consistency, perceptual video quality, and tracking capability alone provide incomplete evidence of motion fidelity between the reference and generated videos. Following your suggestion, we have added Motion, the motion-consistency metric proposed in IC-World [1], to directly compare trajectories reconstructed from the two videos. We also report Motion_dynamic, which restricts the evaluation to foreground regions defined by the DAVIS ground-truth masks.
Specifically, we adapt the IC-World evaluator using SpatialTrackerV2, to reconstruct point trajectories from both reference and generated videos. We align the trajectories using the estimated relative camera pose, match tracks by their temporal-mean 3D positions, and compute the motion-consistency score from their distances at corresponding timestamps. This evaluates whether the generated motion agrees with the reference motion using an external trajectory estimator. The full protocol is provided in Appendix E.4.
DAVIS — Table 1
| Method | MEt3R ↓ | MEt3R_dynamic ↓ | Motion ↑ | Motion_dynamic ↑ | mRotErr (°) ↓ | mTransErr ↓ | mCamMC ↓ |
|---|---|---|---|---|---|---|---|
| GEN3C | 0.290 | 0.328 | 0.434 | 0.431 | 2.538 | 0.127 | 0.163 |
| TrajectoryCrafter | 0.291 | 0.306 | 0.407 | 0.405 | 10.126 | 0.190 | 0.355 |
| CogNVS | 0.333 | 0.346 | 0.404 | 0.398 | 10.439 | 0.228 | 0.400 |
| NeoVerse | 0.302 | 0.323 | 0.426 | 0.424 | 4.705 | 0.159 | 0.228 |
| ReCamMaster | 0.337 | 0.369 | 0.405 | 0.401 | 3.660 | 0.113 | 0.169 |
| Redirector | 0.318 | 0.395 | 0.438 | 0.431 | 1.714 | 0.086 | 0.109 |
| Camera Anything | 0.276 | 0.320 | 0.460 | 0.455 | 2.577 | 0.115 | 0.152 |
| MVTrack4Gen (ReCamMaster) | 0.274 | 0.287 | 0.469 | 0.466 | 1.858 | 0.100 | 0.125 |
| MVTrack4Gen (Redirector) | 0.267 | 0.349 | 0.481 | 0.476 | 1.718 | 0.073 | 0.097 |
On DAVIS, both variants of our method outperform the compared baselines in overall and dynamic-region motion consistency, including depth-warping and reconstruction-based methods. These results support the effectiveness of our supervision in preserving reference motion under viewpoint changes.
iPhone — Table 2
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ | MEt3R ↓ | Motion ↑ |
|---|---|---|---|---|---|
| GEN3C | 10.419 | 0.270 | 0.699 | 0.382 | 0.736 |
| TrajectoryCrafter | 10.271 | 0.276 | 0.766 | 0.379 | 0.718 |
| CogNVS | 10.105 | 0.280 | 0.774 | 0.406 | 0.681 |
| NeoVerse | 11.886 | 0.394 | 0.579 | 0.493 | 0.785 |
| Redirector | 11.447 | 0.329 | 0.720 | 0.539 | 0.780 |
| Camera Anything | 10.455 | 0.287 | 0.663 | 0.425 | 0.731 |
| MVTrack4Gen (ReCamMaster) | 11.521 | 0.340 | 0.640 | 0.325 | 0.800 |
| MVTrack4Gen (Redirector) | 11.969 | 0.338 | 0.633 | 0.347 | 0.788 |
On iPhone, both variants achieve better motion consistency than the compared baselines, including methods that explicitly reconstruct and warp scene geometry. Our Redirector-based model also improves visual quality and geometric consistency over the original backbone.
Neural3D — Table 2
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ | MEt3R ↓ | Motion ↑ |
|---|---|---|---|---|---|
| GEN3C | 16.072 | 0.519 | 0.466 | 0.222 | 0.864 |
| TrajectoryCrafter | 16.380 | 0.523 | 0.496 | 0.257 | 0.865 |
| CogNVS | 16.466 | 0.514 | 0.470 | 0.243 | 0.817 |
| NeoVerse | 15.268 | 0.489 | 0.511 | 0.260 | 0.816 |
| Redirector | 16.170 | 0.511 | 0.462 | 0.213 | 0.870 |
| Camera Anything | 14.307 | 0.473 | 0.517 | 0.244 | 0.854 |
| MVTrack4Gen (ReCamMaster) | 16.078 | 0.500 | 0.461 | 0.210 | 0.875 |
| MVTrack4Gen (Redirector) | 16.694 | 0.527 | 0.447 | 0.199 | 0.878 |
On Neural3D, our Redirector-based model achieves the best results across the reported visual-quality, geometric-consistency, and motion-consistency metrics. Both variants outperform the compared baselines in motion consistency, providing further evidence of improved reference-motion fidelity across datasets.
MultiCamVideo architecture ablation — Table 4
| Configuration | PSNR ↑ | SSIM ↑ | LPIPS ↓ | MEt3R ↓ | Motion ↑ | mRotErr (°) ↓ | mTransErr ↓ | mCamMC ↓ |
|---|---|---|---|---|---|---|---|---|
| ReCamMaster | 13.23 | 0.532 | 0.537 | 0.405 | 0.632 | 5.89 | 20.23 | 24.31 |
| (i) Plücker camera encoding | 13.69 | 0.546 | 0.505 | 0.391 | 0.649 | 4.92 | 20.50 | 23.01 |
| (ii) (i) + Correspondence loss | 14.93 | 0.554 | 0.472 | 0.386 | 0.698 | 4.57 | 18.67 | 21.88 |
| (iii) (i) + Tracking module | 14.49 | 0.571 | 0.456 | 0.378 | 0.713 | 3.47 | 17.86 | 19.43 |
| (iv) (i) + Tracking module + Correspondence loss | 15.05 | 0.571 | 0.465 | 0.363 | 0.719 | 3.90 | 17.48 | 19.31 |
The MultiCamVideo ablation shows that adding either the tracking module or the correspondence loss improves motion consistency over the same camera-encoding baseline. Combining both objectives achieves the strongest geometric and motion consistency, supporting their complementary contributions. Qualitative comparisons in Figure 16 and the supplementary videos further illustrate how our method preserves reference motion across viewpoints.
Evaluation under large camera motions on DAVIS
To assess motion fidelity under large viewpoint changes, we report results on the 125 pairs of DAVIS evaluation samples with the largest camera-motion magnitudes, selected from our existing evaluation set.
| Method | Motion ↑ | Motion_dynamic ↑ |
|---|---|---|
| GEN3C | 0.305 | 0.301 |
| TrajectoryCrafter | 0.275 | 0.273 |
| CogNVS | 0.293 | 0.285 |
| NeoVerse | 0.298 | 0.296 |
| ReCamMaster | 0.274 | 0.272 |
| Redirector | 0.305 | 0.296 |
| Camera Anything | 0.328 | 0.323 |
| MVTrack4Gen (ReCamMaster) | 0.334 | 0.332 |
| MVTrack4Gen (Redirector) | 0.340 | 0.333 |
Both variants of our method outperform all compared baselines in overall and dynamic-region motion consistency on this subset, with our Redirector-based variant achieving the best results on both metrics. These results show that our method maintains its advantage in preserving reference motion even under large camera motions.
Evaluation under occlusion
Quantitative evaluation under occlusion is challenging on real-world datasets because accurately identifying occlusion intervals and obtaining reliable corresponding trajectories across viewpoints is difficult at scale. To investigate this setting, we additionally evaluate our method on synthetic scenes with known object motion and occluder geometry. Specifically, we render a falling ball that becomes temporarily occluded in the reference view and examine whether the generated target video maintains consistent object motion through the occluded interval. As shown in Fig. 17, our method maintains a coherent ball trajectory in the target view even when the ball is temporarily occluded in the reference view, whereas several baselines exhibit missing or misplaced objects and inconsistent motion. These results provide qualitative evidence that our method better preserves motion consistency under reference-view occlusion.
[1] Wu, Fan, et al. “IC-World: In-Context Generation for Shared World Modeling.” European Conference on Computer Vision, 2026.
Dear Reviewer 4KMa,
Thank you for your thoughtful feedback. Your suggestions led us to compare single-view and multi-view supervision and to evaluate motion consistency across viewpoints. These additions strengthen the evidence that our method preserves scene geometry and motion across views.
If you have any further concerns, we would welcome the opportunity to discuss them further. Thank you again for your time and consideration.
Best regards, The Authors
Thank you for your careful review and constructive suggestions. We appreciate your recognition of the improvements across backbones and datasets, the connection between our attention analysis and design choices, and the practical benefit of omitting the tracking head during generation. Below, we address your concerns through additional ablations and independent trajectory-based motion evaluation. The corresponding revisions are highlighted in blue in the revised manuscript.
Regarding W1 and Q1:
We acknowledge that the original manuscript did not provide sufficiently detailed ablation studies to clearly isolate the contributions of our design choices. To address this, we have added an ablation study on DAVIS comparing reference-view-only, target-view-only, and joint reference–target supervision, as reported in Table 3. All configurations use the same ReCamMaster backbone and Plücker camera encoding and are trained for 5K iterations under identical settings. In the single-view variants, both tracking supervision and the correspondence loss are restricted to temporal correspondences within the respective view.
Target-view-only supervision improves geometric and motion consistency, and camera accuracy over the camera-encoding baseline, confirming the benefits of single-view supervision. However, joint reference–target supervision further improves all metrics across these three evaluation axes, while maintaining comparable visual quality. These results demonstrate the additional benefits of explicitly supervising correspondences across viewpoints. Single-view supervision in target view encourages temporally consistent trajectories within the generated video, but does not explicitly align them with the reference video. Joint supervision connects the same scene points across time and viewpoints, helping preserve the reference scene’s geometry and object motion under the target camera trajectory. We have clarified this distinction in Section 5.4 and included qualitative comparisons in Figures 7 and 20.