- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents UniCaMo, a framework for jointly controlling camera and object motion in image-to-video diffusion models. Given a reference image, text prompt, sparse 3D object trajectories, and camera poses, the method constructs motion-consistent initial noise to guide video generation. Specifically, it propagates foreground noise along object trajectories, warps background noise according to camera motion, and introduces spherical Gaussian noise to handle newly revealed regions. These noise components are combined using depth-aware compositing and used to initialize a LoRA-finetuned video diffusion model.
Critical strengths
1.Simple and effective framework. The method provides a unified way to control camera and object motion through structured noise initialization, without introducing additional control modules or modifying the architecture of the pretrained video diffusion model.
2.Strong experimental performance. The method achieves competitive or superior results across multiple benchmarks, particularly in joint camera and object motion control.
3.Reasonable geometric design. The combination of foreground and background noise propagation, spherical noise, and depth-aware composition is intuitive. The ablation studies demonstrate the effectiveness of the proposed components.
Critical weaknesses
The paper lacks discussion and experimental comparisons with closely related recent methods. In particular, CameraNoise (ICML 2026), which explores geometry-guided noise warping for camera control, is not cited or discussed. Moreover, although VerseCrafter (CVPR 2026) has demonstrated joint control of camera and object motion, it is not included in the experimental comparisons.
Questions for authors
1.Non-rigid motion: Can UniCaMo explicitly control articulated or non-rigid object movements through independently specified 3D point trajectories? If so, could the authors provide examples or quantitative evaluations beyond object-level 6-DoF transformations?
-
Robustness to random initialization: Since motion control is achieved through stochastic initial noise, how sensitive is the generated motion accuracy to different random seeds? Reporting the mean and variance of control errors across multiple seeds would help demonstrate the reliability of the method.
-
The proposed spherical representation provides temporally consistent noise for unseen regions, but it does not model their actual depth or geometry. Although the ablation study demonstrates its effectiveness, further evaluation under large camera translations and complex occlusions would help clarify the limitations of this approximation.
-
Trade-off between motion controllability and generation quality. The noise mixing ablation reveals a trade-off between motion accuracy and visual quality. Stronger structured noise improves trajectory fidelity but degrades FVD. This raises questions about how well the proposed noise initialization aligns with the pretrained diffusion prior and whether the trade-off becomes more severe for complex motions.
7 comments
Part 2
Q3: The sphere under large camera motion and complex occlusion
To further stress-test our model, we built Extreme-200 for these cases (Appendix E.2). A vision-language model ranked the real clips in RealCOD-25K by the extremity of their motion. Among the top 600 clips, two authors independently reviewed them and selected the 200 with the most extreme object and camera motion.
| Extreme-200 (UniCaMo / VerseCrafter) | Clips | EPE↓ | RotErr↓ | CamMC↓ |
|---|---|---|---|---|
| All | 200 | 6.68 / 10.42 | 3.58 / 18.06 | 19.27 / 45.59 |
| Camera rotation < 45° | 126 | 5.12 / 7.76 | 2.22 / 10.20 | 15.25 / 36.68 |
| Camera rotation ≥ 45° | 74 | 9.37 / 14.93 | 5.95 / 31.45 | 25.48 / 60.76 |
| Occlusion, then reappearance | 15 | 7.52 / 10.82 | 3.93 / 16.47 | 30.60 / 44.81 |
Errors increase with camera rotation for both methods, but UniCaMo remains clearly ahead in all groups. For rotations beyond 45°, UniCaMo achieves a RotErr of 5.95, compared with 31.45 for VerseCrafter. The limitations are also evident, and we state them in the paper: the sphere cannot provide the content of unseen regions, which must be synthesized by the model; the noise sampled from the sphere only conveys the camera trajectory. Because Extreme-200 clips often contain both camera and object motion, we further isolated UniCaMo's camera-control capability by evaluating the 60 RealEstate10K test clips with the most extreme camera motion: the 30 clips with the largest translation and another 30 clips with the largest rotation (Appendix E.3, Table 11).
| RealEstate10K, 60 most extreme clips | FID↓ | FVD↓ | RotErr↓ | TransErr↓ | CamMC↓ |
|---|---|---|---|---|---|
| Gen3C | 24.60 | 187.92 | 0.57 | 2.34 | 2.59 |
| CameraNoise | 32.93 | 294.66 | 8.59 | 20.10 | 25.52 |
| VerseCrafter | 27.33 | 252.75 | 2.16 | 9.08 | 9.88 |
| SymphoMotion | 27.65 | 233.24 | 2.00 | 7.21 | 8.34 |
| UniCaMo | 19.27 | 157.40 | 0.54 | 1.81 | 2.11 |
UniCaMo outperforms all previous baselines across all five metrics.
Q4: How well the proposed noise initialization aligns with the pretrained diffusion prior and does the control-quality trade-off worsen for complex motion?
According to our reply to reviewer i3S2 Q2, our structured noise is close to i.i.d Gaussian noise spatially. However, our noise is not random Gaussian noise temporally, our structured noise does contain correlation across the time channel to supply diffusion prior with motion cues. Thus, our method does not work with pretrained diffusion model that trained on i.i.d random Gaussian noise in training-free manner but required a light LoRa finetuning phase.
To quantify the severity of the trade-off under complex motion, we repeated the θ study from Table 5(a) on Extreme-200, with θ fixed for each run (Appendix E.2):
| θ on Extreme-200 | FID↓ | FVD↓ | PSNR↑ | SSIM↑ | EPE↓ | RotErr↓ | TransErr↓ | CamMC↓ |
|---|---|---|---|---|---|---|---|---|
| 0.0 | 18.81 | 229.43 | 14.89 | 0.58 | 6.43 | 3.52 | 17.25 | 19.08 |
| 0.2 | 18.43 | 222.20 | 14.84 | 0.58 | 6.68 | 3.58 | 17.46 | 19.27 |
| 0.5 | 18.46 | 220.30 | 14.42 | 0.56 | 7.85 | 4.45 | 18.64 | 21.17 |
The control-quality trade-off remains evident in this more challenging setting. Nevertheless, all settings maintain reasonable performance, with no indication of abrupt degradation or complete failure. Overall, the trade-off does not become more severe under complex motion. This experiment further confirms that if we want better motion-control then we can choose θ=0. In most of our experiment, we choose θ=0.2 to balance visual-quality and control-fidelity.
We would like to thank the reviewer again for your thoughtful review and valuable feedback. We would appreciate it if you could let us know if our responses have addressed your concerns and whether you still have any other questions about our rebuttal. We would be happy to do any follow-up discussion or address any additional comments. Thanks!
Part1
Thank you for the review, and for pointing us to the two missing baselines. Both are now cited, discussed and compared, and we ran the experiments you asked for on seeds and on large motion. All additions to the paper are in red. We have also made an anonymous website to show the requested results explicitly.
Weakness: CameraNoise and VerseCrafter
CameraNoise is the closest work in spirit: it also writes the camera into the noise. It warps the noise with a reprojection flow computed from the camera poses and a constant pseudo-depth, so it uses no scene depth and does not control object motion. We now discuss it in Section 2 and compare with it on RealEstate10K. Overall, Unicamo excell on all 5 metrics using for camera-control. We have updated Table 2 with this new result. | Method | FID↓ | FVD↓ | RotErr↓ | TransErr↓ | CamMC↓ | |---|---:|---:|---:|---:|---:| | CameraNoise | 10.89 | 109.54 | 2.22 | 10.58 | 11.49 | | UniCaMo | 5.99 | 39.93 | 0.26 | 1.89 | 1.98 |
VerseCrafter controls camera and objects jointly through an additional adapter. We compare on camera control (300 RealEstate10K clips, on our joint benchmark (Table 3) and on a new stress set with much larger motion (Extreme-200, Appendix E.2):
UniCaMo is ahead on all eight metrics in both settings, without an additional control module.
Q1: Can UniCaMo manipulates articulated and non-rigid motion ?
Yes. We can explicitly control articulated or non-rigid object movements through independently specified 3D point trajectories, please refer anonymous website and Figure 16 for qualitative results. Given single first frame images, with no reference video, we scripted the 3D tracks by hand via our GUI for non-rigid and articulated manipulation: bending a rope, folding a T-shirt, turning a page, scrubbing a plate, sawing bread, closing a drawer and opening a refrigerator, plus a cyclist taking off into the sky.
Q2: Sensitivity to the random seed
We re-scored the original outputs and evaluated four additional seed runs on the same clips (those with a foreground mask). The second row is the mean and standard deviation of the benchmark averages over these 8 runs; the first row repeats the published result of Table 3:
The results are stable across seeds: benchmark averages vary by about 3% or less on all eight metrics, and in each of the eight runs UniCaMo stays ahead of every baseline of Table 3 on all eight metrics.