- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces Swivel, a framework for sparse 6-DoF motion control in image-to-video generation and inpainting-based video editing. Its annotation pipeline uses pretrained models to recover scene geometry, cameras, human mesh trajectories, and generic 3D point tracks from real videos. It identifies local rigid anchors, estimates their translation and rotation, and filters unreliable estimates. These trajectories are rendered as colored cube videos and used to condition a Wan 2.2-based video diffusion model.
The resulting interface allows users to specify the position and orientation of individual subjects or their parts. The authors evaluate the method on held-out real videos through quantitative baseline comparisons, qualitative demonstrations, a human preference study, and an ablation of cube selection and density.
Critical strengths
-
Automatic 6-DoF supervision from real footage. Automatically extracting translation and rotation supervision from ordinary videos is a practical contribution, reducing reliance on synthetic motion data or specialized capture setups.
-
Sparse, explicit control over local parts. Users can specify the position and orientation of individual subjects or their parts across people, animals, and objects. The interface supports both generation and editing.
-
Favorable reported results. The method outperforms the evaluated baselines across the reported quantitative metrics, and the user study shows substantial preferences for its motion following, subject consistency, and overall quality.
-
Qualitative evidence of responsiveness to motion edits. Different controls applied to the same scene and prompt produce corresponding changes in motion, including reversed windmill rotation and altered clock-hand motion. These examples support the model’s responsiveness to user-authored controls. [Figure 5.]
Critical weaknesses
1. Limited isolation of the proposed method’s contribution
This is my main concern. The experiments show favorable performance of the complete system, but do not clearly isolate the proposed conditioning’s contribution. Swivel uses Wan 2.2 14B, VACE uses Wan2.1-VACE-14B, and the evaluated DaS and 3DTrajMaster implementations use CogVideoX-5B. Differences in backbone and training setup make it difficult to attribute the gains to the proposed components. The only ablation varies control selection and density, without testing the contribution of orientation information under otherwise matched conditions. [Section 4; Table 3; Appendix D.]
A position-only versus position-and-orientation comparison using the same checkpoint, data, and training procedure would help establish the incremental benefit of orientation within the system. This would not fully explain the gains over other methods, but would strengthen the evidence for this component. The reported system-level improvements and potential value of the annotation pipeline remain meaningful.
2. Limited direct assessment of rotation fidelity
Quantitative motion-adherence metrics assess projected anchor positions, without directly measuring orientation. Qualitative examples and the user study support rotation following, but its precision and reliability are less directly characterized. A focused evaluation of orientation accuracy, through a reliable rotation-specific measure or targeted human assessment, would strengthen the evidence. [Section 4; Figures 5-7; Appendices D and E.]
3. Limited comparative coverage leaves broader competitiveness uncertain
The quantitative comparison covers VACE and DaS for image-to-video generation and only VACE for inpainting; 3DTrajMaster appears in the qualitative evaluation and user study. These are relevant comparisons, but provide limited coverage. [Table 2; Section 4.]
Although the examples demonstrate useful rotation control, their visual quality and motion range are not consistently compelling to me. Some motions are modest, leaving competitiveness with strong 2D motion-control methods uncertain. This is a preliminary impression, not an established ranking.
A broader quantitative comparison on motions all methods can represent would better assess whether explicit orientation control comes with competitive visual quality and general motion adherence.
Minor Weaknesses and Reporting Issues
-
The related-work discussion should also cover [1], which uses a flexible reference-video representation for both 2D object manipulation and camera control. For camera control, it reprojects an estimated 3D point cloud along prescribed trajectories, connecting geometric motion specifications to image-space guidance
-
Presentation and clarity. The method section could be clearer, particularly regarding the motivation for the conditioning representation and its relationship to the pretrained model. The experimental narrative could better explain the distinct roles of annotation evaluation, positional adherence, and rotation following.
-
Data provenance is insufficiently documented. The paper describes curation, corpus size, and test-set composition, but does not identify the source datasets or sufficiently explain the training/test split. These details are needed to reproduce the pipeline and interpret the evaluation.
[1] Singer et al., Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising, ICLR 2026.
Questions for authors
-
Contribution of orientation conditioning: How does Swivel compare with a position-only variant trained using the same starting checkpoint, data, and training procedure?
-
Rotation fidelity: Can the authors evaluate orientation following separately from translation by varying the requested rotation direction, angle, or axis while keeping the initial image, prompt, and a suitable single anchor’s center trajectory fixed? A focused rigid-object subset, assessed with a reliable rotation-specific measure or targeted human judgments, could make this evaluation more diagnostic.
-
Broader comparison: Could the authors include an additional strong motion-control baseline on shared tasks, particularly to assess visual quality and motion adherence beyond the current comparisons?
-
Data sources and separation: Which public datasets supply the training and evaluation videos, and how is the held-out split constructed relative to the original source videos?
7 comments
Rebuttal by Authors [2/4]
Extended baselines. The reviewer points out that the baselines use different base models and that some of them are weaker than ours. We agree. It is infeasible for us to retrain every baseline method on our base model, so we add Wan-Move [1] and TTM [2], two recent open-source motion-control baselines built on stronger base models than CogVideoX-5B. TTM was also pointed out by the reviewer. We run TTM on Wan 2.2 14B, which Swivel also builds on, and Wan-Move is built on Wan 2.1 14B. We evaluate all methods on the held-out evaluation set of Table 2 and measure rotation error by running our annotation pipeline on the generated videos. The full results are in the table in our response to W3. Swivel has the lowest rotation error and the best visual quality, including against TTM, which builds on the same Wan 2.2 14B model.
[1] Chu et al., Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance, NeurIPS 2025.
[2] Singer et al., Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising, ICLR 2026.
We will add both comparisons to the revised draft.
W2: Limited direct assessment of rotation fidelity
Quantitative motion-adherence metrics assess projected anchor positions, without directly measuring orientation. Qualitative examples and the user study support rotation following, but its precision and reliability are less directly characterized. A focused evaluation of orientation accuracy, through a reliable rotation-specific measure or targeted human assessment, would strengthen the evidence. [Section 4; Figures 5-7; Appendices D and E.]
Q2: Rotation fidelity: Can the authors evaluate orientation following separately from translation by varying the requested rotation direction, angle, or axis while keeping the initial image, prompt, and a suitable single anchor’s center trajectory fixed? A focused rigid-object subset, assessed with a reliable rotation-specific measure or targeted human judgments, could make this evaluation more diagnostic.
We thank the reviewer for this suggestion. We agree that rotation fidelity should be assessed directly, and we answer in two parts below.
Rotation accuracy. Rotation can be measured directly with our own annotation pipeline. On the held-out evaluation set, we run the pipeline on both the real source videos and the videos generated by each method, and compare the recovered rotations. We include all baselines, together with the two added baselines Wan-Move [1] and TTM [2]. Rotation errors over the evaluation set are reported below. Swivel has the lowest rotation error of all methods.
| Swivel (ours) | VACE | DaS | Wan-Move | TTM | |
|---|---|---|---|---|---|
| Rotation error (°) ↓ | 13.2 | 36.2 | 23.6 | 16.7 | 32.9 |
[1] Chu et al., Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance, NeurIPS 2025.
[2] Singer et al., Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising, ICLR 2026.
Rotation with position held fixed. In real videos, rotation and position usually change together, so changing only the rotation of a real clip often gives an implausible motion, and videos in which a subject turns while its position stays fixed are hard to find. To run the test the reviewer suggests, we therefore build our own cases. We generate 10 images with a commercial image model and use them as the first frames of our test samples. In each sample, we keep the cube at the same position and change only its rotation. We run our method and all I2V baselines on these 10 cases. Each baseline receives the same motion in its own control format. Wan-Move receives point tracks sampled on the cube and moved with it. TTM receives the subject cut out of the first frame and moved with the cube. DaS receives 3D points of the subject moved rigidly with the cube. VACE receives the 2D box of the subject. We check whether the generated subject follows the requested rotation. We report the number of correct cases for each method. A video counts as correct only if the subject turns in the requested direction by about the requested angle. A video counts as a failure if the subject does not turn, turns too little or too much, or shows hallucinated motion. We judge each video by visual inspection, because whether the subject follows the requested rotation is clearly visible in these cases. Every video is shown on the page linked below together with its verdict, so each case can be checked directly.
| Swivel (ours) | VACE | DaS | Wan-Move | TTM | |
|---|---|---|---|---|---|
| Follows the requested rotation ↑ | 10/10 | 1/10 | 2/10 | 5/10 | 5/10 |
Swivel follows the requested rotation in all 10 cases. All generated videos are shown at this anonymous link.
We will add both evaluations to the revised draft.
Rebuttal by Authors [3/4]
W3: Limited comparative coverage leaves broader competitiveness uncertain
The quantitative comparison covers VACE and DaS for image-to-video generation and only VACE for inpainting; 3DTrajMaster appears in the qualitative evaluation and user study. These are relevant comparisons, but provide limited coverage. [Table 2; Section 4.]
Although the examples demonstrate useful rotation control, their visual quality and motion range are not consistently compelling to me. Some motions are modest, leaving competitiveness with strong 2D motion-control methods uncertain. This is a preliminary impression, not an established ranking.
A broader quantitative comparison on motions all methods can represent would better assess whether explicit orientation control comes with competitive visual quality and general motion adherence.
Q3: Broader comparison: Could the authors include an additional strong motion-control baseline on shared tasks, particularly to assess visual quality and motion adherence beyond the current comparisons?
We thank the reviewer for this suggestion. We agree that a broader comparison helps, and we answer in two parts below.
Coverage of baselines and extended experiments. We add two recent open-source motion-control methods, Wan-Move [1] and TTM [2]. We evaluate all methods on the held-out evaluation set of Table 2. All methods receive the same first frame and the same prompt, and follow the motion of the same source video, so the comparison is on shared tasks. We report rotation error measured by running our annotation pipeline on the generated videos, position error, and video quality metrics.
| Rotation (°) ↓ | Position (px) ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ | FVD ↓ | |
|---|---|---|---|---|---|---|
| Swivel (ours) | 13.2 | 14.6 | 21.8 | 0.800 | 0.115 | 263.9 |
| VACE | 36.2 | 52.0 | 17.0 | 0.643 | 0.253 | 460.9 |
| DaS | 23.6 | 29.2 | 20.8 | 0.758 | 0.153 | 353.0 |
| Extended baselines | ||||||
| Wan-Move | 16.7 | 13.7 | 19.2 | 0.715 | 0.148 | 349.6 |
| TTM | 32.9 | 44.0 | 18.7 | 0.707 | 0.189 | 432.8 |
Swivel has the lowest rotation error and the best visual quality. Its position error is comparable to that of Wan-Move and much lower than that of the other baselines. Wan-Move, TTM and DaS do not support inpainting, so VACE remains the baseline in that setting. 3DTrajMaster is text-to-video only, so we compare with it qualitatively and in the user study.
[1] Chu et al., Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance, NeurIPS 2025.
[2] Singer et al., Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising, ICLR 2026.
Visual quality and motion range. To test larger motions directly, we build dedicated rotation tests, where the position is held fixed and only the rotation changes, with turns of up to 90°. We report the results of these tests in our responses to W1 and W2, comparing cubes with a position-only blob in W1 and Swivel with all baselines in W2. For visual quality, Swivel is better than Wan-Move and TTM, two strong recent 2D motion-control methods, on all four quality metrics.
We will add these comparisons to the revised draft.
Rebuttal by Authors [1/4]
Thank you for your time and effort in reviewing our paper. We appreciate that you find it valuable to scalably learn 6-DoF supervision from real-world videos, and that part-level motion control gives better precision. We address each question below.
We thank the reviewer for raising this point. We answer in two parts below.
Controlled training comparison: position-only versus position-and-orientation. We agree that the matched comparison suggested by the reviewer is more convincing. Although training a MoE video model of about 28B parameters is costly, we train two models that differ only in the control signal. One is conditioned on our cubes, which carry position and orientation. The other is conditioned on a blob at the same position, which carries no orientation. Both start from the same public checkpoint, Wan2.2-Fun-A14B-Control, which has a high-noise and a low-noise expert of 14B each and about 28B in total. Both use the same data and training settings. Each model is trained for 6,250 iterations at 240p and then 1,000 iterations at 720p. These two models are trained separately for this comparison with a shorter schedule, and are not the model in Table 2.
To test how well each model controls object rotation, we generate videos from 10 first frames created with a commercial image model. In each case, the position of the subject is held fixed and only its rotation changes. We test two versions of the control. In the first, each subject is marked by one cube or one blob. This tests the extremely sparse case of a single control signal, which our method supports. In the second, we place two smaller cubes or blobs on the left and right parts of the subject, and they move rigidly with it. With two points, even a position-only signal carries some rotation through their relative motion. For each video, we check whether the generated subject follows the requested rotation. A video counts as correct only if the subject turns in the requested direction by about the requested angle. A video counts as a failure if the subject does not turn, turns too little or too much, or shows hallucinated motion. We judge each video by visual inspection, because whether the subject follows the requested rotation is clearly visible in these cases. Every video is shown on the page linked below together with its verdict, so each case can be checked directly.
With two signals, a position-only control can already express some rotation, yet the blob model follows it in only 5 cases, while the cube model follows it in all 10. With a single signal, the blob model does not follow the rotation in any case, while the cube model still follows it in all 10. This shows that the orientation in our control signal is what lets the model follow the requested rotation. All generated videos are shown at this anonymous link.