- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper present 4Director that addresses joint camera and object motion control in image-to-video generation. It represents objects as complete canonical meshes with per-frame rigid transformations and renders the scene into depth videos to condition a pretrained video generator through a VACE-based Motion Adapter. The paper also introduces RealCOD-Rigid, containing 20774 automatically annotated clips, and Identity-Gated IoU (IG-IoU). Evaluation on 100 held-out clips, representation ablations, and a user study report improvements over four baselines in visual quality and motion control.
Critical strengths
-
The control representation is pratical and well motivated. Complete meshes and rigid transformations provide a direct way to specify object position, orientation, and occlusion alongside camera motion. The automatic annotation pipeline makes this representation usable for training at scale.
-
Consistent improvements supported by controlled ablations. IG-IoU improves from 54.8 for the strongest baseline to 60.4, alongside better visual-quality and camera-control scores. Under the same training recipe, removing surface geometry or rotation reduces IG-IoU to 50.3 or 53.0, supporting the value of the proposed representation.
Critical weaknesses
-
The rotation-control claim is not adequately quantified. IG-IoU measures silhouette overlap and identity, but can miss substantial orientation errors when silhouettes remain similar. Rotation examples are persuasive, but no direct object-orientation evaluation is reported. The Qwen-based identity gate also lacks reported validation against human judgments.
-
The reliability of the reported margins is insufficient. Quantitative evaluation uses 100 clips, and camera errors exclude cases where trajectory estimation fails without reporting per-method success rates. Consequently, methods may be compared on different subsets. Confidence intervals are also absent, making the smaller improvements difficult to assess.
Questions for authors
see weaknesses.
4 comments
Response to W2 (Reliability of the reported improvements)
We appreciate your emphasis on the reliability of the reported improvements. We address the two points in turn: camera-estimation failures and confidence intervals.
(1) Camera-estimation failures and different subsets. When the camera of a generated video cannot be reconstructed, no camera error can be computed. This affects only the camera errors: in Table 1, FID, FVD, CLIP-SIM, VBench-I2V, recognition rate and IG-IoU are computed on all 100 clips for every method, and IG-IoU scores a frame in which the object is lost as zero. For the camera errors, our protocol follows common practice in camera-controlled video generation and averages them over the successfully reconstructed videos, as do CamCo (Xu et al., arXiv 2024), CamI2V (Zheng et al., arXiv 2024) and RealCam-I2V (Li et al., ICCV 2025). We agree that the success rates should be reported, and we add two results for the 100 clips of RealCOD-Rigid-Test (original paper): the success rate of every method (Table fef3-R4) and a comparison on the same clips (Table fef3-R5).
Table fef3-R4: Camera-estimation success rate on RealCOD-Rigid-Test (original paper). A clip fails if MegaSaM pose estimation fails or the Sim(3) alignment to the target trajectory fails.
| Method | Success rate (%) ↑ |
|---|---|
| MotionCtrl | 25 |
| PaC | 57 |
| VerseCrafter | 80 |
| SymphoMotion | 8̲6̲ |
| 4Director | 92 |
Because each method's mean error covers a different set of clips, Table fef3-R5 recomputes the errors of each baseline and 4Director on the clips where both succeed. 4Director has lower rotation and translation errors against every baseline.
Table fef3-R5: Camera errors on shared clips of RealCOD-Rigid-Test (original paper). For each baseline, the baseline and 4Director are evaluated on the clips where both succeed.
| Baseline | Shared clips | Rot. (°) ↓: baseline / 4Director | Trans. ↓: baseline / 4Director |
|---|---|---|---|
| MotionCtrl | 22 | 5.05 / 2.54 | 0.232 / 0.119 |
| PaC | 53 | 7.20 / 1.86 | 0.229 / 0.094 |
| VerseCrafter | 77 | 6.17 / 3.24 | 0.171 / 0.109 |
| SymphoMotion | 82 | 3.88 / 3.68 | 0.129 / 0.109 |
(2) Confidence intervals. Table fef3-R6 adds 95% confidence intervals to Table 1 of the paper. Comparing 4Director with the next-best method on the same clips, the gains in IG-IoU (+5.6 over VerseCrafter, 95% confidence interval +3.0 to +8.2) and in translation error (on the shared clips of Table fef3-R5) are significant, whereas the smaller margins in FID, FVD, CLIP-SIM, rotation error and recognition rate lie within the confidence intervals.
Table fef3-R6: Table 1 of the paper with 95% confidence intervals on RealCOD-Rigid-Test (original paper). Camera errors are averaged over each method's successful clips (Table fef3-R4).
| Method | FID ↓ | FVD ↓ | CLIP ↑ | Rot. (°) ↓ | Trans. ↓ | Rec. ↑ | IG-IoU ↑ |
|---|---|---|---|---|---|---|---|
| MotionCtrl | 115.6 (108.0, 123.2) | 1653.3 (1456.7, 1849.9) | 27.5 (26.8, 28.2) | 4.74 (2.99, 6.80) | 0.231 (0.194, 0.268) | 32.6 (27.2, 38.4) | 18.8 (15.6, 22.2) |
| PaC | 55.7 (51.8, 59.6) | 1454.6 (1301.8, 1607.4) | 30.2 (29.6, 30.8) | 7.00 (5.60, 8.60) | 0.228 (0.205, 0.251) | 80.5 (75.6, 85.2) | 40.8 (36.9, 44.8) |
| VerseCrafter | 55.0 (51.0, 59.0) | 484.7 (417.5, 552.0) | 3̲1̲.̲7̲ (31.1, 32.2) | 6.15 (4.65, 8.21) | 0.174 (0.150, 0.199) | 9̲4̲.̲8̲ (92.6, 96.7) | 5̲4̲.̲8̲ (51.8, 57.7) |
| SymphoMotion | 4̲5̲.̲8̲ (42.6, 49.0) | 4̲0̲5̲.̲4̲ (355.4, 455.3) | 30.9 (30.4, 31.5) | 3̲.̲8̲9̲ (2.46, 6.16) | 0̲.̲1̲3̲2̲ (0.112, 0.153) | 90.2 (87.1, 93.1) | 52.9 (48.9, 56.7) |
| 4Director | 44.1 (40.8, 47.4) | 370.4 (312.8, 428.0) | 31.8 (31.3, 32.4) | 3.65 (2.05, 5.99) | 0.122 (0.099, 0.144) | 94.9 (92.8, 96.9) | 60.4 (56.6, 64.1) |
Thanks for the rebuttal. The authors have addressed part of my concerns, I would like to maintain my rating as weak accept.
Thank you very much for taking the time to review our rebuttal. We truly appreciate your thoughtful feedback and your decision to maintain your positive assessment. If you have any further questions, please let us know.
Response to W1 (Object orientation and identity gate validation)
Screenshots of the study form: https://anonymous-author-page.github.io/4Director/#rebuttal-fef3-study
We thank you for identifying these two gaps in the evaluation. We agree that IG-IoU can miss orientation errors when silhouettes remain similar. We added a direct orientation metric and a human check of the identity judge.
(1) Orientation metric. We use Orient Anything V2 (Wang et al., NeurIPS 2025) to estimate how far the object has turned about the vertical axis since the first frame. After compensating for camera motion, we compare the estimated object turn with the target turn at each of 15 later frames. We report the mean angular error and the percentage of frames with an error within 30° (Acc@30°). As a check of the estimator, its median error on rendered 180° turns with known rotations is 8.3°. The target turn differs between the two evaluations:
Table fef3-R1: Object orientation on DAVIS-NewTraj. Errors are measured against the authored rotations. Parentheses indicate 95% confidence intervals.
Table fef3-R2: Object orientation on RealCOD-Rigid-Test (original paper). Differences are measured against rotations estimated from the ground-truth videos. Parentheses indicate 95% confidence intervals.
4Director has the lowest error and highest Acc@30° in both settings.
(2) Human validation of the identity gate. We asked 10 annotators to evaluate the same 50 frames from RealCOD-Rigid-Test (original paper), with one frame per clip. For each of the five generation methods, we sampled two frames from each of five judge-score ranges.
For each frame, every annotator viewed the real and generated crops side by side and answered "same object, not broken?" (Yes/No). The target object was outlined in the real crop; method names and judge verdicts were hidden. The majority vote of the 10 annotators determined the human label for each frame. One frame received five Yes and five No votes and was excluded, leaving 49 frames for comparison.
We then compared each judge's pass/fail decisions with these human labels. The original Qwen3-VL-2B judge achieves the highest weighted accuracy, 85%, compared with 67–80% for the other tested judges (Table fef3-R3).
Table fef3-R3: Agreement with human labels on 49 frames. Accuracy uses sampling weights to represent all judged frames in RealCOD-Rigid-Test (original paper).