- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors notice the challenge of preserving object identity while independently controlling appearance and motion in world simulation. To address it, they propose WoCA, an object-centric framework comprising persistent appearance assets, a trajectory diffusion model, and an asset-conditioned video renderer, disentangling appearance and motion. Appearance features extracted from reference images are paired with object poses, while the trajectory model predicts future 3D states, and the renderer combines global conditioning with localized spatial features. Experiments on MOVi-D and Waymo Open show improved video quality over Neural Assets across controlled rendering tasks, alongside a long-horizon generation example.
Critical strengths
-
The paper addresses an important problem in generative world simulation: preserving object identity while consistently associating appearance and motion with the correct objects. Reliable control over these factors is valuable for scenario editing and exploring alternative futures, especially in rare or difficult-to-capture settings where synthetic data could broaden training coverage and help improve the robustness of downstream models.
-
The proposed method is rational and well-motivated. Maintaining reusable appearance assets independently of evolving object states supports appearance replacement, trajectory editing, and recursive prediction with a shared visual reference, and Global tokens provide access to object appearances and poses, while localized spatial features associate this information with the intended image regions in a complementary manner.
-
The evaluation is comprehensive, covering original generation, appearance swapping, and trajectory editing on both synthetic object-interaction and real-world driving data
Critical weaknesses
-
Separating appearance and motion at the input level does not guarantee faithful control in the generated video. Training primarily relies on reconstruction of original videos, without explicit supervision enforcing appearance preservation under edited trajectories.
-
The evaluation emphasizes video quality rather than editing fidelity. The reported metrics do not directly establish whether appearances are assigned to the correct objects, trajectories are accurately followed, or unedited content remains unchanged. The reference videos used for PSNR, SSIM, and LPIPS under Swap/Edit also require clarification.
-
The main comparison is largely limited to Neural Assets, implemented with the video backbone’s temporal components removed. Some improvements may therefore reflect the benefits of temporal modeling rather than the proposed conditioning design specifically.
-
The framework depends on substantial geometric annotation, including track identities, object boxes, and calibrated cameras; motion prediction additionally requires initial 3D object states. It is better to show how robust it is to the noise and how to deal with the error
Questions for authors
The computational efficiency of the full simulation pipeline remains unclear. The trajectory model uses 50 sampling steps for each four-frame continuation, while video rendering uses 40 sampling steps per window. Although these sampling configurations are provided, end-to-end latency, throughput, and inference memory usage are not reported.
Response to Reviewer vTUu
Thank you for your valuable feedback! Below, we address your concerns point by point.
Our method learns appearance reuse across poses through source-to-target reconstruction: reference appearance is paired with target-frame geometry, while foreground masking prevents the conditioning image from directly supplying the target objects’ appearance. This formulation explicitly enforces appearance preservation under edited trajectories, which is one of the core contributions of our method. We have also empirically verified the resulting control on edited outputs in new Table 1 and new Figure 3 in the updated manuscript.
The reference for PSNR, SSIM and LPIPS is the original GT video, since counterfactual GT videos are unavailable. These metrics characterize full-reference visual fidelity, reflecting both preservation of unchanged scene content and differences introduced by intentional edits, rather than directly measuring editing correctness. Below, we complement these results with explicit identity-preservation and spatial-control evaluations.
We have added identity preservation and spatial control evaluations in new Figure 3 in the updated manuscript. The results show that WoCA achieves the highest identity similarity and box Recall@ 0.5 on both datasets. WoCA also achieves the highest box mIoU on MOVi-D; on Waymo Open, its mIoU is slightly below the new VACE baseline, while providing higher identity similarity and recall. These results support improved identity preservation and object placement across both synthetic and real-world scenes.
We have also conducted an additional ablation study to isolate the effect of video editing by evaluating unedited and edited outputs against the same edited target boxes in new Table 1 in the updated manuscript. Applying the edit increases box mIoU from 0.2975 to 0.5962 and Recall@ 0.5 from 14.93% to 82.02%, demonstrating that the generated objects follow the requested changes. Meanwhile, identity similarity remains close to that of unedited reconstruction evaluated under original targets (0.5854 vs. 0.6158).
Together, these results support WoCA's ability to modify object motion while largely retaining source identity, without requiring counterfactual ground-truth videos for evaluation.
As suggested by Reviewer 6NGh, We have added a video-editing baseline VACE in updated Figure 2 and new Figure 3 in the updated manuscript. For fair comparison, we finetune the official VACE checkpoint with LoRA and construct coarse RGB conditioning videos by placing reference-object crops at the prescribed projected boxes over the initial background. These videos are supplemented with source/donor references and a fixed prompt per task; no future ground-truth RGB frames are provided at inference. The results show that VACE underperforms WoCA in terms video quality, identity preservation and spatial control.
Our approach follows the problem formulation from Neural Assets, which does require geometric inputs, but their source does not necessarily need to be human annotation: estimated tracks, 3D boxes, and camera parameters can in principle use the same conditioning interface after consistent coordinate alignment. Our contribution establishes controllable simulation from these inputs, making integration with automatic perception a natural extension. As already acknowledged in the limitations section in our manuscript, reducing annotation dependence remains an important future work direction.
We have profiled controlled video rendering on Waymo Open using a single NVIDIA H20 GPU under matched generation settings in new Table 2 in the updated manuscript. The results show that our method only has modest increase in latency and peak memory compared to the native Wan2.2 backbone.
We hope that we have adequately addressed your concerns. Please let us know if you have any further questions.