- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies video generation from a single input image with joint control over camera motion, object motion, and weather. Built on a frozen video generation backbone, the method uses a World Adapter to incorporate rendered scene geometry and object bounding-box trajectories, and a State Adapter to introduce weather conditions. Scene–Weather CFG separately weights scene and weather guidance. Training combines real videos, paired simulated weather data, and weather pairs produced through video editing. The intended contribution is to change weather while preserving the underlying scene and prescribed motion.
Critical strengths
The problem is meaningful and practically relevant. Jointly controlling camera motion, object motion, and weather addresses a useful capability in controllable video generation. Preserving scene structure and object consistency under substantial appearance changes is a worthwhile research objective.
The paper presents a coherent system and data construction pipeline. The integration of geometric conditions, object trajectories, and weather prompts is clearly motivated. The reported results provide initial evidence of feasibility, while the ablations offer some support for the geometric inputs and guidance strategy.
Critical weaknesses
The comparisons do not sufficiently establish the benefit of the proposed joint architecture over representative existing systems and straightforward compositions. Recent systems such as WorldPlay, LingBot-World V2, Matrix-Game 3.0, and Cosmos 3 are relevant to camera-conditioned or interactive world generation, while Wan-Move is relevant to multi-object motion control. WorldPlay and Wan-Move are cited but not evaluated. More importantly, two natural alternatives are missing: editing the input image into the target weather before controllable video generation, and applying weather editing after controllable video generation. Both can operate from the same initial image and user instructions without additional observed future frames. Without these comparisons, it remains unclear how much the dual-adapter architecture and joint training improve quality, control accuracy, or computational cost over compositions of existing capabilities.
The evaluation provides limited evidence for preserving joint control under weather changes, particularly in multi-object settings. Weather quality and structural preservation are largely assessed through separate metrics and subsets. Appendix Table 7 does report control errors on the Weather subset, but lacks corresponding baseline results and matched guidance-strength comparisons. The background-preservation metrics in Table 5 are evaluated on the Real subset and therefore do not establish whether stronger weather guidance compromises structure or motion control on weather-conditioned samples. Multi-object robustness also remains insufficiently characterized. The paper does not adequately report object-count distributions or performance under same-category crossings, occlusion, and reappearance. Bounding boxes specify spatial constraints, but stable identity-to-trajectory binding requires further evidence. ObjMC, based on trajectory-center distances and Hungarian matching, does not by itself establish appearance-identity consistency.
Finally, the background evaluation mask described in the appendix depends on both the reference and generated outputs. This appears to allow some erroneous generated regions to be excluded from evaluation. The implementation should be clarified, and the conclusions checked using a valid region determined solely from reference information.
Questions for authors
Baseline selection: What criteria guided the selection of baselines? Could the authors provide representative comparisons with recent camera/world-control and object-motion methods, such as those discussed above, on their shared capabilities? Please clarify differences in conditioning information, model budgets, and practical availability.
Compositional alternatives: How does the proposed method compare with (a) first-frame weather editing followed by controllable generation and (b) controllable generation followed by video weather editing? Comparisons should jointly assess weather quality, scene and object preservation, control accuracy, and inference cost. For the first alternative, any downstream RGB conditions should be made consistent with the edited appearance.
Weather–control trade-off and metric validity: On the same Weather samples, how do camera accuracy, object accuracy, and scene preservation change when weather control is disabled or its guidance strength is varied? Does the background evaluation mask depend on generated pixels in the implementation, how are empty masks handled, and do the conclusions hold with a reference-only mask?
Multi-object control: How are object identities encoded and associated with trajectories, and how are overlapping boxes represented? Could the authors report object-count statistics and results stratified by object count, together with targeted evaluation of crossings, occlusion, and reappearance? How does the evaluation detect identity swaps, trajectory reassignment, or disappearing objects?
Synthetic pair quality: Which editing models and versions were used to construct the weather pairs, and in what proportions? Please clarify the filtering procedure, rejection rate, and residual-error assessment. Is there a quality audit or data ablation that measures the effect of geometric changes, object distortions, and temporal artifacts in the edited targets?