- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies activation interventions for motion control in video diffusion transformers. A BallDiT model trained on synthetic projectile videos provides a controlled setting with pixel-level trajectory measurements. With model weights frozen, an additive activation field is optimized to match counterfactual target latents, typically using 24 fitting clips and 100 held-out evaluation clips. The paper compares shared and token-wise interventions, studies their decomposition and rank, and uses attention-head replacement and pinning to examine intervention dependence. Extensions include inverted synthetic clips in Wan2.1 and natural-prompt facing and color experiments.
Critical strengths
- The controlled experimental design is careful. Matched-noise counterfactuals, pixel-based motion measurements, held-out evaluation, image-quality tracking, and norm/objective/optimization controls help distinguish changes in the generated trajectory from changes in a probe readout. The main intervention is fitted jointly and evaluated on unseen fitting-set-excluded clips, rather than optimized independently for each test video.
- The analysis extends beyond intervention success to decomposition, rank, and attention-head replacement/pinning with matched random-head controls. The paper also reports important negative results, including position sensitivity, poor cross-pair transfer, and successful shared interventions when allowed to vary across layers and denoising steps. This transparency is valuable.
Critical weaknesses
- The central shared-vector versus token-wise comparison does not sufficiently separate intervention structure from capacity and parameter allocation. A shared 384-dimensional vector is compared with a 576-by-384 field with substantially greater flexibility. Its greater effectiveness is therefore not, by itself, a surprising mechanistic finding. Norm matching controls perturbation magnitude, not parameter count; decomposing an optimized full field does not independently resolve this issue either. I appreciate the rank and optimization studies, but low-rank fields still introduce learnable token-specific coefficients. More direct comparisons at similar parameter and optimization budgets are needed to establish what is specifically gained from token-wise structure. Moreover, the layer- and step-dependent shared intervention achieves approximately 86 degrees of rotation, restricting the negative result to the fixed-site intervention family. The optimized field establishes a way to control the model, but does not itself identify its endogenous motion representation, as the paper acknowledges.
- The evidence is concentrated in an overly restricted motion setting relative to the broader question of motion mechanisms in video diffusion models. BallDiT is trained on many videos, so this is not a single-video memorization objection; nevertheless, the core setting uses simple projectile motion, a clean background, and explicit low-dimensional motion conditioning. Held-out clips establish transfer within this setting, not across complex motion distributions. Using Wan does not fully address this limitation because the main quantitative motion experiments still use synthetic projectile clips. Natural-prompt facing and color changes do not establish that the findings hold for articulated dynamics, occlusion, interacting objects, or coupled camera and object motion. The reported position dependence and frequent cross-pair transfer failures further limit the generality. The large number of controlled experiments does not yet translate into sufficiently broad or non-obvious insight about real-world video motion.
Questions for authors
- Can you compare token-wise interventions against alternatives that allocate a similar parameter budget across layers and/or denoising steps, with comparable optimization effort and intervention magnitude? What evidence would distinguish a benefit of token-wise structure from capacity or optimization flexibility?
- Can the central structural and head-dependence findings be tested on at least one genuinely dynamic natural-video setting beyond projectile motion, facing direction, or color? A focused test involving articulated motion, occlusion, or simultaneous camera/object motion would be more informative than additional variants of the same toy setting; a full general-purpose motion controller is not required.
- Given the successful layer/step-dependent shared intervention and limited cross-pair transfer, what precise mechanistic conclusion generalizes beyond the chosen intervention parameterization and synthetic distribution? Please clarify the contribution that remains after these scope restrictions.