- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents a dense point trajectory prediction model based on video diffusion framework. Three components are proposed, i.e., grid-anchor offset encoding module, trajectory-based VAE and the trajectory-based flow matching model. Representing each point as offset from pixel-center reduces the location-dependent bias. The trajectory-based VAE learns a compact latent space and the flow matching model generate future trajectory via diffusion. A benchmark is further designed for evaluation. Extensive experiments conducted on this benchmark validate the effectiveness of the proposal in improving motion realism.
Critical strengths
- Exploring history point trajectories to improve estimation is reasonable, and the proposed formulation is valuable.
- Combining Grid-Anchor Offset Encoding with the spatiotemporal consistency regularizer in TrajLoom-VAE effectively suppresses frame-to-frame jitter and local tearing.
- The proposed unified real-world and synthetic benchmark benefits the research community in trajectory prediction.
- The paper is well-written, logically structured and easy to follow.
Critical weaknesses
- The core contribution, i.e., the grid-anchor offset encoding is essentially a standard zero-centering or cumulative displacement parameterization widely used in point tracking and trajectory forecasting literature. It cannot be packed as a primary algorithm contribution.
- The spatial consistency regularization term (Eq. 2) enforces relative motion smoothness among neighboring grid indices. However, when trajectory evolves into the future forecast window (e.g., frames 82–162), two points that were spatial neighbors on the coarse grid at Frame 1 may have diverged drastically in physical space (e.g., one moving with a foreground object to the right while the other stays on the background to the left) or even undergone topological crossing. Enforcing a heavy spatial neighbor penalty could result in over-smoothing motion boundaries.
- In real-world video dynamics, point occlusion/visibility and 2D spatial trajectories are tightly coupled (e.g., an object rotating out of view or passing behind an occluder). Nevertheless, TrajLoom-Flow models only coordinate offsets during the generative flow-matching process, relegating visibility prediction to a separate, post-hoc 1D temporal convolution head. The performance should be evaluated and discussed.
- The main experimental comparisons against WHN and Tra-MoE suffer from a severe information asymmetry. While WHN and Tra-MoE predict future motion from a single static image and text (a highly ambiguous task with large directional and speed uncertainty), TrajLoom is conditioned on 81 frames of ground-truth history trajectories. The clarification and explanation should be included.
Questions for authors
Please see the weaknesses.
We thank the reviewers for their time, constructive feedback, and recognition of the value of dense motion continuation. We have decided to withdraw this submission. We would also like to clarify several points regarding the WHN comparison, our task formulation, and the evidence supporting the paper’s contributions.
WHN comparison and prediction horizon. Our experiments evaluate WHN (L) in the single-image setting implemented by its official public release. We verified that the released training code selects the first RGB frame and that the inference pipeline accepts one conditioning image per example. Although Appendix A.2 of WHN’s published paper describes a multi-frame extension, the public release we inspected does not implement the corresponding multi-frame conditioning pipeline. Our baseline therefore follows the released configuration. In the relevant comparisons, both methods are evaluated over a common prediction horizon to accommodate WHN (L)’s supported window. TrajLoom additionally demonstrates the capability to generate 81-frame futures jointly.
Why motion continuation uses history. TrajLoom studies the continuation of motion already observed in a scene. A single image generally cannot uniquely determine the ongoing direction or speed: the same visible scene state can correspond to different motions. Video and trajectory history provide evidence about those dynamics and define the motion that the generated future should continue. History conditioning is therefore central to this task formulation. The baseline comparisons evaluate the complete systems under their stated inputs, while the component ablations assess the proposed design choices within our history-conditioned framework. These include the grid-anchor representation, consistency regularization, boundary hints, and on-policy refinement, with the full-budget on-policy ablation providing controlled evidence for the sampling refinement. Together, these components enable long-horizon dense motion continuation and downstream motion-guided video generation.
Spatial consistency and motion boundaries. The spatial term in Eq. (2) matches reconstructed differences between neighboring trajectories to their ground-truth counterparts. When initially neighboring foreground and background points subsequently diverge, that divergence is represented in the target. The grid determines which trajectory pairs are compared; it does not require those points to remain spatial neighbors throughout the forecast. The regularizer therefore encourages faithful reconstruction of relative motion, including differences across motion boundaries.
Complementary evaluation metrics. MSE and Energy Score are reported in the appendix and complement the distributional and coherence measures in the main results. FlowTV and DivCurlE serve as diagnostics of motion structure and should be interpreted alongside the prediction-error and distributional metrics. Together with the component ablations, these evaluations support the paper’s central contribution: generating long-horizon dense trajectories that continue observed scene dynamics and provide useful control for downstream video generation.