- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces XTrajectory, a framework that trains a general-purpose vision-language model to predict future 2D trajectories through its native language interface, without an additional trajectory decoder. It unifies the motion of visual points, objects, human hands, and robot end-effectors using a common coordinate-sequence representation. The authors construct a training corpus from seven sources and introduce XTrajectory-Bench, covering physical and egocentric motion prediction. The training approach combines geometrically verifiable rewards with TS-GSPO, which supplements sequence-level optimization with coordinate-segment objectives. Experiments show improvements over the base VLM, competitive transfer to existing manipulation trajectory benchmarks, and non-uniform transfer to other multimodal tasks.
Critical strengths
-
Useful integration of heterogeneous trajectory prediction tasks. The main strength of this paper is that it brings physical motion, object motion, hand motion, and robot end-effector motion into a unified prediction framework. The authors convert seven datasets into a common format, with sufficient details in Appendix A for reproduction. Since these tasks are usually studied separately, I think a shared dataset and benchmark is a useful contribution. It allows different motion types to be studied and evaluated under the same setting, especially if the authors release the processed data.
-
Consistent improvements over the base model across trajectory benchmarks. XTrajectory improves over Qwen3.5-9B on all five evaluated trajectory benchmarks, including ShareRobot and VABench, whose inputs omit the explicit motion history used during training. These results support the usefulness of trajectory post-training and demonstrate transfer beyond the training input format. The comparisons with specialized trajectory models and embodied VLMs also provide useful context for its performance.
Critical weaknesses
- The contributions of the reward design and TS-GSPO are not clearly isolated.
The reward combines coordinate accuracy, Fréchet distance, motion direction, and a bonus for reasoning longer than 300 characters. However, the paper does not ablate these components. It is unclear whether the added geometric terms outperform a simple coordinate-distance reward, or whether the reasoning length bonus actually helps. The thinking-mode experiments do not answer the latter question.
- The evaluation metrics and baseline comparisons need clarification.
Under the RMSE definition in Eq. 5, discrete Fréchet distance should satisfy for trajectories of equal length evaluated on the same points. This bound also holds when averaging over the same samples. Appendices B.1 and B.3 specify five future points for Physics-IQ and xHOT3D. However, the quadratic baseline in Table 1 gives Fréchet/RMSE ratios of 2.317 and 2.353, both exceeding . Averaging RMSE over spatial dimensions could explain this difference, but that is not the definition given in Eq. 5. The authors should clarify how these metrics are calculated and averaged.
Separately, the SFT baseline trains only on coordinates, without reasoning or semantic labels. The RL method uses a different output format and training setup. This makes it difficult to attribute the reported capability retention to RL itself. Given the near-zero SFT scores on some multimodal benchmarks, it would also help to distinguish formatting failures from incorrect predictions.
Questions for authors
-
Metric consistency: How exactly are RMSE and Fréchet distance calculated in Table 1? Why do the quadratic baseline's Fréchet/RMSE ratios exceed √5 on the two five-step datasets? If this is an implementation issue, do the corrected results change the rankings?
-
Reward design: How does the full reward compare with a coordinate-distance-only reward? Does removing the reasoning-length bonus affect performance?
-
TS-GSPO: What happens when the segment loss uses the global advantage instead of segment-specific advantages? This would help isolate the benefit of local credit assignment.
-
SFT comparison: How was the SFT baseline tuned? Are its low multimodal scores caused by formatting errors or incorrect predictions? Would the conclusions hold under comparable output formats and supervision?