- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GMFlow, a multimodal trajectory prediction framework that explicitly decomposes future trajectories into a goal latent and a motion latent. The goal latent captures intermediate and final goal offsets relative to a constant-velocity reference, while the motion latent captures local trajectory variations between these goals.
A goal-preserving motion basis is designed so that the motion component cannot alter positions specified by the goal latent. The two latent components are jointly generated using conditional flow matching with factorized goal and motion queries, enabling multiple trajectory hypotheses to be generated in a single flow step. The method further uses component-aligned supervision, with endpoint-oriented supervision for the goal component and full-trajectory supervision for the motion component.
Experiments on NBA, SDD, and ETH-UCY report strong prediction accuracy and efficient one-step inference.
Critical strengths
-
Clear and interpretable goal-motion decoupling. This goal-preserving motion basis explicitly prevents motion latent variables from altering the selected goal location; compared to merely partitioning latent representations into two sets of features, this approach achieves stronger semantic separation.
-
Efficient multimodal generation. GMFlow generates multiple trajectory candidates with just a single forward pass of the prediction network, resulting in inference costs far lower than those of alternatives based on iterative diffusion or flow.
Critical weaknesses
-
The conceptual innovation relative to existing goal-conditioned trajectory prediction methods is limited. The core idea of decomposing trajectory uncertainty into "where to go" (goal) and "how to move" (motion) is closely related to a vast body of existing research—including methods conditioned on goals, endpoints, waypoints, and "global-to-local" cues (e.g., PECNet, Y-Net, Graph-TERN, G2LTraj, and related goal-driven modeling schemes). While the formulations of a "goal-preserving basis" and "joint latent flow" are technically distinctive, I view them more as refinements of established modeling principles rather than a fundamentally new prediction paradigm.
-
The ablation studies lack sufficiently robust baselines for goal-conditioned prediction under equivalent conditions. The "w/o Goal-Motion Decomposition" experiment replaces the proposed representation with a unified residual latent variable, which indeed demonstrates the effectiveness of structured decomposition. However, this experiment does not prove superiority over traditional endpoint- or waypoint-conditioned predictors—provided the latter utilize identical history encoders, model capacities, numbers of candidate trajectories, and training strategies. Conducting such controlled comparative experiments would more directly substantiate the claimed advantages of the representation.
-
Experimental validation fails to adequately support the stated application motivation of "autonomous driving." Although autonomous driving is cited as a key application scenario, experiments are limited to the NBA SportVU, SDD, and ETH-UCY datasets. These benchmarks primarily involve sports or pedestrian trajectories and lack critical structural information specific to autonomous driving, such as lane topology, high-definition maps, traffic rules, and interactions among heterogeneous road users. Evaluating the method on modern autonomous driving motion prediction benchmarks would provide stronger evidence of the proposed decomposition method's value regarding generalizability.
-
The evaluation process fails to sufficiently verify the claimed "structured multimodal" characteristics. The primary evaluation metrics are minADE and minFDE (based on the best of 20 candidate trajectories). While these metrics assess whether at least one candidate trajectory approximates the ground truth, they fail to confirm that "goal queries" actually capture distinct destinations or that "motion queries" capture vastly different local paths for a given destination. Incorporating metrics such as diversity, mode coverage, calibration, or direct disentanglement analysis would provide more compelling evidence for this factorized multimodal representation.
Questions for authors
-
Could the authors provide a controlled experiment comparing the proposed method with traditional predictors based on endpoint or waypoint conditions, ensuring that both use identical encoder and decoder capacities, numbers of candidate trajectories, and training strategies?
-
Beyond specific implementation details regarding latent bases and flow matching, how should the core conceptual contributions of this method be distinguished from existing goal-conditioned or "global-to-local" trajectory prediction approaches?
-
Could the authors evaluate GMFlow on modern autonomous driving motion prediction benchmarks that incorporate lane/map structures and heterogeneous road agents?
-
Could the authors directly demonstrate the respective semantic roles of the factorized queries—for instance, by fixing the goal query while varying the motion query, and vice versa?
4 comments
Response to Q3–Q4
Q3. Evaluation on autonomous-driving motion prediction benchmarks.
Thank you for raising this point. We have not evaluated GMFlow on an autonomous-driving motion prediction benchmark. Our experiments on NBA, SDD, and ETH–UCY focus on predicting human trajectories from observed motion. We mention autonomous driving as a downstream application because anticipating where a nearby pedestrian may cross can help a vehicle plan when to slow down or yield. We have clarified in the introduction that our focus is human trajectory prediction.
One possible extension is to condition GMFlow on map features alongside observed trajectories. For example, crosswalk locations could inform both where a pedestrian is likely to cross and the path they may take. Evaluating GMFlow on a driving benchmark would allow us to test whether joint goal–motion generation remains effective when road geometry and interactions among pedestrians, cyclists, and vehicles influence the feasible paths. We leave this evaluation to future work.
Q4. Semantic roles of the goal and motion queries.
Yes. Figure 6 in the revised manuscript shows both comparisons. In the GMFlow panels, comparing the solid and dashed lines of the same color holds the goal query fixed and varies the motion query. Conversely, comparing lines of the same style across colors holds the motion query fixed and shows how the predicted destination changes as the goal query varies.
Section 4.5 quantifies these differences across the NBA test set. Among the 20 candidates per forecast, the mean pairwise endpoint distance is 0.39 m for candidates sharing a goal query and 4.93 m for candidates with different goal queries. Within each goal query, the two motion variants are 0.59 m apart on average at corresponding positions across the 18 non-goal frames. Together, these results show that different goal queries explore distinct destinations and that a single goal query can produce different paths to nearby endpoints.
Candidates sharing a goal query start from the same goal latent. During joint generation, different motion queries can slightly adjust the goal estimate, so the two predicted endpoints may differ. This interaction is intentional: it allows local motion to inform the goal estimate before decoding. Once the generated goal latent is fixed, the decoder preserves its waypoint and endpoint positions exactly as the motion latent varies.
I appreciate the additional baseline comparisons, multimodal analysis, and multi-seed experiments, which address several of my concerns. However, the methodological novelty remains relatively limited, and the controlled comparisons are largely restricted to NBA. Further validation against existing goal-conditioned methods across datasets, particularly autonomous-driving benchmarks, would strengthen the contribution.
Additional controlled comparisons on SDD and ETH–UCY
Thank you for the follow-up. To address your concern that the controlled comparisons were largely restricted to NBA, we evaluated both waypoint-conditioned baselines on SDD and ETH–UCY. The soft baseline uses the predicted waypoints to guide prediction of the full trajectory. The hard baseline fixes the trajectory positions at those waypoints and predicts the remaining positions. Each table entry reports minADE₂₀ / minFDE₂₀.
| Method | SDD (px) ↓ | ETH–UCY average (m) ↓ |
|---|---|---|
| Soft waypoint constraints | 7.03 / 11.27 | 0.19 / 0.32 |
| Hard waypoint constraints | 6.91 / 10.98 | 0.20 / 0.30 |
| GMFlow | 6.69 / 10.28 | 0.17 / 0.28 |
GMFlow achieves lower trajectory and endpoint errors than both baselines on both datasets, including the hard baseline whose trajectories pass exactly through the predicted waypoints. Together with the NBA results, these comparisons show that GMFlow's advantage over the two waypoint-conditioned baselines holds across all three benchmarks.
Scope of evaluation and methodological contribution
We agree that extending the current work to autonomous-driving benchmarks is promising. However, autonomous-driving benchmarks such as nuScenes, Argoverse 2, and Waymo Motion focus on integrating HD vector maps and data from multiple sensors into trajectory prediction. This is a separate topic. Following the evaluation practices of the state-of-the-art methods we compare against in our paper, we currently evaluate our method on ETH–UCY, SDD, and NBA.
We focus on trajectory prediction using only past trajectories, without HD maps or data from other sensors. The benchmarks used in our work cover several common movement patterns, including moving slowly or occasionally remaining stationary, moving rapidly or turning abruptly as in NBA, and walking in dense crowds. Evaluation across these scenarios demonstrates that GMFlow improves upon other methods, especially in cases involving rapid movement or abrupt turns. These findings complement our other results, which indicate the effectiveness of GMFlow’s joint representation. The main difference between GMFlow and existing goal-conditioned and global-to-local trajectory prediction methods is that GMFlow jointly generates goals and local motion, allowing them to influence and adjust each other during generation. GMFlow can therefore predict trajectories more effectively than prior methods.
Response to Q1–Q2
Thank you for your careful reading and constructive comments.
Q1. Controlled comparison with waypoint-conditioned predictors.
We agree that the unified-latent variant in Table 4 does not by itself establish an advantage over conventional waypoint-conditioned designs. To compare GMFlow directly with conventional waypoint-conditioned predictors, we implemented two baselines within our framework. Both use the same history encoder architecture, Agent and Mode Transformer capacity, candidate-assignment strategy, number of candidates (K=20), and training budget as GMFlow. All three models predict residuals relative to the same constant-velocity reference.
Both baselines first predict waypoints at frames 10 and 20, the same goal frames used by GMFlow. The baseline with soft waypoint constraints uses the predicted waypoints as inputs to a decoder that predicts all 20 positions, allowing the decoded trajectory to deviate from those waypoints. The baseline with hard waypoint constraints fixes the trajectory positions at frames 10 and 20 to the predicted waypoints and predicts the remaining 18 positions conditioned on them.
Of the two baselines, the one with soft waypoint constraints achieves the lower minADE₂₀ (0.706 m), while the one with hard waypoint constraints achieves the lower minFDE₂₀ (0.843 m). GMFlow achieves lower errors on both metrics, with a minADE₂₀ of 0.681 m and a minFDE₂₀ of 0.796 m. Generating all 20 candidates requires one network evaluation for GMFlow and two for each baseline.
Q2. Conceptual distinction from goal-conditioned and global-to-local approaches.
The conceptual distinction lies in the role of the goal. In typical goal-conditioned and global-to-local approaches, a predicted goal or set of key positions guides subsequent path generation. In GMFlow, the goal is an explicit component of the trajectory representation and is generated jointly with local motion. At decoding, the goal component directly specifies the waypoint and endpoint positions, while the motion component shapes the intervening path. The two aspects below follow from this design.
(1) Joint generation of goals and local motion, with the generated goal positions preserved at decoding.
Methods such as Y-Net and G2LTraj first determine endpoints or key waypoints and then generate the local path conditioned on them. The goals or waypoints can influence path generation, but the previously selected goals cannot be revised during path generation, so errors in the initial goal estimates may limit forecasting accuracy. IAD uses the pre-estimated endpoint as a guidance condition for diffusion denoising. Denoising updates the entire trajectory but does not re-estimate the guidance endpoint, and the final predicted endpoint may be inconsistent with it.
GMFlow generates the goal latent and the local-motion latent jointly through conditional flow matching, so that the two mutually influence and adjust each other during generation. At decoding, the goal-preserving decoder reconstructs the trajectory from both latents and guarantees that the local-motion latent does not alter the positions specified by the goal latent. To examine the effect of joint generation, we compare GMFlow with a goal-then-motion variant while keeping the trajectory representation, decoder, encoder, number of candidates, and training budget identical. Joint generation reduces minADE₂₀ / minFDE₂₀ from 0.702 / 0.835 to 0.681 / 0.796.
(2) Multimodal modeling via learnable goal and motion queries.
Methods such as PECNet and Graph-TERN mainly produce diverse candidates by sampling different endpoints, whereas IAD performs diffusion sampling over the entire trajectory under intention guidance. In the former, additional endpoint samples mainly expand destination coverage and do not systematically produce more paths for each destination; in the latter, additional whole-trajectory diffusion samples change the endpoint and the path simultaneously, so the added candidates cannot be purposefully allocated to a specific type of uncertainty.
GMFlow crosses L learnable goal queries with S learnable motion queries to generate K=L×S candidates, turning the two types of uncertainty in Fig. 1 (different destinations vs. different paths toward the same destination) into separate budget allocations for goal and motion queries. Table 6 compares different configurations of L and S and shows how the two budget allocations affect trajectory and endpoint errors.