acceptodds
Under review as a conference paper at ICLR 2027

GroupModes: Learning to Couple Goals and Interactions for Human Trajectory Prediction

Abstract

Accurate human trajectory prediction is essential for autonomous driving, mobile robotics, and crowd monitoring. Yet, existing models can achieve low individual trajectory errors without necessarily generating coherent group-level futures that account for interactions among agents. To address this, we propose GroupModes, a trajectory prediction architecture that conditions agent-specific goals and evolving social interactions on a shared group-level mode. Position-aware encoding captures observed motion and spatial context. A structured mode bank generates observation-conditioned modes, augmented with bounded stochastic residuals to represent within-mode variation. Each mode specifies individual goals and modulates time-varying interaction logits through a low-rank parameterization. A goal-conditioned synchronous decoder jointly generates future trajectories for the target and its neighboring agents. Experimental results demonstrate that GroupModes achieves competitive prediction accuracy and that aligning goal and interaction realization under a shared mode improves target and joint forecasting. Further analyses indicate that the learned group modes organize individual trajectory candidates into coordinated joint hypotheses, providing benefits beyond individual candidate coverage. Our code is available at https://anonymous.4open.science/r/groupmodes-review-0F6B.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human trajectory prediction, Joint trajectory prediction, Social interaction modeling, Multimodal forecasting, Shared group modes

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TRdSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies multimodal human trajectory prediction with an emphasis on coherent joint futures rather than individual candidate accuracy alone. GroupModes uses an observation-conditioned bank of shared modes with bounded stochastic residuals. Each mode conditions both agent-specific goals and low-rank modulation of time-varying interaction logits, while a synchronous decoder generates trajectories for the target and its neighbors. Experiments on ETH–UCY, SDD, and NBA demonstrate competitive best-of-20 marginal accuracy. Additional ETH–UCY analyses examine goal–interaction coupling and cross-agent candidate correspondence using joint displacement errors and mean collision rate.

Critical strengths
  • The paper addresses a meaningful prediction problem with an appropriate architectural formulation.
  • The candidate-shuffling experiment provides useful evidence beyond marginal accuracy.
Critical weaknesses
  • The main joint-prediction contribution lacks direct evaluation against established joint predictors. Tables 1 and 2 primarily compare marginal best-of-20 errors, whereas the joint metrics in Tables 4 and 5 compare internal variants of GroupModes. The current experiments do not establish whether its specific goal–interaction coupling improves joint prediction over these alternatives.
  • The evidence attributing the benefits specifically to goal–interaction coupling remains limited. Candidate shuffling establishes meaningful correspondence, but does not by itself distinguish the proposed coupling mechanism from other joint generators. In Table 4, shared versus decoupled modes changes joint ADE/FDE from 0.48/0.87 to 0.45/0.84, while mean collision rates remain close at 8.90% and 8.81%. Appendix A.1 states that all reported training runs use seed 42, leaving the variability of these differences unquantified. The paper should clarify whether the decoupled variant is trained with independent goal–interaction pairing or only perturbed at evaluation, and report variation across training seeds for the central coupling comparison.
Questions for authors

See weaknesses

Reviewer_FdxHOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes GroupModes, a multimodal human trajectory prediction framework that aims to generate coordinated future trajectories for multiple interacting agents. The central motivation is that low individual trajectory prediction errors do not necessarily imply physically or socially coherent joint futures. To address this issue, GroupModes uses a shared group-level mode to simultaneously condition agent-specific destinations and time-varying social interactions. GroupModes achieves SOTA or comparable performance on several benchmarks.

Critical strengths
  1. Competitive results on several established benchmarks. GroupModes achieves strong ADE/FDE on ETH–UCY and SDD, with particularly good short-horizon performance on NBA.
  2. Detailed implementation and efficiency analysis. The appendix provides thorough model dimensions, optimization settings, loss definitions, inference time, and parameter counts.
  3. The paper is well-organized and easy to follow.
Critical weaknesses
  1. The biggest concern is the idea of joint goal conditioning is not new. Previous models like CGPNet [1] and Goal-lbp [2] have explored to simultaneously predict goals of all agents and then leverage the information for final trajectory prediction.
  2. Group-level consistency is limited to individual target-centered neighborhoods. It directly affects the paper's claimed motivation of constructing coordinated group-level futures.
  3. No analysis of trajectory diversity or stochastic analysis.

[1] Conditional Goal-oriented Trajectory Prediction for Interacting Vehicles [2] Goal-lbp: Goal-based local behavior guided trajectory prediction for autonomous driving

Questions for authors

What is the specific technical advantage of coupling goals and interactions through the proposed structured mode bank compared with previous models?

Reviewer_1CEeOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GroupModes, a multimodal human trajectory prediction framework that jointly conditions agent-specific goals and time-varying social interactions on shared group-level modes. The architecture combines a structured mode bank, bounded stochastic residuals, low-rank interaction-logit modulation, and a goal-conditioned synchronous decoder.

Critical strengths
  • Important problem and reasonable architecture. Generating coordinated joint futures is an important problem in multi-agent trajectory prediction. The proposed framework provides an explicit mechanism for conditioning goals and evolving interaction weights on shared latent modes.

  • Competitive prediction accuracy. GroupModes achieves competitive results on ETH–UCY and strong ADE on SDD. The overall model is also relatively compact, with 1.47M parameters.

  • Useful ablation studies. The comparisons with a Conditional Gaussian baseline, static interactions, decoupled modes, and shuffled candidate indices provide encouraging evidence for several architectural choices. The appendix also includes useful implementation details and additional robustness analyses.

Critical weaknesses
  • The primary concern is that the core motivation and several main ideas have already been extensively investigated. AutoBots (ICLR 2022) employs shared discrete latent modes to generate scene-consistent multi-agent futures, incorporating social interactions during decoding. THOMAS (ICLR 2022) addresses cross-agent candidate correspondence by learning to recombine marginal trajectory candidates into coherent joint hypotheses. BiFF (ICCV 2023) couples high-level future intentions with low-level future interactions within joint scene modalities, providing a particularly close conceptual idea.

  • The related-work discussion does not sufficiently differentiate this mechanism from prior work. In particular, THOMAS is not discussed despite its direct relevance to the candidate-correspondence contribution.

  • The current evaluation cannot fully support the claimed improvement in joint forecasting. The comparisons (Tables 1 and 2) only report metrics (ADE/FDE), while key joint metrics (Joint ADE/FDE, collision rates) are restricted to internal ablations. Consequently, the paper does not demonstrate whether GroupModes outperforms established joint predictors like AutoBots or joint-metric-optimized approaches (e.g., Joint Metrics Matter (ICCV 2023)). To validate the main contributions, the authors must directly benchmark GroupModes against representative joint forecasting baselines using matched joint metrics.

  • The analysis of mode specialization is limited. Because all evaluations rely on multi-modal prediction with a large number of modes (K=20), metrics like best-of-20 can artificially inflate performance, which allows even simple models to achieve strong results. To properly evaluate true mode quality and test model sensitivity, the paper should include evaluations under deterministic prediction (K=1) and lower-mode regimes (e.g., K=3, 6).

Questions for authors

N/A

Comments on the paper

1 comment
Area_Chair_NQbSOpenReview ↗

Please enange in discussion with the authors

Dear Reviewers,

Please read the authors' response. Evaluate whether your concerns are addressed or if anything needs more attention. Please engage with the authors while we have time.

Thanks, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.