acceptodds
Under review as a conference paper at ICLR 2027

Bio-TAD: Biomechanical Target-Anchored Diffusion for Multimodal Pedestrian Trajectory Prediction

Abstract

Intelligent systems in autonomous driving necessitate effective approaches for predicting multimodal pedestrian trajectories, especially during highly stochastic maneuvers characterized by minimal early macro-spatial displacement. We present Bio-TAD, an innovative backbone-agnostic generative framework that utilizes fine-grained biomechanical posture cues to effectively anchor a trajectory diffusion model. Instead of passively concatenating continuous features, our method employs an Orthogonal Prototype Memory Bank to convert kinematic signals into discrete behavioral primitives, thereby constraining the generative space. The generation process is regulated by a target-oriented Adaptive Layer Normalization (AdaLN) engine and optimized through a cohesive Multi-Task Diffusion Objective that aligns spatial diffusion errors and destination anchoring concurrently. Comprehensive assessments across three benchmark datasets demonstrate superior performance, yielding up to a 12% reduction in Final Displacement Error relative to the closest pose-augmented prior work, and an absolute improvement of up to 12.11% in semantic Intent Rate over coordinate-only baselines. Through controlled ablation, we further show that discretizing pose into orthogonally-regularized primitives outperforms continuous concatenation under otherwise identical conditions, isolating discretization as the source of the gain. To expose this effect, to which standard minADE/minFDE metrics are structurally blind, we introduce a novel Intent Rate metric that measures semantic mode correctness against a training-decoupled geometric classifier.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Pedestrian Trajectory Prediction, Biomechanical Posture, Autonomous Driving

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pUJfOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes a framwork that integrates human pose information with spatial trajectories for human trajectory prediction. The framework employs an Orthogonal Prototype Memory Bank to convert kinematic signals into discrete behavioral primitives. The evaluation uses three benchmark datasets, showing improvements in trajectory prediction accuracy and intent prediction accuracy.

Critical strengths

The motivation for incorporating human pose information and bounding boxes is clear, with an example showing in Fig.1. The experiment shows that, for exisiting human trajectory prediction methods, incorporating extra biomechanical posture will benefit. The evaluation datasets are also complementary, including synthetic, 2D pose and 3D pose datasets. The ablation studies helps to understand the contribution of each component.

Critical weaknesses
  1. The proposed method, including encoders for trajectories and body poses, with cross attention, and a DiT to denoise future trajectory. The proposed method itself is already a stand-alone human trajectory predictor, but the results of Bio-TAD alone is not reported at all. All results are combining other predictors with Bio-TAD. Could the authors report the performance of Bio-TAD or explain why if it cannot be used by itself?

  2. What does it mean exactly for "backbone-agnostic compatibility"? The paper claims to be compatible with different trajectory prediction approaches. But it is not clear how Bio-TAD is combined with each baseline. Is the encoder of body pose can be plug-in, or the denoiser can be plug-in, or it is like for different approach, it needs to have different ways of exploiting the body pose information?

  3. About social-transmotion baseline comparison. This baseline method already has the version which inlcudes trajectory, 2D/3D pose, and bounding boxes. So does the reported values include social-transmotion method with pose and bounding boxes version in the original paper, or only the version with Bio-TAD? And how to report the ade_min20 for social-transmotion as it only has deterministic prediction? And why in table 3, JTA and Urban results are skipped?

  4. Inconsistency of the results. Some reported values are not consistent over tables. For example, Social-LSTM IR value for T=12 is different in table 6 and table 10.

  5. About the evaluation metric, IR.

  • Among 8 intent categories, only the turns are reported. If the intent evaluation is claimed to be main contribution of the work, all categories can be reported to strengthen the claim.
  • The definitation of IR is: the ratio of the K predicted trajectories that accurately correspond to the ground-truth semantic class. For JRDB, K=1, how does the IR computed in this case, like shown in table 11?
  1. It would be helpful to show statistics signifance for the reported results, especially when the gain is marginal, like Urban dataset results in Table 3. The results include std over multiple random seeds, paired significance tests, would be helpful to strengthen the claim.

  2. The proposed method is a diffusion-based predictor. It would be helpful to include more recent diffusion-based predictors like MoFlow [1] or the intention-aware diffusion models [2], which is mentioned in related works already.

[1] Y. Fu et al. MoFlow: One-Step Flow Matching for Human Trajectory Forecasting via Implicit Maximum Likelihood Estimation based Distillation. CVPR 2025.

[2] Y. Liu et al. Intention-Aware Diffusion Model for Pedestrian Trajectory Prediction. AAAI 2026.

Questions for authors

Same as weaknesses.

11 comments
AuthorsOpenReview ↗

Response to reviewer pUJf

Q1: Standalone "Vanilla Bio-TAD" Evaluation & "Backbone-Agnostic" Terminology

We thank the reviewer for this observation. The reviewer correctly notes that the architecture presented in Figure 2—comprising the spatial/biomechanical encoders, asymmetric cross-attention, and the DiT denoiser—functions as a complete, standalone trajectory predictor.

  • Clarification on "Backbone-Agnostic" Terminology: Our use of the term "backbone-agnostic" was intended to highlight that the Macroscopic Spatial Stream (the trajectory encoder) can be seamlessly replaced with existing foundational architectures (e.g., Social-LSTM, AutoBots). Our primary goal was to demonstrate how our discrete biomechanical integration reliably improves upon established spatial representations. However, we acknowledge that omitting the performance of the native, un-augmented Bio-TAD framework inadvertently obscured its viability as a powerful independent model.
  • New Evaluation (Vanilla Bio-TAD): Bio-TAD is indeed fully capable of functioning on its own. To address this, we have now evaluated Vanilla Bio-TAD, utilizing our native spatial transformer encoder exactly as depicted in Figure 2, without integrating an external baseline backbone:
ModelInput ModalityminADE₂₀ (↓)minFDE₂₀ (↓)Far Left Turn IR (↑)Far Right Turn IR (↑)
Vanilla Bio-TADTraj + Pose0.58 m1.18 m51.5%52.1%

As these new results demonstrate, the standalone Bio-TAD architecture is highly competitive. The native spatial transformer efficiently captures macroscopic routing, while our core innovations—the Orthogonal Prototype Memory Bank and target-oriented AdaLN engine—operate exactly as designed.


Q2: Uniform Integration of External Baselines

We appreciate the reviewer highlighting the need for clearer architectural documentation regarding how external baselines integrate into our framework. The integration mechanism is completely uniform across all baselines; there are no model-specific methods for exploiting body pose information:

  • The Swappable Component (The Backbone): External baseline models (e.g., Social-LSTM, AutoBots, VLMTraj) function exclusively as the Macroscopic Spatial Stream encoder. Their sole responsibility is to process historical coordinate sequences and output their standard spatial hidden state or embedding.
  • The Handoff: Regardless of the baseline architecture, its spatial output is linearly projected to a uniform hidden dimension (d = 256). This standardizes the output, allowing it to act as the query tensor in our Asymmetric Cross-Attention module.
  • The Fixed Bio-TAD Framework: The rest of the pipeline is native to Bio-TAD and remains completely static. The Microscopic Biomechanical Stream (pose encoder), Orthogonal Prototype Memory Bank, Asymmetric Cross-Attention fusion, and the DiT generative denoiser are universally applied exactly as depicted in Figure 2, regardless of the spatial backbone attached.

(Continued in Part 2/3)

AuthorsOpenReview ↗

Response to reviewer pUJf

(Continued from Part 1/3)

Q3.1 & Q3.2: Social-Transmotion Comparisons & Multimodal Adaptation

  • Q3.1 (Comparison Against Native Pose-Integrated Social-Transmotion): Our reported values explicitly include and compare against the original Social-Transmotion method's native pose-integrated version. To strictly isolate the performance gains of our proposed architecture, we evaluated three distinct configurations for Social-Transmotion, as documented in Tables 2 and 3 of the manuscript:
    1. Trajectory-Only: The base model using purely historical coordinate sequences.
    2. Base + Native Pose (Original): The baseline's original continuous concatenation method for integrating pose and bounding boxes (yielding a deterministic ADE/FDE of 0.37/0.73 m on JRDB, as shown in Table 3).
    3. Base + Bio-TAD (Ours): Our proposed framework replacing the native pose integration with our Orthogonal Prototype Memory Bank (yielding a superior deterministic ADE/FDE of 0.35/0.69 m on JRDB, as shown in Table 3). By reporting all three versions, we ensure a fair comparison. The results confirm that our predictive improvements stem specifically from how Bio-TAD discretizes biomechanical data into behavioral primitives, rather than just the mere addition of pose and bounding box inputs.
  • Q3.2 (Multimodal K = 20 Adaptation & Missing JTA/Urban Results): The reviewer correctly points out that the native Social-Transmotion architecture is inherently deterministic. To ensure a fair multimodal comparison (minADE₂₀) against generative frameworks in Table 3, we adapted the unaugmented baseline by appending a standard Conditional Variational Autoencoder (CVAE) output head to its spatial encoder. This standard modification allowed the baseline to generate the K = 20 probabilistic samples required to calculate the metric. In contrast, our deterministic K = 1 evaluations in Table 2 evaluate the unaltered, native architecture. For our proposed Social-Transmotion + Bio-TAD configuration, the K = 20 samples are generated entirely through our framework's integrated Diffusion Transformer (DiT) denoiser, resolving the baseline's deterministic limitation without the need for a CVAE head. The omission of the JTA and Urban results for the native pose-integrated Social-Transmotion baseline in the initial submission occurred because the original authors did not report multimodal (K = 20) metrics for this specific variant (whereas JRDB is deterministic at K = 1, allowing us to directly utilize their published numbers). We have now fully trained the CVAE-adapted version of their pose-integrated architecture on the JTA and Urban datasets to generate the required K = 20 probabilistic samples and updated Table 3 in the revised manuscript with the complete comparative results.

Q4: Social-LSTM Intent Rate Discrepancy Between Table 6 (now Table 7) and Table 10 (now Table 12)

We thank the reviewer for pointing out this clerical error. We have corrected the discrepancy in the revised manuscript.

The Intent Rate (IR) values for the Social-LSTM baseline reported in original Table 10 (Table 12 in the revision) represent our canonical, final evaluation on the 128-sequence JTA test split (using 50-step DDIM at η = 0.0). The discrepancy occurred because the Social-LSTM rows in original Table 6 (Table 7 in the revision, our Capacity-Matched Ablation) inadvertently retained preliminary numbers generated during an early tuning phase on the validation split, which we had used to verify the scaling of the recurrent layers before running our final test suite. While the AutoBots rows and all geometric metrics in original Table 6 (Table 7) were properly updated to the canonical test-split results, we unfortunately missed updating the four Social-LSTM IR columns.

We have updated Table 7 to reflect the canonical test-split results, ensuring complete consistency with Table 12 across the manuscript, and double-checked all other tables to ensure absolute consistency. Importantly, updating these values to the correct test set leaves the core conclusion of our capacity-matched ablation (Appendix A.3) entirely unchanged: scaling the raw parameters of a coordinate-only model (Capacity-Matched) yields only marginal performance gains, whereas integrating the structural innovations of the Bio-TAD framework within the exact same ~11.7M parameter budget drives substantial geometric improvements and semantic gains.

(Continued in Part 3/3)

AuthorsOpenReview ↗

Response to reviewer pUJf

(Continued from Part 2/3)

Q5.1 & Q5.2: Intent Category Selection & Deterministic (K = 1) Intent Rate Definition

  • Q5.1 (Focus on the Four Turn Intent Categories): We appreciate the reviewer's suggestion to include all eight intent categories. We intentionally focused our reporting on the four turn categories (Left, Right, Far Left, Far Right) because they represent the specific, highly stochastic maneuvers where conventional coordinate-only models structurally fail. Categories such as Idle, Stopping, Starting, and Straight represent linear or static behaviors. Standard baselines already achieve exceptionally high intent alignment on these classes because macroscopic coordinate momentum alone is sufficient to predict them. Including these macro-spatially trivial classes in our main tables would saturate the average scores and mask the specific capability Bio-TAD was designed for: leveraging pre-movement biomechanical cues (like pelvic or shoulder rotation) to anticipate abrupt directional changes before spatial displacement occurs.
  • Q5.2 (Meaning of Intent Rate at K = 1 in Table 11 / now Table 13): We thank the reviewer for pointing out this ambiguity in our metric reporting. To clarify, Equation 8 defines the Intent Rate (IR) for a single pedestrian prediction instance. When the evaluation is deterministic (K = 1), as in the JRDB dataset, the per-instance IR simplifies to a binary indicator: it is 1 (100%) if the single predicted trajectory matches the ground-truth intent class, and 0 (0%) if it does not. The values reported in original Table 11 (Table 13 in the revision) are the dataset-wide averages of these per-instance scores for each respective intent category. Therefore, in the K = 1 setting, the aggregate IR mathematically equates to the standard Top-1 semantic classification accuracy for that specific maneuver across the test set. We have updated Section 4.2 in the revised manuscript to explicitly state this dataset-level averaging for both multimodal (K = 20) and deterministic (K = 1) settings.

Q7: Comparison with Contemporary Diffusion Predictors (MoFlow)

We thank the reviewer for this constructive suggestion. We agree that including contemporary diffusion-based predictors provides a more robust benchmark for our DiT-based framework. In our revised manuscript, we have implemented and evaluated the recent MoFlow architecture (alongside the intention-aware diffusion model referenced in our related work). We evaluated both the native MoFlow baseline and our Bio-TAD-augmented version on the JTA dataset.

As shown in our updated Table 3 and Table 12 (original Table 10), the native MoFlow model is highly competitive (achieving a minADE₂₀/minFDE₂₀ of 0.60/1.36 m). However, because it remains a coordinate-only predictor, it still struggles with pre-movement intent alignment during highly stochastic maneuvers (achieving only a 36.50% Far Left Turn IR). When MoFlow is integrated with the Bio-TAD framework to explicitly process discrete biomechanical priors, its spatial error drops to 0.52/1.03 m, and its semantic Intent Rate improves by +17.80% for sharp turns. We have added these models to our baseline suite across all relevant tables to strengthen the comparative evaluation of our diffusion objective.

Reviewer_KoaXOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Bio-TAD uses body-pose cues for multimodal pedestrian trajectory prediction. It combines prototype memories with orthogonality regularization and a diffusion model conditioned on the evolving trajectory endpoint. The paper reports improvements on JTA, JRDB, and Urban across several backbones. It also introduces Intent Rate (IR), which measures how often predicted samples match the observed future's motion class.

Critical strengths

Using pose to anticipate turns is well motivated. The experiments cover three datasets and several backbones, and include relevant comparisons with continuous pose fusion.

Critical weaknesses
  1. The contribution beyond existing pose-based methods is not sufficiently demonstrated. Social-Pose and Social-Transmotion already use pose information for trajectory prediction, and AdaLN is standard in DiT. I am not convinced that the paper establishes the worthy contributions. In particular, orthogonal memory vectors in Eq. (2) do not ensure that different slots represent different behaviors or are even used. The ablations report gains, but there is no analysis connecting slot assignments to poses and predicted modes. The author only provide the Figure 1 which is a schematic; actual test cases, comparisons with a pose-aware baseline, and failures would help substantiate the claim. Table 7 also changes endpoint extraction and conditioning together, leaving the benefit of the evolving endpoint unclear. Finally, the claim in Appendix A.1 that pose is useful only when discretized is contradicted by the gains from continuous fusion in Table 8.
  2. The interpretation of IR and the reported numbers raise concerns about the evaluation. Under Eq. (8), every sample outside the observed ground-truth class receives zero credit, even if it represents another plausible future. Predicting only the ground-truth class gives 100% IR. Higher IR therefore does not establish a better multimodal distribution. Appendix A.1 reports entropy, but this does not resolve whether the predicted diversity is appropriate for each input. I also found unexplained numerical inconsistencies. All 16 trajectory-only IR entries for AutoBots and all 16 for Social-Transmotion are identical between JTA and JRDB in Tables 10 and 11. For Social-LSTM on JTA at T = 12, Table 6 gives far-left IR of 26.40%/48.70% (baseline/augmented), while Table 10 gives 30.17%/47.53%.
Questions for authors
  1. What explains the repeated values in Tables 10 and 11 and the discrepancy between Tables 6 and 10? Please specify any differences in splits, sampling, or evaluation settings.
  2. In Table 8, are inputs, decoder, losses, and training budget held fixed across the pose-integration variants? Which parts of each original backbone are retained?
  3. How is the entropy in Table 4 computed? Is there evidence linking prototype usage to distinct behaviors and showing that the IR gains reflect better multimodal predictions?
10 comments
AuthorsOpenReview ↗

Response to reviewer KoaX

Critical Weakness (CW) - 1 (Points 1–3)

We thank the reviewer for their critical evaluation of our manuscript. We appreciate the opportunity to clarify our core contributions regarding our pose integration mechanism, the application of AdaLN, and our empirical validations.

1. Contribution Beyond Existing Pose-Aware Baselines

  • Reviewer Concern: The contribution beyond existing pose-based methods is not sufficiently demonstrated. Social-Pose and Social-Transmotion already use pose information.
  • Author Response: While prior methods utilize pose data, our contribution lies in the fundamentally different mechanism of integration, which yields strictly superior empirical results. Existing architectures like Social-Pose rely on passively concatenating continuous pose embeddings into an unconstrained latent space, leaving the network to blindly resolve the mapping between high-frequency kinematic noise and multi-future trajectories. In contrast, Bio-TAD introduces an Orthogonal Prototype Memory Bank that quantizes these signals into explicitly regularized, discrete behavioral primitives. In Table 2, Bio-TAD reduces Final Displacement Error (FDE) by ~12% and ADE by ~7% compared to the Social-Pose continuous integration baseline on identical backbones. Furthermore, Table 9 (Appendix A.6) explicitly ablates our pose integration mechanism, proving that transitioning from standard continuous concatenation to our discrete orthogonal memory bank is the precise source of this performance gain.

2. Novel Application of AdaLN

  • Reviewer Concern: AdaLN is standard in DiT. I am not convinced that the paper establishes the worthy contributions.
  • Author Response: We acknowledge that Adaptive Layer Normalization (AdaLN) is a standard component in Diffusion Transformers. However, our novelty lies in how we condition it for trajectory forecasting. Standard DiT architectures utilize AdaLN for static, global class conditioning. Bio-TAD uniquely repurposes AdaLN for dynamic waypoint anchoring. Instead of relying on a separate sub-network to statically predict spatial endpoints upfront, our target-oriented engine dynamically extracts the evolving terminal destination waypoint directly from the latent state after every backward denoising iteration to steer the generative manifold.

3. Memory Slot Utilization and Behavioral Mapping

  • Reviewer Concern: Orthogonal memory vectors in Eq. (2) do not ensure that different slots represent different behaviors or are even used. The ablations report gains, but there is no analysis connecting slot assignments to poses and predicted modes.
  • Author Response: First, our sensitivity analysis in Table 4 (Appendix A.1) quantitatively confirms that additional memory slots are actively utilized rather than left unused: expanding the memory bank from 64 to 128 and 256 slots monotonically increases output Shannon Entropy (0.65 →\rightarrow 1.12 →\rightarrow 1.35) and sharp-turn Intent Rate (46.1% →\rightarrow 50.2% →\rightarrow 52.47%), whereas removing the orthogonality constraint in Table 9 causes partial slot collapse (dropping Far Left/Right Turn IR from 53.12%/54.97% to 48.30%/49.80%). Second, to explicitly verify that the orthogonal memory slots learn distinct, behaviorally meaningful representations, we mapped the Gumbel-Softmax routing distributions during inference across the test set, conditioning activation frequencies on ground-truth semantic intent classes (e.g., "Straight", "Far Left Turn", "Far Right Turn"). The revised manuscript includes a slot activation breakdown (Table 5 in Appendix A.1), showing that all slots are used and specialize by maneuver at above-chance levels (peak selectivity 0.28–0.39 vs. class frequencies of 0.07–0.10 under random routing). We describe this as moderate specialization rather than strict separation.

(Continued in Part 2/4...)

AuthorsOpenReview ↗

Response to reviewer KoaX

(...Continued from Part 1/4)

Critical Weakness (CW) - 1 (Points 4–6)

4. Real Test Cases and Failure Analysis

  • Reviewer Concern: The authors only provide Figure 1 which is a schematic; actual test cases, comparisons with a pose-aware baseline, and failures would help substantiate the claim.
  • Author Response: While the left panel of Figure 1 provides a conceptual schematic alongside empirical test-set FDE and IR diagnostic curves across frames −6-6 to −1-1 on the right panel, we fully agree that qualitative rollout visualizations and failure analyses strengthen the paper. We have added a dedicated Qualitative Evaluation section to the revised manuscript visualizing actual JTA test cases comparing the coordinate-only baseline (AutoBots (Traj-only)), continuous pose fusion (AutoBots + Social-Pose), and AutoBots + Bio-TAD across both sharp mid-stride deflections and abrupt orthogonal pivots, alongside a dedicated failure case analysis. While the coordinate-only baseline linearly extrapolates past momentum and Social-Pose suffers from under-rotation or endpoint drift, Bio-TAD accurately tracks the sharp inflection curvature and terminal destination. Finally, our failure case analysis transparently highlights a boundary limitation during an impulsive, late-onset pivot initiated right at the observation cutoff with minimal prior postural torsion, where strong historical momentum in the asymmetric fusion gate partially restrains the angular deflection despite Bio-TAD still bending closer toward the true turn than both baselines.

5. Disentangling Endpoint Extraction and Conditioning

  • Reviewer Concern: Table 7 also changes endpoint extraction and conditioning together, leaving the benefit of the evolving endpoint unclear.
  • Author Response: We acknowledge that simultaneously modifying the AdaLN mechanism and dynamic endpoint extraction in original Table 7 (Table 8 in the revised version) obscured the individual contribution of the evolving waypoint. To isolate this contribution, we have added a new "Static Waypoint + AdaLN" sub-baseline to Table 8 in the revised manuscript. In this configuration, AdaLN is used for conditioning, but with a static spatial endpoint predicted upfront by a separate MLP, rather than the dynamic endpoint extracted directly from the latent state at step tt. This new baseline successfully isolates the dynamic extraction mechanism, demonstrating that removing dynamic step-tt waypoint refinement degrades minFDE20\text{minFDE}_{20} from 1.13 m to 1.22 m and reduces Far Left Turn IR from 53.12% to 48.65% (Table 8).

6. Clarification on Pose Discretization vs. Continuous Fusion

  • Reviewer Concern: Finally, the claim in Appendix A.1 that pose is useful only when discretized is contradicted by the gains from continuous fusion in Table 8.
  • Author Response: We acknowledge the overstatement in our original text. Stating that pose is "only" useful when discretized contradicts the empirical findings in original Table 8 (Table 9 in the revised version), which show a clear geometric gain from continuous concatenation (reducing the AutoBots baseline ADE from 0.72 m to 0.66 m). We have revised the text in Section 4.5 and Appendix A.1 to accurately reflect these data. Absolute phrases like "only when discretized" have been corrected to "is optimally utilized when discretized." The revised text clarifies that while continuous concatenation provides marginal geometric improvements, it struggles to resolve high-frequency kinematic noise into accurate semantic predictions. As shown in original Table 8 (Table 9 in the revised version), transitioning to a discrete Orthogonal Prototype Memory Bank is strictly required to achieve the +13.72% absolute improvement in Far Left Turn Intent Rate, establishing discretization as the optimal structural mechanism for generative semantic alignment rather than the exclusive method for utilizing pose.

(Continued in Part 3/4...)

AuthorsOpenReview ↗

Response to reviewer KoaX

(Continued from Part 2/4)

Critical Weakness (CW) - 2

We thank the reviewer for this rigorous critique of the Intent Rate (IR) metric and for catching two Appendix table transcription errors.

  • Metric Ensembles and Mode Collapse: IR is not evaluated in a vacuum. Inflating IR via deterministic mode collapse across the dataset would guess the wrong mode in stochastic scenarios, degrading both corpus-level IR and best-of-K geometric error (minADE₂₀ and minFDE₂₀). Table 4 (Appendix A.1) confirms this: restricting memory capacity to 64 slots causes mode collapse (Shannon Entropy drops to 0.65) and lowers sharp-turn IR to 46.1% (vs. 52.47% at 256 slots, Entropy 1.35). Furthermore, Table 3 and Tables 12–14 show Bio-TAD achieves the lowest minADE₂₀ and minFDE₂₀ alongside higher IR (47%–57%, or 10–11 of 20 samples), proving it allocates more probability mass to the true intent while preserving 9–10 samples for multimodal spatial coverage.
  • Single Ground-Truth and Plausible Futures: We agree IR alone can be maximized by collapsing onto the ground-truth class; we revised Section 4.2 to state this explicitly and report IR only jointly with minADE₂₀/minFDE₂₀ and entropy.
  • Input-Appropriate Diversity: While corpus-averaged Shannon Entropy (Appendix A.1) verifies global diversity, per-input diversity in Bio-TAD is governed by our asymmetric cross-attention gate α (Section 3.4, Eq. 3) alongside per-input minADE₂₀/minFDE₂₀. When a pedestrian exhibits clear pre-turn biomechanical torsion (e.g., rotated pelvis/shoulder 1–2 frames before a turn, Figure 1), opposite-direction turns become physically implausible, and α dynamically scales biomechanical priors based on trajectory uncertainty. In Table 8 (Appendix A.4) (original Table 7), ablating α (w/o Asymmetric Gate α) degrades minADE₂₀/minFDE₂₀ from 0.56/1.13 m to 0.60/1.21 m and drops Far Left/Right IR from 53.12%/54.97% to 44.80%/46.50%.
  • Repeated Trajectory-Only (T) Entries in Table 10 (now Table 12, JTA) and Table 11 (now Table 13, JRDB): There is no overlap in splits or settings between JTA (K = 20 on the 128-sequence test split) and JRDB (K = 1 on the JRDB test split; Section 4.2, Appendix B). When formatting Appendix A.7 in LaTeX, the 16 trajectory-only (T) baseline cells for AutoBots and Social-Transmotion from original Table 10 (Table 12, JTA) were inadvertently copied into original Table 11 (Table 13, JRDB), whereas their T+P columns and all T and T+P columns for the other four backbones in Table 13 were properly updated with JRDB results. We have replaced those 16 duplicated cells with the true JRDB T numbers in Table 13.
  • Discrepancy in Social-LSTM Intent Rate (T = 12) Between Table 6 (now Table 7) and Table 10 (now Table 12): Across the paper, all AutoBots metrics are identical across Tables 1, 3, 7, 8, 9, and 12, and the geometric errors (minADE₂₀/minFDE₂₀) for Social-LSTM (Base) (0.58/1.25 m) and Social-LSTM + Bio-TAD (0.51/1.05 m) are identical between Table 3 and original Table 6 (Table 7). Only the four Social-LSTM IR columns (Base, Capacity-Matched, + Bio-TAD) differed between original Table 6 (Table 7) and original Table 10 (Table 12). During the capacity-matched ablation (Appendix A.3), all three Social-LSTM variants were evaluated on the 128-sequence JTA validation split during checkpoint selection for the scaled 11.71M-parameter RNN (d = 1024) to avoid test-set leakage. When assembling original Table 6 (Table 7), we updated the Social-LSTM minADE₂₀/minFDE₂₀ columns and the AutoBots block to the test split, but inadvertently left the Social-LSTM IR columns from the validation-split log instead of the canonical test-split evaluation in Table 12. We have updated Table 7 with the canonical test-split IR values; our Appendix A.3 conclusion remains completely unchanged.

(Continued in Part 4/4)

Reviewer_TgCXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Bio-TAD, a backbone-agnostic generative framework for pedestrian trajectory prediction. Its most compelling feature is how it precisely addresses a "blind spot" in existing models: the phenomenon where a pedestrian’s body posture shifts before their center of mass moves—a critical detail that typically precedes an abrupt change in direction. The authors innovatively employ an orthogonal prototype repository to translate biomechanical postures into discrete behavioral primitives, which effectively constrains the generation manifold of the diffusion model.

Critical strengths

1.Incorporating human pose variations addresses the challenge of predicting sudden changes in pedestrian direction—a task difficult to accomplish using historical coordinates alone—with an intuitive approach that has been empirically validated. 2.The proposed framework is agnostic to the backbone network, allowing for seamless integration with a wide range of existing, state-of-the-art trajectory prediction models to enhance their performance.

Critical weaknesses

My current assessment is positive (leaning toward acceptance) for the following reasons:

  1. The core motivation is grounded in strong physical intuition and holds significant practical value; furthermore, ablation studies confirm the pivotal role of the discretization design.
  2. The framework demonstrates excellent architectural generalization, allowing it to be seamlessly integrated into various mainstream baseline models.
  3. That said, the "Related Work" section is somewhat thin, and the model's robustness in real-world scenarios warrants further discussion. I look forward to the authors' responses to these points during the rebuttal phase.
Questions for authors
  1. Please broaden the Related Work section by discussing and comparing with the omitted literature mentioned above—specifically RL (DADA: Dual-Alignment Domain Adaptation for Pedestrian trajectory prediction), memory-based trajectory prototypes (e.g., Memo-Net), discrete 3D pose primitives (e.g., T2M-GPT), and deep pose-trajectory interaction models (e.g., TBIFormer)—to more accurately position Bio-TAD's contributions.

2.How sensitive is the orthogonal prototype matching and overall framework performance to noisy or partially occluded 3D pose estimations in real-world scenarios?

2 comments
AuthorsOpenReview ↗

Response to reviewer TgCX

Q1: Contextualizing Bio-TAD with Related Works

We sincerely thank the reviewer for pointing out these highly relevant works. Contextualizing Bio-TAD alongside these paradigms strengthens the positioning of our framework, and we have updated the Related Work section in the revised manuscript accordingly. Below is how Bio-TAD fundamentally differs from each method:

  • Memory-Based Trajectory Prototypes (Memo-Net): While Memo-Net uses memory networks to store and retrieve macroscopic spatial trajectory prototypes, Bio-TAD introduces memory at the microscopic kinematic level. Instead of querying spatial patterns, our Orthogonal Prototype Memory Bank quantizes dense 3D biomechanical posture into discrete behavioral primitives before any spatial displacement occurs.
  • Discrete 3D Pose Primitives (T2M-GPT): T2M-GPT uses Vector-Quantized (VQ) codebooks to map discrete 3D pose primitives for text-to-motion generation. Bio-TAD adapts this for trajectory forecasting, using pose primitives not as the final generative output, but as active conditional anchors for a spatial diffusion denoiser. Furthermore, while standard VQ codebooks optimize for reconstruction fidelity, our memory bank optimizes for behavioral diversity via joint slot orthogonality regularization to prevent mode collapse.
  • Deep Pose-Trajectory Interaction (TBIFormer): TBIFormer provides continuous spatial-temporal modeling of body parts and social interactions via continuous feature concatenation. As shown in our ablation study (Appendix A.6), leaving the pose-to-future mapping to an unconstrained continuous latent space struggles with highly stochastic maneuvers. Bio-TAD departs from this by explicitly discretizing posture to constrain the generative manifold, outperforming continuous concatenation under identical conditions.
  • Domain Adaptation / RL (DADA): DADA leverages Dual-Alignment Domain Adaptation to generalize trajectory models across shifting domains. Bio-TAD’s contributions are orthogonal and complementary to macro-level adaptation: we focus on resolving intra-domain high-uncertainty events (like abrupt turns) by extracting localized physiological precursors that coordinate-only models miss.

Q2: Robustness to Noisy Pose Estimations

We thank the reviewer for highlighting real-world robustness. To quantify Bio-TAD’s sensitivity to imperfect pose estimation, we evaluated AutoBots + Bio-TAD on the JTA dataset (T=12T=12) by injecting zero-mean Gaussian noise (σ=0.05 m\sigma=0.05\text{ m} and σ=0.1 m\sigma=0.1\text{ m}) directly into the 3D joint coordinates during inference to simulate off-the-shelf estimator jitter:

Configuration (AutoBots + Bio-TAD, JTA T=12T=12)minADE$_{20}$ (↓\downarrow)minFDE$_{20}$ (↓\downarrow)Far Left IR (↑\uparrow)Far Right IR (↑\uparrow)
Clean Pose (Baseline)0.56 m1.13 m53.12%54.97%
Gaussian Noise (σ=0.05 m\sigma=0.05\text{ m})0.58 m1.18 m51.43%52.80%
Gaussian Noise (σ=0.1 m\sigma=0.1\text{ m})0.62 m1.25 m47.91%49.34%
Coordinate-Only AutoBots (Reference)0.72 m1.50 m29.12%36.13%

Even under severe spatial jitter (σ=0.1 m\sigma=0.1\text{ m}), Bio-TAD experiences only a moderate decline in semantic alignment (Far Left IR drops from 53.12% to 47.91%) and still massively outperforms the coordinate-only baseline (29.12%).

This robustness stems from two architectural choices:

  1. Orthogonal Prototype Memory Bank as a low-pass filter: Because we do not passively concatenate continuous pose features, minor jitters do not propagate directly into the diffusion latent space. Gumbel-Softmax quantization absorbs continuous noise, routing the jittered pose into the correct discrete behavioral slot.
  2. Asymmetric Cross-Attention Gate (α\alpha): When pose inputs are severely corrupted or occluded, the learned trajectory-dependent sigmoid gate reduces the weight of the biomechanical stream, smoothly falling back on macroscopic trajectory momentum rather than collapsing.

We have added this experiment and discussion to Appendix A.8 (Table 10) in the revised manuscript.

Thanks for the authors' careful response. It solves my major concern and I will keep my recoomendation. thx.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Initial global response to all reviewers

We sincerely thank all reviewers for their careful reading, constructive feedback, and insightful suggestions. We have revised the manuscript extensively in response to your comments, and all modified or newly added text and tables are highlighted in blue in the revised manuscript. Below we summarize the major updates:

  • (Reviewer TgCX) Expanded Related Work (Sections 2.1 & 2.2) to contextualize Bio-TAD alongside retrospective memory networks (Memo-Net), vector-quantized 3D motion codebooks (T2M-GPT), and continuous body-part interaction transformers (TBIFormer).
  • (Reviewer TgCX) Added a new Robustness to Noisy Pose Estimations study in Appendix A.8 (Table 10), evaluating Bio-TAD under injected 3D Gaussian joint perturbations (σ=0.05 m\sigma = 0.05\text{ m} and σ=0.10 m\sigma = 0.10\text{ m}) to demonstrate resilience to off-the-shelf pose estimator jitter.
  • (Reviewer KoaX) Added a new Slot Utilization and Disjoint Behavioral Specialization analysis in Appendix A.1 (Table 5), mapping Gumbel-Softmax prototype routing frequencies across the JTA test set against ground-truth semantic intent classes to verify active slot recruitment and above-chance maneuver specialization.
  • (Reviewer KoaX) Added a new Qualitative Evaluation section (Section 4.4, Figure 3) visualizing real JTA test-set trajectory rollouts comparing coordinate-only AutoBots, continuous pose concatenation (AutoBots + Social-Pose), and AutoBots + Bio-TAD across sharp turning maneuvers and a boundary failure case.
  • (Reviewer KoaX) Expanded the architectural component ablation in Appendix A.4 (Table 8) with a new Static Waypoint + AdaLN variant to isolate the contribution of dynamic step-tt terminal waypoint extraction from AdaLN conditioning, and refined Section 4.5 and Appendix A.1 to clarify the geometric vs. semantic roles of continuous pose fusion and discrete prototype quantization (Table 9, Appendix A.6).
  • (Reviewer KoaX, Reviewer pUJf) Expanded Section 4.2 to clarify the dataset-wide formulation of Intent Rate (IR) across both multimodal (K=20K = 20) and deterministic (K=1K = 1) evaluations and its complementary relationship with geometric metrics and Shannon Entropy.
  • (Reviewer KoaX, Reviewer pUJf) Corrected the Appendix table transcription errors in Table 7 (Appendix A.3) (updating the Social-LSTM IR columns to the canonical test-split evaluation) and Table 13 (Appendix A.7) (replacing the duplicated trajectory-only baseline cells with the JRDB test-split values).
  • (Reviewer pUJf) Evaluated the standalone Vanilla Bio-TAD architecture (using our native spatial transformer encoder without an external baseline backbone), clarified the uniform backbone integration handoff (d = 256), and added the multimodal (K=20K = 20) evaluations for the native pose-integrated Social-Transmotion (Traj + Pose) baseline on the JTA and Urban datasets in Table 3.
  • (Reviewer pUJf) Implemented and evaluated the recent trajectory predictor MoFlow (both coordinate-only and augmented with Bio-TAD) on the JTA dataset, updating Section 4.1, Table 3, and Table 12 (Appendix A.7).
AuthorsOpenReview ↗

Overview of reviewer-author discussion #1

Dear Area Chair,

We sincerely thank you for your time, and effort in managing the review process. We also deeply appreciate the reviewers’ thoughtful critiques and constructive engagement throughout the discussion period, which helped us substantially strengthen the manuscript. Below, we summarize the key strengths recognized by the reviewers, followed by an overview of how we addressed all major concerns, including the additional experiments and clarifications provided during the multi-round discussion.

Strengths Recognized by the Reviewers

  • (Reviewers TgCX, KoaX, pUJf) All reviewers recognized that leveraging biomechanical body posture to anticipate abrupt, stochastic pedestrian turns, before macro-spatial center-of-mass displacement occurs, is well motivated, physically intuitive, and addresses an important blind spot in coordinate-only trajectory predictors.
  • (Reviewer TgCX) Highlighted the strong empirical validation of our discretization design via ablations and the framework’s backbone-agnostic architectural generalization across mainstream SOTA baselines.
  • (Reviewers KoaX, pUJf) Appreciated that our evaluation spans three complementary benchmarks (synthetic JTA, 2D-pose JRDB, and 3D-pose Urban), covers diverse backbones, and includes direct comparisons with continuous pose-fusion methods and informative ablation studies.
  • (Reviewer pUJf) Noted that the paper provided a substantially deeper understanding of the problem (Informativeness: 3), and (Reviewer TgCX) noted clear presentation with no substantive soundness concerns (Presentation: 3, Soundness: 3).

Summary of How We Addressed Reviewers' Concerns and Questions

Reviewer TgCX

  • Expanded Related Work (Sections 2.1 & 2.2) & Literature Positioning: Contextualized Bio-TAD against Memo-Net (macroscopic trajectory memory vs. our microscopic kinematic prototypes), T2M-GPT (reconstruction-oriented VQ codebooks vs. orthogonally regularized behavioral anchors), TBIFormer (continuous feature concatenation vs. discrete manifold constraints), and DADA (macro-level domain adaptation vs. intra-domain pre-movement cues).
  • Robustness to Noisy 3D Pose Estimations (Appendix A.8, Table 10): Evaluated AutoBots + Bio-TAD on JTA under zero-mean 3D Gaussian joint jitter (σ=0.05 m\sigma = 0.05\text{ m} and σ=0.10 m\sigma = 0.10\text{ m}). Even under severe jitter (σ=0.10 m\sigma = 0.10\text{ m}), Bio-TAD degrades gracefully (0.56/1.13 m→0.62/1.25 m0.56/1.13\text{ m} \to 0.62/1.25\text{ m}; Far Left IR 53.12%→47.91%53.12\% \to 47.91\%) and still substantially outperforms coordinate-only AutoBots (0.72/1.50 m0.72/1.50\text{ m}; 29.12%29.12\% IR), supported by Gumbel-Softmax discrete routing and the input-adaptive Asymmetric Cross-Attention Gate (α\alpha).
  • Reviewer TgCX confirmed that our response resolved their major concerns and maintained their Weak Acceptance recommendation.
AuthorsOpenReview ↗

Overview of reviewer-author discussion #2

Reviewer KoaX

  • Memory Slot Utilization & Behavioral Specialization (Appendix A.1, Tables 4 & 5): Showed that expanding the memory bank (64→128→25664 \to 128 \to 256 slots) monotonically increases Shannon Entropy (0.65→1.12→1.350.65 \to 1.12 \to 1.35) and sharp-turn IR (46.1%→50.2%→52.47%46.1\% \to 50.2\% \to 52.47\%), whereas removing orthogonality (Table 9) causes partial slot collapse. Added Table 5, mapping Gumbel-Softmax routing frequencies against ground-truth intent classes to verify 100% slot utilization (S=64S = 64, ∣cos⁡θ∣=0.0500|\cos\theta| = 0.0500) and above-chance maneuver specialization (peak selectivity 0.28–0.390.28\text{–}0.39 vs. 0.07–0.100.07\text{–}0.10 random).
  • Qualitative Rollouts & Failure Analysis (Section 4.4, Figure 3): Added real JTA test-set trajectory rollouts (K=20K = 20 fans) comparing coordinate-only AutoBots, continuous AutoBots + Social-Pose, and AutoBots + Bio-TAD across sharp turning maneuvers alongside an explicit boundary failure case analysis.
  • Disentangling Dynamic Waypoint Extraction & Text Corrections (Section 4.5, Appendix A.1, Tables 8 & 9): Added a new Static Waypoint + AdaLN baseline to Table 8, isolating the dynamic step-tt waypoint mechanism (minFDE₂₀: 1.22 m→1.13 m1.22\text{ m} \to 1.13\text{ m}; Far Left IR: 48.65%→53.12%48.65\% \to 53.12\%). Corrected overstated phrasing regarding continuous pose fusion in Section 4.5 and Appendix A.1, and clarified fixed experimental controls across Table 9.
  • Intent Rate (IR) Formulation & Table Transcription Fixes (Section 4.2, Tables 7 & 13): Clarified dataset-wide IR and Shannon Entropy definitions, explicitly noting that IR is evaluated jointly with minADE₂₀/minFDE₂₀ and entropy. Corrected the two transcription errors caught by the reviewer: replaced the 16 duplicated JTA trajectory-only cells in Table 13 (JRDB) with true JRDB values and updated the Social-LSTM IR columns in Table 7 from validation-split logs to the canonical test-split evaluation (Table 12).
  • Follow-up Responses after Score Increase (1→21 \to 2): After Reviewer KoaX raised their score from 1 to 2 and noted remaining concerns on recency of baselines, per-input plausibility of K=20K = 20 samples, and methodological novelty, we provided three targeted follow-ups:
    1. Completed full MoFlow (CVPR 2025) + Bio-TAD evaluations across all three benchmarks (Table 3, Tables 12–14), improving JTA (0.60/1.36→0.52/1.03 m0.60/1.36 \to 0.52/1.03\text{ m}), JRDB (0.39/0.76→0.34/0.70 m0.39/0.76 \to 0.34/0.70\text{ m}), and Urban (0.37/0.66→0.35/0.62 m0.37/0.66 \to 0.35/0.62\text{ m}).
    2. Explicitly acknowledged the single-realization benchmark limitation (Sections 4.2 & 5) and triangulated per-input plausibility via non-monotonic entropy vs. IR (Table 4), unselected deterministic K=1K = 1 gains across all seven backbones (Tables 2, 3, 13), per-input uncertainty gating α\alpha (Tables 8, 10), and coherent K=20K = 20 rollout corridors (Figure 3a–c).
    3. Clarified our novelty as coupling microscopic orthogonal biomechanical discretization (Eqs. 1–3) with iterative macroscopic step-tt dynamic waypoint anchoring inside AdaLN (Section 3.6).

Continued (2/3)......

AuthorsOpenReview ↗

Overview of reviewer-author discussion #3

Reviewer pUJf

  • Standalone Bio-TAD & Uniform Backbone Integration: Evaluated standalone Vanilla Bio-TAD using its native spatial transformer encoder (0.58 m0.58\text{ m} minADE₂₀, 1.18 m1.18\text{ m} minFDE₂₀, 51.5%/52.1%51.5\% / 52.1\% Far Left/Right IR) and documented the uniform backbone handoff (spatial hidden states projected to d=256d = 256 into a fixed Bio-TAD pipeline).
  • Pose-Integrated Social-Transmotion & Multimodal Adaptation (Tables 2 & 3): Clarified our three-way comparison (Traj-only, Base + Native Pose, Base + Bio-TAD) and trained a CVAE-adapted multimodal (K=20K = 20) version of native pose-integrated Social-Transmotion on JTA and Urban for Table 3.
  • Consistency, IR Selection, & Contemporary Baselines: Fixed the Table 7/Table 12 Social-LSTM IR discrepancy, justified focusing on the four stochastic turning categories (vs. linear/static categories where momentum saturates accuracy), defined K=1K = 1 IR as Top-1 semantic accuracy (Section 4.2), and added full three-benchmark MoFlow (CVPR 2025) results (Table 3, Tables 12–14; noting that Liu et al., AAAI 2026 currently lacks runnable evaluation code beyond an undocumented demo).
  • Follow-up Responses (Statistical Significance, Strict DiT Isolation, & Social-Transmotion Gains):
    1. Statistical Validation on Urban (50,000 test samples): Ran 3 random seeds on AutoBots + Bio-TAD, observing minimal variance (±0.005 m\pm 0.005\text{ m} minADE₂₀, ±0.007 m\pm 0.007\text{ m} minFDE₂₀ around 0.35/0.63 m0.35 / 0.63\text{ m}), and confirmed via paired tt-tests across all 50,000 samples that Bio-TAD’s ∼7%–9%\sim 7\%\text{–}9\% relative reductions across backbones are statistically significant (p<0.05p < 0.05), alongside +11.56%+11.56\% to +12.11%+12.11\% sharp-turn IR gains (Table 14).
    2. Isolating Gains Beyond the DiT Decoder & Parameter Count: Transparently clarified Table 11 Row 1 vs. Rows 2–5 and demonstrated that with the exact 4-layer DiT decoder (∼11.37M\sim 11.37\text{M} module) held strictly fixed: (a) orthogonal regularization and waypoint anchoring drive 86.5% of total FDE reduction (Table 6: 0.68/1.45 m→0.56/1.13 m0.68/1.45\text{ m} \to 0.56/1.13\text{ m}, +15.55%+15.55\% IR); (b) discrete orthogonal prototypes drive 67.6% of FDE reduction over continuous 3D pose + DiT (Table 9: 0.66/1.38 m→0.56/1.13 m0.66/1.38\text{ m} \to 0.56/1.13\text{ m}, +13.72%+13.72\% Far Left IR); (c) upgrading visual/pose cues within the fixed DiT module (Table 11, Rows 2–5) cuts minFDE₂₀ by 19.9%19.9\% (AutoBots) and 12.9%12.9\% (Social-Transmotion); and (d) capacity-matched AutoBots (∼12.26M\sim 12.26\text{M} params, Table 7) reaches only 0.68/1.42 m0.68/1.42\text{ m} (31.80%31.80\% IR) vs. 0.56/1.13 m0.56/1.13\text{ m} (53.12%53.12\% IR).
    3. Gains Over Pose-Integrated Social-Transmotion: Showed that while dataset-wide K=1K = 1 ADE is dominated by non-turning trajectories (with each turning maneuver accounting for only 7%–10%7\%\text{–}10\% of test scenes, Appendix A.1), Bio-TAD consistently improves terminal FDE (1.80/1.81 m→1.71 m1.80/1.81\text{ m} \to 1.71\text{ m} on Social-Transmotion K=1K=1; 0.84/1.68 m0.84/1.68\text{ m} on AutoBots + Bio-TAD; 0.35/0.69 m0.35/0.69\text{ m} with 2D pose vs. 0.36/0.72 m0.36/0.72\text{ m} with estimated 3D pose on JRDB), yields 12.5% minADE₂₀ / 12.2% minFDE₂₀ reductions on JTA at K=20K = 20 (Table 3), cuts pre-turn inflection FDE by over 1.01 m1.01\text{ m} (2.934 m→1.924 m2.934\text{ m} \to 1.924\text{ m}, Figure 1), and boosts turning IR by +7.82%+7.82\% to +19.67%+19.67\% (Table 1).

Through these revisions and follow-up rounds, we have addressed every technical, statistical, baseline, and clarity concern raised during the review cycle, including completed multi-benchmark MoFlow evaluations, multi-seed statistical testing on Urban, standalone Vanilla Bio-TAD results, strictly controlled fixed-DiT ablations, pose-noise robustness, and qualitative failure analyses.

We sincerely thank you again for your time and careful consideration of our submission.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.