acceptodds
Under review as a conference paper at ICLR 2027

ParetoTail: Distributional Pareto Adaptation for Long-Tail Motion Forecasting

Abstract

Motion forecasting in naturalistic driving is inherently long-tailed, leaving rare yet safety-critical behaviors severely underrepresented. Existing approaches address this problem either by strengthening tail-sample learning or by generating additional rare behaviors, both increasing the influence of underrepresented behaviors during training. However, we observe a long-tail Pareto problem: stronger tail learning can improve rare-behavior forecasting while destabilizing predictions over the naturalistic distribution. We present ParetoTail, a framework that combines controllable tail generation with distributional Pareto adaptation. ParetoTail represents driving behavior in a continuous space of longitudinal motion, lateral motion, and interaction, and uses a conditional diffusion model to generate diverse rare trajectories in sparsely covered regions. To incorporate these behaviors without unrestricted distributional drift, we model a deficit-weighted tail correction field together with a naturalistic prediction geometry, whose interaction defines a compact tail-efficient adaptation space. Within this space, ParetoTail seeks a distributionally Pareto-dominating update when tail and naturalistic objectives admit co-improvement; otherwise, it maximizes tail improvement under explicit naturalistic-performance and prediction-drift constraints. Experiments demonstrate that ParetoTail improves forecasting on challenging long-tail behaviors while preserving strong performance on common naturalistic scenarios, with generated trajectories remaining controllable, diverse, and physically plausible.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Long-Tail Motion Forecasting, Controllable Trajectory Generation, Distributional Adaptation, Pareto Optimization, Autonomous Driving

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3m7MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper combines synthetic data generation with a constrained adaptation method for motion forecasting. A conditional diffusion model generates rare driving futures from specified longitudinal, lateral and interaction attributes. Samples are refined and checked for behavioral and physical validity, producing 3,000 synthetic trajectories. The forecaster then undergoes 1,000 adaptation updates in a small subspace built from behavior gradients and their effects on existing predictions. It seeks updates that improve both naturalistic and tail objectives; when these conflict, it limits naturalistic degradation while improving the tail objective. On nuScenes, ParetoTail reports minADE5 of 1.17 m and minFDE5 of 2.34 m. Against the same architecture trained on naturalistic data, Common error falls from 1.16 to 1.11 m and Tail error from 2.80 to 2.27 m. Evaluation includes three seeds for the training controls, component ablations and results on NGSIM and HighD. The joint improvement regime activates in 40% of updates, with both losses actually decreasing in 90% of those cases.

Critical strengths
  1. The paper evaluates both sides of its central trade-off using a fixed Common and Tail partition and the same forecasting architecture. The budget ablation is particularly informative: removing the constraint improves Tail error from 2.27 to 2.15 m but worsens Common error from 1.11 to 1.15 m. This provides concrete evidence that the preservation budget affects the intended balance. Adaptation also adds no modules or computation at inference time.

  2. The implementation is carefully worked out. Appendix C avoids dense parameter matrices through a low rank covariance representation, Jacobian products and conjugate gradient solves, making the proposed adaptation procedure concrete. The generator also uses physically meaningful controls, a differentiable decoder and explicit validity checks. Its reported validity and condition hit rates provide useful checks on whether the synthetic supervision meets the requested constraints.

Critical weaknesses

W1. ParetoTail receives 1,000 additional updates, while Naive and Grouped are evaluated at the end of their earlier training. The missing controls are plain fine-tuning from the same checkpoint on naturalistic data, naturalistic plus generated data, and a weighted combination of the two objectives. Without matched update budgets and checkpoint selection, the reported improvement cannot be cleanly attributed to Pareto adaptation.

W2. The paper cites established long-tail learners and GALTraj, the closest generation-based approach, but compares only Natural-only, Grouped and Naive under the shared architecture. It also omits standard alternatives for balancing conflicting objectives, such as MGDA, PCGrad and CAGrad. These comparisons, together with a scalarization-weight sweep, are needed to establish what the proposed machinery adds over existing approaches.

W3. Requested behaviors determine both the synthetic data and the coverage weights used during adaptation, yet their selection procedure, numerical targets and sampling proportions are not sufficiently documented. The main Common/Tail evaluation also uses the same behavioral descriptor that guides adaptation. Sensitivity to different target distributions and an independent difficulty-based partition would help establish broader relevance. Several adaptation hyperparameters are selected on the validation split used for reporting, without a clearly separate final evaluation.

W4. The constraints operate on local surrogates, with curvature estimated from probe updates. A per-step degradation budget does not bound cumulative degradation over 1,000 updates, and a positive mean-minus-standard-deviation tail margin does not ensure improvement in every behavior region. The reported success and violation rates omit the magnitude of failed updates and cumulative drift. Aggregate Common/Tail errors also leave possible regressions within individual behaviors hidden.

W5. Important details remain unclear, including the forecaster architecture and loss, anchor count, kernel bandwidth and subspace refresh schedule. The component ablations also do not fully specify the replacement rule when a component is removed. Each adaptation update requires behavior-conditioned gradients, geometry solves and numerical optimization, but no adaptation time or memory usage is reported. The unchanged inference time does not establish whether these training costs are worthwhile.

W6. The correction field subtracts the same naturalistic gradient from every behavior gradient. This leaves their centered covariance unchanged, while adding the naturalistic gradient back recovers the ordinary deficit-weighted mean gradient. The remaining construction combines a prediction-space metric, a generalized eigenspace and constrained multi-objective optimization. This is a concrete design, but the paper does not clearly identify the new insight that makes this particular combination necessary or preferable.

Questions for authors

Q1. Starting from the same Natural-only checkpoint, please compare 1,000 updates of plain fine-tuning, fine-tuning with the same synthetic pool, a weighted LN+λLDL_N+\lambda L_D objective with λ swept, and MGDA or CAGrad. Please report Overall, Common and Tail errors over three seeds with matched checkpoint selection. Does ParetoTail retain an advantage once additional training and objective balancing are controlled?

Q2. How were the external baseline numbers in Table 3 obtained: reruns, released predictions or published results? Please provide a paired comparison on a fixed difficulty subset defined by Natural-only or a Kalman predictor, and include an established long-tail learner and GALTraj under matched conditions. This would test whether the gains extend beyond the paper’s own controls and behavioral partition.

Q3. How were the requested behavior categories, numerical targets and sampling proportions chosen? Were they fixed using training data alone? Were adaptation hyperparameters selected on the same validation instances used for reporting? Please state the full selection protocol, whether baselines received comparable tuning, and whether an independent final evaluation is available.

Q4. What replaces the update rule when the correction field or two-regime switch is removed in Table A.2? Were the 10% of Regime I updates that failed to improve both losses applied or rejected, and how large were the failures? Please also report cumulative degradation relative to the initial predictor and whether aggregate Tail improvement hides regressions in individual behavior regions.

Q5. Please specify the forecaster architecture and loss, anchor count, kernel bandwidth and subspace refresh schedule. What are the measured wall-clock time and peak memory for adaptation, including all 1,000 updates? How do these compare with base training and ordinary fine-tuning, and what accuracy benefit justifies the additional cost?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 3m7M (Part 1/3)

Thank you for recognizing the motivation and technical contributions of our work, and for your detailed feedback and constructive suggestions.

1. Fair Comparison (W1, Q1).

To isolate the contribution of Pareto adaptation from additional training, we conduct comprehensive comparisons starting from the same pretrained Natural-only checkpoint, with 1,000 subsequent parameter updates and a consistent checkpoint-selection rule. The baselines include naturalistic fine-tuning (FT-Natural), density-grouped fine-tuning (FT-Grouped), direct synthetic-data augmentation (FT-Naive), weighted-objective fine-tuning (Weighted FT), and MGDA. FT-Naive, Weighted FT, and ParetoTail use the same naturalistic data and frozen pool of 3,000 generated trajectories. We further sweep λ∈{0.1,0.3,1,3,10}\lambda\in\{0.1,0.3,1,3,10\} for Weighted FT.

As shown in Table R1 (revised Table 4), ParetoTail achieves the best mean Overall/Common/Tail minADE$_5$ of 1.17/1.11/2.27 m. Compared with the strongest Tail-performing weighted baseline (λ=3\lambda=3), ParetoTail reduces Common and Tail errors by 9.0% and 9.2%, respectively. These results demonstrate the effectiveness of the proposed adaptation strategy beyond continued training and conventional multi-objective optimization.

Table R1. Comparison of adaptation strategies on nuScenes after 1,000 updates. Results are minADE$_5$ (m), reported as mean ±\pm SD over three runs.

MethodOverallCommonTail
Natural-only (Initial)1.25 ± 0.011.16±0.02‾\underline{1.16 \pm 0.02}2.80 ± 0.09
FT-Natural1.24±0.02‾\underline{1.24 \pm 0.02}1.16±0.02‾\underline{1.16 \pm 0.02}2.78 ± 0.11
FT-Grouped1.26 ± 0.031.18 ± 0.022.76 ± 0.07
FT-Naive1.24±0.02‾\underline{1.24 \pm 0.02}1.17 ± 0.032.65 ± 0.10
Weighted FT (λ=0.1\lambda=0.1)1.25 ± 0.021.17 ± 0.022.73 ± 0.08
Weighted FT (λ=0.3\lambda=0.3)1.25 ± 0.011.18 ± 0.032.67 ± 0.12
Weighted FT (λ=1\lambda=1)1.27 ± 0.031.20 ± 0.022.58 ± 0.06
Weighted FT (λ=3\lambda=3)1.28 ± 0.031.22 ± 0.042.50±0.10‾\underline{2.50 \pm 0.10}
Weighted FT (λ=10\lambda=10)1.35 ± 0.051.29 ± 0.052.57 ± 0.15
MGDA1.30 ± 0.041.23 ± 0.032.56 ± 0.09
ParetoTail1.17 ± 0.011.11 ± 0.022.27 ± 0.08

2. Existing Methods and Generalization (W2, Q2).

(1) Existing Methods (W2, Q2). The external baselines in Table 3 were reproduced using their publicly released implementations. These comparisons establish the competitiveness of ParetoTail against existing motion forecasting methods.

To further examine generalization across forecasting architectures, we apply ParetoTail to the MTR backbone. As shown in Table R2, ParetoTail reduces Tail minADE$_5$ by 9.0% compared with weighted fine-tuning, while simultaneously improving Common and Overall performance. These results demonstrate that the proposed adaptation strategy can effectively transfer to another forecasting backbone.

Table R2. Comparison of adaptation strategies on the MTR backbone using nuScenes. Results are minADE$_5$ (m).

MethodOverallCommonTail
MTR (Natural-only)1.20‾\underline{1.20}1.12‾\underline{1.12}2.72
MTR + Weighted FT1.211.142.55‾\underline{2.55}
MTR + ParetoTail1.171.112.32

(2) Paired Evaluation (W2, Q2). We further rank test samples by the prediction errors of Natural-only and evaluate all methods on identical difficulty-defined subsets. As shown in Figure 4 (revised manuscript), ParetoTail achieves pronounced improvements on the most challenging cases across the hardest 1%–100% evaluation curve. This complements Table 3 by directly comparing forecasting performance on the same challenging samples.

3. Request Design and Model Selection (W3, Q3).

The generation requests are constructed exclusively from the naturalistic training data, covering ten rare-event categories across longitudinal, lateral, and interaction behaviors. Numerical targets are selected from the 75th, 90th, and 95th percentiles of the corresponding training attributes, with a normalized tolerance of 0.15. The sampling proportions are set to 30%, 35%, and 35% for longitudinal, lateral, and interaction behaviors, respectively. All generation conditions are fixed using training data alone, independently of the evaluation set.

For model selection, the adaptation hyperparameters rr, κ\kappa, ϵN\epsilon_N, ωN\omega_N, and ωT\omega_T are tuned on the validation set, while final performance is evaluated on the separate test set. For Weighted FT, we consider five coefficients λ∈{0.1,0.3,1,3,10}\lambda\in\{0.1,0.3,1,3,10\} under the same continuation and checkpoint-selection protocol. This separation ensures that the reported test results are not used for hyperparameter selection.

AuthorsOpenReview ↗

Response to Reviewer 3m7M (Part 2/3)

4. Local Adaptation Reliability (W4, Q4).

(1) Ablation Settings (W4, Q4). Removing the Correction Field retains the mean tail direction but excludes heterogeneous corrections (R=0R=0, ΣT=0\Sigma_T=0); removing the Two-Regime Switch applies Regime II throughout adaptation (revised Table 5). These ablations isolate heterogeneous corrections and regime selection.

(2) Update Reliability (W4, Q4). Across three runs, Table R3 shows that 90.0% of Regime I decisions reduce both naturalistic and tail losses, while only 1.0% of Regime II decisions violate the naturalistic budget. This supports the reliability of the two-regime optimization in identifying compatible updates and controlling naturalistic degradation.

Table R3. Reliability of local Pareto adaptation across three independent runs.

RunRegime I DecisionsJoint ImprovementRegime II DecisionsBudget Violations
Run 1400360 (90.0%)6006 (1.0%)
Run 2400359 (89.8%)6006 (1.0%)
Run 3400361 (90.3%)6006 (1.0%)
Overall1,2001,080 (90.0%)1,80018 (1.0%)

(3) Cumulative Performance (W4, Q4). To further assess whether local adaptation leads to cumulative performance degradation, we track naturalistic and tail losses relative to the initial predictor throughout the 1,000 updates. As shown in Table R4, the cumulative Common and Tail losses decrease by 0.70% and 8.20%, respectively. Despite minor fluctuations during adaptation, both objectives improve at the final checkpoint, indicating that the proposed method maintains a favorable Common–Tail balance over the complete adaptation process.

Table R4. Cumulative relative loss changes during ParetoTail adaptation. Negative values indicate loss reduction.

Adaptation StepCommonTail
00.00%0.00%
200-0.18%-2.10%
400-0.32%-3.80%
600-0.25%-5.10%
800-0.45%-6.70%
1,000-0.70%-8.20%

(4) Behavioral Generalization. To further examine the distribution of improvements across different rare behaviors, we evaluate performance across five behavioral groups. As shown in Table R5, ParetoTail achieves substantial Tail improvements in four of the five groups, with reductions of up to 8.50% for mild lateral behaviors and 7.20% for longitudinal behaviors. Interaction-heavy and aggressive lateral behaviors also benefit consistently. Composite behaviors, which involve multiple simultaneous rare attributes, remain more challenging, with a relatively small Tail loss increase of 0.80%. Overall, these results highlight the effectiveness of ParetoTail across diverse long-tail behaviors while identifying composite scenarios as a promising direction for further refinement.

Table R5. Relative loss changes across behavioral groups after adaptation. Negative values indicate improvement.

Behavioral GroupCommonTail
Longitudinal-0.40%-7.20%
Mild lateral-0.65%-8.50%
Aggressive lateral+0.20%-5.30%
Interaction-heavy+0.35%-3.10%
Composite behavior+0.55%+0.80%

5. Implementation and Efficiency (W5, Q5).

(1) Implementation Details (W5, Q5). Our forecaster consists of multiple standard encoder–decoder modules, with a hidden dimension of 64, dropout rate of 0.1, and 10 output modes. The training objective combines minADE$_5$ and Laplace losses with equal weights. The predictor is pretrained for 120 epochs using AdamW. For Pareto adaptation, we use a Gaussian kernel with bandwidth h=0.2h=0.2, retaining 142 of 180 candidate behavioral anchors. We employ 32 quadrature nodes and retain R=2R=2 generalized eigen-directions. Both the adaptation subspace and curvature estimates are refreshed at each update, with conjugate gradients and SLSQP used for geometry solving and constrained optimization, respectively (revised Appendix D.1).

(2) Computational Efficiency (W5, Q5). ParetoTail introduces additional computation only during offline adaptation, without modifying the forecasting architecture. As shown in Table R6, ParetoTail completes 1,000 updates in 9.89 minutes, only 1.9% longer than FT-Naive, while reducing Tail minADE$_5$ by 14.3%. It also achieves the lowest peak GPU memory of 6.23 GiB. Furthermore, its inference time remains approximately 14 ms per batch on an RTX 3090, identical to Natural-only. These results demonstrate a favorable trade-off between adaptation cost and forecasting performance.

Table R6. Computational cost of 1,000 adaptation updates.

MethodWall TimePeak Memory
FT-Natural7.75 min6.41 GiB
FT-Naive9.71‾\underline{9.71} min6.40‾\underline{6.40} GiB
MGDA16.78 min6.51 GiB
ParetoTail9.89 min6.23 GiB
AuthorsOpenReview ↗

Response to Reviewer 3m7M (Part 3/3)

6. Long-Tail Pareto Insight (W6).

The key motivation for defining the correction field c(a)=g(a)−gNc(a)=g(a)-g_N is to characterize how underrepresented behaviors seek to modify the predictor relative to ordinary naturalistic learning. Rather than treating behavior-conditioned gradients as isolated learning signals, we use gNg_N as a common reference to distinguish the baseline learning direction from tail-induced corrections. The deficit-weighted mean correction cˉ\bar{c} captures the overall adaptation demand of underrepresented behaviors, while the correction covariance ΣT\Sigma_T characterizes their heterogeneous learning directions. Subtracting the shared reference does not alter the centered covariance; its purpose is to explicitly separate naturalistic learning, average tail correction, and behavioral heterogeneity.

The central contribution lies in translating this distributional structure into effective adaptation directions. ParetoTail combines the naturalistic gradient gNg_N, the deficit-weighted tail gradient gD=gN+cˉg_D=g_N+\bar{c}, and heterogeneous correction directions selected according to the naturalistic prediction geometry GNG_N. This identifies directions that retain substantial tail-learning benefits while limiting disturbances to naturalistic predictions. The subsequent two-regime optimization exploits compatible Common–Tail improvements and explicitly controls naturalistic degradation when the objectives conflict.

The component ablations further validate this design. Removing the correction field or prediction geometry increases Tail minADE$_5$ from 2.27 m to 2.47 m and 2.50 m, respectively, while disabling the two-regime switch increases it to 2.39 m. Removing the naturalistic preservation budget reduces Tail error to 2.15 m but increases Common error from 1.11 m to 1.15 m. These results demonstrate the importance of jointly modeling heterogeneous tail corrections, naturalistic prediction geometry, and Pareto-aware adaptation.

Thank you for the additional experiments and clarifications. Matching the fine-tuning budget addresses some of my concern. My main concern remains the lack of a direct comparison with long-tail forecasting methods such as GALTraj. The motivation for the correction field is clearer, and the authors acknowledge that subtracting the same naturalistic gradient from each behavior gradient leaves their covariance unchanged. The ablation removes the directions derived from this covariance and the variance penalty together, so it remains unclear whether a simpler formulation could achieve similar gains. The authors mention a separate validation set for model selection but do not specify how it was constructed. No code link is provided for ParetoTail. So I am maintaining my original rating.

Reviewer_3JSVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the common–tail tradeoff in motion forecasting. A conditional diffusion model generates rare multi-agent futures that are checked for physical validity. ParetoTail then compares the desired behavioral coverage with the natural training distribution, estimates behavior-specific forecasting gradients, and selects a compact adaptation space based on how parameter changes affect naturalistic predictions. Within that space, a two-regime optimization seeks joint improvement when the local objectives are compatible and otherwise limits estimated naturalistic degradation. The nuScenes results reported in the paper show lower error on a fixed Tail subset without a loss on the fixed Common subset.

Critical strengths
  1. Each stage of the method has a reasonably clear motivation. In particular, defining the coverage deficit from requested behaviors rather than from the generator's accepted outputs avoids letting generator preference redefine which behaviors are considered important.
  2. The controlled comparisons on the fixed Common and Tail subsets demonstrate that ParetoTail can improve tail performance without sacrificing common-behavior accuracy. The component ablation is also largely consistent with the intended roles of the generator, prediction geometry, and naturalistic budget.
  3. The generator is evaluated beyond qualitative examples, reporting strong automatic joint validity, strict condition-hit rates, and successful manual map and interaction checks. Since generation and adaptation are offline, the final forecasting model does not add any inference-time modules.
Critical weaknesses
  1. The controlled comparisons are not budget-matched. According to Appendix D.1, ParetoTail is evaluated after 1,000 adaptation updates, while Grouped and Naive are evaluated at the end of their natural-data training procedure. The architecture and synthetic pool are controlled, but the optimization and checkpoint protocols are not. This affects how Table 5 should be read: as it stands, the result does not separate the benefit of the Pareto adaptation rule from the benefit of continued training. The most useful control would be to continue Naive and Grouped for the same 1,000 updates, with the same natural and synthetic data exposure and the same checkpoint-selection rule.

  2. The gap between requested and realized coverage is not quantified enough. The deficit is computed from requested conditions, whereas behavior-conditioned gradients use accepted natural and synthetic trajectories. Equation (43) removes anchors with too little local support. This is a reasonable safeguard, but it means some requested tail regions may not contribute to adaptation at all. Given the 80.0% strict condition-hit rate and 72.0% lowest-category rate in Table 1, the paper should show requested and accepted behavioral densities, the fraction of positive deficit discarded by support filtering, and sensitivity to the density bandwidth.

  3. The practical reach of the local Pareto analysis remains uncertain. The co-improvement test is solved in a restricted subspace using local surrogate gains, with curvature coefficients calibrated from finite probe steps. The paper correctly states this local scope, but the construction alone does not guarantee realized validation error or improvement for every under-covered behavior. Section 4.5 reports that Regime I is activated in 40.00% of updates and that 90.00% of these updates decrease both measured losses, with a Spearman correlation of 0.50 between the predicted margin and the realized joint gain; Regime II violates the naturalistic budget in 1.00% of updates. A closer account of the remaining updates and variation across seeds would clarify how reliably the local decision rule translates into actual improvement.

Questions for authors
  1. Can Grouped and Naive be continued for the same 1,000 updates, with the same natural and synthetic data exposure and the same checkpoint-selection rule as ParetoTail? If not, what prevents a budget-matched control?
  2. How closely does the accepted synthetic distribution match the requested distribution after validity filtering? What fraction of the positive coverage deficit is removed by the support threshold in Equation (43), and how sensitive are the results to the density bandwidth?
  3. Are the curvature coefficients recalibrated during adaptation? Could you show predicted and realized naturalistic and tail gains by regime and seed, so that the scope of the local Pareto claim is clear?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 3JSV (Part 1/2)

Thank you for recognizing the motivation and contributions of our work, and for your constructive feedback and suggestions.

1. Fair Comparison (W1, Q1).

To isolate the contribution of Pareto adaptation from additional training, we conduct matched continuation experiments starting from the same pretrained Natural-only checkpoint, with 1,000 parameter updates and a consistent checkpoint-selection rule. The baselines include naturalistic fine-tuning (FT-Natural), density-grouped fine-tuning (FT-Grouped), and direct synthetic-data augmentation (FT-Naive). We additionally compare with weighted-objective fine-tuning and MGDA. FT-Naive, Weighted FT, and ParetoTail use the same naturalistic dataset and frozen pool of 3,000 generated trajectories.

As shown in Table R1 (revised Table 4), ParetoTail achieves the lowest Overall/Common/Tail minADE$_5$ of 1.17/1.11/2.27 m. Compared with FT-Naive, ParetoTail further reduces Tail error by 14.3% while improving Common performance. It also outperforms the strongest Tail-performing weighted baseline (λ=3\lambda=3) on both subsets. These results demonstrate that the improvements extend beyond continued training, synthetic-data augmentation, and conventional objective balancing.

Table R1. Comparison of adaptation strategies on nuScenes after 1,000 updates. Results are minADE$_5$ (m), reported as mean ±\pm SD over three runs.

MethodOverallCommonTail
Natural-only (Initial)1.25 ± 0.011.16±0.02‾\underline{1.16 \pm 0.02}2.80 ± 0.09
FT-Natural1.24±0.02‾\underline{1.24 \pm 0.02}1.16±0.02‾\underline{1.16 \pm 0.02}2.78 ± 0.11
FT-Grouped1.26 ± 0.031.18 ± 0.022.76 ± 0.07
FT-Naive1.24±0.02‾\underline{1.24 \pm 0.02}1.17 ± 0.032.65 ± 0.10
Weighted FT (λ=0.1\lambda=0.1)1.25 ± 0.021.17 ± 0.022.73 ± 0.08
Weighted FT (λ=0.3\lambda=0.3)1.25 ± 0.011.18 ± 0.032.67 ± 0.12
Weighted FT (λ=1\lambda=1)1.27 ± 0.031.20 ± 0.022.58 ± 0.06
Weighted FT (λ=3\lambda=3)1.28 ± 0.031.22 ± 0.042.50±0.10‾\underline{2.50 \pm 0.10}
Weighted FT (λ=10\lambda=10)1.35 ± 0.051.29 ± 0.052.57 ± 0.15
MGDA1.30 ± 0.041.23 ± 0.032.56 ± 0.09
ParetoTail1.17 ± 0.011.11 ± 0.022.27 ± 0.08

2. Behavioral Coverage and Sensitivity (W2, Q2).

To quantify the discrepancy between requested and realized behavioral coverage, we provide a distributional comparison in Figure 3 of the revised manuscript. The accepted trajectories exhibit broad coverage across the three behavioral dimensions, with the longitudinal and lateral distributions following the requested targets more closely than the interaction distribution. The joint total variation distance is 0.475, with the largest marginal discrepancy occurring in the interaction dimension (0.455). Importantly, ParetoTail defines the coverage deficit from the requested distribution rather than the accepted samples, preventing generation bias from altering the intended adaptation targets.

To ensure reliable behavior-conditioned gradient estimation, anchors with insufficient sample support are excluded according to Eq. (43). Under the default threshold, 142 of 180 positive-deficit anchors are retained, preserving 82% of the positive deficit mass. We further evaluate sensitivity to the KDE bandwidth. As shown in Table R2, ParetoTail maintains relatively stable performance across the tested bandwidths, with the default setting achieving the lowest Tail error and consistently low Common errors.

Table R2. Sensitivity to KDE bandwidth on nuScenes (minADE$_5$).

BandwidthOverallCommonTail
h=0.10h=0.101.201.132.43
h=0.15h=0.151.181.122.34
h=0.20h=0.201.171.112.27
h=0.30h=0.301.191.122.42
h=0.40h=0.401.211.142.48
AuthorsOpenReview ↗

Response to Reviewer 3JSV (Part 2/2)

3. Pareto Adaptation Reliability (W3, Q3).

We recalibrate curvature at each step using symmetric finite differences on naturalistic and tail minibatches (Eq. 73). The coefficients are estimated from the 90th percentile of probe curvatures and clipped to nonnegative values. We further compare predicted and realized loss improvements under both optimization regimes.

As shown in Table R3, Regime I achieves realized naturalistic and Tail loss reductions of 0.31% and 1.52%, respectively. Regime II achieves a 1.62% Tail reduction, although its realized naturalistic loss increases slightly, reflecting the approximation error of local surrogates. The reliability diagnostics in Table R4 show 90.0% joint improvement in Regime I and a 1.0% naturalistic-budget violation rate in Regime II.

Table R3. Predicted and realized relative loss improvements. Positive values indicate loss reduction.

RegimePredicted CommonRealized CommonPredicted TailRealized Tail
Regime I+0.42%+0.31%+1.80%+1.52%
Regime II+0.10%−0.16%+1.95%+1.62%

Table R4. Adaptation reliability across three independent runs.

RunRegime I Joint ImprovementRegime II Budget Violations
Run 1360/400 (90.0%)6/600 (1.0%)
Run 2359/400 (89.8%)6/600 (1.0%)
Run 3361/400 (90.3%)6/600 (1.0%)
Overall1,080/1,200 (90.0%)18/1,800 (1.0%)

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts, particularly the budget-matched comparisons. However, the revised manuscript does not highlight the changes, making them difficult to verify.

More importantly, a fundamental concern remains: the joint TV distance between the requested and accepted behavioral distributions is still 0.475, and support filtering excludes 18% of the positive coverage-deficit mass. This raises concerns about whether the method effectively addresses the underrepresented behaviors central to its motivation.

While the additional results are helpful, this concern remains unresolved. Therefore, I will maintain my original rating.

Reviewer_HE2VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ParetoTail addresses long-tail motion forecasting through controllable trajectory generation and distributional adaptation. It constructs a compact parameter-update space from behavior-conditioned gradients and naturalistic prediction geometry, then seeks joint improvement or optimizes tail performance under preservation constraints. Experiments on nuScenes, NGSIM, and HighD report improved forecasting performance.

Critical strengths
  • Improving rare-behavior forecasting while preserving naturalistic performance is a relevant problem.
  • The framework explicitly considers heterogeneous tail corrections and their effects on the forecasting function.
  • Fixed Common–Tail evaluation and component ablations provide useful evidence beyond aggregate accuracy.
  • The adapted predictor requires no additional inference-time modules.
Critical weaknesses

W1. The experiments do not sufficiently isolate the contribution of the proposed adaptation. Appendix D.1 states that Grouped and Naive are evaluated at their training endpoint, whereas ParetoTail is evaluated after 1,000 adaptation updates. Consequently, the comparisons in Tables 4–5 do not distinguish the proposed optimization from the benefit of an additional adaptation stage. Comparable continued-training baselines, with matched data exposure, update budgets, checkpoint selection, and tuning opportunities, are needed. A tuned weighted-objective or gradient-conflict baseline using the same synthetic pool would also help establish whether the proposed formulation improves the attainable common–tail trade-off over simpler alternatives. W2. The robustness and generality of the improvements remain insufficiently established. Table 5 reports promising gains in minADE₅ and minFDE, but minADE₁₀ changes from 1.01 ± 0.01 to 1.00 ± 0.01, and MR₅ from 0.49 ± 0.01 to 0.48 ± 0.01. The paper does not adequately explain this uneven pattern or quantify uncertainty in the differences. Complementary metrics on the fixed Tail subset are particularly important for establishing whether the method reduces forecasting failures. Table 3 evaluates each model’s own highest-error instances. This characterizes error distributions but does not establish improvement on identical rare scenarios. A densely evaluated worst-fraction curve from 1% to 100%, alongside evaluation on independently defined rare-scenario groups, would make the scope of the improvement clearer. Furthermore, the controlled nuScenes experiments use one predictor. Comparisons against other forecasting methods do not establish the incremental benefit of applying ParetoTail to those methods. The additional highway results are useful, but controlled evaluation across stronger backbones and a larger forecasting benchmark remains necessary to support broad generality. W3. The coverage supplied by the route-conditioned generator is insufficiently characterized. Appendix B describes agent-specific route candidates scored using observed history and behavioral conditions. The decoder supports longitudinal acceleration and lateral velocity, so route alignment itself does not exclude braking or lateral motion. However, candidate construction, route selection, lane-change connectivity, and admissible corridor widths are insufficiently specified. Because generation includes route-support penalties and validity filtering, these choices could restrict which maneuvers enter the synthetic pool. Aggregate validity and diversity statistics do not establish coverage of alternative branches, lane changes, or combinations of rare behaviors. Maneuver-specific coverage and rejection statistics would strengthen the evidence. W4. The reliability of adaptation under sparse behavioral support needs stronger validation. The adaptation updates the forecasting network, but its update directions depend on estimated behavioral densities, local gradients, and correction covariance. Appendix C discards insufficiently supported behavioral anchors, while the implementation retains only two heterogeneous correction directions. The paper does not establish how sensitive performance is to support size, density-estimation bandwidth, anchor filtering, or subspace rank. This matters given the reported 1,610 natural-tail training instances and 453 fixed-tail validation instances. Local surrogate improvement within this estimated space also does not guarantee improvement across all rare behaviors or unseen behavioral combinations. Minor presentation issue Define Grouped and Naive clearly in the main text and distinguish the training-strategy controls in Tables 4–5 from the component ablations in Table A.2. Map these experiments to Figure 2 and specify the replacement procedure for each disabled component. The organization alone is not a decisive weakness.

Questions for authors
  1. Do the gains persist under matched continuation and checkpoint-selection procedures, including a tuned weighted-objective baseline?
  2. Do complementary fixed-tail metrics and another strong forecasting backbone show consistent improvements?
  3. How completely does the generator cover lane changes and alternative routes, and which requested behaviors are rejected?
  4. How stable are the results as behavioral support, density-estimation settings, and adaptation-space rank change?
2 comments
AuthorsOpenReview ↗

Response to Reviewer HE2V (Part 1/2)

Thank you for recognizing the motivation and value of our work, and for your thoughtful feedback and constructive suggestions.

1. Fair Comparison (W1, Q1).

To distinguish the benefits of Pareto adaptation from additional training and conventional objective balancing, we conduct comprehensive comparisons starting from the same pretrained Natural-only checkpoint, with 1,000 subsequent updates and a consistent checkpoint-selection protocol across three independent runs. The baselines include naturalistic fine-tuning (FT-Natural), density-grouped fine-tuning (FT-Grouped), direct synthetic-data augmentation (FT-Naive), weighted-objective fine-tuning (Weighted FT), and MGDA. FT-Naive, Weighted FT, and ParetoTail use the same naturalistic data and frozen pool of 3,000 generated trajectories. Weighted FT optimizes LN+λLT\mathcal{L}_N+\lambda\mathcal{L}_T, with λ∈{0.1,0.3,1,3,10}\lambda\in\{0.1,0.3,1,3,10\}.

As shown in Table R1 (revised Table 4), ParetoTail achieves the best Overall/Common/Tail minADE$_5$ of 1.17/1.11/2.27 m. Compared with FT-Naive, ParetoTail reduces Tail error by 14.3% while improving Common performance. Against the strongest Tail-performing weighted baseline (λ=3\lambda=3), it further reduces Common and Tail errors by 9.0% and 9.2%, respectively. The three-run results additionally characterize performance variability, demonstrating the effectiveness of Pareto adaptation beyond continued training, synthetic-data augmentation, and conventional objective balancing.

Table R1. Comparison of adaptation strategies on nuScenes after 1,000 updates. Results are minADE$_5$ (m), reported as mean ±\pm SD over three runs.

MethodOverallCommonTail
Natural-only (Initial)1.25 ± 0.011.16±0.02‾\underline{1.16 \pm 0.02}2.80 ± 0.09
FT-Natural1.24±0.02‾\underline{1.24 \pm 0.02}1.16±0.02‾\underline{1.16 \pm 0.02}2.78 ± 0.11
FT-Grouped1.26 ± 0.031.18 ± 0.022.76 ± 0.07
FT-Naive1.24±0.02‾\underline{1.24 \pm 0.02}1.17 ± 0.032.65 ± 0.10
Weighted FT (λ=0.1\lambda=0.1)1.25 ± 0.021.17 ± 0.022.73 ± 0.08
Weighted FT (λ=0.3\lambda=0.3)1.25 ± 0.011.18 ± 0.032.67 ± 0.12
Weighted FT (λ=1\lambda=1)1.27 ± 0.031.20 ± 0.022.58 ± 0.06
Weighted FT (λ=3\lambda=3)1.28 ± 0.031.22 ± 0.042.50±0.10‾\underline{2.50 \pm 0.10}
Weighted FT (λ=10\lambda=10)1.35 ± 0.051.29 ± 0.052.57 ± 0.15
MGDA1.30 ± 0.041.23 ± 0.032.56 ± 0.09
ParetoTail1.17 ± 0.011.11 ± 0.022.27 ± 0.08

2. Robustness and Generalization (W2, Q2).

(1) Tail Performance (W2, Q2). The fixed Tail subset contains 453 of 9,041 test instances. As shown in Table R2, ParetoTail reduces minADE$_5$ by 18.9% (2.80 to 2.27 m), minFDE$_5$ by 15.5% (5.50 to 4.65 m), and MR$5$ from 72% to 65%, while also improving minADE${10}$. Since Tail cases represent only 5% of the evaluation set, these substantial improvements can coexist with smaller changes in aggregate metrics. This demonstrates the effectiveness of ParetoTail in mitigating severe forecasting errors on underrepresented behaviors.

Table R2. Forecasting performance on the fixed Tail subset of nuScenes.

MethodminADE$_5$minFDE$_5$MR$_5$minADE$_{10}$
Natural-only2.805.500.722.50
FT-Grouped2.765.420.712.45
FT-Naive2.655.250.702.43
Weighted FT (λ=3\lambda=3)2.50‾\underline{2.50}4.91‾\underline{4.91}0.67‾\underline{0.67}2.28‾\underline{2.28}
MGDA2.565.020.682.32
ParetoTail2.274.650.652.20

(2) Hard-case Evaluation (W2, Q2). To ensure a fair comparison on identical challenging scenarios, we rank test samples by the prediction errors of Natural-only and evaluate all methods on the same subsets. As shown in Figure 4, the performance curves spanning the hardest 1% to 100% of samples demonstrate that ParetoTail achieves particularly pronounced improvements on the most challenging cases, further highlighting its effectiveness in mitigating severe forecasting errors.

(3) Backbone Generalization (W2, Q2). To examine generalization across forecasting architectures, we further apply ParetoTail to the MTR backbone. As shown in Table R3, ParetoTail reduces Tail minADE$_5$ by 9.0% compared with weighted fine-tuning, while simultaneously improving Common and Overall performance. These results demonstrate the transferability of the proposed adaptation strategy across different forecasting backbones.

Table R3. Forecasting performance with the MTR backbone on nuScenes. Results are minADE$_5$ (m).

MethodOverallCommonTail
MTR (Natural-only)1.20‾\underline{1.20}1.12‾\underline{1.12}2.72
MTR + Weighted FT1.211.142.55‾\underline{2.55}
MTR + ParetoTail1.171.112.32
AuthorsOpenReview ↗

Response to Reviewer HE2V (Part 2/2)

3. Generator Coverage (W3, Q3).

(1) Route Conditioning (W3, Q3). The generator constructs map-compatible route candidates from connected lane segments and alternative branches, selecting routes according to observed histories and requested behaviors. The route-aligned decoder predicts longitudinal acceleration and lateral velocity, enabling braking, turning, and lane-changing maneuvers. Generated trajectories are subsequently filtered according to behavioral conditions, road feasibility, dynamic constraints, collision avoidance, and interaction consistency.

(2) Behavioral Coverage (W3, Q3). To characterize behavioral coverage after validity filtering, we compare the requested and realized distributions across longitudinal, lateral, and interaction dimensions (revised Figure 3). The accepted trajectories retain a range of behavioral intensities across all three dimensions, with substantial variation in longitudinal and lateral behaviors. This comparison further illustrates how validity filtering shapes the realized behavioral coverage of the synthetic pool.

4. Sensitivity Analysis (W4, Q4).

We examine the sensitivity of ParetoTail to Tail training support, KDE bandwidth, anchor filtering, and subspace rank. As shown in Table R4, the default configuration achieves the lowest Tail error, while Common minADE$_5$ remains within 1.11–1.15 m across all settings.

Reducing Tail support or applying overly strict anchor filtering leads to more pronounced Tail degradation, highlighting the importance of sufficient behavioral coverage. At the default threshold, 142 of 180 positive-deficit anchors are retained, preserving 82% of the deficit mass. These results characterize the robustness of ParetoTail and identify the factors most critical to Tail performance.

Table R4. Sensitivity to Tail support, bandwidth, anchor filtering, and subspace rank. Results are minADE$_5$ (m).

FactorSettingOverallCommonTail
Tail support25% (403)1.221.152.55
50% (805)1.201.132.48
100% (1,610)1.171.112.27
Bandwidth0.5h0.5h1.201.132.43
0.75h0.75h1.181.122.34
hh1.171.112.27
1.5h1.5h1.191.122.42
2h2h1.211.142.48
Anchor threshold0.5×0.5\times1.181.122.34
1×1\times1.171.112.27
2×2\times1.221.152.55
Subspace rankR=1R=11.211.142.48
R=2R=21.171.112.27
R=4R=41.191.132.40

5. Experimental Clarifications (Minor).

Grouped applies density-based grouping to naturalistic training samples, while Naive directly incorporates generated tail trajectories without Pareto adaptation. We also clarify the distinction between training-strategy comparisons and component ablations, together with the corresponding ablation settings.

Comments on the paper

1 comment
Area_Chair_jQUMOpenReview ↗

Reminder: Please Review Authors’ Rebuttal and Provide Feedback

Dear Reviewers,

Thank you for your reviews. If you have not already done so, please read the authors' rebuttal and share your feedback in the discussion thread at your earliest convenience. Your timely responses would be greatly appreciated.

Best,
AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.