- Rating
- 3
- Confidence
- 1
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes HSP (Human-Social Prospect learning), a unified framework for structured behavioral utility learning in interactive autonomous driving. The main idea is to represent surrounding agents through continuous behavioral objective coordinates over four axes: safety, comfort, efficiency, and social accommodation. These coordinates are inferred from observed motion and are used throughout the pipeline: they condition multimodal trajectory prediction, define an energy for gradient-based trajectory refinement (EDTO), and shape agent-conditioned interaction utilities for ego planning.
The planning module combines an ego utility and a social utility through a continuous Social Value Orientation (SVO) angle. It further introduces a Prospect Value Model, which maps normalized interaction utility to decision value via a monotonic Bernstein polynomial with context-adaptive control points. The authors argue that this addresses the “prediction–utility gap”: geometric prediction accuracy alone does not fully capture decision relevance.
The paper evaluates HSP on five benchmarks: nuScenes, NGSIM, HighD for prediction; nuPlan and CommonRoad for planning. Results show strong prediction and closed-loop planning performance. Ablations compare structured behavioral coordinates against learned latent and shuffled coordinates, and compare linear utility, monotonic MLP, and the proposed Prospect Value Model. Additional analyses include matched-accuracy planning comparison, counterfactual interaction analysis, computational efficiency, and qualitative visualizations.
Critical strengths
-
A coherent and operational structured behavioral representation. The continuous behavioral objective coordinates are not just auxiliary features; they are used to condition prediction, define an energy for trajectory refinement, and construct interaction utilities for planning. This is a meaningful attempt to bridge the prediction–planning interface with a shared representation. The structure-controlled ablation in Table 3 supports the claim that the benefit is not merely from adding latent capacity: learned latent coordinates and shuffled coordinates both perform worse. This is consequential because it suggests that the structured objective basis and agent-specific coordinates matter.
- Strong empirical evaluation and useful matched-accuracy analysis. The paper evaluates on five benchmarks and reports competitive or best results on nuScenes, NGSIM, HighD, nuPlan, and CommonRoad. More importantly, the matched-accuracy analysis in Fig. 3 provides direct evidence that planning gains can persist even when geometric prediction accuracy is controlled. This is a valuable contribution to the broader discussion of prediction utility versus prediction accuracy. The counterfactual merge scenario further illustrates how changes in observed motion can shift inferred coordinates and lead to coherent planning responses.
Critical weaknesses
-
The Prospect Value Model is only loosely connected to prospect theory. The paper motivates the value model using prospect theory, but the implemented function is a monotonic Bernstein polynomial with positive increments. This construction guarantees monotonicity and allows curvature variation, but it does not explicitly include reference dependence, loss aversion, or probability weighting---the central ingredients of prospect theory. The learned curves in Fig.~2 are interesting, but without a formal test of prospect-theoretic properties, the claim that the model captures ``prospect-like'' valuation remains largely interpretive. This is consequential because the paper's novelty partly rests on the structured nonlinear valuation mechanism, and the current evidence does not fully distinguish it from a flexible monotonic nonlinearity with context conditioning.
-
Limited isolation of the proposed valuation mechanism from additional conditioning and capacity. The ablation in Table 4 compares linear valuation, a generic monotonic MLP, and the Prospect Value Model. The Prospect Value Model performs best, but the hyper-network also generates context-dependent SVO angles and value-function parameters. It is therefore unclear whether the gains come from the Bernstein parameterization, the context-adaptive conditioning, or the additional capacity of the hyper-network. A more controlled ablation would fix the SVO angle and value-function parameters across contexts, or compare against a context-conditioned monotonic MLP with comparable capacity. Without this, the claim that structured context-dependent nonlinear valuation is the key ingredient is not fully established.
-
Some key design choices and reproducibility details are underspecified. The main text introduces several important components---EDTO, SVO utilities, Prospect Value Model, context-adaptive value planning---but many exact definitions and training details are only in the appendix. Even there, some details remain brief. For example, the normalization of interaction utility is not fully specified in the main text; the architecture of the hyper-network is only sketched; and the exact procedure for selecting the most likely predicted trajectory is not described in detail. These omissions make it harder to assess reproducibility and to compare the proposed method fairly against baselines.
-
The prediction--utility analysis could be strengthened. The matched-accuracy analysis controls only for ADE. However, planning performance may depend on other prediction properties, such as final displacement error, miss rate, mode diversity, or collision rate. A stronger analysis would match on multiple prediction metrics or report the distribution of prediction quality after matching. This would make the claim that ``geometric forecasting accuracy does not fully characterize decision relevance'' more convincing.
-
Questions for authors
-
Prospect-theoretic properties: Can you clarify which specific properties of prospect theory the Prospect Value Model is intended to capture? Does the learned value function exhibit reference dependence, loss aversion, or probability weighting, or is the connection primarily motivational? If the latter, would it be more accurate to describe the model as a structured context-adaptive monotonic nonlinear valuation function rather than a prospect-theoretic one?
-
Isolation of valuation mechanism: How much of the planning gain in Table 4 comes from the Bernstein parameterization itself versus the context-adaptive hyper-network? Could you provide an ablation with a context-conditioned monotonic MLP of comparable capacity, or with fixed SVO and value-function parameters across contexts?
-
Matched-accuracy analysis: The matched-accuracy criterion uses . Could you also report matched results controlling for minFDE, miss rate, or collision rate? Would the planning-score gains persist under a stricter or multi-metric matching criterion?
-
EDTO refinement: The paper claims that EDTO improves prediction quality without degrading feasibility. Table A.8 shows small changes in off-road rate, collision rate, and RMS jerk. Could you provide more detail on how feasibility is enforced during gradient-based refinement? Are there cases where EDTO produces physically infeasible trajectories, and how are they handled?
-
SVO and value-function identifiability: The SVO angle and value-function parameters are generated by a hyper-network conditioned on ego and interaction features. How do you ensure that these parameters are interpretable as social value orientation and valuation curvature, rather than arbitrary context-dependent latent variables? Is there any regularization or supervision that encourages the intended interpretation?
-
Baseline fairness: HSP uses a shared scene encoder, behavioral coordinates, EDTO refinement, and a prospect value model. Are the baselines trained with comparable capacity and comparable input information? In particular, do the baselines also receive the behavioral objective coordinates or an equivalent structured representation?
-
Computational efficiency: The paper reports 12.9 G FLOPs and 19.0 M parameters for planning. How does this compare with the strongest baselines, such as CarPlanner or GameFormer? Is the performance gain achieved at a similar computational cost?
-
3 comments
Response to Reviewer HFWQ (Part 2/3)
3. Implementation Details and Reproducibility (W3).
-
Utility Normalization. Ego and social utilities are normalized using fixed training-set statistics, followed by SVO-weight normalization to obtain a bounded PVM input (revised Sec. 3.4.1, Eqs. (14)–(16)).
-
Hyper-network Architecture. The hyper-network is defined as , where , generating the SVO angle and Bernstein control-point increments (Appendix A.7, Eq. (60)).
-
Trajectory Selection. For each neighboring agent, we select the highest-probability prediction mode using and to construct its agent-conditioned interaction field (Sec. 3.4.1, Eq. (12)).
-
EDTO Implementation. EDTO performs 4 unrolled gradient-based refinement iterations with step size , selected through validation-set tuning. The prediction loss is computed on the refined trajectories and backpropagated through the unrolled optimization steps. The optimization settings and training details are specified in the Appendix C.1.3.
4. Multi-Metric Matched-Accuracy Analysis (W4, Q3).
We extend the matched-accuracy analysis in Fig. 3 to multiple ADE thresholds (0.05, 0.10, and 0.20 m) and stricter joint matching criteria incorporating FDE and miss rate (MR). Table R3 reports the matched-scene counts, retention rates, and mean paired planning-score gains with 95% bootstrap confidence intervals.
Table R2. Planning-score gains under different prediction-matching criteria.
| Split | Matching Criterion | All Paired Scenes | Matched Scenes | Retained (%) | Mean Gain↑ | 95% CI |
|---|---|---|---|---|---|---|
| Val14 | m | 1076 | 86 | 7.99 | +0.15 | [+0.01, +0.29] |
| Val14 | m | 1076 | 149 | 13.85 | +0.20 | [+0.08, +0.32] |
| Val14 | m | 1076 | 356 | 33.09 | +0.17 | [+0.05, +0.29] |
| Val14 | m, m | 1076 | 112 | 10.41 | +0.22 | [+0.07, +0.37] |
| Val14 | m, m, | 1076 | 79 | 7.34 | +0.24 | [+0.04, +0.44] |
| Test14-hard | m | 264 | 14 | 5.30 | +0.27 | [+0.01, +0.49] |
| Test14-hard | m | 264 | 23 | 8.71 | +0.39 | [+0.20, +0.58] |
| Test14-hard | m | 264 | 61 | 23.11 | +0.33 | [+0.05, +0.51] |
| Test14-hard | m, m | 264 | 17 | 6.44 | +0.36 | [+0.14, +0.53] |
| Test14-hard | m, m, | 264 | 11 | 4.17 | +0.40 | [+0.10, +0.58] |
The planning gains remain positive across different ADE thresholds and stricter joint matching criteria. Even under joint ADE/FDE/MR matching, HSP achieves mean gains of +0.24 on Val14 and +0.40 on Test14-hard. These results provide further evidence that geometric prediction accuracy alone does not fully characterize downstream decision utility.
Response to Reviewer HFWQ (Part 3/3)
5. EDTO Refinement and Trajectory Feasibility (Q4).
EDTO adopts a soft, behavior-guided trajectory refinement formulation rather than enforcing strict physical feasibility constraints. This formulation preserves the flexibility of learned multimodal trajectory prediction while encouraging consistency with inferred behavioral objectives. Specifically, EDTO starts from decoder-generated trajectories and performs 4 unrolled gradient-based updates with a step size of . The prediction loss is applied to the final refined trajectories and backpropagated through the refinement process, encouraging realistic motion without requiring explicit feasibility projection.
Since EDTO does not impose hard physical constraints, we evaluate its effects on trajectory feasibility in Table 4. EDTO improves minFDE$_5$ from 2.34 to 2.26, reduces collision rate from 2.63% to 2.51%, and decreases RMS jerk from 2.58 to 2.54, while maintaining a comparable off-road rate (1.21% vs. 1.23%). To further illustrate its effects, we provide additional qualitative ablations in Appendix C.4.4 (Fig. A.2), comparing a geometry-driven baseline, HSP without EDTO, and full HSP across three interactive scenarios. Together, the quantitative and qualitative results demonstrate that EDTO improves prediction accuracy and behavioral consistency while maintaining comparable empirical trajectory feasibility.
6. SVO and Valuation Interpretability (Q5).
The SVO angle and PVM parameters are functionally constrained decision variables rather than unrestricted context-dependent latent features. Specifically, the SVO angle explicitly controls the relative contributions of normalized ego and social utilities through . The fixed utility normalization further provides a consistent scale for interpreting this trade-off. For PVM, normalized positive Bernstein increments ensure , , and strict monotonicity, while their relative variations determine the marginal value sensitivity and curvature, as characterized by Theorem 2.
These parameters are learned from expert driving demonstrations through value-alignment supervision (Appendix B.2) and planning imitation, rather than being assigned arbitrary latent targets. The value-alignment objective encourages demonstrated decisions to receive higher relative decision values, while the learned valuation parameters also condition trajectory generation. Moreover, Fig. 2 visualizes the resulting context-dependent valuation curves, and Table R1 quantifies their marginal sensitivity across interaction contexts. Together, the structural constraints, demonstration-based learning, and empirical analyses support the functional interpretability of the learned SVO and valuation parameters, without requiring direct psychological preference annotations.
7. Baseline Fairness (Q6).
Our main benchmark comparisons follow the official evaluation protocols of the corresponding datasets, using results reported in published studies or official benchmark records. All methods are evaluated on the same benchmark datasets with access to the corresponding benchmark-provided scene information, ensuring consistent evaluation conditions while retaining their respective model architectures and implementations.
To ensure fair comparisons within HSP, we conduct controlled ablations covering behavioral representations (Table 3), valuation functions (Table 5), the geometry-driven baseline (Fig. 3), and individual framework components. These comparisons retain consistent scene observations, prediction–planning architectures, and training settings where applicable, with representation dimensionality and model capacity explicitly matched in the corresponding controlled comparisons. We have expanded the experimental descriptions in the revised manuscript to clarify the configurations and control conditions of these ablations, improving the transparency and reproducibility of our comparisons.
8. Computational Efficiency (Q7).
As shown in Table A.5, HSP reduces FLOPs by 84.9% and parameters by 80.1% compared with GameFormer. Even compared with the lightweight UrbanDriver, HSP reduces FLOPs by 54.7% and parameters by 46.0%, while achieving higher planning scores. These results demonstrate that HSP's performance gains are achieved with substantially lower computational complexity, indicating a favorable performance–complexity trade-off. For CarPlanner, a comparable FLOPs and parameter analysis is unavailable without its official implementation. We therefore report its published planning performance in Table 2, but do not include it in the complexity comparison in Table A.5.
Response to Reviewer HFWQ (Part 1/3)
Thank you for your positive assessment of our work and for your careful review and helpful suggestions.
1. Prospect-Theoretic Motivation and Valuation Properties (W1, Q1).
Inspired by Cumulative Prospect Theory (CPT), our goal is to leverage its nonlinear valuation principles to improve interactive planning, while exploring whether CPT-like valuation patterns can emerge from driving interactions without explicitly prescribing reference points or loss-aversion coefficients. This motivates a fundamental distinction between interaction utility and its decision value: equal changes in utility may carry different decision significance across utility levels and interaction contexts.
To investigate this, HSP introduces PVM as a learnable, context-adaptive decision valuation mechanism. Rather than fitting a predefined value curve, PVM learns from expert driving demonstrations through value-alignment supervision, while its learned parameters directly condition trajectory generation. The normalized Bernstein parameterization preserves utility preference ordering and fixed endpoints while allowing context-dependent marginal sensitivity and curvature, as characterized by Theorem 2. Thus, nonlinear valuation is integrated into policy learning and trajectory generation rather than serving merely as a post-hoc utility transformation.
As illustrated in Fig. 2, the learned value functions exhibit S-shaped profiles and context-dependent curvature changes, with stronger low-utility sensitivity under denser interaction contexts. Importantly, these patterns emerge from learned contextual parameters rather than predefined CPT curve shapes. We further quantify the marginal value sensitivity across utility regions in Table R1, using the number of surrounding agents as a proxy for interaction density.
Table R1. Quantitative characterization of PVM under different interaction contexts on Val14.
50,000 utility–value pairs are sampled per interaction group using scene-stratified sampling. Ratios are computed before rounding.
The higher early-to-late sensitivity ratio (5.08 vs. 3.86) provides quantitative evidence of context-dependent nonlinear valuation, complementing the qualitative patterns in Fig. 2. While the current formulation does not explicitly model reference-dependent gain/loss branches, loss aversion, or probability weighting, these learned patterns provide preliminary evidence of CPT-like nonlinear valuation in interactive driving, motivating further investigation of explicit prospect-theoretic mechanisms.
2. Isolating Context-Adaptive Valuation (W2, Q2).
(1) Controlled Valuation Comparison. Table 5 compares Linear, Monotonic MLP, and Bernstein PVM under controlled settings. All variants share the same scene inputs, prediction–planning backbone, context-adaptive SVO mechanism, and training objectives, with matched model capacity. In particular, the Monotonic MLP is also conditioned on the same interaction context as PVM, allowing us to examine the contribution of the Bernstein valuation structure beyond additional contextual information or model capacity.
On Test14-hard, PVM achieves 77.03 CLS-R, outperforming Monotonic MLP (73.57) and Linear valuation (71.45). These results support the effectiveness of structured Bernstein valuation under matched contextual conditioning.
(2) PVM Learning Pathways. We further conduct a controlled ablation, reported in the revised Table 6 and summarized in Table R2, to distinguish the two mechanisms through which learned valuation influences planning: value-alignment learning and planner conditioning. The prediction framework, adaptive SVO, scene inputs, and model capacity remain matched across variants.
Table R2. Contributions of value-alignment learning and planner conditioning on nuPlan.
On Test14-hard, value-alignment learning alone achieves 75.96 CLS-R, while planner conditioning alone achieves 73.96. Combining both yields 77.03, demonstrating their complementary contributions to the learned planning policy. Together, Tables 5 and R2 provide complementary evidence for the effectiveness of the structured valuation mechanism and its integration into planning.