- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SHRED, a demonstration-free self-play RL framework for jointly learning the behaviors of vehicles, pedestrians, and cyclists. It combines type-specific dynamics, rewards, and policy components with procedural scenario and goal generation, agent-level randomization, and advantage-weighted prioritized replay. The authors evaluate SHRED on WOSAC for traffic simulation and InterPlan for interactive planning, reporting improvements over a demonstration-free baseline and a vehicle-only self-play variant. The paper aims to establish heterogeneous self-play as a unified approach to traffic simulation and autonomous driving planning.
Critical strengths
-
Learning heterogeneous road-user behaviors without expert demonstrations is an important research direction. SHRED provides a coherent framework for jointly training multiple road-user types through closed-loop interaction.
-
Evaluation across simulation and planning is promising. The paper investigates both traffic realism and interactive planning, with ablations suggesting that heterogeneous dynamics and rewards contribute to performance.
Critical weaknesses
-
The distinction between ego planning and agent simulation is insufficiently clarified. During self-play, agents can receive navigation goals, whereas in practical autonomous driving, the ego vehicle's intended route is generally known, but the intentions of surrounding road users are not directly observable. Although the paper introduces goal dropout to reduce dependence on explicit navigation signals, it remains unclear how the policy distinguishes the ego vehicle from other vehicles and how it behaves under these different information conditions. Furthermore, WOSAC and InterPlan have different task setups and evaluation protocols. The paper should clarify how the same framework is configured for simulation and ego planning, what information is available to each agent in each setting, and whether the reported results rely on task-specific modifications.
-
The simulation evaluation does not sufficiently establish competitiveness against recent methods. The main paper emphasizes WOSAC 2023 and 2024 results, while the WOSAC 2025 evaluation is relegated to the appendix. Given the rapid progress in traffic simulation, the latest benchmark results and comparisons with strong contemporary methods should be central to the evaluation. The reported results appear less competitive against more recent approaches, making it difficult to assess the current state of the art and the practical significance of the proposed method. A more comprehensive comparison using the latest available leaderboard results and consistent evaluation protocols is needed.
Questions for authors
-
How does SHRED distinguish the ego vehicle, whose navigation goal is known, from surrounding vehicles whose intentions are generally unknown? How are observations, goals, and policy configurations adapted between WOSAC simulation and InterPlan planning?
-
Why are the WOSAC 2025 results not included in the main paper, and how does SHRED compare with the strongest recent methods under consistent evaluation protocols?
-
SHRED relies on predefined dynamics models and manually designed reward functions for different road-user types. Although these choices facilitate scalable training, they may constrain the range of learnable behaviors and fail to capture the full distribution of human driving and pedestrian behavior. The paper itself acknowledges a fidelity ceiling imposed by the simulator and reward design. More evidence is needed to establish how sensitive the results are to these assumptions and whether the learned behaviors generalize beyond the capabilities encoded by the dynamics and reward functions. More specifically, how sensitive is the learned behavior to the predefined dynamics and hand-designed rewards? Can the authors demonstrate that the method captures plausible behavioral variation beyond what is explicitly encoded in these choices?
-
The validity of procedural scenario initialization is insufficiently established. The method introduces randomized heading perturbations to increase interaction diversity and uses rejection sampling to avoid initial collisions. However, collision-free initialization alone does not guarantee physically plausible or traffic-rule-compliant scenes. For example, perturbed headings may produce vehicles or cyclists facing implausible directions or violating lane and road conventions. The authors should clarify how randomized heading perturbations are constrained by road geometry and agent type to ensure physically plausible, collision-free, and traffic-rule-consistent initial scenes, and provide quantitative validation of initial-state plausibility and traffic-rule compliance.
-
The framework's applicability to end-to-end autonomous driving remains unclear. The policy relies on structured inputs, including map geometry, traffic signals, agent states, and navigation goals. It is therefore unclear whether SHRED is intended as a modular policy operating on structured scene representations or as a framework applicable to end-to-end driving systems. If the authors intend to claim broader relevance to end-to-end autonomous driving, they should discuss how the framework could operate directly from camera observations, or explicitly delimit the current method's scope and perception assumptions.
-
The planning evaluation is too narrow for the breadth of the claims. InterPlan provides evidence of performance in interaction-heavy scenarios, but the paper presents SHRED as useful for both traffic simulation and autonomous planning. The planning results would be more convincing with evaluation on additional established benchmarks, such as NAVSIM v1/v2, where applicable, or other mainstream planning benchmarks with clearly specified protocols. This would help determine whether the reported gains generalize beyond InterPlan's particular scenario distribution.
4 comments
Response for Reviewer ugvE (2/2)
Q4 — Procedural initialization plausibility: Our initialization is designed to produce physically plausible configurations by construction, rather than relying solely on collision rejection. Vehicles spawn on lane centerlines, while cyclists spawn along lanes or road edges, with heading perturbations limited to ±25° relative to lane direction. Pedestrians spawn near road boundaries or crosswalks. Training goals are sampled within type-specific distance ranges: 20–100 m for vehicles, 2–12 m for pedestrians, and 20–35 m for cyclists.
We provide additional ablations separating the effects of initialization and training goals (Table R2). Logged initialization with procedural goals achieves comparable realism to full SHRED (0.724 vs. 0.721), while logged initialization with logged goals achieves 0.716. The latter corresponds to the original “w/o rand. init. & goals” ablation. These results suggest that procedural initialization is a practical alternative to logged initial states, achieving comparable realism without relying on expert demonstrations.
Table R2 — Ablation of initialization and training goals
| Cell | Initialization | Training goals | Overall | Δ vs. A | Vehicle | Pedestrian | Cyclist |
|---|---|---|---|---|---|---|---|
| A — Full SHRED | Procedural | Procedural | 0.721 | — | 0.742 | 0.724 | 0.672 |
| B | Logged | Procedural | 0.724 | +0.003 | 0.738 | 0.719 | 0.680 |
| C | Logged | Logged | 0.716 | −0.005 | 0.731 | 0.716 | 0.675 |
We additionally quantify initial-state plausibility across 75k procedurally initialized agents (Table R9). Compared with logged initialization, procedural initialization achieves substantially lower overlap rates across all agent types, zero vehicle/cyclist off-road rates, and near-zero wrong-way rates. Pedestrian off-road rates are higher (14.49% vs. 6.87%), partly because valid crosswalk placements are counted as off-road under this metric.
Table R9 — Starting-scene plausibility (% of agents)
| Type | Overlap, random | Overlap, log | Off-road, random | Off-road, log | Wrong-way, random | Wrong-way, log |
|---|---|---|---|---|---|---|
| Vehicle | 0.01 | 1.14 | 0.00 | 14.49 | 0.00 | 1.71 |
| Cyclist | 0.01 | 3.93 | 0.00 | 15.73 | 0.12 | 14.04 |
| Pedestrian | 0.00 | 28.20 | 14.49 | 6.87 | n/a | n/a |
Thank you for your response and the additional experimental results, which address most of my earlier concerns. However, several points remain unclear and require further discussion:
-
Based on your response to W1/Q1, goals appear to be an essential input in both setups: all agents rely on map-sampled goals in the sim-agent task, and all surrounding agents (excluding the ego vehicle) do so in the planning task. Could you elaborate on how these map-sampled goals are generated? Specifically, how do you ensure the diversity and behavioral plausibility required for realistic simulation, where agents should exhibit diverse driving behaviors?
-
You position this work as "achieving realistic heterogeneous multi-agent simulation through self-play without expert demonstrations." Given the current performance gap compared to AR-based/diffusion-based approaches (w/ SFT or RFT), could you further justify why avoiding expert demonstrations is practically or scientifically significant for traffic simulation and planning tasks?
-
Since self-play is driven by rule-based rewards, there is an inherent tension between reward optimization and matching the real-world human driving distribution. Could you share your insights on this trade-off and how your framework mitigates potential distribution shift?
-
To provide a more comprehensive picture of the self-play paradigm's effectiveness, it is essential to report SHRED's official performance or ranking on the WOSAC 2025 leaderboard (test split).
Lastly, I recommend incorporating these clarifications and the new experimental results into the revised manuscript.
Q1: Map-sampled goals and behavioral diversity: We first clarify that in planning, SHRED controls only the ego vehicle, while surrounding agents are controlled by the simulation environment (e.g., IDM in nuPlan). In traffic simulation, SHRED is trained with 30% goal dropout, allowing agents to act without explicit goal conditioning. Map-sampled goals are generated by traversing directed lane graphs and randomly selecting among valid successor lanes, ensuring route plausibility while introducing diversity through different route choices. Importantly, goals specify destinations rather than trajectories; given the same goal, agents may exhibit different behaviors depending on surrounding traffic, interactions, and reward parameters conditioning.
Q2:SHRED’s core contribution is demonstrating that vehicles, pedestrians, and cyclists can jointly learn realistic interactive behaviors and effective planning policies through decentralized self-play, without expert trajectories or pretrained imitation policies. Agents learn by interacting and adapting to one another, and this reduces reliance on demonstrations and enables exploration of diverse closed-loop interactions, particularly valuable when representative data are scarce, such as in new regions or for underrepresented road-user types. Our simulation and planning results demonstrate the utility of this approach, positioning self-play as a complementary paradigm to imitation learning instead of a replacement
Q3: Our main insight is the importance of diversity during training. SHRED jointly trains different road-user types while randomizing each agent’s reward weights and dynamics, exposing policies to partners with varied preferences and capabilities. Procedural scenarios further diversify their interactions. This helps mitigate reliance on a single reward configuration or narrow interaction distribution, although it does not guarantee matching the frequency of behaviors in human data.
One challenge in imitation learning is distribution shift during long-horizon closed-loop simulation, where errors can accumulate and lead to collisions or off-road behavior. In contrast, self-play can train agents over extended horizons without being constrained by the duration of logged trajectories available. We observe that SHRED agents exhibit robust closed-loop behavior, as they are explicitly trained to avoid such infractions.
We also observe that distributional realism and desirable behavior are not always aligned. WOSAC evaluates similarity to logged behavior, including collision and off-road statistics. If logged trajectories contain off-road events, reproducing their frequency can improve the corresponding realism metric, even though avoiding those events may be preferable for planning. We therefore interpret realism scores alongside behavioral analysis and closed-loop planning performance, rather than treating realism alone as a measure of safe or effective behavior.
Q4 We have reported SHRED’s official WOSAC 2025 test-set results in original paper's Table9, updated pdf Table 11. Since no prior self-play results are available, direct comparison is limited. Leading imitation-based methods, such as Cat-K, achieve realism scores around 0.785, but rely on demonstrations from the benchmark’s training distribution. While a realism gap remains, the methods differ fundamentally in supervision, and part of the gap may reflect the trade-off between distributional fidelity and desirable closed-loop behavior discussed above.
We thank the reviewer for insightful discussions and have updated the final pdf to include the discussions.
Response for Reviewer ugvE (1/2)
We thank the reviewer for their constructive questions and suggestions. We address each point below:
W1/Q1 — Ego planning versus agent simulation: SHRED uses the same policy architecture for simulation and ego planning. Each agent observes surrounding agents’ states but do not know others' navigation goals. During training, initial states and goals are generated procedurally, goals are renewed on arrival, and goal conditioning is dropped with probability 0.3.
In the WOSAC evaluation, no method receives logged destinations; SHRED uses map-sampled goals and generates a new goal on arrival. In InterPlan, SHRED controls only the ego vehicle and receives goal points along its known route. The distinction is therefore in which agents are controlled and how their goals are supplied.
W2/Q2 — WOSAC 2025 comparison: Imitation-learning methods have been extensively studied in prior work, but direct comparisons are limited because SHRED’s main contribution is achieving realistic heterogeneous multi-agent simulation through self-play without expert demonstrations. Table 3 includes SMART (CAT-K) as an imitation-learning reference. Since prior self-play baselines do not report WOSAC 2025 results, we additionally evaluate SHRED, SPACeR, and HR-PPO using the WOSAC 2025 10k validation set under consistent settings, excluding scenarios unsupported by GPUDrive due to z-axis and agent-count limitations (Table R3). SHRED achieves 0.733, outperforming SPACeR (0.718) and HR-PPO (0.696), consistent with the relative rankings in Table 3.
Table R3 — WOSAC 2025
Q3 — Sensitivity to dynamics and rewards: Table R1 directly evaluates sensitivity to the dynamics and reward design. Sharing dynamics reduces realism from 0.721 to 0.718, while sharing rewards reduces it to 0.714, supporting the benefit of type-specific modeling. In InterPlan and the additional nuPlan evaluations (Table R8 below), SHRED also achieves improved planning performance under nuplan's dynamics different from those used during training. These results provide evidence of transfer beyond the training dynamics.
Table R1 — Heterogeneous modeling
WOSAC 2024 with logged evaluation goals.
Q5 — End-to-end driving scope: We do not claim that SHRED contributes directly to end-to-end autonomous driving. Our focus is on learning interactive behaviors through heterogeneous self-play using structured scene representations. Recent work, such as Scaling Self-Play for End-to-End Driving (Rowe et al., 2026), has demonstrated the potential of extending self-play to pixel-based driving policies, suggesting a promising direction for future work building on SHRED.
Q6 — Broader planning evaluation: We additionally evaluate SHRED on nuPlan Val14 and Test14-hard (Table R8), demonstrating consistent improvements over the vehicle-only baseline across both closed-loop reactive (CL-R) and non-reactive (CL-NR) settings. We also conduct repeated InterPlan-80 evaluations, obtaining 66.41 ± 0.79 for SHRED and 54.20 ± 0.50 for the vehicle-only baseline (mean ± standard deviation).
NAVSIM focuses on end-to-end planning from sensor observations, whereas SHRED operates on explicit object representations, making direct evaluation outside our current scope.
Table R8 — Planning
InterPlan-80 reports mean ± standard deviation over four SHRED and three vehicle-only evaluations.