- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper introduces a LLM-based rl learning method for traffic simulation. It uses two stages: in Stage 1, the authors incorporate a x2-divergence state-occupancy matching module, inspired by SMODICE, into PPO. In Stage 2, HighD trajectories are clustered into three driving styles—Conservative, Moderate, and Aggressive—and the Stage-1 policy is copied into three style-specific PPO policies. An LLM provides high-level corrective actions only when speed-distribution or spacing deviations exceed thresholds; these actions are distilled into the PPO policies through an imitation loss. The method is tested on highD dataset.
Critical strengths
- The idea to combine Dice and PPO is interesting.
- The paper addresses a meaningful practical problem. Long-horizon distribution drift is an important issue for background traffic simulation.
- Using distribution similarity as constraint instead of the single action lable is appealing.
Critical weaknesses
- The paper assumes that llm decision distribution is similar with real human driver decision distribution. But this assumption is not validated.
- The paper validate its method on a single small highD dataset. Its scalability and transferability is unclear. Consider experimenting on more complex waymo or nuplan dataset.
- A lot of related works on traffic simulation are missing such as smart, smart-r1, DecompGAIL.
- The metric is too limited. More metrics on WOSAC can be included.
Questions for authors
- How many LLM queries are required to train each style policy, and what is the total training-time LLM cost?
- Which LLM is used in Stage 2, what is the exact prompt, and what are the trigger thresholds?
- Can the authors add a PPO + Stage-2 LLM fine-tuning baseline without Stage-1 DICE initialization? This seems essential for determining whether occupancy matching contributes to the final result.
2 comments
Thank the authors for the detailed reponse. My concerns are not addressed. I therefore keep my original score.
Response to Reviewer BRKr
Thank you for the detailed review and for recognizing the importance of distribution drift and the potential of occupancy matching.
Validity of LLM guidance. Thank you for questioning the relationship between LLM decisions and human driving. Using an LLM as a teacher does not establish that its decision distribution matches human behavior. We will make this distinction explicit and avoid treating human-like teacher behavior as a validated assumption. We plan to prioritize no-teacher and simpler-teacher controls, together with behavioral and safety measures, to assess the actual effect of guidance.
Evaluation scope, metrics, and related work. Thank you for highlighting these limitations and suggesting SMART, SMART-R1, DecompGAIL, and WOSAC-related evaluation. The current highway-env/highD results do not establish urban-scenario transfer or broad competitiveness. We will improve the related-work discussion and prioritize held-out evaluation, interaction metrics, longer rollouts, and different learned-agent counts. We will examine compatible external baselines and the feasibility of more complex datasets, while clearly acknowledging unvalidated transfer claims.
LLM configuration and training cost. Thank you for identifying these reproducibility requirements. Deployment without an LLM does not eliminate training-time teacher costs. We will explicitly document the model/version, complete prompt, target-style conditioning, trigger statistics and thresholds, checking frequency, and action parsing. We will also verify and report per-style query counts, retries, token usage, monetary or compute costs, and training-time overhead. These require concrete implementation records rather than a general efficiency statement.
Stage-1 ablation. Thank you for proposing PPO followed by Stage 2 without occupancy-matched initialization. We agree that this is essential for isolating Stage 1’s contribution. We plan to prioritize this control using consistent Stage-2 settings and accounting for total training budgets. If the simpler initialization performs comparably, we will revise the necessity claim accordingly.