- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes REARL, a reference-guided online calibration framework for improving the realism of traffic simulation. The method constructs a representative scene library from real-world traffic data and dynamically retrieves reference scenes based on the current simulation state. By detecting discrepancies in speed distributions and vehicle spacing, REARL employs an LLM to select corrective actions while preserving the simulator's original IDM+MOBIL controller. Experiments on the highD dataset demonstrate that REARL improves the statistical alignment between simulated and real traffic, particularly in terms of speed distribution and vehicle spacing.
Critical strengths
- Modular design. The proposed framework preserves the simulator's native controller and introduces a separate online calibration module, allowing integration without retraining the underlying driving policy.
- Detailed component analysis. The paper includes several ablation studies and diagnostic metrics to examine the effects and limitations of different components.
Critical weaknesses
- High computational overhead. Although the paper reports an efficiency comparison, REARL introduces substantial computational overhead due to repeated LLM inference for individual vehicles. This raises concerns about its practicality and scalability for large-scale traffic simulation and autonomous driving evaluation.
- Questionable effectiveness of adaptive interventions. The full REARL configuration triggers interventions at every scheduled checkpoint, effectively resulting in periodic corrections. Moreover, the ablation study shows that periodic intervention can outperform discrepancy-triggered intervention under a matched budget. These results raise questions about the necessity and effectiveness of the proposed adaptive intervention mechanism.
- Lack of downstream validation. Although REARL improves statistical alignment with real-world traffic, it remains unclear whether these improvements benefit the training or evaluation of autonomous driving models. Experiments involving downstream autonomous driving policies or closed-loop evaluation would better demonstrate the practical value of the proposed framework.
- Limited definition of traffic realism. The evaluation primarily relies on speed distributions and vehicle spacing to measure simulation realism. However, matching these statistical properties does not necessarily indicate realistic vehicle interactions or driving behaviors. More comprehensive evaluations of behavioral realism, particularly in safety-critical interactions, are needed.
- Limited evaluation scenarios and generalization. The experiments are conducted on only five traffic scenes. Moreover, these scenes were also used during prompt development, raising concerns about potential overfitting. Evaluation on a larger and more diverse set of unseen traffic scenarios is necessary to demonstrate the generalization and robustness of the proposed framework.
Questions for authors
None.
Cost. Repeated vehicle-level calls are expensive. We did not take a new wall-clock measurement. The figure we cite is the ten-step benchmark, 86,241.309 ms against 0.461 ms for IDM+MOBIL and 15.898 ms for PPO, and it applies to offline calibration. Full-setting rollouts in the new runs use 110 calls each. We do not claim that this cost is appropriate for large-scale traffic simulation or online autonomous-driving evaluation.
Adaptive timing. The full configuration fires through the spacing branch at every scheduled checkpoint, so the realized schedule is periodic. That is already stated in the paper. Both the retrospective matched period and the prospective DeepSeek comparison give lower corresponding errors to the periodic side. An advantage for adaptive timing is not established. The speed and spacing improvement over IDM+MOBIL remains, including on the ten new scenes.
Downstream driving. No downstream policy was trained or evaluated in closed loop. Whether the statistical alignment helps autonomous-driving training or evaluation is unanswered. We do not report a downstream benefit.
Realism. The reported quantities are the speed distribution, Equation (2), the same-lane gap distribution, jerk-distribution distance, and the colliding-vehicle rate. Across the five development scenes, mean absolute jerk of the real trajectories lies between 0.059 and 0.071. One full-bin acceleration or deceleration moves the target speed by 3.375 m/s. Agreement on these statistics does not establish realistic interaction, and this revision adds no evaluation on scenes built for safety-critical events.
Coverage. The submitted evaluation used five scenes that also informed prompt design. The ten additional scenes did not. Their Qwen, DeepSeek, and GLM results are separated by interface in Section 1. The further 15 reserved scenes were not run, so the added evidence stops at those ten scenes.