- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LG-CLFT, a framework for improving tokenized traffic simulation through lookahead-guided pseudo-label construction. It augments a SMART-based simulator with serial multi-step prediction heads and uses distance-oriented beam search to select actions whose predicted continuations better align with recorded expert trajectories. The resulting pseudo trajectories supervise closed-loop fine-tuning, while evaluation retains standard next-token sampling. Experiments on WOSAC report improvements in realism and trajectory alignment over prior methods, with ablations examining prediction depth, lookahead horizon, beam width, and matching criteria.
Critical strengths
-
A practical improvement to closed-loop supervision. The method addresses the concrete limitation of myopic pseudo-label selection and improves both realism and trajectory alignment on the reported test results, while retaining standard next-token decoding at evaluation.
-
Useful component-level evaluation. The paper examines prediction depth, search horizon, beam width, and distance criteria, and reports both full-validation and test results. These experiments provide evidence beyond a single leaderboard comparison.
Critical weaknesses
1.The central lookahead gains are small and lack statistical validation. In Table 5, increasing the horizon from one to two tokens improves RMM only from 0.7714 to 0.7718, while minADE slightly worsens. Most configurations use a single training run, and the ablations use only a 2% validation subset. Given that the test RMM advantage over the strongest listed entry is also only 0.0002, the evidence does not establish that the improvement is reproducible.
2.The quality of the lookahead surrogate is not directly validated. Candidate continuations are generated by auxiliary heads without full backbone rollouts or joint future interaction modeling. The paper does not establish whether candidates ranked favorably by this approximation also yield better continuations under the deployed policy, particularly in strongly coupled interactions. This leaves an important part of the proposed mechanism insufficiently supported.
Questions for authors
- Are the improvements from lookahead matching reproducible across training seeds and on the full validation split? Please compare CAT-K matching, one-step matching, and two-step matching from the same MSP-pretrained checkpoints under matched training settings, reporting mean and variability for RMM and minADE. This would clarify whether lookahead provides a consistent benefit beyond auxiliary pretraining.
- How accurately do MSP-based candidate rankings predict alignment under actual closed-loop continuations? Please compare MSP-based lookahead with full-backbone lookahead under a matched candidate budget, measuring candidate-ranking agreement and the realized alignment after executing selected actions. Including dense-interaction scenarios would help assess the consequences of searching independently for each agent.
I would raise my rating to Weak Acceptance if the additional evidence establishes a reproducible benefit from lookahead beyond auxiliary pretraining and supports the reliability of the approximate candidate evaluation.