acceptodds
Under review as a conference paper at ICLR 2027

Lookahead-Guided Closed-Loop Fine-Tuning for Multi-Agent Traffic Simulation

Abstract

Traffic simulation is crucial for the development of autonomous driving systems. Recent autoregressive approaches formulate this task as sequential motion token prediction, yet inherently suffer from covariate shift between open-loop training and closed-loop deployment. Closed-loop fine-tuning (CLFT) mitigates this issue by sampling pseudo expert trajectories. However, most existing token-level methods rely on greedy single-step sampling, and locally closest matches do not guarantee global trajectory alignment. Inspired by multi-token prediction in large language models, we propose Lookahead-Guided Closed-Loop Fine-Tuning (LG-CLFT), a simple yet effective framework that enables long-horizon matching for pseudo expert trajectory construction. Specifically, we introduce auxiliary multi-step prediction heads to approximate future motion token distributions during open-loop pretraining. During CLFT, we leverage this long-horizon prediction capability to perform beam search over future motion token sequences, evaluating their overall alignment with expert trajectories. This lookahead mechanism enables sampling superior pseudo expert trajectories within the model policy distribution. On the WOSAC 2025 leaderboard, LG-CLFT achieves a Realism Meta score of 0.7854 and a minADE of 1.2865, outperforming prior supervised CLFT methods in both simulation realism and trajectory alignment.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Multi-Agent Traffic Simulation, Autonomous Driving, Closed-Loop Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EQ7KOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LG-CLFT, a framework for improving tokenized traffic simulation through lookahead-guided pseudo-label construction. It augments a SMART-based simulator with serial multi-step prediction heads and uses distance-oriented beam search to select actions whose predicted continuations better align with recorded expert trajectories. The resulting pseudo trajectories supervise closed-loop fine-tuning, while evaluation retains standard next-token sampling. Experiments on WOSAC report improvements in realism and trajectory alignment over prior methods, with ablations examining prediction depth, lookahead horizon, beam width, and matching criteria.

Critical strengths
  1. A practical improvement to closed-loop supervision. The method addresses the concrete limitation of myopic pseudo-label selection and improves both realism and trajectory alignment on the reported test results, while retaining standard next-token decoding at evaluation.

  2. Useful component-level evaluation. The paper examines prediction depth, search horizon, beam width, and distance criteria, and reports both full-validation and test results. These experiments provide evidence beyond a single leaderboard comparison.

Critical weaknesses

1.The central lookahead gains are small and lack statistical validation. In Table 5, increasing the horizon from one to two tokens improves RMM only from 0.7714 to 0.7718, while minADE slightly worsens. Most configurations use a single training run, and the ablations use only a 2% validation subset. Given that the test RMM advantage over the strongest listed entry is also only 0.0002, the evidence does not establish that the improvement is reproducible.

2.The quality of the lookahead surrogate is not directly validated. Candidate continuations are generated by auxiliary heads without full backbone rollouts or joint future interaction modeling. The paper does not establish whether candidates ranked favorably by this approximation also yield better continuations under the deployed policy, particularly in strongly coupled interactions. This leaves an important part of the proposed mechanism insufficiently supported.

Questions for authors
  1. Are the improvements from lookahead matching reproducible across training seeds and on the full validation split? Please compare CAT-K matching, one-step matching, and two-step matching from the same MSP-pretrained checkpoints under matched training settings, reporting mean and variability for RMM and minADE. This would clarify whether lookahead provides a consistent benefit beyond auxiliary pretraining.
  2. How accurately do MSP-based candidate rankings predict alignment under actual closed-loop continuations? Please compare MSP-based lookahead with full-backbone lookahead under a matched candidate budget, measuring candidate-ranking agreement and the realized alignment after executing selected actions. Including dense-interaction scenarios would help assess the consequences of searching independently for each agent.

I would raise my rating to Weak Acceptance if the additional evidence establishes a reproducible benefit from lookahead beyond auxiliary pretraining and supports the reliability of the approximate candidate evaluation.

Reviewer_mRJuOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors aim to push the boundary and address distribution shifts in autoregressive motion-forecasting models (or simulated-agent models, as people call them) by adopting the multi-token prediction paradigm used in LLMs and proposing a Look-ahead-guided CLFT to enable long-horizon matching. The key is auxiliary multi-step prediction heads that learn short continuations conditioned on preceding tokens during open-loop training. During fine-tuning, they use distance-oriented beam search with these heads to compare candidate continuations with the recorded expert trajectory, commit the first token of the best sequence, and then repeat until they get the full generation.

Critical strengths

To begin with, despite how the authors argue, quote In contrast, our method integrates multi-step prediction (MSP) into CLFT for traffic simulation, as their key novelty in this paper, I find it really hard for me to credit them with any substantial achievements, but I will try my best. First, this paper has a very focused scope, targeting a specific weakness in the existing paper: single-step sampling during closed-loop rollout. I appreciate the authors for clearly highlighting the approximations in their proposed method: specifically, candidate searches in the LG-CLFT are per-agent, later continuations are discarded, and there is no full-horizon optimality guarantee. The evaluation in this paper is rather complete. I personally find Table 7 the most useful result since it shows the discrepancy between probability-based and distance-based search strategies.

Critical weaknesses

As I mentioned earlier, my biggest concern about this paper is its significance. This is mainly because it offers only marginal improvements over the baselines and has a confusing codebase implementation. Take Tables 1 and 2 as an example: the proposed method's realism meta-metric is within 0.001 of 4 of the 8 baselines, and it beats them only on minADE. First, I am not a fan of this benchmark, as it only reflects how closely your imitation policy mimics experts. But these metrics themselves are very problematic. Basically, without standard deviations, I can't tell whether your advantage over the baselines comes from randomness, and I know the WOSAC server unfortunately doesn't report them. Meanwhile, I am just curious. Why don't you compute and report the cumulative score in Table 2 using the official metametric_weight in the official devkit at https://github.com/waymo-research/waymo-open-dataset/blob/master/src/waymo_open_dataset/wdl_limited/sim_agents_metrics/challenge_2025_sim_agents_config.textproto so we can compare more intuitively with the results in Table 1?

I took some time reading the code in your supplementary material, and I find one of your implementations quite confusing. Please do correct me if I am wrong, but the three-token branch decodes the second candidate token using token_traj_all[batch_idx_1 % n_agnet, ...] in your distance_oriented_beam_search.py. Let me try to clarify my confusion. For three agents and two beams, that order is [0,0,1,1,2,2], whereas the modulo expression produces [0,1,2,0,1,2]. The token processor constructs different trajectory dictionaries for vehicles, pedestrians, and cyclists, so the mismatch can assign a candidate geometry from another agent type and corrupt the third-token proposal. I am a bit concerned this could affect your overall results, but please bear with my ignorance and let me know if I am wrong.

Questions for authors

For primary concerns, please see the weaknesses I listed above. I also have a few minor follow-up questions. Regarding the experimental design, were Table 5's CAT-K, H=1H=1, and H=2H=2 variants initialized from the same auxiliary-head checkpoint and trained with the same entropy weight, optimizer schedule, and update budget? What exactly differs between Appendix C's entropy sweep and Tables 5–8? Meanwhile, what are the actual label-construction time and peak memory at the default H=2H=2, K=B=32K=B=32 setting, and the corresponding CAT-K cost under the same hardware/batch conditions? Do any existing measurements compare auxiliary continuation rankings with full-backbone continuations? The latter would help explain when the inexpensive approximation improves the committed action.

Reviewer_cziJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This manuscript introduces Lookahead-Guided Closed-Loop Fine-Tuning (LG-CLFT) for multi-agent traffic simulation to address covariate shift in autoregressive models. The proposed method integrates multi-step prediction heads during open-loop pretraining. During closed-loop fine-tuning, these heads enable a distance-oriented beam search to evaluate multi-step candidate continuations against expert trajectories, ultimately committing to the first token of the most globally aligned sequence. The framework is evaluated on the Waymo Open Sim Agents Challenge benchmark, demonstrating performance improvements over prior supervised CLFT methods.

Critical strengths
  1. Applying serial multi-token prediction heads to facilitate long-horizon beam search elegantly mitigates the trajectory drift caused by greedy single-step matching in existing CLFT pipelines.

  2. The approach successfully adapts multi-token predictive techniques from large language modeling to the domain of spatial trajectory alignment.

Critical weaknesses
  1. The absolute performance improvement on the primary Realism Meta metric is extremely limited (e.g., +0.0008 over CAT-K). Furthermore, the empirical evaluation lacks comparisons with several recent state-of-the-art methods on the WOSAC 2025 benchmark, making it difficult to assess the true competitiveness of the proposed approach.

  2. The methodology is strictly restricted to supervised CLFT. It is unclear whether or how this framework could be adapted to RL-based CLFT approaches.

Questions for authors
  1. Given the remarkably small margin of improvement on the Realism Meta score, can you provide variance metrics or statistical significance testing across multiple random seeds to confirm the robustness of this gain?

  2. The ablation studies lack sufficient analytical depth explaining why certain hyperparameters behave as they do. For example, why does MSP (N=1) yield the best pretraining result, while pseudo-trajectory matching optimal displacement error occurs at H=3, and the best final CLFT Realism Meta score peaks at H=2? Please elaborate on the underlying dynamics driving these specific optima.

  3. How does the computational overhead of the distance-oriented beam search during Stage II fine-tuning scale in practice?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.