acceptodds
Under review as a conference paper at ICLR 2027

TSCraft: An Experience-Guided Agent for Evolving Traffic Signal Control Policies

Abstract

Traffic signal policies must adapt to demand while remaining fast and auditable. Rule-based policies need manual revision, deep RL encodes control in learned parameters, and online language-model controllers generate and validate actions inside the control loop. We present TSCraft, an experience-guided agent that evolves compact code policies offline and freezes one for deployment. It maintains multiple policy lineages and revision experience, evaluates anonymous candidates on matched simulator scenes, and maps lane-group demand to legal phases through a topology-normalized interface. The frozen policy runs locally at each intersection without language-model calls, parameter updates, or inter-intersection communication. In one 20-round campaign, TSCraft evaluated 40 candidates and promoted eight distinct non-seed policies, producing an explicit stepwise trajectory. Across matched flows and horizons, each controller is evaluated through its released deployment interface. Under this end-to-end protocol, TSCraft achieves 15.30 s mean waiting on a 23-scenario test constructed after policy selection, versus 25.08 s for MaxPressure, with lower waiting in 22 scenarios. When the same source is deployed at every intersection, it reduces average waiting time relative to MaxPressure by 25.49%, 20.96%, and 22.50% on the evaluated Hangzhou, Jinan, and New York flows. Across 23 sealed episodes, a replay record reconstructs all 5,295 phase choices and execution-aware durations. TSCraft thus places LLM reasoning between releases while fast, auditable code controls the evaluated intersections.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Traffic signal control, transportation systems, LLM agents, program evolution

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZGmAOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes TSCraft, an LLM-guided framework that iteratively evolves rule-based traffic signal control policies using simulation feedback, experience memory, and multi-candidate selection.

Critical strengths
  1. The framework separates offline LLM-guided policy evolution from online TSC, enabling efficient deployment without runtime LLM inference.
  2. The generated rule-based policies are interpretable and computationally efficient.
Critical weaknesses
  1. The manuscript is difficult to follow, as several important methodological details are not clearly explained. In particular, Section 2 introduces numerous concepts and mathematical formulations without clearly describing what the Python-based control policies contain or how they evolve.
  2. Previous studies, such as π-Light (AAAI 2024), have already explored automatically discovering interpretable traffic signal control policies through program search. TSCraft follows a similar idea but relies on LLM-guided code generation and selection. However, the advantages of using LLMs over conventional program search methods remain unclear. Furthermore, π-Light is not included in the experimental baselines, making it difficult to assess the effectiveness of TSCraft relative to existing programmatic approaches.
  3. Although the experiments demonstrate promising results across different intersections and cities, it remains unclear whether this generalization comes from the proposed LLM-guided evolution process or primarily from the topology-normalized representation and the transferability of rule-based policies.
Questions for authors
  1. What are the key advantages of LLM-guided policy evolution over conventional program search methods such as π-Light.
  2. Given that TSCraft ultimately produces a fixed rule-based controller, what justifies its general applicability across different intersections and traffic conditions without further tuning? How does it differ in this regard from classical traffic control formulas, such as Webster's?
  3. How can a single LLM-generated fixed control policy outperform RL-based policies across different traffic scenarios, given that neural network-based RL policies can represent more complex traffic conditions and control strategies? What explains the superior performance of such a relatively simple heuristic policy?
Reviewer_VbptOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

TSCraft uses an LLM to revise traffic signal control programs offline and deploys one selected program without further model calls. The search keeps multiple policy lineages and recent revision history, proposes two candidates per round, and compares them on matched simulation scenes. The resulting controller uses an anonymous phase-to-lane-group interface to choose phases and durations at different intersections. Its recorded execution traces allow individual decisions to be reconstructed. Experiments report improvements on a post-selection test bank and several road networks, with additional studies of search trajectories, component removal, and authoring models.

Critical strengths

-The separation between offline search and online execution makes sense for this application. The final controller is small enough to inspect, and the paper explains its actual phase-selection and duration rules.

-The trace analysis is convincing within its stated scope. Reproducing all 5,295 recorded phase and duration decisions shows that the explanations correspond to the executed program.

-The authors make a serious effort to examine variability and transfer. The three search campaigns, post-selection scenarios, and deployment of unchanged code across networks are valuable parts of the evaluation.

Critical weaknesses

-The headline comparisons give TSCraft more control flexibility than several baselines. It chooses among eight phases with adaptive durations of 5 to 45 seconds, while several released methods retain four actions and a 30-second clock. These differences are disclosed, but they make it hard to attribute the gains to the search method. Table S5 matches interfaces only against FixedTime. The fixed-clock results in Table S8 are more mixed, including worse AWT than MaxPressure on both Jinan flows.

-The omission of yellow and all-red clearance intervals is a significant concern. Adaptive timing is part of the contribution, yet switching phases does not incur the corresponding clearance-time cost in the simulator. This could change which duration rules perform well.

-The closest alternatives are discussed rather than evaluated. A comparison with another traffic-policy program-search method would be more informative about novelty than comparisons with online LLM control alone. I would also like to know whether the LLM selector improves on a fixed rule using the same measured traffic outcomes.

-The component ablations are based on individual search campaigns. Given the large improvements from particular revisions and the variation across full runs, removing memory or lineage breadth once does not establish a reliable effect. The total cost of finding the policy also deserves more attention alongside its low execution cost.

Questions for authors

-How much of the gain remains when TSCraft and the strongest baselines have the same phase set and timing options?

-Do the gains survive realistic yellow/all-red intervals and phase-switching costs?

-How does the search compare with a closely related program-search method or a simpler iterative code-generation procedure at matched budgets?

-Can the component ablations be repeated, and can the total model-call, token, simulation, and wall-clock costs be reported?

Reviewer_GsPGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TSCraft to improve traffic signal control programs through offline LLM search. The agent keeps several programs, records past changes, and uses simulation results to guide new changes. It compares candidates on the same scenes and selects one program for deployment. The final program chooses signal phases and durations without calling an LLM at runtime. The experiments show lower waiting time on several traffic networks. Although the authors have made good efforts, some issues still need to be addressed.

Critical strengths
  1. The final policy is a program with clear phase and duration rules. It runs without online LLM calls and records how each decision is made. This makes the policy fast to run and easier to inspect.

  2. The experiments include single intersections, full networks, and test scenes created after policy selection. The authors also report three complete search runs and reproduce the recorded decisions through replay. These tests help show that the final programs work beyond the scenes used for search.

Critical weaknesses
  1. It is not clear how much improvement comes from the search method. TSCraft uses eight phases and adaptive durations, while some baselines use four actions and fixed durations. These differences may explain part of the gain. The current comparison does not fully separate better control options from better search.

  2. The benefit of the added search steps is not clear enough. TSCraft uses memory, several policy versions, and LLM-based selection. However, the paper does not fully show how much these steps improve over a simpler search process or selection based directly on traffic metrics. It is also unclear whether the gains are worth the extra search cost.

Questions for authors
  1. In Section 3.1 and Appendix S1, the authors should compare methods with the same phase set and decision clock. The current test with the same interface only includes FixedTime. Adding MaxPressure would make this comparison stronger.

  2. The authors should test the same frozen program with fixed and adaptive durations. They should also include the same yellow/all-red transition time for all methods. This would help show which control choices produce the gains.

  3. In Section 3.4, the authors should add a simple LLM search baseline and a fixed selection rule based on traffic metrics. These methods should use the same program interface, number of candidates, and simulation budget.

  4. The authors should repeat the main ablations and report the search cost, including model calls, simulation runs, and total time. This would help show whether memory, multiple policy versions, and LLM selection give stable gains over simpler methods.

Comments on the paper

1 comment

Dear Reviewers,

Please review the authors' rebuttal and participate in the discussion to address any remaining questions before the period closes on Nov 18 AoE.

Thank you!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.