acceptodds
Under review as a conference paper at ICLR 2027

EvolveLight: A Self-Evolving Agentic Framework for Adaptive Traffic Signal Control

Abstract

Traffic signal control (TSC) is fundamental to urban mobility, yet traffic demand is spatially heterogeneous and continuously evolving, making it difficult for a single control algorithm to sustain stable performance across regions and control periods. Existing approaches primarily improve adaptability through predefined control principles, specialized policy architectures, or task-specific training, limiting their adaptability in evolving traffic dynamics. We present EvolveLight, an LLM-based agentic framework that enables continual adaptive TSC through a closed loop of traffic diagnosis, controller construction, feedback, and skill development. EvolveLight coordinates three specialized agents: a planning agent that analyzes traffic dynamics, partitions the road network into spatiotemporally coupled regions, and assigns suitable control strategies; a modeling agent that constructs region-specific control policies; and a skill agent that develops reusable traffic analysis and control policy development skills. These skills are maintained in a persistent versioned skill library. Their utility is evaluated through downstream traffic improvements and fed back into future retrieval and refinement. To mutually improve the reasoning capabilities of the three agents, we introduce a decoupled cooperative multi-agent RL (DCMA-RL) mechanism. It starts from supervised cold-start training with rejection-sampled high-quality demonstrations. Afterwards, DCMA-RL alternates role-wise RL optimization with independently normalized cooperative and role-specific rewards, enabling stable specialization while incorporating delayed cooperative downstream TSC performance feedback into agent learning. Extensive experiments across five real-world traffic datasets demonstrate better or competitive improvements over existing baselines, with superior performance during 24-hour adaptation as experience and reusable skills accumulate. Our project is available at https://anonymous.4open.science/r/EvolveLight-8341.

Primary area:
applications to robotics, autonomy, planning
Keywords:
traffic signal control, llm agent, agentic rl

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bhiTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents EvolveLight, a multi-agent framework for adaptive traffic signal control under changing demand. Its Planning, Modeling, and Skill Development Agents use DCMA-RL and simulation feedback to choose among timing, heuristic, and RL strategies and to develop new controllers. EvolveLight improves traffic metrics on several SUMO networks. However, the experiments do not clearly separate the effects of LLM reasoning, access to different controller families, and the extra training used for generated RL controllers.

Critical strengths
  1. The paper studies how traffic signal controllers can adapt when demand changes across locations and over time.
  2. EvolveLight uses LLMs for occasional planning and controller development, while lightweight controllers make the real-time signal decisions. This design avoids an LLM call at every decision step and makes the deployment setting more practical.
  3. The framework covers planning, controller generation, skill retrieval, simulation feedback, and continued adaptation. The evaluation uses multiple datasets and includes long-horizon experiments, ablations, and an analysis of inference cost.
Critical weaknesses
  1. The experiments do not separate the contributions of LLM reasoning, skill evolution, dynamic controller selection, training the generated controllers, or the individual DCMA-RL design choices. Most importantly, the evaluation lacks a non-LLM selector with access to the same controller portfolio.
  2. The training protocol is not described in enough detail to assess computational fairness. The paper does not explain how a generated RL controller is initialized and trained, or whether the baselines receive comparable training, simulation, and compute budgets.
Questions for authors
  1. What happens after the Modeling Agent produces an RL_PLAN? In particular, when does training begin, how is the controller initialized, and is it trained from scratch or warm-started? Please also report the RL algorithm, simulation horizon, episode or update budget, and stopping criterion.
  2. The released code appears to train an RL controller during TRAIN_AND_SIMULATION, before the candidate strategy is validated and deployed. Does this happen only once before deployment, or again at every strategy-update window during rolling evaluation? Is the checkpoint then frozen for the final evaluation?
  3. How does the trained controller's performance affect optimization of the outer LLM agents? Is the traffic outcome after RL training used as the terminal GDPO reward for the generated trajectory? Does the RL loss directly affect the LLM update, or do the two optimization processes interact only through the resulting traffic performance?
  4. The paper evaluates different levels of initial skill-library availability and includes a frozen-strategy variant. However, these experiments do not separately isolate the contributions of online skill updates, dynamic controller-family selection, generated-controller training, and the individual DCMA-RL design choices. Could the authors disable these components one at a time, especially role-wise optimization, reward decoupling, and reward normalization, to verify their individual contributions?
  5. How would EvolveLight compare with a rule-based or learned non-LLM selector that has access to the same timing, heuristic, and RL controller families? This comparison is needed to distinguish the benefit of LLM reasoning from that of the controller portfolio.
  6. Section 3.2 and Eqs. (7)–(8) assign the same downstream improvement reward to every skill invoked in a training-and-simulation round. How does this mechanism avoid misattributing improvements when multiple skills contribute differently? Separately, the released implementation includes counterfactual evaluation for training the Skill Agent. Was this mechanism used in the reported experiments, and does it affect only the Skill Agent’s DCMA-RL reward or also the per-skill utility scores used for retrieval? Please clarify the relationship between these two credit-assignment mechanisms.
Reviewer_uVwWOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EvolveLight, an LLM-based self-evolving agentic framework for adaptive traffic signal control. The framework consists of three specialized agents responsible for traffic analysis and planning, control policy generation, and reusable skill development. A persistent skill library supports continual adaptation through versioned skill updates and performance-aware retrieval. The authors further introduce Decoupled Cooperative Multi-Agent Reinforcement Learning (DCMA-RL), which combines supervised cold-start training with alternating role-wise optimization and decoupled cooperative and individual rewards.

Critical strengths

The problems are realistic and critical to resolve. The integration of planning, modeling, and skill-development agents provides a flexible alternative to conventional traffic controllers. The versioned skill library is particularly interesting, as it enables reusable capabilities to accumulate and evolve based on downstream traffic performance.

Critical weaknesses
  1. Insufficient consideration of safety and robustness in real-world traffic signal control. The evaluation primarily focuses on traffic efficiency metrics, including average travel time, throughput, queue length, and waiting time. However, traffic signal control is a safety-critical task, where improving efficiency does not necessarily guarantee safe control behavior. The paper does not sufficiently investigate collision risks, traffic conflicts, queue spillback, or safety-critical constraints during automatic controller generation and evolution.

  2. Furthermore, real-world traffic observations are often affected by sensor noise, missing data, communication delays, and sensor failures. These issues are particularly important for a self-evolving framework, as inaccurate observations may affect not only immediate control decisions but also subsequent controller generation and skill updates. The lack of robustness and safety evaluations limits the practical significance of the proposed framework.

3.Real-world intersections exhibit substantial structural heterogeneity, including different numbers of lanes, permitted turning movements, signal phases, and phase-transition constraints. Although the framework supports region-specific control strategies, the experiments do not sufficiently demonstrate its ability to generalize across unseen intersection configurations or heterogeneous action spaces.

4.Moreover, while the 24-hour adaptation experiments demonstrate the benefits of updating control strategies, they do not clearly disentangle the contributions of controller regeneration, skill accumulation, and agent optimization. Additional controlled experiments are needed to establish whether the improvements genuinely arise from progressive skill evolution rather than repeated controller selection or optimization.

Questions for authors
  1. How does EvolveLight ensure that automatically generated or evolved controllers satisfy safety-critical constraints, such as conflict-free phase transitions, minimum green times, yellow/all-red intervals, and pedestrian crossing requirements? Does sandbox validation explicitly check these constraints?

  2. How does the framework perform under realistic sensor failures, noisy observations, missing traffic states, or communication delays? Could the authors evaluate its robustness under different levels of observation corruption?

3.How does EvolveLight handle intersections with different numbers of lanes, phase configurations, permitted turning movements, and action spaces? Can the authors provide experiments on previously unseen intersection topologies to demonstrate generalization?

  1. How does the framework respond to unexpected events, such as traffic accidents, road closures, emergency vehicles, or sudden demand shifts? Can the authors report worst-case traffic performance, queue spillback, or recovery time under such conditions?
Reviewer_BNmxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EvolveLight, an LLM-based framework for adaptive traffic signal control. A planning agent partitions the network and assigns control methods, a modeling agent constructs executable controllers, and an on-demand skill agent develops reusable analysis and controller-development tools. A versioned skill library uses downstream traffic feedback to update utility scores and influence retrieval. The agents undergo supervised cold-start training followed by decoupled cooperative multi-agent RL, which alternates role updates and separately normalizes shared and role-specific rewards. Evaluation uses SUMO simulations with five real-world traffic datasets, including a held-out Guangzhou network for 24-hour adaptation. The paper reports competitive control performance, improvements from post-training, and greater robustness to missing initial skills.

Critical strengths
  1. The framework provides a coherent integration of traffic diagnosis, controller construction, simulation feedback, and reusable tools. Performing LLM reasoning during controller development while deploying executable controllers offers a practical way to limit repeated LLM inference during signal control. The explicit interfaces and versioned skill artifacts make the overall workflow understandable.

  2. The experiments provide encouraging evidence that post-training improves the framework. The Guangzhou evaluation reports a 7.48% reduction in average travel time relative to CityLight, and Figure 3 shows substantially less degradation when initial skills are unavailable. These results suggest that task-specific training produces useful behavior, even though the source and scope of the gains require further validation.

Critical weaknesses
  1. The incremental contribution of the proposed agent organization and skill evolution is insufficiently established. The division into planning, modeling, and skill development is intuitive, but the experiments do not isolate its benefit over a simpler controller-development workflow. EvolveSignal and SignalClaw are discussed but not experimentally compared, leaving the added value over related LLM-based algorithm-development approaches unclear. Table 4 mainly evaluates post-training within the chosen architecture. Similarly, Figure 3 demonstrates robustness to missing initial skills, but does not distinguish improved direct controller generation from benefits specifically attributable to creating and reusing skills. These gaps make it difficult to assess which proposed mechanisms are consequential.

  2. The evaluation does not yet establish a compelling advantage under comparable resources or sufficiently support the claim of continual capability accumulation. Table 2 shows competitive rather than consistently superior performance, with some small differences and no reported run-to-run variability. The direct strong-LLM comparisons are useful, but the cost analysis focuses on deployment inference and does not establish the overall trade-off after demonstration generation, post-training, controller training, and simulation evaluations. The online adaptation opportunities and budgets of the RL baselines also need clarification. Furthermore, Jinan, Hangzhou, and New York are used for post-training and skill initialization, while held-out adaptation is demonstrated mainly on one Guangzhou network over four six-hour windows. This provides limited evidence that reusable development capabilities accumulate and improve subsequent tasks, beyond adapting controllers to that day's traffic.

Questions for authors
  1. What evidence isolates the benefit of the three-role organization and utility-driven skill evolution over a simpler LLM workflow that improves an existing strong controller under matched LLM and simulation-evaluation budgets? How does the approach compare with the related controller-development methods discussed in the paper? Please also clarify the overall cost trade-off against directly using strong LLMs in the same framework, including the additional training costs and their amortization.

  2. For Guangzhou, which components are updated in each window, what historical data are available for candidate development and evaluation, and what online updates and budgets are allowed for the RL baselines? How is the best strategy for EvolveLight-Frozen selected, and can its complete aggregate metrics be reported? These details affect the interpretation of the continual-adaptation results.

  3. In Table 4, does W/O Decouple remove only separate reward normalization, or also replace alternating role updates with simultaneous updates? The description and subsequent interpretation appear to refer to different changes. If both are changed, what evidence distinguishes their individual effects?

  4. Please clarify the vehicle-counting and repeated-run protocols. Table 1 lists 4,365 vehicles for Jinan 1, whereas Table 2 reports throughput of 4,688 for FixedTime and 5,728 for EvolveLight. How are these counts reconciled? How many independent runs underlie the reported results, and what is their variability?

Comments on the paper

1 comment

Dear reviewers,

We have now received the author responses to your raised questions and concerns. Please go through them. Also, have they answered your questions appropriately? Please also look other reviewers' strengths / concerns. Have they changed your perception of the paper? In all cases, please communicate with the authors. It is important that we let them know what relevant issues still exist or have been answered. Please remember we have limited time to get any further responses from them.

Regards,

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.