acceptodds
Under review as a conference paper at ICLR 2027

LLM-Guided Multi-Agent Reinforcement Learning for Emergency Traffic Signal Control

Abstract

Emergency vehicle (EMV) signal preemption requires coordinated multi-agent decisions under uncertain arrival information and changing traffic environments. We propose a large language models (LLM)-guided multi-agent reinforcement learning (MARL) framework integrating probabilistic perception, preference formation, and action grounding. A probabilistic long short-term memory (LSTM) estimates EMV arrival times and uncertainty, and an LLM converts this temporal context and multi-agent traffic state into phase preference distributions that modulate learned Q-values. An uncertainty-aware grounded action transformation (UGAT) reduces the impact of environment-dependent dynamics mismatch on action selection. Experiments on three real-world traffic networks show consistent improvements over MARL and traffic-control baselines. Under heavy congestion, the method reduces overall and EMV travel time by 24.96% and 44.88%, respectively, while improving traffic stability. Cross-environment experiments demonstrate robustness to changes in network structure and traffic demand.

Primary area:
reinforcement learning
Keywords:
Multi-agent reinforcement learning, LLM-guided coordination, Uncertainty-aware control, Traffic signal control, Emergency vehicle signal preemption

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_M4SuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes an LLM-guided multi-agent reinforcement learning framework for emergency vehicle traffic signal control. It combines probabilistic LSTM-based arrival-time prediction, LLM-generated phase preferences, and UGAT for cross-environment adaptation. Experiments cover five scenarios across three urban road networks. In the Jinan high-demand scenario, the method reduces overall vehicle and emergency vehicle average travel time by 24.96% and 44.88%, respectively, compared with fixed-time control.

Critical strengths
  1. The paper addresses the practical trade-off between prioritizing emergency vehicles and maintaining overall traffic efficiency.
  2. Combining arrival-time prediction, LLM-guided phase preferences, and multi-agent reinforcement learning offers a promising direction for emergency vehicle signal control.
Critical weaknesses
  1. Insufficient literature support in the introduction. The discussion of the research background and limitations of existing methods lacks adequate references, making the motivation and its relationship to prior work less clear.
  2. Limited coverage of emergency vehicle control baselines. The evaluation mainly compares the proposed method with fixed-time control, MA2C, and HGAT, but does not include specialized emergency vehicle control methods discussed in the related work, such as EMVLight. This makes it difficult to assess the method's advantages over existing approaches designed specifically for emergency vehicle prioritization.
  3. Lack of direct validation for ETA prediction. The paper reports neither ETA prediction errors on a held-out test set nor uncertainty calibration metrics. It is therefore difficult to assess the accuracy and reliability of this component directly.
  4. Insufficient details on the LLM implementation. The paper mentions DeepSeek but does not specify the exact model version or provide key details such as the prompt templates. These omissions limit reproducibility.
  5. Unclear uncertainty modeling in UGAT. Equation (9) describes the prediction of the next observation but does not explicitly specify the corresponding predictive distribution, leaving some ambiguity regarding the probabilistic forward model described in the paper.
  6. Lack of statistical analysis of experimental results. Table 1 reports only point estimates, without specifying the number of independent training runs or providing standard deviations, confidence intervals, or significance tests. It is therefore difficult to determine whether the reported improvements are robust.
Questions for authors
  1. What are the specific algorithmic differences between the proposed UGAT variant and the method introduced by Da et al. (2023)? Could the authors compare their approach directly with the original UGAT or provide an ablation study isolating the contribution of the modified uncertainty criterion?
  2. Which exact DeepSeek model and version were used? Could the authors provide the prompt templates, decoding settings, and invocation procedure, and compare the proposed method against a preference mechanism that does not use an LLM under otherwise identical settings?
5 comments
AuthorsOpenReview ↗

Research Background and Motivation—Weakness 1 and UGAT Uncertainty Modeling and Comparison with the Original Method—Weakness 5; Question 1

We sincerely thank the reviewer for the positive assessment of the practical relevance of this work and for the constructive comments.

1. Research Background and Motivation—Weakness 1

We appreciate the reviewer's comment on the insufficient literature support in the Introduction. The original manuscript did not provide a sufficiently detailed discussion of the development of emergency-vehicle signal control and the limitations of existing approaches. This made the motivation for combining ETA prediction, LLM-generated phase preferences, and multi-agent reinforcement learning less explicit. The first two paragraphs of the Introduction will cite the EMV and transfer papers, including EMVLight (Su et al., 2022; 2023), the original UGAT (Da et al., 2023) and so on. Each citation will be attached to a specific limitation of that line of work.

2. UGAT Uncertainty Modeling and Comparison with the Original Method—Weakness 5; Question 1

2.1 Forward and Inverse Model Outputs.

The forward model is a deterministic point predictor of the next observation. It is trained using mean squared error and does not output a probability distribution or an observation-uncertainty estimate. The inverse model takes an observed state transition as input and predicts a categorical distribution over possible actions. It is trained using cross-entropy loss. UGAT uses the Shannon entropy of the inverse action distribution. That is not the LSTM ETA variance.

2.2 UGAT methodology and comparison with the original implementation.

The forward–inverse grounding framework comes from Da et al. (2023). Both versions use a target-domain forward model and a source-domain inverse model, and replace the policy action only when uncertainty is low. Ours trains the forward model on SUMO and the inverse model on CityFlow with cross-entropy. Uncertainty is the Shannon entropy of the inverse action distribution, compared with a fixed threshold. Original UGAT uses an evidential inverse model, Dirichlet vacuity, and an adaptive threshold. Our entropy measures action ambiguity, not the joint uncertainty of the two models. The fixed threshold avoids fluctuations from updating the threshold online.

To evaluate these differences, we conducted a direct comparison among the proposed implementation, the original UGAT mechanism, and the configuration without UGAT adaptation. The results are summarized in Table R1.

Table R1. Performance comparison of different uncertainty-guided action grounding strategies on the Hangzhou network.

MethodOverall ATT (s)EMV ATT (s)TPAWC
w/o UGAT Adaptation384.26216.3826931.55
Original UGAT457.7176.424273.23
Ours368.8184.827001.65

Ours has the lowest overall ATT (368.8 s) and the highest throughput (2700). Original UGAT has a lower EMV ATT (176.4 s) but a much worse overall ATT (457.7 s) and throughput (2427). This comparison indicates that the original UGAT provides stronger emergency-vehicle prioritization in this scenario, but at a substantially greater cost to general traffic efficiency.

Figure 16 in the revised manuscript presents the cumulative EMV ATT over simulation time. The original UGAT eventually achieves the lowest EMV ATT, consistent with Table R1, but this advantage is accompanied by worse overall traffic efficiency. These results help distinguish the different operating characteristics of the two grounding mechanisms rather than suggesting that one configuration is uniformly superior. Figure 17 in the revised manuscript further illustrates the screening behavior of the original UGAT. Under its EDL-based mechanism, 51.3% of candidate decisions are accepted. The adaptive threshold changes to 0.51 in response to recent uncertainty estimates.

2.3 Target-environment adaptation and cross-city transfer.

We further compare three settings: F0, the configuration without UGAT adaptation; F1, the proposed method using SUMO records to train the forward model; and F2, a matched-budget fine-tuning baseline initialized from F0 and further trained using a different Hangzhou traffic flow before evaluation on the same test flow. The results are reported in Table R2.

Table R2. Comparison of target-environment adaptation and matched-budget policy fine-tuning on Hangzhou.

SettingOverall ATTEMV ATTTPAWC
F0, proposed ablation384.26216.3826931.55
F1, proposed method368.84184.7527001.65
F2, matched-budget fine-tuning411.43194.1325782.65
AuthorsOpenReview ↗

UGAT Uncertainty Modeling and Comparison with the Original Method—Weakness 5; Question 1 and LLM Implementation and Preference-Generation Comparisons—Weakness 4; Question 2

Relative to F0, F1 lowers overall ATT by 15.42 s and EMV ATT by 31.63 s, and raises throughput by 7 vehicles, while AWC rises from 1.55 to 1.65. F2 is worse than F0 and F1 on overall ATT, throughput, and AWC. In Figures 18 and 19, 90% of F1 trips have a waiting-time share below 9.0%, versus 20.8% for F2. The medians are 1.9% and 3.7%.

The cross-city evaluation is reported separately. A policy trained on Hangzhou is transferred to Jinan-Low without using Jinan-specific data for training or adaptation. Both the Jinan-trained reference policy and the transferred Hangzhou policy are evaluated in SUMO under the same 4365-vehicle demand. The results are shown in Table R3.

Table R3. Cross-city transfer performance from Hangzhou to Jinan-Low.

SettingOverall ATTEMV ATTTPAWCCO2Fuel
Proposed Jinan-Low, trained in Jinan291.67112.2241422.01639.59204.00
Hangzhou policy tested on Jinan-Low409.64106.1130212.53956.02304.93

EMV ATT falls from 112.22 s to 106.11 s, but overall ATT rises from 291.67 s to 409.64 s and throughput falls from 4142 to 3021. Figures 20 and 21 show the resulting congestion, and more than one thousand vehicles remain unfinished.

These results demonstrate that the Hangzhou-trained policy can be evaluated on a different city network without Jinan-specific training, but its overall traffic performance remains substantially below that of the Jinan-trained policy. Accordingly, this experiment is described as cross-city transfer without target-city training, rather than evidence of consistently successful cross-network generalization. The Hangzhou experiment is separately characterized as cross-simulator target-environment adaptation because SUMO data are used to train the forward model. It is therefore not described as strict zero-shot transfer across simulators.

2.4 Reliability of entropy-based uncertainty estimation.

At four central Hangzhou intersections, MADQN and the inverse model stayed frozen, and UGAT did not change the executed phase. After each transition, the inverse model predicted that phase. A mismatch was counted as an error. The 2396 decisions were grouped by entropy.

The aggregate results are reported in Table R4. The estimated error rate increases from 5.9% in the lowest-entropy interval to 39.7%, 68.4%, and 87.9% in the three higher-entropy intervals, respectively. The lowest-entropy interval contains 2146 of the 2396 transitions, accounting for approximately 89.6% of the samples. The individual intersection results show that the error rate in this interval ranges from 4.6% to 7.0%, although the relationship is not strictly monotonic across every interval at every intersection.

Table R4. Inverse-model action-prediction error across entropy intervals.

Entropy intervalNumber of transitionsEstimated incorrect predictionsWeighted error rate
0–0.422,1461275.9%
0.42–0.831415639.7%
0.83–1.25765268.4%
1.25–1.66332987.9%
Total2,39626411.0%

These results indicate that higher inverse-model entropy is generally associated with a higher probability of action-prediction errors, whereas low entropy is associated with substantially more reliable predictions. Nevertheless, low entropy does not guarantee correctness, and the observed relationship should be interpreted as an empirical reliability indicator rather than a formal guarantee of action accuracy.

3. LLM Implementation and Preference-Generation Comparisons—Weakness 4; Question 2

3.1 Model Configuration and Invocation Procedure

The LLM used in the experiments is DeepSeek-Flash. The decoding temperature is set to 0.7, and the maximum number of generated tokens is 2000.

The LLM is invoked through blocking API calls. A new preference update is requested every ten simulation steps and when an emergency-vehicle event triggers an update. The generated preference vector contains eight phase-preference values for each intersection. The preferences are normalized so that the values at each intersection lie in ([0,1]) and sum to one.

The prompt consists of a task instruction and a traffic-information input. The task instruction asks the model to act as a traffic-signal expert and assign a preference to each of the eight phases at each intersection. A higher preference indicates that the corresponding phase should be activated sooner. At each intersection, the eight preferences are constrained to ([0,1]) and must sum to one. The instruction asks the model to consider both queued regular traffic and emergency-vehicle priority, and to increase the preference of phases that allow approaching emergency vehicles to pass.

AuthorsOpenReview ↗

LLM Implementation and Preference-Generation Comparisons—Weakness 4; Question 2 and Direct Evaluation of ETA Prediction and Uncertainty Calibration—Weakness 3

The traffic-information input contains the total number of waiting vehicles associated with each phase at every intersection. For each emergency vehicle, it also provides the vehicle's current speed, remaining distance, the speed and distance of the vehicle ahead, its current intersection, subsequent intersections on its planned route when available, and the phases that permit passage at the current and downstream intersections. If the format check fails, the model is prompted again, with up to three attempts. If the output remains invalid after these attempts, the preference residual is set to zero.

3.2 Comparison with Non-LLM Preference-Generation Mechanisms

On Hangzhou we kept the pretrained MADQN, ETA features, and UGAT fixed, and changed only the preference source: none, a queue rule with an EMV bonus, an MLP using the same numerical inputs, uniform, random, and the LLM.

The LLM has the lowest overall ATT (368.84 s) and EMV ATT (184.75 s). The rule and MLP lower overall ATT but raise EMV ATT above MADQN, to 205.62 s and 196.12 s. Random reaches 189.25 s EMV ATT without a matching overall-ATT gain.

Table R5. Performance comparison of different preference-generation strategies on the Hangzhou 4×4 network.

MethodOverall ATTEMV ATTTPR_change
A0 MADQN404.59193.2525980
A2 MLP381.10196.12269641.1%
A3 Uniform420.32201.6226670
A3 Random409.46189.25268764.5%
A4 Rule-based380.28205.62269640.7%
A5 LLM368.84184.75270039.2%

The action-change ratios are 40.7% (rule), 41.1% (MLP), 64.5% (random), and 39.2% (LLM). Uniform changes no action, because the same value on every phase does not change the ranking. Figures 22 and 23 show that the lowest EMV ATT is not the setting that changes the most actions, and this ordering holds in the second half of the 3600 s run.

4. Direct Evaluation of ETA Prediction and Uncertainty Calibration—Weakness 3

4.1 ETA prediction accuracy and uncertainty calibration.

On trajectory-disjoint Hangzhou data we compare C0μ (always 24.5 s), C0 (distance/speed), C1 (deterministic LSTM), and C2 (heteroscedastic LSTM with Gaussian NLL). Of 89 EMV trajectories, 86 were split by vehicle into 60/13/13 for training, validation, and test. The target is time to first arrival at the current intersection. Nine test vehicles produced 115 overlapping 10-step windows, so the windows are not independent; vehicle-averaged MAE was also checked.

Table R6. ETA prediction accuracy and uncertainty evaluation on the trajectory-disjoint Hangzhou test set.

MethodMAE (s)RMSE (s)NLLPICP@95%MPIW (s)
C0μ Constant training-set mean26.1435.915.000.95146.5
C0 Distance/Speed21.7331.10———
C1 Deterministic LSTM22.5233.115.010.8398.9
C2 Probabilistic LSTM20.5130.784.840.93113.3

C2 achieves the lowest point-prediction errors among the four estimators. Compared with C1, it reduces MAE by approximately 8.93% and RMSE by 7.05%. Compared with C0, the reductions are approximately 5.62% and 1.03%, respectively. These results indicate a modest improvement from probabilistic ETA modeling .

C2's standard deviation is multiplied by 1.71687 and floored at 27.9705 s, both fit on validation only. Its test PICP@95% is 0.93 and its MPIW is 113.3 s. C1 uses one residual standard deviation, 25.2197 s, giving PICP 0.83 and MPIW 98.9 s. C0 has no predictive distribution. C2 is calibrated, but its intervals are wide. The results show that C2 provides useful predictive uncertainty estimates after validation-based calibration, although its relatively wide prediction intervals should be considered when interpreting the coverage result.

4.2 Cross-policy evaluation of the frozen ETA predictor.

Changes in the MADQN policy may alter future traffic conditions and remaining travel-time distributions, potentially introducing prediction bias. To investigate this issue, we conducted an additional cross-policy evaluation.Under the reference MADQN policy (P0), C2 achieved an MAE of 20.51 s. The same predictor was then evaluated on newly collected Hangzhou trajectories generated by a newly trained MADQN controller (P2), using the same additional EMV demand.

Under P2, C2 achieved an MAE of 19.32 s. However, the mean predicted remaining time was 16.8 s, compared with a ground-truth mean of 29.1 s, corresponding to a systematic underestimation of 12.3 s. Figure 24 in the revised manuscript shows the residual distribution.

These findings suggest that the frozen ETA predictor retains a usable aggregate point-error level under the additional policy, but they do not establish robust prediction under arbitrary policy changes or resolve the predictor-controller coupling.

Reviewer_Ko6XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes an LLM-guided multi-agent reinforcement learning (MARL) framework for emergency traffic signal control. An LSTM predicts emergency vehicle (EMV) arrival times and uncertainty. LLM phase preferences are added to MADQN Q-values through residual fusion, while uncertainty-aware grounded action transformation (UGAT) adjusts actions for differences between simulation environments. The framework is trained in CityFlow and tested in SUMO on five datasets from three traffic networks. It reports lower overall and EMV average travel time (ATT) than the evaluated baselines, especially under heavy congestion.

Critical strengths
  1. Clearing an EMV route quickly can delay other traffic, especially during congestion. The paper addresses this trade-off across multiple intersections.
  2. The system combines ETA prediction, LLM phase preferences, and UGAT with MARL. Invoking the LLM less often than the MADQN controller avoids an LLM call at every signal decision.
  3. The five datasets span three networks and several demand levels. The method reduces overall and EMV ATT against the reported baselines, with clear gains under heavy congestion. The ablations offer some evidence that the modules help.
Critical weaknesses
  1. The framework combines established techniques in relatively independent modules, but does not show whether their interaction adds value. Several design choices also lack clear justification.
  2. It is unclear how the preference adapter and gate are trained, and parts of the supplementary code differ from the paper.
  3. The evaluation lacks EMV-specific baselines, comparisons with simpler modules, and statistical analysis.
Questions for authors
  1. Figures 1 and 2 introduce a learnable adapter and gate, and Eqs. (7) and (8) define their fusion. Yet the TD loss in Appendix C.4 (Eqs. (20) and (21)) uses only the original Q-values. Appendix F mentions further optimization in Stage 3 without specifying an objective for these parameters. How are the adapter and gate trained, and where do their gradients come from?
  2. Eq. (3) includes ETA mean and log-variance in the MADQN observation, while the supplementary code appears to use only the ETA estimate. Was predictive uncertainty used in the reported control policy? If so, how?
  3. Why predict ETA explicitly instead of feeding traffic history to a temporal MARL policy? Signal actions affect EMV arrival times, yet the LSTM is trained under a pretrained policy and then frozen. How is this coupling handled as the MADQN policy changes?
  4. The AI Use Statement says the LLM uses probabilistic EMV arrival estimates, but neither Section 4.3.1 nor Appendix D includes ETA mean or variance in the prompts. What ETA information, if any, was given to the LLM?
  5. The ablation does not show that an LLM is needed. How would rule-based or MLP preference generators perform?
  6. Why compare only with Fixed-time, MA2C, and HGAT, and not an EMV-specific method such as EMVLight?
  7. Section 3 says each intersection selects the EMV with the shortest remaining distance; the supplementary code selects the smallest predicted ETA. Which rule produced the reported results?
5 comments
AuthorsOpenReview ↗

Clarification of Module Inputs, Interactions, and Design Choices—Weakness 1; Questions 2, 3, 4, 5 and 7

1. Clarification of Module Inputs, Interactions, and Design Choices—Weakness 1; Questions 2, 3, 4, 5 and 7

We thank the reviewer for the careful assessment of the relationships among the proposed modules and for identifying several points requiring clarification.

1.1 Contributions and Complementary Roles of the Proposed Modules.

The proposed framework contains three main components: an ETA prediction module, an LLM-based phase-preference generation module, and an uncertainty-guided action grounding module (UGAT). Their contributions were examined through several complementary comparisons.

First, the original manuscript included three ablation configurations:w/o ETA Refinement; Repl . Preference w/ State; w/o UGAT Adaptation. These configurations examine the effects of removing or changing individual components within the proposed framework.

To further isolate the contribution of LLM-generated phase preferences, an additional comparison was conducted using a MADQN controller without the LLM preference module. This configuration retains the ETA and UGAT components but does not use LLM-generated phase preferences. It is compared with MLP-based, uniform, random, rule-based, and LLM-based preference-generation strategies under the same Hangzhou 4×4 network and traffic scenario. All six configurations receive the same input information, including the available EMV route information; only the preference-generation strategy differs. The results are as follows:

Table R1. Performance comparison of different preference-generation strategies on the Hangzhou 4×4 network.

MethodOverall ATTEMV ATTTPR_change
A0 MADQN404.59193.2525980
A2 MLP381.10196.12269641.1%
A3 Uniform420.32201.6226670
A3 Random409.46189.25268764.5%
A4 Rule-based380.28205.62269640.7%
A5 LLM368.84184.75270039.2%

The LLM-based strategy achieves the lowest overall ATT and EMV ATT, together with the highest throughput. Although the MLP and rule-based strategies reduce overall ATT relative to MADQN without LLM preferences, their EMV ATT values are higher. The random strategy produces the highest action-change rate but does not achieve the best overall or EMV performance. These results suggest that the quality of the generated phase preferences, rather than simply the frequency of action changes, is important to the observed performance.

Figure 22 in the revised manuscript further compares the action-change ratio with EMV ATT, showing that the LLM-based method achieves the lowest EMV ATT without having the highest action-change ratio. In particular, the random setting changes substantially more actions but does not achieve the same performance. Figure 23 in the revised manuscript tracks the cumulative EMV ATT during the 3600 s evaluation and shows that the performance ordering is maintained during the second half of the simulation. Together, these results suggest that the effectiveness of preference guidance depends on the direction and quality of the preference signal, rather than simply on how frequently it changes the original MADQN actions.

A further comparison evaluates the complete framework against a base MADQN controller that simultaneously removes ETA prediction, LLM-based phase preferences, and UGAT. The results are as follows:

Table R2. Performance comparison between the base MADQN controller and the complete framework on Hangzhou.

MetricMADQNOurs
Overall ATT436.37368.84
EMV ATT191.00 (5 completed EMVs)184.75 (8 completed EMVs)
AWC1.661.65
CO2965.47799.12
Fuel307.94254.88

On the Hangzhou 4×4 network, the complete framework achieves an overall ATT of 368.84 s, an EMV ATT of 184.75 s, an average waiting count of 1.65, CO₂ emissions of 799.12 g/vehicle, and fuel consumption of 254.88 g/vehicle. The base MADQN obtains 436.37 s, 191.00 s, 1.66, 965.47 g/vehicle, and 307.94 g/vehicle, respectively.

Figure 35 in the revised manuscript further compares the travel times of the completed EMVs within each method. At each shared rank from one to five, the proposed method has a shorter travel time than the base MADQN. However, this vehicle-level comparison is also subject to the different completion counts.

Taken together, the original ablations, the preference-generation comparison, and the complete-model comparison provide complementary evidence of the contributions of the three components. The ablations examine the effects of removing or changing individual components, the preference-generation comparison isolates the contribution of LLM-generated phase preferences, and the complete-model comparison evaluates the combined effect of the proposed components relative to the base controller. These experiments support the usefulness and complementary roles of the proposed modules.

AuthorsOpenReview ↗

Clarification of Module Inputs, Interactions, and Design Choices—Weakness 1; Questions 2, 3, 4, 5 and 7

1.2 Whether ETA Variance Is Used by the MADQN Controller.

The reviewer correctly identifies an inconsistency between the observation definition in Eq. (3) and the actual implementation. The probabilistic LSTM predictor estimates both the ETA mean and variance during training using a Gaussian negative log-likelihood objective. However, the MADQN controller directly receives only the predicted ETA mean as an observation feature. The predicted log-variance is not explicitly concatenated into the controller observation.

To examine whether the variance provides additional information for signal control, an additional comparison was conducted between the controller using the ETA mean alone and a variant that additionally includes the predicted ETA variance as an observation feature. The additional variance feature did not show a clear performance improvement under the tested setting. Therefore, the current implementation retains the ETA mean as the controller input.

The variance remains relevant to the probabilistic training of the ETA predictor and can influence the learned prediction mean through the shared model parameters. Nevertheless, it is not directly used as an additional observation feature by the MADQN controller.

To ensure consistency between the mathematical description and the implementation, Eq. (3) and its accompanying explanation will be revised to specify that only the predicted ETA mean is included in the MADQN observation.

1.3 Explicit ETA Prediction and Its Relationship to Policy Learning.

The explicit ETA predictor is introduced to provide an interpretable estimate of the time remaining before an EMV reaches the current intersection. This intermediate estimate can be evaluated independently and used directly by the signal-control policy, rather than requiring the policy to infer EMV arrival time implicitly from a history of traffic observations.

The reviewer also raises an important concern that signal-control actions can affect vehicle trajectories and consequently change the ETA distribution.

To investigate the effect of this policy change, we conducted an additional cross-policy evaluation. The C2 predictor, including its calibration parameters, was kept frozen without retraining. Under the reference MADQN policy (P0), C2 achieved an MAE of 20.51 s. The same predictor was then evaluated on newly collected Hangzhou trajectories generated by a newly trained MADQN controller (P2), using the same additional EMV demand.

Under P2, C2 achieved an MAE of 19.32 s. However, the mean predicted remaining time was 16.8 s, compared with a ground-truth mean of 29.1 s, corresponding to a systematic underestimation of 12.3 s. Figure 24 in the revised manuscript shows the residual distribution, where residuals are defined as predicted ETA minus ground-truth ETA. Although the aggregate point-prediction error remained around 19 s in this additional evaluation, the systematic early bias indicates that the frozen predictor is not policy-invariant.

These findings suggest that the frozen ETA predictor retains a usable aggregate point-error level under the additional policy, but they do not establish robust prediction under arbitrary policy changes or resolve the predictor-controller coupling. The current framework adopts a freeze-and-reuse strategy rather than online ETA adaptation. If a substantially different controller induces larger prediction bias, collecting trajectories under that controller and refitting or fine-tuning the ETA predictor would be a practical adaptation strategy. This adaptation has not yet been implemented.

1.4 Clarification of the Information Used by the LLM.

The reviewer identifies an ambiguity in the AI Use Statement. The intended meaning is that the LLM independently assesses EMV arrival urgency from the traffic and vehicle information provided in its prompt. It does not receive the probabilistic ETA predictions generated by the LSTM.

Specifically, the LLM prompt provides directly observed information, including phase-level queue conditions, EMV speed and remaining distance, the speed and distance of the preceding vehicle, the current intersection, downstream route intersections when available, and feasible phases for EMV passage. Based on these inputs, the LLM assesses the urgency of EMV passage and generates phase preferences. Therefore, the LLM's assessment of arrival urgency should not be confused with receiving or using the LSTM's predicted ETA values.

1.5 EMV Selection Rule.

The reviewer correctly points out that the manuscript's description of selecting the EMV with the shortest remaining distance does not match the implementation. The actual controller selects the EMV with the smallest estimated arrival time, rather than the shortest remaining distance. The manuscript will be corrected to state that the EMV with the minimum estimated arrival time is selected.

AuthorsOpenReview ↗

Training Mechanism and Implementation Consistency of the Preference Adapter and Gate—Weakness 2; Question 1 and Additional Baselines and Statistical Evaluation—Weakness 3; Questions 6

2. Training Mechanism and Implementation Consistency of the Preference Adapter and Gate—Weakness 2; Question 1

We thank the reviewer for identifying the insufficient description of the preference adapter and gate optimization. The revised explanation will clarify their parameterization, gradient flow, optimization objective, and interaction with the replay buffer and target network.

Given an eight-dimensional phase-preference vector π\pi, the Adapter maps it to a learnable residual vector:

hpref=Adapter(π).h_{\mathrm{pref}}=\mathrm{Adapter}(\pi).

The Gate is a single globally shared learnable scalar, rather than a separately parameterized value for each phase or intersection. Its output is g=σ(s)g=\sigma(s), and the final Q-values are computed as Qθ(s,a)+g⋅hprefQ_{\theta}(s,a)+g\cdot h_{\mathrm{pref}}.

The Gate parameter is initialized with (s=0.5), corresponding to an initial gate value of approximately 0.6225.

The Adapter, Gate, and MADQN Q-network are jointly optimized using the same RMSprop optimizer and the TD mean squared error. Implementation probes verify that the Gate receives non-zero gradients approximately (6.5×10−3)(6.5\times10^{-3}) and that the Adapter's 280 trainable parameters receive non-zero gradients and change during optimization. The Gate value increases from 0.6225 before training to 0.7996 after training, with a standard deviation of approximately 0.00024 over the final 10% of training. These results provide implementation-level evidence that the preference modules are trainable and that the Gate stabilizes during the final training stage.

The TD formulation in the original manuscript also requires correction. We acknowledge that Eqs. (20)–(21) used the original Q-values in their description and therefore did not accurately represent the implementation. In the revised formulation, the current Q-value incorporates the preference residual.

Accordingly, both the current Q-value and the TD target incorporate the preference residual, and action selection is also based on QfinalQ_{\mathrm{final}}. The target network periodically copies only the parameters of the Q-network. The preference residual in the TD target is calculated using the current Adapter and Gate parameters, while the entire target branch is detached from gradient computation. Therefore, the target branch does not propagate gradients to the Adapter or Gate.

For replay-memory updates, each transition stores both the current preference π\pi and the next-step preference π′\pi'. These stored preferences are retrieved during TD updates, rather than generated again by invoking the LLM. This keeps the preference signals associated with each stored transition fixed during replay.

The failure-handling mechanism is also clarified. The LLM is queried every ten simulation steps and when an EMV event triggers an update. If a request times out or fails, or if the output fails three format checks, the preference residual is set directly to zero. So the controller falls back to the MADQN Q-values without introducing an invalid preference residual. Between successful LLM updates, the most recent valid preference vector is retained.

The revised manuscript and supplementary implementation description will be aligned to make the above parameterization, optimization procedure, gradient flow, replay-buffer handling, and fallback behavior explicit.

3. Additional Baselines and Statistical Evaluation—Weakness 3; Questions 6

We appreciate the reviewer's suggestions for stronger baseline comparisons and statistical evaluation. Additional experiments have been conducted, include a max-pressure with EMV priority controller,an EMVLight-Based controller and evaluate the sensitivity of the proposed method to training randomness. The existing multi-network and multi-demand experiments are also retained to demonstrate performance across different traffic environments.

3.1 Comparison with Non-Learning Max-Pressure Controllers.

To provide a non-learning, EMV-aware signal-control baseline, two additional SUMO controllers were implemented and evaluated using the same 3,600 s simulation trips as the paper results: classical Max-pressure (MP) and Max-pressure with emergency-vehicle priority (MP-EMV). MP selects the green phase that maximizes the difference between inbound and outbound lane vehicle counts, without explicitly prioritizing EMVs. MP-EMV identifies the next traffic light on the EMV route, restricts the available actions to phases whose start lanes contain the EMV's exact current lane, and then applies the Max-pressure criterion among the eligible phases.

Table R3 summarizes the results across Hangzhou, the three Jinan demand levels, and New York. The effectiveness of explicit EMV prioritization depends on both traffic demand and network conditions. On Hangzhou, MP-EMV slightly reduces overall ATT from 415.79 to 414.08 s and EMV ATT from 205.75 to 198.62 s. The proposed Full model achieves lower values of 368.84 s and 184.75 s, respectively.

Reviewer_3NbuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the traffic signal control problem at multi-intersections prioritizing emergency vehicles (EMVs), aiming to reduce emergency vehicle travel time while minimizing congestion and disruption to general traffic. This method is based on a multi-agent deep Q-network (MADQN), integrating three modules: a heterogeneous variance LSTM to predict emergency vehicle arrival times and their uncertainties; a low-frequency operating LLM to generate signal phase preferences, adjusting Q values ​​through residuals and gate mechanisms; and uncertainty-aware action transition (UGAT) to handle traffic dynamic differences between CityFlow and SUMO. The main identifiable contribution of this paper is the integration of time prediction, LLM preference guidance, and cross-simulator adaptation into the emergency vehicle signal control process. Experiments covered three urban road networks and five traffic scenarios, but the originality and effectiveness of each module still require further comparison and validation.

Critical strengths

Strengths of this paper: • Cross-simulator evaluation adds value: Section 3 clearly states that prioritizing emergency vehicles cannot only pursue localized speed; it must also consider downstream clearing, backlog, and general traffic efficiency. The problem setting is reasonable. • The temporal prediction inputs reflect relevant traffic mechanisms: Section 4.1, in addition to distance and speed, incorporates preceding vehicle speed, vehicle spacing, queue length, and current phase, which helps describe congestion and waiting phenomena that cannot be captured by simply dividing distance by speed. • Separating the frequencies of LLM guidance and real-time control is a reasonable design: Section 4.3 allows MADQN to make continuous decisions, with LLM providing lower-frequency preference signals and setting up format checks and backup outputs, showing that the authors considered inference delays and output reliability. • Cross-simulator evaluation adds value: Table 1 includes indicators such as travel time, throughput, number of waiting times, and emissions, allowing observation of whether priority is accompanied by other traffic costs. • Cross-simulator evaluation increases the experimental value. CityFlow training and SUMO assessment provide opportunities to examine differences in traffic dynamics; the appendix also provides time-series curves, travel time distributions, and road network heatmaps to supplement information beyond the average values.

Critical weaknesses
  1. The novelty of UGAT and its relationship with existing research are unclear.
    The authors list UGAT as a major contribution, but Da et al. (2023), which has already been cited, proposed a UGAT with the same name and also used forward/backward dynamic models and uncertainty judgments to choose whether to switch actions. It is currently insufficient to determine whether this paper proposes a new adaptation method or mainly integrates existing methods into the EMV system. This directly affects the establishment of the core academic contribution (see Sections 1, 2, and 4.4; Appendices E and F.) Please compare the differences between existing UGAT and this paper in model architecture, sources of uncertainty, threshold rules, and multi-agent processing. This paper uses backward action distribution entropy; its differences from existing methods should be explained, and a direct comparison of the original UGAT under fair data budget should be added. See reference [7].

  2. Existing ablation is insufficient to prove that LLM is a necessary source of performance improvement. Repl. Preference w/ State compares preference guidance and semantic feature encoding, and cannot rule out the possibility that simple rules, small neural networks, or additional route information can bring the same benefits. LLM receives information such as downstream routes, so improvements may also stem from increased information rather than language model inference.
    Please add control groups such as completely removing LLM, regular phase preference, small MLPs with the same input, and uniform/random preference; keep the input information, training steps, and model capacity as consistent as possible, and report the gate value and the proportion of actions actually changed by LLM.

  3. The training and backup behavior of the residual preference module lacks a complete definition. Action selection uses Q_final, but the TD target and loss only list the original Q_i, without clearly explaining how the adapter and gate obtain gradients, or whether the target network contains preference branches. Even after uniform preference is processed by a biased MLP, uneven action correction may still occur, and it cannot be directly considered a neutral backup (see Section 4.3.3, Equations (7) to (8); Appendix C.4, Equations (20) to (22); Appendix F). Please provide a complete training algorithm, clearly outlining how update parameters, TD targets, and preference signals are stored or reconstructed at each stage. If invalid outputs require reverting to the original MADQN, the residual should be explicitly set to zero or the gate should be closed, and the behavior verified.

  4. Ambiguity in the Execution Location and Generalization Scope of Cross-Environment Adaptation. The main text describes UGAT correcting actions before execution, but the appendices mainly describe the use of UGAT during CityFlow retraining, followed by direct deployment of SUMO. These involve different mapping directions and deployment costs. Training already uses SUMO interactive data, therefore it cannot be interpreted as zero-sample transfer without contact with the target environment; individual training and testing across cities does not equate to cross-network generalization (see beginning of Section 4; Section 4.4; Figures 1 and 7; Appendices E and F.).

  5. Insufficient comparative baselines and lack of methods directly targeting emergency vehicles. The only external baselines are Fixed-time, MA2C, and HGAT; the cited EMVLight was not included in the experiment. EMVLight has addressed the joint improvement of emergency vehicle passage and general traffic efficiency, and is a highly relevant comparison object. Please prioritize the inclusion of EMVLight, Max-pressure with EMV priority rules, and the complete backbone MADQN. This paper uses a preset route, while EMVLight includes dynamic routing. A separate control comparison with the same fixed route should be set up to avoid confusion of signal control benefits due to route differences.

  6. Insufficient Statistical Evidence and Some Claims Exceed Table Support. Table 1 does not provide independent training seeds, standard deviations, or confidence intervals. The full model is not optimal in all metrics: Jinan-Medium's EMV ATT is 125.89 seconds, slightly worse than the 125.56 seconds with ETA removed; Hangzhou's AWC is 1.65, worse than the 1.55 with UGAT removed; Jinan-High's traffic volume is also lower than the partially ablation version.

  7. The effectiveness of probabilistic ETA and uncertainty is not directly verified. Existing ablation methods simultaneously remove the temporal model and uncertainty, failing to distinguish the contributions of both. Gaussian NLL training does not guarantee that the prediction interval is calibrated; the low entropy of the inverse action distribution does not guarantee correct transformation.

  8. Oversimplification of Multiple Emergency Vehicles and Actual Traffic Signal Limits. Considering only the EMV with the shortest remaining distance at the same intersection may ignore competing requests from other directions. Legitimate, conflict-free phases are insufficient to describe time constraints such as yellow lights, all-red lights, shortest green lights, and pedestrian clearance, limiting the method's applicability to complex events and real-world deployments.

Questions for authors

• Please clearly list the data sources, action execution environment, whether UGAT is retained in the final deployment, and record the original or transformed actions in the replay buffer. Differentiate between zero-sample migration, adaptation using target data, and cross-network generalization, and add fine-tuning baselines with the same target data budget.

• Use multiple independent training and traffic demand seeds, and report paired differences and confidence intervals. Calculate improvement rates separately for external baselines and internal ablation; the 24.96% and 44.88% in the abstract should be noted as being compared to Fixed-time, and the "consistent improvement" statement should be corrected.

• Please report the MAE, RMSE, NLL, prediction interval coverage, and width of the ETA; compare deterministic LSTM, input-only average ETA, and average plus variance. Examine the relationship between UGAT entropy and transformation error. ETA data comes from a frozen baseline strategy; prediction shifts after strategy changes should also be examined.

• Please add stress tests for simultaneous EMV arrivals from different directions, reporting worst-case or high-quantile delays; explicitly define flag switching constraints and ensure all comparison methods use the same constraints.

• Section 4, page 3, lines 150–151: “end-to-end rather than trained in isolation” is inconsistent with the phased training and frozen parameter descriptions in Appendix F. Closed-loop integration and end-to-end joint optimization should be distinguished.

• Section 4.2.2, Equation (4), Appendix C.3: P_i,t is located in [0,1], therefore the fixed penalty −1 is the maximum penalty magnitude in this range, but the priority effect still depends on the role duration, discount rate, and global feedback. The normalization method and weights should be supplemented.

• Appendix C.3, Equation (18): traffic pressure is a weighted indicator of the number of vehicles entering the lane, queuing, and delays, and does not include the upstream and downstream differences of typical Max-pressure. It is recommended to rename it a congestion indicator, or explain the differences.

• Section 4.3.1 and Appendix D: The summary indicates the probability of LLM use ETA, but the mean and variance of the input tuples in Equation (24) are not explicitly listed. A complete prompt should be provided to confirm whether the LLM has received this information.

• Figure 2, Section 4.3.3: Supplement with the exact DeepSeek version, decoding parameters, call frequency, triggering conditions, and expired preference handling; asynchrony itself does not prove that latency has no impact.

• Section 4.4, Equations (9) to (12): The forward model is referred to as a probabilistic model, but only a single predicted observation is listed. The output distribution and training loss should be defined, distinguishing between ETA variance and inverse action entropy.

• Section 5.1, Appendix G.1, Figure 8: Distinguish between veh/h and the total number of vehicles in the simulation, and supplement with the warm-up, demand input, and clearing period lengths.

• Tables 1, 2, Appendix G.4: Define whether ATT includes vehicles with incomplete trips; emissions and fuel consumption are per vehicle averages or total network volume; queue volume in Figure 10 uses veh, while Table 2 uses m; this should be standardized or explained. • Appendix G.2, Figure 9: Supplement the rules for EMV quantity, dispatch time, and route generation, confirming that all methods use the same EMV task.

• Figures 1, 2, and 7: High information density; it is recommended to enlarge the font size and clearly indicate the training and deployment process, data sources, and gradient update direction.

6 comments
AuthorsOpenReview ↗

UGAT Novelty and Cross-Environment Adaptation—Weakness 1, 4; Question 1

We now separate our UGAT from Da et al. (2023) and separate simulator adaptation from cross-city transfer.

1.1 UGAT methodology and comparison with the original implementation.

The forward–inverse grounding framework comes from Da et al. (2023). Both versions use a target-domain forward model and a source-domain inverse model, and replace the policy action only when uncertainty is low. Ours trains the forward model on SUMO and the inverse model on CityFlow with cross-entropy. Uncertainty is the Shannon entropy of the inverse action distribution, compared with a fixed threshold. Original UGAT uses an evidential inverse model, Dirichlet vacuity, and an adaptive threshold. Our entropy measures action ambiguity, not the joint uncertainty of the two models. A fixed threshold was adopted because it provides a consistent acceptance criterion and avoids additional temporal fluctuations introduced by dynamically updating the threshold in response to changing traffic conditions.

To evaluate these differences, we conducted a direct comparison among the proposed implementation, the original UGAT mechanism, and the configuration without UGAT adaptation. The results are summarized in Table R1.

Table R1. Performance comparison of different uncertainty-guided action grounding strategies on the Hangzhou network.

MethodOverall ATT (s)EMV ATT (s)TPAWC
w/o UGAT Adaptation384.26216.3826931.55
Original UGAT457.7176.424273.23
Ours368.8184.827001.65
Ours has the lowest overall ATT (368.8 s) and the highest throughput (2700). Original UGAT has a lower EMV ATT (176.4 s) but a much worse overall ATT (457.7 s) and throughput (2427). This comparison indicates that the original UGAT provides stronger emergency-vehicle prioritization in this scenario, but at a substantially greater cost to general traffic efficiency. Figure 16 in the revised manuscript shows the same pattern over time: original UGAT ends with the lowest cumulative EMV ATT and worse overall efficiency. Figure 17 shows that its rule accepts 51.3% of candidate decisions, and the adaptive threshold moves to 0.51.

1.2 Target-environment adaptation and cross-city transfer.

The revision separates zero-shot transfer, target-environment adaptation, and cross-network generalization. On Hangzhou, one hour of SUMO decisions across 16 intersections trains the forward model. The inverse model and policy are trained in CityFlow. During adaptation, a low-entropy inverse prediction may replace the selected phase, and the replay buffer stores the executed phase. At the final SUMO test, replacement is disabled, so the result is the adapted policy, not test-time replacement.

F0 has no UGAT adaptation. F1 trains the forward model on SUMO records. F2 is a matched-budget fine-tune of F0 on a different Hangzhou flow, then tested on the same flow.

Table R2. Comparison of target-environment adaptation and matched-budget policy fine-tuning on Hangzhou.

SettingOverall ATTEMV ATTTPAWC
F0, proposed ablation384.26216.3826931.55
F1, proposed method368.84184.7527001.65
F2, matched-budget fine-tuning411.43194.1325782.65

Relative to F0, F1 lowers overall ATT by 15.42 s and EMV ATT by 31.63 s, and raises throughput by 7 vehicles, while AWC rises from 1.55 to 1.65. F2 is worse than F0 and F1 on overall ATT, throughput, and AWC. In Figures 18 and 19, 90% of F1 trips have a waiting-time share below 9.0%, versus 20.8% for F2. The medians are 1.9% and 3.7%.

A Hangzhou-trained policy is tested on Jinan-Low with no Jinan training or adaptation. Both policies are evaluated in SUMO at the same 4365-vehicle demand.

Table R3. Cross-city transfer performance from Hangzhou to Jinan-Low.

SettingOverall ATTEMV ATTTPAWCCO2Fuel
Proposed Jinan-Low, trained in Jinan291.67112.2241422.01639.59204.00
Hangzhou policy tested on Jinan-Low409.64106.1130212.53956.02304.93

EMV ATT falls from 112.22 s to 106.11 s, but overall ATT rises from 291.67 s to 409.64 s and throughput falls from 4142 to 3021. Figures 20 and 21 show the resulting congestion, and more than one thousand vehicles remain unfinished. These results demonstrate that the Hangzhou-trained policy can be evaluated on a different city network without Jinan-specific training, but its overall traffic performance remains substantially below that of the Jinan-trained policy. The Jinan run is cross-city transfer without target-city training. The Hangzhou run is cross-simulator adaptation, because SUMO data train the forward model. It is not strict zero-shot transfer.

AuthorsOpenReview ↗

UGAT Novelty and Cross-Environment Adaptation—Weakness 1, 4; Question 1 and Necessity of LLM Guidance and Adapter/Gate Validation—Weakness 2, 3; Questions 8, 9

1.3 Reliability of entropy-based uncertainty estimation.

At four central Hangzhou intersections, MADQN and the inverse model stayed frozen, and UGAT did not change the executed phase. After each transition, the inverse model predicted that phase. A mismatch was counted as an error. The 2396 decisions were grouped by entropy.

The error rate rises from 5.9% to 39.7%, 68.4%, and 87.9%. The lowest bin contains 2146 of 2396 transitions (89.6%). Within that bin, the four intersections range from 4.6% to 7.0%, and the trend is not strictly monotonic at every intersection.

Table R4. Inverse-model action-prediction error across entropy intervals.

Entropy intervalNumber of transitionsEstimated incorrect predictionsWeighted error rate
0–0.422,1461275.9%
0.42–0.831415639.7%
0.83–1.25765268.4%
1.25–1.66332987.9%
Total2,39626411.0%

Higher entropy is associated with more action errors, but low entropy does not guarantee a correct action. These comparisons, transfer protocols, and the entropy limitation are now in the revised manuscript.

2. Necessity of LLM Guidance and Adapter/Gate Validation—Weakness 2, 3; Questions 8, 9

2.1 LLM Implementation and Preference-Generation Comparisons

On Hangzhou we kept the pretrained MADQN, ETA features, and UGAT fixed, and changed only the preference source: none, a queue rule with an EMV bonus, an MLP using the same numerical inputs, uniform, random, and the LLM.

The LLM has the lowest overall ATT (368.84 s) and EMV ATT (184.75 s). The rule and MLP lower overall ATT but raise EMV ATT above MADQN, to 205.62 s and 196.12 s. Random reaches 189.25 s EMV ATT without a matching overall-ATT gain.

Table R5. Performance comparison of different preference-generation strategies on the Hangzhou 4×4 network.

MethodOverall ATTEMV ATTTPR_change
A0 MADQN404.59193.2525980
A2 MLP381.10196.12269641.1%
A3 Uniform420.32201.6226670
A3 Random409.46189.25268764.5%
A4 Rule-based380.28205.62269640.7%
A5 LLM368.84184.75270039.2%

The action-change ratios are 40.7% (rule), 41.1% (MLP), 64.5% (random), and 39.2% (LLM). Uniform changes no action, because the same value on every phase does not change the ranking. Figures 22 and 23 show that the lowest EMV ATT is not the setting that changes the most actions, and this ordering holds in the second half of the 3600 s run.

The prompt does not include the LSTM-predicted ETA mean or its uncertainty. These quantities are model estimates rather than direct observations, and are therefore not presented to the LLM as measured traffic facts. Instead, the LLM receives the observed traffic information needed to assess the urgency of each emergency vehicle.

The prompt consists of a task instruction and a traffic-information input. The task instruction asks the model to act as a traffic-signal expert and assign a preference to each of the eight phases at each intersection. A higher preference indicates that the corresponding phase should be activated sooner. At each intersection, the eight preferences are constrained to ([0,1]) and must sum to one. The instruction asks the model to consider both queued regular traffic and emergency-vehicle priority, and to increase the preference of phases that allow approaching emergency vehicles to pass. The input gives phase-level waiting vehicles and, for each EMV, speed, remaining distance, the speed and distance of the vehicle ahead, the current intersection, downstream intersections when available, and feasible phases. The output is JSON only. After three failed format checks, the preference residual is set to 0.

The LLM is DeepSeek-Flash, with the decoding parameters set to temperature=0.7\mathrm{temperature}=0.7 and maxtokens=2000\mathrm{max_tokens}=2000. The API call is blocking, so the corresponding decision step waits for the response. A preference update is triggered every 10 simulation steps and by an emergency-vehicle event.

2.2 Adapter/Gate Validation

Given an eight-dimensional preference π\pi, the Adapter maps it to a residual:

hpref=Adapter(π).h_{\mathrm{pref}}=\mathrm{Adapter}(\pi).

The Gate is one shared scalar, g=σ(s)g=\sigma(s), and Qθ(s,a)+g⋅hprefQ_{\theta}(s,a)+g\cdot h_{\mathrm{pref}} is the value used for action selection. It is initialized at s=0.5s=0.5, so g≈0.6225g\approx 0.6225.

The Adapter, Gate, and Q-network are trained jointly with RMSprop on the TD squared error. The Gate gradient is about 6.5×10−36.5\times10^{-3}, and all 280 Adapter parameters receive gradients. After training, gg rises from 0.6225 to 0.7996, with a standard deviation of about 0.00024 over the last 10% of training.

Eqs. (20)–(21) previously used the raw Q-values. The current Q-value now includes the preference residual.

AuthorsOpenReview ↗

ETA Prediction and Uncertainty Reliability—Weakness 7; Questions 3, 10 and Additional Baselines and Statistical Evaluation—Weakness 5, 6; Question 2

Action selection and the TD target both use QfinalQ_{\mathrm{final}}. The target network copies only Q-network parameters. Its residual uses the current Adapter and Gate but is detached, so it does not train them.

Each transition stores the current preference π\pi and the next preference π′\pi'. Replay reuses them and does not call the LLM again.

A uniform preference through the Adapter is not a neutral backup: it still changed 0.39% of actions in a fixed-Q test. An invalid call, a timeout, or three failed format checks now sets the residual to 0.

That decision then matches MADQN exactly. The previous preference is not reused after a failed update.

The preference comparison, prompt, Adapter/Gate training, corrected TD target, replay storage, and zero-residual fallback are now described in the revision.

3. ETA Prediction and Uncertainty Reliability—Weakness 7; Questions 3, 10

We thank the reviewer for raising concerns about the reliability of ETA prediction and the use of predictive uncertainty in emergency traffic control.

3.1 ETA prediction accuracy and uncertainty calibration.

On trajectory-disjoint Hangzhou data we compare C0μ (always 24.5 s), C0 (distance/speed), C1 (deterministic LSTM), and C2 (heteroscedastic LSTM with Gaussian NLL). Of 89 EMV trajectories, 86 were split by vehicle into 60/13/13 for training, validation, and test. The target is time to first arrival at the current intersection. Nine test vehicles produced 115 overlapping 10-step windows, so the windows are not independent; vehicle-averaged MAE was also checked.

Table R6. ETA prediction accuracy and uncertainty evaluation on the trajectory-disjoint Hangzhou test set.

MethodMAE (s)RMSE (s)NLLPICP@95%MPIW (s)
C0μ Constant training-set mean26.1435.915.000.95146.5
C0 Distance/Speed21.7331.10———
C1 Deterministic LSTM22.5233.115.010.8398.9
C2 Probabilistic LSTM20.5130.784.840.93113.3

C2 has the lowest errors: MAE and RMSE are about 8.93% and 7.05% below C1, and 5.62% and 1.03% below C0. The gain is modest. Distance/speed also beats the constant mean.

C2's standard deviation is multiplied by 1.71687 and floored at 27.9705 s, both fit on validation only. Its test PICP@95% is 0.93 and its MPIW is 113.3 s. C1 uses one residual standard deviation, 25.2197 s, giving PICP 0.83 and MPIW 98.9 s. C0 has no predictive distribution. C2 is calibrated, but its intervals are wide. The results show that C2 provides useful predictive uncertainty estimates after validation-based calibration, although its relatively wide prediction intervals should be considered when interpreting the coverage result.

3.2 Effect of ETA mean and variance information on traffic control.

Adding the predicted variance as an observation did not clearly beat the ETA mean alone, so the controller keeps the mean. The variance is still used to train the predictor.

3.3 Cross-policy evaluation of the frozen ETA predictor.

The predictor is fit under one frozen controller and then held fixed, so a new policy can shift the arrival-time distribution. C2, including its calibration, was not retrained. Under the reference policy P0 its MAE is 20.51 s. Under a newly trained MADQN policy P2, with the same extra EMV demand, the MAE is 19.32 s.

Under P2 the mean prediction is 16.8 s against a true mean of 29.1 s, a bias of −12.3 s. Figure 24 shows this early bias, so the frozen predictor is not policy-invariant.

These findings suggest that the frozen ETA predictor retains a usable aggregate point-error level under the additional policy, but they do not establish robust prediction under arbitrary policy changes or resolve the predictor-controller coupling. The current framework adopts a freeze-and-reuse strategy rather than online ETA adaptation. If a substantially different controller induces larger prediction bias, collecting trajectories under that controller and refitting or fine-tuning the ETA predictor would be a practical adaptation strategy. This adaptation has not yet been implemented.

4. Additional Baselines and Statistical Evaluation—Weakness 5, 6; Question 2

4.1 Run-to-Run Stability Across Independent Training Seeds.

The full model was retrained with seeds 3047, 42, 1234, 2026, and 7777 on the same Hangzhou demand. Seed 3047 is the reported run.

Table R7. Hangzhou results over five training seeds.

seedOverall ATTEMV ATTTPAWCCO₂Fuel
3047368.84184.7527001.65799.1254.9
42369.09188.6327001.68800.5255.3
1234369.53194.0026991.70801.1255.5
2026369.28198.1326941.68800.0255.2
7777369.29194.6327051.68802.2255.9
Mean ± SD369.20 ± 0.26192.03 ± 5.302699.6 ± 3.91.68 ± 0.02800.6 ± 1.1255.3 ± 0.4

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.