acceptodds
Under review as a conference paper at ICLR 2027

AMHSTrafficLab: An Industrial Benchmark for Routing and Regional Traffic Control in Large-Scale Resource-Competitive Transport Networks

Abstract

In industrial settings, such as automated material handling systems (AMHS) in modern semiconductor manufacturing, transport networks must serve uneven, continuously arriving demand with large vehicle fleets and limited track space. Routing decisions interact with rules that control resource competition, including vehicle-following rules, right-of-way mechanisms, and regional admission control; together, they shape traffic flow and affect the risks of congestion, deadlock and throughput collapse. However, comparisons across industrial routing studies are difficult to reproduce when simulator implementations and accompanying control policies are incompletely documented. We introduce AMHSTrafficLab, an industrial benchmark for reproducibly evaluating and developing routing and regional traffic control in large-scale AMHS: it provides industrially grounded vehicle dynamics, track layouts, workloads, and standardized interfaces for classical, heuristic, and learning-based policies. Using the benchmark, we evaluate 13 routing methods across 7 fleet–workload scenarios and 3 regional-control settings. Our results show that without regional-control, the system deadlocks frequently regardless of the routing algorithm used, demonstrating that routing performance cannot be interpreted independently of the accompanying traffic-control policy. A proposed regional admission-control baseline reduces throughput-collapse runs from 265/910 to 9/910 and substantially suppresses local deadlocks. With regional control, we further compare different routing methods and find that learning-based methods do not consistently outperform congestion-aware heuristics, and their relative performance varies with workload and fleet density. These findings highlight the benchmark's value for evaluating routing and traffic control together and for developing future algorithms.

Primary area:
datasets and benchmarks
Keywords:
industrial benchmark, multi-agent routing, traffic control, automated material handling systems, lifelong pickup and delivery, deadlock prevention, reinforcement learning.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1Uh6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AMHSTrafficLab, an industrially motivated benchmark for evaluating routing and regional traffic-control algorithms in AMHS, particularly OHT networks used in semiconductor manufacturing. The benchmark models large-scale directed transport networks with persistent vehicle fleets, dynamically arriving pickup-and-delivery tasks, vehicle-following dynamics, shared-resource competition, intersection access coordination, and regional traffic-control mechanisms. A central motivation is that routing algorithms cannot be evaluated independently of the traffic-control policies governing access to congested regions and shared resources. To support reproducible evaluation, AMHSTrafficLab provides configurable simulation scenarios, standardized policy interfaces, and implementations of classical, heuristic, and learning-based routing algorithms. The main results show that throughput collapse occurs frequently without regional control. Across 910 runs, the number of collapse cases decreases from 265 without regional control to 9 under regional admission control, and further to 3 when empty-vehicle rebalancing is added.

Critical strengths

Strength 1

The paper addresses an important limitation of existing routing evaluations: the behavior of a routing policy depends not only on its path-selection strategy but also on the traffic-control mechanisms governing access to shared resources. AMHSTrafficLab provides a unified simulation environment. It allows researchers to study routing policies under realistic forms of resource competition and evaluate system-level failures that may not be adequately reflected by conventional shortest-path or travel-time metrics. 

Strength 2

The paper evaluates 13 routing algorithms across seven scenarios, three regional-control settings, and ten evaluation seeds per configuration. The benchmark covers different network layouts, fleet sizes, traffic loads, and routing strategies, including classical shortest-path algorithms, congestion-aware heuristics, and reinforcement-learning-based approaches. The experiments use common task streams and initial vehicle states within matched scenario–seed configurations, improving the comparability of the results. An important finding is that learning-based methods do not consistently outperform congestion-aware heuristics.

Critical weaknesses

Weakness 1

The experimental evidence does not sufficiently establish how closely the simulated system reproduces the behavior of real industrial AMHS deployments. In particular, the paper does not provide a direct comparison between simulated and observed operational measurements, such as vehicle travel times, queue lengths, congestion patterns, task cycle-time distributions, or deadlock frequencies. The benchmark also relies on workloads generated from fixed From–To demand matrices. Although tasks arrive dynamically, the underlying spatial demand distributions remain fixed within each scenario. In actual semiconductor manufacturing environments, transport demand can vary substantially with production schedules, equipment availability, and process disruptions. 

Weakness 2

The paper evaluates routing performance using several complementary metrics, which is a strength. However, the primary efficiency comparisons are based on averages computed only over runs that do not experience throughput collapse. A routing policy that performs well in its successful runs but occasionally experiences severe congestion or collapse may appear competitive in terms of mean cycle time, even though its overall operational reliability is inferior to that of a consistently stable policy. This issue is especially relevant in industrial transport systems, where rare but severe failures can be more consequential than modest improvements in average travel efficiency. The detailed appendices are valuable, but documentation alone does not fully substitute for an independently executable benchmark.

Questions for authors

Q1

The paper introduces a comprehensive industrial simulation benchmark for routing and regional traffic control in automated material handling systems. While the benchmark appears practically valuable, its primary contributions seem more closely related to industrial simulation, transportation optimization, and operations research than to machine learning methodology. Could the authors clarify the intended contribution of AMHSTrafficLab to the broader machine learning community and, specifically, its relevance to ICLR? What novel machine learning research questions does this benchmark enable that cannot be adequately investigated using existing multi-agent reinforcement learning or transportation benchmarks? 

Q2

The benchmark focuses on routing and regional traffic control in industrial transport networks, a problem closely related to established operations research and combinatorial optimization. This raises a fundamental question regarding the motivation for introducing this problem as a machine learning benchmark. In particular, what unique machine learning challenges does AMHSTrafficLab introduce that cannot be adequately addressed by existing optimization or control methods? Without demonstrating such opportunities, the contribution appears primarily to be an industrial simulation and algorithm-comparison platform rather than a benchmark that addresses a compelling machine learning research problem. 

Q3

The benchmark incorporates layouts and workloads derived from semiconductor manufacturing environments, but the paper does not present a direct comparison with observed operational measurements. Could the authors clarify whether any simulation components have been calibrated or validated against real AMHS operational logs? 

Q4

The paper shows that admission control substantially reduces throughput collapse, but the baseline relies on predefined regional water-level thresholds and region configurations. How sensitive are the reported improvements to these choices?

4 comments

We thank the reviewer for recognizing the value of our work and for the insightful feedbacks. We explain the reviewer's concerns and answer the questions below.

1. Relevance to ICLR and the learning problem (Q1, Q2)

Our first contribution is that we expose the AMHS scene, a scene of great practical importance, as a complex joint-action traffic MDP, and provides a platform for publicly learning and fairly evaluating policies targeting this scene. In the MDP, vehicle routers choose paths and regional controllers regulate entry or rebalance idle vehicles; both are agents whose decisions determine fleet movement and task completion.

This MDP is a complex one: even with a fixed regional policy, two routes for each of mm vehicles give 2m2^m route assignments. A vehicle's travel time depends on the routes and entry permissions of others. A queue outside one region can extend through a merge and delay vehicles whose routes never hit that region. Targeting routers in traffic systems of such complexity, we raise a question: With regional control fixed, which parts of the traffic state should a router observe, what history should it retain, and how should it represent value or cost to improve fleet performance?

Under admission control, exponentially averaged delay routing (EDR) had the lowest mean task cycle time among collapse-free methods in all seven scenarios (Table 3). Its decision process uses two designs:

  • Explicit memory of edge traversal times. Whenever any vehicle completes edge ee, EDR updates an exponential moving average of its observed traversal time. This measurement includes delays from vehicle following, merge queues, and regional admission.
  • Shared, scalable cost for each edge. EDR periodically publishes each t^e\hat t_e as an edge cost TeT_e. At node ii with destination tt, it chooses the successor jj minimizing Q~EDR(i,t,j)\widetilde Q_{\mathrm{EDR}}(i,t,j), the shortest-path cost from ii to tt via jj under the published costs (Eq.(5)). A single set of ∣E∣|E| costs serves all vehicles and destinations (Appendix D.2).

We added a matched DQN experiment (Section 5.2). DQN learns destination-conditioned next-hop values from traffic features. We trained and evaluated DQN with or without Q~EDR\widetilde Q_{\mathrm{EDR}} as an additional feature for each candidate next hop. Independently, we either froze DQN's weights during evaluation or updated them online from new traversal times. Below, each cell reports throughput (tasks/h) / mean task cycle time (s) over ten matched seeds (0–9), with 20,000 simulated seconds per run under CAC_A. All 120 runs were collapse-free.

EDR featureOnline weight updatesSMALL 150MID 225LARGE 300
OffOff3039.9 / 126.466010.5 / 96.297265.2 / 96.92
OffOn3098.4 / 125.186010.1 / 96.367264.7 / 97.07
OnOff3099.3 / 122.836010.5 / 95.637265.8 / 96.30
OnOn3097.9 / 123.606010.8 / 95.577265.2 / 95.94

Results show that under regional control, DQN achieves shorter task cycles in all comparisons when supplying fleet-wide traversal history as a shared, scalable cost, motivating future machine learning algorithms to utilize or exploit these two aspects.

2. Industrial calibration (Q3, W1)

In earlier collaboration with our industrial partners, we anonymized facility maps, calibrated From–To demand from industrial process routes, and set vehicle-motion parameters using manufacturer specifications. The operational data and manufacturers' identities cannot be disclosed for confidentiality. However, certain aspects we adopted in layouts, transport demand, and vehicle motion already appear in relevant literature. We added Appendix H.1–H.3 to the paper for explanation.

The general idea of admission control comes from industry, but the control regions and water-level thresholds are specifically engineered for our benchmark topologies. See the answer to Q4 below.

3. Region and threshold sensitivity (Q4)

We amended Appendix H.4 to explain the design rationale of regional control and to exhibit a sensitivity analysis using 10 alternative regional configurations in LARGE 300. Without regional control, Dijkstra, EDR, and QLBWR experienced throughput collapse in 10/10, 6/10, and 9/10 runs, respectively; across the 10 alternative admission configurations, the highest collapse counts for the same methods were 4/10, 0/10, and 3/10, respectively (Tables 51 and 55). The reduction in collapse therefore persists across the tested regional configurations with From–To demand held fixed.

4. Interpreting the metrics (W2)

We thank the reviewer for pointing this out, and we admit that the original paper did not make it clear. Because sustained service is essential, collapse counts over all seeds take precedence. The reported cycle-time and throughput means should be fairly comparable only among policies with zero observed collapses on the same scenario and seeds.

Hi Reviewer 1Uh6,

We are grateful for your thorough feedback, which helped us better articulate the paper's relevance to the ML community and strengthen our real-world validation and reliability evaluation. Before the discussion period concludes, we wanted to check in on whether our responses have resolved your concerns. Please let us know if there is anything else we can clarify—we would be happy to follow up. Thank you again for your valuable input.

Thank you for your detailed rebuttal and supplementary experiments.

However, I still do not believe that the proposed benchmark makes a sufficiently compelling contribution to the machine learning community.

Despite the additional experiments, the study remains primarily focused on the fields of industrial simulation and traffic optimization, and lacks sufficient evidence to demonstrate that it addresses unique machine learning challenges or offers methodological insights relevant to ICLR.

Therefore, I regret to say that I am upholding my original score.

Dear Reviewer 1Uh6,

Thank you for reading our rebuttal and for your candid feedback. We understand and respect your decision, and we truly appreciate your time throughout the review process.

Best regards, The Authors

Reviewer_3s24OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

AMHSTrafficLab is a simulation benchmark for routing and regional traffic control in semiconductor material-handling networks. It combines directed layouts, continuous vehicle motion, pickup-and-delivery demand, shared-resource coordination, and extensible policy interfaces. The evaluation covers thirteen routing methods, seven fleet-workload scenarios, three control settings, and ten evaluation seeds. Admission control reduces throughput-collapse runs from 265 of 910 to nine of 910, while also reducing confirmed circular waits. Under admission control, the online travel-time heuristic EDR achieves the lowest mean cycle time among collapse-free methods across the seven scenarios. The benchmark highlights the dependence of routing performance on the accompanying control policy.

Critical strengths
  1. The benchmark captures consequential interactions between routing and resource coordination. The simulator description specifies vehicle following, merge permissions, region admission, task execution, and extensible interfaces. The evaluation separates persistent circular waits from throughput collapse, allowing researchers to study local blocking and system-level service loss without conflating these distinct outcomes.

  2. The control comparison is broad and produces a substantial effect. The evaluation crosses thirteen routers with seven scenarios and three control settings under matched task streams and initial states. Admission reduces collapse from 265 to nine runs and detector-positive runs from 295 to six. The transfer evaluation further tests frozen-checkpoint transfer across fleet sizes and layouts.

Critical weaknesses
  1. The industrial grounding of the control result is insufficiently validated. The workload generator constructs stationary demand from process-route mixtures, and the regional-control configuration adjusts regional thresholds using operational experience. The paper does not quantify agreement with measured traffic or sensitivity to engineered choices. Provide calibration evidence or a threshold and demand perturbation study to establish how broadly the benchmark findings apply.

  2. The learning-versus-heuristic comparison does not characterize training uncertainty or equal adaptation opportunities. The evaluation uses one frozen checkpoint per method and scenario, whereas EDR continues updating travel times online. Ten evaluation seeds do not measure training variability, and the reported results give conditional means without confidence intervals. Report training variability and distinguish frozen-policy comparison from adaptation-matched comparison.

Questions for authors
  1. Which motion, demand, and regional-control parameters were calibrated against measured operations, and which were manually selected? Please provide the available calibration error or sensitivity evidence for the choices that determine the collapse reduction.

  2. What training seeds, budgets, checkpoint-selection criteria, and training control settings produced the learned baselines? Are additional training runs available, and does the routing ranking persist when online adaptation is permitted or disabled consistently across the compared methods?

  3. The experimental setup reports Python 3.12.13, while the distribution instructions specify a Python 3.11 compiled wheel. Which review-accessible release reproduces the reported matrix, and what equivalence checks connect that release to the experimental runtime?

2 comments

We thank the reviewer for acknowledging our work and for providing detailed feedback. Below we provide explanations targeting the reviewer’s concerns and questions.

Q1. Industrial calibration and regional-control sensitivity

In earlier collaboration with our industrial partners, we anonymized facility maps, calibrated From–To demand from industrial process routes, and set vehicle-motion parameters using manufacturer specifications. The underlying operational data and the identities of the equipment manufacturers cannot be disclosed for commercial confidentiality. However, certain settings we adopted in layouts, transport demand, and vehicle motion already appear in relevant literature. We have added Appendix H.1–H.3 to elaborate on this part.

The general idea of admission control comes from industry. The specific control regions and water-level thresholds were engineered for our benchmark topologies. We added Appendix H.4 to explain their idea of the design and to report a sensitivity analysis using 10 alternative configurations in LARGE 300. Without admission control, Dijkstra, EDR, and QLBWR experienced throughput collapse in 10/10, 6/10, and 9/10 runs, respectively; across the 10 alternative admission configurations, the highest collapse counts for the same methods were 4/10, 0/10, and 3/10, respectively (Tables 51 and 55). The reduction in collapse therefore persists across the tested regional configurations with From–To demand held fixed.

Q2. Training details and online adaptation

Training details. The main evaluation uses one trained checkpoint per method–scenario pair, trained under admission control (CAC_A) and evaluated across seeds 0–9 and all three control settings (C0C_0, CAC_A, and CARC_{AR}).

For NQ, DQN, and GDQN, training begins with 5,000 shortest-path teacher examples over 100 initialization epochs, followed by 20,000 simulated seconds of TD training. The initialization/TD-training seed pairs are 2030/2031 for SMALL (100, 150), 2040/2041 for MID (175, 200, 225), and 2050/2051 for LARGE (275, 300). Each scenario has its own checkpoint, saved at the end of training without selection based on evaluation performance. Method-specific hyperparameters are documented in Appendix D.3.

The main matrix uses one training realization per method–scenario pair. Thus, the ten evaluation seeds capture environmental variability, while variability across independently trained checkpoints remains unquantified. A uniform hyperparameter-tuning budget across methods is also not established by the available records.

Online adaptation. We thank the reviewer for raising this question. We added a preliminary DQN evaluation in three scenarios under admission control (CA)(C_A), comparing online parameter updates enabled versus disabled. Each entry below reports throughput (tasks/h) / mean task cycle time (s) averaged over 10 seeds (0-9). We witnessed no throughput collapse in these experiments.

ScenarioOnline updates OFFOnline updates ON
SMALL 1503039.9 / 126.463098.4 / 125.18
MID 2256010.5 / 96.296010.1 / 96.36
LARGE 3007265.2 / 96.927264.7 / 97.07

These preliminary results suggest a marginal, mixed effect from enabling online updating alone. None of these performances surpass EDR, the heuristic routing algorithm with best performance, in terms of average cycle time. We will complete the remaining experiments and report the full comparison across scenarios and online-adaptation settings, including whether the routing ranking persists.

Q3. Experimental runtime and release reproducibility

The reported experiments were conducted with Python 3.12.13. Appendix G describes the earlier Python 3.11 wheel (v0.1.1). The current reviewer-accessible release is v0.1.2 for CPython 3.11 on Linux x86-64. It provides the runner and configurations for the 2,730 main-matrix runs, together with the 49 learned-policy checkpoints.

When preparing the original paper, we compared the experimental runtime and the Python 3.11 compiled distribution using the same seeds and configurations. We checked each and every scenario, routing method and regional policy, finding that the simulated performance metrics agreed. Separately, the public validation record for v0.1.2 verifies the full experiment plan and documents 20-second Dijkstra and GDQN runs on the released wheel. Numerical agreement over all 2,730 full-length runs has not been evaluated as part of those release checks.

Hi Reviewer 3s24,

Thank you for the constructive review and insightful suggestions on validating our benchmark and ensuring fair baseline comparisons. As the discussion phase wraps up soon, we wanted to follow up and see if our rebuttal has resolved your questions. Please feel free to let us know if anything remains unclear—we would be glad to clarify further. Thanks again for your time and effort!

Reviewer_eTixOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces AMHSTrafficLab, a simulation benchmark for routing and regional traffic control in automated material-handling systems. It models vehicles sharing directed tracks, merge permissions, following constraints, and regional admission rules. The main evaluation compares 13 routing methods across seven fleet–workload scenarios and three regional-control settings, using ten environment seeds per combination. Admission control reduces throughput-collapse runs from 265/910 to 9/910. Under admission control, the evaluated learning-based configurations do not consistently outperform the strongest congestion-aware heuristic across all seven scenarios. The appendices provide the complete evaluation matrix and cross-fleet and cross-topology transfer results.

Critical strengths
  1. A useful common environment for a practically relevant problem.

The benchmark brings routing, vehicle following, merge permissions, and regional admission into one evaluation setting. This makes it possible to study interactions that can be missed when routing is evaluated with incompletely specified traffic control. The platform and policy interfaces are a useful contribution for industrial transport research. The manuscript describes these mechanisms and interfaces in Sections 3–4 and Appendices A–B.

  1. A systematic evaluation with clear evidence for the main control effect.

The main matrix covers 13 routing methods, seven scenarios, three control settings, and ten environment seeds. Configurations share the same task stream and initial vehicle states for a given scenario and seed . The reported reduction from 265 to nine collapse runs is substantial, and recomputing the appendix table totals gives the same counts. The paper also distinguishes persistent circular waits from throughput collapse and provides transfer experiments. These are valuable parts of the evaluation.

Critical weaknesses

1、Industrial structure is documented more clearly than industrial dynamics are validated. Demand uses fixed From–To rates, service durations are fixed, and regional thresholds are adjusted using operational experience. These are understandable modeling choices, but their effect on the main failure patterns and rankings has not been quantified. Anonymized aggregate calibration or targeted sensitivity tests would strengthen the claimed industrial grounding. 2、Per-call latency does not establish total cost or scaling. Route ms excludes learning updates, periodic observation outside the call, and dispatching, while methods have different call frequencies. The reported experiments also use a substantial CPU server. Total wall-clock cost, memory, preprocessing, and training cost are needed to judge the resources required to use the benchmark and extend it beyond 300 vehicles. 3、The evaluation does not quantify variation across independently trained models. The manuscript evaluates one frozen checkpoint per method and scenario. Ten environment seeds assess variation in tasks and initial states, but they do not measure variation caused by training. The checkpoint-selection rule, independent training seeds, and total tuning budget are also important for interpreting the rankings. The TD-loss example does not by itself establish that transport performance has converged. 4、QLBWR's signed normalization can reverse the intended action preference. The potential-based update in Eq. (14) adjusts the learned route costs without guaranteeing positive Q values. The subsequent Boltzmann rule divides by their signed mean. Whether the reported runs encounter it depends on the actual implementation and learned values.

Questions for authors

1、Clarification of the learning target. Does the next-state maximization in Eq. (17) use the same admissible-action set as Algorithm D.3? Please identify the implementation for NQ, DQN, and GDQN. If training uses a different action set, how is it related to the evaluation constraint? 2、Clarification of QLBWR. How are negative or zero Q-value means handled in the Boltzmann rule? Please provide the actual normalization, observed value ranges, and any safeguards. If the printed expression is used directly, what changes after correction? 3、Cost and scaling. Could you report cumulative routing, observation, control, dispatching, and training costs, alongside memory use and call counts? How do they change with fleet size, network size, and region count? 4、Sensitivity of the routing comparison. What motivates the different stretch bounds and discount conventions? Could targeted sensitivity results separate their effects from the algorithm changes? What fraction of evaluated decisions has only one admissible action? 5、Training evidence. What are the independent training seeds, interaction budgets, tuning budgets, training control settings, and checkpoint-selection rules? Do the main rankings hold across independently trained checkpoints? 6、Industrial validation and control configuration. Which dynamic quantities have been checked against real logs or an independent reference? How were regions and thresholds selected, and were the main test outcomes used in that process? Which relevant parameter variations preserve the admission-control result?

3 comments

We thank the reviewer for the careful assessment and constructive questions. We have revisited the implementation and experimental records to clarify the evaluation protocol and provide additional evidence on computational cost and parameter sensitivity.

Q1. Learning target and admissible actions

The stretch constraint is applied to action selection during both training and evaluation, including exploratory actions during training. The executed actions therefore follow the same stretch-constrained selection rule in both phases.

Eq. (18) uses all outgoing successors N+(i′)N^+(i') for the next-state TD maximization, while Algorithm D.3 selects executed actions from As(i,t)A_s(i,t). In the reported implementation, the bootstrap can consequently include successors outside the stretch-bounded action set.

For NQ, the current network evaluates the next-state features of all outgoing successors. DQN uses its target network for this maximization, and GDQN follows the DQN update with its graph-aggregated feature representation. All three methods apply the stretch constraint during action selection.

Q2. QLBWR normalization

The reported experiments use the following normalization for QLBWR. For candidate Q-values qjq_j, the implementation computes

T=\max(10^{-9},\tau),\qquad z_j=-\frac{q_j}{Tm}.$$ $$w_j=\exp\left(\operatorname{clip}(z_j-\max_k z_k,-50,0)\right),\qquad p_j=\frac{w_j}{\sum_k w_k}.$$ The positive floor on the denominator preserves the preference for lower-cost actions when the candidate mean is non-positive. Maximum-logit subtraction and exponent clipping provide numerical stability. We also inspected the seven released QLBWR checkpoints. Across these checkpoints, stored Q-values range from -10.5497 to 316.7428. Four checkpoints contain negative entries, and all stored values are finite. These safeguards were already present in the implementation used for the reported experiments. No post-review correction was applied to obtain the reported results. ### Q3. Computational cost and scaling To complement the per-call Route ms metric, we extracted cumulative costs and routing-call counts from the original logs. Each simulation uses one CPU worker, with 32 independent simulations executed concurrently. The following results are per-run medians across ten 20,000-s evaluations on L300 under $C_A$. | Method | Routing calls (k) | Cumulative routing (s) | Traversal feedback (s) | Wall time (min) | | :--- | ---: | ---: | ---: | ---: | | D | 82.5 | 24.3 | 1.9 | 61.8 | | EDR | 499.8 | 157.5 | 5.6 | 64.2 | | DQN | 515.7 | 194.8 | 52.4 | 53.7 | | MAPPO | 510.5 | 1,909.7 | 2.0 | 85.0 | Feedback time includes traversal-time processing and feature preparation. EDR's separately timed observation callback adds a median of 3.09 s per run. Learned-policy checkpoints remain frozen during evaluation, while EDR continues updating its travel-time estimates online. Wall time covers initialization, simulation, traffic control, dispatch, and result exports. Control and dispatch were not timed separately; per-run peak memory and separate preprocessing costs were not recorded. The measurements reflect the original shared-server conditions. For scaling context, median wall times under $C_A$ are: | Scenario | OHTs | Nodes | Tracks | Regions | EDR (min) | DQN (min) | | :--- | ---: | ---: | ---: | ---: | ---: | ---: | | S150 | 150 | 3,334 | 3,419 | 56 | 45.3 | 48.5 | | M225 | 225 | 2,782 | 3,174 | 186 | 62.1 | 52.1 | | L300 | 300 | 3,522 | 4,014 | 224 | 64.2 | 53.7 | The scenarios vary jointly in fleet size, topology, workload, and region configuration, so the wall times reflect their combined execution costs. Training costs are recorded separately. For L300 DQN, teacher initialization took 17.6 s, followed by 49.3 min of TD training over 20,000 simulated seconds, including 31.8 s of minibatch updates. The released checkpoints can be reused across evaluation seeds and control settings without retraining.

Q4. Stretch bounds, discount conventions, and sensitivity

Stretch bounds limit excessive detours in resource-constrained networks. The reported configurations use 1.05 (SMALL/MID) and 1.02 (LARGE) for NQ/DQN, 1.10 for GDQN, 1.05 for MAPPO, and 1.50 for QECV (based on free-flow time). These are fixed baseline configurations throughout experiments.

Discount conventions reflect each method's transition structure: NQ/DQN/GDQN discount per branching-node decision, MAPPO discounts by elapsed time, and tabular Q-routing methods update over track traversals. Thus, the effective time horizon depends on the convention as well as the numerical discount factor.

We admit that a full-scale sensitivity test has not been done, but we conducted a small-scale targeted sensitivity analysis for DQN on MID225 under CAC_A, varying stretch and discount separately, as an indicator.

SettingThroughputCycle time (s)
Reported stretch (1.05)6010.096.52
Stretch: 1.006010.597.39
Stretch: 1.106010.496.44
Stretch: 1.256009.996.97
Stretch: unrestricted6004.5107.95
Discount: 0.906009.696.68
Discount: 0.956010.497.14
Discount: 0.996010.396.60

All configurations have zero throughput-collapse runs. Across bounded stretch values (1.00–1.25), cycle time varies by less than 1%. Removing the constraint increases cycle time by 11.8%, supporting its role in preventing costly detours. The discount sweep (0.90–0.99) also shows limited variation, with cycle times between 96.60 and 97.14 s. Importantly, EDR achieves 92.7 s on MID225 (Table 3 of the paper), outperforming all tested DQN configurations. Thus, the DQN–EDR comparison newly discussed in the paper is robust to the tested parameter changes in this scenario.

Q5. Training evidence

The main evaluation uses one trained checkpoint per method–scenario pair, reused across evaluation seeds 0–9 and all three regional-control settings (C0C_0, CAC_A, and CARC_{AR}). All checkpoints were trained under admission control (CAC_A). The same policy is evaluated across control settings under consistent training conditions.

For NQ, DQN, and GDQN, initialization uses 5,000 shortest-path teacher examples for 100 epochs, followed by 20,000 simulated seconds of TD training. The training configurations are summarized below.

ScenarioInitialization seedTD-training seedTD-training horizonTraining control
S1002030203120,000 sCAC_A
S1502030203120,000 sCAC_A
M1752040204120,000 sCAC_A
M2002040204120,000 sCAC_A
M2252040204120,000 sCAC_A
L2752050205120,000 sCAC_A
L3002050205120,000 sCAC_A

The training horizon measures simulated environment interaction time. Each checkpoint was saved at the end of its training run and used directly for evaluation. No checkpoint selection was performed using the evaluation seeds or regional-control results. Method-specific hyperparameters are documented in Appendix D.3.

The current evaluation includes one training realization per method–scenario pair. The ten evaluation seeds characterize performance variation across task streams and initial vehicle states for a fixed checkpoint. Variability across independently trained checkpoints remains unquantified, so the reported rankings should be interpreted as results for the documented baseline configurations.

The primary benchmark findings concern the interaction between routing and regional traffic control. Evaluating fixed policies across all three control settings provides a consistent basis for examining this interaction.

Q6. Industrial validation and control configuration

In earlier collaboration with our industrial partners, we anonymized facility maps, calibrated From–To demand from industrial process routes, and set vehicle-motion parameters using manufacturer specifications. Commercial confidentiality prevents disclosure of operational data and manufacturer identities. However, many settings we adopted in layouts, transport demand, and vehicle motion already appear in relevant literature. We added Appendix H.1–H.3 for explanation of this part.

While the general idea of admission control comes from industry, the specific control regions and water-level thresholds were engineered for our benchmark topologies. Appendix H.4 explains their design rationale and reports a sensitivity analysis using 10 alternative configurations in LARGE 300. Without admission control, Dijkstra, EDR, and QLBWR experienced throughput collapse in 10/10, 6/10, and 9/10 runs, respectively; across the 10 alternative admission configurations, the highest collapse counts for the same methods were 4/10, 0/10, and 3/10, respectively (Tables 51 and 55). The reduction in collapse therefore persists across the tested regional configurations with From–To demand held fixed.

Hi Reviewer eTix,

We sincerely appreciate your thoughtful feedback, which helped us strengthen our work and improve both our algorithmic clarity and empirical validation. With the discussion period ending soon, we wanted to check if our responses have adequately addressed your concerns. Please let us know if there are any remaining questions or if further clarification would be helpful. Thank you once again for your time and effort in reviewing our submission.

Comments on the paper

1 comment

We thank the reviewers for their careful review and constructive feedback. In response, we have revised the manuscript to clarify its contribution to learning research, provide additional empirical analysis, and improve presentation. Alongside our point-by-point responses, we summarize the main revisions below. Changes are highlighted in blue in revised paper.

  1. Clarification of the relevance to ICLR (Abstract, Introduction, Section 3, and Conclusion). To make the learning questions more explicit, we now formulate the AMHS traffic system as a joint-action MDP in which routing and regional-control decisions interact through shared resources. The revised discussion identifies state abstraction, traffic memory, and scalable value representations as learning questions that can be studied reproducibly in this benchmark.

  2. Additional analysis of traffic memory and value representations (Section 4 and Appendix D). We describe the information retained and used by the 13 routing baselines. In particular, we explain how EDR combines traversal-history estimates with shared edge costs and shortest-path search to construct candidate cost-to-go values across destinations. This analysis connects the empirical routing comparison to specific choices of memory and value representation.

  3. Additional controlled DQN comparison (Section 5). We report a matched 2-by-2 experiment varying the use of an EDR-derived cost-to-go feature and online weight updates in SMALL 150, MID 225, and LARGE 300. Each configuration is evaluated over ten seeds and 20,000 simulated seconds. Adding the EDR feature reduces mean task cycle time in all six matched comparisons, while the effects of online weight updates vary across settings. All 120 runs are free of throughput collapse. These results provide a concrete example of how structured traffic history can support learned routing policies.

  4. Further explanation of industrial grounding and control sensitivity (Appendix H). We have clarified the industrial fidelity of our benchmark in terms of network layouts, transport demand, and vehicle motion parameters. We have also provided further details on the engineering design of regional control. We report sensitivity experiments on region design and water-level thresholds in LARGE 300 using Dijkstra, EDR, and QLBWR.

  5. Revisions to presentation and organization (Sections 3 and 5, and Appendix E). We shortened the platform description and moved the full Dijkstra regional-control table to Appendix E.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.