- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces a large-scale, road-grounded corpus for EVRP-TW and a benchmark for it.
Critical strengths
The dataset scale of this paper is far larger than that of previous works.
Critical weaknesses
-
One important difference between the proposed dataset and previous ones is the travel representation. But actually, users themselves can get it through the real road network (e.g., from OSRM) and the coordinates, which has been widely used by many ride-hailing simulators.
-
As a dataset and benchmark paper, the data seems all collected from websites instead of new ones, and the compared methods are limited without standard deviation provided.
-
Even though there are some findings through the evaluation, most of them are very intuitive without providing enough new insight.
Questions for authors
-
What is the reason for choosing these methods in your benchmark?
-
What are the core differences between this work and previous EVRP-TW dataset works that should make this paper be considered for publication in ICLR?
5 comments
Response to Reviewer yjwD (W2/Q1): Method Selection and Statistical Reporting
2. W2 (methods and statistics) / Q1 — Why these methods, and how reliable are the comparisons?
The evaluated set covers one exact model, four heuristics, and five neural designs, and standard deviations are now reported. We added HMA and RoutingBlocks in the revision.
| Family | Methods | Implementation source | Role in the comparison |
|---|---|---|---|
| Exact | Gurobi | Our MILP formulation (Appendix A.2) | Solves each instance within the budget |
| Heuristic | VNS/TS, ALNS | Local implementations based on the published descriptions | Established search designs |
| Heuristic (added) | HMA, RoutingBlocks | Authors' C++ solver (HMA); public Python/C++ framework (RoutingBlocks) | Stronger search references |
| Neural | AM-EVRPTW, TERRAN | Adapted from released code | Omit explicit pairwise travel inputs |
| Neural | EVRPTW-RL | Paper-guided implementation | Aggregates travel-time rows per node |
| Neural | DRL-TS, RRNCO | Paper-guided implementation (DRL-TS); adapted from released code (RRNCO) | Retain explicit pairwise travel relations |
The neural policies also differ in attention/recurrent architecture and training procedure. Appendix E.2 identifies each method's source, adaptations, and restrictions.
The complete comparison uses the same 500 T1 instances at N=1000. Exact and heuristic runs receive 1,800 seconds, including initialization for heuristics; neural policies use fixed D1000 checkpoints and 30 trajectories. VNS/TS and ALNS share identical initial routes, RoutingBlocks uses its own ALNS Init construction, and HMA uses native initialization (Appendix E.3).
| Reported statistic | Where |
|---|---|
| Mean cost with SD | Tables 3 and 5; Appendix F.2 for the configurations in Tables 2 and 4 |
| Median, P90, P99, and paired wins | Appendix F.1 |
| Search progress at 60–1,800 seconds | Appendix F.1 |
| Definitions | Appendix E.5 |
SD is the sample SD across the 500 test instances for a fixed configuration. Main-text mean cost is reported only for fully feasible cohorts. For example, HMA's mean/SD is 1,945.47/438.08 versus DRL-TS's 1,997.51/430.31; Gurobi returns accepted incumbents on 2/500 tasks within the budget.
Initialization controls address the vehicle count in the submitted VNS/TS result: its 683.340-vehicle, 288,943.07-cost result becomes 17.220 vehicles and 7,369.50 with ALNS Init. RoutingBlocks improves from 454/500 accepted outputs in its original control to 500/500. Appendix F.1 reports these controls, and E.2 records search restrictions. HMA supplies the strongest mean-cost reference in this complete comparison.
Response to Reviewer yjwD (W3): Experimental Findings and Implications
3. W3 — What do the experiments reveal beyond expected training or scale effects?
The experiments show three things that are not implied by scale alone: the training corpus changes which architecture is cheapest, a strong heuristic still leads the best neural policy, and feasibility holds where cost does not.
| Finding | Evidence | Where |
|---|---|---|
| (a) Road training saves mainly through fewer vehicles | Mean cost falls by 31.4–65.6% at N=1000 with matched 100-location training; the vehicle cost term accounts for 81.8–86.9% of the saving | Section 6.1; Appendix F.2 |
| (a) The training corpus changes the lowest-cost architecture | At N=500 and N=1000, RRNCO has the lowest reported mean cost among the E100 policies, while DRL-TS leads the D100 policies | Section 6.1, Table 2 |
| (a) The training stage matters for checkpoint selection | The checkpoint matching N=100, 500, or 1000 has the lowest mean cost at that size among the three evaluated stages; all evaluated stages are fully feasible, including at N=2000 | Section 6.3; Appendix F.2 |
| (b) Strong search leaves room for neural solution quality | HMA 1,945.47 vs. DRL-TS 1,997.51 on the same 500 T1 tasks; 477/500 paired wins; lower mean cost in all ten cities; Gap 2.7% | Section 6.2; Appendix F.1 |
| (b) Road-information inputs align with cost | The two lowest-cost D1000 policies retain explicit pairwise road information; EVRPTW-RL, which aggregates travel-time rows, costs 19.79% more than TERRAN, which omits explicit pairwise input | Table 3; Appendix E.2 |
| (c) Full feasibility coexists with city-dependent cost sensitivity | Usable capacity 100%→50%: every policy stays fully feasible while cohort mean costs rise by 15.42–59.14%; Los Angeles has the largest increase for every policy (43.37–93.22%) | Appendix F.6 |
On (a), earlier checkpoints therefore remain useful for smaller tasks, and Section 6.5 summarizes the cross-experiment findings. On (b), the worked example in Appendix E.2 establishes that row aggregation loses destination identity; the empirical rankings motivate controlled study of road-relation representations. On (c), the capacity study reruns decoding as a separate evaluation, using 30 trajectories per instance at each capacity level and holding the selected D1000 checkpoint fixed within each method; each percentage uses that policy's own full-capacity baseline from this study. These results show why feasibility should be accompanied by operating cost when evaluating deployment conditions.
Response to Reviewer yjwD (W1/W2/Q2): Dataset Contribution and Construction
Response to Reviewer yjwD
1. W1 / W2 (data) / Q2 — What does the resource contribute beyond public inputs and road matrices?
Thank you for recognizing the scale of the dataset. EVRP-TW-D/B combines a road-grounded task corpus, a reusable delivery-day generator, and a shared evaluation protocol with released reference implementations. This combination supports policy training and comparisons across new days, workload sizes, held-out communities, and an unseen city. Table 1 positions these capabilities relative to existing resources; Sections 4–5 describe their construction and evaluation roles.
We agree that road distances between given coordinates can be computed with tools such as OSRM, and Table 1 lists an existing generator with directed-road travel. The contribution is the routing task built on those roads: which locations are active customers, their joint demand, service-time, and time-window tuples, the charging roster selected for each delivery day, single-customer feasibility certificates, and fixed splits for new days, held-out communities, and an unseen city. The public inputs and construction procedures are documented for reproducibility, and redistribution follows the component-specific licenses (Appendix D.3).
The 71,500 benchmark views span 100–2,000 delivery locations in 11 cities and share 7,500 parent families. Constructing these tasks links residential service candidates and facilities to directed roads, combines reference speeds with road topology, and transfers intact package-volume/service/window tuples under feasibility checks. Appendix C.1 identifies each public source's role; C.3–C.6 specify speed construction, location/facility layers, and instance materialization. C.4 documents the building-source reconciliation and residential-unit assignment used for the benchmark.
The nested views and held-out partitions support training and tests at distinct workload sizes and geographic settings (Appendix C.7). EVRP-TW-B adds a common task, cost objective, budgets, and independent replay so solver outputs can be compared consistently (Section 5; Appendix E). Code, checkpoints, and sample data are provided with the submission; the full corpus will be released at camera-ready. The implementation descriptions distinguish author-code adaptations from paper-guided local implementations.