- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper identifies a setting that the neural-VRP literature has not addressed: lifelong learning of routing solvers when the problem distribution drifts continually over time, so that a very large number of locally-stationary tasks arrive in sequence, each with only insufficient training resources. The authors first demonstrate that this drift occurs in real logistics data (Amazon last-mile / LaDe streams) by tracking the Gromov–Wasserstein distance between instance distributions over time (Figure 2). They then formalise the setting as lifelong RL over a drifting MDP distribution and propose DREE (Dual Replay with Experience Enhancement): PIR (problem instance replay) relearns buffered instances; KR (knowledge replay) learns from buffered solver behaviours via confidence-weighted behaviour matching (following LLR‑BC); and EE (experience enhancement) re-evaluates buffered instances during replay and refreshes their stored behaviours whenever a better solution is found, so that replay targets improve instead of staying frozen. Evaluation covers synthetic CVRP/TSP streams over five task orders, two additional drift scenarios (periodically stationary; drift with sudden changes), three extra base solvers (Omni, INViT, CoEKS), real-world LaDe/Amazon streams, and an unseen-instance generalisation metric.
Critical strengths
- Broad, well-organised evaluation. Five task orders with dispersion, two additional drift regimes, four base solvers, synthetic and real-world streams, plus a generalisation metric. This is more than is typical for the area, and it makes the method's applicability claim credible.
- The core insight of the paper is simple, correct, and useful. More specifically, I find the central idea to be clear, well motivated, and technically sound.
Critical weaknesses
- The novelty is limited: the method is essentially a combination of existing methods. PIR is a standard instance-replay approach, and KR adopts LLR-BC’s behaviour matching wholesale.
- Data retention is a first-class concern for this method. DREE’s memory holds real delivery instances, including customer coordinates, demands, and solution trajectories from LaDe and Amazon Last Mile. EE repeatedly re-solves those stored instances, so the retained records are not inert. These retained records pose serious privacy-leakage risks, and replay creates a potential for “cheating”; I am therefore somewhat pessimistic about methods that rely on replay mechanisms.
Questions for authors
refer to weaknesses.
4 comments
References
[1] Wang, L., Zhang, X., Su, H., & Zhu, J. (2024). A comprehensive survey of continual learning: Theory, method and application. IEEE transactions on pattern analysis and machine intelligence, 46(8), 5362-5383.
[2] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., ... & Tuytelaars, T. (2021). A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7), 3366-3385.
[3] Khalil, E., Dai, H., Zhang, Y., Dilkina, B., & Song, L. (2017). Learning combinatorial optimization algorithms over graphs. Advances in neural information processing systems, 30.
[4] Barrett, T., Clements, W., Foerster, J., & Lvovsky, A. (2020). Exploratory combinatorial optimization with reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 04, pp. 3243-3250).
[5] Luo, F., Lin, X., Liu, F., Zhang, Q., & Wang, Z. (2023). Neural combinatorial optimization with heavy decoder: Toward large scale generalization. Advances in Neural Information Processing Systems, 36, 8845-8864.
[6] Joe, W., & Lau, H. C. (2020). Deep reinforcement learning approach to solve dynamic vehicle routing problem with stochastic customers. In Proceedings of the international conference on automated planning and scheduling (Vol. 30, pp. 394-402).
[7] Li, J., Cao, Z., Wu, Y., & Liu, T. (2025). Enhancing the Cross-Size Generalization for Solving Vehicle Routing Problems via Continual Learning. arXiv preprint arXiv:2510.10262.
[8] Feng, S., Lin, Z., Zhou, J., Zhang, C., Li, J., Chen, K. W., ... & Ong, Y. S. (2026). Lifelong learner: Discovering versatile neural solvers for vehicle routing problems. IEEE Transactions on Intelligent Transportation Systems.
Thank you for the author's response and clarification. I do not deny the importance of replay in continual learning. However, current SOTA continual learning methods are shifting toward approaches such as LoRA fine-tuning that do not rely on replay, thereby directly avoiding potential risks associated with data leakage. I have increased my score, but I still cannot give a positive rating.
We are grateful to the reviewer for increasing the score, and we appreciate the reviewer's feedback, which helps us better understand the reviewer's concern and offers us a chance to share our thoughts on this.
In the studied scenario, replay in DREE does not lead to the potential for data leakage.
Indeed, in the scenario in which data are assumed sensitive and cannot be stored, approaches without replay directly avoid potential data leakage risks from replay. However, we would like to take this opportunity to clarify that in our focus scenario, in which instances can be stored, DREE does not lead to the potential for data leakage.
"leakage is the introduction of information about the data mining target that should not be legitimately available to mine from." [1] All replayed objectives of DREE are directly accessible during learning and legitimately available to be stored in the scenario. Thus, no leakage happens in DREE according to the definition above.
Progress in other applications and paradigms of lifelong learning does not invalidate the motivation, novelty, or contributions of this paper, which addresses a specific VRP scenario.
DREE addresses the issue faced by existing lifelong neural VRP solvers and achieves promising performance on both synthetic and real-world streams. Progress in other paradigms of continual learning may offer valuable insights for future neural VRP solver research. Addressing some specific applications where data cannot be stored is also a potential research direction. However, these research directions are outside the scope of this paper and do not invalidate our motivation, novelty, and contributions in addressing our target problem.
Recent replay-based lifelong/continual learning studies and SOTA methods
Although some other paradigms are developed in the field of lifelong/continual learning, like the LoRA-based methods mentioned by the reviewer, as far as we know, replay-based methods are still one of the major directions. New replay-based methods are still proposed and are SOTA in some applications [2,3,4].
References
[1] Kaufman, S., Rosset, S., Perlich, C., & Stitelman, O. (2012). Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data (TKDD), 6(4), 1-21.
[2] Liu, Z., Li, Y., Gao, H., Li, Y., Kong, L., Sun, L., & Huang, W. (2026). IDER: IDempotent Experience Replay for Reliable Continual Learning. ICLR.
[3] Lai, S., Ma, C., Zhu, F., Zhao, Z., Lin, X., Meng, G., & Zhang, Q. (2025). Gradient-guided epsilon constraint method for online continual learning. NeurIPS.
[4] Wang, X., Li, S. Y., Zhang, J., & Chen, S. (2025). Cut out and replay: A simple yet versatile strategy for multi-label online continual learning. ICML.
We thank the reviewer for the comments. We provided point to point response as below, and had revised the paper accordingly, with changes marked in green and red.
W1.
Novelty and reliance on existing replay methods:
Our novelty lies in adaptively utilizing and integrating complementary sources of experience and incrementally improving the quality of buffered experiences, rather than proposing a specific new replay mechanism. DREE is a framework designed to address the issues faced by existing lifelong neural solvers in the continually drifting scenario to improve them instead of entirely replacing them. Its default implementation uses LLR-BC's consolidation because of LLR-BC's reported SOTA effectiveness in lifelong neural routing. DREE does not rely on LLR-BC, and other methods can also be embedded in DREE. We already implemented DREE-Li, which uses Li's reference-model distillation instead of the behavior consolidation of LLR-BC (Section 5.3 and Appendix F.3). DREE-Li improved AP from 6.12 to 3.57 on CVRP and from 3.64 to 1.40 on TSP compared with Li (Table 3). We suggest that designing totally new replay mechanisms is not compulsory for addressing the issue and achieving substantial performance improvement in the continually drifting scenario. We had clarified this framework-level contribution in Section 4 and KR's support for model-based and behavior-based knowledge preservation in Sections 1, 2.2, and 4.3.
W2.
Legitimacy of experience replay:
We respectfully disagree with the claim that replay creates a potential for “cheating.” Experience replay is an established approach in machine learning and one of the major paradigms of lifelong learning methods [1,2]. Experience replay is also widely used in NCO [3,4] as part of the training algorithm. In DREE, all replayed objectives are directly accessible during the learning of each task and do not require external information like an optimal-solution oracle, test reference solutions, future task distributions, or knowledge of the direction of drift. We believe this should be interpreted as an effective approach to knowledge consolidation and performance improvement instead of "cheating."
Data retention is widely used, as it aligns with real-world settings and needs, and is not an issue:
This work, following the majority of existing works on neural VRP solvers (e.g., [5,6]), focuses on the case where problem instances can be stored for later training. Furthermore, in the context of lifelong neural solvers, the need to store problem instances was not first introduced by our work. Retention of problem instance information was already included in previous lifelong neural solver studies, including LLR-BC, Li [7], and Feng [8]. Our approach alleviates the need for instance retention, as it provides more efficient utilization of knowledge and thus reduces the number of problem instances that need to be stored.
Specifically, in LLR-BC, BC constrains the full trainable solver, including its encoder, which takes the original instance information as input. Those inputs must therefore remain available for knowledge consolidation. This access is directly verifiable in the official implementation of LLR-BC. Li and Feng assume access to the generators of past tasks when constructing replay samples. In the real-data setting considered here, such arbitrary regeneration is unavailable, so, as suggested by Feng et al. [8], historical replay requires access to retained training samples.
DREE improves the efficiency of this historical-data use by improving the learning value extracted from retained instances, and therefore makes progress in reducing problem-instance storage. In the experiments, DREE used a buffer of 256 instance-batch entries, compared with LLR-BC's 1,000 experience-batch entries, which contain information from 1,000 batches of instances. The results supported more effective learning within a compact active buffer, offering a direction for reducing dependence on retained inputs. Actual storage bytes and privacy protection still require separate assessment because the two methods retain different representations. We had clarified training-information access in Appendix A.1 and historical-input access and retention in Appendix A.4, with the storage-matched comparison in Appendix A.3.4 (Table 11).