acceptodds
Under review as a conference paper at ICLR 2027

Keep Rehearsing and Refining: Lifelong Learning Vehicle Routing under Continually Drifting Tasks

Abstract

Existing neural solvers for vehicle routing problems (VRPs) are typically trained either in a one-off manner on a fixed set of pre-defined tasks or in a lifelong manner with tasks arriving sequentially, assuming sufficient training on each task. Both settings overlook a common real-world property: problem patterns may drift continually over time, yielding massive tasks sequentially arising, each with only limited training resources. In this paper, we propose a novel lifelong learning paradigm for neural VRP solvers under continual task drift over time, where each task is locally stationary at one learning time step but receives only insufficient training resources. We empirically demonstrate that such continual drift arises in practice using real-world logistics datasets. We then propose Dual Replay with Experience Enhancement (DREE), a general framework to improve learning efficiency and mitigate catastrophic forgetting under such drift. Extensive experiments based on real-world logistics datasets and commonly used synthetic datasets show that, under such continual drift, DREE effectively learns new tasks, preserves prior knowledge, improves generalization to unseen tasks, and can be applied to various existing neural solvers.

Primary area:
optimization
Keywords:
Vehicle routing problem, Neural combinatorial optimization, Lifelong learning for optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eT2POpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies a setting that the neural-VRP literature has not addressed: lifelong learning of routing solvers when the problem distribution drifts continually over time, so that a very large number of locally-stationary tasks arrive in sequence, each with only insufficient training resources. The authors first demonstrate that this drift occurs in real logistics data (Amazon last-mile / LaDe streams) by tracking the Gromov–Wasserstein distance between instance distributions over time (Figure 2). They then formalise the setting as lifelong RL over a drifting MDP distribution and propose DREE (Dual Replay with Experience Enhancement): PIR (problem instance replay) relearns buffered instances; KR (knowledge replay) learns from buffered solver behaviours via confidence-weighted behaviour matching (following LLR‑BC); and EE (experience enhancement) re-evaluates buffered instances during replay and refreshes their stored behaviours whenever a better solution is found, so that replay targets improve instead of staying frozen. Evaluation covers synthetic CVRP/TSP streams over five task orders, two additional drift scenarios (periodically stationary; drift with sudden changes), three extra base solvers (Omni, INViT, CoEKS), real-world LaDe/Amazon streams, and an unseen-instance generalisation metric.

Critical strengths
  1. Broad, well-organised evaluation. Five task orders with dispersion, two additional drift regimes, four base solvers, synthetic and real-world streams, plus a generalisation metric. This is more than is typical for the area, and it makes the method's applicability claim credible.
  2. The core insight of the paper is simple, correct, and useful. More specifically, I find the central idea to be clear, well motivated, and technically sound.
Critical weaknesses
  1. The novelty is limited: the method is essentially a combination of existing methods. PIR is a standard instance-replay approach, and KR adopts LLR-BC’s behaviour matching wholesale.
  2. Data retention is a first-class concern for this method. DREE’s memory holds real delivery instances, including customer coordinates, demands, and solution trajectories from LaDe and Amazon Last Mile. EE repeatedly re-solves those stored instances, so the retained records are not inert. These retained records pose serious privacy-leakage risks, and replay creates a potential for “cheating”; I am therefore somewhat pessimistic about methods that rely on replay mechanisms.
Questions for authors

refer to weaknesses.

4 comments

We thank the reviewer for the comments. We provided point to point response as below, and had revised the paper accordingly, with changes marked in green and red.

W1.

Novelty and reliance on existing replay methods:

Our novelty lies in adaptively utilizing and integrating complementary sources of experience and incrementally improving the quality of buffered experiences, rather than proposing a specific new replay mechanism. DREE is a framework designed to address the issues faced by existing lifelong neural solvers in the continually drifting scenario to improve them instead of entirely replacing them. Its default implementation uses LLR-BC's consolidation because of LLR-BC's reported SOTA effectiveness in lifelong neural routing. DREE does not rely on LLR-BC, and other methods can also be embedded in DREE. We already implemented DREE-Li, which uses Li's reference-model distillation instead of the behavior consolidation of LLR-BC (Section 5.3 and Appendix F.3). DREE-Li improved AP from 6.12 to 3.57 on CVRP and from 3.64 to 1.40 on TSP compared with Li (Table 3). We suggest that designing totally new replay mechanisms is not compulsory for addressing the issue and achieving substantial performance improvement in the continually drifting scenario. We had clarified this framework-level contribution in Section 4 and KR's support for model-based and behavior-based knowledge preservation in Sections 1, 2.2, and 4.3.

W2.

Legitimacy of experience replay:

We respectfully disagree with the claim that replay creates a potential for “cheating.” Experience replay is an established approach in machine learning and one of the major paradigms of lifelong learning methods [1,2]. Experience replay is also widely used in NCO [3,4] as part of the training algorithm. In DREE, all replayed objectives are directly accessible during the learning of each task and do not require external information like an optimal-solution oracle, test reference solutions, future task distributions, or knowledge of the direction of drift. We believe this should be interpreted as an effective approach to knowledge consolidation and performance improvement instead of "cheating."

Data retention is widely used, as it aligns with real-world settings and needs, and is not an issue:

This work, following the majority of existing works on neural VRP solvers (e.g., [5,6]), focuses on the case where problem instances can be stored for later training. Furthermore, in the context of lifelong neural solvers, the need to store problem instances was not first introduced by our work. Retention of problem instance information was already included in previous lifelong neural solver studies, including LLR-BC, Li [7], and Feng [8]. Our approach alleviates the need for instance retention, as it provides more efficient utilization of knowledge and thus reduces the number of problem instances that need to be stored.

Specifically, in LLR-BC, BC constrains the full trainable solver, including its encoder, which takes the original instance information as input. Those inputs must therefore remain available for knowledge consolidation. This access is directly verifiable in the official implementation of LLR-BC. Li and Feng assume access to the generators of past tasks when constructing replay samples. In the real-data setting considered here, such arbitrary regeneration is unavailable, so, as suggested by Feng et al. [8], historical replay requires access to retained training samples.

DREE improves the efficiency of this historical-data use by improving the learning value extracted from retained instances, and therefore makes progress in reducing problem-instance storage. In the experiments, DREE used a buffer of 256 instance-batch entries, compared with LLR-BC's 1,000 experience-batch entries, which contain information from 1,000 batches of instances. The results supported more effective learning within a compact active buffer, offering a direction for reducing dependence on retained inputs. Actual storage bytes and privacy protection still require separate assessment because the two methods retain different representations. We had clarified training-information access in Appendix A.1 and historical-input access and retention in Appendix A.4, with the storage-matched comparison in Appendix A.3.4 (Table 11).

References

[1] Wang, L., Zhang, X., Su, H., & Zhu, J. (2024). A comprehensive survey of continual learning: Theory, method and application. IEEE transactions on pattern analysis and machine intelligence, 46(8), 5362-5383.

[2] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., ... & Tuytelaars, T. (2021). A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7), 3366-3385.

[3] Khalil, E., Dai, H., Zhang, Y., Dilkina, B., & Song, L. (2017). Learning combinatorial optimization algorithms over graphs. Advances in neural information processing systems, 30.

[4] Barrett, T., Clements, W., Foerster, J., & Lvovsky, A. (2020). Exploratory combinatorial optimization with reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 04, pp. 3243-3250).

[5] Luo, F., Lin, X., Liu, F., Zhang, Q., & Wang, Z. (2023). Neural combinatorial optimization with heavy decoder: Toward large scale generalization. Advances in Neural Information Processing Systems, 36, 8845-8864.

[6] Joe, W., & Lau, H. C. (2020). Deep reinforcement learning approach to solve dynamic vehicle routing problem with stochastic customers. In Proceedings of the international conference on automated planning and scheduling (Vol. 30, pp. 394-402).

[7] Li, J., Cao, Z., Wu, Y., & Liu, T. (2025). Enhancing the Cross-Size Generalization for Solving Vehicle Routing Problems via Continual Learning. arXiv preprint arXiv:2510.10262.

[8] Feng, S., Lin, Z., Zhou, J., Zhang, C., Li, J., Chen, K. W., ... & Ong, Y. S. (2026). Lifelong learner: Discovering versatile neural solvers for vehicle routing problems. IEEE Transactions on Intelligent Transportation Systems.

Thank you for the author's response and clarification. I do not deny the importance of replay in continual learning. However, current SOTA continual learning methods are shifting toward approaches such as LoRA fine-tuning that do not rely on replay, thereby directly avoiding potential risks associated with data leakage. I have increased my score, but I still cannot give a positive rating.

We are grateful to the reviewer for increasing the score, and we appreciate the reviewer's feedback, which helps us better understand the reviewer's concern and offers us a chance to share our thoughts on this.

In the studied scenario, replay in DREE does not lead to the potential for data leakage.

Indeed, in the scenario in which data are assumed sensitive and cannot be stored, approaches without replay directly avoid potential data leakage risks from replay. However, we would like to take this opportunity to clarify that in our focus scenario, in which instances can be stored, DREE does not lead to the potential for data leakage.

"leakage is the introduction of information about the data mining target that should not be legitimately available to mine from." [1] All replayed objectives of DREE are directly accessible during learning and legitimately available to be stored in the scenario. Thus, no leakage happens in DREE according to the definition above.

Progress in other applications and paradigms of lifelong learning does not invalidate the motivation, novelty, or contributions of this paper, which addresses a specific VRP scenario.

DREE addresses the issue faced by existing lifelong neural VRP solvers and achieves promising performance on both synthetic and real-world streams. Progress in other paradigms of continual learning may offer valuable insights for future neural VRP solver research. Addressing some specific applications where data cannot be stored is also a potential research direction. However, these research directions are outside the scope of this paper and do not invalidate our motivation, novelty, and contributions in addressing our target problem.

Recent replay-based lifelong/continual learning studies and SOTA methods

Although some other paradigms are developed in the field of lifelong/continual learning, like the LoRA-based methods mentioned by the reviewer, as far as we know, replay-based methods are still one of the major directions. New replay-based methods are still proposed and are SOTA in some applications [2,3,4].

References

[1] Kaufman, S., Rosset, S., Perlich, C., & Stitelman, O. (2012). Leakage in data mining: Formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data (TKDD), 6(4), 1-21.

[2] Liu, Z., Li, Y., Gao, H., Li, Y., Kong, L., Sun, L., & Huang, W. (2026). IDER: IDempotent Experience Replay for Reliable Continual Learning. ICLR.

[3] Lai, S., Ma, C., Zhu, F., Zhao, Z., Lin, X., Meng, G., & Zhang, Q. (2025). Gradient-guided epsilon constraint method for online continual learning. NeurIPS.

[4] Wang, X., Li, S. Y., Zhang, J., & Chen, S. (2025). Cut out and replay: A simple yet versatile strategy for multi-label online continual learning. ICML.

Reviewer_zGBbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies lifelong learning for neural vehicle routing solvers when task distributions change at every learning time step and training on each task is insufficient. The proposed DREE framework combines problem instance replay, confidence-weighted behavior replay, and experience enhancement: buffered behaviors are replaced whenever re-solving a historical instance yields a better objective. Experiments cover synthetic TSP and CVRP streams, logistics-derived datasets, and several base solvers. The results show improved final solution quality and a favorable adaptation–retention trade-off relative to the evaluated lifelong learning baselines.

Critical strengths

S1 A relevant learning setting The focus on continual distribution drift with limited per-task training is practically motivated. The analysis of logistics records supports the relevance of changing customer distributions, and the resulting benchmark challenges the assumption that each stationary task can be learned sufficiently before the next task arrives. This setting should interest researchers in neural combinatorial optimization and continual reinforcement learning. S2 A coherent framework with useful empirical support Combining instance replay with behavior preservation and updating replay targets using improved solutions is a sensible design. Table 1 reports lower final average gaps than the evaluated lifelong learning methods, while Table 19 supports a contribution from each component. Results on logistics-derived streams and different backbones strengthen the empirical case within the evaluated setting.

Critical weaknesses

W1 Insufficient characterization of progressive improvement and attainable performance The central claim of experience enhancement is that repeated replay can refine initially low-quality historical behaviors (Section 4.4). Figure 4 primarily shows aggregate test performance on principal tasks, with DREE remaining relatively stable over substantial portions of training. These curves demonstrate useful retention but do not directly quantify progressive improvements to buffered experiences or their transfer to held-out instances. Table 19 already supports the contribution of EE to final performance, and a stable test curve does not disprove refinement. The remaining concern is how much improvement accumulates through repeated replay and what performance level it ultimately reaches. Please track a fixed cohort of historical instances over replay events and report newly generated and best-so-far buffered objective values, successful replacement rates, and cumulative gains. Include held-out instances from the corresponding historical distributions to distinguish improvement of cached targets from improvement of the current solver. A matched no-EE trajectory would further clarify the contribution of updating stored behaviors. The interaction between PIR and KR also needs a clearer explanation. PIR optimizes solutions to historical instances, whereas KR penalizes departures from stored action distributions on historical states. These objectives need not be incompatible, since KR is a soft constraint, but their balance may limit departures from suboptimal historical behaviors. The current evidence does not distinguish improvements due to additional optimization on replayed instances, positive transfer from incoming tasks, or repeated stochastic solution generation. Nor does it establish which task relationships permit positive backward transfer. These are alternative explanations to be tested, not established failure modes. A focused analysis could compare frozen-policy repeated solution generation, replay-only learning without incoming-task updates, and the full training procedure, controlling historical solution-construction counts and reporting total compute. Subsequent tasks with different degrees of similarity and selected KR weights would clarify when new-task learning helps historical instances and when behavior preservation inhibits improvement. Report current-policy mean solution quality separately from best-so-far cached values: an occasional better sampled route does not by itself demonstrate improved expected policy performance. Representative controlled cases would be sufficient; an exhaustive study is not required. To put these improvements in context, please compare the final DREE solver with an offline joint-training reference that accesses the full collection of tasks and is trained until a documented validation plateau. Preferably use the same unique incoming training instances available over the DREE run, shuffled across tasks, and report the stopping rule, learning curve, total sample exposures, and compute. MT-Ref is a useful existing reference, but Appendix B notes that finite training and task interference can limit its performance; neither convergence nor identical training instances is established. Thus, closeness to MT-Ref alone does not quantify how close continual refinement brings DREE to adequately trained offline performance. The proposed comparison should report the remaining gap overall and per task, treating the converged offline solver as an empirical reference rather than a guaranteed upper bound. Together with the replay trajectories, this would establish both the progression and the extent of improvement.

Questions for authors

The following questions request evidence or clarification that could materially change my assessment; they are not merely suggestions for future work. Q1 How much improvement accumulates during repeated replay For a fixed cohort of historical instances, how do newly generated solution quality, best-so-far buffered quality, and cumulative replacement gains change across replay events? Do these improvements transfer to held-out instances from the same historical distributions? Please provide corresponding trajectories for DREE and its no-EE variant under matched replay schedules. Q2 Why can PIR improve solutions under KR constraints While PIR seeks improved solutions to historical instances, KR encourages matching their stored behaviors. What enables improvement under this constraint: further optimization on historical data, positive transfer from incoming tasks, or repeated stochastic solution generation? Can a focused comparison of frozen-policy resampling, replay-only learning without incoming-task updates, and full DREE distinguish these sources under matched historical solving budgets? Under what new–old task relationships and KR weights is refinement facilitated or inhibited? Please distinguish improvements in current-policy mean quality from improvements in the best-so-far cached solution. Q3 How close is DREE to converged training on all tasks Can the authors compare DREE’s final performance with an offline joint solver trained on the full task collection until a documented validation plateau, preferably using the same unique incoming instances available over the DREE run? Please report the overall and per-task performance gaps, together with the offline learning curve, stopping criterion, sample exposures, and compute. Can MT-Ref be extended to provide this reference, with its existing budget and convergence status clarified? This comparison would quantify how far repeated refinement improves DREE and how much gap remains to adequately trained offline performance.

3 comments

We thank the reviewer for the comments. We present point to point response as below. We had revised the paper accordingly and marked the corresponding changes in blue.

Q1.

EE provided a sufficient cumulative objective improvement, reducing the objective from 9.2288 to 7.3931, according to the experiment below.

We re-ran DREE and its no-EE variant under matched incoming streams and replay schedules, tracking a fixed cohort collected at epoch 1 on TSP task order 1. Newly generated objectives measure the quality of solutions constructed during PIR. Stored-target objectives measure the quality of the buffered behaviors used by KR. EE updates these targets when replay improves their objectives, while no-EE retains the original targets.

The following table reports average objectives during lifelong learning.

MethodNew objectiveStored objectiveCumulative gainHeld-out gap
DREE, no-EE6.60869.22880.00002.1619
DREE6.60577.39311.83562.1320

We had added the cohort analysis and replay trajectories to Appendix F.4.1 (Table 30 and Figure 19).

Objective changes across replay events:

With EE, average newly generated and stored-target objectives were 6.6057 and 7.3931, compared with 6.6086 and 9.2288 without EE. The average cumulative cached-objective gain from EE was 1.8356. DREE accepted 13,145 replacements over 23,680 comparisons (55.51%), while no-EE left the stored targets unchanged. The results verified that EE did effectively improve the quality of solutions for the stored problem instances.

Improvements transferred to held-out instances:

The average gap on independent instances was 2.1320 with EE versus 2.1619 without EE.

Q2.

Objective can be improved in PIR as KR imposes a weighted penalty rather than a hard constraint. The model is updated toward improving objetive when PIR sufficiently contributes in reducing the loss. We conducted an additional experiment comparing frozen-policy resampling, replay-only learning with EE, full training without EE, and full DREE from the same epoch-200 checkpoint and initial buffer of TSP task order 1. Every control kept the historical pool fixed and ran for 100 epochs, ending at epoch 300, with matched historical solution-construction counts. The current-policy mean objective averages the objectives of newly generated solutions on the fixed historical cohort. The best-so-far objective averages each instance's lowest objective across replay events.

MethodCurrent-policy meanBest-so-far
Starting point6.95466.9438
Frozen-policy resampling6.95436.9211
Replay-only + EE6.93636.9077
Full training, no-EE6.93486.9066
Full DREE6.93426.9003

In a separate experiment, we trained on C-100, GM-100, or U-20 for 10 epochs from the same epoch-200 checkpoint, with KR weights of 0, 10, and 100. C-100 and GM-100 differ in spatial distribution at the same problem size, while U-20 also changes the size. All settings used the same historical cohort.

Incoming taskKR weightCurrent-policy meanBest-so-far
Common checkpoint, epoch 200—6.95466.9438
C-10006.95106.9307
C-100106.95056.9308
C-1001006.94886.9308
GM-10007.01876.9401
GM-100107.00956.9402
GM-1001006.98356.9396
U-2007.05186.9399
U-20107.03806.9399
U-201007.00686.9402

KR does not prevent solution improvement from PIR:

KR imposes a weighted penalty rather than a hard constraint. PIR can reduce the RL loss enough to offset this penalty, and EE refreshes the stored teacher. Repeated stochastic solution generation alone reduced the best-so-far objective from 6.9438 to 6.9211, while the current-policy mean changed only from 6.9546 to 6.9543. Replay-only learning without incoming-task updates achieved a current-policy mean of 6.9363 and a best-so-far objective of 6.9077. Full training without EE reached 6.9348 and 6.9066, respectively, and full DREE further reduced them to 6.9342 and 6.9003. These results supported contributions from historical optimization and incoming-task learning beyond repeated sampling, with an additional benefit from EE.

Refinement varied under different task relationships and KR weights:

At KR=100, C-100 improved the current-policy mean from 6.9546 to 6.9488 and the best-so-far objective from 6.9438 to 6.9308. Increasing KR from 0 to 100 reduced the current-policy mean from 7.0187 to 6.9835 for GM-100 and from 7.0518 to 7.0068 for U-20. Both remained above the checkpoint mean of 6.9546 despite cached improvements, showing that stronger KR reduced interference while task compatibility still affected net policy improvement. We had clarified KR in Section 4.3 and added the controlled comparisons to Appendices F.4.2 (Table 31) and F.5.1 (Table 32).

Q3.

DREE is close to the budget-matched offline reference (original MT-Ref), as shown in the original experiments. Although the performance difference increases when the offline reference receives a substantially larger budget, DREE is still the best evaluated lifelong learning method, considering the constraints of the lifelong learning scenario we studied (limited training and no knowledge of future tasks).

MT-Ref can be extended to provide this converged reference. We extended MT-Ref using DREE's exact incoming-instance pool, including intermediate tasks, shuffled across tasks. For the training counts below, we counted each geometrically augmented input as one instance. The reference trained for 5,000 epochs, completing five passes through the same pool of 4,096,000 incoming training instances for each problem. An exposure counts one presentation of an incoming training instance. AP was averaged over all six principal tasks.

MethodEpochsIncoming-instance pool sizeIncoming-instance exposures
DREE (original)1,0004,096,0004,096,000
MT-Ref (original)1,0004,096,0004,096,000
Extended MT-Ref5,0004,096,00020,480,000

The following table retains the original Table 1 results and adds the extended reference.

Solver / referenceCVRP APTSP AP
MT-Ref (Table 1)3.040.91
DREE (Table 1)3.111.19
Extended MT-Ref2.480.86

Relative to DREE's Table 1 AP, the extended MT-Ref's AP was lower by 0.6294 on CVRP and 0.3256 on TSP. The matched-data per-task results below report the remaining differences explicitly.

Principal taskDREEExtended MT-RefDREE − Extended MT-Ref
TSP E-501.62510.82710.7980
TSP C-1003.38821.74841.6398
TSP Grid-500.34670.00720.3395
TSP U-200.41130.22820.1831
Ring-200.10660.05520.0514
TSP GM-1003.38152.32011.0614
CVRP E-503.41602.46480.9512
CVRP C-1004.58873.43301.1557
CVRP Grid-503.34202.42800.9140
CVRP U-201.98941.74820.2412
CVRP Ring-201.87671.60490.2718
CVRP GM-1003.46063.20480.2558

DREE learned tasks sequentially with 4,096,000 incoming-instance exposures, one fifth of the extended reference's 20,480,000. Over the final 500 epochs (4500–5000), the extended MT-Ref's AP improved by >0.01 on both TSP and CVRP, demonstrating convergence in performance. We had clarified the original MT-Ref budget in Appendix B and added the extended reference and learning curves to Appendix F.1.1 (Tables 23–25 and Figure 18).

AuthorsOpenReview ↗

Kindly ask for feedback from Reviewer zGBb

Dear Reviewer zGBb,

Following your valuable comments, we have carefully revised the manuscript and clarified the points you raised in our responses. With this discussion period approaching its conclusion, we would greatly appreciate it if you could take a moment to re-evaluate the paper in its current form. If you find that your main concerns have been satisfactorily resolved, we would very much appreciate it if you could consider adjusting your score accordingly.

We are thankful for your careful review and for the thoughtful remarks you have provided.

Best regards,

Paper 39606 Authors

Reviewer_ZczVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies lifelong learning for neural vehicle routing solvers in a setting where task distributions change continuously over time, with limited training available for each task. The authors motivate this setting using real-world logistics data and propose DREE, a framework that combines problem instance replay, knowledge replay, and experience enhancement. The main idea is to preserve both past instances and the model's learned behaviors, while progressively updating the stored behaviors whenever better solutions are found during replay. The method also adjusts replay frequency based on how often these improvements occur.

The authors evaluate DREE on TSP and CVRP using synthetic task sequences and real-world data from LaDe and Amazon. The experiments compare against existing lifelong learning methods and examine forgetting, generalization to unseen tasks, and applicability to different neural solvers. The reported results generally show improved solution quality and reduced forgetting compared with the considered baselines.

Critical strengths

(1) Relevant and well-motivated problem setting. The paper addresses an interesting limitation of existing lifelong learning approaches for neural VRP solvers, which typically assume that each task remains stationary long enough for sufficient training. The authors make a reasonable case for studying continual task drift, supported by observations from real-world logistics datasets. I think this is a worthwhile problem to investigate, particularly given the practical need for routing solvers that can adapt to changing distributions without forgetting previously learned knowledge.

(2) Encouraging empirical results across different settings. DREE consistently outperforms the considered lifelong learning baselines on the synthetic TSP and CVRP benchmarks, particularly in terms of final solution quality and forgetting. The experiments with different neural solvers also suggest that the approach is not tied to a specific architecture. Although the gains are not always substantial, the overall results provide reasonable evidence that combining instance replay with behavioral consolidation and experience refinement is beneficial.

Critical weaknesses

(1) Limited novelty. My main concern is that DREE builds quite heavily on existing replay methods, particularly LLR-BC. The experience enhancement mechanism is a sensible addition, but the idea of updating stored behaviors whenever a better solution is found seems fairly straightforward. The gains from this component in the ablation study are also rather modest. Another issue is that DREE stores complete trajectories, unlike LLR-BC, which stores individual state-behavior pairs. Without a comparison under the same memory budget, it is hard to tell how much of the improvement comes from the proposed method itself rather than the additional information being stored. This makes me somewhat less convinced of the overall technical contribution.

(2) Concerns about the experimental setting. I am not entirely convinced that the experiments fully capture the continual drift scenario that motivates the paper. The synthetic tasks are generated by interpolating between a small set of predefined distributions, and some consecutive epochs actually share the same distribution. For the real-world experiments, many training instances are obtained by resampling customers from the same daily instance. While I understand the practical reasons for doing this, it is unclear how well this reflects learning from genuinely evolving routing problems. The evaluation on future days is also fairly limited, and the improvements over the baselines are not consistent across all regions. Since the continual drift setting is one of the main contributions of the paper, I think stronger validation would be needed to make the results more convincing.

Post-rebuttal update — 19 November 2026

I thank the authors for their detailed response and the additional experiments. I appreciate that they have addressed some of my concerns, particularly regarding the synthetic task construction and memory comparison. However, I still have reservations about the real-world validation and the overall novelty of the approach. While the rebuttal is helpful, it does not change my overall assessment, so I will maintain my rating of 2 (Weak Rejection).

Questions for authors

(1) In the real-world experiments, many training instances are generated by resampling customers from the same daily instance. Could the authors clarify how much overlap there is between these generated instances and those used for evaluation on historical tasks? I would also be interested to know whether the results are sensitive to the resampling procedure. Since the real-world setting is an important part of the paper's motivation, this would help me better judge how representative the experiments are.

(2) DREE stores complete trajectories, whereas LLR-BC stores individual state-behavior pairs. Could the authors provide a comparison where both methods have the same memory budget, measured in actual storage rather than buffer size? I think this is important for understanding how much of the reported improvement comes from the proposed mechanisms rather than from retaining more information about previous tasks.

(3) Regarding the synthetic experiments, the task distributions are obtained by interpolating between a small number of predefined tasks. Given that the number of nodes is discrete, some consecutive epochs appear to share the same distribution. Could the authors clarify how frequently this happens and whether they have checked the results under a setting where the distribution genuinely changes at every step? I think this matters because continual task drift is one of the main assumptions motivating the paper.

5 comments

We thank the reviewer for recognizing the practical motivation of continual task drift and the empirical benefits of preserving and refining historical experience. We answer the reviewer's questions and address the concerns below. We had revised the paper accordingly and marked the corresponding changes in red.

Q1.

We thank the reviewer for raising this point, which helps strengthen our conclusion. Overlap may exist, and in most cases the probability is negligible. However, an additional experiment further showed that the conclusions were not sensitive to the resampling procedure. DREE on a task stream without resampling still outperformed other lifelong learning methods. Thus, DREE outperformed all other lifelong learning methods regardless of whether there was overlap. Evidence is provided as follows.

Overlap:

Since training instances were resampled on the fly without being recorded, we analyzed overlap under independent uniform subset sampling. The average probability that one training instance appears among T=100T=100 test instances is pˉ=1−1D∑d=1D(1−1(Ndkd))T\bar p=1-\frac{1}{D}\sum_{d=1}^{D}\left(1-\frac{1}{\binom{N_d}{k_d}}\right)^{T}, where DD is the number of historical evaluation dates and NdN_d and kdk_d are the raw and resampled node counts on date dd, respectively.

DatasetRegion / stationAvg. Prob.
LaDesh-20.073269%
LaDehz-120<10−10<10^{-10}%
LaDejl-310.545951%
LaDeyt-1677.362619%
LaDecq-14431.359254%
AmazonDBO2<10−10<10^{-10}%
AmazonDBO3<10−10<10^{-10}%
AmazonDSE4<10−10<10^{-10}%
AmazonDSE5<10−10<10^{-10}%

In most cases, the probability is negligible (<10−10<10^{-10}%). For small regions, particularly when a large fraction of the original nodes is retained, the number of distinct routing subinstances can be limited, and overlap exists. However, it does not invalidate the promising performance of DREE as shown in the additional experiment below.

Additional experiment without resampling -- DREE still outperformed other lifelong learning methods and DREE's advantage was not sensitive to resampling:

Additionally, we conducted an experiment on the complete daily instances without resampling to investigate sensitivity to resampling in region yt-167 (with the problem scale closest to the resampling target). Each epoch corresponded to one day, with repeated training on that day's raw instance instead of resampled instances. Experimental results were as follows.

MethodAPAFBAMFBABPlFuture-day Gen
Fine-tuning6.34844.36779.10481.980712.7716
LLR-BC2.30041.10392.75901.19659.1469
DREE2.14040.95461.97841.18588.2151

DREE still outperformed other methods, indicating that the experimental conclusions were not sensitive to resampling. We had added the overlap analysis and resampling-sensitivity experiment to Appendices C.5 (Table 15) and E.1 (Table 20), respectively.

Q2.

Additional experiment -- DREE still outperformed LLR-BC under matched storage and the advantage of DREE did not rely on greater memory usage:

We conducted a storage-matched experiment on synthetic TSP task order 1. DREE retained an AP advantage of 0.1569 (11.77% relative improvement) over LLR-BC under the same storage budget. This supported the contribution of DREE's replay and refinement mechanisms beyond a larger storage allocation.

MethodAPAFBAMFBABPl
LLR-BC1.33250.17520.30971.1573
DREE1.17560.12610.25661.0495

We first re-ran DREE and measured the maximum buffer storage, 36.4 GiB. We then set LLR-BC's maximum buffer storage to the same size, allowing it to retain more state–behavior pairs rather than restricting its number of entries. Notably, our implementation was not optimized for memory and retained redundant data structures and tensors. According to our calculations, the DREE buffer can be reduced to 8.93 GiB with lossless reformatting and data packing, preserving full FP32 accuracy. Furthermore, each retained trajectory batch was sampled about once every 3.1 minutes on average, so the history could also be stored on disk at a lower storage cost. The memory cost of DREE is thus affordable and worthwhile, considering its promising performance. We had added the storage-matched comparison and storage discussion to Appendix A.3.4 (Table 11).

Q3.

We are grateful to the reviewer for raising this question, which helps to clarify our statements. Distribution repetition occurs across consecutive epochs, leading to changes in the duration of intermediate tasks. However, adjacent tasks still change progressively, and our assumption and motivation are not invalidated. DREE on a task stream without distribution repetition still outperformed other lifelong learning methods, verifying that this repetition does not invalidate our conclusion from the original experiments. Evidence is provided as follows.

Repeated synthetic distributions:

The reviewer correctly notes that integer-valued node allocation can leave adjacent epochs with the same generating distribution. The generating distribution remains unchanged when the scale and the numbers of nodes drawn from the two principal tasks remain unchanged. The length of an interval of adjacent epochs with the same distribution depends on the scales of the principal tasks that form the intermediate tasks. In our experimental setting, the shortest (20→10020\rightarrow100) and longest (20→2020\rightarrow20) intervals are as follows.

Principal-task scalesAverage interval with the same distribution
20→10020\rightarrow100200/108≈1.85200/108\approx1.85 epochs
20→2020\rightarrow20200/20=10.00200/20=10.00 epochs

Consecutive epochs with the same generating distribution form a single task. Integer rounding therefore extends the duration of some intermediate tasks (distributions) during lifelong learning, particularly at small problem scales. However, adjacent tasks still change progressively, preserving the gradual transition between principal tasks. Thus, repeated distributions across adjacent epochs are consistent with the assumption of continual task drift. We had revised the paper to avoid implying that the generating distribution changes between every pair of consecutive epochs.

Additional experiment -- DREE still outperformed LLR-BC in the case without repeated distributions and the advantage of DREE did not rely on repeated distributions:

We conducted an additional experiment with a stricter drift stream. We shortened the intervals between principal tasks according to their problem scales so that the generating distribution changed at every epoch. We compared DREE and LLR-BC on TSP and CVRP task order 1 under this construction. The principal-task intervals were 50, 50, 20, 20, 20 epochs, forming a stream of 160 epochs in total with 160 unique distributions. The experimental results were as follows.

ProblemMethodAPAFBAMFBABPl
TSPLLR-BC2.24880.31300.37951.9358
TSPDREE1.79590.07040.08351.7255
CVRPLLR-BC5.46051.47171.85783.9887
CVRPDREE4.04820.26250.42123.7857

The results showed that DREE still outperformed LLR-BC on both problems, consistent with the original experiments. These results verified that the original experimental conclusion did not rely on repetition and supported DREE's effectiveness under the stricter requirement that the generating distribution changed at every training epoch. We had clarified the relationship between time steps and epochs in Appendix D.1 (Table 16), and added the strict-drift results to Appendix D.4 (Table 17).

W1.

Novelty at the framework level, and no reliance on existing replay methods:

In addition to the novelty in proposing, formulating, and analyzing the new and practical scenario, our novelty also lies in adaptively utilizing and integrating complementary sources of experience and incrementally improving the quality of buffered experiences, rather than proposing a specific new replay mechanism. DREE is a framework designed to address the issues faced by existing lifelong neural solvers in the continually drifting scenario to improve them instead of entirely replacing them. Its default implementation uses LLR-BC's consolidation because of LLR-BC's reported SOTA effectiveness in lifelong neural routing. DREE does not rely on LLR-BC, and other methods can also be embedded in DREE. We already implemented DREE-Li, which uses Li's reference-model distillation instead of the behavior consolidation of LLR-BC (Section 5.3 and Appendix F.3). DREE-Li improved AP from 6.12 to 3.57 on CVRP and from 3.64 to 1.40 on TSP compared with Li (Table 3). The results indicated that designing totally new replay mechanisms is not compulsory for addressing the issue and achieving substantial performance improvement in the continually drifting scenario.

Specifically, the problem addressed by DREE is not that existing knowledge consolidation mechanisms fail but that the source of knowledge and the consolidation targets are weak under the continually drifting scenario. Earlier lifelong routing methods overlooked this problem as they were primarily developed for long stationary task intervals, with substantial training before knowledge preservation. DREE addresses this in a straightforward but effective way: combining multiple experience sources and replacing the buffered experiences when the solution improves. This simplicity is actually an advantage, as DREE can be simply implemented to improve different existing lifelong neural solvers (Section 5.3, Table 3, and Appendix F.3).

Improvement gain from EE:

In optimization, improving an already strong solver typically becomes more difficult, making further gains especially valuable. The ablation study showed that the experience enhancement mechanism (EE) achieved further gains at negligible additional computational cost. Although its improvement was smaller than those of PIR and KR, both of those modules incur non-negligible additional computational and memory costs. Considering the cost, the improvement from EE is promising. Moreover, beyond the performance gain, with EE we identified, empirically examined, and provided a way to address the low-quality experience issue overlooked by earlier work. It offers insight for future research about the quality of experience. We had clarified the framework-level contribution in Section 4, the alternative KR implementations in Section 4.3, and EE's benefit in Section 5.4, with refinement diagnostics in Appendix F.4.1 (Table 30 and Figure 19).

Storage matching:

We address the issue of storage in the response to Q2. We had added the storage-matched comparison to Appendix A.3.4 (Table 11).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.