- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a reinforcement learning framework that trains a multitask VRP constructor using feedback from deterministic DP splitting and bounded local refinement. Its central observation is that distinct constructed candidates can produce identical refined outcomes, motivating diversity credit at the post-processing level. The framework combines refined-cost rewards with a Refined-Edge Prior (REP), which learns adjacency frequencies from refined candidates and guides decoding, and Refined-Outcome Entropy (ROE), which assigns credit using refined-solution and route-set frequencies. Optional inference-time route recombination further exploits candidate complementarity. Evaluation includes 16 VRP variants, matched-post-processing neural comparisons, component ablations, distribution shifts, unseen constraint combinations, larger instances, and CVRPLIB. The full pipeline reports macro-average gaps of 0.059% and 0.087% to PyVRP@60s at mean latencies of 0.110s and 0.560s for 50 and 100 customers, respectively.
Critical strengths
A meaningful contribution to learning from downstream optimization, supported by relevant ablations. Previous multitask methods primarily improve constraint representation, expert sharing, or transfer, while solver-feedback approaches such as RL-RFCS already learn from deterministic splitting costs. This paper adds a distinct mechanism: assigning diversity credit using the frequencies of outcomes that survive refinement. This directly addresses redundancy in the candidates used for final selection and route recombination. Table 4 provides substantial supporting evidence: adding the full ROE objective reduces the M20 gap from 0.230% to 0.117% and the DFS gap from 0.173% to 0.059%, while increasing unique refined endpoints from 72.86 to 220.07. The action-entropy comparison further supports the importance of the chosen diversity measure. Strong empirical results with comparisons that account for classical post-processing. The five main neural baselines receive the same candidate counts, DP splitting, local refinement, and route recombination settings. This makes the results informative about the learned candidate-generation strategy. Improvements are already present in the core pipeline: for example, at n=50, it achieves a 0.117% gap in 0.105s, compared with 0.265% in 0.256s for CCL with M20. The full pipeline reports the lowest neural objectives across all 32 size–variant combinations. The quality–latency curves, competitive CVRP200 results against HGS-CVRP, and matched-post-processing comparison with Scale-Net provide additional evidence of practical value.
Critical weaknesses
The comparison with action entropy does not sufficiently isolate the source of ROE’s advantage. Appendix B.6 selects the action-entropy coefficient from three values using training statistics only through epoch E4. In contrast, ROE uses per-instance RMS normalization and substantially changes its diversity weights between training stages. Consequently, Table 4 compares methods that differ in both the diversity measure and its optimization configuration. Although the results demonstrate the effectiveness of the implemented ROE objective, they do not establish how much of its advantage comes specifically from measuring diversity after refinement rather than from differences in regularization strength and scheduling. Moreover, the component ablations are restricted to n=50, leaving the corresponding mechanism insufficiently examined at n=100.
Questions for authors
Could the authors provide a validation-based sensitivity analysis of the action-entropy coefficient evaluated after the complete training schedule, particularly following the switch to stronger refinement? This would help determine whether ROE’s advantage in Table 4 persists with a baseline calibrated for the later training stage. Corresponding ROE ablations at n=100 would also clarify whether the component-level conclusions extend beyond the smaller problem size. Could the authors separate REP’s auxiliary supervision from its decoding prior and report their individual contributions? A comparison with refined-trajectory imitation using the same constructor and deterministic refiner would further clarify the value of the proposed edge-frequency targets relative to existing refinement-based supervision. Under the default K=32 and gamma=0.25, how frequently does solution-level ROE produce a nonzero advantage during training, and how does this frequency vary across tasks and problem sizes? Appendix A.5 explains that equal endpoint frequencies, including the case where all retained endpoints are distinct, yield zero solution-level advantage. Training-time statistics would help establish when this signal contributes and how its role differs from route-level credit.
2 comments
Dear Reviewer 6XmR,
Thank you again for your feedback. We added AE comparisons, n=100 and REP ablations, and ROE signal statistics, and discussed imitation as future work. Revision locations are summarized in our first response. We would be happy to discuss any further questions.
We thank the reviewer for recognizing the contribution of learning from refined outcomes and for the constructive suggestions on baseline calibration and mechanism analysis. We address these points below with additional experiments and clarifications, referring to our response to Reviewer rgbL and 5Eeb where the concerns overlap.
Response: Action-entropy coefficient sensitivity
The AE ablation tests whether a conventional action-entropy bonus can replace ROE. We retain REP and replace both ROE terms with mean action entropy added to the loss using a fixed coefficient. To extend the original early-stage screening, all three AE configurations were trained from scratch through the complete 100-epoch schedule.
Rebuttal Table 9
Results use n=50 and A8S50, with unique endpoints measured after M20. ROE achieves lower gaps at all three stages than every tested fixed-coefficient AE baseline.
Response: Ablations at n=100 and REP decomposition
Please see our response to Reviewer rgbL’s W1/Q2 for additional CVRP50/CVRP100 ablations, including a refined-cost-only baseline, route-level ROE comparisons, and REP decomposition across two training seeds. Full REP consistently improves M20 quality over both the prior-only and REP-disabled configurations, while its gains after DFS are smaller and inconsistent. These experiments use a common 10-epoch training-from-scratch protocol.
Response: Refined-trajectory imitation
We thank the reviewer for this suggestion. Refined-trajectory imitation is a useful alternative supervision design that could be integrated into our framework. Our current experiments focus on the contributions of REP and ROE; we leave a controlled comparison with imitation-based supervision to future work and do not claim superiority over this alternative.
Response: Frequency of nonzero ROE signals
Complementing the redundancy statistics in our response to Reviewer 5Eeb’s W4, we report nonzero ROE signal frequencies below.
Rebuttal Table 10
Each checkpoint is replayed on 1,024 fixed instances per task with K=32, temperature 1, γ=0.25, and M12. Percentages count instances whose gated, centered credit vector has RMS > 10⁻⁸. Solution-level ROE becomes less frequently active as refined solutions become distinct, especially at n=100. Route-level credit remains nonzero on nearly all instances at E90/E99, providing a complementary signal even when complete solutions are unique.