- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DRLOS-LSMOEA, a deep reinforcement learning-assisted evolutionary algorithm for large scale multiobjective optimization. Its main idea is to expand the operator pool by combining four conventional evolutionary operators with four offspring generation procedures derived from existing optimization algorithms. A DQN selects one operator at each generation using a two-dimensional population state based on convergence and diversity. The reward combines adaptively scaled improvements in these two measures. Additionally, a dimension control mechanism restricts conventional operators to subsets of decision variables during later search stages to promote local exploitation.
Critical strengths
- The paper is generally well written and easy to follow. The framework and its main components are clearly explained.
- The experiments cover two established benchmark suites and decision dimensions ranging from 500 to 8000, with repeated runs and statistical significance tests. The component ablations and parameter analyses provide additional empirical support.
- Treating offspring-generation procedures from existing algorithms as selectable operators provides a practical way to combine complementary search mechanisms within a unified framework.
Critical weaknesses
- The paper explains the motivation for selecting the large-scale dedicated operators, but does not sufficiently establish why this four large-scale dedicated operators combination is needed. Removing operators one at a time, testing smaller or larger subsets, or replacing individual operators would help assess sensitivity to the composition and size of the pool. These experiments would also clarify whether most of the gains come from one or two strong operators.
- The experiments do not include LMOEA-LGCS, RLDS-LSMOEA, or LSDMC, although these methods are discussed in the related work. Please compare against these closely related methods or explain why such comparisons are not feasible. Since the proposed pool incorporates the offspring-generation procedure of LMOEA-LGCS, a comparison with the full algorithm would be particularly useful for assessing the added benefit of the proposed scheduling framework.
- The ablation study removes the large-scale dedicated operators LS and DCM separately, but does not test a variant with both removed. Adding this configuration would help distinguish the individual contributions of the two components and examine their interaction.
- The reward function assigns weights of 0.7 and 0.3 to convergence and diversity improvements, respectively, but the basis for this choice is unclear. Please explain how these weights were chosen and provide a sensitivity analysis to determine whether the reported gains depend on this particular setting.
Questions for authors
- The paper explains the motivation for selecting the large-scale dedicated operators, but does not sufficiently establish why this four large-scale dedicated operators combination is needed. Removing operators one at a time, testing smaller or larger subsets, or replacing individual operators would help assess sensitivity to the composition and size of the pool. These experiments would also clarify whether most of the gains come from one or two strong operators.
- The experiments do not include LMOEA-LGCS, RLDS-LSMOEA, or LSDMC, although these methods are discussed in the related work. Please compare against these closely related methods or explain why such comparisons are not feasible. Since the proposed pool incorporates the offspring-generation procedure of LMOEA-LGCS, a comparison with the full algorithm would be particularly useful for assessing the added benefit of the proposed scheduling framework.
- The ablation study removes the large-scale dedicated operators LS and DCM separately, but does not test a variant with both removed. Adding this configuration would help distinguish the individual contributions of the two components and examine their interaction.
- The reward function assigns weights of 0.7 and 0.3 to convergence and diversity improvements, respectively, but the basis for this choice is unclear. Please explain how these weights were chosen and provide a sensitivity analysis to determine whether the reported gains depend on this particular setting.
4 comments
Response(2/4)
2. LMOEA-LGCS, RLDS-LSMOEA, and LSDMC.
We added comparisons with the LMOEA-LGCS on the LSMOP and UF benchmarks (LSMOP1–9 and UF1–10, M ∈ {2,3}, 20 runs each). The comparison results are reported in Appendix C.7 (Tables S.35–S.41), including a summary in Table S.41 (Please refer to Response(2/3) to VZ6S). The two methods have complementary strengths, i.e., LMOEA-LGCS wins on a subset of the LSMOP problems, with the largest gaps on LSMOP1, LSMOP8 and LSMOP9, while the proposed method is markedly better on the UF suite. Specifically, in terms of IGD, DRLOS-LSMOEA is significantly better on 68, significantly worse on 51, and comparable on 21 instances, respectively, than LMOEA-LGCS. From the HV perspective, these counts are 64, 43, and 33, respectively. In summary, the proposed DRLOS-LSMOEA performs slightly better than the LMOEA-LGCS on the 140 tested instances. However, the LMOEA-LGCS shows advantages in the LSMOP benchmarks. Notably, the results in Appendix C.8-3) show that none of the versions with single large-scale operator can beat the complete DRLOS-LSMOEA, indicating that the two-direction sampling strategy in LMOEA-LGCS may not act as the key contributor for its good behaviors. However, we only adopt its first stage, i.e., the two-direction sampling, which may explain why our algorithm underperforms it on the LSMOP benchmark suite. Please refer to Appendix C.7 for more details.
For RLDS-LSMOEA and LSDMC, as far as we know, there are no public implementations available, and it could be time-consuming to reproduce these codes. Thus, we have to admit that a faithful comparison is not possible at this time. But we have conducted comparisons against their underlying search components (LMOEA-DS and competitive learning), which are both stand-alone baselines and members of our pool.
Response(3/4)
3. Ablation with both large-scale operators and the DCM removed.
We have added experiments on the variant with the DQN retained and both the four large-scale operators and the DCM removed in Appendix C.8 (Tables S.46–S.49). Results show that on 90 LSMOP instances, it is significantly worse than the complete DRLOS-LSMOEA on all 90 instances by IGD and on 79 of 90 by HV (11 ties, no wins). The mean IGD grows from about two orders of magnitude above the complete DRLOS-LSMOEA at D = 500 to more than three orders at D = 8000. Combined with the existing ablations (w/o LS: significantly worse on 90/90 instances; w/o DCM: 51/90 by IGD), the individual contributions are clearly differentiated. Large-scale operators deliver the primary performance gain, DCM stabilizes the search, while removing both is significantly worse than the complete method on all 90 instances, confirming that the DQN cannot compensate for the loss of either component. Please refer to the Appendix C.8 (Tables S.46–S.49) for more details.
Table S.46: IGD results of the DQN-only variant and DRLOS-LSMOEA on 2-objective LSMOPs
| Problem | D | DQN-only | DRLOS-LSMOEA |
|---|---|---|---|
| LSMOP1 | 500 | 8.7575e-1 (1.57e-1) - | 1.9131e-1 (3.91e-2) |
| LSMOP1 | 1000 | 1.4201e+0 (1.61e-1) - | 1.5907e-1 (5.22e-2) |
| LSMOP1 | 2000 | 1.8739e+0 (1.49e-1) - | 1.6144e-1 (6.00e-2) |
| LSMOP1 | 5000 | 2.4002e+0 (1.55e-1) - | 1.6605e-1 (5.49e-2) |
| LSMOP1 | 8000 | 2.7899e+0 (2.77e-1) - | 1.5976e-1 (3.61e-2) |
| LSMOP2 | 500 | 4.9364e-2 (9.96e-4) - | 4.3299e-3 (7.44e-4) |
| LSMOP2 | 1000 | 3.0255e-2 (6.82e-4) - | 2.8453e-3 (3.62e-4) |
| LSMOP2 | 2000 | 2.0691e-2 (3.31e-4) - | 2.0896e-3 (1.34e-4) |
| LSMOP2 | 5000 | 9.7271e-3 (1.04e-4) - | 1.6625e-3 (9.73e-5) |
| LSMOP2 | 8000 | 6.7350e-3 (7.32e-5) - | 1.5761e-3 (6.36e-5) |
| LSMOP3 | 500 | 1.9003e+1 (1.33e+0) - | 1.5627e+0 (4.20e-3) |
| LSMOP3 | 1000 | 2.4359e+1 (1.03e+0) - | 1.5666e+0 (6.55e-3) |
| LSMOP3 | 2000 | 2.7709e+1 (8.20e-1) - | 1.5688e+0 (8.12e-3) |
| LSMOP3 | 5000 | 2.9919e+1 (4.99e-1) - | 1.5733e+0 (4.67e-3) |
| LSMOP3 | 8000 | 3.0334e+1 (4.62e-1) - | 1.5732e+0 (6.58e-3) |
| LSMOP4 | 500 | 6.7254e-2 (4.05e-3) - | 2.1180e-2 (4.18e-3) |
| LSMOP4 | 1000 | 5.0751e-2 (9.42e-4) - | 1.3549e-2 (1.60e-3) |
| LSMOP4 | 2000 | 3.2322e-2 (2.77e-4) - | 7.3519e-3 (6.89e-4) |
| LSMOP4 | 5000 | 1.6851e-2 (2.45e-4) - | 3.9211e-3 (2.89e-4) |
| LSMOP4 | 8000 | 1.2418e-2 (3.02e-4) - | 2.8881e-3 (2.54e-4) |
| LSMOP5 | 500 | 1.7815e+0 (1.92e-1) - | 6.3878e-1 (1.49e-1) |
| LSMOP5 | 1000 | 3.4723e+0 (5.34e-1) - | 6.9313e-1 (1.29e-1) |
| LSMOP5 | 2000 | 4.6138e+0 (6.01e-1) - | 6.8549e-1 (1.48e-1) |
| LSMOP5 | 5000 | 5.9489e+0 (6.87e-1) - | 7.2992e-1 (3.59e-2) |
| LSMOP5 | 8000 | 6.5747e+0 (1.30e+0) - | 6.8775e-1 (1.47e-1) |
| LSMOP6 | 500 | 7.6168e-1 (5.49e-2) - | 2.5170e-1 (7.36e-2) |
| LSMOP6 | 1000 | 7.4983e-1 (6.02e-2) - | 2.8013e-1 (4.36e-2) |
| LSMOP6 | 2000 | 7.5328e-1 (7.35e-4) - | 2.5967e-1 (5.46e-2) |
| LSMOP6 | 5000 | 7.4671e-1 (2.24e-4) - | 2.2744e-1 (6.50e-2) |
| LSMOP6 | 8000 | 7.4496e-1 (2.35e-4) - | 2.4910e-1 (5.97e-2) |
| LSMOP7 | 500 | 6.5223e+2 (1.53e+2) - | 1.5026e+0 (8.73e-4) |
| LSMOP7 | 1000 | 2.3593e+3 (7.65e+2) - | 1.5107e+0 (4.03e-4) |
| LSMOP7 | 2000 | 4.0273e+3 (8.16e+2) - | 1.5141e+0 (4.71e-4) |
| LSMOP7 | 5000 | 6.8654e+3 (2.49e+3) - | 1.4879e+0 (1.26e-1) |
| LSMOP7 | 8000 | 8.5633e+3 (4.12e+3) - | 1.5165e+0 (4.00e-4) |
| LSMOP8 | 500 | 1.5881e+0 (2.48e-1) - | 4.9898e-1 (2.10e-1) |
| LSMOP8 | 1000 | 2.6583e+0 (2.32e-1) - | 5.1317e-1 (2.05e-1) |
| LSMOP8 | 2000 | 3.6650e+0 (3.31e-1) - | 5.3692e-1 (2.11e-1) |
| LSMOP8 | 5000 | 4.4150e+0 (6.97e-1) - | 4.0872e-1 (2.13e-1) |
| LSMOP8 | 8000 | 4.9078e+0 (6.41e-1) - | 5.0793e-1 (1.90e-1) |
| LSMOP9 | 500 | 1.2992e+0 (8.31e-2) - | 4.5978e-1 (1.92e-1) |
| LSMOP9 | 1000 | 3.9692e+0 (1.12e+0) - | 4.7902e-1 (1.45e-1) |
| LSMOP9 | 2000 | 7.9143e+0 (9.05e-1) - | 4.7058e-1 (1.54e-1) |
| LSMOP9 | 5000 | 8.2044e+0 (9.97e-1) - | 4.2479e-1 (1.71e-1) |
| LSMOP9 | 8000 | 8.5450e+0 (1.26e+0) - | 4.0674e-1 (1.58e-1) |
| +/-/= | 0/45/0 |
Response(4/4)
4. Reward weights.
We have added ablation experiments on more reward weights settings 0.9/0.1, 0.3/0.7, and 0.1/0.9 on the same 10 LSMOP instances used in Appendix C.4 (i.e., LSMOP1, 4, 6, 8, 9; M ∈ {2,3}; D = 500; 20 runs per instance) in Appendix C.9 (Tables S.54–S.57). Results show that none of these three alternatives is ever significantly better than the default setting, i.e., 0.7/0.3. The 0.3/0.7, 0.9/0.1, and 0.1/0.9 settings are significantly worse on 1, 1 and 2 of the 10 instances in terms of IGD, and on 1, 2 and 3 in terms of HV, and statistically indistinguishable from the default on the rest. The weights 0.7/0.3 is therefore not a knife-edge choice, and the larger weights on the diversity reward hurt performance on a few instances with no compensating gain elsewhere. Overall, the current weights, i.e., 0.7/0.3, for the convergence and diversity in the reward constitutes a reasonably sound choice. Please refer to Appendix C.9 (Tables S.54–S.57) for more details.
Table S.54: IGD results of DRLOS-LSMOEA with different reward weights on 2-objective LSMOPs
| Problem | M | D | (default) | |||
|---|---|---|---|---|---|---|
| LSMOP1 | 2 | 500 | 1.6921e-1 (5.30e-2) = | 1.8879e-1 (5.27e-2) = | 1.8872e-1 (4.98e-2) = | 1.9131e-1 (3.91e-2) |
| LSMOP4 | 2 | 500 | 2.3913e-2 (3.66e-3) - | 2.3587e-2 (2.95e-3) = | 2.4947e-2 (3.14e-3) - | 2.1180e-2 (4.18e-3) |
| LSMOP6 | 2 | 500 | 2.6823e-1 (4.83e-2) = | 2.8756e-1 (3.39e-2) = | 2.8429e-1 (3.56e-2) = | 2.5170e-1 (7.36e-2) |
| LSMOP8 | 2 | 500 | 4.7685e-1 (1.76e-1) = | 5.1755e-1 (1.84e-1) = | 4.5215e-1 (1.94e-1) = | 4.9898e-1 (2.10e-1) |
| LSMOP9 | 2 | 500 | 4.9761e-1 (1.91e-1) = | 5.6545e-1 (9.93e-2) - | 5.9407e-1 (9.83e-2) - | 4.5978e-1 (1.92e-1) |
| +/-/= | 0/1/4 | 0/1/4 | 0/2/3 |
Table S.55: IGD results of DRLOS-LSMOEA with different reward weights on 3-objective LSMOPs
| Problem | M | D | (default) | |||
|---|---|---|---|---|---|---|
| LSMOP1 | 3 | 500 | 1.8308e-1 (6.07e-2) = | 1.8992e-1 (6.91e-2) = | 1.8294e-1 (7.40e-2) = | 1.7559e-1 (6.78e-2) |
| LSMOP4 | 3 | 500 | 7.4644e-2 (4.87e-3) = | 7.7659e-2 (7.91e-3) = | 7.7918e-2 (4.17e-3) = | 7.6111e-2 (5.82e-3) |
| LSMOP6 | 3 | 500 | 6.4044e-1 (5.80e-2) = | 6.2729e-1 (2.79e-2) = | 6.3808e-1 (4.50e-2) = | 6.2284e-1 (4.45e-2) |
| LSMOP8 | 3 | 500 | 7.7035e-2 (4.23e-2) = | 8.6457e-2 (6.24e-2) = | 8.7991e-2 (4.44e-2) = | 8.5860e-2 (4.97e-2) |
| LSMOP9 | 3 | 500 | 5.8754e-1 (1.12e-3) = | 5.8732e-1 (9.97e-4) = | 5.8756e-1 (1.02e-3) = | 5.8732e-1 (1.58e-3) |
| +/-/= | 0/0/5 | 0/0/5 | 0/0/5 |
We hope these experiments and clarifications address your concerns, and would be grateful if you would reconsider your assessment.
Response(1/4)
e thank you for the detailed review. We have added ablation experiments on variants with single large-scale operator, variant with the DQN-only, variants with different reward weights, and comparisons with the LMOEA-LGCS.
1. Why these four large-scale operators?
We ran experiments with four variants that keep the DQN, the DCM, and the four conventional operators, and retain exactly one large-scale operator (competitive learning, LMOEA-DS, MOPSO, or two-direction sampling). On the 18 LSMOP instances at D = 500 (M ∈ {2,3}, 20 runs each), every variant is significantly worse than the complete DRLOS-LSMOEA on the great majority of the instances (Wilcoxon rank-sum, p < 0.05), and only two isolated significant wins occur across all four variants on both metrics (competitive learning on one instance by IGD and LMOEA-DS on one instance by HV). No single large-scale operator recovers the performance of the heterogeneous pool. This indicates that the gains come from the synergy of all these operators instead of one super operator. Please refer to Appendix C.8 (Tables S.50–S.53) for more details.
Table S.52: IGD results of variants with single large-scale operator and DRLOS-LSMOEA on 3-objective LSMOPs (D = 500) (For 2-objective LSMOPs, please refer to Response(2/3) to VZ6S)
This is consistent with Appendix C.3 (Table S.9 and Fig. S.2), which shows that each operator is selected at a non-trivial rate. In the late search stage, the selection probability of competitive learning, two-direction sampling, and LMOEA-DS are all significantly above the random baseline, and the conventional SBX and PM fall significantly below it. No operator approaches a dominating share. The reward analysis (Table S.10) also shows distinct utility. Four large-scale operators gain positive average rewards, while most conventional operators gain negative ones, and statistically significant differences are observed within the large-scale operator group. These show that the heterogeneous pool can provide complementary search behaviours, and the DQN can dynamically exploit them at different stages. Moreover, the DCM then confines conventional operators to variable subgroups late in the search, which prevents them from undoing the progress of the directional large-scale operators.