acceptodds
Under review as a conference paper at ICLR 2027

Dynamic Algorithm Configuration with Evolution-Aware State Representation for Multi-Objective Combinatorial Optimization

Abstract

Recently, deep reinforcement learning-based dynamic algorithm configuration (DRL-DAC) has been widely studied in evolutionary computation. However, existing DRL-DAC methods mainly focus on the structure of the current population while neglecting inter-generational evolutionary dynamics, leading to an insufficient characterization of the evolutionary state. To address this limitation, we propose an evolution-aware state representation that incorporates the Kullback–Leibler (KL) divergence between consecutive populations to capture inter-generational transitions. Combined with search-stage information, this metric guides the feature-wise reweighting of node embeddings, providing the RL agent with a more discriminative state representation for accurate parameter adaptation. Extensive experiments on representative multi-objective combinatorial optimization problems demonstrate that our method significantly improves overall optimization performance, achieving superior convergence and diversity compared to state-of-the-art baselines. Furthermore, evaluations under large-scale and many-objective settings highlight its strong generalization capabilities. The source code is available at https://github.com/AetherrehteA/DAC.

Primary area:
optimization
Keywords:
Dynamic Algorithm Configuration, Multi-Objective Combinatorial Optimization, deep reinforcement learning, Kullback–Leibler divergence

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_y3jkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes an evolution-aware state representation for deep-RL-based dynamic algorithm configuration (DAC) of multi-objective evolutionary algorithms. Existing DAC state encoders (e.g., GS-MODAC) represent only the current population, so populations at the same search stage with different rates of change look alike. The authors address this by (i) fitting multivariate Gaussians to consecutive populations' objective vectors and computing the closed-form KL divergence D_KL(P_{t-1}||P_t) as a scalar summary of inter-generational change, (ii) concatenating this normalized scalar with normalized search progress t/T into a 2-D "evolution-aware" feature e_t, (iii) mapping e_t through an MLP (tanh) to a feature-wise weight vector w_t that element-wise reweights the GCN node embeddings of a Pareto-dominance population graph, and (iv) concatenating the pooled representation with e_t as the policy state. A PPO agent outputs continuous crossover/mutation probabilities for NSGA-II, trained with a normalized HV-improvement reward. Validation covers Bi/Tri/Penta-FJSP and Bi-CVRP (12 settings), cross-size generalization, zero-shot cross-problem transfer (DAFJS-SDST variants, CVRPLIB Set X), an ablation, a reweighting visualization, and a runtime analysis.

Critical strengths
  1. Well-motivated, lightweight mechanism. Capturing recent evolutionary dynamics with a 2-D feature rather than a full temporal graph is appealing and empirically adds almost no test-time overhead (Table 7: within ±6%). This is consequential because it keeps DAC practical on larger instances.
  2. Broad and systematic evaluation. Two problem families, varying objective counts, cross-size/cross-problem transfer, real CVRPLIB Set X instances, ablation, and a visualization of the reweighting process together make a fairly thorough empirical study.
Critical weaknesses
  1. Marginal and possibly non-significant empirical gains, overstated as "significant." Improvements over GS-MODAC are tiny: Bi-FJSP-25j5m 9.51e4 vs 9.54e4, Tri-FJSP-25j5m 2.15e7 vs 2.16e7, Penta-FJSP-25j5m 5.62e12 vs 5.63e12, Bi-CVRP-100 1.35e2 vs 1.36e2. The only clearly larger margin is Bi-CVRP-500 (2.35e2 vs 2.42e2, ~3%). Yet the paper repeatedly claims "significant" improvements. Crucially, statistical testing is performed only against GS-MODAC (the underlined entries), not against irace or NSGA-II, and OURS's standard deviation is frequently higher. This directly undermines the central claim of superiority.
  2. The KL component is shown to contribute little, and the closest SOTA baseline is missing. In the ablation (Table 5), "OURS (w/o KL guidance)" nearly matches the full model (e.g., Bi-CVRP-200 1.61e2 vs 1.62e2; Bi-CVRP-500 2.41e2 vs 2.42e2), whereas "w/o feature reweighting" drops below GS-MODAC. This suggests the gain comes mainly from the generic reweighting mechanism, not the proposed evolution-aware/KL signal, and no significance test is given for the KL contribution. Moreover, STN-MODAC—the most closely related recent method explicitly modeling multi-generation dynamics—is discussed but never compared against, which is a serious omission for a novelty claim built on contrast with it.
Questions for authors
  1. Are the improvements over GS-MODAC/irace/NSGA-II statistically significant? Please report the Wilcoxon test results for all baselines, not only GS-MODAC, and clarify whether the sub-1% differences are significant.
  2. Given that "w/o KL guidance" is nearly identical to the full model, what is the demonstrable marginal contribution of the KL signal in isolation (with significance)? Does the reweighting mechanism alone explain most of the gain?
  3. Why is STN-MODAC not included as a baseline? How does the proposed method compare to it under identical budgets?
  4. What justifies modeling population objective vectors as Gaussians, and how sensitive are results to the covariance regularization ε=1e-6 and to the directional choice of KL?
  5. Why do MADAC and irace sometimes underperform fixed-parameter NSGA-II, and does this indicate tuning/training issues that affect fairness?
6 comments
AuthorsOpenReview ↗

Statistical Significance and Interpretation of Performance Improvements [W1 / Q1]

We thank the reviewer for emphasizing rigorous statistical testing and careful interpretation of modest performance improvements.

W1: Several improvements over GS-MODAC are numerically small and potentially non-significant. Statistical comparisons with other baselines are incomplete, and OURS sometimes exhibits higher standard deviations.

Q1: Are the improvements over GS-MODAC, irace, and NSGA-II statistically significant, particularly when the numerical differences are below 1%?

We acknowledge that several improvements over GS-MODAC are numerically modest, and we agree that the magnitude of an improvement should be distinguished from its statistical significance. Our original presentation could have more clearly differentiated these two aspects.

To provide stronger statistical evidence, we extended the two-sided paired Wilcoxon signed-rank tests to include NSGA-II and STN-MODAC, in addition to GS-MODAC. The results are reported in Table 7 of Appendix D.3.

Table 7: Wilcoxon Signed-Rank Test Results for Mean HV

Problem Settingvs. GS-MODACvs. NSGA-IIvs. STN-MODAC
Bi-FJSP-5j5m1.659e-18.787e-134.915e-15
Bi-FJSP-10j5m9.506e-19.549e-102.490e-12
Bi-FJSP-25j5m2.367e-22.747e-121.400e-10
Tri-FJSP-5j5m5.985e-61.231e-132.797e-12
Tri-FJSP-10j5m7.779e-13.471e-166.604e-11
Tri-FJSP-25j5m3.911e-23.897e-183.897e-18
Penta-FJSP-5j5m1.400e-103.201e-108.580e-12
Penta-FJSP-10j5m4.053e-13.750e-171.809e-4
Penta-FJSP-25j5m3.897e-183.897e-183.737e-15
Bi-CVRP-1003.832e-39.487e-81.975e-9
Bi-CVRP-2002.865e-105.817e-171.442e-13
Bi-CVRP-5004.159e-123.897e-181.143e-11

Specifically, OURS achieves statistically significant improvements over GS-MODAC in 8 of the 12 settings, over NSGA-II in all 12 settings, and over STN-MODAC in all 9 evaluated FJSP settings (p < 0.05).

Notably, the four settings highlighted by the reviewer—Bi-FJSP-25j5m, Tri-FJSP-25j5m, Penta-FJSP-25j5m, and Bi-CVRP-100—yield p-values of 0.023670.02367, 0.039110.03911, 3.897×10−183.897{\times}10^{-18}, and 0.0038320.003832 against GS-MODAC, respectively. Although the numerical improvements are small in these cases, they are statistically significant under the reported tests. We nevertheless acknowledge that statistical significance does not necessarily imply a substantial practical improvement.

Regarding irace and MADAC, independently reproducing their results under precisely matched experimental conditions requires detailed implementation and hyperparameter settings, some of which could not be fully verified from the available descriptions. To maintain consistency with the established benchmark comparisons and avoid introducing discrepancies from potentially different configurations, we adopted the results reported in the GS-MODAC study. However, since the corresponding instance-level results are unavailable, statistical significance test cannot be reliably conducted. We acknowledge this limitation and do not claim statistically significant superiority over these two baselines. We also acknowledge that OURS does not consistently achieve lower standard deviations than GS‑MODAC. Therefore, our results should be interpreted as improvements in solution quality across several evaluated settings, rather than universal improvements in both performance and stochastic stability.

AuthorsOpenReview ↗

Marginal Contribution of KL Guidance and Additional Ablation Evidence [W2 (part) / Q2]

We thank the reviewer for encouraging a clearer assessment of the respective contributions of KL guidance and feature-wise reweighting.

W2 (part): Removing KL guidance produces only small performance reductions, whereas removing feature-wise reweighting causes larger degradation, suggesting that the improvement may come primarily from feature reweighting.

Q2: What is the demonstrable marginal contribution of KL guidance, and does feature-wise reweighting alone explain most of the improvement?

We acknowledge that feature-wise reweighting contributes substantially to the performance improvements, particularly on Bi-CVRP, where removing KL guidance results in relatively small reductions in HV. However, this observation does not necessarily imply that the KL signal is redundant.

In our framework, feature-wise reweighting adaptively adjusts the contributions of latent node-embedding dimensions, while KL divergence supplies complementary information about inter-generational population changes. Their integration allows recent evolutionary dynamics to influence the construction of the population representation.

To further investigate their contributions beyond Bi-CVRP, we extended the ablation experiments to Penta-FJSP, covering three problem scales (5j5m, 10j5m, and 25j5m), as reported in Table 6 of Appendix D.3. Table 6: Ablation Study on Penta-FJSP

Method5j5m Mean5j5m Max5j5m Std10j5m Mean10j5m Max10j5m Std25j5m Mean25j5m Max25j5m Std
GS-MODAC6.17×10106.17\times10^{10}6.59×10106.59\times10^{10}3.13×1093.13\times10^{9}4.14×10114.14\times10^{11}4.48×10114.48\times10^{11}2.38×10102.38\times10^{10}5.62×10125.62\times10^{12}6.07×10126.07\times10^{12}3.20×10113.20\times10^{11}
OURS (w/o feature-wise reweight)6.16×10106.16\times10^{10}6.55×10106.55\times10^{10}2.96×1092.96\times10^{9}4.14×10114.14\times10^{11}4.49×10114.49\times10^{11}2.52×10102.52\times10^{10}5.58×10125.58\times10^{12}6.06×10126.06\times10^{12}3.35×10113.35\times10^{11}
OURS (w/o KL guidance)6.15×10106.15\times10^{10}6.59×10106.59\times10^{10}3.48×1093.48\times10^{9}4.13×10114.13\times10^{11}4.49×10114.49\times10^{11}2.59×10102.59\times10^{10}5.62×10125.62\times10^{12}6.11×10126.11\times10^{12}3.26×10113.26\times10^{11}
OURS (FULL)6.18×1010\mathbf{6.18\times10^{10}}6.63×1010\mathbf{6.63\times10^{10}}3.33×1093.33\times10^{9}4.14×1011\mathbf{4.14\times10^{11}}4.50×1011\mathbf{4.50\times10^{11}}2.62×10102.62\times10^{10}5.63×1012\mathbf{5.63\times10^{12}}6.12×1012\mathbf{6.12\times10^{12}}3.46×10113.46\times10^{11}

The full model achieves the highest or tied-highest mean and maximum HV across these settings. Notably, on Penta-FJSP-5j5m, removing KL guidance reduces mean HV from 6.18×10106.18\times10^{10} to 6.15×10106.15\times10^{10}, whereas removing feature-wise reweighting results in 6.16×10106.16\times10^{10}. This indicates that the relative contributions of the two components can vary across optimization settings. Furthermore, the Wilcoxon signed-rank tests associated with Tables 5 and 6 indicate statistically significant improvements of the full model over GS-MODAC and both ablation variants in the evaluated Bi-CVRP and Penta-FJSP settings (p < 0.05).

To further examine whether Gaussian KL provides useful information beyond simpler temporal descriptors, we additionally compared it with mean displacement and Wasserstein distance on Bi-CVRP-200. Gaussian KL achieves the highest mean HV (162.38) and maximum HV (178.94), compared with mean displacement (160.15 and 176.28) and Wasserstein distance (161.71 and 177.99).

Table: Comparison of Alternative Temporal Metrics on Bi-CVRP-200

Temporal MetricMean HVMax HVStd
Mean displacement160.15176.289.53
Wasserstein distance161.71177.999.39
Gaussian KL162.38178.949.81

These results support the interpretation that KL guidance and feature-wise reweighting provide complementary benefits, although feature-wise reweighting may account for a larger share of the improvement in some settings.

AuthorsOpenReview ↗

Direct Comparison with STN-MODAC [W2 (part) / Q3]

We thank the reviewer for pointing out the need for a direct comparison with STN-MODAC under comparable experimental settings.

W2 (part): The absence of STN-MODAC, a closely related method that models multi-generational evolutionary dynamics, weakens the novelty and empirical comparison.

Q3: Why was STN-MODAC not included, and how does the proposed method compare with it under comparable computational budgets?

We agree that the absence of a direct STN-MODAC comparison was an important limitation of the original experimental evaluation.To address this concern, we added STN-MODAC as a baseline across nine FJSP settings, covering two, three, and five objectives at three different problem scales.

The overall comparisons are summarized in Table 1. Our method achieves the highest or tied-highest mean HV across these settings. Furthermore, Table 7 reports statistically significant improvements over STN-MODAC in all nine settings according to paired Wilcoxon signed-rank tests (p < 0.05). (Due to the character limit, Table 1 is provided in the next comment. Table 7 is presented in the first comment.)

Reviewer_hxuuOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates dynamic algorithm configuration for multi-objective combinatorial optimization and proposes an evolution-aware state representation method. By incorporating the KL divergence between consecutive populations and search progress information, the proposed method dynamically reweights GCN-based population features. It further employs PPO to adaptively adjust the crossover and mutation probabilities of NSGA-II. The authors conduct performance comparisons, transfer experiments, and ablation studies on FJSP and CVRP. The experimental results show that the proposed method achieves higher HV values across multiple test settings.

Critical strengths

The paper is well motivated and incorporates inter-generational evolutionary information into dynamic algorithm configuration. Experiments on FJSP and CVRP problems of different sizes demonstrate its potential practical value.

Critical weaknesses

1.The proposed method is closely related to GS-MODAC (ICML 2025) and STN-MODAC (ECAI 2025). Its main modifications involve KL divergence and feature-wise reweighting, but their novelty is not sufficiently established. Moreover, the lack of a direct comparison with STN-MODAC makes it difficult to assess the advantages over existing methods.

2.The paper uses KL divergence to measure distributional changes between consecutive populations, but its effectiveness in capturing evolutionary search states is not sufficiently justified. Several experimental results are very close to those of GS-MODAC, and the ablation studies are limited to Bi-CVRP, leaving the effectiveness of the proposed components across different objective counts and problem types insufficiently validated.

Questions for authors

1.STN-MODAC has already incorporated evolutionary history into dynamic algorithm configuration. The proposed method mainly uses KL instead of explicit trajectory modeling. The authors should better clarify its novelty and include a direct comparison with STN-MODAC in experiments.

2.Several results in Table 1 are very close to those of GS-MODAC, while the ablation studies are limited to Bi-CVRP. Additional ablation experiments on FJSP and a more detailed analysis of performance gains are needed.

3.The choice of KL divergence for measuring population changes is not sufficiently justified. A comparison with alternative distribution distance metrics is recommended.

4.The paper lacks an analysis of how crossover and mutation probabilities evolve across generations. Parameter trajectories would help explain the behavior of the learned configuration policy.

5.On Bi-CVRP-500, the training time increases by 32.38% compared with GS-MODAC. The authors should discuss whether this additional computational cost is justified by the performance gains.

6 comments
AuthorsOpenReview ↗

Novelty and Differentiation from GS-MODAC and STN-MODAC [W1 / Q1]

We thank the reviewer for raising important questions about the novelty of our representation and its distinction from GS-MODAC and STN-MODAC.

W1: The proposed method is closely related to GS-MODAC and STN-MODAC, and its novelty is insufficiently established. The lack of a direct STN-MODAC comparison further limits the empirical evidence.

Q1: How does the proposed method differ from STN-MODAC, and how does it perform against this temporal DAC baseline?

First, our contribution is not simply the introduction of an additional temporal feature, but the integration of inter-generational evolutionary information into the population representation through adaptive feature-wise modulation.

We acknowledge that our framework builds upon the graph-based population representation of GS-MODAC. However, GS-MODAC primarily represents the current population, whereas our method additionally incorporates recent evolutionary changes to adaptively recalibrate latent node embeddings.Compared with STN-MODAC, our approach differs in two main aspects:

1.Evolutionary information modeling. STN-MODAC uses a Search Trajectory Network and a temporal graph model to represent solution transitions across multiple generations. In contrast, our method adopts a population-level perspective, summarizing the distributional changes between consecutive populations using Gaussian KL divergence without explicitly maintaining multi-generational solution trajectories.

2.Evolutionary information integration. Rather than using temporal information solely as an additional state descriptor, our method combines KL divergence and normalized search progress to generate feature-wise weights. These weights adaptively modulate the GCN node embeddings before global pooling, allowing evolutionary dynamics to influence the contributions of different latent feature dimensions.

Second, we added STN-MODAC as a direct baseline across nine FJSP settings, covering two, three, and five objectives at three problem scales.

As reported in Table 1, our method achieves the highest or tied-highest mean HV across these settings. The paired Wilcoxon signed-rank tests in Table 7 of Appendix D.3 further indicate statistically significant improvements over STN-MODAC in all nine settings (p < 0.05).(Due to the character limit, Table 1 is provided in the next comment.)

Table 7: Wilcoxon Signed-Rank Test Results for Mean HV

Problem Settingvs. GS-MODACvs. NSGA-IIvs. STN-MODAC
Bi-FJSP-5j5m1.659e-18.787e-134.915e-15
Bi-FJSP-10j5m9.506e-19.549e-102.490e-12
Bi-FJSP-25j5m2.367e-22.747e-121.400e-10
Tri-FJSP-5j5m5.985e-61.231e-132.797e-12
Tri-FJSP-10j5m7.779e-13.471e-166.604e-11
Tri-FJSP-25j5m3.911e-23.897e-183.897e-18
Penta-FJSP-5j5m1.400e-103.201e-108.580e-12
Penta-FJSP-10j5m4.053e-13.750e-171.809e-4
Penta-FJSP-25j5m3.897e-183.897e-183.737e-15
Bi-CVRP-1003.832e-39.487e-81.975e-9
Bi-CVRP-2002.865e-105.817e-171.442e-13
Bi-CVRP-5004.159e-123.897e-181.143e-11

Overall, the key distinction is our compact, population-level evolution-aware representation, which directly guides latent feature modulation rather than explicitly modeling multi-generational search trajectories.

AuthorsOpenReview ↗

Table 1 Hypervolume results

MBi-FJSP-5j5mBi-FJSP-10j5mBi-FJSP-25j5m
mean,max,stdmean,max,stdmean,max,std
NSGA-II1.87×1041.87{\times}10^4,2.02×1042.02{\times}10^4,1.21×1031.21{\times}10^33.82×1043.82{\times}10^4,4.11×1044.11{\times}10^4,2.29×1032.29{\times}10^39.41×1049.41{\times}10^4,9.93×1049.93{\times}10^4,4.84×1034.84{\times}10^3
irace1.92×104\mathbf{1.92{\times}10^4},2.04×104\mathbf{2.04{\times}10^4},1.06×1031.06{\times}10^33.90×1043.90{\times}10^4,4.11×1044.11{\times}10^4,1.95×1031.95{\times}10^39.52×1049.52{\times}10^4,9.97×1049.97{\times}10^4,4.03×1034.03{\times}10^3
MADAC1.82×1041.82{\times}10^4,1.95×1041.95{\times}10^4,7.53×1027.53{\times}10^23.69×1043.69{\times}10^4,3.98×1043.98{\times}10^4,4.47×1034.47{\times}10^39.24×1049.24{\times}10^4,9.72×1049.72{\times}10^4,3.09×1033.09{\times}10^3
GS-MODAC1.92×104\mathbf{1.92{\times}10^4},2.04×104\mathbf{2.04{\times}10^4},1.07×1031.07{\times}10^33.91×104\mathbf{3.91{\times}10^4},4.14×1044.14{\times}10^4,1.88×1031.88{\times}10^39.51×1049.51{\times}10^4,9.98×1049.98{\times}10^4,4.46×1034.46{\times}10^3
STN-MODAC1.88×1041.88{\times}10^4,2.03×1042.03{\times}10^4,1.07×1031.07{\times}10^33.82×1043.82{\times}10^4,4.11×1044.11{\times}10^4,2.36×1032.36{\times}10^39.43×1049.43{\times}10^4,9.96×1049.96{\times}10^4,4.97×1034.97{\times}10^3
OURS1.92×104\mathbf{1.92{\times}10^4},2.04×104\mathbf{2.04{\times}10^4},1.07×1031.07{\times}10^33.91×104\mathbf{3.91{\times}10^4},4.15×104\mathbf{4.15{\times}10^4},1.95×1031.95{\times}10^39.54×104\mathbf{9.54{\times}10^4},10.0×104\mathbf{10.0{\times}10^4},4.60×1034.60{\times}10^3
MTri-FJSP-5j5mTri-FJSP-10j5mTri-FJSP-25j5m
mean,max,stdmean,max,stdmean,max,std
NSGA-II2.06×1062.06{\times}10^6,2.22×1062.22{\times}10^6,1.32×1051.32{\times}10^55.53×1065.53{\times}10^6,5.95×1065.95{\times}10^6,3.09×1053.09{\times}10^52.05×1072.05{\times}10^7,2.18×1072.18{\times}10^7,1.13×1061.13{\times}10^6
irace2.11×1062.11{\times}10^6,2.26×1062.26{\times}10^6,1.16×1051.16{\times}10^55.47×1065.47{\times}10^6,5.82×1065.82{\times}10^6,2.65×1052.65{\times}10^52.07×1072.07{\times}10^7,2.20×1072.20{\times}10^7,1.07×1061.07{\times}10^6
MADAC1.99×1061.99{\times}10^6,2.14×1062.14{\times}10^6,8.86×1048.86{\times}10^45.39×1065.39{\times}10^6,5.87×1065.87{\times}10^6,5.97×1055.97{\times}10^52.09×1072.09{\times}10^7,2.20×1072.20{\times}10^7,1.97×1061.97{\times}10^6
GS-MODAC2.11×1062.11{\times}10^6,2.25×1062.25{\times}10^6,1.17×1051.17{\times}10^55.67×1065.67{\times}10^6,6.08×1066.08{\times}10^6,2.98×1052.98{\times}10^52.15×1072.15{\times}10^7,2.28×1072.28{\times}10^7,1.05×1061.05{\times}10^6
STN-MODAC2.10×1062.10{\times}10^6,2.25×1062.25{\times}10^6,1.26×1051.26{\times}10^55.56×1065.56{\times}10^6,6.01×1066.01{\times}10^6,3.43×1053.43{\times}10^52.12×1072.12{\times}10^7,2.26×1072.26{\times}10^7,1.24×1061.24{\times}10^6
OURS2.13×106\mathbf{2.13{\times}10^6},2.27×106\mathbf{2.27{\times}10^6},1.25×1051.25{\times}10^55.68×106\mathbf{5.68{\times}10^6},6.09×106\mathbf{6.09{\times}10^6},3.19×1053.19{\times}10^52.16×107\mathbf{2.16{\times}10^7},2.29×107\mathbf{2.29{\times}10^7},1.12×1061.12{\times}10^6
MPenta-FJSP-5j5mPenta-FJSP-10j5mPenta-FJSP-25j5m
mean,max,stdmean,max,stdmean,max,std
NSGA-II6.01×10106.01{\times}10^{10},6.48×10106.48{\times}10^{10},3.70×1093.70{\times}10^93.96×10113.96{\times}10^{11},4.31×10114.31{\times}10^{11},2.42×10102.42{\times}10^{10}5.08×10125.08{\times}10^{12},5.48×10125.48{\times}10^{12},2.75×10112.75{\times}10^{11}
irace6.08×10106.08{\times}10^{10},6.49×10106.49{\times}10^{10},2.98×1092.98{\times}10^94.03×10114.03{\times}10^{11},4.38×10114.38{\times}10^{11},2.35×10102.35{\times}10^{10}5.18×10125.18{\times}10^{12},5.63×10125.63{\times}10^{12},2.92×10112.92{\times}10^{11}
MADAC5.82×10105.82{\times}10^{10},6.28×10106.28{\times}10^{10},2.72×1092.72{\times}10^93.91×10113.91{\times}10^{11},4.29×10114.29{\times}10^{11},4.36×10104.36{\times}10^{10}5.10×10125.10{\times}10^{12},5.74×10125.74{\times}10^{12},5.01×10115.01{\times}10^{11}
GS-MODAC6.17×10106.17{\times}10^{10},6.59×10106.59{\times}10^{10},3.13×1093.13{\times}10^94.14×1011\mathbf{4.14{\times}10^{11}},4.48×10114.48{\times}10^{11},2.38×10102.38{\times}10^{10}5.62×10125.62{\times}10^{12},6.07×10126.07{\times}10^{12},3.20×10113.20{\times}10^{11}
STN-MODAC6.07×10106.07{\times}10^{10},6.55×10106.55{\times}10^{10},3.57×1093.57{\times}10^94.07×10114.07{\times}10^{11},4.47×10114.47{\times}10^{11},2.65×10102.65{\times}10^{10}5.58×10125.58{\times}10^{12},6.05×10126.05{\times}10^{12},3.65×10113.65{\times}10^{11}
OURS6.18×1010\mathbf{6.18{\times}10^{10}},6.63×1010\mathbf{6.63{\times}10^{10}},3.33×1093.33{\times}10^94.14×1011\mathbf{4.14{\times}10^{11}},4.50×1011\mathbf{4.50{\times}10^{11}},2.52×10102.52{\times}10^{10}5.63×1012\mathbf{5.63{\times}10^{12}},6.12×1012\mathbf{6.12{\times}10^{12}},3.46×10113.46{\times}10^{11}
MBi-CVRP-100Bi-CVRP-200Bi-CVRP-500
mean,max,stdmean,max,stdmean,max,std
NSGA-II1.34×1021.34{\times}10^2,1.47×1021.47{\times}10^2,7.567.561.56×1021.56{\times}10^2,1.72×1021.72{\times}10^2,9.129.122.27×1022.27{\times}10^2,2.48×1022.48{\times}10^2,1.28×1011.28{\times}10^1
irace1.34×1021.34{\times}10^2,1.48×1021.48{\times}10^2,8.028.021.57×1021.57{\times}10^2,1.72×1021.72{\times}10^2,9.539.532.27×1022.27{\times}10^2,2.48×1022.48{\times}10^2,1.26×1011.26{\times}10^1
MADAC1.35×1021.35{\times}10^2,1.49×1021.49{\times}10^2,8.018.011.61×1021.61{\times}10^2,1.76×1021.76{\times}10^2,9.229.222.33×1022.33{\times}10^2,2.54×1022.54{\times}10^2,1.29×1011.29{\times}10^1
GS-MODAC1.35×1021.35{\times}10^2,1.48×1021.48{\times}10^2,7.957.951.60×1021.60{\times}10^2,1.76×1021.76{\times}10^2,9.509.502.35×1022.35{\times}10^2,2.59×1022.59{\times}10^2,1.41×1011.41{\times}10^1
STN-MODAC1.34×1021.34{\times}10^2,1.47×1021.47{\times}10^2,7.947.941.58×1021.58{\times}10^2,1.74×1021.74{\times}10^2,8.838.832.37×1022.37{\times}10^2,2.60×1022.60{\times}10^2,1.34×1011.34{\times}10^1
OURS1.36×102\mathbf{1.36{\times}10^2},1.51×102\mathbf{1.51{\times}10^2},8.138.131.62×102\mathbf{1.62{\times}10^2},1.79×102\mathbf{1.79{\times}10^2},9.809.802.42×102\mathbf{2.42{\times}10^2},2.65×102\mathbf{2.65{\times}10^2},1.36×1011.36{\times}10^1
AuthorsOpenReview ↗

Effectiveness of KL Guidance and Generality of the Proposed Components [W2 / Q2,Q3]

We thank the reviewer for encouraging a more comprehensive evaluation of KL guidance and the generality of the proposed components.

W2: The effectiveness of Gaussian KL in representing evolutionary states is insufficiently justified, several improvements over GS-MODAC are small, and the original ablations are limited to Bi-CVRP.

Q2: Can additional FJSP ablation experiments and performance analysis establish the effectiveness of the proposed components across different objective counts and problem types?

Q3: Why use Gaussian KL rather than alternative distributional distance metrics?

We agree that additional evidence is needed to clarify both the evolutionary information captured by KL divergence and the contributions of our proposed components beyond Bi-CVRP. To address these concerns, we have made three additional analyses.

First, we visualized the evolution of consecutive population distributions at three representative stages (Generations 4–6, 24–26, and 44–46), as presented in Table 3 of Appendix D.4. During the early and intermediate stages, the population distributions and non-dominated fronts exhibit noticeable changes, accompanied by relatively large KL values. In contrast, at the late stage, the populations become more concentrated, and KL values decrease to 0.106 and 0.039, indicating smaller inter-generational distributional changes.

These observations suggest that Gaussian KL can reflect changes in population location and dispersion, providing temporal information that is not explicitly represented by a single-generation population graph.

Second, we extended the ablation experiments to Penta-FJSP in Table 6 of Appendix D.3, covering 5j5m, 10j5m, and 25j5m. The complete model achieves the highest or tied-highest mean and maximum HV across these settings. Notably, on Penta-FJSP-5j5m, removing KL guidance reduces mean HV from 6.18×10106.18{\times}10^{10} to 6.15×10106.15{\times}10^{10}, whereas removing feature-wise reweighting results in 6.16×10106.16{\times}10^{10}. These results suggest that the relative contributions of KL guidance and feature-wise reweighting depend on the problem setting, rather than one component consistently accounting for all improvements.

Table 6: Ablation Study on Penta-FJSP

Method5j5m Mean5j5m Max5j5m Std10j5m Mean10j5m Max10j5m Std25j5m Mean25j5m Max25j5m Std
GS-MODAC6.17×10106.17\times10^{10}6.59×10106.59\times10^{10}3.13×1093.13\times10^{9}4.14×10114.14\times10^{11}4.48×10114.48\times10^{11}2.38×10102.38\times10^{10}5.62×10125.62\times10^{12}6.07×10126.07\times10^{12}3.20×10113.20\times10^{11}
OURS (w/o feature-wise reweight)6.16×10106.16\times10^{10}6.55×10106.55\times10^{10}2.96×1092.96\times10^{9}4.14×10114.14\times10^{11}4.49×10114.49\times10^{11}2.52×10102.52\times10^{10}5.58×10125.58\times10^{12}6.06×10126.06\times10^{12}3.35×10113.35\times10^{11}
OURS (w/o KL guidance)6.15×10106.15\times10^{10}6.59×10106.59\times10^{10}3.48×1093.48\times10^{9}4.13×10114.13\times10^{11}4.49×10114.49\times10^{11}2.59×10102.59\times10^{10}5.62×10125.62\times10^{12}6.11×10126.11\times10^{12}3.26×10113.26\times10^{11}
OURS (FULL)6.18×1010\mathbf{6.18\times10^{10}}6.63×1010\mathbf{6.63\times10^{10}}3.33×1093.33\times10^{9}4.14×1011\mathbf{4.14\times10^{11}}4.50×1011\mathbf{4.50\times10^{11}}2.62×10102.62\times10^{10}5.63×1012\mathbf{5.63\times10^{12}}6.12×1012\mathbf{6.12\times10^{12}}3.46×10113.46\times10^{11}

Furthermore, the Wilcoxon signed-rank tests associated with Tables 5 and 6 indicate statistically significant improvements of the full model over GS-MODAC and both ablation variants in the evaluated Bi-CVRP and Penta-FJSP settings (p < 0.05).

Third, we compared Gaussian KL with mean displacement and Wasserstein distance on Bi-CVRP-200.

Table: Comparison of Alternative Temporal Metrics on Bi-CVRP-200

Temporal MetricMean HVMax HVStd
Mean displacement160.15176.289.53
Wasserstein distance161.71177.999.39
Gaussian KL162.38178.949.81

Gaussian KL achieves the highest mean HV (162.38) and maximum HV (178.94) among the evaluated descriptors. Mean displacement characterizes population-center movement, whereas distribution-based metrics can also describe population dispersion. The results suggest that Gaussian KL provides useful complementary information for the learned configuration policy, although its numerical advantage over Wasserstein distance remains modest.

Overall, the additional visualizations, cross-problem ablations, and descriptor comparisons support the usefulness of Gaussian KL as a compact descriptor of evolutionary changes.

Reviewer_NEGZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents an evolution-aware state representation that incorporates the Kullback–Leibler (KL) divergence between consecutive populations to capture intergenerational transitions. A PPO policy adjusts NSGA-II’s crossover and mutation probabilities. Experiments on multi-objective scheduling and routing report improved hypervolume, transfer across sizes and problem variants, and modest evaluation overhead.

Critical strengths
  1. The work offers a simple way to incorporate recent population changes into graph-based DAC, with limited reported evaluation overhead.
  2. Experiments provide useful evidence of practical effectiveness across multiple problems, objective counts, and transfer settings.
Critical weaknesses
  1. The KL contribution is insufficiently established. In Table 5, removing KL guidance yields a mean HV lower than the full model.
  2. STN-MODAC, a directly relevant temporal DAC method discussed in the paper, is not evaluated. This limits the evidence for the proposed representation’s advantage over existing temporal approaches.
Questions for authors

Please clarify whether consecutive populations share the same normalization and whether the KL ablation removes the signal from both the gating network and the policy state. Direct comparisons with simpler temporal statistics would help establish the specific value of the Gaussian KL.

4 comments
AuthorsOpenReview ↗

Contribution of KL Guidance and Comparison with Alternative Temporal Descriptors [W1 / Q1 (part)]

We thank the reviewer for the thoughtful comments on the contribution of KL guidance and the need for comparisons with simpler temporal descriptors.

W1: The contribution of KL guidance is insufficiently established.

Q1 (part): Can comparisons with simpler temporal statistics establish the specific value of Gaussian KL?

We acknowledge that the performance differences between the full model and the variant without KL guidance are relatively small in some Bi-CVRP settings, and that feature-wise reweighting contributes substantially to the overall improvement.

However, KL guidance and feature-wise reweighting serve complementary roles. Feature-wise reweighting adaptively adjusts the contributions of latent embedding dimensions, whereas KL divergence provides information about inter-generational distributional changes. This additional temporal information helps the model distinguish evolutionary states that may have similar current population representations but different recent search dynamics.

To further examine their contributions, we extended the ablation experiments to Penta-FJSP, covering three problem scales (5j5m, 10j5m, and 25j5m), as reported in Table 6 of Appendix D.3.

Table 6: Ablation Study on Penta-FJSP

Method5j5m Mean5j5m Max5j5m Std10j5m Mean10j5m Max10j5m Std25j5m Mean25j5m Max25j5m Std
GS-MODAC6.17e106.59e103.13e94.14e114.48e112.38e105.62e126.07e123.20e11
OURS (w/o feature-wise reweight)6.16e106.55e102.96e94.14e114.49e112.52e105.58e126.06e123.35e11
OURS (w/o KL guidance)6.15e106.59e103.48e94.13e114.49e112.59e105.62e126.11e123.26e11
OURS (FULL)6.18e106.63e103.33e94.14e114.50e112.62e105.63e126.12e123.46e11

The full model achieves the highest or tied-highest mean and maximum HV across these settings. Moreover, the paired Wilcoxon signed-rank tests associated with Tables 5 and 6 indicate statistically significant improvements over GS-MODAC and both ablation variants in the evaluated Bi-CVRP and Penta-FJSP settings (p < 0.05).

We also compared Gaussian KL with two alternative temporal descriptors, mean displacement and Wasserstein distance, on Bi-CVRP-200:

Table: Comparison of Alternative Temporal Metrics on Bi-CVRP-200

Temporal DescriptorMean HVMax HVStd
Mean displacement160.15176.289.53
Wasserstein distance161.71177.999.39
Gaussian KL162.38178.949.81

Gaussian KL improves mean HV by approximately 1.39% over mean displacement and 0.41% over Wasserstein distance. These results suggest that incorporating distributional changes beyond population-center movement can benefit dynamic configuration, although the advantage over Wasserstein distance is relatively modest. Collectively, the ablation results in Table 6 and visual comparisons in Figure 4 verify the unique evolutionary information provided by KL guidance. Although feature reweighting dominates the performance gain, KL serves as a valid and non-redundant complementary component.

AuthorsOpenReview ↗

Direct Comparison with STN-MODAC [W2]

We thank the reviewer for emphasizing the importance of a direct experimental comparison with STN-MODAC.

W2: STN-MODAC is a directly relevant temporal DAC method but was not included in the original experimental comparisons. We agree that a direct comparison with STN-MODAC is necessary to better establish the empirical effectiveness of our approach relative to existing temporal DAC methods.

To address this concern, we added STN-MODAC as an additional baseline and evaluated it on nine FJSP settings, covering two, three, and five objectives at three problem scales.

The overall comparisons are summarized in Table 1. The results show that the proposed method achieves the highest or tied-highest mean HV across these settings. Furthermore, the paired Wilcoxon signed-rank tests in Table 7 show statistically significant improvements over STN-MODAC in all nine evaluated FJSP settings (p < 0.05).(Due to the character limit, Table 1 is provided in the next comment.)

Table 7: Wilcoxon Signed-Rank Test Results for Mean HV

Problem Settingvs. GS-MODACvs. NSGA-IIvs. STN-MODAC
Bi-FJSP-5j5m1.659×10−11.659\times10^{-1}8.787×10−13\mathbf{8.787\times10^{-13}}4.915×10−15\mathbf{4.915\times10^{-15}}
Bi-FJSP-10j5m9.506×10−19.506\times10^{-1}9.549×10−10\mathbf{9.549\times10^{-10}}2.490×10−12\mathbf{2.490\times10^{-12}}
Bi-FJSP-25j5m2.367×10−2\mathbf{2.367\times10^{-2}}2.747×10−12\mathbf{2.747\times10^{-12}}1.400×10−10\mathbf{1.400\times10^{-10}}
Tri-FJSP-5j5m5.985×10−6\mathbf{5.985\times10^{-6}}1.231×10−13\mathbf{1.231\times10^{-13}}2.797×10−12\mathbf{2.797\times10^{-12}}
Tri-FJSP-10j5m7.779×10−17.779\times10^{-1}3.471×10−16\mathbf{3.471\times10^{-16}}6.604×10−11\mathbf{6.604\times10^{-11}}
Tri-FJSP-25j5m3.911×10−2\mathbf{3.911\times10^{-2}}3.897×10−18\mathbf{3.897\times10^{-18}}3.897×10−18\mathbf{3.897\times10^{-18}}
Penta-FJSP-5j5m1.400×10−10\mathbf{1.400\times10^{-10}}3.201×10−10\mathbf{3.201\times10^{-10}}8.580×10−12\mathbf{8.580\times10^{-12}}
Penta-FJSP-10j5m4.053×10−14.053\times10^{-1}3.750×10−17\mathbf{3.750\times10^{-17}}1.809×10−4\mathbf{1.809\times10^{-4}}
Penta-FJSP-25j5m3.897×10−18\mathbf{3.897\times10^{-18}}3.897×10−18\mathbf{3.897\times10^{-18}}3.737×10−15\mathbf{3.737\times10^{-15}}
Bi-CVRP-1003.832×10−3\mathbf{3.832\times10^{-3}}9.487×10−8\mathbf{9.487\times10^{-8}}1.975×10−9\mathbf{1.975\times10^{-9}}
Bi-CVRP-2002.865×10−10\mathbf{2.865\times10^{-10}}5.817×10−17\mathbf{5.817\times10^{-17}}1.442×10−13\mathbf{1.442\times10^{-13}}
Bi-CVRP-5004.159×10−12\mathbf{4.159\times10^{-12}}3.897×10−18\mathbf{3.897\times10^{-18}}1.143×10−11\mathbf{1.143\times10^{-11}}

These additional experiments provide empirical evidence that the proposed population-level evolution-aware representation is competitive with existing method that explicitly model multi-generational search trajectories.

AuthorsOpenReview ↗

Table 1 Hypervolume results

MBi-FJSP-5j5mBi-FJSP-10j5mBi-FJSP-25j5m
mean,max,stdmean,max,stdmean,max,std
NSGA-II1.87×1041.87{\times}10^4,2.02×1042.02{\times}10^4,1.21×1031.21{\times}10^33.82×1043.82{\times}10^4,4.11×1044.11{\times}10^4,2.29×1032.29{\times}10^39.41×1049.41{\times}10^4,9.93×1049.93{\times}10^4,4.84×1034.84{\times}10^3
irace1.92×104\mathbf{1.92{\times}10^4},2.04×104\mathbf{2.04{\times}10^4},1.06×1031.06{\times}10^33.90×1043.90{\times}10^4,4.11×1044.11{\times}10^4,1.95×1031.95{\times}10^39.52×1049.52{\times}10^4,9.97×1049.97{\times}10^4,4.03×1034.03{\times}10^3
MADAC1.82×1041.82{\times}10^4,1.95×1041.95{\times}10^4,7.53×1027.53{\times}10^23.69×1043.69{\times}10^4,3.98×1043.98{\times}10^4,4.47×1034.47{\times}10^39.24×1049.24{\times}10^4,9.72×1049.72{\times}10^4,3.09×1033.09{\times}10^3
GS-MODAC1.92×104\mathbf{1.92{\times}10^4},2.04×104\mathbf{2.04{\times}10^4},1.07×1031.07{\times}10^33.91×104\mathbf{3.91{\times}10^4},4.14×1044.14{\times}10^4,1.88×1031.88{\times}10^39.51×1049.51{\times}10^4,9.98×1049.98{\times}10^4,4.46×1034.46{\times}10^3
STN-MODAC1.88×1041.88{\times}10^4,2.03×1042.03{\times}10^4,1.07×1031.07{\times}10^33.82×1043.82{\times}10^4,4.11×1044.11{\times}10^4,2.36×1032.36{\times}10^39.43×1049.43{\times}10^4,9.96×1049.96{\times}10^4,4.97×1034.97{\times}10^3
OURS1.92×104\mathbf{1.92{\times}10^4},2.04×104\mathbf{2.04{\times}10^4},1.07×1031.07{\times}10^33.91×104\mathbf{3.91{\times}10^4},4.15×104\mathbf{4.15{\times}10^4},1.95×1031.95{\times}10^39.54×104\mathbf{9.54{\times}10^4},10.0×104\mathbf{10.0{\times}10^4},4.60×1034.60{\times}10^3
MTri-FJSP-5j5mTri-FJSP-10j5mTri-FJSP-25j5m
mean,max,stdmean,max,stdmean,max,std
NSGA-II2.06×1062.06{\times}10^6,2.22×1062.22{\times}10^6,1.32×1051.32{\times}10^55.53×1065.53{\times}10^6,5.95×1065.95{\times}10^6,3.09×1053.09{\times}10^52.05×1072.05{\times}10^7,2.18×1072.18{\times}10^7,1.13×1061.13{\times}10^6
irace2.11×1062.11{\times}10^6,2.26×1062.26{\times}10^6,1.16×1051.16{\times}10^55.47×1065.47{\times}10^6,5.82×1065.82{\times}10^6,2.65×1052.65{\times}10^52.07×1072.07{\times}10^7,2.20×1072.20{\times}10^7,1.07×1061.07{\times}10^6
MADAC1.99×1061.99{\times}10^6,2.14×1062.14{\times}10^6,8.86×1048.86{\times}10^45.39×1065.39{\times}10^6,5.87×1065.87{\times}10^6,5.97×1055.97{\times}10^52.09×1072.09{\times}10^7,2.20×1072.20{\times}10^7,1.97×1061.97{\times}10^6
GS-MODAC2.11×1062.11{\times}10^6,2.25×1062.25{\times}10^6,1.17×1051.17{\times}10^55.67×1065.67{\times}10^6,6.08×1066.08{\times}10^6,2.98×1052.98{\times}10^52.15×1072.15{\times}10^7,2.28×1072.28{\times}10^7,1.05×1061.05{\times}10^6
STN-MODAC2.10×1062.10{\times}10^6,2.25×1062.25{\times}10^6,1.26×1051.26{\times}10^55.56×1065.56{\times}10^6,6.01×1066.01{\times}10^6,3.43×1053.43{\times}10^52.12×1072.12{\times}10^7,2.26×1072.26{\times}10^7,1.24×1061.24{\times}10^6
OURS2.13×106\mathbf{2.13{\times}10^6},2.27×106\mathbf{2.27{\times}10^6},1.25×1051.25{\times}10^55.68×106\mathbf{5.68{\times}10^6},6.09×106\mathbf{6.09{\times}10^6},3.19×1053.19{\times}10^52.16×107\mathbf{2.16{\times}10^7},2.29×107\mathbf{2.29{\times}10^7},1.12×1061.12{\times}10^6
MPenta-FJSP-5j5mPenta-FJSP-10j5mPenta-FJSP-25j5m
mean,max,stdmean,max,stdmean,max,std
NSGA-II6.01×10106.01{\times}10^{10},6.48×10106.48{\times}10^{10},3.70×1093.70{\times}10^93.96×10113.96{\times}10^{11},4.31×10114.31{\times}10^{11},2.42×10102.42{\times}10^{10}5.08×10125.08{\times}10^{12},5.48×10125.48{\times}10^{12},2.75×10112.75{\times}10^{11}
irace6.08×10106.08{\times}10^{10},6.49×10106.49{\times}10^{10},2.98×1092.98{\times}10^94.03×10114.03{\times}10^{11},4.38×10114.38{\times}10^{11},2.35×10102.35{\times}10^{10}5.18×10125.18{\times}10^{12},5.63×10125.63{\times}10^{12},2.92×10112.92{\times}10^{11}
MADAC5.82×10105.82{\times}10^{10},6.28×10106.28{\times}10^{10},2.72×1092.72{\times}10^93.91×10113.91{\times}10^{11},4.29×10114.29{\times}10^{11},4.36×10104.36{\times}10^{10}5.10×10125.10{\times}10^{12},5.74×10125.74{\times}10^{12},5.01×10115.01{\times}10^{11}
GS-MODAC6.17×10106.17{\times}10^{10},6.59×10106.59{\times}10^{10},3.13×1093.13{\times}10^94.14×1011\mathbf{4.14{\times}10^{11}},4.48×10114.48{\times}10^{11},2.38×10102.38{\times}10^{10}5.62×10125.62{\times}10^{12},6.07×10126.07{\times}10^{12},3.20×10113.20{\times}10^{11}
STN-MODAC6.07×10106.07{\times}10^{10},6.55×10106.55{\times}10^{10},3.57×1093.57{\times}10^94.07×10114.07{\times}10^{11},4.47×10114.47{\times}10^{11},2.65×10102.65{\times}10^{10}5.58×10125.58{\times}10^{12},6.05×10126.05{\times}10^{12},3.65×10113.65{\times}10^{11}
OURS6.18×1010\mathbf{6.18{\times}10^{10}},6.63×1010\mathbf{6.63{\times}10^{10}},3.33×1093.33{\times}10^94.14×1011\mathbf{4.14{\times}10^{11}},4.50×1011\mathbf{4.50{\times}10^{11}},2.52×10102.52{\times}10^{10}5.63×1012\mathbf{5.63{\times}10^{12}},6.12×1012\mathbf{6.12{\times}10^{12}},3.46×10113.46{\times}10^{11}
MBi-CVRP-100Bi-CVRP-200Bi-CVRP-500
mean,max,stdmean,max,stdmean,max,std
NSGA-II1.34×1021.34{\times}10^2,1.47×1021.47{\times}10^2,7.567.561.56×1021.56{\times}10^2,1.72×1021.72{\times}10^2,9.129.122.27×1022.27{\times}10^2,2.48×1022.48{\times}10^2,1.28×1011.28{\times}10^1
irace1.34×1021.34{\times}10^2,1.48×1021.48{\times}10^2,8.028.021.57×1021.57{\times}10^2,1.72×1021.72{\times}10^2,9.539.532.27×1022.27{\times}10^2,2.48×1022.48{\times}10^2,1.26×1011.26{\times}10^1
MADAC1.35×1021.35{\times}10^2,1.49×1021.49{\times}10^2,8.018.011.61×1021.61{\times}10^2,1.76×1021.76{\times}10^2,9.229.222.33×1022.33{\times}10^2,2.54×1022.54{\times}10^2,1.29×1011.29{\times}10^1
GS-MODAC1.35×1021.35{\times}10^2,1.48×1021.48{\times}10^2,7.957.951.60×1021.60{\times}10^2,1.76×1021.76{\times}10^2,9.509.502.35×1022.35{\times}10^2,2.59×1022.59{\times}10^2,1.41×1011.41{\times}10^1
STN-MODAC1.34×1021.34{\times}10^2,1.47×1021.47{\times}10^2,7.947.941.58×1021.58{\times}10^2,1.74×1021.74{\times}10^2,8.838.832.37×1022.37{\times}10^2,2.60×1022.60{\times}10^2,1.34×1011.34{\times}10^1
OURS1.36×102\mathbf{1.36{\times}10^2},1.51×102\mathbf{1.51{\times}10^2},8.138.131.62×102\mathbf{1.62{\times}10^2},1.79×102\mathbf{1.79{\times}10^2},9.809.802.42×102\mathbf{2.42{\times}10^2},2.65×102\mathbf{2.65{\times}10^2},1.36×1011.36{\times}10^1
AuthorsOpenReview ↗

Population Normalization and KL Ablation Implementation [Q1 (part)]

We thank the reviewer for requesting clarification of the population normalization procedure and the implementation of the KL ablation.

Q1 (part): Do consecutive populations share the same normalization, and does the KL ablation remove the KL signal from both the gating network and the policy state?

First, GCN node-feature normalization and KL computation are handled separately.

For GCN node features, we apply dimension-wise min-max normalization to the objective values. The normalization bounds are initialized using the objective-wise extrema of the initial population and dynamically updated in each generation according to newly evaluated offspring. Consequently, consecutive generations do not necessarily share identical normalization bounds. However, all individuals within the same generation share the same set of normalization bounds, ensuring consistent feature scaling within each population.

In contrast, the KL divergence is calculated directly from the original objective vectors of consecutive populations, without additional generation-dependent normalization. Thus, both population distributions are estimated in the same original objective coordinate system, avoiding artificial distributional changes caused by different normalization references. The resulting KL value is then transformed as D^tKL=DtKLDtKL+1\displaystyle \widehat{D}_t^{\mathrm{KL}} = \frac{D_t^{\mathrm{KL}}}{D_t^{\mathrm{KL}} + 1} to obtain a bounded feature within [0, 1).

Second, the KL signal is used in both the feature-wise gating network and the policy state.

In our framework, the evolution-aware feature vector consists of two components: normalized search progress and the KL divergence between consecutive populations. This feature vector is used in two pathways:

1)It serves as the contextual input to the feature-weight generation network, which produces the feature-wise weights used to modulate the GCN node embeddings.

2)It is concatenated with the graph-level population representation and directly provided to the actor and critic networks as part of the policy state.

In the w/o KL guidance variant, the KL component is removed from the shared evolution-aware feature vector in both pathways, while the normalized search-progress component is retained.Therefore, the ablation removes the KL signal from both the feature-wise gating mechanism and the policy state, while keeping the remaining architecture unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.