- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes GIFT-Evo, a learning-based evolutionary optimizer designed for black-box optimization under a limited budget of function evaluations. The method combines offline-trained operators with the individual's search state that records information. During iteration, the model parameters remain fixed while the search state continuously evolves. Experimental evaluations include the B2Opt-based BBOB implementation, multi-dimensional deployment, component ablation studies, and validation on two scenarios.
Critical strengths
The method described in the text is clear and detailed, the experimental results and analysis are quite comprehensive.
Critical weaknesses
My main concern is that these surprising and prominent results might stem from the interaction between the construction of the benchmark tests and the update rules that depend on the origin. Table 16 particularly surprised me: under 1000-dimensional conditions, GIFT-Evo achieved nearly optimal performance on many functions after just 3100 function evaluations; in contrast, other algorithms still showed significant gaps when using the original evaluations. Similar large gaps were observed on several other functions. These results have raised my concern: is this method actually learning to locate the unknown global optimum, or is it merely converging toward a particular point that has been specially favored?
This issue is highly similar to Kudela's concern raised in "A critical problem in benchmarking and analysis of evolutionary computation methods" (Nature Machine Intelligence, 2022): when the optimal point of a benchmark function is placed at the center, operators biased toward the center might perform exceptionally well; however, their performance may change significantly when the problem is shifted.
-
Section 4.1 and Appendix E.3 indicate that the experiment uses an implementation of BBOB paired with B2Opt. In the publicly available B2Opt evaluation script, exps/main.py lines 344–356 explicitly sets xopt to zero for all functions except F05 and F24, and sets fopt to zero during instance generation. Lines 378–391 then loads these instances for evaluation. This directly places the optimal solution x for the majority of functions at the zero point.
-
F05 provides a noteworthy contrast. According to lines 236-238 of utils.py, the optimal point of this function lies at boundary corner points, not at zero. In Figure 9, when d=100 and all methods use the same 3100 evaluation budget, GIFT-Evo does not demonstrate its significant convergence advantage observed on many other functions, and significantly lags behind in both comparisons. The results for F05 further support the aforementioned concerns.
-
Having only test functions does not necessarily mean the algorithm is exploiting such properties. Therefore, I carefully examined the specific method definitions of GIFT-Evo. Eq. (7) combines three sets of coefficients to generate the absolute coordinates of candidate solutions.
Appendix A.4 explicitly states that the sum of these three coefficients does not have to equal 1.
Let . The update rule can be rewritten as
When the candidate solutions in each branch are close to one another, this update approximately becomes
Specifically, by setting and , the operator can implement
For a simple function where the optimal point lies at zero, adopting such a contraction rule is a shortcut to reducing training objectives. The same rule can naturally generalize across dimensions without requiring the model to actually learn the location of unknown optimal points. Even if the coefficient network takes relative values as input, the final update still depends on the absolute coordinates of the candidate solutions.
-
What particularly puzzles me is that the authors trained only on a few simple functions CECF1 to CECF3 with d=10, and yet report such broad and significant generalization advantages. This result contradicts the trend observed in Lange et al. "Discovering Evolution Strategies via Meta-Black-Box Optimization", which analyzes the relationship between training distribution and generalization performance: their analysis suggests that incorporating more complex, multi-modal, and ill-conditioned training functions helps improve generalization, and that the training process also randomizes the location of optimal solutions.
In contrast, this paper claims to achieve strong cross-function and cross-dimensional transfer capabilities using only extremely limited and simple training functions, without providing a clear explanation for why such a simple training distribution could support these results. This discrepancy requires a compelling mechanistic explanation and empirical evidence, especially one that rules out the shortcut of shared optimal solution locations between training and test functions.
Current deployment-phase ablation experiments are insufficient to address this issue, as they fail to distinguish whether the model has learned a search strategy adapted to different target functions or simply a rule that contracts toward the origin based on feedback.
The authors should first clearly define the specific forms of CECF1–CECF3 and the distribution of their optimal solutions, and then verify whether the reported generalization advantages still hold when both training and test functions have randomized optimal solution locations. Additionally, a comparison should be made between the trained model and a simple contraction baseline on both original and shifted instances, recording the sum of fusion coefficients and population trajectories, and evaluating a retrained variant where the sum of fusion coefficients is constrained to 1 but negative weights are allowed.
Without such evidence, I find it difficult to accept that the prominent results achieved under such a simple training setup can be directly attributed to broad, transferable meta-learning capabilities; contraction toward the origin remains a plausible alternative explanation.
Questions for authors
-
Did the actual BBOB instances used in this paper retain the handling of setting xopt to zero as in B2Opt? Please provide evidence to clearly identify the actual optimal points for each test function. When the optimal points are independently randomly translated, or when using official COCO instances, does GIFT-Evo maintain its advantage under low evaluation budgets, especially its strong performance in high-dimensional cases highlighted in Table 16?
-
Please report the fusion coefficients of GIFT-Evo on functions F01, F03, and F05, as well as the evolution of the sum with iteration, and analyze whether the update exhibits a continuous trend of converging toward the origin, based on the population center trajectory. Compared to a simple fixed contraction baseline, how significant is GIFT-Evo's advantage on both original and translated instances? If the sum of fusion coefficients is constrained to 1, while retaining negative weights and retraining, do the main performance conclusions still hold?
-
Please clearly define CECF1–CECF3, specify the instance generation method, and describe the distribution of optimal point locations. How to explain that such remarkable cross-function and cross-dimensional generalization performance can be achieved by training only on three simple functions at d=10, when LES analysis shows that a richer training distribution helps improve generalization?
For thanks
Thank you for your careful analysis and for highlighting this potential confounding factor. The central objective of GIFT-Evo is to investigate how individual search feedback can guide learned evolutionary operators while keeping model parameters fixed during deployment.
As you noted, origin-related bias is a broader benchmarking concern in evolutionary optimization. Our evaluation follows a B2Opt-derived instance-construction protocol, making your observations directly relevant to the evidence presented in our paper. Although addressing coordinate-origin bias was not the primary focus of this work, ruling out this confound is necessary to substantiate our performance and generalization claims.
In a future revision, we will explicitly document the training functions, instance-generation procedures, and optimum-location distributions. We will incorporate evaluations with independently shifted optima and validated benchmark instances, comparisons against fixed contraction baselines, and a retrained fusion variant whose coefficients sum to one while allowing negative weights. We will also report the fusion coefficients and population trajectories to distinguish origin-dependent contraction from effective search behavior.
We appreciate your suggestions and will revise the framework and conclusions according to the resulting evidence.