- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes an LLM feedback-driven operator evolution framework for constrained multi-objective optimization. Instead of asking an LLM to generate executable operator code, the method keeps the original GA/DE variation interface and parameterizes candidate search behaviors through four directional weights—random exploration, constraint improvement, objective convergence, and diversity restoration—plus a step size. Candidate operators are screened online using constraint progress, feasible-offspring rate, Tchebycheff-based objective improvement, and diversity contribution. Weak candidates are periodically reconstructed by an LLM, which returns only bounded parameters, not code. The original operator remains as an anchor, and reconstructed candidates must re-enter screening before participating in main evolution. The method is integrated into five CMOEAs (CCMO, C3M, C-TAEA, CMOEA-MS, MOEA/D-DAE) and evaluated on LIR-CMOP, DAS-CMOP, RWMOP, and eight MOOPF cases, with IGD/HV or IGD+/HV, 30 runs, Wilcoxon tests, parameter sensitivity for δ and γ, and an ablation study on MOOPF.
Critical strengths
-
By restricting the LLM to four weights and a step size and by preserving the original variation interface, the framework avoids the validity, repair, and interface problems that plague LLM-generated operator code. This makes the LLM intervention auditable and comparatively easy to integrate into existing CMOEAs.
-
It separates constraint progress, feasible-offspring generation, objective improvement, and diversity, and it aggregates them according to the population’s feasible ratio. This is a reasonable way to credit operators under changing feasibility regimes, and it gives the reconstruction step a meaningful signal.
-
The empirical scope is broad. The paper tests five base CMOEAs, 35 CMOPs, 12 RWMOP cases, and 8 MOOPF cases, with 30 runs and two metrics. If the tables were clean and the comparisons were fully reported, this would provide useful evidence about where the framework helps and where it does not.
Critical weaknesses
-
The LLM returns only four nonnegative weights and a scalar step size, i.e., a five-dimensional bounded proposal. There is no comparison against random search, Bayesian optimization, CMA-ES, or even a simple grid/random perturbation over the same parameter space under the same screening budget. The LR ablation replaces LLM reconstruction with parameter transfer from the reference operator, which tests a different hypothesis. Without a random/BO baseline, the paper cannot establish that the LLM is doing anything beyond proposing a low-dimensional parameter vector. This is consequential because the central claim is “LLM feedback-driven operator evolution,” not merely “bounded operator parameter adaptation.”
-
The paper reports large gains on LIR-CMOP but also clear degradations on DAS-CMOP5 and DAS-CMOP7 for CCMO, and several RWMOP IGD results are statistically similar or worse. The runtime increases roughly 2× to 9× depending on the base CMOEA (e.g., CCMO 0.4→3.8 minutes, CMOEA-MS 0.2→1.7 minutes), without a corresponding analysis of when this cost is justified. More seriously, Tables 1–3 and several appendix tables are garbled in the submitted PDF, so I could not verify the main quantitative claims, the win/loss counts, or the sensitivity results. The ablation is also limited to eight MOOPF cases and reports mainly Friedman ranks; it does not cover LIR-CMOP or DAS-CMOP.
-
The framework introduces many hyperparameters and control thresholds without adequate sensitivity analysis.
-
The paper does not report the exact number of LLM calls per run, API cost, latency, validation-failure rate, or variance across repeated runs. Temperature is set to 0, but that does not guarantee deterministic outputs across API calls. The cross-LLM experiments with MiMo-V2.5-Pro and Qwen-Plus are mentioned but not shown in the provided main text or appendix extracts. There is no code release, no output logs, and no discussion of how invalid or malformed proposals are handled beyond “strict mode terminates.” These omissions make it difficult to judge whether the reported gains are stable or reproducible.
Questions for authors
-
Can you add a baseline that replaces the LLM with random search, Bayesian optimization, or CMA-ES over the same five-dimensional bounded parameter space, using the same screening budget and reconstruction schedule? This would directly test whether the LLM contributes beyond low-dimensional parameter adaptation.
-
Can you provide clean, complete tables for all five CMOEAs on LIR-CMOP, DAS-CMOP, and RWMOP, with per-problem means/standard deviations and explicit win/loss/tie counts? Can you also extend the ablation beyond MOOPF to at least LIR-CMOP and DAS-CMOP?
-
Can you report the number of LLM calls per run, API cost, latency, validation-failure rate, and run-to-run variance? How deterministic are repeated runs under temperature 0, and where are the cross-LLM results for MiMo-V2.5-Pro and Qwen-Plus?
-
Can you justify and sensitivity-test K, S, H, Trec, ρ_max, μ_sw, β_tr, and ω_O? In particular, how often does a non-anchor candidate actually participate in main evolution, and how often does LLM reconstruction change the operator’s behavior in a way that survives screening?