- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies online hyperparameter control for population-based metaheuristics using frozen LLMs. It proposes HypaEvo, which separates search-state assessment from numerical actuation. Experiments cover TSP, CVRP, FSSP, and UAV trajectory optimization with GA, ACO, and PSO backbones. Comparisons with AutoEP and other baselines, component ablations, scorer-replacement experiments, and cost analyses support the practical effectiveness of the proposed architecture.
Critical strengths
- Separating LLM assessment from numerical actuation is an attractive design choice. Asking an LLM to assess search conditions while deterministic code handles arithmetic, parameter bounds, and update constraints reduces the burden of generating absolute parameter values. This provides a practical and interpretable interface for LLM-based optimization control without controller-specific training.
- The experimental design of the paper is relatively rigorous. HypaEvo improves solution quality over the re-run AutoEP baseline on the reported TSP comparisons and extends to CVRP, FSSP, and UAV optimization with different metaheuristic families. In addition, the paper provides sufficient ablation experiments and sensitivity analysis.
Critical weaknesses
- A little technical concern. HypaEvo prespecifies "in which direction each hyperparameter should be adjusted to promote exploration or exploitation". However, reliance on fixed directional priors may be brittle when hyperparameter effects are strongly state-dependent or interaction-dominated; the method does not adapt these priors online.
- The claim that only ordinal assessment is required is not fully consistent with the actual control rule. Section 4.3 states that the scores need only be ordinally comparable under a shared rubric. However, Section 4.4 uses their difference,
z = (u_E - u_X) / 10, to determine both update direction and magnitude, and their sum,a = min(1, (u_E + u_X) / tau_ref), to determine the urgency gate. A shared ordering can determine the sign of the difference, but cannot determine these magnitudes: they also depend on numerical spacing and consistency between the two assessors' scales. An order-preserving nonlinear recoding can preserve score rankings while changing both the update size and the gate. The authors should describe the scores more precisely as bounded numerical control signals and test sensitivity to monotone recoding and cross-role calibration differences. - The contribution of the scoring interface is not yet fully separated from the contribution of action constraints. The same-coordinator experiment establishes the usefulness of LLM-derived scores relative to the tested alternative scorers, while the scoring/direct-generation comparisons evaluate different actuation interfaces. Neither fully isolates whether scoring itself is more reliable than numerical generation once both receive the same local action restrictions, directional priors, step-size bounds, gating, and damping. The LLM-actuator ablation also changes how these structural guarantees are enforced. A stronger control would let an LLM generate a bounded signed update that is then passed through the same deterministic constraints. Without this comparison, some of the gains attributed to capability-aligned scoring may come from restricting and regularizing the action space.
- Some empirical conclusions would benefit from more precise uncertainty and budget comparisons. The paper reports repeated runs and variability in several analyses, and Appendix D.1 clarifies the matched search budgets and seed sets. However, the principal TSP table and component-ablation tables mainly report means, making small differences difficult to assess. Some backbone-study quality differences are also small relative to the reported standard deviations. Paired uncertainty estimates or significance tests would strengthen these claims. In addition, reduced runtime relative to AutoEP does not by itself establish an advantage over cheaper non-LLM controllers under equal wall-clock budgets; quality-versus-time comparisons would better characterize when LLM control is worthwhile.
Questions for authors
- What numerical comparability is actually required between the Explorer and Exploiter scores? Can the authors test order-preserving nonlinear recodings and role-specific score compression or offsets, and compare the current coordinator with a sign-only or rank-based controller? These tests would clarify the gap between the ordinal interpretation and magnitude-dependent actuation.
- Can the authors compare score-based control with direct generation of a bounded signed update under the same direction map, step-size bounds, gate, damping, feature access, and decision schedule? This would help distinguish the effect of the scoring interface from that of action constraints.
- How sensitive is HypaEvo to incorrect directional priors or state-dependent reversals in parameter effects? Are there examples where the shared scalar update prevents useful independent changes to different hyperparameters?
- Can the authors provide paired uncertainty estimates for the main quality comparisons and evaluate solution quality under equal wall-clock budgets against AutoEP and inexpensive non-LLM controllers?
On author responses
Dear reviewers,
We have now received the author responses to your raised questions and concerns. Please go through them. Also, have they answered your questions appropriately? Please also look other reviewers' strengths / concerns. Have they changed your perception of the paper? In all cases, please communicate with the authors. It is important that we let them know what relevant issues still exist or have been answered. Please remember we have limited time to get any further responses from them.
Regards,