- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Uncertainty Calibrated Evolution (UCE), a stage-aware scoring paradigm that calibrates candidate evaluations according to the collective performance within each fixed-parent stage. Specifically, UCE contextualizes each candidate’s score using the mean and dispersion of its stage-level candidate pool, transforming an isolated evaluation into a relative signal for subsequent selection and feedback. This design reduces the disproportionate influence of low or otherwise atypical scores on the LLM’s iterative decisions, without modifying the program generator or requiring additional model training.
Critical strengths
-
The proposed two challenges are quite clear and well-presented.
-
This paper is relatively well-written.
-
The results in Table 3 are promising; the performance demonstrated on frontier models highlights the scaling ability of this approach.
Critical weaknesses
- As the greatest weakness, this paper is merely the same as a paper produced in 2024 [1]. Both Qube and UCE introduce uncertainty to judge the score. The author fails to compare against Qube or even discuss its existence, which is unacceptable. I require the author to provide clear comparative experiments and a discussion of related work before I will consider a higher score.
[1] Chen, Zijie, et al. "Qube: enhancing automatic heuristic design via quality-uncertainty balanced evolution." arXiv preprint arXiv:2412.20694 (2024).
- This paper exhibits several characteristics that fall short of standard academic research criteria: figures are provided as screenshots rather than vector graphics; experimental parameters are often vaguely defined; and instead of employing conventional constructive or ACO (Ant Colony Optimization) frameworks, the authors have opted for the OR-Tools suite.
Questions for authors
-
Please provide a clearer description of the experimental setup for the main experiment.
-
Which large language model (LLM) was used in the main experiment?
-
Were the results for the main experiment obtained by averaging across multiple runs?