- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
Meta-Delta augments LLM-driven evolutionary optimization with a revisable knowledge bank. Structured Delta Cards record search lessons, applicability conditions, and supporting evidence. Routine updates incorporate individual experiments, while Global Review uses three complementary perspectives to reconsider accumulated guidance and propose subsequent directions. Evaluation spans 11 optimization tasks and two scientific-law discovery settings. The framework is well motivated and includes informative case studies. However, the current evidence leaves substantial uncertainty about computational fairness, robustness of the reported gains, and the specific contribution of knowledge revision.
Critical strengths
- Careful evidence management. The distinction between supported and exploratory Cards, explicit evidence provenance, and separation of implementation failures from strategy failures are sensible design choices.
- Broad empirical coverage. The evaluation spans mathematical, scientific, and systems optimization, with promising results on tasks such as Signal Processing and EPLB.
- Informative qualitative analysis. The case studies trace Card revisions to subsequent plans and candidate changes, include an unsuccessful hypothesis, and acknowledge attribution limits.
- Useful controlled evidence. The same-checkpoint continuation study provides additional support beyond selected trajectories, although its scope is limited.
Critical weaknesses
- Computational fairness and mechanism attribution remain unresolved. Methods are compared using equal outer-round budgets, while Meta-Delta can perform multiple candidate evaluations per round and additional Global Review calls. The released configuration explicitly treats outer iterations as the budget unit. Moreover, disabling Global Review simultaneously removes additional computation, broader historical context, exploratory suggestions, and joint Bank revision. This ablation therefore does not isolate the contribution of persistent knowledge revision.
- The strongest scientific-benchmark comparisons lack uncertainty estimates. In Table 1, the numeric-fit advantage over DeltaEvolve is 49.30 versus 49.12, and the structure-score advantage over the strongest external baselines is 48.02 versus 47.53. Relative to w/o GR, the structure-score improvement is only 0.21 percentage points, while scientific validity remains tied at 84.24. Without per-task differences and uncertainty estimates across seeds, the robustness of these margins is unclear. For the 11-task suite, best@3 reporting similarly leaves average performance and variability unresolved.
- The MMD results do not support the stated Global Review attribution. Section 5.2 counts earlier plateau attainment on both MMD tasks as a benefit of enabling Global Review. However, Full reaches both plateaus at round 1, whereas Algorithm 1 places Global Review after that round’s candidate generation and evaluation. Under the described procedure, these first-round outcomes cannot result from preceding Global Review. The timing or interpretation requires clarification.
- The incremental contribution needs a closer mechanism-level comparison. The paper appropriately discusses evolving experience libraries and applicability refinement, including ACE and AlphaOPT. A budget-matched generic global reflection or single-manager global update would help establish the added value of the proposed review structure. The present comparisons leave its contribution relative to simpler alternatives uncertain.
Questions for authors
- Under matched token or candidate-evaluation budgets, how does Meta-Delta compare with w/o GR and a simpler global-reflection control? Please report actual resource usage.
- How can Global Review explain the round-1 MMD results? Is there an initialization-stage review omitted from Algorithm 1?
- Can the authors clarify the actual stopping/trigger settings and final candidate-selection protocol, including how these correspond to Algorithm 1?