- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes WikiEvolve, an LLM-driven self-evolving framework for algorithm improvement that adds a fixed foundational wiki to experience-based evolution. The wiki is a graph of hyperlinked Markdown pages: source cards, synthesized concept pages, and an index node. A curator agent powered by Claude Opus 4.6 builds it offline from a task-specific literature corpus. In each iteration, the agent formulates a research agenda and research questions, retrieves relevant wiki knowledge and past experiences, and proposes local modifications to the current best algorithm. A surrogate model then selects promising candidate algorithms for implementation and evaluation. Experiments cover three DCOP tasks and two BO tasks.
Critical strengths
-
The motivation is reasonable: experiential memory records how well past attempts performed, but carries no information about mechanisms the agent has not yet explored. The paper connects research planning, the structured foundational wiki, experience retrieval, and candidate selection into a single loop. Each candidate algorithm also cites the source cards it builds on, which links foundational knowledge to the experiential memory.
-
The paper specifies the wiki schema and construction pipeline, the retrieval scores, the robust standardization and noise model of the GP surrogate, the candidate selection rule, and the utility normalization. The evaluation spans two quite different domains and two LLMs. It also provides several ablation studies and concrete examples of algorithms.
Critical weaknesses
-
WikiEvolve runs iterations with up to implemented candidates each. The Single-Iteration baselines use only 50 evaluations. Could the authors report, for each method, the number of evaluations, token usage, and time? Ideally, all methods would also be compared under a matched evaluation budget.
-
WikiEvolve greedily updates a single current best algorithm, which is closer to hill climbing than to population-based evolution. Could the authors evaluate an existing evolutionary framework (e.g., OpenEvolve) given access to the same foundational wiki through standard retrieval? Could the authors also compare against other recent open-source evolutionary frameworks?
Typos and grammatical errors: "sugggesting" (Sec. 4.2), "This design help to agent" (Sec. 3.2), "algorithms that specifically works" (Sec. 4.2), "forms a pool the top 16 … that having" (App. B.2), and "and and" (Fig. 8 caption).
Questions for authors
Please see the questions under Weaknesses.
4 comments
Response to Reviewer Hbth (Part 2/3)
W2. WikiEvolve greedily updates a single current best algorithm, which is closer to hill climbing than to population-based evolution. Could the authors evaluate an existing evolutionary framework (e.g., OpenEvolve) given access to the same foundational wiki through standard retrieval? Could the authors also compare against other recent open-source evolutionary frameworks?
W2. Population-based evolution and existing evolutionary frameworks with access to foundational knowledge
We thank the reviewer for this question. We agree that WikiEvolve updates a single current best algorithm greedily, which resembles hill climbing with several candidates per step. However, WikiEvolve explicitly balances exploitation and exploration when it generates and selects candidates. The candidates are proposed from the foundational knowledge retrieved from the wiki, so a candidate can introduce a mechanism from a method that is dissimilar to the current algorithm. On Graph Coloring, for example, the lineage of the best-found algorithm cites DPOP and Bounded Max-Sum, which rank 16 and 11 in similarity to the best-found algorithm (App. I, Fig. 10). Candidate selection then balances exploitation of candidates with higher predicted utility and exploration of candidates with uncertain utility under the surrogate model (Sec. 3.3, App. D).
We provide additional results below comparing the performance of OpenEvolve with and without access to our wiki knowledge base, alongside the results of EoH and ReEvo with similar token budget. And for EoH, ReEvo and OpenEvolve with wiki, we use BM25 to provide 16 documents for the agent to use per iteration, which is the same setting as our method.
| Task | Method | Improvement (%) ↑ | Total token (M) |
|---|---|---|---|
| Graph Coloring | EoH (Given sources) | 8.25 | 11.98 |
| ReEvo (Given sources) | 2.99 | 11.96 | |
| OpenEvolve (Given wiki) | 4.81 | 11.40 | |
| OpenEvolve | 0.00 | 10.10 | |
| WikiEvolve (Ours) | 18.01 | 11.87 | |
| Max-DisCSP | EoH (Given sources) | 22.11 | 14.48 |
| ReEvo (Given sources) | 17.81 | 14.48 | |
| OpenEvolve (Given wiki) | 20.08 | 14.94 | |
| OpenEvolve | 18.68 | 13.68 | |
| WikiEvolve (Ours) | 26.74 | 14.39 | |
| Weighted DCOP | EoH (Given sources) | 1.82 | 5.63 |
| ReEvo (Given sources) | 1.92 | 5.65 | |
| OpenEvolve (Given wiki) | 5.11 | 5.53 | |
| OpenEvolve | 5.06 | 5.07 | |
| WikiEvolve (Ours) | 6.79 | 5.58 |
Coupling WikiEvolve with population-based evolution is a promising direction for future work, and we have added it to Sec. 6.
Response to Reviewer Hbth (Part 3/3)
Typos. "sugggesting" (Sec. 4.2), "This design help to agent" (Sec. 3.2), "algorithms that specifically works" (Sec. 4.2), "forms a pool the top 16 … that having" (App. B.2), and "and and" (Fig. 8 caption).
Typos
We thank the reviewer and have corrected all of them in the revised paper.
We hope that our clarifications address your concerns and improve your opinion of our work. We would be happy to answer any further questions.
Follow-up on Our Response
Dear Reviewer Hbth,
As the discussion period is coming to an end, we would like to follow up on our response. Please let us know if you have any remaining concerns, and we will be glad to address them before the discussion closes.
If our clarifications and additional experiments have resolved your concerns, we would be grateful if you could consider updating your score accordingly.
Thank you once again for your time and constructive feedback.
Best regards,
The Authors
Response to Reviewer Hbth (Part 1/3)
We thank the reviewer for the thorough review, for recognizing our motivation and method design, and for pointing out the typos. In summary, we (i) report the realized tokens and evaluations of all methods under comparable token budgets, and (ii) evaluate OpenEvolve with BM25 retrieval over the same foundational wiki, together with EoH and ReEvo. Additionally, we have revised the PDF version of the paper, with all revisions marked in blue. We apologize that, due to time constraints, we have not yet added the results for the DeepSeek model. We plan to include these results in a subsequent revision.
W1. Evaluations and token usage
We thank the reviewer for the request. We do not report wall-clock time because the experiments were run in parallel on a shared server, where varying CPU/GPU load, resource contention, and process queuing introduce substantial variability. The table below reports, for each method, the improvement, the total tokens, the implemented candidates, and the successful evaluations of a self-evolution run on the three DCOP tasks (App. J, Table 12). In our experiments, we primarily use total token consumption as the criterion for matching computational budgets across methods. We acknowledge that the budgets are not fully matched in the single-iteration setting. To address this concern, we have conducted additional experiments under more comparable budgets, with the results also presented in the table below.
We have added this table to App. J (Table 12), which additionally reports OpenEvolve (Given wiki), EoH (Given sources), and ReEvo (Given sources). We also report the token efficiency analysis (Section 4.6, Figure 3). These comparisons show that WikiEvolve achieves higher utility improvement under comparable total-token budgets on all three DCOP tasks (App. J, Table 12), including against the Single-Iteration baselines run with comparable token budgets. Its advantage therefore does not simply result from spending more tokens. The trajectories in Sec. 4.6 (Fig. 3) show substantial improvements with relatively few tokens on Graph Coloring and Weighted DCOP, and continued improvement as more tokens are consumed on Max-DisCSP. WikiEvolve also outperforms the baselines given access to foundational knowledge, showing that access alone does not account for its gains. Together, these results support the effectiveness of the structured wiki, research planning, and evaluation feedback in achieving greater utility improvement within comparable token budgets.