acceptodds
Under review as a conference paper at ICLR 2027

LinkEvo: Learning When to Recombine Modular Prompts

Abstract

Genetic prompt optimization improves modular language-model systems by evolving instruction bodies while preserving their interfaces, tools, and control flow. However, recombination can disrupt dependencies between modules, and the value of inheriting an instruction block changes with its recipient context and evaluation cost. We introduce LinkEvo, a bounded-population genetic optimizer that measures interactions through controlled block substitutions, builds candidate groups from these measurements, and allocates mutation and crossover effort using context-weighted signed gate gains under an explicit adaptation budget. Across six benchmarks and six executor families, LinkEvo achieved positive observed differences in 34 of 36 settings against uniform genetic-action allocation and 33 against GEPA with merging, with median gains of 1.59 and 0.82 percentage points, respectively, and Qwen3.6 retrieval gains over GEPA with merging of 1.20 points on HotpotQA and 1.33 on HoVer. These results identify a practical role for optimization history in learning which instruction combinations to inherit and when their predicted utility warrants further evaluation, supporting gains across several model families while exposing small mixed differences on reasoning tasks.

Primary area:
optimization
Keywords:
large language models, prompt optimization, evolutionary algorithms, linkage learning, black-box optimization, compound AI systems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Jg6MOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This article proposes LinkEvo, a genetic Prompt optimization method for modular LLM programs. This method evaluates the interaction relationship between different instructions through controlled module replacement, constructs module dependency hierarchy, and dynamically allocates Mutation and Crossover operations based on context weighted historical benefits and computational costs. The experiment covers six benchmark tasks and six model families. The results indicate that LinkEvo outperforms Uniform Genetic Allocation and GEPA+Merge in most settings, achieving performance of 64.40% and 56.27% on HotpotQA and HoVer, respectively.

Critical strengths

The paper models the interaction relationship between modules separately from the actual modification benefits, which are used to guide module grouping and genetic operation selection. This design can avoid simply equating strong dependencies between modules with the effectiveness of joint modifications, making the search process more targeted.

The author not only compared with methods such as MIPROv2, GEPA, and GEPA+Merge, but also analyzed the contributions of each component through ablation experiments such as random dependency hierarchy, removal of contextual information, removal of benefit guidance, and Mutation only. In addition, the frozen state experiment provides further evidence for the effectiveness of the module grouping strategy.

The method showed consistent performance advantages in six tasks and six model families, and detailed statistics were conducted on model invocation, token consumption, and search budget. The main retrieval task also provides results and confidence intervals from five independent runs, enhancing the credibility of the experimental conclusions.

Critical weaknesses

In each controlled replacement comparison, the author uses the same 4 Screen task samples to evaluate the original instruction, two partially replaced versions, and the joint replaced version, and calculates the module interaction value (I) based on this. Due to the limited number of evaluation samples, interaction estimation may be affected by task selection and execution randomness. At present, there is a lack of consistency analysis between these estimates and larger scale independent evaluation results, so it is unclear whether the learned dependency relationships can stably reflect the true functional interactions between modules.

After selecting the best modification scheme from four Screen tasks, the author compared the performance of this scheme with the parent task using eight additional Gate task samples that did not overlap with Screen. The signed profit (z) obtained by Gate is used to determine whether to allow offspring to enter the subsequent population evaluation, and is also saved as historical information to guide future genetic operations. Although this staged evaluation is reasonable, the paper lacks a direct analysis of the relationship between historical predicted returns and subsequent actual returns, making it difficult to determine whether the allocation mechanism has truly learned stable and transferable operational return rules.

The current experiment mainly uses four editable instruction modules, and as the number of modules increases, the potential module interaction relationships and candidate recombination combinations will rapidly grow. Meanwhile, the algorithm has strict matching criteria for historical records, which may result in insufficient valid historical evidence. The paper has not yet verified whether dependency learning and search scheduling can still maintain their advantages in more complex multi module programs.

Questions for authors

The paper evaluates four module replacement combinations using four Screen tasks and calculates the interaction value (I) based on them. Has the author analyzed the stability of this interaction value under different task samples? For example, when recalculating interaction values on a larger independent task set, do their signs and intensities remain consistent with the original estimates?

After selecting the best modification plan in the Screen stage, the paper evaluates its relative benefits (z) to the parent using an additional 8 non overlapping Gate tasks, and uses this benefit for subsequent operation allocation. Can the author further explain the correlation between historical predicted returns and subsequent actual returns? Furthermore, in the actual search process, how many candidate operations can obtain sufficient valid historical support?

The current experiment is mainly based on four editable instruction modules. When the number of modules increases to 8 or 16, the potential interaction relationships and candidate recombination combinations will significantly increase. Has the author evaluated the impact of module quantity on search efficiency, historical sparsity, and final performance? Can a fixed population of four members still maintain sufficient diversity of instructions?

In Appendix B.3, the author reported that the interaction effect between historical records and cache access was approximately 0.07 percentage points on HotpotQA, but based on the data in Table 9 and the provided calculation formula, the result appears to be 0.49 percentage points. Can you clarify this difference and indicate whether different statistical calibers were used?

Reviewer_N18POpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a text optimization method for the prompts in DSPy-style programs via genome-based evolutionary algorithms. On top of the existing mutation algorithms found in works such as GEPA, the proposed method adds some novelties in deciding which instruction/prompt slot to change in the agentic program. This is done by measuring interactions with small tests and allocating budget from past results in a cost-aware setting. Experiments show that under a fixed budget, the methods gain marginally over GEPA and variants on HotpotQA and HoVer.

Critical strengths

The paper studies prompt optimization in agentic programs, which is a valid problem, and proposes a well-engineered algorithm on top of existing baselines and demonstrates some improvements. The experiments are conducted in a cost-aware setup, which is necessary, and the ablations provide further results on the benefits of individual actions in the pipeline.

Critical weaknesses

I do not clearly see what the scientific contribution of this paper is. Admittedly, the overall system is well designed and implemented, yet genome-based mutation algorithms for prompt optimization are well established, and the specific novelty of doing this in agentic programs, considering the interactions among groups, is also found in previous works, such as GOMEA. The four-point interaction is also similarly discussed in subsequent follow-up works of GOMEA. On top of the limited novelty, the empirical benefits are also very small.

Questions for authors

Please see the weaknesses.

Reviewer_ZD9TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents LinkEvo, an evolutionary approach to optimizing instructions in modular language-model programs. Rather than treating modules independently, LinkEvo estimates their interactions to identify groups that benefit from joint optimization. It also adaptively chooses between mutation and crossover based on the outcomes of previous optimization steps. The method is evaluated on HotpotQA and HoVer using Qwen3.6, where it achieves scores of 64.4 and 56.3, outperforming GEPA+Merge (63.2/54.9) and uniform allocation (62.0/52.6) under the same optimization budget. Across 36 benchmark–executor configurations, LinkEvo outperforms uniform allocation in 34 cases and GEPA+Merge in 33.

Critical strengths
  1. The paper has quite a careful experimental evaluation. It makes a good effort to ensure fair comparisons by using the same optimization budget across methods, including the overhead of exploring and evaluating candidates. It also provides ablation studies to examine the contribution of different components and reports negative results on BBEH.

  2. Results are nice, they show consistent improvements across tasks. For example, the paper has shown improvements across a diverse set of benchmarks covering retrieval, coding, and tool use. I also found the example in Appendix G helpful, as it illustrates why optimizing related modules together can be more effective than optimizing them independently.

Critical weaknesses
  1. My main concern is whether the proposed mechanisms offer a substantial contribution beyond existing evolutionary optimization methods.

    • The main ideas have close precedents in evolutionary computation: pairwise interaction estimation (LINC), linkage-based recombination (LTGA/GOMEA), and adaptive operator selection. The related work discusses only one GOMEA paper. How does it differ from prior methods? It is unclear for me upon reading.

    • All evaluated programs have only four modules. Linkage is estimated using roughly 60 comparisons on small, four-task screens. It is unclear whether these estimates are reliable or useful at this scale.

    • LinkEvo improves over GEPA+Merge by only 1.2–1.3 points (roughly four tasks out of 300). The mutation-only variant already outperforms GEPA+Merge, while crossover adds about one point. It's quite surprising that randomizing the linkage hierarchy hurts more than removing crossover entirely. It makes the ablation results a bit more difficult to interpret.

    • I would like to see experiments on programs with substantially more modules, where learning module dependencies should matter more.

  2. in general there're a few questions to improve the consistency of the results

    • Table 1 averages five runs on the same 300 binary-reward tasks, so the reported means should be consistent with increments of 1/1500. But 14 of 26 entries do not appear to match this under the reported rounding. Similar issues seen in Tables 9 and 12, and Appendix B.3 reports an interaction of approximately 0.07 where Table 9 gives 0.49.

    • The comparison reports only performance differences, without absolute scores, run counts, or error bars.

    • Clarifying these discrepancies and providing stronger uncertainty analysis would help to understand how robust the reported gains are.

Questions for authors
  1. Can you give per-run success counts for every row of Table 1 and explain the values that are not multiples of 1/1500 (e.g., were some runs or tasks excluded)? Please also reconcile App. B.3 with Table 9.

  2. For the 36-cell grid, what are the absolute scores, number of runs, test-set sizes, and error bars? The executor ranking against Uniform is identical in five of the six benchmarks, which is surprising given the run-level noise in Table 15, so per-run numbers would help.

  3. How were the constants chosen (decay scale, kernel weights, exploration rate, panel / screen / gate sizes), on which data, and how sensitive are the results to them?

  4. GEPA's Pareto set is drawn from the training split here, and the budget is smaller than in the GEPA paper. MIPROv2, GEPA, and GEPA+Merge end up within 0.15 points of each other on HotpotQA. How do they perform with their native settings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.