- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This article proposes LinkEvo, a genetic Prompt optimization method for modular LLM programs. This method evaluates the interaction relationship between different instructions through controlled module replacement, constructs module dependency hierarchy, and dynamically allocates Mutation and Crossover operations based on context weighted historical benefits and computational costs. The experiment covers six benchmark tasks and six model families. The results indicate that LinkEvo outperforms Uniform Genetic Allocation and GEPA+Merge in most settings, achieving performance of 64.40% and 56.27% on HotpotQA and HoVer, respectively.
Critical strengths
The paper models the interaction relationship between modules separately from the actual modification benefits, which are used to guide module grouping and genetic operation selection. This design can avoid simply equating strong dependencies between modules with the effectiveness of joint modifications, making the search process more targeted.
The author not only compared with methods such as MIPROv2, GEPA, and GEPA+Merge, but also analyzed the contributions of each component through ablation experiments such as random dependency hierarchy, removal of contextual information, removal of benefit guidance, and Mutation only. In addition, the frozen state experiment provides further evidence for the effectiveness of the module grouping strategy.
The method showed consistent performance advantages in six tasks and six model families, and detailed statistics were conducted on model invocation, token consumption, and search budget. The main retrieval task also provides results and confidence intervals from five independent runs, enhancing the credibility of the experimental conclusions.
Critical weaknesses
In each controlled replacement comparison, the author uses the same 4 Screen task samples to evaluate the original instruction, two partially replaced versions, and the joint replaced version, and calculates the module interaction value (I) based on this. Due to the limited number of evaluation samples, interaction estimation may be affected by task selection and execution randomness. At present, there is a lack of consistency analysis between these estimates and larger scale independent evaluation results, so it is unclear whether the learned dependency relationships can stably reflect the true functional interactions between modules.
After selecting the best modification scheme from four Screen tasks, the author compared the performance of this scheme with the parent task using eight additional Gate task samples that did not overlap with Screen. The signed profit (z) obtained by Gate is used to determine whether to allow offspring to enter the subsequent population evaluation, and is also saved as historical information to guide future genetic operations. Although this staged evaluation is reasonable, the paper lacks a direct analysis of the relationship between historical predicted returns and subsequent actual returns, making it difficult to determine whether the allocation mechanism has truly learned stable and transferable operational return rules.
The current experiment mainly uses four editable instruction modules, and as the number of modules increases, the potential module interaction relationships and candidate recombination combinations will rapidly grow. Meanwhile, the algorithm has strict matching criteria for historical records, which may result in insufficient valid historical evidence. The paper has not yet verified whether dependency learning and search scheduling can still maintain their advantages in more complex multi module programs.
Questions for authors
The paper evaluates four module replacement combinations using four Screen tasks and calculates the interaction value (I) based on them. Has the author analyzed the stability of this interaction value under different task samples? For example, when recalculating interaction values on a larger independent task set, do their signs and intensities remain consistent with the original estimates?
After selecting the best modification plan in the Screen stage, the paper evaluates its relative benefits (z) to the parent using an additional 8 non overlapping Gate tasks, and uses this benefit for subsequent operation allocation. Can the author further explain the correlation between historical predicted returns and subsequent actual returns? Furthermore, in the actual search process, how many candidate operations can obtain sufficient valid historical support?
The current experiment is mainly based on four editable instruction modules. When the number of modules increases to 8 or 16, the potential interaction relationships and candidate recombination combinations will significantly increase. Has the author evaluated the impact of module quantity on search efficiency, historical sparsity, and final performance? Can a fixed population of four members still maintain sufficient diversity of instructions?
In Appendix B.3, the author reported that the interaction effect between historical records and cache access was approximately 0.07 percentage points on HotpotQA, but based on the data in Table 9 and the provided calculation formula, the result appears to be 0.49 percentage points. Can you clarify this difference and indicate whether different statistical calibers were used?