- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates whether LLM evolutionary search, supported by human expert input and an exact evaluator, can find improved solutions to a specific mathematical problem: constructing binary linear codes with high minimum distance. The proposed framework combines LLM-generated search programs, established construction strategies, and verified evaluation feedback. It discovers seven codes that improve previously known lower bounds, yielding improvements to 22 table entries through standard code modifications. The paper also compares the framework with directly prompting a strong LLM on two target parameter pairs.
Critical strengths
New constructions that improve existing records. The strongest contribution is the discovery of seven codes that improve previously known minimum-distance bounds, with additional improvements obtained through standard modifications. These are concrete mathematical results whose correctness can be checked independently. The reported exact verification and independent checking by the CodeTables maintainer strengthen confidence in the contribution.
Critical weaknesses
-
The contribution of the framework’s individual components is unclear. The method includes multiple agent roles, specialized search islands, a large hand-written strategy toolkit, construction-based initialization, secondary fitness criteria, and human intervention. Without appropriate ablations or comparisons against simpler alternatives, it is difficult to determine which components are responsible for the improvements. In particular, the experiments do not separate the benefit of the proposed LLM orchestration from that of expert-selected targets, strong starting constructions, and existing search strategies. This limits the methodological insight provided by the paper.
-
The comparison with direct prompting is insufficient to establish an advantage. The comparison covers only two target parameter pairs. The framework improves minimum distance from 65 to 66 on one target and matches distance 77 on the other, while obtaining fewer minimum-weight codewords. These results are encouraging, but provide limited evidence of a consistent advantage over direct prompting. Moreover, the baseline protocol is insufficiently specified: it is unclear whether direct prompting received comparable mathematical hints, initial constructions, tool access, human assistance, or computational budgets. Consequently, the observed differences cannot confidently be attributed to the proposed search framework.
Questions for authors
-
Clarification: What exactly was the direct-prompting protocol? Please specify the prompts, mathematical hints, starting constructions, available tools, and human feedback, and explain how these compare with the assistance provided to LinCodeEvolve.
-
Clarification: What were the budgets and stopping criteria for each approach, including model calls, computational evaluation, and repeated attempts? Were the reported direct-prompting results the best of multiple attempts, and are repeated-run results available for either approach?
-
Additional evidence: Can the authors provide ablations or controlled comparisons that isolate the benefit of the proposed framework beyond expert initialization and the strategy toolkit? Particularly informative comparisons would include toolkit-only search and the underlying evolutionary framework without the added agent roles and specialized islands.
Dear Reviewer,
Thank you very much for your insightful feedback. On behalf of all the authors, we truly appreciate the time and effort you dedicated to evaluating our work. Your comments have pinpointed important questions to address. In order to address the major revisions, we have decided to withdraw our paper. We are grateful for your guidance, which will strengthen the paper's final version.