acceptodds
Under review as a conference paper at ICLR 2027

Researcher-in-the-loop evolutionary discovery of entanglement-assisted quantum code families

Abstract

LLM-guided program evolution can produce mathematical constructions, but validity, generality, and reproducibility require distinct evidence. We develop a researcher-in-the-loop discovery pipeline that combines program evolution, exact evaluation, mathematical generalization, certified nonexistence checks, and formal proof checking. We apply it to entanglement-assisted quantum error-correcting codes (EAQECCs). Starting from tasks that did not specify a family formula, evolved programs produced finite qubit constructions whose structure researchers generalized to the family for prime powers at and for at odd . Lean checks the algebraic certificate underlying this family. Together with entanglement lifting, the family closes 54 gaps in archived EAQECC tables. The released evidence includes 115 verified code records, ten nonexistence results with checked DRAT proofs, family implementations, and preserved search programs for exact checks and seeded replay. In a controlled cohort of eight paired generation blocks, validation-selected iterative programs achieve success in 46.1% and 31.6% of held-out executions at unseen lengths , compared with 19.5% and 3.1% for independently generated programs. In this setting, program evolution and researcher analysis turn finite computational discoveries into a proved family with independently checkable evidence.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
program evolution, human-in-the-loop, entanglement-assisted quantum error correcting codes, formal verification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UyTLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a researcher-in-the-loop framework combining LLM-based program evolution with exact evaluation and mathematical generalization to discover entanglement-assisted quantum error-correcting codes (EAQECCs). Starting from finite qubit constructions generated by evolved programs, researchers generalize the results to a general code family, with key algebraic properties formalised and checked. The overall approach closes 54 open gaps in standard EAQECC reference tables and includes DRAT-certified nonexistence proofs for selected parameter bounds. Controlled generation experiments demonstrate that iterative feedback improves program success rates on longer, unseen code lengths.

Critical strengths
  1. The mathematical generalization from finite computational discoveries into an infinite code family is a solid contribution that closes 54 listed table gaps and is formally verified in Lean. 2. The empirical ablation setup is rigorous, using controlled generation blocks and held-out code lengths to cleanly demonstrate the benefit of iterative feedback in program evolution.
Critical weaknesses
  1. The limitations of related work in quantum code construction and automated program evolution are not sufficiently detailed, making it hard to see where existing frameworks fail. 2. The paper lacks a detailed description clearly distinguishing novel search components from pre-existing AlphaEvolve components. 3. The specific features of quantum code discovery that make general LLM search algorithms inadequate without researcher intervention are not clearly articulated. 4. Experimental evaluations are mostly restricted to small binary parameters and specific distance targets, lacking evaluation on broader benchmark suites or larger field sizes.
Questions for authors
  1. Section 2.2: What specific domain characteristics of EAQECC matrix search make existing program evolution tools like FunSearch or AlphaEvolve unsuitable without researcher intervention? Please clarify what fails when running these out-of-the-box. 2. Section 3.2: Could you provide a clear top-level algorithmic description (or detailed flowchart) of the complete pipeline that explicitly separates what is built on AlphaEvolve versus what is custom-designed for quantum codes? 3. Section 3.4 & Reproducibility Statement: The setup relies on proprietary components like Vertex AI and Gemini models. How can an independent researcher without Vertex AI access reproduce the search runs and tool flow? 4. Section 4.1 & Tables 1-2: The experiments primarily focus on k=1 and low field sizes q. How does the search pipeline perform on higher dimensions (k > 2, q > 5) or more challenging code parameters? 5. Section 5.1 & Figure 4: Results are presented quite concisely. Could you add explicit lists of observations discussing why certain blocks show minimal improvement under iterative evolution compared to independent proposals?
2 comments
AuthorsOpenReview ↗

Response to Reviewer UyTL

Question 1. Section 2.2: What specific domain characteristics of EAQECC matrix search make existing program evolution tools like FunSearch or AlphaEvolve unsuitable without researcher intervention? Please clarify what fails when running these out-of-the-box.

AlphaEvolve requires a program, evaluator and specification. We tested EAQECC formulations, not an engine baseline. A target couples generator/Gram ranks for k,ck,c with minimum non-radical dual weight; improving distance can violate rank. For our binary targets, dim⁡S=2n−4\dim S=2n-4 and dim⁡S⊥=4\dim S^\perp=4. Dualization changes representation and moves, not the feasible code set. The initial program solves 1/64 at n=9,11n=9,11. Broad dual-space advice raises independent/iterative area by +0.316/+0.346; screening then independent generation reaches 0.314 (Figure 7, §5.2). Cruz-Benito et al. also evolve quantum-code programs and prove structural results; we target EAQECC families (§2.2).

Question 2. Section 3.2: Could you provide a clear top-level algorithmic description (or detailed flowchart) of the complete pipeline that explicitly separates what is built on AlphaEvolve versus what is custom-designed for quantum codes?

Figure 1 separates our targets, specification, initial program and evaluator from AlphaEvolve’s unmodified generation, scoring, database and parent selection. Researchers and Fable 5 analyze codes and develop checked proofs; SAT refines bounds. Magma checks parameters, DRAT-trim exclusions and Lean the algebraic core. Controlled comparisons use our Python controller with Gemini 3.5 Flash (§3).

Question 3. Section 3.4 & Reproducibility Statement: The setup relies on proprietary components like Vertex AI and Gemini models. How can an independent researcher without Vertex AI access reproduce the search runs and tool flow?

Install requirements.txt; in the released repository, run:

python3 scripts/reproduce_eaqecc.py --tier deterministic
python3 scripts/eaqecc_ablation/audit_replication.py
python3 scripts/reproduce_eaqecc.py --claim search-determinism

These check constructions, 54 closures and study records, and replay twelve archived search lists without model access. Controlled replay needs the recorded macOS runtime. The artifact guides cover external-tool setup. Fresh generation uses Vertex/AlphaEvolve; no other backend is implemented. Please report replay issues (§3.4).

Question 4. Section 4.1 & Tables 1-2: The experiments primarily focus on k=1 and low field sizes q. How does the search pipeline perform on higher dimensions (k > 2, q > 5) or more challenging code parameters?

Theorem 1 covers all prime powers q≥3q\ge3 at n≥2q+1n\ge2q+1 and binary odd n≥5n\ge5. Eight exactly enumerated q=7q=7 instances include [[15,1,14;7]]7[[15,1,14;7]]_7. Verified examples include [[13,6,6;6]]2[[13,6,6;6]]_2, [[10,1,9;4]]4[[10,1,9;4]]_4 and [[12,1,11;5]]5[[12,1,11;5]]_5 (§4.2, manuscript Table 2, Appendix A.2). These verify constructions; policy comparisons fix k=1,q=2k=1,q=2. Exact evaluation enumerates qn+k−cq^{n+k-c} dual vectors; dual parametrization does not remove this cost (Appendix A.1).

Question 5. Section 5.1 & Figure 4: Results are presented quite concisely. Could you add explicit lists of observations discussing why certain blocks show minimal improvement under iterative evolution compared to independent proposals?

A block pairs generation runs: seven candidates per policy, with training/validation selection. Broad dual-space advice lets both policies implement dual search, with varying selected programs. In original replication A, run 3, iteration solves 35/64 versus 24/64 but area is 0.217 versus 0.218 because successes occur later. Policy differences are +0.099 in A (5/8 positive, p=0.33p=0.33) and +0.079 in repeat B (4/8, p=0.53p=0.53), neither established. Submitted Figure 4 is now Figure 6; Figure 9 shows the timing example.

Table 1 (this response). Ten-run study: behaviors among 70 candidates; area difference = iterative − independent.

AdviceIndependent behaviorsIterative behaviorsArea difference
None7067−0.067
Representation only5464−0.024
Objective2159−0.001
Search policy222+0.203

Behaviors differ in recorded training score, call count, distance ratio, matrix or error. Counts are post-selection descriptions. Under search-policy advice, 69/70 independent candidates repeat one behavior; nine iterative selections add distance terms and eight change counter resets. Its area gain is positive in 10/10 runs (98.75% interval [+0.103,+0.287]); no-advice/representation differences are unestablished and objective results are near the floor. Longer-code pooled success rises from 13.4%/0.9% to 62.5%/33.4% at n=13,15n=13,15 (ten programs × 32 seeds per length). The cost ratio is 1.31 without a matched-cost control. See revised Figure 4 and Appendix C.9 for results and assumptions.

I would like to thank the authors for the responses. In light of the rebuttal and other reviews, I keep my score.

Reviewer_MvtZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This manuscript presents a researcher-in-the-loop (RITL) framework and uses LLMs to guide the discovery of algorithms; focusing on entanglement-assisted quantum error correcting codes (EAQECC) tables. The main findings of their work are centered around their provable Theorem 1, compared to previous results, which usually test instances computationally. This finding distinguishes this work from other recent approaches of being verifiable rather than testable. Furthermore, they present previously 54 unsolved/unresolved EAQECC tables, which are significant advancements by themselves. The Authors also present a substantial Appendix, with formal LEAN proofs and a detailed methodology. Generally, the manuscript is well structured and illustrated with Figures, Tables, and Algorithms. While the RITL idea nor the LLM guided program evolution are novel by themselves, the experimental showcase that mathematical proofs can be obtained for EAQECC tables are novel.

Critical strengths

Strength 1: Theorem 1 and its proof (and how the pipeline discovered Theorem 1) are the main strengths of this work.

Strength 2: The Authors present formal algebraic and LEAN proofs for their proposed mathematical results.

Critical weaknesses

Weakness 1: The methodology does not provide sufficient details of the human RITL and only broadly covers the researchers contributions: how many researchers were used in the study (single, multipole, coordinated or single-acting); and what was the decision process regarding the researchers; when did the researchers need to intervene between the campaigns; how did the generalization and the proving part happen from the researchers' perspective? Even after several careful readings, this Reviewer failed to envision the main takeaway from the manuscript: is it a methodological work focusing on RITL + LLM guided program discovery?; or does it focus on the new mathematical results? This discrepancy makes the paper difficult to assess and navigate when it comes to comparison to parallel works in the field. From a methodological point of view, neither LLM guided program discovery nor the RITL ideas can be claimed to be novel.

Weakness 2: The presented results (Tables 1 and 2) show insufficient information to conclude the contribution of RITL. If the Authors' main claim are related to the RITL pipeline, there is not enough convincing evidence that RITL+LLM guidance works better than just pure LLM guided program evolutions. For example, there is no clear evidence, that the presented RITL pipeline resulted in the formulation of Theorem 1. For example, how did the researcher recognize the pattern? Furthermore, prior works (such as Benito et al. https://arxiv.org/html/2606.02418v1) showed that LLM-guided program evolution can work on unassisted code. Therefore, this Reviewer is not yet convinced that the presented RITL pipeline adds enough novelty that justifies a strong acceptance.

These two collections of weaknesses are of major concern. However, given the fact the methodology resulted in quantifiable improvements in the EAQECC tables, the presented manuscript might be considered for publication, given the authors address the above weaknesses and the questions below.

Questions for authors
  1. How many researchers were used in this work?
  2. What level of education do these human researchers have?
  3. What level of uncertainty do different researchers introduce to the work (if any)?
  4. What is the decision criteria of the researchers?
  5. What do researchers focus on when they need to intervene?
  6. How patterns are recognized by the researchers? Were there suggested a priori by the LLM?
  7. What is the clear advantage of using RITL over simple LLM-based program discovery? State precisely what is the advantage and quantify this advantage with measured differences across the recent work by others.
  8. How does this work compare to OpenEvolve and the work by Benito et al?
1 comment
AuthorsOpenReview ↗

Response to Reviewer MvtZ

We present an EAQECC discovery pipeline with verified results: Theorem 1's family closes 54 gaps with entanglement lifting, alongside ten DRAT-checked exclusions. We combine established program evolution with EAQECC-specific inputs, mathematical analysis and independent checks (§§1–4, Figure 1).

Question 1. How many researchers were used in this work?

Three researchers formed one team. Two proposed ideas, designed campaigns and reviewed programs and proof drafts; a senior professor checked the construction and proof.

Question 2. What level of education do these human researchers have?

One holds a mathematics master's degree and is an AI doctoral student; the second holds a mathematics doctorate and is a postdoc. The third is a senior mathematics professor with over 100 related publications, on our team.

Question 3. What level of uncertainty do different researchers introduce to the work (if any)?

One team participated. Repetitions measure program-generation variation under fixed advice, not between researchers. Proofs, Magma, DRAT-trim and Lean's algebraic checks separately assess mathematical validity.

Question 4. What is the decision criteria of the researchers?

We intervened based on unresolved bounds, verified-code patterns and stalled fixed-budget searches. Codes required agreement between exact evaluation and independent recomputation. Generalizations required proof review. We fixed selection in advance for the controlled study.

Question 5. What do researchers focus on when they need to intervene?

We focused on targets, evaluation, representation and generalization. We set 68 targets, requested structural analysis and reused the checked family in 329-target specifications. For controlled generation, the AI doctoral student discussed dual-space search with an OpenAI model and the mathematics postdoc. We evaluated a fixed program before generation (Figure 1, Appendix B.3).

Question 6. How patterns are recognized by the researchers? Were there suggested a priori by the LLM?

Fable 5 first noted that the verified n=7,9,11n=7,9,11 codes follow [[2m+1,1,2m;2m−2]]2[[2m+1,1,2m;2m-2]]_2 during our researcher-directed analysis on 13 July. At our request it analyzed the three matrices and drafted the construction and proof; an exact program checked admissible odd lengths through 63. It proposed the prime-power extension on 23 July. We refined and checked the drafts, the professor reviewed the mathematics, and Lean checked the algebraic statements. Initial campaigns supplied cyclic-shift suggestions but no family formula; the formula entered campaign 3. This records discovery history, not a causal experiment. The proof is in Appendix A.2; Figure 2 concerns later program evolution.

Question 7. What is the clear advantage of using RITL over simple LLM-based program discovery? State precisely what is the advantage and quantify this advantage with measured differences across the recent work by others.

We isolated one advice paragraph while fixing model, seeds, budgets and selection. In eight paired runs, broad dual-space advice raises area at n=9,11n=9,11 by +0.316 under independent generation and +0.346 under iterative evolution (8/8 and 7/8 positive, p=0.008,0.016p=0.008,0.016). Area rewards earlier success and is zero for a miss. Independent generation after screening one shared pool of 32 initial programs reaches area 0.314 without that advice. Its longer-code success is 14.8%, versus 25.4% with advice under independent generation; costs differ (Figures 7–10, Appendices C.6–C.8).

The ten-run study prespecified supporting advice effects: the mean over both policies minus the no-advice mean. Its representation-only advice is narrower.

Table 1 (this response). Supporting proposal effects at n=9,11n=9,11.

AdviceMean effect98.33% interval
Representation only+0.194[+0.086,+0.303]
Objective−0.036[−0.091,+0.004]
Search policy+0.196[+0.129,+0.277]

The objective interval includes zero. The search-policy contrast is +0.203 (10/10 positive; model-cost ratio 1.31; revised Figure 4, Appendix C.9). These compare fixed advice and policies, not researcher presence: all conditions retain researcher-designed targets, evaluator and selection. Tasks and budgets differ across papers, preventing a numerical framework comparison.

Question 8. How does this work compare to OpenEvolve and the work by Benito et al?

OpenEvolve is an open program-evolution framework. We use AlphaEvolve for discovery and a local controller for controlled studies. Cruz-Benito et al. study bivariate-bicycle codes, use staged distance estimates with exact MILP-certified cases, and prove structural results. Our distinct results are EAQECC families for admissible lengths/fields, exact reported distances, certified nonexistence and Lean-checked algebraic statements. See §2.2.

Reviewer_G4yvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper concerns the application of program evolution techniques (in the style of AlphaEvolve) supported by formal verification tools to the specific problem of entanglement-assisted quantum error-correcting codes. Guided by a researcher, this leads to the discovery of a family of improved entanglement-assisted quantum error-correcting codes.

Critical strengths
  • Discovery and generalisation of a family of entanglement-assisted quantum error-correcting codes seems to be a convincing case study of the potential of human--AI research pipelines

  • Lean formalisations are used to verify the algebraic core

  • The well-documented anonymised code repository boosts reproducibility

Critical weaknesses
  • There is little by way of general guidance on when and how researcher interventions lead to successful discoveries.

  • Despite some useful analysis in the appendices, the main paper provides limited insight into why the evolved search programs work, or justification for the many numerical constants and probabilities (i.e. there appears to be a lot of magic numbers).

  • Evidence for the general benefit of iterative evolution seems limited.

Questions for authors

Clarifications:

  • What specific interventions by the researcher were essential to the main discovery presented in the paper? How can we separate and understand the researchers' contributions to the discovery from those of the automated program evolution?

  • Beyond the distance-shortfall penalty analysed in Appendix C.5, which evolved changes accounted for the gains?

  • Given the small number of trials and unequal token costs in your experiments, how confident can we be that iterative evolution works reliably better than independent generation?

3 comments
AuthorsOpenReview ↗

Response to Reviewer G4yv

Question 1. What specific interventions by the researcher were essential to the main discovery presented in the paper? How can we separate and understand the researchers' contributions to the discovery from those of the automated program evolution?

We selected 68 unresolved targets, wrote the initial program and exact evaluator, and suggested directions including cyclic shifts without a family formula. Evolved programs returned finite codes. Fable 5 noted the n=7,9,11n=7,9,11 pattern and, at our request, analyzed the matrices and drafted the construction and proof. We revised and checked both; the senior professor reviewed the mathematics, and Lean checked its algebraic core. We then reused the family in campaigns 3–4.

For controlled generation, we developed dual-space advice with an OpenAI model and the mathematics postdoc, then calibrated a fixed program. Removing broad dual-space advice lowers area from 0.369 to 0.053 under independent generation and from 0.448 to 0.102 under iterative evolution (eight paired runs, p=0.008,0.016p=0.008,0.016). Independent generation after screening 32 initial candidates reaches area 0.314. We measure a supplied proposal, not human necessity. Our procedure calibrates changes, fixes the comparison and checks generalizations before reuse (Figure 1; Appendices B.3, C.6, C.8).

Question 2. Beyond the distance-shortfall penalty analysed in Appendix C.5, which evolved changes accounted for the gains?

Algorithm 1 retains weights 5000/1000 for entanglement/dimension mismatch and counts nontrivial logical operators below target distance. It evolves target weighting, stored-candidate reuse (probability 0.2) and row rotations. The 1500 limit counts attempts without strict improvement. These are search settings (§4, Figure 2).

Algorithm 2 has a separate lineage: parent, child and nine variants were replayed on 16 new seeds per length at n=13,15n=13,15 (352 executions). Restoring parent objective weights lowers the child's successes from 15/16 to 10/16 at n=13n=13; disabling best-state return gives 11/16. Restoring parent mutations or annealing instead improves results to 16/16 and 14/16 at the two lengths, versus the child's 15/16 and 7/16. Thus not every edit helps. Perturbing constants by 20% gives 5–8/16 successes at n=15n=15; we do not claim optimal values. See manuscript Table 10, Appendix C.7.

Question 3. Given the small number of trials and unequal token costs in your experiments, how confident can we be that iterative evolution works reliably better than independent generation?

After model-free calibration, we froze a Gemini 3.5 Flash study with four advice conditions, ten paired runs and seven candidates per policy (560 calls). Independent generation receives the fixed initial program/feedback; iteration receives the best and latest candidates/feedback. Both select by training and validation. Test seeds are shared across runs, so inference is conditional on those seeds. Area averages 1−h/B1-h/B for success at evaluation hh within budget BB, zero for a miss.

Table 1 (this response). Area at n=9,11; difference = iterative − independent.

AdviceIndependentIterativeDifference
None0.0850.018−0.067
Representation only0.2570.233−0.024
Objective0.0160.015−0.001
Search policy0.1460.349+0.203

Search-policy advice specifies generator replacement/multiplication by 1–3-qubit Paulis and restart limit 1000. Its gain is positive in 10/10 runs (98.75% interval [+0.103,+0.287], skewness −0.58). The interaction is +0.270 (10/10, 98.33% interval [+0.122,+0.437], Holm p=0.006p=0.006, skewness 0.58). Sign-flip inference assumes independent symmetric differences and a shift model for intervals. The interaction includes 0.067 from the unestablished no-advice estimate. The first two policy intervals include zero; objective results are near the floor.

At excluded n=13,15n=13,15, area averaged across the two lengths favors iteration in all ten runs; combined success counts favor it in nine, with one tie. Pooled rates are 13.4%→62.5% and 0.9%→33.4% (320 executions per length/policy, not independent policy trials). Search budgets are 10,000 evaluations per target and 30,000 at longer lengths. See revised Figure 4 and Appendix C.9.

Iteration costs 1.31 times as much in this condition (input/output/reasoning tokens 1.44/1.00/1.57); no cost-matched arm was run. Earlier broad-advice differences were +0.099 and +0.079 (5/8 and 4/8 positive, p=0.33,0.53p=0.33,0.53); its input-token-matched 12-candidate control gave +0.105 (p=0.45p=0.45). Those controls do not match costs for the new proposal. Source inspection finds repeated training outputs in 69/70 independent candidates and added distance terms/counter changes in 9/10 and 8/10 iterative selections. These are post-selection observations, not causal tests. The gain is proposal-specific.

Thank you for the additional experiments and clarifications. These help address my questions, particularly by showing which program changes help and which do not.

The new results suggest that the benefit of iterative evolution depends strongly on the advice supplied. Please explain this finding clearly in plain language in the revised paper, alongside the additional token cost and absence of a cost-matched comparison.

Please also clarify the discovery narrative to reflect the respective roles of program evolution, Fable 5 and the researchers in identifying the family and developing its proof.

I remain positive about the verified mathematical contribution. My main remaining concern is that the reader must work too hard to extract the methodological lessons from the experimental detail.

Thank you for the follow-up. We have revised the paper to explain the controlled findings and discovery process more directly. Results, figures, tables and proofs are unchanged. In the revised PDF, wording changed since the previous revision is dark green, while earlier changes remain blue.

Controlled findings and cost (Sections 1 and 5, Appendix C.9). Section 5 defines normalized success-curve area before presenting the results. Each execution scores 1−h/B1-h/B if it solves the target at exact evaluation hh within a budget of BB evaluations, and zero otherwise. We average these scores. In ten paired runs, iterative evolution yields a clear gain under the search-policy (mutation-and-restart) proposal. No difference is established in either direction without a proposal or with representation-only advice. Under objective advice, both policies score near zero, making that contrast uninformative. Section 5.1 reports the observed program changes separately from their interpretation. Nine iterative selections add an unrequested distance term, and eight change the restart-counter rule. These observations do not isolate the individual effects of those edits. Iterative evolution uses 1.44, 1.00 and 1.57 times the input, output and reasoning tokens, respectively, giving an estimated model cost of 1.31 times that of independent generation. Candidate counts are equal, but no matched-cost comparison was run. Sections 1 and 6 and Appendix C.9 retain that qualification.

Evaluation and discovery roles (Sections 1 and 3.2). Section 3.2 distinguishes the oracle's exact parameter computation, the search program's internal objective, and the outer score used to select parent programs. Section 1 describes the discovery sequence. Specifications suggested directions including cyclic shifts but supplied no family formula. AlphaEvolve produced finite qubit codes. Researcher-directed analysis with Claude Fable 5 led to the qubit family and the prime-power construction and proof drafts for Theorem 1. We revised and checked those drafts, and Lean verified the formalized algebraic core. Section 3 and Appendix B.3 detail the team's review roles.

Methodological lesson (Section 6). The conclusion now explains why the proposal's effect and the additional benefit of iterative feedback should be evaluated separately. Representation-only advice does not establish a policy difference, while mutation-and-restart advice supports an iterative gain in all ten paired runs and higher success at longer lengths excluded from generation and selection. We therefore compare generation policies within each proposal condition. Program inspection helps describe the resulting edits, while their individual causal effects and the absence of a matched-cost comparison remain explicit.

Please let us know if any point remains unclear.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.