acceptodds
Withdrawn from ICLR 2027

Agentic Program Evolution of Executable Numerical Operators for PDE Dynamics

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Time-dependent partial differential equation (PDE) simulations support forecasting and design in fluid dynamics, pattern formation, wave propagation, and many other physical systems. Although the governing equations may be known, computational budgets require them to be realized on finite grids and time steps. Accuracy and stability therefore depend on expert choices of derivative approximation, nonlinear discretization, filtering, operator splitting, time integration, and numerical constants, especially when relevant scales are unresolved. We introduce APEX (Agentic Program Evolution of eXecutable numerical operators), a framework that treats these coupled choices as a typed executable program and improves them through evaluator-guided evolution. A deterministic compiler rejects candidates that violate type or resource constraints, while a frozen evaluator supplies structured diagnostics of accuracy, stability, spectral behavior, and cost. A Program Evolution Graph retains successful and failed trials, and recurring subprograms are promoted to a reusable operator library. Our analysis characterizes finite-horizon error propagation, structured search state, and semantics-preserving operator abstraction. Experiments across canonical PDE families show that APEX improves long-horizon accuracy over learned neural operators while remaining competitive with task-specific coarse solvers. Combined library-and-history reuse improves a related PDE under the same target-side call budget. In one seven-call synthesis run on under-resolved 2D Navier–Stokes, APEX synthesizes a near-cutoff shadow-state program whose final error is lower than the standard coarse solver. APEX provides an explicit, auditable, and reusable approach to numerical-method design.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
partial differential equations, executable numerical operators, agentic program synthesis, program evolution, scientific machine learning, numerical methods, long-horizon dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5EjTOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces Apex, which relies on an LLM to iteratively develop a PDE solver in a structured fashion.

Critical strengths

Nice description of related works, bucketed into two clusters.

Elegant idea of using a more structured approach for LLM-based solving.

Critical weaknesses

Parts of the paper are really hard to understand, especially for an ML/AI audience. For instance, bullet 3 in the intro, which is the main summary of empirical results, was incomprehensible to me. More generally, I think the language used in this paper is often unnecessarily complicated, and gets in the way of understanding the work (the text felt very much AI generated).

The theoretical analysis section appears a bit out of the blue. It's not mentioned in the intro or anywhere else previously. I think it should either be more sufficiently motivated or removed.

Questions for authors

I'm curious about the experimental setup, both in terms of the settings and the baselines.

  • Settings: It looks like you introduced your own benchmarking tasks. Why did you do that rather than reusing benchmarking tasks the were considered in previous works?
  • Baselines: How did you choose these baselines? In particular, if I'm understanding correctly, the language model-driven automation methods like CodePDE are your closest competing methods, and I'm curious why none of these methods were included as baselines?
Reviewer_7Ry6OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper builds a search loop called APEX. The PDE is already known. An agent edits a typed solver program. A compiler throws out bad types and over-budget programs. A frozen scorer returns one-step error, rollout error, gradient error, spectrum error, growth, failure, and cost. A graph stores wins and failures. Repeated pieces can become library operators.

The tests use five made-up PDE sets: Burgers, Kuramoto–Sivashinsky, 2D Navier–Stokes, Gray–Scott, and a 3D wave. APEX is compared with a coarse solver and with FNO, Transolver, LinearNO, and Tucker-FNO. The headline example is one seven-call run on a coarse 2D Navier–Stokes grid. That run’s final error is 19.3 times lower than the coarse solver.

Critical strengths

The task is a real one. A known PDE still needs a grid scheme, a time scheme, and constants. Writing the solver as a typed program is a clear choice. The primitive list is concrete: FFT, mask, RK4, split, and an auxiliary state (Appendix A.2).

The protocol has some good locks. The agent cannot change the PDE, the reference data, or the scorer. Test trajectories are not used to pick the program. Failed trials stay in the graph. The authors say the 19.3 times result is one search path, and that they did not knock out each term (Section 5.5). They also say call count is not the same as candidate count (Appendix A.4.5).

Appendix A.6 is the most useful table. On the same program space and the same number of unique candidates, APEX reaches R40 = 2.60×10⁻³. Random search reaches 1.54×10⁻². Evolutionary search reaches 6.91×10⁻³. That is a real comparison inside the DSL.

On the main five tasks, the coarse solver is close. APEX is a bit better on four and a bit worse on the 3D wave at the default budget (Table 1: wave R40 is 1.52×10⁻³ versus 1.42×10⁻³). The paper does not hide that cell.

Critical weaknesses

1. The theory section restates textbook facts.

Equation (14) says the rollout error grows like L^N e0 plus a sum of one-step defects. That is the standard one-step recursion in numerical ODE books (Hairer et al., 1993; LeVeque, 2007). Theorem A.1 proves it by the triangle inequality and induction. It does not describe APEX, and the text says it is not a stability guarantee.

Proposition A.3 says a finer view of the search state cannot make the best rule worse, because the coarse rule is a special case of the fine rule. Proposition A.4 says the same thing for a larger policy set. Both are true because one set sits inside the other. They do not show that this LLM uses the extra diagnostics well. The authors say that. Then these propositions should not be offered as analysis of the method.

Lemma A.5 says a library name that expands to the same tree has the same meaning. That is the definition of a macro. DreamCoder already learns reusable program pieces from search (Ellis et al., 2021). The length claim in Proposition A.6 is the usual “a macro can shorten the tree” fact. It does not say the search gets faster. The paper says that too.

This is the same move as a weak-journal note that reprints a known method and then prints a known error bound. Abascal Fuentes et al. (2022), in the Journal of Advances in Mathematics and Computer Science, take Horner’s rule, split the polynomial into blocks, and then analyze the rounding error. The venue is a low-bar ScienceDomain title. The scientific habit matches Section 4 here: a familiar object, a small wrapper, and a bound that was already true before the wrapper.

Bohannon (2013) sent a badly flawed paper to many open-access journals. More than half accepted it. A basic reader was supposed to see the flaw. Here, a reader who knows numerical PDE can see that Equation (14) is the standard rollout bound, and that the 19.3 times line is one run against a coarse scheme.

2. The 19.3 times line is one search path against a weak coarse scheme.

Figure 2 gives R40 = 7.34×10⁻³ for APEX and 1.42×10⁻¹ for the coarse spectral solver. The ratio is about 19.3. The text says this path was not repeated, so it does not estimate how often the search finds it. It also says there is no term-by-term knockout.

The coarse solver in that setting is skew convection, a hard two-thirds dealias, and RK4 with four substeps, on a 64² grid cut down from 128² (Appendix A.4.4). That is not a strong large-eddy baseline. The usual tools for a cutoff grid are a dynamic Smagorinsky term (Germano et al., 1991), a scale-similarity term (Bardina et al., 1980), approximate deconvolution (Stolz and Adams, 1999), and spectral viscosity (Tadmor, 1989). None of these is in the table. Learned coarse corrections already exist too (Bar-Sinai et al., 2019; Kochkov et al., 2021; Stachenfeld et al., 2022).

The found update is an extra field z near the cutoff, with fitted numbers 2.125, 0.925, and 6.38, plus weak-RA constants A = 0.03125 and q0 = 0.63. The start library already contains a generic extra-state primitive. So the search was allowed to add a hidden field before it “discovered” one. The missing test is the same program with z removed, and the same program with the feedback term removed.

3. Beating the neural nets is an easy win. The real gap to the coarse solver is small.

On Burgers, the coarse solver already has R40 = 8.66×10⁻⁵. FNO has 4.38×10⁻². The equation is known, and a spectral reference exists (ETDRK4, Kassam and Trefethen, 2005). A neural operator trained for 100 epochs of five-step push-forward is the wrong main rival. FNO, Transolver, LinearNO, and Tucker-FNO are for maps when you do not want to write the scheme (Li et al., 2021; Wu et al., 2024). Here the scheme is the object being searched.

Against the coarse solver, the R40 drops are about 26% (Burgers), 24% (KS), 23% (Navier–Stokes), and 19% (Gray–Scott). On the wave, APEX is worse. The ten-run wave row in Table 9 picks the best of ten validation runs and then looks better (1.20×10⁻³ versus 1.42×10⁻³). That is extra search, not the default method. The paper’s own selected programs for Burgers, KS, Gray–Scott, and the wave are not printed. Only the hard Navier–Stokes program is shown. A reader cannot see whether APEX found a new scheme or a known one, such as ETDRK4, CNAB2, or a dealias mask (Orszag, 1971; Ascher et al., 1995).

4. The reuse test stays inside one equation family.

Table 3 sends a Burgers library and a KS library to a PDE that is Burgers plus a fourth-derivative term. Full reuse beats scratch (R40 2.05×10⁻⁴ versus 7.20×10⁻⁴) and beats WENO5+FD4+IMEX. The two sources are the two pieces of the target. The paper says library and memory are transferred together, so the table does not say which one helped. CodePDE, AutoNumerics, and FunSearch already search solver code with an evaluator (Li et al., 2026; Du et al., 2026; Romera-Paredes et al., 2024). Those systems are cited and not run on this DSL.

5. The data and the agent cannot be checked.

All five sets are new. No PDEBench trajectory is reused (Takamoto et al., 2022). The agent is gpt-5.6-sol through a closed API (Appendix A.4.4). The prompt is not in the paper. Three synthesis seeds are not enough for the 19.3 times claim, because that claim is a single path.

Questions for authors
  1. Please print the selected program for each of the five main PDEs, in the same style as Figure 2. If it is ETDRK4, CNAB2, or a two-thirds dealias plus RK4, please say so.

  2. On the hard Navier–Stokes case, please rerun the search at least ten times from the same root and report how often R40 beats 1.42×10⁻¹ by 19 times. Please also score three knockouts: no z, no feedback into ω, and no near-cutoff mask.

  3. Please add Germano et al. (1991), Bardina et al. (1980), and Stolz and Adams (1999) on the same 64² grid, with the same 128² reference. Please add the learned coarse corrector of Kochkov et al. (2021) as well.

  4. Equation (14) is the standard Lipschitz rollout bound (Hairer et al., 1993). What new fact does Theorem A.1 give that this bound does not already give? If the answer is none, please move it to a remark.

  5. Propositions A.3 and A.4 say a larger set of rules can do at least as well as a smaller set. Please show one edit in a logged run where the spectrum or the growth ratio changed the next edit, and the scalar score alone would have chosen a worse edit.

  6. Table 4 does not match the number of compiled candidates. Please rerun “no feedback,” “no structural edits,” and “no memory” at the same unique-candidate count used in Table 8.

  7. For the reuse test, please run library-only and memory-only. The target is Burgers plus KS. Please also try one far target, such as Gray–Scott or the 3D wave, with the Burgers–KS library frozen.

  8. Why are FNO and Transolver the main learned baselines when the coarse solver is already about 100 times better than FNO on Burgers? Please state the claim as a comparison with classical schemes.

  9. Table 9 selects one wave program from ten runs. Please report the mean and the spread of all ten test scores, not only the validation winner.

  10. Please release the DSL, the prompts, the selected ASTs, and the five datasets. Without them, a closed model name is not a reproducible search.

References

  • Abascal Fuentes, P., Fueyo Tirado, F., García Quintas, D., Jiménez Meana, J., and Palacio, A. (2022). On the fast evaluation of polynomials. Journal of Advances in Mathematics and Computer Science, 37(6). doi:10.9734/jamcs/2022/v37i630457.
  • Ascher, U. M., Ruuth, S. J., and Wetton, B. T. R. (1995). Implicit-explicit methods for time-dependent partial differential equations. SIAM Journal on Numerical Analysis, 32(3), 797–823.
  • Bardina, J., Ferziger, J. H., and Reynolds, W. C. (1980). Improved subgrid-scale models for large-eddy simulation. AIAA Paper 80-1357.
  • Bar-Sinai, Y., Hoyer, S., Hickey, J., and Brenner, M. P. (2019). Learning data-driven discretizations for partial differential equations. PNAS, 116(31), 15344–15349.
  • Bohannon, J. (2013). Who’s afraid of peer review? Science, 342(6154), 60–65.
  • Du, J., Sun, Y., and Yang, H. (2026). AutoNumerics. arXiv:2602.17607.
  • Ellis, K., Wong, C., Nye, M., Sablé-Meyer, M., Morales, L., Hewitt, L., Cary, L., Solar-Lezama, A., and Tenenbaum, J. B. (2021). DreamCoder. PLDI / Communications of the ACM lineage; the paper cites Ellis et al. (2021).
  • Germano, M., Piomelli, U., Moin, P., and Cabot, W. H. (1991). A dynamic subgrid-scale eddy viscosity model. Physics of Fluids A, 3(7), 1760–1765.
  • Hairer, E., Nørsett, S. P., and Wanner, G. (1993). Solving Ordinary Differential Equations I. Springer.
  • Kassam, A.-K., and Trefethen, L. N. (2005). Fourth-order time-stepping for stiff PDEs. SIAM Journal on Scientific Computing, 26(4), 1214–1233.
  • Kochkov, D., Smith, J. A., Alieva, A., Wang, Q., Brenner, M. P., and Hoyer, S. (2021). Machine learning–accelerated computational fluid dynamics. PNAS, 118(21), e2101784118.
  • LeVeque, R. J. (2007). Finite Difference Methods for Ordinary and Partial Differential Equations. SIAM.
  • Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2021). Fourier neural operator. ICLR.
  • Li, S., et al. (2026). CodePDE. The paper’s reference list.
  • Orszag, S. A. (1971). On the elimination of aliasing in finite-difference schemes by filtering high-wavenumber components. Journal of the Atmospheric Sciences, 28, 1074.
  • Romera-Paredes, B., et al. (2024). Mathematical discoveries from program search. Nature.
  • Stachenfeld, K., Fielding, D. B., Kochkov, D., Cranmer, M., Pfaff, T., Godwin, J., Cui, C., Ho, S., Battaglia, P., and Sanchez-Gonzalez, A. (2022). Learned coarse models for efficient turbulence simulation. ICLR.
  • Stolz, S., and Adams, N. A. (1999). An approximate deconvolution procedure for large-eddy simulation. Physics of Fluids, 11(7), 1699–1701.
  • Tadmor, E. (1989). Convergence of spectral methods for nonlinear conservation laws. SIAM Journal on Numerical Analysis, 26(1), 30–44.
  • Takamoto, M., et al. (2022). PDEBench. NeurIPS.
  • Wu, H., Luo, H., Wang, H., Wang, J., and Long, M. (2024). Transolver. ICML.
Reviewer_Lfu7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes APEX, an agentic framework for automatically designing numerical schemes for time-dependent PDEs. It combines a typed numerical operator language, deterministic compilation, evaluator-guided program evolution, search memory, and reusable operator libraries. Experiments across five PDE families demonstrate improvements over neural operators and selected classical numerical solvers. The paper also reports an interesting numerical mechanism discovered for under-resolved Navier–Stokes equations.

The problem is important, and the results are promising. However, I have several major concerns.

  1. Missing comparisons with existing PDE solver agents. This is a significant weakness. The paper discusses existing frameworks such as AutoNumerics, CodePDE, AutoPDE, and LLM-PDEveloper, but none are included in the numerical comparisons. Since the primary contribution is an agentic framework for numerical algorithm design, comparisons with neural operators and conventional numerical solvers are insufficient to establish improvements over existing agentic approaches. Direct comparisons with representative PDE solver agents under comparable computational budgets are necessary.

  2. Insufficient comparison with established numerical methods. It remains unclear whether APEX can generate numerical solvers that outperform, or even match, well-established state-of-the-art numerical methods. The experiments compare against only one conventional solver per PDE, without demonstrating that these baselines are carefully optimized. The classical baselines predominantly use finite-difference discretizations, despite the availability of many sophisticated alternatives. In particular, Fourier pseudo-spectral methods are natural choices for the periodic PDEs considered and are already used to generate the reference solutions. The selection of spatial discretizations, time integration schemes, and time steps is not sufficiently justified. Without comparisons against carefully optimized classical methods, it is difficult to determine whether APEX discovers competitive numerical algorithms or merely improves upon suboptimal baselines.

  3. Limited methodological novelty. Most components of APEX, including evaluator-guided program evolution, structured feedback, search memory, and reusable operator libraries, have been explored in previous work. Although the typed numerical operator language provides a useful framework, the paper does not clearly establish a fundamentally new algorithmic contribution beyond integrating existing techniques.

  4. Limited theoretical contributions. The theoretical results in Section 4 are largely elementary. The finite-horizon error bound follows from a standard Lipschitz argument, while the remaining propositions are consequences of information inclusion, policy-class inclusion, and semantics-preserving program substitution. These results do not provide substantial insight into the convergence, stability, or approximation properties of the numerical methods discovered by APEX.

  5. Insufficient validation of the most significant discovery. The reported 19.3× improvement on under-resolved Navier–Stokes is potentially the most interesting contribution. However, this result comes from a single synthesis trajectory without repeated independent discoveries or component-level ablations. Consequently, it is unclear whether the discovered mechanism is reproducible, which components are responsible for the improvement, or whether it generalizes to different flow regimes.

  6. Incomplete computational efficiency evaluation. The paper emphasizes fixed LLM-call budgets, but the numbers of generated candidates and numerical evaluations are not necessarily matched. More importantly, the runtime efficiency of the resulting numerical solvers is not systematically compared against established numerical methods. Accuracy-versus-runtime comparisons, together with the total computational costs of synthesis, are essential for evaluating the practical benefits of the framework.

Overall, the paper addresses an important research problem, but the experimental evidence is insufficient to establish the competitiveness of the generated numerical solvers or the superiority of APEX over existing agentic approaches. Together with the limited methodological novelty and theoretical contributions, these concerns prevent me from supporting acceptance in its current form.

Critical strengths
  1. Important problem. The paper addresses automated numerical method design for PDEs, an interesting research direction with substantial practical potential.

  2. Promising numerical results. The reported improvement on under-resolved Navier–Stokes suggests that agentic program evolution may discover useful numerical mechanisms beyond standard solver configurations.

Critical weaknesses
  1. Inadequate baseline comparisons. The paper neither compares APEX with existing PDE solver agents nor establishes its competitiveness against carefully optimized state-of-the-art numerical solvers. Consequently, its main claims about automated numerical algorithm design are not convincingly supported.

  2. Limited novelty and insufficient validation. The framework largely integrates existing techniques, while the most significant numerical discovery lacks repeated independent validation and component-level ablations.

Questions for authors
  1. Why are existing PDE solver agents, such as AutoNumerics, CodePDE, and AutoPDE, excluded from the comparisons? Can the authors provide evaluations under comparable computational budgets?

  2. How were the classical numerical baselines selected and optimized? Why are alternative discretizations, particularly Fourier pseudo-spectral methods, not included? Were the time integration schemes and time steps systematically optimized?

  3. Can the authors demonstrate that APEX generates solvers competitive with state-of-the-art numerical methods through accuracy-versus-runtime comparisons?

  4. How reproducible is the reported 19.3× improvement on under-resolved Navier–Stokes across independent synthesis runs? Which components of the discovered numerical program are responsible for the improvement?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.