acceptodds
Under review as a conference paper at ICLR 2027

Differentiate the Evaluator: Co-Searching Programs and Parameters with Exact Gradients

Abstract

Scientific models can be expressed as executable programs whose code represents mechanisms, equations, control flow, and numerical procedures. An AI agent may therefore propose the correct scientific mechanism yet have it rejected because its continuous constants are poorly calibrated. In program search, calibration is not merely post-processing: it helps determine which scientific ideas survive. Language models and gradient-based optimization offer complementary strengths, with language models proposing discrete program structure and gradients efficiently calibrating continuous constants. Existing approaches, however, either restrict candidates to fixed differentiable templates or require new tracing, compilation, or implementation work for each proposed program. We instead differentiate the evaluator once, allowing programs supplied as data to inherit exact reverse-mode gradients with respect to embedded constants. Our native differentiable virtual machine executes these programs at runtime and batches many parameter candidates through a single structural execution, making gradient calibration practical as the inner loop of AlphaEvolve-style evolutionary program search. We evaluate the method by co-searching nonlinear WENO shock-capturing schemes for computational fluid dynamics. From language-model-proposed programs, the system reliably recovers the expert-designed WENO-Z+ mechanism and recalibrates its parameters through full solver rollouts. The resulting schemes reduce error by 34–46% relative to WENO-Z on three unseen problems (32–41% against independent reference solutions) and, for one seed, by 26–47% relative to published WENO-Z+, while retaining fifth-order accuracy; a stronger shock test exposes a small undershoot that WENO-Z avoids. Ablations show that evolutionary search identifies the mechanism, while exact-gradient calibration produces most of the cross-problem improvement. Discrete structure also acts as a regularizer: given the same gradient freedom, a strong expert-designed family crosses a stability cliff and develops large shock oscillations, whereas every discovered program keeps a fixed exponent that stays clear of it. A second co-search over state-space programs exercises loops, branches, and matrix operations, and an accompanying optimizer study shows that gradients become increasingly important as parameter dimensionality grows. These results suggest that differentiating the evaluator rather than each candidate program enables AI-driven co-search over discrete scientific structure and continuous parameters without per-program differentiation or engineering effort.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
program synthesis, program search, differentiable programming, automatic differentiation, large language models, bilevel optimization, scientific machine learning, differentiable interpreters, evolutionary program search, symbolic regression

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RmY8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses a setting in which AI models search over programs with the goal of finding executable models that explain observed phenomena (e.g., physics simulations). One approach in this setting is a two-level approach: first, search over program structures; then, for a given program structure optimize continuous parameters with gradient descent. However this can be expensive, since a candidate program must be compiled into a reverse-mode AD graph. The paper proposes a differentiable virtual machine which amortizes the cost of differentiating candidate programs, and an approach to batched execution of a given program to further accelerate search. The paper evaluates on a fluid simulation, finding that the proposed approach successfully finds a program and corresponding parameters that match behavior of held-out data.

Critical strengths
  • To the best of my knowledge, the proposed approach is technically sound
  • The work has an interesting and important core conceit, that of accelerating the speed at which AI agents can iterate in the task of scientific discovery; doing this via accelerating the inner search in SGA is an interesting approach
Critical weaknesses
  • To the best of my understanding, the core contribution of the paper is in the speedup of NDVM vs Jax (and not in the general approach of using gradient descent in the inner loop). However, the evaluation does not actually demonstrate that this speedup leads to more accurate discovery. For example, I would have expected to see a plot showing something along the lines of held-out fitness vs wall-clock time showing that NDVM results in faster optimization than a naive approach.
    • While there is some discussion of the performance characteristics in Section 3, it’s not clear how the various different cited wall-clock numbers add up to compare against a well-tuned Jax baseline.
  • The paper also does not sufficiently describe NDVM: what are the set of programs that one can write in this language? What are the semantics of these programs? How does the batching executor work with substantially divergent program traces (e.g., looping for different numbers of iterations, nested branches, etc)?
Questions for authors

I would be willing to raise my score conditioned on an evaluation demonstrating that NDVM-based optimization is more efficient than baseline approaches, and on a more complete description of the NDVM language.

Reviewer_S3xGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose a new method to find gradients when optimizing constants in program search: the native differentiable virtual machine (NDVM). The setup is using LLM in the outer loop to search program structure, and use optimizer to calibrate constants for a given program structure, where NDVM supplies the gradients. NDVM is an interpreter with a native automatic differentiation implementation and batched numerical execution. It obtains gradients by recording the numerical operations in the forward pass, and using automatic differentiation to obtain gradients in the reverse pass. The key idea is to separate program structure from numerical values in the implementation, so that evaluations with the same program structure can be evaluated in batches. In this way, NDVM provides an efficient implementation. Compared to compilation-based methods like JAX, NDVM is much slower per gradient step. However, since it saves compilation time, when a program does not need to be re-evaluated for many times, NDVM can provide a meaningful speedup over JAX.

This paper then apply its search pipeline using NDVM on 2 tasks: a fluid solver and state-space model structure. For the fluid solver, this paper's pipeline successfully recovers a more advanced program structure (WENO-Z+) using Fable. This paper also noted WENO-Z+ is a published structure so it may be inside the training data of Fable, so the discovery is not a finding. The main results are calibration of the constants using NDVM provides significant improvement over a few metrics, while keeping the program structure the same. This experiments show calibration of constants is helpful for program search, and NDVM is a good candidate for that.

The state-space model structure experiments show this pipeline can be used beyond simple equation-style program search. However, in this experiment, calibration of constants does not seem to be helpful, and it also does not show advantages of NDVM.

Critical strengths

This paper provides a new implementation for computing gradients in the calibration of constants in program search: the native differentiable virtual machine (NDVM). It is a concrete implement with codes provided. The key ideas: separating program structure from numerical constants is clear and helpful. This paper also clearly layouts the advantages and disadvantages of NDVM: Compared to compilation-based methods like JAX, NDVM is much slower per gradient step. However, since it saves compilation time, when a program does not need to be re-evaluated for many times, NDVM can provide a meaningful speedup over JAX.

Therefore, the provided codes and ideas in this implementation can be reused and applied to circumstances where NDVM has advantages if such circumstances can be identified.

Critical weaknesses
  1. No demonstrated evidence for the practical advantages of NDVM over other auto-differentiation methods.

The efficiency benefits of NDVM over other auto-differentiation methods like JAX is a core claimed contribution of this paper. However, this advantage is only showed in standalone benchmarks, but not in any practical experiments. Although the micro-benchmark for per-program cost (Table 6) in this paper shows that NDVM can provide meaningful speedup in gradient computations when a program does not need to be re-evaluated for many times, the actual experiment presented in this paper does not show an advantage of NDVM in efficiency.

In Appendix B, it says "Median recorded end-to-end times at the 40-step budget are 74.5, 17.4, and 10.5 s, respectively. JAX compilation has median 4.21 s in that same subset". This suggests that NDVM-based pipeline does not have efficiency advantage in the actual WENO-Z experiment. There is also no comparison of execution time for the state-space experiment. Therefore, it is unclear for which practical tasks NDVM has an actual advantage over other auto-differentiation methods.

  1. Limited evidence for the benefits of having calibration as an inner loop in program search

This paper clearly shows the benefits of calibration of constants in program search. However, this paper does not show that having calibration as an inner loop in program search is meaningfully better than performing calibration after program search. A key motivation of this paper is having calibration as an inner loop can prevent promising structures from being discarded because of poorly chosen constants. It is also the only provided use case of NDVM. I think such evidence is important to claim the usefulness of NDVM.

In the WENO-Z experiment, Fable is able to recover WENO-Z+ structure without calibration, and when we already have WENO-Z+ structure, the constants can be tuned to achieve similar performance as the program search with NDVM-based calibration (as show in Table 1). In the state-space model experiment, Table 3 does not show any meaningful difference between program search with and without calibration. Therefore, experiments in this paper do not suggest having calibration can lead to better program structure. On the other hand, they seem to support that LLM-based program search achieves similar program structures with and without calibration. If that is true for most cases, fitting constants after program structure search may be an easier and more practical method, and thus weakening the value of NDVM.

Questions for authors
  1. What are the practical circumstances that NDVM has actual advantage over other auto-differentiation methods? Do you have new experiment results to support that?

  2. Are there evidence showing the benefits of having calibration as an inner loop in program search? If there are tasks where having calibration as an inner loop can lead to better program structure, how does NDVM perform in those tasks?

Reviewer_UNCsOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a new algorithm search and optimization framework that decouples an algorithm into an executable program and its tunable parameters. Under a heuristic search paradigm, the outer loop searches discrete program forms (e.g., in the OpenEvolve framework, LLMs generate program code), while the inner loop computes the gradient of the evaluation function with respect to the parameters and calibrates the parameter values using gradient-based optimizers. To support this process, the authors introduce NDVM, which interprets programs conforming to the supported forms and automatically computes parameter gradients. The proposed method is evaluated on the co-search of nonlinear WENO shock-capturing schemes, achieving an accuracy–robustness balance competitive with, and in some cases better than, that of expert-designed schemes.

Critical strengths
  1. The framework assigns discrete program construction to an LLM-driven outer search and continuous parameter calibration to a gradient-based inner optimizer. This division of responsibilities is reasonable: the proposer explores alternative computational structures, while the optimizer refines numerical constants using the task objective. By design, NDVM can integrate with frameworks like OpenEvolve as a parameter-calibration inner loop, giving it the potential to be widely applied across multiple domains.

  2. The paper is clearly written, with well-articulated motivation, task background, and proposed approaches.

  3. The experiment results on WENO and state-space tasks are comprehensive. The authors also discuss unfavorable outcomes, including oscillation, weaker smooth-solution accuracy, mixed recovery results, and runtime overhead. This reporting provides a useful account of both capabilities and limitations

Critical weaknesses
  1. NDVM’s advantage over existing autograd frameworks remains unclear. For example, an alternative is to have the LLM generate PyTorch programs and calibrate their parameters directly using autograd. This approach could benefit from mature tooling, broader operator and hardware support, and standard code-generation workflows. Table 6 mainly compares against a tensor-hosted interpreter, while NDVM is slower than the eager kernel on WENO. Moreover, Appendix H demonstrates gradient calibration using PyTorch directly. A controlled end-to-end comparison is needed to establish NDVM’s incremental value.

  2. Complete co-search experiments cover only WENO and synthetic state-space models, which seems limited compared to its potential application scenarios. OpenEvolve and related frameworks have already been applied to domains such as kernel and AI/ML algorithm optimization. As a reusable parameter-calibration component within OpenEvolve, NDVM could build on this breadth of application to demonstrate its effectiveness across more varied tasks with suitable differentiable objectives.

Questions for authors
  1. Could you provide more implementation details about NDVM, including its programming language and dependencies? Under what workloads does it offer advantages over directly implementing candidate programs in a mature framework such as PyTorch and using its autograd, and what evidence supports these advantages?

  2. Beyond WENO and state-space modeling, could you give concrete examples of tasks for which NDVM would be suitable or unsuitable? What requirements or limitations determine its applicability to these tasks?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.