acceptodds
Under review as a conference paper at ICLR 2027

GAV: A scientific experimentation environment for PDE agents

Abstract

Reliable feedback is essential for coding agents to develop programs for partial differential equations (PDEs). Yet evaluating these programs on new PDE problems often requires expensive reference computations, while fixed datasets restrict the range of experiments. We introduce Generator–Agent–Verifier (GAV), a scientific experimentation environment that makes new PDE problems verifiable by construction. The Generator learns a distribution of continuous reference solutions using data from numerical simulations or experimental measurements to represent a target physical regime. Using the method of manufactured solutions, it works backward from the generated reference solutions to construct matching PDE problems without a separate numerical solve. The Agent designs experiments, investigates failures, and iteratively improves its program. The Verifier evaluates candidate programs against the reference solutions and provides numerical feedback on accuracy and computational cost. This feedback can also guide the generation of new problems that probe the current program's weaknesses. We provide a theoretical analysis of evaluation reliability and guided development. Experiments on Navier–Stokes and Vlasov–Poisson–Fokker–Planck tasks across different data sources demonstrate the effectiveness of GAV. In particular, on the numerical Navier–Stokes task, GAV reduces the mean test error of three existing PDE agents by more than an order of magnitude relative to their original setups.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
LLM agents, scientific experimentation environments, partial differential equations, AI for science, generative models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ar4GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces an agentic environment (or harness) called GAV to improve the performance of agentic PDE solvers. GAV uses the method of manufactured solutions to cheaply generate PDE problems to help PDE solvers improve their outputs.

Critical strengths
  • This paper is studying an interesting question of enhancing the environment/harness used by PDE solver agents.
  • The the specific- approach of cheaply generating PDE problems in a curriculum of sorts to help that agent improve is also an elegant idea.
Critical weaknesses
  • It is not clear to me that GAV improves agentic performance on tasks of interest. Figure 5 for instance does clearly show improvement of GAV/GAV-Full over the Original environment, but if I'm understanding correctly, the evaluations are done on PDE problems that GAV itself generates. It would be more compelling IMO to demonstrate that GAV combined with something like CodePDE could improve CodePDE performance on the benchmarks that CodePDE identified. If this were teh case, I'd be much more bullish on this work.
  • Figure 5 seems to show that GAV is better than GAV-Full, thus calling into question the effectiveness of guided generation and perturbation.
Questions for authors
  • The theoretical results provide a reasonable conceptual framing, but it's not clear that they guide any concrete design choice for the proposed environment. If they do, can you explain how?
6 comments
AuthorsOpenReview ↗

Response to Reviewer ar4G (1/2)

We thank the reviewer for finding the question of enhancing the environment of PDE agents interesting and the cheap generation of a problem curriculum elegant. We address performance on tasks of interest (W1) in this comment, and GAV-Full (W2) and how the theory guides the design (Q1) in the next. Changes in the revised PDF are highlighted in blue, and the new experiments are collected in App. F.


W1 — Performance on tasks of interest

Answer: GAV improves CodePDE on CodePDE's own benchmarks, on top of an already strong baseline. With gpt-5.6-sol, CodePDE alone reaches nRMSE below 10⁻⁴ on all three tasks we test; on the official problems and against the official reference solutions, CodePDE + GAV still lowers its error by 2.8× on reaction–diffusion and 1.4× on compressible NS, and on Burgers both versions reproduce the official reference to round-off. On our NS task, GAV's gains also hold beyond GAV-generated problems: on original forward problems with their prescribed forcing, GAV improves all three agents by 15–107×. In the revision, the main GAV-Full program is also 3.4× more accurate than the best of three established solvers.

1. CodePDE's own benchmarks

We compare CodePDE and CodePDE + GAV on three of CodePDE's native tasks: 1D Burgers (ν=0.01\nu=0.01), 1D reaction–diffusion (ν=0.5\nu=0.5, ρ=1\rho=1), and 1D compressible NS (η=ζ=0.1\eta=\zeta=0.1, γ=5/3\gamma=5/3). Both versions use the same 50 official development problems, model, and development procedure (three candidate generations and three refinements). CodePDE + GAV additionally obtains new development problems built by recombining PDE parameters and applying MMS, without training a Generator, and queries inputs at chosen coordinates. Both select their final program on the same 50 development problems and are tested on the same 100 official problems, with the official initial conditions, parameters, boundary conditions, output grids, and numerical reference solutions. We report mean nRMSE (App. F.3, Table 12).

Native taskCodePDECodePDE + GAVCodePDE / GAV
1D Burgers1.47 × 10⁻⁴⁵1.47 × 10⁻⁴⁵1.0×
1D reaction–diffusion6.99 × 10⁻⁹2.50 × 10⁻⁹2.8×
1D compressible NS8.17 × 10⁻⁵5.91 × 10⁻⁵1.4×

GAV thus further improves an already highly accurate agent wherever improvement is possible. Since GAV builds its development problems here from parameter recombination and MMS alone, it extends to new PDE families without training a Generator.

2. Original NS forward problems

We evaluate the frozen final programs of Sec. 4.3 on 20 original NS forward problems, keeping their prescribed forcing and using high-accuracy Nektar++ reference solutions, so no test problem is constructed by GAV (App. F.2, Table 11).

AgentOriginalGAVOriginal / GAV
CodePDE4.445%0.289%15.4×
AutoNumerics7.129%0.0666%107×
PDE-SHARP5.656%0.302%18.7×

GAV improves all three agents by 15–107× on the original problems, so its gains carry over to the target task itself.

3. Accuracy against established solvers

In the revision, three established solvers solve the main numerical NS test problems with the same geometry, viscosity, boundary conditions, and continuous input queries, and are scored at the same points (App. F.1, Table 10).

SolverMean test error ↓
Nektar++1.9042%
COMSOL1.6352%
FEniCSx0.3510%
Program developed with GAV-Full0.1046%

The program developed with GAV-Full is 3.4× more accurate than the best established solver, and its accuracy carries over to the interpolated problems (0.1256%).

AuthorsOpenReview ↗

Response to Reviewer ar4G (2/2)

W2 — GAV-Full versus GAV

Answer: GAV-Full is an optional extension for robustness under distribution shift, not the core algorithm. GAV's core (learning the problem distribution from data, exact problems through MMS, and continuous input queries) already improves all three agents by 11–16× over Original (Fig. 5).

GAV-Full adds two tools for problems that move away from the learned distribution:

  • Guided generation samples problems that are hard for the current program while staying within a KL neighborhood of the learned distribution, the worst-case objective in Eq. (5). Every guidance update produced harder problems (App. D.2).
  • Perturbation checks whether an improvement still holds on problems moved off the learned distribution (App. D.3).

Fig. 5 instead measures average error on problems from the learned distribution itself, where the core GAV is already highly effective. Even there, GAV-Full improves all three agents over Original (3.4–55×) and gives PDE-SHARP the lowest error in Fig. 5 (0.069%). The revision makes this role of GAV-Full explicit in Sec. 3.3.


Q1 — How the theory guides the design

Answer: Yes. The theory directly shapes three design choices in GAV; the revision spells out the first two in Sec. 3.2.

  1. Learning the problem distribution from target data. Proposition 1 bounds the Verifier's relative evaluation error through the KL divergence between learned and target problem distributions. This motivates learning the distribution from target-regime data rather than using a generic prior. Since MMS cannot increase this divergence (data-processing inequality [1]), learning the solution distribution suffices. Replacing the learned distribution with a Gaussian-process prior increases test error 3.3× (Sec. 4.4, Table 1).
  2. Evaluating on problems unseen during development. Proposition 1 requires the evaluated program to be chosen without using the test problems. Accordingly, the Verifier never reveals reference solutions to the Agent, and final evaluation uses a fixed test set separate from the selection problems (App. C.2).
  3. Guided generation as bounded tilting. Theorem 1 shows that, in an idealized convex setting, if each guidance step tilts the learned distribution toward high-error problems with bounded strength, stabilized revisions monotonically reduce the worst-case loss and converge. Guided generation therefore targets the loss-tilted distribution in Eq. (6), with its strength capped relative to the flow velocity (Table 4) so that guided problems stay close to the learned regime.

References

[1] Cover and Thomas. Elements of Information Theory. Wiley, 2006.

Thanks for sharing, thoug it looks like you only considered 3 of the 5 datasets included in Table 1 of the CodePDE paper. Moreover, providing similar results for the same benchmarking tasks used by AutoNumerics and PDE-Sharp would go a long way.

Overall, I think these results are promising, but they don't change my mind about the current version of the paper. I think the work would benefit from more systematically making this point in a future submission.

Reviewer_dekBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a scientific experimentation environment (GAV) which helps existing auto PDE solving system reducing the cost of acquiring high resolution reference solution. It’s designed to enlarge the scale of the available PDE testing problems through MMS and added guiding mechanism for leveraging the difficulty of the generated problems. The experiments involving two main PDE families: Navier- Stokes and Fokker-Planck; they demonstrated the effective of GAV on three existing auto PDE solving systems. Contributions: 1. The paper uses generative modeling with MMS to learn the PDE physical regime and generate large scope of new PDE testing problems as well as providing high reference solution. 2. The paper proposes guiding and perturbing mechanism for challenging problem generation with related theoretical analysis. 3. The paper creates a double agent system to verify the effectiveness of the GAV and GAV-Full environment Validation: 1. The paper applied both GAV and GAV-Full to three existing auto PDE solving systems to demonstrate the effectiveness of their environments 2. The paper applied the double agent system in their GAV-Full and compared the generated solver with solvers created by 4 classic software.

Critical strengths

The idea of using generative modeling with MMS to enlarge the availability of testing problems as well as using guidance and perturbation in leveraging the problem quality. (This is indeed a bottleneck for autoPDE solving system which heavily rely on the problem quality and quantity)

Critical weaknesses

The consistency between the forcing distribution of generated PDE problems using MMS and that of the original trajectories is not showcased. (This is critical since it may violate the physical regime.)

Questions for authors
  1. What’s the forcing distribution of the generated MMS problems compare to the distribution of original trajectories. Any quantitative results available to show the physical regime preservation? 2.For the agent comparison study, it shows a pattern that with GAV and GAV Full, the performance of the three systems promoted. However, the primary cause behind this is not direct, since it may be due to the availability of data points for evaluation under the GAV environment in contrast to each agent’s original setting. Could the authors provide more quantitative study on this?
3 comments
AuthorsOpenReview ↗

Response to Reviewer dekB (1/2)

We thank the reviewer for recognizing that combining generative modeling with MMS addresses a key bottleneck of automated PDE solving: the quality and quantity of available problems. We address the forcing distribution (W1/Q1) in this comment and the source of the gain (Q2) in the next. Changes in the revised PDF are highlighted in blue, and the new experiments are collected in App. F.


W1/Q1 — Forcing distribution and physical-regime preservation

Answer: We now provide this quantitative comparison. Generated forcing matches the original forcing in strength and spatial scale, the two forcing properties that set the physical regime. On every distribution statistic we test, it is as close to the original as two independent sets of original problems are to each other. Programs developed on generated problems also remain accurate on the original problems with their prescribed forcing, 15–107× more accurate than Original.

1. Strength and spatial scale

We compare the prescribed forcing of 64 original problems held out from Generator fitting, as used in their simulations, with the MMS forcing of 64 unguided, unperturbed generated problems. Both are evaluated at the same 32 times and 3,878 fluid points. For each problem, we compute:

  • Strength A=mean⁡t,x∥f∥22A=\sqrt{\operatorname{mean}_{t,\mathbf x}\lVert\mathbf f\rVert_2^2}, the root-mean-square forcing;
  • Spatial variation S=∥∇f∥2/∥f∥2S=\lVert\nabla\mathbf f\rVert_2/\lVert\mathbf f\rVert_2, how fast the forcing varies in space relative to its strength (an inverse length scale), computed with central differences that never cross an obstacle.

At fixed viscosity and geometry, the strength and length scale of the forcing set its Grashof number, which governs the flow regime.

StatisticOriginal, median [95% CI]Generated, median
Strength AA0.493 [0.382, 0.762]0.505
Spatial variation SS11.61 [9.79, 12.48]10.94

Intervals come from 2,000 bootstrap resamples of whole problems (App. F.4, Table 13). Both generated medians lie within the original intervals, so MMS construction preserves the original physical regime. Generated forcing also varies no faster in space than the original: differentiating the generated solutions adds no small-scale content.

2. Distribution of the full forcing fields

Each forcing field has 32 × 3,878 × 2 ≈ 2.5 × 10⁵ values, so a direct distance between the two distributions cannot be estimated reliably from tens of problems (curse of dimensionality [1]). We therefore compare statistics that each capture one property of the forcing. As a yardstick, we compute each statistic between two independent sets of 64 original problems, 500 times; this range shows the difference expected from sampling alone (App. F.4, Table 13).

StatisticWhat it comparesGenerated vs. originalOriginal vs. original, median [2.5%, 97.5%]
PCA projections, sliced W1W_1 [2]Main space–time patterns0.2900.265 [0.198, 0.372]
Structure function in xx, W1W_1Change over distance in xx0.3120.219 [0.128, 0.460]
Structure function in yy, W1W_1Change over distance in yy0.2500.192 [0.097, 0.422]
Structure function in tt, W1W_1Change over time0.1810.179 [0.088, 0.399]
Classifier, held-out AUC [3]Can a classifier tell them apart? (0.5 = chance)0.5490.492 [0.343, 0.634]

Every statistic lies within the original-vs-original range, and the classifier performs at chance: generated forcing differs from the original no more than two independent sets of original problems differ from each other.

3. Accuracy on original problems

The decisive test of regime preservation is whether programs developed on generated problems stay accurate on the original problems themselves. We evaluate the frozen final programs of Sec. 4.3 on 20 original NS forward problems with their prescribed forcing, against high-accuracy Nektar++ reference solutions (App. F.2, Table 11).

AgentOriginalGAVOriginal / GAV
CodePDE4.445%0.289%15.4×
AutoNumerics7.129%0.0666%107×
PDE-SHARP5.656%0.302%18.7×

Programs developed with GAV reach 0.07–0.30% on the original problems, at the level of their 0.22–0.35% on generated problems (Fig. 5). The generated problems therefore capture the original physical regime: programs developed on them transfer directly to the original problems.

AuthorsOpenReview ↗

Response to Reviewer dekB (2/2)

Q2 — Whether the gain comes from more evaluation data

Answer: The gain does not come from more evaluation data. With four times as many fixed development problems, Original's error does not decrease and stays 18× above GAV's. The decisive feature is continuous input queries, which give agents initial velocity and forcing at the coordinates their discretization requires, not additional reference data; removing them alone raises GAV's error 11.8×.

1. A four-times larger fixed problem pool

For CodePDE in Sec. 4.3, we enlarge Original's fixed collection from 64 to 256 problems with the same fixed-grid inputs and references, again without new problems or continuous queries during development. All programs are evaluated on the 96 test problems of Fig. 5 (App. F.5, Table 14).

EnvironmentDevelopment problemsInputsMean test error ↓
Original64 fixedFixed grid3.59%
Original-256256 fixedFixed grid4.05%
GAV64 + new on requestContinuous queries0.224%

Quadrupling the evaluation data available to Original does not reduce its error, which remains 18× above GAV's.

2. New problems without continuous queries

In the Sec. 4.4 ablation, the w/o continuous input queries condition still lets agents request new problems and receive Verifier feedback throughout development, but provides inputs only on a fixed 64×6464\times64 grid (App. C.4). Its error is 11.8× that of GAV on the same test set (5.211% vs. 0.441%, Table 1) and 13.5× on independently interpolated problems (5.915% vs. 0.440%, App. D.1, Table 8).

More data, whether a larger fixed pool or new problems on request, does not close the gap; access to inputs at any coordinates does.

References

[1] Weed and Bach. Sharp Asymptotic and Finite-Sample Rates of Convergence of Empirical Measures in Wasserstein Distance. Bernoulli, 2019.

[2] Bonneel et al. Sliced and Radon Wasserstein Barycenters of Measures. Journal of Mathematical Imaging and Vision, 2015.

[3] Lopez-Paz and Oquab. Revisiting Classifier Two-Sample Tests. ICLR, 2017.

I thank the authors for the clarification. I will maintain my positive score.

Reviewer_fmwUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes GAV (Generator–Agent–Verifier), an experimentation environment for LLM agents developing PDE solvers and reduced-order models. The Generator learns continuous solution distributions from simulation or experimental data and uses the method of manufactured solutions (MMS) to construct PDE problems with exact references. The Agent iteratively develops programs using numerical feedback from the Verifier. Guided generation and perturbation further support targeted experimentation. The paper provides theoretical guarantees under distributional and convexity assumptions. Experiments cover simulated and PIV-based Navier–Stokes (NS) problems and Vlasov–Poisson–Fokker–Planck (VPFP) problems.

Critical strengths
  1. Efficient, verifiable problem generation. GAV generates new PDE problems with exact continuous references without requiring expensive numerical solves for each problem. Learning from simulation or experimental data also helps characterize the target physical regime. This is a valuable capability for interactive scientific agents.

  2. Improvements across multiple agent frameworks. GAV improves CodePDE, AutoNumerics, and PDE-SHARP, suggesting that the benefits are not restricted to a particular agent architecture. The ablation studies also provide useful evidence about the contributions of different environment components.

Critical weaknesses
  1. The sources of improvement are insufficiently disentangled.

    • Compared with Original, GAV provides both additional development problems and continuous input queries. The ablation results show that continuous queries have the largest effect (11.8×), but the contributions of problem diversity, input accessibility, and interactive feedback are not sufficiently isolated.
    • GAV-Full performs substantially worse than GAV for CodePDE (1.07% vs. 0.22%) and AutoNumerics (0.97% vs. 0.35%). This raises questions about the actual benefits of guided generation and perturbation.
    • The large performance differences between Sections 4.2 and 4.4 are difficult to interpret because the agent configurations, test sets, and execution budgets differ. More controlled comparisons are needed.
  2. Limited evidence of generalization beyond manufactured PDE problems.

    • All evaluation problems, including the independently interpolated sets, are constructed using MMS. The independent NS evaluation retains the same continuous representation, while the independent VPFP evaluation contains only eight problems.
    • The PIV experiment uses a rectangular observation region without the cylinder obstacle, assumes zero pressure, and derives forcing from reconstructed measurements. It therefore does not directly evaluate solvers on the original physical flow problem.
    • No results are reported on the original forward-simulation problems with their prescribed forcing. Moreover, repeated-run statistics and quantitative validation of the generated PDE-input distribution are missing.
    • The theoretical guarantees rely on assumptions that are not empirically established. In particular, accurate modeling of solution distributions does not necessarily imply a small divergence between the corresponding PDE-input distributions after applying differential operators.
Questions for authors
  1. Experimental consistency: What explains the substantial performance differences between Sections 4.2 and 4.4? Can the authors evaluate the resulting programs on a common test set under matched execution limits?

  2. Guided generation: Why does GAV-Full underperform GAV for two of the three agents? Could this be related to the fixed error predictor used in Section 4.3?

  3. Generalization: Can the final programs be evaluated on the original NS forward-simulation problems, using their prescribed forcing and numerical reference solutions, without reconstructing forcing through MMS?

  4. Baseline fairness: Can the authors separately evaluate the effects of additional development problems and continuous input queries, for example by increasing Original's fixed problem pool or enabling continuous queries without introducing new problems?

  5. Reliability and fidelity: Can the authors report variance over multiple independent runs and quantify the Generator's fidelity, particularly the discrepancy between original and generated PDE-input distributions?

3 comments
AuthorsOpenReview ↗

Response to Reviewer fmwU (1/3)

We thank the reviewer for recognizing that GAV generates verifiable PDE problems efficiently and improves three different agent frameworks. We address each point below: the sources of improvement (W1) in this comment, generalization beyond manufactured problems (W2) in the next, and the theoretical assumptions and the PIV study in the last. Changes in the revised PDF are highlighted in blue, and the new experiments are collected in App. F.


W1/Q4 — Additional problems versus continuous input queries

Answer: Continuous input queries account for most of GAV's gain; additional problems alone do not reproduce it.

1. A four-times larger fixed problem pool

Following the reviewer's suggestion, we enlarge CodePDE's fixed Original collection (Sec. 4.3) from 64 to 256 problems from the same source, with the same fixed-grid inputs and references and no new problems or continuous queries. All programs are tested on the 96 problems of Fig. 5 (App. F.5, Table 14).

EnvironmentDevelopment problemsInputsMean test error ↓
Original64 fixedFixed grid3.59%
Original-256256 fixedFixed grid4.05%
GAV64 + new on requestContinuous queries0.224%

Quadrupling the fixed pool does not reduce the error, which remains 18× above GAV's.

2. New problems without continuous queries

The w/o continuous input queries ablation (Sec. 4.4) keeps GAV's new problems and Verifier feedback but provides initial velocity and forcing only on a fixed 64×6464\times64 grid (App. C.4). Its error is 11.8× GAV's (5.211% vs. 0.441%, Table 1) and 13.5× on interpolated problems (5.915% vs. 0.440%, Table 8).

Without continuous queries, neither more fixed problems nor new problems with feedback come within an order of magnitude of GAV. Continuous queries let the program obtain inputs wherever its numerical method needs them (Sec. 3.1); for example, the re-run main solver queries them on 1008×10081008\times1008 and 1056×10561056\times1056 grids (App. D.3).


W1/Q2 — GAV-Full versus GAV

Answer: GAV-Full is an optional extension for robustness under distribution shift, not the core algorithm. GAV's core (learning the problem distribution from data, exact problems through MMS, and continuous input queries) already gives every agent its main gain, 11–16× over Original (Fig. 5).

GAV-Full adds two tools for problems that move away from the learned distribution:

  • Guided generation samples problems that are hard for the current program while staying within a KL neighborhood of the learned distribution, the worst-case objective in Eq. (5). Every guidance update produced harder problems (App. D.2).
  • Perturbation checks whether an improvement still holds on problems moved off the learned distribution (App. D.3).

Fig. 5 instead measures average error on problems from the learned distribution itself, where the core GAV is already highly effective. Even there, GAV-Full improves all three agents over Original (3.4–55×) and gives PDE-SHARP the lowest error in Fig. 5 (0.069%). Its extra gain over GAV varies with the agent and, as the reviewer suggests, relates to the fixed error predictor in Sec. 4.3. The predictor is fitted once before development, and on 64 independent problems its predictions are essentially uncorrelated with the final GAV-Full programs' errors (rank correlations −0.19 to 0.04). The revision makes the role of GAV-Full explicit in Sec. 3.3.


W1/Q1 — Sections 4.2 and 4.4 under a common protocol

Answer: The difference came from the problem banks: in the submitted version, Sec. 4.2 drew its problems from a different NS Generator than Secs. 4.3–4.4, so the two error levels are not comparable. In the revision, all numerical NS studies share one Generator, and we re-ran the main experiment (Sec. 4.2, Fig. 3). On a test set common to every NS program, the 96 interpolated problems of App. D.1, the main GAV-Full program reaches 0.1256%, 3.5× below GAV in Sec. 4.4.

The interpolated problems are built from held-out numerical solutions without sampling any Generator, and every program is scored at the same 128 points per problem.

ProgramEnvironmentDevelopmentMean error, interpolated problems ↓
Sec. 4.4GAV3 rounds × 2 h0.440%
Sec. 4.2, re-runGAV-Full5 rounds × 4 h0.1256%

The long-horizon GAV-Full program is the more accurate, as expected from its larger budget and targeted experimentation. It is also below every Sec. 4.3 program developed with GAV (0.1890–0.3086%, Table 7) and close to its 0.1046% on generated test problems.

AuthorsOpenReview ↗

Response to Reviewer fmwU (2/3)

W2/Q3 — Original NS forward problems with prescribed forcing

Answer: Yes. On original NS forward problems with their prescribed forcing and numerical reference solutions, without MMS, GAV programs are 15–107× more accurate than Original programs.

We fix 20 original problems in advance, keeping each problem's initial velocity, geometry, boundary conditions, viscosity, time interval, and prescribed forcing, with high-accuracy Nektar++ references. The frozen final programs of Sec. 4.3 are scored at the same 128 points per problem under the 300 s limit (App. F.2, Table 11).

AgentOriginalGAVOriginal / GAV
CodePDE4.445%0.289%15.4×
AutoNumerics7.129%0.0666%107×
PDE-SHARP5.656%0.302%18.7×

GAV programs keep their accuracy on the original problems (0.07–0.30%, versus 0.22–0.35% on generated test problems, Fig. 5), so accuracy developed on generated problems carries over to the original forward problems.


W2/Q5 — Repeated runs and Generator fidelity

Answer: The effect of continuous queries is stable over three independent runs. On every statistic we test, generated forcing is as close to the original prescribed forcing as two independent sets of original problems are to each other.

1. Repeated runs

We repeat the continuous-query ablation of Sec. 4.4 with two new seeds for both conditions, keeping the model, initial program, budget, 24 selection problems, and 96 test problems (App. F.6, Table 15).

EnvironmentSeed 101 (Table 1)Seed 102Seed 103Mean ± std
GAV0.441%0.476%0.817%0.578 ± 0.208%
w/o continuous input queries5.211%5.944%4.173%5.109 ± 0.890%

The two conditions never overlap: the best run without continuous queries is still 5.1× worse than the worst GAV run.

2. Generator fidelity

We compare the MMS forcing of 64 unguided generated problems with the prescribed forcing of 64 held-out original problems (App. F.4, Table 13). Each forcing field has 32 × 3,878 × 2 ≈ 2.5 × 10⁵ values, so a direct distance between the two distributions cannot be estimated reliably from tens of problems (curse of dimensionality; Weed & Bach, 2019). We therefore compare statistics that each capture one property of the forcing.

Strength and spatial variation. For each problem, AA is the root mean square of ∥f∥2\lVert\mathbf f\rVert_2, and S=∥∇f∥2/∥f∥2S=\lVert\nabla\mathbf f\rVert_2/\lVert\mathbf f\rVert_2 measures how fast the forcing varies in space (an inverse length scale).

StatisticOriginal, median [95% CI]Generated, median
Strength AA0.493 [0.382, 0.762]0.505
Spatial variation SS11.61 [9.79, 12.48]10.94

Both generated medians lie within the original bootstrap intervals. Differentiation errors would add small-scale content and raise SS; the generated SS is no higher than the original.

Distribution statistics. As a yardstick, we compute each statistic between two independent sets of 64 original problems, 500 times; this range shows the difference expected from sampling alone.

StatisticWhat it comparesGenerated vs. originalOriginal vs. original, median [2.5%, 97.5%]
PCA projections, sliced W1W_1Main space–time patterns0.2900.265 [0.198, 0.372]
Structure function in xx, W1W_1Change over distance in xx0.3120.219 [0.128, 0.460]
Structure function in yy, W1W_1Change over distance in yy0.2500.192 [0.097, 0.422]
Structure function in tt, W1W_1Change over time0.1810.179 [0.088, 0.399]
Classifier, held-out AUCCan a classifier tell them apart? (0.5 = chance)0.5490.492 [0.343, 0.634]

Every statistic lies within the original-vs-original range, and the classifier performs at chance: generated forcing differs from the original no more than two independent sets of original problems differ from each other.

AuthorsOpenReview ↗

Response to Reviewer fmwU (3/3)

W2 — Theoretical assumptions

Answer: The key assumption holds both theoretically and empirically: the KL divergence in Assumption 1(i) cannot increase under MMS, and errors measured on generated problems carry over to original problems.

MMS maps target and generated solutions to problems through the same deterministic map Φ(λ,u)=(λ,Nλ[u],B[u])\Phi(\lambda,u)=(\lambda,\mathcal N_\lambda[u],\mathcal B[u]). In a well-posed family, the problem distribution is the image of the solution distribution under Φ\Phi, so the data-processing inequality [4] gives DKL(P∥P^)≤DKL(Pu∥P^u)D_{\mathrm{KL}}(P\Vert\widehat P)\le D_{\mathrm{KL}}(P_u\Vert\widehat P_u), with equality since Φ\Phi is injective. In words, MMS cannot enlarge the KL divergence between the learned and target distributions. Differential operators can amplify pointwise or Wasserstein discrepancies, but not KL. The revision adds this argument after Assumption 1 (Sec. 3.2).

Empirically, generated forcing matches the original on every statistic we test, without small-scale amplification (answer to W2/Q5), and GAV programs reach 0.07–0.30% on original forward problems, at the level of their 0.22–0.35% on generated problems (answer to W2/Q3), as Proposition 1 predicts. Theorem 1 states the principle behind guided generation, bounded tilting toward the current program's errors, which GAV implements by capping the guidance strength relative to the flow velocity (Table 4).


W2 — PIV experiment

Answer: The PIV study shows that GAV learns a physical regime directly from experimental measurements, and its conventions leave the velocity task unchanged.

Zero pressure is a gauge choice: adding ∇ϕ\nabla\phi to the forcing only shifts the pressure to p+ϕp+\phi and leaves the velocity unchanged, so p=0p=0 moves the unmeasured pressure gradient into the forcing. The observation window defines a well-posed initial–boundary-value problem: the cylinder lies outside it, and its influence enters through the time-dependent boundary velocities. The non-gradient part of the MMS forcing absorbs effects that the two-dimensional window does not resolve, such as three-dimensional motion and measurement noise, so every generated velocity field is an exact solution of its problem. Forward evaluation with prescribed forcing is given in the answer to W2/Q3.

References

[1] Weed and Bach. Sharp Asymptotic and Finite-Sample Rates of Convergence of Empirical Measures in Wasserstein Distance. Bernoulli, 2019.

[2] Bonneel et al. Sliced and Radon Wasserstein Barycenters of Measures. Journal of Mathematical Imaging and Vision, 2015.

[3] Lopez-Paz and Oquab. Revisiting Classifier Two-Sample Tests. ICLR, 2017.

[4] Cover and Thomas. Elements of Information Theory. Wiley, 2006.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.