acceptodds
Under review as a conference paper at ICLR 2027

A Robust In-Context Model for Conservation Laws: Injecting Context into Flux Neural Operators via Recurrent Vision Transformers

Abstract

PDE foundation models adapt to new systems from context but do so with generic backbones that ignore the numerical structure of the equations they solve. We study the opposite design for one class: families of hyperbolic conservation laws, whose finite-volume structure guarantees discrete conservation. HFluxNO infers a numerical flux operator from a short observed trajectory, with no access to the governing equation or its coefficients: a temporally recurrent vision transformer encodes the trajectory, a hypernetwork maps the encoding to the parameters of a Flux Neural Operator, and the generated flux advances the state through the conservative finite-volume update, so that conservation holds exactly for every generated solver. Across one- and two-dimensional scalar and vector-valued conservation laws, including the compressible Euler equations, HFluxNO improves rollout accuracy over recent in-context solvers, remains the most accurate model under zero-shot shifts to a flux family never seen in training and, in all but one setting, to shock-dominated initial conditions, and conserves the state exactly in every case. Code: https://anonymous.4open.science/r/in_context_flux_no/.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operators, In-Context Learning, Conservation Laws, Scientific Machine Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_AyEFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HFluxNO, an in-context neural solver for families of hyperbolic conservation laws. Given a short trajectory from an unknown system, a recurrent vision transformer encodes the trajectory into a context vector, a hypernetwork uses that vector to generate the parameters of a neural numerical-flux model, and the resulting flux is applied through a finite-volume update. The finite-volume structure guarantees exact conservation by construction. The authors evaluate on five families spanning 1D/2D, scalar/vector systems, different fluxes, and distribution shifts, including an unseen sine flux and shock-dominated initial conditions. HFluxNO generally outperforms DPOT and DISCO, while maintaining exact conservation, although it develops grid-scale oscillations on longer Euler rollouts and loses to the baselines on shock-dominated shallow water.

Critical strengths

One strength of the paper is the combination of in-context adaptation with an exact conservation guarantee. The finite-volume construction gives a real structural guarantee that the generic in-context baselines do not have, and the empirical results suggest that this is useful for long autoregressive rollouts. The comparison is also reasonably thorough, including unseen fluxes and shock-dominated initial conditions rather than only standard interpolation tests. I also like the component-level analysis. In particular, the authors compare against alternatives that receive the same context and separately replace the encoder, conditioning mechanism, and conservative target. This makes the architectural claims substantially better supported than many papers in this area.

Critical weaknesses

My main concern is that the paper somewhat overstates what is demonstrated about robustness to unseen conservation laws. The most striking zero-shot result is training on the cubic family and testing on a sine flux, but this is still a relatively controlled synthetic setting with the same periodic Cartesian discretization and the same general scalar conservation-law structure. More importantly, the model is trained on a fairly narrow family of fluxes and the paper does not establish how far the inferred flux generalizes beyond this type of functional variation. I would be more convinced by substantially more diverse unseen flux families or a clearer analysis of what information in the 20-frame context actually makes the new flux identifiable. There is also a substantive limitation in the claimed "robust" solver: conservation is guaranteed, but stability and physical admissibility are not. The paper itself shows that HFluxNO develops grid-scale oscillations in Euler at longer horizons and performs worse than the baselines on shock-dominated shallow water. This is not a minor implementation issue, since shocks and admissibility are central to the motivation for choosing hyperbolic conservation laws in the first place. The paper is candid about this limitation, but I think it weakens the broader claim that the numerical structure provides a robust foundation for in-context PDE modeling.

Questions for authors
  1. What evidence supports the claim that the context window is actually identifying the underlying flux, not learning a good predictor for the distribution of trajectories seen during training? In particular, can you quantify how accurately the inferred solver/flux predicts dynamics for substantially different coefficient regimes or flux families?
  2. How sensitive are the results to the particular choice of 20 context states? Does increasing or decreasing the context length materially change the ability to generalize to the unseen sine flux? How does performance change with context length, time step/CFL number, and rollout horizon?
  3. Since stability and entropy admissibility are not guaranteed, how often does the generated flux produce nonphysical states or unstable rollouts across the test distribution? The long-horizon Euler oscillations suggest that exact conservation alone may be insufficient for the intended shock-dominated setting.
  4. Can the same in-context framework be extended to an entropy-stable or monotone numerical flux without sacrificing the ability to generate the solver parameters from context?
  5. Can the authors compare against relevant in-context operator-learning methods such as ICON, PDE generalization of ICON, In-Context Neural PDE, and Nonlocal Attention Operator, rather than only conventional neural-operator baselines?
  6. What are the training and inference costs, including the size of the generated numerical-flux network and the per-step runtime/memory overhead?
  7. Since the positional encoding is tied to the training grid and the time step is fixed, does the method transfer to different resolutions or CFL conditions?
5 comments
AuthorsOpenReview ↗

Response to Reviewer AyEF (Q1 and Q6)

We thank the reviewer for the detailed and balanced review. We address Q1 (whether the context identifies the flux) and Q6 (inference cost) here with a new experiment; the remaining questions will be addressed in a separate response.

Q1. Does the context identify a solver, or does the model learn a trajectory predictor?

In the default rollout, HFluxNO re-encodes the most recent 20 frames at every step. As the rollout proceeds, these frames are increasingly the model's own predictions, so the flux network is regenerated from a context that it has itself produced. If the encoder and hypernetwork were only learning a good next-frame predictor for the training distribution, the generated flux network should be useful only for the step it was generated for.

To test this, we decouple flux generation from time stepping. The flux network is generated once from the current window and then held fixed for nn consecutive finite-volume steps; only after nn steps is it regenerated from the latest window. n=1n = 1 recovers the original rollout exactly (we verified a maximum absolute difference of 00 in float32). Results on the 2D cubic family, 20-step rollout, final-frame error (×10−1\times 10^{-1}, mean ±\pm std over 3 seeds):

Refresh every nn stepsIn-dist. Rel. ℓ2\ell^2In-dist. Rel. ℓ∞\ell^\inftyShock IC Rel. ℓ2\ell^2Shock IC Rel. ℓ∞\ell^\infty
11 (original)0.131±0.00650.131 \pm 0.00650.638±0.0340.638 \pm 0.0341.76±0.241.76 \pm 0.245.65±0.755.65 \pm 0.75
220.125±0.00560.125 \pm 0.00560.603±0.0290.603 \pm 0.0291.69±0.241.69 \pm 0.245.53±0.775.53 \pm 0.77
440.112±0.00400.112 \pm 0.00400.545±0.0210.545 \pm 0.0211.59±0.221.59 \pm 0.225.27±0.735.27 \pm 0.73
880.100±0.0029\mathbf{0.100 \pm 0.0029}0.487±0.016\mathbf{0.487 \pm 0.016}1.44±0.20\mathbf{1.44 \pm 0.20}4.86±0.67\mathbf{4.86 \pm 0.67}

A flux network generated once and applied for eight steps is not only usable but more accurate: the 20-step relative ℓ2\ell^2 error decreases by 25% in distribution and by 18% on shock-dominated initial conditions. The improvement is monotone in nn and holds for every seed individually. The generated flux therefore behaves as a self-contained numerical solver that remains valid over many time steps without access to the context, which is what one expects if the context determines the dynamics rather than the next frame. The gain also suggests that per-step re-encoding feeds the model's own prediction error back into the generated solver; holding the solver fixed removes this feedback path.

Q6. Inference cost

The same experiment shows that the encoder and hypernetwork need not be evaluated at every step. With refresh every nn steps, they are called 20/n20/n times per 20-step rollout instead of 2020 times, and every other step evaluates only the generated flux network inside the finite-volume update. With n=8n = 8, the context is processed 33 times instead of 2020 per rollout while the error is reduced, so the cost of parameter generation is amortized over many solver steps rather than paid at every step.

AuthorsOpenReview ↗

Reply to reviewer AyEF (1/4)

We thank the reviewer for these detailed questions. The general comment lists the revisions and defines the terms that we use below (context length, rollout, the two trivial baselines). This reply follows the order of the questions and completes our earlier reply on Q1 and Q6.

Q1. What does the context identify? Our earlier reply showed that a solver generated once and held for several steps is more accurate than one regenerated at every step. The revision reports this on 1D and 2D cubic (Appendix E, Table 10, which holds the 2D numbers of that reply). Those numbers are 20-step errors, taken over all 20 steps as defined in Section 5, and not final-frame errors as that reply labeled them. With the solver held for eight steps, the 20-step error falls by 20% and 24% in distribution, the latter where we wrote 25% before.

To answer the question further, we performed additional experiments that systematically corrupt or exchange the context frames read by the encoder, while the solver advances the true state (Section 6.1, Appendix G). A change in the prediction can then come only from a change in the generated solver. All errors are one-step errors on 1D cubic.

  • The dynamics are needed. With the 20 context states in a random order, or with the last state repeated 20 times, the error rises to that of the identity baseline (0.94 and 0.98 of it, against 0.07 with the true context). The states alone, without their time evolution, are of no use to the model.
  • The equation is read from the context. A context from a trajectory with other coefficients makes the prediction worse than the identity baseline (1.23 of its error). The cost grows with the distance between the two fluxes, from 7.4 to 22.4 times the true-context error, measured among replacement contexts that cover all values of the advanced state (Table 15).
  • The flux is represented over the visited states. A context from another solution of the same equation is not free. Its cost depends on its coverage, the fraction of the values of the advanced state that lie within the range of values in the replacement context. The error falls from 11.3 to 2.2 times that of the true context as the coverage rises from below one half to one (Table 14).

We therefore claim identification in a limited form, and the revision states the limit (Sections 6.1 and 7). The model does not recover the flux as a function, or its coefficients. It generates a solver that is accurate for the equation shown in the context, over the states that the context visited. A factor of 2.2 remains even at full coverage, which we attribute to the coarseness of our coverage measure and have not resolved. We also do not test extrapolation to a disjoint coefficient range inside the cubic family, since the test coefficients are unseen draws from the training range.

Different flux families. The submitted sine flux was nearly a cubic, as Reviewer DHwv pointed out. The revision replaces it with a rescaled sine flux, f(u) = (a/|b|)·sin(bu) in four bins of |b| up to 4, and the Buckley–Leverett flux of two-phase flow in three bins of the mobility ratio. Both move away from the cubic family in measured steps while the maximum wave speed stays fixed (Fig. 3, Appendix J.2). The distance is the relative residual of the wave speed f′ from that of the closest cubic, 0.00 to 0.36 across the sine bins and 0.19 to 0.43 for Buckley–Leverett (Table 24). The cubic-trained models are evaluated without retraining.

  • HFluxNO has the lowest 20-step error on all eleven sets. In relative ℓ² its error is 20% to 40% below DISCO's and 44% to 72% below DPOT's.
  • On smooth data its error doubles across the sine sweep (0.046 to 0.102) and stays below that of the identity baseline, which DPOT reaches.
  • On Buckley–Leverett it is flat at 0.073, against 0.110 to 0.120 for DISCO and 0.161 to 0.174 for DPOT.
  • The limit is visible on the farthest sine flux, where HFluxNO's fronts fall behind the reference within three context lengths (Fig. 10(b)). Section 7 states it.

These tests remain in the setting that the reviewer describes, which is scalar, one-dimensional and on the training grid. We state the claim for that setting.

AuthorsOpenReview ↗

Reply to reviewer AyEF (2/4)

Q2. Context length, time step, CFL number and rollout horizon (Appendix I). All errors here are one-step errors on 1D cubic, so that they are not mixed with the accumulation of error in a rollout.

  • Context length. We trained models with 5, 10, 20 and 40 context states, three runs each. In distribution the error (×10⁻³) is 5.11, 4.30, 3.80 and 3.77, so it stops improving at 20 states. On the unseen sine fluxes 40 states still help, by 11% and 12% in the two bins nearest to the training family and by 4% and 1% in the two farthest (Table 19). A trained model does not transfer to another length. The submitted model, run with 5, 10 or 40 states, reaches 1.17, 0.59 and 0.53 of the error of the identity baseline, against 0.07 with 20 (Table 20).
  • CFL number. The CFL number is the number of cells that the fastest wave crosses in one time step. At the training grid and time step it varies across the test trajectories from 0.05 to 10.7, because the wave speed does. We sort the test segments into twelve bins of equal count. HFluxNO has the lowest error in every bin, and its error does not grow with the CFL number (Fig. 9).
  • Time step. The encoder does not receive Δt, and the finite-volume update takes it explicitly, so the generated solver can be applied at another step. With the context at the training step, one update at 0.5Δt or 2Δt gives 1.7 and 10 times the error at Δt. These errors are 0.23 and 0.37 of the error of the identity baseline, against 0.07 at Δt (Table 21). The generated flux is thus a flux for the training step. The baselines share this restriction, since DPOT maps frames to the next frame and DISCO integrates its vector field over a fixed unit interval.
  • Rollout horizon. Fig. 4(a) follows the error over 80 steps, four context lengths, together with the two trivial baselines. Table 12 counts the rollouts of each model that are worse than trivial by 20, 40 and 80 steps (see Q3).

In summary, the accuracy holds across what varies inside the training distribution, the wave speed, and the model is tied to what is fixed during training, the context length and the time step. Section 7 lists both among the limitations.

Q7. Resolution and CFL transfer. We answer this question here because it continues Q2. We tested the CFL number only as it varies across test trajectories at the training grid and time step, where the accuracy holds. We did not test another resolution, and the current model cannot be run at one, because the learned positional encoding of the encoder ties it to the training grid, as Section 7 and Appendix A.1 state. The patch-based encoders of the baselines have the same restriction. Lifting the restriction to one grid, time step and context length is among the next steps of Section 8.

Reviewer_R8HrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HFluxNO, a context-conditioned neural operator for conservation laws. A recurrent Vision Transformer encodes short solution trajectories, and a hypernetwork generates the parameters of a Flux Neural Operator. The resulting model predicts numerical fluxes and advances the solution using a conservative finite-volume update. Experiments on cubic conservation laws, shallow-water equations, and viscous Burgers show improvements in prediction accuracy, long-term rollout stability, and generalization to unseen flux functions.

Critical strengths
  1. Combining in-context adaptation with a conservative finite-volume formulation is a reasonable and useful approach. The flux-difference structure provides a meaningful physical inductive bias.

  2. The experimental results are encouraging, particularly for long-term prediction and unseen sine-flux dynamics. The encoder and target-network ablations also provide useful evidence for the architectural choices.

Critical weaknesses
  1. The contribution of context conditioning is not sufficiently isolated. The paper compares different encoders and target networks, but does not include a Flux NO baseline without context conditioning. It is therefore difficult to determine how much of the improvement comes from the learned context representation rather than the conservative architecture itself.

  2. The OOD robustness claims are not fully supported by all results. In Table 2, HFluxNO performs substantially worse than both DPOT and DISCO in single-step relative L2 error on the shock-dominated shallow-water dataset. Its rollout L2 error is also higher than DISCO's. This suggests that the benefit of the proposed architecture may depend on the equation and distribution shift. Moreover, all main experiments are restricted to one-dimensional problems, limiting the evidence for broader generalization.

Questions for authors
  1. How does HFluxNO compare with a Flux NO trained without context conditioning? Would shuffling or replacing the context trajectories affect performance?

  2. Why does HFluxNO perform worse than the baselines on the shallow-water OOD benchmark?

  3. Can the authors provide conservation-error measurements or other numerical stability diagnostics to support the claimed physical robustness?

2 comments
AuthorsOpenReview ↗

Reply to reviewer R8Hr (1/2)

We thank the reviewer for the follow-up and for stating the two remaining concerns clearly. We address them directly below. The general comment defines the terms that we use in our replies (context length, rollout, the two trivial baselines).

Shock-dominated shallow water (W2, Q2). This result came from a mismatch in the test set. The submitted set used step initial conditions and also drew β from [8, 12], ten times the training range of [0.8, 1.2], so it tested a tenfold coefficient extrapolation together with the new initial conditions. The submission listed the test values as the training family by mistake. Item 1 of the general comment gives the two files of the submitted anonymous code in which this can be checked. The revised set changes only the initial condition, as the shock-dominated sets of the other families do, and the same trained models are evaluated. On it HFluxNO has the lowest 20-step error in both norms (0.064 in relative ℓ², against 0.075 for DPOT and 0.104 for DISCO). At a single step DPOT is lower (6.0e-3 against 6.5e-3), which we report as an exception in Appendix J.1. Under the combined shift of the submitted set HFluxNO did trail both baselines at one step and DISCO over 20 steps, and we do not claim otherwise.

Instability in long Euler rollouts. We agree that exact conservation does not guarantee stability, and no longer suggest it in the revision. We have removed "Robust" from the title. The contributions state that stability and positivity are learned properties, and Section 7 opens with "HFluxNO guarantees conservation and nothing else." We also add to the revision measurements of when and how each model fails (Section 6.2, Fig. 4, Appendix F).

  • We would also like to clarify the context in which HFluxNO's error overtakes the baselines' at about t = 0.4. Fig. 4(a) now shows two trivial baselines: identity, which repeats the last true state, and spatial mean, which predicts a constant field. HFluxNO's median error overtakes the baselines' only after the median error of all three models has exceeded the spatial-mean baseline. By step 40, two context lengths, 14% of HFluxNO's Euler rollouts have reached that level, against 42% for DISCO and 99.9% for DPOT. By step 80 nearly all rollouts of all models have (Table 12). Accuracy and physical admissibility are separate matters, however. At the same step 40, 58% of HFluxNO's rollouts have already violated positivity somewhere in the domain, against 75% for DPOT and 99% for DISCO (Fig. 7).
  • The failures differ in kind. At one and two context lengths HFluxNO reproduces the shock structure, while DPOT's prediction has broken into patch-aligned blocks and DISCO's is covered by ringing (Fig. 4(b)). HFluxNO's oscillation starts at isolated strong compressions and grows from there, and conservation holds through it.

So the late crossing occurs in a regime where the baselines have already failed, and our accuracy claims rest on the earlier 20-step results. A guarantee would require dissipation built into the generated flux, as conservation is built in now, which Section 8 names as the next step.

Q3. Diagnostics (Appendix F). We follow rollouts of 80 steps on all five families.

  • Conservation. The drift of the total of each conserved variable stays at round-off for HFluxNO, at the level of the reference solver, while that of DPOT and DISCO is several orders of magnitude higher and grows (Fig. 6).
  • Worse than trivial. By 80 steps in distribution, 25%, 37%, 3% and 0% of HFluxNO's rollouts on 1D cubic, shallow water, viscous Burgers and 2D cubic have an error above that of the spatial-mean baseline, against 78%, 75%, 35% and 86% for DPOT and 55%, 75%, 99% and 21% for DISCO (Table 12).
  • Total variation. HFluxNO's total variation stays close to that of the reference for about two context lengths on every family, which indicates the absence of spurious oscillations over that horizon (Fig. 8).
  • Positivity. Height, density and pressure must stay positive, and no model guarantees it. This property shows the smallest advantage. HFluxNO has the fewest violations on Euler, DPOT is comparable on shallow water at short horizons, and by 80 steps nearly every Euler rollout of every model has violated it (Fig. 7).
AuthorsOpenReview ↗

Reply to reviewer R8Hr (2/2)

Q1. Shuffling or replacing the context (Section 6.1, Appendix G). We have performed an additional experiment on this point, on the 1D cubic family. The state that the solver advances is always the true last frame, and only the context frames that the encoder reads were changed. A change in the prediction can therefore come only from a change in the generated solver. The encoder receives one of five inputs.

  • True context. The 20-frame window that ends at the state being advanced.
  • Shuffled frames. The same 20 states in a random order, which keeps the states and destroys their temporal order.
  • Last frame repeated. Twenty copies of the last state, which shows the current state and no motion.
  • Another trajectory, same coefficients. The context of a different solution of the same equation.
  • Another trajectory, other coefficients. The context of a solution of a different equation of the family.

One-step relative ℓ² error, divided by the error of the identity baseline (Table 13):

Encoder inputError ÷ identity
True context0.07
Shuffled frames0.94
Last frame repeated 20 times0.98
Context of another trajectory, same coefficients0.45
Context of a trajectory with other coefficients1.23

The results show three things.

  • The model uses the dynamics in the context. A context without usable dynamics, shuffled or static, removes the advantage almost entirely, although every true state is still present.
  • The generated solver is specific to the equation in the context. A context from another equation makes the prediction worse than the identity baseline, which is what one expects when a solver built for one equation is applied to a state of another.
  • The solver is accurate over the states that the context visited. A context from another solution of the same equation is better than any other replacement, but it is not free. Its cost depends on how many of the values of the advanced state lie within the range of values that the replacement context contains, which we call its coverage. The error falls from 11.3 to 2.2 times that of the true context as the coverage rises from below one half to one. A context from another equation costs 17 to 18 times at any coverage (Table 14). With the coverage at one, the cost grows with the distance between the two fluxes, from 7.4 to 22.4 times (Table 15).

Together with the fixed Flux NO of Table 1, which has no context and 6.9 times the error of HFluxNO, these results isolate the contribution of the context. The gain comes from what the model extracts from the time evolution of the context, and not from the conservative architecture alone. We do not claim that the model recovers the governing equation uniquely. We also comment that a context from the same equation with full coverage still costs 2.2 times the true-context error, which we attribute to the coarseness of the metric definition (Appendix G, Summary paragraph).

Reviewer_DHwvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HFluxNO, an in-context neural operator for solving conservation laws direclty instead of knowing the euqations or coefficients. Compared to PDE foundation models, HFluxNO explicitly preserves conservation through a finite-volume formulation. It uses a recurrent Vision Transformer to infer dynamics from 20 observations, a hypernetwork to generate flux prediction parameters, and a flux neural operator to evolve the solution. Experiments across five PDE benchmarks show that HFluxNO generally outperforms DPOT and DISCO, particularly on unseen coefficients and flux functions. However, it struggles with long-term stability on Euler equations and performs worse than DISCO on shock-dominated shallow-water problems.

Critical strengths

S1. Integration of conservation into in-context learning.

A key strength of HFluxNO is that it combines in-context learning with a conservative finite-volume method. Instead of directly predicting future states, the model predicts numerical fluxes, which ensures discrete conservation by construction. This provides a useful physical constraint and makes the approach more reliable than purely data-driven methods in terms of conservation.

Critical weaknesses

W1. Conservation does not guarantee numerical stability

Although HFluxNO guarantees discrete conservation, it does not ensure numerical stability or convergence to physically correct solutions. This is particularly evident in the 2D Euler experiments (Figures 8–11), where the model develops oscillations near strong shocks during long rollouts, and its error eventually exceeds the baselines around t=0.4t=0.4. This raises concerns about the robustness of the proposed method.

W2. Potential issue with the conservation-error metric

Equation (12) defines the normalized conservation error as Dn=∣Mn−M0∣/∣M0∣D^n=|M^n-M^0|/|M^0|. However, Appendix B.5 states that the 2D cubic conservation-law initial conditions are generated with mean subtraction, meaning M0M^0 should be close to zero. This makes the normalization potentially ill-conditioned. It is therefore unclear how Table 11 reports conservation errors around 10−910^{-9}, and whether the reported metric is reliable.

W3. Limited difficulty of the unseen-flux experiment

The paper evaluates generalization from cubic flux functions to a sine flux. However, for the tested parameter and state ranges, the sine function can be closely approximated by a cubic Taylor expansion, with an error below approximately 0.0083 for ∣u∣≤1|u|\leq1. Therefore, the unseen sine flux may not represent a sufficiently challenging distribution shift, making the generalization claim less convincing.

W4. Limited evidence for the hypernetwork's effectiveness

Table 1 shows that replacing the hypernetwork with simple context concatenation only slightly increases the relative error from 0.521 to 0.544, with overlapping standard deviations. This raises questions about whether generating the full flux-network parameters is necessary. Moreover, generating around one million parameters and repeatedly processing the context may introduce considerable computational overhead without a clear performance benefit.

Questions for authors

Refer to the weakness

4 comments
AuthorsOpenReview ↗

Response to W4 (computational overhead of the hypernetwork)

We thank the reviewer for raising the cost of generating the flux parameters. The reviewer is right that regenerating them from the context at every step is costly, and a new experiment shows that this is not necessary.

We generate the flux network once from the current context window and hold it fixed for nn finite-volume steps before regenerating it. On the 2D cubic family, the 20-step relative ℓ2\ell^2 error decreases as nn grows: from 0.131±0.00650.131 \pm 0.0065 (n=1n = 1, the original rollout) to 0.100±0.00290.100 \pm 0.0029 (n=8n = 8) in distribution, and from 1.75±0.261.75 \pm 0.26 to 1.44±0.201.44 \pm 0.20 on shock-dominated initial conditions (×10−1\times 10^{-1}, 3 seeds). With n=8n = 8, the encoder and hypernetwork are evaluated 33 times per 20-step rollout instead of 2020 times. Generating the flux network is therefore a cost amortized over many steps, and the generated network can be reused as a standalone solver. Full results are in our response to Reviewer AyEF (Q1).

AuthorsOpenReview ↗

Reply to reviewer DHwv (1/3)

We thank the reviewer for these questions, which led to most of the new analyses. The general comment defines the terms that we use below (context length, rollout, the two trivial baselines).

W1. Conservation does not guarantee stability. We agree. The revision says so and measures stability directly instead (Section 6.2, Appendix F). The revised contributions and Section 7 state that conservation is the only guarantee and that stability and positivity are learned, and we have removed "Robust" from the title. We follow rollouts of 80 steps, four context lengths, on every family and report four diagnostics.

  • Conservation. We measure how far the total of each conserved variable drifts from its initial value. HFluxNO's drift stays at round-off on every family, at the level of the reference solver. The drift of DPOT and DISCO is several orders of magnitude larger and grows along the rollout (Fig. 6).
  • Total variation. We compare the total variation of the prediction with that of the reference. A ratio above one indicates spurious oscillations. HFluxNO's ratio stays close to one for about two context lengths on every family (Fig. 8). For the scalar families this is a physical check, because their entropy solutions do not increase total variation.
  • Positivity. We count the rollouts in which the height (shallow water) or the density or pressure (Euler) has become non-positive. Positivity is learned and not guaranteed, and it shows the smallest advantage (Fig. 7). In distribution at 40 steps, 8% of HFluxNO's shallow-water rollouts and 58% of its Euler rollouts have violated positivity, against 10% and 75% for DPOT and 20% and 99% for DISCO. DPOT has fewer violations than HFluxNO on shallow water at 20 steps. By 80 steps nearly every Euler rollout of every model has violated it.
  • Worse than trivial. We count the rollouts whose error has exceeded that of the spatial-mean baseline (Table 12). By 80 steps in distribution this happens to 25%, 37% and 0% of HFluxNO's rollouts on 1D cubic, shallow water and 2D cubic, against 78%, 75% and 86% for DPOT and 55%, 75% and 21% for DISCO. HFluxNO has the lowest fraction on nearly every test set at every horizon.

The Euler result. We acknowledge the instability, but would like to clarify that the performance loss against the baselines due to the instability happens only after the median error of all three models has exceeded that of the spatial-mean baseline. We demonstrate this by adding the two trivial baselines to Fig. 4(a): identity, where the last frame of the initial context is used as the prediction, and spatial mean, where the prediction is a constant field. The updated figure shows that HFluxNO's median error overtakes the baselines' at about t = 0.4, by which time all three medians lie above the spatial-mean baseline. By step 40, 14% of HFluxNO's Euler rollouts are worse than trivial, against 42% for DISCO and 99.9% for DPOT. By step 80 nearly all rollouts of all models are (Table 12).

We also comment that the model failures differ in kind. HFluxNO's error starts as a grid-scale oscillation at isolated strong compressions and spreads from there, while the baselines lose the structure of the whole field within one context length (Fig. 4(b), Appendix K). The oscillation appears in Fig. 8 as a steep rise of the total variation after two context lengths, and conservation holds through it. We relate it to the odd-even decoupling of low-dissipation classical fluxes, whose cure is added dissipation. Building dissipation and positivity into the generated flux, as conservation is built in now, is the next step that our updated conclusion names (see also our reply to AyEF Q4).

AuthorsOpenReview ↗

Reply to reviewer DHwv (2/3)

Shallow water. The result that the reviewer's summary mentions came from a mismatch in the test set used, as explained in item 1 of the general comment. The submitted shock-dominated set drew the coefficient β from ten times the training range, in addition to changing the initial condition. On the revised set, which changes only the initial condition, HFluxNO has the lowest 20-step error in both norms, and DPOT has the lowest single-step error.

W2. The conservation metric. As in our earlier reply, the denominator is the L¹ norm of the initial state, which does not vanish for mean-subtracted data. The revision corrects the formula (Eq. 14) and reports the drift on all five families (Fig. 6), not on 2D cubic alone.

W3. The unseen sine flux is close to a cubic. We agree. For the submitted ranges the sine flux is a cubic to third order, so that test was nearly in distribution. We have replaced it (Fig. 3, Appendices B.2 and J.2) with the following:

  • Rescaled sine flux, f(u) = (a/|b|)·sin(bu) with a ~ U[−1, 1] and |b| in the bins (0, 1], (1, 2], (2, 3] and (3, 4]. The factor 1/|b| fixes the maximum wave speed, so only the shape of the flux changes. The bin (0, 1] is the near-cubic regime that the reviewer describes.
  • Buckley–Leverett flux, the non-convex flux of two-phase flow, in three bins of the mobility ratio, scaled to the same maximum wave speed.
  • Distance from the training family (Table 24), the relative L² residual of the wave speed f′ from that of the closest cubic: 0.00, 0.02, 0.15 and 0.36 across the sine bins, and 0.19, 0.33 and 0.43 for Buckley–Leverett.

The models are trained on the cubic family only and are evaluated on these fluxes without retraining. 20-step relative ℓ² error, sine flux on smooth initial conditions (Table 25):

Bin of abs(b)HFluxNODISCODPOTIdentity
(0, 1]0.0460.0730.1610.222
(1, 2]0.0580.0810.1630.198
(2, 3]0.0800.1050.1730.174
(3, 4]0.1020.1280.1810.151

HFluxNO has the lowest error on all eleven sets, in both norms. The eleven sets are the four sine bins on smooth and on shock-dominated data and the three Buckley–Leverett bins. Its advantage holds at every distance from the training family, with a relative ℓ² error 20% to 40% below DISCO's and 44% to 72% below DPOT's. On the sine flux with shock-dominated data its error is 0.068 to 0.084. On Buckley–Leverett, the family farthest from the cubics by our measure, it stays flat at 0.073, against 0.110 to 0.120 for DISCO and 0.161 to 0.174 for DPOT. HFluxNO also remains well below the error of the identity baseline on every set, whereas DPOT reaches that level on the fluxes farthest from the training family.

As expected, the error of every model grows along the sine sweep, as the flux moves away from the training family. HFluxNO's error doubles across the sweep on smooth data, and in ℓ∞ its margin over DISCO becomes small in the two farthest bins (Table 26). On the farthest flux its fronts begin to lag the reference within three context lengths (Fig. 10(b)). Section 7 reports this limit.

AuthorsOpenReview ↗

Reply to reviewer DHwv (3/3)

W4. The hypernetwork. We agree that generation and concatenation are comparable in accuracy (4%, Table 1), and no longer call every component indispensable. Section 4.4 now states that the encoder and the conservative target are crucial for accuracy, and that the hypernetwork is chosen for the standalone solver it produces (Appendix E). We also correct a number. The generated network has 71,681 parameters in 1D, not of order 10⁶, which was the size of the hypernetwork's output layer.

  • Cost of generation (Appendix D). The output layer of the hypernetwork holds 88% of the parameters but takes 1.4% of the FLOPs of a step in 1D and 0.1% on Euler (Table 9). Regenerating at every step costs mostly the context encoder, which DPOT and DISCO also run at every step. HFluxNO and the concatenation variant take nearly the same time per step, 0.036 and 0.038 ms per trajectory when a batch of 2000 is advanced together on one A100 (Table 8).
  • Reuse (Appendix E). The generated solver is an explicit finite-volume scheme that runs without the encoder, so it need not be regenerated at every step. We generate it once from the current context, hold it for r steps, and only then regenerate it. Held for eight steps, it cuts the time per step 4.9 times in a batch and 3.1 times for a single trajectory in 1D (Table 11), which makes HFluxNO faster than DPOT in both settings. It also lowers the 20-step error, by 20% on 1D cubic and 24% on 2D cubic (Table 10). We attribute the lower error to a feedback that reuse removes. In the default rollout a prediction error enters the context and therefore changes the solver that is generated next.
  • Reuse is not specific to the hypernetwork. We ran the same test on the concatenation variant. Holding its context vector for eight steps lowers its last-frame error by 23%, against 22% for HFluxNO in the same measure. The agreement supports our explanation that the gain comes from removing the feedback of prediction errors into the inferred dynamics. Nevertheless, we believe that the standalone solver of the hypernetwork design is the more convenient form to reuse. A context-concatenated model must still receive its frozen conditioning input at every step.

All other results in the paper regenerate the solver at every step, which is the standard inference mode of autoregressive solvers. In our earlier reply on W4, the shock-dominated entry at n = 1 should read 1.76 ± 0.24, as in Table 10.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General comment to all reviewers (1/2)

We thank all three reviewers for their careful reading. Their questions led to new experiments and to several corrections, all of which are in the revised PDF, where changed text is set in blue. This comment lists the revisions, and the replies to each reviewer refer to its items.

Terms used in all replies. HFluxNO reads the last 20 states of a trajectory, its context, and predicts the next state. A rollout feeds the predictions back into the context. One context length is 20 steps, or t = 0.1. We compare every model with the two trivial baselines of Fig. 4(a). The identity baseline repeats the last true state, so it predicts no change. The spatial-mean baseline predicts a constant field, the spatial mean of each conserved variable. A rollout is worse than trivial once its error has exceeded that of the spatial-mean baseline.

Revisions

1. A correction to the shock-dominated shallow-water test set (R8Hr W2 and Q2; raised also by DHwv and AyEF). The submitted set differed from the training family in two ways. It used step initial conditions for h, and it also drew β from [8, 12], while training used [0.8, 1.2] (and a variance of 0.5 for the initial momentum, where training used 1). Table 2 and Appendix B.3 of the submission listed these test values as the training family by mistake. Both can be checked in the submitted anonymous code: scripts/data/configs/pde/shallow_water_1d.yaml holds the training range, and the cell of notebooks/data_generation/shallow_water_1D.ipynb that generates the step set passes "beta": (8, 12). The submitted set therefore tested a tenfold extrapolation in a coefficient together with the change of initial condition.

The revised set changes only the initial condition, as the shock-dominated sets of the other families do, and the same trained models are evaluated, which is why the in-distribution numbers are unchanged. We also compared the coefficient ranges of every other test set with its training set and found no other mismatch. On the revised set HFluxNO has the lowest 20-step error (0.064 in relative ℓ², against 0.075 for DPOT and 0.104 for DISCO), and its single-step error is comparable to DPOT's (6.5e-3 against 6.0e-3). The submitted numbers remain a valid measurement of the combined shift. On it HFluxNO trailed both baselines at one step and DISCO over 20 steps in relative ℓ².

2. Harder unseen fluxes (DHwv W3, AyEF Q1; Fig. 3, Appendix J.2). The sine flux of the submission was close to a cubic. It is replaced by a sine sweep that moves away from the cubic training family in measured steps, and the Buckley–Leverett flux of two-phase flow is added as a second family. HFluxNO has the lowest 20-step error on all eleven new test sets, in relative ℓ² 20% to 40% below DISCO's and 44% to 72% below DPOT's.

3. Conservation and stability in long rollouts (DHwv W1 and W2, R8Hr Q3, AyEF Q3; Section 6.2, Fig. 4, Appendix F). We follow rollouts of 80 steps on every family and measure four properties: the drift of the conserved totals, the positivity of height, density and pressure, the total variation, and the fraction of rollouts that are worse than trivial. HFluxNO conserves every variable to round-off on every family, while the baselines drift by several orders of magnitude more. The properties that HFluxNO does not guarantee are also better than the baselines'. Its rollouts stay better than trivial for longest on nearly every test set, its total variation follows the reference for about two context lengths on every family, and its positivity violations grow most slowly on the Euler system. On Euler, 14% of its rollouts are worse than trivial after two context lengths, against 42% for DISCO and 99.9% for DPOT. Our reply to DHwv W1 gives the numbers.

4. What the model reads from the context (R8Hr W1 and Q1, AyEF Q1; Section 6.1, Appendix G). We corrupt or exchange the 20 context states while the state being advanced stays the true one. A shuffled or static context brings the error to that of the identity baseline, and a context from another equation makes it worse. A context from another solution of the same equation works to the extent that it visits the same range of states. The generated solver therefore depends on the time evolution in the context and on the equation that the context shows.

5. Cost and reuse of the generated solver (DHwv W4, AyEF Q6; Appendices D and E, Section 4.4). We report parameters, FLOPs, time and memory of all models. The output layer of the hypernetwork, which generates the solver, accounts for 1.4% of the FLOPs of a step on 1D cubic, and most of the rest is the context encoder. The generated solver can also be held for several steps without rerunning the encoder. Held for eight steps, it makes a step three to five times cheaper in 1D and lowers the 20-step error by 20% to 24%.

AuthorsOpenReview ↗

General comment to all reviewers (2/2)

6. Sensitivity (AyEF Q2 and Q7; Appendix I). We performed an inspection of the model's sensitivity to changes in wave speed (CFL number), context length and time step. HFluxNO has the lowest error at every wave speed of the test set, and its error does not grow with the CFL number. The context length of 20 states is sufficient in distribution. The context length and the time step are fixed by training, and the baselines share the restriction to one time step.

7. Scope of the claims (AyEF weaknesses and Q5, R8Hr, DHwv W4). The claims are now stated at the strength supported by the evidence. The abstract and contributions compare with autoregressive in-context solvers. The contributions state that conservation is guaranteed and that stability and positivity are learned, and they name the encoder and the conservative target as the components that carry the accuracy. Section 7 opens with "HFluxNO guarantees conservation and nothing else."

We have also removed "Robust" from the title, which is now "An In-Context Model for Conservation Laws: Injecting Context into Flux Neural Operators via Recurrent Vision Transformers".

8. Smaller corrections (DHwv W2 and W4). The generated network has 71,681 parameters in 1D and 180,360 on Euler, not of order 10⁶ (Sections 4.2 and A.2). Table 4 lists B = 4 blocks for the 1D families. The conservation metric divides by the L¹ norm of the initial state (Eq. 14). Appendix B.1 states that test trajectories extend to t = 0.5 while training uses t ≤ 0.4. Appendix B.3 states where the generalized shallow-water system is hyperbolic.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.