- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces ORIA, an in-context model for inferring an unknown operator from a few input–output function pairs. Given these demonstrations and a new input function, ORIA predicts the response as a joint Gaussian over many locations, in a single forward pass and without any test-time training. It uses quadrature-weighted attention so that the same model works across different grids and resolutions. ORIA is trained on synthetic priors over operators, following the prior-fitted network idea, so that its output approximates the Bayesian posterior predictive under that prior. The experiments have four parts. The first compares ORIA with the exact posterior on a linear-Gaussian operator family and shows that good pointwise calibration can hide errors in the joint distribution. The second tests few-shot prediction on three finite-element (FEM) problem families and on public Navier–Stokes (NS) data, against neural-process, FNO-based and classical baselines. The third uses ORIA's uncertainty to choose which measurements to take next. The fourth studies what happens when the test data drift away from the training prior, including a diagnostic for detecting the drift and an attempt to fix it by revising the prior.
Critical strengths
- The paper's strongest point is about joint uncertainty. In the exact-Bayes experiment, ORIA models with covariance rank 8, 9 and 16 all reach about 91% pointwise coverage. Yet their joint KL to the true posterior differs by about two orders of magnitude (Figure 1e). The FEM experiments show why this matters in practice. If you keep ORIA's predictions but drop the off-diagonal covariance entries, coverage of spatial averages falls from about 0.90 to about 0.63, while pointwise coverage stays the same (Table 10). This is a clean, convincing demonstration. Anyone who reports only pointwise coverage for function-valued predictions should take note.
- Another strength is how openly the authors report their results. The appendices spell out which cohorts are paired and which are not. They also say the public NS block was reused while revising the prior, and they report results that hurt their own story. Examples are the 71% coverage on NS and the intervals that get narrower under an unseen nonlinearity. This made the paper much easier to judge fairly, and I think the joint-uncertainty finding alone is worth publishing. My concerns are about how far the rest of the evidence supports the paper's larger claims.
Critical weaknesses
- My main concern is that the "Bayesian operator inference" claim has only been checked in the easiest possible case. The only comparison against an exact posterior uses a linear-Gaussian prior with nine modes. In that case the true posterior predictive is itself a low-rank Gaussian, which is exactly the shape ORIA's output head produces. For every other experiment, the true posterior is non-Gaussian, and the paper never checks how close ORIA gets to it. The fidelity results therefore can't be extended to the physical systems.
- Even in the easy case, the model stops contracting its posterior beyond the number of demonstrations it was trained on. Trained with up to 8 demonstrations, its KL at 16 demonstrations is about 50 times its KL at 1, and its intervals are about 35% wider than the exact ones (Tables 1 and 2). Appendix F suggests a likely reason. The inter-function attention normalizes over the demonstrations, so copying every demonstration leaves the prediction unchanged. A true posterior would contract if those copies were new, independent measurements. The architecture therefore can't represent the Bayesian posterior in general. The paper mentions this but doesn't connect it to the contraction failure or try to fix it. On the technical side, the method mostly combines existing parts: PFN-style training, continuum attention and the low-rank Gaussian output of TNP-ND and related models. That is fine, but it means the evidence has to carry the paper, and the Bayesian part of that evidence is thin.
- The accuracy comparisons are weaker than the headline numbers suggest. The "order of magnitude" gains mostly come from baselines that never see the training prior, such as per-episode FNOs fit to 8 pairs, spectral fitting and the context mean. That shows the prior helps, which isn't surprising. The baselines that do use the prior look under-tuned. Adapted TNP-ND gets worse on cooling as the number of demonstrations goes from 4 to 8 (0.32 to 0.46). It also barely improves on the cavity problem, where its error is still around 0.81 at 8 demonstrations. Pretrained FNO with fine-tuning gets worse with more demonstrations on conduction (0.73 to 1.29). Behavior like this usually points to a bug or a poorly tuned recipe, not a fair reference point. TNP-ND also got only four 10k-step pilot runs to select its configuration.
- Several baselines were evaluated on separate archived cohorts with a different ORIA checkpoint, some with only 12 to 21 episodes. The closest methods are also missing entirely. ICON, GenICON and Zebra are discussed in related work, and GenICON produces probabilistic predictions too, but none of them appears in the experiments.
- The NS test is narrower than it looks as well. All episodes come from one fixed viscosity and forcing, so this tests transfer to a single operator, not inference of unseen operators. At one demonstration, FNO (0.300) and Poseidon-B (0.313) both beat ORIA (0.367). The prior revision that fixes ORIA's NS coverage was tuned on the same reused block it is evaluated on.
- The measurement-selection result doesn't use the joint covariance on the physical problems. The FEM experiments pick points by marginal variance, and against a simple diversity rule the conduction gain is 2.6%, with an interval that includes zero.
- I'd tone down the claim that ORIA points toward models "that know what they do not know." The 71% coverage on NS and the narrower intervals under an unseen nonlinearity suggest otherwise. Running the joint selection rule on the FEM problems would make the measurement-selection section match its framing. The paper uses five separately trained models (linear-Gaussian, 1D grammar, FEM, 2D physics-mixture and NS), and the main text switches between them quickly. A small table listing each model, its prior and where it is used would help a lot.
Questions for authors
- Can you check posterior fidelity on a nonlinear prior where a reference posterior can still be computed, for example with MCMC or importance sampling on a low-dimensional version of the 1D grammar prior? This would show whether the Bayesian claim holds when the true posterior is not Gaussian.
- Does a fix for the replication invariance, such as a context-size feature or unnormalized aggregation over demonstrations, stop the KL from growing between 8 and 16 demonstrations?
- Why are ICON, GenICON and Zebra missing from the experiments? GenICON in particular seems like the most direct comparison for probabilistic in-context operator prediction.
- Can you explain why TNP-ND and the pretrained FNO get worse with more demonstrations? Could you rerun all baselines on the shared test bank with the same ORIA checkpoints and a tuning budget closer to ORIA's?
- Can you test on data where the operator actually varies across episodes, for example NS with different viscosity or forcing? Can you also evaluate the revised priors on a block of data that hasn't been used before?