acceptodds
Under review as a conference paper at ICLR 2027

Neural Operator Processes for Probabilistic Operator Learning under Partial Observations

Abstract

Neural operators learn mappings between function spaces, but are typically developed with dense input–output training fields and fully observed inputs at inference. Many scientific problems require instead predicting solution fields from sparse, irregular, or partial observations under uncertainty. We introduce Neural Operator Processes (NOPs), a framework that unifies neural-process conditioning with neural-operator decoding to predict full output fields from limited context. NOPs condition on sparse joint input–output observations and support deterministic and probabilistic prediction within a shared encoder–decoder architecture. We study two conditioning strategies, convolutional pooled summaries and query-aligned attention, and analyze how their interaction with latent stochastic variables depends on PDE geometry. Across function regression and three PDE benchmarks, we find that sparse conditional operator learning is viable and can match dense-grid behavior in several regimes, that preserving local context-query geometry is essential in non-periodic settings but less so in spectrally smooth periodic regimes, and that uncertainty-aware operator learning succeeds when latent conditioning complements rather than overwrites the local geometric pathway. These results provide a basis for probabilistic operator learning under partial observations and help bridge operator learning and probabilistic meta-learning in function space.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, neural processes, operator learning, uncertainty quantification, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_BcGjOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Neural Operator Processes (NOPs) for predicting full PDE solution fields using complete input fields and sparse output observations. It combines Neural Processes with Neural Operators, using SetConv and attention to handle sparse observations. It also adds latent variables to estimate uncertainty. The results show that this approach works reasonably well on several PDE benchmarks, but the improvements over simpler methods are not always clear. Also, the sparse observations are sampled from simulation data, not real sensors. So it is still unclear how well the method would work with noisy measurements or more complex PDE problems. The experiments also do not clearly show the benefits of probabilistic modeling.

Critical strengths

The experiments provide some useful insights into how different PDE problems respond to sparse observation conditioning. In particular, they show that the choice between SetConv and attention matters depending on the PDE. The paper also explores how these methods interact with probabilistic latent variables. These findings help better understanding the role of different model components and their trade-offs.

Critical weaknesses

I'm not sure this really counts as sparse-to-dense operator learning. The model still sees the full input field, and training uses full-field supervision. Only the output observations given at test time are sparse. These observations are also sampled directly from simulation results, rather than collected from real sensors. So the experiments are mostly about using sparse observations under ideal conditions. It's still unclear how the method would handle noisy measurements, sensor errors, or differences between simulations and real-world data.

Some NOP variants perform better than standard FNO, but the comparison is not entirely fair since NOP also gets to see part of the true output. The paper includes some baselines with the same observations, but the gains are not always clear.

The probabilistic models can estimate uncertainty, but they seem to lose quite a bit of accuracy. On Darcy, for example, the error goes from 0.0226 for the best deterministic model to 0.0657 for the best probabilistic one. Cross-resolution generalization is another concern. Burgers works reasonably well, but the results on Darcy and Navier–Stokes are much worse. On Darcy, DNOP's error jumps from 0.0250 to 0.3316. Most experiments are also done on regular grids, so we don't know how well the method would work on complex geometries, unstructured meshes, or more challenging turbulent flows.

Questions for authors

I'm wondering how much NOP actually improves over a standard FNO that has access to the same sparse observations. Is the proposed architecture really making a difference, or is most of the gain simply coming from the extra information?

I'm also not sure how much the probabilistic part really helps. The prediction accuracy gets noticeably worse, especially on Darcy. Are the uncertainty estimates useful enough to justify this trade-off, particularly when the observations are noisy or the PDE problems become more challenging?

2 comments

Thank you for your review. Please find our responses below:

  • We should notice three distinct properties: sparsity of test-time response observations, availability of the operator input, and density of training supervision. Our problem is explicitly defined in Section 3.1 as conditional operator prediction from a fully available input field a_Q and sparse response context C={(x_i,a_i,u_i)}_{i=1}^{n_C}. The objective is to predict the full response field U_Q. Dense training supervision does not imply access to unobserved response values at inference. This constitutes a sparse-to-dense response-conditioning problem. Extending NOP to jointly sparse input and response observations would require handling partial operator-input information alongside sparse response conditioning. This introduces additional representation and inference challenges and constitutes a natural, more demanding extension. We already consider this a meaningful extension of the proposed framework rather than a prerequisite for the conditional operator-learning problem studied here and will clarify in the Discussion section. Sampling context observations from held-out simulated solutions implements the stated inference protocol without target leakage. Robustness to sensor noise, irregular geometries, and simulation-to-reality transfer remains an important direction for extending the framework beyond controlled PDE benchmarks.
  • The concern that improvements over FNO arise entirely from access to additional response observations is directly addressed in Appendix C.6, Table 19. Masked-FNO receives the same complete operator input, sparse response values, and observation locations as NOP. Comparisons use identical held-out tasks and context realizations. On Darcy, geometry-matched Masked-FNO achieves rel-L^2=0.02429, compared with 0.02255 for DANOP (95% CI [0.00135,0.00219]). On Navier–Stokes, Masked-FNO achieves 0.03389, compared with 0.02840 for DNOP (95% CI [0.00414,0.00689]). On Burgers, Masked-FNO and DANOP are statistically indistinguishable. This indicates that learned conditioning provides benefits beyond additional response information in selected PDE regimes, while identifying cases where direct rasterization is sufficient. The standard FNO comparison in Table 2 provides a conventional operator-learning reference; Table 19 establishes the architectural comparison under matched information.
  • The probabilistic variants can sacrifice point-prediction accuracy, particularly on Darcy. However, probabilistic operator learning targets predictive-distribution quality and uncertainty quantification in addition to reconstruction accuracy. Accuracy (mainly measured here with relative L^2) alone therefore does not capture the full objective. In Appendix B.5, Table 10 directly evaluates these additional objectives against a deterministic ensemble with validation-calibrated residual variance. PNOP-HE and PANOP-HE achieve significantly lower NLL and CRPS across all three GP families, with paired bootstrap confidence intervals excluding zero. These improvements occur without consistently improving predictive means, demonstrating that probabilistic modeling provides benefits beyond point accuracy in this setting. For PDEs, the probabilistic variants produce full-field predictive distributions with benchmark-dependent calibration. The results do not establish superiority over context-matched probabilistic FNOs or ensembles, nor uniformly nominal coverage. The observed accuracy–uncertainty trade-off is therefore a property of the evaluated architectures, and the probabilistic objective should not be assessed solely through deterministic reconstruction error.
  • Appendix C.4 explicitly documents the 2D transfer failures, including Darcy DNOP's increase from 0.0250 to 0.3316. We clearly indicate the limitated discretization-transfer capability, but the results do not invalidate the operator-learning formulation. Parameter sharing across resolutions, including through FNO layers, does not guarantee resolution-independent prediction accuracy for a learned conditional operator. Appendix C.6 further establishes decoder-family dependence: local convolutional decoding improves results on Darcy but performs substantially worse on Burgers and Navier–Stokes. This indicates that the interaction between sparse conditioning and decoder inductive bias is relevant, although the available experiments do not fully isolate the cause of cross-resolution degradation. The benchmark suite examines distinct PDE geometries and dynamics under controlled protocols. Evaluation on irregular meshes, noisy physical sensors, and more turbulent regimes would extend the empirical scope, but the current experiments do not establish performance in those settings.

Thanks for the response. Overall, the authors have clarified several points, but my two main concerns remain.

First, I am still not convinced of the benefits of probabilistic modeling for PDE problems. Comparing Tables 2 and 3, the best probabilistic models have roughly 89%, 191%, and 69% higher prediction errors than the best deterministic models on the three PDE benchmarks, respectively. I would be fine with this trade-off if the probabilistic models provided better uncertainty estimates than alternative methods, or showed better robustness to noisy observations. However, the authors do not provide such evidence. As they acknowledge in their response:

The results do not establish superiority over context-matched probabilistic FNOs or ensembles, nor uniformly nominal coverage.

The strongest evidence for probabilistic modeling still comes from GP regression rather than PDE problems.

There is also a more fundamental question. These PDEs are deterministic, and the full input field is already known. Assuming the PDE parameters and boundary conditions are fixed, what exactly is the latent variable modeling? Is it uncertainty due to limited training data, model approximation errors, or uncertainty in the physical system itself? I think this distinction is important and deserves more discussion.

Second, I remain unconvinced by the architectural novelty. I acknowledge that NOP performs better than Masked-FNO on some benchmarks. However, the best architecture varies across PDEs. Darcy favors SetConv + Attention, Navier–Stokes favors SetConv alone, and Burgers shows little benefit from either over direct masked-grid conditioning.

Moreover, SetConv, cross-attention, and FNO are all existing methods. Showing that their combination works well in some cases does not necessarily establish a strong architectural contribution. Interestingly, in Table 21, the simpler ConvCNP-style model achieves an error of 0.016389 on Darcy, compared with 0.022549 for DANOP.

To me, these results mainly suggest that different PDEs benefit from different conditioning and decoding mechanisms. This is a useful empirical observation, but it does not make a strong case for the proposed architecture itself.

Overall, I appreciate the clarifications, but my concerns about novelty and the experimental support for probabilistic operator learning remain. I therefore maintain my original rating.

Reviewer_QEuWOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper seeks to predict the full PDE solution field from a known input field a(x)a(x) and a sparse variable set of joint observations, which is given by (x,a(x),u(x))(x, a(x), u(x)) observations with optional uncertainty. The method combines a SetConv context encoder, an optinonal query-aligned attention, and an FNO decoder. A global Gaussian latent and a Gaussian output head can also be combined for probabilistic variants. The experimental setting supplies the operator input over the entire query grid and observes the response at a sparse subset of locations. They test out this framework on different benchmark problems (that were generated by the authors): GP regression, Burgers, Darcy and NS. Their results seem to match or slightly beat standard FNOs.

Critical strengths
  • The paper seems well motivated and clearly explains the problem setting that they're seeking to tackle
  • The formulation of the problem to incorporate arbitrary context points makes sense.
  • One aspect which seems particularly strong is that the results show robustness to sensor layouts when attention is incorporated
  • Good reporting of results where it does fall short with rather extensive experiments reported in the Appendix.
Critical weaknesses
  • Current benchmark suite seems rather narrow. While I think the problems are good (GP regression, Darcy, NS) are relevant, it seems restricted to rather low resolutions and small dataset sizes / easier regimes (low Re). Further, it is entirely self-generated.
    • For a mesh-free encoder motivated by sparse sensing and data assimilation, the absence of irregular geometry and real sensor data seems like a gap.
    • It seems like the problems which are considered or its variants are also available in standard suites such as PDEBench or TheWell, and they come with standard reference baselines. It would be helpful to have this in place because then there is an extensive literature of results which are available, which makes comparison a little bit more fair. While sparse sampled results may not be standard, subsampling contexts from established datasets would make the results comparable to published numbers.
    • The cost profile (as outlined in the appendix) suggests harder problems are feasible
  • Seems like the model isn't discreization-invariant in 2D as mentioned the paper (NS and Darcy seem to spike in error). Doesn't this go against the fact that you're learning a model in function space? Can you possibly clarify? FNO layers should be resolution-invariant on their own.
  • Not a queryable field model: outputs are locked to a fixed grid, and the input a(x)a(x) must be dense. SetConv is evaluable at arbitrary query locations but it is only evaluated on the query grid, the FNO operates on that grid, and predictions are always made over a dense query grid. For a model motivated by sparse sensing, the natural use is to predict the field (and its uncertainty) at any location. Query-based decoders that do this exist (e.g., cross-attention or coordinate-conditioned decoders as in CViT [Wang et al., ICLR 2025], AROMA [Serrano et al., NeurIPS 2024], or GINO [Li et al., NeurIPS 2023]). Doesn't this mean that this means the model currently learns sparse conditioning on one grid rather than an operator in the function-space sense?
    • My mental model comes from say something like a Darcy problem where we may not have permeability across all locations (even this is sparsely informed samples), but essentially we want pressure at some arbitrary queryable location. It doesn't seem like this framework can run that, can it?
  • Probablistic extensions seem to show little value: It seems like probabilistic variants are consistently worse than the deterministic ones and than the context-free FNO that never sees uu. While acknowledged, in its current form a user gets error bars at the price of a worse mean than a baseline that ignores the context.
    • Further, it doesn't seem like there's PDE UQ baselines. Can you potentially look into doing this? Compare against standard options that have been coming out in the space (ensembles, mc-dropout, other probablistic NOs)
  • Extra test-time context doesn't help and sometimes hurts. Can you explain this? Shouldn't it be strictly better?
Questions for authors

Please see weaknesses and address the points accordingly

2 comments

Thank you for your review. Please find our responses below:

  • The experiments use common controlled benchmark families for both NPs and NOs to isolate the effects of sparse response conditioning, observation geometry, latent inference, and decoder architecture. The 1,000/200/200 task splits and moderate resolutions support these comparisons and exploration of components contributions but do not establish scalability to more complex physical regimes as indicated in limitations. PDEBench and The Well would provide valuable external validation but direct comparison with published errors requires matching the underlying PDE distributions, input–output mappings, discretizations, and evaluation protocols. Sparse response-conditioned evaluation would additionally require constructing comparable context sets. We therefore distinguish the controlled architectural evidence established here from broader generalization to public benchmarks or real sensor data.
  • The distinction between function-space operator learning and arbitrary-location decoding is important. An operator maps between function spaces, learning its approximation does not require independently queryable outputs at arbitrary coordinates. FNO itself operates on structured discretizations of functional inputs and outputs. In NOP we extend this through conditional inference, \widehat U_Q = f_\theta(Q,C). where C is a variable-size set of joint input–response observations. The SetConv representation can be evaluated at arbitrary coordinates, but the FNO decoder currently operates on a structured query grid. Thus, our PDE implementation does not support independent off-grid output queries without constructing an appropriate grid representation. It also assumes the operator input is available over the prediction domain. Appendix C.3 further quantifies the benefit of geometry-aware conditioning under sensor-layout shift. When trained on uniform contexts and evaluated on clustered observations, DANOP reduces rel-(L^2) on Darcy and NS relative to DNOP. Thus, there are relevant architectural improvements in robustness under the same context-budget and sampling protocols. We consider that these are architectural scope conditions, not contradictions of the operator-learning formulation. Arbitrary-query decoding and partially observed operator inputs are distinct extensions to the proposed framework that could be incorporated using alternative decoder and input-representation mechanisms.
  • The ability to evaluate FNO layers across resolutions does not guarantee resolution-independent prediction accuracy. The learned conditional operator additionally depends on context-to-function lifting, input projections, discretized field representations, and their interaction with the decoder. Appendix C.4 directly evaluates this distinction using paired coarse/fine fields. DANOP retains rel-L^2 approx 0.0048 under Burgers (256 to 512) transfer, whereas Darcy and NS increases. Increasing the fine-grid context budget does not resolve these failures with the current components/framework. Appendix C.6 provides further evidence that decoder choice matters: local convolutional decoders outperform FNO-based decoding on Darcy, while FNO remains stronger on Burgers and NS. Thus, we make no claim of universal discretization invariance and state it in limitations and further extensions.
  • The probabilistic variants generally trade some deterministic point accuracy for predictive-distribution modeling. Their contribution must consequently be evaluated using distributional metrics alongside prediction error. Table 10 in Appendix B.5 provides a quantitative comparison against a validation-calibrated deterministic ensemble. PNOP-HE and PANOP-HE achieve significantly lower NLL and CRPS across all three GP families, with paired bootstrap confidence intervals excluding zero. These improvements are not attributable to uniformly better predictive means. For PDEs, the probabilistic models provide conditional predictive distributions, but point accuracy is generally lower than the strongest deterministic variants and nominal coverage is not uniformly achieved.
  • More observations do not guarantee strictly lower realized prediction error. For an exact Bayes predictor under squared-error loss, conditioning on additional information cannot increase expected Bayes risk when the information sets are nested. This does not imply strict improvement for individual predictions, nor does it establish monotonicity for an approximate learned predictor evaluated with relative L^2. Appendix C.2 evaluates fixed checkpoints trained under a base context distribution while varying the test-time context budget. Such changes modify the conditioning distribution without updating the learned representation. The results exhibit that Burgers DNOP degrades when moving from base to high test-time context, whereas DANOP remains nearly unchanged. Similar stability is observed for geometry-aware variants in other settings.

Thank you for your response. I have raised my score to reflect the clarifications, but my main concerns remain.

Reviewer_HyawOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper combines neural-process conditioning with an FNO decoder. Sparse response measurements help predict the full output field. The input field stays fully observed, and training still uses dense output labels. A shared Gaussian latent adds probabilistic predictions, tested on GP regression and three PDE benchmarks. I see a workable approach, but its advantages depend heavily on the benchmark, and the 2D uncertainty results leave me unconvinced.

Critical strengths
  • I appreciate the controlled appendix experiments. Matching response observations, boundary features, and decoder capacity makes the comparisons much more informative. The authors also show substantial failures instead of hiding them.

  • The GP uncertainty results give me something solid. PNOP and PANOP beat the validation-calibrated DNOP ensemble on NLL and CRPS across all three GP families in Table 10. That supports the probabilistic extension in this setting.

Critical weaknesses
  • The headline FNO comparison gives NOP extra test-time response measurements. Both methods already receive full input fields and dense training supervision. I’d put much more weight on the matched controls.

  • PDE calibration is uneven even without distribution shift. Table 3 reports 99.8% coverage on Darcy and 90.2% on Navier–Stokes for nominal 95% intervals in their strongest variants. That’s too uneven for me to call the uncertainty dependable.

  • Generalization remains brittle. More test-time response measurements often bring little benefit, and finer 2D grids wreck both accuracy and coverage. That seriously limits the practical value of the operator-learning setup.

Questions for authors
  • What survives against a context-matched FNO ensemble or probabilistic FNO? I’d need PDE NLL, CRPS, calibration curves, and prediction error under comparable tuning budgets.

  • How much does the model actually use the observed response values? Please remove or shuffle those values while keeping the input field and sensor locations fixed.

  • Are the PDE observations noiseless, and what does the conditional variance represent? I need that clarified before accepting the “epistemic” and “aleatoric” labels in the figures.

  • Why does paired 2D resolution transfer fail so badly? Would a context-matched FNO fail by the same amount, or does the NOP conditioning introduce an additional problem?

1 comment

Hi, thank you for your thoughtful review. Please find our responses below:

  • Information-matched comparisons. Appendix C.6 (Table 19) directly controls for the additional response information. Masked-FNO receives the same full input field, sparse response values, and observation locations as NOP, with identical evaluation tasks and context realizations. On Darcy, DANOP achieves rel-L2=0.02255\mathrm{rel}\text{-}L^2 = 0.02255, compared with 0.024290.02429 for geometry-matched Masked-FNO (95% CI [0.00135,0.00219][0.00135, 0.00219]). On Navier–Stokes, DNOP achieves 0.028400.02840 versus 0.033890.03389 for Masked-FNO (95% CI [0.00414,0.00689][0.00414, 0.00689]). On Burgers, DANOP and Masked-FNO are statistically indistinguishable. This show benefits from learned conditioning beyond direct response-channel injection in Darcy and Navier–Stokes, while showing that the advantage is more problem-dependent. Table 2 provides the conventional FNO reference; Table 19 isolates the architectural contribution under matched information.

  • Probabilistic performance and calibration. As shown in the UQ comparison of Table 10, PNOP-HE and PANOP-HE significantly outperform a validation-calibrated deterministic ensemble across all three GP families, with paired bootstrap confidence intervals excluding zero. Also, it is true that PDE calibration is less consistent, as noted with the mentioned coverage numbers of 99.8% on Darcy and 90.2% on Navier–Stokes. As indicated, the probabilistic prediction showed benchmark-dependent calibration during ablations, not uniformly calibrated uncertainty. A context-matched probabilistic FNO comparison on PDEs is not included, and deciding on the strategy for the probabilistic FNO is a relevant question, thus, we do not claim superiority over that class of models.

  • Conditioning and uncertainty interpretation. The context-budget and sensor-geometry ablations demonstrate sensitivity to context availability and placement, but do not isolate the contribution of response values (uCu_C) independently of sensor locations. Am additional ablation using response-value permutation control would address that distinct question.

The PDE experiments use simulated response fields without an explicit measurement-noise model. Our uncertainty decomposition is

Var⁡(Uj∣Q,C)=Var⁡z∼pθ(z∣C)[μj(z)]⏟Latent component+Ez∼pθ(z∣C)[σj2(z)]⏟Conditional likelihood component.\begin{aligned} \operatorname{Var}(U_j \mid Q,C) &= \underbrace{ \operatorname{Var}_{z \sim p_\theta(z \mid C)} \left[\mu_j(z)\right] }_{\text{Latent component}}\\ &\quad+ \underbrace{ \mathbb{E}_{z \sim p_\theta(z \mid C)} \left[\sigma_j^2(z)\right] }_{\text{Conditional likelihood component}}. \end{aligned}

The epistemic component denotes variability across global-latent samples; the aleatoric component denotes conditional likelihood variance. These are operational model components, not an identified decomposition into physical epistemic uncertainty and irreducible measurement noise. In particular, conditional variance may absorb residual model error.

  • Cross-resolution transfer. Appendix C.4 shows successful Burgers transfer but substantial deterioration on the 2D benchmarks. For DANOP, coarse-to-fine rel-L2\mathrm{rel}\text{-}L^2 increases on Darcy and Navier–Stokes. Probabilistic coverage deteriorates correspondingly. Although the available additional experiments do not completely isolate whether this failure originates in the conditional representation, the FNO decoder, or their interaction, in Appendix C.6 we provide relevant evidence of decoder sensitivity: replacing FNO with a local convolutional decoder improves Darcy prediction but substantially worsens Burgers and Navier–Stokes. We therefore interpret the 2D transfer failures as limitations of the evaluated conditional operator architectures, without attributing them exclusively to sparse conditioning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.