acceptodds
Under review as a conference paper at ICLR 2027

Debiased Neural Operators

Abstract

Neural operators are widely used to approximate solution maps of complex physical systems. In many applications, however, the goal is not to recover the full solution trajectory, but to summarize the solution trajectory via a scalar target quantity (e.g., a functional such as time spent in a target range, time above a threshold, accumulated cost, or total energy). In this paper, we introduce DOPE (debiased neural operator): a semiparametric estimator for such target quantities of solution trajectories obtained from neural operators. DOPE is broadly applicable to settings with both partial and irregular observations and can be combined with arbitrary neural operator architectures. We make three main contributions. (1) We show that, in contrast to DOPE, naive plug-in estimation can suffer from first-order bias. (2) To address this, we derive a novel one-step, Neyman-orthogonal estimator that treats the neural operator as a high-dimensional nuisance mapping between function spaces, and removes the leading bias term. For this, DOPE uses a weighting mechanism that simultaneously accounts for irregular observation designs and for how sensitive the target quantity is to perturbations of the underlying trajectory. (3) To learn the weights, we extend automatic debiased machine learning to operator-valued nuisances via Riesz regression. We demonstrate the benefits of DOPE across various numerical experiments.

Primary area:
causal reasoning
Keywords:
neural operators, functionals, debiasing, autoDML, Riesz representer, irregular sampling, partial differential equations

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_4tBYOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the problem of estimating low-dimensional scalar functionals of solution trajectories mapped by neural operators under partial and irregular observations. It demonstrates that standard plug-in estimators suffer from a severe systematic first-order bias because approximation errors propagate nonlinearly through the functional. To overcome this, the authors introduce DOPE (Debiased Neural Operator), a semiparametric framework that constructs a one-step Neyman-orthogonal estimator using a weighting mechanism based on Riesz representers. Theoretical estimate and numerous experiments are performed.

Critical strengths

Extending automatic debiased machine learning from function-valued nuisances to operator-valued nuisances via Riesz regression and autodiff-based Jacobian-vector products is a good, non-trivial methodological contribution.

Critical weaknesses
  1. Training the extra operator-valued nuisance (β^\hat{\beta}) via Riesz regression requires solving a separate optimization problem alongside the primary neural operator surrogate, which increases training complexity and compute costs.

  2. While DOPE relaxes the convergence rate requirement on the neural operator, the validity of the estimator still relies heavily on the design overlap assumption (Assumption 3.1) and the accurate estimation of both nuisance components. In data-sparse or extremely irregular sampling regimes, estimating the debiasing weight operator stably can be challenging.

Questions for authors
  1. How sensitive is the performance of DOPE to the choice of the regularization parameter λ\lambda in the Riesz regression objective? Is cross-validation practical for tuning λ\lambda across different PDE/operator systems?

  2. Under extreme observation sparsity or heavy adaptive sampling concentration (e.g., large ρ\rho), does the estimation variance of β^\hat{\beta} increase significantly, and how does this affect the empirical coverage of the CLT-based confidence intervals?

  3. After learning β^\hat{\beta} through Riesz regression, if it is very unsmooth or extrapolated (when ρ\rhois large), is regularization necessary? How does regularization affect orthogonality?

4 comments

Thank you very much for your helpful review of our paper! We improved our paper as a result of your comment, and highlighted all changes in blue color in our revised rebuttal PDF.


Responses to Weaknesses:

W1: Computational complexity

Thank you. While DOPE requires fitting an additional nuisance operator in the general case, we emphasize that this introduces only constant-factor training overhead, and does not change the asymptotic computational order. Specifically, the Riesz representer is learned using the same neural operator architecture as the solution operator, and the optimization requires only standard automatic differentiation via Jacobian–vector products (see Supplement F). Importantly, no additional PDE solves or observations are required.

Moreover, the additional optimization is not always necessary. By Proposition 4.7, the debiasing weight decomposes as β(A)(x)=ξ(A)(x)wg(S(A))(x)\beta(A)(x)=\xi(A)(x)w_g(S(A))(x). When the sampling design and functional derivative are known, we can directly construct β(A)(x)\beta(A)(x), which eliminates the need for a separate Riesz regression entirely. Hence, the automatic Riesz regression provides flexibility for settings where such closed-form constructions are unavailable.

Importantly, this modest additional cost enables reduction of bias by an order of magnitude, as demonstrated by our theoretical results and experiments.

Finally, training separate nuisance models is standard practice in double/debiased machine learning [1,2,3], where auxiliary nuisance components are estimated before constructing the Neyman-orthogonal estimator. Therefore, it is not an overhead specific to our method.

Action: We highlight the modest computational overhead in our revised Section 4, and emphasize that no PDE solves or similar are required.


W2a: Overlap

Thank you. Design overlap is a fundamental identifiability requirement, not a restriction specific to DOPE.

If a region of the solution domain has positive measure under μ\mu but is never observed under P(⋅∣A)P(\cdot\mid A), even the solution operator S0S_0 itself cannot, in general, be identified on that region from the available observations. Two distinct solution operators can agree on every observed location while differing on unobserved regions.

Hence, no estimator (including conventional plug-in neural-operator estimation) can consistently recover the full solution operator or arbitrary functionals depending on these regions without additional structural assumptions. Extremely weak overlap may increase variance, but this is a general statistical limitation of inference from sparse measurements, not a failure of Neyman orthogonality.

Action: In our revised Assumption 3.1, we clarify that overlap is fundamental for the identifiability and stable estimation of the underlying solution operator and its functionals, and is not specific to our debiased estimator.


W2b: Nuisance errors

Thank you for asking this question!

It is exactly the contrary:

Robustness to nuisance estimation errors is precisely the central contribution of DOPE. It is not the case that DOPE requires highly accurate nuisance estimates. Instead, our Neyman-orthogonal score is exactly constructed to eliminate their first-order effect on the function estimation bias (Proposition B.3). By Theorem 4.3, the remaining bias satisfies

∣Bias(θ^)∣≲∣S^−S0∣L2(P)2|\mathrm{Bias}(\widehat\theta)| \lesssim |\widehat S-S_0|_{L^2(P)}^2

+∣β^−β0∣L2(P)2.+|\widehat\beta-\beta_0|_{L^2(P)}^2.

In contrast, the plug-in estimator generally incurs first-order bias in the neural operator error (Proposition 4.1). Consequently, DOPE permits substantially slower nuisance convergence rates (e.g., op(n−1/4)o_p(n^{-1/4}) for both nuisances), and not the op(n−1/2)o_p(n^{-1/2}) sufficient rate required for plug-in estimation.

Schematically, if for example the solution operator or the correction weight has error δ<1\delta<1, it only enters at order O(δ2)O(\delta^2) into the final estimate for DOPE, while the former stays at order O(δ)O(\delta) for the plug-in estimator.

This is also directly validated by our Figure 3, where DOPE exhibits quadratic instead of linear sensitivity to nuisance perturbations.

Thus, DOPE explicitly reduces sensitivity to nuisance estimation errors.

Action: In our revised Section 4.3, we highlight that robustness to nuisance errors is exactly the property of our proposed method.

Responses to Questions:

Q1: Riesz regularization and cross-validation

Thank you! This is correct, the regularization parameter λ\lambda can be selected using standard cross-validation, as in other neural estimation methods. Importantly, our Riesz regression formulation provides a natural validation objective. We do not need access to the unknown Riesz representer β0\beta_0.

Specifically, by Theorem 4.8, the population Riesz risk satisfies

R(β)\mathcal R(\beta)

=E[β(A)(X)2−2DgS0(A)(β(A))]= \mathbb E\left[\beta(A)(X)^2 - 2Dg_{S_0(A)}(\beta(A))\right]

=E[β(A)(X)2−2β0(A)(X)β(A)(X)]= \mathbb E\left[\beta(A)(X)^2 - 2\beta_0(A)(X)\beta(A)(X)\right]

=∥β−β0∥L2(P)2= \| \beta-\beta_0 \|_{L^2(P)}^2

−∥β0∥L2(P)2- \| \beta_0 \|_{L^2(P)}^2

where the latter term is constant w.r.t. β\beta.

Hence, minimizing the Riesz risk is equivalent to minimizing the L2L^2 estimation error of the debiasing weights. We can therefore select λ\lambda by evaluating the empirical Riesz objective on held-out validation samples. This requires neither oracle Riesz weights nor additional PDE solves and is applicable to any PDE system.

Importantly, cross-validation can be performed entirely within the training folds of our cross-fitting procedure. The resulting estimator retains the Neyman-orthogonal score construction.

Finally, λ\lambda controls the usual regularization trade-off: stronger penalties can stabilize weight estimation under sparse observations, whereas excessive regularization may introduce approximation bias. Our framework does not require a particular fixed value of λ\lambda, and it can be tuned separately for each application using the same Riesz objective.

Action: In our revised Section 4.4, we highlight that cross-fitting can be used for tuning of λ\lambda, and show this in our Algorithms 1 and 2. Further, we show an empirical example of selecting λ\lambda in our new Supplement D.4.


Q2: Irregular observations

Thank you. We have additionally conducted a new variance analysis across 50 independent seeds in our Supplement D.2. Further, our existing experiments already investigate how increasingly concentrated observation designs affect estimation accuracy and confidence-interval coverage. We would like to address your concern with three points:

(i) New analysis of variance stability: To address your question directly, we additionally analyzed the estimated variance of the debiased score across all nine sampling regimes from our main experiments in our new experimental results in Supplement D.4. From ρ=0\rho=0 to ρ=1\rho=1, the score variance increases by only 0.28% for AUC, 0.85% for smooth TAT, and 3.57% for soft Cmax. Across the entire ρ\rho grid, variance fluctuations remain within 3.6–5.5% of the uniform-sampling value, with no systematic increasing trend.

Yet, the RMSE of the learned Riesz representer β^\hat \beta increases by 109–153% under the most concentrated sampling design. Thus, although estimating the debiasing weights becomes more challenging, these additional nuisance-estimation errors do not translate into increases in the variance of the final estimator. This is because DOPE is robust against first order error terms even in β\beta, not only SS. We emphasize that Riesz RMSE measures estimation error relative to the oracle, rather than the training-induced variance of β^\hat \beta itself.

(ii) Robustness under irregular sampling: In our main experiments (Table 1), we systematically vary the sampling-irregularity parameter ρ\rho across nine levels, from uniform sampling (ρ=0\rho=0) to strongly concentrated sampling (ρ=1\rho=1). DOPE maintains stable estimation accuracy and consistently outperforms plug-in estimation. For example, the AUC RMSE changes only from 0.00720.0072 to 0.00750.0075. Our Darcy-flow experiments (Table 3) further demonstrate improvements under adaptive observation designs.

(iii) Confidence-interval validity: Our existing CLT experiments (Supplement D.2) confirm this stability. Empirical coverage of nominal 95% intervals remains between 90% and 96% for AUC and smooth TAT, and between 90% and 98% for soft Cmax, without systematic deterioration as ρ\rho increases. Between ρ=0\rho=0 and ρ=1\rho=1, mean confidence-interval lengths increase by just 0.08%, 0.39%, and 1.47%, respectively. In contrast, plug-in confidence intervals exhibit substantially poorer coverage (68–88%).

=> Theoretical explanation: Neyman orthogonality makes the estimator first-order insensitive to nuisance-estimation errors. In particular, if S^=S0\hat S = S_0, the correction has zero expectation regardless of the accuracy of β^\hat \beta; more generally, errors in β^\hat \beta enter the bias only at second order. This explains why substantial Riesz-estimation errors need not produce comparable deterioration in the final estimator.

Taken together, our new variance analysis and existing CLT experiments provide direct evidence that DOPE maintains stable inferential performance even as adaptive sampling concentration makes nuisance estimation considerably more difficult.

Action: We added the new study to our new Supplement D.2. Therein, we include the new score-variance analysis and figures, and explicitly highlight the confidence-interval results.

Q3: Regularization and orthogonality

Thank you. Regularization affects only the estimation of the Riesz representer, not the construction of the Neyman-orthogonal score for the target functional. Specifically, our debiasing score (Equation 12) and its Neyman-orthogonality property (Proposition B.3) are defined independently of the regularization parameter λ\lambda. Hence, regularizing β^\hat \beta does not alter the debiasing construction or its orthogonality.

Regularization is an optional, standard mechanism for controlling the complexity and variance of the learned Riesz weights, particularly under irregular sampling. Importantly, our theoretical framework explicitly allows estimation errors in both nuisance components, which includes those arising from regularization: by Theorem 4.3, these errors enter the final estimation bias of the target functional only through second-order terms.

Put simply: One can trade a little increase bias in β^\hat \beta for a large reduction in variance of β^\hat \beta. The error in β^\hat \beta is then effectively reduced. Then, the smaller error enters the final stage estimator of our target functional, but its debiasing construction remains completely valid, and benefits from smaller errors in β^\hat \beta. Further, these errors only enter at second order. Thus, regularization can improve numerical stability without modifying the underlying debiasing construction of the target functional.

Action: In our revised Section 4.4, we highlight that regularization does not affect orthogonality w.r.t. nuisance estimates and the debiasing effect of the final estimator.


[1] Dylan J. Foster and Vasilis Syrgkanis. Orthogonal statistical learning. The Annals of Statistics, 53(3): 879–908, 2023.

[2] Victor Chernozhukov, Whitney Newey, Victor Quintas-Martinez, and Vasilis Syrgkanis. RieszNet and ForestRiesz: Automatic debiased machine learning with neural nets and random forests. In ICML, 2022a.

[3] Edward H. Kennedy. Semiparametric doubly robust targeted double machine learning: A review. arXiv preprint, 2203.06469, 2022.

Reviewer_6wfEOpenReview ↗
Rating
3(+2)
Confidence
3(−1)
Informativeness
3
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper proposes a statistical de-biasing method for the estimation of smooth functionals gg (typically of integral type) of function-valued operators. It focuses on the plug-in estimation approach: one first estimates the "true" operator S0S_0 by a neural operator SS, then estimates g∘S0g\circ S_0 by g∘Sg\circ S, where gg is assumed known. First, the authors show that g∘Sg\circ S suffers from a first-order bias of order roughly the error of S−S0S-S_0, which precludes fast estimation of the functional. To instead obtain a leading bias of order the square of the error in S−S0S-S_0, the authors use a Taylor expansion technique to cancel the bias, leading to a method they call "DOPE". This method requires estimating another operator β\beta (a Riesz representer of the derivative map of gg), which they propose to do empirically using neural operators and "Riesz regression''. In all methods proposed, the authors consider estimating the averaged value of g∘Sg\circ S against the data-generating distribution. The whole framework is quite general; it is applicable to any neural operator architecture and irregular function space discretizations and noise models. Numerical experiments on real and synthetic data support the potential of the framework.

Critical strengths

The concept is highly interesting and relevant for operator learning in the sciences and engineering. The extreme flexibility of DOPE is also a major strength and serves to increase its potential impact.

Critical weaknesses

(EDIT: The paper was significantly improved during the rebuttal period)

My main criticism is that as a paper positioned for the operator learning community, the central objective on page 4 equation 8 is not motivated or immediately relevant to this field. This is the main weakness of the paper and why I think the paper is not yet ready for publication. Indeed, θ=EAg(S0(A))∈R\theta=\mathbb{E}_{A}g(S_0(A))\in\mathbb{R} is a double-averaged scalar "semiparametric" quantity. The first averaging comes from gg, and the second from the outer expectation over random function AA. It is highly unclear why the averaged functional θ\theta is relevant for operator learning. All of the motivation in Sec 1 of the paper concerns the nonparametric quantity f=g∘S0 ⁣:a↦g(S0(a))f=g\circ S_0\colon a\mapsto g(S_0(a)). Most use cases of operator learning concern the estimation of ff (i.e., instance-wise computation of drag on a vehicle's surface given the geometry as an input); see the reference [1] below. Further averaging over A=aA=a defeats the instance-wise need for rapid-to-evaluate neural surrogates for amortization. Formulating the paper around estimating θ\theta makes the mathematics easier and closer to the semiparametric inference literature in mathematical statistics, but a much stronger justification should be given for this decision.

The errors that I would expect to want to control are strong-type errors like EEA∣f(A)−f^(A)∣\mathbb{E}\mathbb{E}_A|f(A)-\hat{f}(A)| instead of weak errors like ∣θ−EEAf^(A)∣|\theta-\mathbb{E}\mathbb{E}_A\hat{f}(A)| considered in the paper.

The authors briefly remark on these points in Remark 4.4 page 7. However, the paper would be significantly improved if Remark 4.4 could become the central starting point/objective of the paper. I understand that this significantly disrupts the current structure and results of the paper, which is why I advocate for rejection and resubmission to another venue unless the authors can clearly articulate, with backing from existing literature, why θ\theta is interesting for operator learning.

Similarly, the authors mention that there are no other methods to compare to. However, one can consider the "end-to-end" estimation approach both in theory and in the experiments, where one estimates ff based on samples (Ai,g(S0(Ai))(A_i, g(S_0(A_i)) and directly supervises the learning on these scalar labels. While usually sub-optimal compared to plug-in estimation, this end-to-end approach is considered in the following reference:

[1] Daniel Zhengyu Huang, Nicholas H. Nelsen, and Margaret Trautner. "An operator learning perspective on parameter-to-observable maps." Foundations of Data Science, Vol. 7, No. 1, pp. 163–225, 2025.

Minor: The experiments for the Darcy flow problem in Appendix H.2 uses such a coarse discretization (17 by 17) and only 9 cosine modes that the problem becomes far from infinite-dimensional operator learning.

Questions for authors
  • In Remark 4.4, please explain in more detail what the "first and second stage regressions" are.

  • I would like to see a comparison to the semiparametric de-biasing methods in a long (nearly a decade) line of work by Vladimir Koltchinskii. In particular, comparing the proposed approach to similar methods that can handle Banach-valued nuisance parameters would better position the work. For example: Vladimir Koltchinskii. Minghao Li. "Functional estimation in high-dimensional and infinite-dimensional models." Bernoulli 32 (2) 849 - 873, May 2026. https://doi.org/10.3150/25-BEJ1882

  • Page 2, the first two related work paragraphs are repetitive and likely a result of previous editing not finished.

  • Line 255 on "highly unrealistic" rates for first order root-n error: The following paper shows that for many PDE operator learning problems, the near parametric rate is visible in practice. Do the authors have more concrete evidence to support their claim in Line 255? The Cost-Accuracy Trade-Off in Operator Learning with Neural Networks by de Hoop, Maarten V and Huang, Daniel Zhengyu and Qian, Elizabeth and Stuart, Andrew M. Journal of Machine Learning, Vol 1, No 3, pp. 299--341, 2022.

  • What concrete examples of interest satisfy Assumption 3.1 on design overlap?

  • Does the DOPE framework accommodate the setting in which gg is only partially known in some sense?

  • Thm 4.3: what is the meaning of "cross-fitting"?

11 comments

Thank you very much for your helpful review of our paper! We improved our paper as a result of your comment, and highlighted all changes in blue color in our revised rebuttal PDF.


Responses to Weaknesses:

W1: Target estimand

Thank you. We have (i) extended our framework to explicitly accommodate input-specific estimation, (ii) established its debiasing properties in a new Theorem 4.4, and (iii) added new experimental input-specific results. Importantly, this extension requires only a minor modification of our original procedure. At the same time, we emphasize that population-average functionals are themselves central scientific quantities of interest. We clarify these points below.

(i) Our framework readily extends to input-specific functionals. While our primary estimand is the population-average functional θ0=E[g(S0(A))]\theta_0=\mathbb E[g(S_0(A))], the same framework can also easily accommodate the input-specific target θ0(a)=g(S0(a))\theta_0(a)=g(S_0(a)).

The extension is straightforward: Instead of averaging the debiased pseudo-outcomes across subjects, we perform a second-stage regression (see answer Q1) of their residual corrections on the functional inputs AiA_i. Specifically, we learn the conditional correction r^(A)\hat r(A) and construct

θ^(a)\hat\theta(a)

=θ^plug(a)⏟Plug-in=\underbrace{\hat\theta_{\mathrm{plug}}(a)}_{\text{Plug-in}}

+r^(a)⏟Conditional correction.+\underbrace{\hat r(a)}_{\text{Conditional correction}}.

Here, θ^plug(a)\hat \theta_{\mathrm{plug}}(a) is the original neural-operator plug-in prediction g(S^(a)g(\hat S (a), while r^(a)\hat r (a) estimates its systematic conditional correction. This requires only an additional scalar-output regression, and reuses the same nuisance operators, Riesz construction, and sparse-observation model.

Crucially, no ground-truth functional labels or complete solution fields are required for training. Algorithm 2 in our new Supplement E.2 formalizes this procedure, including cross-fitting.

Further, we update our Objective in our revised Section 3.

(ii) We establish a new theorem proving conditional debiasing. To rigorously justify this extension, we introduce a new Theorem 4.4, Conditional Debiasing for Input-Specific Functionals. It establishes that the corrected conditional functional is Neyman-orthogonal with respect to both nuisance operators. Moreover, the conditional bias of the cross-fitted corrected functional satisfies

∣θS^,β^(a)−θ0(a)∣|\theta_{\hat S,\hat\beta}(a)-\theta_0(a)|

≲∣S^(a)−S0(a)∣L2(P(⋅∣a))2\lesssim |\hat S(a)-S_0(a)|_{L^2(\mathbb P(\cdot\mid a))}^2

+∣β^(a)−β0(a)∣L2(P(⋅∣a))2.+|\hat\beta(a)-\beta_0(a)|_{L^2(\mathbb P(\cdot\mid a))}^2.

Thus, our first-order bias cancellation extends directly from population-average estimation to input-specific functionals. This follows the established principle of orthogonal pseudo-outcome regression (Chernozhukov et al., 2022a; Kennedy, 2022), which we now formally establish in our neural-operator setting. The complete proof is provided in the new Supplement B.3.

(iii) New experiments demonstrate improvements in input-specific prediction. Beyond the theoretical extension, we conducted new input-specific experiments in our new Table 2 on the PK dataset, where we evaluated AUC and TAT across all nine sampling regimes and 50 independent repetitions. Further, we evaluate our debiasing procedure for input-specific functionals our new Supplement D.1 on Darcy flow

We measure strong, subject-level RMSE against the true input-specific functional values, and compare input-specific DOPE with the direct operator plug-in (i.e., the one that uses g(S^(a))g(\hat S(a)) without correction) with the same learned solution operator and test subjects. DOPE achieves lower RMSE in all configurations.

These results demonstrate that DOPE improves not only the estimation of population averages but also the prediction of individual quantities of interest. We report the results in our new Table 2 and new Supplement D.1.

(continued below)

(iv) Population-average functionals remain fundamental scientific objectives. Independently of this extension, we respectfully disagree that estimating an expected functional is an irrelevant objective for operator learning. Expected quantities of interest are central to uncertainty quantification, stochastic PDEs, and engineering optimization and have been studied for decades (Ghanem & Spanos, 1991; Xiu & Karniadakis, 2002; Kleywegt et al., 2002, Vidal-Codina et al., 2016). The importance of average functionals is also highlighted by reviewer CX8N.

Our original estimand θ0=E[g(S0(A))]\theta_0=\mathbb E[g(S_0(A))] represents a physical quantity of interest averaged over uncertain input realizations. Examples include expected heat flux or structural stress under random PDE coefficients, expected drag and lift under uncertain operating conditions, and average drug exposure across heterogeneous patient populations.

Importantly, the functional gg need not itself be an averaging operator: it can, for example, represent nonlinear quantities such as peak concentration. The functional gg extracts the physical quantity of interest from an individual solution, whereas the outer expectation accounts for uncertainty across input realizations. These are distinct scientific operations.

Furthermore, population averaging does not diminish the computational motivation for neural operators; it strengthens it. Direct Monte Carlo estimation of θ0\theta_0 requires potentially thousands of full PDE solves across independent inputs. Neural operators amortize this cost by learning the solution map once and evaluating it rapidly across new realizations. This is precisely why surrogate-based methods are attractive for uncertainty quantification.

Finally, strong input-specific prediction and accurate population-average estimation are complementary, not competing, objectives. The former requires accurate predictions for individual inputs; the latter targets a scientifically meaningful expectation without necessarily requiring equally accurate individual predictions. Our revised framework now addresses both.

Action: We have (1) extended the objective of DOPE to input-specific functional estimation in our revised Sections 1 and 3, (2) introduced a new Theorem 4.4 that guarantees Neyman-orthogonality and debiasing for input-specific functionals, with a formal proof in the Supplement B.3, and (3) added new experiments for input-specific functionals demonstrating consistent improvements in strong RMSE across all 18 configurations in Table 2 and Supplement D.1. We also (4) strengthened the motivation for population-average functionals in the revised Section 3.

Thus, the revised paper demonstrates that DOPE is not restricted to population-average targets. The same debiasing principle easily enables both population-level inference and input-specific functional estimation.


W2: Applicability of baseline

Thank you. Huang et al. (2025) study end-to-end parameter-to-observable learning. However, this requires a fundamentally different supervision regime from the one considered in our work.

Specifically, end-to-end learning directly regresses g(S(Ai))g(S(A_i)) on AiA_i. Thus, it requires ground-truth scalar functional labels for every training input. In contrast, our observation model provides only sparse, noisy point evaluations of the solution field, and these scalar functional labels are precisely what we aim to estimate. Therefore, their method requires stronger supervision labels than our setting allows.

Obtaining them would require either direct measurements of the target functional or solving the underlying PDE (or reconstructing the full solution trajectory) for each training input, and thereby defeats the purpose of our partial-observation setting.

Their approach is therefore a complementary strategy for settings where such scalar labels are available, not a directly comparable baseline under our observation model.

Further, they do not provide any debiasing strategy when their labels are somehow learned from data.

Hence, our plug-in estimator is the natural baseline in our setting because it uses the same learned solution operator and the same sparse observations as DOPE, without the proposed debiasing correction.

Action: We have added the above reference to our revised extended related work and clarify why it requires stronger supervision than our setting allows. To strengthen the empirical comparison, we have additional observation-only baseline in our revised Table 1. This baseline estimates the target functionals directly from sparse, noisy measurements, with oracle access to the observation-design weights, but without learned operator predictions or ground-truth functional labels. DOPE achieves lower RMSE, which demonstrates that its improvements extend beyond the comparison with the plug-in estimator.

Responses to Questions:

Q1: Second-stage regression

Thank you. Second-stage regression follows a well-established principle in semiparametric estimation and causal machine learning: first construct an orthogonal pseudo-outcome, then regress it on the covariates (in our setting, functional inputs) to recover the conditional target (e.g., Chernozhukov et al., 2022a; Kennedy, 2022).

In our setting, the procedure consists of two stages:

First stage (debiased pseudo-outcome construction): We estimate the nuisance operators S^\hat S and β^\hat \beta from sparse observations and construct

ψ~=g(S^(Ai))+1Ki∑k=1Kiβ^(Ai)(Xik)(Yik−S^(Ai)(Xik)).\widetilde \psi = g(\hat S(A_i)) + \frac{1}{K_i}\sum_{k=1}^{K_i}\hat\beta(A_i)(X_{ik})\big(Y_{ik}-\hat S(A_i)(X_{ik})\big).

Averaging these pseudo-outcomes yields our population-average DOPE estimator. To estimate the input-specific functional θ0(a)=g(S0(a))\theta_0(a)=g(S_0(a)), we instead perform a second-stage regression.

Second stage (conditional functional estimation): We regress the cross-fitted residual corrections ψ~i−g(S^(Ai))\widetilde \psi_i -g(\hat S(A_i)) on the functional inputs AiA_i. Specifically, we estimate the conditional correction through

r^=arg⁡min⁡rϕ∈R1n∑i=1n(ψ~i−g(S^(Ai))−rϕ(Ai))2.\hat r = \arg\min_{r_\phi\in\mathcal R} \frac{1}{n}\sum_{i=1}^n \left( \tilde\psi_i - g(\hat S(A_i)) - r_\phi(A_i) \right)^2.

We then construct the input-specific debiased predictor. That is, we preserving the original plug-in prediction and add the learned conditional correction:

θ^(a)\hat \theta(a)

=θ^plug(a)⏟Plug-in= \underbrace{\hat \theta_{\mathrm{plug}}(a)}_{\text{Plug-in}}

+r^(a)⏟Learned conditional correction,+\underbrace{\hat r(a)}_{\text{Learned conditional correction}},

where θ^plug(a)=g(S^(a)).\hat \theta_{\mathrm{plug}}(a)= g(\hat S(a)).

Thus, the second-stage regression learns the systematic conditional correction to the original neural-operator prediction.

Importantly, we now formally establish the debiasing property for input-specific targets in a new main-text theorem. Our Conditional Debiasing Theorem 4.4 proves that the corrected conditional functional is Neyman-orthogonal and that its conditional bias is second-order in the nuisance estimation errors.

Thus, first-order nuisance bias cancels not only for population averages but also for input-specific functionals. The learned second-stage regression approximates this conditional correction.

We additionally provide new experimental evidence supporting this extension. Across 50 repetitions and all 18 functional–sampling configurations on PK data, input-specific DOPE consistently improves strong RMSE over the direct operator plug-in that learns g(S^(A))g(\hat S (A)) (Table 2). We further provide input-specific results on Darcy flow in our new Supplement D.1.

Action: We added a new main-text theorem 4.4 that shows Neyman orthogonality and second-order nuisance bias for input-specific functionals, provided its proof in the Supplement, updated the setup and introduction, and introduced a corresponding second-stage algorithm in our new Algorithm 2. We also add new input-specific experimental results in Table 2 and Supplement D.2, demonstrating consistent improvements across all sampling regimes and functionals.


Q2: Koltchinskii and Li (2026)

Thank you for pointing us to this work. Koltchinskii and Li (2026) develop a general framework for estimating smooth functionals of Banach-valued statistical parameters, using sample splitting and higher-order Taylor expansions. While their abstract framework is broad, it does not provide a direct construction for our observation model, in which an unknown solution operator is observed only through sparse, noisy, input-dependent point evaluations. Hence, it is not directly applicable.

In particular, their approach relies on independent estimators of the underlying Banach-valued parameter to estimate the terms in the Taylor expansion (Section 2, Equations 2.1–2.3). In our setting, the full solution operator is never observed, and constructing such estimators is precisely the challenge. Instead, we derive a conditional Riesz representation that translates the functional derivative of an entire solution field into an observable correction based on sparse pointwise residuals (Equation 11 and Proposition 4.7). This representation simultaneously accounts for the observation design and the local sensitivity of the functional. We further develop its automatic estimation via neural-operator Riesz regression (Theorem 4.8).

Action: We added Koltchinskii and Li (2026) to our revised related work.


Q3: First two paragraphs in RW

Thank you for pointing this out!

Action: We fixed this in our revised Section 2.

Reviewer_CX8NOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The problem formulation of this work is the real contribution. All existing neural operator work is primarily consed on obtained the solution field; however this paper estimates the population mean of a scalar quantity of interest.This idea is quie useful and has direct application.

Critical strengths

The most critical strength is that the correction is independent of the surrogate. DOPE turns any predictor into an estimator with a second-order error and a valid confidence interval. The predictor can be a neural operator, a physics-informed operator, or a coarse numerical solver. It separates two questions that are usually mixed. The first is how good the surrogate is. The second is whether the final estimate of θ can be trusted.

Critical weaknesses

The most critical weakness is that the experiments never isolate the effect of debiasing. DOPE uses the observed values at the evaluation inputs, and the plug-in estimator does not. So “DOPE beats plug-in” may only show that using the measurements helps. That is trivially expected. The paper claims that plug-in is “the only existing baseline” (lines 372, 801). That is not true. There is an obvious baseline that uses only the observations and no surrogate. The AUC case shows it most clearly.

Questions for authors
  1. Add the observation-only estimator, using the known design weights \xeta and no surrogate, as a baseline in every table. For nonlinear g, the analogous baseline is a weighted interpolation of the observed points with g evaluated on the result.
  2. State exactly which samples are used to train \hat{S} for the plug-in estimator. This shows whether plug-in ever sees the evaluation-sample observations.
  3. What fails if standard automatic DML is applied directly? Which step of Chernozhukov et al. (2022) or van der Laan et al. (2025) does not apply to this setting? This would justify the claim of a “non-trivial extension”
  4. Is the 1/K averaging efficient? When K varies across samples, equal weighting within each sample is not generally efficient. Did the authors consider weighting samples by K or by the variance of their correction term?
  5. Is there a proof for Remark 4.4?
  6. What happens when overlap fails? Assumption 3.1 requires every region of \Omega to have positive sampling probability. Fixed sensor locations, which are common in practice, violate this. Is \beta then defined at all?
  7. How does DOPE compare with direct solves? In PK and Darcy, the PDE is known and cheap to solve. The authors already solve 2000 PDEs for the ground truth. How does DOPE compare in cost and error with direct-solver Monte Carlo?
  8. Can the authors test a misspecified model? Train the surrogate on a PDE that differs from the one generating the observations. Show whether DOPE corrects the resulting bias, and whether a physics-informed plug-in fails.
  9. Which samples train \hat{S} and \hat{\beta}? Table 9 says “2-fold cross-fitting on the test split.” The ETTh1 section says the plug-in and DOPE use “the same frozen fitted solution operator.” Please state exactly which samples train for both, and which samples the final average uses, for every experiment.
6 comments

Thank you very much for your positive review of our paper! We improved our paper as a result of your comment, and highlighted all changes in blue color in our revised rebuttal PDF.


Responses to Weaknesses:

W1: Isolating the effect of debiasing

Thank you. We added the suggested experiments. Specifically, we have implemented the requested observation-only baseline and added it to our experiments in our revised Table 1 (see our response to Q1 below).

To put the results in context, we first discuss why our experiments successfully isolate the effect of debiasing, and why the improvements from DOPE are non-trivial. For this, we would like to clarify three important points.

(i) Our comparison with plug-in estimation directly evaluates the benefit of the proposed debiasing procedure. Both estimators use exactly the same training data and the same fitted neural operator S^\hat S.

The only methodological difference is that DOPE augments the plug-in estimate with our theoretically derived, Riesz-weighted residual correction. Importantly, using observed residuals is not an experimental confounder, but exactly the defining contribution of one-step debiased estimation [1,2].

The purpose of our comparison is to validate whether this correction improves functional estimation relative to standard plug-in evaluation of the same learned solution operator. Adding residual corrections does not trivially guarantee improved estimation accuracy. Poorly constructed corrections can increase variance or introduce additional bias. The improvement depends on the specific weighting mechanism, which is exactly what DOPE derives.

(ii) Our experiments already isolate and validate the mathematical mechanism of debiasing. All of our experiments compare the plug-in approach against our debiased neural operator. Further, Figure 3 directly tests Neyman orthogonality by systematically perturbing the nuisance operators around their oracle values.

Here, we see that the plug-in estimator exhibits first-order bias (log–log slope approx. 1.16), whereas DOPE exhibits second-order bias (slopes approx. 2.03 and 2.02 for individual and joint nuisance perturbations, respectively). These results show that improvement of DOPE is not simply due to observed measurements, but because of the cancellation of first-order nuisance errors through our carefully constructed orthogonal correction.

These results directly validate our theoretical results (Proposition 4.1, Proposition B.3, and Theorem 4.3).

(iii) Observation-only estimation addresses a different question. A direct observation-based estimator can be a useful additional comparison for simple linear functionals such as AUC. Hence, to address your concern, we added a new baseline to our main experimental results in our revised Section 5 (please see Q1 below for details).

However, for general nonlinear functionals, applying gg to an interpolated sparse trajectory does not generally yield an unbiased estimator and introduces additional interpolation and approximation errors. Thus, observation-only estimation is not a universally applicable solution for the operator-specific question. Further, this does not invalidate the plug-in comparison, which is the natural baseline for quantifying the benefits of debiasing a learned neural operator.

Action: In our revised paper, we clarify the different roles of the plug-in and observation-only estimators in our revised Section 5. We emphasize the dedicated orthogonality experiment in Figure 3, which already directly isolates and validates the proposed debiasing mechanism. Finally, we add a new baseline in our revised Table 1.

Responses to Questions:

Q1: Observation only baseline

Thank you for this suggestion! We have implemented the requested observation-only baseline and added it to our experiments in Table 1.

For the AUC functional, we use direct oracle design-weighted estimation from the sparse observations, without interpolation or a surrogate. For nonlinear functionals, we reconstruct each evaluation trajectory independently with oracle design-weighted Gaussian RBF smoothing, and subsequently evaluate gg on the reconstructed trajectory. The smoothing hyperparameters are selected on validation data.

Importantly, this baseline receives oracle access to the true observation-design weights (ξ0\xi_0), which are not available to DOPE in its fully learned variant.

=> DOPE outperforms the observation-only estimator across AUC and TAT with the same sparse evaluation measurements. For example, at (ρ=1\rho=1), DOPE reduces RMSE by up to 17.6% for AUC and 41.8% for TAT. All results are based on 50 independent seeds.

These findings directly address the concern that the improvement of DOPE is merely a trivial consequence of accessing evaluation measurements. Even an observation-only estimator with oracle knowledge of the sampling design is less accurate than DOPE. This demonstrates the benefit of combining learned solution structure with a principled residual correction.

We emphasize that the observation-only estimator is complementary, not the primary baseline for isolating the effect of debiasing. The main comparison is DOPE versus plug-in, which uses the identical learned neural operator but omits the derived correction. Our controlled perturbation experiment (Figure 3) further isolates the debiasing mechanism, and demonstrates the predicted transition from first-order to second-order bias.

Action: We add a new baseline to our comparison Table 1. Although the baseline receives oracle information, DOPE strongly outperforms it across both functionals.


Q2: Samples for solution operator

Both plug-in and DOPE use exactly the same neural operator S^\hat S, and are trained exclusively on the training set, with the validation set; neither method uses evaluation-sample observations to train S^\hat S. DOPE uses the observed evaluation residuals through its theoretically derived orthogonal correction, which is precisely the intended methodological distinction and the effect our comparison aims to quantify.

Action: We updated Supplement G to avoid any ambiguities.

Q3: Related work

Thank you. A direct, out-of-the-box application of existing automatic DML does not yield a valid estimator. The critical difficulty is not only that our nuisance is infinite-dimensional, but that the target depends on complete solution fields while the data consist exclusively of sparse, noisy, input-dependently sampled point evaluations. Three concrete steps are required.

(i) The standard observed-outcome formulation is not applicable. Chernozhukov et al. (2022a;b) formulate automatic debiasing around regression functionals and observable regression residuals. In our setting, neither the full output S0(Ai)S_0(A_i) nor the scalar label g(S0(Ai))g(S_0(A_i)) is observed. Treating the sparse measurements as ordinary scalar functional outcomes would target the wrong quantity. We instead formulate the estimation problem at the level of the entire solution operator. Here, we use subject-level sparse observations and construct the observable residual correction that is required for the full-field functional.

(ii) Naively applying the functional Riesz representer destroys orthogonality. The Fréchet derivative of gg is represented under the full-field reference measure μ\mu, whereas residuals are observed under P(⋅∣A)P(\cdot\mid A). Simply using the functional sensitivity wg(S0(A))w_g(S_0(A)) as the correction weight generally gives

DgS0(a)(h)≠E[wg(S0(a))(X)h(X)∣A=a].Dg_{S_0(a)}(h) \neq \mathbb E[w_g(S_0(a))(X)h(X)\mid A=a].

=> Thus, the first-order plug-in bias is not canceled, and the resulting score is not Neyman-orthogonal. Our Proposition 4.7 establishes the required conditional change-of-measure identity.

Neither inverse-design weighting alone nor functional sensitivity alone provides the correct correction for general nonlinear gg.

(iii) The resulting correction must be learned from sparse operator observations. Existing automatic DML formulations do not directly supply an implementable neural-operator learner for this conditional representer. We derive its population optimization objective (Theorem 4.8) and implement neural-operator Riesz regression using automatic-differentiation Jacobian–vector products (Supplement F). This provides an end-to-end procedure from sparse observations to debiased functional estimates, without full-field labels, analytical functional derivatives, or explicit estimation and inversion of the observation density.

Finally, van der Laan et al. (2025) provide general theory of smooth functionals of Hilbert-space M-estimands. Their general framework could accommodate our problem once its observation-level loss, target derivative, and appropriate Riesz representation have been established. However, these are precisely the non-trivial, problem-specific constructions developed in our work; they are not obtained by off-the-shelf application of an existing estimator.

In summary, our contribution is to make automatic debiasing mathematically valid and computationally realizable for nonlinear functionals of partially observed neural operators. Our derivation is new and requires custom steps including the conditional representation under the actual observation law and its automatic estimation from sparse measurements.

Action: We make these distinctions explicit in our revised related work Section 2 and highlight the specific difficulties in our setting at the end of our revised Section 3.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to all reviewers

We thank the reviewers for the comprehensive and helpful feedback on our work.

To improve our work, we made the following additions to our revised paper (highlighted in blue\color{blue}{\text{blue}}):

  1. Extension to input-specific functional estimation, with a new theorem and new experiments: We extend DOPE beyond population-average estimation to input-specific functionals. Importantly, this requires only a minor addition to our original framework.
    • We update our objective formulation to allow for input-specific functionals
    • We establish a new Theorem 4.4, which proves Neyman orthogonality and second-order nuisance bias for input-specific functionals, with a formal proof in our new Supplement B.3 and an implementation in our new Algorithm 2.
    • We also provide new input-specific experiments in our new Table 2 (PK dataset) and new Supplement D.1 (Darcy flow dataset). DOPE improves strong RMSE over all datasets, all irregularity parameters and all functionals. These results show that DOPE improves both population-average estimation and individual target-specific functional prediction.
  2. New observation-only baseline: In our revised Table 1, we add an observation-only baseline that estimates functionals from sparse measurements with oracle observation-design weights. Despite this advantage, DOPE achieves lower RMSE of up to 41.8%. We also emphasize that DOPE and plug-in use the identical learned operator, while Figure 3 directly confirms first-order versus second-order bias.
  3. New variance and confidence-interval robustness analysis: In our new Supplement D.2, we analyze all nine sampling regimes across three functionals and 50 seeds. Although Riesz estimation RMSE increases, the score variance increases by DOPE increases by only 0.28%–3.57%, and confidence-interval lengths by just 0.08%–1.47%. Empirical coverage remains 90%–98%, which demonstrates effective robustness to errors in nuisance estimates.
  4. New comparisons against direct numerical solvers: We add two experiments in new Supplement D.3, which compares direct Monte Carlo with DOPE on estimation RMSE and empirical coverage at equal confidence-interval width. These results reconfirm the fundamental advantage of neural operators: amortizing expensive numerical solves across input realizations.
  5. New regularization sensitivity study and cross-validation procedure: We clarify oracle-free selection of the Riesz regularization parameter λ\lambda using the held-out Riesz objective in revised Section 4.4 and Algorithms 1–2. A new experiment in Supplement D.4 demonstrates that cross-validation consistently selects small penalties. We also clarify that regularization does not alter Neyman orthogonality at the true nuisances.
  6. Theoretical clarifications: We clarify that design overlap is fundamental for any estimator and not specific to DOPE in our revised Assumption 3.1, and explain within-subject versus population averaging in our revised Section 4.2.
  7. Strengthened theoretical and computational explanations: We clarify the second-order bias guarantee and weaker nuisance convergence requirements, and explain why sparse, input-dependent observations require a tailored conditional Riesz construction beyond standard automatic DML. We further highlight in revised Section 4 and Supplement F that Riesz regression incurs only constant-factor training overhead without additional PDE solves and can be avoided entirely when the debiasing weights are analytically available.
  8. Motivation and related work: We strengthen revised Section 3 to better motivate population-average functionals. We also expand our Section 2 and Supplement A to discuss related work.
  9. We add explicit limitations regarding structural model misspecification.

We are confident that, with the help of the reviews and our improvements, our work will be an important contribution to the community. Finally, we wanted to say “thank you” – we truly appreciate the help, which allowed us to revise our work and make important improvements.

Area_Chair_qEdMOpenReview ↗

Please engage in the discussion

Reviewers: if you have not responded to authors' reply, please read it and engage in the discussion. Thanks!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.