- Rating
- 3(+1)
- Confidence
- 1(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper addresses the problem of estimating low-dimensional scalar functionals of solution trajectories mapped by neural operators under partial and irregular observations. It demonstrates that standard plug-in estimators suffer from a severe systematic first-order bias because approximation errors propagate nonlinearly through the functional. To overcome this, the authors introduce DOPE (Debiased Neural Operator), a semiparametric framework that constructs a one-step Neyman-orthogonal estimator using a weighting mechanism based on Riesz representers. Theoretical estimate and numerous experiments are performed.
Critical strengths
Extending automatic debiased machine learning from function-valued nuisances to operator-valued nuisances via Riesz regression and autodiff-based Jacobian-vector products is a good, non-trivial methodological contribution.
Critical weaknesses
-
Training the extra operator-valued nuisance () via Riesz regression requires solving a separate optimization problem alongside the primary neural operator surrogate, which increases training complexity and compute costs.
-
While DOPE relaxes the convergence rate requirement on the neural operator, the validity of the estimator still relies heavily on the design overlap assumption (Assumption 3.1) and the accurate estimation of both nuisance components. In data-sparse or extremely irregular sampling regimes, estimating the debiasing weight operator stably can be challenging.
Questions for authors
-
How sensitive is the performance of DOPE to the choice of the regularization parameter in the Riesz regression objective? Is cross-validation practical for tuning across different PDE/operator systems?
-
Under extreme observation sparsity or heavy adaptive sampling concentration (e.g., large ), does the estimation variance of increase significantly, and how does this affect the empirical coverage of the CLT-based confidence intervals?
-
After learning through Riesz regression, if it is very unsmooth or extrapolated (when is large), is regularization necessary? How does regularization affect orthogonality?
4 comments
Responses to Questions:
Q1: Riesz regularization and cross-validation
Thank you! This is correct, the regularization parameter can be selected using standard cross-validation, as in other neural estimation methods. Importantly, our Riesz regression formulation provides a natural validation objective. We do not need access to the unknown Riesz representer .
Specifically, by Theorem 4.8, the population Riesz risk satisfies
where the latter term is constant w.r.t. .
Hence, minimizing the Riesz risk is equivalent to minimizing the estimation error of the debiasing weights. We can therefore select by evaluating the empirical Riesz objective on held-out validation samples. This requires neither oracle Riesz weights nor additional PDE solves and is applicable to any PDE system.
Importantly, cross-validation can be performed entirely within the training folds of our cross-fitting procedure. The resulting estimator retains the Neyman-orthogonal score construction.
Finally, controls the usual regularization trade-off: stronger penalties can stabilize weight estimation under sparse observations, whereas excessive regularization may introduce approximation bias. Our framework does not require a particular fixed value of , and it can be tuned separately for each application using the same Riesz objective.
Action: In our revised Section 4.4, we highlight that cross-fitting can be used for tuning of , and show this in our Algorithms 1 and 2. Further, we show an empirical example of selecting in our new Supplement D.4.
Q2: Irregular observations
Thank you. We have additionally conducted a new variance analysis across 50 independent seeds in our Supplement D.2. Further, our existing experiments already investigate how increasingly concentrated observation designs affect estimation accuracy and confidence-interval coverage. We would like to address your concern with three points:
(i) New analysis of variance stability: To address your question directly, we additionally analyzed the estimated variance of the debiased score across all nine sampling regimes from our main experiments in our new experimental results in Supplement D.4. From to , the score variance increases by only 0.28% for AUC, 0.85% for smooth TAT, and 3.57% for soft Cmax. Across the entire grid, variance fluctuations remain within 3.6–5.5% of the uniform-sampling value, with no systematic increasing trend.
Yet, the RMSE of the learned Riesz representer increases by 109–153% under the most concentrated sampling design. Thus, although estimating the debiasing weights becomes more challenging, these additional nuisance-estimation errors do not translate into increases in the variance of the final estimator. This is because DOPE is robust against first order error terms even in , not only . We emphasize that Riesz RMSE measures estimation error relative to the oracle, rather than the training-induced variance of itself.
(ii) Robustness under irregular sampling: In our main experiments (Table 1), we systematically vary the sampling-irregularity parameter across nine levels, from uniform sampling () to strongly concentrated sampling (). DOPE maintains stable estimation accuracy and consistently outperforms plug-in estimation. For example, the AUC RMSE changes only from to . Our Darcy-flow experiments (Table 3) further demonstrate improvements under adaptive observation designs.
(iii) Confidence-interval validity: Our existing CLT experiments (Supplement D.2) confirm this stability. Empirical coverage of nominal 95% intervals remains between 90% and 96% for AUC and smooth TAT, and between 90% and 98% for soft Cmax, without systematic deterioration as increases. Between and , mean confidence-interval lengths increase by just 0.08%, 0.39%, and 1.47%, respectively. In contrast, plug-in confidence intervals exhibit substantially poorer coverage (68–88%).
=> Theoretical explanation: Neyman orthogonality makes the estimator first-order insensitive to nuisance-estimation errors. In particular, if , the correction has zero expectation regardless of the accuracy of ; more generally, errors in enter the bias only at second order. This explains why substantial Riesz-estimation errors need not produce comparable deterioration in the final estimator.
Taken together, our new variance analysis and existing CLT experiments provide direct evidence that DOPE maintains stable inferential performance even as adaptive sampling concentration makes nuisance estimation considerably more difficult.
Action: We added the new study to our new Supplement D.2. Therein, we include the new score-variance analysis and figures, and explicitly highlight the confidence-interval results.
Q3: Regularization and orthogonality
Thank you. Regularization affects only the estimation of the Riesz representer, not the construction of the Neyman-orthogonal score for the target functional. Specifically, our debiasing score (Equation 12) and its Neyman-orthogonality property (Proposition B.3) are defined independently of the regularization parameter . Hence, regularizing does not alter the debiasing construction or its orthogonality.
Regularization is an optional, standard mechanism for controlling the complexity and variance of the learned Riesz weights, particularly under irregular sampling. Importantly, our theoretical framework explicitly allows estimation errors in both nuisance components, which includes those arising from regularization: by Theorem 4.3, these errors enter the final estimation bias of the target functional only through second-order terms.
Put simply: One can trade a little increase bias in for a large reduction in variance of . The error in is then effectively reduced. Then, the smaller error enters the final stage estimator of our target functional, but its debiasing construction remains completely valid, and benefits from smaller errors in . Further, these errors only enter at second order. Thus, regularization can improve numerical stability without modifying the underlying debiasing construction of the target functional.
Action: In our revised Section 4.4, we highlight that regularization does not affect orthogonality w.r.t. nuisance estimates and the debiasing effect of the final estimator.
[1] Dylan J. Foster and Vasilis Syrgkanis. Orthogonal statistical learning. The Annals of Statistics, 53(3): 879–908, 2023.
[2] Victor Chernozhukov, Whitney Newey, Victor Quintas-Martinez, and Vasilis Syrgkanis. RieszNet and ForestRiesz: Automatic debiased machine learning with neural nets and random forests. In ICML, 2022a.
[3] Edward H. Kennedy. Semiparametric doubly robust targeted double machine learning: A review. arXiv preprint, 2203.06469, 2022.
Thank you very much for your helpful review of our paper! We improved our paper as a result of your comment, and highlighted all changes in blue color in our revised rebuttal PDF.
Responses to Weaknesses:
Thank you. While DOPE requires fitting an additional nuisance operator in the general case, we emphasize that this introduces only constant-factor training overhead, and does not change the asymptotic computational order. Specifically, the Riesz representer is learned using the same neural operator architecture as the solution operator, and the optimization requires only standard automatic differentiation via Jacobian–vector products (see Supplement F). Importantly, no additional PDE solves or observations are required.
Moreover, the additional optimization is not always necessary. By Proposition 4.7, the debiasing weight decomposes as β(A)(x)=ξ(A)(x)wg(S(A))(x). When the sampling design and functional derivative are known, we can directly construct β(A)(x), which eliminates the need for a separate Riesz regression entirely. Hence, the automatic Riesz regression provides flexibility for settings where such closed-form constructions are unavailable.
Importantly, this modest additional cost enables reduction of bias by an order of magnitude, as demonstrated by our theoretical results and experiments.
Finally, training separate nuisance models is standard practice in double/debiased machine learning [1,2,3], where auxiliary nuisance components are estimated before constructing the Neyman-orthogonal estimator. Therefore, it is not an overhead specific to our method.
Action: We highlight the modest computational overhead in our revised Section 4, and emphasize that no PDE solves or similar are required.
Thank you. Design overlap is a fundamental identifiability requirement, not a restriction specific to DOPE.
If a region of the solution domain has positive measure under μ but is never observed under P(⋅∣A), even the solution operator S0 itself cannot, in general, be identified on that region from the available observations. Two distinct solution operators can agree on every observed location while differing on unobserved regions.
Hence, no estimator (including conventional plug-in neural-operator estimation) can consistently recover the full solution operator or arbitrary functionals depending on these regions without additional structural assumptions. Extremely weak overlap may increase variance, but this is a general statistical limitation of inference from sparse measurements, not a failure of Neyman orthogonality.
Action: In our revised Assumption 3.1, we clarify that overlap is fundamental for the identifiability and stable estimation of the underlying solution operator and its functionals, and is not specific to our debiased estimator.
Thank you for asking this question!
It is exactly the contrary:
Robustness to nuisance estimation errors is precisely the central contribution of DOPE. It is not the case that DOPE requires highly accurate nuisance estimates. Instead, our Neyman-orthogonal score is exactly constructed to eliminate their first-order effect on the function estimation bias (Proposition B.3). By Theorem 4.3, the remaining bias satisfies
∣Bias(θ)∣≲∣S−S0∣L2(P)2
+∣β−β0∣L2(P)2.
In contrast, the plug-in estimator generally incurs first-order bias in the neural operator error (Proposition 4.1). Consequently, DOPE permits substantially slower nuisance convergence rates (e.g., op(n−1/4) for both nuisances), and not the op(n−1/2) sufficient rate required for plug-in estimation.
Schematically, if for example the solution operator or the correction weight has error δ<1, it only enters at order O(δ2) into the final estimate for DOPE, while the former stays at order O(δ) for the plug-in estimator.
This is also directly validated by our Figure 3, where DOPE exhibits quadratic instead of linear sensitivity to nuisance perturbations.
Thus, DOPE explicitly reduces sensitivity to nuisance estimation errors.
Action: In our revised Section 4.3, we highlight that robustness to nuisance errors is exactly the property of our proposed method.