acceptodds
Under review as a conference paper at ICLR 2027

Deep Minimax Estimation for Nonparametric Instrumental Variables Regression

Abstract

We study nonparametric instrumental-variable (IV) mean, expectile, and quantile regression using a common neural minimax estimator. ReLU networks represent the structural function and a bounded instrument-indexed critic. Estimation minimizes the largest absolute empirical residual-score moment without separately estimating a first-stage conditional law. Mean and expectile scores enter directly, whereas the quantile indicator is approximated by an integrated kernel. For exact empirical minimizers, we establish nonasymptotic bounds separating statistical error, approximation errors for both networks, and quantile-smoothing error. Under the stated regularity conditions and H\"older smoothness \(\zeta\), the expected adversarial-moment risk admits an upper bound of order \(n^-\zeta/(d_\max+2\zeta)\), up to logarithmic factors, where \(d_\max\) is the larger of the regressor and instrument dimensions. Additional localization, critic-link, and spectral conditions yield structural \(L^2(P_X)\) upper bounds. Simulation studies assess finite-sample structural recovery, while an application to randomized-encouragement diary data illustrates differences among fitted response curves.

Primary area:
learning theory
Keywords:
Adversarial estimation, Deep neural networks, Instrumental variables, Expectile regression, Quantile regression, Convergence rates

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_noctOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies nonparametric instrumental variable regression for mean, expectile, and quantile estimation under a unified minimax framework. Two DNNs are used to represent the structural function and the critic. The authors establish bounds for adversarial moment risk and derive structural (L2L^2) convergence rates under additional assumptions. The paper also includes simulation studies and a real-data application.

Critical strengths
  1. The paper extends NPIV estimation beyond the standard mean regression setting to expectile and quantile regression. In particular, handling the nonlinear expectile score and the smoothing error in quantile estimation adds some technical interest to the theoretical analysis.
  2. The authors explicitly distinguish moment-risk convergence from structural recovery, which is important in ill-posed inverse problems.
Critical weaknesses
  1. The theoretical assumptions seem strong to me. In particular, the structural L2L^2 convergence result in Theorem 4.2 relies on the localization condition P(f^∈Uβ)→1P(\hat f \in U_\beta) \to 1, where UβU_\beta imposes spectral regularity restrictions on the estimation error. The paper does not establish why the proposed estimator should satisfy this localization condition. Therefore, the applicability of the structural convergence guarantee remains unclear.

  2. The experimental evaluation is limited. The simulations mainly show that estimation errors decrease as the sample size increases. There are no direct comparisons with existing neural IV methods or other nonparametric IV estimators.

Questions for authors
  1. The theoretical motivation for using DNNs. The DNN approximation and covering number bounds in this paper are based on existing results. Moreover, the convergence rate in Theorem 4.1 still depends on the ambient dimension and does not appear to exploit potential advantages of DNNs under low intrinsic dimension. Could the authors explain why DNNs are particularly suitable for this problem and whether they offer theoretical advantages over traditional nonparametric estimators? Given the existing literature on minimax conditional moment estimation and structural recovery, what new mathematical difficulties arise when handling the expectile and quantile scores?

  2. Could the authors include an ablation study on critic network complexity (e.g., width or depth) to examine how sensitive the proposed estimator is to the choice of critic class?

1 comment

We thank the reviewer for the positive assessment of our technical contributions and for the constructive feedback. Below, we address your concerns point-by-point.

W1: The theoretical assumptions seem strong.

We thank the reviewer for pointing this out. We agree that it is helpful to further clarify how this assumption operates in practice.

The spectral source condition is indeed a standard assumption in the NPIV literature (e.g., Bennett et al., 2023; Wang et al., 2022). It is primarily used to control the high-frequency tail of the estimation error when inverting a compact operator. While providing a rigorous verification for an unregularized neural ERM remains an open challenge, in practice, the network's complexity is effectively constrained by explicit regularization (e.g., weight decay) or implicit regularization (e.g., early stopping), which prevents it from fitting severely ill-posed high-frequency components.

Furthermore, our theoretical results actually require a weaker condition. Instead of the strict pointwise spectral decay ∑jj2β⟨f^−f0,uj⟩2≲1\sum_j j^{2\beta}\langle \hat{f}-f_0, u_j\rangle^2 \lesssim 1, the proof in Appendix E.5 only relies on a tail-energy control ∑j>J⟨f^−f0,uj⟩2≲wJ2\sum_{j>J}\langle \hat{f}-f_0, u_j\rangle^2 \lesssim w_J^2 with wJ≍J−βw_J \asymp J^{-\beta}. This relaxed condition is closely connected to the approximate and variational source conditions widely adopted in nonlinear inverse problems (e.g., Hein and Hofmann, 2009; Hofmann et al., 2007). Under this weaker condition, we can still obtain the exact same convergence rate as stated in Theorem 4.2.

W2: The experimental evaluation is limited.

We thank the reviewer for this helpful comment. Our primary focus in this paper is to establish rigorous non-asymptotic theoretical bounds. Therefore, the simulations are primarily designed to validate these specific theoretical findings (e.g., verifying how the estimation errors decrease with sample size nn, and the effect of the smoothing bandwidth hh).

The absence of direct comparisons with existing nonparametric IV estimators is due to structural mismatches in problem settings. Traditional nonparametric IV estimators (such as sieve or kernel methods) suffer from severe performance degradation in high-dimensional settings (d=p=10d=p=10 in our simulations) and are typically limited to low-order basis functions. Meanwhile, existing neural IV methods (e.g., DeepIV by Hartford et al., 2017) are primarily designed for conditional mean or two-stage procedures, and they do not directly accommodate the discontinuous indicator score required for quantile estimation or the adversarial minimax formulation used in our framework.

Q1: The theoretical motivation for using DNNs and the dependence on the ambient dimension.

This is an excellent question. DNNs are particularly suitable here because traditional sieve or kernel methods require manual basis selection (e.g., polynomial or B-spline bases), which becomes exponentially difficult in high dimensions. DNNs provide a flexible, data-driven basis that can adapt to the underlying structure of the data without manual tuning.

The dependence on the ambient dimension dmax⁡d_{\max} in Theorem 4.1 reflects the intrinsic worst-case cost of ReLU network approximation in the ambient space. While deep networks can theoretically exploit low-dimensional compositional structures, deriving a rate based on the intrinsic dimension d∗d^* for a nonlinear adversarial minimax objective with discontinuous scores remains a highly non-trivial open problem.

A fundamental mathematical challenge we solve is handling the expectile and quantile scores. Due to their nonlinearity and discontinuity, standard mean-IV RKHS analyses are inapplicable. We introduce novel pathwise weighted local linearizations (Appendix C.6) to rigorously analyze structural recovery. This constitutes a significant theoretical advancement not covered by existing literature.

Q2: Ablation study on critic network complexity.

We thank the reviewer for this constructive suggestion. Regarding the sensitivity to the choice of critic class: our theoretical analysis indicates that the estimator is relatively robust, provided the critic network is sufficiently complex to achieve the required approximation rates (as specified in Eq. 18-19). Once this complexity threshold is met, further increasing the width or depth does not fundamentally alter the convergence behavior.

Since the primary goal of our numerical experiments is to empirically validate our main theoretical bounds (e.g., convergence rates with respect to sample size nn and bandwidth hh), we did not include an ablation study focusing solely on the critic's width and depth in the current version. We plan to conduct systematic experiments in our future work to explicitly examine how the critic network's complexity affects the final structural estimation.

We hope these responses address your concerns.

Reviewer_GuAuOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies non-asymptotic bounds for adversarial formulations of NPIV problem with neural network approximations.

Critical strengths

This paper addresses a relevant and interesting problem in a causal machine learning.

Critical weaknesses

The paper's introduction contains many instances of sloppy writing and jargon undefined when first introduced, as in for instance "We use sub-Gaussian outcome control for the mean and expectile analyses, and integrated-kernel smoothing for quantile estimation.", where "integrated-kernel smoothing" is undefined. In another example, "The question studied here is how approximation of both neural players and quantile smoothing affect finite-sample risk under a common bounded-critic criterion, and what further conditions are needed for structural recovery." It is unclear what a "common bounded-critic criterion" means. These writing deficiencies hinder readers' understanding of the papers' contribution and distinction from prior literature.

"we obtain the expected adversarial-moment risk bound". it is unclear what this means. This is quite serious as it makes it hard for readers to assess exactly what is being achieved here.

The primary contribution seems to be a straightforward combination of ReLU neural network approximation theory with existing analysis of NPIV. Many related works are not cited. The results are not novel. Many existing analyses studied adversarial or minimax NPIV formulation with neural network approximation, establishing a large body of statistical and optimisation results, as well as deep results from the lens of semiparametric efficiency theory. None of these results are discussed in detail.

The ill-posedness based non-asymptotic bounds appear to be a standard and crude adaptation of classical bounds from the work of Xiaohong Chen and others, yielding few new insights. The bounds also do not appear to be sharp (i.e. admit matching lower bounds.)

Missing references. Chen et al. Towards a Unified Analysis of Neural Networks in Nonparametric Instrumental Variable Regression: Optimization and Generalization, Petrulionyte et al. Functional Bilevel Optimization for Machine Learning, NIPS 2024, Wang et al. Spectral Representation Learning for Conditional Moment Models, Zhu et al. A Mean-Field Analysis of Neural Stochastic Gradient Descent-Ascent for Functional Minimax Optimization, JMLR 2026 Chen and Christensen. Optimal Sup-norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regression Quantitative Economics Chen, Christensen and Kankanala. Adaptive Estimation and Uniform Confidence Bands for Nonparametric Structural Functions and Elasticities, Bennett et al. Inference on Strongly Identified Functionals of Weakly Identified Functions, JRSSB 2026 Bennett et al. Source Condition Double Robust Inference on Functionals of Inverse Problems, Olivas-Martinez and Rotnitzky. Source-Condition Analysis of Kernel Adversarial Estimators, 2025

Among the above references, Chen et al. and Zhu et al. on analysis of NN in NPIV directly study the optimisation aspect of neural network optimization in NPIV, for both two-stage least squares and adversarial approach. Chen and Christensen already studies nonlinear functionals such as the quantile function and obtained optimal rates. Although their estimator is based on sieve methods, past results in Kim et al. have established that optimal rates under even more refined conditions can be established for neural network methods, and obtained in addition to minimax optimal convergence rates, adaptivity of neural network to low regularity regimes and separation from kernel methods. Papers by Bennett et al. and Olivias-Martinez, among many others in the field, established rich statistical guarantees for minimax estimation of NPIV model. The current work does not meaningfully advance the theory of NPIV with neural networks in relation to state of the art in the literature.

Finally, a recent reference that comes up from Google search. I would concede that this can be viewed as concurrent work, nevertheless it is useful to compare briefly against these related approaches. Feng et al. Conditional Diffusion for Nonparametric Instrumental Variable Quantile Regression 2026

Questions for authors
  1. What distinguishes your work from the vast body of existing theoretical analysis for NPIV problems, both two-stage and minimax games approach, which have already considered both RKHS and neural networks with various activation functions as function classes, studies both dose response curve and quantile estimation, addressed sharp statistical rates under rich function class assumptions and optimisation guarantees for neural network based methods?
3 comments

We sincerely thank the reviewer for the detailed and critical assessment. We highly appreciate the references provided, which are very relevant to our work. Below, we address your concerns point-by-point following the order of your comments.

W1: "The paper's introduction contains many instances of sloppy writing and jargon undefined when first introduced... it is unclear what a 'common bounded-critic criterion' means... It is unclear what this means [referring to 'expected adversarial-moment risk bound']."

We thank the reviewer for pointing this out. We agree that these terms were introduced too abruptly. To clarify:

  • "Integrated-kernel smoothing" refers to replacing the discontinuous quantile indicator score with a Lipschitz-continuous integrated kernel (Eq. 14), allowing gradient-based optimization.
  • "Common bounded-critic criterion" refers to the unified minimax framework (Eq. 4) where the structural function minimizes the worst-case empirical moment over a bounded neural network critic class.
  • "Expected adversarial-moment risk bound" refers to Theorem 4.1, which provides a non-asymptotic upper bound on the expected population adversarial risk. It explicitly separates the statistical error, neural approximation errors, and quantile smoothing error.

Moreover, in the revised manuscript, we have thoroughly improved the Introduction to explicitly define these terms upon their first appearance. This enhances readability and clearly highlights our contributions and how they distinguish from prior literature.

W2: "The primary contribution seems to be a straightforward combination of ReLU neural network approximation theory with existing analysis of NPIV. Many related works are not cited..."

We sincerely apologize for the missing citations and thank the reviewer for providing such a comprehensive list. We have now cited and discussed all the suggested works (e.g., Chen et al., 2024; Petrulionyte et al., 2024; Wang et al., 2022; Zhu et al., 2026; Chen & Christensen, 2018; Bennett et al., 2023, 2026; Olivas-Martinez & Rotnitzky, 2025). While these works provide foundational analyses for NPIV, our contribution is distinct: we provide the first unified nonasymptotic analysis for discontinuous quantile and nonlinear expectile scores within a neural minimax framework, explicitly decoupling the smoothing error from the network approximation error (Sec 4.3).

W3: "The ill-posedness based non-asymptotic bounds appear to be a standard and crude adaptation of classical bounds... The bounds also do not appear to be sharp (i.e. admit matching lower bounds)."

We respectfully clarify that our bounds are not a crude adaptation of classical results. Classical nonparametric IV theory relies on the strong convexity of the squared loss and linear sieves. Our adversarial minimax framework operates as a non-convex zero-sum game between two ReLU networks, and our empirical risk involves a supremum of an absolute moment (L1L^1-type envelope). This lack of convexity and the first-order absolute envelope inevitably lead to a loss of 1/21/2 order in the approximation error compared to squared-loss regression. Therefore, our rates rigorously characterize the intrinsic costs of Neural ERM.

Regarding sharpness, we agree that our upper bounds do not claim to be minimax optimal. Obtaining a matching lower bound for nonlinear adversarial estimators with discontinuous scores is currently an extremely challenging open problem. Standard Fano or Le Cam lower-bound techniques rely on the differentiability and convexity of the hypothesis class and loss function, both of which are fundamentally violated in our setting. Our work provides a rigorous first step toward establishing sharpness.

W4: "Finally, a recent reference that comes up from Google search... Feng et al. A Conditional Diffusion Model Approach to Deep Nonparametric Instrumental Variables Quantile Regression, 2026."

We thank the reviewer for bringing this to our attention. We agree that Feng et al. (2026) is a highly relevant concurrent work. However, our work differs fundamentally in both methodology and theoretical focus:

  • Methodology: Feng et al. propose a two-stage estimator that first trains a conditional diffusion model to learn the joint density px,y∣zp_{\mathbf{x}, y|\mathbf{z}}, and then minimizes a kernel-smoothed empirical moment over a ReLU DNN class using Monte Carlo samples. In contrast, our work utilizes a single-stage adversarial minimax framework with a bounded neural critic, bypassing the need to separately estimate the conditional density.
  • Theoretical Focus: The theoretical challenge addressed by Feng et al. is bounding the end-to-end total variation (TV) error of the diffusion model under unbounded support. Our theoretical contribution is entirely different: we establish a unified non-asymptotic oracle inequality that explicitly decouples the quantile smoothing error from the statistical and neural approximation errors (Sec 4.3), and we derive structural L2L^2 recovery rates via spectral regularization (Sec 4.4). These aspects are not covered by Feng et al.

Q1: "What distinguishes your work from the vast body of existing theoretical analysis for NPIV problems...?"

We thank the reviewer for this excellent summarizing question. While prior literature has indeed established rich NPIV theories, our work is distinguished in three fundamental ways:

  • Unified Treatment of Discontinuous and Nonlinear Scores: Existing minimax/RKHS NPIV theories (e.g., Dikkala et al., 2020) primarily handle linear mean residuals E[Y−h(X)∣Z]=0\mathbb{E}[Y-h(X)|Z]=0, while sieve-based quantile analyses are restricted to classical basis functions. We provide the first unified neural minimax framework that simultaneously handles the mean, nonlinear expectile, and discontinuous quantile scores.
  • Theoretical Decoupling of Smoothing Error: Because the quantile indicator score is discontinuous, standard empirical process theory cannot be directly applied. We introduce an integrated-kernel smoothing and establish a novel non-asymptotic oracle inequality (Sec 4.3) that rigorously decouples the smoothing bias from the neural approximation and statistical errors.
  • Nonlinear Structural Recovery: While sharp statistical rates for linear NPIV rely on standard source conditions for the conditional expectation operator, our analysis of the structural L2L^2 norm (Sec 4.4) requires novel pathwise weighted local linearizations for the expectile and quantile maps. This addresses a substantially harder mathematical challenge than the linear mean-IV case.

We hope these responses clarify our contributions and adequately address the reviewer's concerns.

I thank the authors for their detailed rebuttal. Unfortunately, I continue to have reservations regarding the writing clarity and novelty of the work. Using a kernel smoother to smooth an indicator function is quite a common technique in past work. For these reasons I maintain my score.

Reviewer_ofW9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper considers nonparametric instrumental variable (IV) regression for mean, expectile, and quantile estimation. The authors propose a neural-network-based minimax estimator by minimizing an adversarial loss. For quantile regression, kernel smoothing is used to handle the discontinuous score function. The authors establish nonasymptotic convergence rates for the adversarial-moment risk and, under additional regularity and spectral assumptions, derive L2L^2 convergence rates for mildly and severely ill-posed problems. Simulation studies and a real-data application are used to illustrate the performance of the proposed approach.

Critical strengths
  1. The paper provides a unified framework for nonparametric IV mean, expectile, and quantile regression using neural-network-based adversarial estimation, which is useful from both theoretical and practical perspectives.
Critical weaknesses
  1. I am not fully convinced about the novelty of the paper. To my understanding, the main contribution is the estimation of conditional mean, expectile, and quantile functions under an instrumental variable setup using adversarial learning. However, adversarial estimation of conditional moment models has already been extensively studied in the literature (including with neural networks), and many of these relevant papers have been cited in the manuscript. For ERM problem, the theory of neural-network estimation under various loss functions is also well established. Therefore, it seems that the main novelty lies in combining neural-network-based adversarial estimation with the particular expectile and quantile score functions, which appears to be a quite narrow. I think the authors should clearly explain what new theoretical insights or technical developments their work provides beyond applying existing methods to these specific score functions.

  2. The presentation of the results needs improvement. In particular, I think it is important to clearly state the key assumptions and define all relevant parameters before presenting the main theorem statements, instead of referring readers entirely to the appendix. For example, the parameters (ζ,β,b)(\zeta,\beta,b) appearing in Theorem 4.2 are introduced only informally in the main text, which makes it difficult to understand their exact meanings and roles in the convergence rate.

  3. It is not clear whether the convergence rate in Theorem 4.2 is (minimax) optimal, even up to logarithmic factors. For example, under polynomial singular-value decay σj≍j−a\sigma_j\asymp j^{-a} and a smoothness condition with index β\beta, the classical minimax-optimal rate is n−β/(2a+2β+1)n^{-\beta/(2a+2\beta+1)} under the corresponding regularity conditions (see, e.g., Chen and Reiss, 2011). In contrast, Theorem 4.2 gives the rate n−ζβ/{(dmax⁡+2ζ)(a+b+β)}n^{-\zeta\beta/\{(d_{\max}+2\zeta)(a+b+\beta)\}} (up to log factor). Although the assumptions are not exactly the same, it would be helpful if the authors could carefully compare these rates under comparable conditions and discuss whether the additional dependence on ζ\zeta and bb is necessary (e.g., whether their rate is minimax optimal under the assumptions of the paper). In particular, is the rate in Theorem 4.2 optimal under the stated assumptions, or is it potentially an artifact of the proof technique?

  4. Furthermore, Theorem 4.2 requires P(f^∈Uβ)→1\mathbb P(\widehat f\in U_\beta) \to 1, where UβU_\beta is a local set satisfying the spectral source restriction ∑j=1∞j2β⟨f−f0,uj⟩L2(PX)2≤1.\sum_{j=1}^{\infty}j^{2\beta} \langle f-f_0,u_j\rangle_{L^2(P_X)}^2\le1. So, this condition depends on the estimation error f−f0f-f_0 instead of only f0f_0. Therefore, the proposed neural-network-based estimator needs to satisfy this spectral regularity condition. However, it is unclear whether the network architecture and parameter constraints considered in the paper guarantee this. I think the authors should provide sufficient conditions under which P(f^∈Uβ)→1\mathbb P(\widehat f\in U_\beta)\to1, or at least discuss how restrictive this assumption is for the NN-based estimator.

Questions for authors

Kindly see the weakness section above.

2 comments

We thank the reviewer for the positive assessment of our unified framework. Below, we address the specific concerns raised.

W1: Novelty of the framework.

We respectfully clarify that this paper is not simply applying existing adversarial estimators to new scores. The unified framework itself is a new theoretical contribution. Existing adversarial NPIV literature primarily handles linear residuals E[Y−h(X)∣Z]=0\mathbb{E}[Y-h(X)|Z]=0. Extending this to expectile and quantile restrictions introduces severe theoretical obstacles—most notably, the discontinuity of the quantile indicator score, which breaks standard empirical process analyses. To solve this, we introduce integrated-kernel smoothing and derive a completely new nonasymptotic oracle inequality (Sec 4.3) that decouples smoothing error, network approximation, and statistical error. Furthermore, our simulation studies are specifically designed to validate this new theory across all three scores.

W2: Definitions of (ζ,β,b)(\zeta, \beta, b).

We thank the reviewer for this helpful suggestion. We agree that these parameters were introduced too informally in the main text. We have now explicitly added their definitions in Section 4.4 (immediately preceding Theorem 4.2), where we state that ζ\zeta is the Hölder smoothness index, β\beta is the index in the spectral source restriction, and bb is the exponent in the projected critic norming condition, ensuring they are formally defined before the main result.

W3: Minimax optimality and rate sharpness.

We thank the reviewer for this deep and important question. The convergence rate in Theorem 4.2 is OP(rnβ/(a+b+β))O_P(r_n^{\beta/(a+b+\beta)}), where rn=n−ζ/(dmax⁡+2ζ)(log⁡n)3/2r_n = n^{-\zeta/(d_{\max}+2\zeta)}(\log n)^{3/2}. We agree that this differs from the classical minimax optimal rate n−β/(2a+2β+1)n^{-\beta/(2a+2\beta+1)} (e.g., Chen & Reiss, 2011). We respectfully clarify that this gap is not an artifact of our proof technique, but an intrinsic consequence of the loss function and the adversarial estimation framework. Specifically, our rate loses orders in two fundamental aspects compared to classical squared-loss regression:

  • Statistical error under non-convex loss: Classical nonparametric regression relies on the strong convexity of the squared loss, which allows the application of standard empirical process theory to yield tight generalization error bounds. However, our adversarial minimax framework operates as a zero-sum game between two ReLU networks, and the resulting loss is non-convex and non-smooth (especially for the discontinuous quantile score). This lack of convexity prevents us from obtaining the same tight variance-bias decomposition as in standard M-estimation, which inevitably leads to a loss in the statistical rate.

  • Approximation error with an absolute envelope: In standard squared-loss regression, the approximation error is analyzed via the L2L^2 norm. In our framework, the empirical risk involves the supremum of an absolute moment, sup⁡g∣E[ϑf(X,Y)g(Z)]∣\sup_{g} |\mathbb{E}[\vartheta_f(X,Y)g(Z)]|, which has an L1L^1-type envelope. Because the approximation error is bounded by a first-order absolute value (rather than a squared term), it contributes an error term whose order is 1/21/2 relative to the squared-loss counterpart. This means the approximation error cannot tightly match the true lower bound and acts as an unavoidable penalty in the overall convergence rate.

  • On the matching lower bound: Obtaining a matching lower bound for nonlinear adversarial estimators with ReLU networks is currently an extremely challenging open problem. Standard Fano or Le Cam lower-bound techniques—which establish estimation lower bounds via hypothesis testing—rely on the differentiability and convexity of the hypothesis class and loss function, both of which are violated here. Therefore, while our upper bound does not claim to be minimax optimal, it provides a rigorous, non-trivial characterization of how statistical error (under non-convexity) and approximation error (under L1L^1 envelopes) interact.

W4: The localization condition.

We thank the reviewer for this constructive suggestion. We agree that discussing the restrictiveness of this assumption is valuable.

The spectral source condition is indeed a standard assumption in the NPIV literature. It appears in classical work on minimax IV estimation (e.g., Bennett, Kallus, Mao, Newey, Syrgkanis, and Uehara, "Minimax Instrumental Variable Regression and L2L_2 Convergence Guarantees without Identification or Closedness," PMLR, 2023), which derives L2L_2 rates under a source condition while avoiding the closedness condition, and in recent analyses of spectral methods for IV regression (e.g., Wang, Luo, Li, Zhu, and Schölkopf, "Spectral Representation Learning for Conditional Moment Models," arXiv:2210.16525, 2022). Its role is to control the high-frequency tail of the estimation error when inverting a compact operator.

Regarding how restrictive this is for the NN-based estimator: we acknowledge that the condition P(f^∈Uβ)→1\mathbb{P}(\hat{f} \in U_\beta) \to 1 is stated in terms of the estimation error, which makes direct verification for unregularized neural ERM challenging. However, this does not mean the NN estimator is inherently incapable of satisfying it. In practice, explicit regularization (weight decay) or implicit regularization (early stopping) constrains the effective complexity of the learned function, which prevents the network from fitting the severely ill-posed high-frequency components of the error. This mechanism is well understood in the inverse problems literature as spectral regularization.

More importantly, a weaker condition suffices for our proof. Instead of the pointwise spectral decay ∑jj2β⟨f^−f0,uj⟩2≲1\sum_j j^{2\beta}\langle \hat{f}-f_0, u_j\rangle^2 \lesssim 1, the proof in Appendix E.5 only requires a tail-energy control of the form ∑j>J⟨f^−f0,uj⟩2≲wJ2\sum_{j>J}\langle \hat{f}-f_0, u_j\rangle^2 \lesssim w_J^2 with wJ≍J−βw_J \asymp J^{-\beta}. This type of condition is closely related to approximate source conditions (e.g., Hein and Hofmann, "Approximate source conditions for nonlinear ill-posed problems—chances and limitations," Inverse Problems, 2009) and variational source conditions (e.g., Hofmann, Kaltenbacher, Pöschl, and Scherzer, "A convergence rates result for Tikhonov regularization in Banach spaces with non-smooth operators," Inverse Problems, 2007), which are well-established tools for obtaining convergence rates in nonlinear ill-posed problems without imposing strict range-type source conditions. Under this weaker tail-energy condition, the same spectral truncation argument yields the identical rate in Theorem 4.2.

We believe these clarifications address the reviewer's concerns and accurately reflect the contributions of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.