acceptodds
Under review as a conference paper at ICLR 2027

Operator-Robust Instrumental Variable Regression under Joint Distribution Shifts

Abstract

Instrumental variable regression identifies causal effects under unobserved confounding, but only when moment conditions hold. However, weak identification increases the sampling error, while invalid instruments and distribution shifts can violate its moment restrictions. Instead of indirectly protecting against perturbations to the data distribution, we directly robustify the identifying moments. By tracing joint distributional perturbations through these moments, we derive a common score that jointly perturbs the treatment operator and outcome moment while preserving their relationship. This pushforward construction creates an operator-robust worst-case criterion and tractable convex average-case surrogates. A special case provides a nonparametric operator analogue of k-class regularization. We characterize how score robustness changes identification and when it preserves the causal target, and bound structural error in terms of identification strength, estimation error, and robustness-induced biases. Across a wide range of synthetic, semi-synthetic, and real-world settings, our method improves over state-of-the-art IV estimators, including under weak identification, invalid instruments, and distribution shift.

Primary area:
causal reasoning
Keywords:
instrumental variable regression, causal inference, distributionally robust optimization, conditional moment restrictions, kernel methods, distribution shift, weak instruments, invalid instruments

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Jm3LOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

In nonparametric additive IV (NPIV) models a structural function (here denoted f0f_0) is identified as the unique (under an injectivity condition) solution to a conditional moment restriction. This paper examines distributional robustness of such models. The paper takes an RKHS approach and considers replacing the usual conditional moment restriction (expressed using the RKHS) with a maximum over perturbations of the distribution of observables. The paper uses this as both an analytical device in order to investigate the robustness of identification in NPIV to certain types of misspecification, and also to motivate a robust estimator, which minimizes worst-case (over perturbations) NPIV objective function. There is a trade-off here: more robustness (i.e.,allowing for larger perturbations) can potentially lead to bias when the conditional moment restriction is correctly specified (i.e., it holds at the true f0f_0). The paper investigate the trade-offs this using semi-synthetic and real data.

Critical strengths

The idea here is interesting. Robustness is important in NPIV because these models are always misspecified (additive separabiltiy of the causal model is a strong assumption and is at best a good approximation). The proposed approach to distributional robust estimation seems intuitively reasonable in this setting. The paper is fairly thorough in its empirical investigations and the analysis is well thought-through.

Critical weaknesses

The writing could do with some improvements. The introduction is difficult to understand without having read the paper first. There are typos (e.g., "functiosn" on page 2, "conencted" on page 3, "varint" on page 5). ρ\rho is used before it is defined. Some discussion of the literature seems to contrast with what is in those papers, for example the paper claims that Newey & Smith, 2004 "seek the smallest shifts under which the CMR is satsifed and moments approach zero" which is a very strange reading of that paper. One of the biggest problems is that the paper does not clearly state the empirical method it actually proposes: it never points to a particular formula but instead readers are expected to figure this out from the discussion.

The paper does not appear to provide a complete asymptotic theory/inference approach for the proposed method. However, this appears to be a complicated problem so this is a reasonable omission in my opinion.

Questions for authors

How does the present work connect this to recent papers in the econometrics literature which explicitly analyse the failure identification when the conditional moment restriction is relaxed? In particular, https://www.aeaweb.org/articles?id=10.1257/aer.20231765 and https://arxiv.org/abs/1901.01241, the latter of which proposes a robust estimator. The approach in the present work is quite different, perturbing the distribution of observables rather than the moment condition, but the ideas are clearly related. That work should probably also be cited in the paper.

Regarding the proposed empirical method, there are some questions left unanswered. Does the distributionally robust method impart some implicit form of regularization (important given the ill-posedness of the NPIV estimation problem)? Proposition 2 suggests it might, but it was not clear to me whether that interpretation is correct?

1 comment

Thank you for the careful review, for your assessment of why robustness matters in nonparametric IV, and for the pointers to the econometrics literature.

Summary

  • Implicit regularization: yes, a Tikhonov-type term centered at the least-squares solution (Prop. 2, Fig. 2).
  • Deaner: we now cite and contextualize both papers in Sec. 3 and related work (Prop. 1).
  • The method in one formula: Eq. (15), with TAN–RMS written out in Eq. (16) and all criteria in Table 2 (App. D).
  • Newey and Smith, typos, undefined symbol: corrected.
  • Statistical theory: in App. F, we provide a fixed-feature sampling bound. A complete asymptotic and inference theory remains future work.

The tradeoff for TAN–RMS on the main design (first-stage R2=0.1R^2=0.1, Fig. 2):

κ\kappa01310
Squared bias of the population target0.430.080.120.31
Estimation error around it1.400.070.050.02

Details

Regularization. The response term adds τΣq\tau\Sigma_q to the normal matrix (1−τ)A⊤A(1-\tau)A^\top A (Prop. 2). For TAN–RMS the estimation error falls with the radius and the bias toward least squares grows beyond κ=1\kappa=1 (Sec. 6.2, Fig. 2). Removing the term at least doubles the error at every evaluated sample size on the main design under the same EB selection rule (App. J). The response term shrinks the IV fit toward the weighted least-squares fit of Y on T, whereas the ridge shrinks the coefficients toward zero. We keep both, and Corollary 1 gives the population target under both. k-class estimators often shrink toward least squares. For linear features and whitened instruments of constant norm, TAN–RMS is a k-class estimator (App. D).

Deaner. By reweighting along a=ε0h(Z)a=\varepsilon_0h(Z) of norm ρ\rho, we obtain Deaner's relaxed restriction EQ[ε0∣Z]=ν0(Z)E_Q[\varepsilon_0\mid Z]=\nu_0(Z), with ∥ν0/σ∥=ρ\|\nu_0/\sigma\|=\rho (Prop. 1, Sec. 3). In Corollary 1, we show how such a violation ν0\nu_0 affects our estimator: it passes through the same regularized inverse as the sampling error, so the radius limits the effect of both.

Presentation. We revised the introduction: it no longer refers to undefined terms, and it now explains why controlling small perturbations of the data helps when instruments are weak. We also corrected the related-work paragraph on generalized empirical likelihood. Moreover, we defined the ambiguity set earlier, and streamlined the notation.

We hope this resolves your concerns and would be grateful if you would reconsider your assessment. We are happy to clarify further.

Reviewer_XowoOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper studies instrumental variable regression under perturbations of the identifying moments and proposes IV estimators that are, by construction, robust to the considered perturbations. The authors adopt the DRO-like construction (1+a)dP(1+a)dP for the perturbed data distribution, and define the resulting distributional ambiguity set which translates into a structured uncertainty set for the treatment operator and outcome moment. The authors formulate the resulting worst-case moment violation and argue that it cannot be computed in closed form, and hence a lower and two upper bounds are proposed. Robustness to two types of shifts is considered: shifts in the instrument distribution only, which preserves the conditional moment restriction (KS), and shifts in the joint distribution, which could lead to it's violations, making the optimal solution not necessarily equal to f0f_0 (TAN). The authors then propose estimators for both cases which minimize one of the lower/upper bounds. The estimators are tested and compared to baselines on a range of synthetic and semi-synthetic datasets (where the structural error is measured) and real-world data (where the held-out projected residual error is measured).

Critical strengths

Some interesting insights from the paper are the translation of the distributional ambiguity set into the structured operator ambiguity set and the respective formulation of the worst-case moment violation, as well as separation into shifts which preserve the CMR and shifts that could violate it. The experiments cover many datasets and settings. The appendix seems well-constructed and sound.

Critical weaknesses

In my opinion, the main weaknesses of the paper are presentation&writing and the limited (or not well articulated) novelty. The theory mainly combines known components (maximum moment restriction, DRO), and their combination has been introduced before [1,2]. The proposed estimators are conceptually closely connected to k-class estimators, as studied in the appendix. In experiments, the proposed methods are also not compared against [1].

The motivation for why the IV estimator being robust to arbitrary shifts (TAN) helps the objective (estimating h0h_0) is not well-established. If we cannot guarantee that the resulting estimator is close to h0h_0, and the goal is not distributional robustness (or domain generalization), why should be a robust version preferred against a non-robust version for IV? For CMR-preserving shifts, the objective adds no information in population. It could for instance win in finite sample settings, but this is not studied in the paper.

Further, I find the conclusions from experiments flawed: the authors propose multiple estimators (3 are shown in the main plots). There is no clear winner among the 3, however, only the best of 3 is discussed in the main text. No strategy is given how to select one of the 3 estimators ahead of seeing test data, thus, it seems like the best estimator is read out from its held-out performance.

In total, the writing is hard to follow, and contains numerous typos. "ORION" as a method is never defined or depicted in the plots, but discussed multiple times in the main text as the best method. The notation is dense, and sometimes changed in the middle of the paper (T to X, f0f_0 to h0h_0, for which there is no conceptual need).

[1] Qu, Zhaonan, and Yongchan Kwon. "Distributionally robust instrumental variables estimation." arXiv preprint arXiv:2410.15634 (2024).

[2] Jakobsen, M. E., & Peters, J. (2022). Distributional robustness of K-class estimators and the PULSE. The Econometrics Journal, 25(2), 404-432.

Questions for authors
  • How should a practitioner choose between KS and TAN, and between the RMS/spectral/Frobenius version? How does your evaluation change if the choice is performed ahead of evaluation based on the proposed criteria?
  • How do the results change if all methods use the same tuning method (either all CV, or all EB)?
  • How was κ=3\kappa=3 chosen, was it selected using the reported benchmarks? With this κ\kappa, the resulting ambiguity set against which the methods "protect" seems far smaller than the shifts ($ \chi^2 $ up to 1) on which the methods are evaluated in the plots. Thus, it is unclear whether the benefit of the methods lies in protection against shifts, or in an unrelated effect. Could you elaborate on how the optimal $ \kappa $ varies with shift strength and sample size empirically?
  • Can you give a high-level motivation of why distributional robustness is beneficial in the general setting if one's goal is to estimate h0h_0?
3 comments

Thank you for the detailed review, and for noting the structured operator ambiguity set and the split into shifts that preserve or violate the moment restriction.

Summary

  • Why robustness helps estimate f0f_0: controlling moment sensitivity regularizes weakly identified directions (Cor. 1, Fig. 2).
  • Separate estimators: every result refers to exactly one criterion fixed in advance, and we removed the "ORION" name.
  • Notation and typos: the treatment is TT and the structural function ff throughout, and we corrected the typos.
  • Comparison with [1] and [2]: added (Table B, Fig. 17). If accepted, we will move these comparisons into the main figures.
  • Same tuning for all: reported with EB for all and with cross-validation for all (Table A, Figs. 15–16).
  • κ\kappa: we fixed κ=3\kappa=3 beforehand.
  • The evaluated shifts: they can exceed the ball; the gains are consistent with regularization, and the lowest-error κ\kappa remains between 1 and 3 across shifts (Fig. 9).

Table A: Same tuning, structural MSE on the main design by first-stage R2R^2. MMR-IV is a non-robust kernel moment estimator. Among the competitors, FGEL has the lowest mean error at most of these strengths under cross-validation.

First-stage R2R^20.020.050.10.150.20.30.50.7
TAN–RMS, EB0.620.370.230.180.170.160.150.12
MMR-IV, EB0.700.640.620.550.440.260.150.09
Sieve 2SLS, EB0.840.640.450.350.310.260.180.09
TAN–RMS, CV0.710.480.340.280.300.300.270.20
MMR-IV, CV0.700.640.620.550.440.260.150.09
FGEL, CV0.610.490.420.330.300.230.150.10

Table B: Comparison with [1] and [2], structural MSE on our main design under exclusion violations (Z→YZ\to Y) and instrument endogeneity (U–Z):

validZ→YZ\to Y 0.2Z→YZ\to Y 0.4U–Z 0.2U–Z 0.4
TAN–RMS0.160.581.560.240.39
DRIVE [1]0.290.681.600.350.46
PULSE [2]0.430.781.530.440.53

Details

Why robustness helps. Corollary 1 characterizes the bias, and Fig. 2 shows that TAN–RMS reduces estimation error as κ\kappa increases, while TAN–Frobenius has less bias and more estimation error.

Contribution. The k-class is a special case of our derivation. The broader contribution is the joint perturbation of treatment and outcome moments, the choice of score space and loss, and explicit conditions for preserving or moving the structural target (Props. 1–3, Cor. 1). This is why we study both RMS and the worst-case envelopes. Both [1] and [2] treat linear models, and [1] perturbs the treatment and outcome moments separately, whereas our perturbation moves them together.

Comparison with [1] and [2]. TAN–RMS has lower error than (kernelized) DRIVE and PULSE on our DGP design except at the largest exclusion violation, where the errors are similar. On the strongly identified linear design of [1], TAN-RMS (linearized, bootstrapped κ\kappa) matches DRIVE without violations (Fig. 17).

Which estimator. TAN reweights the full observation. It reduces the variance in weakly identified directions but moves the target, and likely works best with weak, low-dimensional instruments (Fig. 1). KS reweights the instruments only and keeps the target. It has the lower error with image instruments and with few observations (Figs. 1 and 6). Frobenius has less bias, RMS less estimation error (Fig. 2). Spectral is the tighter worst-case bound (Fig. 5). A rule for choosing between TAN and KS from data is open.

Instrument-only scores. They add no information in the population, as you note. The response term matters in finite samples: at R2=0.15R^2=0.15, removing it (κ=0\kappa=0) raises the error of KS–RMS from 0.28 to 0.96 (App. J).

Same tuning. For cross-validation, all methods use the same folds, validation criterion and grid of eleven penalties (App. H). TAN–RMS then has a lower error than every competitor for R2R^2 from 0.05 to 0.2 or 0.3 (with a 1-SE rule). With image instruments, CV does not perform well for us (Fig. 16). When using EB for every method, MMR-IV and Sieve 2SLS barely change, and TAN–RMS keeps its lead for R2≤0.3R^2\le 0.3 (Fig. 15).

Radius and shifts. We set ρ=κ/n\rho=\sqrt{\kappa/n} for the worst case and τ=κ/n\tau=\kappa/n for RMS, motivated by sampling error rates (n−1/2n^{-1/2}). The evaluated training shifts can exceed the ball, so the worst-case bound may not cover them. The observed gains are consistent with the regularization of Cor. 1 and Sec. 6.2. The lowest-error κ\kappa is 1 at almost every shift strength and 3 under the strongest joint shift (Fig. 9). Across sample sizes from 250 to 4 000 it is κ=1\kappa=1 or 2 (App. J).

We hope this resolves your concerns and would be grateful if you would reconsider your assessment.

I thank the authors for their comment and improving the presentation. Apart from the presentation improvement, my concerns regarding the novelty and evaluation details remain. I therefore raise the score by 1.

Thank you very much for acknowledging the improved clarity---much appreciated!

We'd like to briefly respond to your remaining concerns.

1. Re novelty. Both [1] and [2] study linear models. Our estimators are nonparametric, perturb the treatment and outcome moments jointly, and come with results that tell us when the structural target is preserved or moved (Props. 1-3, Cor. 1), which is a novelty. Not unlike how results on DRO estimators turn an ambiguity set to a known regularizer, we show that a specialization of our family can recover a classical k-class IV estimator. That is, we specialized to linear features and whitened instruments of constant norm, one member of the family, TAN-RMS, reduces to a k-class estimator (App. D).

In Tbl. B, we compare against [1] and [2]. We would also like to point out that all main figures already compare against SMM (Sinkhorn Method of Moments), a published, nonparametric, optimal-transport-based IV estimator.

2. Re evaluation. Every reported result is for a single criterion with a clearly-stated defined radius selection (kappa), and a selection rule (EB or CV). The radius and the EB rule were fixed before the experiments, and no parameter or rule is chosen using test data or the ground truth. We never report the best member of the family. We report each criterion / estimator individually. In Table A, we compare all under the same tuning rule and tuning budget for all methods, including the settings where cross-validation may perform worse than EB.

We consider selecting between KS and TAN as a design choice about which shifts to protect against (instrument-only or the full observation). However, we agree that selecting the estimator from data would be interesting.

Once again, thank you! We hope this resolves your remaining concerns and that you might consider our response further.

Reviewer_6rcaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a family of robust IV estimators that account for coupled changes in treatment operators and outcome moments under distribution shifts. It characterizes when the robust objectives are minimized at the true structural function and when their minimizers are biased, and evaluates the methods in weak-instrument, invalid-instrument, and shifted settings.

Critical strengths

The paper characterizes the tradeoff between robustness and structural function estimation bias. Proposition 2 explicitly characterizes the population bias of TAN–RMS, while Proposition 3 establishes conditions under which the upper-bound objectives remain minimized at the true structural parameter. Overall, I found the analyses interesting and potentially useful in practice.

Critical weaknesses

My major concern is the clearity of this paper. This is a technical paper. Although I have some related background, I found the paper difficult to read and understand, and I think it will be even more difficult for readers without a strong background in this area. In particular, the paper introduces several optimization objectives, but without explicitly defining the concrete estimators and explaining how they are used in practice. Readers have to piece together these details from different sections.

Questions for authors
  1. Please see the weakness part.
  2. Propositions 2 and 3 analyze population objectives without an additional ridge penalty, while the experiments use regularized empirical objectives. To what extent do the theoretical results apply to the estimators used in the experiments?
  3. How should the robustness parameters τ\tau and ρ\rho be selected in practice? Can Proposition 3 provide practical guidance?
  4. Some terminologies are introduced without explanation. For example, what is ORION in line 321, and what is HC0 moment transform in line 296? If these are standard terms, please provide references. Besides, there are some typos in the paper. I encourage authors to carefully proofread the paper.
2 comments

Thank you for the careful review and for your positive assessment of the tradeoff analysis in Props. 2 and 3.

Summary

  • Concrete estimator: now stated in one place, Sec. 3, Eqs. (15)–(16), with all criteria in Table 2 (App. D).
  • Props. 2–3 and the ridge: the new Corollary 1 covers the ridge penalty and violated moments (Sec. 6.2, Fig. 2).
  • Choosing ρ\rho and τ\tau: one number κ\kappa sets both, and we fixed κ=3\kappa=3 before we ran experiments (App. J, Figs. 13–14).
  • Prop. 3 as guidance: not directly, since its thresholds for target preservation depend on the unknown θ0\theta_0.
  • ORION, HC0, typos: "ORION" is removed. HC0 is the heteroskedasticity-consistent covariance estimate of White (1980), which we use to whiten the instrument features (Sec. 3, App. H). The typos are corrected.

Structural MSE of TAN–RMS along the radius (main design, first-stage R2=0.3R^2=0.3):

nκ=0\kappa=00.31310
2501.180.560.520.570.70
1 0001.140.250.240.260.41
4 0000.580.170.140.160.23

Details

Estimator. In Sec. 3, we now state the empirical estimator in Eq. (15), with TAN–RMS written out in Eq. (16). In the preceding paragraph, we clarified the whitening transform, sample moments, and response matrix; App. H specifies the empirical-Bayes rule for selecting λ\lambda on the training sample.

Ridge. In Corollary 1, we show the population target is displaced by three terms. That is, the pull toward least squares, a moment violation, and the ridge shrinkage. Measuring the fitted estimators, we see that the estimation error of TAN–RMS falls with the radius and its bias toward least squares grows beyond κ=1\kappa=1 (Sec. 6.2, Fig. 2). In Prop. 3 we derive the radii up to which the truth remains the minimizer of the population criterion without ridge, and in App. A, we extend this to a ridge. So, we distinguish penalty from coefficient shrinkage.

Radius. This is a re-parametrization: after whitening, the sampling error is at n−1/2n^{-1/2}, so we set ρ=κ/n\rho=\sqrt{\kappa/n} and τ=κ/n\tau=\kappa/n. In the aggregate synthetic experiments, TAN–RMS is better than the competitors for κ\kappa from 0.3 to 3, and removing the term (κ=0\kappa=0) at least doubles its error at every evaluated sample size in the main design using EB (App. J, Fig. 13). New results show that bootstrapping selects κ\kappa from the training data without structural truth or held-out observations. For TAN–RMS, it matches or reduces the error of fixed κ=3\kappa=3 in 19 of 20 settings. For KS–RMS, bootstrapping improves at every synthetic-data first-stage strength and at strong image instruments (R2=0.5R^2=0.5). On weak image instruments (R2≤0.1R^2\le 0.1), a fixed κ=3\kappa=3 performs well (Fig. 14). Practical radius selection is addressed by this sensitivity analysis and the bootstrap procedure.

Terms. We removed the undefined name "ORION". Every criterion is now named by its score space and loss, for example TAN–RMS, and is clearly defined in Sec. 3 and App. D.

We hope this resolves your concerns and would be grateful if you would reconsider your assessment. We are happy to clarify further.

Thank you for the rebuttal. My questions are addressed and I will keep my positive evaluation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.