acceptodds
Under review as a conference paper at ICLR 2027

Outstrumental Variable Regression

Abstract

Estimating the causal effect of a treatment on an outcome from observational data in the presence of unobserved confounding requires additional structure or assumptions. Existing identification strategies such as instrumental variable re- gression exploit exogenous variation in only the treatment . We show that the causal effect can also be identified if the outcome mechanism varies across a finite set of environments, even if the treatment distribution is unchanged. Concretely, we introduce Outstrumental Variable Regression (OVR), which proceeds in three steps: (i) compute the conditional expectation of given in each environ- ment; (ii) form the vector of differences in these conditional expectations between all environments and a reference environment (we call this vector contrast); and (iii) regress Y on X while adjusting for the contrast and environment. We prove that this yields the true causal effect if the following two conditions hold: inclusion restriction, i.e., the contrast determines the conditional confounding bias; and pos- itivity, i.e., the treatment varies sufficiently strongly after adjustment. We prove that a finite-sample version of OVR is consistent under regularity conditions and demonstrate the procedure’s effectiveness on synthetic and real-world data.

Primary area:
causal reasoning
Keywords:
causal inference, causal effect identification, unobserved confounding, multi-environment learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6wbGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Outstrumental Variable Regression (OVR), which identifies a linear causal effect under unobserved confounding by exploiting variation in the outcome mechanism across environments, rather than in the treatment as IV does. OVR differences environment-specific conditional means to cancel the causal term, then regresses YY on XX after partialling out these contrasts. The authors prove identification and consistency under an inclusion restriction and positivity, and evaluate the method on synthetic data and KuaiRand-Pure.

Critical strengths
  1. Original idea. Using heterogeneity in the outcome mechanism across environments to identify causal effects, instead of variation in the treatment as IV does, is new and interesting. It complements existing multi-environment and IV approaches.

  2. Clear presentation. The three-step procedure and the worked example (§2.2, Fig. 3) make the intuition easy to follow.

  3. Honest treatment of limitations. The paper includes a bias identity and sensitivity bound for IR violations, discusses positivity failures, and supports the smoothing step with ablations.

Critical weaknesses
  1. Restrictive inclusion restriction. IR requires the contrasts to fully determine the baseline nuisance. The sufficient settings in §4 rely on special structure: a common low-rank confounding factor with enough environments, an unconfounded environment, or a recoverable latent state. IR also cannot be tested from observational data, and it must hold together with positivity, which rules out injective nuisances such as the scalar linear-Gaussian case. It is unclear how often this holds in practice. Weakening IR, or giving guidance on calibrating δIR\delta_{IR} in the sensitivity bound, would broaden applicability.

  2. Connection to double ML. The estimation step is essentially Robinson's (1988) partialling-out estimator in cross-fitted DML form (Chernozhukov et al., 2018), with W=(C(X),E)W = (C(X), E) as the adjustment variable. Neither is cited.

  3. Real-data benchmark. KuaiRand-Pure includes a randomized arm, so the benchmark mainly tests recovery of the randomized-arm slope, not adjustment for unmeasured confounding in observational data. It also reflects an assignment shift, not the outcome-mechanism shift the paper motivates. An observational dataset with outcome-mechanism variation would be more convincing.

  4. Lack of baselines. OLS, FE and anchor (γ = 2) nearly coincide when E⊥XE \perp X, so they add little. Missing comparisons include DML with an oracle contrast (to separate identification from first-stage error), environment IV/2SLS where E→XE \to X, per-environment OLS on KuaiRand, and the method of Kivva et al. (2025) in a scalar linear setting where it applies.

Questions for authors

What does βOVR\beta_{OVR} estimate under nonlinear or heterogeneous effects? IV has LATE and its extension to variable treatment intensity (Angrist & Imbens, 1995). For a nonlinear invariant effect gg with scalar XX, and assuming IR still holds, βOVR\beta_{OVR} appears to be a positively weighted average of g′g'. Can the authors characterize this estimand and its weights, including for vector XX?

2 comments

We thank the reviewer for highlighting the originality of OVR, the clarity of the three-step procedure and worked example, and our discussion of when the assumptions fail. We address their questions below.

W1 "Restrictive inclusion restriction. IR requires the contrasts to fully determine the baseline nuisance. The sufficient settings in §4 rely on special structure: a common low-rank confounding factor with enough environments, an unconfounded environment, or a recoverable latent state. IR also cannot be tested from observational data, and it must hold together with positivity, which rules out injective nuisances such as the scalar linear-Gaussian case. It is unclear how often this holds in practice. Weakening IR, or giving guidance on calibrating δIR\delta_{IR} in the sensitivity bound, would broaden applicability."

A: We agree that IR is a real restriction, that it cannot be verified from data, and that it excludes the scalar linear-Gaussian case (Sections 3.1 and 5). IR and positivity hold together when the confounding bias depends on XX only through a coarser summary than XX itself, for example when the confounder takes few values, as in the new light-tunnel experiment (Section 6.3), or when it acts through a low-dimensional summary of a multivariate treatment (Section 4.3). Positivity can be assessed from data (Appendix B.1). In the light tunnel, where a randomized reference is available, we can also check IR empirically: the nuisance is a linear function of the contrast with R2>0.99R^2>0.99 in all three assignments (Appendix E.8, Figure 12(c)).

Exact IR is not required. OVR recovers β0\beta_0 if and only if the nuisance remaining after adjustment is uncorrelated with the residual treatment (Section 3.1, "Relaxing IR"), and otherwise its bias is bounded in terms of δIR\delta_{\mathrm{IR}}. Synthetic settings (d)–(f) and the new airport experiment (Section 6.4) do not impose IR, and OVR still has lower error than all baselines there (in (d)–(f) for n>2,000n>2{,}000).

We have now added guidance on calibrating δIR\delta_{\mathrm{IR}} to Section 3.1. It is measured in outcome units: the root-mean-square part of the baseline confounding bias that the contrasts do not explain. For scalar XX, the bound implies that the true slope can be zero only if δIR≥∣βOVR∣E[X~2]\delta_{\mathrm{IR}}\geq|\beta_{\mathrm{OVR}}|\sqrt{\mathbb E[\widetilde X^2]}. The right-hand side can be estimated, and a practitioner can compare it with the unexplained confounding they consider plausible, as in standard sensitivity analyses.

W2 "Connection to double ML. The estimation step is essentially Robinson's (1988) partialling-out estimator in cross-fitted DML form (Chernozhukov et al., 2018), with W=(C(X),E)W=(C(X),E) as the adjustment variable. Neither is cited."

A: We agree and apologize for the omission. We have now added both references in Section 5: given W=(C(X),E)W=(C(X),E), the second step of OVR is Robinson's partialling-out estimator, and we use cross-fitting as in double machine learning.

W3 "Real-data benchmark. KuaiRand-Pure includes a randomized arm, so the benchmark mainly tests recovery of the randomized-arm slope, not adjustment for unmeasured confounding in observational data. It also reflects an assignment shift, not the outcome-mechanism shift the paper motivates. An observational dataset with outcome-mechanism variation would be more convincing."

A: We agree that KuaiRand contains a randomized arm and corresponds to the setting of Proposition 2; we have now clarified this in the paper (Section 6.2 and Appendix E.7). We would not describe it as only an assignment shift. The policy changes which video a user sees, and XX contains only six attributes of that video. A recommender selects videos using much more than these attributes, so we expect the policy to affect watch time also through video features that are not in XX.

To directly test an outcome-mechanism shift, we have now added the light-tunnel experiment: rotating a polarizer changes the hidden light source’s contribution to the outcome while keeping the joint treatment–confounder assignment fixed (Section 6.3; Appendix E.8). We have now also added an observational airport-delay benchmark with taxi-in time withheld from the estimators (Section 6.4; Appendix E.9).

W4 "Lack of baselines. OLS, FE and anchor (γ=2\gamma=2) nearly coincide when E⊥XE\perp X, so they add little. Missing comparisons include DML with an oracle contrast (to separate identification from first-stage error), environment IV/2SLS where E→XE\to X, per-environment OLS on KuaiRand, and the method of Kivva et al. (2025) in a scalar linear setting where it applies."

A: We have now added all four requested comparisons to the revised manuscript (Appendices E.4 and E.7).

DML with an oracle contrast. This oracle method replaces DOVR's estimated contrast with the population contrast C(X)C(X), keeping the sample split and residualization fixed (Appendix E.4, new Figure 8). In settings (a) and (c), at n=10,000n=10{,}000, the oracle contrast reduced the mean ℓ2\ell_2 coefficient error of DOVR from 0.161 to 0.038 and from 0.330 to 0.108, respectively, so most of the remaining error there comes from estimating the contrast. We did not compute C(x)C(x) in (b), where it requires numerical integration over the two-dimensional confounder conditional on X=xX=x in each environment, or in (d)–(f), where we would need to approximate E[Y∣X=x,E=e]\mathbb{E}[Y\mid X=x,E=e] for each environment.

Environment IV. In settings (b), (e), and (f), 2SLS had substantially higher mean ℓ2\ell_2 coefficient error than SOVR (Appendix E.4, Figure 8). In (a) and (d), EE is independent of XX, so the population first stage is zero (making population 2SLS inapplicable). In (c), four environments provide at most three independent differences in mean treatment, which are insufficient to identify four treatment coefficients. We have now added a setting where EE is a valid instrument and OVR's inclusion restriction fails (Figure 9(b)), as expected, 2SLS had the lowest mean absolute error (0.0310.031), compared with 0.4960.496 for SOVR and 0.4950.495 for DML with an oracle contrast.

Per-environment OLS on KuaiRand. OLS on the standard-policy data has distance 0.198 to the held-out reference and OLS on the random-policy data 0.010, against 0.019 for SOVR (Table 4).

Kivva et al. (2025). Their estimator assumes a scalar treatment and a linear SCM with independent noise. Where this holds, it is the most accurate non-oracle method: in a scalar experiment we have now added (Appendix E.4, Figure 9(a); mean absolute error 0.0260.026, followed by SOVR with 0.1340.134) and in the additive light-tunnel assignment (1.4% against 6.3% for SOVR; Table 2). In the binary and three-level light-tunnel assignments, which are outside its assumptions, its error is 244% and 370%, against 2.3% and 2.9% for SOVR. The six original synthetic settings (a)–(f) have four-dimensional treatments, so it cannot be used there.

Q1 "What does βOVR\beta_{OVR} estimate under nonlinear or heterogeneous effects? IV has LATE and its extension to variable treatment intensity (Angrist & Imbens, 1995). For a nonlinear invariant effect gg with scalar XX, and assuming IR still holds, βOVR\beta_{OVR} appears to be a positively weighted average of g′g'. Can the authors characterize this estimand and its weights, including for vector XX?"

A: We thank the reviewer for this suggestion. The reviewer's intuition is correct. Let W=(C(X),E)W=(C(X),E) and let X′X' be a conditionally independent copy of XX given WW. Write D=X−X′D=X-X' and M=E[DD⊤]=2ΣCM=\mathbb E[DD^\top]=2\Sigma_C. Under IR, the nuisance is constant given WW, so the residual regression defining OVR gives

βOVR=M−1E[D{g(X)−g(X′)}]\beta_{\mathrm{OVR}}=M^{-1}\mathbb E[D\lbrace g(X)-g(X')\rbrace]

whenever positivity holds. For scalar XX, this is an average of the secant slopes (g(X)−g(X′))/(X−X′)(g(X)-g(X'))/(X-X') with weights D2/E[D2]D^2/\mathbb E[D^2]. If gg has a bounded continuous derivative on the treatment interval, then

w(t)=\frac{\mathbb E\left[|D|\mathbf{1}\lbrace\min(X,X')\le t<\max(X,X')\rbrace\right]} {\mathbb E[D^2]}.$$ Thus $w(t)\geq 0$ and $\int w(t)dt=1$. A treatment level $t$ receives weight from pairs that straddle it within the same contrast and environment. For vector $X$, the same argument gives matrix weights that average to the identity but can mix partial derivatives with signed coefficients, so the positive-weight interpretation need not hold coordinate by coordinate. On heterogeneous effects: Equation (1) restricts only the average response, so effects may differ between individuals; slopes that differ between environments are treated in the new Appendix H. We have now added the weight formulas and their derivation as the new Appendix G of the revised manuscript.
Reviewer_7pCyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper provides a novel causal effect estimation method that uses the variance of the conditional means across multiple environments to account for confounding and identify the causal effect on this outcome variable. In a sense, this approach is the complement of the classic instrumental variable regression technique, in that it uses exogenous variation of the conditional means rather than the treatment -- as in instrumental variable regression -- to adjust for unobserved confounding and isolate a causal effect. Given this, the authors term their method "outstrumental" regression. The basic idea is to use the differences of the outcomes between environments, what the paper calls contrasts, to estimate confounding, and then regress the outcome on such contrasts and the treatment variable to estimate the causal effect. The key assumptions the authors use to prove identifiability in this set up are: 1) linear caudal effects, 2) the confounding bias in the reference environment, can be read off from how the environments’ outcomes differ, and 3) causal effects are invariant across environments. They provide synthetic and real data experiments to test their method.

Critical strengths

An interesting and novel new causal estimation method that is the complement of classic instrumental variables.

Critical weaknesses

I'm struggling a little with your real-world experiment. The two environments here appear to be the assignment policies: the standard recommender and random exposure. A policy here decides which video a user sees, so the policy acts directly on the treatment channel and changes the confounding, rather than the outcome mechanism directly. As the environment changes treatment, this appears to be more in line with classic instrumental variable regression, rather than outstrumental regression. Can you help me understand what's being shown in this experiment, and why it's a direct test of your method?

My reading of this is as follows: This is essentially the experimental-grounding case from your proposition 2, rather than the direct outcome-shift story. If the random policy makes q_random(x) roughly constant, then C(x) = q_standard(x) − q_random(x) \approx q_standard(x) − const, so the contrast directly reveals the recommender’s selection bias, and IR holds trivially. Is this reading correct? If so, a discussion of this in the paper would be very useful.

Finally, as environment appears to directly impact treatment (i.e. the video recommended), an IV-type approach would also seem to be applicable here as a baseline. Do you agree? if so, it would be quite instructional to see this baseline for comparison. Moreover, a fairer baseline in this experiment would simply be to regress on the random-policy training data alone. I would also be very interested in seeing that baseline.

If these points could be addressed adequately I would raise my score.

Questions for authors

The need for the causal invariance assumption, as well as the assumption that confounding varies in a sufficient way across environments, seem reasonable to me. However the need for linear causal effect seems a key ingredient to the proof -- without it, the causal effect won't "cancel" in the environment contrasts and they won't capture confounding alone. Can this be weakened in anyway? Moving beyond contrasts to a model that aims to look for differences between environments that can only be explained by variance across environment rather that within? One immediate would be to keep the linear decomposition between the causal effect and the confounding, but allow the causal effect itself to be a non-linear function of the treatment. In this setting, the causal effect would still cancel in the environment contrasts.

One thing that I think would improve the readability of the paper would be to provide in the introduction at least one clear, real world motivational examples that illustrate in what settings one might use outstrumental regression and which could help show practitioners the clear difference between it and IV regression.

2 comments

We thank the reviewer for describing OVR as an interesting and novel complement to instrumental variable regression, and we address their questions below.

W1 and W2 "[...] A policy here decides which video a user sees, so the policy acts directly on the treatment channel and changes the confounding, rather than the outcome mechanism directly. As the environment changes treatment, this appears to be more in line with classic instrumental variable regression, rather than outstrumental regression. Can you help me understand what's being shown in this experiment, and why it's a direct test of your method?" and "My reading of this is as follows: This is essentially the experimental-grounding case from your proposition 2, rather than the direct outcome-shift story. If the random policy makes qrandom(x)q_{\mathrm{random}}(x) roughly constant, then C(x)=qstandard(x)−qrandom(x)≈qstandard(x)−constC(x)=q_{\mathrm{standard}}(x)-q_{\mathrm{random}}(x)\approx q_{\mathrm{standard}}(x)-\mathrm{const}, so the contrast directly reveals the recommender's selection bias, and IR holds trivially. Is this reading correct? If so, a discussion of this in the paper would be very useful."

A: The reviewer's reading in W2 is correct: KuaiRand corresponds to the setting of Proposition 2 (one unconfounded environment). On W1, we agree that the policy changes which video a user sees. But XX contains only six attributes of the video, and a recommender selects videos using much more than these. We therefore cannot assume that the policy affects watch time only through XX, as IV would require. OVR does not need this assumption: with qrandom(x)=cq_{\mathrm{random}}(x)=c, the contrast C(x)=qstandard(x)−cC(x)=q_{\mathrm{standard}}(x)-c is the recommender's selection bias, so IR holds. What the experiment shows is that OVR, without being told which policy is random, recovers the slope of the held-out random-policy data (distance 0.019, against 0.129 for OLS). We have now added this discussion to Section 6.2 and Appendix E.7.

The light-tunnel experiment we added to the revised manuscript (Section 6.3; Appendix E.8; see also the general response for details) more directly tests an outcome-mechanism shift by changing how strongly the hidden light source affects the sensor while holding treatment assignment fixed.

W3 "Finally, as environment appears to directly impact treatment (i.e. the video recommended), an IV-type approach would also seem to be applicable here as a baseline. Do you agree? if so, it would be quite instructional to see this baseline for comparison. Moreover, a fairer baseline in this experiment would simply be to regress on the random-policy training data alone. I would also be very interested in seeing that baseline."

A: We have now added both baselines.

We list OLS on the random-policy data as an oracle because it must be told which policy is random. OVR is not given this information. OLS on the random-policy training data has distance 0.010 to the held-out reference (Table 1). It is the same regression as the reference, fitted on disjoint data, so this distance reflects sampling noise only. Smoothed OVR reaches 0.019 and pooled OLS 0.129.

For IV, the policy indicator is a single binary instrument, so 2SLS cannot estimate six coefficients on KuaiRand, in addition to the exclusion concern discussed above. We therefore added the comparison in simulations (Appendix E.4). In a new design where the environment is a valid instrument and IR fails, IV is the most accurate method at n=10,000n=10{,}000 (error 0.031, against 0.496 for Smoothed OVR; Fig. 9b). In the three settings of Fig. 5 where the environment shifts the treatment, it also changes the outcome mechanism, and the error of IV is more than ten times that of OVR (Fig. 8). The two methods thus cover different settings.

Q1 "[...] However the need for linear causal effect seems a key ingredient to the proof – without it, the causal effect won't "cancel" in the environment contrasts and they won't capture confounding alone. Can this be weakened in anyway? Moving beyond contrasts to a model that aims to look for differences between environments that can only be explained by variance across environment rather that within? One immediate would be to keep the linear decomposition between the causal effect and the confounding, but allow the causal effect itself to be a non-linear function of the treatment. In this setting, the causal effect would still cancel in the environment contrasts."

A: The reviewer is right: any causal response gg that is shared across environments cancels from the contrasts, linear or not. Linearity is needed in the next step. With μe(x)=g(x)+qe(x)\mu_e(x)=g(x)+q_e(x), the data determine gg only up to an additive function of the contrast: replacing g(x)g(x) by g(x)+h(C(x))g(x)+h(C(x)) and every qe(x)q_e(x) by qe(x)−h(C(x))q_e(x)-h(C(x)) changes neither the observed regressions nor IR. We can thus learn how gg differs between treatment values with the same contrast, but not between values with different contrasts, and an unrestricted gg is not identified.

A restriction on gg removes this ambiguity, and it need not be linearity in xx. If g(x)=θ⊤ϕ(x)g(x)=\theta^\top\phi(x) for known features ϕ\phi, for example ϕ(x)=(x,x2)⊤\phi(x)=(x,x^2)^\top, the same procedure with ϕ(X)\phi(X) in place of XX identifies θ\theta under IR, provided ϕ(X)\phi(X) retains variation after adjusting for the contrast and the environment. We have now added this extension as Appendix F. If OVR is instead applied unchanged when gg is nonlinear and XX is scalar, it returns a positively weighted average of the derivative g′g' (Appendix G).

Q2 "One thing that I think would improve the readability of the paper would be to provide in the introduction at least one clear, real world motivational examples that illustrate in what settings one might use outstrumental regression and which could help show practitioners the clear difference between it and IV regression."

A: We have now added such an example to the introduction (Section 1). Consider an electrically heated building observed over two winters, before and after its outdoor lighting was replaced by LEDs. We want the effect of outdoor temperature XX on daily electricity use YY. The season HH confounds the two: midwinter days are colder and darker, so they need more heating and more lighting. The retrofit (the environment EE) changes how much electricity an hour of darkness costs, but not how much heating a degree of cold requires.

This is the opposite of an instrument. An instrument would have to change the temperature without otherwise affecting electricity use. The retrofit does not change the temperature at all and affects electricity use directly, so IV cannot use it. OVR instead uses the change in the seasonal pattern between the two winters to remove the confounding; for this it also needs days with different temperatures but the same expected darkness (positivity). The (new) light-tunnel experiment (Section 6.3) realizes this structure physically, with both conditions satisfied.

Reviewer_AhwzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Outstrumental Variable Regression (OVR), which constructs confounding-adjustment variables from differences between environment-specific outcome regressions. The authors establish identification under a common linear causal response, environment overlap, an inclusion restriction (IR), and a residual-variation condition, and prove consistency of a smoothed estimator under additional regularity conditions. The method is evaluated on six synthetic settings and KuaiRand-Pure.

Critical strengths
  1. The paper uses changes in outcome mechanisms to construct confounding-adjustment signals, offering an original complement to conventional instrumental-variable methods based on exogenous treatment variation. Under its identification conditions, OVR can identify causal coefficients even when the treatment distribution remains unchanged across environments.

  2. The paper provides identification results, sufficient structural conditions, a bias bound for IR violations, and consistency analysis for the smoothed estimator, with supporting proofs in the appendix.

  3. The component ablations and smoothing-scale sensitivity analyses in the appendix help explain the roles of individual components and the sources of empirical performance.

Critical weaknesses
  1. Equation (1) assumes a linear causal response with the same slope across environments. I find it hard to see why environmental changes would alter confounding but leave this slope unchanged. The examples show that this is mathematically possible, but give limited practical support for this assumption holding together with IR and the residual-variation condition.

  2. The OLS coefficients from random-policy data serve as an empirical reference, but getting closer to them does not establish that OVR identifies causal effects in this application. It remains unclear why short-video recommendation satisfies the assumptions needed by OVR.

  3. The analysis covers violations of IR, but all synthetic settings keep the causal slope unchanged across environments. The results therefore provide limited evidence about what happens when Equation (1) fails because causal effects differ across environments.

Questions for authors
  1. What practical mechanisms or prior studies support the common linear causal response in Equation (1)? When is it plausible together with IR and the residual-variation condition?

  2. Which real-world settings are suitable or unsuitable for OVR? In short-video recommendation, what supports a common causal slope and the assumption that regression contrasts capture the conditional confounding bias?

  3. What do OVR estimates represent when causal slopes differ across environments? How do the estimates change as these differences grow?

2 comments

We thank the reviewer for highlighting the consistency result, and for noting that the ablations clarify how the method works. We address their questions below.

W1, Q1 and Q2 "Equation (1) assumes a linear causal response with the same slope across environments. I find it hard to see why environmental changes would alter confounding but leave this slope unchanged. [...]" and "What practical mechanisms or prior studies support the common linear causal response in Equation (1)? When is it plausible together with IR and the residual-variation condition?" and "Which real-world settings are suitable or unsuitable for OVR? [...]"

A: The causal effect is constant over the environments (up to environment-specific intercepts) whenever the treatment and the confounder reach the outcome through separate pathways whose contributions are additive, and the environment acts on the confounder's pathway (this would allow the confounding to change). Equation (1) states this: on average, the environment does not modify the effect of XX. It restricts only the average response; the confounding term is otherwise unrestricted.

In the light-tunnel experiment we have now added (Section 6.3), the treatment LED shines directly on the sensor, while light from the hidden source HH passes through the polarizers. Rotating the polarizer (the environment) reduced the effect of HH on the sensor by 77% and changed the randomized treatment slope by 0.5%. The other two conditions hold alongside: the assignment of (H,X)(H,X) is the same at both angles, so the contrast is proportional to the expected hidden-light contribution (IR), and several treatment values share the same contrast (residual variation).

A structural coefficient shared across environments is the standard assumption of multi-environment causal methods (Rothenhäusler et al., 2019; 2021; Kivva et al., 2025; see also Section 4.1). Equation (1) is weaker than the linear SCMs used in these works: only the average causal response is linear.

In general, OVR applies when (i) the environment changes how strongly a hidden factor reaches the outcome through a pathway the treatment does not use (the light tunnel), or (ii) one environment is unconfounded, even if we do not know which (Proposition 2). In both cases the confounding bias must depend on XX only through a coarser summary, such as a discrete confounder state, so that treatment variation remains. It is unsuitable when (i) the environment modifies the treatment effect itself (Appendix H); (ii) the confounding bias is the same in every environment, so the contrast carries no information and an instrument is needed (Section 5); or (iii) the contrast is one-to-one in XX, as in scalar linear-Gaussian models, so no treatment variation remains.

Case (iii) can be detected from data with the diagnostic in Appendix B.1. IR, like the exclusion restriction in IV, needs justification from the application; the bound in Section 3.1 quantifies the bias if it holds only approximately.

W2 and Q2 "The OLS coefficients from random-policy data serve as an empirical reference, but getting closer to them does not establish that OVR identifies causal effects in this application. It remains unclear why short-video recommendation satisfies the assumptions needed by OVR." and "[...] In short-video recommendation, what supports a common causal slope and the assumption that regression contrasts capture the conditional confounding bias?"

A: Under the random policy, the video shown does not depend on the user. The users who see videos with attributes xx are therefore comparable for every xx, and the random-policy regression E[Y∣X=x,E=rand]\mathbb{E}[Y\mid X=x,E=\mathrm{rand}] equals the expected watch time if we showed a video with attributes xx to a randomly chosen user, that is, E[Y∣do(X=x)]\mathbb{E}[Y\mid \mathrm{do}(X=x)]. (The reference coefficient does not tell us about the effect of changing a single attribute of a given video; we now state this explicitly in Section 6.2.) The standard policy shows the same videos to the same users on the same interface, with the same number of interactions per user in both environments, and changes only which video is matched to which user. We therefore assume that the causal effect is shared, and only the confounding differs. Because the random environment is unconfounded, IR holds by Proposition 2 if the effect is linear. Even if the slopes differed between the policies, OVR would recover (under positivity) the slope of the random environment, because that environment is unconfounded (see the new Appendix H).

The experiment checks this argument empirically: OVR is not told which environment is randomized, yet its estimate lies at distance 0.019 from the held-out randomized slope, compared with 0.129 for OLS and 0.010 for OLS fitted on the random environment alone.

W3 and Q3 "The analysis covers violations of IR, but all synthetic settings keep the causal slope unchanged across environments. The results therefore provide limited evidence about what happens when Equation (1) fails because causal effects differ across environments." and "What do OVR estimates represent when causal slopes differ across environments? How do the estimates change as these differences grow?"

A: Equation (1) allows environment-specific intercepts; what must be shared is the slope. If the slopes βe\beta_e differ, OVR still returns a single coefficient, and it does not depend on which environment is used as the reference for the contrasts. The bias bound of Section 3.1 then holds for every environment rr, with βr\beta_r in place of β0\beta_0: OVR is close to βr\beta_r whenever the contrasts explain the confounding bias of environment rr (Appendix H). In particular, if one environment is unconfounded, OVR recovers (under positivity) the slope of that environment, whatever the slopes of the others.

As the differences grow, so does the term (βe−βr)⊤x(\beta_e-\beta_r)^\top x that they add to the contrasts. The contrasts then mix slope differences with confounding, and adjusting for them can remove treatment variation in the directions in which the slopes differ. Both effects can loosen the bound. In the light tunnel, the randomized slopes differ by 0.5% between the two angles, and OVR's error is 2.3%.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

General response

We thank all reviewers for their time and the valuable feedback we received. Many reviewers mentioned real-world experiments and practical applicability of OVR. To address this we have now added two real-world experiments to the revised manuscript (§6.3 and §6.4). More precisely, we consider real-world data for which a credible reference for β0\beta_0 is available (without it evaluation is impossible). The reference is obtained either by randomizing the treatment (the new causal chambers experiments and the KuaiRand experiment) or from a known accounting identity whose remaining term is withheld from the methods (the new airport experiment).

Causal Chambers light tunnel (new). We used the light tunnel of Gamella, Peters and Bühlmann (2025) (Section 6.3; Appendix E.8). The outcome YY is the reading of an infrared light sensor. The sensor receives light directly from a local LED placed next to the sensor with brightness XX (the treatment) and from a light source placed behind two polarizers with brightness HH, which in this experiment we consider hidden. To introduce confounding, we assigned XX according to HH. We withheld HH from OVR and competitors. The environment EE is the angle of the second polarizer relative to the first (0∘0^\circ or 60∘60^\circ). Light from the red source passes through the polarizer and its transmitted intensity thus decreases depending on EE; the light from the treatment LED (XX) reaches the sensor unaffected by the polarizers. We thus have X→Y←EX \rightarrow Y \leftarrow E and X←H→YX \leftarrow H \rightarrow Y. We used the same assignment rule for XX and HH at both values of EE. In separate measurements that set XX, HH and the angle independently, raising HH from 3232 to 192192 increased the mean of YY by 909909 units in E=0∘E= 0^\circ and 210210 units in E=60∘E=60^\circ. To evaluate the estimators, we measured β0\beta_0 by randomizing XX. We compare three assignments of XX and HH. In the binary assignment, HH takes two brightness levels. We set XX depending on HH so that the distribution of HH given XX is the same for pairs of treatment values, such as X=12X=12 and X=20X=20, which leaves variation in XX after adjusting for (C(X),E)(C(X), E). The three-level assignment adds an intermediate brightness level for HH and sets XX so that, for the same pairs of treatment values, the distribution of HH given XX differs but its mean contribution to YY does not. This illustrates that IR concerns only the conditional mean of the confounding contribution. In the additive assignment, we set X=a+bH+εXX=a+bH+\varepsilon_X, with noise independent of HH, matching the treatment model of Kivva et al. (2025). The following table reports the mean error in % of the randomized reference: our methods SOVR and DOVR ranked first and second among non-oracle methods in both the binary and three-level assignments; in the additive assignment, (as expected, as this is a special case matching the model of Kivva et al. (2025)) the moment estimator of Kivva et al. (2025) ranked first, followed by DOVR and SOVR.

MethodBinaryThree-levelAdditive
OLS24.617.437.2
FE24.217.437.6
Anchor (γ=2\gamma=2)25.017.436.8
LNN39.429.949.4
DOVR3.43.54.2
SOVR2.32.96.3
Kivva et al.2443701.4
DML with an oracle contrast1.11.72.8
OLS with observed HH (oracle)0.20.10.1

Airport delays (new). We used 200,000200{,}000 flights from the July 2024 U.S. Bureau of Transportation Statistics records (Section 6.4; Appendix E.9). The outcome YY is arrival delay. The treatment vector XX contains departure delay, taxi-out time, and airborne time minus scheduled gate-to-gate duration, all in minutes. With taxi-in time HH (which, in this experiment, we consider hidden), the flight-time accounting identity gives

Y=X1+X2+X3+H.Y=X_1+X_2+X_3+H.

Thus the coefficient target (1,1,1)⊤(1,1,1)^\top is known; it is the effect of XX at fixed taxi-in time. During estimation, the reported estimators do not use HH. The two environments are groups of ten destination airports. Using U.S. Census regions, East (E=0E=0) contains the Northeast and South (ATL, BOS, CLT, DCA, DFW, EWR, JFK, LGA, MCO, and PHL), and West (E=1E=1) contains the Midwest and West (DEN, DTW, LAS, LAX, MSP, ORD, PHX, SEA, SFO, and SLC). We selected the twenty destinations with the most completed, non-diverted arrivals on July 8–31, before applying treatment, outcome or taxi-in filters. After range and timing-identity checks, we sampled 100,000100{,}000 flights without replacement within each geographic group. Airport selection and estimation use only July 8–31. We formed twenty bootstrap samples by resampling the 2424 calendar days with replacement, jointly across environments, and refitted all methods with a fixed training seed. The table reports mean coefficient errors ∥β^−β0∥2\lVert\widehat\beta-\beta_0\rVert_2 and Monte Carlo standard errors of the means (the error standard deviation divided by 20\sqrt{20}).

MethodMean errorMonte Carlo s.e.
OLS0.08200.0007
FE0.08330.0007
Anchor (γ=2\gamma=2)0.08080.0006
LNN0.08980.0061
DOVR0.05860.0021
SOVR0.05900.0042

Environment IV has one instrument for three treatment coefficients. The population contrast required by oracle-contrast DML is unknown, and the implemented Kivva et al. (2025) estimator requires a scalar treatment. These methods therefore do not apply here.

None of the reviews mentioned the appendix section on VOVR. Since it is technical and slightly out of scope, we have removed it from the revised paper. We are happy to restore it if a reviewer disagrees.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.