acceptodds
Under review as a conference paper at ICLR 2027

Confounder Detection via Treatment Intent: A New Observational Study Design

Abstract

Understanding the effects of interventions is central to scientific progress, with randomized controlled trials (RCTs) regarded as the gold standard for causal inference in many applied fields. However, RCTs are costly, time-consuming, and often constrained by ethical or practical limitations, motivating the need for causal methods able to draw conclusions from observational data. While such data is collected at ever larger scale, making its use for causal inference is often hindered by the fact that not all variables affecting treatment allocation and the outcome are observed – an issue known as unobserved confounding. In this paper, we introduce a new study design called confounder detection via treatment intent. The idea is to query a human expert who makes treatment decisions, and ask them to compare pairs of units proposed by a principled matching strategy, with the goal of eliciting unobserved variables that explain why treatment decisions differ. We provide a theoretical basis for such a procedure, ascertaining conditions under which such a study design may elicit unobserved confounders. Building on this newly established foundations, we study treatment effects of interventions in the intensive care unit (ICU). First, we show empirical evidence strongly indicating that electronic health records (EHRs) collected in ICUs are subject to unobserved confounding. By using clinical text notes as a proxy for physicians' knowledge and leveraging natural language processing, we provide a proof of concept for our methodology in a semi-synthetic environment with a known ground truth.

Primary area:
causal reasoning
Keywords:
Causal Inference, Experimental Design, Confounder Detection

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dJKQOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Confounder Detection via Treatment Intent (CDTI), an observational study design that uses explanations of treatment decisions to elicit candidate unobserved confounders. It combines the principled selection of treated–untreated pairs with a separate explanation extraction mechanism. The theoretical analysis draws on stochastic dominance to justify the matching strategies and compare their effectiveness, under explicit assumptions and an idealized extraction model. Synthetic and MIMIC-III-based semi-synthetic experiments evaluate the matching strategies. In the latter, adding selected candidates reduces downstream ETT bias. On real data, however, text ablation yields clinically plausible concepts but no significant change in the ETT estimate.

Critical strengths
  1. The paper tackles incomplete confounder measurement by seeking additional information, rather than relying on a better treatment-effect estimator.
  2. Separating matching from explanation extraction makes the framework easier to assess, and the theoretical analyses help distinguish what the different matching strategies can achieve.
  3. The semi-synthetic experiments also check the downstream effect: whether adding the selected candidates actually reduces ETT bias, not just whether candidates can be detected.
Critical weaknesses
  1. I may be missing a detail, but I am not sure that Theorem 3 holds as stated for discrete confounders. Continuity is assumed only for ZZ, whereas Eqs. (43)–(44) appear to require conditional continuity of UU to exclude ties and establish P(Ui(l)>Uj(l))≥1/2P(U_i^{(l)} > U_j^{(l)}) \geq 1/2. For binary confounders, as used in the semi-synthetic experiments, ties seem to invalidate this step. Could the authors clarify how ties are handled and, if necessary, provide an additional assumption or a revised bound?

  2. I would appreciate some clarification on whether the ρ=0.05\rho=0.05 setting in Figure 3 satisfies the global condition in Proposition 4. Under the stated untruncated Gaussian model, the negative logistic cross-derivative appears to vanish in the tails, while the positive Gaussian dependence term persists, seemingly violating Eq. (11). The numerical ordering may still hold, but this setting does not appear to verify the result under the proposition’s assumptions. Clarifying this distinction would help readers interpret the experiment.

  3. The experiments provide useful initial evidence, but they rely on either perfect extraction or BERT-based text ablation, without directly querying decision-makers. This leaves the central premise of human elicitation largely untested. It would strengthen the paper to assess whether matched comparisons reliably elicit previously unrecorded factors in practice and whether the resulting explanations faithfully reflect physicians’ treatment intent.

  4. I would encourage a more cautious interpretation of the real-data results. Appendix F reports no statistically significant change in the ETT estimate after incorporating the identified concepts, so these findings do not establish bias reduction. Similarly, the 17/20 clinical plausibility assessment is informative, but does not by itself validate the concepts as confounders. The noisy-text-proxy explanation is plausible; presenting it as a hypothesis that requires further validation would make the conclusions more precise.

Questions for authors

See Weaknesses.

2 comments
AuthorsOpenReview ↗

Review Response

We thank the reviewer for their time and effort in reviewing our paper. Based on the comments, we made important changes to the paper, which are highlighted in blue in the revised version. We answer each point below.


(W1: Ties) Thank you for raising this important point. To avoid duplication, we kindly ask you to see our response to Reviewer FoXG (W1) for a full treatment, including a revised bound for discrete variables, as you note.


(W2: Ex. 4's ρ\rho in the tails) This is a good catch, and we thank the reviewer for the careful reading. As Ex. 4 illustrates, the condition holds in the tails only for ρ≤0\rho \leq 0, so ρ=0.05\rho = 0.05 does not satisfy it globally. We have revised the experiment to use ρ=0\rho = 0, where the condition holds everywhere, to make this very precise. The original setting with ρ=0.05\rho = 0.05 gives a mild violation: the condition holds except in the propensity tails, and the same ordering between Z-dominance and Z-matching is observed. Therefore, this illustrates that our conditions are sufficient, but not necessary (e.g., the condition holding in most of the space can still be enough). The paper is updated accordingly.


(W3: Human interaction aspect) We fully agree that the human aspect is very interesting, and we are looking forward to exploring it in future work. Studying how well physicians' explanations reflect their treatment intent is a natural next step. At the same time, we believe it is very important to first establish the idea and its theoretical foundations, and to do so in a venue that can assess these claims. Our goal with this paper is to lay this groundwork, and we hope to pursue the mentioned aspects in the future.


(W4: Cautious interpretation in App. F) Agreed, and have revised the text of App. F accordingly (see change highlighted in blue). We now state that the 17/20 concepts are clinically plausible candidates, which does not by itself establish them as confounders. We also present the noisy-proxy explanation explicitly as a hypothesis that requires further validation, e.g., through prospective measurement of the elicited concepts.


Thanks again for your comments. Please let us know if any open questions remain!

Dear Reviewer,

The discussion period is nearing its end, and we haven’t yet seen a reply from you. We encourage you to respond to the authors’ comments, especially if you have further questions about points that may affect your assessment.

Thank you for taking the time to engage in the discussion.

Best,

Your AC

Reviewer_FoXGOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Confounder Detection via Treatment Intent (CDTI). This proposed study design first selects cases receiving different treatments, then asks decision-makers to explain differences in treatment assignment, with the aim of eliciting candidate unobserved confounders. The authors decompose this process into case pairing and explanation extraction, and analyze three matching methods: covariate matching, propensity score matching, and Z-dominance matching. Under conditions of monotonicity, log-supermodularity, and additional dependency structures, the paper establishes stochastic dominance results for the distribution of latent variables and compares matching strategies under specific extraction models. The experiments include synthetic simulations, a semi-synthetic study based on MIMIC-III, and a real-data analysis using clinical notes. Semi-synthetic results show that adjusting for candidate variables identified through prioritization can reduce bias in ETT estimates. However, while the analysis of real-world data yielded clinically plausible candidate factors, adjusting for these candidates did not significantly change the ETT estimates.

Critical strengths

The research question addressed is of practical significance in casual reasoning applications, and the motivation for the study is well-founded. The decomposition of the matching and extraction processes provides a clear analytical framework for future research.

In terms of formal analysis, the likelihood ratio argument for covariate matching and the argument establishing stochastic dominance for propensity score matching through the balancing property of the propensity score are presented clearly.

The semi-synthetic experiments preserve the joint distribution of clinical variables and provide known ground truth for evaluation. Finally, the authors clearly articulate the paper's limitations and honestly report the lack of significant changes in ETT estimates in the real-data analysis.

Critical weaknesses
  • Applicability of Theorem 3 to discrete unobserved confounders. 
My primary concern relates to Theorem 3. I hope the authors will further clarify its applicability to discrete unobserved confounders.
The continuity assumption explicitly stated in Theorem 3 concerns each marginal distribution of the observed covariates Z, whereas Appendix A uses the condition “two independent univariate continuous random variables” before Equation (43). Subsequently, the authors apply this result to the components of U and obtain the lower bound in Equation (44). However, the theorem does not explicitly state whether the conditional distributions of these components, given the propensity score and treatment status, must also be continuous. My concern is that when a component of U is binary, two individuals may have the same value. Therefore, stochastic dominance alone does not imply that a value drawn from the treated group is strictly greater than an independently drawn value from the untreated group with probability at least one-half. The derivation from Equation (43) to Equation (44) needs to explicitly address ties, and this derivation is subsequently used to obtain Equation (17) in Theorem 3.

This issue is directly related to the experimental setting, as all unobserved confounders in the semi-synthetic experiments are binary. The authors need to clarify whether the theorem requires additional conditional continuity assumptions, or how the derivation should be adjusted to account for discrete variables. Until this point is clarified, the theoretical support provided by Theorem 3 for the binary-confounder setting remains incomplete. This concern pertains to the applicability of the lower bound, rather than the stochastic dominance conclusion itself.



  • Evidence for the practical benefit of the matching strategies.

    An additional concern is that the evidence for reduced ETT estimation bias comes mainly from the semi-synthetic experiments. The success rate in these experiments is calculated directly under the assumption of perfect extraction, and candidate rankings rely on comparisons among known unobserved variables. These experiments therefore do not directly test how accurately real decision-makers or text-based extractors can recover the relevant explanations.

Questions for authors
  1. How does Theorem 3 handle ties resulting from binary unobserved confounders? Consider Z to be a standard normal variable, with U independent of Z and taking on 0 and 1 with equal probability. When U = 0 and U = 1, the treatment probabilities are 0.4 and 0.6, respectively. In this case, the observed propensity score is always 0.5, and the utility difference between propensity score matching and random pairing is 0; however, according to the paper’s AUC definition based on strict inequalities, AUC(Z) = 0.5 and AUC(U) = 0.36, resulting in a lower bound of 0.14 for Equation (17). Can the authors clarify whether this setup is ruled out by any specific assumption?
  2. Under the same budget and sampling process, how much gain does the matching strategy provide compared to random pairing? Appendix F uses 2,000 case pairs and limits each patient to a maximum of 3 appearances. Could the authors provide control results for random pairing under the same BERT model, conceptual ablation rules, 2,000-case-pair budget, and patient reuse restrictions?
4 comments
AuthorsOpenReview ↗

Review Response - Part I

We thank the reviewer for their time and effort in reviewing our paper. Based on the comments, we made important changes to the paper, which are highlighted in blue in the revised version. We answer each point below.

(W1: Binary confounders UU) Thank you for raising this important point. Indeed, most of our theoretical developments implicitly assume continuous random variables. For instance, our definition of the AUC, P(Vi>Vj)P(V_i > V_j), coincides with the standard AUC only in the continuous case. For binary UU, things need to be handled slightly differently, as the reviewer correctly points out.

First, for discrete variables, the AUC is given by P(Vi>Vj)+12P(Vi=Vj)P(V_i > V_j) + \tfrac12 P(V_i = V_j), i.e., ties are broken at random. To handle ties explicitly, for a variable VV and a pair (i,j)(i, j) we introduce the following notation p+=P(Vi>Vj),p−=P(Vi<Vj),p==P(Vi=Vj),p++p−+p==1.p^+ = P(V_i > V_j), \quad p^- = P(V_i < V_j), \quad p^= = P(V_i = V_j), \qquad p^+ + p^- + p^= = 1. In our framework, a tie is not a success: if ii and jj agree on VV, then VV cannot explain the different treatment decisions. Therefore, NN and DD count only p+p^+, which we can write as p+=12(1−p=+Δ)p^+ = \tfrac12 (1 - p^= + \Delta) with Δ=p+−p−\Delta = p^+ - p^-. Under π\pi-matching, the two parts of the argument behave as follows. For each Z(k)Z^{(k)}, the matched Zi,ZjZ_i, Z_j are i.i.d. given π(Z)\pi(Z), so Δ=0\Delta = 0 by exchangeability, without any continuity assumption. For each U(l)U^{(l)}, the stochastic dominance of Prop. 3 gives Δ≥0\Delta \geq 0. Ties enter only through p=p^=, which may differ between the two strategies; we denote this difference by δ==pπ=−pmarg=\delta^= = p^=_{\pi} - p^=_{\text{marg}}.

With this notation, Thm. 3 for discrete ZZ and UU becomes Uπ−Umarg  ≥  12[α∑kΔZ(k)marg−∑lΔU(l)marg]+12[α∑kδZ(k)=−∑lδU(l)=].\mathcal{U}^{\pi} - \mathcal{U}^{\text{marg}} \;\geq\; \frac12 \Big[\alpha \sum_k \Delta^{\text{marg}}_{Z^{(k)}} - \sum_l \Delta^{\text{marg}}_{U^{(l)}}\Big] + \frac12 \Big[\alpha \sum_k \delta^=_{Z^{(k)}} - \sum_l \delta^=_{U^{(l)}}\Big]. Note that in the continuous case, we have Δ=2AUC−1\Delta = 2\mathrm{AUC} - 1, so the first term recovers Thm. 3. The second term is new and accounts for ties. It also gives a nice interpretation: more ties on ZZ under π\pi-matching help (fewer competing observed explanations), while more ties on UU hurt.

Thus, the result requires only the assumptions of Prop. 3 (monotonicity and log-supermodularity), and no continuity. In the revision, we add this more general theorem and its proof to the appendix. In the main text, we now state the (conditional) continuity of UU as an explicit assumption in Thm. 3. Interestingly, after considering your question carefully, we realized that the continuity assumption on ZZ is not needed.

(W2: Semi-synthetic evidence only) We thank the reviewer for this comment. We would first like to note that the paper does include a real-data application: in App. F, we apply the full pipeline to MIMIC-III, using π\pi-matching with 2,000 pairs and a text-based extractor that ablates concepts from clinical notes. 17/20 of the elicited concepts are clinically highly plausible drivers of mechanical ventilation. The semi-synthetic experiments are used to provide what real data cannot, that is a known ground truth, against which success rates and ETT bias can be measured.

Our main goal in this paper is to introduce the idea of eliciting confounders through matched comparisons, and to understand when it can work. We see a study with practicing clinicians as the natural next step, which we are excited about. Such a study involves ethics approval, integration into clinical workflows, and physicians' time, and we felt it was important to first establish the design on solid ground. We now mention this direction more explicitly in the Limitations section.

AuthorsOpenReview ↗

Review Response - Part II

(Q1: Ties) We thank the reviewer for this very helpful example; the reviewer's computation is correct. Since π(Z)=0.5\pi(Z) = 0.5 everywhere, π\pi-matching and marginal matching produce the same pairs, so the true utility difference is 00. With P(U=1∣X=1)=0.6P(U = 1 \mid X = 1) = 0.6 and P(U=1∣X=0)=0.4P(U = 1 \mid X = 0) = 0.4, the notation we introduced in response (W1) gives, for both strategies, p+(U)=0.36,p−(U)=0.16,p=(U)=0.48,Δ(U)=0.2,p^+(U) = 0.36, \quad p^-(U) = 0.16, \quad p^=(U) = 0.48, \quad \Delta(U) = 0.2, while Δ(Z)=0\Delta(Z) = 0 and the tie terms coincide (δ==0\delta^= = 0). The bound of the discrete theorem then reads −12Δ(U)=−0.1≤0-\tfrac12 \Delta(U) = -0.1 \leq 0, and therefore holds. Thus, we believe the revised theorem fixes the raised problem. Thanks again for constructive feedback!

(Q2: Random pairing baseline in App. F) We thank the reviewer for suggesting this control. We have now performed the experiment, comparing π\pi-matching with random treated-control pairing on the same held-out set, using the same fitted BERT model, concept-ablation rules, budget of 2,000 pairs, and limit of three appearances per ICU stay. Random pairing uses the same seed as the data split and propensity-model fitting.

Random pairing produces at least one retained candidate for 49.60% of pairs (992 out of 2,000), compared with 57.15% (1,143 out of 2,000) under π\pi-matching. Thus, π\pi-matching improves the nonempty-extraction rate by 7.55 percentage points, yielding 151 additional nonempty comparisons at the same budget. Furthermore, the top-20 concepts elicited by random pairing fail to include 2 out of 8 Group (A) confounder candidates, and instead include clinically less relevant features. Overall, in the experiment, π\pi-matching seems to perform better than random pairing.

In the revised pdf, we have added this comparison to App. F, together with a table of the 20 most frequently discovered candidate confounders under random pairing.


Thanks again for the comments; please let us know if any questions remain!

Dear Reviewer,

The discussion period is nearing its end, and we haven’t yet seen a reply from you. We encourage you to respond to the authors’ comments, especially if you have further questions about points that may affect your assessment.

Thank you for taking the time to engage in the discussion.

Best,

Your AC

Thanks for the informative response. Most of my concerns are resolved. I will raise my score to support the acceptance of this paper.

Reviewer_F7NpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CDTI (Confounder Detection via Treatment Intent), a new observational study design: rather than asking a domain expert to enumerate unobserved confounders in the abstract, one shows the decision-maker (DM) matched treated/untreated pairs and asks why the treatment decision differed. The authors formalize a matching strategy M (Z-match, π-match, Z-dominance, marginal) and an extraction strategy ε , and prove stochastic-dominance results: under monotonicity of π(z,u) in u and log-supermodularity of P(u∣z) , conditioning on X=1 shifts U upward (Prop. 1–2); the result transfers to π -matching via the balancing property (Prop. 3); and Z-dominance holds under a two-channel (collider vs. bidirected) cross-derivative condition (Prop. 4). Strategy comparisons follow: Z-dom ≥ Z-match in expected accuracy (Thm. 2), and π-match ≥ marginal in a linearized surrogate E[N−αD] (Thm. 3). Empirically, the paper argues that EHR-based ETT estimates for mechanical ventilation and vasopressors are implausibly harmful (hence confounded), verifies the theorems on synthetic Gaussians, and runs a semi-synthetic MIMIC-III study plus a real-data pipeline using BERT-ablation extraction.

Critical strengths

S1. The core reframing is genuinely good. "Any U that confounds X→Y must have been available to the DM at decision time — so query the decision, not the abstraction" is a clean and clinically sensible idea. It converts an open-ended elicitation problem into a constrained pairwise comparison, which is the right move.

S2. The theory is correct as far as I checked. I verified Props. 1–4 and Thms. 2–3 end to end. The Bayes-ratio step in Prop. 1, the Karlin–Rinott verification in Prop. 2 (the cross-term (1−π(u))π(u ′ )≤(1−π(u∧u ′ ))π(u ′ ∨u ′ ) does follow from coordinatewise monotonicity), the common-mixing preservation in Prop. 3, and the two-chain decomposition in Prop. 4 all hold. Example 2's reduction of log-supermodularity to (Σ −1 ) kl ​ ≤0 is correct.

S3. Unusual intellectual honesty. Example 3 constructs a counterexample to log-supermodularity; Examples 5–6 construct two distinct failure modes for Prop. 4; Panel 2 of Fig. 3 deliberately shows Z-dominance losing; and the Limitations section states outright that the method "does not certify whether any specific detected variable is a confounder." This is rare and I credit it.

Critical weaknesses

W1 (central). The theory identifies causes of treatment, not common causes of treatment and outcome. Fig. 2 posits both U→X and U→Y , but every result in Sec. 2.1–2.3 uses only π(z,u)=P(X=1∣z,u) . If you set the U→Y coefficient to zero, every theorem in the paper remains true verbatim — a pure instrument satisfies Prop. 1–4, Thm. 1–3, and would be extracted and reported as a "confounder." The evaluation inherits the same blind spot: the ground-truth importance score in App. E is I l ​

p ^ ​

l ​ (1− p ^ ​

l ​ )∣γ X (l) ​ ∣ , which uses only the treatment coefficient γ X ​ and never γ Y ​ . So the paper never once scores whether a recovered variable is associated with the outcome.

This is not a minor caveat, and App. F is the evidence. There, 2,000 π-matched pairs with BERT-ablation extraction yield 20 concepts, 17/20 judged "clinically highly plausible" — yet adjusting for them does not move the ETT toward zero, and no adjustment set differs significantly from Z -only. That is exactly the signature you would expect if the extracted variables are treatment drivers (or proxies for them) rather than confounders. The paper attributes this to noise in the binary note-derived proxies, which may be true, but the more parsimonious explanation is the one the theory permits: the design has no Y-side signal at all. A one-line Y-side screen (conditional association with Y given X,Z , or a placebo-outcome check) would have distinguished these, and its absence is the paper's biggest gap.

Questions for authors

Q1. Given W1: is there any step in the pipeline that uses the U→Y arrow? If not, how do you distinguish a detected variable from an instrument, and would you consider adding a Y-side screen (e.g. association with Y given X,Z , or a placebo-outcome test) to the success criterion? Would App. F's null result survive such a screen?

Q2. For Thm. 3: what is E[N∣π] when γ>0 ? The bound is d U ​ /2 and is exact only at γ=0 . If the true value is materially above d U ​ /2 , where does the κ=1 threshold move? And since κ contains AUC(U l ​ ) , how is a practitioner supposed to evaluate the criterion?

Q3. Example 4 shows ρ≤0 is required in the propensity tails. Large δ lives in the tails. Is there an admissible (δ,ρ,βγ) region where Z-dominance is both justified and materially better than Z-matching? Can you give a rule for choosing δ ?

3 comments
AuthorsOpenReview ↗

Review Response

We thank the reviewer for their time and effort in reviewing our paper. Based on the comments, we made important changes to the paper, which are highlighted in blue in the revised version. We answer each point below.

(W1, Q1: U→YU \to Y link) We thank the reviewer for this excellent point, with which we fully agree. The design elicits drivers of treatment, and U→YU \to Y is not guaranteed. We considered this in detail during manuscript development, but ended up not discussing it explicitly in the submission, which we now rectify.

Revision. We add a new appendix (App. G) stating that elicited candidates should be validated by testing U(l)⊥!!!⊥Y∣X,ZU^{(l)} \perp\\!\\!\\!\perp Y \mid X, Z (which holds if U(l)U^{(l)} is an instrument), prospectively and with multiplicity correction. We also mention possibly suitable tests (partial correlation / logistic LRT, GCM, KCI).

Update to App. F. Following the reviewer's suggestion, we ran this screen on the 20 elicited concepts. Partial correlation with mortality YY, given (X,Z)(X, Z), is reported below; we also report the total variation (TV), defined as P(y∣U(l)=1)−P(y∣U(l)=0)P(y \mid U^{(l)}=1) - P(y \mid U^{(l)}=0).

GroupConceptP(U(l)=1)P(U^{(l)}=1) (%)TV (%)Partial cor.pp-value
APleural effusion16.3+8.8+0.055<0.01
APneumonia10.9+8.4+0.044<0.01
ADyspnea10.5+5.9+0.027<0.01
APulm. edema7.6+8.1+0.031<0.01
AAtelectasis21.2+4.6+0.017<0.01
AHypoxia5.0+10.5+0.042<0.01
APneumothorax8.8+4.0+0.0100.08
AInfiltration9.2+6.2+0.022<0.01
BFever8.4+1.8−0.031<0.01
BCommun. disease8.1+4.7+0.0040.50
BSinus tachycardia8.2+4.6+0.0110.05
CHemorrhage16.4+3.5+0.0090.13
CIntracranial hem.2.7+9.0+0.035<0.01
CFracture11.9+0.9+0.0030.57
DHypertensive dis.9.1+0.6−0.033<0.01
DCoronary athero.7.3−1.3−0.0090.10
DPericardial eff.8.0+0.8−0.0060.31
EEdema10.6+5.2+0.016<0.01
EPain11.3+0.8−0.021<0.01
ENausea5.0−0.3−0.022<0.01

In summary, these are our findings:

GroupPositive TVPositive partial cor.Positive & p<0.05p<0.05
A8/88/87/8
B3/32/30/3
C3/33/31/3
D2/30/30/3
E2/31/31/3
All18/2014/209/20

Group (A), the most clinically plausible concepts, passes almost uniformly (8/8 positive, 7/8 significant). Therefore, we do not believe the main elicited variables (Group A) are instruments; rather, we believe it is the retrospective nature of the data and the partial reporting (which reduces prevalence) which results in our findings.

Thanks again for raising this important point.


(Q2: E[N∣π]E[N | \pi]) We thank the reviewer for this question. We write Uπ\mathcal{U}^{\pi} and Umarg\mathcal{U}^{\text{marg}} for the utilities introduced in the paper. Let SZ=∑k(AUC(Z(k))−12)S_Z = \sum_k (\mathrm{AUC}(Z^{(k)}) - \tfrac12) and SU=∑l(AUC(U(l))−12)S_U = \sum_l (\mathrm{AUC}(U^{(l)}) - \tfrac12). Then, we define the slack c:=E[N∣Mπ-match]−dU2=∑l(P(Ui(l)>Uj(l)∣Mπ-match)−12).c := \mathbb{E}[N \mid \mathcal{M}^{\pi\text{-match}}] - \tfrac{d_U}{2} = \sum_{l} \Big(P(U_i^{(l)} > U_j^{(l)} \mid \mathcal{M}^{\pi\text{-match}}) - \tfrac12\Big). By Prop. 3, P(U(l)∣π=p,X=1)⪰stP(U(l)∣π=p,X=0)P(U^{(l)} \mid \pi = p, X = 1) \succeq_{st} P(U^{(l)} \mid \pi = p, X = 0), so each summand is non-negative and c≥0c \geq 0. This gives the identity Uπ−Umarg=αSZ−SU+c,\mathcal{U}^{\pi} - \mathcal{U}^{\text{marg}} = \alpha S_Z - S_U + c, so π\pi-matching dominates iff κ=SUSZ≤α+cSZ.\kappa = \frac{S_U}{S_Z} \leq \alpha + \frac{c}{S_Z}. The threshold therefore moves up by exactly c/SZc / S_Z. SZS_Z is computable from data, while SUS_U is not (unless we also measure UU prospectively). Therefore, the criterion should be interpreted as a sensitivity statement: π\pi-matching is preferred unless the unobserved confounders explain more treatment variation than the observed ones.


(Q3: δ\delta rule) We thank the reviewer for this insightful question, and we clarified this point in the revision.

Condition (iii) of Prop. 4 is a condition on the entire (z,u)(z, u)-space, and does not depend on δ\delta. Hence, whenever it holds, Z-dominance is justified for every δ\delta. In Ex. 4, this is the region ρ≤0\rho \leq 0, for any βγ>0\beta\gamma > 0.

However, the reviewer is right that for ρ>0\rho > 0 the condition fails in the tails, and in that case no δ\delta is formally covered. Still, within the admissible region, the chain argument of Prop. 4 applies to any z′′<z′<zz'' < z' < z, so the success probability is non-decreasing in δ\delta. This also gives rise to a simple rule: choose δ\delta as large as the data allows, trading off the dominance gap against the number of available dominated pairs.

We thank the reviewer again for their comments. As the discussion period is approaching its end (Nov 18 AoE), we would be glad to hear back from the reviewer, to understand if our responses addressed their concerns. If any concerns remain unresolved, we would be happy to respond before the deadline.

Dear Reviewer,

The discussion period is nearing its end, and we haven’t yet seen a reply from you. We encourage you to respond to the authors’ comments, especially if you have further questions about points that may affect your assessment.

Thank you for taking the time to engage in the discussion.

Best,

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.