- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Confounder Detection via Treatment Intent (CDTI), an observational study design that uses explanations of treatment decisions to elicit candidate unobserved confounders. It combines the principled selection of treated–untreated pairs with a separate explanation extraction mechanism. The theoretical analysis draws on stochastic dominance to justify the matching strategies and compare their effectiveness, under explicit assumptions and an idealized extraction model. Synthetic and MIMIC-III-based semi-synthetic experiments evaluate the matching strategies. In the latter, adding selected candidates reduces downstream ETT bias. On real data, however, text ablation yields clinically plausible concepts but no significant change in the ETT estimate.
Critical strengths
- The paper tackles incomplete confounder measurement by seeking additional information, rather than relying on a better treatment-effect estimator.
- Separating matching from explanation extraction makes the framework easier to assess, and the theoretical analyses help distinguish what the different matching strategies can achieve.
- The semi-synthetic experiments also check the downstream effect: whether adding the selected candidates actually reduces ETT bias, not just whether candidates can be detected.
Critical weaknesses
-
I may be missing a detail, but I am not sure that Theorem 3 holds as stated for discrete confounders. Continuity is assumed only for , whereas Eqs. (43)–(44) appear to require conditional continuity of to exclude ties and establish . For binary confounders, as used in the semi-synthetic experiments, ties seem to invalidate this step. Could the authors clarify how ties are handled and, if necessary, provide an additional assumption or a revised bound?
-
I would appreciate some clarification on whether the setting in Figure 3 satisfies the global condition in Proposition 4. Under the stated untruncated Gaussian model, the negative logistic cross-derivative appears to vanish in the tails, while the positive Gaussian dependence term persists, seemingly violating Eq. (11). The numerical ordering may still hold, but this setting does not appear to verify the result under the proposition’s assumptions. Clarifying this distinction would help readers interpret the experiment.
-
The experiments provide useful initial evidence, but they rely on either perfect extraction or BERT-based text ablation, without directly querying decision-makers. This leaves the central premise of human elicitation largely untested. It would strengthen the paper to assess whether matched comparisons reliably elicit previously unrecorded factors in practice and whether the resulting explanations faithfully reflect physicians’ treatment intent.
-
I would encourage a more cautious interpretation of the real-data results. Appendix F reports no statistically significant change in the ETT estimate after incorporating the identified concepts, so these findings do not establish bias reduction. Similarly, the 17/20 clinical plausibility assessment is informative, but does not by itself validate the concepts as confounders. The noisy-text-proxy explanation is plausible; presenting it as a hypothesis that requires further validation would make the conclusions more precise.
Questions for authors
See Weaknesses.
2 comments
Dear Reviewer,
The discussion period is nearing its end, and we haven’t yet seen a reply from you. We encourage you to respond to the authors’ comments, especially if you have further questions about points that may affect your assessment.
Thank you for taking the time to engage in the discussion.
Best,
Your AC
Review Response
We thank the reviewer for their time and effort in reviewing our paper. Based on the comments, we made important changes to the paper, which are highlighted in blue in the revised version. We answer each point below.
(W1: Ties) Thank you for raising this important point. To avoid duplication, we kindly ask you to see our response to Reviewer FoXG (W1) for a full treatment, including a revised bound for discrete variables, as you note.
(W2: Ex. 4's ρ in the tails) This is a good catch, and we thank the reviewer for the careful reading. As Ex. 4 illustrates, the condition holds in the tails only for ρ≤0, so ρ=0.05 does not satisfy it globally. We have revised the experiment to use ρ=0, where the condition holds everywhere, to make this very precise. The original setting with ρ=0.05 gives a mild violation: the condition holds except in the propensity tails, and the same ordering between Z-dominance and Z-matching is observed. Therefore, this illustrates that our conditions are sufficient, but not necessary (e.g., the condition holding in most of the space can still be enough). The paper is updated accordingly.
(W3: Human interaction aspect) We fully agree that the human aspect is very interesting, and we are looking forward to exploring it in future work. Studying how well physicians' explanations reflect their treatment intent is a natural next step. At the same time, we believe it is very important to first establish the idea and its theoretical foundations, and to do so in a venue that can assess these claims. Our goal with this paper is to lay this groundwork, and we hope to pursue the mentioned aspects in the future.
(W4: Cautious interpretation in App. F) Agreed, and have revised the text of App. F accordingly (see change highlighted in blue). We now state that the 17/20 concepts are clinically plausible candidates, which does not by itself establish them as confounders. We also present the noisy-proxy explanation explicitly as a hypothesis that requires further validation, e.g., through prospective measurement of the elicited concepts.
Thanks again for your comments. Please let us know if any open questions remain!