- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies the problem that the temporal provenance of covariates may not match their causal eligibility. Either keeping all covariates or deleting all post-index covariates would impact the performance of the treatment effect estimation. So, the paper proposes a preprocessing framework(TEAR) to address the problem through a soft routing gate that attributes the contribution of all covariates and lowers the adverse impact of post-treatment covariates without losing important information. The main contributions are formulating covariate preprocessing as continuous feature-level routing and combining metadata-guided gates with finite-dimensional aggregation. The paper finally validates TEAR on synthetic controlled data and semi-synthesized MIMIC data using multiple CATE estimators such as DML, R-Learner, TARNet, and DragonNet. The results show that TEAR performs best when both eligible delayed information and treatment descendants exist in data.
Critical strengths
The paper has a clear and easy-to-follow story. The problem is well-defined, and the authors conduct relatively comprehensive experiments to validate TEAR under different settings, including different eligibility regimes, multiple CATE estimators, ablation studies, observed-data selection, and provenance corruption. These experiments help us understand under what scenarios TEAR could work better and when its advantage would become weaker.
Critical weaknesses
-
The evidence of comparing the TEAR with naive inclusion or deletion of covariates is too weak. It would be better to compare it with careful causal work.
-
The gate does not shrink the weight of post-treatment covariates completely to zero, meaning that some post-treatment information leakage is still present. This could convince us that bias would be minimized or controlled from post-treatment covariates.
Questions for authors
-
The mismatch between recorded time and causal eligibility of covariates is plausible. But is it a common scenario in the real world that the clinically meaningful measurement time is missing or the recorded time is not rigorously monitored, so that this problem becomes important enough to require a learned routing framework?
-
Only comparing TEAR mainly with these naive methods does not fully demonstrate the contribution of TEAR. I would like to see it compared with careful causal work. Here is the paper I found that might be used as a stronger baseline: Doutreligne et al. (2025), “Step-by-step causal analysis of EHRs to ground decision-making.” Another control could be to adjust the cutoff timestamps for inclusion or deletion instead of just including or deleting all post-index covariates.
-
TEAR might be able to achieve a lower PEHE, but the gate does not shrink the weight of post-treatment covariates completely to zero, meaning that some post-treatment information leakage is still there. I would like the authors to clarify whether the remaining leakage is reduced to an acceptable level.
-
Please discuss the following related work and consider comparisons where applicable:
- Nagalapatti et al., Leveraging a Simulator for Learning Causal Representations from Post-Treatment Covariates for CATE (TMLR 2025).
- Lin and Crespo, Variance reduction combining pre-experiment and in-experiment data (CLeaR 2026).
-
Could you clarify which function computes the reported root-PEHE, defined as ? The supplied
compute_caie_rmseaverages over both the treatment-effect error and a zero reference column, yielding root-PEHE divided by for binary treatment—for example, 0.7071 instead of 1 for unit error. Was this function used for the reported tables? If not, please identify the implementation used.
4 comments
reply to updates
Thanks for all the updates to improve our understanding of the paper. I have read both the reply to me and the replies to the other two reviewers to capture all details. I appreciate the additional baseline comparisons, including the validation-tuned temporal cutoff. But I still have some confusion that needs clarification:
-
Metadata vs. routing: It is still unclear whether TEAR's improvement comes from learned routing or its use of richer metadata. Could the authors clarify what the existing ablations establish?
-
Residual leakage: Lower PEHE and CATE bias in controlled settings do not necessarily guarantee valid causal estimation. Could the authors clarify the limitations of retaining post-treatment information?
-
PEHE calculation: The authors acknowledge the normalization bug but have not confirmed whether it affected the reported results. Could they clarify which results have been verified?
These clarifications would help me better assess the contribution and reliability of the paper.
And I know the rebuttal period is closing. So, I would not expect running more experiments. Maybe for some questions, I just misunderstood or missed something in the manuscript. So, it is fine to just use your existing work or understanding to remove my concerns.
Follow-up Response to Reviewer
Thank you for your thoughtful follow-up, for reading our replies across all three reviews, and for acknowledging the tuned-cutoff baseline. We also appreciate your patience as we revisited the evidence and its limitations. As you suggested, we address your three points using existing results.
1. Metadata versus Learned Routing
Both contribute, and the existing ablations separate their roles. In the controlled mixed regime (DML, 50 matched seeds), App. H.1 (Tables 17–18) varies only gate training, with the same covariates and timestamp/schema inputs:
| Gate configuration | PEHE ↓ | Post-index AUROC ↑ | Delayed / descendant h |
|---|---|---|---|
| Full TEAR | 0.121 | 0.90 | 0.82 / 0.11 |
| No prior loss or prewarming | 0.139 | 0.76 | 0.61 / 0.29 |
| Frozen prior-only gate | 0.150 | 0.51 | 0.06 / 0.05 |
With the same metadata, the frozen gate closes both post-index feature types, whereas joint training separates delayed-neutral records from descendants and lowers PEHE by 19.3%. Even without the gate-prior loss or prewarming, joint training achieves a post-index AUROC of 0.76. The post-index AUROC of 0.90 (Table 18) is computed among post-index features; the overall gate AUROC is 0.95 (Table 1B).
App. H.4 (Table 21) holds the architecture fixed and varies metadata availability: PEHE is 0.121 with full metadata, 0.136 with schema only, and 0.148 with time only (Deletion: 0.156). Metadata richness therefore sets how much the gate can separate, and joint learning converts that metadata into routing. Because Eq. (4) also feeds metadata into the embeddings, we report these as complementary ablations rather than an additive decomposition. Table 1C further separates coordinate-wise gating, random projection, and eligibility-guided aggregation.
2. Scope of Retaining Post-Treatment Information
We agree that lower PEHE and valid causal identification are distinct claims. In the controlled mixed regime, TEAR assigns descendants a mean gate of 0.11 (normalized ineligible exposure 0.09) and reaches PEHE / absolute CATE bias of 0.121 / 0.011, compared with 0.156 / 0.014 for Deletion, 0.193 / 0.083 for Raw, and 0.103 / 0.006 for the oracle (Table 10). This shows reduced error under the evaluated generator, and the oracle provides a reference for the remaining estimation gap.
The scope of this result follows from our theory. Proposition 1 bounds representation and fixed-predictor perturbations by the sum of exposed ineligible mass and omitted eligible mass, under bounded embeddings and a fixed Lipschitz host; a nonzero gate on a genuine descendant can still admit bias in a refitted estimator. Lemma B.1 identifies CATE when its eligible-representation, balancing, and heterogeneity-sufficiency assumptions hold, and for a learned gate these remain assumptions. Known descendants belong outside the adjustment set by causal design; TEAR is designed for features whose eligibility recorded time leaves uncertain. Its boundary is also measurable: at 100% provenance corruption, fixed TEAR reaches PEHE 0.159 versus 0.156 for Deletion, and observed-data selection then chooses Deletion in 87% of seeds (Fig. 1; Table 9), so the selector falls back to the conservative rule when provenance carries no signal.
3. Root-PEHE Implementation and Evaluation Scope
Thank you for following up on the metric implementation. We traced the public code and confirmed the source of the normalization issue. The compute_caie_rmse helper includes the zero-reference column when computing binary-treatment RMSE, introducing a factor of 1/√2. Its only call site in the released package is an optional diagnostic within OrbitTrainer.fit(), evaluated on the adapter's training inputs.
For any binary-treatment comparison evaluated uniformly with this helper, correcting the normalization multiplies absolute PEHE values, paired differences, and confidence-interval endpoints by √2. The common factor leaves method rankings, relative improvements, validation-PEHE selection, and scale-invariant paired tests unchanged. Within each comparison, candidates share the downstream host and evaluation protocol, differing in their preprocessing. If their binary-treatment PEHE values were computed through a common scoring path, the normalization correction preserves the tuned-cutoff and routing comparisons, including their relative gains and paired statistical conclusions. The actual table-generation path remains to be verified. Gate diagnostics, routing weights, exposure measurements, and observed-data selection scores do not depend on this PEHE normalization.
The released repository does not establish the scoring path for each manuscript table, so we distinguish the confirmed helper defect from table-level verification. We will correct the released helper and document any table-level numerical corrections supported by the verified evaluation paths.
Thank you again for the opportunity to clarify these points.
Thank you again for your thoughtful engagement throughout the discussion and for taking the time to consider our additional evidence!
We have now provided detailed clarifications addressing your three follow-up questions, and we sincerely hope they help resolve your remaining concerns.
As the discussion period draws to a close, we would be grateful for any final feedback you may have. If our responses have strengthened your assessment of the paper, we would also greatly appreciate your consideration of updating your overall rating accordingly.
Thank you once again for your time and constructive feedback.
Thank you for recognizing the clarity of the causal-eligibility problem and the breadth of the controlled evaluations. We took your tunable-cutoff suggestion as a direct test of whether a stronger temporal rule can replace feature-specific routing; we also clarify residual descendant exposure, clinical provenance, and the metric implementation.
W1 & Q2: Stronger Controls—Including Your Proposed Cutoff
We evaluated a validation-tuned global timestamp cutoff over seven predeclared candidates, selected on validation data and assessed once on matched test splits (50 paired seeds; new App. H.2, Table 19). The same fixed TEAR configuration improves mean test PEHE in all three settings:
Thus tuning one global temporal boundary does not recover the gain from feature-specific routing in these settings, despite allowing the cutoff seven validation candidates versus a fixed TEAR operating point.
Existing controls also include L0 selection (Table 3), Gate×Raw and random projection (Table 1C), and a 50-seed causal-representation comparison (Table 15). On Synth-High/DML, TEAR-B0 / TEDVAE / D2VD-nonlinear have mean PEHE 9.23 / 10.20 / 17.71, with P95 23.77 / 26.65 / 55.59. Together, these test temporal rules, learned sparsity, compression, and causal representations.
W2 & Q3: Residual Descendant Exposure and CATE Error
A post-index timestamp does not by itself imply treatment-induced content. The controlled mixed regime separates delayed-neutral records from genuine descendants. TEAR gives them mean gate weights 0.82 vs. 0.11, with normalized ineligible exposure 0.09 (Table 1B; App. D.1). Crucially, we also measure effect-estimation error rather than relying on gate weights alone (Table 10):
The controlled evidence shows lower ineligible exposure than Raw, and lower CATE estimation error than both Raw and Deletion. Nonzero descendant mass remains, and the oracle benchmark makes the remaining gap explicit; the claim is reduction in this setting, not guaranteed elimination of causal bias.
Q1: Why the EHR Timing Mismatch Is Operationally Real
MIMIC-IV
charteventscontains distinctcharttime(observation time) andstoretime(entry/validation time), illustrating that when a value was observed and when it entered a structured export need not coincide (revised App. E.2). TEAR targets this distinction after the estimand and admissibility question have been specified. Our EHR-shaped tests use semi-synthetic outcomes; they test the mechanism, not its prevalence across hospitals.Q4: Relation to the Suggested Causal Literature
We added all three references (revised Sec. 2). Doutreligne et al. address target-trial design and expert identification, which are necessary upstream decisions rather than a plug-in feature router. Nagalapatti et al. use simulator-generated supervision for post-treatment CATE representations; Lin & Crespo study pre-/in-experiment data for randomized-experiment ATE variance reduction. Their supervision and estimands differ from this static provenance-guided CATE setting. The directly comparable controls here are learned feature selection, causal representations, and the proposed tuned cutoff.
Q5: Root-PEHE Function and Reported Values
Thank you for the precise numerical check. We reproduced the public
compute_caie_rmsehelper's binary-treatment normalization error: including the zero-reference contrast returns 0.7071 rather than 1.0000 for unit effect error. Correct root-PEHE uses only the nonreference treatment contrast (Sec. 5.1; App. D.1). The available repository does not identify the implementation used to generate the manuscript tables, so their absolute scale is not verified against this helper. Uniform use of this helper would rescale PEHE and paired intervals by 2 without changing relative improvements; which tables require correction depends on their actual calculation path.The new cutoff test, clinical timestamp example, and comparisons with the cited literature are incorporated in the revised PDF (Sec. 2; Sec. 5.3; Apps. E.2, H.2).