acceptodds
Under review as a conference paper at ICLR 2027

Causal Autoencoding and Treatment Conditioning for Counterfactual Estimation over Time

Abstract

Counterfactual estimation over time is important in various applications, such as personalized medicine. However, time-dependent confounding bias in observational data still poses a significant challenge in achieving accurate and efficient estimation. We introduce causal autoencoding and treatment conditioning (CAETC), a novel method for this problem. Built on adversarial representation learning, our method leverages an autoencoding architecture to learn a partially invertible and treatment-invariant representation, where the outcome prediction task is cast as applying a treatment-specific conditioning on the representation. Our design is independent of the underlying sequence model and can be applied to existing architectures such as long short-term memories (LSTMs) or temporal convolution networks (TCNs). We conduct extensive experiments on synthetic, semi-synthetic, and real-world data to demonstrate that CAETC yields significant improvement in counterfactual estimation over existing methods.

Primary area:
causal reasoning
Keywords:
causal inference, causal machine learning, counterfactual estimation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1Qt4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Causal Autoencoding and Treatment Conditioning (CAETC), a framework for longitudinal counterfactual estimation under time-dependent confounding. The proposed approach combines partial autoencoding to encourage local invertibility and preserve historical covariate information, adversarial learning to obtain treatment-balanced representations, and treatment-specific conditioning to predict future outcomes. The authors also provide other theoretical results and deriving a bound on counterfactual estimation error. Experiments on synthetic, semi-synthetic, and real-world datasets show promising preliminary results regarding the model's robustness to changes in treatment assignment policies, particularly when trained on confounded data and evaluated on non-confounded data.

Critical strengths

The paper addresses an interesting and relevant problem in longitudinal counterfactual estimation. The authors have clearly invested substantial effort in developing the proposed framework, providing theoretical analyses, and conducting an extensive experimental evaluation across synthetic, semi-synthetic, and real-world datasets. The investigation of robustness to changes in treatment assignment policies is also an interesting aspect of the experimental study.

Critical weaknesses

Despite the relevance of the problem and the effort invested in this work, I have two main concerns regarding the novelty and experimental contributions of the proposed approach, as detailed below.

The conceptual novelty of CAETC appears somewhat limited, particularly compared to Causal CPC (CCPC, Bouchattaoui et al., NeurIPS 2024). Both methods address the same fundamental challenges of preserving historical information while learning treatment-balanced representations, although through different mechanisms.

The experimental claims should also be more nuanced. In particular, the statement that "CAETC-LSTM/TCN demonstrates strong improvement across all time steps" (raw 446 and 462) is not fully supported by Tables 1 and 2. The improvements over CCPC are generally modest, and CCPC even performs better at certain prediction horizons. Moreover, the relative performance of CAETC-LSTM and CAETC-TCN varies across datasets and horizons, which deserves further investigation. Indeed, For Table 2 CAETC-TCN is always better Than CAETC-LSTM but in table 1 CAETC-TCN is better until tau=3 then CAETC-LSTM becomes better. A more thorough analysis is needed to isolate the contributions of the proposed components, particularly to establish whether partial autoencoding offers advantages over the InfoMax objective used in CCPC. Comparing both mechanisms under the same architecture and training protocol would be particularly informative.

Finally, regarding the novelty of the theoretical contributions, there is substantial overlap between Theorem 3.1 of CAETC and Theorem 5.4 of CCPC, as the authors themselves acknowledge in raw 270. The authors should more clearly articulate what is theoretically new compared to this prior result. Moreover, the choice of encouraging only local invertibility deserves further justification, as preserving information from the entire history may intuitively be more beneficial, particularly for long-horizon counterfactual estimation.

Questions for authors
  1. Novelty and theory: 


1.1. To what extent does CAETC represent a conceptual advance over Causal CPC (CCPC), beyond replacing the InfoMax objective with a reconstruction loss and the CLUB-based adversarial objective with entropy maximization? What are the fundamental advantages of these alternative formulations? 


1.2. Then, Given that Theorem 5.4 of CCPC already establishes treatment-balanced representation distributions, what additional theoretical insight or guarantee does the corresponding balancing result in CAETC provide? Could the authors clarify the distinction between these results, particularly in light of the connection acknowledged around raw 270? 


1.3. Invertibility assumption: How can the application of Theorem 3.2, which assumes an invertible representation, be justified for an architecture that only encourages local reconstruction? Under what conditions would the proposed reconstruction objective guarantee the required invertibility?


1.4. Local versus global information preservation: What motivates the choice of local reconstruction rather than an objective that preserves information from the entire observed history, as in CCPC? Could local reconstruction lead to information loss that becomes particularly detrimental for long-horizon counterfactual predictions?

  1. Experimental results and improvements:


2.1. The improvements over CCPC in Tables 1 and 2 appear relatively modest, and CCPC outperforms CAETC at certain prediction horizons. How do the authors interpret these results in relation to their claim of strong improvements across all time steps? Are the observed differences statistically significant?


2.2. Ablation and comparison: Could the authors provide a controlled comparison between the reconstruction objective of CAETC and the InfoMax objective of CCPC, using the same backbone, training protocol, and hyperparameter tuning budget? This would help isolate the actual contribution of the proposed information-preservation mechanism.


2.3. Robustness to treatment-policy shifts: Figure 5 suggests promising improvements when models are trained under strong confounding and evaluated on non-confounded data. Could the authors study and analyse these improvements and investigate treatment-policy shifts (between train and test)? This appears to be a particularly interesting direction for strengthening the paper's contribution.


2.4. Causal estimation versus distribution-shift robustness and generalization : In Figure 5, CAETC achieves better predictive performance when trained on confounded data and evaluated on non-confounded data. However, how can the authors establish whether these improvements reflect more accurate causal effect estimation or greater robustness to changes in treatment assignment policies? Since ground-truth causal effects are available in the synthetic setting, could the authors directly evaluate treatment-effect estimation errors to support their interpretation (rather than relying exclusively on RMSE)?


2.5. Architecture-dependent performance: CAETC-LSTM and CAETC-TCN exhibit different relative performances across datasets and prediction horizons. What explains these differences, and what do they reveal about the respective contributions of the architecture and the proposed learning objectives?

3 comments

Thank you, Reviewer 1Qt4, for your time in reviewing our work. Here, we hope to partially resolve some of the misunderstandings and concerns. Please let us know if you want further clarification on any details.

1.1 + 1.4

CAETC and CCPC are indeed two ways to solve a problem, though we argue CAETC is much simpler than CCPC. For example, the CLUB loss in CCPC requires access to both the joint and product of marginal distributions (CCPC's Eq. (9)), which the authors approximated by shuffling over the batch dimension. Adversarial entropy maximization in CAETC (Eq. (15)) directly targets the classifier outputs, while the simplified form is also directly equivalent to the divergence term in Theorem 3.2. On the other hand, the proposed InfoMax regularization in CCPC implicitly encourages reconstruction but is computationally expensive in practice and requires additional approximation. We also emphasize that CCPC does not directly encourage global reconstruction but optimizes a lower bound of a lower bound (CCPC's Eq. (8)), which only achieves global reconstruction at the global optimum. CAETC's local reconstruction is an explicit, cheap, and simple heuristic for partial invertibility. So while CCPC has some nice theoretical motivations, the authors have to resort to approximation, which practically impacts their claimed theoretical properties. We acknowledge the practical tradeoff and therefore propose CAETC. Hence, CCPC and CAETC encourage invertibility from different perspectives, one implicitly on the lower bound of lower bound of global reconstruction, whereas the other is explicitly local reconstruction. As such, it is entirely possible that CAETC outperforms CCPC in certain settings and vice versa.

1.2

Theorem 3.1 of CAETC, as well as Theorem 5.4 of CCPC, is also similar to Theorem 1 of CRN and Theorem 4.1 of CT. These are proofs that the equilibrium for the proposed adversarial game of each method satisfies the treatment-invariant representation, and are not unique by themselves. However, in practice, we may not converge to equilibrium. Theorem 3.2 of CAETC establishes the performance bound in a more general case. CAETC's adversarial game is different and simpler than CCPC, while also minimizing the exact quantity desired by Theorem 3.2 instead of adding an additional divergence term as in Eq. (12).

1.3

Please see the response to W1 and Q1 of Reviewer Y15e.

2.1 + 2.2 + 2.5

Please see the response to W2 and Q3 of Reviewer Y15e.

2.3 + 2.4

It has been established before that there is a connection between the generalization bound of counterfactual estimation and domain adaptation by Johansson et al. (2022). Similarly, the techniques used in CRN, CT, CCPC, and CAETC share similarities with techniques in the domain adaptation literature. Therefore, we can indeed look at Fig. 5 through the lens of domain adaptation. However, another alternative view is simply that we are removing the shortcut induced by confounding, which the model may rely on during training. A model trained to be counterfactually generalizable should not rely on this shortcut. This is true for the setting in both Figures 4 and 5 and Table 1. Note that for the binary case, the treatment effect is straightforwardly defined as the difference between the two outcomes. However, for categorical treatment, for K options, there are K(K-1)/2 ways to compute treatment effects between outcome pairs. Therefore, we opt to evaluate outcomes only.

Answers to 1.1 and 1.4 :

Thank you for this clarification. I find the argument that CAETC provides a simpler alternative to CCPC while achieving comparable or slightly better predictive performance reasonable and potentially convincing. However, I do not think this positioning is sufficiently clear in the current manuscript. If the main contribution of CAETC lies in providing a simpler and more practical alternative to CCPC, rather than a major conceptual advance, I believe this should be explicitly reflected in the paper and supported by appropriate evidence. In particular, the claimed advantages could be substantiated through comparisons of computational efficiency (training time, memory, computational cost), optimization stability (sensitivity to initialization and hyperparameters), and implementation complexity (auxiliary modules and optimization steps). Regarding the trade-off between local reconstruction and global information preservation, I acknowledge that the current experimental results partially address my initial concern. Nevertheless, experiments involving longer prediction horizons or more complex temporal dependencies could help identify potential limitations of this simplification. Overall, I believe that an incremental methodological contribution can be scientifically valuable if it offers clear and sufficiently significant practical advantages. I would therefore encourage the authors to clarify CAETC's positioning relative to CCPC and provide stronger evidence supporting the claimed benefits of its simpler formulation. If these aspects are already adequately discussed or evaluated in the manuscript, I would appreciate it if the authors could point me to the relevant sections and precise line numbers, as I may have overlooked them.

Regarding question 1.2, I find the authors' clarification satisfactory.

Answers to question 1.3 : Thank you for this clarification. I understand the practical compromise adopted by the authors. However, local reconstruction does not guarantee the invertibility required by Theorem 3.2. I believe this limitation, and its implications for the applicability of the theoretical bound to the proposed architecture, should be explicitly acknowledged in the manuscript.

Answers to questions 2.1., 2.2, 2.5 : 
Thank you for these clarifications. I appreciate the authors' acknowledgment of the modest differences with CCPC and their willingness to revise the claims accordingly. However, Question 2.2 remains largely unanswered. A controlled comparison using the same backbone and training protocol would help isolate the contribution of the proposed learning objectives from architectural differences. This is particularly relevant if CAETC is positioned as a simpler alternative to CCPC.

Answers to questions 2.3, 2.4 :

Thank you for this clarification. I agree that the results in Figure 5 can be interpreted from both a domain adaptation and a confounding-induced shortcut perspective. Regarding Question 2.4, I am not fully convinced by the justification for evaluating outcomes only. The existence of multiple pairwise treatment effects in the categorical setting does not prevent their evaluation. For instance, reporting the average treatment-effect estimation error across treatment pairs would provide valuable evidence to distinguish improvements in causal estimation from robustness to distribution shifts.

We thank the reviewer for the engagement, which helped to improve our work. Please see the revised version with changes highlighted in red to assist in the discussion below.

1.1+1.3+1.4. We've already discussed details related to CAETC and CCPC, as well as full invertibility and partial invertibility in lines 67-74, 155-159. However, we agree with the reviewer that these details may not be sufficiently clear in the manuscript. Therefore, we moved the limitations section from the appendix to the main paper and revised the text to explicitly acknowledge these trade-offs. Similarly, we revised the text for the semi-synthetic and real-world data results to reflect the previously discussed details.

2.2. We added a new result with CAETC using a GRU backbone in Table 3 to facilitate comparison with CCPC. The training protocol has already been designed to be consistent for all methods (please see Appendix G).

2.4. While we agree that additional results on treatment effects are useful, we don't find them to support our claim for counterfactual estimation, which is our explicit setting for both the theoretical and experimental results. Our experimental setting also follows a similar design to prior works (RMSN, CRN, CT, CCPC), which directly evaluate the factual and counterfactual outcomes.

Finally, while we understand that the reviewer may not agree with us on every detail to warrant an accept score, we hope that the discussion is sufficient to clarify many concerns. If it is the case, we hope the reviewer consider revising the score. Thank you for your time engaging with our work.

Reviewer_Y15eOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CAETC, a framework for counterfactual outcome estimation from longitudinal observational data. It combines partial autoencoding to mitigate historical information loss, adversarial balancing through prediction-entropy maximization, and FiLM-based treatment conditioning to model interactions between patient representations and planned treatments. The authors connect the adversarial objective to a generalized Jensen–Shannon divergence and present an error bound under representation invertibility assumptions. The framework is evaluated using LSTM and TCN backbones on synthetic tumor-growth data, semi-synthetic MIMIC-III data, and factual prediction tasks on real-world MIMIC-III records. Experiments examine different confounding strengths, prediction horizons, and component ablations.

Critical strengths

This paper provides a coherent and practically motivated architecture. CAETC combines partial reconstruction with adversarial balancing to address historical information loss, while FiLM-based treatment conditioning explicitly models interactions between patient representations and planned treatments. These complementary components provide a useful architectural approach to longitudinal counterfactual estimation, with potential relevance to personalized treatment assignment.

Critical weaknesses
  1. Gap between the theoretical assumptions and the implemented representation. Theorem 3.2 assumes an invertible representation, whereas CAETC encourages only partial reconstruction of current time steps. This does not guarantee that sequential ignorability is preserved after compressing HtH_t into Φ(Ht)\Phi(H_t); discarded information may introduce representation-induced confounding bias. The paper does not establish when partial reconstruction provides sufficient information for causal identification, limiting the applicability of its theoretical guarantee to the implemented model.

  2. Inconsistent empirical improvements. Tables 1 and 2 provide limited evidence of consistent gains over existing methods. Although Figure 5 shows more encouraging gains under strong confounding, the broader claims of strong improvement across all time steps are insufficiently supported. With only three random seeds and no paired uncertainty analysis, the reliability of the smaller gains remains unclear.

Questions for authors
  1. Theorem 3.2 assumes an invertible representation, whereas CAETC encourages only partial reconstruction. Under what additional conditions does sequential ignorability remain valid given Φ(Ht)\Phi(H_t)? Which guarantees apply to the implemented estimator when full invertibility is not achieved?

  2. Under the squared-error loss used in the experiments, what explicit assumptions on outcomes and predictions ensure a finite and informative bound on SS in Eq. (20)? How should the weights π\pi be chosen, and how does this choice relate to the treatment-probability weighting in the adversarial objective?

  3. Can you provide paired per-seed comparisons or confidence intervals against CCPC for Tables 1 and 2? The reported gains are small in several settings, and CAETC-TCN has a higher average RMSE than CCPC on semi-synthetic data. Which settings substantiate the claim of strong improvement across all time steps?

1 comment

Thank you, Reviewer Y15e, for your time in reviewing our work. We hope to address some of your concerns below, but please let us know if any details deserve additional clarification.

W1 and Q1

Our analysis is inspired by the seminal work of Shalit et al. (2017) and Johansson et al. (2022) on generalization bounds for treatment effect estimation and inherits the same invertibility assumption and, therefore, the limitation. Johansson et al. (2022) also further discuss the effect of non-invertibility and suggest that the marginal risk can be bounded using additional assumptions, while also pointing to some existing network architectures that have been studied and shown to be approximately invertible. However, in general, we typically trade one assumption for another. We are also aware of the representation-induced confounding bias work by Melnychuk et al. (2023). Notably, in the same work, the authors mention that invertibility is not the only option to ensure valid estimation. However, it remains difficult to bridge these theoretical analyses and practical implementation, especially in the time series setting. Therefore, we opt to use a practical design of partial invertibility, as full reconstruction may also affect performance (see lines 154-158).

This is a known problem where we aim to partially ensure conformance to theoretical requirements and observe empirical gains, instead of disregarding these requirements as in some prior works.

Q2

Since the bound in Theorem 3.2 applies to any fixed Φ\Phi and GYG^Y, besides the continuity assumption, which is satisfied by neural networks, we only additionally need the domain of V,Z\mathbf{V},\mathbf{Z} to be bounded, which is satisfiable for many types of data in healthcare, e.g., heart rate (see lines 295-298).

The weight π\pi is defined by the adversarial game. For example, in CRN, π=[1/K,...,1/K]\pi=[1/K, ..., 1/K] and for CAETC, π=[P(1)A,...,P(K)A]\pi=[P^A_{(1)},..., P^A_{(K)}]. We proved the general case with any π>0\pi>0. Note that Eq. (10) and Eq. (16) are exactly Jensen-Shannon divergence minimization with respective weights, so we just need to use those weights in the bound in Theorem 3.2.

W2 and Q3

Thank you for identifying our oversights in claiming "across all time steps" and with respect to CCPC. Except for CCPC, the gaps to other methods are much more discernible. We are aware of the small differences between time steps compared to CCPC, which is why we provide the final average column to demonstrate cumulative error over time steps, as both metrics matter. Since both CAETC and CCPC aim to solve similar problems from different lenses, it is expected that these two methods will have a similar performance range. Furthermore, architectural differences are expected to result in performance variations, and CAETC-TCN suffers mostly on the semi-synthetic dataset. Due to the limited number of seeds, it is difficult to obtain statistically significant results compared to CCPC. We agree that the writing should be more nuanced, which was omitted previously due to space and time constraints. We will expand the writing in the revised manuscripts to discuss the negligible difference between CAETC and CCPC, as these methods address similar issues from different lenses, and the larger difference from other methods. In summary, the claim should be reduced to "demonstrates strong improvement compared to methods overlooking invertibility".

Reviewer_VymNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CAETC for counterfactual estimation over time. CAETC combines adversarial treatment balancing with partial autoencoding to preserve historical information and treatment conditioning for outcome prediction. The design is applied to both LSTM and TCN backbones. Experiments include synthetic tumor-growth data, semi-synthetic MIMIC-III, and real-world MIMIC-III, with comparisons against RMSN, CRN, CT, CCPC, and an unbalanced LSTM. Component ablations are also provided. 33757_Causal_Autoencoding_and_

Critical strengths

The paper addresses the information-loss problem of adversarial balancing by explicitly introducing reconstruction and treatment conditioning. The method is evaluated with two sequence backbones, and the ablation study separately examines treatment conditioning, reconstruction, and adversarial balancing.

Critical weaknesses

Although treatment-invariant representation learning is central to the method, the experiments mainly evaluate outcome errors and do not directly quantify the achieved treatment balance or residual treatment information in the learned representation. The real-world experiment evaluates observable factual outcomes because counterfactual outcomes are unavailable, so it cannot directly establish real-world counterfactual accuracy.

Questions for authors

See the above.

1 comment

Thank you, Reviewer VymN, for your time reviewing our work. Here, we want to clarify some potential misunderstandings. However, please do help us clarify your intention if our answer does not sufficiently address your concerns.

Treatment-invariant representation. Theorem 3.2 implies that there exists an inherent tradeoff between observational error (first term, right side) and treatment-invariant representation (second term, right side). Unlike prior works that only consider the equilibrium case (CRN - Theorem 1, CT - Theorem 4.1, and CCPC - Theorem 5.4), which requires a stricter invariant representation, we established both the equilibrium case (Theorem 3.1) and the generalization bound (Theorem 3.2). The second term is exactly the divergence between the conditional representation distributions. We emphasize that under Theorem 3.2, methods can achieve good performance even when the representation is not fully treatment-invariant by having smaller observational errors. The treatment-invariant representation term is therefore similar to a regularization term, and evaluating the term does not sufficiently reflect the performance, which is analogous to a smaller final L2 regularization value not meaning better performance. Instead, we directly evaluate an approximation of the factual and counterfactual prediction performance (Figures 4, 5, and Table 1), which is the target for our work.

Counterfactual in real-world experiments. We emphasize that existing works also suffer from this problem since it is, by definition, impossible to evaluate on counterfactual outcomes in real-world data, i.e., the fundamental problem of causal inference. Therefore, existing works, including ours, circumvent this problem by theoretical analysis and experimental results on synthetic data, where we control the data generation process and can generate counterfactuals. However, it is worth a sanity check to evaluate whether CAETC significantly impacts the factual prediction performance via the factual experiments. Our results show that CAETC does not deteriorate in terms of the factual outcome performance.

Comments on the paper

1 comment

We thank all the reviewers for your time in reviewing our paper. We have revised our paper to reflect the details from our discussion with the reviewers. Please see the revised version with changes highlighted in red. A summary of changes is as follows:

  • Revise the text for semi-synthetic and real-world data results.
  • Move the limitations section from the appendix to the main paper and revise the text to acknowledge additional limitations.
  • Add an additional result on CAETC with a GRU backbone in Table 3.
  • Minor typography fixes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.