- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Causal Autoencoding and Treatment Conditioning (CAETC), a framework for longitudinal counterfactual estimation under time-dependent confounding. The proposed approach combines partial autoencoding to encourage local invertibility and preserve historical covariate information, adversarial learning to obtain treatment-balanced representations, and treatment-specific conditioning to predict future outcomes. The authors also provide other theoretical results and deriving a bound on counterfactual estimation error. Experiments on synthetic, semi-synthetic, and real-world datasets show promising preliminary results regarding the model's robustness to changes in treatment assignment policies, particularly when trained on confounded data and evaluated on non-confounded data.
Critical strengths
The paper addresses an interesting and relevant problem in longitudinal counterfactual estimation. The authors have clearly invested substantial effort in developing the proposed framework, providing theoretical analyses, and conducting an extensive experimental evaluation across synthetic, semi-synthetic, and real-world datasets. The investigation of robustness to changes in treatment assignment policies is also an interesting aspect of the experimental study.
Critical weaknesses
Despite the relevance of the problem and the effort invested in this work, I have two main concerns regarding the novelty and experimental contributions of the proposed approach, as detailed below.
The conceptual novelty of CAETC appears somewhat limited, particularly compared to Causal CPC (CCPC, Bouchattaoui et al., NeurIPS 2024). Both methods address the same fundamental challenges of preserving historical information while learning treatment-balanced representations, although through different mechanisms.
The experimental claims should also be more nuanced. In particular, the statement that "CAETC-LSTM/TCN demonstrates strong improvement across all time steps" (raw 446 and 462) is not fully supported by Tables 1 and 2. The improvements over CCPC are generally modest, and CCPC even performs better at certain prediction horizons. Moreover, the relative performance of CAETC-LSTM and CAETC-TCN varies across datasets and horizons, which deserves further investigation. Indeed, For Table 2 CAETC-TCN is always better Than CAETC-LSTM but in table 1 CAETC-TCN is better until tau=3 then CAETC-LSTM becomes better. A more thorough analysis is needed to isolate the contributions of the proposed components, particularly to establish whether partial autoencoding offers advantages over the InfoMax objective used in CCPC. Comparing both mechanisms under the same architecture and training protocol would be particularly informative.
Finally, regarding the novelty of the theoretical contributions, there is substantial overlap between Theorem 3.1 of CAETC and Theorem 5.4 of CCPC, as the authors themselves acknowledge in raw 270. The authors should more clearly articulate what is theoretically new compared to this prior result. Moreover, the choice of encouraging only local invertibility deserves further justification, as preserving information from the entire history may intuitively be more beneficial, particularly for long-horizon counterfactual estimation.
Questions for authors
- Novelty and theory:
1.1. To what extent does CAETC represent a conceptual advance over Causal CPC (CCPC), beyond replacing the InfoMax objective with a reconstruction loss and the CLUB-based adversarial objective with entropy maximization? What are the fundamental advantages of these alternative formulations?
1.2. Then, Given that Theorem 5.4 of CCPC already establishes treatment-balanced representation distributions, what additional theoretical insight or guarantee does the corresponding balancing result in CAETC provide? Could the authors clarify the distinction between these results, particularly in light of the connection acknowledged around raw 270?
1.3. Invertibility assumption: How can the application of Theorem 3.2, which assumes an invertible representation, be justified for an architecture that only encourages local reconstruction? Under what conditions would the proposed reconstruction objective guarantee the required invertibility?
1.4. Local versus global information preservation: What motivates the choice of local reconstruction rather than an objective that preserves information from the entire observed history, as in CCPC? Could local reconstruction lead to information loss that becomes particularly detrimental for long-horizon counterfactual predictions?
- Experimental results and improvements:
2.1. The improvements over CCPC in Tables 1 and 2 appear relatively modest, and CCPC outperforms CAETC at certain prediction horizons. How do the authors interpret these results in relation to their claim of strong improvements across all time steps? Are the observed differences statistically significant?
2.2. Ablation and comparison: Could the authors provide a controlled comparison between the reconstruction objective of CAETC and the InfoMax objective of CCPC, using the same backbone, training protocol, and hyperparameter tuning budget? This would help isolate the actual contribution of the proposed information-preservation mechanism.
2.3. Robustness to treatment-policy shifts: Figure 5 suggests promising improvements when models are trained under strong confounding and evaluated on non-confounded data. Could the authors study and analyse these improvements and investigate treatment-policy shifts (between train and test)? This appears to be a particularly interesting direction for strengthening the paper's contribution.
2.4. Causal estimation versus distribution-shift robustness and generalization : In Figure 5, CAETC achieves better predictive performance when trained on confounded data and evaluated on non-confounded data. However, how can the authors establish whether these improvements reflect more accurate causal effect estimation or greater robustness to changes in treatment assignment policies? Since ground-truth causal effects are available in the synthetic setting, could the authors directly evaluate treatment-effect estimation errors to support their interpretation (rather than relying exclusively on RMSE)?
2.5. Architecture-dependent performance: CAETC-LSTM and CAETC-TCN exhibit different relative performances across datasets and prediction horizons. What explains these differences, and what do they reveal about the respective contributions of the architecture and the proposed learning objectives?
3 comments
Answers to 1.1 and 1.4 :
Thank you for this clarification. I find the argument that CAETC provides a simpler alternative to CCPC while achieving comparable or slightly better predictive performance reasonable and potentially convincing. However, I do not think this positioning is sufficiently clear in the current manuscript. If the main contribution of CAETC lies in providing a simpler and more practical alternative to CCPC, rather than a major conceptual advance, I believe this should be explicitly reflected in the paper and supported by appropriate evidence. In particular, the claimed advantages could be substantiated through comparisons of computational efficiency (training time, memory, computational cost), optimization stability (sensitivity to initialization and hyperparameters), and implementation complexity (auxiliary modules and optimization steps). Regarding the trade-off between local reconstruction and global information preservation, I acknowledge that the current experimental results partially address my initial concern. Nevertheless, experiments involving longer prediction horizons or more complex temporal dependencies could help identify potential limitations of this simplification. Overall, I believe that an incremental methodological contribution can be scientifically valuable if it offers clear and sufficiently significant practical advantages. I would therefore encourage the authors to clarify CAETC's positioning relative to CCPC and provide stronger evidence supporting the claimed benefits of its simpler formulation. If these aspects are already adequately discussed or evaluated in the manuscript, I would appreciate it if the authors could point me to the relevant sections and precise line numbers, as I may have overlooked them.
Regarding question 1.2, I find the authors' clarification satisfactory.
Answers to question 1.3 : Thank you for this clarification. I understand the practical compromise adopted by the authors. However, local reconstruction does not guarantee the invertibility required by Theorem 3.2. I believe this limitation, and its implications for the applicability of the theoretical bound to the proposed architecture, should be explicitly acknowledged in the manuscript.
Answers to questions 2.1., 2.2, 2.5 : Thank you for these clarifications. I appreciate the authors' acknowledgment of the modest differences with CCPC and their willingness to revise the claims accordingly. However, Question 2.2 remains largely unanswered. A controlled comparison using the same backbone and training protocol would help isolate the contribution of the proposed learning objectives from architectural differences. This is particularly relevant if CAETC is positioned as a simpler alternative to CCPC.
Answers to questions 2.3, 2.4 :
Thank you for this clarification. I agree that the results in Figure 5 can be interpreted from both a domain adaptation and a confounding-induced shortcut perspective. Regarding Question 2.4, I am not fully convinced by the justification for evaluating outcomes only. The existence of multiple pairwise treatment effects in the categorical setting does not prevent their evaluation. For instance, reporting the average treatment-effect estimation error across treatment pairs would provide valuable evidence to distinguish improvements in causal estimation from robustness to distribution shifts.
We thank the reviewer for the engagement, which helped to improve our work. Please see the revised version with changes highlighted in red to assist in the discussion below.
1.1+1.3+1.4. We've already discussed details related to CAETC and CCPC, as well as full invertibility and partial invertibility in lines 67-74, 155-159. However, we agree with the reviewer that these details may not be sufficiently clear in the manuscript. Therefore, we moved the limitations section from the appendix to the main paper and revised the text to explicitly acknowledge these trade-offs. Similarly, we revised the text for the semi-synthetic and real-world data results to reflect the previously discussed details.
2.2. We added a new result with CAETC using a GRU backbone in Table 3 to facilitate comparison with CCPC. The training protocol has already been designed to be consistent for all methods (please see Appendix G).
2.4. While we agree that additional results on treatment effects are useful, we don't find them to support our claim for counterfactual estimation, which is our explicit setting for both the theoretical and experimental results. Our experimental setting also follows a similar design to prior works (RMSN, CRN, CT, CCPC), which directly evaluate the factual and counterfactual outcomes.
Finally, while we understand that the reviewer may not agree with us on every detail to warrant an accept score, we hope that the discussion is sufficient to clarify many concerns. If it is the case, we hope the reviewer consider revising the score. Thank you for your time engaging with our work.
Thank you, Reviewer 1Qt4, for your time in reviewing our work. Here, we hope to partially resolve some of the misunderstandings and concerns. Please let us know if you want further clarification on any details.
1.1 + 1.4
CAETC and CCPC are indeed two ways to solve a problem, though we argue CAETC is much simpler than CCPC. For example, the CLUB loss in CCPC requires access to both the joint and product of marginal distributions (CCPC's Eq. (9)), which the authors approximated by shuffling over the batch dimension. Adversarial entropy maximization in CAETC (Eq. (15)) directly targets the classifier outputs, while the simplified form is also directly equivalent to the divergence term in Theorem 3.2. On the other hand, the proposed InfoMax regularization in CCPC implicitly encourages reconstruction but is computationally expensive in practice and requires additional approximation. We also emphasize that CCPC does not directly encourage global reconstruction but optimizes a lower bound of a lower bound (CCPC's Eq. (8)), which only achieves global reconstruction at the global optimum. CAETC's local reconstruction is an explicit, cheap, and simple heuristic for partial invertibility. So while CCPC has some nice theoretical motivations, the authors have to resort to approximation, which practically impacts their claimed theoretical properties. We acknowledge the practical tradeoff and therefore propose CAETC. Hence, CCPC and CAETC encourage invertibility from different perspectives, one implicitly on the lower bound of lower bound of global reconstruction, whereas the other is explicitly local reconstruction. As such, it is entirely possible that CAETC outperforms CCPC in certain settings and vice versa.
1.2
Theorem 3.1 of CAETC, as well as Theorem 5.4 of CCPC, is also similar to Theorem 1 of CRN and Theorem 4.1 of CT. These are proofs that the equilibrium for the proposed adversarial game of each method satisfies the treatment-invariant representation, and are not unique by themselves. However, in practice, we may not converge to equilibrium. Theorem 3.2 of CAETC establishes the performance bound in a more general case. CAETC's adversarial game is different and simpler than CCPC, while also minimizing the exact quantity desired by Theorem 3.2 instead of adding an additional divergence term as in Eq. (12).
1.3
Please see the response to W1 and Q1 of Reviewer Y15e.
2.1 + 2.2 + 2.5
Please see the response to W2 and Q3 of Reviewer Y15e.
2.3 + 2.4
It has been established before that there is a connection between the generalization bound of counterfactual estimation and domain adaptation by Johansson et al. (2022). Similarly, the techniques used in CRN, CT, CCPC, and CAETC share similarities with techniques in the domain adaptation literature. Therefore, we can indeed look at Fig. 5 through the lens of domain adaptation. However, another alternative view is simply that we are removing the shortcut induced by confounding, which the model may rely on during training. A model trained to be counterfactually generalizable should not rely on this shortcut. This is true for the setting in both Figures 4 and 5 and Table 1. Note that for the binary case, the treatment effect is straightforwardly defined as the difference between the two outcomes. However, for categorical treatment, for K options, there are K(K-1)/2 ways to compute treatment effects between outcome pairs. Therefore, we opt to evaluate outcomes only.