acceptodds
Under review as a conference paper at ICLR 2027

Evaluating Counterfactual Quantities Using Cross-World Factor Models

Abstract

The evaluation of counterfactual queries is a central objective of causal inference. We develop an identification strategy for counterfactual queries using cross-world factor models, which provide a compact representation of causal relationships across treatment levels using scalar latent factors. First, we characterize when this representation is saturated, that is, when it can be adopted without loss of generality relative to an unrestricted structural causal model, and show that saturation requires at least as many latent factors as treatment levels (). Next, under conditional mutual independence and additional regularity and loading-structure assumptions, we derive identification results for the joint distribution of the potential outcomes from their conditional marginal distributions. Our identification results require no more latent factors than treatment levels (), implying that a model satisfying both saturation and our identification conditions must use the same number of latent factors as treatment levels (). We further develop a parametric maximum likelihood estimation procedure and demonstrate the proposed approach using a real-world randomized recommendation dataset.

Primary area:
causal reasoning
Keywords:
causal inference, counterfactual quantities, cross-world factor models, saturated models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_FBdSOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the identification of joint potential outcome distributions using a specified pattern of shared latent factors. Specifically, the paper gives rank conditions for model saturation and identification under factor independence and equal shared factor loadings. The paper then develops maximum likelihood estimation and evaluates counterfactual rankings on synthetic data and randomized recommendation data.

Critical strengths
  1. The paper studies the identification of joint dependence across potential outcomes, an important problem beyond identifying individual outcome distributions.
  2. The identification argument is clear and constructive, and the rank conditions and examples help explain when the proposed representation is sufficient.
  3. The paper provides finite-sample experiments, uncertainty estimates, and sensitivity analysis.
Critical weaknesses
  1. The choice of the factor sharing matrix M is not well justified. Identification assumes that M is specified, but observed marginal distributions do not generally determine the correct structure.
  2. The ranking conclusion is sensitive to small changes in the factor loadings. Appendix F shows that slightly changing lambda1 can change which treatment is most likely to be best, and fewer than half of the tested settings satisfy the variance conditions.
  3. The synthetic experiments estimate only the location parameters while treating the other NIG parameters as known, whereas the real application estimates all parameters. Therefore, the experiments provide limited evidence for the full estimation problem.
  4. The real application lacks sufficient model checks. Viewing time is nonnegative and contains zeros, while the NIG model allows negative values, and each treatment includes many different videos.
Questions for authors
  1. How should M be selected, and how do the conclusions change under alternative compatible structures?
  2. How should the sensitivity results be interpreted when the variance conditions fail or the fitted marginal distributions change?
  3. What are the estimation errors and interval coverage when all distribution parameters are estimated?
  4. What video sampling rule defines each treatment, and how well does the fitted distribution describe the observed viewing times?
3 comments

We thank the reviewer for their time and valuable feedback. We have uploaded a revised manuscript addressing the reviewers’ feedback, with major revisions highlighted. Below, we reproduce your comments and respond to each in turn.

Comment: 1. The choice of the factor sharing matrix MM is not well justified. Identification assumes that MM is specified, but observed marginal distributions do not generally determine the correct structure.

Questions 1. How should MM be selected, and how do the conclusions change under alternative compatible structures?

Our Response: MM is the matrix that specifies the factor-sharing structure, which is the core of our assumptions, as discussed in Examples 1–3. MM cannot be determined from the marginal distributions alone and must be specified from domain knowledge. In our application, each tag is assigned an additive response component, and MM follows the sharing relationships among the tag sets {39}\{39\}, {39,43}\{39,43\} and {20,43}\{20,43\} (Section 6 and Appendix F). In practice, we recommend comparing the results across several plausible factor-sharing structures. The conclusions may differ depending on the assumptions, as the comparison with rank preservation (RP) in Section 6 and the sensitivity analysis in Appendix F illustrate; the extent of this dependence varies across datasets.

Comment: 2. The ranking conclusion is sensitive to small changes in the factor loadings. Appendix F shows that slightly changing lambda1 can change which treatment is most likely to be best, and fewer than half of the tested settings satisfy the variance conditions.

Questions 2. How should the sensitivity results be interpreted when the variance conditions fail or the fitted marginal distributions change?

Our Response: We acknowledge that the results do not support a single conclusion about which treatment is best. Each grid point should be interpreted conditional on its assumed loadings. At grid points that fail the sample-variance check in (73), the sample variances conflict with conditions that Assumption 1 implies under those loadings, so these settings deserve less weight; however, passing the check does not confirm a setting.

Some patterns persist across the explored grid. First, across all 81 grid points, the PoB of television drama never exceeds 0.60, in contrast to 0.998 under RP. Second, the PoB of animation remains between 0.33 and 0.40, so animation is the best option for roughly one-third of users in every setting. Third, 32 of the 36 settings in which urban short drama has the largest PoB fail the check; among the 38 settings that pass, urban short drama has the largest PoB at only 4. Regarding the fitted marginals, the identification theory keeps the marginal distributions fixed, whereas our implementation refits the NIG model at each setting, so the results at each grid point reflect both the assumed loadings and the refitted marginals. In the revision, we have added these results to Appendix F and summarized the main findings in Section 6.

Comment: 3. The synthetic experiments estimate only the location parameters while treating the other NIG parameters as known, whereas the real application estimates all parameters. Therefore, the experiments provide limited evidence for the full estimation problem.

Questions 3. What are the estimation errors and interval coverage when all distribution parameters are estimated?

Our Response: We conducted a synthetic experiment using the same three-treatment, eight-parameter model as in the real-data analysis, estimating all location, scale, and shape parameters, with the same group sizes as the real data (3,939, 8,069, and 5,028), and report it in Appendix E (Table 7). Over 100 simulated datasets, the absolute bias of PoB was at most 0.0051, and the RMSEs were 0.033, 0.032, and 0.007 for urban, TV, and animation, respectively. The intervals covered the true PoB in 90%, 90%, and 94% of the datasets, with average widths (0.120, 0.120, and 0.029) close to those in the real-data analysis (0.115, 0.115, and 0.032). Coverage for urban and TV was slightly below the nominal level, which may partly reflect the small number of bootstrap samples (100, owing to time constraints); we will increase both the number of bootstrap samples and the number of simulated datasets in the final version. This supplements the existing experiments in Appendix E by assessing the accuracy of the same estimation procedure used in the real-data analysis.

Comment: 4. The real application lacks sufficient model checks. Viewing time is nonnegative and contains zeros, while the NIG model allows negative values, and each treatment includes many different videos.

Questions 4. What video sampling rule defines each treatment, and how well does the fitted distribution describe the observed viewing times?

Our Response: In KuaiRand’s random exposure, a video is sampled uniformly from 7,583 candidates and displayed. Our analysis is restricted to videos with the specified tag sets, and each treatment is defined as randomly presenting a video from the candidates in that category (66, 152, and 89 videos for urban, TV, and animation, respectively). Because the video is drawn at random within each category, each potential outcome includes variation across the videos in that category; the treatment is the randomized presentation of a video from the category. We have stated this sampling rule in Appendix F. We use NIG as a continuous approximation to represent the skewness and tails of viewing times. To assess this approximation, we added distributional checks: the observed proportion of zero viewing times was 0.24–0.34%, the fitted probability of negative values was at most 0.11%, and the maximum difference from the empirical CDF was 0.015–0.021. These results indicate a close overall fit to the marginal distributions, with negligible probability assigned to negative values. We have added these results and group-specific QQ plots to Appendix F (Figure 5).

Thank you for the clarifications and additional experimental results. My concerns have been addressed, and I maintain my original rating.

Reviewer_qUbqOpenReview ↗
Rating
3(+1)
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes cross-world factor models for identifying joint counterfactual distributions from marginal potential-outcome distributions. It derives saturation and identification conditions, develops an MLE-based estimator, and evaluates the approach on synthetic data and KuaiRand-Pure.

Critical strengths
  1. The paper addresses an important identification problem in causal inference.
  2. The theoretical framework is clearly formulated, with both synthetic experiments and a real-world application.
Critical weaknesses
  1. The main results rely on latent-factor independence and homogeneous loadings across treatments. These assumptions are difficult to verify and are not adequately justified in the real-world setting.
  2. The saturation condition follows directly from matrix rank, while the MLE asymptotic results largely rely on standard estimation theory. The contribution beyond the assumed factorization is somewhat limited.
  3. The synthetic experiments assume the proposed model is correctly specified and estimate only location parameters. There is little evidence of robustness to model misspecification.
  4. The factor-sharing structure is manually specified without sufficient justification. The large differences from rank preservation in Table 2 further highlight how strongly the conclusions depend on modeling assumptions, with no ground truth to establish which estimates are reliable.
Questions for authors
  1. How is the factor-sharing structure selected in practice, and how sensitive are the results to alternative structures?
  2. How does the method perform when the latent factors are correlated?
  3. Can you provide evidence supporting the distributional assumptions used in the real-world experiment
4 comments

We thank the reviewer for their time and valuable feedback. We have uploaded a revised manuscript addressing the reviewers’ feedback, with major revisions highlighted. Below, we reproduce your comments and respond to each in turn.

Comment: 1. The main results rely on latent-factor independence and homogeneous loadings across treatments. These assumptions are difficult to verify and are not adequately justified in the real-world setting.

Our Response: We acknowledge that Assumptions 1 and 3 cannot be fully verified from the observed data and require substantive justification in the real-world setting. In general, identifying cross-world quantities from treatment-specific marginals requires additional assumptions, since these marginals do not determine the joint distribution. Our framework offers an alternative to common assumptions such as rank preservation and independent potential outcomes by allowing partial dependence through shared latent factors. Its applicability depends on whether our assumptions are credible in the domain.

Substantive interpretation. In our application, latent factors represent interpretable characteristics: users’ responses to content attributes indexed by observed tags. Tag 39 is shared by urban short drama and television drama, tag 43 by television drama and animation, and tag 20 is specific to animation. Assumption 3 then states that a user’s response to a given tag contributes equally regardless of which content type carries it, and Assumption 1 states that responses to different tags are independent. Domain experts can judge whether these assumptions are plausible. These assumptions may be plausible when the tags capture distinct sources of preference and their contributions remain stable across categories. They may fail when preferences for different tags are correlated or when a tag's contribution depends on the surrounding content. We have made this interpretation explicit in Section 6.

The assumptions can also be partially checked and their influence assessed.

Testable implications. Assumptions 1 and 3 jointly imply the variance conditions in (18). In the application, the component variances implied by the sample variances are all positive (5.89×1085.89 \times 10^8, 2.36×1082.36 \times 10^8, and 0.50×1080.50 \times 10^8), so the data do not contradict the assumptions in this respect, although this condition is necessary but not sufficient.

Sensitivity to homogeneous loadings. Appendix F reports a sensitivity analysis over λ1,λ2∈[0.8,1.2]\lambda_1, \lambda_2 \in [0.8, 1.2]. The PoB of animation remains stable (0.33–0.40), whereas the PoBs of the other two treatments, and hence the top-ranked treatment, vary with λ1\lambda_1.

We have summarized this analysis and the sample-variance check in Section 6, stating explicitly that the identity of the top-ranked treatment depends on Assumption 3. Our response to Question 2 reports an additional correlation analysis (Appendix F, Table 16) that further assesses sensitivity to Assumption 1.

Comment: 2. The saturation condition follows directly from matrix rank, while the MLE asymptotic results largely rely on standard estimation theory. The contribution beyond the assumed factorization is somewhat limited.

Our Response: We agree that the saturation condition and MLE asymptotics use standard tools. Our central contribution is a new model class for cross-world dependence, together with a characterization of its representational capacity and conditions for identification.

Model. Cross-world factor models make assumptions about cross-world dependence explicit and interpretable through the factor-sharing structure. This formulation yields novel conditions under which cross-world quantities are identifiable from treatment-specific marginals.

Identification. Theorem 3 gives sufficient conditions for identifying the joint distribution of the potential outcomes from the marginals, while Theorems 2 and 5 establish non-identifiability results when J>KJ > K or when Assumption 3 is dropped. Together with Theorem 1, they show that J=KJ = K is necessary for saturation and identification to hold together under our conditions. Theorem 2 does not follow from rank arguments alone: its proof shows that, when the columns of MM are distinct and nonzero, the row space of MM cannot be closed under componentwise multiplication if rank⁡(M)<J\operatorname{rank}(M) < J.

Estimation behavior. Beyond the standard asymptotics, Table 1 illustrates how factor-sharing structure affects estimation precision as KK increases: the variance decreases slightly with a shared baseline factor but grows sharply when each outcome loads on many factors.

Comment: 3. The synthetic experiments assume the proposed model is correctly specified and estimate only location parameters. There is little evidence of robustness to model misspecification.

Our Response: We have conducted a synthetic experiment with the same three-treatment, eight-parameter model as in the real-data analysis, treating all location, scale, and shape parameters as unknown, and report it in Appendix E (Table 7). Over 100 simulated datasets, the absolute bias of PoB was at most 0.0051, the RMSEs were 0.033, 0.032, and 0.007 for urban, TV, and animation, respectively, and the 95% bootstrap percentile intervals covered the true PoB in 90%, 90%, and 94% of the datasets. Coverage for urban short drama and television drama was slightly below the nominal level, which may partly reflect the small number of bootstrap samples; we will increase both the number of bootstrap samples and the number of simulated datasets in the final version.

To investigate sensitivity to structural misspecification, we examined violations of independence that leave the marginals unchanged (see our response to Question 2) and departures from homogeneous loadings (Appendix F). For potential-outcome correlations up to 0.18, the asymptotic bias in PoB is at most 0.021 in absolute value and animation remains the treatment with the largest PoB; at 0.44, the bias reaches 0.071 (Table 16). Over the evaluated grid λ1,λ2∈[0.8,1.2]\lambda_1,\lambda_2\in[0.8,1.2], the PoB of animation stays within 0.33--0.40, whereas those of urban short drama (0.01--0.52) and television drama (0.09--0.60) vary substantially, so the top-ranked treatment depends on Assumption 3 (Table 15).

Comment: 4. The factor-sharing structure is manually specified without sufficient justification. The large differences from rank preservation in Table 2 further highlight how strongly the conclusions depend on modeling assumptions, with no ground truth to establish which estimates are reliable.

Questions 1. How is the factor-sharing structure selected in practice, and how sensitive are the results to alternative structures?

Our Response: In practice, we specify the factor-sharing matrix MM based on domain knowledge about which latent components are shared across treatments. In our application, overlapping content tags motivate the proposed structure: categories sharing a tag share its associated latent factor. This is a substantive modeling assumption.

We agree that the counterfactual ground truth is unavailable for validation in this application. Accordingly, the differences from rank preservation (RP) in Table 2 demonstrate sensitivity to cross-world assumptions.

When several structures are substantively plausible, we recommend evaluating the target queries under each and reporting which conclusions persist. The variance conditions in (18) can also exclude candidate structures that are incompatible with the marginals. For example, the shared-baseline structure of Example 2 requires Var(Y2)≥Var(Y0)\mathrm{Var}(Y_2) \ge \mathrm{Var}(Y_0) under Assumptions 1 and 3, which the sample variances violate (2.87×1082.87 \times 10^8 vs. 5.89×1085.89 \times 10^8; Table 9).

Questions 2. How does the method perform when the latent factors are correlated?

Our Response: We newly examined this (Appendix F, Table 16) in our application by adding a common component to V0V_0 and V2V_2, which induces a positive correlation between the urban and animation potential outcomes while leaving the marginal distribution under each treatment unchanged. The estimator under the independence assumption therefore has the same limit, and its difference from the true PoB is the asymptotic bias caused by violating latent-factor independence. When the potential-outcome correlation Cor(Y0,Y2)\mathrm{Cor}(Y_0,Y_2) was 0.09, 0.18, and 0.35, the asymptotic bias in the PoB for animation was approximately +0.01, +0.02, and +0.05, respectively. At correlations of 0.09 and 0.18, animation remained the treatment with the largest PoB. Table 16 reports the results for all settings.

Since MM specifies the components expected to be strongly shared, this analysis examines residual dependence among factors under the specified structure. The results show both stability and a limitation. Under weak dependence, the bias is small and the top-ranked treatment is unchanged. Under stronger dependence, the conclusion can change: at the tested correlations of 0.26 and above, television drama has the largest PoB. Because such dependence leaves the marginals unchanged, it cannot be detected from the data.

Questions 3. Can you provide evidence supporting the distributional assumptions used in the real-world experiment

Our Response: We use NIG as a continuous approximation to represent the pronounced skewness and long tails of viewing times. To assess its adequacy, we added treatment-specific goodness-of-fit diagnostics. Across treatments, the maximum absolute discrepancies between the fitted and empirical CDFs (KS distances) are 0.015–0.021. Although NIG distributions allow negative values, the fitted probability below zero is at most 0.11%. These results indicate a close overall fit to the marginal distributions, with negligible probability assigned to negative values. We have added these diagnostics and treatment-specific Q–Q plots to Appendix F (Figure 5).

thank you for your detailed explanation and I decide to raise my score

Reviewer_RwKKOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes an identifications method for counterfactual queries using cross-world factor model. The identification results implies obtaining the joint distribution of the potential outcomes from their conditional marginal distributions under the assumptions of conditional mutual independence, additional regularity and loading structure. The authors also characterize saturation of the factor models which can represent every relationship allowed by an unrestricted SCM, under condition. Finally, the authors show empirical analysis on a real-world short video recommendation dataset.

Critical strengths

I really appreciate the illustrative example in section 2. The authors provided intuition of their approach at different parts of the paper. The authors provided empirical results on synthetic dataset and an interesting real-world dataset collected from a short-video platform to show the best potential outcome.

Critical weaknesses

Below I share my concerns:

  1. The author should define “factor structure” or give an example in the intro.
  2. The concepts of “Saturated model class“, “Unrestricted SCM” should be defined more intuitively. Some intuitions should be provided in the introduction as well.
  3. Section 4 and 5 should be accompanied with more simpler discussion.
  4. Too many references to the Appendix.
  5. The authors should discuss the significance of the proposed method against existing counterfactual identification approaches. For example, I would request the authors to discuss how their approach similar and different from [1][2][3].
  6. In the experiment section, the authors did not compare against any baselines. Can the above approaches be used as baselines to estimate any counterfactual queries?
  7. What type of causal graphs can the proposed approach deal with? To my understanding, the empirical performance is shown mainly on X->Y graphs. More extensive, synthetic experiments should be provided. With larger causal graphs, high-dimensional covariates etc.

[1] Shpitser, Ilya, and Judea Pearl. "What counterfactuals can be tested." arXiv preprint arXiv:1206.5294 (2012).
[2] Maiti, Aurghya, Drago Plecko, and Elias Bareinboim. "Counterfactual identification under monotonicity constraints." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 25. 2025.
[3] Nasr-Esfahany, Arash, Mohammad Alizadeh, and Devavrat Shah. "Counterfactual identifiability of bijective causal models." International conference on machine learning. PMLR, 2023.

Questions for authors

Below I share my questions:

  1. What is g in equation 1?
  2. How do we obtain the conditional marginal distributions in practice? Given observational data only, can we identify the conditional marginal distributions?
  3. What is the significance of h and \bar{U} in Definition 1? Are h and \bar{U} defined?
4 comments

We thank the reviewer for their time and valuable feedback. We have uploaded a revised manuscript addressing the reviewers’ feedback, with major revisions highlighted. Below, we reproduce your comments and respond to each in turn.

Comment:1. The author should define “factor structure” or give an example in the intro.

Our Response: We have added the paragraph below to the Introduction, after the paragraph following Equation (3).

By the factor-sharing structure, we mean the specification of which latent factors enter which potential outcomes. Each latent factor UjU_j represents an unobserved individual characteristic, and sharing a factor across treatments links the corresponding potential outcomes across worlds. For example, on a short-video platform showing urban short drama, television drama, or animation, a user’s general viewing tendency may affect viewing time under every content type, whereas a preference for animation may affect it only when animation is shown (Example 2 in Section 2). With positive loadings, users with a high general viewing tendency then tend to watch longer under every content type. The structure is summarized by a binary matrix MM whose (x,j)(x, j) entry indicates whether UjU_j may enter YxY_x. Figure 1 illustrates three such structures as parallel-world graphs.

Comment: 2. The concepts of “Saturated model class“, “Unrestricted SCM” should be defined more intuitively. Some intuitions should be provided in the introduction as well.

Our Response: We have added the following intuition to the Introduction:

An unrestricted SCM places no restriction on how the outcome depends on its exogenous noise, so the potential outcomes may have any joint distribution. A representation is saturated if, with an unrestricted latent distribution, it can reproduce every such SCM, so that it can be adopted without loss of generality. For example, Example 1 is saturated, whereas a one-factor linear representation is not when K>1K>1, since it forces Y1Y_1 to be an affine function of Y0Y_0. In general, depending on the factor-sharing structure and the loadings, our representation may exclude some of these joint distributions.

Comment: 3. Section 4 and 5 should be accompanied with more simpler discussion.

Our Response: We have opened each section with a plain-language summary. For Section 4:

Under our assumptions, each potential outcome is a sum of independent components. Since the characteristic function of a sum of independent components is the product of theirs, under the rank condition the marginal distributions determine each component’s distribution; for example, in Example 2, the distribution of Y1−Y0Y_1 - Y_0 is obtained by “dividing out” the distribution of Y0Y_0 from that of Y1Y_1.

For Section 5, the opening paragraph now states that, instead of recovering the component distributions by numerical deconvolution, we fit a parametric factor model jointly to all treatment groups by maximum likelihood and that standard likelihood theory gives asymptotic normality.

Comment: 4. Too many references to the Appendix.

Our Response: We apologize for the frequent references to the Appendix. We have moved key content into the main text within the page limit; in particular, Section 6 now summarizes the sensitivity analysis (detailed in Appendix F, Figure 6), the sample-variance check, and the tag-based interpretation of the assumptions.

Comment: 5. The authors should discuss the significance of the proposed method against existing counterfactual identification approaches. For example, I would request the authors to discuss how their approach similar and different from [1][2][3].
[1] Shpitser, Ilya, and Judea Pearl. "What counterfactuals can be tested." arXiv preprint arXiv:1206.5294 (2012).
[2] Maiti, Aurghya, Drago Plecko, and Elias Bareinboim. "Counterfactual identification under monotonicity constraints." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 25. 2025.
[3] Nasr-Esfahany, Arash, Mohammad Alizadeh, and Devavrat Shah. "Counterfactual identifiability of bijective causal models." International conference on machine learning. PMLR, 2023.

Our Response: Thank you for suggesting these comparisons. [1] gives complete algorithms for identifying counterfactuals from the causal graph and experimental distributions, without restrictions on the functional form of the mechanisms. Its Lemma 7 (attributed to Avin et al., 2005) shows that, when XX is a parent of YY, the joint distribution P(yx,yx′′)P(y_x, y'_{x'}) of the same outcome under two treatment values is not identifiable; hence cross-world quantities such as PoB require additional assumptions. [2] and [3] provide such assumptions by restricting the mechanisms: [2] imposes monotonicity of mechanisms in their observed parents, formulated mainly for binary variables, and [3] assumes mechanisms that are bijective in the exogenous noise. Our assumptions instead restrict cross-world dependence directly through a specified factor-sharing structure, conditional mutual independence of the latent factors, and homogeneous loadings, under which the conditional joint distribution of the potential outcomes is identified from the conditional marginals (Theorem 3).

The significance is an alternative, interpretable route to cross-world identification when monotonicity or rank preservation is implausible. Multiple partially shared factors allow units' relative ranks to vary across treatments, and independent potential outcomes and rank preservation with location-shifted marginals arise as special cases (Proposition 1). In our application, this choice matters: rank preservation gives television drama a PoB of 0.998, whereas our model gives 0.327 (Table 2). Our results complement existing approaches; we do not claim uniformly weaker assumptions or broader applicability. We have added Appendix G, which cites and compares these papers (Table 17).

Comment: 6. In the experiment section, the authors did not compare against any baselines. Can the above approaches be used as baselines to estimate any counterfactual queries?

Our Response: Tables 2 and 3 already report a rank-preservation (RP) baseline. Regarding the suggested approaches, [1] does not impose monotonicity, and PoB is not identifiable under its criterion, so it cannot serve as a baseline. The monotonicity in [2] is formulated mainly for binary variables, whereas our paper targets continuous outcomes. In the unconfounded X→YX \to Y setting with scalar outcome and noise, [3] coincides with RP when the mechanisms are monotone in the noise in the same direction, so the RP baseline also represents [3] in this setting.

Comment: 7. What type of causal graphs can the proposed approach deal with? To my understanding, the empirical performance is shown mainly on X->Y graphs. More extensive, synthetic experiments should be provided. With larger causal graphs, high-dimensional covariates etc.

Our Response: Our approach focuses on identifying the joint distribution of potential outcomes across KK treatment levels from their conditional marginal distributions, rather than providing an identification procedure for arbitrary causal graphs. Over the observed variables, the causal graph is X→YX \rightarrow Y or, with covariates WW, W→XW \rightarrow X, W→YW \rightarrow Y, and X→YX \rightarrow Y, with no unobserved confounding between XX and YY given WW. Within this setting, we study additional structure: how latent factors are shared across potential-outcome worlds, as illustrated by the parallel-world graphs in Figure 1. Appendix E evaluates these structures with large KK (up to K=100K=100) treatment levels. These experiments assess scalability in the number of treatment worlds, but do not establish performance on larger causal graphs or with high-dimensional covariates. High-dimensional covariates pose additional challenges for estimating the conditional marginals P(Yx∣W=w)P(Y_x \mid W = w). We will clarify this scope and highlight estimation with high-dimensional covariates and extensions to more general causal graphs as important directions for future work.

Questions: 1. What is gg in equation 1?

Our Response: g:RK→Rg:\mathbb R^K\to\mathbb R is a measurable function defining the counterfactual quantity of interest, with E[ ∣g(Y0,…,YK−1)∣∣W=w ]<∞\mathbb E[\,|g(Y_0,\dots,Y_{K-1})| \mid W=w\,]<\infty. For example, g(y)=y1−y0g(\boldsymbol y)=y_1-y_0 gives the conditional average treatment effect, and g(y)=I{yr>max⁡x≠ryx}g(\boldsymbol y)=\mathbb I\{y_r>\max_{x\ne r}y_x\} gives the conditional probability that treatment rr achieves the best potential outcome (PoB). We have added an explanation immediately after Equation (1).

Questions: 2. How do we obtain the conditional marginal distributions in practice? Given observational data only, can we identify the conditional marginal distributions?

Our Response: With observational data, under conditional exchangeability Yx⊥X∣WY_x \perp X \mid W and positivity, these distributions are identified as P(Yx∣W=w)=P(Y∣X=x,W=w)P(Y_x \mid W = w) = P(Y \mid X = x, W = w). In randomized experiments, exchangeability follows from the randomization design. We have added this to Section 4.

Questions: 3. What is the significance of hh and Uˉ\bar{U} in Definition 1? Are h and Uˉ\bar{U} defined?

Our Response: hh is the generating function of the representation under consideration, and Uˉ\bar U is the latent vector used in that representation. Definition 1 fixes hh and defines saturation in terms of whether the potential outcomes of any SCM can be jointly reproduced by appropriately choosing the distribution of Uˉ\bar U. The factor model in Definition 2 is the case in which hh takes the form of a linear factor representation. We have stated this relationship explicitly in the revised manuscript.

I thank the authors for their rebuttal. I believe this work has significance and thus I keep my score as weak acceptance.

Reviewer_SV42OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper develops a framework for counterfactual identification with cross-world factor models. The paper characterizes a sufficient and necessary condition on the latent factors for the cross-world factor model to be expressive enough to represent any SCM. Then, under additional assumptions, the authors derive identification results for the joint distribution of potential outcomes from their marginals. For practical estimation, the authors propose a parametric maximum likelihood estimator and establish its asymptotic properties. The authors evaluate the proposed approach through synthetic experiments and a real-world randomized recommendation dataset.

Critical strengths
  • The paper is well-written and easy to follow, with a clear problem formulation and well-organized presentation. The authors clearly state the definitions and assumptions for the main theoretical results.
  • The paper addresses an important problem in causal inference. While several identification strategies have been studied, counterfactual identification remains challenging. The paper proposes a framework based on latent factor models and establishes identification results for counterfactual queries.
Critical weaknesses
  • My first concern relates to the practical aspect of the proposed framework. The paper proposed an identification framework for counterfactual queries under the assumption that latent factors are conditionally mutually independent. The assumptions seem to be more flexible than some established assumptions, such as monotonicity and independent potential outcomes, but they are not a strict relaxation of existing assumptions. So it is less clear how the proposed framework compares to the existing counterfactual identification approaches. Furthermore, although the identification results are established under a more general setting, the proposed estimator and theoretical guarantees rely on a specified parametric model, which limits the framework's practical applicability. I believe the paper would benefit from a more systematic comparison with alternative counterfactual identification approaches.

  • Another concern is related to the saturation and identification results. If I understand correctly, Theorem 1 establishes that the proposed factor representation is sufficiently expressive for any SCM. However, this saturation result does not require conditional independence of latent factors. So it is not obvious which models are identifiable under Assumptions 1-3 and which are saturated. This might be ok if the conditional independence requirement is not restrictive to the factor model in Section 3. But this should be discussed more explicitly in the paper.

Questions for authors
  • Could the authors clarify the relationship between the identifiable and saturated models? In particular, do assumptions 1-3 impose additional restrictions on the set of SCMs that the proposed factor model can represent?

  • Could the authors provide a more detailed comparison with existing approaches for counterfactual estimations? And provide some intuition about the practical aspects of the proposed framework, such as under what circumstances the proposed factor model assumptions would be more appropriate than alternative identification assumptions.

4 comments

We thank the reviewer for their time and valuable feedback. We have uploaded a revised manuscript addressing the reviewers’ feedback, with major revisions highlighted. Below, we reproduce your comments and respond to each in turn.

Comment: My first concern relates to the practical aspect of the proposed framework. The paper proposed an identification framework for counterfactual queries under the assumption that latent factors are conditionally mutually independent. The assumptions seem to be more flexible than some established assumptions, such as monotonicity and independent potential outcomes, but they are not a strict relaxation of existing assumptions. So it is less clear how the proposed framework compares to the existing counterfactual identification approaches. Furthermore, although the identification results are established under a more general setting, the proposed estimator and theoretical guarantees rely on a specified parametric model, which limits the framework's practical applicability. I believe the paper would benefit from a more systematic comparison with alternative counterfactual identification approaches.

Comment: Could the authors provide a more detailed comparison with existing approaches for counterfactual estimations? And provide some intuition about the practical aspects of the proposed framework, such as under what circumstances the proposed factor model assumptions would be more appropriate than alternative identification assumptions.

Our Response:

Relation to existing assumptions. We agree that our assumptions are not a strict relaxation of existing identification assumptions. Our contribution is an alternative set of restrictions on cross-world dependence, expressed through interpretable shared and treatment-specific latent factors.

Monotonicity, often used for binary outcomes, restricts the sign (and in some variants the magnitude) of individual effects; rank preservation (RP) requires each unit to keep its rank across potential-outcome distributions. The independent potential outcomes assumption imposes (conditional) independence between potential outcomes. Our framework represents cross-world dependence through shared and unshared factors, without requiring a common ranking or a uniform direction of treatment effects, with independence and RP with location-shifted marginals as special cases. We have added Proposition 1 and Table 17 in Appendix G to formalize these relationships and compare existing approaches.

In particular, under the stated conditions, M=IKM=I_K yields independent potential outcomes, while a single factor shared across all worlds yields RP with location-shifted marginals. The framework also includes the binary-treatment assumption (Y1−Y0)⊥Y0∣W(Y_1-Y_0)\perp Y_0\mid W as a special case. These inclusions concern specific subclasses; they do not establish that our assumptions are uniformly weaker than RP or monotonicity.

When our assumptions are appropriate. Strong or perfect dependence, as imposed by RP, and independence are often implausible in practice; in such cases, our assumptions provide an alternative. Our assumptions are most natural when treatments share interpretable components, as in our application, where video tags determine which factors are shared (Example 3). RP is natural when all potential outcomes are strictly increasing functions of a single common latent trait. Table 2 illustrates how different cross-world assumptions lead to different estimates from the same data.

Identification versus practical estimation. Our saturation and identification results (Theorems 1–3) make no parametric assumptions and provide a foundation for developing estimators beyond the particular parametric family used here. We adopt a parametric model for estimation to avoid the instability of nonparametric deconvolution, which involves ratios of estimated characteristic functions (e.g., ψ2=φ0φ2/φ1\psi_2 = \varphi_0\varphi_2/\varphi_1 in Example 3). We acknowledge that model misspecification can limit the estimator’s practical applicability and have therefore supplemented the application with goodness-of-fit diagnostics for the fitted marginal distributions (Appendix F, Figure 5). Developing more flexible estimators is an important direction for future work.

Comment: Another concern is related to the saturation and identification results. If I understand correctly, Theorem 1 establishes that the proposed factor representation is sufficiently expressive for any SCM. However, this saturation result does not require conditional independence of latent factors. So it is not obvious which models are identifiable under Assumptions 1-3 and which are saturated. This might be ok if the conditional independence requirement is not restrictive to the factor model in Section 3. But this should be discussed more explicitly in the paper.

Could the authors clarify the relationship between the identifiable and saturated models? In particular, do assumptions 1-3 impose additional restrictions on the set of SCMs that the proposed factor model can represent?

Our Response: Yes, these assumptions impose restrictions. Saturation is a property of representational capacity before restrictions are imposed on the latent distribution, and Theorem 3 establishes identification within the class satisfying Assumptions 1–3. The restriction is particularly explicit in the square case:

"When J=KJ=K, MM is invertible, and the loadings are nonzero, the joint distributions admitting a representation that satisfies Assumptions 1 and 3 are precisely those for which the components of M−1YM^{-1}\boldsymbol Y are mutually independent conditional on WW."

Assumption 2 imposes additional regularity conditions on densities and characteristic functions. Since marginal distributions alone do not determine the joint distribution, some restrictions are indispensable for point identification.

We have clarified this relationship in Section 4 (paragraph on the interpretation of Assumption 1) and in the caption of Figure 2.

Thanks to the authors for the detailed reply and revision of the manuscript. Sorry for the late response.

Regarding the relationship with existing assumptions, Proposition 1 mostly resolves my concerns. Although the question still remains: how would one choose the coupling between counterfactual worlds (in this case, the matrix M and dimensions J and K), I view this more as a fundamental difficulty of the problem than a limitation of the paper.

Regarding the saturated and identifiable condition. I agree with the authors that some restrictions must be applied for identification. And the paper states that J=KJ=K is a necessary (not sufficient) condition, which is correct. If the saturation property does not hold in general under the restrictive model, it is less clear what the expressiveness of the proposed factor model is under the identifiable conditions. I understand this is not part of the paper's main contribution; I think this should be stated more clearly.

Overall, the authors' response addresses my main concerns. And I remain positive about the paper.

Thank you for your positive assessment. In the final version, we will state more clearly how the identification conditions restrict the expressiveness of the model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.