acceptodds
Under review as a conference paper at ICLR 2027

Factual-Anchored Counterfactuals for Causal-Diagnostic Explanations

Abstract

A counterfactual explanation of an observed case requires two choices: how alternative values are coupled to the values that actually occurred, and how much of the observed case the alternative world inherits downstream. We introduce factual-anchored counterfactuals (FACs), which encode these choices through natural-value-dependent laws. Scoped variables are revised by stochastic rules explicitly anchored at their realized factual values, while downstream propagation proceeds either through the original unit-level mechanisms (unit-persistent, u-FAC) or by sequential resampling from endogenous cross-world laws (resampling, r-FAC). We study diagnostic queries that condition simultaneously on the factual case and on a desired alternative outcome. Both variants admit the same Bayes decomposition into a factual-anchored revision law and a target-success score. However, they differ in their need for latent-background abduction. The u-FAC diagnostic law is generally not identified by the causal graph, revision law, and observed data. Conversely, r-FAC is point identified when its resampling laws are defined from the observed distribution, whilst giving up interventionist unit-level semantics. Finally, the revision law can be specified as an exponential tilt of the observed law, revealing connections between FAC-based explanations and feature-space, disturbance-based, and optimal-transport approaches to counterfactual explanation.

Primary area:
causal reasoning
Keywords:
causality, explainability, graphical models, counterfactuals

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ArHHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces two new definitions of counterfactuals, with the intention of leveraging the notions for providing counterfactual explanations, for instance in settings such as algorithmic recourse (when flipping of a decision is concerned).

Critical strengths

(S1) Some of the thinking in the paper is interesting. The paper explores a possibly worthwhile formulation for counterfactual explanations.

Critical weaknesses

(W1) The motivating example is difficult to follow, and does not seem to provide crystal-clear motivation as one would hope.

The example describes the setting with X -> A -> Y, and asks the question about alternative values of AA, labeled A∗A*, which may result in a positive decision. Then the example states this is considered through a revision law which conditions on A=a,X=xA = a, X = x: however, it is not clear why there is no conditioning on Y=yY = y in this law? I could understand the query motivation much better with Y=yY = y behind the conditioning bar, which actually happens later on in the paper (Section 5, where conditioning on history H=hH = h appears).

Similarly, an alternative is discussed, where PY(⋅∣A∗)P_Y(\cdot \mid A*) appears suddenly, once again without a crisp reason.

Overall, this leaves the feeling that the introductory example could be much better.

(W2) Why is the restriction to Vj,Mj,Mj∗V_j, M_j, M_j* a good idea?

The u-FAC definition seems to be a Layer 3 analogue of the concept of soft-intervention in Layer 2, which instead of atomic interventions, allows for distributions over the intervened value.

However, u-FAC conceptually departs heavily from this. Firstly, why does the definition require Γ\Gamma to condition on Vj,Mj,Mj∗V_j, M_j, M_j*? For instance, a soft-intervention may allow dependence on a new parent, whereas u-FAC does not; also, in Layer 3, one may consider distributions which depend on other downstream variables (e.g., Y=yY = y in the motivating example), yielding another restriction which is not clearly motivated.

Furthermore, the PVj(vj∗∣mj∗)P_{V_j}(v_j* | m_j*) with exp⁡{−βdis}\exp \{-\beta \mathrm{dis}\} proposal also seems to be a pretty arbitrary choice, which also depends on a specific notion of discrepancy.

Additionally, soft interventions also lead to the σ\sigma-calculus, which may help solve questions of identification, whereas this does not seem to be the case for this paper, beyond a non-ID counterexample in Thm. 1 and ID-by-construction in Prop. 5.

Therefore, this key definition seems to be tailored w.r.t. a downstream problem that is being solved (to satisfy something like a Bayesian factorization property), rather than being motivated as a fundamental new notion of a counterfactual. Put differently, the paper’s ideas would probably be leveraged much better if it was framed as a method for solving a specific explanation/recourse problem, rather than introducing a new notion of a counterfactual.

(W3) **Is Def. 2 even a counterfactual quantity? **

From reading Definition 2, it seems unclear whether this construct is actually a Layer 2, interventional construct, rather than a counterfactual. If yes, the paper stretches the name of “counterfactual” too far, and does not really sit well with existing literature.

Relatedly, for the case where the “null revision” is used (as in Prop. 1), doesn’t r-FAC violate the consistency axiom for SCMs, since D∗≠DD* \neq D can happen after resampling? It would be important to discuss the implication of this possible side effect.

(W4) Does the discrepancy form allow multivariate notion of distance/cost in the recourse formulation?

In the formulation, discrepancy of (vj,mj)(v_j, m_j) to (vj∗,mj∗)(v_j*, m_j*) is considered. However, a cost of changing one variable may depend on its children (e.g., V1 = number of credit lines -> V2 = total debt on credit lines, where the cost of changing V1 may depend on V2). Can the formulation be used to model such scenarios?

Questions for authors

(Q1) Can the authors point to the exact results that require the most significant theoretical lifting?

4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer ArHH: 1/2

We sincerely thank the reviewer for their careful assessment and for identifying where the motivating example needs more precision. We present our rebuttal below.

The motivating example and the role of the factual outcome (W1)

Why does the revision law does not condition on the factual outcome Y=yY=y? We have made this clearer by strengthening the motivating example with the diagnostic query:

Given this applicant’s financial history Z=zZ=z, observed debt-to-income ratio A=aA=a, and credit denial Y=0Y=0, how likely is each alternative ratio A⋆A^\star among scenarios in which the application would have been approved, Y⋆=1Y^\star=1?

This diagnostic query does indeed conditions on the factual outcome because it is part of the case we seek to explain.

The revision law is an input to this query. For instance, on the graph Z→A→YZ\to A\to Y, the auditor can specify the revision ΓA⋆(⋅∣a,z)\Gamma_{A^\star}(\cdot\mid a,z) using the factual exposure and pre-outcome information (before learning the decision). This restriction gives the law a direct interpretation as a resampling rule and connects it to natural-value-dependent policies, which use the exposure that arose naturally to define its alternative.

Writing h=(z,a,0)h=(z,a,0), Bayes’ rule gives Eq. (11) for discrete proposals:

\propto \Gamma_{A^\star}(a^\star\mid a,z)\, P(Y^\star=1\mid A^\star=a^\star,H=h).$$ **Question on first factor: could $\Gamma_{A^\star}$ also condition on $Y=y$?** Allowing $\Gamma_{A^\star}(a^\star\mid a,z,y)$ would introduce no cycle: the factual decision $Y$ is already realized when $A^\star$ is drawn, and revising $A^\star$ cannot change that earlier decision. Thus, Bayes’ rule still applies. The concern is how much discretion this gives the auditor. In the example, after seeing a denial, the auditor could give a particular debt-to-income ratio a higher proposal weight than they would after an approval. Even if two ratios have the same probability of success for this applicant, the more frequently proposed ratio will receive greater diagnostic weight. An outcome-dependent proposal can therefore strongly influence the explanation. Asking the auditor to specify the weights **among proposals that yield approval would go further and amount to supplying the explanation itself**. Moreover, if the proposal depends on the factual outcome, Bayes’ rule still applies, but Lemma 1’s implication $A^\star\perp Y\mid(Z,A)$ need no longer hold. We therefore favor our original modular proposal rule: the auditor can specify the revision, **before the decision**, using only the applicant’s history and observed ratio. As said before, the final diagnostic query still conditions on the factual denial. **Question on second factor: why does $P_{Y^\star}(\cdot\mid A^\star)$ appear?** The general success score in Eq. (11) would be $P(Y^\star=1\mid A^\star=a^\star,H=h)$. Since $A$ is the only parent of $Y$ **in the provided example**, the score reduces to $P(Y^\star=1\mid A=a^\star)$. This is why $P_{Y^\star}(\cdot\mid A^\star)$ appears in the example. We have revised the motivating example to clarify how the factual outcome enters the diagnostic query. ## Why restrict the kernel inputs, and how does u-FAC relate to soft interventions? (W2) **Local inputs.** We agree with the reviewer’s analogy. A soft intervention can assign a value stochastically, and a policy may use information beyond the node’s original parents. u-FAC likewise permits stochastic assignment, anchors it at the *observed* value, and evaluates downstream equations using the same unit’s factual disturbances. Proposition 2 makes this connection precise for fixed assignments and deterministic shifts of factual values. Following the reviewer questions, we now discuss broader inputs and two caveats. Conditioning on a *factual* descendant gives the auditor more discretion to shape the explanation and can invalidate Lemma 1’s anchoring independence. Conditioning on a *starred* descendant can create a cycle because its value may itself depend on the proposed revision. These considerations motivate our choice of modular revision laws that respect the topological order. **Why the exponential discrepancy/cost?** Equation (5) is optional. Its rationale is specific: if the analyst wants a revision law $q$ that trades off expected discrepancy/cost against departure from a reference law $Q$, minimizing $\mathrm{KL}(q|Q)+\beta E_q[\mathrm{cost}]$ yields $dq\propto e^{-\beta\,\mathrm{cost}}dQ$ (Appendix B, Lemma 2). Thus the exponential form follows from a penalized information-theory criterion. **This is the choice typically made for backtracking counterfactuals**
AuthorsOpenReview ↗

Rebuttal to Reviewer ArHH: 2/2

What Definition 2 represents under a null revision (W3)

Is r-FAC a Layer 2 intervention? In general, no. Definition 2 specifies a joint law for the factual and alternative records. The reviewer's intuition is correct: Layer 2 quantity arises in an important special case. In a DAG, under Eq. (7) and the conditions of Proposition 2, the downstream distribution at a fixed revision agrees with the corresponding interventional distribution. In the loan example, the approval probability then equals P(Y=1∣do(A=a⋆))P(Y=1\mid \mathrm{do}(A=a^\star)).

Under a more general r-FAC rule, two applicants with the same factual ratio aa and revised ratio a⋆a^\star can have different probabilities of alternative approval because their factual decisions differed. The interventional distribution alone does not determine that pairing.

Does r-FAC satisfy SCM consistency? In general, no, but u-FAC does. Consistency requires Y(a)=YY(a)=Y whenever A=aA=a. Under a null revision, u-FAC evaluates the same downstream equations with the same factual disturbances, so it reproduces all affected values: D⋆=DD^\star=D almost surely (Proposition 1). Under r-FAC, downstream values may be resampled even when A⋆=AA^\star=A. For example, under Eq. (7), an applicant denied at ratio aa can receive approval on a fresh draw at that same ratio. Thus, this provides no evidence that keeping the ratio unchanged could have reversed the original decision.

Multivariate costs and the credit-line example (W4)

Can the cost of changing credit lines depend on total debt? Yes, but with an extension to Eq. (5). The discrepancy can already involve several variables and their interactions, but the local proposal for V1V_1 only uses V1V_1 and its pillow. On V1→V2V_1\to V_2, this excludes the child V2V_2. The potential extension depends on whether the relevant cost is determined by current or resulting debt.

  • Current debt v2v_2. Suppose closing one credit line costs more for an applicant with a larger outstanding balance. A cost c(v1,v1⋆,v2)c(v_1,v_1^\star,v_2) captures this directly. An extended proposal ΓV1⋆(⋅∣v1,v2)\Gamma_{V_1^\star}(\cdot\mid v_1,v_2) can then give less weight to more costly changes. This introduces no cycle because factual debt is already observed.

  • Resulting debt v2⋆v_2^\star. If the cost depends on the balance after the change, one can first propose v1⋆v_1^\star, generate v2⋆v_2^\star downstream, and evaluate the joint cost c(v1,v2,v1⋆,v2⋆)c(v_1,v_2,v_1^\star,v_2^\star). This cost can rank successful scenarios. If it should instead guide the initial proposal, one can use the expected cost under the downstream debt law for each candidate v1⋆v_1^\star.

Which results carry the main theoretical contribution? (W2, Q1)

Theorem 1 and Propositions 4, 5 and 6.

Proposition 4 and Theorem 1 establish the identification limit for same-unit explanations. Proposition 4 expresses the success probability of each revision as an average over the disturbances compatible with the factual history. Theorem 1 proves that even exact knowledge of the causal graph, observed law, and revision law can leave the u-FAC diagnostic weights undetermined. The missing information is how the same unit’s responses under different revisions are related.

Proposition 5 gives an explicit identification functional for r-FAC. When the downstream resampling kernels are specified functionals of the observed law, Eqs. (13)–(14) show how to compose them in topological order and average over omitted factual inputs using P(L∣H)P(L\mid H). Lemma 1 justifies this averaging under the stated pillow and scope conditions. This result concerns identification under that law. Whether it represents same-unit responses is a separate modeling question.

Proposition 6 establishes why factual anchoring matters. A fixed marginal distribution of proposed revisions can yield different diagnostic weights for the same observed case. A case-specific explanation therefore requires specifying how alternative values are paired with factual values.


We thank the reviewer for pushing us to clarify the interpretation of resampling, the scope of our identification result, and the role of downstream costs. Their comments helped us explain these points more clearly in the revised manuscript.

Since the edits are expository and the theoretical results stand, we hope the clearer presentation will be reflected in an updated score. We welcome further discussion of any remaining questions.

Reviewer_ArHHOpenReview ↗

Thanks for the response

Thank you for the response.

(W1) Adding the explicit diagnostic query is helpful, but the discretion argument is unconvincing: Γ\Gamma is already fully auditor-specified, and an auditor may legitimately want it to depend on Y=yY = y (in the recourse setting, this would be the typical problem in fact; those with a positive decision will never be audited in a recourse setting). The remaining reason seems to be technical convenience.

Also, does the reduction of P(Y⋆=1∣A⋆=a⋆,H=h)P(Y^\star = 1 \mid A^\star = a^\star, H = h) to P(Y=1∣A=a⋆)P(Y = 1 \mid A = a^\star) hold under u-FAC? There the score is P(Ya⋆=1∣z,a,y)P(Y_{a^\star} = 1 \mid z, a, y), which may depend on hh?

(W2) The response still does not motivate restricting Γ\Gamma to (Vj,Mj,Mj⋆)(V_j, M_j, M_j^\star). Non-descendant inputs beyond the pillow are not discussed, and factual descendants may be of interest, as discussed in (W1). Also, the framing concern remains unaddressed: I still cannot tell if this is a new notion of a counterfactual vs. a method for a specific explanation problem. Whichever it is, the semantics could be far clearer.

(W3) The authors confirm that r-FAC is neither interventional in general nor consistent. If r-FAC is neither an L2 nor an L3 object, why call it a counterfactual?

Based on the above (which I believe are significant concerns), I will maintain my score.

AuthorsOpenReview ↗

Followup Rebuttal to Reviewer ArHH

We thank the reviewer for the follow-up.

Outcome-dependent proposals and the success score (W1)

Can the auditor use the factual denial when proposing revisions? Yes. As noted in our first response, this is mathematically possible and, as the reviewer suggests, can indeed represent some types of recourse more directly when only denied applicants are audited. The extension can invalidate the independence A⋆⊥Y∣(Z,A)A^\star\perp Y\mid(Z,A) in Lemma 1. For a denial, Bayes’ decomposition still applies with proposal factor ΓA⋆(a⋆∣z,a,0)\Gamma_{A^\star}(a^\star\mid z,a,0). To accommodate this choice, Lemma 1 should condition on all factual inputs to the proposal, here (Z,A,Y)(Z,A,Y). If A⋆A^\star is drawn using only these inputs and fresh randomization, it is conditionally independent of any remaining factual variables given (Z,A,Y)(Z,A,Y).

We will add a new appendix section developing this extension and the adjustment to Lemma 1.

Moreover, the second paragraph of Section 3 was ambiguous, and our previous reply (mentioning target-success) added to the confusion. We intended PY(⋅∣A=a⋆)P_Y(\cdot\mid A=a^\star) as one example of how the analyst might resample YY under r-FAC. The general resampling law is ΓY⋆(⋅∣y,a,a⋆)\Gamma_{Y^\star}(\cdot\mid y,a,a^\star), which may use the factual outcome yy and both the factual and revised Markov pillow values aa and a⋆a^\star. The law Γ\Gamma is an input and a designed choice. The analyst chooses which inputs to use, and the paper is written in its most general form: with all inputs used. That paragraph illustrates a simpler choice. It says "For example, drawing from PY(⋅∣A=a⋆)P_Y(\cdot\mid A=a^\star)....". That is, drawing a fresh outcome y⋆y^\star from the observational law of YY given its parent, using the revised a⋆a^\star in place of aa.

We will clarify that Section 3 paragraph to not confuse it the target-success score introduced later in Section 5.

Why local inputs, and what problem does FAC solve? (W2)

Why restrict Γ\Gamma to (Vj,Mj,Mj⋆)(V_j,M_j,M_j^\star)? This choice lets the auditor specify each revision locally: cases agreeing on these inputs receive the same proposal distribution at that node. We agree that a non-descendant outside the pillow, such as a baseline variable determining which revisions are feasible, can legitimately enter the rule. A factual descendant can also enter a retrospective proposal because its value is already known.

We will discuss both extensions, how they preserve Bayes’ decomposition, with omitted inputs handled as in W1.

What is the intended contribution? We favor the reviewer’s suggested framing. FAC addresses a specific diagnostic question: given this factual case, which proposed revisions are most likely among alternatives that achieve the target? The proposal determines how often each revision is considered. The downstream rule determines how often it succeeds.

We will foreground this question in the introduction and present u-FAC and r-FAC as two ways to answer it.

Is r-FAC counterfactual? (W3)

The reviewer raises a distinction that also arises in many topic in causal inference like principal strata and mediation analyses. On an augmented graph, r-FAC can be represented exactly as a sequence of stochastic interventions on the starred nodes [1]. With only single-world inputs, the resulting marginal outcome law is a Layer 2. When the resampler uses both factual aa (or yy) and revised a⋆a^\star, the diagnostic query is cross-world, and thus it is Layer 3. For u-FAC, shared disturbances give the diagnostic query its same-unit interpretation, which is undisputed Layer 3 when the resampling law uses cross-world inputs.

The average treatment-effect on the treated (ATT) offers a familiar analogy, as the query E[Y(0)∣A=1]E[Y(0)\mid A=1] evaluates an alternative outcome among units selected by factual treatment [2]. Mediation also illustrates why this distinction, L2L2 vs L3L3, matters. The natural direct effect uses the same unit’s M(a0)M(a_0) in Y(a1,M(a0))Y(a_1,M(a_0)), a Layer 3 quantity [3]. The interventional direct effect instead draws the mediator from a reference distribution, giving a Layer 2 interpretation and allowing identification under weaker conditions [4]. This mirrors u-FAC vs r-FAC!

Because _counterfactual_ can carry different meanings across frameworks, including SCMs and Rubin’s causal model, we will use the more precise SCM terms “Layer 2” and “Layer 3” to distinguish these queries in the final paper.

Thank you again!

References

[1] Correa, J., & Bareinboim, E. (2020). A Calculus for Stochastic Interventions: Causal Effect Identification and Surrogate Experiments. AAAI, 34(6), 10093–10100

[2] Shpitser, I., & Pearl, J. (2009). Effects of Treatment on the Treated: Identification and Generalization. UAI, 522–529

[3] Pearl, J. (2001). Direct and Indirect Effects. UAI, 411–420

[4] Vansteelandt, S., & Daniel, R. (2017). Interventional Effects for Mediation Analysis with Multiple Mediators. Epidemiology, 28, 258–265

Reviewer_pYNsOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies retrospective counterfactual explanations conditioned on a factual case and a desired alternative outcome. It proposes factual-anchored counterfactuals (FACs), separating stochastic revisions of selected variables from downstream propagation through either the original mechanisms and factual disturbances (u-FAC) or endogenous resampling laws (r-FAC). The authors derive a common Bayes decomposition, establish general non-identification for u-FAC and identification for observed-law-defined r-FAC under stated conditions, and relate revision laws to existing explanation methods. An Adult benchmark illustration for one selected factual case and two revision scopes shows that downstream semantics can change the resulting diagnostic distributions; it does not establish causal effects on income or comparative predictive superiority.

Critical strengths
  • The methodology subsumes relevant prior approaches, and its exposition appears comprehensive. This helps situate the framework relative to existing counterfactual explanation methods (Lines 079–085, 247–284; Appendix B).
  • The experiments are reasonable. The Adult comparison examines two revision scopes and alternative downstream constructions, providing an empirical illustration of the proposed distinction (Lines 451–474).
Critical weaknesses
  • I find the paper difficult to follow, particularly its opening explanation of the central concepts. Terms such as “coupled,” “inherits downstream,” and “natural-value-dependent laws” are unclear to me, as is the abstract’s identification statement (Lines 011–024). The introduction of semi-Markovian models and Γ also needs clearer explanation at first use (Lines 075–077, 114–118). This makes it difficult for me to assess the contribution confidently.
  • The practical takeaway remains unclear to me. After reading the conclusion’s discussion of the identification burden, resampling laws, and choices not determined by observed data, I do not know what to do with this as a reader (Lines 478–485).
Questions for authors
  • Clarification: Could you explain concretely what “coupled,” “inherits downstream,” and “natural-value-dependent laws” mean, and what the abstract’s identification distinction amounts to for a factual instance (Lines 011–024)?
  • Clarification: Are the stated null-revision properties not immediate consequences of the two constructions? What is non-obvious about these results (Lines 076–078)?
  • Clarification: Why is abduction described as “generally” required? My understanding is that inferring and then preserving the factual exogenous background requires abduction; please clarify the qualification (Lines 045–047).
  • Clarification: What should a reader take away from the conclusion’s distinction between u-FAC and r-FAC, given the remaining choices of scopes, revision laws, and downstream resampling laws (Lines 478–485)?
1 comment
AuthorsOpenReview ↗

Rebuttal to Reviewer pYNs

We thank the reviewer for identifying where the terminology obscures the central diagnostic question. We will use the loan example to introduce the definitions and revise the conclusion to explain how a reader can choose and interpret an explanation.

Terminology (W1, Q1)

Consider an applicant whose loan was denied. The central diagnostic question is: given this applicant’s financial history, debt-to-income ratio, and denial, which alternative ratios are most likely among scenarios that yield approval? Three terms describe how we construct those scenarios.

  • "Coupled" and "natural-value-dependent": Coupled means that the augmented causal model determines a joint distribution of the factual and counterfactual records. Given what actually happened, this distribution tells us how likely each alternative value is for the explanatory/exposure variable. Natural-value-dependent describes the revision law. The "natural value" is the exposure value that actually occurred; and the revision can depend on it, and more generally on all pre-outcome factual information for the case.

  • "Downstream (inheritance)": This asks what carries over when we evaluate a proposed ratio. Under u-FAC, we apply the original decision rule with the same unobserved circumstances that contributed to this applicant’s decision. Under r-FAC, a specified rule draws an alternative decision and determines how much the factual denial informs that draw.

We will also explain semi-Markovian at first use. It means here that the causal model allows two observed variables to share an unobserved cause.

What does identification mean? Suppose we knew the population distribution exactly, including the approval rate at every debt-to-income ratio. Would that knowledge settle which ratio deserves the most weight for this denied applicant? Under u-FAC, two models can agree on every approval rate yet disagree on which denied applicants would have been approved at another ratio (Theorem 1). Under r-FAC, once the proposal and downstream resampling rules are specified from observable quantities, the data determine the corresponding weights under the support conditions of Proposition 5. The analyst must still justify why that resampling rule answers the question of interest.

What the null revision shows (Q2)

Yes, the null-revision properties follow directly from the constructions. Proposition 1 makes their practical implication explicit. If we give the applicant their original ratio, u-FAC reproduces the (factual) denial because both the inputs and the unobserved circumstances are the same. Under r-FAC, a fresh decision at that ratio can yield approval. Such an approval tells us what can happen under reassessment. This is, resampling (r-FAC) can change the outcome even under no change in the explanatory variable, due to sampling noise, but it requires no abduction. The main theoretical distinction is what the data can determine under each construction (Theorem 1 and Proposition 5).

Why abduction is generally required (Q3)

The reviewer’s understanding is correct. Abduction means using what happened to infer which unobserved circumstances are compatible with the case. If several circumstances could explain the denial and they predict different decisions at the revised ratio, u-FAC must account for their probabilities given the factual record. If every circumstance compatible with the denial predicts approval at the revised ratio, the success probability is one regardless of their relative probabilities. The same-unit interpretation still applies, but there is no need to compute those probabilities explicitly.

What a reader should do in practice (W2, Q4)

Start by stating which variables may be revised directly and which alternatives are plausible for this case. Then choose the downstream rule according to the question. Use u-FAC to ask what would have happened to the same person, and state the assumptions about that person’s responses. Use r-FAC to ask what would happen under a specified reassessment procedure, and describe what it retains from the factual case. When several choices are credible, vary one at a time and report whether the explanation changes.

Figure 2 illustrates why this matters. With education alone in scope and the same education proposal, every successful u-FAC draw increased education. Under r-FAC at βd=0.5\beta_d=0.5, 34.5% of successful draws retained the original education level because downstream features could change on resampling. Thus, for this fitted case, whether an education increase appears necessary depends on what the alternative scenario preserves.


We thank the reviewer for these questions and recommendations. We have incorporated their suggestions into the revised manuscript and hope the clarifications help readers assess the assumptions and interpret the resulting explanations.

Reviewer_a9zMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors focus on the factual-based counterfactual (CF) generations. In my opinion, this can be viewed as a slightly different angle of ensuring the plausibility of CF generations from a factual base. The proposed approach builds on top of Markov theory and causal inference, which looks like a principled approach. The paper separates two choices that are usually conflated. One is the coupling that pairs an alternative value with this case. The other is the semantics that govern how the alternative propagates downstream. It contrasts a unit-persistent construction (u-FAC) with a resampling construction (r-FAC). It establishes well-posedness for both.

Critical strengths

The paper is disciplined about identification. It keeps separate what can be recovered from the observed law and what has to be assumed, and it reports that the unit-persistent construction is not identifiable even when the full causal graph is known. I think this is the paper's most useful contribution, as it tells the reader which claims rest on data and which rest on modelling choices.

Critical weaknesses
  1. The notation and the problem statement make the paper hard to follow. The notation is heavy and inconsistent throughout. Also, we need a clear problem formulation section to define what our target is. As far as I can tell there is no single statement of the inputs, the valid output and what makes an explanation good. Without a more explicit statement of what is being measured, I find it difficult to judge whether the approaches are effective.

  2. Some key assumptions are not listed until they are needed in the proof (they don't even appear in the theorem either). This is a bad practice for a theoretical study, because the reader cannot tell which result holds under which condition. It looks like the assumptions are doing the work rather than the theorems. For instance, Lemma 1 carries a requirement that the decomposition depends on. As I read it, that requirement is stated once in the lemma and then dropped by every proposition that uses it. Also, in the proof of theorem 1, "t(0),t(1)∼Bernoullit(0), t(1) \sim Bernoulli" is stated while Bernoulli is never mentioned in the main body.

  3. The experiments are too limited. Everything empirical is one case from one dataset. Backtracking is the only competing construction implemented, and it is not matched to u-FAC on its kernel. None of the reported quantities is a quality metric: target probability, the mean change in education and the feature distance describe the sampled distribution, not whether the explanation is correct. Moreover, key theoretical results without empirical support are not quite acceptable.

Questions for authors
  1. The notation is abused heavily throughout (even though the authors claim it is slightly abused). The conditioning argument is written two ways, sometimes with events and sometimes with bare values, apparently for the same object, and the paper never says whether the two are interchangeable. For example, eq (8) writes dPA⋆(a⋆∣H=h,Y⋆=y⋆)dP_{A^\star}(a^\star \mid H=h, Y^\star=y^\star), ie., the first argument is a bare value, while the conditioning set is written with events. Eq (11) puts both conventions in one display, dPA⋆(a⋆∣h,y⋆)∝dPY⋆(y⋆∣A⋆=a⋆,H=h)dP_{A^\star}(a^\star \mid h, y^\star) \propto dP_{Y^\star}(y^\star \mid A^\star=a^\star, H=h). Is hh meant to be the same object as H=hH=h, and y⋆y^\star the same as Y⋆=y⋆Y^\star=y^\star? The same symbols also shift meaning: dPdP serves for both a density and a probability element, MM for both the SCM and its posterior, and the star for both a variable and an event.

  2. Lemma 1 assumes condition (9) and additionally that Z⊇⋃rMAr∖AZ \supseteq \bigcup_r M_{A_r} \setminus A. If I read Propositions 3, 4 and 5 correctly, that second assumption is then dropped by all three, which cite the lemma without it. Section 5.1 also seems inconsistent about where it comes from: one sentence attributes it to the way ZZ is chosen, and a later one to condition (9). These cannot both be right.

  3. Proposition 4 states only "under Definition 1 and condition equation 9", but I believe eq (12) fails when the outcome is in the scope. Is Y∉SY \notin S intended as a standing assumption? The proof also invokes a conditional independence that looks too weak for the conclusion; as I read it, Lemma 1 gives a stronger statement than the one quoted.

  4. Proposition 5 is described as "estimable from observed data." What is the estimator, what is its rate, and has it been checked on a synthetic case where the structural truth is known? The kernels actually used depend on fitted quantities, so my concern is that this is estimation uncertainty rather than identification, and I would like to know which one the proposition is claiming.

  5. The paper does not state what makes a diagnostic law a good explanation. Equation (11) makes the answer a product of a score and a revision law, but I could not find a criterion separating a correct weighting from a merely computable one. No ground truth, downstream task or human study is offered either.

  6. How is the scope SS chosen? The paper says which variables must be in the scope, but never how to pick SS. The two scopes used are fixed without justification, and no sensitivity analysis over SS is run. "Scientifically chosen" would be more useful if it explained the choosing procedure, and sensitivity analysis is promised but not performed.

  7. There are math symbol typos, e.g. "Equations equation 4" and "Equation equation 12", etc.

  8. Theorem 1's example is built from machinery that is never introduced in the main body. The word "Bernoulli" appears only in the appendix, while Section 5.2 already interprets the example in the main body. Should the example, or at least its setup, be moved to Section 5.2?

  9. This approach is computationally heavy. An SCM, Markov pillows, abduction, a tilt kernel, a downstream resampling law and Monte Carlo are stacked on each other. Given the few experiments, it is difficult to tell what this buys over a simpler method. What would a simpler method get wrong?

4 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer a9zM: 1/2

We sincerely thank the reviewer for their time and careful assessment of our work, and for raising these insightful questions. We address their main concerns below.

Notation and diagnostic query (W1, Q1, Q7)

Q1-Q7. In the conditioning arguments of Eqs. (8) and (11), hh and y⋆y^\star abbreviate H=hH=h and Y⋆=y⋆Y^\star=y^\star. We now introduce this convention earlier. The star indexes the alternative/counterfactual world: Y⋆Y^\star is a variable, y⋆y^\star a value, and Y⋆=y⋆Y^\star=y^\star the target event. We now use P′P^\prime and Γ′\Gamma^\prime instead of dPdP and dΓd\Gamma for densities/mass functions. Calligraphic M\mathcal{M} represents an SCM, and MjM_j a Markov pillow. We corrected the duplicated equation references and added a notation table in the appendix.

We now introduce the diagnostic query early on, together with its inputs and output:

Given the factual case history h=(z,a,y)h=(z,a,y), which revisions to AA could help explain how the outcome might have differed, and how likely is each revision among counterfactual worlds attaining Y⋆=y⋆Y^\star=y^\star for the same case?

  • Inputs. The factual case, target outcome, causal graph (with its topological ordering), the scope/set of directly revisable variables, and their revision law. Downstream propagation uses an SCM under u-FAC and specified resampling laws under r-FAC.

  • Output. The conditional law of A⋆A^\star given H=hH=h and Y⋆=y⋆Y^\star=y^\star.

What makes a counterfactual explanation "correct"? (Q5)

The reviewer raises a very interesting question: what would count as evidence that an explanation assigns the right weight to a revision? Equation (11) tells us how to compute that weight once the revision law and downstream mechanism are specified. Assessing its correctness depends on whether the SCM is known or not.

When the SCM is known:

For u-FAC, a known SCM plus revision law provide a same-unit ground truth. Given H=hH=h, Pearl’s Abduction–Action–Prediction procedure infers the distribution of disturbances compatible with the factual history, applies a revision, and predicts the outcome while retaining those disturbances. Across draws from the revision law, the "correct" benchmark is each revision’s frequency among counterfactual worlds attaining Y⋆=y⋆Y^\star=y^\star.

For r-FAC, Proposition 5 gives correctness a statistical/population-level interpretation. Under its assumptions, the observed distribution and revision laws determine each revision’s frequency among target-attaining worlds.

We have added new synthetic example illustrating the discrepancy in recovering both these frequencies and the disagreement versus u-FAC.

When the SCM is unknown:

The same-unit weights may be only partially identified. Holding the revision law fixed, sharp bounds can still be computed to give the lowest and highest possible weights across compatible SCMs. Weights producing estimated diagnostic query outside such bounds are incompatible with every such SCM. Yet, non-informative/wide bounds are common in counterfactual inference, and may leave several explanations plausible.

We report such observational-bound analysis in the revised Adult application, along other proxy measures of appropriateness.

Assumptions behind Lemma 1 and Propositions 3–5 (W2, Q2, Q3)

Q2. Where does the requirement on ZZ come from? It comes from the choice of factual information. The reviewer is right that the two requirements concern different inputs Let A\mathcal{A} be the scope of explanatory variables to revise, then:

  • Z⊇(⋃rMAr)∖AZ\supseteq(\bigcup_r M_{A_r}) \setminus \mathcal{A} ensures that (Z,A)(Z,A) supplies every factual pillow argument. If a kernel reads an omitted factual variable, observing YY may reveal information about that variable and change the revision law.

  • Condition (9), Mj∩D⊆AM_j\cap D\subseteq \mathcal{A} for each scoped node jj, ensures that every starred pillow argument is an earlier scoped revision. On A1→L→A2A_1\to L\to A_2, for instance, scoping only A1,A2A_1,A_2 leaves the second kernel dependent on L⋆L^\star, which must be generated or integrated out.

We now state both conditions separately and more clearly in Propositions 3–5. Together they yield the ordered product in Eq. (10) and the independence in Lemma 1.

Q3. Is YY outside the scope, and which independence is needed? Yes. The intended downstream query takes YY as a descendant of the scope, but not part of the scope, which we state explicitly. If YY is scoped, fixing the scope fixes Y⋆Y^\star, so the success score is the indicator that its outcome coordinate equals the target.

The revised proof invokes Lemma 1 directly. It gives A⋆⊥(V,U)∣(Z,A)A^\star\perp(V,U)\mid(Z,A). Since YY is a component of VV, decomposition and weak union give A⋆⊥U∣(Z,A,Y)A^\star\perp U\mid(Z,A,Y). Thus P(U∣A⋆,H)=P(U∣H)P(U\mid A^\star,H)=P(U\mid H) almost surely, precisely the equality used in Eq. (12). We removed the weaker statement, which is not needed under Lemma 1.

AuthorsOpenReview ↗

Rebuttal to Reviewer a9zM: 2/2

What a simpler method would miss (W2, Q8, Q9)

Q8. As recommended, we moved the Bernoulli setup defining the two counterexample SCMs and its numerical conclusion to Section 5.2.

Q9. Consider a simple baseline: weight each proposed revision by its arm success rate. Both SCMs in Theorem 1 randomize AA uniformly over [0,1,2][0,1,2], with success rates (1/2,1/2,1)(1/2,1/2,1). For a person observed at (A,Y)=(1,0)(A,Y)=(1,0), the baseline therefore assigns weights (1/3,2/3)(1/3,2/3) to equally proposed revisions 0 and 2. What can it miss? The factual outcome tells us that this person’s response under exposure 1 was t(1)=0t(1)=0. In model 1, t(0)t(0) is independent of t(1)t(1), so revision 0 still succeeds with probability 1/21/2. In model 2, t(0)=1−t(1)t(0)=1-t(1), so revision 0 succeeds with certainty. Revision 2 always succeeds. Thus the baseline correctly favors revision 2 in model 1 but gives the same ranking in model 2, where the two revisions deserve equal weight.

The SCMs have identical observed laws, so every method using only that law must give the same answer in both, even with unlimited data. For a same-unit explanation, u-FAC requires the SCM to specify how a person's responses across exposures are related. Abduction then uses the factual outcome to update each revision's success probability. For a distributional explanation, r-FAC lets the analyst specify downstream resampling directly. Here, two arm proportions give the exact distributional r-FAC ranking.

Identification vs estimation (W3, Q4, Q5)

Prop. 5 makes an identification claim. Given the observed inputs, the r-FAC target is identifiable when downstream kernels are functionals of the observed law, and the required conditionals and target event fulfill positivity. We removed "estimable" because we give no conditions for root-nn inference. The estimation rate depends on the kernel estimators.

For a new synthetic benchmark, we used the two-SCM counterexample in Appendix A.6. For the factual case (A,Y)=(1,0)(A,Y)=(1,0), the appendix considers equally likely revisions to 0 and 2 and derives the exact weight on revision 0 among worlds attaining Y⋆=1Y^\star=1. We used these analytical values for two new numerical checks:

  • Same-unit benchmark. With 1M draws per SCM, the frequencies were 0.3340.334 in SCM 1 (exact u-FAC weight 1/31/3) and 0.5000.500 in SCM 2 (exact weight 1/21/2).

  • Estimation from data. Both SCMs yield the same distributional r-FAC weight of 1/31/3. Its plug-in estimate is p^0/(p^0+p^2)\hat p_0/(\hat p_0+\hat p_2), where p^a\hat p_a is arm aa's observed success proportion. Across 10k datasets per sample size, mean absolute errors were 0.0170.017, 0.0060.006, and 0.0020.002 at n=300;3k;30kn=300; 3k; 30k

In SCM 2, the exact r-FAC and u-FAC weights differ by 1/61/6 even when the observed law is known. This gap measures the effect of downstream propagation.

Adult benchmark and kernel matching. Target frequency measures how often the fitted model gives the selected person a favorable decision. Among successful revisions, mean education change measures the average shift in education, while feature distance measures overall departure from the factual case. Since Adult's SCM is unknown, we check whether proposed same-unit weights fall within the observational bounds for the query.

We match the revision kernel and final classifier across u-FAC and r-FAC to isolate downstream propagation. With education alone in scope, every successful u-FAC draw increased education, while 34.5% of successful r-FAC draws retained factual education at βd=0.5\beta_d=0.5. Backtracking uses the same scope but a different kernel by construction, so its comparison with u-FAC cannot isolate propagation.

Choosing the scope (W3, Q6)

The scope specifies which variables the auditor revises directly. Unscoped descendants may still respond. Criteria to choose it include:

  • Regret. A regret question may restrict the scope to choices/decisions the person could have made differently at the time.

  • Policy mechanism. The scope may contain the variables a policy manipulates directly in the future.

  • Preferences. Certain feasible revisions may be preferred by scientific curiosity.

  • Law or mandate. An auditor may reconsider only variables covered by a mandate, and nothing protected by law.

In Adult, hours may respond downstream under EE and may also be revised directly under (E,J)(E,J). Allowing direct revision of hours lowers this person's mean education increase among successful u-FAC draws from 2.23 to 1.13 education-levels. Before Figure 2, we will explain this choice and identify the comparison as a scope sensitivity analysis.


The reviewer’s constructive comments strengthened the paper. We sincerely appreciate the time and care devoted to our work. We hope this rebuttal addresses the main concerns.

Since clearer terminology and fuller explanations address the issues raised, we would respectfully appreciate an updated score. We are glad to clarify or discuss any remaining concerns.

Reviewer_a9zMOpenReview ↗

Thanks for the response

Thank you for the response. I think it clarifies some of my questions.

There are still quite some concerns from my end.

  1. The presentation in the paper is incredibly hard to follow. It is not only about the writing flow, but also numerous minor issues. The notation choice can be far better improved. Although authors promise to do revisions, I still lack confidence that the necessary changes can be made in time. Also, the promise here is pretty vague.

  2. The fact of lacking strong experiments seems not possible to be fixed in time. A new synthetic data is promised but still, it is synthetic (yet I'm not sure what it is exactly from the response).

  3. A few promises are made with vague details. A few examples are also vague, e.g.

Same-unit benchmark. With 1M draws per SCM, the frequencies were in SCM 1 (exact u-FAC weight) and in SCM 2 (exact weight).

What is the setup of the SCM? What data is it for? To name but a few.

  1. Also I was asking what defines a good counterfactual, not what defines a correct counterfactual. There may never be a answer to the correct counterfactual question.

The response still cannot fully help me understand the real impact of the work. And the paper presentation is a major issue, in particular for publishing in a top venue. My current score is the maximum I can suggest.

AuthorsOpenReview ↗

Followup Rebuttal to Reviewer a9zM

We thank the reviewer for their remarked engagement. We are happy to know our rebuttal has already clarified some of the questions. We answer the remaining comments.


1. Exposition

We understand the paper is dense. We have submitted an edited version where examples, cases and more pedagogical expositions have been put in the foreground. Some of the topics covered, like cross-world counterfactual inference and natural-value-dependent distributions, are niche, technically heavy, and require a step-by-step introduction.

We will further simplify the language and presentation, on top of the edits already made.

2. Experimental evidence and the synthetic benchmark

To keep the manuscript cohesive, we develop one example throughout. The same two SCMs M1\mathcal{M}_1 and M2\mathcal{M}_2 establish non-identification in Theorem 1, illustrate "What an arm success rate misses" in Section 5.2, and generate the synthetic data. The experiment separates estimation error from the discrepancy between r-FAC and same-unit u-FAC targets.

Both models have A→YA\to Y, with AA randomized uniformly over {0,1,2}\{0,1,2\} independently of the latent response vector tt, and Y=t(A)Y=t(A). Both set t(2)=1t(2)=1 and t(1)∼Bernoulli(1/2)t(1)\sim\mathrm{Bernoulli}(1/2). In M1\mathcal{M}_1, t(0)t(0) is an independent fair Bernoulli draw. In M2\mathcal{M}_2, t(0)=1−t(1)t(0)=1-t(1). They therefore induce the same observed law, with arm success rates (1/2,1/2,1)(1/2,1/2,1).

For the factual case (A,Y)=(1,0)(A,Y)=(1,0), we propose actions 0 and 2 equally often. Action 2 always succeeds, while action 0 succeeds with probability 1/21/2 in M1\mathcal{M}_1 and certainty in M2\mathcal{M}_2. Its share among successful alternatives is therefore 1/31/3 versus 1/21/2. Sampling one million drawn observations/samples per each one of these SCMs, the frequencies were 0.334 and 0.500..

Weighting proposals by observed arm success rates gives the distributional r-FAC weight 1/31/3 in both models. Increasing the sample size improves estimation of this quantity, while its discrepancy from the u-FAC target in M2\mathcal{M}_2 remains 1/61/6. Knowing the generating mechanisms lets us measure this gap exactly. The setup is synthetic in two ways: (1) the data are generated from models, and (2) the models are fully known; so counterfactual queries can be computed to analyze theoretical discrepancies. The setup and simulation protocol appear in Section 5.2 and Appendix A.7.

We will make the shared role of the two SCMs explicit throughout the manuscript, and clarify the purpose and limitations of both experiments.

3. What makes a counterfactual explanation "good"?

The reviewer is right that our previous answer focused on correctness, as one theoretical criterion for "good". In our opinion, a good counterfactual explanation must serve the intended user and purpose. That definition may include many points beyond correctness. For recourse, that includes, for instance:

  • achieving the desired outcome under a credible model,
  • respecting feasible and permitted changes,
  • keeping costs acceptable to the individual, and
  • remaining understandable and sufficiently stable under plausible modeling choices or misspecification.

These criteria can conflict, so the intended use and the individual’s constraints matter.

FAC makes several of these choices explicit. The scope specifies which variables may change. The revision law can encode feasibility, costs, and preferences. The downstream law specifies the reassessment or same-unit interpretation under which success is evaluated. Thus, we recommend running sensitivity analyses comparing explanations across plausible choices, to study how sensitive the recommendation is to those assumptions.

We will revise the introduction and discussion to distinguish quality for the intended use from correctness. The benchmark evaluates the latter. We will make clear that establishing the former also requires domain knowledge, user constraints, and evidence about whether the proposed changes can be carried out. We will also present a protocol for sensitivity analysis.


Even if the reviewer remains unconvinced and maintains the given score, we would still like to thank you again for the thoughtful discussion, recommendations, and candid feedback, which have greatly improved our paper. Thank you so much!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.