- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Causally Fair Generation with Large Language Models (CFG), a framework for controlling demographic disparities in LLM-generated content using causal fairness principles. The approach targets scenarios in which an analyst uses an LLM to generate synthetic attributes of individuals based on a partially specified profile. For instance, an analyst may supply age and gender and request the generation of education, employment status, and salary. Such applications may be useful for generating synthetic records for testing or evaluating public-sector systems while limiting the use of sensitive real-world information.
The main contribution is a framework that allows users to selectively remove undesirable causal effects of sensitive attributes while retaining effects considered justifiable under the concept of business necessity (BN). The approach relies on the decomposition of Total Variation (TV) into Direct Effects (DE), Indirect Effects (IE), and Spurious Effects (SE). The user specifies which components should be removed and which causal pathways may be retained. CFG supports both topological queries, where the supplied attributes causally precede the generated ones, and non-topological queries, where the model may be asked to generate an upstream attribute, including a sensitive attribute, from its descendants. The proposed framework combines population grounding, causal-diagram acquisition, causal adaptation, and prompt construction. Theorem 1 provides the main theoretical guarantee, establishing conditional distributional invariance under a policy that removes all directed causal effects of the protected attribute. The experimental evaluation involves four LLMs, three real-world datasets, and one synthetic dataset with known causal ground truth. The results indicate that CFG can reproduce reference-population causal effects and substantially reduce selected causal effects in generated outputs, for both topological and non-topological queries.
Critical strengths
-
The paper is generally well organized, and its objectives and contributions are clearly articulated. The distinction between fair prediction and fair generation is useful, and the illustrative examples, particularly the applicant-profile scenario, make the proposed framework easier to understand.
-
The paper addresses the problem of controlling discriminatory effects in LLM-generated attributes, rather than merely detecting statistical disparities. The ability to distinguish causal pathways and retain those considered justifiable through business-necessity constraints offers a more nuanced fairness mechanism than unconditional statistical parity.
-
The framework may be valuable in situations where analysts need to generate realistic synthetic profiles from limited available information, particularly when real individual-level data cannot be fully disclosed due to privacy or safety considerations.
-
The paper provides a theoretical guarantee for full causal-effect removal and evaluates the proposed approach across multiple LLMs and datasets. The results show substantial reductions in targeted effects and demonstrate that the framework can accommodate different query directions and selected business-necessity exceptions.
Critical weaknesses
My main concern is the relatively narrow application scenario motivating the proposed framework. CFG primarily targets situations in which an analyst generates synthetic attributes of human subjects based on other known attributes. Although the applicant-profile example is intuitive, it remains unclear how common such applications are in practice and to what extent they justify a dedicated framework for causally fair LLM generation. More generally, CFG operates on identifiable, structured attributes extracted from natural-language prompts and requires a suitable reference population. This restricts its applicability to domains where the variables can be clearly defined, representative population data are available, and a meaningful causal model can be constructed.
- The proposed approach relies heavily on the decomposition of total variation into direct, indirect, and spurious effects: $ x\text{-}DE_{x_0,x_1}(y\mid x_0)$, , Although this decomposition is established in the causal fairness literature, its practical use for identifying and selectively removing individual effects depends on important assumptions. In particular, the decomposition involves nested counterfactual quantities such as (). These are cross-world counterfactuals, whose identification generally requires assumptions beyond ordinary observational associations. The paper adopts the Standard Fairness Model and additionally assumes a Markovian structure among generated attributes, including independence of the corresponding exogenous variables. Such assumptions rule out certain forms of latent confounding and may be restrictive in applications involving demographic, educational, and socioeconomic variables.
Moreover, the selective identification of path-specific effects is not always possible, even when the causal graph is known. The paper introduces a backward-closure condition for business-necessity sets and discusses stronger quantile-preservation assumptions in Appendix A.2. However, these requirements and their practical implications deserve substantially more attention in the main text. In particular, quantile preservation specifies a particular coupling between factual and counterfactual outcomes that cannot generally be verified from observational data.
A dedicated discussion distinguishing the assumptions required for the decomposition, causal identification, selective effect removal, and practical estimation would help clarify the scope of the proposed framework.
- The approach assumes access to a credible causal diagram describing the relationships between sensitive attributes, background covariates, and generated outcomes. However, constructing reliable causal graphs remains one of the fundamental challenges of causal inference. CFG obtains causal diagrams either from a curated repository or through an LLM-based agent. Although the latter may facilitate practical deployment, an LLM-generated diagram does not necessarily represent the true underlying causal structure. Missing edges, incorrect causal directions, or unobserved confounding may substantially change the identification and interpretation of direct, indirect, and spurious effects. Although the authors acknowleged this limitation in the very end of the paper, but they do not sufficiently investigate the sensitivity of its results to causal-graph misspecification. An experimental analysis introducing controlled errors into the graph would provide valuable information about the robustness of the approach.
Furthermore, Theorem 1 establishes guarantees specifically for the full-removal policy (\rho^\star), under which all directed effects of (X) on (Y) are eliminated through recursive substitution: This implies, under the assumed causal model, However, this guarantee does not directly extend to the more general and arguably more interesting setting involving selective effect removal and business-necessity exceptions. These settings require additional identification assumptions and are not covered by an equivalent general theoretical guarantee.
The paper should more clearly distinguish the theoretical guarantees established for full effect removal from the assumptions and empirical evidence supporting partial effect removal.
Questions for authors
The framework relies on the decomposition of total variation into direct, indirect, and spurious effects, and on identifying selected causal pathways for removal. Could the authors clarify the minimal assumptions required for the identification of these components and for the correctness of the proposed adaptation procedure? In particular, how would unobserved confounding, violations of the Markovian assumption, or failure of the quantile-preservation assumption affect the results? It would also be useful to clarify which guarantees remain valid when business-necessity exceptions are introduced.
Could the authors provide additional concrete application scenarios motivating LLM-based causal fairness beyond the generation of synthetic individual profiles? In particular, what is the added value of using an LLM rather than directly generating structured synthetic records from the adapted population distribution? Additionally, how robust is CFG to errors in the causal graph, especially when the graph is proposed by an LLM rather than supplied by domain experts?
4 comments
Part II
(W2: Causal assumptions) Thank you, this is a very important comment. We agree that the assumptions deserve a dedicated discussion, which we now provide in the new Appendix A.3 of the revised pdf. Assumptions fall into three classes, in the increasing order of strength:
- (i) By construction: statements that hold in any SCM and require no substantive assumptions.
- (ii) Causal assumptions falsifiable from interventional data: absence of unobserved confounding, which cannot be tested from observational data, but could in principle be falsified by experiments.
- (iii) Individual-level coupling: assumptions on the joint distribution of factual and counterfactual outcomes of the same unit, which are not falsifiable from any data.
The steps of CFG require different levels of assumptions, and here is the detailed description:
-
Decomposition (class (i)): the TV decomposition is obtained by adding and subtracting counterfactual terms and holds in any SCM; the SFM topology can always be constructed by setting , and . In other words, the decomposition can always be defined, but its identification may be more difficult.
-
Identification (class (ii)): the SFM assumes no unobserved confounding between and , between and , or within . Here, we also make an important note regarding cross-world quantities: one can also consider the interventional analogue , with drawn independently from . In the literature, these are sometimes called interventional effects. Crucially, these effects have exactly the same identification expression (i.e., adopting these effects changes nothing about our approach).
-
Full removal (class (ii)): in full removal, the target distribution is interventional, and Thm. 1 requires only and . Strictly speaking, Markovianity within is not needed for full removal.
-
Partial removal with backward-closed BN sets (class (ii)): for partial removal under backward closure, the target is interventional, under (see new Thm. 2 in Appendix A.2). This setting additionally requires and overlap .
-
General BN sets (class (iii)): when an attribute must take both its factual and its counterfactual value within the same unit, we rely on the quantile preservation assumption. This assumption is not falsifiable; in the fairness context, it can be viewed as a normative specification, namely that an individual retains their relative rank within their group, during data adaptation.
-
Estimation: finally, for estimation, overlap is required. This can be checked from observational data.
(W3: Credibility of the causal diagram) We agree that CFG requires a credible causal diagram. However, as argued in (W1), we view this requirement as fundamental to any serious (causal) fairness claim.
Furthermore, we fully agree that the LLM-proposed diagram need not coincide with the true one, and thank you for suggesting a sensitivity analysis. We have now included it in the revised paper (Appendix F.1). We added a sensitivity analysis on Census-Synth by omitting subsets of three edges from the assumed causal diagram. The mean absolute remaining effect increases from 1.65 to 5.08 percentage points as the number of omitted edges increases from zero to three, illustrating the sensitivity of effect removal to missing edges.
(W4: Guarantees for partial removal) Thank you, this is a great point. We have now extended Theorem 1 to partial removal (see Thm. 2 in Appendix A.2 of the revised paper). To avoid duplication, we kindly ask you to see our response (W2) to Reviewer bQRj for details.
(Q1: Minimal assumptions and their violations) Please see (W2).
(Q2: Application scenarios and the role of the LLM) Please see (W1).
(Q3: Robustness to errors in the causal graph) Please see (W3).
We thank the reviewer again for their comments. As the discussion period is approaching its end (Nov 18 AoE), we would be glad to hear back from the reviewer, to understand if our responses addressed their concerns. If any concerns remain unresolved, we would be happy to respond before the deadline.
I appreciate the additional real-world use cases, which help better motivate the proposed framework, although I still believe its practical applicability remains relatively limited. My main concern, however, remains the reliance on the decomposition of total variation into direct, indirect, and spurious causal effects, which is central to the proposed approach. The identification and selective removal of these effects depend on strong causal assumptions that may be difficult to justify in practice, particularly in the presence of unobserved confounding or uncertain causal structures. I maintain my original score (Weak Rejection).
Part I
We thank the reviewer for their time and effort in reviewing our paper. Based on the comments, we introduced important changes to the manuscript, which are highlighted in blue color in the revised pdf. We answer each point below.
(W1: Narrow application scenario) Thank you for this thoughtful comment; this is really worth thinking about! Synthetic profiles, which are highlighted in this paper, are not the only application scenario. We are aware of other, quite related observations in the broader literature:
(i) Educational stories. [1] analyzes 500,000 stories generated by five LLMs about learning, work, and relationships. Minority characters are disproportionately portrayed as struggling students, so the protected attribute shapes several co-generated attributes of the same character.
(ii) Clinical vignettes. [2] shows that GPT-4, when generating vignettes for medical education, exaggerates known demographic prevalence differences in 89% of diseases, relative to US prevalence estimates. This closely mirrors our setting, with population data serving as the reference.
(iii) Reference letters. [3] finds significant gender biases in LLM-generated recommendation letters, including when the model is given background information such as a biography, which acts as a natural mediator.
(iv) Text-to-image generation. [4] shows that occupation prompts amplify racial and gender disparities. Relatedly, Google had a temporary suspension of person generation in Gemini [5], after indiscriminate diversity adjustments produced historically inaccurate images. This illustrates why removing all dependence on X is undesirable, and why retaining permissible pathways (business necessity) matters (put differently, this very example is a strong argument for the causal approach).
We believe that the proposed methodology, with further work, could address many of the above described settings (which do not take a causal approach).
Requirements for fairness: we agree, however, that CFG requires well-defined variables, a reference population, and a causal diagram. In our view, though, these ingredients are needed for any serious attempt at causal fairness: without them, it is unclear which effects should be removed, and relative to which population. In other words, fair generation without them is, we believe, very hard to do in a principled way.
[1] Shieh, Evan, et al. "Laissez-faire harms: Algorithmic biases in generative language models." arXiv preprint arXiv:2404.07475 (2024).
[2] Zack, T., et al. "Assessing the potential of GPT‑4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit Health. 2024; 6 (1): e12–22."
[3] Wan, Yixin, et al. "“Kelly is a warm person, Joseph is a role model”: Gender biases in LLM-generated reference letters." Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.
[4] Bianchi, Federico, et al. "Easily accessible text-to-image generation amplifies demographic stereotypes at large scale." Proceedings of the 2023 ACM conference on fairness, accountability, and transparency. 2023.
[5] Gemini image generation got it wrong. We’ll do better. https://blog.google/products-and-platforms/products/gemini/gemini-image-generation-issue/