acceptodds
Under review as a conference paper at ICLR 2027

Causally Fair Generation with Large Language Models

Abstract

Large language models (LLMs) are increasingly used to generate, complete, and transform information in settings where their outputs can shape consequential decisions, raising concerns about their impact on demographic disparities. In this context, causal inference provides a principled basis for assessing fairness, because it attributes observed disparities to the mechanisms that generated them, which a purely statistical approach cannot do even with infinite data. In LLM generation, a query may request several causally related variables, each of which is both an outcome of interest and a possible cause of other outputs, and the information supplied in the prompt need not follow a topological or a temporal order. This calls for methods that can analyze and selectively remove disparities from such a flexible generation process. In this paper we introduce Causally Fair Generation with LLMs (CFG, for short). CFG extracts relevant concepts, grounds generation in a reference population and causal diagram, and removes user-selected causal effects. CFG also allows pathways deemed justifiable for the task’s utility to be retained, which is known in legal literature as business necessity. Further, we provide formal guarantees for our method when eliminating all discriminatory causal effects in the adapted population model, under appropriate causal assumptions. We evaluate CFG with four LLMs in three real-world settings based on population data and on a synthetic dataset with a known causal ground truth.

Primary area:
causal reasoning
Keywords:
causal fairness, large language models, structured generation, algorithmic fairness, causal inference, path-specific effects, bias mitigation, controlled generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7XwwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Causally Fair Generation with Large Language Models (CFG), a framework for controlling demographic disparities in LLM-generated content using causal fairness principles. The approach targets scenarios in which an analyst uses an LLM to generate synthetic attributes of individuals based on a partially specified profile. For instance, an analyst may supply age and gender and request the generation of education, employment status, and salary. Such applications may be useful for generating synthetic records for testing or evaluating public-sector systems while limiting the use of sensitive real-world information.

The main contribution is a framework that allows users to selectively remove undesirable causal effects of sensitive attributes while retaining effects considered justifiable under the concept of business necessity (BN). The approach relies on the decomposition of Total Variation (TV) into Direct Effects (DE), Indirect Effects (IE), and Spurious Effects (SE). The user specifies which components should be removed and which causal pathways may be retained. CFG supports both topological queries, where the supplied attributes causally precede the generated ones, and non-topological queries, where the model may be asked to generate an upstream attribute, including a sensitive attribute, from its descendants. The proposed framework combines population grounding, causal-diagram acquisition, causal adaptation, and prompt construction. Theorem 1 provides the main theoretical guarantee, establishing conditional distributional invariance under a policy that removes all directed causal effects of the protected attribute. The experimental evaluation involves four LLMs, three real-world datasets, and one synthetic dataset with known causal ground truth. The results indicate that CFG can reproduce reference-population causal effects and substantially reduce selected causal effects in generated outputs, for both topological and non-topological queries.

Critical strengths
  1. The paper is generally well organized, and its objectives and contributions are clearly articulated. The distinction between fair prediction and fair generation is useful, and the illustrative examples, particularly the applicant-profile scenario, make the proposed framework easier to understand.

  2. The paper addresses the problem of controlling discriminatory effects in LLM-generated attributes, rather than merely detecting statistical disparities. The ability to distinguish causal pathways and retain those considered justifiable through business-necessity constraints offers a more nuanced fairness mechanism than unconditional statistical parity.

  3. The framework may be valuable in situations where analysts need to generate realistic synthetic profiles from limited available information, particularly when real individual-level data cannot be fully disclosed due to privacy or safety considerations.

  4. The paper provides a theoretical guarantee for full causal-effect removal and evaluates the proposed approach across multiple LLMs and datasets. The results show substantial reductions in targeted effects and demonstrate that the framework can accommodate different query directions and selected business-necessity exceptions.

Critical weaknesses

My main concern is the relatively narrow application scenario motivating the proposed framework. CFG primarily targets situations in which an analyst generates synthetic attributes of human subjects based on other known attributes. Although the applicant-profile example is intuitive, it remains unclear how common such applications are in practice and to what extent they justify a dedicated framework for causally fair LLM generation. More generally, CFG operates on identifiable, structured attributes extracted from natural-language prompts and requires a suitable reference population. This restricts its applicability to domains where the variables can be clearly defined, representative population data are available, and a meaningful causal model can be constructed.

  1. The proposed approach relies heavily on the decomposition of total variation into direct, indirect, and spurious effects: $ x\text{-}DE_{x_0,x_1}(y\mid x_0)$, x-IEx1,x0(y∣x0)x\text{-}IE_{x_1,x_0}(y\mid x_0), x-SEx1,x0(y).x\text{-}SE_{x_1,x_0}(y). Although this decomposition is established in the causal fairness literature, its practical use for identifying and selectively removing individual effects depends on important assumptions. In particular, the decomposition involves nested counterfactual quantities such as (Yx1,Wx0Y_{x_1,W_{x_0}}). These are cross-world counterfactuals, whose identification generally requires assumptions beyond ordinary observational associations. The paper adopts the Standard Fairness Model and additionally assumes a Markovian structure among generated attributes, including independence of the corresponding exogenous variables. Such assumptions rule out certain forms of latent confounding and may be restrictive in applications involving demographic, educational, and socioeconomic variables.

Moreover, the selective identification of path-specific effects is not always possible, even when the causal graph is known. The paper introduces a backward-closure condition for business-necessity sets and discusses stronger quantile-preservation assumptions in Appendix A.2. However, these requirements and their practical implications deserve substantially more attention in the main text. In particular, quantile preservation specifies a particular coupling between factual and counterfactual outcomes that cannot generally be verified from observational data.

A dedicated discussion distinguishing the assumptions required for the decomposition, causal identification, selective effect removal, and practical estimation would help clarify the scope of the proposed framework.

  1. The approach assumes access to a credible causal diagram describing the relationships between sensitive attributes, background covariates, and generated outcomes. However, constructing reliable causal graphs remains one of the fundamental challenges of causal inference. CFG obtains causal diagrams either from a curated repository or through an LLM-based agent. Although the latter may facilitate practical deployment, an LLM-generated diagram does not necessarily represent the true underlying causal structure. Missing edges, incorrect causal directions, or unobserved confounding may substantially change the identification and interpretation of direct, indirect, and spurious effects. Although the authors acknowleged this limitation in the very end of the paper, but they do not sufficiently investigate the sensitivity of its results to causal-graph misspecification. An experimental analysis introducing controlled errors into the graph would provide valuable information about the robustness of the approach.

Furthermore, Theorem 1 establishes guarantees specifically for the full-removal policy (\rho^\star), under which all directed effects of (X) on (Y) are eliminated through recursive substitution: firw(x0,Z,Y<iρ⋆,Ui)f_i^{rw}(x_0,Z,Y_{<i}^{\rho^\star},U_i) This implies, under the assumed causal model, Prw(y∣x0,z)P_{\mathrm{rw}}(y\mid x_0,z) However, this guarantee does not directly extend to the more general and arguably more interesting setting involving selective effect removal and business-necessity exceptions. These settings require additional identification assumptions and are not covered by an equivalent general theoretical guarantee.

The paper should more clearly distinguish the theoretical guarantees established for full effect removal from the assumptions and empirical evidence supporting partial effect removal.

Questions for authors

The framework relies on the decomposition of total variation into direct, indirect, and spurious effects, and on identifying selected causal pathways for removal. Could the authors clarify the minimal assumptions required for the identification of these components and for the correctness of the proposed adaptation procedure? In particular, how would unobserved confounding, violations of the Markovian assumption, or failure of the quantile-preservation assumption affect the results? It would also be useful to clarify which guarantees remain valid when business-necessity exceptions are introduced.

Could the authors provide additional concrete application scenarios motivating LLM-based causal fairness beyond the generation of synthetic individual profiles? In particular, what is the added value of using an LLM rather than directly generating structured synthetic records from the adapted population distribution? Additionally, how robust is CFG to errors in the causal graph, especially when the graph is proposed by an LLM rather than supplied by domain experts?

4 comments

We thank the reviewer for their time and effort in reviewing our paper. Based on the comments, we introduced important changes to the manuscript, which are highlighted in blue color in the revised pdf. We answer each point below.

(W1: Narrow application scenario) Thank you for this thoughtful comment; this is really worth thinking about! Synthetic profiles, which are highlighted in this paper, are not the only application scenario. We are aware of other, quite related observations in the broader literature:

  • (i) Educational stories. [1] analyzes 500,000 stories generated by five LLMs about learning, work, and relationships. Minority characters are disproportionately portrayed as struggling students, so the protected attribute shapes several co-generated attributes of the same character.

  • (ii) Clinical vignettes. [2] shows that GPT-4, when generating vignettes for medical education, exaggerates known demographic prevalence differences in 89% of diseases, relative to US prevalence estimates. This closely mirrors our setting, with population data serving as the reference.

  • (iii) Reference letters. [3] finds significant gender biases in LLM-generated recommendation letters, including when the model is given background information such as a biography, which acts as a natural mediator.

  • (iv) Text-to-image generation. [4] shows that occupation prompts amplify racial and gender disparities. Relatedly, Google had a temporary suspension of person generation in Gemini [5], after indiscriminate diversity adjustments produced historically inaccurate images. This illustrates why removing all dependence on XX is undesirable, and why retaining permissible pathways (business necessity) matters (put differently, this very example is a strong argument for the causal approach).

We believe that the proposed methodology, with further work, could address many of the above described settings (which do not take a causal approach).

Requirements for fairness: we agree, however, that CFG requires well-defined variables, a reference population, and a causal diagram. In our view, though, these ingredients are needed for any serious attempt at causal fairness: without them, it is unclear which effects should be removed, and relative to which population. In other words, fair generation without them is, we believe, very hard to do in a principled way.

[1] Shieh, Evan, et al. "Laissez-faire harms: Algorithmic biases in generative language models." arXiv preprint arXiv:2404.07475 (2024).

[2] Zack, T., et al. "Assessing the potential of GPT‑4 to perpetuate racial and gender biases in health care: a model evaluation study. Lancet Digit Health. 2024; 6 (1): e12–22."

[3] Wan, Yixin, et al. "“Kelly is a warm person, Joseph is a role model”: Gender biases in LLM-generated reference letters." Findings of the Association for Computational Linguistics: EMNLP 2023. 2023.

[4] Bianchi, Federico, et al. "Easily accessible text-to-image generation amplifies demographic stereotypes at large scale." Proceedings of the 2023 ACM conference on fairness, accountability, and transparency. 2023.

[5] Gemini image generation got it wrong. We’ll do better. https://blog.google/products-and-platforms/products/gemini/gemini-image-generation-issue/


(W2: Causal assumptions) Thank you, this is a very important comment. We agree that the assumptions deserve a dedicated discussion, which we now provide in the new Appendix A.3 of the revised pdf. Assumptions fall into three classes, in the increasing order of strength:

  • (i) By construction: statements that hold in any SCM and require no substantive assumptions.
  • (ii) Causal assumptions falsifiable from interventional data: absence of unobserved confounding, which cannot be tested from observational data, but could in principle be falsified by experiments.
  • (iii) Individual-level coupling: assumptions on the joint distribution of factual and counterfactual outcomes of the same unit, which are not falsifiable from any data.

The steps of CFG require different levels of assumptions, and here is the detailed description:

  • Decomposition (class (i)): the TV decomposition is obtained by adding and subtracting counterfactual terms and holds in any SCM; the SFM topology can always be constructed by setting Y=de(X)\mathbf Y = \mathrm{de}(X), and Z=V∖{Y,X}\mathbf Z = \mathbf V \setminus \{\mathbf Y, X\}. In other words, the decomposition can always be defined, but its identification may be more difficult.

  • Identification (class (ii)): the SFM assumes no unobserved confounding between XX and Y\mathbf Y, between Z\mathbf Z and Y\mathbf Y, or within Y\mathbf Y. Here, we also make an important note regarding cross-world quantities: one can also consider the interventional analogue Yx1,W~x0Y_{x_1, \widetilde{\mathbf W}_{x_0}}, with W~x0\widetilde{\mathbf W}_{x_0} drawn independently from P(Wx0∣Z)P(\mathbf W_{x_0} \mid \mathbf Z). In the literature, these are sometimes called interventional effects. Crucially, these effects have exactly the same identification expression (i.e., adopting these effects changes nothing about our approach).

  • Full removal (class (ii)): in full removal, the target distribution P(Yx0=y∣z)P(\mathbf Y_{x_0} = \mathbf y \mid \mathbf z) is interventional, and Thm. 1 requires only Yx0⊥X∣Z\mathbf Y_{x_0} \perp X \mid \mathbf Z and Prw(x0∣z)>0P_{\mathrm{rw}}(x_0 \mid \mathbf z) > 0. Strictly speaking, Markovianity within Y\mathbf Y is not needed for full removal.

  • Partial removal with backward-closed BN sets (class (ii)): for partial removal under backward closure, the target is interventional, under do(X=x0,Y≤i=y≤i)do(X = x_0, \mathbf Y_{\leq i} = \mathbf y_{\leq i}) (see new Thm. 2 in Appendix A.2). This setting additionally requires U≤i⊥ ⁣ ⁣ ⁣⊥U>i\mathbf U_{\leq i} \perp\!\!\!\perp \mathbf U_{>i} and overlap Prw(x0∣z,y≤i)>0P_{\mathrm{rw}}(x_0 \mid \mathbf z, \mathbf y_{\leq i}) > 0.

  • General BN sets (class (iii)): when an attribute must take both its factual and its counterfactual value within the same unit, we rely on the quantile preservation assumption. This assumption is not falsifiable; in the fairness context, it can be viewed as a normative specification, namely that an individual retains their relative rank within their group, during data adaptation.

  • Estimation: finally, for estimation, overlap Prw(x∣z,y)>0P_{\mathrm{rw}}(x \mid \mathbf z, \mathbf y) > 0 is required. This can be checked from observational data.


(W3: Credibility of the causal diagram) We agree that CFG requires a credible causal diagram. However, as argued in (W1), we view this requirement as fundamental to any serious (causal) fairness claim.

Furthermore, we fully agree that the LLM-proposed diagram need not coincide with the true one, and thank you for suggesting a sensitivity analysis. We have now included it in the revised paper (Appendix F.1). We added a sensitivity analysis on Census-Synth by omitting subsets of three edges from the assumed causal diagram. The mean absolute remaining effect increases from 1.65 to 5.08 percentage points as the number of omitted edges increases from zero to three, illustrating the sensitivity of effect removal to missing edges.


(W4: Guarantees for partial removal) Thank you, this is a great point. We have now extended Theorem 1 to partial removal (see Thm. 2 in Appendix A.2 of the revised paper). To avoid duplication, we kindly ask you to see our response (W2) to Reviewer bQRj for details.


(Q1: Minimal assumptions and their violations) Please see (W2).

(Q2: Application scenarios and the role of the LLM) Please see (W1).

(Q3: Robustness to errors in the causal graph) Please see (W3).

We thank the reviewer again for their comments. As the discussion period is approaching its end (Nov 18 AoE), we would be glad to hear back from the reviewer, to understand if our responses addressed their concerns. If any concerns remain unresolved, we would be happy to respond before the deadline.

I appreciate the additional real-world use cases, which help better motivate the proposed framework, although I still believe its practical applicability remains relatively limited. My main concern, however, remains the reliance on the decomposition of total variation into direct, indirect, and spurious causal effects, which is central to the proposed approach. The identification and selective removal of these effects depend on strong causal assumptions that may be difficult to justify in practice, particularly in the presence of unobserved confounding or uncertain causal structures. I maintain my original score (Weak Rejection).

Reviewer_bQRjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies causal fairness in LLM generation when requested attributes are causally related and prompts may supply information about later variables. It proposes Causally Fair Generation with LLMs (CFG), which grounds generation in a reference population and an explicit causal diagram. CFG extracts concepts from the query, uses fairadapt to modify the population distribution according to selected effect-removal policies, and samples a joint record for the LLM to express in text. The formulation allows direct, indirect, and spurious effects to be targeted while retaining pathways designated as permissible through business-necessity exceptions. It also supports queries that condition on descendants rather than following the causal order. The paper provides formal guarantees for full directed-effect removal under stated assumptions. Experiments with four LLMs on three survey-based populations and a synthetic causal model assess effect recovery, selective removal, and preservation of sampled attributes through text generation, reporting reduced targeted disparities and high sampled-annotated agreement.

Critical strengths
  1. The paper studies a meaningful problem on the causal fariness in LLM generataion. The paper distinguishes adaptation of the joint distribution from conditioning that distribution for a query, which an intereting point. Theorem 1 and Appendix A.3 also explain when user-supplied descendants prevent recovery of the fully adapted joint. This is useful for profile completion, where the supplied context can itself contain the disparities being removed.

  2. In general, the empirical results support that CFG gives explicit control over the reference population and fairness policy. The method exposes the data, causal assumptions, removal choices, and pathway exceptions before sampling a joint record, which makes its intended behavior inspectable. The experimental results, especially sampled-annotated agreement of in Appendix C.2 supports the feasibility of preserving those controlled attributes through different LLMs.

Critical weaknesses

The theorem does not specify the conditions needed for all claimed input cases. The argument in Appendix A.1 leverages the independence of the unobserved noise variables in Equations (4) and (5). However, Equation (11) uses the outcome distribution in the reference group for every zz, without requiring that this group occurs at each such zz. Furthermore, positive probabilities for the unobserved variables do not guarantee this overlap. As the division in Equation (16) also requires a nonzero denominator, the proof therefore does not establish Equation (12) for contexts with zero probability or density in the adapted population unless the method explicitly defines the conditional distribution for these inputs. Moreover, while Theorem 1 covers full directed-effect removal, Table 5's aggregate IE under business necessity does not separate effects along allowed pathways from effects along pathways that should have been removed. These limitations affect both the claimed range of queries and the evidence for selective removal.

Questions for authors
  1. Does Equation (11) require Prw(X=x0∣Z=z)>0P_{rw}(X = x_0 \mid Z = z) > 0 for every allowed z? If this can be zero, how is the reference-group conditional distribution defined and identified?

  2. If Equation (16) has a zero denominator, how is Equation (12) defined? How does CFG use Prw(x∣z)P_{rw}(x | z) when the supplied context has zero probability or density under the adapted distribution?

  3. Under what assumptions does fairadapt produce the joint distribution in Equation (15), and how does its quantile transformation handle ties for categorical attributes?

2 comments

We thank the reviewer for their time and effort in reviewing our paper. Based on the comments, we introduced important changes to the manuscript, which are highlighted in blue color in the revised pdf. We answer each point below.

(W1: Overlap) Thank you, this is an excellent catch, and you are definitely right that an overlap condition is needed. We now state it explicitly. Here are the key points:

  • For Theorem 1, we need Prw(x0∣z)>0P_{\mathrm{rw}}(x_0 \mid \mathbf z) > 0 for all z\mathbf z in the support. This ensures that Eq. (11) is well defined. The denominator in Eq. (16) then equals Prw(y∣x0,z)P_{\mathrm{rw}}(\mathbf y \mid x_0, \mathbf z), which is nonzero exactly on the support of the adapted distribution; outside it, (z,y)(\mathbf z, \mathbf y) has probability zero, and we define the conditional as Prw(x∣z)P_{\mathrm{rw}}(x \mid \mathbf z).
  • For estimation, however, a stronger condition is needed: fairadapt's conditional quantile models and the subsequent effect estimators both require Prw(x∣z,y)>ϵ>0P_{\mathrm{rw}}(x \mid \mathbf z, \mathbf y) > \epsilon > 0.

Therefore, for the Theorem, we could get away with a slightly weaker version of overlap; but this would ignore the issues related to estimation. Therefore, we use the stronger condition in the throughout the paper (please see revised pdf).


(W2: Partial removal) Thank you for raising this. Prompted by your comment, we extended Theorem 1 to partial removal under business necessity (new Appendix A.2, with proof). Let B=Y≤i\mathbf B = \mathbf Y_{\leq i} be a backward-closed BN set, and let ρi⋆\rho^\star_i remove all directed effects of XX on Y\mathbf Y except those along pathways through B\mathbf B. Assuming the SFM and Prw(x0∣z,y)>0P_{\mathrm{rw}}(x_0 \mid \mathbf z, \mathbf y) > 0, we have:

\begin{align} P_{\mathrm{adapt}}^{\rho^\star_i}(\mathbf y_{\leq i} \mid x, \mathbf z) &= P_{\mathrm{rw}}(\mathbf y_{\leq i} \mid x, \mathbf z), \ P_{\mathrm{adapt}}^{\rho^\star_i}(\mathbf y_{>i} \mid x, \mathbf z, \mathbf y_{\leq i}) &= P_{\mathrm{rw}}(\mathbf y_{>i} \mid x_0, \mathbf z, \mathbf y_{\leq i}), \end{align}

and, for non-topological queries, Padaptρi⋆(x∣z,y)=Prw(x∣z,y≤i).P_{\mathrm{adapt}}^{\rho^\star_i}(x \mid \mathbf z, \mathbf y) = P_{\mathrm{rw}}(x \mid \mathbf z, \mathbf y_{\leq i}).

Interpretation: In words, BN attributes keep their real-world distribution, while the non-BN attributes follow the reference group x0x_0 given z\mathbf z and the BN attributes; using i=0i = 0 (empty BN set) recovers the setting of Thm. 1. Further insights can also be obtained from the theorem. For i=k−1i = k-1, partial removal eliminates only X→YkX \to Y_k, giving Yk⊥X∣Z,Y<kY_k \perp X \mid \mathbf Z, \mathbf Y_{<k} and hence DE =0= 0. Using the above result, Table 6 now reports the target value of each effect under the BN policy alongside its estimate, separating permitted pathways from residual effects along removed ones.


(Q1: Overlap for Eq. (11)) Yes. Please see our response to (W1).

(Q2: Zero denominator in Eq. (16)) Please see our response to (W1). Outside the support of the adapted distribution, we define the conditional as Prw(x∣z)P_{\mathrm{rw}}(x \mid \mathbf z); but to make things more straightforward, we assume a stronger version of overlap throughout.

(Q3: fairadapt assumptions and ties) fairadapt produces the joint distribution in Eq. (15) under three conditions:

  • (i) the SFM, i.e., a correctly specified causal diagram;
  • (ii) overlap, Prw(x∣z,y)>0P_{\mathrm{rw}}(x \mid \mathbf z, \mathbf y) > 0 (response (W1) has more details);
  • (iii) consistent estimation of the conditional quantiles.

Quantile preservation is not needed for Eq. (15): it determines only the unit-level coupling (individual), but it is not needed for the population-level adapted distribution. Finally, we remark that for categorical attributes, ties are broken conditionally at random, so the adapted values follow the target conditional distribution exactly.


Please let us know if any of the questions remain; and thanks again for the comments.

Thank you for the detailed response and revisions. While the explicit overlap conditions and additional theorem for partial removal help address my original concerns, I retain some concern about the accuracy of effect removal when reference-group observations are scarce for the supplied context, since the population-level theorem does not establish finite-sample estimation accuracy in these settings. The evidence for selective removal also warrants some caution. The new theorem covers backward-closed business-necessity sets, while broader policies require stronger assumptions, and the residual direct effect leaves some uncertainty about how accurately the intended removal is achieved in practice.

Reviewer_sgEROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies causal fairness when an LLM generates several related attributes, for example a synthetic person profile with education, job and salary. The user picks which direct, indirect or spurious effects of a protected attribute to remove, and can keep some paths as business necessity. CFG fits a population model on survey data and debiases it with fairadapt using a causal diagram. It then samples the attribute values and asks the LLM to write a story with them. There is a theorem for removing all directed effects, for both topological and non-topological queries. Experiments use NSDUH, BRFSS, ACS and a synthetic SCM with four open LLMs.

Critical strengths

I like how the paper sets up the problem. Treating generation as several causally related outputs, and also handling queries where the protected attribute is the thing being asked (the non-topological case), makes sense for LLMs. I also liked the business necessity option since it is close to how fairness is handled in practice. The method works with black-box models and the selected effects go close to zero for all four models when no paths are exempted.

Critical weaknesses

My main concern is that the LLM does little of the causal work. The debiasing is done by fairadapt on tabular data and the sampled values are put in the prompt. The LLM then mostly copies them (about 99% agreement in Table 1). So I think the results are mainly a property of fairadapt plus a sampler, and the similar numbers across LLMs are what you would expect from this setup. The paper also says CFG is the first fair generation method, but fair data adaptation (fairadapt itself) and causally aware fair synthetic data such as DECAF (van Breugel et al., NeurIPS 2021) already do the core step, and DECAF is not cited.

I also have some concerns about the experiments. Vanilla, CoT and RAG are only used in the recovery experiment and they do not try to remove anything, and the removal experiments in Sec. 3.2 have no baseline at all. For the three survey datasets the recovery test also looks in-sample, because the DRF that CFG samples from is fitted on the same 8192 records used for the reference effects. So a near zero error there is not very surprising. Also, only the extracted attributes are evaluated, but the free text can still carry bias (the ethics statement says this too). Smaller points: there is only one generation seed, and some baseline errors (like 1076.38 for Gemma with RAG in Table 2) come from the estimator failing under weak overlap.

Questions for authors
  1. Can you add a baseline for effect removal, for example DECAF or fairadapt output passed to a simple prompt? An out-of-sample recovery test, with reference effects computed on held-out records, would also help.
  2. In Table 2, the non-topological CFG rows for Gemma-4-31B and Qwen3-4B have exactly the same numbers. Is this a copy error, or do both models really give the same annotated data? Please also report the sampled vs annotated agreement for non-topological queries.
  3. In Table 5, after direct effect removal with B = {education}, DE is still 2.25 (topological) against 6.65 in the reference, even though DE has no BN exception. Where does this remaining effect come from?
3 comments

We thank the reviewer for their time and effort in reviewing our paper. We answer each point below.

(W1: the LLM does little causal work) Thank you for this question. We agree that CFG deliberately moves the causal computation out of the LLM. However, we don't think this makes the LLM's role entirely minor. In CFG, the LLM:

  1. recognizes whether a query is relevant to fairness, and requires processing;
  2. extracts the concepts and maps them onto population variables, separating supplied from requested ones (App. D);
  3. proposes the causal diagram when none is in the repository (explored in App. F);
  4. calls the population and adaptation tools;
  5. turns the sampled record into free-form text.

This division of labor is not entirely different from the literature on tool use and code-execution harness [1,2]. For instance, a model that recognizes a math question and calls a Python interpreter gets the same answer whichever LLM issues the call, and the reasoning is still credited to the system. CFG applies the same principle to causal fairness: the LLM recognizes the problem and calls a tool that is provably correct for it.

Various aspects of our experiments also show why this is needed. Even when the relevant conditional probabilities are given in the prompt (RAG baseline), the LLMs fail to reproduce them (e.g., final-outcome TV error of 12.42 vs. 0.63 for CFG with Llama 3-8B; see Table 3). For the same reason, we see the similar results across LLMs as the intended outcome rather than a weakness: the fairness behavior is guaranteed regardless of the generator, while each model keeps its own way of expressing the record.

New empirical test: preserving semantics. To further corroborate the above point, we now add the following experiment.

(a) building a classifier for guessing the LLM writer. We first fine-tune a ModernBERT-base to classify which of the four investigated LLMs generated a specific story. In particular, we used 8,192 stories from topological and non-topological queries. The accuracy was 99.79% on a held-out set (958/960; against random guessing 25%). Therefore, our classifier was able to distinguish almost perfectly which model generated each original story.

(b) using the classifier on CFG-adapted stories. After this, we adapted each of the stories with CFG, to achieve a causal fairness guarantee (Appendix K). We then re-tested the same classifier on adapted stories, and found the accuracy to still be extremely high (99.79%). Therefore, signatures specific to the LLM generator remain detectable after effect removal; showing that our method achieves fairness but keeps the model output recognizable.

[1] Schick, Timo, et al. "Toolformer: Language models can teach themselves to use tools." Advances in neural information processing systems 36 (2023): 68539-68551.

[2] Gao, Luyu, et al. "Pal: Program-aided language models." International conference on machine learning. PMLR, 2023.


(W2: Prior work on fair generation) Thank you for raising this point. Methods for causally fair data do exist, but their scope differs considerably from ours.

fairadapt, for instance, is a data-adaptation method: it transforms an existing tabular dataset into its counterfactually fair version, but it does not generate new records, and not from unstructured inputs such as text. In CFG, it serves as a component of the overall framework.

CFG instead addresses generation with LLMs: queries in natural language whose variables, and which of them are supplied or requested, change from query to query (including non-topological queries), with target-specific removal of effects and business-necessity exceptions, all expressed in free-form text. Our claim of novelty refers to fair generation in this sense, rather than more broadly.

Also, thank you for sharing the DECAF reference, which is indeed relevant to our work. We have now added it to the paper.

(W3, Q1: In-sample recovery; Baselines for effect removal)

The main experiments use the same reference records for population fitting and generation, as in Example 1, where the analyst wants to reproduce a reference population in generated stories while removing selected sources of disparity (e.g., the task is really in-sample in that sense).

However, we recognize the importance of your comment: one would certainly wish this guarantee to hold out-of-sample as well. We added a new set of experiments, in Appendix I, showing that the same pattern holds for held-out data: CFG matches reference causal effects better than baselines and brings the selected effects toward zero for both topological and non-topological queries.

Baseline for effect removal. Further, we have now extended Vanilla, CoT, and RAG to effect removal and evaluated them against CFG on held-out records with an empty BN set (Appendix J, ``Baseline Comparison for Effect Removal''). CFG achieves lower mean absolute effects than every baseline in all 14 reported comparisons under topological and non-topological queries.


(W4: Smaller points)

  • (i) Bias in free text. Thank you, we agree. CFG controls the attributes represented as variables, and the free text may still carry disparities along dimensions that are not measured. We now acknowledge this explicitly in the limitations: the more attributes are measured and included in the causal diagram, the more comprehensive the fairness guarantee becomes.
  • (ii) Single generation seed. We repeated all eight non-topological recovery settings for Qwen3-4B with an additional sampling and generation seed. Averaging equally over datasets, target positions, and the four effects, the mean absolute recovery error was 1.15 percentage points for the original seed and 1.09 for the additional seed, showing similar recovery accuracy across the two runs. The full set of experiments required 2,500 GPU hours on NVIDIA GH200 GPUs, and repeating generation and annotation across additional seeds would further increase this cost.
  • (iii) Estimator failure under weak overlap (Table 2 in the submission, now Table 3). For Gemma with RAG on NSDUH, almost all generated records fall in the same protected-attribute category (8,037/8,192), leaving poor overlap between groups and exposing a problem in the estimator's handling of discarded records that inflates the reported value. We added a footnote to the recovery table clarifying this limitation.

(Q2: Identical non-topological rows in Table 2 in the submission, now Table 3) This is not a copy-paste error. Both models used the same DRF samples and generated different stories, but the attributes used for effect estimation were identical, resulting in equal numbers in Table 3.

(Q3: Remaining DE after direct-effect removal with B={education}\mathbf B=\{\text{education}\}) The BN exception for education does not exempt the direct effect, whose target remains zero. A diagnostic on the adapted tabular data already shows a residual direct effect before story generation, so the remaining disparity cannot be attributed to the LLM. In this setting, the fairadapt reduces but does not eliminate the estimated direct effect. However, the confidence interval extends close to zero, and finite-sample estimation error may contribute to the observed residual.

We thank the reviewer again for their comments. As the discussion period is approaching its end (Nov 18 AoE), we would be glad to hear back from the reviewer, to understand if our responses addressed their concerns. If any concerns remain unresolved, we would be happy to respond before the deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.