acceptodds
Under review as a conference paper at ICLR 2027

Where Does Bias Enter RAG? A Causal Attribution Framework

Abstract

Large language models (LLMs) are increasingly used to support or automate consequential decisions, and such systems often leverage historical data through retrieval-augmented generation (RAG). Given a query case, similar past cases are retrieved and an LLM reasons over them before producing a decision. When the final decision exhibits a disparity between demographic groups, a fundamental question arises: is the disparity inherited from the historical data, induced by the retrieval algorithm, or introduced by the LLM's reasoning over the retrieved evidence? Existing bias audits measure the disparity of the deployed system as a whole, and are therefore unable to attribute it to individual components. In this paper, we make two contributions towards answering this question. First, we develop an attribution framework that decomposes the disparity of a RAG pipeline into the contributions of (a) the retrieval database, (b) the retrieval algorithm, and (c) the LLM aggregator, by modeling the pipeline as a sequence of predictors related to the same underlying causal model. This attribution method applies to any disparity measure defined on the score itself. Second, for causal fairness measures, we derive decomposition results in which each component's contribution is further attributed to the direct, indirect, and spurious causal pathways, yielding a granular bias analysis of the pipeline. We establish identification conditions for the required quantities, and construct estimators that accommodate free-text mediators such as clinical notes or loan descriptions. We demonstrate the framework in two real-world settings, emergency department triage and microfinance loan applications.

Primary area:
causal reasoning
Keywords:
AI Fairness, Causal Fairness, Retrieval-Augment Generation, Causal Inference, Graphical Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_AsWAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper proposes a causal attribution framework for identifying where demographic disparities arise in RAG systems. It decomposes disparities across three components: historical data, retrieval, and LLM aggregation. It further attributes each contribution to direct, indirect, and spurious causal pathways, yielding a 3*3 bias decomposition. Experiments on semi-synthetic data validate the estimation accuracy, while evaluations on emergency triage (MIMIC-IV-ED) and microfinance lending (Kiva) show that retrieval and LLM aggregation can amplify, mitigate, or introduce disparities through different causal pathways, with the effects of LLM aggregation depending on the retrieved evidence.

Critical strengths
  1. The paper provides a useful framework for diagnosing demographic disparities at both the component and causal pathway levels in RAG systems. By distinguishing the contributions of historical data, retrieval, and LLM aggregation, it offers a fine-grained and actionable insights than conventional fairness evaluations that focus on aggregate disparities.

  2. The paper provides meaningful empirical validation through both semi-synthetic and real-world experiments. The semi-synthetic experiments demonstrate that the proposed estimators can recover known causal effects, while experiments on emergency triage and microfinance lending reveal how retrieval and LLM aggregation influence different causal pathways.

Critical weaknesses
  1. The proposed disparity attribution depends on the choice of the intermediate retrieval predictor. Different reasonable choices may lead to different component-level attributions, even when the final RAG outputs remain unchanged, raising concerns about the robustness and reliability of the attribution results.
Questions for authors
  1. How sensitive are the component-level attribution results to the choice of the intermediate retrieval predictor?
4 comments
AuthorsOpenReview ↗

Review Response

We thank the reviewer for their comments. We reply to the key points below.

(W1, Q1: Choice of the intermediate predictor) Thanks for the question, it is a good one. Since we see two possible interpretations of the question, we address each separately.

(a) Different retrievers, same output. One interpretation is: two retrievers A and B return different neighborhoods, NKA(q)≠NKB(q)N_K^A(q) \neq N_K^B(q), yet produce the same output Y^RAG\widehat{Y}_{\text{RAG}}, while the attributions differ.

First, this setting is unlikely to occur in practice: the LLM conditions on the retrieved neighborhood and its labels, so different neighborhoods mean different evidence, and the output should generally change. In our experiments, for instance, dense and lexical retrieval yield different ΔLLMIE\Delta^{\text{LLM}}\text{IE} for the same aggregator (Sec. 3).

Second, suppose the outputs do coincide. This would mean that between the two settings, the same output is produced by the LLM while using different evidence; in our view, this does warrant a different attribution to the LLM component. For instance, consider two cases:

  • under A, the retriever is biased (Δret=δ\Delta^{\text{ret}} = \delta), and the LLM corrects it (ΔLLM=−δ\Delta^{\text{LLM}} = -\delta);
  • under B, neither stage shifts the signature (Δret=ΔLLM=0\Delta^{\text{ret}} = \Delta^{\text{LLM}} = 0).

The final outputs stay the same, but the mechanisms differ: with a calibrated aggregator LLM (reporting the average retrieved label), setting A would exhibit the retrieval bias δ\delta, whereas B would still have Δret=0\Delta^{\text{ret}} = 0. A good attribution should distinguish the two cases.

(b) Same neighborhood, different summary. The second interpretation concerns the intermediate predictor itself.

Y^ret\widehat{Y}_{\text{ret}} summarizes the retrieved labels by their mean 1K∑i∈NKYi\frac{1}{K}\sum_{i \in N_K} Y_i. One may ask whether a different summary of the same neighborhood (e.g., majority voting) would change the attribution, with Y^RAG\widehat{Y}_{\text{RAG}} unchanged.

The answer is affirmative: a different summary function would change the attribution. However, since the bias signature depends on a score only through its conditional mean (see Sec. 2.1), the role of Y^ret\widehat{Y}_{\text{ret}} is to estimate μY(x,z,w)=E[Y∣x,z,w]\mu_Y(x, z, w) = \mathbb{E}[Y \mid x, z, w] from the retrieved cases. The mean is the canonical estimator of μY\mu_Y (specifically, it is KK-NN estimator): if the retrieved cases match the query on (x,z,w)(x, z, w), retrieval is calibrated (Def. 1) and Δret=0\Delta^{\text{ret}} = 0 (Prop. 1). A different function, such as majority vote for instance, would estimate 1μY>1/2{1}\\{\mu_Y > 1/2\\}, and not preserve the conditional mean. This is why the mean function is a principled choice for the intermediate predictor.

Please let us know if the above answers the main concern; thanks again.

Thank the authors for the clarification. However, my main concern is about the sensitivity of the attribution results to this choice. Other reasonable alternatives, such as similarity-weighted averaging, can also estimate the conditional mean. It remains unclear whether such alternatives would substantially change the component-level attributions or affect the main empirical conclusions. I would appreciate further clarification on this point.

(Follow-up: Sensitivity to the summary function) Thank you for the clarification; we now fully understand the concern. We can actually address it both theoretically and empirically.

Theoretical. Let Y^w(q)=∑i∈NK(q)ai(q)Yi\widehat{Y}_{\text{w}}(q) = \sum_{i \in N_K(q)} a_i(q) Y_i with ai(q)∝exp⁡{(λsi(q))}a_i(q) \propto \exp\{(\lambda s_i(q))\}, that is, a weighted softmax based on similarity, rather than a simple average. Let Δs(q)=max⁡i,j∈NK(q)∣si(q)−sj(q)∣\Delta s(q) = \max_{i,j \in N_K(q)} |s_i(q) - s_j(q)| be the spread of similarities within the retrieved neighborhood. Then, we can bound the weighted-mean and simple-mean difference as ∣Y^w(q)−Y^ret(q)∣≤12(eλΔs(q)−1).|\widehat{Y}_{\text{w}}(q) - \widehat{Y}_{\text{ret}}(q)| \le \tfrac{1}{2}\big(e^{\lambda \Delta s(q)} - 1\big). This is true because softmax weights satisfy e−λΔs/K≤ai≤eλΔs/Ke^{-\lambda \Delta s}/K \le a_i \le e^{\lambda \Delta s}/K, so ∑i∣ai−1/K∣≤eλΔs−1\sum_i |a_i - 1/K| \le e^{\lambda \Delta s} - 1. Since both sets of weights sum to 1, it follows that ∑i(ai−1/K)=0\sum_i (a_i - 1/K) = 0. If Yi∈[0,1]Y_i \in [0,1] (or bounded in general, with a small proof adaptation), we have ∣∑i(ai−1/K)Yi∣=∣∑i(ai−1/K)(Yi−12)∣≤12∑i∣ai−1/K∣|\sum_i (a_i - 1/K) Y_i| = |\sum_i (a_i - 1/K)(Y_i - \tfrac12)| \le \tfrac12 \sum_i |a_i - 1/K|.

Asymptotically, as the database grows, the retrieved neighbors become arbitrarily similar to the query, so Δs→0\Delta s \to 0 and the weights converge toward uniform. Since each causal effect is a linear functional of the conditional expectation, under strict overlap, for any effect CE, we can show that: ∣CE(Y^w)−CE(Y^ret)∣≤CE[eλΔs(Q)−1],\big|\text{CE}(\widehat{Y}_{\text{w}}) - \text{CE}(\widehat{Y}_{\text{ret}})\big| \le C \mathbb{E}[e^{\lambda \Delta s(Q)} - 1], which tends to 00 for a fixed λ\lambda as ∣D∣→∞|\mathcal{D}|\to \infty. Therefore, the attributions are equivalent, asymptotically. Intuitively, we are leveraging the fact that K-NN and weighted-K-NN estimators are asymptotically the same.

Empirical. We also tested this empirically. On MIMIC-ED, we replaced the mean by the similarity-weighted average on the same neighborhoods. The shape parameter λ\lambda from above was chosen so that the effective neighborhood size Keff=1/∑iai2K_{\text{eff}} = 1/\sum_i a_i^2 is about 77 or 44 (out of K=10K = 10). Specifically, we report the change cCE:=CE(Y^w)−CE(Y^ret)c_{\text{CE}} := \text{CE}(\widehat{Y}_{\text{w}}) - \text{CE}(\widehat{Y}_{\text{ret}}) (in %), which measures how much the attribution moves:

RetrieverKeffK_{\text{eff}}cTVc_{TV}cDEc_{DE}cIEc_{IE}cSEc_{SE}
Dense≈7+0.53 ± 0.40+0.76 ± 0.47+0.25 ± 0.33−0.02 ± 0.09
Dense≈4+0.82 ± 0.73+0.69 ± 0.80−0.10 ± 0.44−0.03 ± 0.16
Lexical≈7+0.27 ± 0.40−0.02 ± 0.45−0.30 ± 0.23+0.01 ± 0.09
Lexical≈4+0.44 ± 0.77+0.18 ± 0.82−0.25 ± 0.35+0.00 ± 0.17

All changes stay below one percentage point, and most are within one standard deviation of zero; they are small relative to the effects reported in Sec. 3.1, and the main conclusions remain unaffected.

We would be happy to include the result and the experiment in the paper, if the reviewer finds it helpful.

Please let us know if any concern remains! Thank you.

Thank you for the further clarification. I have updated my Soundness score to 3.

Reviewer_KfmQOpenReview ↗
Rating
3(+1)
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the fairness attribution problem in RAG decision system, mainly about when RAG outputs exhibit disparities across demographic groups, how much of this disparity stems from the historical retrieval database, how much from the retrieval algorithm, and how much from the LLM aggregator. The authors model the pipeline as a sequence of three predictors anchored on a shared Standard Fairness Model. By applying xx-DE, xx-IE and xx-SE decompositions to each predictor and taking differences between adjacent stages, they derive a bias signature across "3 components ×\times 3 causal paths". The empirical evaluation covers two scenarios. In MIMIC-IV-ED emergency triage retrieval eliminates direct effects; LLM aggregation under dense retrieval reverses indirect effects; and direct effects emerge only when race is explicitly disclosed, with their direction varying across model families. In Kiva micro-lending, the dominant direct effect of gender in the historical data is eliminated by retrieval, though the LLM re-introduces a portion of it along indirect paths. Semi-synthetic experiments demonstrate that the proposed estimator accurately attributes artificially injected bias to its corresponding stage and pathway.

Critical strengths
  1. The observation that effects depend on their mechanisms solely through the conditional mean E[S∣x,z,w]\mathbb{E}[S|x,z,w] bridges calibration with invariance of the bias signature.
  2. The experiments shows that indirect path shifts are primarily driven by the "retriever ×\times aggregator" combination rather than the aggregator alone; retrievers and race disclosure act on indirect and direct paths respectively; medical fine-tuning drastically changes output levels yet barely alters the bias signature.
Critical weaknesses
  1. All inferences are conditioned on D\mathcal{D}, and confidence intervals do not reflect the variance introduced by retriever library sampling, whereas databases change continuously in actual deployment.
  2. Twenty-four configurations ×\times four metrics were performed without multiple comparison corrections, and some conclusions are on the borderline of significance.
Questions for authors
  1. All inferences are conditioned on the database D\mathcal{D}. If D\mathcal{D} is resampled (e.g., using different estimation/retrieval splits), how large is the variance of each Δ\Delta? How does this uncertainty compare in magnitude with existing confidence intervals?
  2. Were multiple comparison corrections considered for the significance judgments of the 24 configurations? After correction, does the conclusion "the sign of direct effect under race disclosure varies by model family" still hold?
  3. How does this framework handle cases where LLM outputs are continuous probabilities?
2 comments
AuthorsOpenReview ↗

Review Response

We thank the reviewer for their time and effort in reviewing our paper. Based on the review, we made some important changes to the manuscript, which are highlighted in blue color in the revised pdf.

(W1, Q1: Conditioning on D\mathcal{D}) Thanks for the comment; however, we emphasize that conditioning on D\mathcal{D} is the intended target for the audit.

An audit evaluates the deployed system Y^RAG(⋅;D)\widehat{Y}_{\text{RAG}}(\cdot; \mathcal{D}), and the remedies it suggests (curating D\mathcal{D}, changing the retriever) act on this particular D\mathcal{D}. The average ED[CE]\mathbb{E}_{\mathcal{D}}[\text{CE}] does not describe a system that is actually deployed. This mirrors the distinction between the error of the model at hand and the average error over training sets, where the former is the quantity of interest for a deployed model [1].

That said, the reviewer's underlying point is indeed a very interesting one: databases change over time, and one would like to know how much the attribution moves when they do. We therefore re-ran the full pipeline with dense retrieval, Qwen 2.5 72B, race withheld, on 1010 random re-splits of the MIMIC-ED cohort into estimation sample and database.

Findings. Across re-splits, the variability of most components was of the same order as our influence-function standard errors. The exception was ΔretSE\Delta^{\text{ret}}\text{SE}, whose variability exceeds that expected from sampling alone, i.e., it depends on the database draw. However, the main findings of the paper remain stable even after considering re-splitting: ΔretDE\Delta^{\text{ret}}\text{DE} and ΔLLMIE\Delta^{\text{LLM}}\text{IE} are significant in all 1010 re-splits (the latter ranging over 4.14.1–5.55.5\\%), as is every component of B(Y)\mathcal{B}(Y). The smaller retrieval shifts along the indirect and spurious pathways are significant in 4/104/10 re-splits. Full results and the description of the analysis are in the new App. G of the revised pdf.

We thank the reviewer for raising this interesting point.

[1] Bates, Stephen, Trevor Hastie, and Robert Tibshirani. "Cross-validation: what does it estimate and how well does it do it?." Journal of the American Statistical Association 119.546 (2024): 1434-1445.


(W2: Multiple comparisons) Thank you, this is an excellent catch. We now adopt the following protocol, stated in Sec. 3 of the revised pdf:

(i) Primary estimands. B(Y)\mathcal{B}(Y) and Δret\Delta^{\text{ret}} are, for each dataset, a small set of quantities fixed by the design of the framework; we report them unadjusted.

(ii) Theory-implied nulls. DE(Y^ret)=0\text{DE}(\widehat{Y}_{\text{ret}}) = 0 (Prop. 3) and ΔLLMDE=0\Delta^{\text{LLM}}\text{DE} = 0 with race withheld are predictions of the framework. Claims of non-significance only become more conservative under adjustment (adjustment can only strengthen them).

(iii) Configuration sweeps. This is the key point of the comment. The ΔLLM\Delta^{\text{LLM}} shifts across configurations are exploratory, so in line with the multiple testing literature, we now apply Benjamini–Hochberg to control the false discovery rate at the 5% level within each dataset, over all aggregation-stage quantities reported across configurations (96 tests in MIMIC-ED; 24 in Kiva).

On MIMIC-ED, 38 of the 42 originally-significant shifts survive. All indirect-effect results survive, so the claims about the indirect pathway are unchanged. With race stated under dense retrieval, the significant direct effects are affected: the significance reduces from four to two aggregators, Aloe-Qwen (−2.0%-2.0\%) and Aloe-Llama (+1.9%+1.9\%). We have revised Sec. 3.1 accordingly. On Kiva, the main claim that the aggregator restores part of the indirect effect remains significant; the corresponding shift under lexical retrieval does not, and we have revised App. E accordingly. All changes are highlighted in the revised pdf.


(Q1: Resampling D\mathcal{D}) Please see (W1).

(Q2: Multiple comparisons) Please see (W2).

(Q3: Continuous LLM outputs) Thanks for the question. The framework actually applies unchanged to continuous outputs: identification (Prop. 2) and the efficient influence function depend on the score SS only through μS(x,z,w)=E[S∣x,z,w]\mu_S(x, z, w) = \mathbb{E}[S \mid x, z, w], and are valid for any bounded SS, binary or continuous. Continuous outputs are actually easier to handle, from the statistical perspective. For a probability score S=P(yes∣prompt)S = P(\text{yes} \mid \text{prompt}), the residual S−μSS - \mu_S has smaller variance than a residual based on a decision D∼Bernoulli(S)D \sim \text{Bernoulli}(S) sampled from SS, written D−μSD - \mu_S. Thus, for continuous scores, we expect the resulting confidence intervals to be tighter.


Please let us know if any of the raised questions remain open; thank you again for the comments.

Thank you for the detailed rebuttal and the additional experiments. I appreciate the authors' efforts to address my concerns, particularly the new re-splitting analysis and the multiple-comparison correction. The re-splitting experiment provides useful evidence that the main attribution findings are robust to changes in the retrieval database and the Benjamini–Hochberg correction satisfactorily addresses my concern regarding multiple testing. Overall, I believe that my main concerns have been substantially addressed. While some limitations remain regarding the generalizability of the database robustness analysis, they no longer constitute major soundness concerns. I am therefore inclined to increase my rating.

Reviewer_5BvDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The study primarily presents a formalization of causal attributions of bias sources within RAG-based LLM pipelines.

Critical strengths
  • The presented formalization seems rigorous and generalizable. It should be applicable especially to many high-stakes applications of LLMs.
  • The study presents some interesting findings that look different from the majority of studies one would expect in this domain. They reveal some less explored patterns in the community.
Critical weaknesses
  • The entire study gives a very strong weight to the scale of RAG being used in LLM decision-making. This may not be totally accurate, given the many other tools that are coupled with RAGs toward recent “context engineering” efforts. Relatedly, other ways of integrating RAGs (beyond the simplified process currently used) seem to be uncovered.
  • The paper needs to either acknowledge or address its limitations, including not considering other sources of biases (e.g., use-related) and not considering intersectional biases.
  • While the study presents some interesting and important findings, it rarely tries to establish the overall method against different baselines or some sort of ground truth.
Questions for authors

How would the framework in Fig 1.a extend to other implementations of context engineering in LLMs?

4 comments
AuthorsOpenReview ↗

Review Response - Part I

We thank the reviewer for their time and effort in reviewing our paper. We made some important changes to the manuscript, which are highlighted in blue color in the revised version.

(W1, Q1: Emphasis on RAG) Thanks for the comment; however, we wish emphasize that our framework is not specific to just RAG, and applies to any sequence of predictors, including the more recent context engineering advances.

First, we point to the paragraph following Remark 1 (Sec. 2.3), which notes that the proposed decomposition extends to richer pipelines (reranking, weighted aggregation, retrieval-free scoring, and other related methods). We now also make this more explicit in the revision, and add the following result to Sec. 2.3 (see revised pdf):

Corollary (Multi-Stage Pipelines). Let S0=YS_0 = Y, and let S1,…,SmS_1, \dots, S_m be a sequence of scores attached to the SFM. For each CE∈{x-DE,x-IE,x-SE}\text{CE} \in \{x\text{-DE}, x\text{-IE}, x\text{-SE}\}, CE(Sm)=CE(Y)+∑j=1mΔCE(Sj−1→Sj).\text{CE}(S_m) = \text{CE}(Y) + \sum_{j=1}^m \Delta\text{CE}(S_{j-1} \to S_j). Moreover, if stage jj preserves the conditional mean, E[Sj∣X,Z,W]=E[Sj−1∣X,Z,W]\mathbb{E}[S_j \mid X, Z, W] = \mathbb{E}[S_{j-1} \mid X, Z, W] a.s., then ΔCE(Sj−1→Sj)=0\Delta\text{CE}(S_{j-1} \to S_j) = 0.

The proof follows that of Thm. 1, just generalizing the approach. Thus, any context-engineering operation that maps (q,D)(q, \mathcal{D}) to a score (retrieval, reranking, filtering, compression) enters as one additional term Δj\Delta_j.

Regarding Q1: the SFM in Fig. 1a remains unchanged, and each stage adds a predictor Y^j\widehat{Y}_j with parents {X,Z,W}\{X, Z, W\}.

To strengthen the above point, we also add a new experiment. For the MIMIC-ED triage example, we add a reranking stage: dense retrieval returns the top-5050 candidates, and a cross-encoder (Qwen3-Reranker-4B) reads the query and each candidate jointly, selecting the top-1010 neighbors, which are passed to the aggregator (Qwen 2.5 72B, race withheld). The richer pipeline Y→Y^ret→Y^rr→Y^RAGrrY \to \widehat{Y}_{\text{ret}} \to \widehat{Y}_{\text{rr}} \to \widehat{Y}_{\text{RAG}}^{\text{rr}} then decomposes as B(Y^RAGrr)=B(Y)+Δret+Δrr+ΔLLM.\mathcal{B}(\widehat{Y}_{\text{RAG}}^{\text{rr}}) = \mathcal{B}(Y) + \Delta^{\text{ret}} + \Delta^{\text{rr}} + \Delta^{\text{LLM}}.

Here is the summary of the decomposition:

(%)DEIESE
B(Y)\mathcal{B}(Y)4.5∗4.5^*−2.6∗-2.6^*−3.1∗-3.1^*
Δret\Delta^{\text{ret}}−4.4∗-4.4^*1.1∗1.1^*−0.8∗-0.8^*
Δrr\Delta^{\text{rr}}0.6∗0.6^*0.8∗0.8^*0.2∗0.2^*
ΔLLM\Delta^{\text{LLM}}−1.2∗-1.2^*3.5∗3.5^*−0.7∗-0.7^*
B(Y^RAGrr)\mathcal{B}(\widehat{Y}_{\text{RAG}}^{\text{rr}})−0.5-0.52.8∗2.8^*−4.4∗-4.4^*

The reranking stage itself does not substantially shift the signature, but it significantly reduces the indirect effect added by the aggregator from 5.3%5.3\% to 3.5%3.5\%, a pattern visible only through the stage-wise attribution. Re-ranking also significantly improves the aggregator's final accuracy. Details of the experiment are shown in App. F of the revised pdf.

Thanks again for this helpful suggestion.


(W2: Other sources of bias) We thank the reviewer for the comment, which we now address explicitly in the Limitations section (please see revised pdf). We give a short summary below.

Intersectional bias. The framework in principle applies to any protected attribute, and can be run separately for each one. Further, intersectional groups can be handled by taking XX to be multi-valued (e.g., X=(race,sex)X = (\text{race}, \text{sex}), with sex removed from ZZ), since the xx-specific effects are defined for any pair of levels x0,x1x_0, x_1; we refer to the causal fairness literature that handles this [1].

Use-related bias. Deployment-related bias [2], i.e., a mismatch between how a system is designed and how it is used, can actually be nicely addressed by our framework. Suppose decision-makers act on Y^RAG\widehat{Y}_{\text{RAG}} differently across groups. Then, the final human decision Y~\widetilde{Y} is a further score attached to the SFM, contributing an additional term Δhuman=B(Y~)−B(Y^RAG)\Delta^{\text{human}} = \mathcal{B}(\widetilde{Y}) - \mathcal{B}(\widehat{Y}_{\text{RAG}}) also through the new corollary discussed in (W1). This is possible to compute whenever Y~\widetilde{Y} is observed.

[1] Plečko, Drago, and Elias Bareinboim. "Causal Fairness Analysis: A Causal Toolkit for Fair Machine Learning." Foundations and Trends® in Machine Learning 17.3 (2024): 304-589.

[2] Suresh, Harini, and John Guttag. "A framework for understanding sources of harm throughout the machine learning life cycle." Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. 2021.

AuthorsOpenReview ↗

Review Response - Part II

(W3: Baselines and ground truth) We thank the reviewer for this comment. We would like to point out that the paper does validate the method against ground truth (see App. D, referenced at the start of Sec. 3).

Specifically, we consider semi-synthetic MIMIC-ED data with known, closed-form expressions for outcome, retrieval and aggregation mechanisms. We establish that our method (a) recovers B(Y)\mathcal{B}(Y) within its confidence intervals; (b) recovers DE(Y^ret)=0\text{DE}(\widehat{Y}_{\text{ret}}) = 0, as implied by Prop. 3; and (c) attributes an injected δ=10\delta = 10\\% to the aggregation stage and the direct pathway (ΔLLMDE=10.2%\Delta^{\text{LLM}}\text{DE} = 10.2\%; IE and SE ≈0\approx 0).

Regarding baselines: most existing audits report the TV(Y^RAG)\text{TV}(\widehat{Y}_{\text{RAG}}) measure; therefore, the TV itself serves as the most commonly used baseline in the literature. The TV, however, is the sum of the nine terms in Thm. 1, and cannot attribute disparity to components or pathways, which is exactly the gap our approach fills.


Please let us know if any further questions remain, thanks again for the comments.

This reviewer thanks the authors for their responses. Their response partially addresses the raised concerns. In particular, the concerns about the baselines must be addressed explicitly in the main text. I would maintain my score.

(Follow-up: Baselines in the main text) Thank you for the follow-up. Following your suggestion, we have added a paragraph on baselines and ground truth to the introduction of Sec. 3 (highlighted in blue, in the latest update of the revised pdf). It states that existing audits report aggregate disparities, so that TV(Y^RAG)\text{TV}(\widehat{Y}_{\text{RAG}}) is the natural baseline, and explains how our decomposition refines it. We also highlighted an example in Sec. 3.1 where the TV masks offsetting effects (TV≈0.2%\text{TV} \approx 0.2\%, while the indirect and spurious effects are ≈3.9%\approx 3.9\% and ≈−3.9%\approx -3.9\%).

As you mentioned that our response partially addresses your concerns, we would be grateful if you could let us know which points remain open. We would be glad to address them before the end of the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.