acceptodds
Withdrawn from ICLR 2027

DF-Audit: Scalable Fairness Auditing for Disentangling Legitimate-Evidence Use from Discriminatory Reliance in LLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

LLMs are increasingly used in socially consequential decision-making settings such as hiring, raising concerns about differential treatment across groups defined by protected attributes (e.g., race, nationality, and gender). Existing fairness benchmarks typically generate a fixed set of instances that compare model decisions under interventions on such protected information while holding other information fixed. However, observed decision outcomes alone cannot determine whether a decision error reflects inadequate use of legitimate evidence or sensitivity to protected information, while predefined test conditions provide limited control over decision difficulty, making it difficult to stress-test increasingly capable models. We introduce DF-Audit, a dynamic fairness auditing framework that disentangles failures in legitimate decision making from dependence on protected information through controlled contrasts. We formulate decision quality as a pairwise ranking problem and progressively introduce protected-information variation for increasingly fine-grained fairness diagnosis. Within-group comparisons first establish whether the model preserves rankings implied by legitimate evidence; cross-group comparisons then test whether the same legitimate ordering changes with protected-group composition; and counterfactual interventions finally hold legitimate evidence fixed while manipulating protected information to probe finer direct and proxy-enabled pathways. Beyond diagnosis, DF-Audit scales the audit through a breadth-to-depth process: breadth explores diverse configurations and randomly sampled instances across legitimate and protected-information conditions, while depth progressively increases difficulty within selected conditions by lengthening the context, narrowing differences in legitimate evidence, increasing conflicts among legitimate cues, and strengthening protected-information challenges.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLMs, fairness, audit, evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_f9bvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DF-Audit, a fairness auditing framework for LLM decision-making in hiring that generates pairwise comparisons from O*NET occupational requirements. The legitimate evidence fixes the correct ordering by construction, independently of any protected information. The framework separates evidence-following (within-group order preservation, OPR_within) from dependence on protected information (cross-group contrast ∆c and direct/proxy-enabled counterfactual interventions τ), and adds configurable stress tests that vary the number of evidence items, the differing items, the signal strength, and the presence of conflicting evidence. Across seven LLMs and four binary attributes, the authors report three findings: dependence persists in models that otherwise follow the evidence well (mostly favoring younger and U.S. candidates), group-level and intervention-based signals rarely agree, and harder configurations reveal dependence that the main audit does not detect.

Critical strengths

The construction in §4.2 and Appendix B.1 yields an unambiguous ground-truth ordering, since the winner dominates the loser on every requirement and R is generated before X. The invariant is also verified after generation (Appendix B.5).

The diagnosis matrix (Table 3) reads |∆c| against the within-group error 1 − OPR_within, which gives a principled way to distinguish “protected information overrides usable evidence” (GPT-4o-mini, age) from “protected information breaks ties among unresolved pairs” (Qwen3-14B).

The statistical treatment is careful: paired sign-flip tests with Holm correction, a δ-sensitivity analysis (Table 12), a tie-permitted robustness check (Table 14), and suitably hedged discussions of what is not identified (Appendix D.4 on indirect effects, Appendix E.3 on proxy-only presentation).

The stress tests produce concrete, non-obvious results. GPT-4o-mini’s age gap becomes sustained only from 20 evidence items onward, while OPR_within remains at or above roughly 0.95 (Table 16, Fig. 3a,d). GPT-4.1, which is clean in the main audit, shows a significant preference for younger candidates at five of the six conflicting-evidence levels (Table 17).

Critical weaknesses

The “one signal is not enough” finding (Obs. 2) is likely in part an artifact of the intervention design. Interventions use identical-evidence pairs with ties permitted, and several models simply tie every such pair: Qwen3-14B has τ = 0 and 0% flips on all attributes (Tables 4 and 13), and GPT-4.1 ties all 200 units in Table 17(iii). In these cases τ is structurally floored rather than independently informative. Interventions on pairs with a small legitimate gap, or under forced ranking, would test whether the two views genuinely diverge.

The main configuration is highly artificial: 40 templated items, a single “slightly above” vs. “slightly below” difference, and an explicit attribute line (“Race: white”, “The candidate is 62 years old”) placed next to the proxy (Fig. 4). The dominant age finding may partly reflect models treating stated age as job-relevant under the instruction to “use job-relevant evidence,” rather than discrimination in realistic screening. Adding a free-text résumé condition, and extending the proxy-only analysis beyond race and two models (Appendix E.3), would strengthen external validity.

Each prompt is queried once and some decoding is non-deterministic (Appendix C), yet no noise floor is measured. Re-querying unchanged prompts would show how much of the flip rates and the small ∆c values (e.g. Grok-3-mini age, −1.4 pp) exceeds run-to-run variability. Decoding settings are also not reported for GPT-4.1 (main audit), MiniMax-M3, Gemini-3.5-Flash-Lite and Qwen3.7-Plus, which limits both interpretation and reproducibility.

The stress testing that motivates the “dynamic/scalable” framing covers only two models with hand-chosen sweeps, and the conflicting-evidence target depends on an importance-weighted score that the prompt never specifies, with acceptance margins as small as 0.002 (Appendix B.1). Where OPR_within is 0.55–0.75, the “legitimate” ordering is partly an author convention. Running the sweeps on all seven models and reporting results as a function of the margin would make Obs. 4 more convincing.

Questions for authors

See weaknesses

Reviewer_wsNEOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a method for fairness auditing of LLMs, in an attempt to distinguish whether the LLM is basing its decision on legitimate or illegitimate sources of disparity. The paper adopts a causal framework, and uses what is described as controlled generation for probing the LLM decision mechanism.

Critical strengths

(S1) The paper explores a timely and important problem, of understanding bias in LLM-produced predictions or decisions.

(S2) The idea of using controlled generation (or, put differently, synthetic data) is possibly a useful idea, and an interesting avenue to explore.

(S3) Some of the experiments appear to be interesting.

Critical weaknesses

(W1) The paper has considerable space to improve on the mathematical rigor. Many of the concepts in the paper are described in words, which often results in lack of precision in some places. Several places where this occurs explicitly:

  • L80-82: “how can we distinguish failures in legitimate decision making …”
  • L92: “whether its decision changes when protected information is varied under matched evidence”
  • L102: “makes the test condition itself configurable rather than fixed”

This makes the story difficult to follow in many places. Simple mathematical descriptions would remove the ambiguity entirely.

Most importantly, this also occurs in the contribution bullets, e.g., “protected information dependence persists despite competent decision-making” -> here, the statements are not linked to a specific Proposition, Theorem, or experimental result. Describing contributions purely in words, without formal results, comes across as a significant weakness.

(W2) A clear motivating example, early on, would also help clarity significantly.

(W3) Methodologically, a significant issue is encoded in the assumptions in Figure 2.

Firstly, the notion of legitimate evidence is probably related to the concepts of resolving variables [1] and business necessity [2]. It would be great to connect explicitly with these names.

However, the above cited literature also signals something important: legitimate evidence and “proxy information” cannot be separated easily (for instance, a PhD in computer science is both a proxy for sex in the current world, and possibly legitimate evidence). In other words, R and B variables may be entangled through their own causal structure. As a consequence, it is unclear whether nodes R and B can be separated in the graph as described. Also, the assumption of no R->B and B->R arrows seems like a strong restriction.

Overall, the assumed causal model seems to restrict generality much more than is desirable.

(W4) Constructing the legitimate decisions paragraph: implicitly, here, there seems to be a clear monotonic relationship of the covariates and the outcome. It is really worth explaining whether this is a necessary assumption of the framework. Currently, it is difficult to infer this from the paper.

(W5) Group-constrained variants paragraph: at this stage, the notation (G1,G1)(G_1, G_1), … is not even defined. This impedes the reading.

(W6) The notation in Y^x1,∅\hat Y_{x_1, \emptyset} in L298 is not the standard notation in the structural causal models framework, and could be misinterpreted very easily.

(W7) The overall contribution needs sharpening.

It seems that the key idea in the paper is to use some form of “synthetic data” to investigate the LLM’s decision-making boundary. This is a good idea.

However, one could look at just the data D=(X,B,R,Y^)D = (X, B, R, \hat Y), and gain a lot of insight into discrimination, just by applying known tools in causal fairness (e.g., analyzing direct/indirect/spurious/path-specific effects).

The comparative advantage of the proposed approach needs better justification: e.g., what is the advantage compared to just looking at static data DD from above? Is it related to improved statistical efficiency? Can it handle issues of non-overlap between populations (violations of positivity), which is usually needed for standard methods working with observational data? Justifying the benefit crisply would strengthen the paper.

A related question that arises: if the synthetic data is different from the observational distribution, is there a risk that the fairness quantification is performed in a non-existent, possibly shifted, population?

[1] Kilbertus, Niki, et al. "Avoiding discrimination through causal reasoning." Advances in neural information processing systems 30 (2017).

[2] Plečko, Drago, and Elias Bareinboim. "Causal Fairness Analysis:: A Causal Toolkit for Fair Machine Learning." Foundations and Trends® in Machine Learning 17.3 (2024): 304-589.

Questions for authors

Please see (W7).

Reviewer_eCQgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents DF-Audit, a dynamic fairness auditing framework for LLMs that aims to distinguish between failures in evidence-based decision making and reliance on protected attributes. The authors test the framework on seven LLMs, covering four protected attributes (sex, age, race, and nationality), using a synthetic hiring scenario based on O*NET occupational requirements. The results suggest that even when LLMs make fairly accurate decisions based on job-related evidence, they may still rely on protected information. The paper also shows that group-based comparisons and counterfactual interventions can uncover different fairness issues, while more challenging evaluation settings help reveal biases that might otherwise go unnoticed.

Critical strengths

The paper looks at an important problem in LLM fairness evaluation. When we see different decisions across demographic groups, it is not always clear whether this is because the model fails to use the relevant evidence properly or because it relies on protected attributes. I think DF-Audit provides a useful way to look at this issue by considering both decision accuracy and the influence of protected information. This is especially meaningful in hiring scenarios, where we care about both making correct decisions and treating different groups fairly.

Another strength is the way the authors design the evaluation. By changing legitimate evidence and protected attributes separately, they can better understand what actually affects the model's decisions. The experiments are also quite comprehensive, covering seven LLMs and four protected attributes. One interesting finding is that models can still be affected by protected information even when they perform well on evidence-based decisions. I also like that the authors explore more challenging settings, since some fairness issues might not show up in simpler tests. However, all experiments are based on synthetic hiring scenarios, so I am not fully convinced that the same findings would hold in real-world hiring applications.

Critical weaknesses

One major concern is the limited generalizability of the framework. The experiments are only conducted in a synthetic hiring setting, where the evidence is generated based on O*NET requirements and the correct decisions are defined by predefined rules. While the authors present DF-Audit as a general fairness auditing framework, it is still unclear whether it can be applied to other domains, such as lending, university admissions, or healthcare. These domains have different types of evidence and decision criteria, and constructing reliable counterfactual examples may not be as straightforward as in the current setting.

I am also concerned that the main evaluation tasks are relatively simple. In most cases, one candidate is clearly better than the other across all relevant evidence, so the model does not really need to deal with difficult trade-offs or conflicting information. As a result, the high order preservation rates may just show that the models can identify obvious differences between candidates, rather than demonstrate strong evidence-based reasoning ability. The authors do include some more challenging cases in the stress tests, which is useful. However, the ground truth in these cases is still based on predefined weighted scores, and I am not sure how well these reflect real-world hiring decisions.

Finally, I find the main findings somewhat expected. For example, high decision accuracy does not necessarily mean a model is fair, different fairness metrics can reveal different issues, and more challenging tasks often lead to more failures. These are already well-known observations in fairness research. While the paper provides experimental evidence for these points, I would like to see more discussion of what new insights DF-Audit brings beyond confirming what we already know.

Questions for authors

How can DF-Audit be applied to decision-making domains beyond synthetic hiring, where legitimate evidence and ground-truth decisions may be difficult to define?

How do the authors validate that the proposed diagnostic measures identify genuine discriminatory reliance rather than artifacts of different evaluation conditions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.