acceptodds
Under review as a conference paper at ICLR 2027

Asymmetrically Unfair Decision Making and Reasoning in LLMs

Abstract

As LLMs are being deployed in higher-stakes tasks such as hiring, healthcare, and law, developers are making efforts to align them such that their decision making and processes are fair and unbiased, to avoid harms and discrimination. Yet, existing benchmarks only measure fairness at a surface level, risking missing subtle discrimination. To address this, we introduce the FairStress benchmark, which includes 300 decision-making cases across 29 critical domains (e.g., job candidate selection) with 4 degrees of demographic identity signaling, from implicit to explicit. In contrast to previous fairness benchmarks, FairStress examines LLM decisions as a full process, including output-level, (a)symmetric robustness to adversarial prefixes that subtly push towards certain identities, and verbalized and internal reasoning. Contrary to the stereotypical LLM behavior reported by prior work, comprehensive benchmarking with \FairStress shows anti-stereotypical overcorrection, an asymmetry that emerges more strongly in how robustly the model makes decisions for one person over another. With explicit identity, accuracy differs by only 2.9 points between groups, yet one added sentence makes 15 models abandon correct answers favoring the majority 9.3 points more often, and flip ties toward the minority 23.9 points more often. To test mitigations to address this, we use 9 post-hoc debiasing methods and show that they do not reliably reduce the asymmetry. We further trace the behavior back to the training data and find asymmetric demographic framing, with minorities portrayed and protected more positively. Therefore, we argue that fairness evaluations should test both the direction and robustness of a decision.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
fairness, alignment, benchmark, robustness, bias, evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_FcHvOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper focuses on the construction of the FairStress benchmark to investigate the phenomenon of asymmetric unfairness in Large Language Models (LLMs). Specifically, the authors argue that merely observing a model's final decision may fail to fully reveal its unfairness; even if the final output appears relatively fair, the processes of making, maintaining, and justifying that decision can still be influenced by demographic attributes. To explore this, the authors developed a benchmark requiring LLMs to choose between two candidates across 29 decision-making domains, including recruitment, healthcare, and the judicial system. First, GPT-4o-mini was used to generate 300 base scenarios according to predefined rules, mostly presented in two forms: (1) GT (Ground Truth): one candidate’s merit is clearly superior to the other’s, implying a "correct answer"; and (2) AMB (Ambiguous): both candidates have distinct strengths and weaknesses, with no objectively correct answer. Demographic attributes were then swapped between the two candidates across varying levels of identity signals (D0–D3). Additionally, the authors designed 78 adversarial prefixes to test whether the models would alter their initial decisions when exposed to additional information. The raw dataset comprised approximately 13 million test items; Item Response Theory (IRT) was subsequently used to select about 50,000 items to form "FairStress-Core."

Using these datasets, the authors conducted experiments across four dimensions: Decision Fairness, Robustness Fairness, Reasoning Fairness, and Internal State. The results indicate that while group disparities in final decision accuracy were relatively minor, the models exhibited significant "anti-stereotypical overcorrection" under external pressure—a phenomenon that typically intensified as demographic attributes became more explicit. The paper also evaluated existing debiasing methods, finding that most failed to consistently mitigate this asymmetry, while some incurred a significant cost in accuracy. Finally, the study further characterized these asymmetric behaviors and explored their potential causes through analyses of reasoning language, internal activations, and training data audits.

Critical strengths
  1. Clear narrative arc and logical progression: The paper presents a coherent narrative centered on the phenomenon of asymmetric unfairness in LLMs. It moves systematically from problem definition to the proposal of an evaluation framework (including benchmark construction and data synthesis), and then delves deeper—progressing from direct decision outcomes to the effects of perturbations, reasoning/explanation analysis, and finally, internal model activations. The experimental design is well-structured and effectively addresses the core research questions. The writing is profound yet accessible, and the figures serve as excellent aids to understanding. I particularly appreciate Figure 1 and the "takeaway" sections for the research questions; they enhance the overall presentation and ensure the paper remains internally consistent regarding the issues it addresses.

  2. Comprehensive experiments: The paper goes beyond superficial, outcome-oriented evaluations of fairness. It designs a series of in-depth experiments to examine asymmetric robustness under stress, asymmetries in model reasoning, and demographic-related signals within internal model activations. Furthermore, the paper combines experimental results with reasoned arguments to offer plausible explanations for the observed fairness phenomena, thereby providing valuable insights to the fairness research community.

  3. Valuable research problem: Decision outcomes that appear fair can indeed mask deeper-seated fairness issues. While much current AI fairness evaluation focuses on final results, this paper highlights two critical points: 1) Latent fairness risks beneath the surface outcomes can cause seemingly fair results to deteriorate under different conditions (such as the stress scenarios explored here or in entirely different contexts); and 2) In many human-in-the-loop decision-making scenarios, stakeholders consider not only the final outcome but also the model's explanations and reasoning traces. Consequently, uncovering these deeper fairness risks holds significant practical value.

Critical weaknesses
  1. There are potential issues regarding the comparability of demographic identity signal strengths: The paper categorizes identity information into four levels (D0–D3) representing increasing specificity; however, the actual construction reveals inconsistencies in how identity signals manifest across different demographic attributes. For instance, while D1 relies on names to indirectly imply attributes like race and gender, it employs explicit descriptions for age and disability status—to the extent that the D1 and D3 representations for these groups become effectively identical. Furthermore, the strength of identity cues may also vary by name; some names are strongly associated with specific groups, while others offer weaker signals, potentially introducing additional noise. Although the authors outline specific handling measures and data quality controls in the appendix, the comparability of signal strengths across different demographic attributes lacks sufficient validation. This limitation may, to some extent, affect the interpretation of the relationship between identity information salience and asymmetric fairness.

  2. The paper’s novelty is relatively limited; its research positioning and contributions could be more clearly defined: Several key research themes addressed in this paper—such as the potential misalignment between outcome fairness and process fairness, model fairness robustness under adversarial perturbations, and anti-stereotypical overcorrection—have already been explored in existing literature. The four evaluation levels proposed here also rest on well-established research foundations. Consequently, the paper’s primary contribution appears to lie in systematically organizing existing fairness issues into a unified evaluation framework and providing further empirical evidence through comprehensive, in-depth experiments, rather than introducing entirely new research questions or discovering these phenomena for the first time. While this does not diminish the value of the experiments, it is recommended that the authors provide a more thorough discussion of related work—particularly recent studies on the divergence between outcome and process fairness and anti-stereotypical overcorrection—within the Introduction and Related Work sections to better articulate the paper's incremental contributions relative to existing research.

  3. Complete reliance on synthetic decision scenarios, lacking external validation against real-world data: The foundational decision scenarios in FairStress were entirely generated by GPT-4o-mini based on predefined rules. Although the authors conducted both automated and manual verification, these checks primarily ensured adherence to experimental parameters rather than establishing a robust link to real-world decision-making contexts. In particular, the paper presents decision scenarios in two forms—GT (scenarios with a clear correct answer) and AMB (scenarios where both options have distinct advantages and disadvantages). While this design facilitates variable control and experimentation, it somewhat idealizes the nature of actual decision-making problems. Real-world domains such as recruitment, healthcare, and the judicial system often involve far more complex conditions and information distributions that do not necessarily map neatly onto these two categories. While the experimental results based on controlled synthetic data are comprehensive, the absence of independent validation using real-world decision cases leaves some uncertainty regarding the generalizability of the findings and the actual severity of the observed unfairness. Therefore, it is recommended that the authors more clearly define the scope of applicability for their conclusions and further discuss this limitation.

Questions for authors
  1. Appendix B.3 notes that explicit identity descriptors are used for age and disability groups starting at level D1, and that D3 is identical to D1. How do the authors address the issue of non-comparable identity signal strengths across these different demographic attributes? Based on existing experimental results, could the authors provide the primary trends for D0–D3 with age and disability contrasts excluded, in order to verify the robustness of the conclusions drawn from Figure 2?

  2. In the full FairStress dataset, the "Tie-Break Lean" for D0 is reported as 50.0%; however, in the debiasing experiments using FairStress-Core, the D0 baselines for the three models reach 55.9%, 56.2%, and 58.2%, respectively (Appendix Tables 53–55). How do the authors explain this discrepancy? Did the IRT-based selection process for FairStress-Core preserve the complete identity-mirroring pairs, and does this affect the interpretation of the relevant fairness metrics?

5 comments
AuthorsOpenReview ↗

Response to Reviewer FcHv (1/3)

Thank you for your careful and thorough review! We are glad you found our work to have a "clear narrative arc and logical progression", with "comprehensive experiments" offering "valuable insights to the fairness research community", and that "uncovering these deeper fairness risks holds significant practical value". Below, we address each concern with more analyses and clarifications; changes are marked in red in the revised PDF.


TL;DR

  • Identity-signal comparability (W1, Q1): without age and disability, the degree trend is unchanged, and how well models recognize a name does not predict its effect (r = 0.04).
  • Novelty (W2): we extended the related work and the positioning in the Introduction. FairStress's design of a validated correct answer, mirrored pressure, and one decision scored at four levels enables us to measure procedural fairness and show toward whom a model gives up a correct answer. We also find a framing asymmetry that replicates in 4 other post-training datasets.
  • Real vs. synthetic inputs (W3): on résumés, names and job ads from a real field experiment, all 7 models show the same asymmetry.
  • Core D0 baseline (Q2): IRT selection did not keep mirrored pairs; comparing each Core item with the same item without names shows that identity still shifts choices toward the minority in every model, and we release a mirror-complete subset.

W1, Q1. The degree trend holds with age and disability removed, and name strength does not drive it

Identity signals may not be comparable across attributes: age and disability are explicit from D1, and names vary in strength.

Thank you for this important point; we agree, and we move the App. B.3 note into the main text (Sec. 2.1).

Without age and disability, the trend is unchanged (Type 1, mean over 15 models; Table 40, App. D.11):

ContrastsTie-Break Lean D0 / D1 / D2 / D3Tie-Break Shift D0 / D1 / D2 / D3
All 2650.0 / 52.0 / 57.4 / 61.7+0.1 / +4.1 / +14.1 / +24.1
24 name-based50.0 / 51.4 / 56.6 / 61.9+0.1 / +2.9 / +12.8 / +24.9

The trend stays monotone, and D3 barely moves. Disability alone raises the pooled D1 point (lean 68.8% from D1 on); age is near zero at every degree.

Name strength does not drive the effect. We asked all 15 models which group each of our 108 names belongs to, and correlated how strongly they recognize a name with its Tie-Break Lean at D1. Recognition varies across names, but it does not predict the effect (r = 0.04 over the 60 names with enough tie items; r = 0.10 within contrast; App. D.11).

So the degree trend depends neither on the age and disability descriptors nor on how recognizable a name is.

AuthorsOpenReview ↗

Response to Reviewer FcHv (2/3)

W2. What is new: FairStress shows toward whom a model gives up a correct answer

The themes have prior work; related work should be discussed more fully.

Thank you for pushing us to sharpen our positioning. We agree that each theme and each of the four levels has precedent.

  • Related Work (Sec. 4) now covers the two areas you name: outcome vs. process fairness (Popoola & Sheppard, 2026; Rajwal et al., 2025; Wu et al., 2025) and anti-stereotypical overcorrection (Kumar et al., 2026; Rozado, 2025; Tamkin et al., 2023).
  • The Introduction now ends with an itemized list of what is new.

FairStress combines three design choices to answer a new question: does a model hold a correct decision equally for both groups, and does an identical push have asymmetrical power depending on its direction?

  1. A validated correct answer. Overcorrection studies measure preferences on tasks without a correct answer (Kumar et al., 2026; Tamkin et al., 2023). With one, overcorrection becomes a measurable error and shows how much performance a model is willing to sacrifice: models stay accurate on clear-cut cases but asymmetrically give up correct answers under small adversarial prefixes (Sec. 3.1).
  2. Mirrored pressure. Robustness benchmarks ask whether a model gives way under pressure, for example whether it can be jailbroken (FLEX; Sharma et al., 2025). By mirroring every push and identity, we give each failure a direction and ask for whom the model changes its answer: the Robustness Gap (+9.3 at D3, positive in all 15 models) is about three times the Accuracy Gap (+2.9; Sec. 3.1), and the same asymmetry holds on a real field experiment's résumés (W3). This shows that unfairness can appear in different ways, and that robustness is asymmetrical across demographics.
  3. One decision at four levels. Scoring the same decision in its answer, under pressure, in its reasoning and in its activations shows where a model looks fair on one level but not another, so a single score can mislead. Together, these levels show how different models or training algorithms affect each level.

For example, Tulu-3 and Tulu-3.1 differ only in their RL stage. Tulu-3.1 leans less on ties than Tulu-3 (52.0% vs. 57.4%), so on this metric it looks fairer, and one might conclude that the RL change made it fairer. But FairStress's four levels show that it loses 6.1 points of accuracy and nearly doubles its Accuracy Gap (Table 87). A single metric would not show the full picture. The reasoning level also shows that this overcorrection does not actually serve minorities: on ties, models justify picking the minority candidate by identity and the majority candidate by merit (Sec. 3.3). So FairStress reveals an interesting phenomenon: the overcorrection in the answers comes with reasoning that overlooks minority candidates' qualifications and hard work (Sec. 5).

We also find a matching asymmetry in post-training data (App. I.7). As prompted by another reviewer, we checked whether the asymmetric framing in OLMo-3 replicates, and it does in 4 other post-training datasets! Share of passages framing a group positively:

Post-training dataMinority-coded groupsMajority-coded groups
OLMo-3 (fully open)36.4%11.3%
Tulu-3 SFT38.4%15.5%
HH-RLHF25.9%6.7%
UltraFeedback37.1%13.6%
SmolTalk40.2%11.7%
Four datasets pooled34.1%10.1%
  • By group: men 4.5%, Christians 5.4%, White people 7.2%, against Black 40.9%, transgender 50.0% and disabled people 62.1%.
  • What it suggests: prior work attributes overcorrection to alignment in general (Kumar et al., 2026); we hypothesize that this auditable property of the data may contribute (Sec. 5). This points to a potential limitation in how post-training datasets are designed, and we need more studies on how to design them!
AuthorsOpenReview ↗

Response to Reviewer FcHv (3/3)

W3. Real vs. synthetic inputs: on a real field experiment's résumés and names, models show the same asymmetry!

All scenarios are synthetic, and the GT/AMB design idealizes real decisions.

Thank you for this important point! We agree:

  • Scope. In Sec. 5 we scope our conclusions to controlled two-candidate decisions. FairStress needs a controlled setting to measure the asymmetry cleanly at each level, and real decision records lack that control.
  • A new real-world check (App. D.15). Encouraged by your comment, we use the Bertrand & Mullainathan (2004) field experiment: its real job ads, coded résumé attributes (education, experience, gaps, skills), first names, and the real employers' callbacks.
  • Setup. Two blind judges select 600 clear-cut pairs from the same ad and 400 ties. We add names (D1) or names with explicit labels (D3), mirror every pair, and add four prefixes that never mention a group. Seven models: OLMo-3-7B, OLMo-3.1-32B, Qwen2.5-7B, Llama-3.1-8B, Gemma-3-27B, GPT-4o-mini, GPT-5.5.

Results:

  • The cases work without identity: D0 accuracy on clear-cut pairs is 98.1–100%.
  • Ties and pressure (D3): every model picks the Black-coded candidate more often on ties and gives way more readily to prefixes favoring them (Table 41). Each pair is a Black-coded vs. a white-coded candidate of the same gender:
ModelTie-Break Lean, % (women / men)Robustness Gap (women / men)Accuracy Gap (women / men)
OLMo-3-7B63.5 / 65.1+15.0 / +2.8+0.2 / +0.0
Qwen2.5-7B67.7 / 64.5+38.1 / +30.1+0.8 / +0.3
Llama-3.1-8B68.1 / 68.1+25.1 / +21.7+13.3 / +5.2
Gemma-3-27B55.4 / 58.4+51.4 / +53.2+2.9 / +0.7
OLMo-3.1-32B81.9 / 85.3+46.5 / +46.5+0.3 / +0.3
GPT-4o-mini57.3 / 57.2+48.8 / +47.8+0.9 / +0.0
GPT-5.562.9 / 73.3+18.2 / +17.6+8.3 / +14.1
  • Severity: on clear-cut pairs, GPT-5.5 picks a weaker Black-coded résumé over a stronger white-coded one in 10.5% of cases, but a weaker white-coded résumé over a stronger Black-coded one in only 0.9%.

Q2. Clarifying the D0 baseline on FairStress-Core, and a matched estimate that confirms the effect

Why is D0 Tie-Break Lean 50.0% on the full corpus but 55.9–58.2% on FairStress-Core, and did IRT selection preserve mirrored pairs?

Thank you for this precise question!

  • Why D0 differs. On the full corpus, D0 is exactly 50.0% by construction. FairStress-Core was selected item by item for informativeness, so IRT selection did not keep most mirrored pairs, and its D0 lean reflects which case profile a model prefers among the selected items, not identity. We now state this in Sec. 2.1 and App. D.8.
  • A matched estimate on every Core item. We compare each item with its D0 twin (same case, candidate order and prefix, no identity) and measure how much more often the model picks the minority candidate once identity is shown. At D3 this effect is positive, with every 95% interval above zero, for all 12 frontier runs (6 models in answer and reasoning modes; +4.7 to +12.7 points) and for the three debiasing baselines (OLMo-3-7B +6.9, Gemma-3-12B +8.6, Qwen2.5-7B +10.2).
  • Core-M, a mirror-complete subset (17,274 Type-1 D3 items, each case with both identity assignments, so case content cancels). We release it and run our new analyses on it: sampling (App. D.10), explicit policies (App. G.8) and R1-style reasoning models (App. H.10). The asymmetry holds in all three.

The full-corpus results are unaffected, and each debiasing method is compared with its own baseline on the same Core items. We recommend:

GoalSet
Exact mirrored comparisonsCore-M
Broad coverageFairStress-Core
Most precise estimatesFull corpus

Thank you again; your review made our claims more precise. We hope this addresses your concerns, and if so, we would be grateful if you would consider raising your score. If you have any further concerns, please let us know and we would be happy to address them.

Reviewer_ce6XOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces FairStress, a benchmark for studying biases in LLMs. It contains 300 core scenarios, generated using GPT-4o-mini, spanning 29 domains such as hiring and lending. The design principles motivating the benchmark are: (1) LLM benchmarks should study procedural fairness, not just output fairness; and (2) fairness claims must be robust, i.e., one should study whether a decision holds despite the introduction of additional context.

The benchmark is constructed to: (1) measure both model correctness (samples containing a correct answer) and unfairness under ambiguity (no clear answer); (2) manipulate the identity information revealed to the model (none at all, name alone, name plus an associated activity, and name plus explicit demographics); and (3) compare "majority" vs. "minority" groups given the same context, in addition to a control case that compares different "minority" vs. "minority" groups.

The authors evaluate model fairness, i.e., differences in model behavior between two groups, at four levels: (1) output/decision fairness, (2) "robustness" to the introduction of new context, (3) verbalized reasoning, and (4) internal activations. Their key empirical claims are: (1) while decisions appear fair, they shift in favor of minority candidates when identity information is revealed, which the authors claim existing bias benchmarks do not account for; (2) common debiasing methods do not achieve the desired alignment/fairness; (3) verbalized reasoning shows different justification patterns for majority vs. minority groups; and (4) an examination of pre-training data from one open-weight model finds no samples that exemplify what a "merit-based" decision-making process should look like.

Critical strengths

Overall, this is important work that studies algorithmic fairness in LLMs. The specific strength of the paper that motivates acceptance is:

  • Data design, empirical coverage, and human validation. The authors largely construct the benchmark carefully, with valid justifications for its construction. Their model coverage is good (patterns are validated on 15 models), as is their data coverage (~13 million text samples). They also validate the LLM-judged responses, which is much appreciated and timely validation to ensure that LLM judgments are valid.
Critical weaknesses
  1. Limited novelty. At a minimum, the related work section is incomplete and misses key papers that are directly relevant to this work. This lack of contextualization undermines the novelty / significance of this work. Seen alongside these works, the paper appears to do some careful unifying work, but it largely builds on premises and produces outcomes that are already known in the field:
  • The need for robustness checks in fairness (the second guiding principle of the dataset) has already been addressed in many papers. See [1], which studies fairness through the lens of robustness, i.e., whether LLMs "hold" their decisions under "extreme conditions" (similar to the conditions the authors of this paper call "pressure").
  • Reasoning has already been investigated (as a proxy for procedural bias) together with outcomes. See [2] and [3]. Additionally, [4] addresses outcome fairness in the presence of reasoning bias. This brings into question the novelty of the first guiding principle of the dataset, which is to study not only outcome fairness but also its relation to verbalized reasoning (this paper additionally studies activations).
  • Similarly, prior work also discusses claims such as observed anti-stereotypical bias, potential overcorrection resulting from alignment, and refusal to assign negative traits to minority groups. See [5] which studies this premise and produces similar outcomes.
  1. Weakly corroborated claims and under-specifications. Some claims are weakly corroborated. In some instances, setup details required for an adequate and complete assessment of the work are also missing. See the questions below.

References:

[1] Jung, D., Lee, S., Moon, H., Park, C., & Lim, H. S. (2025, April). Flex: A benchmark for evaluating robustness of fairness in large language models. In Findings of the Association for Computational Linguistics: NAACL 2025 (pp. 3606-3620).

[2] Rajwal, S., Garg, S., Abdel-Salam, R., & Zayed, A. (2025). Do Biased Models Have Biased Thoughts?. arXiv preprint arXiv:2508.06671.

[3] Wu, X., Nian, J., Wei, T. R., Tao, Z., Wu, H. T., & Fang, Y. (2025). Does reasoning introduce bias? a study of social bias evaluation and mitigation in llm reasoning. arXiv preprint arXiv:2502.15361.

[4] Popoola, G., & Sheppard, J. (2026). Fairness of Explanations in Artificial Intelligence (AI): A Unifying Framework, Axioms, and Future Direction toward Responsible AI. arXiv preprint arXiv:2605.09852.

[5] Kumar, D., Gupta, I., Birur, N. A., Baswa, T., Agarwal, S., & Harshangi, P. (2026). Redirected, Not Removed: Task-Dependent Stereotyping Reveals the Limits of LLM Alignments. arXiv preprint arXiv:2604.02669.

Questions for authors

Clarification requests

  1. What are the "steps" referred to in line 302?
  2. Line 395 states: "The model's reasoning focuses on merits for majority candidate and identity for minority candidates when deciding correctly for both." It can be argued that, under the principle of substantive equality, evaluating candidates differently makes sense when standard measures of achievement do not reflect an unequal starting point. The authors' position must be grounded in a philosophical definition or principles of fairness.
  3. How do the authors think sampling stochasticity does or does not contribute to the key contributions and claims made in the paper? What temperature values are used to sample responses?
  4. OLMo-Think is one of the models used, but do the reasoning claims and takeaways hold for "reasoning" R1-style models?
  5. In line 257, it appears you are comparing "Muslim women" with "atheist men." Are the majority/minority groups intersectional? If so, this raises a concern about whether the compared quantities vary minimally. That is, a conclusion drawn from comparing Muslim women with atheist men does not clearly attribute the change to gender, because the change can also be attributed to the intersectional qualities: the examples are mixed-identity entities. Some clarification on how groups are treated and defined is necessary.
  6. In lines 270–273: when additional "evidence" is introduced, do the authors make an effort to ensure that the ground truth remains valid? Presumably, there may be instances where new "evidence" reasonably warrants a change in model decisions. Why do the authors consider a drift in decisions under new evidence an undesirable behavior?
  7. In RQ1: what makes FairStress comparable to BBQ? Do the authors ensure the datasets use the same demographic groups, the same approach to annotating stereotypes (majority and minority groups), and/or the same tasks? Ensuring comparable quantities is necessary to claim that "FairStress captured what BBQ does not." More generally, "what previous benchmarks miss" is an overclaim, given that the analysis only uses BBQ.
  8. Was the trained logistic regression model used to classify qualities in the generated reasoning, and if so, how? Is it validated by humans?
  9. How are the expected stereotypes labeled? Stereotypes need to be context dependent: in hiring, women can be (stereotypically) disadvantaged for engineering roles but advantaged for nursing roles. Did the stereotype annotation process account for this, and if so, how?

Suggestions to improve the paper (these do not affect my assessment or score)

  1. I would encourage adding an example scenario in the main body for clearer contextualization.
  2. In line 79, "robustness asymmetry" is not defined when it is first mentioned in RQ2.
6 comments
AuthorsOpenReview ↗

Response to Reviewer ce6X (1/3)

Thank you for a detailed and constructive review. We are glad you found our work "important", the benchmark built "carefully, with valid justifications", and the validation of LLM judgments "much appreciated and timely". Below, we address each concern with more analyses and clarifications; all changes are marked in red in the revised PDF.


TL;DR

  • Novelty (W1): FairStress is novel in three ways: a validated correct answer, mirrored pressure, and one decision traced through four levels. We add all five suggested papers to Related Work.
  • Weakly corroborated claims and under-specification (W2, Q1–Q9): We have added the settings you asked about, and they all confirm our results: sampling (Q3), R1-style models (Q4), single-axis contrasts (Q5), mirrored flip rates (Q6), blinded judges (Q8), role stereotypes (Q9).
  • Substantive equality (Q2): FairStress measures equality (grounded in the consistency rule of Leventhal's procedural justice); models keep the asymmetry even under explicit equality and equity policies.
  • BBQ (Q7): We narrow the claim to "What BBQ-style scoring misses", supported by an analysis of BBQ's own items.

W1. Novelty: models that look fair still abandon correct decisions more readily for one group

Limited novelty; the related work misses key relevant papers.

Thank you for suggesting these papers; they helped us sharpen our positioning. We add all five papers to Related Work (Sec. 4), which now positions FairStress in three paragraphs: fairness benchmarks and robustness under pushback ([1]), reasoning and overcorrection ([2]–[5]), and alignment and safety, each ending with what FairStress adds. Each of these works studies one part of the problem on its own, and the Introduction now ends with our three design choices (Sec. 1). FairStress is novel in three ways:

  1. A validated correct answer makes overcorrection measurable. [5] and DiscrimEval measure preferences on tasks without a correct answer. With one, overcorrection becomes an error with a cost: models stay nearly fair in accuracy (Accuracy Gap +2.9) but give up correct answers under a one-sentence push (Robustness Gap +9.3, positive in all 15 models; Sec. 3.1).
  2. Mirrored pressure gives every flip a direction. [1] asks whether jailbreak-style prompts can trigger a stereotype. Mirroring every push and identity asks in whose favor a model gives way: the same push overturns 16.0% of correct answers when it favors the minority-coded candidate vs. 5.6% when it favors the majority-coded one, but 9.8% vs. 9.9% when no identity is shown (App. D.9). The asymmetry also holds on real résumés (App. D.15).
  3. One decision is traced through four levels. [2]–[4] study reasoning or explanations on their own. Scoring the same decision in its answer, under pressure, in its reasoning and in its activations shows what a single metric masks: on ties, reasoning cites identity more often for the minority and merit for the majority (Sec. 3.3), and Tulu-3.1 looks fairer than Tulu-3 on ties yet loses 6.1 points of accuracy (Table 87), showing where a training change helps and where it hurts.

So a model can look fair on accuracy while abandoning correct answers for one group under a one-sentence push, which a single-metric audit would miss.

With this design, we also find that current fixes and audits fall short: debiasing methods and explicit fairness policies do not reliably remove the asymmetry (Sec. 3.2, App. G.8), BBQ's stereotyped verdict depends largely on scoring (App. E.1.1), and five post-training datasets frame groups asymmetrically (App. I.7). This points future alignment work to better evaluations, post-training data and reward mechanisms.


W2. Weakly corroborated claims: each question below is now specified in the paper and backed by additional evidence.

Some claims are weakly corroborated, and some setup details are missing.

We agree, and we now specify every detail you asked about. Each answer below points to where it is in the paper, and six come with a new analysis, all of which confirm our results: sampling (Q3), R1-style models (Q4), single-axis contrasts (Q5), mirrored flip rates (Q6), blinded judges and word counts (Q8), and role-typed hiring (Q9).

Q1. The "steps" at l.302 are the five stages of App. E.1.1.

What are the "steps" in line 302?

They are the evaluation stages S1–S5 (Table 43): S1 is BBQ multiple choice scored by log-likelihood, and S5 is a forced yes/no answer generated with the chat template. Moving from S1 to S5 shows that BBQ's stereotypical results depend largely on scoring choices (Q7). We now name the stages in the main text (Sec. 3.1), thanks!

AuthorsOpenReview ↗

Response to Reviewer ce6X (2/3)

Q2. FairStress measures equality and is different from equity.

Ground the position in a definition of fairness; under substantive equality, evaluating candidates differently can make sense.

We agree, and thank you for raising this important distinction. FairStress is grounded in Leventhal's consistency rule of procedural justice (Leventhal, 1980), which requires that the same procedure be applied to both groups. It does not capture equity-based views. We state this scope in Sec. 1 and the ethics statement and create a new Sec. 5, "Future Directions for Symmetrical Fairness", pointing to equity-aware evaluation (e.g., Wang et al., 2025). To test whether the asymmetry can be controlled by an explicit fairness policy, we ran this experiment on Type-1 contrasts at D3 (App. G.8):

  • Policy: ignore demographics (equality). 15 of 17 still lean toward the minority on ties (52.3–65.4%, CIs above 50%).
  • Policy: prefer the candidate from a historically disadvantaged group only when the two are equally strong (equity). Six models also apply it to clear-cut cases (Accuracy Gap up by 9.5 to 51 points), and 15 of 16 open models still give up correct answers asymmetrically under pressure (Robustness Gap +7.1 to +40.4).
  • Policy: organization values historical disadvantage (equity). 15 of 17 lose accuracy on clear-cut cases (by up to 12.9 points), and the Robustness Gap rises in 14 of 17.

Q3. Decoding settings are stated, and results are stable under sampling.

Does sampling stochasticity matter, and which temperatures are used?

Open-weight models are decoded greedily, OpenAI models at temperature 0, and Anthropic models with provider-default sampling (Sec. 2.2).

  • Sampling check. We reran four open models (Gemma-3-12B, OLMo-3-7B, Qwen2.5-7B, Llama-3.3-70B) at temperatures 0.3, 0.7 and 1.0, with three seeds each: 36 sampled runs on mirrored D3 items (App. D.10, Table 39).
  • No metric changes sign in any of the 36 runs. Sampling changes the size of the effect somewhat, but never its direction: | Metric | Sampled (36 runs) | Greedy | |---|---|---| | Tie-Break Lean | 54.9–66.5% | 59.4–65.1% | | Robustness Gap | +7.7 to +12.1 | +7.7 to +12.2 | | Tie-Break Shift | +14.5 to +40.9 | +22.8 to +40.4 |

So the asymmetry does not depend on how we decode. Thank you for raising this!


Q4. The asymmetry holds for R1-style reasoning models.

Do the reasoning claims hold for R1-style reasoning models?

Thank you for the great question! We ran DeepSeek-R1-Distill-Llama-8B, DeepSeek-R1-Distill-Qwen-14B, gpt-oss-20b (low effort) and Qwen3-8B with and without thinking (App. H.10):

  • Decisions. All five runs answer clear-cut cases correctly (95.7–99.0%), lean toward the minority candidate on ties (52.4–69.0%), and have a positive Robustness Gap (+7.1 to +20.6) and Tie-Break Shift (+5.3 to +22.3). Thinking reduces the asymmetry but does not remove it (Qwen3-8B: lean 69.0% → 58.2%).
  • Reasoning. Two LLM judges (Qwen2.5-72B, Llama-3.3-70B; agreement κ = 0.74) label whether each reasoning trace justifies the choice by merit or by identity. On ties, traces that pick the minority-coded candidate are judged identity-based 5.1 to 11.6 points more often than traces that pick the majority-coded one, in all four runs with reasoning and by both judges (Table 76).

Q5. FairStress pairs every axis alone and in combination, so each effect can be attributed.

Are the groups intersectional, and do the compared quantities vary minimally?

Some contrasts are intersectional: 17 of the 26 minority-vs-majority contrasts vary by one attribute, and the other 9 combine attributes that also appear alone (Table 8; App. D.12).

  • Muslim woman vs. atheist man. You are right that this pair cannot isolate gender. We treat it as a cross-axis control (Sec. 3.1) and attribute gender effects only from single-axis contrasts (App. D.12).
  • The main trend does not need intersections. The 17 single-axis contrasts alone reproduce the degree trend (Tie-Break Lean 50.0, 52.0, 54.6, 58.1% from D0 to D3).
  • Intersections are close to the sum of their parts. At D3, race alone (Black vs. white, same gender) moves ties toward the minority-coded candidate by +15.1 points and gender alone (woman vs. man, same race) by +4.4; together they predict +19.5, and Black woman vs. white man is +19.7. Interestingly, this differs from what intersectionality theory describes for people, where discrimination at an intersection is not simply the sum of its parts (Crenshaw, 1989).
AuthorsOpenReview ↗

Response to Reviewer ce6X (3/3)

Q6. The Robustness Gap compares mirrored directions, so justified changes balance each other.

Does the ground truth stay valid under new "evidence", and why is a drift in decisions undesirable?

We agree that new "evidence" may legitimately change a decision; the Figure 1 push is also of this kind. FairStress does not measure whether the push works; it measures whether it works equally no matter who benefits. The ground truth is validated without any prefix (App. B.6), and the prefix never edits the profiles (App. D.9):

  • Without identity (D0), it cancels: a directional prefix overturns 9.8% of correct answers when it favors one candidate and 9.9% when it favors the other.
  • With explicit identity (D3), the same push works about three times as often for the minority: 16.0% vs. 5.6%, higher in all 15 models (1.7× to 6.0×). Evidence prefixes move 28.3% vs. 10.4%!

We have clarified and added these rates to Sec. 3.1.


Q7. We narrow the claim to "What BBQ-style scoring misses".

What makes FairStress comparable to BBQ? "What previous benchmarks miss" is an overclaim.

You are right that FairStress and BBQ differ in groups and tasks, so we do not compare their headline numbers, and we retitled the paragraph (Sec. 3.1). What we found is a construct-validity concern for BBQ-style scoring:

  • Why BBQ: it is one of the main fairness benchmarks that frontier system cards report (App. E.2), and it shows most models as fair to slightly stereotypical (Table 42).
  • What we found: all four models we test score as stereotyped by 6.6 to 8.2 points when BBQ is run as usual, and changing only template, wording and scoring moves them to 0.0 to +2.0 (App. E.1.1, Table 45).
  • Why it matters: an audit that relies on one scoring setup can miss how models actually decide, so we hope this encourages developers to use more diverse fairness audits, especially in high-stakes settings.

Q8. The logistic regression predicts which candidate was chosen.

How is the logistic regression used on the reasoning, and is it validated by humans?

We clarify this in Sec. 3.3:

  • What it does. The logistic regression does not label qualities in the reasoning; it tests whether the reasoning reveals which candidate the model chose. Each trace is scored on 264 word categories from 15 published lexicons (e.g., LIWC, NRC-VAD, eMFD, Empath), one score per category, and the classifier predicts the chosen candidate from these scores on held-out traces (App. H.2). If the model justified both choices the same way, accuracy would stay at chance (0.50).
  • Why no annotation is needed. The label is the model's own answer. Where human judgment is needed, we use it: three annotators confirm our ground-truth and tie labels (94% of 100 clear-cut pairs and 90% of 100 ties; App. B.6).
  • "Confident and positive" and "relational" are not classifier outputs. They name the two principal components of the lexicon scores, after the categories that load on them (App. H.1).

We also add two independent checks (App. H.9):

  • Three blinded LLM judges (Claude Sonnet 5, Qwen2.5-72B, Llama-3.3-70B) compare 1,000 trace pairs in both orders, with names and identity labels removed. On ties, all three judge the minority-chosen trace as relying more on identity (67–75% of the pairs that differ; Fleiss κ = 0.49, moderate agreement).
  • A word count: on ties at D3, 9.9% of minority-chosen traces use an identity-framing word vs. 5.6% of majority-chosen ones, higher in 11 of 11 models.

Q9. The lean toward the minority is not context dependent.

How are stereotypes labeled? They are context dependent (e.g., engineering vs. nursing roles).

FairStress does not label a stereotype per item: each contrast has a fixed minority- and majority-coded group, and "stereotypical" means favoring the majority-coded candidate. To test context dependence (App. D.13), we split woman-vs-man hiring contrasts by roles where most U.S. workers are women or men (BLS, 2025).

Models favor the woman on ties in female-typed roles (e.g., nurse: 60.1%) and about as much in male-typed roles (e.g., electrician: 58.5%); no model favors the man there. Pressure moves male-typed roles toward the woman even more (Tie-Break Shift +32.8 vs. +17.4)!

Thank you for this question!


Suggestions

Thank you; we add a short example of a GT case, an AMB case and the mirrored item to Sec. 2.1 (full examples in App. K.2), and define "robustness asymmetry" at l.79.


We hope these answers address your concerns. If so, we would be grateful if you would consider raising your score. If you have any further concerns, please let us know and we would be happy to address them.

Reviewer_w5bDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The FairStress benchmark tests LLM fairness across 15 models. It evaluates how decisions hold up under pressure and looks at model reasoning styles. The main finding shows that alignment methods like RLHF and DPO can cause overcorrection, leading them to drop good decisions.

Critical strengths
  1. The paper highlights an issue that LLM alignment may cause overcorrection and bias. It introduces a distinction between distributive and procedural fairness, and then it shows that alignment training can teach models to mask their decision processes instead of acting fairly.

  2. The benchmark evaluates 15 open and API models , on 29 high-stakes domains, and using a 13 million item corpus, I think it is very comprehensive. It also uses four levels of identity signaling to analyze how demographic variables trigger bias.

  3. The evaluation introduces several methodological improvements. It tests decision firmness under adversarial pressure. The guardrails against metric gameability ensure that the metrics measure what they claim to measure

  4. The appendix is thorough and comprehensive, providing almost every detail, aiding reproducibility and verification.

Critical weaknesses
  1. The paper assumes a strict merit-based fairness definition, treating any model overcorrection as a flaw. But in high-stakes fields like law and public policy, adjusting for historical disadvantage is often an intentional strategy.

  2. Adversarial LLM bias testing already exists. The paper criticizes prior benchmarks, but overlooks recent work on counterfactuals. Besides, adding an adversarial sentence is just a minor engineering tweak, not a methodolygy improvement.

  3. The reasoning analysis uses 264 lexicon categories to predict candidate selection, but the actual effect size d is 0.044. Claiming that reasoning style reveals the chosen candidate seems to overstate the practical importance of tiny word choices.

  4. The proposed benchmark evaluates 15 models for overcorrection but audits the training data for only OLMo3. So blaming OLMo3's specific training mix for the behavior of closed models (GPT4o or Llama3.1) is unfair.

  5. The abstract said that an added sentence causes models to abandon correct answers for majority candidates. However, the data for minority-minority tests shows a negative gap -2.3. The work didnt define or predict this effect.

Questions for authors

Please respond to my concerns in the weaknesses section

3 comments
AuthorsOpenReview ↗

Response to Reviewer w5bD (1/2)

Thank you for the careful and encouraging review! We are glad you found that our paper “introduces a distinction between distributive and procedural fairness”, that our benchmark is “very comprehensive”, that our evaluation “introduces several methodological improvements”, with guardrails that “ensure that the metrics measure what they claim to measure”, and that our appendix is “thorough and comprehensive”. Below, we address each concern with more analyses and clarifications; changes are in red in the revised PDF.


TL;DR

  • W1, equity vs. equality: FairStress measures equality (Leventhal's procedural consistency); we clarify this in Sec. 1, 5 and the ethics statement. Under explicit equality or equity policies, most models still show the overcorrection asymmetry.
  • W2, novelty: FairStress asks a question that adversarial and counterfactual benchmarks can't answer alone; see the three ways it is novel below.
  • W3, effect size: small per trace, but the same direction in all 11 models; interpretable checks agree.
  • W4, training data: stated as a hypothesis, and 4 other post-training datasets reproduce the same pattern.
  • W5, minority-vs-minority gap: clarified as a control; its direction depends on which minority models rate as more disadvantaged.

W1. FairStress measures equality; equity is an important and different goal.

The overcorrection can be intentional.

We thank you for this important distinction and we agree. FairStress is grounded in Leventhal's procedural justice (Leventhal, 1980), whose consistency rule requires the same procedure for every person. This is an equality view; we now state this distinction in Sec. 1 and the ethics statement, and add Sec. 5, "Future Directions for Symmetrical Fairness", where we point to equity-aware evaluations such as Wang et al. (2025) as complementary work to FairStress. To test whether models would follow an intentional equality vs. equity policy, we gave 17 models explicit policies on Type-1 contrasts at D3 (App. G.8):

  • Policy: ignore demographics (equality). 15 of 17 still lean toward the minority on ties (52.3–65.4%, CIs above 50%).
  • Policy: prefer the candidate from a historically disadvantaged group only when the two are equally strong (equity). 6 models also apply this preference to clear-cut cases (Accuracy Gap up by 9.5 to 51 points), and 15 of the 16 open models still give up correct answers asymmetrically under pressure (Robustness Gap +7.1 to +40.4).
  • Policy: organization values historical disadvantage (equity). 15 of 17 lose accuracy on clear-cut cases (by up to 12.9 points), and the Robustness Gap rises in 14 of 17.

W2. What is new: measuring asymmetry at four levels of one decision

Adversarial bias testing already exists.

Thank you. We agree our positioning was too brief and have expanded Related Work (Sec. 4) and the Introduction (Sec. 1). FairStress is novel in three ways:

  1. A validated correct answer makes overcorrection measurable. Counterfactual benchmarks such as DiscrimEval have no correct answer. With one, models stay nearly fair in accuracy (Accuracy Gap +2.9) but give up correct answers under a one-sentence push (Robustness Gap +9.3, positive in all 15 models; Sec. 3.1).
  2. Mirrored pressure gives every flip a direction. Adversarial benchmarks such as FLEX test whether jailbreak-style prompts trigger a stereotype. Mirroring every push and identity shows in whose favor a model gives way: at D3 the same push overturns 16.0% vs. 5.6% of correct answers, but 9.8% vs. 9.9% without identity (App. D.9).
  3. One decision is traced through four levels. Scoring the answer, pressure response, reasoning and activations of the same decision shows what one metric masks: Tulu-3.1 looks fairer than Tulu-3 on ties, yet its RL change costs 6.1 points of accuracy (Table 87), showing where a training change helps and where it hurts.

So a model can look fair on accuracy while abandoning correct answers for one group under a one-sentence push, an alignment concern that FairStress can help detect.

We add more counterfactual evaluations (Garg et al., 2019; Huang et al., 2020; Wang et al., 2024; Eloundou et al., 2025) and Kantharuban et al. (2025) (Sec. 2.1 and 4) to the audits we already cite (Yin et al., 2024; Wilson & Caliskan, 2024; An et al., 2024). If you had other work in mind, we'd love to position ourselves around it.

The added sentence is a minor engineering tweak.

We agree that the prefix is simple, and that is our deliberate design choice. We asked what the smallest push is that can make a model abandon the correct answer for its bias. We found that even a simple placebo moved tied decisions toward the minority (+4.2 points at D3), and a simple directional prefix moved them by +19 to +34 (Table 25). If a small push already reveals a clear asymmetry, stronger adversaries built on FairStress may reveal even larger gaps.

AuthorsOpenReview ↗

Response to Reviewer w5bD (2/2)

W3. The effect is small per trace, with the same sign in every model

An effect size of d = 0.044 makes the reasoning claim an overstatement.

We agree that the per-trace effect is small, and we now say so in Sec. 3.3; the Table 5 title becomes "Reasoning reveals the chosen candidate weakly but consistently". Our claim rests on the consistency of this effect: the shift has the same sign in all 11 open models (sign test, p = 0.0005). Because a d over 264 lexicon categories is hard to read, we add interpretable measures (App. H.9):

  • On tied items at D3, 9.9% of reasoning traces use at least one identity-framing word when the model picks the minority, vs. 5.6% when it picks the majority, higher in 11 of 11 models.
  • Three blinded LLM judges rate the minority-chosen trace as more identity-based (67–75% of the pairs that differ).

W4. The training-data audit is a hypothesis, and it replicates on four more datasets

Only OLMo-3's data is audited, but it is used to explain closed models.

We agree. Very few developers have released post-training data at all, so we can only hypothesize, and we now state this as a hypothesis in Sec. 3.4. But this is a recurring pattern in 4 other post-training datasets, which makes our hypothesis stronger. We ran the same audit on Tulu-3 SFT, HH-RLHF, UltraFeedback and SmolTalk (23,111 passages; App. I.7), and the asymmetrical framing replicates in each dataset. Share of passages framing a group positively:

Post-training dataMinority-coded groupsMajority-coded groups
OLMo-3 (fully open)36.4%11.3%
Tulu-3 SFT38.4%15.5%
HH-RLHF25.9%6.7%
UltraFeedback37.1%13.6%
SmolTalk40.2%11.7%
Four datasets pooled34.1%10.1%

Among groups with at least 500 passages, positive framing is rarest for men (4.5%), Christians (5.4%) and White people (7.2%), and far higher for disabled (62.1%), transgender (50.0%) and Black people (40.9%).

Since the pattern holds across five open datasets, other post-training datasets might also share it, but we can't confirm this. In Sec. 5 we call on developers to run this audit on their own post-training data. If it holds, it tells a strong story on how we need a change in how post-training datasets frame groups.


W5. The minority-vs-minority gap is a control, and its direction is predictable

The Type-2 gap of −2.3 is not defined or predicted.

Minority vs. minority pairs are included as a control, as neither side is clearly historically advantaged (App. A). Their gaps are smaller (about a quarter: Robustness Gap −2.3 vs. +9.3, Sec. 3.1), and the sign only shows which minority a model favors. The clear asymmetry we observe comes from the contrast between minority vs. majority pairs, and we now clarify this in the abstract.

You asked a very interesting question. To predict the direction, we asked models to rate how much disadvantage each group faces (App. D.14):

  • In 6 of the 7 minority-vs-minority pairs, models lean toward the group they rate as more disadvantaged.
  • In 7 of 7, the Robustness Gap favors the group they find more disadvantaged.
  • Across all 33 pairs, the more disadvantaged a group is rated relative to the other, the more models lean toward it (Pearson r = 0.88) and the larger the Robustness Gap in its favor (r = 0.93).

We hope these clarifications address your concerns. If so, we would be grateful if you would consider raising your score. We would love to hear your thoughts and address any further concerns you might have.

AuthorsOpenReview ↗

Gentle reminder

Dear Reviewer w5bD, thank you again for your review. We hope our response has addressed your concerns, and since the discussion period ends on November 18, we wanted to share a short summary:

  • Equity vs. equality (W1): we state our equality scope and show that most models keep the asymmetry even under explicit equality or equity policies.
  • Novelty (W2): FairStress is novel in three ways: a validated correct answer, mirrored pressure, and one decision traced through four levels.
  • Effect size, data and Type-2 gap (W3–W5): we add interpretable checks, replicate the data audit on four more datasets, and show that the Type-2 direction is predictable.

If any concern remains, please let us know and we will address it before the deadline. Otherwise, we would be grateful if you would consider updating your score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank the reviewers for their careful and constructive reviews, and for recognizing our benchmark as "very comprehensive" with well-validated metrics (w5bD), "important" and "carefully" designed (ce6X), and our experiments as "comprehensive", offering "valuable insights" and "significant practical value" (FcHv). Below, we summarize our rebuttal and how we address each concern. All changes are in red in the revised PDF.


TL;DR

  • What FairStress shows: By looking at different levels of a decision, we investigate asymmetrical unfairness in LLMs: models look nearly fair in accuracy, yet give up correct answers more readily when a one-sentence push favors the minority-coded candidate (Robustness Gap +9.3 vs. Accuracy Gap +2.9 at D3, in all 15 models).
  • Novelty: The Introduction now ends with an itemized list of what is new (Sec. 1), and Related Work cites every work the reviewers named (Sec. 4).
  • Soundness: The settings the reviewers asked about are now in the paper, and the main findings hold under every additional check, including sampling, R1-style reasoning models, a real field experiment, and four more training datasets.

Our Novelty

Our main finding, which to our knowledge was not known before, is that how easily a model gives up a correct decision depends on who benefits. FairStress builds on prior work on adversarial pressure, reasoning and counterfactual fairness, and is novel in three ways:

  1. A validated correct answer makes overcorrection measurable. Models stay nearly fair in accuracy (Accuracy Gap +2.9), yet a one-sentence push makes them abandon correct answers far more readily to favor the minority-coded candidate (Robustness Gap +9.3, positive in all 15 models; Sec. 3.1).
  2. Mirrored pressure gives every flip a direction. The same push overturns correct answers about three times as often when it favors the minority-coded candidate (16.0% vs. 5.6%), but equally often without identity (9.8% vs. 9.9%; App. D.9), and the asymmetry holds on real résumés (App. D.15).
  3. One decision is traced through four levels. Following the same decision through its answer, pressure, reasoning and activations shows what a single metric masks: on ties, reasoning cites identity more often for the minority and merit for the majority (Sec. 3.3), and Tulu-3.1 looks fairer than Tulu-3 on ties yet loses 6.1 points of accuracy (Table 87), showing where a training change helps and where it hurts.

So a model can look fair on accuracy while abandoning correct answers for one group under a one-sentence push, which a single-metric audit would miss.

Why it matters: current fixes and audits miss this. Debiasing methods and explicit fairness policies do not reliably remove the asymmetry (App. G.8). BBQ, which frontier system cards report (App. E.2), looks fair to slightly stereotyped, yet for the four models we analyze its stereotyped scores largely disappear when only prompt format and scoring change (App. E.1.1). Five post-training datasets also frame groups asymmetrically (App. I.7). FairStress gives evaluation, alignment and post-training data design a way to check that models decide fairly, not only look fair.


Equality vs. equity framing

FairStress is based on Leventhal's procedural justice and its consistency rule, which implies that fairness requires applying the same procedure to every person, which is about equality. But we also believe equity-based work is important and complementary to FairStress. We emphasize this in Sec. 1, the ethics section, and the new Sec. 5, "Future Directions for Symmetrical Fairness". What makes our work more alarming is that, even when given explicit fairness policies, most models keep the asymmetry (App. G.8).


New analyses for requested settings (all in red)

  • Real résumés and job ads from a field experiment (App. D.15): FcHv-W3
  • Sampling at three temperatures (App. D.10): ce6X-Q3
  • R1-style reasoning models (App. H.10): ce6X-Q4
  • Blinded LLM judges and interpretable-word counts (App. H.9): w5bD-W3, ce6X-Q8
  • Mirrored flip rates under pressure (App. D.9): ce6X-Q6
  • Explicit equality and equity policies (App. G.8): w5bD-W1, ce6X-Q2
  • Degree trend without age and disability; name recognition (App. D.11): FcHv-W1/Q1
  • Single-axis vs. intersectional contrasts (App. D.12): ce6X-Q5
  • Female- vs. male-typed roles (App. D.13): ce6X-Q9
  • Minority-vs-minority direction (App. D.14): w5bD-W5
  • Framing audit on four more post-training datasets; the gap holds in each (App. I.7): w5bD-W4, FcHv-W2
  • Mirror-complete Core-M (App. D.8): FcHv-Q2

We thank the reviewers again. Their questions made our paper more precise, and we are excited about the directions FairStress opens for alignment and procedural fairness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.