- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper focuses on the construction of the FairStress benchmark to investigate the phenomenon of asymmetric unfairness in Large Language Models (LLMs). Specifically, the authors argue that merely observing a model's final decision may fail to fully reveal its unfairness; even if the final output appears relatively fair, the processes of making, maintaining, and justifying that decision can still be influenced by demographic attributes. To explore this, the authors developed a benchmark requiring LLMs to choose between two candidates across 29 decision-making domains, including recruitment, healthcare, and the judicial system. First, GPT-4o-mini was used to generate 300 base scenarios according to predefined rules, mostly presented in two forms: (1) GT (Ground Truth): one candidate’s merit is clearly superior to the other’s, implying a "correct answer"; and (2) AMB (Ambiguous): both candidates have distinct strengths and weaknesses, with no objectively correct answer. Demographic attributes were then swapped between the two candidates across varying levels of identity signals (D0–D3). Additionally, the authors designed 78 adversarial prefixes to test whether the models would alter their initial decisions when exposed to additional information. The raw dataset comprised approximately 13 million test items; Item Response Theory (IRT) was subsequently used to select about 50,000 items to form "FairStress-Core."
Using these datasets, the authors conducted experiments across four dimensions: Decision Fairness, Robustness Fairness, Reasoning Fairness, and Internal State. The results indicate that while group disparities in final decision accuracy were relatively minor, the models exhibited significant "anti-stereotypical overcorrection" under external pressure—a phenomenon that typically intensified as demographic attributes became more explicit. The paper also evaluated existing debiasing methods, finding that most failed to consistently mitigate this asymmetry, while some incurred a significant cost in accuracy. Finally, the study further characterized these asymmetric behaviors and explored their potential causes through analyses of reasoning language, internal activations, and training data audits.
Critical strengths
-
Clear narrative arc and logical progression: The paper presents a coherent narrative centered on the phenomenon of asymmetric unfairness in LLMs. It moves systematically from problem definition to the proposal of an evaluation framework (including benchmark construction and data synthesis), and then delves deeper—progressing from direct decision outcomes to the effects of perturbations, reasoning/explanation analysis, and finally, internal model activations. The experimental design is well-structured and effectively addresses the core research questions. The writing is profound yet accessible, and the figures serve as excellent aids to understanding. I particularly appreciate Figure 1 and the "takeaway" sections for the research questions; they enhance the overall presentation and ensure the paper remains internally consistent regarding the issues it addresses.
-
Comprehensive experiments: The paper goes beyond superficial, outcome-oriented evaluations of fairness. It designs a series of in-depth experiments to examine asymmetric robustness under stress, asymmetries in model reasoning, and demographic-related signals within internal model activations. Furthermore, the paper combines experimental results with reasoned arguments to offer plausible explanations for the observed fairness phenomena, thereby providing valuable insights to the fairness research community.
-
Valuable research problem: Decision outcomes that appear fair can indeed mask deeper-seated fairness issues. While much current AI fairness evaluation focuses on final results, this paper highlights two critical points: 1) Latent fairness risks beneath the surface outcomes can cause seemingly fair results to deteriorate under different conditions (such as the stress scenarios explored here or in entirely different contexts); and 2) In many human-in-the-loop decision-making scenarios, stakeholders consider not only the final outcome but also the model's explanations and reasoning traces. Consequently, uncovering these deeper fairness risks holds significant practical value.
Critical weaknesses
-
There are potential issues regarding the comparability of demographic identity signal strengths: The paper categorizes identity information into four levels (D0–D3) representing increasing specificity; however, the actual construction reveals inconsistencies in how identity signals manifest across different demographic attributes. For instance, while D1 relies on names to indirectly imply attributes like race and gender, it employs explicit descriptions for age and disability status—to the extent that the D1 and D3 representations for these groups become effectively identical. Furthermore, the strength of identity cues may also vary by name; some names are strongly associated with specific groups, while others offer weaker signals, potentially introducing additional noise. Although the authors outline specific handling measures and data quality controls in the appendix, the comparability of signal strengths across different demographic attributes lacks sufficient validation. This limitation may, to some extent, affect the interpretation of the relationship between identity information salience and asymmetric fairness.
-
The paper’s novelty is relatively limited; its research positioning and contributions could be more clearly defined: Several key research themes addressed in this paper—such as the potential misalignment between outcome fairness and process fairness, model fairness robustness under adversarial perturbations, and anti-stereotypical overcorrection—have already been explored in existing literature. The four evaluation levels proposed here also rest on well-established research foundations. Consequently, the paper’s primary contribution appears to lie in systematically organizing existing fairness issues into a unified evaluation framework and providing further empirical evidence through comprehensive, in-depth experiments, rather than introducing entirely new research questions or discovering these phenomena for the first time. While this does not diminish the value of the experiments, it is recommended that the authors provide a more thorough discussion of related work—particularly recent studies on the divergence between outcome and process fairness and anti-stereotypical overcorrection—within the Introduction and Related Work sections to better articulate the paper's incremental contributions relative to existing research.
-
Complete reliance on synthetic decision scenarios, lacking external validation against real-world data: The foundational decision scenarios in FairStress were entirely generated by GPT-4o-mini based on predefined rules. Although the authors conducted both automated and manual verification, these checks primarily ensured adherence to experimental parameters rather than establishing a robust link to real-world decision-making contexts. In particular, the paper presents decision scenarios in two forms—GT (scenarios with a clear correct answer) and AMB (scenarios where both options have distinct advantages and disadvantages). While this design facilitates variable control and experimentation, it somewhat idealizes the nature of actual decision-making problems. Real-world domains such as recruitment, healthcare, and the judicial system often involve far more complex conditions and information distributions that do not necessarily map neatly onto these two categories. While the experimental results based on controlled synthetic data are comprehensive, the absence of independent validation using real-world decision cases leaves some uncertainty regarding the generalizability of the findings and the actual severity of the observed unfairness. Therefore, it is recommended that the authors more clearly define the scope of applicability for their conclusions and further discuss this limitation.
Questions for authors
-
Appendix B.3 notes that explicit identity descriptors are used for age and disability groups starting at level D1, and that D3 is identical to D1. How do the authors address the issue of non-comparable identity signal strengths across these different demographic attributes? Based on existing experimental results, could the authors provide the primary trends for D0–D3 with age and disability contrasts excluded, in order to verify the robustness of the conclusions drawn from Figure 2?
-
In the full FairStress dataset, the "Tie-Break Lean" for D0 is reported as 50.0%; however, in the debiasing experiments using FairStress-Core, the D0 baselines for the three models reach 55.9%, 56.2%, and 58.2%, respectively (Appendix Tables 53–55). How do the authors explain this discrepancy? Did the IRT-based selection process for FairStress-Core preserve the complete identity-mirroring pairs, and does this affect the interpretation of the relevant fairness metrics?
5 comments
Response to Reviewer FcHv (2/3)
W2. What is new: FairStress shows toward whom a model gives up a correct answer
The themes have prior work; related work should be discussed more fully.
Thank you for pushing us to sharpen our positioning. We agree that each theme and each of the four levels has precedent.
- Related Work (Sec. 4) now covers the two areas you name: outcome vs. process fairness (Popoola & Sheppard, 2026; Rajwal et al., 2025; Wu et al., 2025) and anti-stereotypical overcorrection (Kumar et al., 2026; Rozado, 2025; Tamkin et al., 2023).
- The Introduction now ends with an itemized list of what is new.
FairStress combines three design choices to answer a new question: does a model hold a correct decision equally for both groups, and does an identical push have asymmetrical power depending on its direction?
- A validated correct answer. Overcorrection studies measure preferences on tasks without a correct answer (Kumar et al., 2026; Tamkin et al., 2023). With one, overcorrection becomes a measurable error and shows how much performance a model is willing to sacrifice: models stay accurate on clear-cut cases but asymmetrically give up correct answers under small adversarial prefixes (Sec. 3.1).
- Mirrored pressure. Robustness benchmarks ask whether a model gives way under pressure, for example whether it can be jailbroken (FLEX; Sharma et al., 2025). By mirroring every push and identity, we give each failure a direction and ask for whom the model changes its answer: the Robustness Gap (+9.3 at D3, positive in all 15 models) is about three times the Accuracy Gap (+2.9; Sec. 3.1), and the same asymmetry holds on a real field experiment's résumés (W3). This shows that unfairness can appear in different ways, and that robustness is asymmetrical across demographics.
- One decision at four levels. Scoring the same decision in its answer, under pressure, in its reasoning and in its activations shows where a model looks fair on one level but not another, so a single score can mislead. Together, these levels show how different models or training algorithms affect each level.
For example, Tulu-3 and Tulu-3.1 differ only in their RL stage. Tulu-3.1 leans less on ties than Tulu-3 (52.0% vs. 57.4%), so on this metric it looks fairer, and one might conclude that the RL change made it fairer. But FairStress's four levels show that it loses 6.1 points of accuracy and nearly doubles its Accuracy Gap (Table 87). A single metric would not show the full picture. The reasoning level also shows that this overcorrection does not actually serve minorities: on ties, models justify picking the minority candidate by identity and the majority candidate by merit (Sec. 3.3). So FairStress reveals an interesting phenomenon: the overcorrection in the answers comes with reasoning that overlooks minority candidates' qualifications and hard work (Sec. 5).
We also find a matching asymmetry in post-training data (App. I.7). As prompted by another reviewer, we checked whether the asymmetric framing in OLMo-3 replicates, and it does in 4 other post-training datasets! Share of passages framing a group positively:
| Post-training data | Minority-coded groups | Majority-coded groups |
|---|---|---|
| OLMo-3 (fully open) | 36.4% | 11.3% |
| Tulu-3 SFT | 38.4% | 15.5% |
| HH-RLHF | 25.9% | 6.7% |
| UltraFeedback | 37.1% | 13.6% |
| SmolTalk | 40.2% | 11.7% |
| Four datasets pooled | 34.1% | 10.1% |
- By group: men 4.5%, Christians 5.4%, White people 7.2%, against Black 40.9%, transgender 50.0% and disabled people 62.1%.
- What it suggests: prior work attributes overcorrection to alignment in general (Kumar et al., 2026); we hypothesize that this auditable property of the data may contribute (Sec. 5). This points to a potential limitation in how post-training datasets are designed, and we need more studies on how to design them!
Response to Reviewer FcHv (3/3)
W3. Real vs. synthetic inputs: on a real field experiment's résumés and names, models show the same asymmetry!
All scenarios are synthetic, and the GT/AMB design idealizes real decisions.
Thank you for this important point! We agree:
- Scope. In Sec. 5 we scope our conclusions to controlled two-candidate decisions. FairStress needs a controlled setting to measure the asymmetry cleanly at each level, and real decision records lack that control.
- A new real-world check (App. D.15). Encouraged by your comment, we use the Bertrand & Mullainathan (2004) field experiment: its real job ads, coded résumé attributes (education, experience, gaps, skills), first names, and the real employers' callbacks.
- Setup. Two blind judges select 600 clear-cut pairs from the same ad and 400 ties. We add names (D1) or names with explicit labels (D3), mirror every pair, and add four prefixes that never mention a group. Seven models: OLMo-3-7B, OLMo-3.1-32B, Qwen2.5-7B, Llama-3.1-8B, Gemma-3-27B, GPT-4o-mini, GPT-5.5.
Results:
- The cases work without identity: D0 accuracy on clear-cut pairs is 98.1–100%.
- Ties and pressure (D3): every model picks the Black-coded candidate more often on ties and gives way more readily to prefixes favoring them (Table 41). Each pair is a Black-coded vs. a white-coded candidate of the same gender:
| Model | Tie-Break Lean, % (women / men) | Robustness Gap (women / men) | Accuracy Gap (women / men) |
|---|---|---|---|
| OLMo-3-7B | 63.5 / 65.1 | +15.0 / +2.8 | +0.2 / +0.0 |
| Qwen2.5-7B | 67.7 / 64.5 | +38.1 / +30.1 | +0.8 / +0.3 |
| Llama-3.1-8B | 68.1 / 68.1 | +25.1 / +21.7 | +13.3 / +5.2 |
| Gemma-3-27B | 55.4 / 58.4 | +51.4 / +53.2 | +2.9 / +0.7 |
| OLMo-3.1-32B | 81.9 / 85.3 | +46.5 / +46.5 | +0.3 / +0.3 |
| GPT-4o-mini | 57.3 / 57.2 | +48.8 / +47.8 | +0.9 / +0.0 |
| GPT-5.5 | 62.9 / 73.3 | +18.2 / +17.6 | +8.3 / +14.1 |
- Severity: on clear-cut pairs, GPT-5.5 picks a weaker Black-coded résumé over a stronger white-coded one in 10.5% of cases, but a weaker white-coded résumé over a stronger Black-coded one in only 0.9%.
Q2. Clarifying the D0 baseline on FairStress-Core, and a matched estimate that confirms the effect
Why is D0 Tie-Break Lean 50.0% on the full corpus but 55.9–58.2% on FairStress-Core, and did IRT selection preserve mirrored pairs?
Thank you for this precise question!
- Why D0 differs. On the full corpus, D0 is exactly 50.0% by construction. FairStress-Core was selected item by item for informativeness, so IRT selection did not keep most mirrored pairs, and its D0 lean reflects which case profile a model prefers among the selected items, not identity. We now state this in Sec. 2.1 and App. D.8.
- A matched estimate on every Core item. We compare each item with its D0 twin (same case, candidate order and prefix, no identity) and measure how much more often the model picks the minority candidate once identity is shown. At D3 this effect is positive, with every 95% interval above zero, for all 12 frontier runs (6 models in answer and reasoning modes; +4.7 to +12.7 points) and for the three debiasing baselines (OLMo-3-7B +6.9, Gemma-3-12B +8.6, Qwen2.5-7B +10.2).
- Core-M, a mirror-complete subset (17,274 Type-1 D3 items, each case with both identity assignments, so case content cancels). We release it and run our new analyses on it: sampling (App. D.10), explicit policies (App. G.8) and R1-style reasoning models (App. H.10). The asymmetry holds in all three.
The full-corpus results are unaffected, and each debiasing method is compared with its own baseline on the same Core items. We recommend:
| Goal | Set |
|---|---|
| Exact mirrored comparisons | Core-M |
| Broad coverage | FairStress-Core |
| Most precise estimates | Full corpus |
Thank you again; your review made our claims more precise. We hope this addresses your concerns, and if so, we would be grateful if you would consider raising your score. If you have any further concerns, please let us know and we would be happy to address them.
Response to Reviewer FcHv (1/3)
Thank you for your careful and thorough review! We are glad you found our work to have a "clear narrative arc and logical progression", with "comprehensive experiments" offering "valuable insights to the fairness research community", and that "uncovering these deeper fairness risks holds significant practical value". Below, we address each concern with more analyses and clarifications; changes are marked in red in the revised PDF.
TL;DR
W1, Q1. The degree trend holds with age and disability removed, and name strength does not drive it
Thank you for this important point; we agree, and we move the App. B.3 note into the main text (Sec. 2.1).
Without age and disability, the trend is unchanged (Type 1, mean over 15 models; Table 40, App. D.11):
The trend stays monotone, and D3 barely moves. Disability alone raises the pooled D1 point (lean 68.8% from D1 on); age is near zero at every degree.
Name strength does not drive the effect. We asked all 15 models which group each of our 108 names belongs to, and correlated how strongly they recognize a name with its Tie-Break Lean at D1. Recognition varies across names, but it does not predict the effect (r = 0.04 over the 60 names with enough tie items; r = 0.10 within contrast; App. D.11).
So the degree trend depends neither on the age and disability descriptors nor on how recognizable a name is.