- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Interesting paper - The paper compares yes/no judgements with comparison between two targets using social-bias benchmarks. The paper analysis finds that models tend to show greater [references when asked to compare (especially with CoT), but the gap get smaller when the context provides enough information to answer. Neural options and random instructions are also tested. I think the main takeaway is the way we ask questions can substantially change the bias we measure - unsurprising finding, but good that we have a rigorous student to examine this.
Critical strengths
- S1: The main strength of this paper is that the research questions are well-structured with experiments designed to answer these important questions. The findings also shows that the prompt has substantial effects on the results.
Critical weaknesses
-
W1: My main concern is whether ISO and CMP are measuring sufficiently similar things to support the paper's interpretation. Asking whether a sentence is unusual and asking which of two sentences is more unusual are different judgment tasks. Technically, both sentences could receive the same yes/no answer while still being distinguishable in a comparison. - I suspect this design choice has a strong influence on how we should interpret the results.
-
W2: I think one of the key concern I have is the effect of abstention, and the interpretation of neural answers. For instance, on BBQ, some models abstain almost all the time (Table 4). Their remaining answers can still be biased, but abstention may nevertheless greatly reduce the number of biased decisions overall.
Questions for authors
-
Does the ISO–CMP gap persist with graded isolated judgments, rather than binary yes/no answers?
-
How do neutral options change unconditional disparity, and how often is PG determined by the no-definitive-answer default?
7 comments
W2 & Q2: Abstention and the parity default We thank the reviewer for this point. For W2 and the first part of Q2 (unconditional disparity), as also replied to Reviewer uMNo (R1.Q3), the first table below compares |PG(CMP)| with the neutral option computed only on the definitive answers (Def., as in Table 5) and on all the answers (All). The bias drops where models often abstain (e.g., BBQ, DiscrimEvalGen) and barely changes where they rarely do (StereoSet). We agree that abstention reduces the overall number of biased decisions and have revised our claims accordingly. Nevertheless, its practical effect depends on the deployment. Indeed, for example, when a selection of N candidates is required (e.g., hiring settings), a neutral answer only leads to querying the model again, so the biased decision is delayed rather than avoided.
|PG(CMP)|% with the neutral option, computed only on the definitive answers (Def.) and on all the answers (All), gender, CoT. Bold: statistically significant difference between Def. and All (p < 0.05).
| Model | SS (Def.) | SS (All) | RB (Def.) | RB (All) | BBQ (Def.) | BBQ (All) | DE-Gen (Def.) | DE-Gen (All) | DT-Tox (Def.) | DT-Tox (All) |
|---|---|---|---|---|---|---|---|---|---|---|
| DS-LMa-70B | 63.2 | 63.0 | 82.4 | 72.5 | 8.2 | 0.3 | 17.0 | 3.2 | 98.6 | 67.3 |
| DS-Qw-32B | 60.2 | 59.1 | 79.2 | 73.9 | 15.0 | 0.8 | 31.0 | 4.8 | 96.6 | 79.3 |
| LMa3.1-70B-I | 60.8 | 60.0 | 80.4 | 67.6 | 14.4 | 1.1 | 32.6 | 4.7 | 96.6 | 72.7 |
| LMa4-17B-16E-I | 59.6 | 59.4 | 71.8 | 71.6 | 21.4 | 10.7 | 35.0 | 19.3 | 99.6 | 99.7 |
| Min3-14B-R | 49.0 | 48.6 | 50.0 | 43.4 | 37.2 | 9.5 | 38.2 | 5.6 | 96.8 | 83.1 |
| OSS-120B | 65.6 | 64.6 | 76.8 | 70.0 | 10.6 | 0.9 | 1.4 | 1.6 | 99.8 | 51.9 |
| Qw2.5-32B-I | 59.0 | 58.3 | 83.2 | 62.7 | 0.4 | 0.0 | 24.2 | 3.2 | 96.6 | 84.1 |
| Qw3-30B-A3B-I | 59.0 | 55.1 | 70.8 | 44.5 | 3.0 | 0.7 | 3.8 | 1.3 | 96.4 | 65.3 |
| Qw3.5-27B | 63.8 | 58.7 | 82.2 | 66.0 | 3.0 | 0.2 | 4.0 | 1.5 | 63.2 | 12.6 |
| Qw3.6-35B-A3B | 60.4 | 53.0 | 81.8 | 55.7 | 5.4 | 1.1 | 4.6 | 0.5 | 93.2 | 70.2 |
As for the second part of Q2, PG falls back to the default only when the model always selects the neutral option. The second table below compares CMP without the neutral option (A/B) and with it (A/B/C, Table 5). It shows how much |PG(CMP)| changes from A/B to A/B/C, as a percentage change, and splits this change into two parts. Non-Def. is the change due to the neutral answers. Def. is the change due to the A or B answers.
For example, for OSS-120B on BBQ, |PG| decreases by about 54%, as it decreases by 65.7% due to the neutral answers but increases by 11.4% due to the A or B answers. Where PG decreases, mainly on BBQ and DiscrimEvalGen, this is mostly due to the neutral answers. On BBQ, this may partly reflect familiarity (data contamination) with the original benchmark, the only one whose original design includes a neutral option, which is also the correct answer in the ambiguous split we evaluate (lines 392-395; Appendix G, lines 1758-1761).
Importantly, when the model picks one of the two targets in A/B/C, its answers are more biased than in A/B in 31 of the 50 model-benchmark pairs, and almost equally biased (less than 0.5% of change) in 7. So the neutral option may lower the measured bias because the model often selects it, not because its choices become fairer, consistently with Section 4.3 (lines 395-401). If a neutral answer leads to asking the question again, as when a decision is required, the final decision comes from the A or B answers, which in most cases are more biased than in A/B.
We have added these analyses to Appendix E.8, with a pointer from Section 4.3. The first one is marked in gold (lines 1512-1538), as it was also requested by Reviewer uMNo, and the second one in violet (lines 1541-1555).
Change (%) of |PG(CMP)| from A/B to A/B/C, relative to A/B, due to the neutral answers (Non-Def.) and to the A or B answers (Def.). Negative values reduce |PG|, positive values increase it.
| Model | SS | RB | BBQ | DE-Gen | DT-Tox | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Non-Def. | Def. | Non-Def. | Def. | Non-Def. | Def. | Non-Def. | Def. | Non-Def. | Def. | |
| DS-LMa-70B | 0.0 | +0.7 | -0.3 | +1.2 | -70.7 | +4.3 | -46.1 | -19.8 | 0.0 | +0.6 |
| DS-Qw-32B | 0.0 | -0.1 | 0.0 | +1.8 | -51.6 | +14.7 | -19.2 | +21.1 | 0.0 | +1.9 |
| LMa3.1-70B-I | 0.0 | -0.3 | +0.3 | +7.5 | -30.7 | -13.6 | -57.5 | -0.4 | 0.0 | +1.3 |
| LMa4-17B-16E-I | 0.0 | +0.7 | 0.0 | -0.6 | -37.0 | +2.5 | +5.3 | +56.4 | 0.0 | +0.2 |
| Min3-14B-R | 0.0 | -3.8 | 0.0 | -2.8 | -10.4 | +24.5 | 0.0 | -2.8 | 0.0 | +3.8 |
| OSS-120B | 0.0 | +0.7 | 0.0 | -0.6 | -65.7 | +11.4 | -73.5 | -11.9 | 0.0 | +17.7 |
| Qw2.5-32B-I | 0.0 | +0.3 | -0.2 | +12.9 | -92.9 | -1.2 | -27.8 | +77.5 | 0.0 | -0.4 |
| Qw3-30B-A3B-I | 0.0 | +2.3 | -0.5 | +4.0 | -84.5 | +5.6 | -59.7 | +62.0 | 0.0 | -0.4 |
| Qw3.5-27B | 0.0 | +10.2 | -1.2 | +16.5 | -59.9 | -12.0 | -52.2 | -8.8 | -15.8 | +57.2 |
| Qw3.6-35B-A3B | 0.0 | +5.6 | 0.0 | +18.1 | -78.3 | -5.9 | -139.9 | +180.0 | 0.0 | +6.3 |
Dear Reviewer 6GmE,
Thank you for the positive review. We are glad that you found our research questions well structured and our experiments suited to answering them.
Below, we address your concerns regarding weaknesses (W) and questions (Q). In the revised manuscript, the changes made in response to your comments are marked in violet. The line numbers below refer to the revised manuscript.
W1: Similarity of the ISO and CMP tasks We agree that ISO and CMP are distinct judgment tasks, absolute and relative, as stated in Section 3.2 (lines 189-192). Since benchmarks use the two formats interchangeably (Table A.2), our aim is to quantify how this choice alters bias conclusions. The two formats still capture a related signal, as per-pair ISO and CMP preferences are moderately correlated (mean r between 0.34 and 0.58, Appendix C.1). We also agree that both sentences could receive the same yes/no answer while still being distinguishable in a comparison. The distinction that emerges in the comparison is not arbitrary, as models consistently judge the anti-stereotypical sentence as the more unusual one (Section 4.1, lines 291-292). The gap is also not specific to this question or to the binary answer. It appears across all five benchmarks (lines 286-289), it persists when ISO uses a graded 1-to-5 scale (Q1 below), and it is rarely statistically significant when the context determines a correct answer, as in disambiguated BBQ, WinoBias, and MMLU (lines 306-312; Appendix E.3).
Q1: Graded isolated judgments This is indeed a reasonable and interesting setting, which the submitted version listed as future work in the Limitations. Nevertheless, we provide here a limited analysis on a subset of models. We replaced Yes/No in ISO with a 1-to-5 scale (1 = definitely no, 2 = probably no, 3 = uncertain, 4 = probably yes, 5 = definitely yes). As shown in the table below, graded ISO remains close to binary ISO and far below CMP, with a significant gap in 14 out of 15 cases. We have added this analysis to Appendix E.5.
PG (%) under binary ISO, graded ISO (1-5 scale) and CMP (gender, CoT). Bold: statistically significant difference between graded ISO and CMP.