acceptodds
Withdrawn from ICLR 2027

QuantiBias: Benchmarking Quantization-Induced Bias in LLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Almost every large language model that reaches a broad audience is quantized: trained in full precision, then compressed for efficiency. This step is assumed harmless and its safety is rarely re-checked. We find its principal side effect is increased bias that standard safety evaluation misses. Holding the model, its training, and the prompts fixed, a quantized model still refuses harmful requests, still avoids over-refusing benign prompts, and still selects the unbiased multiple-choice answer. Yet asked an open-ended question, the same model volunteers stereotypes in all eight languages we probe, in roughly one in four open-ended answers under an independent judge ( 24% to  27% across the compression ladder): it passes every standard check and still reaches users measurably more biased. The selective gap is a robust finding; whether open-ended bias further increases with compression is less certain, sensitive to the judge that scores it. We address both with QuantiBias, a benchmark that pairs a generative, multilingual stereotype probe with the refusal and multiple-choice controls that isolate open-ended generation, contrasts each build with and without reasoning, and rates the content severity of what it generates. Across two backbone models (Qwen and Gemma), a five-family screen, and eight benchmarks, quantizers allocate their extra precision by capability data that carries no bias-prevention signal, and reasoning before answering roughly halves the effect on some families while doing nothing on others. A quantized build must be re-evaluated for open-ended bias, not only on the short-form safeguards it already passes.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
quantization, bias, fairness, safety evaluation, benchmark, large language models, multilingual, stereotypes, LLM-as-judge, reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_VXiiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how quantization affects bias in LLMs. It introduces QuantiBias, an evaluation protocol combining multilingual open-ended stereotype probes with refusal, over-refusal, multiple-choice bias, and capability benchmarks as controls. The main finding is that quantized models can maintain their performance on standard safety checks while still generating stereotypes in open-ended tasks. Whether open-ended bias increases with stronger compression is less certain, as the trend depends on the judge scoring the responses. The paper also compares reasoning on and off: reasoning roughly halves open-ended bias on Qwen under the lean judge, but does not show the same benefit on Gemma. Overall, the results suggest that standard safety checks alone are insufficient for evaluating bias in quantized models, and that both model configuration and judge choice matter.

Critical strengths
  • The paper identifies a useful gap in how quantized models are evaluated. Refusal and multiple-choice bias scores remain nearly unchanged, while the same models still generate stereotypes in open-ended tasks. This motivates evaluating open-ended behavior rather than relying only on standard checks.

  • I like the comparison between reasoning on and off. Reasoning roughly halves the stereotype endorsement rate on Qwen under the lean judge, but does not show the same benefit on Gemma. This shows that reasoning alone cannot be assumed to provide the same safeguard across models.

  • The evaluation includes multiple languages and a second generative benchmark, CEB. Comparing the same prompts across precision levels and reasoning settings is also useful for examining behavioral changes.

  • The authors acknowledge the limitations of LLM-as-judge and cross-check the lean judge with two independent judges. Reporting the disagreement, including its effect on the compression trend, makes the results easier to assess.

  • Measuring effective bits per weight rather than relying on nominal quantization labels is a practical strength. The evaluation protocol and scoring harness could also be useful for checking models before deployment.

Critical weaknesses
  • The main results are highly dependent on the judge. On Qwen, the lean judge reports endorsement rates increasing from 4.2% to 7.8%, while the independent judge reports much higher but roughly flat rates. The two independent judges agree, but this does not validate the lean judge, which the authors report as uncorrelated with both.

  • The distinction between stereotype endorsement and responding to a loaded prompt needs further validation. Of the 48 one-bit cases labeled as endorsements by the lean judge, 10 received a severity score of zero from the independent assessment. This raises questions about how the rubric separates factual or contextual statements, prompt compliance, and actual stereotype endorsement.

  • Several central comparisons contain only two or three precision levels, and the five-family screen does not resolve the compression slope within its uncertainty intervals. The reasoning benefit is also primarily supported by the lean judge. These results support model-specific observations, but broader claims about compression or reasoning need stronger evidence.

Minor comment

  • The text in Figure 1 is too small to read comfortably.
Questions for authors
  • Are all evaluations single-turn? In a multi-turn conversation, does stereotype endorsement persist, become reinforced, or fade as the conversation continues?

  • How do the judges distinguish factual statements about group differences (true statistics) from stereotype endorsement?

  • How do the compression trends compare between MBTP and CEB? The independent judge reports a roughly flat trend on MBTP but an increasing trend on CEB. What might explain this difference?

  • The exact judge prompts and endorsement criteria are not provided in the appendix. In particular, how are judges instructed to distinguish stereotype endorsement from quoting, rejecting, or discussing a stereotype, factual statements about group differences, and compliance with a loaded prompt?

  • Model-specific bias is already well documented in prior works including [1], and the evidence linking stronger compression to increased bias is inconclusive and judge-dependent. What new insight does this study establish specifically about quantization, beyond showing that standard safety scores do not capture open-ended bias?

[1] Jeong, Hyejun, Shiqing Ma, and Amir Houmansadr. "Bias Similarity Measurement: A Black-Box Audit of Fairness Across LLMs." ICLR. Vol. 2026.

Reviewer_92YGOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates the impact of quantization on bias in large language models and introduces QuantiBias, a benchmark combining open-ended stereotype evaluation, standard safety metrics, and analysis of reasoning modes. Experiments across different models and quantization levels show that standard safety metrics remain largely stable, while open-ended generation reveals a substantial rate of stereotypical responses. However, the relationship between quantization level and bias remains inconclusive and sensitive to the choice of evaluator.

Critical strengths
  1. The paper highlights an important limitation of conventional LLM safety evaluations by demonstrating a gap between performance on standard safety benchmarks and the presence of stereotypes in open-ended generation.
  2. The authors systematically compare models across quantization levels while controlling generation settings and accounting for actual weight precision. The use of multiple benchmarks, control metrics, and reasoning on/off comparisons provides a broader assessment of behavioral changes.
Critical weaknesses
  1. The lack of accessible QuantiBias code and implementation limits the ability to verify the proposed methodology. The reliability of bias evaluation is also a concern, as the results depend strongly on the choice of LLM judge, while independent validation and statistical analysis have limited coverage.
  2. Despite the paper's focus on quantization-induced bias, the evidence does not convincingly establish that quantization systematically increases bias. The absence of BF16 baselines on MBTP and CEB prevents a direct comparison with the original model. Furthermore, the observed increases in bias are not consistently supported across benchmarks and model families.
Questions for authors
  1. Could the authors include BF16 baseline results on MBTP and CEB to assess changes in bias relative to the original model?
  2. The lean judge's assessments differ substantially from those of independent judges. How reliable are these results without systematic human validation? Providing the prompts used for bias evaluation would also improve transparency.
  3. On MBTP, the independent judge reports a nearly constant bias rate as quantization becomes more aggressive, whereas CEB shows a substantial increase. How do the authors interpret this discrepancy between benchmarks?
  4. How reliable is the multilingual data validation procedure based on automated translation and back-translation? Was any additional native-speaker validation conducted, particularly for examples with identified translation discrepancies?
  5. Since QuantiBias primarily combines existing benchmarks and evaluation methods, it would be helpful to clarify its distinct methodological contribution and advantages over existing approaches.
  6. Could the severity analysis be extended across different quantization levels to examine how stereotype severity changes with compression? It would also be useful to specify which independent LLM judge was used for severity assessment and provide the corresponding evaluation prompt.
  7. How are Pass/Fail decisions determined in QuantiBias, and how is the statistical significance of changes in bias taken into account?
Reviewer_LkEiOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how post-training quantization affects social bias in LLMs. It argues that quantized builds continue to pass standard short-form safety checks (e.g., refusal, over-refusal, and multiple-choice bias) while producing stereotype-endorsing content in open-ended generation. To assess this gap, the authors introduce QuantiBias, a setup dedicated to this purpose. Experiments on Qwen and Gemma precision ladders show that, under an independent LLM judge, roughly one in four open-ended answers contains a stereotype. The paper further reports that reasoning reduces this effect.

Critical strengths
  • The paper targets a timely and practically important topic: the deployed models are typically compressed, and the safety and bias side of quantization has received far less systematic attention than the general capability side.
Critical weaknesses
  • The introduction contains only two citations, which is highly unusual for a well-established research area spanning LLM bias and quantization. Many of its claims and much of its background lack supporting references. Just to name a new:

    • “A language model is evaluated at full precision and typically deployed compressed … so the build a user interacts with is usually not the one the evaluation measured” → what’s the evidence that “the build a user interacts with is usually not the one the evaluation measured”?
    • “Reports on aggressively quantized builds tend to document capability retention across benchmarks while saying less about safety or bias.” → What are such “Reports”?
    • “Prior studies of quantization and safety often disagree, and much of the disagreement follows from what the standard safety metric examines.” → What are the “Prior studies”?
    • “… reasoning halves the open-ended bias on Qwen but leaves it unchanged on Gemma.” → This is the first appearance of both model families in the main text, and they should be cited there. Proper referencing would better situate the work and substantiate its motivation as well as improve scientific rigor.
  • The section font appears to be different (smaller) from the typeface of the official ICLR 2027 template. The authors should confirm that they used the correct ICLR 2027 style files since modifying the style files may be grounds for rejection.

  • It is difficult to understand what exactly QuantiBias proposes, and whether it constitutes a new benchmark or an evaluation protocol. Section 4 describes it as a reusable protocol with four components, with the probe items attributed to MBTP. Overall, QuantiBias reads more as a combination of existing resources than as a new technically contribution.

Questions for authors
  • Why was MBTP not evaluated on the BF16 builds (Figure 2, Table A3)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.