- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a method to find and reduce biases in VLM judges by examining their internal representations using SAE. The method identifies features that explain differences between VLM judgments and human preferences, and then uses feature steering to improve the judges' decisions.
Critical strengths
-
The proposed methodology is intuitive, well-motivated, and makes sense.
-
The attempt to quantitatively measure biases internally is interesting.
-
Using human preferences as a reference standard for identifying biases is reasonable. In fact, recent studies have also adopted human-grounded standards for bias identification. I suggest that the authors discuss these studies in the related work section and further explain the rationale for using human preferences as the reference standard in this work.
-
The experiments provide empirical justification for the authors' design choices, demonstrating that the methodology was carefully designed and validated.
Critical weaknesses
-
The proposed method is limited to discovering biases that can be captured and interpreted through the model's internal features. It may therefore overlook biases that are not represented by the extracted features.
-
The proposed method cannot be directly extended to black-box models, as it requires access to internal model representations.
-
I have several additional questions regarding the proposed method, and depending on the authors' responses, some of these concerns may also be considered weaknesses
Questions for authors
1.Could the authors examine which of the discovered biases are novel? I am curious whether the found bias features are novel compared to known biases and how novel are they. It woudl be interesting if they do this analysis
2.Could the authors explain the advantages of the proposed method over simply using stronger frontier black-box models as judges?
3.Does the proposed hidden-state subtraction account for position bias in pairwise judgments? It is unclear to me whether subtraction sufficiently addresses position bias. Judging only by the authors' description in the paper, it seems that the subtraction is intended to isolate comparative reasoning features rather than mitigate position bias. So I was wondering if the authors have accounted position bias.
4.Since the proposed steering method aims to mitigate discovered biases, could the authors examine whether the absolute bias scores really decrease toward zero after steering?
5.How do the authors determine whether a nonzero bias score represents a meaningful bias rather than statistical noise? For example, how can we determine whether a score of 0.2 is merely noise or is sufficiently large to be considered a bias?
6.Could the authors evaluate whether the discovered biases and steering policies generalize across datasets? Specifically, does a feature discovered on MM-RLHF exhibit a similar bias score on VisionArena-Battle without retraining the SAE? Also, does a steering policy trained on MM-RLHF improve judgment accuracy on VisionArena-Battle without retraining?
7 comments
Response to reviewer Z8W9 (2/3)
Q2: Could the authors explain the advantages of the proposed method over simply using stronger frontier black-box models as judges?
Human preferences vary based on the population and setting, and no judge can match all sets of preferences at the same time, regardless of how strong the underlying model is. For example, as we discuss in Section 4.3, the refusal bias for the same judge appears in opposite directions depending on the dataset and its annotators. So we need a way of using data from a specific group of humans to align a judge to that group's preferences, and our method provides that.
W2: The proposed method cannot be directly extended to black-box models, as it requires access to internal model representations.
It is true that our method cannot be used on a black-box model. However, biases that the method discovers in other models can still be candidates to look for in a black-box model. For example, Gemma 4's refusal, image grounding, and length bias types also appear in Qwen3.5, a model of a different family and architecture. Developers of black-box models can go further and use the method directly to improve their models, as they have access to the hidden states.
Q3: Does the proposed hidden-state subtraction account for position bias in pairwise judgments? It is unclear to me whether subtraction sufficiently addresses position bias.
Yes. Each SAE feature's activation is computed from the difference between the two orderings' hidden states, so the sign flips when the two responses are swapped, while position bias stays constant. So no feature can represent position bias, and it cannot be mixed into any other bias our method discovers. The rest of our method and evaluation also uses both orderings and controls for position bias, including the bias score, the steering policy, and the reported accuracies. Any position preference that the judge has is still visible in its verdicts: on MM-RLHF, Gemma 4 chooses position B 50.8% of the time, and Qwen3.5 53.9%. We will add the explanation to Section 3.1, where we introduce the subtraction.
Q4: Since the proposed steering method aims to mitigate discovered biases, could the authors examine whether the absolute bias scores really decrease toward zero after steering?
We compute the bias scores of the three steered features from Table 2 on the MM-RLHF test set, before and after steering. The table shows the scores × 100.
| Bias feature | Before | After |
|---|---|---|
| Refusal | −4.45 | +1.98 |
| Image grounding | +5.84 | +5.97 |
| Length | +4.20 | +0.85 |
The absolute scores for refusal and length become smaller and statistically indistinguishable from zero. The decrease is statistically significant for length, while for refusal it has a wide confidence interval that includes zero. Image grounding's score stays similar.
However, the bias score after steering may no longer cleanly reflect the judge's own bias. The feature activation that goes into the bias score is changed by the steering policy, which injects the human preference that it is trained on, so part of the score comes from this introduced preference and not the judge. This may be why image grounding's score does not fall: its activation becomes more correlated with human preference (from 0.119 to 0.136), which pushes the positive score upward. We therefore consider the accuracy improvement on relevant task subsets, which we report in the paper, to be more direct evidence of bias mitigation.
Q5: How do the authors determine whether a nonzero bias score represents a meaningful bias rather than statistical noise?
We determine this by testing for statistical significance. We compute a bootstrap confidence interval for each bias score, and conclude that the score represents a real bias rather than noise if the interval excludes zero. We also correct for multiple testing when checking many features at once.
Nine of the ten bias features in Table 2 have a 95% confidence interval that excludes zero, and we consider them true biases. The remaining one is the user-language bias feature. Since its interval alone does not settle the question, we check it directly: on 883 VisionArena-Battle tasks where only one of the two candidate responses matches the user's language, the judge chooses the user-language response 59.6% of the time, while for humans the rate is 64.8%. So even though the feature has the smallest absolute bias score in the table and its confidence interval includes zero, it corresponds to a statistically significant 5.2 pp gap in measured behavior.
Significance holds for features outside the table as well. For MM-RLHF, every feature with an absolute bias score above 0.015 is statistically significant after Benjamini–Hochberg correction at 5%.
Response to reviewer Z8W9 (3/3)
Q6: Could the authors evaluate whether the discovered biases and steering policies generalize across datasets? Specifically, does a feature discovered on MM-RLHF exhibit a similar bias score on VisionArena-Battle without retraining the SAE? Also, does a steering policy trained on MM-RLHF improve judgment accuracy on VisionArena-Battle without retraining?
We apply Gemma 4's MM-RLHF features to VisionArena-Battle without retraining and compute the bias scores. The table shows the scores × 100 for the seven MM-RLHF features from Table 2 on the two datasets.
| Bias feature | MM-RLHF | VisionArena-Battle |
|---|---|---|
| Helpful information vs. minimal refusal | −2.93 | +1.88 |
| Contextual interpretation vs. listing isolated details | +2.45 | +2.76 |
| Image-grounded reasoning vs. unsupported answer | +5.74 | +2.01 |
| Detailed reasoning vs. minimal answers | +2.86 | −1.44 |
| Honest uncertainty vs. confident task completion | −1.67 | +1.13 |
| Negative conclusion vs. affirmative claim | −2.69 | −2.50 |
| Literal instruction adherence vs. intent fulfillment | −4.35 | −6.37 |
Bias scores transfer when the human preferences match. This is the case for four of the seven features, which show the same bias on both datasets, with statistically significant scores of the same sign. The honest uncertainty feature's score is not significant on VisionArena-Battle, so we do not detect this bias there. The refusal and length bias features reverse their sign. For these features, the judge preference is similar across the two datasets, so the reversal comes from the human preferences, which agrees with our finding for refusal in Section 4.3.
Next we check whether Gemma 4's three MM-RLHF steering policies improve accuracy on VisionArena-Battle when used unchanged. Steering pushes the judge toward the candidate response chosen by the policy, so we first measure how often this response matches the human choice on the tasks that each policy steers. On MM-RLHF, the policy's choice agrees with human preference more often than the unsteered judge's verdicts do (83–85% vs. 77–80%), but on VisionArena-Battle it agrees less often (58–69% vs. 73–74%).
As a result, steering improvements do not transfer to VisionArena-Battle. The accuracy stays about the same for image grounding (−0.1 pp). It decreases for refusal and length, the two reversed biases (−1.8 pp and −1.2 pp). For these two, simply applying the policy's predicted steering with the opposite sign does not help (−2.9 pp and −5.7 pp), because the policy's original choice is still right more often than wrong. These results support our point in Section 4.3 that the human preference data should be selected to represent the preferences that the judge is intended to follow.
W1: The proposed method is limited to discovering biases that can be captured and interpreted through the model's internal features. It may therefore overlook biases that are not represented by the extracted features.
Yes, our method can only discover biases that the internal features represent. Every discovery method is limited to the space it searches, and ours looks inside the judge, which makes it complementary to methods that look at the responses.
Response to reviewer Z8W9 (1/3)
Thank you for your review! We are glad that you find our internal measurement of biases interesting and the method carefully designed and validated. We have run additional experiments and analyses to answer your questions, and we will update the paper with new results.
Thank you for the suggestion. Our paper describes bias as a mismatch between human and judge preferences, following Li et al. [1] and Wedgwood et al. [2]. Other recent studies take the same human-grounded view. Koo et al. [3] identify bias factors from cases where judges disagree with humans, measure them by rewriting responses, and remove their estimated effect with a regression. Polo et al. [4] model how a judge's ratings deviate from human preferences as a function of response properties, and use this to recalibrate the ratings. These studies analyze the responses and the judge verdicts, while our method discovers and corrects biases in the judge's internal states.
We use human preferences as the reference because a judge model is a substitute for human evaluation. Correcting a judge relative to a different standard can remove something that humans also value, as Polo et al. [4] note. We will add this rationale and the discussion of the studies to the paper.
We compare each of the ten biases in Table 2 with biases reported in prior work. The table shows how novel each bias is, to our knowledge.
Two of the ten biases are novel, and two are partly novel. For two of the known biases, the sign of our bias score indicates a bias in the direction opposite to what is reported in the literature. This is in line with our finding that biases depend on the human preferences that serve as the reference (Section 4.3).
Another way to assess novelty is to measure how strongly a feature is correlated with a known bias. We do this for length bias and compute the correlation between each feature's activation and the difference in length between the two candidate responses. The length bias feature has the largest correlation, 0.43, as expected, while the remaining nine features average 0.05 in magnitude, with none above 0.20, so they are distinct from length bias.
References
[1] Li et al. "Dissecting Human and LLM Preferences." ACL 2024.
[2] Wedgwood et al. "Automated Concept Discovery for LLM-as-a-Judge Preference Analysis." ICLR 2026 Workshop on Unifying Concept Representation Learning.
[3] Koo et al. "Auditing the Judge: Human-Grounded Bias Discovery, Quantification, and Mitigation in LLM Judges." NeurIPS 2026.
[4] Polo et al. "Bridging Human and LLM Judgments: Understanding and Narrowing the Gap." NeurIPS 2025.
[5] Lee et al. "MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge." ACL 2026.
[6] Zou et al. "When Vision-Language Models Judge Without Seeing: Exposing Informativeness Bias." ACL 2026.
[7] Zheng et al. "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena." NeurIPS 2023.
[8] Lee et al. "Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the Effect of Epistemic Markers on LLM-based Evaluation." NAACL 2025.
[9] Yu et al. "Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment." NAACL 2025.
[10] Zhou et al. "Fairness or Fluency? An Investigation into Language Bias of Pairwise LLM-as-a-Judge." arXiv 2026.