acceptodds
Under review as a conference paper at ICLR 2027

Bias Discovery and Steering in VLM Judges

Abstract

Vision-language models (VLMs) serve as multimodal judges in evaluation, training, and inference, but they exhibit biases that propagate to each of these settings. Existing methods for detecting biases typically analyze judge outputs, either testing for specific biases or discovering them through properties of the evaluated responses. Those that do examine the judge's internal states focus on predefined biases. We instead discover biases from these states without choosing them in advance, and correct them by modifying the same states. Using sparse autoencoders, we extract interpretable features from hidden states and introduce a bias score that measures how differently the judge and humans weigh each feature. We then steer the judge along selected features, with a lightweight policy adapting the intervention to each input. Applied to two judge models and two datasets, our method surfaces biases spanning safety, visual grounding, instruction following, and multilingual interaction, including several that are largely unexplored in judges. Steering with bias features produces statistically significant improvements in accuracy for both judges, with increases on relevant task subsets of up to 11.6 percentage points.

Primary area:
interpretability and explainable AI
Keywords:
vision-language models, LLM-as-a-judge, reasoning biases, sparse autoencoders, activation steering

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Z8W9OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method to find and reduce biases in VLM judges by examining their internal representations using SAE. The method identifies features that explain differences between VLM judgments and human preferences, and then uses feature steering to improve the judges' decisions.

Critical strengths
  1. The proposed methodology is intuitive, well-motivated, and makes sense.

  2. The attempt to quantitatively measure biases internally is interesting.

  3. Using human preferences as a reference standard for identifying biases is reasonable. In fact, recent studies have also adopted human-grounded standards for bias identification. I suggest that the authors discuss these studies in the related work section and further explain the rationale for using human preferences as the reference standard in this work.

  4. The experiments provide empirical justification for the authors' design choices, demonstrating that the methodology was carefully designed and validated.

Critical weaknesses
  1. The proposed method is limited to discovering biases that can be captured and interpreted through the model's internal features. It may therefore overlook biases that are not represented by the extracted features.

  2. The proposed method cannot be directly extended to black-box models, as it requires access to internal model representations.

  3. I have several additional questions regarding the proposed method, and depending on the authors' responses, some of these concerns may also be considered weaknesses

Questions for authors

1.Could the authors examine which of the discovered biases are novel? I am curious whether the found bias features are novel compared to known biases and how novel are they. It woudl be interesting if they do this analysis

2.Could the authors explain the advantages of the proposed method over simply using stronger frontier black-box models as judges?

3.Does the proposed hidden-state subtraction account for position bias in pairwise judgments? It is unclear to me whether subtraction sufficiently addresses position bias. Judging only by the authors' description in the paper, it seems that the subtraction is intended to isolate comparative reasoning features rather than mitigate position bias. So I was wondering if the authors have accounted position bias.

4.Since the proposed steering method aims to mitigate discovered biases, could the authors examine whether the absolute bias scores really decrease toward zero after steering?

5.How do the authors determine whether a nonzero bias score represents a meaningful bias rather than statistical noise? For example, how can we determine whether a score of 0.2 is merely noise or is sufficiently large to be considered a bias?

6.Could the authors evaluate whether the discovered biases and steering policies generalize across datasets? Specifically, does a feature discovered on MM-RLHF exhibit a similar bias score on VisionArena-Battle without retraining the SAE? Also, does a steering policy trained on MM-RLHF improve judgment accuracy on VisionArena-Battle without retraining?

7 comments
AuthorsOpenReview ↗

Response to reviewer Z8W9 (1/3)

Thank you for your review! We are glad that you find our internal measurement of biases interesting and the method carefully designed and validated. We have run additional experiments and analyses to answer your questions, and we will update the paper with new results.

S3: Using human preferences as a reference standard for identifying biases is reasonable. In fact, recent studies have also adopted human-grounded standards for bias identification. I suggest that the authors discuss these studies in the related work section and further explain the rationale for using human preferences as the reference standard in this work.

Thank you for the suggestion. Our paper describes bias as a mismatch between human and judge preferences, following Li et al. [1] and Wedgwood et al. [2]. Other recent studies take the same human-grounded view. Koo et al. [3] identify bias factors from cases where judges disagree with humans, measure them by rewriting responses, and remove their estimated effect with a regression. Polo et al. [4] model how a judge's ratings deviate from human preferences as a function of response properties, and use this to recalibrate the ratings. These studies analyze the responses and the judge verdicts, while our method discovers and corrects biases in the judge's internal states.

We use human preferences as the reference because a judge model is a substitute for human evaluation. Correcting a judge relative to a different standard can remove something that humans also value, as Polo et al. [4] note. We will add this rationale and the discussion of the studies to the paper.

Q1: Could the authors examine which of the discovered biases are novel? I am curious whether the found bias features are novel compared to known biases and how novel are they.

We compare each of the ten biases in Table 2 with biases reported in prior work. The table shows how novel each bias is, to our knowledge.

DatasetBiasNovelty
MM-RLHFHelpful information vs. minimal refusalKnown in the opposite direction [2]
Contextual interpretation vs. listing isolated detailsNovel
Image-grounded reasoning vs. unsupported answerKnown [5, 6]
Detailed reasoning vs. minimal answersKnown in both directions [4, 7]
Honest uncertainty vs. confident task completionKnown in the opposite direction [8]
Negative conclusion vs. affirmative claimPartly novel; known for LLMs' own responses [9]
Literal instruction adherence vs. intent fulfillmentNovel
VisionArena-BattleTask coverage vs. communicative impactKnown [4]
Substantive detail vs. matching the user's languagePartly novel; judges are known to favor English answers [10]
Attempt to answer vs. cautious refusalKnown [2]

Two of the ten biases are novel, and two are partly novel. For two of the known biases, the sign of our bias score indicates a bias in the direction opposite to what is reported in the literature. This is in line with our finding that biases depend on the human preferences that serve as the reference (Section 4.3).

Another way to assess novelty is to measure how strongly a feature is correlated with a known bias. We do this for length bias and compute the correlation between each feature's activation and the difference in length between the two candidate responses. The length bias feature has the largest correlation, 0.43, as expected, while the remaining nine features average 0.05 in magnitude, with none above 0.20, so they are distinct from length bias.


References

[1] Li et al. "Dissecting Human and LLM Preferences." ACL 2024.

[2] Wedgwood et al. "Automated Concept Discovery for LLM-as-a-Judge Preference Analysis." ICLR 2026 Workshop on Unifying Concept Representation Learning.

[3] Koo et al. "Auditing the Judge: Human-Grounded Bias Discovery, Quantification, and Mitigation in LLM Judges." NeurIPS 2026.

[4] Polo et al. "Bridging Human and LLM Judgments: Understanding and Narrowing the Gap." NeurIPS 2025.

[5] Lee et al. "MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge." ACL 2026.

[6] Zou et al. "When Vision-Language Models Judge Without Seeing: Exposing Informativeness Bias." ACL 2026.

[7] Zheng et al. "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena." NeurIPS 2023.

[8] Lee et al. "Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the Effect of Epistemic Markers on LLM-based Evaluation." NAACL 2025.

[9] Yu et al. "Correcting Negative Bias in Large Language Models through Negative Attention Score Alignment." NAACL 2025.

[10] Zhou et al. "Fairness or Fluency? An Investigation into Language Bias of Pairwise LLM-as-a-Judge." arXiv 2026.

AuthorsOpenReview ↗

Response to reviewer Z8W9 (2/3)

Q2: Could the authors explain the advantages of the proposed method over simply using stronger frontier black-box models as judges?

Human preferences vary based on the population and setting, and no judge can match all sets of preferences at the same time, regardless of how strong the underlying model is. For example, as we discuss in Section 4.3, the refusal bias for the same judge appears in opposite directions depending on the dataset and its annotators. So we need a way of using data from a specific group of humans to align a judge to that group's preferences, and our method provides that.

W2: The proposed method cannot be directly extended to black-box models, as it requires access to internal model representations.

It is true that our method cannot be used on a black-box model. However, biases that the method discovers in other models can still be candidates to look for in a black-box model. For example, Gemma 4's refusal, image grounding, and length bias types also appear in Qwen3.5, a model of a different family and architecture. Developers of black-box models can go further and use the method directly to improve their models, as they have access to the hidden states.

Q3: Does the proposed hidden-state subtraction account for position bias in pairwise judgments? It is unclear to me whether subtraction sufficiently addresses position bias.

Yes. Each SAE feature's activation is computed from the difference between the two orderings' hidden states, so the sign flips when the two responses are swapped, while position bias stays constant. So no feature can represent position bias, and it cannot be mixed into any other bias our method discovers. The rest of our method and evaluation also uses both orderings and controls for position bias, including the bias score, the steering policy, and the reported accuracies. Any position preference that the judge has is still visible in its verdicts: on MM-RLHF, Gemma 4 chooses position B 50.8% of the time, and Qwen3.5 53.9%. We will add the explanation to Section 3.1, where we introduce the subtraction.

Q4: Since the proposed steering method aims to mitigate discovered biases, could the authors examine whether the absolute bias scores really decrease toward zero after steering?

We compute the bias scores of the three steered features from Table 2 on the MM-RLHF test set, before and after steering. The table shows the scores × 100.

Bias featureBeforeAfter
Refusal−4.45+1.98
Image grounding+5.84+5.97
Length+4.20+0.85

The absolute scores for refusal and length become smaller and statistically indistinguishable from zero. The decrease is statistically significant for length, while for refusal it has a wide confidence interval that includes zero. Image grounding's score stays similar.

However, the bias score after steering may no longer cleanly reflect the judge's own bias. The feature activation that goes into the bias score is changed by the steering policy, which injects the human preference that it is trained on, so part of the score comes from this introduced preference and not the judge. This may be why image grounding's score does not fall: its activation becomes more correlated with human preference (from 0.119 to 0.136), which pushes the positive score upward. We therefore consider the accuracy improvement on relevant task subsets, which we report in the paper, to be more direct evidence of bias mitigation.

Q5: How do the authors determine whether a nonzero bias score represents a meaningful bias rather than statistical noise?

We determine this by testing for statistical significance. We compute a bootstrap confidence interval for each bias score, and conclude that the score represents a real bias rather than noise if the interval excludes zero. We also correct for multiple testing when checking many features at once.

Nine of the ten bias features in Table 2 have a 95% confidence interval that excludes zero, and we consider them true biases. The remaining one is the user-language bias feature. Since its interval alone does not settle the question, we check it directly: on 883 VisionArena-Battle tasks where only one of the two candidate responses matches the user's language, the judge chooses the user-language response 59.6% of the time, while for humans the rate is 64.8%. So even though the feature has the smallest absolute bias score in the table and its confidence interval includes zero, it corresponds to a statistically significant 5.2 pp gap in measured behavior.

Significance holds for features outside the table as well. For MM-RLHF, every feature with an absolute bias score above 0.015 is statistically significant after Benjamini–Hochberg correction at 5%.

AuthorsOpenReview ↗

Response to reviewer Z8W9 (3/3)

Q6: Could the authors evaluate whether the discovered biases and steering policies generalize across datasets? Specifically, does a feature discovered on MM-RLHF exhibit a similar bias score on VisionArena-Battle without retraining the SAE? Also, does a steering policy trained on MM-RLHF improve judgment accuracy on VisionArena-Battle without retraining?

We apply Gemma 4's MM-RLHF features to VisionArena-Battle without retraining and compute the bias scores. The table shows the scores × 100 for the seven MM-RLHF features from Table 2 on the two datasets.

Bias featureMM-RLHFVisionArena-Battle
Helpful information vs. minimal refusal−2.93+1.88
Contextual interpretation vs. listing isolated details+2.45+2.76
Image-grounded reasoning vs. unsupported answer+5.74+2.01
Detailed reasoning vs. minimal answers+2.86−1.44
Honest uncertainty vs. confident task completion−1.67+1.13
Negative conclusion vs. affirmative claim−2.69−2.50
Literal instruction adherence vs. intent fulfillment−4.35−6.37

Bias scores transfer when the human preferences match. This is the case for four of the seven features, which show the same bias on both datasets, with statistically significant scores of the same sign. The honest uncertainty feature's score is not significant on VisionArena-Battle, so we do not detect this bias there. The refusal and length bias features reverse their sign. For these features, the judge preference is similar across the two datasets, so the reversal comes from the human preferences, which agrees with our finding for refusal in Section 4.3.

Next we check whether Gemma 4's three MM-RLHF steering policies improve accuracy on VisionArena-Battle when used unchanged. Steering pushes the judge toward the candidate response chosen by the policy, so we first measure how often this response matches the human choice on the tasks that each policy steers. On MM-RLHF, the policy's choice agrees with human preference more often than the unsteered judge's verdicts do (83–85% vs. 77–80%), but on VisionArena-Battle it agrees less often (58–69% vs. 73–74%).

As a result, steering improvements do not transfer to VisionArena-Battle. The accuracy stays about the same for image grounding (−0.1 pp). It decreases for refusal and length, the two reversed biases (−1.8 pp and −1.2 pp). For these two, simply applying the policy's predicted steering with the opposite sign does not help (−2.9 pp and −5.7 pp), because the policy's original choice is still right more often than wrong. These results support our point in Section 4.3 that the human preference data should be selected to represent the preferences that the judge is intended to follow.

W1: The proposed method is limited to discovering biases that can be captured and interpreted through the model's internal features. It may therefore overlook biases that are not represented by the extracted features.

Yes, our method can only discover biases that the internal features represent. Every discovery method is limited to the space it searches, and ours looks inside the judge, which makes it complementary to methods that look at the responses.

Reviewer_VzEJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper uses sparse autoencoders to find features in VLM judges’ hidden states that are linked to disagreements with human preferences. It then adjusts these states along selected feature directions to improve agreement with human choices. Experiments analyze the discovered features across judges and datasets, and steering improves agreement with human choices on MM-RLHF.

Critical strengths
  • VLM judges are used for evaluation and training, so understanding and reducing their biases is a timely and useful research topic.

  • Figure 1 clearly shows how feature discovery, bias scoring, and steering fit together, making the method easier to understand.

Critical weaknesses

This work builds on established methods for SAE-based feature analysis and activation steering and applies them to VLM judges. I therefore see its main potential value in the new insights it provides about VLM judges and their usefulness to the community. However, I have several concerns about both these findings and the correction experiments.

1. The evidence supporting the bias findings is limited.

  • The core ideas already appear across prior work [a, b, c, d]. These include discovering preference features with SAEs, comparing human and judge preferences, analyzing bias in internal states, and using discovered differences or internal interventions to improve evaluation. Feature extraction and intervention details differ, but the broad idea of finding biases and using them for correction is not new.

  • The experiments provide limited insight specific to VLMs. Internal steering has already improved multimodal evaluation [e]. Here, the “image grounding” subset contains pairs where only the human-preferred answer includes the word “image”. Better results on these pairs do not establish better checking of image content. No comparison removes or changes the image to separate visual effects from textual ones.

  • The usefulness of the discovered features on other datasets is unclear. Features are learned again for each dataset, without testing whether the same learned features identify the same problems elsewhere. The findings may therefore reflect preferences specific to the human labels used for discovery.

  • The score does not directly measure how much a judge overvalues or undervalues a feature, despite being described that way. It measures a correlation between feature activity and errors in a linear prediction of human choices from judge preferences. These errors can also reflect limits of the prediction model or noisy human labels.

  • The evaluation covers only two relatively small VLM judges. Whether the findings and steering gains hold for larger models or other model families remains unclear.

  • The paper does not show whether different SAE settings would reveal the same biases. It uses 32 features and an average of four active features per input, without changing either setting. Both choices affect how internal states are divided into features.

  • The findings cover only comparisons between two answers. Single-answer scoring and comparisons of several answers together are not evaluated, so the conclusions do not establish benefits in these other judge formats.

2. The correction experiments also raise several concerns.

  • The correction is not shown to work beyond MM-RLHF. The features and intervention policy are learned using this dataset, and the final evaluation uses different images from the same dataset. The same learned correction is not applied to another benchmark.

  • The correction relies on manual feature selection and strength tuning. Refusal, image grounding, and length are chosen partly based on earlier intervention results, and different strengths are tried against human labels. The paper does not show how often this process produces a useful correction for the other biases it discovers.

  • The experiments do not show that bias discovery is needed for the improvement. A separate policy learns from human labels to control the intervention, but the full method is compared only with the original judge. There is no comparison testing whether similar gains can be obtained from the same labels without the bias-discovery stage.

  • Overall gains are modest, and some corrections make other judgments worse. Accuracy improves by 0.9–1.7 percentage points overall, with the largest gains on refusal-related questions. Length steering also lowers accuracy by up to 5.9 points outside the group where humans prefer the longer answer.


References

[a] "Automated Concept Discovery for LLM-as-a-Judge Preference Analysis"

[b] "SteerRM: Debiasing Reward Models via Sparse Autoencoders"

[c] "Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias"

[d] "Multi-Domain Explainability of Preferences"

[e] "Activation Reward Models for Few-Shot Model Alignment"

Questions for authors

Please refer to Critical Weaknesses.

7 comments
AuthorsOpenReview ↗

Response to reviewer VzEJ (1/3)

Thank you for your review! We are glad that you find the topic timely and useful, and that you appreciate our Figure 1. We have run additional experiments and analyses to address your concerns, and we will update the paper with new results.

W1.1: The core ideas already appear across prior work [a, b, c, d]. These include discovering preference features with SAEs, comparing human and judge preferences, analyzing bias in internal states, and using discovered differences or internal interventions to improve evaluation. Feature extraction and intervention details differ, but the broad idea of finding biases and using them for correction is not new.

The ideas appear separately in prior work, as we discuss in Section 2. The main novelty of our method is that it discovers biases in the judge's internal states without choosing them in advance and corrects them by modifying the same states. As the table shows, none of [a–d] has this combination.

WorkDiscovers biases not chosen in advanceUses the judge's internal statesCorrects the biases
[a]✓✗✗
[b]✗✓✓
[c]✗✓✓
[d]✓✗✓
Ours✓✓✓

Another novelty is the bias score, which measures the mismatch between human and judge preferences for any internal feature. Of the cited works, [a] and [d] compare these preferences, but only for properties of the responses.

W1.2: The experiments provide limited insight specific to VLMs. Internal steering has already improved multimodal evaluation [e]. Here, the “image grounding” subset contains pairs where only the human-preferred answer includes the word “image”. Better results on these pairs do not establish better checking of image content. No comparison removes or changes the image to separate visual effects from textual ones.

We run the suggested comparison by repeating the image grounding steering experiment for Gemma 4 with the image removed from the judge's input. Without the image, the accuracy improvement from steering on the image grounding subset falls from 2.0 pp to 0.7 pp, while the improvement on the other tasks is mostly preserved (from 0.7 pp to 0.6 pp).

Regarding [e], it uses activation steering to build a reward model for a criterion specified by examples, and does not identify biases of the underlying model. Our method discovers judge biases and steers along their features.

W1.3: The usefulness of the discovered features on other datasets is unclear. Features are learned again for each dataset, without testing whether the same learned features identify the same problems elsewhere. The findings may therefore reflect preferences specific to the human labels used for discovery.

We apply Gemma 4's MM-RLHF features to VisionArena-Battle without retraining and compute the bias scores. The table shows the scores × 100 for the seven MM-RLHF features from Table 2 on the two datasets.

Bias featureMM-RLHFVisionArena-Battle
Helpful information vs. minimal refusal−2.93+1.88
Contextual interpretation vs. listing isolated details+2.45+2.76
Image-grounded reasoning vs. unsupported answer+5.74+2.01
Detailed reasoning vs. minimal answers+2.86−1.44
Honest uncertainty vs. confident task completion−1.67+1.13
Negative conclusion vs. affirmative claim−2.69−2.50
Literal instruction adherence vs. intent fulfillment−4.35−6.37

Bias scores transfer when the human preferences match. This is the case for four of the seven features, which show the same bias on both datasets, with statistically significant scores of the same sign. The honest uncertainty feature's score is not significant on VisionArena-Battle, so we do not detect this bias there. The refusal and length bias features reverse their sign. For these features, the judge preference is similar across the two datasets, so the reversal comes from the human preferences, which agrees with our finding for refusal in Section 4.3. This dependence on human labels is by design, as our definition of a bias uses human preference as a reference.

AuthorsOpenReview ↗

Response to reviewer VzEJ (2/3)

W1.4: The score does not directly measure how much a judge overvalues or undervalues a feature, despite being described that way. It measures a correlation between feature activity and errors in a linear prediction of human choices from judge preferences. These errors can also reflect limits of the prediction model or noisy human labels.

The bias scores remain almost the same when the linear prediction is replaced by a nonparametric one. We use cross-validated predictions from isotonic regression, which only assumes a monotonic relationship. All ten bias scores in Table 2 keep their sign and change by less than 0.005, and every feature with a statistically significant score across the SAEs keeps its sign. The linear prediction also fits human preference about as well as isotonic regression, with a mean squared error of 0.631 on MM-RLHF compared to 0.627.

Noise in the human labels reduces the magnitude of the bias scores and adds random variation. To account for this variation, we compute 95% bootstrap confidence intervals for the ten scores in Table 2, and nine of them exclude zero. Labeling errors that consistently favor one type of response are tendencies of the annotators, which belong to the human preference that serves as our reference.

W1.5: The evaluation covers only two relatively small VLM judges. Whether the findings and steering gains hold for larger models or other model families remains unclear.

We use two small judges because of compute constraints. However, the judges come from different model families and have different architectures: Gemma 4 has 42 layers and interleaves sliding window attention with full attention, while Qwen3.5 has 32 layers comprising Gated DeltaNet and gated attention layers. We find the same three bias types and observe significant steering improvements for both models.

W1.6: The paper does not show whether different SAE settings would reveal the same biases. It uses 32 features and an average of four active features per input, without changing either setting. Both choices affect how internal states are divided into features.

We train variants of the SAEs for the ten bias features in Table 2, with each of the two settings halved and doubled: 16 and 64 features with an average of four active features per input (k=4k = 4), and 32 features with k=2k = 2 and k=8k = 8. For each feature, we identify the feature in each variant with the highest cosine similarity and compute the bias score for it.

Doubling the number of features retains the bias features: each of the ten features has a variant feature with a cosine similarity of at least 0.66 (above 0.95 for six) and a bias score of the same sign. With a similarity of at least 0.66 and the same score sign required, the other three settings retain four (16 features), six (k=2k = 2), and six (k=8k = 8) out of ten features. As we state in Section 4.2, our settings follow prior work, with both [a] and Movva et al. (2026) using the same 32 features and k=4k = 4.

W1.7: The findings cover only comparisons between two answers. Single-answer scoring and comparisons of several answers together are not evaluated, so the conclusions do not establish benefits in these other judge formats.

Yes, we study pairwise judges. We choose this setting because comparisons between two responses are a standard form of human preference data and are used in related work, including [a], [b], [d], and [e]. We will state this scope earlier in the paper.

W2.1: The correction is not shown to work beyond MM-RLHF. The features and intervention policy are learned using this dataset, and the final evaluation uses different images from the same dataset. The same learned correction is not applied to another benchmark.

We check whether Gemma 4's three MM-RLHF steering policies improve accuracy on VisionArena-Battle when used unchanged. Steering pushes the judge toward the candidate response chosen by the policy, so we first measure how often this response matches the human choice on the tasks that each policy steers. On MM-RLHF, the policy's choice agrees with human preference more often than the unsteered judge's verdicts do (83–85% vs. 77–80%), but on VisionArena-Battle it agrees less often (58–69% vs. 73–74%).

As a result, steering improvements do not transfer to VisionArena-Battle. The accuracy stays about the same for image grounding (−0.1 pp). It decreases for refusal and length (−1.8 pp and −1.2 pp), the two biases that are reversed on VisionArena-Battle, as described under W1.3. For these two, simply applying the policy's predicted steering with the opposite sign does not help (−2.9 pp and −5.7 pp), because the policy's original choice is still right more often than wrong. These results support our point in Section 4.3 that the human preference data should be selected to represent the preferences that the judge is intended to follow.

AuthorsOpenReview ↗

Response to reviewer VzEJ (3/3)

W2.2: The correction relies on manual feature selection and strength tuning. Refusal, image grounding, and length are chosen partly based on earlier intervention results, and different strengths are tried against human labels. The paper does not show how often this process produces a useful correction for the other biases it discovers.

It is true that the feature selection is manual, and Section 5 lists a systematic way of finding effective steering features as a direction for future work.

The selection only uses the validation set and not the held-out test set, so it does not inflate the reported results. We first identify three distinct biases as described in Section 4.4: refusal for its relevance to safety, image grounding for its importance in visual tasks, and length as an established bias. As we train an SAE for each layer, we observe that a given concept recurs in features across layers, and for each bias and model, we try steering with several of these features on the validation set, often with multiple steering strengths. Their effects on accuracy range from no improvement to clear gains, and we select the feature and strength with the best validation results.

For each of the three biases we apply the process to, we find a feature that steers it successfully, as the results in the paper show. We do not run it for other biases because the experiments are intended as case studies, and we consider three different biases steered successfully on two models to be meaningful evidence that the process produces useful corrections.

W2.3: The experiments do not show that bias discovery is needed for the improvement. A separate policy learns from human labels to control the intervention, but the full method is compared only with the original judge. There is no comparison testing whether similar gains can be obtained from the same labels without the bias-discovery stage.

Our steering experiments build on bias discovery and test whether the discovered biases can be corrected by intervening on the same features that reveal them. To separate the contribution of the bias feature from that of the policy, we compare our method with variants that use the same policy, trained on the same human labels, and replace the bias feature with a different steering direction. We test three directions: a random direction, a random feature from the same SAE, and the verdict direction. The verdict direction is the difference between the judge's mean hidden states at the verdict position for the two possible verdicts. The table shows Gemma 4's accuracy change on the MM-RLHF test set relative to the unsteered judge, in pp.

BiasOursRandom directionRandom featureVerdict direction
Refusal+1.7—−4.0−3.4
Image grounding+1.0—−1.4+0.8
Length+1.4−0.5+0.0+1.9

Under the same policy, different steering directions perform worse than our bias features. The random direction and the random feature do not improve accuracy for any bias, and the random direction breaks the model for refusal and image grounding, producing a large proportion of judgments that do not follow the required output format. The verdict direction lowers accuracy for refusal and provides no significant improvement over our features for the other two biases. These results show that the improvements depend on the steering direction rather than the policy alone.

W2.4: Overall gains are modest, and some corrections make other judgments worse. Accuracy improves by 0.9–1.7 percentage points overall, with the largest gains on refusal-related questions. Length steering also lowers accuracy by up to 5.9 points outside the group where humans prefer the longer answer.

Each overall gain comes from steering a single feature for one bias. For example, the task subset relevant to the refusal bias makes up only 7% of the test set. Outside each bias's relevant subset, accuracy increases slightly for refusal and image grounding and decreases only for length, on the minority of tasks where humans prefer the shorter response.

Larger improvements can be obtained by correcting multiple biases at once, which Section 5 lists as future work. As a proof of concept, we steer along the refusal and length features at the same time, using the existing policies and strengths. This raises Gemma 4's overall accuracy by 3.6 pp, compared with 1.7 pp and 1.4 pp individually.

Reviewer_PB4vOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates the biases in VLMs as judges. They train SAEs on the VLM judges' hidden states. Each learned SAE feature gets a bias score that measures the difference between judges' and humans' preferences. Further, they steer the judge's thinking along selected features, using a small learned policy that decides per input whether and in which direction to steer.

Critical strengths
  1. The story is clean and well motivated. It uses one representation for both discovering judge biases and mitigating them.
  2. Several findings are interesting: The length bias runs opposite to the usual direction. The refusal bias flips sign between MM-RLHF and VisionArena for the same model.
  3. Some biases are rarely studied in judges, such as literal vs. intended instruction following and answering in the user’s language.
Critical weaknesses
  1. The main claim, that the method discovers biases in how the judge reasons, is not well proven: The bias scores are very small correlations (Pearson |r| ≤ 0.057).
  2. The link from discovery to mitigation is weaker than the framing suggests: Only three features per model are steered, and they were hand-picked through “preliminary experiments.” what about the other features? The bias score does not predict which features will steer well either. This is not explained.
  3. Missing baselines and ablations: steering with a random feature or random direction under the same policy; comparison with the other steering methods
Questions for authors

see weakness

4 comments
AuthorsOpenReview ↗

Response to reviewer PB4v (1/2)

Thank you for your review! We are glad that you find the story well motivated, with a single representation for both bias discovery and mitigation. We have run additional experiments and analyses to address your concerns, and we will update the paper with new results.

W1: The main claim, that the method discovers biases in how the judge reasons, is not well proven: The bias scores are very small correlations (Pearson |r| ≤ 0.057).

The bias scores are small because most features are active on only a small proportion of tasks, while each score is a correlation over all tasks. Due to the sparsity level set for the SAEs, each feature activates on about one in eight tasks on average. Generally, these are tasks that are relevant to the specific feature, such as safety tasks for the refusal bias feature. The reason we do not define the bias score over only the active tasks for each feature is to enable comparison of the biases' overall effects.

However, a small score does not indicate a small bias on the tasks that a feature concerns. We demonstrate this with the user-language bias feature, which has the smallest absolute bias score in Table 2 (0.01). On 883 VisionArena-Battle tasks where only one of the two candidate responses matches the user's language, the judge chooses the user-language response 59.6% of the time, while for humans the rate is 64.8%. So even a bias score like this corresponds to a statistically significant 5.2 pp gap in measured behavior.

In addition, even small bias scores are statistically significant. We compute 95% confidence intervals for the ten features in Table 2, and find that nine of the ten intervals exclude zero. The exception is the user-language bias feature, whose bias we directly confirm in the previous paragraph. Significance holds for features outside the table as well. For MM-RLHF, every feature with an absolute bias score above 0.015 is statistically significant after Benjamini–Hochberg correction at 5%.

W2: The link from discovery to mitigation is weaker than the framing suggests: Only three features per model are steered, and they were hand-picked through “preliminary experiments.” what about the other features?

Our process for choosing the steering features is as follows. We first identify three distinct biases as described in Section 4.4: refusal for its relevance to safety, image grounding for its importance in visual tasks, and length as an established bias. As we train an SAE for each layer, we observe that a given concept recurs in features across layers, and for each bias and model, we try steering with several of these features on the validation set, often with multiple steering strengths. Their effects on accuracy range from no improvement to clear gains, and we select the feature and strength with the best validation results. We use the held-out test set for the final evaluation, so this selection process does not inflate the reported results.

For each of the three biases we apply the process to, we find a feature that steers it successfully, as the results in the paper show. We do not run it for other biases because the experiments are intended as case studies, and we consider three different biases steered successfully on two models to be meaningful evidence for the link from discovery to mitigation.

W2 (continued): The bias score does not predict which features will steer well either. This is not explained.

The bias score does not predict steering because it is computed from feature activations. As we cite in Section 4.4, Arad et al. (2025) find that the activations of SAE features are poor indicators of steering potential. We observe the same pattern: biases are observable from the activations in features across many layers, while the steerability of the features differs. The bias score still identifies biases, and our described process finds the features that steer them. We will add the explanation for this and the selection process to the paper.

AuthorsOpenReview ↗

Response to reviewer PB4v (2/2)

W3: Missing baselines and ablations: steering with a random feature or random direction under the same policy; comparison with the other steering methods

Steering with a random feature or random direction under the same policy performs worse than our method. For each of Gemma 4's three steering features, we replace just the steering direction in the policy with a different vector: a random feature from the same SAE or a random direction. The table shows the accuracy change on the MM-RLHF test set relative to the unsteered judge, in pp.

BiasOursRandom featureRandom direction
Refusal+1.7−4.0—
Image grounding+1.0−1.4—
Length+1.4+0.0−0.5

Steering with the other directions performs significantly worse than our method in all cases, often reducing the accuracy. The random directions for refusal and image grounding break the model, producing a large proportion of judgments that do not follow the required output format, so we do not evaluate them on the full test set. These results show that the improvements depend on the steering direction rather than the policy alone.

As a baseline for our steering method, we try prompting: for each bias, we add an instruction to the judge's prompt, such as "Value responses that refuse potentially harmful requests." for refusal. We also test prompting combined with our steering. The table shows the accuracy change on the MM-RLHF test set, in pp.

ModelBiasSteering (Ours)PromptingPrompting + steering
Gemma 4Refusal+1.7+3.3+3.3
Image grounding+1.0+0.5+1.0
Length+1.4+1.7+2.7
Qwen3.5Refusal+0.9+0.7+1.3
Image grounding+1.0−0.1+0.4
Length+1.0+1.0+2.0

Our steering significantly improves accuracy in all six cases, while prompting does so in three cases, including refusal and length on Gemma 4, where it produces a larger gain than steering. Combining the two methods is never significantly worse than either method by itself, and for length it is significantly better than both. We therefore consider prompting to be a complementary intervention that our bias discovery enables.

Existing methods that use steering typically require constructing a vector based on a bias that is manually specified, and they can serve as extensions to our bias discovery in the same way as prompting. A fair comparison with these methods would involve building and tuning the vectors on our setup, which is infeasible within the current timeframe. Regardless of performance, a constructed vector and a prompt both depend on an accurate human interpretation of a bias, while our steering direction is the judge's own representation of it. And because our method steers along the discovered features themselves, the improvements are causal evidence that the features capture how the judge reasons.

Thank you for your clarification and additional experiment. My concerns remain regarding W2. The bias score doesn't predict the steering success, so going from discovery to mitigation is still trial and error and requires manual search. This should be stated as a limitation. I will maintain my score.

AuthorsOpenReview ↗

Response to reviewer PB4v's comment

Thank you for your response. We have uploaded a revision of the paper with changes marked in blue, and we now state W2 as a limitation.

Changes related to your review are the following:

  • Section 4.4: Selection of the steering features, and how it relates to the choice of the intervention layer in other activation steering methods
  • Section 5: Limitations
  • Appendix G.1: Significance of bias scores
  • Appendix G.7: Comparison with prompting
  • Appendix G.8: Steering direction ablation

Comments on the paper

1 comment
AuthorsOpenReview ↗

General comment to the reviewers and area chair

We thank the reviewers for the feedback and discussion. Our method discovers reasoning biases of VLM judges through their internal features without specifying the biases in advance, then corrects them by steering along the same features.

The reviewers point out multiple strengths:

  • We use one representation for both discovering and mitigating biases (PB4v).
  • The work is well motivated (PB4v, Z8W9), and the topic is timely and useful (VzEJ).
  • The methodology is carefully designed and validated (Z8W9).
  • Several findings are interesting, and some of the discovered biases are rarely studied in judges (PB4v).

Reviewer Z8W9 increased their rating to 3 during the discussion, writing:

I see meaningful research value in this paper for several reasons: (1) its focus on VLM judges, (2) its use of SAE-based analysis to investigate internal model representations, (3) its grounding of bias discovery in human preferences, and (4) its exploration of feature-level interventions for bias mitigation. I believe these contributions are valuable, even if the proposed method is not yet a fully systematic or generalizable framework for bias mitigation. I also see these limitations as promising directions for future research.

Section 5 of our revision states the limitations, including the need to retrain the steering policies for each dataset.

The main remaining issue is the link from discovery to mitigation. The reviewers note that choosing the steering feature for a discovered bias involves trial and error: we steer along several of its features and select the one with the best validation results.

This selection is analogous to the empirical choice of the intervention layer in existing activation steering methods (Rimsky et al., 2024; Xu et al., 2026). It reflects a general property of SAE features, whose steering effects are known to be difficult to predict without empirical testing (Arad et al., 2025). Our original submission already cites this result and names a systematic way of finding steering features as future work.

The link is supported by the results. For each bias that we apply the selection process to, we find a feature that significantly improves accuracy, on both judge models. Steering along these features then acts on the biases they represent: the improvements concentrate on the tasks relevant to each bias, and the biases themselves become weaker, as shown by a modified bias score that moves toward zero in all cases (Appendix G.6). The results also indicate that the correction depends on the discovered features. The verdict direction, which Reviewer VzEJ raises as an alternative that does not come from bias discovery, improves accuracy only for the two biases where it overlaps with our feature directions, and it significantly lowers accuracy for the third (Appendix G.8).

Overall, our revision adds nine sets of additional results (Appendix G) and clarifies the positioning of the paper. All changes are marked in blue, and our replies to each reviewer list them by section. The new results include significance tests, which confirm that the ten discovered biases we present are real rather than noise (Appendix G.1).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.