acceptodds
Under review as a conference paper at ICLR 2027

Do LLM Evaluators Prefer Themselves for a Reason?

Abstract

Large language models (LLMs) are increasingly used as automatic evaluators in applications such as benchmarking, reward modeling, and self-refinement. Prior work suggests that these LLM evaluators exhibit *self-preference*: they favor their own outputs, often more strongly as model capability increases. However, whether self-preference behavior is really harmful, or simply reflects the genuinely higher-quality outputs of stronger models, has been overlooked. To resolve this, we study self-preference on verifiable benchmarks spanning mathematical reasoning, factual knowledge, and code generation. This setting enables us to separate *legitimate* self-preference (favoring objectively better responses) from *harmful* self-preference (favoring worse ones), different from past studies that focus on subjective tasks, where preferences are hard to verify objectively. Across large-scale experiments on Llama, Qwen, Gemma, Mistral, Phi, GPT, and DeepSeek models, we find three main results: *(1)* Stronger models show greater self-preference, but much of it is legitimate and tracks their genuinely better generations. *(2)* Harmful self-preference persists when models make mistakes as generators, and it becomes more pronounced in stronger models, suggesting they struggle more to recognize when they are wrong. *(3)* Inference-time scaling strategies, such as generating a long Chain-of-Thought before evaluation, effectively reduce harmful self-preference. Notably, we conduct further experiments with LMArena and observe similar patterns, showing our findings extend beyond verifiable tasks to subjective, real-world settings. Overall, our results offer a sharper picture of when self-preference is benign versus problematic, and point to practical ways to make LLM-based evaluation more reliable.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, LLM Evaluators, LLM-as-a-Judge

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_jsVHOpenReview ↗
Rating
2(+1)
Confidence
3(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper empirically investigates whether LLM evaluators’ preference for their own responses reflects superior response quality or harmful evaluation errors. Using verifiable benchmarks in mathematical reasoning, factual knowledge, and code generation, the authors distinguish legitimate self-preference from cases where a judge favors its own incorrect response over a correct alternative. Across multiple model families and scales, they find that stronger models generally exhibit more legitimate self-preference, but can also show a higher conditional tendency to favor themselves when their own responses are incorrect. The paper further examines these patterns on LMArena data and evaluates standard CoT prompting and distilled reasoning models as strategies for reducing harmful self-preference.

Critical strengths

Relevant research question. The paper investigates whether LLM self-preference reflects genuinely better responses or incorrect judgments in favor of the model’s own outputs. This distinction is practically relevant to LLM-based evaluation and self-improvement.

Systematic empirical evaluation. The study covers mathematical reasoning, factual knowledge, and code generation across multiple model families and scales. Using verifiable answers provides a stronger basis for assessing correctness than subjective preferences alone. The fixed evaluatee pool and response-order swapping also improve comparability and reduce position bias.

Useful descriptive findings. Separating legitimate self-preference from harmful self-preference highlights behavior that aggregate win rates can obscure. The same-model comparison between direct judgments and standard CoT provides useful evidence that additional reasoning can reduce erroneous self-favoring judgments.

Critical weaknesses

Limited incremental contribution relative to prior work. The distinction between response quality and self-preference bias has already been explicitly studied, notably in Beyond the Surface: Measuring Self-Preference in LLM Judgments [1]. Although this paper introduces verifiable benchmarks and conditional metrics, it does not sufficiently explain what substantially new understanding these additions provide beyond existing quality-aware analyses. So it is not appropriate to say "However, whether self-preference behavior is really harmful, or simply reflects the genuinely higher-quality outputs of stronger models, has been overlooked" in the abstract. Could the author reframe the words in the rebuttal phase?

The central interpretation is not adequately established. HSPP measures how often a model selects its own incorrect response over a correct alternative, but this does not by itself isolate self-preference bias. A model may reproduce the same misconception during generation and evaluation. Furthermore, stronger models may fail on systematically harder questions, making their conditional error subsets different from those of weaker models. Difficulty-controlled comparisons and independent judges evaluating the same response pairs would help distinguish these explanations.

The long-CoT results contain important confounders. Comparing instruction-tuned models with DeepSeek-R1-distilled models changes the model weights and training procedure, in addition to reasoning length and decoding settings. Consequently, the observed improvements cannot be attributed specifically to longer reasoning. The standard-CoT comparison offers cleaner evidence, but the broader inference-time scaling claim requires stronger controls.

Some conclusions are broader than the evidence supports. The correlation between model capability and harmful self-preference is weak on MBPP+ (r = 0.150), limiting the generality of the claimed trend. The LMArena extension also lacks a fixed query set and evaluatee pool across judges, and human preference labels should not be treated as equivalent to objective correctness.

[1] Chen, Zhi-Yuan, Hao Wang, Xinyu Zhang, Enrui Hu, and Yankai Lin. 2025. Beyond the Surface: Measuring Self-Preference in LLM Judgments. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 1653–1672. Association for Computational Linguistics.

Questions for authors

Could the authors move more results to main page but not in the appendix?

Additionally, For the reference: Israel Cohen, Yiteng Huang, Jingdong Chen, Jacob Benesty, Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correlation coefficient. Noise reduction in speech processing, pp. 1–4, 2009.

“Pearson Correlation Coefficient” contains duplicate names. The correct author order is Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen, with each author listed once.

Reviewer_iPcWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether LLM evaluators' preference for their own responses reflects response quality or harmful bias. The authors evaluate self-preference on verifiable math, knowledge, and code tasks, distinguishing correct self-selections from cases where the model prefers its own incorrect response. They find that stronger models often favor correct responses but still make harmful self-selections, and investigate CoT-based mitigation and generalization to Arena human preferences.

Critical strengths
  1. Distinguishing correct self-selections from harmful ones is useful for understanding self-preference. The results show that better overall judging performance can coexist with substantial errors when the model's own answer is wrong.

  2. The experiments cover several model families and use a fixed opponent pool and swapped response ordering. The capability-matched comparisons and the standard-CoT experiment using the same checkpoint provide useful additional evidence.

Critical weaknesses
  1. I am concerned about the interpretation that stronger models have more difficulty recognizing their own mistakes. HSPP in Figure 6 is calculated on each model's own incorrect responses, so stronger models may be evaluated on harder remaining errors. The capability-matched LSPR results help address differences in opponent quality, but do not resolve this difference in HSPP evaluation subsets. Table 27 also omits the conditional sample counts, and the correlation on MBPP+ is only 0.150, so the strength and scope of this conclusion need further examination.

  2. The long-CoT comparison changes more than the amount of inference-time reasoning. Unlike standard CoT, it uses distilled model weights and a different decoding temperature, and Appendix B does not clearly specify whether Figure 7 uses the same candidate responses throughout. These differences make it difficult to attribute the additional improvement to longer reasoning itself. A lower HSPP also does not show whether more responses are judged correctly or more pairs are assigned a tie.

Questions for authors
  1. Can the authors provide the HSPP numerators and denominators with uncertainty estimates? Does the MATH500/MMLU trend remain when error difficulty is controlled?
  2. Are the candidate answers and conditional subsets fixed across Figure 7's settings? How do the correct-choice, wrong-choice, and tie rates change, and is a reasoning-budget comparison within one checkpoint available?
  3. Which exact Arena source models are matched to each evaluator, and how are similar-family matches handled? Since the labels represent human preferences, I suggest describing these results as agreement with human preferences rather than objective correctness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.