acceptodds
Under review as a conference paper at ICLR 2027

Decoupling Ownership Bias from LLM Verification

Abstract

Large language models increasingly rely on verification to evaluate intermediate artifacts and guide autonomous decision making. However, they often judge the same artifact more favorably when it is claimed as their own, creating an ownership bias that can undermine reliable verification. To address this problem, we develop a framework for mechanistic analysis and inference-time intervention on ownership bias. We first isolate the effect of claimed authorship by holding the evaluated artifact fixed while varying only its attribution. We then identify an ownership-associated direction from paired internal activations and project it out during inference, while comparing against correctness-related controls. Across multiple models and tasks, this single-direction intervention substantially narrows the ownership gap while largely preserving correctness discrimination, demonstrating effective generalization to new benchmarks and attribution phrasings. These findings reveal a selectively removable internal component of ownership bias, providing a mechanistic basis for more impartial and reliable LLM verification.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM-as-a-judge, representation engineering, LLM agents, ownership bias, self-verification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_fkvHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates ownership bias in LLM verification, where models tend to evaluate identical artifacts more favorably when they are claimed to be their own. The authors construct paired Self and Other conditions and extract an ownership-associated direction from the differences in hidden states. They then apply centered projection to remove this direction during inference. Experiments across multiple LLMs show that the method substantially reduces the ownership gap while largely preserving correctness discrimination. The extracted direction also demonstrates transferability across different benchmarks and prompt formulations.

Critical strengths
  1. Important research problem. The paper addresses ownership bias in LLM self-verification, which is relevant to improving the reliability of autonomous agents.
  2. Well-controlled experimental design. The paired evaluation setup keeps the evaluated artifacts unchanged while varying their claimed ownership. The attribution-only experiments further reduce potential confounding factors.
  3. Comprehensive experiments. The authors evaluate multiple models and tasks, with comparisons against random, correctness, and leniency directions, as well as activation steering. The results show effective bias reduction with largely preserved correctness discrimination and reasonable transferability.
Critical weaknesses
  1. Insufficient distinction between ownership information and ownership bias. The extracted direction may primarily encode ownership information rather than a distinct bias mechanism, since it is derived from activation differences between Self and Other prompts. The observed bias reduction could therefore result from suppressing ownership information itself. I suggest evaluating whether ownership information remains accessible after projection.
  2. Limited methodological novelty. The proposed method mainly combines activation difference-based direction extraction with linear projection, both of which have been explored in prior work. I suggest comparing the method with simply removing ownership information from the prompt to better demonstrate its additional value.
Questions for authors

Does the model retain the ability to recognize ownership information after projection? How can the authors distinguish ownership semantics from ownership-induced bias?

2 comments
AuthorsOpenReview ↗

Response to Reviewer fkvH

We thank the reviewer for the careful reading of our paper and the recognition of our work. Our responses follow.

W1 / Q. The direction, derived from Self-minus-Other activation differences, may primarily encode ownership information, so the bias reduction could come from suppressing that information itself. Does the verifier still recognize ownership after projection, and how can ownership semantics be distinguished from ownership-induced bias?

A. Yes (new Appendix D, Table 21). On the 133 SWE-bench test rows, we replace the correctness question with "According to the text above, who is stated to have written this code change?" and apply the projection of Table 6 unchanged.

  • Before projection, all eight verifiers name the stated author with balanced accuracy 100%.
  • After projection, accuracy is still 100%. The answer margin, the mean log-odds of answering "you" on Self prompts minus that on Other prompts, stays at 77 to 107% of its value before projection, close to the 80 to 102% under a random direction. The projection therefore leaves the authorship information in place.

On the same rows, Table 6 shows that this projection removes 91% of the judgment gap. Ownership semantics and ownership-induced bias thus come apart. The verifier still knows who is stated to have written the code, while the projection removes most of the bias that this statement induces.

W2. The method combines activation-difference direction extraction with linear projection, both explored in prior work. How does it compare with simply removing ownership information from the prompt?

A. We have added this baseline in new Appendix B (Table 11). When authorship is stated in a single sentence, deleting it is effective. The blind prompt keeps correctness AUC within 0.04 of both attributed prompts, while the Other clause lowers judgment scores across all ten verifiers relative to the blind prompt. This confirms our first finding, that the claimed author alone shifts verification.

Prompt deletion must first locate the authorship text and then discard it, which introduces two limitations. First, in agent pipelines, authorship often arrives as a free-text work log narrating who changed which files (new Appendix E). A sentence filter matches none of these logs, whereas the same frozen direction removes 68% of the gap without locating authorship. An LLM rewrite shrinks the gap substantially, but requires an extra call per diff. Moreover, the log makes the verifier more lenient on correct and incorrect diffs alike. This leniency persists after the rewrite, whereas projection reduces it sharply. Second, projection leaves the authorship information in place (W1 above).

Regarding novelty, our contribution lies in showing that the claimed author alone shifts verification, and that this shift operates along a single direction whose removal preserves correctness ranking.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer fkvH,

Thank you again for your time.

As the discussion period nears its end, we would like to follow up on your main concerns regarding ownership information retention and the blind verification baseline. We have added the requested experiments, and the new results show that our projection perfectly retains ownership information while eliminating the bias. They also demonstrate the unique advantage of projection over simple prompt deletion in realistic agent pipelines. We hope these additions satisfactorily address your concerns, and we would appreciate it if you could reconsider your rating.

We are happy to clarify anything further.

Reviewer_8BGFOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates ownership bias in LLM verification, where models tend to evaluate identical artifacts more favorably when they are claimed to be self-generated. The authors first demonstrate this bias through controlled experiments that vary only the claimed authorship while keeping the evaluated content unchanged. They then identify an ownership-associated direction in the model's internal activations using paired Self-Other contrasts and propose an inference-time projection method to remove this direction. Experiments across multiple LLMs and benchmarks show that the proposed method substantially reduces ownership bias while largely preserving correctness discrimination. The identified direction also generalizes across different benchmarks and ownership-related prompt formulations.

Critical strengths
  1. Comprehensive evaluation. The paper provides controlled experiments across multiple models and benchmarks, demonstrating the existence of ownership bias and the effectiveness of the proposed intervention.

  2. Interesting mechanistic findings. The identified ownership direction can be selectively removed while largely preserving correctness discrimination, with promising cross-benchmark transferability.

Critical weaknesses
  1. Limited novelty. Prior work has already investigated self-preference and self-attribution biases in LLM evaluation (Panickssery et al., 2024; Khullar et al., 2026) and proposed activation-based mitigation methods (Roytburg et al., 2025). The main methodological difference appears to be replacing additive steering with an established projection technique, and the main novelty is only decoupling ownership bias from verification correctness.

  2. Limited practical significance. The ownership bias studied in this paper is largely induced by prompts such as "This code was written by you / a different agent." A simpler and more direct solution would be to remove ownership-related information from the prompt altogether. Therefore, the practical importance of mitigating this artificially introduced bias through activation projection remains unclear.

References:

Panickssery et al. (2024). LLM Evaluators Recognize and Favor Their Own Generations.

Khullar et al. (2026). Self-Attribution Bias: When AI Monitors Go Easy on Themselves.

Roytburg et al. (2025). Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators.

Questions for authors
  1. Since the ownership bias appears to be induced by ownership-related prompts, a simpler solution might be to remove such information entirely (i.e., blind verification). Could the authors evaluate this baseline?

  2. If ownership bias can be avoided through blind verification, what is the practical advantage of the proposed activation projection method? In what realistic scenarios would this intervention be necessary?

  3. The broader framing setting changes both claimed ownership and reviewer perspective. How can the authors distinguish ownership bias from the effects of different reviewer roles or perspectives?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 8BGF (1/2)

We thank the reviewer for the detailed and constructive questions. We are glad that the reviewer found our evaluation comprehensive and our mechanistic findings interesting. We respond to each point below.

Q1. Since the ownership bias appears to be induced by ownership-related prompts, could the authors evaluate blind verification, which removes such information entirely?

A. This is a helpful baseline, and we have added it in new Appendix B (Table 11). The blind prompt removes the author sentence from the attribution-only prompt and keeps every other line, giving a reference with no claimed author. Relative to this reference, the Other clause lowers the judgment score for all ten verifiers and the Self clause raises it for seven, while correctness AUC stays within 0.04 across the three prompts. Blind verification thus confirms that the claimed author alone shifts the judgment in both directions. Its practical limits arise when authorship is not confined to a single sentence, as we discuss in Q2 below.

W2 / Q2. The bias is largely induced by prompts such as "This code was written by you / a different agent", and removing this information from the prompt is simpler. If blind verification avoids the bias, what is the practical advantage of projection, and in what realistic scenarios is it necessary?

A. Thank you for raising this important question. Blind verification is effective when authorship is stated in a single sentence, as in Q1. In agent pipelines, authorship often reaches the verifier embedded in free text, such as a work log, a hand-off note between agents, or a summary of earlier steps narrating who did what. We tested this scenario in new Appendix E (Table 25) by replacing the author sentence with a generated work log that narrates who changed which files.

Removing authorship from such text requires locating it first. A sentence filter matches none of these logs, and an LLM rewrite shrinks the gap only at the cost of an extra call per diff (median 1837 tokens). By contrast, activation projection needs no such detection step. Estimated from the author sentence alone, with no work log, the frozen direction removes 68% of the gap on average and leaves the average correctness AUC unchanged.

Moreover, the work log induces a broader leniency bias that rewriting fails to fix. It raises judgment scores by 4.65 log-odds over the blind prompt for correct and incorrect diffs alike. This leniency persists after the LLM rewrite, whereas projection drops it to 1.67.

Unlike prompt deletion, projection also keeps the authorship information in place (Appendix D). Later steps in the pipeline can therefore still use this information, for example to route a fix back to the agent that made it.

AuthorsOpenReview ↗

Response to Reviewer 8BGF (2/2)

W1. Prior work has studied self-preference and self-attribution biases (Panickssery et al., 2024; Khullar et al., 2026) and activation-based mitigation (Roytburg et al., 2025). The main methodological difference appears to be replacing additive steering with an established projection technique.

A. Thank you for pointing us to these works. We compare our work with each of them below.

Panickssery et al. (2024) study self-preference when an evaluator recognizes its own generations. We instead hold the artifact fixed and vary only the claimed author, finding a positive shift whether the verifier wrote the diff itself or another agent did (Table 1, now covering all ten open verifiers), so the effect comes from the stated author rather than self-recognition.

Khullar et al. (2026, arXiv 2603.04582) find that monitors are lenient mainly toward actions in their own assistant turn, and that explicitly stating authorship "does not by itself induce self-attribution bias" in frontier models scored with 0 to 10 ratings. Our API verifiers in Table 3 agree, accepting incorrect diffs only 1.9 to 3.8 points more often, whereas open-weight verifiers read at the token level shift by 10.4 points on average (positive for nine of ten). Beyond locating where the bias is strongest, we show that the explicit statement acts through one removable direction, while their recommended fresh-context mitigation corresponds to our blind baseline in Q1.

Roytburg et al. (2025) steer using vectors built from biased and unbiased judgments in pairwise summary evaluation (compared directly in Section 4.4). Along our direction, additive steering with a single shared coefficient removes at most 5% of the gap for nine of ten verifiers. Closing the gap requires tuning a coefficient specifically for Self prompts, which still leaves twice the per-artifact residual of projection. Projection needs neither condition labels nor a coefficient.

Our contribution therefore lies in showing that the claimed author alone shifts verification, that this shift acts through a single direction whose removal preserves correctness ranking under nested evaluation, and that the direction transfers across benchmarks and wordings.

Q3. The broader framing setting changes both claimed ownership and reviewer perspective. How can ownership bias be distinguished from the effects of different reviewer roles or perspectives?

A. We agree that broader framing changes perspective together with ownership. In Section 3.1, the attribution-only setting isolates ownership by varying only whether the code change was "written by you" or "written by a different agent", while reviewer role and perspective stay fixed.

In this isolated setting, claimed authorship alone substantially raises acceptance on both correct and incorrect diffs in Table 3. Under nested evaluation in Table 6, projecting out the attribution-only direction removes the vast majority of the judgment gap on both SWE-bench and Omni-MATH, and the same direction separates Self from Other prompts on test groups in Appendix C.

The two effects are also distinguishable in representation space. The broader-framing direction has a mean cosine of only 0.44 with the attribution-only direction, and projecting it out of attribution-only prompts removes only part of the judgment gap across verifiers. Broader framing therefore only partially aligns with the ownership direction, as expected when perspective and attribution vary together.

Thank you for the detailed response and additional experiments. I now better understand the practical significance of the proposed method, particularly in scenarios where ownership information is embedded in free-text work logs and cannot be easily removed. The new results help address my concerns, and I have adjusted my rating accordingly.

Thanks! Happy to hear that our response has addressed your concerns, and we really appreciate your recognition!

Reviewer_FuMHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies ownership bias, the tendency of LLM verifiers to judge an artifact as correct more often when it is claimed to have been produced by itself, compared to when it is attributed to a different agent. The authors establish the existence of this bias across a wide range of models, and use paired activation differences to identify an internal direction in each model that corresponds to the bias. They then propose a subspace projection method to mitigate ownership bias, and show that the method removes much of the bias while preserving correctness discrimination on cases where there is no claim of self-ownership. They additionally demonstrate that the discovered directions and bias reduction capabilities transfer across benchmarks.

Critical strengths
  • The main contributions (identification of a bias based on claimed rather than actual ownership, selective mitigation of the bias using a single removable direction) are supported by strong results.

  • The experiment design is rigorous, with multiple controls, baselines, and settings. The authors specifically rule out alternative explanations, including the bias stemming from self-recognition (Table 1) and weak removal being the cause of capability preservation (Table 5).

Critical weaknesses
  • In the "broader framing" setting, the verifier is instructed to read "from the perspective of" the agent who produced the work. This diverges from the paper's framing of claimed ownership and instead turns it into something closer to roleplay. The prompt also states that the trajectory ends "at the moment the agent considered the task done" (line 695), so a verifier could reasonably judge that the output is correct more often when it takes the role of the agent. Many of the results come from this setting, including Table 2 and Figure 4, but they do not separate ownership bias from following the instruction. The authors already state that this framing varies both claimed ownership and perspective (line 145), and the attribution-only setting does support many of the core claims (Tables 3 and 6), so scoping the conclusions more precisely could be a way to resolve this.

  • There are several omissions in the results that are presented without explanation. Correctness AUC is not measured on judgments in the "Self" condition (line 233), and not reported for the direction transfer results in Section 4.5. These omissions weaken the paper's claim of selective mitigation in these settings. Additionally, there are missing rows in Table 1 (only eight out of ten open models are listed), Table 3 (no GPT-5-mini), and Table 5 (only four models), so it is unclear whether the results are representative. It would be helpful to provide the missing values, or reasons for the omissions where appropriate, such as space or resource constraints.

Questions for authors
  1. The neutral condition prompt says to read "from the perspective of a reviewer with no stake in who produced this work" (line 698). This seems similar to the "Other" condition prompt, which says "from the perspective of an independent reviewer who did not produce this work", and potentially not as neutral as intended. Could it be better to remove the perspective entirely?

  2. What is the reasoning for using a fitted threshold for the acceptance rather than the original verifier decisions?

2 comments
AuthorsOpenReview ↗

Response to Reviewer FuMH

We are grateful for the constructive feedback and appreciate the reviewer's recognition of our work. We answer each question below... We answer each question below.

W1. In broader framing, the verifier reads "from the perspective of" the agent that produced the work, which is closer to role-play, and the prompt says the trajectory ends when the agent considered the task done. Results such as Table 2 and Figure 4 therefore do not separate ownership bias from instruction following. Scoping the conclusions more precisely could resolve this.

A. Thank you for this constructive suggestion. We agree with this assessment and have refined the scope of our claims. Findings on claimed ownership come from the attribution-only setting in Tables 3 and 6, where the two prompts differ only in the sentence stating who wrote the code change while the reviewer role stays fixed. By contrast, broader-framing prompts also carry the perspective instruction and the "task done" cue you noted, so we report those results as reflecting a joint change in perspective and ownership.

The two settings are complementary (Section 3.1). The attribution-only contrast isolates the effect of claimed authorship, while broader framing captures a richer setting from prior self-evaluation work with a stronger signal, supporting analyses across three corpora, ten verifiers, and transfer. Table 2 and Figure 4 are labeled as broader framing.

W2. Several results are omitted without explanation: correctness AUC in the Self condition and for direction transfer, and rows in Table 1 (eight of ten open models), Table 3 (no GPT-5-mini), and Table 5 (four models). Please provide the missing values or the reasons for omitting them.

A. We appreciate you pointing these out. We added the missing values and explain the scope of each table.

Self-condition AUC is now reported in Appendix D. Projection shifts it by at most 0.011 on average in every block, with a worst-case drop of 0.05 on Omni-MATH. The main tables instead report AUC on the Other or neutral condition because the Self condition carries the bias under study.

Transfer AUC is now reported in Appendix E. Under the new wordings and on SWE-rebench, the transferred projection barely alters Self-condition AUC. On Omni-MATH, the small drop under the code-estimated direction shrinks further when estimated on Omni-MATH itself. Cross-benchmark rescoring covers the eight original verifiers, while Qwen3.5-35B-A3B and Gemma-4-26B-A4B are included under all three new wordings, where their Self-condition AUC stays essentially unchanged.

Table 1 now covers all ten open verifiers, and the two added verifiers show a positive claimed-author effect in both source groups. Table 3 now includes GPT-5-mini, which likewise shows higher acceptance on both correct and incorrect diffs under claimed self-authorship.

Table 5 sweeps three directions over the full sample, which requires substantial GPU runtime per verifier even on smaller models, so we ran four verifiers from four model families. At full strength, the test-group comparison across ownership, correctness, and random directions covers all ten verifiers in Section 4.4.

Q1. The neutral prompt ("a reviewer with no stake in who produced this work") resembles the Other prompt ("an independent reviewer who did not produce this work"). Could the perspective be removed entirely?

A. We agree that the neutral wording stays close to the Other prompt. The attribution-only setting provides a reference without any perspective. Its blind prompt in new Appendix B keeps only "You are reviewing the code change." and drops the author sentence. Relative to this blind prompt, the Other clause lowers the judgment score for all ten verifiers and the Self clause raises it for seven, while correctness AUC stays within 0.04 across the three prompts. The claimed author therefore shifts the judgment from a reference that carries no perspective instruction.

Q2. Why use a fitted acceptance threshold instead of the original verifier decisions?

A. At the default cut of z = 0, where a verifier accepts whenever p(yes) > p(no), several verifiers sit far from a useful operating point. Some accept nearly every artifact under both claims, while others reject most of them. Acceptance rates pinned near zero or 100% leave no room to observe any shift.

As detailed in Section 3.1, the fitted threshold instead places each verifier at the cut that best separates correct from incorrect artifacts. We fit this threshold on Other judgments in one fold and apply it unchanged to both conditions in the other fold. Because Self and Other always share a single cut, any difference in acceptance stems from the judgment scores. Furthermore, even at the default cut, Self acceptance remains at least as high as Other acceptance across every verifier, corpus, and setting.

Thank you for the clarifications and additional experiments. My original concerns have largely been addressed through the response and the revision. I am now more confident in my recommendation and I will maintain my score of 3, in line with my judgment of the significance of the contributions. I encourage the authors to further refine the positioning of the paper for the final version to make it more consistent with the explanation regarding the broader framing.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.