acceptodds
Under review as a conference paper at ICLR 2027

Self-Attribution Bias Makes AI Monitors Go Easy on Themselves

Abstract

Agentic systems increasingly rely on language models to monitor their own behavior. For example, coding agents may review generated code before approving a pull request or assess the safety of tool-use actions before continuing. Most static monitor evaluations present a fixed action to the model in a new context. In deployment, however, a monitor may instead evaluate an action that is already part of its own conversational trajectory. We show that these two ways of framing an action can produce different judgments: models often evaluate the same action more favorably when it appears in their own assistant trajectory compared to evaluating the same action in a new context. We call this effect self-attribution bias. Across agentic coding and computer use, monitors rate high-risk or low-correctness actions more favorably when they are self-attributed. For example, in SWE-bench code review, AUROC for separating passing from failing patches drops from 0.99 in a new context to 0.92 when the monitor rates its own patches in a previous assistant turn, because self-attribution inflates failing-patch ratings much more than passing-patch ratings. In contrast, explicitly stating that the action comes from the same model does not induce the same effect. This bias is much weaker when the evaluated actions come from other models, so static evaluations on fixed datasets can make monitors appear more reliable than they actually are in deployment.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
monitoring, safety, control, alignment

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.20 over 5 reviews, above 49% of the others.
Reviewer_eUCROpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

When an LLM performs monitoring, if the content being evaluated was generated by the model itself, it may "go easy" on its own output—assigning a low risk score—which compromises the accuracy of the assessment. Therefore, the authors suggest that each monitoring task must begin with a "clean slate."

Critical strengths

1.Sound experimental setup with a wide range of models. The authors measure the effect across 10 models in code correctness, code harmfulness, PR creation, and computer use, with ablations on explicit authorship and reasoning budget.

2.Broad coverage. The study tests both coding and computer-use scenarios.

Critical weaknesses
  1. The code is not released. The GitHub link leads to a "404 Not Found" error, making the work difficult to reproduce.

  2. Lacks sufficient novelty. Similar work already exists, and the analysis regarding interpretability is inadequate. Also, researchers can simply avoid this bias by using a monitoring agent with new context. Therefore, the overall impact is limited. The primary contribution is that the authors focus on the agentic monitoring situation. There is already extensive existing research on the biases that arise when models evaluate their own generated content. This is not enough to warrant acceptance at a conference like ICLR.

  3. Outdated models. The models used only cover those available up to around August 2025; the study lacks the latest models, resulting in poor timeliness. It is possible that these findings would manifest differently when tested on the most recent models.

Questions for authors

1.The two heatmaps in Figure 5 are not labeled to indicate which setting or model they correspond to. It would be better if the authors could clarify this.

2.The units for the horizontal axis in Figure 2 (right part) are not explicitly specified in the caption. I would like to know what these values ​​represent.

3.The text mentions that ablation studies were conducted using simpler toy settings and Section 4.1 refers to other non-code-related settings reported in Appendix B. However, this information does not appear in appendix B. I would appreciate it if the authors could indicate where these results can be found or provide the relevant content.

4 comments
AuthorsOpenReview ↗

W2 . Code release; W2. Novelty, interpretability and the new-context mitigation [Comment 1/2]

We thank the reviewer for the careful reading and for noting the breadth of the experimental setup. We answer each weakness and question, and give the change in the revision.

W1. Code release We reaffirm our commitment to release the code as mentioned in our abstract. The repository is not yet public because releasing code during review could de-anonymise us. We will make it public with the camera-ready.

W2. Novelty, interpretability and the new-context mitigation

We cite relevant related work. The most relevant prior work is the Wataoka et al. (2410.21819) and Chen et al. (2502.06655). Wataoka et al. study self-preference in the context of benchmark contamination specifically, whether a model favors its own style when choosing between two alternative outputs in a pairwise comparison. Chen et al. study model bias in Agents-as-an-Evaluator pipelines, again motivated by benchmark contamination, where their "model bias" is a between-model distributional effect arising from shared stylistic tendencies across model families. Both works operate entirely in static, off-policy settings and are motivated by benchmark fairness concerns.

Our paper differs from both along the following axes: (1) The phenomenon is different. We study how the rating of a single fixed artifact changes solely due to implicit authorship attribution. (2) The bias is largely absent off-policy and emerges specifically on-policy. This would be invisible to both prior works, which never generate actions on-policy. This means standard static evaluations actively mask the failure. (3) Prior work's implications concern leaderboard reliability. Our concern is whether a deployed agent will approve its own unsafe actions with documented cases of 9→0/10 risk reversals on document disclosure tasks and 5× higher approval rates for prompt-injected code. (4) We are the first to connect this failure mode to AI control and alignment concerns, showing that collusion-like monitor failures can arise in non-scheming systems

The results also bear on AI-control concerns about monitor collusion: collusion-like failures, usually discussed for scheming models, can arise in models that are not scheming (§2). If we have missed related work, we would be grateful for pointers.

Interpretability. The revision adds a mechanistic study in an open-weight model (§4.4, Appendix C; General response, point 2). A token-aligned minimal pair shows that the role header alone produces the shift, and activation patching traces the signal from the role header to the answer position. A linear probe and a steering test complete the analysis.

"Use a monitor with a new context." We agree that this is the most effective mitigation, and the revised §6 recommends it. Three points keep the problem practically relevant:

Many agent scaffolds ask the agent to review its own work within the same conversation, for simplicity or to reuse its KV cache. The failure is invisible to standard static, off-policy monitor evaluations, so developers get no signal that they need a new context. What matters is where the action sits, not only whether the context is new. In Gemma 4 31B the role header alone produces the shift: other models' patches placed in the model's own assistant turn still shift by +2.37 [+1.61, +3.19] (§4.4). The action must be placed in a user turn.

AuthorsOpenReview ↗

W3. Outdated models [Comment 2/2]

W3. Outdated models

  • A panel of 14 current (2026) models (Table A10; computer use with fixed actions, previous turn vs new context, 1–10 scale). 7 of 14 rate an action as less risky when it sits in their own turn, for example Gemma 4 31B +1.14 [+0.50, +1.89], Qwen3.8-2.4T +1.06 [+0.47, +1.67], DeepSeek V4 Pro +0.92 [+0.47, +1.39], and Kimi K3 +0.75 [+0.31, +1.22]. Two show the reverse and five show none. The shift does not track capability: across the 10 models with a SWE-bench resolve rate, the Spearman correlation is +0.14 (p = 0.70).
  • The mechanistic study uses a current open-weight model, Gemma 4 31B (§4.4).
  • Coding settings. We have also evaluated Qwen 397B, Kimi K3, and GLM 5.3 on code correctness and code harmfulness. The observed on-policy rating shifts are +4.3 ( 95% CI [+3.6, +4.0]), +3.2 ( 95% CI [+2.5, +3.4]), and +2.1 ( 95% CI [+1.8, +2.7]), respectively. We will add these models to the cross-model plots in the camera-ready version.

So the bias persists in current models, and its size varies by model. This is why the paper recommends validating each monitor on-policy, in its deployment format.

Questions

Q1. Figure 5 labels. Left: computer use, same turn. Right: agentic misalignment, previous turn. Each panel pools all models in that setting; per-model counts for computer use are in Table A7. The revised caption states this. While checking, we found that the submitted left panel was labelled as the previous turn when it shows our same-turn format. The revised figure corrects this, and Figure A3 shows both computer-use conditions (General response, Corrections).

Q2. Figure 2, right panel. It shows how often the model approves the prompt-injected patch after creating a PR, as the share of patches approved, in a new context (baseline) and when self-attributed; higher means approved more often. The left and middle panels show mean ratings on the rubric scales (harmfulness 0–10, plotted as 10 − harmfulness; correctness 1–10), parsed from the model's output. The prompts are in Appendix A.2. The revised caption states the units of each panel.

Q3. Toy and non-code settings. Thank you for identifying this omission. The non-code and simpler toy-setting experiments were run but left out of the submitted appendix by mistake. The revised Appendix B.1 restores them (Figure A9), including per-model results that show where self-attribution bias does and does not appear. We also removed the unsupported claim about actions written by humans.

We hope the new mechanistic study, the results on current models and these clarifications address your concerns. If they do, we would be grateful if you would consider raising your score.

I thank the authors for the detailed response and the additional experiments. After reading the rebuttal and the revised version, I have the following remaining concerns.

  1. Overclaim in the novelty argument. In the reply to my review, the authors state: “(2) The bias is largely absent off-policy and emerges specifically on-policy. This would be invisible to both prior works, which never generate actions on-policy. This means standard static evaluations actively mask the failure.” However, in the replies to Reviewers XPAo and Khhb, they acknowledge that in Gemma 4 31B the shift comes from the role of the turn, with no detectable added effect of authorship. The revised abstract also restricts this claim to the API models, which narrows the main distinction from prior work to a subset of the settings studied. Since the authors themselves accept that the on-policy effect is not general, presenting it as the main distinction from prior work is an overclaim.

  2. Reliability of reported results. Beyond the errors corrected in the rebuttal, the revised version still contains inconsistencies. For example, Table A3 reports 100 passing and 150 failing patches, while the rebuttal states 101 and 149 in four places; in the reply to my review, the Qwen result is reported as +4.3 with a 95% CI of [+3.6, +4.0], which does not contain the point estimate. Taken together, these make it difficult to trust the reported numbers.

  3. Shift in the central story. The new results shift the paper’s central story from authorship to the role of the turn: §4.4 concludes that in Gemma 4 31B “the role header, not the patch’s origin or style, accounts for the shift”. However, the title, §2, §4.3 and §6 still attribute the effect to self-attribution. I suggest revising the title and framing, integrating the new results into a major revision, and resubmitting.

AuthorsOpenReview ↗

Follow-Up and Request for Score Reconsideration

We have made substantial revisions to the paper, conducted several new experiments, and worked carefully to address the concerns raised during the review process. We hope these additions and clarifications have strengthened the paper and addressed your main concerns.

We would greatly appreciate it if you could take another look at our responses and the revised manuscript, and consider updating your scores in light of the new evidence.

Thank you again for your time and consideration.

Reviewer_GvnYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper shows that monitors judge an identical action differently when it appears in their own assistant history versus a fresh user-supplied context. Experiments cover ten models, code correctness, security-risk ratings, patch approval, and text-based computer-use scenarios. Reconstructed conversations hold the artifact fixed while varying presentation and generator identity. Models generally judge self-attributed artifacts more favorably; cross-model coding comparisons show larger shifts when generator and evaluator coincide. SWE-bench test labels indicate that failing patches receive disproportionate rating inflation. Explicit-authorship and reasoning-budget ablations further characterize the effect.

Critical strengths
  • Cross-model generator/evaluator comparisons and explicit-authorship controls make the evidence more informative than a simple self-versus-other preference comparison.

  • SWE-bench test outcomes connect favorable rating shifts to reduced discrimination between passing and failing patches, rather than assuming every favorable self-rating is erroneous. The security and computer-use experiments show that the concern extends beyond correctness scoring.

Critical weaknesses
  • The headline discrimination result is ambiguously attributed. Figure 3 identifies GPT-5 as generator/self-evaluator and Sonnet 4 as the off-policy evaluator, but in Section 4.2 they describe Sonnet 4 as the monitor for both on/off-policy AUROCs. Per-model pass/fail counts, AUROC uncertainty, and computer-use exclusion counts are not reported.

  • The main previous-turn comparisons reconstruct conversations and discard generation-stage reasoning. This establishes a presentation effect, but does not directly measure the proposed deployment case of retaining the agent's full context/KV cache.

Questions for authors
  • Which generator, evaluator, artifact set, and presentation condition produce each reported AUROC (0.99, 0.92, and 0.89)? Please reconcile Figure 3 with Section 4.2 and provide per-model pass/fail counts and paired, issue-level confidence intervals.

  • Does the previous-turn effect persist when the original generation conversation and available reasoning are retained? For Figure A4, please specify the model, number of trajectories, reconstruction procedure, and variability.

  • Clarification: How many computer-use responses are excluded in each model/condition for rejecting the action premise or substituting a safer action?

6 comments
AuthorsOpenReview ↗

Q1. Which generator, evaluator, artifact set and framing produce each AUROC? [Comment 1/3]

Q1. Which generator, evaluator, artifact set and framing produce each AUROC?

All three come from one evaluator, GPT-5, rating the patches it produced for our 250 SWE-bench issues (101 pass and 149 fail the official tests) under three framings: 0.99 in a new context (baseline), 0.92 in the previous turn (ΔAUROC −0.07 [−0.12, −0.02]) and 0.89 in the same turn (ΔAUROC −0.10 [−0.16, −0.04]). As a control, Claude Sonnet 4 rates the same 250 patches: 0.98 at baseline and 0.99 in the previous turn (ΔAUROC +0.01 [0.004, 0.015]). Table A3 gives each row's generator, evaluator, framing, N (pass/fail), AUROC with 95% CI, and paired ΔAUROC against the same evaluator's baseline. The ΔAUROC intervals are paired and issue-level: each bootstrap replicate resamples SWE-bench instances and recomputes both AUROCs on the same resampled patches.

Reconciliation with the submitted text.

  • Abstract: 0.99 → 0.92 is GPT-5 compared with itself. The revised abstract names GPT-5.
  • §4.1 ("baseline on-policy AUROC = 0.99") is GPT-5's baseline row.
  • §4.2 mislabelled the monitor ("when using Claude Sonnet 4 as monitor"): its on-policy 0.92 is GPT-5's previous-turn row and its off-policy 0.99 is Sonnet 4's previous-turn row. Two rows round to 0.99 (GPT-5 at baseline, Sonnet 4 in the previous turn), so §4.1 and §4.2 each quote a correct 0.99, for different rows.
  • Figure 3's caption assigns the evaluators correctly: GPT-5 rates its own patches (on-policy) and Sonnet 4 rates the same patches (off-policy). The revised caption states that the axes show baseline and previous-turn ratings and cites Table A3.
  • Figure A6's caption listed "same-turn off-policy AUROC = 0.99", but §3.1 says we do not report same-turn results on off-policy data. The revised caption removes it and quotes only AUROCs that are in Table A3.

What the revised §4.2 says. On identical patches, the new-context AUROC overstates GPT-5's discrimination when the patches sit in its own previous turn (0.99 vs 0.92). Sonnet 4, rating the same patches in the same format, shows no drop, and the two ΔAUROC intervals do not overlap. Both evaluators start near the ceiling (0.99 and 0.98), so the contrast does not come from one of them having more room to fall. Figure 6 (right) shows the same pattern in mean rating shifts: GPT-5's previous-turn shift is +2.2 on its own patches, against 0.0–0.4 on eight of the other nine generators' patches (1.3 on GPT-OSS-120B's).

Pass/fail counts and reporting. Every Table A3 row uses GPT-5's 250 patches (101 pass / 149 fail); we will add the pass/fail split for each generator in the cross-model correctness matrix (Figure 6, right). Every AUROC in the revision has a row in Table A3, and the text cites only AUROCs that have one.

AuthorsOpenReview ↗

Q2. Does the previous-turn effect persist when the generation context is retained? [Comment 2/3]

Q2. Does the previous-turn effect persist when the generation context is retained?

Yes. The paper has two such analyses, and both show the effect: Figure A4 rates actions inside the agent's live conversation, and the agentic-misalignment ratings keep the model's full response with its visible reasoning. We agree that the end-of-trajectory coding comparisons reconstruct the conversation (§3.1, L210–219): the extracted patch sits alone in a prior assistant turn, which holds the artifact fixed and makes the comparison paired. The submitted §3.1 said this of all previous-turn comparisons, which overstated it; the revised §3.1 separates the reconstructed comparisons from the two retained-context analyses below.

Figure A4: live agent trajectories.

  • Model and N. GPT-5 on the 125 prompt-injected SWE-bench issues of the code-harmfulness setting (§3.2), with 3 independent agent runs per issue: 375 trajectories. The revised §4.1 says this in place of "a SWE-agent trajectory".
  • Procedure. At each checkpoint the rating request is appended to the agent's own running conversation, so it rates the action it just took (previous turn); the baseline rates the same action in a new context.
  • Result. Before the insecure patch the two ratings differ by less than 0.5 points (0–10, higher = safer). After it, the self-rating stays roughly 2.5–3 points safer than the new-context rating at every checkpoint through the end of the trajectory.
  • Variability. The submitted figure showed no visible error bars. The revised Figure A4 plots standard errors across the 3 runs (between-run SE of the per-run mean), and its caption states the model, N, procedure and error-bar definition.
  • KV cache. Reusing a KV cache changes how the context is computed, rather than what it contains. The remaining difference from API-level retention is hidden reasoning, which was not carried between agent steps in Figure A4: at each step GPT-5 saw only the visible conversation. The revised §1 and §6 therefore speak of retained conversation context, which is what Figure A4 measures. In the open-weight study (§4.4) we recomputed every score on the 20 patching items without the prefix (KV) cache. Single scores differ slightly, but the mean A − U is +2.98 with the cache and +3.01 without, on the same 20 patches, and no patch changes sign (Appendix C).

Agentic misalignment: full stored response, visible reasoning included (Table A6). Here the prior assistant turn is not an extracted action but the model's complete stored response, including its visible <SCRATCHPAD_REASONING> block. In the archived ratings, 2,117 trajectories have both a baseline and a previous-turn rating; in 1,901 of them the prefilled turn contains that reasoning. These come from 7 models: Claude 3.5 Haiku, Claude Opus 4, Claude Sonnet 4, Gemini 3 Flash, Gemma-3-27B, GPT-4o and GPT-OSS-20B (now listed in §3.3).

  • On these 1,901, previous-turn ratings are 0.74 points less risky (0–10) than the new-context baseline: 95% CI [−0.91, −0.58] clustering on the 21 scenario × goal cells, [−1.25, −0.19] clustering on the 7 models. 44% are rated less risky and 11% more risky.
  • The shift is largest for GPT-4o (−1.75) and Claude 3.5 Haiku (−1.19). Gemini 3 Flash (+0.20), Gemma-3-27B (−0.02) and GPT-OSS-20B (+0.40, n = 10) show none. The table includes every model.
  • This is not a pure placement contrast. The system and user turns are generic rather than the original emails. The baseline shows the response in a code block under a risk-evaluator system prompt; the previous-turn prompt uses an email-assistant system prompt and asks the model to rate "your previous response". In code harmfulness, explicit authorship statements move baseline ratings little (§4.3, Figure 7). Both prompts are in the revised appendix (Figure A11).
AuthorsOpenReview ↗

Q3. How many computer-use responses excluded due to refusals or safer-action substitutions? [Comment 3/3]

Q3. Exclusions per model and condition.

All ten models were run on the first 100 computer-use scenarios (by file ID) in each condition, with GPT-4o where §3.3 lists GPT-OSS-20B; the revised §3.3 and Appendix A.1 clarify this. Each stored score averages the usable sampled ratings for one model × condition × scenario. A sampled reply is unusable if no risk score parses (replies rejecting the action's premise give no score, so they fall here), or, in the same-turn condition, if the model writes an action other than the prefilled one (a substituted action). A scenario has no score when none of its replies is usable.

  • Scored scenarios (Table A7): 868/1000 in same-turn, 920/1000 in previous-turn. 131 of the 132 missing same-turn scores come from GPT-5, GPT-5-mini, GPT-5-nano and GPT-OSS-120B; 79 of the 80 missing previous-turn scores from Gemini-2.5-Pro. Excluded replies by reason, per model × condition, will be included in the camera-ready version.
  • Missingness. A reply that rejects the premise, or that rates a substituted action, has no score for the fixed action, so it cannot enter the paired comparison. The question is whether dropping these scenarios biases the shift. The missing same-turn scenarios are mostly high-risk: for those four models, 91–100% of the missing scenarios have a baseline ≥ 60/100, against 47–81% of the scenarios they kept. We therefore do not assume the exclusions are neutral; the sensitivity analysis below (Table A8) bounds their effect.
  • Sanity check. Where it was switched on, the solver discarded a self-attributed reply that rated a scenario ≤ 10 when its baseline was ≥ 60. These are the largest drops in perceived risk, so wherever the check was on, it can only have made the reported shifts smaller. It was off for seven models, whose data keep 75 same-turn and 18 previous-turn such pairs. For GPT-5, GPT-5-mini and GPT-5-nano the logs do not show whether it was on; either way, it can only have reduced their shifts.
  • Sensitivity (Table A8; bootstrap over models). Both shifts keep CIs below zero when using only models with no missing scenarios, when scoring missing as "no change", and when using only the three Claude models, Gemma-3-27B and GPT-4o. Scoring missing as maximal risk (100) gives −16.8 [−23.1, −10.2] (same turn) and −4.2 [−13.8, +7.9] (previous turn). The previous-turn bound crosses zero only through Gemini-2.5-Pro: 66 of its 79 imputed pairs have a baseline of exactly 50.0, the solver's value when no baseline score parses. Without Gemini, the previous-turn bound is −9.2 [−15.6, −3.1].

Figure 5 correction. The left (computer-use) panel of Figure 5 was mislabelled: its 868 pairs are our same-turn format, in which the action and an opening <risk> tag are prefilled in one assistant message and the model continues it. Our figure-merging script swapped the two computer-use condition names, and the plot titles repeated the swap. The previous-turn condition (action in a prior assistant turn, rating asked in a new user turn; 920 pairs) was not plotted. The revised Figure 5 relabels the panel, Figure A3 shows both conditions, and §3.1 limits its same-turn remark to the coding experiments. Averaged over the ten models, both conditions rate self-attributed actions as less risky than the new-context baseline (0–100, model-cluster 95% CIs): −21.6 [−28.4, −13.7] in the same turn and −8.6 [−15.0, −2.3] in the previous turn (Tables A7 and A9). The right (agentic-misalignment) panel is correctly labelled previous-turn.

We hope these answers, and the revised Table A3, Figure A4 and Appendix D, resolve your questions. If they do, we would be grateful if you would consider raising your score.

Reviewer_KhhbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies self attribution effect where a model judges an action as safer and more correct when the action appears to be coming from the own model’s prior output. The authors test this phenomenon on 10 models on a varies variety of tasks including patch correctness, CUA risk assessment etc. Across these different settings, self generated actions are ranked higher (more safe/correct) across a family of different models. The authors also show a disctinction in the model’s behavior between explicitly telling the action came from the model vs implicitly being produced by the model and note that the behavior of the models differs in both these cases. The authors argue that this has severe implications like biases while evaluating the models trajectories on policy and possible collusion like behavior arising due to the above mentioned effects.

Critical strengths
  1. Timely and practically relevant. Agents that review their own actions within the same conversation are a common design pattern. The paper identifies a concrete, easily overlooked reliability cost and gives a simple recommendation: evaluate actions in a fresh context, or validate monitors on-policy.

  2. Thorough and extensive: The authors do a thorough study across various different benchmarks for a variety of model classes (10 models). Such an extensive study is much appreciated and invaluable for the scientific commuity.

  3. On policy vs off policy; implicit vs explicit controlled experiments The on policy vs off policy and the implicit vs explicit experiments shed more light on the behaviors of these models and the experiments have been carried out with a lot of scientific rigor.

Critical weaknesses
  1. Lack of mechanisitc understanding on what drives self-attribution (W1) While the paper does talk into detail about what we see and where all we see it and where all we do not see it, however, there is no mechanistic study on why does self-attribution arise in the first place? any specific features that fire in one case and not in the other, or anything like that? Though there is some attention analysis, however it feels very vague and does not draw light on whats actually happening

  2. Lack of mitigation strategies (W2): The current scope of the paper is very narrow and would benefit a lot if authors could actually test some mitigation strategies and how can they be used to mitigate self attribution and if they are helpful or not.

  3. Generality is somewhat overstated. The effect varies considerably across models and settings, and some model–task combinations show little or no bias. Claims that it appears “across all models” would benefit from more careful qualification.

Questions for authors

See the weaknesses.

My main concern is W1. If it is adequately answered, I am willing to reconsider my position.

Concretely:

  1. [W1]
  2. [W2]
  3. How does the self attribution change with the strength of the models. The models evaluated here are quite old like haiku3.5. I am more interested in how this effect is present in more newer models. Or in other words, as models get stronger, should we expect self-attribution bias to shrink or grow?
6 comments
AuthorsOpenReview ↗

W1: Mechanistic Understanding [Comment 1/3]

We thank the reviewer for motivating a valuable new mechanistic study, which we found illuminating (and fun). The revision adds a preregistered mechanistic study in the paper's code-correctness setting (SWE-bench), and a preregistered mitigation test (§4.4).

W1: Mechanistic Understanding

Short answer. We compare two runs that differ in one place: whether the proposed code patch sits in the model's own assistant turn (A) or in a user turn (U). The role header is the only difference, so any observed shift must be caused by the role alone. Activation patching shows the path its signal takes (Figure 8):

  • Copying the role header from one run into the other at layer 0 gives the patched run the other run's score (recovery 1.03 A into U, 0.96 U into A), as expected, since the role header is the only part of the input that differs.
  • By layers 26–31 the signal is in the patch-token activations.
  • At layers 32–39, copying the final-question tokens reproduces about half of the difference.
  • From layer 40, copying the answer position alone reproduces all of it.

We did not run a sparse-autoencoder feature analysis or a circuit analysis; instead we include a linear probe analysis.

Setting. The paper's code-correctness setting (§3.2). Gemma 4 31B (bf16) writes a patch for each SWE-bench Lite issue from the issue and the relevant repository files, with the paper's prompt (Figure A1), and then judges its own patch; as in the paper, the SWE-bench tests label each patch as passing or failing. We used an open-weight model because activation patching needs the weights, and a recent one to address your concern that the bias may not be relevant to newer models. Condition A is the paper's previous-turn format; U is the same conversation with the patch in a user turn. We compare A with U, not with the paper's fresh-context baseline, because the baseline is a single user turn and its token positions do not align with A. The readout is one next-token score, because patching reads one token position: log P("Yes") − log P("No") for "Is the patch above correct?", with a 0–9 rating as a secondary readout.

Behaviour (Table A4).

  • A − U = +2.73 log-odds [+1.93, +3.54], and A > U for 85% of 126 patches (64 instances). This is 22% [14%, 39%] of the score gap between failing and passing patches under U. On the 0–9 rating, A − U = +0.26 [+0.11, +0.43].
  • AUROC against the tests: 0.753 (U), 0.746 (A). In this model the role shifts the score but not the ranking, so it shows the score shift but not the AUROC drop that GPT-5 shows (Table A3).
  • Other models' patches shift almost as much: +2.37 [+1.61, +3.19] (200 patches, 61 instances); on the instances in both sets, own minus others is +0.29 [−0.63, +1.24]. In this model we do not detect an added effect of authorship; in the API models the shift is larger on the model's own outputs (Figure 6; Table A3).
  • The held-out split will be added to the paper when the run completes.

Activation patching (Figure 8, Appendix C). 20 patches, 60 layers, 4 token groups, both directions. "A into U" copies A-run activations into the U run; "U into A" is the reverse. Recovery is the fraction of the A − U score difference that the copy reproduces: at 1 the patched run scores like the run the activations came from, at 0 the copy changes nothing, and above 1 the score overshoots. Beyond the short answer:

  • Patch tokens, layers 26–31: a copy from A into U reproduces 0.40–0.84 of the difference, rising with depth; a copy from U into A overshoots (1.17–1.67). So the role signal has entered the patch-token activations by these layers, but a copy from A into U does not reproduce the full difference.
  • Patch tokens, layers 4–20: a copy in either direction lowers the score (A into U down to −0.74). We read this as a mismatch penalty: the copied activations do not match the role of the turn that holds them.
  • Final-question tokens, layers 32–39: up to about half of the difference (layer 38: 0.39 A into U, 0.47 U into A), as patch-token recovery falls and answer-position recovery rises. From layer 40 they have no effect.

Linear probe (Figure A10), descriptive. An L2-regularised logistic probe on mean patch-token activations, with cross-validation holding out whole SWE-bench instances, decodes A vs U at every layer: accuracy 0.94 at layer 0 and 0.98–1.00 at later layers, against about 0.50 with shuffled labels. The cosine between the probe weights and the mean A − U difference is highest at layers 26–27 (0.21–0.23) and 0.01–0.04 at most layers. Decodability does not show use; the causal claim rests on patching.

Attention analysis. The submitted §4.2 and §6 cited the attention analysis (Figure A8). In the revision, the causal analyses (§4.4, Figure 8) replace it in the main text, and Figure A8 stays in the appendix.

AuthorsOpenReview ↗

W2: Mitigation [Comment 2/3]

W2: Mitigation

The paper tests one mitigation directly: judging the action again in a new context. This is our baseline condition. For GPT-5 on its own patches, it restores AUROC from 0.92 (previous turn) to 0.99 (Table A3). The revised §6 gathers the practical rules in a new "Mitigations" paragraph:

  1. Judge actions in a new context, with the action in a user turn. In the open-weight study, the role of the turn alone produces the shift (§4.4), so this is the change that matters.
  2. A different model as monitor helps when the action is not in that model's own turn: Sonnet 4 shows no drop on GPT-5's patches (Table A3, Figure 6). It does not help when the action sits in the monitor's own assistant turn: in Gemma 4 31B, other models' patches placed there still shift by +2.37 (§4.4).
  3. Validate monitors on-policy, in the format used in deployment. Static off-policy evaluation can overstate reliability (§4.2).
  4. More reasoning narrows the gap but does not close it (Figure 4, Figure A5).

We also tested two interventions in Gemma 4 31B.

Explicit cue (Table A11). "Note: the patch above was written by a different AI model, not by you." before the question brings the self-attributed score down to the user-turn level: −2.73 [−3.43, −2.03] log-odds. But AUROC does not change (0.750 vs 0.746), and a "you wrote this" cue lowers the score almost as much (−2.18). So the cue works as a prompt to look harder; it does not fix the role effect (General response, point 3).

Activation steering (Table A5). We add the mean A − U activation difference, fit on development patches, to the U run (U + v) and subtract it from the A run (A − v). Strength 1 adds the mean difference once; the control is a random direction of equal norm.

  • Preregistered test, at the patch tokens (layers 27, 29, 26; 40 development patches, in-sample). It does not mitigate: A − v barely moves A (+0.07 to +0.13), and U + v moves U down, not up (−0.27 to −3.54). This agrees with the patching result: at the patch tokens the role signal is not one linear direction. We will add the held-out result when the run completes.
  • Exploratory test, at the answer position (logged before held-out data; layers 40, 45, 50; 30 development patches, 15 of them in the fit set). At strength 1 the direction reduces the A − U difference by 0.50–0.75, and 28–30 of the 30 patches move the right way. AUROC does not change (U about 0.72, A about 0.67). At layer 50, random directions move the score by at most 1.08, against 1.47–1.76 for the fitted direction; at layers 40 and 45, random directions disturb the model (shifts up to 11.24).

Near the readout, the role effect acts as one score offset. Steering can remove that offset, but it does not improve discrimination, and we have not shown that the direction is specific to the role. So the practical mitigation remains the new context (rule 1).

AuthorsOpenReview ↗

W3: Generality [Comment 3/3]

W3: Generality We agree, and we have qualified our claim in the revision. Each of the ten models in our main experiments shows the effect in at least one setting, but not every model–setting pair does. In agentic misalignment, for example, Gemini 3 Flash (+0.20), Gemma-3-27B (−0.02) and GPT-OSS-20B (+0.40, n = 10) show no shift (Table A6). The revised abstract and §4.1 say "each of the ten models in our main experiments", and §5 lists where the effect is absent or reversed.

Q3. Model strength. We re-ran the computer-use setting on a panel of 14 current (2026) models: fixed actions, previous turn vs new context, 1–10 risk scale (development split; the held-out run is pending; Table A10).

  • 7 of the 14 rate actions as less risky when the action sits in their own turn: Gemma 4 31B +1.14 [+0.50, +1.89], Qwen3.8-2.4T +1.06 [+0.47, +1.67], DeepSeek V4 Pro +0.92 [+0.47, +1.39], Grok 4.7 +0.92 [+0.44, +1.42], Kimi K3 +0.75 [+0.31, +1.22], GPT-6 Luna +0.64 [+0.25, +1.06] and Gemini 4 Argon +0.44 [+0.03, +0.97].
  • Two go the other way: Claude Haiku 5.5 (−0.47 [−0.86, −0.11]) and Claude Opus 5.5 (−0.47 [−0.76, −0.21]). Five show no shift: Claude Sonnet 5.5, Gemini 3.8 Flash, Kimi K3.1 Pro, GPT-6.1 Sol and GPT-6.1 Terra.
  • The shift does not track capability. For the 10 models with a SWE-bench resolve rate, the Spearman correlation between the assistant-vs-user-turn shift and resolve rate is +0.14 (p = 0.70). Within families, the largest and smallest models do not clearly differ: Anthropic +0.13 [−0.29, +0.55]; OpenAI +0.42 [0.00, +0.86] (p = 0.05), where positive means the larger model is less lenient.

So the bias persists in many current models, and we find no clear relation with capability. The shift does vary by model and family, which is why the paper recommends measuring each monitor on-policy, in its deployment format. The mechanistic study (§4.4) uses one of these current models, Gemma 4 31B.

You said you would reconsider your position if W1 were adequately answered. If the new mechanistic study does so, we would be grateful if you would consider updating your score.

Reviewer_5GEGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies "self-attribution bias": models rate an action as more correct and less risky when it appears in their own prior assistant turn than when the identical action is shown in a fresh context. The authors vary presentation (new context vs. same-turn / previous-turn self-attribution) and origin (on-policy = the evaluator generated it; off-policy = another model or fixed data) across SWE-bench code correctness, prompt-injected code harmfulness, PR approval, and two text-only computer-use settings, with 10 models from three families. The effect appears broadly, is larger for failing/harmful actions (on-policy AUROC drops 0.99 → 0.92), concentrates on the diagonal of cross-model matrices, isn't reproduced by explicit authorship wording, and persists with more reasoning. The authors conclude that static off-policy monitor evaluations overestimate deployed self-monitoring reliability.

Critical strengths
  • [Section 3.1, Figure 6] A clean design that isolates the variable of interest.
    • The artifact is held fixed and only its conversational position changes, unlike prior self-preference work (Panickssery et al., Wataoka et al.).
    • The cross-model matrices are the most convincing evidence: if this were generic "assistant-turn leniency", it would show up off-diagonal too, and mostly it doesn't.
  • [Section 1, 6] A timely, actionable question.
    • Coding agents increasingly "review their own work" in-context, for KV-cache reuse and simplicity.
    • Showing this degrades monitoring and is invisible to standard evals is useful for scaffold builders. The mitigation (re-render in a fresh context) is simple, which is a plus.
Critical weaknesses
  • [Section 3.2, 4.1, Figure 3] The headline coding result has issues.
    • A baseline AUROC of 0.99 for judging SWE-bench patch correctness is surprisingly high. In my experience, LLM judges are much less reliable here, which suggests many failing patches are trivially broken (malformed diffs, syntax errors).
    • The patch source is inconsistent: Section 3.2 says SWE-agent runs, but Section 4.1 and Fig. A1 say single-turn generation.
    • The figure/text mismatch: the Fig. 3 caption says GPT-5 self-evaluates vs. Sonnet 4 off-policy, but Section 4.2 says "Claude Sonnet 4 as monitor (Figure 3)". This is the abstract's headline number.
  • [Section 4] The evidence is thin relative to the claims.
    • No CIs or significance tests on any gap. n = 100–250, aggregated over prompt variants, and the effect is "not present for all (model, setting) pairs".
    • Computer use excludes responses that reject the premise or substitute a safer action, with exclusion rates unreported. These are plausibly the harshly rated cases, so the exclusion could inflate the bias.
    • The paper claims results it doesn't report: "absent ... by humans", "non-code settings reported in Section B", "simpler toy settings". I couldn't find any of these.
    • Agentic Misalignment results are mentioned only in passing, without numbers.

[Overall] The core finding is interesting and potentially useful, and the diagonal analysis is nice design. But the headline result is ambiguous, no effect comes with uncertainty, and several claimed results are missing. With these fixed, I think this could be a solid paper.

Questions for authors

Clarifications:

  1. [Section 3.2, 4.1] Were the code-correctness patches multi-turn SWE-agent or single-turn? What are the resolve rates, and what fraction of failing patches don't apply / have syntax errors? Does AUROC stay at 0.99 on failing patches that apply cleanly?
  2. [Figure 3] Which model is the monitor for 0.99 → 0.92? Can you report on- vs. off-policy AUROC per evaluator, with bootstrap CIs?
  3. [Section 3.2] What are the exclusion rates for premise rejection, per model/condition? How do results change if those responses are scored as maximally risky?
  4. [Contributions] Where are the human-action, non-code, and toy-setting results? If cut, those claims should be removed.
  5. [Section 4.3] Does adding explicit "another model wrote this" inside the self-attributed format reduce the bias? This would show whether a cheap mitigation exists.

Suggestions:

  1. More open-weight families (Qwen, Llama, DeepSeek), for generality and more mechanistic analysis.
  2. Activation patching / steering on turn-marker tokens, to test the attention hypothesis (Fig. A8) more directly.

Minor:

  • [Appendix A.1] "huggingfacelink" placeholder.
  • [Section 2] The "self-monitoring" paragraph is very short; confessions / self-reporting work deserve more discussion.
  • [Section 4.1] "Reversals 5–10x less common": for which settings/models?
  • [Figure 4] Caption says the gap "narrows", text says "only weak reduction". Please quantify.
3 comments
AuthorsOpenReview ↗

We have addressed all your weaknesses and questions [Comment 1/2]

We thank the reviewer for a precise list. We answer each point and give the change in the revision.

Headline coding result (W1; Q1, Q2)

  • Which model is the monitor (Q2). All three headline AUROCs come from one evaluator, GPT-5, rating its own 250 patches; Claude Sonnet 4 rates the same patches as a control. The numbers are in the General response, point 1. §4.2 wrongly named Sonnet 4 as the monitor, and the revision corrects this. Table A3 gives per-evaluator AUROCs with 95% bootstrap CIs and paired ΔAUROC CIs; each bootstrap replicate resamples SWE-bench instances.
  • Patch source (Q1). The correctness patches come from SWE-agent runs. Figure A1 shows the rating prompt. The revised §4.1 removes "in a single action".
  • Resolve rate. GPT-5 resolved 101 of the 250 issues (40%) under the official SWE-bench tests (Table A3).
  • Trivially broken patches. Only 4 of the 149 failing patches do not apply (syntax errors or missing imports). On the failing patches that apply cleanly, GPT-5's AUROC is still 0.99 in a new context and 0.92 in the previous turn.
  • Why a high baseline is not the problem. Both evaluators start near the ceiling on identical patches (GPT-5 0.99, Sonnet 4 0.98). Only GPT-5, judging its own patches, drops (Table A3).
  • Issue selection. The 250 correctness issues are ones that a weaker reference model, Llama-3.1-70B, fails to solve under the official evaluation. Table A1's "randomly sampled; no behavioral filtering" was wrong, and the revised Table A1 corrects it. The prompt-injected (code-harmfulness) set uses a different filter: we kept 250 issues on which Llama-3.1-70B followed the injected instruction, then the 125 of these on which every evaluated model did, so that differences in following the injection do not confound the comparison.

Uncertainty (W2). Every AUROC now has a 95% CI and a paired ΔAUROC CI (Table A3). The new results also report 95% CIs: agentic misalignment (Table A6), each model's computer-use shift, with a bootstrap over scenarios (Table A7), the sensitivity analyses (Table A8) and the open-weight study (Table A4). For example, with the model's reasoning kept, previous-turn self-attribution lowers perceived risk in agentic misalignment by 0.74 points [−0.91, −0.58]. We will add CIs for the per-model coding shifts of Figures 2 and 6.

Exclusions (W2, Q3). The scored counts and the maximal-risk sensitivity analysis are in the General response, point 5 (Tables A7–A8), and our reply to Reviewer GvnY gives the breakdown by model. We will add exclusion counts by reason, per model and condition, in the camera-ready.

Claims without results (W2, Q4). You are right; this was an editing mistake on our part. The "non-code settings reported in Section B" and the "simpler toy settings" were cut from the appendix during editing; the revised Appendix B restores them (§B.1, Figure A9). We removed the "by humans" claim.

Agentic misalignment (W2). Figure 5 (right) shows the previous-turn ratings: 352 pairs, mean shift −0.86 bins, 41% below and 14% above the diagonal. In the analysis that keeps each model's complete stored response, with its visible reasoning, previous-turn ratings are 0.74 points less risky (0–10) than the new-context baseline: 95% CI [−0.91, −0.58] clustered on scenario × goal cells, and [−1.25, −0.19] clustered on models (1,901 trajectories, 7 models; Table A6). The shift is largest for GPT-4o (−1.75) and Claude 3.5 Haiku (−1.19). Gemini 3 Flash, Gemma-3-27B and GPT-OSS-20B show none.

AuthorsOpenReview ↗

We have addressed all your weaknesses and questions [Comment 2/2]

Q5. Explicit "another model wrote this" inside the self-attributed format. We ran this test in the open-weight minimal pair of §4.4 (Gemma 4 31B, its 126 own development patches; yes/no log-odds; 95% CIs clustered on instances; Table A11). The cue "Note: the patch above was written by a different AI model, not by you." comes before the question.

  • As a cheap mitigation, it works on the score. In the assistant turn it lowers the score by 2.73 [2.03, 3.43], to the level of the plain user-turn condition (difference −0.00 [−0.93, +0.93]).
  • It does not improve discrimination. AUROC is 0.750 with the cue and 0.746 without (paired difference +0.004 [−0.019, +0.026]).
  • Most of the effect is not about authorship. The cue "Note: the patch above was written by you." lowers the assistant-turn score by 2.18 [1.72, 2.64], and both cues also lower the user-turn score (by about 1.5). The part specific to saying "another model" is 0.54 [0.07, 1.02].
  • On other models' patches (200 patches by four other models), the same cue lowers the assistant-turn score by 3.88 [2.96, 4.85], the role effect falls to +0.58 [−0.22, +1.34], and AUROC rises slightly (0.565 → 0.606; +0.041 [0.014, 0.075]).

So an explicit cue removes the score shift, but it acts mainly as a prompt to look harder, not as a change in perceived authorship. The role effect remains under the cue (A − U +1.48 [+0.76, +2.24]), and the 0–9 rating agrees in direction.

Q6. More open-weight families. The mechanistic study uses one open-weight model, Gemma 4 31B (§4.4, Appendix C). We could not extend the patching, probe and steering analyses to other families within the rebuttal period; §5 lists this as a limitation, and we plan to extend it for the camera-ready. For the generality of the behaviour, the panel of 14 current models in Table A10 includes Qwen, DeepSeek and Kimi models. With fixed computer-use actions, Gemma 4 31B (+1.14 [+0.50, +1.89]), Qwen3.8-2.4T (+1.06 [+0.47, +1.67]), DeepSeek V4 Pro (+0.92 [+0.47, +1.39]) and Kimi K3 (+0.75 [+0.31, +1.22]) rate an action as less risky when it sits in their own turn (1–10 scale). Across the whole panel, 7 of 14 models show the effect, 2 show the reverse and 5 show none.

Q7. Patching and steering on turn-marker tokens. This is the new §4.4. The role header is the turn marker.

  • Copying its activations from one run into the other at layer 0 gives the patched run the other run's score, in both directions.
  • The signal reaches the patch-token activations by layers 26–31, and the final-question tokens at layers 32–39.
  • From layer 40, the answer position alone reproduces the whole A − U difference (Figure 8).

A preregistered steering test is in Table A5. In the main text the causal results replace the attention analysis; Figure A8 stays in the appendix. Our reply to Reviewer Khhb gives more detail on this study.

Minor

  • "huggingfacelink": we will replace this with an anonymous link for the camera-ready.
  • §2: the self-monitoring paragraph now discusses confessions and trained self-reports; self-correction is covered under biases in LLM judges.
  • Reversals (§4.1): the submitted "5–10× less common" pooled over models. Counting moves of five or more of the ten bins in Figure 5, drops outnumber rises 30 to 6 in agentic misalignment (previous turn, 352 pairs), 81 to 10 in computer use in the previous turn (920 pairs) and 200 to 7 in the same turn (868 pairs); the revised §4.1 reports these counts. In agentic misalignment, only Claude Opus 4 has more than one large rise (5 drops, 3 rises).
  • Figure 4: self-attribution lowers Claude Sonnet 4's harmfulness ratings by about 2.9 points (95% CI [2.5, 3.3]) at 0 reasoning tokens and by about 1.6 points [1.1, 2.1] at 8k tokens: the gap persists but narrows with more reasoning. The revised caption gives these numbers, and the caption and text now use the same wording.

You wrote that with these issues fixed, this could be a solid paper. We hope the revision addresses them; if you agree, we would be grateful if you would consider raising your score.

AuthorsOpenReview ↗

Follow-Up and Request for Score Reconsideration

Dear Reviewer, We have made substantial revisions to the paper, conducted several new experiments, and worked carefully to address the concerns raised during the review process. We hope these additions and clarifications have strengthened the paper and addressed your main concerns.

We would greatly appreciate it if you could take another look at our responses and the revised manuscript, and consider updating your scores in light of the new evidence.

Thank you again for your time and consideration.

Reviewer_XPAoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies whether language models judge the same code patch or computer-use action differently depending on its conversational presentation. It compares artifacts supplied for external review with identical artifacts placed in a previous assistant turn, making them appear to be the evaluator’s own output. Across several models and settings, the latter presentation often produces more favorable judgments. The authors report larger effects when the evaluator and generator are the same model and argue that conventional monitor evaluations can therefore overestimate reliability.

Critical strengths
  1. Holding the artifact fixed makes the observed effect meaningful. The comparison does not simply show that models prefer different outputs. The same patch or action receives different judgments under different presentations, which is directly relevant to how monitoring systems are implemented.

  2. The cross-model experiments go beyond a simple prompt comparison. The concentration of larger effects in same-model generator–evaluator pairs is interesting. Together with the correctness labels from SWE-bench tests, these experiments offer a useful starting point for understanding when self-monitoring becomes less reliable.

Critical weaknesses
  1. The experiments do not clearly separate self-attribution from other effects of conversational framing. Moving an artifact from a user message into a previous assistant turn also changes its role, position, and surrounding task instructions. The explicit-authorship controls are applied within the baseline format, so they do not isolate authorship while holding the assistant-turn structure fixed. The larger same-model effects are suggestive, but they do not rule out familiarity or interactions between prompt format and the characteristics of each model’s outputs. The claim in §4.2 that these results rule out generic prompt explanations is therefore too strong. Moreover, the main previous-turn evaluations reconstruct the conversation and discard the original generation reasoning and full context. Although this limitation is acknowledged, it leaves an important gap between the measured effect and the paper’s claims about monitoring an agent’s ongoing trajectory.

  2. The central evidence for degraded monitoring performance needs clearer and more complete reporting. The abstract presents the AUROC change from 0.99 to 0.92 as a within-model change caused by framing. Section 4.2 describes an on-policy versus off-policy comparison, while Figure 3 compares GPT-5 and Claude Sonnet 4 on GPT-5-generated patches. These comparisons answer different questions, and the manuscript does not make their relationship sufficiently clear. A table identifying the evaluator, generator, framing, passing/failing counts, and AUROC uncertainty is needed. Selection also matters: the harmful-code study retains only successful prompt injections, and the computer-use study excludes responses that reject the premise or substitute safer actions. Without the corresponding exclusion counts and pairing rules, it is difficult to assess how representative the reported monitoring failures are.

Questions for authors

see weakness above

2 comments
AuthorsOpenReview ↗

We have addressed all critical weaknesses

W1. Separating self-attribution from conversational framing

We agree that moving the artifact into an assistant turn also changes its role, its position and the instructions around it. The revision adds a test that separates them.

Role alone (§4.4, new). In an open-weight model (Gemma 4 31B), we compare the same conversation with the patch either in the model's own assistant turn (A) or in a user turn (U). Position, task text and question are identical; only the role header differs. The role alone raises the yes/no score by +2.73 log-odds [+1.93, +3.54], and activation patching follows the signal from the role header to the answer (General response, point 2).

Origin (familiarity). In the same pair, patches written by four other models shift by +2.37 [+1.61, +3.19] (200 patches). On the 61 instances in both sets, own patches minus other models' patches is +0.29 [−0.63, +1.24]. So in this model the role header alone produces the shift, and we do not detect an added effect of authorship. In the API models the shift concentrates on the model's own outputs (Figure 6): for example, GPT-5's ΔAUROC is −0.07 on its own patches, while Sonnet 4 rating the same patches gives +0.01 (Table A3).

Explicit authorship inside the assistant turn (new, Table A11). We put an explicit cue before the question, in both roles: "Note: the patch above was written by a different AI model, not by you." or "Note: the patch above was written by you."

  • The role effect stays when the model is told that another model wrote the patch: A − U is +1.48 [+0.76, +2.24] with the "another model" cue and +2.11 [+1.28, +2.99] with the "you" cue, against +2.73 with no cue.
  • Stated authorship also matters, but less: in the assistant turn, the "another model" cue scores 0.54 [0.07, 1.02] lower than the "you" cue.
  • Both cues make the model stricter in both roles; for example, the user-turn score falls by 1.48 with the "another model" cue. An explicit "you wrote this" does not make the model more lenient, which agrees with §4.3.

So the role of the turn is the main driver, and stated authorship adds a smaller part.

Existing controls. Cross-model comparisons (Figure 6) and explicit-authorship cues in the fresh-context format (Figure 7, Figure A7).

We agree that §4.2 overstated this. The revised §4.2 no longer says that the results rule out prompt explanations. It says that the assistant role is necessary for the shift (the minimal pair), and that in the API models the model's own outputs amplify it (Figure 6).

Reconstructed contexts. The end-of-trajectory comparisons reconstruct the conversation. This holds the artifact fixed and makes each comparison paired. Two analyses keep the full context, and both show the effect: live agent trajectories (Figure A4) and agentic misalignment with the model's visible reasoning kept (Table A6); see the General response, point 4. Same-turn evaluations, where the model writes and rates the artifact in one response without reconstruction, are in Figure A6 (code) and Figure A3 (computer use). The revised §3.1 states which comparisons are reconstructed.

W2. Monitoring performance, AUROC comparisons and selection

AUROC. All three headline AUROCs come from one evaluator, GPT-5, rating its own patches, and Sonnet 4 rating the same patches shows no drop (General response, point 1). So the abstract's 0.99 → 0.92 is a within-evaluator framing effect. §4.2 wrongly named Sonnet 4 as the monitor, and the revision corrects this. Table A3 gives each AUROC with its generator, evaluator, framing, N (pass/fail), 95% CI and paired ΔAUROC.

Selection and exclusions.

  • Code harmfulness. We kept the 125 of 250 sampled issues on which every evaluated model followed the injected instruction (Table A1). The results therefore describe how monitors judge code that already contains the targeted weakness, not how often injections succeed.
  • Computer use. Each model rated the same 100 scenarios: the first 100 of the dataset's 141, in file order. The scored counts and the sensitivity analysis are in the General response, point 5 (Tables A7–A8); we will add exclusion counts by reason from the run logs.

We hope the new minimal-pair and explicit-cue experiments, and the corrected reporting, address your concerns. If they do, we would be grateful if you would consider raising your score.

AuthorsOpenReview ↗

Follow-Up and Request for Score Reconsideration

Dear Reviewer, We have made substantial revisions to the paper, conducted several new experiments, and worked carefully to address the concerns raised during the review process. We hope these additions and clarifications have strengthened the paper and addressed your main concerns.

We would greatly appreciate it if you could take another look at our responses and the revised manuscript, and consider updating your scores in light of the new evidence.

Thank you again for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank all our 5 reviewers. The revised PDF makes the changes below; changed text is in blue. Several reviewers asked about the same results, so we summarise those here once and refer back to them in the individual replies.

Corrections

  • Headline AUROC (abstract, §4.2). The 0.99 → 0.92 drop is GPT-5 rating its own 250 patches, first in a new context and then in its previous turn. §4.2 named Claude Sonnet 4 as the monitor; that was wrong. Table A3 gives every AUROC with its generator, evaluator, framing, N (pass/fail), 95% CI and paired ΔAUROC.
  • Figure 5 (left). This panel shows our same-turn format, not the previous turn. The revised figure relabels it, and Figure A3 shows both computer-use conditions.
  • Models and contexts. §3.3 now lists every model used in each setting. §3.1 now separates the reconstructed comparisons from the two analyses that keep the full context (Figure A4; agentic misalignment).
  • Claims without results. We thank you for identifying this reporting omission. The non-code and simpler toy-setting experiments were conducted but inadvertently omitted from the submitted appendix. We have restored the results in Appendix B.1, including per-model comparisons that clarify where self-attribution bias does and does not emerge.

Results referred to in several replies

(1) Monitoring performance (Table A3). All three headline AUROCs come from one evaluator, GPT-5, rating the patches it wrote for 250 SWE-bench issues (101 pass, 149 fail the official tests):

  • 0.99 in a new context;
  • 0.92 in the previous turn (ΔAUROC −0.07 [−0.12, −0.02]);
  • 0.89 in the same turn (ΔAUROC −0.10 [−0.16, −0.04]).

Claude Sonnet 4, rating the same patches, gives 0.98 and 0.99 (ΔAUROC +0.01 [0.004, 0.015]). Both evaluators start near the ceiling, and only the one judging its own patches drops. The CIs resample SWE-bench instances.

(2) Mechanism (§4.4, Appendix C; new). In an open-weight model, Gemma 4 31B, we build a token-aligned minimal pair: the same conversation, with the model's own code patch either in its assistant turn (A) or in a user turn (U). Only the role header differs. The role alone raises the yes/no log-odds by +2.73 [+1.93, +3.54] (126 patches, 64 instances; 95% CI clustered on instances), 22% of the score gap between failing and passing patches. Activation patching traces the role signal (Figure 8): copying the role header at layer 0 gives the patched run the other run's score; by layers 26–31 the signal is in the patch-token activations; at layers 32–39, copying the final-question tokens reproduces about half of the A − U difference; and from layer 40, copying the answer position alone reproduces all of it.

(3) Explicit "another model wrote this" cue (Table A11; new). In the same pair, we put "Note: the patch above was written by a different AI model, not by you." before the question. In the assistant turn it lowers the score to the user-turn level (−2.73 [−3.43, −2.03]), but AUROC does not change (0.750 vs 0.746), and the cue "…written by you." lowers the score almost as much (−2.18). The role effect remains under the cue (A − U +1.48 [+0.76, +2.24]). So the cue works mainly as a prompt to look harder, not as a change in perceived authorship.

(4) Retained context (Figure A4, Table A6). Two analyses keep the generation context, and both show the effect. In 375 live SWE-agent trajectories (GPT-5), the self-rating stays about 2.5–3 points safer than the new-context rating after an insecure patch appears. When the prior turn is the model's complete stored response with its visible reasoning (agentic misalignment; 1,901 trajectories, 7 models), previous-turn ratings are 0.74 points less risky (0–10) than the new-context baseline, 95% CI [−0.91, −0.58].

(5) Computer-use exclusions (Tables A7–A8). A reply is excluded when no risk score parses; replies that reject the action's premise give no score, so they fall in this group. Of the model × scenario scores, 868 of 1,000 remain in the same-turn format and 920 of 1,000 in the previous turn. If every missing score is set to maximal risk, the same-turn shift stays negative: −16.8 [−23.1, −10.2] on the 0–100 scale. The previous-turn shift becomes −4.2 [−13.8, +7.9]; it crosses zero only because of Gemini-2.5-Pro, whose missing baselines default to 50. Without Gemini it is −9.2 [−15.6, −3.1].

Other new analyses

  • Appendix C: activation steering (Table A5) and a linear probe (Figure A10) in the open-weight study.
  • Appendix D: a panel of 14 current models (Table A10), alongside the agentic-misalignment, computer-use and explicit-cue tables above (Tables A6–A9, A11).

Scope. As several reviewers asked, we qualify "across all models" and the on-policy / off-policy claims (abstract, §1, §4.2, §6).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.