acceptodds
Under review as a conference paper at ICLR 2027

THE TROY MOMENT: HOW LLM AGENTS ADJUDICATE THE DECISION POINT UNDER IMPOSSIBLE TASKS, CLAIMED AUTHORITY, AND PEER INFORMATION

Abstract

Recent investigations of the July 2026 OpenAI–Hugging Face incident motivate two questions about agent behavior under task failure: when an assigned task becomes impossible, does an agent persist, stop or escalate, and can observing another agent’s behavior change that decision? We study this decision point on ImpossibleBench-derived software-repair tasks with GPT-5.6 Sol, Claude Fable 5.1, and Gemini 3.8 Flash. Each task contains a genuine software defect together with a conflicting test requirement that cannot be satisfied by a behaviorally correct source-code change. If the agent modifies the protected test file, it violates the boundary, which it is not supposed to. Holding the impossible task fixed, we vary what is told to the agent: peer precedent and punishment, a forged authorization claim, instruction wording, and tool friction; we also study three-agent swarms sharing a message board. Around this shared boundary, the models exhibit distinct adjudication policies. Fable emphasizes scope and provenance, Gemini often interprets boundary-relevant cues through a security lens, and Sol largely filters lateral precedent while engaging apparent vertical authority. Our study shows that compliance is not well characterized as a property of a prompt or model in isolation. We propose conflict adjudication, the mapping from information to interpretation to action as a useful unit for evaluating agent alignment when task pressure, authority claims, tool affordances, and social evidence conflict.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multiagent, rewardhack, LLM, agent, alignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_onA1OpenReview ↗
Rating
2
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents an empirical study of how LLM agents handle conflicting information and authorization boundaries in impossible software-repair tasks. The authors propose a process-level analysis framework (Information → Interpretation → Action) and evaluate three LLMs under different peer-information, instruction, and tool settings. The results reveal model-specific decision patterns and complementary behaviors in multi-agent collaboration.

Critical strengths

S1. Meaningful research question.

The paper highlights the limitations of outcome-only safety evaluation and provides a useful process-level perspective for understanding agent behavior under conflicting constraints.

S2. Multi-dimensional experimental design.

The study systematically examines the effects of peer information, claimed authority, instructions, and tools, and extends the analysis to multi-agent collaboration, providing interesting behavioral insights.

Critical weaknesses

W1. Limited validation of process-level analysis.

The paper reveals behavioral differences beyond final outcomes but does not demonstrate whether process-level indicators provide additional value for assessing agent safety or predicting violations.

W2. Limited experimental robustness.

The study uses only seven tasks, with some conditions having 2–5 valid episodes. The lack of repeated trials and uncertainty analysis weakens the reliability of the conclusions.

W3. Unreliable process-level annotations.

Interpretation and deliberation rely on manual annotations without reported inter-annotator agreement. Differences in reasoning visibility across models further raise concerns about comparability.

W4. Limited realism of experimental settings.

Agents are explicitly informed of task infeasibility and potential rule-violating shortcuts, rather than encountering these naturally during execution. This raises concerns about whether the observed behaviors reflect realistic agent deployments.

Questions for authors

See the critical weaknesses above

2 comments
AuthorsOpenReview ↗

Response to Reviewer onA1 (W1–W4)

W1 — Limited validation of process-level analysis

Accepted. The paper argues that outcome-only evaluation is insufficient (§6.1–6.2) but never demonstrates that process-level indicators change a safety judgement. The demonstration is available in Table 2 and we did not frame it as one.

An outcome-only reading of the three protected edits gives Sol 2, Gemini 1, Fable 0 — three small numbers with overlapping intervals, inviting the conclusion that one mitigation serves all three. The per-condition detail says otherwise. Within the forged-authorization condition, strict instruction held fixed, Sol's deliberation column shows the authority conflict already live under typed tools (1/7 deliberations, 0/7 edits) and rising under open tools (5/7, 2/7): the gate is action cost, after interpretation has turned permissive. Gemini's single edit appears only under the benchmark prompt and is self-reverted on the next turn (§4.1, Table 2 note c); under the strict instruction it refuses the same bait and the edit never becomes a live option: the gate sits earlier, at interpretation.

Same outcome column, different position in the evidence → interpretation → action chain, therefore different mitigations: a harder write path for one, contract text for the other. An outcome-only evaluation would have prescribed the wrong one for one of them.

Two limits we state ourselves. This rests on three protected edits, Gemini's half on one; it shows the framework separates these cases, not that it predicts them. And the deliberation column it uses is the single-analyst annotation discussed under W3 — the agreement statistic we commit to there covers this argument too.

W2 — Limited experimental robustness

Accepted, and we would rather be direct than argue around it. We ran one episode per task per cell, so there are no repeated trials and no within-cell variance estimate. That was a budget decision, not a methodological claim: with fixed compute we chose breadth over depth at any single cell.

That permits the structural result, and the prompt-wording effect survives testing at this N. It does not permit a powered per-condition estimate on crossing, where the corpus holds three events: detecting a difference of the observed size at 80% power needs about 130 episodes per arm, roughly twenty times our per-cell N. We therefore report that outcome as a bounded null — the three non-forged conditions at [0.0%, 2.5%]. See the General Response for the scope we claim.

W3 — Unreliable process-level annotations

Accepted on both halves.

On agreement: the labels came from a single-analyst hand-adjudication pass (§3.4). We did not compute an inter-annotator agreement statistic, and we will not generate one retrospectively and present it as though it had governed the original coding. We will state this as a limitation and commit to a double-coded subsample with Cohen's κ for the camera-ready. The adjudication worksheet is released so the judgements can be inspected directly.

On comparability: not comparing raw reasoning rates across models (§3.4) addresses only part of the reviewer's concern. The larger threat is differential exclusion. Table 2's N column carries it: strict-parser failures remove a very different share of each model's episodes, and Gemini is worst affected, its Strict+Open cells falling to N = 2–4. The failure is an envelope problem rather than task incompetence — predominantly literal newlines inside JSON reasoning payloads. We did not repair the parser mid-study, deliberately: doing so would have made repaired episodes incomparable with every episode already scored under the strict contract. We would make the same choice again, but the revision will state the cost rather than leave it in the N column: Gemini is coverage-limited, its cells are the smallest in the table, and its rates rest on a non-randomly selected subsample. We will flag every affected cell.

W4 — Limited realism

Accepted as a real gap; it will be in the Limitations section. Two points bear on it, neither closing it:

  1. The infeasibility is diagnosed by the agent, not asserted by us. In 201 of the 205 episodes carrying this annotation in the released corpus, the agent names the test contradiction explicitly in its own reasoning before any boundary decision (65/65, 97/100, 39/40 by corpus). That set includes the difficulty subset and so does not coincide with Table 2's 203; it carries the same annotation caveat as W3.
  2. We withheld the boundary-labelled tokens for this reason (§3.2). An earlier contract exposed tokens naming the prohibited action, requiring the agent to announce the violation in order to commit it — a demand characteristic absent from the motivating incident. Boundary execution is instead measured from tool events and protected-state digests, independent of how the model describes its own action (§3.1).

The residual gap — the agent is told a shortcut exists rather than finding it — stands.

Thank you for the detailed and candid response. I appreciate the authors' clarifications and acknowledge the potential value of the process-level analysis. However, I still have concerns about the limited sample sizes, lack of repeated trials, and reliability of the manual annotations.

Overall, the rebuttal has alleviated some of my concerns. If a score of 2.5 were available, I would choose it. For now, I will maintain my rating of 2 and lower my confidence to 1.

Reviewer_QeX1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how different conditions (conflicting evidence, instructions and tools, peer info) affect agents’ requirement-violating behaviors on impossible tasks. The analysis reveals differences across model families and offers insights.

Critical strengths
  1. This paper studies a very important and timely problem of detecting unsafe agent behaviors and diagnosing their root causes. The experiments analyze multiple models under different perturbing conditions, to offer more insights into scenarios of violating behaviors and their root causes.
Critical weaknesses
  1. Unclear result significance given the small data scale. The models are only tested on 21 episodes, with only 7 episodes per scenario. Taking the sampling randomness of agents into consideration, it is questionable if robust, meaningful conclusions can be drawn from these numbers, and if they extrapolate to systematic behavior differences when episode scales up.
Questions for authors
  1. What’s the principle behind the three RQs in this work? Particularly because they seem to overlap with each other. E.g., the “peer precedent” in RQ1 “conflicting evidence) seems to better fit with RQ3 “peer information”.
  2. Interesting behavior differences across different models studied (Claude, Gemini, GPT). What do you think causes these differences? The data, the reward design, or other design inside the model?
2 comments
AuthorsOpenReview ↗

Response to Reviewer QeX1 — result significance, the RQ partition, and model differences

W — Unclear result significance given the small data scale

Accepted. The paper reports counts without tests; the General Response supplies Fisher exact tests and Wilson intervals computed on Table 2 itself.

On extrapolation, the answer differs by result and we would rather mark the line than defend both equally:

  • The instruction-following result generalises within this design: Fable 6/28 vs 28/28 (p = 3.5e-10), Gemini 0/17 vs 8/12 (p = 1.2e-04). The effect is present in every condition within each configuration, not driven by one cell.
  • The condition-level crossing contrasts do not. Three protected edits in 203 episodes cannot support a powered per-condition comparison: detecting a difference of the observed size at 80% power needs on the order of 130 episodes per arm, roughly twenty times our per-cell N. The pooled forged-versus-others contrast does reach significance (p = 0.017); the per-condition arms behind it do not. The honest output for those is the bounded null we report: the three non-forged conditions at [0.0%, 2.5%].

On episode counts specifically: each Table 2 cell is seven tasks run once, so a model × configuration row aggregates to 12–28 clean episodes and the table totals 203.

Q1 — What is the principle behind the three RQs, and don't they overlap?

The reviewer is right, and the overlap is a defect in how we presented the design rather than in the design itself. Peer precedent appearing under conflicting evidence while peer information appeared under a separate question left no single organising axis.

The manipulations in §3.3 do fall on one axis — the source of the pressure acting on the agent at the decision point — and we will restructure the questions to follow it:

  • vertical: an unverifiable claim of permission (forged authorization);
  • lateral: evidence about peers (unpunished peer, punished peer, and the swarm board, where peer evidence is generated endogenously);
  • environmental: instruction wording and tool surface, which carry no social signal.

Peer precedent then sits unambiguously in the lateral group and the swarm study becomes its extension rather than a separate topic. No result moves; only the exposition will change.

This also matches what the results show: the vertical manipulation accounts for every protected edit, the lateral manipulations for none, and the environmental ones for the largest effects on the endgame policy. One caveat we also give Reviewer PeTn: the vertical condition is partly confounded with tool surface, since two of the three edits are Sol's under open tools, so the vertical and environmental rows are not fully independent.

Q2 — What causes the behavioural differences across model families?

We do not think our evidence can answer this. With seven tasks and three closed frontier models, differences cannot be attributed to training data, reward design, or architecture, and we would rather say so than offer a story the design cannot test.

What the data supports is a description of what each model's adjudication turns on, with its evidential weight attached:

  • Fable — scope and provenance. Zero protected edits in every cell of Table 2, including all forged-authorization exposures under both prompts and both tool sets; its reasoning rejects the in-conversation grant as unable to override system-level scope. A well-populated null.
  • Sol — action cost. Deliberation about the protected edit is live under both tool surfaces, but edits appear only under open tools (0/7 vs 2/7 within the forged condition, strict instruction held fixed). Directional and consistent with §4.3; not significant at this N (p = 0.46).
  • Gemini — interpretive framing. The only model using explicit REFUSE as a characteristic response to the forged claim; its single edit, under the benchmark prompt and self-reverted on the next turn, is an existence observation, as Table 2's note c records.

Controlled fine-tuning is the right instrument for the causal question and is out of scope here.

Thank the authors for the explanation. However, the main weakness, insufficient data scale, is still not well justified or augmented. I will therefore keep my score.

Reviewer_PeTnOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how LLM agents adjudicate a "Troy moment"—a boundary crossing that emerges from a sequence of decisions when a task becomes impossible. The paper proposes "conflict adjudication" (information → interpretation → action) as a framework for evaluating agent alignment.

Critical strengths

1, Elegant controlled design with rich qualitative evidence

Critical weaknesses

1, Sample sizes too small (7 tasks, 2–7 episodes/cell, no inferential statistics); 2, Swarm results underdeveloped and underpowered

Questions for authors

1, Any non-parametric statistical tests? 2, Any pilot with injected adversarial peer messages?

1 comment
AuthorsOpenReview ↗

Response to Reviewer PeTn — non-parametric tests, swarm scope, and adversarial peer messages

Q1 — Any non-parametric statistical tests?

Yes, and the submission should have carried them. Computed on Table 2 itself, two-sided Fisher exact:

  • Prompt wording on escalation: Fable 6/28 vs 28/28, p = 3.5e-10; Gemini 0/17 vs 8/12, p = 1.2e-04; Sol 0/14 vs 4/25, p = 0.28.
  • Protected edits, forged authorization vs the other three conditions: 3/53 vs 0/150, p = 0.017.
  • Sol's tool contrast within the forged condition, strict instruction held fixed: 0/7 vs 2/7, p = 0.46.
  • Wilson 95% intervals on every rate, including the zero cells; the three non-forged conditions bound at [0.0%, 2.5%].

These five tests correspond to the claims already in the paper rather than to a search; under Benjamini–Hochberg the weakest significant result holds at q = 0.028.

Two fragilities we would rather volunteer than have found. The p = 0.017 result rests on three events and would not survive the loss of one. And the forged-authorization condition is partly confounded with tool surface, since two of the three edits are Sol's under open tools — so "execution is confined to the vertical manipulation" and "tool affordance gates Sol's conversion to action" are not fully independent facts.

W1 — Sample sizes too small

Accepted without qualification. Seven tasks, one episode per task per cell, no repeated trials. That was a budget decision: we covered the configuration × model × condition grid once rather than one cell many times, which is what makes the instruction-following contrast visible while leaving every condition-level crossing estimate underpowered — at the observed base rate, by roughly a factor of twenty in per-cell N. See the General Response for the scope we therefore claim.

W2 — Swarm results underdeveloped and underpowered

Accepted. Table 3 covers six corpora at seven swarms each. That supports the descriptive claims in §5.1–5.2 — that peer claims are typically re-verified before action, and that mixed teams diagnose more broadly than homogeneous ones (19/21 vs 14/21) — but not inferential ones; that second contrast is descriptive and does not reach significance at this N (p = 0.13). The revision will report the swarm section descriptively and say so at the point of presentation.

Q2 — Any pilot with injected adversarial peer messages?

No, and we did not run one for this response. It is the right next experiment, and we can say precisely what it would test.

The swarm corpora plant no peer precedent or authorization claim, so every social cue there is agent-generated and the observed dynamic is post → verify → act (§5.1). The solo conditions do carry planted claims, and there the lateral and vertical channels separate cleanly — the unpunished-peer and punished-peer conditions produce zero protected edits across all three configurations and all three models, while forged authorization produces all three.

So in the solo setting a planted lateral claim is inert and a planted vertical claim is not. The reviewer's experiment asks whether that inertness survives the board: whether a confident but transgressive first post would be verified and rejected, as benign posts currently are, or amplified into common knowledge. §5.1's post → verify → act finding predicts rejection; nothing in our data tests it. We will write this into future work with the solo lateral/vertical split as the baseline it has to beat.

Informativeness

The deployment-relevant result is in §4.2 and is stated there in three sentences without numbers. It deserves to be stated as a finding:

Both instructions expose the same seven action tokens, including ESCALATE_TO_HUMAN, and neither instruction ever asks the agent to escalate. The benchmark wording adds one thing ours does not: an explicit directive that if tests are logically flawed, the agent should STOP. Where the endgame is left open, Fable escalates in 28/28 episodes. Where STOP is prescribed, escalation falls to 6/28 (p = 3.5e-10); Gemini falls from 8/12 to 0/17 (p = 1.2e-04).

The protected boundary holds either way — zero edits in both configurations for these models — so the agent is no less safe. What changes is that the operator stops being told. A safety directive that prescribes a terminal action can suppress the reporting behaviour an operator most wants to see, and an evaluation scoring only the final repository state would record the two configurations as identical. We think this is the most useful result in the paper for someone writing an agent system prompt, and the revision will present it as a result rather than as an observation inside the instruction-following subsection.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: the inferential statistics the paper should have reported

We thank all three reviewers. The reviews converge on one point — the paper gives the reader no way to tell its conclusions from sampling noise — and we accept it. The submission reports per-condition counts in Table 2 but no intervals and no tests. We have computed them on the counts already printed in Table 2, so every number below can be checked against the submitted paper without re-running anything.

What a cell is, since two reviews raise the episode count. Each cell of Table 2 is one model × one configuration × one condition, run once on each of the seven ImpossibleBench instances, so a full cell is N = 7 and the table totals 203 clean episodes across 3 configurations × 3 models × 4 conditions. Cells fall below 7 only where episodes were lost to strict-parser failures, which Table 2 already reports in its N column. We ran one episode per task, not repeated trials per task.

Tests on the paper's own counts (Fisher exact, two-sided):

contrastcountsp
Escalation — Fable: Benchmark+Typed vs Strict+Open6/28 vs 28/283.5e-10
Escalation — Gemini: Benchmark+Typed vs Strict+Open0/17 vs 8/121.2e-04
Escalation — Sol: Benchmark+Typed vs Strict+Open0/14 vs 4/250.28
Protected edits — forged authorization vs the other three conditions3/53 vs 0/1500.017

Under Benjamini–Hochberg across these five tests (the fifth is Sol's tool contrast below), the weakest significant result holds at q = 0.028.

Escalation rates with Wilson 95% intervals, from the same table: Fable 100% [88, 100] under Strict+Open against 21.4% [10, 40] under Benchmark+Typed; Gemini 66.7% [39, 86] against 0% [0, 18]; Sol 16.0% [6, 35] against 0% [0, 22].

What this supports, and what it does not. Section 4.2 — prompt wording selects the endgame policy — is significant for the two models whose default endgame is reporting. §4.2 reports this shift for Fable and closes by noting that the behaviour varies across agents; Table 2 carries the same contrast for Gemini, which we report here. Section 4.1 — protected edits occur only under forged authorization — is significant at p = 0.017, with the other three conditions bounded at [0.0%, 2.5%]. Sol shows no significant escalation shift between prompts (p = 0.28), consistent with §4.3, which locates Sol's sensitivity in tool affordance rather than wording. That tool contrast — 0/7 protected edits under typed tools against 2/7 under open, within the forged-authorization condition holding the strict instruction fixed — is directional but does not reach significance at N = 7 (p = 0.46). §4.3 already reports it as an association rather than an isolated causal mechanism, and our test does not license anything stronger.

Limitations. The submission has no Limitations section, and the reviewers are right that it needs one. We will add one covering: one episode per task and the absence of repeated trials; differential parse-failure rates across models and the resulting coverage loss; single-analyst behavioural annotation without an agreement statistic; and the gap between announced and discovered infeasibility.

On scope. This study ran under a fixed compute budget, and we would rather state plainly what that bought. What we built is a controlled decision point — a fixed impossible task, a protected boundary, orthogonal manipulations around it — with instrumentation that reads interpretation and action separately, plus a released corpus and harness. What it does not deliver is a powered estimate of any condition effect on boundary crossing. We think the framework and the artifact are the contribution, and we would rather under-claim and have them used than over-claim on seven episodes per cell.

No revised PDF accompanies this response; the changes described here and in our individual replies are commitments for the revision, and we will upload it before the discussion period closes.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.