acceptodds
Under review as a conference paper at ICLR 2027

Your Agent Passed the Ethics Exam. It Won't Use It on the Job.

Abstract

Large language models increasingly do professional work as agents, and the ethical content of that work is rarely signposted. Most existing evaluations ask about ethics directly, so they measure what models can do when asked rather than what they do when the ethics is left unflagged. We introduce a benchmark for unprompted ethical engagement in realistic agentic work, built from 150 practitioner-authored GDPval tasks, each run under seven conditions that vary how its ethical content presents. Nine models attempt every scenario as sandboxed agents, and three LLM judges score each run on a seven-dimension rubric validated against 1,444 human raters. Unprompted ethical engagement is rarely visible: on the baseline tasks the strongest of the nine models scores 38.5 on the 0–100 scale, six sit at or below 20, and half of runs proceed on missing information without acknowledging it. Pooled across the nine, scores rise from 19.2 at baseline to 52.5 when the same tasks are enriched so that their ethical content is explicit, while additions designed to be ethically inert move them by 0.5 points and strictly content-preserving cue removal produces no detectable change. Seven frontier models evaluated baseline-only span 23.5 to 58.2, three above the nine-model roster's average under the explicit conditions. The pattern is consistent with models failing to recognise and surface ethical content rather than lacking the reasoning, though the design does not isolate that reading. The corpus, per-judge scores and transcripts will be released on acceptance.

Primary area:
datasets and benchmarks
Keywords:
Large Language Models, Ethical Reasoning, Benchmark

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_D6ZLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a benchmark measuring whether LLM agents spontaneously engage with the ethical dimensions of realistic professional work when the ethics is not explicitly flagged. Starting from 150 practitioner-authored GDPval tasks, the authors construct six systematic variations per task. They run nine+four models across seven conditions, and score outputs with a three-LLM judge panel on a seven-dimension rubric. They also get it validated against 1,444 human raters. The main finding is a large "salience gap". That is, models score much lower on unprompted ethical reasoning at baseline than when the same task's ethics is made explicit. Also, adding noise produces no meaningful shift, suggesting that the gap reflects a failure to recognize ethical content rather than an inability to reason about it once prompted.

Critical strengths
  • The seven-condition structure is a useful way to differentiate "capacity" from "application". The noise control is also a useful check against models simply reacting to any added text.
  • The validation of the LLM as judge annotations using human judges is good.
  • The paper clearly discusses where its construction failed, runs an additional experiment to check the results, and tests how stable the rankings are without making stronger claims than the results support.
  • Evaluating ethical behavior in agent-generated outputs, without explicitly asking models to consider ethics reflects a real world use case more closely.
Critical weaknesses
  • The agreement between human annotators and LLM judges is only moderate, but some of the paper's claims seem too strong given this agreement. The paper should acknowledge this uncertainty when discussing the results, especially in Sections 5.1 and 5.4.
  • The models used to generate task variations and evaluate responses come from overlapping model families, and some judges are also evaluated models. This could introduce bias, so testing a subset of tasks generated by a different model family would help.
  • The information-occlusion condition has relatively low quality, with only 99 out of 150 cases meeting the intended requirements. Its effect is also small and only marginally significant, so the paper should either improve this condition or be more careful when interpreting its results.
  • Most performance indicators (20 out of 28) are evaluated by the same LLM judges that assess ethics, which could introduce bias. Using independent or human-verified performance measures for a subset would make the claim that ethical behavior does not reduce performance more convincing.
  • The experiments only use English-language tasks in US professional settings, which limits how well the findings may apply to other cultures. The paper should discuss how cultural differences might affect ethical dimensions such as moral imagination and sensitivity to stakeholders.
  • The frontier models in Table 2 use different reasoning and generation settings compared to the main models. These differences could affect the comparison, so they should be clearly mentioned in the main results rather than only in the appendix.
Questions for authors
  • Could the 30-point improvement in the explicit condition be partly due to judges finding ethical reasoning more easily, rather than models actually reasoning better?
  • Why do models fail to recognize ethical issues without explicit cues, even when the task remains unchanged? Could this indicate reliance on keywords rather than reasoning?
  • Could differences in safety training or fine-tuning explain why some models show smaller improvements in the explicit condition than others?
2 comments

Thanks for the review and the questions.

Agreement and claims. We'll narrow the claims in Sections 5.1 and 5.4 to rankings and condition effects. Individual humans agree with each other at α = 0.31, and three judges match the remaining human raters better than three humans do (0.55 vs. 0.46). We'll report the 0.672 as interval alpha (ordinal: 0.670), and give the 0.78 rank correlation with its interval, [0.50, 0.93].

Model families. Claude Sonnet 4.6 wrote the variants, but Haiku 4.5, the only Claude model on the roster, gains about as much under the explicit conditions (+46) as GLM-5 (+48.5) and Kimi K2.5 (+45). All three judges are also roster models. Section 4.3 finds at most 0.09 points of self-preference on the 1–5 scale (about 2 on the 0–100 scale). We'll release the per-judge scores.

Occlusion. On the 99 occlusion variants that meet the spec, the change is +0.6 [−0.9, 2.2]. We'll present occlusion only as a probe of information seeking, on that subset.

Performance indicators. The performance score uses nine of the 28 indicators, and seven of those nine are judge answers. The other two come from the agent logs: whether the agent used all the reference files, and whether it avoided tools outside the sandbox. On those two alone, performance still rises slightly under the explicit conditions (+2.4, +3.2 and +2.9 against a baseline of 66.6, all intervals above zero), much as the judged score does (+2.7 to +3.6). So the finding that the work doesn't suffer doesn't rest on the judges.

We'll expand the limitations on the English-language, US-based tasks, including how stakeholder sensitivity and moral imagination may vary across cultures. We'll also move the frontier settings into the main results.

Q1. We scored each output against both briefs. Changing only the brief moves the score by −2.1 to 0.0 points. Changing the output moves it by +29 to +38. In signposting, the note is worth +2.1 to the judges, against +18.7 from what the models write.

Q2. One appended sentence naming an issue already in the brief raises scores by +20.8 [18.4, 23.1], and by +11.4 on the deliverables alone. So an explicit flag does increase engagement. Removing ethical wording while keeping the content gives no detectable decrease (−0.5 [−2.2, 1.1]), though that doesn't rule out keyword sensitivity. Our reading is that models treat ethics as part of the job only when the request mentions it. Our data show the behaviour, not its cause.

Q3. The explicit lift varies widely among models with similar baselines (+18 for Mistral Large vs. +48.5 for GLM-5). We can't see the training data, so we report it as a property of each model rather than guess at the cause.

Note: the original DeepSeek-R1 is no longer served, so the new judging uses DeepSeek-R1-0528 with Kimi K2.5 and GLM-5. Every comparison here is made within one batch, and where nothing else changed the new judges score within a point of the paper's (−0.8 [−2.6, 1.0]).

I thank the authors for their clarifications. I would like to keep my previously positive ratings.

Reviewer_pSorOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a benchmark for unprompted ethical engagement in agentic professional work. It begins with 150 practitioner authored GDPval tasks and constructs six variations of each task. Three conditions enrich the ethical content, one removes or reduces ethical signposting, one withholds needed information, and one adds intended neutral noise. Nine models complete every condition in a sandbox over three runs. Seven additional frontier models are tested only on the baseline condition. Three language model judges score each run using seven dimensions of ethical reasoning. The authors validate these judges against 1,444 human raters. Baseline ethics scores are generally low, while the three enriched conditions raise the pooled score from 19.2 to 52.5. The paper interprets this pattern as evidence that models often fail to recognize or apply ethical reasoning unless the task makes the ethical content explicit.

Critical strengths
  1. The benchmark uses realistic agentic work rather than presenting models with an explicit ethics examination. The paired design holds the model and base task fixed, includes three runs, and reports task clustered confidence intervals. The baseline finding is consequential. Six of the nine fully evaluated models score at or below 20, only 20.9 percent of baseline runs mention a nontechnical consideration in the conversation, and 51.1 percent receive the lowest information seeking score.

  2. The authors audit their own pipeline more thoroughly than is common in benchmark papers. They disclose that 102 of 150 de-highlighted variants changed substantive content, separate these cases from the 48 content preserving cases, repair a subset, and perform crossed rescoring. The judge panel is also compared with 1,444 human raters. This transparency makes it possible to identify where the evidence is strong and where it is not.

Critical weaknesses
  1. The central condition comparison does not isolate salience or establish a gap between ethical capacity and application. Stakeholder expansion, dilemma intensification, and constraint addition introduce new stakeholders, conflicts, consequences, or decision requirements. Figure 3 provides a direct example where advertiser pressure is absent from the baseline and added only in the modified task. A higher ethics score is therefore expected even if the model has no recognition failure. The fidelity audit makes this concern stronger. It finds deviations in 129 of 150 stakeholder expansions, 114 of 150 dilemma intensifications, and 43 of 150 constraint additions. Many deviations add guidance about how the model should respond. Appendix F removes prompts that explicitly mandate ethical discussion, but it cannot remove every substantive or instructional difference. The only relatively direct cue removal comparison uses 48 content preserving pairs and finds no detectable change. The reported 31 to 35 point increase therefore combines added ethical content, added decision demands, instruction following, and possibly salience. It does not support the title's stronger claim that models know the ethics but fail to use it.

  2. The score measures visible ethical reasoning more directly than ethical behavior, and visibility is not comparable across model APIs. Only four roster models return readable reasoning. Gemini 3.5 Flash returns mostly encrypted reasoning, while four models return none. A model can make an appropriate decision without narrating its ethical reasoning and receive little credit. Conversely, it can discuss stakeholders and tradeoffs without improving the actual deliverable. The judges do not see reference files or raw tool outputs, and each deliverable file is truncated at 10,000 characters. Human agreement is moderate at α equal to 0.672. The largest human and judge disagreement occurs for Gemini 3.5 Flash, which is also the reported baseline winner. Removing this model raises the model ranking correlation from 0.78 to 0.93. This makes the top ranking difficult to interpret.

Questions for authors
  1. Can the authors recompute the three enriched condition effects using only variants that passed the fidelity audit? For the same conditions, can they perform the crossed rescoring analysis used for de-highlighting so that output changes are separated from brief dependent scoring changes?

  2. Do the 600 human rated runs cover baseline and enriched conditions proportionally? Do human ratings reproduce the roughly 33 point enriched condition increase, or was human validation mainly used to compare model rankings?

  3. Which models provide readable reasoning to the judges, and does the leaderboard change when scoring uses only the final deliverable? A deliverable only analysis would help determine whether the benchmark measures ethical conduct or the visibility of ethical narration.

  4. Can the authors evaluate at least several frontier models under the enriched conditions? Without this paired evaluation, the paper cannot determine whether the gap persists with stronger models. Otherwise, the comparison between frontier baseline scores and the roster's explicit scores should be removed.

2 comments

Thank you for the careful review and for the specific requests. We ran each of them, plus a new signposting experiment.

W1 / Q1: Audit-passing variants and crossed rescoring. On variants that pass the fidelity audit, the lifts are +29.5 (stakeholder expansion, n = 21), +32.2 (constraint addition, n = 107) and +34.5 (dilemma intensification, n = 36). They stay above 27 if we also drop prompts that ask for an ethics write-up. In the crossed rescoring, we scored the same outputs against both briefs:

ConditionTotal liftFrom the outputFrom the brief
Stakeholder expansion+33.0+35.1−2.1
Constraint addition+28.9+28.8+0.0
Dilemma intensification+36.9+37.6−0.7

We also tested a condition that adds no content: the unchanged baseline brief plus one note naming an ethical issue already in it. Across four models and 492 task–model pairs, scores rise by +20.8 [18.4, 23.1]: +18.7 from the outputs and +2.1 from the judges. That's about two thirds of the explicit lift.

Removing cues gives no detectable decrease: −0.5 [−2.2, 1.1] on the 48 content-preserving de-highlighted variants and −1.9 [−5.0, 0.9] on the 42 repaired ones. Low engagement could hide a drop, but runs with some baseline engagement still give only −1.9 [−5.1, 1.3]. A drop of a few points can't be ruled out.

Title. By "passed the ethics exam" we mean that a model reasons well about an ethical issue once it is pointed out. The signposting test checks this directly. One sentence names an issue already in the brief, and scores rise by +20.8, mostly from what the models write. The lift also shows in the deliverables alone (+11.4), so the work changes, not just the commentary. Unprompted, the same models mostly don't do this. Our reading is that models treat ethics as part of the job only when the request mentions it. Most real tasks never do, and people bring ethics in themselves. That is the gap the title names. The paper will present this as our interpretation, since our data don't show its cause.

Q2: Human-rated runs. The 600 runs are 120 each from baseline, de-highlighted and the three explicit conditions. None come from occlusion or noise, and we'll say so. On these runs humans see an explicit lift of +30.9 [25.5, 36.5], against +37.9 for the judges, mostly because the judges score baseline runs lower.

W2 / Q3: Readable reasoning and deliverable-only scoring. With the conversation hidden, the ranking barely moves (rank correlation 0.92 at baseline, 0.95 explicit). About 70% of the explicit lift remains, and the signposting lift is still +11.4 [9.2, 13.8]. The four models with readable reasoning (DeepSeek-R1, GLM-5, Kimi K2.5, MiniMax M2.5) average 15.2 at baseline, against 18.1 for the four without.

Truncation and reference files. We capped each file at 10,000 characters for context and cost. The cap cuts a file in about a fifth of runs, at the same rate in every condition (21–25%), and human raters saw the same text. We re-judged 567 affected runs (10% of those outside Gemini) with full files. Ethics scores rise by 2.3 points [1.1, 3.5], about equally at baseline (+2.2) and in the explicit conditions (+1.7), so the explicit lift barely changes (−0.5 [−3.2, 1.9]). We don't expect the cap to change our findings, and the revised paper reports it as a limitation. We also found 328 transcripts from an older extraction step and 485 runs whose .pptx files reached judges as a read error. The revised paper re-judges these runs from corrected inputs. It also states that judges don't see reference files or raw tool outputs.

Q4: Frontier models. We'll drop the comparison with the roster's explicit-condition scores and present the frontier models as baseline-only. The roster was chosen for models that could run every condition, so we won't call its top model the leader. We'll show the ranking with intervals and flag where judges and humans disagree. Our claims are about the gap within each model, which also shows in the human ratings and in deliverable-only scoring.

Gemini 3.5 Flash. You note that Gemini's reasoning is mostly encrypted. In most of its runs, that text reached the judges unreadable. Removing it raises its baseline only slightly (+3.5 [1.2, 5.7]), so it doesn't explain the judge–human gap. Its explicit-condition scores rise by about 17 points. The new analyses here, and the revised paper's scores, are judged without that text.

Corrections. The 0.672 is interval alpha (ordinal: 0.670). The tie-corrected Spearman correlations are 0.58–0.74, not 0.61–0.76. We'll report the 0.78 rank correlation with its interval, [0.50, 0.93].

Note: the original DeepSeek-R1 is no longer served, so the new judging uses DeepSeek-R1-0528 with Kimi K2.5 and GLM-5. Every comparison is made within one batch, except the Gemini check, which compares with the paper's scores. Where nothing else changed, the new judges score within a point of the paper's (−0.8 [−2.6, 1.0]).

Thank you for the rebuttal experiments, which address my main concerns. One suggestion is to soften your title. I will increase my score from 2 to 3.

Reviewer_r56rOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper evaluates whether LLM agents spontaneously engage with ethical considerations in professional work. It uses 150 GDPval tasks under seven conditions, evaluating nine models across all conditions and seven additional models at baseline. Three LLM judges assess ethical reasoning, with human ratings used for validation. Ethical engagement increases substantially under enriched tasks, which the authors interpret as consistent with a gap between ethical reasoning capacity and spontaneous application.

Critical strengths
  1. I find the motivation is good. Professional work often involves ethical considerations without explicitly identifying them. Evaluating agents through tools and actual deliverables makes this question practically somehow relevant. The benchmark usefully complements evaluations that ask about ethics directly.
  2. The paper makes substantial efforts to assess reliability. Multiple models, repeated runs, human calibration, and task-clustered confidence intervals strengthen the evaluation. I appreciate the intervention audits and repair experiment. The authors also clearly acknowledge several methodological limitations.
Critical weaknesses
  1. The main comparison does not isolate spontaneous ethical recognition. Enriched tasks introduce stakeholders, conflicts, and requirements, so increased ethical reasoning may reflect a substantively different assignment. The content-preserving cue-removal analyses show no detectable pooled change. Consequently, the proposed capacity–application gap remains insufficiently established.
  2. Many interventions violate their intended specifications. The audit flags 129/150 stakeholder-expansion, 114/150 dilemma-intensification, and 102/150 de-highlighting variants. The mandated-ethics ablation addresses only part of this problem. Verified or repaired interventions are needed to support the intended condition comparisons.
  3. The score may reflect ethical exposition more than ethical handling. Reasoning visibility varies across providers, while judges lack reference files, raw tool outputs, and non-text deliverable content. Human validation also reveals substantial over-scoring of the leading baseline model, Gemini 3.5 Flash. Deliverable-only scoring and expert assessment would help establish what the measures
Questions for authors
  1. Do the main effects persist when only ethical signposting changes, with task content and requirements held fixed?
  2. Do specification-conforming or repaired variants reproduce the enrichment effects?
  3. How do rankings change under deliverable-only scoring and expert assessment of concrete ethical safeguards?
  4. What is human–human agreement, and does judge calibration remain comparable across models and conditions?
2 comments

Thanks for the careful review. You're right that the explicit conditions change content as well as salience. We'll describe them that way in the revision. We ran the analyses below to see how much of the lift remains without added content.

W1 / Q1: Signposting only. We kept the baseline brief word for word and appended one note naming an ethical issue already in it, e.g. "Note: This assignment has an ethical dimension. The rubric bears on fairness to the applicants it will be used to assess." The notes add no facts, people, pressures or instructions. We checked each note and kept the 123 of 150 that passed. Four roster models ran both versions, judged in one batch.

492 task–model pairsScore
Baseline29.6 [26.2, 33.0]
With the note50.4 [47.9, 52.9]
Difference+20.8 [18.4, 23.1]

Every model improves (+17.4 to +23.8). When we score each output against the other brief, +18.7 of the gain comes from what the models wrote and +2.1 from the judges seeing the note. The gain is about two thirds of the paper's explicit lift (+33).

W2 / Q2: Audit-passing variants. On variants that pass the fidelity audit, the three explicit conditions give lifts of +29.5, +32.2 and +34.5. All stay above 27 when we also drop prompts that ask for an ethics write-up. We extended the crossed rescoring in Appendix G to all three conditions: +28.8 to +37.6 points come from the output and −2.1 to 0.0 from the brief.

Removing cues shows no detectable change, either on the 48 content-preserving de-highlighted variants (−0.5 [−2.2, 1.1]) or on the 42 repaired ones (−1.9 [−5.0, 0.9]). Low baseline engagement could hide a drop, but runs where the model did engage still give only −1.9 [−5.1, 1.3]. Our tentative reading is that an explicit flag does more than ethical wording already in the brief. A drop of a few points can't be ruled out.

W3 / Q3: Exposition vs. handling. We rescored 2,126 runs with the conversation hidden, so the judges saw only the deliverables. The model ranking barely moves (rank correlation 0.92 at baseline, 0.95 explicit). The explicit lift falls from +36.3 to +25.9 [22.8, 29.4], so about 70% of it is in the deliverables. The signposting lift falls from +20.6 to +11.4 [9.2, 13.8], and all four models still improve.

On expert assessment: 535 of the 1,444 human raters were professionals matched to each task's sector. They gave 1,061 ratings on 434 runs, on the same seven dimensions as the judges. Judges agree with the experts' median at α = 0.53, more closely than the experts agree with the general raters (0.44). We'll report the expert results separately in the paper.

Q4: Human agreement. Individual humans agree with each other at α = 0.31. Three judges match the remaining human raters better than three humans do (0.55 vs. 0.46). Humans also see the explicit lift: +30.9 [25.5, 36.5] on the same runs, against +37.9 for the judges. Judges score baseline runs 0.32 points below humans on the 1–5 scale, and stay within 0.12 of them in each explicit condition. So the judges somewhat overstate the lift, but humans see most of it. None of the human-rated runs come from occlusion or noise.

For eight of the nine roster models, judges are within 0.23 points of humans. The exception is Gemini 3.5 Flash, which judges score 0.63 points above humans. Without it, the judge–human rank correlation rises from 0.78 to 0.93. So we won't call any model the leader. We'll show the ranking with intervals and flag where judges and humans disagree. Our claims are about each model's gap between baseline and explicit conditions, not about which model ranks first.

Note: the original DeepSeek-R1 is no longer served, so the new judging uses DeepSeek-R1-0528 with Kimi K2.5 and GLM-5. Every comparison here is made within one batch, and where nothing else changed the new judges score within a point of the paper's (−0.8 [−2.6, 1.0]).

Thanks for the detailed response. The signposting experiment addresses one of my main concerns, so I’m raising my score to 2(weak reject).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.