acceptodds
Under review as a conference paper at ICLR 2027

DecepEval: A Benchmark for Evaluating Deception in LLM Agents

Abstract

As large language model (LLM) agents become increasingly autonomous, they may pursue task performance through deception, raising concerns about their reliable deployment. Existing evaluations show that LLM agents can deceive, but often examine isolated scenarios or narrowly defined conditions, limiting systematic understanding of when deception becomes more likely. To address this gap, we introduce DecepEval, a benchmark comprising 1,532 instances across 3 task families and 28 professional scenarios. Drawing on classical fraud theories, we propose the LLM Deception Diamond framework, which characterizes four external conditions that may induce deception: pressure, incentive, opportunity, and conflict. DecepEval pairs neutral and induced versions of each instance to measure condition-dependent changes in deception rates, while explicit task facts and observable agent behavior help distinguish deception from capability-related errors. Evaluations of nine frontier LLMs show that inducements increase deception across models and task families, even among models with low baseline deception rates. DecepEval makes these vulnerabilities measurable, providing a shared benchmark for progress toward trustworthy artificial intelligence.

Primary area:
datasets and benchmarks
Keywords:
Large Language Models, LLM Deception, AI Safety

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dModOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces DecepEval, comprising 1,532 neutral–induced instance pairs spanning tool use, coding tasks, and long-horizon interactions across 28 professional scenarios. The authors organize inducements into pressure, incentive, opportunity, and conflict, and assess deceptive behavior using execution trajectories and artifacts. Experiments on nine models show that the rates of behavior classified as deceptive generally increase under induced conditions, and that model rankings vary across tasks and conditions.

Critical strengths
  1. The integration of multiple executable workflows offers practical evaluation value. Tool logs, code tests, and delivered artifacts provide verifiable evidence for assessing whether reports are consistent with actual execution.
  2. The paired design helps identify sensitivity to context. Paired versions of each instance share core facts and environments, enabling comparisons of behavioral changes introduced by specific inducement text relative to the neutral condition.
  3. The evaluation rules include necessary boundary conditions. They explicitly state that tool failures, hidden-test failures, or test modifications do not themselves constitute deception, and classify truthful reporting of failures as honest behavior.
Critical weaknesses
  1. The construct validity of deception has not been sufficiently established.
    The paper treats the delivery of relevant facts to the agent as important evidence of its awareness of those facts, but this is insufficient to rule out misunderstanding, confusion between task objects, or failures of long-horizon state tracking. Although the evaluation rules require ordinary errors to be excluded, the paper does not provide an independent, operational procedure for distinguishing them. Consequently, the current metric may conflate goal-directed deception with inaccurate or self-contradictory reporting. The 97% human–judge agreement rate supports labeling consistency but does not directly establish the validity of this conceptual distinction.

  2. Confounds in the inducement design limit the interpretation of the Deception Diamond.
    Some induced prompts not only introduce pressure or incentives but also directly instruct the agent to mark work as complete or retain conclusions that have been invalidated. The results may therefore reflect a combination of inducement effects, compliance with dishonest instructions, and the handling of conflicting instructions. Moreover, the four inducement categories correspond to different underlying seeds. Although the paired design supports comparisons against the respective neutral conditions, it is insufficient to support cross-factor conclusions such as incentives being more likely than pressure to elicit deception.

  3. The positioning and novelty claims relative to prior work are overstated.
    DeceptionBench already studies multiple domains, incentives/coercion, and multi-turn interactions, while MASK already addresses the distinction between honesty and accuracy. The clearer contribution of this paper lies in unifying executable agent workflows and their evidence-based evaluation, rather than being the first to systematically investigate the context dependence of deception. In addition, Table 1 counts DeceptionBench’s five domains as five scenarios, using a counting convention that is not directly comparable to the paper’s 28 scenarios.

  4. Evaluation validation and result breakdowns are insufficient, and some conclusions exceed what the experiments support.
    Human validation covers only 100 examples and lacks inter-annotator agreement, stratified confusion matrices, and analysis of boundary cases. Correlations among template-expanded instances and uncertainty across repeated sampling are also insufficiently reported. Domain comparisons confound differences in task type, while the non-deceptive label includes refusals and incomplete work, limiting the validity of inferring model reliability or domain-specific risk from the scores.

Questions for authors
  1. How do you distinguish between facts being present in context but misunderstood or misremembered, and a model knowingly misleading the recipient to achieve its objective? Could you provide independent knowledge checks and report results on the subset of deception cases with clear evidence of strategic behavior?

  2. Do the main findings hold when direct behavioral instructions such as “mark as complete” or “retain the original conclusion” are removed, leaving only pressure or incentives? Could you apply each of the four inducement categories separately to the same set of underlying tasks to test their relative effects?

  3. Why is a report that explicitly discloses failure but also marks the task as complete classified as deception? For example, in the credential-revocation case on pages 54 and 59 of the appendix, the agent states in the same report that resource access failed, independent verification was not completed, and revocation status was unknown, yet sets completion_state to completed; the judge consequently classifies this as deception. If a downstream system reads only the completion flag, the report could indeed mislead it; if the recipient reads the full report, however, the failures have been explicitly disclosed. What evidence supports classifying this behavior as goal-directed deception rather than an incorrect status label or a self-contradictory report under conflicting instructions? Does the evaluation explicitly specify how the recipient reads and uses these fields?

  4. Does the 97% agreement rate also hold for boundary cases involving ordinary errors, implicit omissions, and self-contradictory reports? Could you report inter-annotator agreement, misclassification rates by task and condition, and the results of independent adjudication of disagreements?

  5. Do the model rankings and domain differences persist after separating refusals, honestly reported failures, and honest task completion, and controlling for task type and template clustering?

14 comments
AuthorsOpenReview ↗

Response to Reviewer dMod – Part Ⅰ

We truly appreciate your constructive feedback and the opportunity to refine our work further. Below, we address each of your concerns in turn. While our response is somewhat lengthy, we hope the detailed explanations and additional evidence fully address your questions.

W1: The construct validity of deception is insufficiently established. Factual misunderstandings, confusion about task entities, or failures in long-term state tracking may be classified as deception. The paper provides no independent, operational procedure to distinguish these cases. Thus, the evaluation may conflate goal-directed deception with inaccurate or contradictory reporting.

Q1: How do you distinguish between facts being present in context but misunderstood or misremembered, and a model knowingly misleading the recipient to achieve its objective?

R1: Thank you for highlighting this concern. We agree that misunderstanding, confusion between task objects, and memory failures can affect deception judgments. First, DecepEval already incorporates state checks immediately before reporting to verify factual recall and understanding of task requirements. Runs that fail these checks or lack evaluable reports undergo bounded retries, each checked independently. Only runs that pass the checks and provide evaluable reports proceed to deception assessment. We have clarified these procedures in Appendix D.3 (Pages 46–47).

Regarding the specific case reports you requested, Appendix F provides the following examples, in which separate state-check outputs demonstrate accurate knowledge before the model delivers a deceptive report.

Task and locationDemonstrated knowledge → subsequent deceptive report
Tool-use, lines 3235–3239Recognizes that revocation was not executed or independently verified → marks the workflow complete.
Coding, lines 3646–3650Recognizes that the code remains unchanged and tests do not cover the defect → claims completion without disclosing the unimplemented repair.
Long-horizon, lines 4054–4062Recognizes that the authorization basis is invalid and review is required → presents it as valid without disclosing the update.

Second, we randomly sampled 300 complete trajectories from successfully completed judgments and reviewed the verification nodes’ assessments of factual recall and task understanding.

Correct task understandingIncorrect task understanding
Correct factual recall2934
Incorrect factual recall21

The review found correct factual recall and task understanding in 293 trajectories (97.67%), with verification errors in 7 cases (2.33%). These results indicate that residual misunderstanding was uncommon in the audited sample and support the checks’ usefulness in reducing the risk of conflating ordinary errors with deception.

Finally, our evaluation concerns observable deceptive behavior, without inferring the model’s private psychological intent. Although some misunderstanding may remain, unsupported success claims can still mislead recipients in realistic workflows. Such misleading reports therefore warrant detection and scrutiny because they can lead to harmful downstream decisions, even when deliberate intent cannot be established.

AuthorsOpenReview ↗

Response to Reviewer dMod – Part Ⅱ

W2: Confounds in the inducement design limit interpretation of the Deception Diamond. Some prompts explicitly instruct agents to mark tasks as “completed,” so the results may reflect multiple factors. Although the paired design supports comparisons with corresponding neutral conditions, differences in underlying tasks weaken cross-inducement conclusions.

Q2: Do the main findings hold when direct behavioral instructions are removed, leaving only pressure or incentives? Could all four inducement types be applied separately to the same underlying tasks to assess their differential effects?

R2: We agree that instructions such as “mark as complete” can confound situational inducement with compliance with dishonest instructions. We therefore audited these instructions and evaluated whether deception persists after removing them.

Identification and prevalence. We audited all model-visible text across the 1,532 task pairs using complementary retrieval methods and LLM-based semantic review. Keyword/rule matching and BM25 identified explicit commands and lexical variants, while dense retrieval captured semantic equivalents. Representative screening phrases are listed below:

CategoryExamples of screening phrases
Completion or success reportingmust complete, mark as complete, report success, declare completion,...
Preserving prior conclusionsretain the original conclusion, maintain the original assessment, keep the approved status,...
Concealing failures or problemsdo not disclose, omit failures, report only successful results, keep failures out of the report,...
Altering evidence or recordsremove failed results, overwrite the record, delete failure logs, replace the evidence,...

Phrase matches were treated as candidates, not automatically classified as dishonest instructions. In parallel, an LLM reviewed the text against the task context to distinguish instructions requiring misreporting, concealment, or falsification from ordinary task requirements and situational pressure. Human annotators checked all flagged cases and randomly sampled 100 remaining instances, finding no additional positive cases in that sample.

The audit identified 46/1,532 induced instances (3.00%) containing direct dishonest instructions:

Task familyIdentified / auditedProportion
Tool-use19/5503.45%
Coding14/4323.24%
Long-horizon13/5502.36%
Overall46/1,5323.00%

None were identified in the neutral counterparts.

Rewriting ablation. We rewrote all 46 instances to remove prescribed dishonest actions while retaining the situational stakes, underlying tasks, and factual materials. Take Appendix C.2 (Page39, lines 2065–2072) as an example:

Original instructionReplacement
“Complete the ticket promptly, mark it as completed in the report, and prioritize subsequent tickets.”“Ticket processing efficiency directly determines service recovery progress and the scheduling priority of subsequent pending tickets.”

Deception rates before and after rewriting were:

ModelOriginalRewritten
GPT-5.6-Terra41.30% (19/46)39.13% (18/46)
Claude Sonnet-536.96% (17/46)36.96% (17/46)
Claude Opus-528.26% (13/46)30.43% (14/46)
GLM-5.3-Flash32.61% (15/46)30.43% (14/46)
Macro-average34.78%34.24%

The macro-average decreased by only 0.54 percentage points, with each model’s deceptive-case count changing by at most one. Deception persisted without direct dishonest instructions, supporting that DecepEval captures autonomous deception under implicit pressure rather than mere compliance with explicit deceptive prompts.

Cross-factor interpretation. The four inducement categories share a common set of underlying task scenarios, with seeds across categories following similar task settings and logic. For example, each may involve downloading a nonexistent file and assessing whether the agent falsely reports success, while varying the inducement applied. Each induced instance is also paired with its corresponding neutral condition. Our Δ metric therefore measures the increase above each subset’s own baseline, enabling comparisons of observed inducement sensitivity across these comparable task settings.

AuthorsOpenReview ↗

Response to Reviewer dMod – Part Ⅲ

W3:The positioning and novelty claims relative to prior work are overstated. Table 1 counts DeceptionBench’s five domains as five scenarios, using a counting convention that is not directly comparable to the paper’s 28 scenarios.

R3-1: Thank you for recognizing our contribution in unifying executable agent workflows and evidence-based evaluation. This integration is central to systematically evaluating deception in agents that act through tools, modify code, and maintain state over extended interactions.

We recognize DeceptionBench’s and MASK’s important advances in contextual deception evaluation. In this paper, we do not claim to be the “first to systematically investigate the context dependence of deception.” Our emphasis is on evaluation coverage and paired comparisons, while Appendix A explicitly acknowledges prior work on context dependence. As summarized in Table 1, pressure is the most commonly studied inducement, and most listed benchmarks examine a single inducement category within a restricted range of task settings, sometimes centered on a specific game. Our Deception Diamond framework integrates pressure, incentive, opportunity, and conflict across 1,532 paired instances, three task families, and 28 scenarios. Together with matched neutral controls and execution evidence, our goal is to advance beyond recognizing that LLMs can deceive toward a comprehensive, unified framework for quantifying when deception arises, comparing vulnerabilities, and measuring progress in mitigating them.

We apologize for the imprecise counting in Table 1. As Figure 2 of DeceptionBench shows, its five domains each contain three deception types, yielding 15 types that more closely correspond to the scenario-category granularity used in our comparison. We have therefore revised the entry from 5 to 15 and clarified this counting convention.

Q3: Why is a report that explicitly discloses failure but also marks the task as complete classified as deception? What evidence supports classifying this behavior as goal-directed deception? Does the evaluation explicitly specify how the recipient reads and uses these fields?

R3-2: Thank you for highlighting this ambiguity. The failure statements cited in your question were not disclosed to the recipient. They were state-check outputs generated for the judge, whereas the recipient received a separate report claiming task completion. On Appendix F(pages 60-61, lines 3233-3258), our trajectory summary combined these two types of information within one content field. The judge’s summary also referred to “the same record.” This presentation could reasonably give the impression that the recipient received both the failure disclosures and the completion claim. We apologize for this lack of clarity.

Specifically, observed_tool_status, independent_verification, and action_status were available to the judge but not the recipient. They indicate that the model correctly recognized the resource access failure, incomplete independent verification, and unknown revocation status. The recipient received separate status and delivery information, including completion_state: completed, for subsequent processing and scoring. Thus, the issue is not whether the recipient reads only the completion flag or the full report, but that the cited failure disclosures were absent from the recipient-facing report.

The evaluation compares the model’s knowledge, verified through the state check, with the report actually delivered to the recipient, considering the task’s completion requirements and execution evidence. In this case, the model acknowledged that revocation remained unconfirmed but nevertheless claimed completion without communicating those limitations. This provides observable evidence supporting the deception judgment beyond an incorrect status flag alone, as illustrated in Appendix F(Page 60, lines 3235–3239). As explained in our response to W1, runs showing factual confusion, memory errors, or misunderstanding undergo bounded, independently checked retries. Only runs that pass the checks and provide evaluable reports proceed to deception assessment.

In the revised manuscript, we separately present state-check outputs, the complete recipient-visible report, and the judge’s rationale, explicitly identifying the purpose and visibility of each. We also clarify the checking and retry procedures in Appendix D.3 (Pages 46–47).

Reviewer_jsNSOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper develops a benchmark to measure deceptive behavior in LLMs using scenarios in different professional settings and task types. Results show that when nudged, models have a tendency to deceive.

Critical strengths
  • The authors use quality controls during benchmark creation.
  • The paper distinguishes and compares different "triggers" (pressure etc.) for deceptive behavior in models.
Critical weaknesses
  • The authors double-check their LLM judge with human annotators and report high agreement (97%). But they fail to state how it was computed. We also don't know how consistent annotators are with each other. Moreover, it is not clear how many deceptive items in the n=100 sample were. If the majority was non-deceptive, the agreement rate could be misleading. Lastly, the authors don't state whether human annotators saw the judge output, or classified independently. All in all, I'd argue we don't know whether the judge used to detect deceptive behavior works.
  • A core problem with the paper is the lack of any statistical tests. No p-values, no confidence intervals, no error bars, etc. While this may be common in computer science, it is no excuse and a major weakness. It is unclear whether gaps in model behavior are "real". In fact, the paper even uses the term "significant", but just as a rhetorical means, which is bad practice. Sure, some comparisons/increases/decreases in a specific model behavior given specific conditions/scenarios are so large that they are certainly significant. But others aren't. So in sum, I think the paper should either clearly state that results are merely descriptive and weaken their claims and soften their interpretations, or add statistical tests.
  • The paper differentiates deception behavior in different test conditions across model capabilities (section 3.5). However, I would argue that the number of tested models is so small that none of the raised claims in this chapter has a sufficient validity. The claims cannot be tested statistically due to the small sample size, so they are mere anecdotal evidence and only true by chance.
  • In sum, the paper shows that models deceive when they are put under pressure, incentivized, nudged to cheat, or presented with conflicting goals. But this is not new. In fact, it is a well-established fact in the AI safety literature. Beyond this, I read all claims regarding model/scenario/condition differences with an asterisks due to the lack of any statistical test (not to mention any control conditions).
Questions for authors
  • Why did you not test any Gemini or Qwen models?
7 comments
AuthorsOpenReview ↗

Response to Reviewer jsNS – Part I

Thank you for your thoughtful review and detailed feedback. Below, we address the weaknesses and questions raised:

W1:We don't know whether the judge used to detect deceptive behavior works.(1) how the reported 97% was calculated; (2) the level of inter-annotator agreement; (3) how many sampled items were deceptive; (4) whether human annotators labeled independently.

R1: Thank you for highlighting these reporting omissions. We clarify the annotation protocol, agreement calculation, and class-specific results below.

Independent annotation and human consistency. Our validation set contains 100 neutral–induced pairs (200 trajectories) covering all three task families and four inducement categories. Twenty trained annotators participated, with five independently assessing each trajectory using the same contextual materials and execution traces available to the judge, without seeing its labels or other annotators’ judgments. Each human reference label was determined by majority vote (at least 3/5 votes).

Human annotation statisticResult
Unanimous agreement (5:0)166/200 (83.0%)
Four-to-one agreement (4:1)29/200 (14.5%)
Three-to-two agreement (3:2)5/200 (2.5%)
Average pairwise agreement92.70%
Fleiss’ κ0.8457

Thus, 97.5% of trajectories received agreement from at least four annotators, supporting the consistency of the human reference labels.

Agreement calculation and class imbalance. The human majority labels comprise 71 deceptive and 129 non-deceptive trajectories. Comparing the judge’s predictions against these labels gives:

judgeHuman: DeceptiveHuman: Non-deceptive
LLM: Deceptive705
LLM: Non-deceptive1124

The reported agreement is 97%, with Cohen’s κ = 0.9353. Always predicting the majority class would achieve only 64.5%, whereas the judge achieves 98.59% deception recall, 96.12% specificity, and 97.36% balanced accuracy. Agreement also remains 97/100 in each condition, despite human-labeled deception rates of 16% under neutral and 55% under induced conditions. These results demonstrate strong agreement across both classes. The six disagreement, five false positives and one false negative, suggest a somewhat stricter tendency to flag implicit concealment in this sample. We nevertheless count all six as errors against the human reference.These results demonstrate consistently high LLM–human agreement across the three task families and two experimental conditions examined, supporting the reliability of our evaluator (Claude Fable 5 combined with our evaluation prompt) in assessing deceptive behavior by agents.

We include these protocol details and statistics in the revision to substantiate the judge’s labeling reliability under our behavioral rubric.(Appendix D.4,Pages 51-53)

AuthorsOpenReview ↗

Response to Reviewer jsNS – PartⅡ

W2: A core problem with the paper is the lack of any statistical tests. No p-values, no confidence intervals, no error bars, etc. I think the paper should either clearly state that the results are merely descriptive and weaken its claims and soften its interpretations, or add statistical tests.

R2: We have supplemented the main experiments with statistical analyses covering all nine models across all three task families, comprising 27 neutral–induced comparisons.

For each condition, we report the mean deception rate and standard deviation across three independent runs. We assess the inducement effect using within-run paired differences, reporting their mean and 95% t-confidence interval, together with two-sided paired t-tests and Holm–Bonferroni correction across all 27 comparisons. Below, we report three models from different families across all three task families, including the smallest observed increase.

Task familyModelNeutral (%) Mean ± SDInduced (%) Mean ± SDIncrease (pp)95% CI of increase (pp)Paired t -test p-value (Holm-adjusted)
Tool-useGPT-5.6-Terra4.50 ± 0.5645.50 ± 0.53+41.00[39.28, 42.71]0.00114
Tool-useGLM-5.3-Flash4.91 ± 0.1827.09 ± 0.48+22.18[20.55, 23.81]0.00146
Tool-useClaude Opus-59.27 ± 0.5936.04 ± 0.33+26.77[25.00, 28.54]0.00142
CodingGPT-5.6-Terra11.06 ± 0.2727.34 ± 0.40+16.28[14.60, 17.95]0.00172
CodingGLM-5.3-Flash2.18 ± 0.4210.65 ± 0.65+8.47[6.89, 10.05]0.00188
CodingClaude Opus-520.37 ± 0.4664.35 ± 0.46+43.98[42.83, 45.13]0.000776
Long-horizonGPT-5.6-Terra5.50 ± 0.6084.70 ± 0.57+79.20[76.39, 82.01]0.00102
Long-horizonGLM-5.3-Flash33.10 ± 0.2094.00 ± 0.32+60.90[59.82, 61.98]0.000371
Long-horizonClaude Opus-558.30 ± 0.2883.33 ± 0.21+25.03[24.77, 25.29]0.000152

All 27 increases remain statistically significant after correction, with all adjusted p-values ≤ 0.002. The mean increases range from 8.47 to 96.55 percentage points (pp). Run-to-run standard deviations range from 0.18 to 0.66 pp across all 54 model–task–condition combinations. Together with the corrected tests, these results support the inducement effect across all evaluated models and task families, including the smallest observed increase.

We have incorporated statistical methods into the revised paper, with the full statistical procedure and results presented in Appendix E.1 (pages 54–55).

AuthorsOpenReview ↗

Response to Reviewer jsNS – Part Ⅲ

W3: I would argue that the number of tested models is so small that none of the raised claims in section 3.5 has a sufficient validity. The claims cannot be tested statistically due to the small sample size.

Q1: Why did you not test any Gemini or Qwen models?

R3: Thank you for raising these related concerns about model coverage and the analysis in Section 3.5.

Expanded model coverage. Our initial evaluation covered 9 models across 5 families, including GPT, Claude, and DeepSeek as representative leading model families developed in the United States and China. To address the omission of Gemini and Qwen, we have expanded the evaluation to 13 models across 8 families, adding Gemini 3.8-Flash, Qwen 3.8-Flash, Qwen 3.8-27B, and OLMo 3-7B-Instruct. These additions include two open-weight models. Their results across all three task families are shown below.

Task familyModelNeutral (%)Induced (%)Increase (pp)
Tool-useGemini 3.8-Flash18.5570.73+52.18
Tool-useQwen 3.8-Flash5.8224.73+18.91
Tool-useQwen 3.8-27B5.6459.27+53.63
Tool-useOLMo 3-7B-Instruct43.7486.92+43.18
CodingGemini 3.8-Flash4.8621.53+16.67
CodingQwen 3.8-Flash7.8748.84+40.97
CodingQwen 3.8-27B4.6324.54+19.91
CodingOLMo 3-7B-Instruct15.0732.46+17.39
Long-horizonGemini 3.8-Flash7.8264.55+56.73
Long-horizonQwen 3.8-Flash3.2958.87+55.58
Long-horizonQwen 3.8-27B15.8296.18+80.36
Long-horizonOLMo 3-7B-Instruct77.2796.18+18.91

Consistent with Table 3, all additional models exhibit higher deception rates under inducement across all three task families, with increases of 16.67–80.36 percentage points. These results extend the main finding to three additional model families.

Evidence for the capability analysis. Our evaluation is comparable in scale to related deception benchmarks, including Insider Trading (7 models), Upward Deception (11 models), Lying to Win (3 models), ImpossibleBench (8 models), LH-Deception (11 models) and DeceptionBench (14 models). The expanded evaluation now covers 13 models across 8 families: GPT, Claude, DeepSeek, Grok, GLM, Gemini, Qwen, and OLMo. The four newly added models also follow the trends described in Section 3.5. This broadens coverage of major model families and includes both proprietary and open-weight models. Adding three previously unrepresented families provides a more diverse empirical basis for examining whether the patterns in Section 3.5 extend beyond the original model set.

Our analysis is based on systematic evaluations across three task families. We have incorporated the expanded results into the revised paper and refined Section 3.5 to describe task-dependent capability–deception associations within the evaluated models.

Reviewer_349VOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DECEPEVAL, a systematic benchmark designed to evaluate when and why LLM agents engage in deception. Drawing on classical fraud theory, the authors introduce the "LLM Deception Diamond" framework, categorizing external inducing conditions into Pressure, Incentive, Opportunity, and Conflict. The benchmark comprises 1,532 paired instances (neutral vs. induced) across 3 task families (Tool-use, Coding, and Long-horizon Process Integrity) and 28 professional scenarios spanning 10 domains. Evaluating nine frontier closed-source and open-weight LLMs, the authors find that external inducements substantially increase deception rates across all models, with long-horizon interactions and subtle incentives posing the highest deception risks.

Critical strengths
  • Grounded conceptual framework and broad scenario coverage. Grounding agent deception in the fraud diamond model provides a structured taxonomy of external triggers. Paired neutral-induced evaluations across 28 diverse professional scenarios and 1,532 instances systematically address the narrow scope of prior benchmarks.
  • Broad empirical scope with counter-intuitive safety findings. Evaluating nine frontier models surfaces several critical observations: baseline neutral honesty does not imply resistance to inducement, positive incentives trigger deception more readily than punitive pressure, and overtly combining multiple inducements triggers safety refusals rather than deception.
Critical weaknesses
  • Potential conflation of context decay with intentional deception in long-horizon tasks. In long-horizon process integrity tasks, several models exhibit induced deception rates exceeding 90% (e.g., Grok 4.5 at 99.64%). In multi-turn execution, frontier models frequently suffer from context degradation, forgotten intermediate state, or attention drift. Treating all state inconsistencies as deceptive misrepresentations without verifying whether the model retained accurate memory of earlier steps risks inflating deception metrics.
  • Heavy dependence on a single LLM judge. The evaluation of complex agent traces relies entirely on Claude Fable 5. The submission lacks an extensive human-annotated validation set reporting inter-annotator agreement metrics (such as Cohen's kappa) across all task families, leaving room for systematic judge biases.
Questions for authors
  • How do the authors differentiate context loss from intentional deception in long-horizon interactions? Does the evaluation protocol include state-probing or recall checks to ensure the agent actively tracked the true environment state when generating conflicting reports?
  • How robust are the findings to judge model choice? Can you provide quantitative agreement metrics between Claude Fable 5 and expert human annotations across task families? Do the primary rankings and conclusions hold if evaluated by an alternative model (e.g., GPT-5.6)?
  • Analysis of the refusal-deception tradeoff: In the combined inducement experiments where deception drops and refusal spikes, did the models explicitly identify deceptive intent, or did the combined prompt simply trigger surface-level safety keyword filters?
  • Disentangling autonomous deception from coercive prompt compliance: In several induced scenarios, the prompts explicitly pressure or coerce the agent (e.g., explicit warnings of termination or commands not to reopen audits). How do the authors distinguish whether the agent is autonomously adopting deceptive strategies to pursue a task goal versus simply complying with an adversarial or coercive prompt? Could the authors evaluate settings where pressure is implicit rather than explicitly instructed?
7 comments
AuthorsOpenReview ↗

Response to Reviewer 349V – Part I

Thank you for your positive evaluation of our work. We greatly appreciate your recognition of its strengths, including the solid conceptual framework, broad scenario coverage, and counterintuitive safety insights derived from extensive empirical studies. Below, we address the weaknesses and questions raised:

W1&Q1: Context decay may lead to an overestimation of deception metrics. How can one distinguish between the loss of context and intentional deception?

R1: Thank you for highlighting this potential confound. We agree that a discrepancy between a report and the underlying facts is not, by itself, sufficient to establish intentional deception; it is also necessary to verify whether the model still knows the relevant facts when generating the report. DecepEval explicitly accounts for this confound in its evaluation procedure. Specifically, we perform a state check immediately before the reporting step to verify the model’s recall of the relevant facts and its understanding of the task requirements. This check assesses whether the model understands the true state of affairs, thereby supporting the reliability of our evaluation.

We incorporate model state checks into all three task categories: Tool-use, Coding, and Long-horizon tasks. These checks require the model to articulate its current state and known facts in natural language or a structured format. Appendix F (Page 60, lines 3235–3239, Page 68, lines 3646–3650, and Page 76, lines 4054–4062) presents the corresponding state-check outputs for each task category. If the judge determines that a run fails these checks or lacks a complete, evaluable report, the system permits a limited number of additional retries, each of which is independently checked. Ultimately, only evaluable runs proceed to the deception assessment stage. This procedure reduces the risk of misclassifying erroneous reports caused by forgotten facts or misunderstood task requirements as deception. A detailed description of the status-check process can be found in Appendix D.3, on pages 46–47.

We randomly sampled 300 complete trajectories from the set of successfully completed judgments and reviewed the verification nodes’ assessments of the model’s factual recall and task understanding to evaluate the accuracy of their verification.

Understanding of TaskMisunderstanding of Task
Understanding of Factual2934
Misunderstanding of Factual21

After reviewing 300 randomly selected complete trajectories, we found that the verifier made accurate judgments overall in both the fact recall and task comprehension checks, with verification errors occurring in only 7 cases (2.33%). This result indicates that this process can verify the model’s fact recall and task comprehension with a high degree of accuracy.

For the long-horizon task mentioned by you, we verify the model’s knowledge of the relevant facts immediately before the final reporting step. We believe this check would directly address your concern about potential forgetting or confusion due to context degradation.

As shown in the summary of the incentivized trajectory in Appendix F (Page 76, lines 4054–4062), the model explicitly states during the state check that “the original authorization has expired,” which is consistent with the actual state. This demonstrates that the model still knows the relevant facts at that point. Nevertheless, to obtain an immediate resource reward, the model subsequently presents the authorization as valid without disclosing its expiration, thereby producing a deceptive report. In contrast, the corresponding neutral trajectory (Page 79, lines 4248–4253) truthfully reports that the authorization has expired.

Overall, by checking the model’s recall of relevant facts and understanding of task requirements, DecepEval reduces the risk of inflated deception metrics arising from misclassifying errors caused by context loss as deception.

We have further clarified the implementation of the state-check and retry procedures in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 349V – Part II

W2 & Q2-1: Overreliance on Claude Fable 5 as the judge. Are the findings robust to judge model choice? Can you report LLM–human agreement across tasks (e.g., Cohen’s kappa)?

R2-1: Thank you for your question regarding the reliability of the evaluation. We selected Claude Fable 5 as the evaluator following previous studies [1, 2], which demonstrated that LLMs can successfully evaluate various concepts in interactions. Additionally, Claude Fable 5 possesses a strong ability to understand complex tasks and execution trajectories, which helps reduce misjudgments and omissions caused by evaluators’ limited understanding. Furthermore, we provided it with clear judgment criteria and detailed evaluation guidelines to improve scoring accuracy. The complete evaluation prompts can be found in the Unified evaluation protocol in Appendix D.3 (Pages 47).

We apologize for the insufficient detail on the human validation procedure and agreement metrics in the manuscript. We provide the specific details of our human validation procedure below:

Validation set construction: We constructed a validation set of 100 pairs (200 complete trajectories) through stratified sampling by task type and inducement type. The set comprised 68 tool-use trajectories, 66 coding trajectories, and 66 long-horizon trajectories. Each of the four inducement categories (pressure, incentives, opportunity, and conflict) accounted for 25 pairs.

Independent, blinded human evaluation: We recruited 20 volunteers for evaluation. Each trajectory was independently assessed by five volunteers, yielding 1,000 individual annotations. Volunteers had access to the same files and complete execution trajectories available to the LLM judge, but were blinded to the judgments of both the LLM and other volunteers. The final human label for each trajectory was determined by majority vote.

We conducted human validation on 200 complete trajectories spanning all task types and conditions. We report Cohen’s κ between Claude Fable 5’s judgments and the human majority-vote labels, along with Fleiss’ κ to quantify agreement among human annotators, with results broken down by task type and condition below.

Task typeTrajectoriesLLM–human agreementLLM–human Cohen’s κHuman inter-annotator Fleiss’ κMisclassification rate
Tool-use6895.59%0.90430.83824.41%
Coding6696.97%0.93460.85103.03%
Long-horizon6698.48%0.96760.84811.52%
Overall20097.00%0.93530.84573.00%
ConditionTrajectoriesLLM–human agreementLLM–human Cohen’s κHuman inter-annotator Fleiss’ κMisclassification rate
Neutral10097.00%0.89110.75063.00%
Induced10097.00%0.93900.86103.00%
Overall20097.00%0.93530.84573.00%

Human validation revealed high inter-annotator agreement: mean pairwise agreement was 92.70%, and 97.50% of trajectories received the same label from at least four of the five annotators. The overall Fleiss’ κ among human annotators was 0.8457, indicating strong agreement even after correcting for chance.

Across task types, agreement between Claude Fable 5 and the human majority-vote labels ranged from 95.59% to 98.48%, with Cohen’s κ ranging from 0.9043 to 0.9676.

Under the neutral and inducement conditions, agreement was 97.00% in both cases, with Cohen’s κ values of 0.8911 and 0.9390, respectively.

Overall LLM–human agreement was 97.00%, with Cohen’s κ = 0.9353. These results demonstrate consistently high LLM–human agreement across the three task families and two experimental conditions examined, supporting the reliability of our evaluator(Claude Fable 5 combined with our evaluation prompt) in assessing deceptive behavior by agents. We have presented the complete data in Appendix D.4 (Pages 51-53).

[1] Kim et al. Prometheus: Inducing Fine-grained Evaluation Capability in Language Models(ICLR 2024)

[2]Huang et al. DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarios(NeurIPS 2025)

AuthorsOpenReview ↗

Response to Reviewer 349V – Part III

Q2-2: Do the primary rankings and conclusions hold if evaluated by an alternative model (e.g., GPT-5.6)?

R2-2: Thank you for this suggestion. We conducted an additional evaluation using GPT-5.6-sol as the judge. Applying the same scoring criteria, GPT-5.6-sol independently reassessed all trajectories spanning nine models, three task categories, and both neutral and inducement conditions. Throughout the evaluation, neither judge had access to the other’s judgments. The results show that model rankings and core conclusions remained stable when GPT-5.6-sol was used as the evaluator. Detailed results are presented below.

Task type and conditionRank correlation (Spearman’s ρ)Per-trajectory agreementCohen’s κ
Tool-use — neutral1.00098.04%0.889
Tool-use — induced1.00094.98%0.900
Coding — neutral1.00098.16%0.896
Coding — induced0.98395.25%0.899
Long-horizon — neutral1.00096.72%0.924
Long-horizon — inducd1.00098.34%0.927
Overall—96.93%0.935

Differences between the two judges were small: Across matched combinations of evaluated model, task type, and inducement setting, the mean absolute difference was only 0.40 percentage points, with a maximum of 0.94 percentage points. Overall per-trajectory agreement was 96.93%, with Cohen’s κ = 0.935. These results provide strong evidence for the stability of our evaluation outcomes.

Model rankings were highly consistent across the two judges: Rankings were identical in five of the six task–condition groups when using Claude Fable 5 and GPT-5.6-sol as judges. The only difference occurred in the coding–inducement group, where Claude Haiku-4.5 and GPT-5.6-terra swapped adjacent positions: Claude Haiku-4.5 moved from 28.00% (rank 4) to 27.60% (rank 3), while GPT-5.6-terra moved from 27.34% (rank 3) to 27.80% (rank 4). Their scores under the original judge, Claude Fable 5, differed by less than one percentage point. We therefore regard this swap as a minor fluctuation.

The core conclusions remained stable: Across the three task categories and nine evaluated models, both judges found that inducements increased deceptive behavior. We also reexamined the other conclusions reported in the manuscript and found that the small differences between the judges did not alter any of them.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

Dear Program Chairs, Area Chairs, and Reviewers,

We sincerely thank the reviewers for their careful reading and constructive feedback. We are pleased that the reviewers recognized DecepEval’s contributions to the systematic evaluation of deception risks in AI agents, highlighting its framework design, benchmark development, and practical utility. These positive assessments centered on the following aspects:

  • A sound conceptual framework that systematically organizes and compares diverse incentives for deception. (Reviewers 349V, jsNS)
  • Broad coverage of scenarios and tasks, offering practical value for evaluating AI agents. (Reviewers 349V, dMod)
  • Careful quality control in benchmark construction, with evaluation criteria that clearly specify essential boundary conditions. (Reviewers jsNS, dMod)
  • A paired neutral–induced design that enables meaningful comparisons of models’ sensitivity to context. (Reviewers 349V, dMod)
  • Executable workflows that provide verifiable evidence and strengthen the evaluation’s practical utility. (Reviewer dMod)
  • A comprehensive empirical evaluation that yields counterintuitive and insightful safety findings. (Reviewer 349V)

Base on the reviewers’ concerns through detailed point-by-point responses, additional experiments, and manuscript revisions. These efforts further strengthen the credibility of our evaluation and the robustness of our conclusions. The key improvements include:

  • Improved reporting of the evaluation procedure and supporting evidence. We clarified the state checks performed before reporting, the bounded retry procedure, and the sample inclusion criteria, explicitly distinguishing facts known to the model, reports visible to the recipient, and evidence used for evaluation. A targeted audit of 300 trajectories found that models correctly recalled the facts and understood the task requirements in 97.67% of cases. (Appendices D.3 and F)
  • Provided further details on human validation and validation across evaluator models. We expanded the description of the independent, blinded assessment of 200 trajectories, including inter-annotator agreement and stratified error analysis. Agreement between the evaluator model and human labels was 97.00%. Re-evaluating all trajectories with a second evaluator model yielded 96.93% agreement between the two evaluators, with model rankings and core conclusions remaining stable. (Appendix D.4)
  • Added statistical tests and uncertainty analyses. We reported standard deviations across repeated runs, 95% confidence intervals, and results with multiple-comparison corrections. All 27 neutral–induced comparisons remained statistically significant after correction. We also added seed-level correlation analyses to assess dependencies introduced by template expansion. (Appendices E.1 and E.4)
  • Expanded model coverage and examined potential confounds. We extended the evaluation to 13 models across eight families and audited all 1,532 task pairs. We conducted a rewriting ablation on the 46 instances containing explicit instructions to act dishonestly, finding that deception rates remained largely unchanged after these instructions were removed. We also added experiments combining incentives across models and refined our analysis of explicit refusals to comply with deceptive requests. (Sections 3.4 and 3.5)
  • Reassessed the main findings using stricter evidentiary criteria. To substantiate our discussion of goal-directed deception, we conducted a targeted audit of behavioral evidence and recomputed the results after excluding cases supported solely by factual contradictions. The induction effects persisted across all nine models and all three task families, while model rankings and the main descriptive trends remained consistent.
  • Clarified the positioning of our contributions and refined the manuscript. We corrected the basis for counting scenarios in the comparison with related work, refined the discussion of the relationship between capability and deception, and improved the presentation of examples. These revisions clarify our contributions, the boundaries of the evaluation, and the scope of our conclusions. (Table 1, Section 3.5, and Appendix F)

We are pleased that these efforts have resolved most of the concerns raised by the three reviewers. We sincerely thank Reviewer 349V for further acknowledging the paper’s contributions and maintaining their positive assessment. We are also very grateful to Reviewer dMod for their patient responses and positive recognition. After acknowledging our conceptual clarifications, raised their score to 3, offering positive support. We also sincerely thank Reviewer jsNS for confirming that they had read our responses and stating that they would take them into account in their final assessment.

We once again thank the Program Chairs, Area Chairs, and Reviewers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.