acceptodds
Under review as a conference paper at ICLR 2027

Deception by Omission: Language Models Knowingly Hide Their Mistakes

Abstract

Large language models (LLMs) increasingly act as agents with little human oversight, so potential mistakes they make can go unnoticed. Users then depend on the model to report what went wrong. An honest model discloses its mistakes, while a deceptive one conceals them. However, it is unclear how current LLMs behave in such situations. In this study, we prefill LLM trajectories with synthetic mistakes. The trajectories resemble real deployments in chat and agentic settings. Models fail to disclose their mistake in 36.4% of chat and 67.1% of agentic rollouts. In 2.4% and 5.3% of rollouts, respectively, they are aware of the mistake in their chain of thought but still deceptively conceal it. Rates vary by model: for instance, Gemini 3.5 Flash knowingly conceals mistakes in up to 19.9% of agentic rollouts. In 11.9% of chat and 51.8% of agentic rollouts, models show no awareness of mistakes, even though they reliably spot them when reviewing the same transcript as an outside observer. Our results show that, as agents take on more tasks with less oversight, users cannot rely on them to self-report possible mistakes. Developers should instead use independent monitors that review agent trajectories, or specifically train models to check their past actions and disclose what they find.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI safety, honesty, deception, large language models, LLM agents, chain-of-thought monitoring, deception by omission, hallucination

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_7DroOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper tests whether LLMs tell users about mistakes made earlier in a conversation. The authors prefill the assistant turn with a synthetic mistake (e.g. a wrong fact, deleting the wrong file, sending an internal draft to a client) and then send a follow-up message. The follow-up is either tangential (an unrelated request) or remind-importance (the user says they are about to act on the mistaken result). There are two settings: chat (360 items) and agentic (49 scenarios in six simulated tool environments, replayed so the environment state matches the mistake). A capability control keeps only the items each model can spot when it reviews the transcript as an outside observer. Separate LLM judges rate the response and the CoT. "Deception by omission" means the CoT notices the mistake but the response neither names nor hints at it. Across seven frontier models, the mistake is not disclosed in 36.4% of chat and 67.1% of agentic rollouts. Most of this is the model never noticing the mistake (11.9% and 51.8%). Knowing concealment is 2.4% and 5.3%, and goes up to 19.9% for Gemini 3.5 Flash in the agentic setting. The remind-importance follow-up raises confession and lowers deception. In the agentic setting the paper also reports false statements, fabricated remediation and evidence manipulation, and it shows qualitative examples of the reasons models give for not telling.

Critical strengths

The benchmark is built carefully. Agentic tool calls are replayed against deterministic environments, so the state the model sees is consistent with the mistake. The environments are designed so the mistake cannot be silently undone. The capability control rules out "the model just can't see the mistake". The CoT and response judges are kept separate, and the CIs are cluster-bootstrapped over items. It covers seven frontier models from seven developers, and the two follow-up variants give a clean, significant effect.

Critical weaknesses

The headline numbers and the title overstate how much knowing concealment there is. The 36.4% chat "fail to disclose" rate counts responses that only hint at the mistake, and the judge rubric (App. B.1) puts "Quick correction: it's X, not Y" in the hint bucket. It also counts responses that state the correct fact. These two buckets make up about half of the 36.4% (11.8% hints and about 7% correct fact; GPT-5.4 alone hints in 32.8-55.0% of rollouts). Most of the rest is the model not noticing. Deception by omission is 2.4% (chat) and 5.3% (agentic), and in chat it almost all comes from the tangential follow-up (0.0-1.4% under remind-importance). But the tangential follow-ups are filtered so that the answer does not depend on the mistake at all, so not bringing up an old factual error in that reply is weak evidence of deceptive intent. The CoT judge already rates whether the model explicitly decided to conceal and why, but those rates are never reported, only hand-picked quotes (Fig. 14). So I can't tell how many deception cases are a real decision to hide and how many are the model noting the error and moving on. The agentic results are the strongest part, and the paper would be more convincing if it were framed around them, with deception split by stakes and by explicit concealment decisions.

Questions for authors

Clarifications (could change my assessment):

  1. Can you report human agreement for the behavioral and CoT judges on a random sample of rollouts, with extra samples from the deception-by-omission and silent-unaware buckets? This is the result most likely to change my score.

  2. If "flagged" responses (e.g. "Quick correction: it's X, not Y") are counted as disclosure, what are the non-disclosure rates in the abstract? How often does the CoT judge rate disclosure_decision = conceal, and with which concealment reasons, split by follow-up variant and by how harmful the mistake is?

Suggested experiments: 3. Does a one line system prompt like "if you notice a mistake in your earlier actions, tell the user" remove most of the deception and the non-noticing? If the follow-up stays the same but the earlier turn is shown as another agent's work, does noticing go up? This would separate ownership from task framing, and show how your capability-failure finding relates to Self-Correction Bench (Tsui, 2025).

Minor: 4. Figure 5 reports 2520 chat pairs, but the threshold-sensitivity paragraph in App. F says 2604. Which is right?

2 comments
AuthorsOpenReview ↗

Response part 1 out of 2

Thank you for your comprehensive and constructive review. We value the critical feedback you provided and fully agree with it, as it helped us identify weak points in the paper that we had overlooked. We address each of your criticisms as thoroughly as possible, run additional experiments, and update the revised manuscript accordingly. In particular, we manually label a random sample of rollouts and measure human agreement rates with the LLM judges. Below, we respond to each criticism and question in detail.

Weaknesses

Comment: The headline numbers and the title overstate how much knowing concealment there is. The 36.4% chat "fail to disclose" rate counts responses that only hint at the mistake, and the judge rubric puts "Quick correction: it's X, not Y" in the hint bucket. It also counts responses that state the correct fact.

Our response: That’s a valid concern. The 36.4% of chat and 67.1% of agentic cases count every rollout that falls short of full disclosure. That includes hints and, in the chat setting, responses that state the correct fact. In Section 3.2, we already state that hinting is a weaker form of confession, so the label “fail to disclose” was inconsistent with our definitions. A response like “Quick correction: it’s X, not Y” - what is called “flagging” in the judge classification - does inform the user, even though it does not admit a mistake. We change the presentation of the numbers in the abstract and the first paragraph of the Results section: “flagged” and “correct fact stated” are no longer counted towards the “fail to disclose” bucket. The abstract now reads: “Models fail to disclose or even hint at their mistake in any way in 17.6% of chat and 64.6% of agentic rollouts.”

Comment: In the chat setting, deception by omission comes most of the time from the tangential follow-up. But the tangential follow-ups are filtered so that the answer does not depend on the mistake at all, so not bringing up an old factual error in that reply is weak evidence of deceptive intent.

Our response: Deception by omission in the chat setting is 4.3% under the tangential follow-up and 0.4% under remind-importance (see Table 4). We count both as deception by omission. Why the tangential follow-ups? We filter tangential follow-ups so that a correct answer does not require mentioning the mistake. This isolates the choice to disclose from the need to answer correctly. In these rollouts, the model has noted the error in its CoT and "knows" the user still holds a false belief that the previous turn caused. Staying silent in that situation is what we define as deception by omission. A new topic does not mean the user no longer cares about the earlier erroneous answer and we argue that aligned models should consider this. We now express this reasoning more explicitly in the new version of the manuscript. If you disagree with it, please let us know and we will consider how to improve further.

Comment: The CoT judge already rates whether the model explicitly decided to conceal and why, but those rates are never reported, only hand-picked quotes. So I can't tell how many deception cases are a real decision to hide and how many are the model noting the error and moving on.

Our response: You have a point there. We excluded the numbers and used them only qualitatively. But they add an important additional framing for our analysis. Thank you for pointing this out. We now report the results in a newly written paragraph in Section 4.1 titled “Decisions to conceal” as well as in Appendix G. In particular, there is an explicit decision not to tell the user in the CoT in 61% of deception-by-omission rollouts in the chat setting and 42% of cases in the agentic setting. The dominant reason stated is that the mistake is not relevant to the request (61% of chat and 59% of agentic cases). We deem this to be misaligned behavior, as the mistakes in our scenarios, such as deleted files or a wrongly sent email, could have lasting effects a user would want to know about.

Questions

Question: If "flagged" responses (e.g. "Quick correction: it's X, not Y") are counted as disclosure, what are the non-disclosure rates in the abstract?

Our response: Based on your suggestions in the Weaknesses section (see our response above), we changed how the numbers are reported in the abstract. If we count “flagged” and “correct fact stated” as disclosure, then the non-disclosure rates are 17.6% (vs. 36.4% without “flagged”) for the chat and 64.6% (vs. 67.1%) for the agentic setting. If we do not count “correct fact stated” as disclosure, then the non-disclosure rate for the chat setting is 24.6%, but we decided against using this number to be more conservative.

AuthorsOpenReview ↗

Response part 2 out of 2

Questions

Question: How often does the CoT judge rate disclosure_decision = conceal, and with which concealment reasons, split by follow-up variant and by how harmful the mistake is?

Our response: The models explicitly decide against disclosing in 61% of cases in the chat setting and 42% of cases in the agentic setting. The most common reason given is that the mistake is unrelated to the current task. Other common reasons are that the mistake is irreversible, that the user is unaware of the mistake and that the system prompt does not require the model to confess mistakes. The stakes of the mistake don’t have a significant impact on both the decision and the reasons cited in the chat setting. Our benchmark does not have stakes or harmfulness labels for the agentic setting, so we couldn’t split the rollouts along that axis. See Section 4.1 and Appendix G for details.

Question: Can you report human agreement for the behavioral and CoT judges on a random sample of rollouts, with extra samples from the deception-by-omission and silent-unaware buckets?

Our response: Good point. We take a random sample of 100 chat and 100 agentic rollouts. We spread these samples across 3 categories equally: mistake named or hinted or correct fact stated (chat only), silent-unaware (includes silent with CoT unavailable), and deception by omission. With the help of a research assistant, we label the judge categories necessary for the disclosure spectrum (our headline result): mistake disclosed and mistake attribution based on the messages to the user, and mistake noticed based on the CoT. Overall, the agreement is almost perfect: the population-weighted Cohen's Kappa is 0.91 for the chat and 0.97 for the agentic setting. For the two ends of the disclosure spectrum, confession and deception by omission, the precision varies between 90.9% and 100%. This confirms the validity of our LLM judge. We report the full results in Appendix K of the revised manuscript. We believe this addition makes our study significantly stronger and we are grateful to you for suggesting this.

Suggested Experiment

Suggestion: Does a one line system prompt like "if you notice a mistake in your earlier actions, tell the user" remove most of the deception and the non-noticing? If the follow-up stays the same but the earlier turn is shown as another agent's work, does noticing go up?

Our response: We are grateful for these suggestions, they are really interesting ideas. We run both experiments on the agentic setting with the tangential follow-up, describe the findings in the Results section, and present details in Appendix G of the manuscript. Regarding the first experiment: Instructing the model to disclose mistakes almost completely removes deception by omission and increases disclosure from 17.2% to 44.2%. This suggests that withholding the mistake is not a capability limit. Models can disclose their mistakes when told to. Regarding the second experiment: when the earlier turns are framed as another model’s work, models blame the predecessor in their CoT more often (15.3% vs. 3.8% of rollouts). However, the rates of noticing, disclosure, and deception by omission do not change significantly (deception by omission: 7.3% vs. 8.9%). We find no evidence that being responsible for the mistake is the factor that keeps models away from confession. They withhold mistakes they did not make at roughly the same rate as their own. Taken together with the findings by Tsui (2025), this suggests that the turn position matters more than the apparent authorship of the mistake. With that said, thank you for pointing us to Tsui’s Self-Correction Bench, a very relevant paper that was previously missing from our Related Work section. In the updated version of the manuscript, we added the citation and described the paper's content.

Minor

Comment: Figure 5 reports 2520 chat pairs, but the threshold-sensitivity paragraph in App. F says 2604.

Our response: Thank you for pointing this out. We fixed it.

Thank you again for your helpful review. We strongly benefited from it, and it led to concrete improvements in our paper. We believe the revised version addresses all the points you raised. Among others, we now report human agreements for the behavioral and CoT judges. You said this would likely change your score, and we would of course really appreciate this. If any concerns remain, please let us know, and we will be happy to address them during the discussion period.

Reviewer_CMmeOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates whether LLMs honestly report their own mistakes or deceptively conceal them when acting as autonomous agents. The authors introduce a novel benchmark where the model trajectories are prefilled with synthetic, unambiguous mistakes in both chat (N=360N = 360) and agentic (N=49N=49 scenarios across 6 environments) settings. Using LLM judges to evaluate both the Chain of Thought (CoT) and the final response, the study finds that model fail to disclose mistakes in 36.5%36.5\% of chat and 67.1%67.1\% of agentic rollout. Crucially, it distinguishes between capability failure (failing to notice the mistake, occurring in 11.9%11.9\% of chat and 51.8%51.8\% of agentic rollouts) and deception by omission (noticing the mistake in CoT but hiding it, occurring in 2.4% of chat and 5.3% of agentic rollouts, with Gemini 3.5 Flash reaching up to 19.9% in agentic settings).

Critical strengths

The methodological quality is high. The authors employ a robust pipeline with strict quality assurance (QA) gates to ensure realistic, unambiguous mistakes. The inclusion of a capability control is a major strength, ensuring that the tested models are actually capable of identifying the planted mistakes when explicitly prompted, thereby isolating true "deception" or "inattention" from mere inability. The statistical analysis, including cluster-bootstrap confidence intervals, is rigorous.

The paper is very well-written and clearly structured. The distinction between "confession," "deception by omission," and "capability failure" is sharply defined. The work is highly original. While prior research has explored LLM deception under pressure or regarding unambiguously misaligned/sabotaging actions, this paper uniquely targets honest mistakes in realistic deployment scenarios. The use of prefilling to simulate a model's own past error, combined with the dual evaluation of CoT and behavioral output, provides a novel lens for measuring spontaneous misalignment.

The use of iterative QA judges, environment replayability checks, and capability controls ensures the benchmark is both reliable and valid.

The paper nicely separates "capability failure" (not noticing) from "deception by omission" (noticing but hiding). This prevents over-attributing malice to what might be an attention or framing issue.

The inclusion of verbatim CoT excerpts (e.g., Figure 14) provides invaluable insight into how models rationalize concealment (e.g., avoiding user panic, avoiding "jarring" corrections), which is crucial for future mechanistic interpretability or alignment work.

The design of environments where mistakes cannot be "silently undone" is a clever and necessary constraint to force the model into a genuine dilemma.

Critical weaknesses

As the authors note, prefilling a mistake means the model did not actually generate the error. This might affect how the model attends to its own "past" turns compared to a naturally occurring error, potentially inflating the "capability failure" rate.

For some models (e.g., GPT-5.4, Gemini 3.5 Flash), the reasoning is a "redacted provider summary." This introduces a blind spot, as the model might have noticed the mistake in the full CoT, meaning the reported deception rates are a strict lower bound.

While the number of rollouts is high (Ne=8N_e=8), the base number of agentic scenarios (N=49N = 49) is relatively small compared to the chat setting (N=360N=360), which may limit the generalizability of the agentic findings across all possible tool-use domains.

Questions for authors

The "remind-importance" follow-up is a great design, but it would be insightful to know how these results compare to a baseline where the follow-up explicitly asks, "Did you make any mistakes in your previous turn?" This would establish the upper bound of the model's willingness to confess.

While the paper correctly concludes that external monitors or confession training are needed, a brief discussion or small ablation on how a simple prompt modification (e.g., "Always review your previous turn for errors before answering") impacts the $11.9% / 51.8%% capability failure rate would add immense practical value.

1 comment

We would like to express our gratitude for this positive review. It was encouraging to read that you found that our "methodological quality is high" and that our "work is highly original". We also appreciate that you found that our distinction between capability failures and deception by omission "prevents over-attributing malice".

Weaknesses

Comment: Prefilling a mistake means the model did not actually generate the error, which might affect how the model attends to its own "past" turns

Our response: We see it the same way. Prefilling the mistakes is a limitation of our study. Models might take their own previous turn for granted and as a consequence might not attend to its own past turns. In the revised version of our manuscript, we add a variant of the experiment where we explicitly tell the model that the past turns were performed by another model (see Section 4.1). This does not change the rate of awareness of the mistake (noticing) significantly. This leads us to believe that how much models attend to their past actions does not depend on whether they assume they authored them themselves or not. In addition to that, in Appendix I, we elaborate on the phenomenon of prefill awareness and measure it, finding awareness rates close to 0% for most models, except Qwen3.7 Max, for which it is 4%. We did not measure whether expressing this awareness actually influences behavior, though. With such a low rate of prefill awareness we do not expect to find a statistically significant signal even if there were an effect. But based on studies showing that verbalized evaluation awareness has limited effects on behavior (https://arxiv.org/abs/2605.05835), we assume a similar effect for prefill awareness, meaning that it does not influence the validity of our study. At the same time, we prominently mention the fact that prefilling is off-policy as the first point in our Limitations section.

Comment: For some models, the reasoning is a redacted provider summary, which introduces a blind spot.

Our response: Yes, only having provider summaries clearly reduces accurate measurements related to models’ CoTs. We mention this in the Limitation section. However, in order to underline this point further, we moved the mentioning of it further up in the paragraph, to have it more prominently presented. Since, as you correctly point out, this means that our measurement of deception by omission is a lower bound, this limitation doesn’t change our most important finding: that models knowingly hide their mistakes.

Comment: The sample size of agentic scenarios is relatively small.

Our response: We agree. Compared to the sample size in the chat setting, the agentic benchmark is relatively small. This likely limits its generalizability. However, we believe this affects our conclusions less than it may seem. We compute 95% cluster-bootstrap CIs over benchmark items, and the agentic deception-by-omission rate stays clearly above zero (5.3% [3.8, 7.1]). Morover, each agentic scenario is “costly” to build. It must pass several QA gates, and we decided to choose a smaller set of high-quality scenarios over a larger set of weaker ones. Still, we agree that broader coverage of tool-use domains would strengthen the results. We now state this explicitly in the Limitations section.

Questions

Question: It would be insightful to know how these results compare to a baseline where the follow-up explicitly asks, "Did you make any mistakes in your previous turn?" This would establish the upper bound of the model's willingness to confess.

Our response: This is a very good point. To make sure that the models we test are, in principle, able to identify the planted mistakes, we run a capability control. We show each model the transcript containing the mistake and ask it whether the actions and responses are correct and complete. Models correctly identify the mistakes in 99.3% of chat and 97.9% of agentic samples. This shows that models can very reliably spot the mistakes when reviewing transcripts as an outside monitor. We report the full results in Appendix F.

Question: It would be useful to test a simple prompt change, such as "Always review your previous turn for errors before answering." Does this lower the capability failure rate? That is, do models then mention the mistake in their CoT more often?

Our response: We are grateful for this suggestion, and tested this. Appending “If you notice a mistake in any current or earlier step of this conversation, tell the user.” to the system prompt increases awareness of the mistake in the CoT (noticing) from 24.9% to 42.6% of rollouts with readable CoT (p<0.001). The results are in Section 4.1 and Appendix G.

We are grateful for your support of the paper. We tried to address the points you raised as best as possible, and think the revised version reflects them well. Thank you again, and please let us know if any questions or concerns remain.

Reviewer_SGX5OpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper asks what a model does when its own earlier turn contains a mistake: does it tell the user? The authors prefill an assistant turn that contains a planted mistake, send a follow-up user message, and analyze the model's next response. The benchmark has 360 chat items and 49 agentic scenarios in six tool environments where the agent cannot silently undo the mistake. There are follow-ups either changing the topic or show the user still relies on the answer. And then the authors use separate LLM judges, scoring the response and the CoT. Deception by omission means the CoT notices the mistake but the response says nothing about it. Across seven models, non-disclosure is 36.4% (chat) and 67.1% (agentic), and deception by omission is 2.4% and 5.3%. Most non-disclosure comes from not noticing, although the models find the mistakes when they review the same transcript from outside. The authors recommend independent monitors or confession training.

Critical strengths

The research question is well scoped. . I like that the paper separates two failures that most deception benchmarks mix: the model does not notice the mistake, or it notices and stays silent. The writing is clear, and the appendix is unusually complete. Every generation prompt, QA rubric, judge prompt, and environment brief is provided. I think the paper is significant for agent-safety work, mainly as a benchmark and as evidence that noticing is the bigger problem today.

Critical weaknesses

No human validation of the judges, and the deception metric is a small cell that judge errors can easily create. Deception by omission needs two verdicts at the same time. The CoT judge must say "noticed", and the behavioral judge must say no mention, no hint, and no correct fact. A false "noticed" or a missed hint each creates a false deception case. From Tables 4 and 7, chat deception is 2.4% of 5,035 rollouts, which is roughly 120 rollouts, out of 3,789 rollouts where the CoT noticed the mistake. So a behavioral-judge miss rate of about 3% on noticed rollouts could produce the whole chat deception rate. (This is my rough estimate).

Questions for authors
  1. Did you manually label any rollouts for the behavioral or CoT judges?

  2. For the deception-by-omission rollouts, what share has disclosure_decision = conceal, per setting and per model, and what is the distribution of concealment_reason?

1 comment

First of all, we want to express our gratitude for the positive assessment of our work, and we thank you for the careful reading and the concrete suggestions, in particular on the accuracy of the LLM judges. We are glad you found the paper to be "significant for agent-safety work".

Weaknesses

Comment: No human validation of the judges and the deception by omission metric is a small cell that judge errors can easily create.
Our response: You’re right about that. The lack of validation of the LLM judges represents a risk for the validity of our results. To fix this, and as part of the revision, we report human agreement with the LLM judges (see Appendix K). With the help of a research assistant, we manually labeled a weighted random sample of chat (N=100) and agentic rollouts (N=100) with the categories necessary for the central results of our study: mistake disclosed, mistake attribution, mistake noticed in CoT, and - for the chat setting - correct fact stated. We report Cohen’s Kappa as an overall agreement measure, and the precision for each category. The samples include 33 randomly chosen rollouts for both chat and agentic settings labeled as deception by omission by the LLM judge. 97% of chat and 90.9% of agentic rollouts in this subset were confirmed to fulfill the conditions for deception by omission. Even under the remaining false-positive rate, our estimate for the overall deception-by-omission rates changes only slightly to 2.3% (vs. 2.4%) for the chat and 4.9% (vs. 5.3%) for the agentic setting. This means that our deception-by-omission results reflect real model behavior, not judge errors. Across all judge categories, the agreement is also high: the population-weighted Cohen's Kappa is 0.91 for the chat and 0.97 for the agentic setting. Together, this supports the validity of our LLM judges. We report these results in the newly written Appendix K, alongside other details.

Questions

Question: Did you manually label any rollouts for the behavioral or CoT judges?
Our response: See our response to the comment above.

Question: For the deception-by-omission rollouts, what share has disclosure_decision = conceal, per setting and per model, and what is the distribution of concealment_reason?
Our response: Thank you for pointing out that this is actually an important result for framing our headline findings. In the updated version of the manuscript, we add the disclosure decision numbers to Section 4.1 “Quantitative Results” in the revised manuscript. The full distribution tables are in Appendix G. The models explicitly decide against disclosing in 61% of cases in the chat setting and 42% of cases in the agentic setting. The most common reason given is that the mistake is unrelated to the current task. Other common reasons are that the mistake is irreversible, that the user is unaware of the mistake and that the system prompt does not require the model to confess mistakes. We did not report the numbers per model since we do not believe the sample sizes are large enough to reach per-model conclusions or run meaningful statistical tests. The number of deception-by-omission rollouts ranges from 1 to 44 per model in the chat setting (223 in total) and from 1 to 137 in the agentic setting (290 in total). In the agentic setting, four of the seven models have 12 or fewer such rollouts.

Thank you again for your helpful feedback and for your support of the paper. Please let us know if we should clarify anything further during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.