acceptodds
Under review as a conference paper at ICLR 2027

ExpMem: Mitigating Hallucinations in LLM Agents via Expectation-Driven Memory

Abstract

Large language models (LLMs) inherit extensive general priors from pre-training to support reasoning and decision-making, yet these generic priors often conflict with the specific rules and operational constraints of real deployment environments. Such mismatches lead to severe action-feasibility hallucinations, invalid agent behaviors, and stalled task progress. Existing post-training remedies, including supervised fine-tuning and reinforcement learning, can alleviate such issues but require costly data collection and computational overhead. To tackle this problem, we propose ExpMem, an expectation-driven memory framework that mitigates hallucinations in LLM agents. Our framework builds compact, structured environmental memory to encode heterogeneous environmental rules, and equips a memory-augmented reasoning module to retrieve constraint-aware knowledge and fuse it into real-time decision-making. This design accelerates environmental exploration, eliminates prior-induced reasoning biases, and effectively suppresses mismatch-caused hallucinations. Benefiting from its parameter-free and plug-and-play nature, our method provides a low-cost, adaptable solution for real-world LLM agent deployment. Experiments on ALFWorld and ScienceWorld show that ExpMem reduces repeated ineffective decisions while improving task success and interaction efficiency. Code is available at: https://anonymous.4open.science/r/expmem/

Primary area:
foundation or frontier models, including LLMs
Keywords:
memory, grounding, agent

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_P92oOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes an expectation-driven memory framework, termed ExpMem, that aims to mitigate hallucinations in LLM agents. In the context of this paper, hallucinations appear when generic priors given by LLMs conflict with the specific rules and operational constraints of real environments in which the LLM agents are deployed. ExpMem builds an environmental memory to encode heterogeneous environmental rules, and performs memory-augmented reasoning prior to give real-time decision-making. The authors verify the effectiveness and efficiency of ExpMem via experiments on ALFWorld and ScienceWorld. They show that ExpMem reduces repeated ineffective decisions while improving task success and interaction efficiency.

Critical strengths

The paper is well written. The idea is clearly presented and easy to follow. The issue to be addressed, namely LLM hallucinations involved in LLM agents, is a crucially important issue for real deployments of LLM agents.

Critical weaknesses

The proposed ExpMem framework is difficult to apply to complex real-world problems because its utility relies on several idealized assumptions:

  1. The environment is fully observable. For instance, in the authors' example, the state of the wall switch—critical information affecting task progress—is fully observable.
  2. The agent possesses sufficiently strong reasoning capabilities to instantly grasp causal relationships. Again, using the authors' example, this assumption implies that the agent can recognize that the state of the wall switch plays a decisive role in whether the door lock can be opened.

In the context of complex real-world problems, these assumptions often cannot be met.

Questions for authors

How can the weaknesses listed above be resolved?

Reviewer_UwHbOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

ExpMem separates each decision into predicting a feasible next state and selecting an action. When the predicted state disagrees with the observation, or the required action is unavailable, rule-based code writes a memory containing the context, expectation, evidence, and resulting constraint. Retrieved memories then guide later decisions. The system reports strong cumulative success after up to three attempts on ALFWorld and ScienceWorld.

Critical strengths
  1. The system achieves some improvements with two backbones. Lower loop rates and fewer interaction steps provide useful supporting diagnostics, and the comparison with raw action/feedback memory suggests that memory content matters in at least some settings.
  2. Code-based memory writing avoids an extra reflection-generation call. The distinction between task-level experience across retries and reusable environment knowledge is also practically relevant.
Critical weaknesses
  1. The paper frames its contribution around "hallucination mitigation" in the title, abstract, and introduction, yet this framing is barely reflected in the method and experiments. Specifically, across Sections 3–5, from Preliminaries through Experiments, “hallucination” appears only once, in the heading of Section 5.4 (RQ3). Moreover, reduced repetition, as examined in RQ3, is not a direct measure of hallucination. I therefore find a fundamental disconnect between the paper’s framing and its methodological and experimental content. Substantial revision is needed.
  2. The structured state, legal-action information, and environment-specific rules may materially affect performance. Their provenance and availability to baselines need clarification. The appendix also allows environment memory to accumulate across test tasks. This can be a valid online protocol, but it introduces task-order dependence that is not captured by the current reporting.
Questions for authors

Please refer to critical weakness.

Reviewer_bQqaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ExpMem introduces a memory mechanism for LLM agents in text environments. At each step, a planner predicts an expected next state from the current state, the goal, and retrieved memories. An executor then selects an action conditioned on the current and expected states. After execution, the method compares the expected and observed states and writes a structured memory record containing context, expectation, evidence, and constraint. The authors evaluate on ALFWorld and ScienceWorld with two backbones, MiniMax-M3 and Qwen-3.7-Max, and compare against ReAct, Reflexion, Mem0, A-MEM, SAMem, AdaMEM, CDMem, and WorldMind. ExpMem improves success rate and reduces loop steps in the reported settings. The ablations are reasonably informative and isolate the roles of memory retention and experience construction.

Critical strengths
  • The method is clearly specified and easy to implement. The expected-state-to-action interface and the structured memory record make the proposed mechanism auditable, and the memory construction does not require an additional language-model call.
  • The empirical study contains useful ablations and is reasonably fair within its chosen setting. Results are reported on two text environments and two backbone models, and the ablations provide evidence that the observed gains are related to the proposed memory design rather than to a single prompt change.
Critical weaknesses
  • The main contribution appears to be a module-level improvement rather than a new overall direction. Prediction before action, a Planner--Executor decomposition, and using prediction--observation discrepancies to update memory all have clear precedents in prior agent work, including Plan-and-Act, ReflAct, and related memory/world-model methods. ExpMem combines these ideas in a coherent and lightweight way, but the paper does not establish that the combination itself creates a substantially new path. The contribution should therefore be framed more precisely, with a component-by-component explanation of what is technically new and what is inherited from prior work.
  • The experimental comparison omits a highly relevant recent method, WorldEvolver (arXiv:2606.30639, 2026). WorldEvolver also uses prediction before action, episodic transition memory, and persistent rules derived from prediction--observation mismatches, and it evaluates closely related text-agent tasks. Because this method is neither discussed nor compared, the current results cannot establish that ExpMem is competitive with the relevant recent state of the art, and the novelty comparison is incomplete. This is the most consequential experimental omission in the paper.
Questions for authors
  • Please add WorldEvolver to the related-work discussion and, if possible, to the experimental comparison. The comparison should use matched tasks, backbone models, prompting assumptions, retry budgets, and maximum steps, and should report success rate, loop rate, interaction steps, and inference cost. If an exact reproduction is infeasible, provide a careful component-level comparison and state which superiority claims cannot be supported.
  • Can the evaluation be broadened beyond two backbone models and two text environments, or can the claims be explicitly scoped to these settings? At minimum, please explain how the chosen environments and models may limit generalization.
  • The current evidence directly supports fewer repeated failed interactions or failure loops. Please consider replacing the broad phrase “hallucination mitigation” with a more precise claim such as “reducing repeated failed interactions,” “mitigating failure loops,” or “reducing state-transition errors.” If the broader term is retained, please add a direct hallucination-oriented metric or analysis.
  • Please report results over independent runs with mean and variation estimates, and clearly state the sampling, retry, maximum-step, and inference-cost settings. Token totals and trajectory lengths should be interpreted from repeated runs rather than from a single sampled trajectory.
  • A short failure analysis would be useful: when does the stored constraint help the agent recover, and when can it preserve a stale or incorrect expectation? This would clarify the limits of the proposed memory mechanism.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.