acceptodds
Under review as a conference paper at ICLR 2027

When Agents Should Not Remember: Habitual Execution for Retrieval-Efficient Long-Running LLM Agents

Abstract

Long-running LLM agents often retrieve long-term memory even on stable, repeated workflows, which adds retrieval cost, stale context, and operational complexity. Reducing retrieval is not simple caching: the agent must decide, before retrieval, which workflows are SOP-stable, and must interrupt an automatic routine when policy, tool, or user-goal signals show that the context has changed. We propose habitual execution, a complementary control path. A Habitual Memory Layer compiles verified successful trajectories into guarded routines that can bypass explicit retrieval when the context is stable, and return control to retrieval when the routine no longer applies. On tool-using airline and retail customer-service tasks, HM-Gated RAG matches always-retrieve RAG in mean success over four trials (airline 0.667 vs. 0.675; retail 0.497 vs. 0.506; 95% CIs on the paired gap are [-8.3,+6.7] and [-6.9,+5.0] percentage points, both crossing zero) while cutting retrieval calls by about 59% and 51% (95% CIs [40.7%,76.1%] and [40.0%, 61.9%]). The saving requires offline task-type tags; keyword and no-tag routers do not produce it. The method is not a general way to reduce local tokens or latency.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
LLM agents, agent memory, retrieval-efficient generation, habitual execution, tool-using workflows

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_5DgQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether recurring tool-use workflows can reduce policy retrieval while maintaining task success. Its Habitual Memory Layer uses reference-action routines and offline task-type tags, with fallback triggered by model declarations or tool-message keywords. Across four qwen-plus trials on airline and retail tasks, retrieval calls decrease by about 59% and 51%, respectively, but the reported success-rate intervals do not establish noninferiority and end-to-end cost changes differ by domain.

Critical strengths
  1. The paper addresses a practical efficiency question by making retrieval bypass and fallback explicit control decisions. Its implementation and stated reliance on observable task types make the intended operating conditions relatively clear.

  2. The reported bootstrap intervals support a substantial reduction in retrieval calls across both domains. The paper also discloses the mixed token/latency results and provides route-specific diagnostics and a selected-sample manual failure audit.

Critical weaknesses
  1. The additional contribution of routines and guards is not isolated from routing and avoided retrieval. Although Procedural Prompt and Habitual Memory controls are included, the evaluation lacks a per-episode retrieval cache and a same-router/no-routine control. The existing trajectory examples and failure audit are descriptive; they do not establish the causal benefit of fallback. Moreover, the wrong-activation metric conditions on episode success and excludes incorrect habitual actions in failed episodes.

  2. The success-rate precision is insufficient for a claim of preserved performance. No acceptable loss margin is specified, and the paired 95% intervals of [-8.3, +6.7] and [-6.9, +5.0] percentage points permit meaningful losses. This is separate from the supported retrieval savings, which also do not imply lower total cost: the disclosed airline token and latency changes are +16.2% and +12.4%. The practical benefit therefore depends on the workload and retrieval cost.

Questions for authors
  1. Additional validation: Under the same task-type inputs and reference demonstrations, how does the method compare with caching one retrieval per episode and with a same-router/no-routine control? A fallback on/off comparison under fixed initial routes and independent action-validity labels would help assess the guard mechanism.

  2. Clarification: What success-rate loss would be acceptable for the retrieval savings, and how do the intervals compare with that margin? Please also clarify which normally available deployment inputs provide the task-type tags and how formation trajectories are separated from evaluation tasks.

Reviewer_31LVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether a tool-using agent can treat memory access as a pre-retrieval control decision and thereby reduce explicit memory retrieval. Specifically, the authors propose a Habitual Memory Layer that organizes successful trajectories into routines with triggers, action steps, guards, confidence scores, and fallback mechanisms. Before execution, a router selects between habitual execution for stable workflows and retrieval-augmented generation (RAG) for tasks requiring policy information. When a disruption is detected during habitual execution, the agent falls back to RAG. The experiments include 30 airline episodes and 80 retail episodes, covering both routine and disrupted conditions. The main results use qwen-plus and show that retrieval calls can be reduced when task-type tags are available, although overall execution costs do not consistently decrease. The authors explicitly acknowledge this limitation.

Critical strengths
  • The paper addresses a concrete engineering problem. Reducing retrieval calls can be valuable when retrieval systems are remote, billed per call, or expensive to operate.

  • The paper is candid about its limitations. It explicitly discloses unimplemented components, performance degradation on a weaker model, and the lack of consistent improvements in token usage and latency.

Critical weaknesses
  • The study focuses on a narrow setting: customer-service agents. This substantially limits its demonstrated applicability and practical value. I also find the novelty of both the problem formulation and the proposed method limited. The paper does not sufficiently establish how its contribution differs from adaptive memory access in existing agent frameworks and harnesses, such as Claude Code and Codex.

  • The presentation quality needs substantial improvement, particularly in the figures. Figure 1 is blurry, making some labels difficult to read. Figure 2 lacks complete method labels and a legend that clearly identifies the bars. Although AI-assisted manuscript polishing is reasonable, the final figures still require careful checking and editing. These issues make the paper harder to follow and suggest insufficient attention to presentation quality.

  • The experimental evaluation remains limited. The main results across four trials rely on qwen-plus. Although the paper also reports results for qwen-turbo and qwen2.5-7b-instruct, these are limited to a single run on the airline domain. This provides limited evidence that the findings generalize across models.

Questions for authors
  • To what extent does the proposed method apply beyond customer-service agents? Could you provide evidence or further discussion of its applicability to other types of agent workflows?

  • What is the key technical distinction between the Habitual Memory Layer and existing mechanisms for adaptive memory access or selective retrieval in agent frameworks and harnesses, such as Claude Code and Codex?

  • Could you improve the resolution and readability of Figure 1 and add complete method labels or a clear legend to Figure 2?

  • Could you extend the evaluation beyond the main qwen-plus experiments, using repeated runs across both domains and models from additional families, to establish whether the retrieval savings and task-success results generalize?

Reviewer_mvPuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HM-Gated RAG, which skips retrieval on repetitive workflows using routines constructed from reference actions and restores RAG when disruption signals appear. On adapted τ²-bench airline and retail tasks, it reduces retrieval calls by approximately 59% and 51%, with slightly lower mean success rates. The main results depend on offline task-type tags.

Critical strengths

The paper addresses a useful operational question. Stable, repetitive workflows may not require repeated policy retrieval. The reported reductions of approximately 59% and 51% show that substantial retrieval savings are possible under the evaluated routing setup, which could matter when retrieval is remote or costly.

The reporting makes the limitations visible. The paper explicitly discloses its reliance on offline tags and reference actions, reports unsuccessful routing variants, and separates retrieval savings from end-to-end costs. This helps readers assess where the approach works and avoids presenting fewer retrieval calls as a general reduction in latency or token usage.

Critical weaknesses
  1. The retrieval savings depend on supplied routing information. The main system uses offline task-type tags to decide whether to retrieve. Without tags, retrieval reaches 102.0% of RAG; keyword routing reaches 100.9% (Table 3). The experiment therefore shows that retrieval can be reduced given a suitable task classification, but does not establish how the agent obtains that classification. Routines also use benchmark reference actions, leaving generalization to unseen tasks unclear.

  2. The added value and reliability of the full method remain unestablished. On retail, zero-retrieval Habitual Memory matches RAG’s mean success (0.506), while HM-Gated achieves 0.497 with 417 retrieval calls. No guard/fallback ablation establishes the benefit of these components. Evaluation covers only 15 airline and 40 retail task pairs, with repeated trials on one model and incomplete baseline coverage. The success-gap confidence intervals permit losses of 8.3 and 6.9 percentage points, which does not support the claim of preserved performance.

Questions for authors
  1. How are task-type tags obtained? Do retrieval savings hold when routing uses only deployment-available information?

  2. Are routines built from the test tasks’ own reference actions? How does the method perform on held-out tasks?

  3. Could you ablate guards and fallback separately to establish their contribution to success and disruption handling?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.