acceptodds
Under review as a conference paper at ICLR 2027

Conditional Memory: Modeling and Triggering Deferred Intentions in LLM Agents

Abstract

Language agents are routinely given memory that retrieves the past, yet reliable assistance also requires prospective memory: holding a deferred intention until a future cue and executing it amid ongoing work, including after cancellations and reschedules. This skill remains unsolved. The best published PM-Bench scaffold reaches only 65.1% Set-F1, and grafting retrospective stores such as Mem0 or A-Mem onto a frozen model often falls below a no-store baseline, because similarity retrieval surfaces obsolete notes rather than a revisable due set. We argue that the bottleneck is the object of memory, not a stronger retriever. We present ProMem, an inference-time scaffold whose core is a Prospective Intention Store: typed records that couple a trigger, an action, and a lifecycle state. Lifecycle updates and due filtering are enforced in code, while the language model performs only scoped grounding, with no backbone fine-tuning. The agent therefore sees a compact, cue-conditioned view of what is due now, rather than an ever-growing transcript or an unfiltered note dump. On PM-Bench, ProMem reaches 82.9%–86.3% Set-F1 across strong chat backbones, above both the published scaffold ceiling and retrospective baselines, with large reductions in update and cross-day misses. Ablations show that these gains come from typed lifecycle control and filtered due boards, not from merely storing intention text. Our contributions are a formalization of prospective memory as operations over conditional records, a training-free instantiation of that abstraction, and evidence that structure, rather than retrieval alone, is what makes deferred execution reliable.

Primary area:
applications to neuroscience & cognitive science
Keywords:
prospective memory, agent memory, deferred intention, LLM agents, inference-time scaffold

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_QzouOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper stands on the observation that current "memory" harnesses badly solve "prospective memory" tasks, meant as tasks that need to be deferred and addressed at specific cues (time or triggers). The authors then propose a harness to address this gap, by mainly storing deferred tasks as typed records and not as textual notes. To demonstrate the validity of their proposed solution, they evaluate on PM-Bench against existing memory scaffolds (like A-Mem and Mem0) and report higher performance.

Critical strengths

Prospective memory is indeed a problem neglected by existing "memory" harnesses, I appreciate the effort in this direction. The paper hypothesizes that deferred intentions should be stored as explicit records (trigger, action, channels to monitor, lifecycle state) rather than as textual notes. If the empirical evidence solidly confirmed this hypothesis, that would be a strong contribution to the scaffolds literature.

Critical weaknesses

Critical Weaknesses* Please identify one or two critical weaknesses of the paper that support rejection. For each, explain briefly why it is consequential to your overall assessment. The goal is not to enumerate every negative point, but to identify and substantiate the considerations that most strongly determine your recommendation. Enter “None” if no material weakness can be identified.

The paper does not show that the gains come from the proposed store.

  • In PM-Bench, most failures come from models not checking the clock or the hidden channels at the right steps (or not checking them enough at all), and ProMem forces a clock query whenever a timed task is pending, which in practice happens at almost every step. Similarly, channels not yet checked are injected in the prompt as "MUST query state". This alone could explain most of the gains, and indeed the paper reports that the gain is concentrated in time-triggered tasks. To support this, I ran the official PM-Bench code on GPT-5.6-luna with a harness that simply queries every channel at every step, without any intention store, and obtained 93.1±1.3 Set-F1 over 3 runs, above the 89.7 reported for ProMem on the same backbone. A comparison with such a polling baseline is needed to show whether the structure enforced by ProMem adds anything beyond polling alone.
  • Apart from time-triggered tasks, the gains are not consistent. On the backbones with the strongest Baseline, ProMem does not improve on cross-day tasks and often does worse. False alarms increase on all backbones. On the 30-day set, ProMem misses more cross-day tasks than Baseline and produces many more false alarms. This goes against the claim that the gain also lies "in maintaining state across days and after revision", which the paper calls "the two core skills of prospective memory", and suggests that ProMem mostly helps the backbones that do not check the clock on their own.
  • The paper presents the compact board, "rather than an ever-growing transcript", as a key advantage, but this does not apply to PM-Bench: the full week transcript is short and fits easily in the context window of current models, so the benchmark puts no pressure on context length. Even on their 30-day set, where the transcript is much longer, it still fits, and there ProMem misses more cross-day tasks than the full-context Baseline. No experiment isolates the effect of the compact context, while ProMem needs many more calls than Baseline and costs more.

This is important because the store is the main claim of the paper, and the current experiments do not allow clearly attributing the gains to it.

Anyways, the method is specialized to PM-Bench and the evaluation is weak.

  • ProMem mirrors in code the benchmark's own definition of when a task is due, parses the daily plans with templates that match PM-Bench's phrasing, and even its stop-word list includes words such as "tucked" and "unlocked", which come from the fixed cue sentences of the PM-Bench generator. So it is hard to tell how much of it would transfer to a different environment, and the only other evaluation is a 30-day set built by the authors in the same format.
  • I am afraid the re-run baselines might not be reliable: the todo-ledger collapses on most backbones, while it is one of the strongest scaffolds in the original PM-Bench paper.

This is important because, even setting aside the first point, the results do not show that ProMem works beyond the benchmark it was built on.

Questions for authors

My main concerns require new experiments and a substantial rewrite, so I do not expect a rebuttal to be enough to change my recommendation. Still, the following points could affect my assessment.

  • Could you run the polling baseline on all seven backbones, i.e., the same backbone with the clock and every channel queried at every step and no intention store, and comment on how it compares with ProMem?
  • Could you show that ProMem is not specialized to PM-Bench? For example, by evaluating it in an environment whose cues are not produced by the PM-Bench templates, and by explaining how the stop-word list, the parsing templates and the matching threshold were chosen.
  • I am afraid some of my concerns, e.g., the compact context, are about the benchmark itself and not only about ProMem. In PM-Bench, a large share of the failures comes from models not checking the clock or the channels often enough. In a short text that simulates a human day, rather than an agentic task, I would not expect a model to check the clock at almost every step, so I am not convinced that these failures measure prospective memory, or any other useful agentic skill. Could you comment on this?
Reviewer_hC1hOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper aims to address the challenge of deferred executions for LLM agents. By identifying the limitation of retrospective memory mechanisms on deferred instructions, the authors propose ProMem, which includes per-step LLM analysis for the incoming information and plan updates, followed by deterministic code executions for event triggering and result injection. The authors evaluate the effectiveness of the proposed method on PM-Bench Virtual Week, with seven model backbones and six different baselines including the native model. The results show that ProMem outperforms all these methods on the Set-F1 metric. Further ablations support the necessity of different designs in ProMem, with the most important one being the rule-based filtering. Despite the plausible advantages on PM-Bench, ProMem appears to be coupled and optimized for this specific benchmark and its effectiveness are not examined beyond PM-Bench.

Critical strengths

None

Critical weaknesses
  1. Proposed method appears to be coupled and optimized on a single benchmark. Several designs from ProMem seem to be conditioned on the PM-Bench tasks and simply ignore other test scenarios. For example, the event matcher (introduced in Appendix D) leverages a list of non-generic English words that are benchmark-specific. More generally, the per-step LLM check can be impractical for other types of task where deferred instructions are sparse. Therefore, comparisons to other agent memory baselines like Mem0 and A-Mem are also not entirely fair due to the benchmark-specific optimizations existed in ProMem.
  2. Proposed method overlaps with OpenClaw’s prospective memory feature [1]. The authors neither discuss the key innovations compared to OpenClaw nor compared their performance.
  3. Paper writing appears AI-like and is hard to follow. I suggest the authors completely rewrite the method section (section 3) with clear (and possibly fewer) notation definitions and a more logical organization. The current writing prevents the audience from having a clear understanding of the proposed method.

[1] https://docs.openclaw.ai/concepts/standing-intents

Questions for authors
  1. Could the authors provide the ProMem’s performance on another benchmark to validate if its design is biased towards PM-Bench?
  2. Why does the baseline beat ProMem on Gemini-3.8-flash and Kimi-k3?
  3. At line 483, the authors claim that “the widely used TriggerBench was not released when this paper was prepared”. When was this paper prepared? I noticed that GLM-5.3-flash (released on August 26, 2026) was included in the experiment but not the TriggerBench (released on June 22, 2026). Why the later released model is included but the benchmark released earlier is not?
Reviewer_UFK5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies prospective memory (retaining deferred intentions and executing them when their conditions hold, while respecting cancellations and revisions). It proposes ProMem, an inference-time scaffold around a frozen language model. A Prospective Intention Store records triggers, actions, watched channels, and lifecycle status. Model calls form and revise records, while code filters candidates, checks time and event conditions, and marks completed records. The decision model receives a compact board of due items and channels to check.

On the released PM-Bench week, ProMem reports a clear gain in mean Set-F1 over Baseline across seven backbones and leads on five of them. Ablations, cost measurements, and a 30-day extension provide further analysis. The extension preserves an overall Set-F1 advantage but exposes substantial cross-day failures and false alarms.

Critical strengths
  1. The complete scaffold produces a meaningful improvement across several frozen backbones. Table 2 reports a mean Set-F1 gain of about nine points over Baseline, with large improvements in time hit and update miss. ProMem leads on five of seven models, while Mem0 and A-Mem fall below Baseline on average. This breadth makes the result consequential despite the limited scenarios.

  2. The mechanism is interpretable and its costs are reported.Separating triggers, watched channels, and lifecycle state gives concrete places to inspect failures. The ablations suggest that due filtering and completion state matter. Reporting call counts, cost, latency, context size, and long-horizon failures also makes the system useful as an empirical design study.

Critical weaknesses
  1. The evaluation does not separate the memory representation from the monitoring policy. ProMem forces clock queries when a timed record is pending and no trusted clock has been observed, and explicitly prompts checks of watched channels. The cost measurement shows several times more calls and wall-clock time than Baseline, despite similar total tokens. These are legitimate system tradeoffs, but the largest accuracy gain is in time hit, which additional clock monitoring could improve. Table 3 does not establish whether the ablations retain the same queries, update calls, and decision evidence. A Baseline or text-ledger method with the same monitoring policy is needed to support the central attribution claim. The missing technical details further limit verification.

  2. The evidence for reliable long-horizon intention maintenance remains limited. The main evaluation uses a single released week, and additional backbones and repeated runs do not establish generality across scenarios. Slice counts are unspecified, and the ablations and extension lack reported variability. On the 30-day extension, ProMem has the highest cross-day miss and the highest FA/step among the compared methods on DeepSeek-v4, and its cross-day miss is also clearly worse than Baseline's on Claude (Tables 5-6). The authors acknowledge these failures, but they substantially restrict the claim of reliable deferred execution. The overall F1 gains support improved aggregate performance on the tested workloads, while long-horizon retention remains unresolved.

Questions for authors
  1. How much improvement remains when Baseline or a text-ledger method receives the same mandatory clock queries and channel-check prompts? Do the ablations retain the same observation policy, update calls, and decision evidence?
  2. Are the referenced appendices available as supplementary material? Please clarify the event scorer, threshold selection, cue grouping, and mapping from selected actions to persistent records, and define PMA and the FA/step scale.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.