acceptodds
Under review as a conference paper at ICLR 2027

When Should Agents Check External State? Budgeting Observations for Stored Intentions

Abstract

Prospective memory allows an agent to retain an intention tied to a future condition, but the stored intention does not reveal whether that condition currently holds. Checking it may require web access, multi-step tool use, and paid calls. Existing systems decide when intentions require attention, but do not allocate the resulting observations under a shared budget. We introduce the first resource-allocation for- mulation for the external observations required by stored intentions under a shared episode budget. BudgetPM offers two policy variants that share a hard-budget executor. BudgetPM-Static uses a lightweight Logistic scorer to learn whether a check improves the current decision. BudgetPM-Sequential distills full-episode hindsight schedules into a lightweight policy that decides when to spend or reserve capacity using only pre-query information at deployment. We evaluate BudgetPM against two public memory-agent systems, five matched controls, and four hand-designed monitoring or budget-adaptation rules. Across two benchmarks and three backbones, BudgetPM-Static outperforms adapted Mem0 and PMA workflows. On PM-Bench, its Logistic scorer reaches competitive quality–cost operating points alongside higher-capacity scorers and retains 99.9–100% of unconstrained quality with 42–54% fewer observations. Under severe scarcity and the same hard caps, BudgetPM-Sequential exceeds the strongest tested natural monitoring schedule by 1.92–2.58 Set F1 points. It reaches the same Set F1 and on-time recall with 16–33% fewer observations. Matched attribution, exact-cost analysis, and a fixed-budget load intervention link this gain to competition between present and future opportunities. These results yield a demand–capacity design rule: local gating works when capacity covers demand, while future-aware supervision adds value when observations compete across time.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Prospective memory, memory-augmented agents, budgeted observation, sequential resource allocation, cost-aware agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HHECOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper formulates how the external-state checks required by stored intentions(clock, calendar, verifier queries, etc) should share a per-episode budget. It proposes two lightweight policies sharing a hard-cap executor: BudgetPM-Static, trained on replay-constructed labels of whether a check improves the current decision, and BudgetPM-Sequential, distilled from hindsight-DP full-episode schedules. Validation spans two benchmarks (PM-Bench, GoodAI) and three LLM backbones: end-to-end comparison against adapted Mem0/PMA, matched single-variable controls isolating the supervision label and scorer capacity, exact-cost structural analysis, and a fixed-budget load intervention. The central finding is a demand–capacity boundary: future-aware temporal allocation helps at B5/B10 but not at B15/B20, and the boundary moves with opportunity load.

Critical strengths
  1. Attribution methodology. Matched controls that change exactly one ingredient, paired bootstrap with prespecified power analysis, and exact-cost decomposition cleanly attribute gains to specific mechanisms. This makes the paper's sign-level claims credible even where effect sizes are small, and sets a good example for the field.
  2. Clear applicability boundary. The paper demonstrates that "future-aware supervision" only works under resource constraints and is useless at B15/B20. This gives practitioners a clear, actionable rule on when to avoid complex timing setups, raising the work above standard benchmark-chasing.
Critical weaknesses
  1. No proper comparison from the cited literature. When2Tool, BATS, CostBench, INTENT, PIS, and knapsack-bandit routers are cited as the closest lines of work but never run. In Table 1, Mem0/PMA run without an episode-level cap while their very low scores (0.201–0.513) plausibly reflect adapter/workflow mismatch rather than missing allocation, so "outperforms adapted memory-agent workflows" does not evidence the allocation contribution itself.
  2. The central finding rests on one benchmark, one generator, and unstated assumptions. The scarcity boundary is established only on PM-Bench (GoodAI is never tested at B5–B20); opportunities are near-uniform over episode time (193/197/204/207), a distribution favorable to temporal allocation; and supervision construction assumes queries do not change world state (Appendix A.3), a restriction never stated and violated whenever checking is itself an action. I think the assumptions are far from the real-world, so the paper's main contribution appears to reduce to a property of its own benchmark.
  3. Over-complicated framing. The method is essentially a budgeted tool-call gate that scores and executes queries under a hard cap; it does not manage memories itself. Terminology like "stored intentions" and "observation capacity" creates unnecessary confusion and mispositions the paper's novelty. The true novelty compared to budgeted tool-calling baselines (e.g., When2Tool) lies in its one-query-serves-many-intentions demand structure, not memory. The authors may frame the work as general budgeted allocation, treat PM as the application, and plainly emphasize this shared-resource structure.
Questions for authors
  1. Does the B5/B10 boundary persist under deadline-clustered or bursty opportunity processes? Your paired generator appears capable of injecting clustered opportunities.
  2. Is the method fundamentally restricted to read-only (non-state-changing) checks? If checking can alter the world (e.g., prompting a new supplier), how would the Outcome labels and DP teacher need to change?
  3. Given that policies are frozen after offline fitting, how would the framework handle distribution drift over long deployments — is periodic re-collection of trajectories the intended answer?
  4. The hindsight-DP teacher requires caching outcomes for every feasible query at development time and a DP over (TP, FP) states at each integer cost. Both are tractable at benchmark scale but may not scale to realistic settings. Please report the DP's runtime/memory scaling with T, B, and candidate-set size, and clarify the practical limits of this supervision pipeline outside the benchmark.
Reviewer_Cnj4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BudgetPM, a framework for allocating external observation resources under a limited budget in Prospective Memory. Prospective Memory focusing on storing and retrieving past information, it requires agents to monitor external states and execute stored intentions when the time arrive. BudgetPM-Static evaluates whether an external query improves the agent's current decision by comparing outcomes with and without the query. BudgetPM-Sequential further uses Dynamic Programming to generate optimal query schedules from complete historical trajectories and distills these schedules into a lightweight policy that decides when to query or reserve the remaining budget. Experiments on PM-Bench and GoodAI LTM show that BudgetPM performed better than Mem0 and PMA systems.

Critical strengths
  1. The paper is generally easy to read and follow.
  2. The proposed method is simple while efficient, keeping the online computation cost low. The experiments also show that BudgetPM reduces external queries while maintaining comparable task performance
  3. The paper goes beyond reporting performance improvements and investigates when future-aware allocation is beneficial. Through budget sweeps, exact-cost analysis, and controlled-load experiments, the authors show that local query selection is sufficient when the budget is adequate, while future-aware allocation becomes more useful when observation opportunities compete for limited resources. These experiments provide useful insights into why and when the proposed methods work.
Critical weaknesses
  1. The proposed methods mainly rely on existing techniques, including Logistic Regression, Dynamic Programming, and Policy Distillation. Although applying these techniques to observation budget allocation in Prospective Memory is meaningful, the algorithm novelty is relatively limited. The paper could better clarify how its approach differs from existing budget-aware tool-use and sequential resource allocation methods.
  2. The paper evaluates BudgetPM on only two benchmarks, PM-Bench and GoodAI LTM. More importantly, the experiments for Sequential are mainly conducted on PM-Bench, making it unclear whether the method generalizes to other tasks and more dynamic environments. Additional evaluations on different Agent workflows would strengthen the paper.
Questions for authors
  1. Why is Sequential mainly evaluated on PM-Bench rather than both benchmarks? Could the authors provide additional experiments or explain whether Sequential can generalize to other environments?
  2. In Table 1, BudgetPM is compared with Mem0 and PMA under different workflows and budget settings. Could the authors explain how these differences affect the results? Would it be possible to compare the methods under a more consistent setup?
Reviewer_Qxv7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how agents should allocate external-state checks for stored intentions under a shared observation budget. BudgetPM-Static learns whether a check improves the current decision, while BudgetPM-Sequential imitates offline hindsight schedules. Experiments examine local gating, temporal allocation under scarcity, and the relationship between observation demand and available capacity.

Critical strengths
  1. The Sequential-versus-Myopic comparison is convincing. The policies share features, model capacity, fitting and calibration procedures, and execution constraints; only their supervision differs. Sequential improves by 0.85–1.83 F1 points across two disjoint tests with equal or slightly lower observation cost. This provides useful evidence for future-aware supervision.

  2. The analysis goes beyond comparing final scores. Matching the oracle comparisons at exact total cost, then fixing the query count at each step, helps identify where temporal allocation creates value. The resulting lightweight policy is also attractive for a component that must run repeatedly.

Critical weaknesses
  1. The headline cost savings do not include the strongest adaptive baseline. The 16–33% savings are measured against the three natural monitoring schedules, although Pacing-LB performs better than Deadline at all four reported scarcity points. Pacing-LB is omitted from the full cost frontier. Against it, Sequential’s boundary-B5 advantage is only 0.0036 F1, with a confidence interval spanning zero (Table 19). The savings are therefore established against the cadence rules, but not yet against the strongest tested budget-adaptive control.

  2. The demand–capacity conclusion is stronger than the learned-policy evidence. The controlled-load interaction is +0.00429 F1 with a 95% interval of [−0.00064, 0.00922] (Table 22), and Sequential uses roughly 0.9 additional queries at both loads (Table 23). The intervention clearly creates exact-cost hindsight headroom, but does not establish that higher demand increases the deployed policy’s advantage at matched cost. This distinction should be clearer in the proposed design rule.

Questions for authors

NA

1 comment

Thank you for the detailed response and your efforts to address my concerns. The added comparison with Pacing-LB and the matched-mean-cost analysis address most of my concerns, and I appreciate the more careful framing of the conclusions. However, I still have some reservations about the robustness and generality of the gains. I am therefore inclined to maintain my original rating.

Comments on the paper

1 comment

I appreciate the authors' detailed response. However, I still have some reservations about the novelty, I am inclined to keep my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.