acceptodds
Desk rejected at ICLR 2027

M-Plan: Comprehensive Planning For LLM Agents with Memory Arrangement

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language model (LLM)-based agents have recently demonstrated outstanding performance in single-turn tasks, yet they continue to struggle with long-horizon tasks, particularly in environments without prior training. A key limitation of single LLMs is their difficulty in processing long contexts, which stems from weaknesses in continuous decision-making and reasoning capabilities. Although existing multi-agent systems (MAS) equipped with long-term memory bases can partially address this gap, they mostly rely on knowledge reuse or summarization—an approach that is often constrained by the specific requirements of the current task. In contrast, MAS characterized by task-specific guidance often lack comprehensive long-term memory designs. To address these challenges, we propose M-Plan, a closed-loop planning framework that integrates long-term and short-term memory. By combining global-local and past-current guidance, M-Plan enhances agents' performance on multi-turn tasks while maintaining a plug-and-play design. During task execution, agents receive task-specific milestones and insights, and are provided with local hints only when confusion arises. These guidance generation and utilization construct the short-term memory. After task completion, relevant information about the current task is optimized or reorganized before being sent to the long-term memory storage for future retrieval. Extensive experiments on multi‑turn tasks including ALFWorld, WebShop, and PDDL across diverse models demonstrate that M‑Plan outperforms multiple baseline methods in both accuracy and efficiency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent, memory, plan

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EtUeOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes M-Plan, a closed-loop multi-agent framework for long-horizon LLM agents that integrates non-parametric long-term memory with a short-term planning and guidance mechanism. The short-term memory comprises a Planner (milestones), an Insighter (task-specific global insights), and a conditionally triggered Hinter (local hints which is activated only when the agent falls into loops or invalid actions), while an Optimizer refines completed trajectories and reorganizes milestones/insights before storing them back into a graph-structured long-term memory.

Experiments on ALFWorld, Webshop, and a PDDL-based benchmark show consistent gains in success rate, reward, step efficiency, and an introduced token-efficiency metric over representative memory-reuse and step-wise guidance baselines.

Critical strengths

M-plan introduces a well-structured combination of global (milestones + insights) and conditional local hints, avoiding expensive per-step hinting while still correcting common multi-turn pitfalls. A simple graph-based one-hop expansion on top of vector retrieval is a reasonable, low-cost way to diversify references for short-term guidance without heavy step-wise retrieval.

Evaluates across three standard multi-turn benchmarks with different characteristics (embodied text environment, web navigation, symbolic tasks), and with two backbone model sizes, showing robustness. The conditional triggering logic for the Hinter is described (loose vs strict regimes) and justified by environment characteristics. Overall the paper addresses the core challenge of token/latency costs in per-step MAS while retaining strong corrective power via conditional hinting.

Critical weaknesses

The Ft/Ct token-efficiency metric is ad-hoc. The analysis contains a mathematical misstatement (it claims Ft attains a maximum at SR=0.5 whereas Ft is monotonic in SR, and it is the first derivative that peaks around SR=0.5). Also conditional hinting rules are heuristic (similar other are - a) one-hop memory expansion policy; b) criteria for “polishing” milestones; and c) the profit/efficiency metric) and depend on environment feedback. Paper provide no analysis on false positives/negatives or sensitivity to trigger thresholds.

The reported token usage (At) appears surprisingly low given multi-agent prompting and environment interactions; token accounting methodology (input vs output, environment observation size, system prompts) is insufficiently specified. I also find that comparisons largely omit strong recent hybrid/planning baselines that couple LLMs with symbolic planners or programmatic policies, which are directly relevant to the PDDL-style setting and many other step-wise or retrieval agents (e.g., ReAct variants, step-wise thought retrieval etc.) are mentioned but not included as strong baselines in the main tables. Some references are provided here

[1] Yao, Shunyu, et al. "React: Synergizing reasoning and acting in language models." arXiv preprint arXiv:2210.03629 (2022). [2] Shinn, Noah, et al. "Reflexion: Language agents with verbal reinforcement learning." Advances in neural information processing systems 36 (2023): 8634-8652. [3] Verma, Nikhil, and Manasa Bharadwaj. "LEAP & LEAN: Look-ahead Planning and Agile Navigation for LLM Agents." Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track). 2025. [4] Liu, Yuxin, et al. "MAP: A Map-then-Act Paradigm for Long-Horizon Interactive Agent Reasoning." arXiv preprint arXiv:2605.13037 (2026).

Questions for authors
  • How exactly are tokens counted for At? Do you include both input and output tokens, system prompts, environment observations, and all multi-agent calls? What tokenizer and accounting tooling are used?
  • For the Optimizer, what concrete rules distinguish “redundant” milestones in practice, and how often does milestone polishing materially change subsequent performance? An ablation on Optimizer sub-components would help (polish-only, refine-only, sparsify-only).
  • On PDDL tasks, could you include at least one hybrid symbolic baseline (e.x. TwoStep, PSALM, or a programmatic-planner baseline like LMP LAN/GenePlan) to contextualize M-Plan’s performance on symbolic reasoning tasks?
Reviewer_Q1DGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes M-Plan, a multi-agent framework, for long-horizon tasks, linking long-term memory with short-term memory for task-specific guidance in a closed loop. The long-term memory, starting from scratch, stores each finished task in a vector database, linking similar tasks using a graph format. The short-term memory retrieves two successes and one failure, which are then used by a Planner to write milestones, and by an Insighter to write insights for the current task. A Hinter is called only when the Actioner repeats an action or makes an invalid one. The hint gives the current state, current milestone, what is missing, and a corrected action. The Optimizer revises the insights by comparing a failed trajectory with the most similar success, and removes repeated or invalid steps from successful trajectories. M-Plan with Qwen3-14B and Qwen3-32B, achieves the best success rate across three multi-turn benchmarks (ALFWorld, WebShop, PDDL). Ablation studies justify the importance of different modules of M-Plan.

Critical strengths

M-Plan achieves the best success rate and reward against all three baselines, on all three benchmarks, at both model scales. The Hinter module is called only on invalid or repeated actions, thereby reducing the token cost associated with per step guidance and also contributes significantly to performance improvements.

Critical weaknesses

The evaluations are currently limited to one model family and generalization to more realistic environments like WebArena remains to be seen. Also the triggering rules for the hinter module, which is one of the key components of M-Plan, seem a bit fuzzy and benchmark dependent.

Questions for authors
  1. How was HiPlan set up for PDDL?

  2. Can the authors try on another model family (like Llama3)?

  3. Do the authors have any insights as to why even after ablating all modules (especially retrieval and hinter), beat all memory based baselines on PDDL by a big margin? Does this also hold for other benchmarks?

  4. Is long term memory built over the evaluation tasks? Is the method sensitive to ordering of tasks? Is there any held out test set after memory is built?

Reviewer_FiWsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Long-horizon multi-turn agents need to reuse what worked on past tasks, give local hints only when the current step goes off track, and feed refined experience back into memory after each task. M-Plan implements this loop with a graph-augmented long-term memory, a short-term planner that issues milestones and task-specific insights, and a conditionally-triggered hint module plus a post-task optimizer. On two Qwen3 sizes across ALFWorld, Webshop and PDDL, it reports the best success rate and a roughly two-thirds raw token cut on ALFWorld.

Critical strengths
  1. Global milestones address coarse plans, task-specific insights address missing detail, and conditionally-triggered hints replace per-step prompting; the PDDL ablation shows each module contributes — removing Hinter drops SR from 0.528 to 0.411, removing Retrieval to 0.425 (Table 4).
  2. The Hinter fires only on invalid or immediately-repeated actions (App. D), and on ALFWorld Qwen3-32B the raw At column already shows 0.03M tokens vs HiPlan's 0.09M — about a 2/3 cut before any normalization (Table 3).
  3. Gains are consistent across the two Qwen3 sizes and three benchmarks, and long-term memory starts empty and accumulates across tasks, matching deployments without an expert-demo library.
Critical weaknesses
  1. The HiPlan baseline is a self-reproduction. No official trajectories are available, so the authors built 6 ALFWorld + 25 Webshop expert traces with Qwen3-32B (App. C); whether this reproduction matches published HiPlan numbers or decoding settings is not reported, so the gap to "HiPlan" reflects the authors' implementation rather than a verified comparison.
  2. The long-memory gain is not separated from memory maturation. Final SR is aggregated over the whole task pool with no task-order or early-vs-late curve, so closed-loop accumulation cannot be told apart from the Planner/Insighter/Hinter design; a warm-up breakdown would show where in the task order M-Plan's gap grows.
  3. The ablation covers one benchmark and one backbone. Table 4 only ablates PDDL with Qwen3-32B (where M-Plan's absolute gain over G-Memory is largest), and Hinter trigger rules are tuned per environment (loose for ALFWorld/Webshop, strict for PDDL; App. D), so module contribution cannot be separated from trigger-rule tuning.
  4. Does the one-hop graph expansion actually help over plain vector retrieval? Table 4 only removes Retrieval as a whole. It does not test different retrieval counts or the fixed mix of two successful cases and one failed case, so it is unclear how much these choices matter.
Questions for authors

Refer to above weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.