- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ForeNote, which maintains a compact JSON history note to prevent an agent with a limited context window from forgetting searched locations, target locations, or completed steps. During training, an environment-specific rule tracker constructs current and next-step reference notes. History-Note (HN) supervision scores the current note and contributes a note advantage to GRPO, while Future Prediction (FP) asks the model to predict how the note will change after its action and supplies an auxiliary cross-entropy target during SFT/RL. At test time the tracker is removed and the agent writes its own note. The experiments are extensive and include many baselines, training-stage/objective ablations, note-erasure and tracker-note interventions, seen/unseen ALFWorld scenes, WebShop, and ScienceWorld. The results are strongest on ALFWorld, while gains are smaller or less clear on the other environments.
Critical strengths
- The empirical study is thorough for an agent-memory paper. It includes many comparison methods, controlled HN/FP ablations, note erasure, tracker-note replacement, seen/unseen ALFWorld evaluation, and behavioral diagnostics such as repeated searches and taking back already placed objects.
- FP directly supervises which state fields an action will change, addressing the shared-omission blind spot of comparing only current notes. The intervention results also show that a test-time tracker provides only a small additional gain over the agent's own note, suggesting that the model can internalize the structured state representation under the proposed training recipe.
Critical weaknesses
- The core contribution appears to be an environment-specific training recipe that combines manually designed note schemas, rule-based trackers, SFT, and RL, rather than a substantially new learning principle. The paper should more clearly distinguish the contribution of the note interface, HN reward, and FP from the broader claim of general agent memory learning.
- Training and evaluation remain within the same benchmark families, simulator semantics, observation language, and action interfaces. Although unseen ALFWorld scenes provide limited layout generalization, the experiments do not establish transfer to a new environment or to settings where a reliable tracker is unavailable, such as a real-world environment. The method therefore primarily demonstrates tracker-to-self internalization within a fixed environment family, not sim-to-real or tracker-free generalization.
Questions for authors
- Can the authors evaluate ForeNote under a meaningful environment shift, such as a different simulator, altered observation/action formatting, or a real-world setting where the manually designed tracker is unavailable? At minimum, please report whether a new environment requires a newly hand-designed note schema and tracker, and provide a tracker-free transfer experiment or discuss the associated development cost and generality limitations.