- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper asks a useful question for long-lived agents: once an agent has a world model that captures regularities of its environment, what information from an individual experience still needs to be stored explicitly? The central idea is that persistent memory should be divided into shared world knowledge and episode-specific information. If the world model can reliably reconstruct some part of an experience, repeatedly storing that information is wasteful; episodic memory should preserve primarily what is both difficult to predict and consequential for future tasks. The proposed framework, MemoWM, implements this idea by encoding observations into a compact, query-independent discrete representation and using an action-conditioned world model to predict each coordinate from previously reconstructed states.
Critical strengths
- The core idea is both conceptually clean and technically substantive. Treating a world model as shared side information for episodic memory is a natural but nontrivial reframing.
Critical weaknesses
The task-aware componen relies substantially on the offline task/query distribution, and very-long-horizon robustness remains only partially demonstrated.
Questions for authors
N.A.
7 comments
Response for Weakness 2
Weakness 2: very-long-horizon robustness remains only partially demonstrated.
• Robustness over longer interaction horizons.
The original evaluation covers rollouts up to 40 steps. To examine longer horizons, we collected additional WebArena trajectories and extended the closed-loop evaluation to 150 steps, using the same comparison with a teacher-forced oracle:
Table 7. Closed-loop reconstruction relative to a teacher-forced oracle on WebArena rollouts up to 150 steps. Extra NLL is in bits/answer; extra storage is in bits/state. (Appendix G of the revised manuscript.)
| Step range | Extra NLL (bits) | Extra storage (bits/state) |
|---|---|---|
| 1–25 | 0.010 | 15 |
| 26–50 | 0.014 | 19 |
| 51–75 | 0.018 | 21 |
| 76–100 | 0.022 | 23 |
| 101–125 | 0.025 | 24 |
| 126–150 | 0.027 | 25 |
Both additional NLL and storage increase gradually over the 150-step rollout, without a sharp deterioration. Even in the final 25 steps, the additional NLL is only 0.027 bits, with 25 additional bits/state of storage. These results extend our original 40-step evaluation (revised manuscript, Section 4.6; Figure 4(b)) to 150 steps and provide further evidence of stable closed-loop reconstruction over longer interactions. Thank you again for highlighting these two aspects of robustness. Your feedback helps us clarify when the offline statistics remain useful and the interaction horizons covered by our evaluation.
We hope these clarifications and additional results address your concerns, and we look forward to your further feedback.
Dear Reviewer vTf3,
Thank you again for your careful review and constructive suggestions. We would like to briefly follow up on our responses and the additional experiments.
When you have a chance, we would be grateful to hear whether the new results and clarifications address your main concerns, or whether any points would benefit from further explanation. We would also appreciate an updated assessment of the paper in light of these additions and revisions.
Thank you for your time and consideration.
Best regards, The Authors
Response for Weakness 1
We sincerely thank you for your thoughtful review and thorough understanding of our work. We especially appreciate your recognition of the core idea as conceptually clean and technically substantive, and of the value of treating world models as shared side information for episodic memory. Your insightful feedback on offline task statistics and longer-horizon robustness motivated the held-out analyses and 150-step closed-loop evaluation presented below.
We first clarify the role of the offline statistics and their held-out evaluation, then present the longer-horizon results.
•Why use offline task-impact statistics?MemoWM is designed to construct query-independent memory, meaning that memory is written before future questions are known. Therefore, task importance cannot be estimated using a specific future query at writing time. Instead, we estimate two shared statistics offline: (1) a coordinate-level task-impact prior Vⱼ, which measures how much a prediction error affects future answers, and (2) an entropy-to-error lookup gⱼ(H), which estimates prediction error probability from predictive entropy. Both are fixed before deployment and shared by the writer and decoder. This allows the same memory to support different future queries without requiring an additional Reader call during memory writing.
These estimates depend on the offline data distribution. We evaluate their usefulness on held-out trajectories and unseen-query variants below. First, the lowest-impact coordinates remain relatively unimportant across different trajectories and query formulations:
Table 3(c). Answer-NLL damage conditioned on prediction errors for original queries and independently constructed unseen-query variants (10⁻³ bits/answer). (Section 4.5 of the revised manuscript.)
Even on independently constructed unseen-query variants, omitting these coordinates causes much less damage than omitting equally sized random subsets.
Second, we evaluate the frozen entropy-to-error lookup on held-out trajectories:
Table 3(b). Frozen entropy lookup versus observed prediction error (%). (Section 4.5 of the revised manuscript.)
The observed prediction error increases monotonically from 8.1% to 88.8% as entropy increases, and the frozen lookup preserves the same trend. Although its absolute estimates are not perfectly calibrated, it provides a useful signal for ranking prediction risk on held-out data. Together, these results support reusing offline task-impact and prediction-risk statistics across different states and queries. A substantially different deployment task distribution may require recalibration. We will clarify the conditions for reusing these offline statistics in the revised manuscript.