- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes VerMem to address challenges faced by long-horizon LLM agents, such as heterogeneous memory coordination, historical context recovery, and multi-granularity credit assignment. The authors claim that the method achieves unified memory control with explicit history recovery, verifier-guided hierarchical credit assignment over different comparison sets, and full-command policy learning with uniform command-level weights. Single-run results on a limited evaluation suite suggest improvements over conventional methods.
Critical strengths
- The flow diagrams are well drawn (e.g., Figures 1 and 2, and Figures A.1 and A.2). Their minimalist composition makes them relatively easy to follow.
Critical weaknesses
1. The paper reads very poorly. Specifically:
- First, the algorithm and experiment sections repeatedly point to the appendix, and the main text alone does not let readers clearly grasp the whole algorithm. I would suggest presenting the entire algorithm compactly in pseudocode with a detailed natural-language explanation, while moving to the appendix the parts that do not affect the overall exposition (e.g., the rationale for choosing specific parameters).
- Second, the placement of figures and tables is haphazard: Figures 3–6 are all crowded onto page 8 even though Figure 3 is first referenced on page 6; small tables such as Tables 2, 3, and 4 could be placed on the same row or otherwise worked into the running text.
- Third, when presenting experiments, the paper should first explain what each symbol denotes, then report the results, and only then analyze them. The current experimental section is fragmented and in places reports results before giving definitions, which confuses the reader.
2. The paper is hard to reproduce.
In the algorithm section, the authors genuinely try to show readers the full VerMem pipeline through detailed description. But the pipeline involves many computation steps, each with several parameters, and there is insufficient evidence to support all of these design choices and settings; a reader therefore cannot determine from the paper alone whether the method is reproducible.
Questions for authors
- Lines 162–165 mention token-count definitions such as 8192 tokens. What is their significance, and how were these parameters chosen? Similarly confusing concrete values include {0, 1, 2, 3, 4} at line 174.
- The experimental results appear to come from single runs. How is the robustness of this work assessed?
- I could not understand how the multiple experiments support the paper's key claim — namely that the results show VerMem achieves effective memory control, history recovery, credit assignment, and policy learning relative to prior work, and delivers better performance. Could you explain whether the experiments strongly support this claim, and how each of them does so?
The authors did not respond early in the discussion period. In my assessment, the work still has substantial room for improvement, and I encourage the authors to continue developing it.