- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Hippo improves agents with frozen model parameters by comparing multiple trajectories, reflecting on failures and fortuitous successes, and updating reusable textual memories. On WebArena and SWE-bench, memories are updated after all attempts on a task and used by subsequent tasks. Experiments also cover Mind2Web and a longer learning stream.
Critical strengths
- Tables 1–2 show consistent improvements across four backbones. With GPT-5.6 Sol, success increases from 64.2% to 72.9% on WebArena and from 81.5% to 88.5% on SWE-bench Verified.
- The long-horizon experiment includes both revisited and unseen tasks, with a useful append-only control: full Hippo achieves 86%/82% versus 72%/66% after 400 learning tasks (Table 11).
- The appendix provides concrete memory examples, failure cases, and a human audit of the judge.
Critical weaknesses
My main concerns are the limited methodological novelty, gaps in the experimental evaluation, and the lack of code provided to reproduce the reported results.
-
Novelty and baseline coverage remain insufficiently established. Training-Free GRPO (preprint, 2025) already derives semantic advantages from grouped rollouts and updates textual experience; ExpeL (AAAI 2024) compares successes and failures and revises insights. Mem0 (ECAI 2025) also updates and deletes existing memories, albeit in a different application setting. The external memory baselines are limited to AWM and ReasoningBank, leaving out representative alternatives such as ACE (ICLR 2026) for incremental maintenance and MemRL (NeurIPS 2026) for utility-based retrieval. Surprise handling and learning from all-failure batches are plausible distinctions, but the judge ablation removes both filtering and diagnostic text, preventing clear attribution. The paper needs a sharper account of its incremental contribution and a focused comparison under matched conditions.
-
The Best-of-N and efficiency questions are not directly answered. Table 3 reports only No Memory's mean per-attempt success (64.3%) and any-of-eight success (73.6%); Hippo has no corresponding any-of-eight or verifier-selected result. These cannot be compared directly with Hippo's main-table mean success, and the domain aggregation also differs. Moreover, the reported cost increase from $1.93 to $2.17 per task (+12.4%) compares two methods that both use eight rollouts. It measures overhead beyond repeated sampling, not the full cost relative to single execution. The cost proxy states that assessment is included, but the call breakdown does not clearly locate trajectory-judge calls. Thus, superiority over direct retries at equal total budget remains unestablished.
-
The evaluation supports online adaptation more directly than independent generalization. The main experiments update memory throughout the evaluated stream. WebArena uses reference-assisted memory construction, including for the memory baselines; Mind2Web uses gold-derived candidates and carries memory across splits. These protocols need to be distinguished from evaluation with frozen memory on independent tasks. Section 4.2 does include a held-out set, but the task composition and learning/test relationship are insufficiently specified. The main experiments hold task order fixed across seeds, and the long-horizon results lack an order-robustness check.
-
Reproducibility is difficult to assess. No separate supplementary materials were submitted. Although the main PDF includes appendices, some implementation and evaluation details remain insufficiently specified, making it difficult to establish whether the reported results can be independently reproduced.
-
Minor comments. Clarify which concrete design choices follow from the hippocampal replay analogy. “Misleading or overly trajectory-specific lessons” would be clearer than the undefined “overfitting a lesson”. Fix “basedline” and the duplicated “(i)” in Section 3.3. The improvement reported in Section 4.3 is 0.9 percentage points, not a 0.9% relative increase.
Questions for authors
- Can you compare against one close trajectory-comparison method (Training-Free GRPO or ExpeL) and one maintenance/retrieval alternative (ACE or Mem0 or MemRL), adapting them to the same feedback and task stream?
- Can Hippo and No Memory be evaluated with the same success/selection criterion and total budget, including judge and memory costs, across different rollout counts?
- Could you provide a little more detail on the Mind2Web evaluation setup and how tasks are selected and divided in the long-horizon experiment? This would help readers better understand the scope of the reported generalization results.