- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to retain and retrieve useful history for long-horizon GUI tasks. MementoGUI adds a trainable memory controller to a frozen action model. It maintains working memory through textual summaries and local screenshot crops, and retrieves episodic memory from completed trajectories. The authors automatically construct training data and evaluate on GUI-Odyssey, MM-Mind2Web, and the new offline MementoGUI-Bench. The results show improved action prediction across several backbones, with contributions from local visual evidence and two-stage retrieval.
Critical strengths
- The method combines textual summaries, local images, and experience retrieval into a trainable memory layer that works with frozen GUI action models, offering a concrete approach to using interaction history more effectively.
- Experiments across backbones and component ablations support the design. Table 14 compares trained and untrained controllers of the same size while fixing the memory pipeline and action model, providing stronger evidence for training gains than simply adding context.
Critical weaknesses
- The central goal is better long-horizon control, but experiments mainly use recorded interface states and do not test recovery after incorrect actions, subsequent memory updates, or final task completion. Better offline action matching does not establish long-horizon gains in live interaction. This is the main limitation affecting the overall assessment.
- VAM and TPS read memory, potentially conflating memory–action consistency with action correctness. The action-only scores in Table 13 provide additional evidence but lack corresponding human validation; the human comparison in Table 9 also includes only configurations using the proposed memory. More independent evidence is needed to interpret improvements in action quality and task progress.
Questions for authors
- Could you provide closed-loop long-horizon results comparing final completion, repeated actions, and forgotten constraints?
- In light of Table 15, is the main advantage of the compact trained controller over the prompt-only Gemini controller latency, inference cost, or deployment requirements? Do existing measurements support this trade-off?