- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper tackles memory retrieval for GUI agents adapted to a specific application: given the current screenshot and a user goal, which previously observed state should be retrieved to guide the next action? The authors formalize this as goal-conditioned graph localization, define functional continuation classes — recursively, a state's optimal goal-directed transitions together with the continuation classes of their successors — and use these as supervision to distill a lightweight screenshot–goal representation (frozen unimodal encoders + a small projection). At inference, retrieval is a single cosine-similarity ranking with no graph traversal and no extra VLM call (29.6 ms, 205M params). A new corpus of 49 Android/HarmonyOS apps (7,903 states / 39,154 transitions / 36,351 intents) is built via VLM annotation with human auditing. GCFSim achieves the best NA@1 (.517) among nine baselines and consistently improves first-action selection for seven frozen GUI agents.
Critical strengths
S1: Application-memory retrieval is important for efficient GUI agents, and the distinction between visual similarity and functional similarity is well motivated.
S2: Motivation, design, and evaluation are coherently aligned: the paper's selling point is zero added inference latency, and it delivers exactly that (29.6 ms / 205M vs. 420 ms / 8.5B for LLM re-ranking). The paper includes multiple retrieval baselines, frozen-agent evaluations, continuation-depth analysis, robustness tests, and failure analysis.
S3: The experiments indicate that functional continuation supervision improves retrieval beyond visual similarity, goal distance, and immediate-action supervision.
Critical weaknesses
W1: The main metric and downstream experiments focus on first-action retrieval. The paper does not evaluate complete task success, long-horizon navigation, action efficiency, or recovery from mistakes.
W2: GCFSim is trained separately for each application, while training and test goals come from the same explored graph. Generalization to unseen applications, cross-platform transfer, and application-version changes is not demonstrated.
W3: The five-term loss (BCE + flip + sup + NCE + hard, weights 1.0/2.0/1.0/0.5/0.5) is never decomposed; no λ sensitivity.
W4: Notation and baseline citation issues. The Introduction introduces without defining , , or . In addition, the tables do not consistently identify which methods are prior work and which are author-implemented baselines. External models such as SigLIP, VisualGNN should have direct citations.
Questions for authors
-
Complete task-level evaluation: Does retrieved functional memory improve complete task success, total interaction steps, execution time, or recovery after incorrect actions, rather than only first-action accuracy? Can you run even an end-to-end evaluation (e.g., AndroidWorld or MobileWorld) reporting full-task success with/without GCFSim memory?
-
Generalization and data splits: Are the training, validation, and test goals strictly state-disjoint or path-disjoint within each application? Can GCFSim transfer to unseen applications, different platforms, or new application versions without training a separate model from scratch?
-
Loss decomposition and sensitivity: What is the individual contribution of the five loss terms—BCE, flip, supervised contrastive, NCE, and hard-negative loss? How sensitive is performance to the coefficients (1.0/2.0/1.0/0.5/0.5), particularly the weight of the flip term?
-
Fix the notation and citation problem.
Reminder: Please Review Author Rebuttals and Participate in Discussions
Dear Reviewers,
Thank you for your time on the reviews. The authors have now submitted their rebuttals, and I encourage you to read their responses and participate in the discussion.
Please consider whether the rebuttals address your concerns and, where clarification is still needed, engage with the authors through the discussion forum. I also encourage you to exchange perspectives with your fellow reviewers and update your review or rating if appropriate.
Your participation will help ensure a thorough and fair evaluation of each paper. Thank you again for your time and effort.
Best regards,
AC