- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PonderAgent, a model-agnostic safety framework for mobile GUI agents. It combines task-level pre-screening, semantic risk routing, RAG-enhanced safety review, and cross-step memory to selectively escalate actions for deeper review. On MobileSafetyBench, the framework improves harm prevention for open-weight actors from 1–3% to 65–66%, achieves a 93.3% prevention rate against indirect prompt injection in the default configuration, and is estimated to reduce average latency to roughly half that of uniform deep review.
Critical strengths
- The motivation is reasonable. Allocating review resources according to contextual risk can reduce unnecessary deep review and addresses a clear engineering need.
- The framework is modular and does not require access to the actor’s internal reasoning. Its rule and policy knowledge bases can be updated independently, facilitating integration with different actors.
- The experiments cover multiple actors, module ablations, risk categories, and injection attacks, and include comparisons with random routing. The authors also report limitations such as weak semantic discrimination, false rejections, and high latency.
Critical weaknesses
- The methodological novelty and GUI-specific challenges are insufficiently explained. Rule-based filtering, semantic matching, RAG-based review, and historical memory are established techniques, and lightweight screening followed by expensive review is an intuitive design. The paper should clarify what makes reducing review costs for GUI agents different from doing so for general agents or LLMs, and how these differences motivate its specific design. The discussion of contextual and cross-step risks does not sufficiently establish why this combination goes beyond applying a general safety framework to GUI tasks.
- The contribution of semantic risk scoring remains unclear. The R-Gate matches action context against approximately 100 risk patterns, but its semantic score achieves an AUC of only 0.53. Although random-routing comparisons and ablations support the complete gate, they do not establish the additional value of semantic matching. Moreover, supervised logistic regression over the same embeddings achieves an AUC of 0.96. While it requires labeled training data, the authors should explain the trade-off motivating their training-free design and consider an end-to-end comparison.
- The practical advantage over existing safety approaches remains unclear. Under the default configuration, benign-task success drops from 53% to 40%, while each slow-path review takes approximately 354 seconds. These costs do not invalidate the method, but comparisons with bare actors or uniform deep review alone are insufficient to establish its practical competitiveness. Comparisons with relevant safety frameworks, such as OS-Sentinel and SafeMobile, at comparable safety or task-utility levels would help determine whether PonderAgent offers a better overall trade-off in latency and cost.
Questions for authors
Please see the weakness.