- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ContextGuard, a context-conditioned neuro-symbolic safety guardrail for mobile GUI agents. It grounds GUI observations, task instructions, proposed actions, and interaction history into soft semantic predicates, which are evaluated against predefined safety constraints to produce fine-grained runtime interventions (PERMIT, CONFIRM, or STOP). Experiments report improvements in safety detection and intervention accuracy over existing guardrail methods.
Critical strengths
- Meaningful problem setting. Context-dependent safety intervention is an important problem for mobile GUI agents, as the safety implications of an action can vary with user intent and interaction history.
- Interpretable framework. Combining neural semantic grounding with explicit safety constraints provides an interpretable mechanism for fine-grained intervention decisions.
Critical weaknesses
-
Limited evidence of generalization and advantages over model-based approaches. The proposed neuro-symbolic framework relies on predefined safety predicates and constraints, raising concerns about its scalability and generalization to open-ended real-world agent scenarios. While such rules may work well within a predefined safety policy space, it remains unclear how effectively they can handle unseen applications, tasks, or safety risks. More importantly, the paper does not sufficiently demonstrate the advantages of this rule-dependent design over model-based approaches in more diverse and realistic settings.
-
Insufficient experimental validation and methodological clarity. The main evaluation relies on a newly constructed benchmark with manually designed or audited safety labels, which may introduce subjectivity and potentially favor the proposed rule-based framework. Furthermore, the paper does not clearly specify the architecture, training procedure, or supervision of the Neural Grounding Model, which is a critical component of the proposed method and essential for assessing its effectiveness and reproducibility.
Questions for authors
N/A
4 comments
Thank you for the clarification. I understand that Neuro-Symbolic, or more specifically, rule-based approaches can help improve the interpretability and controllability of mobile agent safety. However, I still believe that, given the diversity and complexity of realistic mobile agent scenarios, model-based approaches should be prioritized over rule-based ones. The paper also does not provide sufficient evidence or experimental results to substantiate this choice. Therefore, I will maintain my current score.
Thank you for sharing your further thoughts. We agree that model-based approaches play an important role in addressing the diversity and complexity of realistic mobile agent scenarios. However, we believe that explicit rule-based reasoning offers complementary capabilities that are particularly valuable for runtime safety, where safety constraints and intervention decisions need to be interpretable, controllable, and verifiable.
Importantly, ContextGuard does not aim to establish the universal superiority of rule-based approaches over model-based ones. Instead, it demonstrates how context-conditioned symbolic reasoning can provide a principled mechanism for enforcing safety constraints during mobile agent execution. By grounding UI observations, task instructions, and interaction history into semantic predicates, ContextGuard explicitly conditions safety-rule applicability on task intent and workflow state, enabling fine-grained intervention decisions that are difficult to achieve through safety prediction alone.
Our experimental results provide empirical support for this design. On AndroidLab, we compare ContextGuard against four representative state-of-the-art model-based guardrails: Safiron, AgentAuditor, OS-Sentinel, and SeerGuard. ContextGuard achieves the best binary safety recognition and fine-grained intervention accuracy among all evaluated methods, outperforming these model-based baselines. We further evaluate ContextGuard on MobileSafetyBench, where it is compared with OS-Sentinel and SeerGuard. The results demonstrate its ability to transfer to a separate benchmark, including applications not encountered during AndroidLab evaluation, without benchmark-specific tuning. Together, these results provide strong empirical evidence for the effectiveness of context-conditioned neuro-symbolic reasoning in mobile agent runtime safety, including in comparison with state-of-the-art model-based approaches.
We view explicit symbolic safety reasoning not as a replacement for model-based approaches, but as an important complementary mechanism. Combining the flexible semantic understanding of neural models with the explicit constraints and controllable intervention provided by symbolic reasoning is a promising direction, and the current work establishes a concrete foundation for this direction. Thank you again for your thoughtful feedback.
Thank you for the thorough rebuttal, the error analysis, and the per-category breakdown. While these additions make the evaluation much clearer, your clarifications confirm my main soundness concerns. First, there is a big gap between how the paper is framed and what the system actually does: the draft presents ContextGuard as a neural grounding and differentiable reasoning approach, yet the rebuttal confirms it uses purely deterministic XML parsing, 26 hand-crafted rules with fixed weights, and no learned components. Second, the headline 87.8% intervention accuracy leans heavily on a separate lexical pattern matcher that handles the ~40% content-risk subset (where accuracy is 89–99%), whereas categories requiring actual GUI context reasoning perform much worse—such as Data Loss (64.7% recall), System Availability (62.5% recall), and Financial Loss (40.0% FPR). Combined with the alignment bias of testing hand-written rules and keyword patterns on an author-constructed benchmark whose labels match those exact outputs, the core methodological issues remain, so I will keep my current rating.
Follow-up on Rebuttal Discussion
Dear Reviewer KERj,
Thank you again for your thoughtful feedback. We have carefully addressed the concerns raised in your review and provided additional clarifications and experimental results in our rebuttal. We have also uploaded the latest version of the revised manuscript PDF, which incorporates these changes.
As the discussion period is approaching its deadline, we would greatly appreciate it if you could review our responses and the updated manuscript and share any remaining concerns or feedback before the discussion closes. We would be happy to provide further clarification if needed.
Thank you for your support and valuable time.