acceptodds
Under review as a conference paper at ICLR 2027

GAP: GATING ACTION WITH A PLAN FOR SAFE SERVICE AGENTS

Abstract

Customer service relies on trained professionals to resolve complex requests through multi-turn interactions. Large language model (LLM) agents can automate such work, but executing write actions such as refunds and cancellations is risky: their legality depends on backend facts the user may misstate. Adversarial users exploit this to induce unauthorized write actions—fabricating facts, expanding a request’s scope, exploiting ambiguous confirmations, or pressuring the agent to skip checks. Yet even strong LLM agents struggle over long interactions: they may omit decisive checks, skip confirmation, lose evidence, and execute a write action before the required conditions are met. We propose GAP (Gating Action with a Plan), which equips each request with an action-gating plan: a checklist of facts to verify, missing information to obtain, and the write action to confirm. GAP maintains the plan across turns, permits the write action only after all required checks pass and the user confirms the exact action, and closes the gate when verification returns counter-evidence. To train this behavior, we adapt on-policy self-distillation to distill a privileged teacher’s ability to construct and maintain action-gating plans through dense, token-level supervision, and use role-separated advantage estimation to give planning and execution their own learning signals. Across four customer-service domains, GAP-trained Qwen3-14B policies achieve 80.7% task success while limiting the attack success rate to 8.6%. Compared with DeepSeek-V4-Pro, GAP raises task success by 5.5 percentage points and lowers the attack success rate by 1.5 points.

Primary area:
reinforcement learning
Keywords:
Agentic Safety, LLM-based Agents, Reinforcement, Conversational AI, Adversarial Learning, Robustness, Post-Training, Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_ea6QOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Customer service agents must handle adversarial users while ensuring that required checks and gates are satisfied before taking further action. The authors show that prompting alone can address some of these challenges but does not guarantee adherence to the gates specified in a plan. To address this limitation, they introduce GAP, a training method that combines OPSD with separate advantage estimation to improve agents’ adherence to these gates during execution. Extensive experiments shows the effectiveness of the approach compared to baselines.

Critical strengths

The paper presents detailed experiments and ablation studies that demonstrate the method’s improvements and clarify the contributions of individual components.

The paper is well written, with a clear motivation and logically structured arguments.

The separation of planning from action, together with explicit gates that govern execution, is an interesting and effective approach to enforcing procedural constraints.

Critical weaknesses

None

Questions for authors

Is the formulation of the loss for OPSD same as what is standard? If not, then what is the purpose behind the different approach.

Reviewer_QFGSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces GAP, a framework designed to safeguard agents from executing unauthorized write actions during multi-turn interactions. GAP addresses these vulnerabilities by establishing a persistent action gating checklist for every identified request. The agent must check critical facts using read only tools, collect missing information, and secure explicit approval before performing any actions. Crucially, discovery of contradictory evidence immediately abandons the pending action. To effectively achieve this behavior, GAP combines supervised fine-tuning with on-policy self-distillation from a teacher model. Evaluated across four distinct domain datasets, a fine-tuned model achieves reliable tradeoff between TSR and ASR.

Critical strengths
  1. The problem studied in this work is critical.
  2. This work conducts extensive experiments with well-known commercial models and open-sourced models to show the superiority of the proposed method.
Critical weaknesses
  1. Methodology part includes too many notations, resulting in several notation typos. For example, in Eq. 6, the ϵ=10−6\epsilon = 10^{-6}. However, in the line 296, ϵ\epsilon is also used to control the update strength of GRPO. Additionally, in Eq. 5, there is a clear equation typos. Overall, this methodology section has many types of typos. I believe the work need to be further refined.

  2. In line 53, TSR and ASR are used for the first time. However, the paper does not clearly define these terms upon their first appearance, only defining them later in line 204.

  3. Although this work improves agent safety, it does not analyze the additional costs introduced by the proposed method. Since multiple ask actions and confirmation queries require human approval, the latency and time costs of agent operations remain unclear.

  4. The baselines only include standard base models and RL-based methods. Closely related methods from prior work that address similar problems are missing.

Questions for authors

See weakness

Reviewer_fGuVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies customer-service language-model agents that must modify databases for legitimate users without being manipulated into unauthorized actions. Existing agents skip checks, act before confirmation, or lose counter-evidence over long dialogues, and prompting them to plan does not fully fix this. GAP maintains a checklist for each request, with a status machine that allows writes after confirmation and closes the gate when verification contradicts the user's claim. Supervised fine-tuning on relabeled trajectories is followed by reinforcement learning with separate advantages for planning and execution, plus self-distillation from the same model given the gold action and attack type. Tests use simulated adversarial users across two benchmark domains and two author-built domains. The trained model has higher task success and comparable attack success to the best proprietary baseline; the paper also reports ablations and tests with unseen simulators.

Critical strengths
  • The empirical gain is strong.
  • The considered setting is important, and trustworthy AI systems are crucial.
Critical weaknesses
  • The novelty is limited. It's basically a plan-then-execute framework with on policy self-distillation.
  • The gate is enforced by the same model doing execution. This can cause serious reliability issues. What if the model misses an important check? What if it finds shortcuts in the training signal, that could skip necessary checks in some scenarios?
  • For customer-service systems with structured backend data, it's much easier and more reliable to use mandatory preconditions written in code. It is unclear to me why train a model.
  • There is only in-domain testing and no cross-domain generalization evidence. Also the results seem to limited to one model (Qwen3-14B).
  • The paper quality also needs improvements. Many of the figures are not vectorized.
Questions for authors
  • Any evidence on cross-domain generalization after training?
  • The baselines, without explicit plan-then-execute, can starve for training signals initially. Are baselines also initiated from the SFT checkpoints before GRPO?
Reviewer_bJdiOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GAP, a framework for enabling customer-service agents to safely execute write actions such as refunds, cancellations, and exchanges. GAP maintains a persistent verify-ask-confirm plan for each request and permits a write action only after the required checks and explicit user confirmation are completed. The method combines privileged-teacher-based OPSD with DualAdv, which assigns separate learning signals to planning and execution tokens.

GAP is evaluated by separately training Qwen3-14B policies for four domains: Retail, Airline, Hotel, and Food Delivery. It achieves an average TSR of 80.7% and an ASR of 8.6%. The paper validates the method through training-stage ablations, attack-type breakdowns, plan-quality evaluation, and experiments with unseen attack strategies and user simulators.

Overall, GAP achieves strong performance in the evaluated settings. However, the methodological novelty is limited, and the lack of comparably informed baselines and external safety-layer comparisons leaves the main claims insufficiently supported. Generalization across service domains, model families, and parameter scales is also not established. Furthermore, despite having only 80 underlying test tasks, the paper does not provide a sufficiently systematic case analysis of when prompt-only agents and GAP succeed or fail. Addressing these issues would make the paper’s contribution substantially more convincing.

Critical strengths

1. Author addresses an important and practically consequential safety problem.

GAP focuses on the safe execution of database-modifying actions, such as refunds, cancellations, and exchanges, rather than only on response-level safety. Evaluating both legitimate task completion and unauthorized action success provides a meaningful view of the trade-off between utility and safety in customer-service agents.

2. The components of the training pipeline are evaluated systematically within the studied setting.

Using the same Qwen3-14B setup, the paper incrementally compares prompting, SFT, GRPO, OPSD, and DualAdv, and additionally evaluates unseen attack strategies and user simulators. Although these experiments do not establish broad transferability, they provide evidence that the proposed components work complementarily in the evaluated domains and yield more reliable behavior than prompting alone.

Critical weaknesses

1. The comparison design does not establish the necessity or advantage of domain-specific GAP training.

GAP is trained separately for each evaluation domain through SFT and RL, with supervision from a privileged teacher that observes the user's legitimate intent, gold action, and adversarial context. In contrast, the frontier-model baselines in Table 1 receive the runtime policy and zero-shot instructions without comparable domain adaptation. Therefore, the result that GAP-trained Qwen3-14B outperforms larger models conflates the benefit of the proposed gating mechanism with that of domain-specific training.

Vanilla DAPO and Vanilla OPSD are useful objective-level controls within the GAP pipeline, but they do not establish that the overall training procedure is necessary. In particular, the +prompt condition uses detailed zero-shot instructions but no few-shot demonstrations or retrieval. Since the authors already possess successful training trajectories and request-specific gold plans, a stronger training-free baseline could retrieve relevant plan templates or demonstrations from the training split. The current experiments show that zero-shot instructions are insufficient, but not that in-context adaptation is insufficient or that per-domain SFT and RL are required.

The paper also discusses symbolic guardrails, policy engines, reviewing agents, and pre-write reflection in Section 2.2, but does not evaluate these alternatives. An external verifier or deterministic gate could check the required evidence and explicit confirmation immediately before a write action, potentially avoiding expensive per-domain training. Without comparisons to such alternatives, the practical necessity of learning a self-enforced gate inside the policy remains unclear.

2. The methodological novelty is limited, while transferability and residual failure modes are insufficiently examined.

GAP combines established components, including GRPO, OPSD, role-separated advantage estimation, explicit planning, and checklist-based verification, within a persistent verify-ask-confirm plan and request-state machine. This may be a useful system design, but the contribution can also be viewed as a domain-specific integration of existing techniques rather than a fundamentally new learning principle. Strong evidence of transferability and a detailed account of the resulting behavioral improvements are therefore important.

However, all GAP results use a single Qwen3-14B backbone, and a separate policy is trained for each service domain. The OOD evaluations vary the attack strategy and user simulator, but not the service domain, policy, or tool schema. The paper does not evaluate:

  • joint multi-domain training,
  • transfer to a held-out service domain,
  • adaptation to a new policy or tool schema, or
  • application to other model families and model scales.

It is therefore unclear whether every new domain requires a new round of data generation, SFT, and RL, or whether the learned behavior transfers across service settings.

The failure analysis is also limited despite the evaluation containing only 80 underlying test tasks. The paper reports aggregate attack-type metrics, LLM-judged plan scores, and three illustrative cases in which GAP succeeds, but these cases are not paired under identical adversarial interactions. A systematic analysis should distinguish:

  • prompt failure and GAP success,
  • failure by both methods, and
  • prompt success and GAP failure.

The remaining attack successes and legitimate-task failures should also be categorized into planning omissions, verification errors, confirmation mistakes, cross-request state confusion, over-refusal, and execution failures. Such an analysis is important for determining what the learned gate actually fixes, what regressions it introduces, and why it still fails after domain-specific training.

Questions for authors
  1. Comparison with in-context adaptation.
    Were the frontier-model baselines in Table 1 given the same complete runtime protocol and domain policy as GAP? Did the authors evaluate stronger in-context baselines using few-shot demonstrations, retrieved gold-plan templates, or successful trajectories from the training split? If so, please provide the results. If not, please explain how the necessity of per-domain SFT and RL can be established from comparisons against zero-shot prompting alone.

  2. Comparison with external safety mechanisms.
    Please include a comparison with the external defenses discussed in Section 2.2, such as a reviewing agent, symbolic guardrail, policy engine, or deterministic pre-write gate. In particular, a simple external mechanism could check whether all required verifications were completed, whether counter-evidence was found, and whether the user explicitly confirmed the exact action before permitting a write. Without such an experiment under the same evaluation protocol, it is difficult to determine whether learned self-gating through domain-specific SFT and RL is necessary or preferable to a training-free external safety layer.

  3. Generalization across backbones and model scales.
    All trained GAP results are limited to Qwen3-14B, making it difficult to assess the method's generality and novelty. Are results available for other Qwen model sizes or other model families? Does GAP retain a comparable relative benefit as the backbone becomes larger or improves in instruction following and tool use? If not, what evidence supports treating GAP as a model-independent method rather than a training recipe specialized to Qwen3-14B? Additional evaluation across model families and parameter scales is necessary to substantiate this claim.

  4. Transfer across service domains.
    Must a separate GAP policy be trained for every service domain? Please provide any available results from joint multi-domain training or from training on a subset of domains and evaluating on a held-out domain. Otherwise, please clarify the data-generation and retraining requirements when applying GAP to a new domain, policy, or tool schema.

  5. Systematic analysis of successes and failures.
    Using the existing evaluation logs, please report how frequently the following outcome categories occur: zero-shot prompt failure with GAP success, failure by both methods, and zero-shot prompt success with GAP failure. Please further categorize GAP's remaining failures into planning, verification, confirmation, state-maintenance, and execution errors. If few-shot demonstrations or retrieved plans or trajectories were evaluated, please provide the same case analysis and explain which zero-shot prompting failures these in-context methods resolve and where they still underperform GAP.

Comments on the paper

1 comment
Reviewer_bJdiOpenReview ↗

Reviewer bJdi’s Response to the Rebuttal

Thank you for the additional experiments and analysis. I still have substantial concerns about the reproducibility of the evaluation and the novelty of the method.

On reproducibility, the Qwen3-8B results are useful, but the paper does not explain why the smaller model achieves a lower ASR in some settings. Both the 8B and 14B models are from the Qwen family, so it remains unclear whether GAP would provide similar gains on another model family. I would also find results for GAP across different model sizes and families more informative than the Qwen3.5 35B and 122B baseline results. Reducing that baseline table to three or four of the strongest closed models and using the space to examine GAP’s generality would address an important concern.

The new few-shot results also change how I read the main comparison. GAP’s TSR and ASR differences from DeepSeek-V4-Pro + few-shot are not statistically significant (Table 10). Its TSR advantage is concentrated in Hotel, a domain constructed by the authors with only ten test tasks. The paper does not examine why the difference is so much larger there, making it difficult to judge how much weight to place on the overall result. Appendix U says the full few-shot prompts are in the supplementary material, but I could not find the demonstrations in the supplementary/prompts files available to me. Table 18 also does not show which tasks are handled differently by GAP and the new few-shot baseline. The experiments show that the full GAP pipeline works, but they do not establish why training an internal gate is preferable to request-specific retrieval or an external gate applied under comparable training conditions.

On novelty, I remain unsure what methodological contribution goes beyond combining existing approaches to planning, distillation, and role-specific learning. My original question about the need for domain-specific training and learned self-gating has therefore not been resolved. The few-shot baseline uses fixed, manually written examples; it does not retrieve relevant plans or successful trajectories from the training data.

A smaller model with strong performance could be a practical benefit. However, the throughput experiment in Appendix H fixes the input and output lengths and excludes the cost of generating and maintaining GAP’s plans. Table 19 shows that GAP produces substantially more output tokens and takes much longer per dialogue than base Qwen, while providing no end-to-end cost comparison with DeepSeek. The current results therefore do not establish that GAP offers comparable performance more efficiently when run locally.

As a minor presentation issue, I cannot tell what Table 15 is intended to show. It appears that something went wrong with the LaTeX table formatting.

For these reasons, I am keeping my rating. I appreciate the additions. However, I do not think they resolve the central concerns. The advantage over the strongest few-shot baseline is not statistically established, much of the observed TSR difference comes from the small Hotel test set, and I cannot verify the construction of the few-shot prompts from the files available to me. It also remains unclear whether domain-specific training and an internal gate are needed relative to the alternatives above, or whether the method transfers beyond the Qwen family.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.