acceptodds
Under review as a conference paper at ICLR 2027

AdaGuard: An Adaptive Guard Model with User-defined Policies

Abstract

Guard models support the safe deployment of language model agents, but fixed risk taxonomies limit their ability to accommodate requirements that vary across applications and tasks. Under user-defined policies, detecting violations requires interpreting both the applicable rules and the agent's behavior, since identical actions can receive different judgments under different policies. To support learning this capability, we introduce AdaptiveSafety, a dataset of 10,939 training examples and 1,000 test examples covering policies with 1–100 rules. The dataset combines trajectories from multiple sources with policy and behavioral counterfactuals, pairing each example with an explanation and the complete set of violated rules. These counterfactuals expose changes that alter compliance, while structural augmentations provide supervision for consistency under rule reordering and identifier remapping. Building on this supervision, we propose SafePO, a reinforcement learning algorithm for refining violation identification while balancing explanatory reasoning and final verdicts. SafePO uses structured rewards to assess prediction correctness, retains group-relative advantages at the response level, and employs a separately trained value model to modulate token weights within explanation and verdict regions. Separate normalization controls their relative contribution to training despite differences in length. Through supervised initialization followed by SafePO, we develop AdaGuard, a family of 0.6B, 4B, and 8B guard models that assess agent trajectories under policies supplied at inference time. Our 4B model achieves binary accuracies of 89.30% on AdaptiveSafety and 71.82% on DynaBench. The anonymous project repository is available at https://anonymous.4open.science/r/AdaGuard-A733.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adaptive Safety Benchmark, Agent Guard

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jUe1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper targets an important problem. Namely, agents face changing risk taxonomies, and their actions need to be evaluated against user-supplied policies. The paper “AdaGuard: An Adaptive Guard Model with User-Defined Policies” makes three main contributions. First, it introduces the AdaptiveSafety dataset. It consists of 10,939 training and 1,000 test samples, which cover policies with 1–100 rules. The dataset is built from agent trajectories with policy and behavioral counterfactuals. Second, the paper introduces a GRPO-style objective called SafePO. Third, it develops AdaGuard, a family of guard models built using SFT followed by SafePO.

Critical strengths
  1. The dataset design targets an important problem, i.e., changing risk taxonomies. The dataset provides two kinds of counterfactuals. First, policy counterfactuals rely on changed rules and fixed behavior. Second, behavioral counterfactuals rely on fixed rules and changed actions or outcomes.
  2. The AdaGuard models developed in the paper perform strongly on the AdaptiveSafety dataset. In accuracy, they outperform competing models such as GPT-5.2 by 13 percentage points and DynaGuard-4B by 22 percentage points.
Critical weaknesses
  1. The paper lacks critical ablations and competing methods for SafePO. For example, the paper does not compare SafePO to GRPO under the same reward setting. Ablations for the value-guided modulation and region budgets are also missing.
  2. Table A4 compares SafePO against SFT using a different input-budget calculation and single runs which makes the gains against SFT hard to judge.
  3. On the DynaBench dataset (Table 1 and TableA1), DynaGuard-8b model and three API-based models (GPT-5.2, Gemini-3.1-Flash-Lite and Jev) outperform AdaGuard-8B. And AdaGuard-0.6B achieves only 51.38 percent accuracy.
  4. Section 3.1 does not specify whether the AdaptiveSafety dataset’s labels and test set were produced and validated by humans or an LLM. Moreover, the paper does not report agreement statistics.
Questions for authors

Please see the Critical Weaknesses section.

4 comments
AuthorsOpenReview ↗

Rebuttal 1

We thank the reviewer for recognizing the importance of adaptive risk taxonomies, policy and behavioral counterfactuals, and AdaGuard’s strong AdaptiveSafety performance. Following the review, we conducted matched 8B controls and a comprehensive audit of the annotation pipeline.

Q1. What do the matched controls establish about SafePO?

A1. We evaluated all four checkpoints with identical examples, prompts, parsing, greedy decoding, and input/output budgets. The RL conditions share the same SFT initialization, training data, structured reward, group size, optimization budget, KL coefficient, and checkpoint-selection rule. Uniform GRPO uses whole-response token weights. Fixed-region SafePO uses a 1:2 analysis-body to structure-and-verdict budget without value modulation. Full SafePO additionally enables within-region modulation.

8B conditionRegion weightingValue mod.AS Acc.AS Set-EMDB Acc.DB Set-EM
Matched SFTN/AN/A88.676.474.5967.22
GRPO, same rewarduniform tokensno88.976.775.3268.14
SafePO, fixed regions1:2no89.277.076.4370.53
Full SafePO1:2yes89.577.176.8070.72

Uniform GRPO improves over matched SFT by 0.3 points on both AS metrics and by 0.73 and 0.92 points on DB accuracy and Set-EM. Fixed regional weighting adds 0.3, 0.3, 1.11, and 2.39 points. Learned modulation adds a further 0.3, 0.1, 0.37, and 0.19 points. The uniform-to-fixed comparison isolates regional budgeting, while the fixed-to-full comparison isolates value modulation. The value model is used only during training, so all variants retain the same inference architecture and cost. We will replace the historical comparison in Appendix A.4 with this matched experiment and add the exact seed and training-cost records.

Q2. How should the DynaBench results be interpreted?

A2. DynaGuard-8B reaches 80.11% binary accuracy and 77.31% violation-positive F1, compared with 76.80% and 76.05% for AdaGuard-8B. The three API systems obtain 80.85–83.61% accuracy. AdaGuard-8B has higher recall than DynaGuard-8B, 74.91% versus 68.91%, and additionally predicts the complete policy-local violation set with 70.72% Set-EM, whereas DynaGuard provides a binary interface.

The model-size trend is clear: DynaBench accuracy increases from 51.38% at 0.6B to 71.82% at 4B and 76.80% at 8B. Our revised conclusion is that AdaGuard-8B supports complete rule attribution beyond AdaptiveSafety and improves over its matched SFT initialization, while the strongest DynaBench systems retain higher aggregate binary accuracy.

Q3. Were AdaptiveSafety labels produced by humans or LLMs, and how were they checked?

A3. Both train and test reference labels were produced by an LLM-based annotation pipeline. The primary annotator received only the current policy and canonicalized interaction, without the inherited label, transformation name, or intended counterfactual direction. A different model then produced its assessment and conducted proposal-aware verification for omitted or unsupported rules, policy-local validity, analysis faithfulness, and evidence grounding.

Annotation auditTrainTestTotal
Retained examples10,9391,00011,939
Direct cross-model exact-set acceptance9,63184110,472
Accepted after adjudication1,3081591,467
Validated evidence pointers15,8941,45617,350
Same primary and verifier model000

Direct cross-model exact-set acceptance is 87.71% overall and 84.10% on the test set. Every retained positive rule is linked to supporting interaction content; all 17,350 pointers passed address validation, quoted-text occurrence, and positive-label coverage checks. We report this cross-model acceptance rate as the agreement statistic and explicitly identify the annotations as LLM-generated.

The split is lineage-disjoint. A base interaction and all associated policy, behavioral, and identifier variants remain in one split. Train-test overlap is zero for base identity, interaction-content, candidate, policy-content-pair, contrast-group, and reference-analysis hashes. All six data collections and nine construction categories are represented in both splits.

Thank you again for the careful review. We believe the matched controls, calibrated DynaBench interpretation, and explicit annotation audit directly address the principal concerns. We welcome further questions and would be grateful if the reviewer would reconsider the score in light of this evidence.

We greatly appreciate your time and constructive feedback. Should you have any questions regarding our rebuttals, please feel free to contact us, we would be more than happy to discuss them with you and make further revisions as needed.

Thank you for the detailed response and the additional experiments. Overall, the responses address several of my concerns and provide useful additional evidence. In particular, the matched 8B controls and the description of the annotation pipeline are helpful. However, regional budgeting and value modulation each add only 0.1–0.3 points on AdaptiveSafety, and value modulation adds at most 0.37 points on DynaBench. Since no variance across seeds is reported, these gains cannot be separated from run-to-run variance. All AdaptiveSafety labels, including those in the test set, are LLM-generated. The reported agreement is between two LLMs, and no human validation is reported. I appreciate the authors’ efforts and will maintain my original rating.

Thank you for your thoughtful response and constructive suggestions. We continue to believe AdaGuard offers a useful contribution to agent safety through AdaptiveSafety, a dataset centered on user-defined policies, and guard models that assess agent behavior against those policies. We understand your remaining concerns and respect your decision, although we regret that the additional evidence did not change your assessment. Thank you again for your time and careful consideration of our work.

Reviewer_nnqgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to assess recorded agent behavior under user-defined policies. It introduces AdaptiveSafety, with 10,939 training and 1,000 test examples containing explanations and complete violation sets. It also proposes SafePO, which combines structured rewards with value-guided token weighting and separate budgets for explanation and verdict regions. The resulting models are finally evaluated on benchmarks AdaptiveSafety and DynaBench.

Critical strengths
  1. The proposed task is well-motivated and rigorously formulated. Specifically, Sections 2.1 and 2.2 establish clear distinctions between malicious user requests and actual agent violations, as well as between attempted actions and completed operations. Furthermore, fine-grained identification of all violated rules offers substantially richer diagnostic utility than a coarse-grained binary judgment.
  2. AdaGuard-8B demonstrates strong empirical performance on AdaptiveSafety, achieving 89.50% accuracy, 77.10% exact match, and 74.05% rule micro-F1 (Tables 1 and 2). Notably, the authors maintain transparency regarding the method's limitations by explicitly acknowledging inter-evaluator discrepancies and inconsistent performance gains across different training stages.
Critical weaknesses
  1. The SafePO's contribution remains unclear. Appendix A.4 compares historical SFT results with SafePO using different input-budget calculations, and component ablations are missing. The matched comparisons with SFT and GRPO, together with ablations of region weighting and value-guided modulation, are needed.
  2. AdaptiveSafety's annotation reliability is hard to assess. Section 3.1 provides limited information about data sources, annotation, adjudication, and verification of complete violation sets. These labels determine both evaluation scores and training rewards, yet the public release contains only four synthetic schema examples without benchmark data.
Questions for authors

Q1: Can the authors provide matched baselines and ablations to isolate SafePO's contribution? Q2: How were complete violation sets annotated and verified? Is there any evidence can support the reliability of the violation sets?

3 comments
AuthorsOpenReview ↗

Rebuttal Part1

We thank the reviewer for recognizing the practical importance of policy-conditioned trajectory assessment, complete rule attribution, and our counterfactual construction. Following the review, we conducted matched 8B controls and a comprehensive audit of AdaptiveSafety.

Q1. What does SafePO contribute under a matched pipeline?

A1. We evaluated all four checkpoints with identical examples, prompts, parsing, greedy decoding, and input/output budgets. The RL conditions share the same SFT initialization, training data, structured reward, group size, optimization budget, KL coefficient, and checkpoint-selection rule. Uniform GRPO uses whole-response token weights. Fixed-region SafePO assigns one-third of the weight mass to the analysis body and two-thirds to structure-and-verdict tokens, with value modulation disabled. Full SafePO additionally enables within-region modulation.

8B conditionRegion weightingValue mod.AS Acc.AS Set-EMDB Acc.DB Set-EM
Matched SFTN/AN/A88.676.474.5967.22
GRPO, same rewarduniform tokensno88.976.775.3268.14
SafePO, fixed regions1:2no89.277.076.4370.53
Full SafePO1:2yes89.577.176.8070.72

Relative to uniform GRPO, fixed regional weighting improves AS accuracy and Set-EM by 0.3 points each and DB accuracy and Set-EM by 1.11 and 2.39 points. Learned modulation adds a further 0.3, 0.1, 0.37, and 0.19 points. These contrasts separately measure the observed effects of regional budgeting and value modulation.

SafePO retains response-level group advantages and uses prefix values only to redistribute weight within fixed regional masses. This differs from TEMPO’s tree-structured credit assignment and CSCR’s counterfactual-sensitivity reallocation. The value model is training-only, so inference retains the actor’s parameter count and latency. We will add the exact run seeds, GPU-hours, and peak-memory measurements to the revised appendix.

Q2. How were complete violation sets produced and audited, and what does the split hold out?

A2. Both train and test labels are LLM-generated. The primary annotator received the current policy and canonicalized interaction without the inherited label, transformation name, or intended counterfactual direction. A different model then produced its assessment and performed proposal-aware verification of the proposed labels, analysis, and evidence. Direct acceptance required exact-set agreement, sufficient confidence, and quality scores of at least 9/10.

Audit itemTrain / Test
Examples10,939 / 1,000
Collections A–F3,300/300; 3,097/206; 1,348/135; 1,120/130; 1,092/129; 982/100
Construction categories C1–C9956/82; 310/33; 982/100; 3,271/295; 1,065/105; 292/9; 1,003/104; 2,950/261; 110/11
Direct exact-set acceptance9,631 / 841
Corrective adjudication1,308 / 159
Evidence pointers15,894 / 1,456
Overlap across six audited lineage keys0

Direct cross-model acceptance is 87.71% overall and 84.10% on the test set. All 17,350 evidence pointers passed address, quoted-text-occurrence, and positive-label-coverage checks. All variants derived from one base interaction remain in one split. All six collections and nine construction categories occur in both splits; the evaluation therefore holds out interaction lineages rather than complete source or construction families. We will add the annotation prompts, adjudication criteria, per-example audit fields, and split manifests to the revision and release.

AuthorsOpenReview ↗

Rebuttal Part2

Q3. How will counterfactual sensitivity and identifier consistency be evaluated directly?

A3. We have replaced the earlier group-level interpretation with a pair-resolved protocol. A pair is retained only when both endpoints share a verified base lineage, differ by the designated intervention, and satisfy an auditable gold transformation. Policy and behavioral pairs are scored by joint endpoint correctness and set-transition accuracy. Identifier variants are scored after applying the verified inverse mapping, with mapped consistency reported separately from joint Set-EM. Pair eligibility is fixed before examining model predictions, and every result is reported with its exact numerator and denominator.

The current audit identifies 13 clean rule-order and identifier-remapping pairs. The policy-deletion, semantic-policy, and behavioral records are candidate transformations rather than pair denominators. We will release the frozen pair manifest and evaluate matched SFT, fixed-region SafePO, and full SafePO on precisely the same pairs. Figure 4 will remain a policy-length analysis rather than evidence for paired adaptation.

Thank you again for the constructive review. We believe the matched controls, annotation audit, and pair-resolved evaluation directly address the central concerns. We welcome further questions and would be grateful if the reviewer would reconsider the score in light of these additions.

We greatly appreciate your time and constructive feedback. Should you have any questions regarding our rebuttals, please feel free to contact us, we would be more than happy to discuss them with you and make further revisions as needed.

Reviewer_HQxtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper study guards that assess an agent trajectory against a policy supplied at inference time and identify all violated rules. It introduces a framework adaptiveSafety, with policy and behavioral counterfactuals, and trains AdaGuard through supervised fine-tuning followed by SafePO. SafePO combines structured verdict rewards with separate weight budgets for explanation and verdict tokens, then redistributes those weights using a learned value model. Results are generally good.

Critical strengths
  1. The counterfactual construction is interesting. Keeping a trajectory fixed while changing its governing policy directly tests something a fixed harmful-content taxonomy cannot represent. Behavioral counterfactuals and rule-identifier permutations are also sensible ways to discourage shortcuts. Predicting the complete violation set makes the task more useful than binary screening alone.

  2. The adaGuard is competitive at a practical model size. The 4B model reaches 89.3% binary acc. and 76.6% complete-set acc. on AdaptiveSafety. Table A5 shows that the result is not confined to user-only requests: trajectory accuracy is 89.01%.

Critical weaknesses
  1. I cannot tell whether SafePO is needed. The appendix A.4 explicitly states that there are no component ablations and that the historical SFT checkpoint was evaluated with different preprocessing. Even that comparison is mixed: for 4B, AdaptiveSafety acc. deteroriates from 89.90% to 89.30%, and DynaBench F1 goes down as well. The 8B improvements are encouraging, but single runs without a matched SFT evaluation do not isolate the structured reward, region balancing, or learned value modulation. Since SafePO is a central contribution and adds a separate trained value model, a comparison with the same reward and fixed region weights is particularly important.

  2. The benchmark needs a more auditable account of its labels and generalization split. SEc. 3.1 explains the transformations and keeps related variants together, but did not give source-wise counts, the annotation procedure, or an audit of whether all violated rules were recovered. IT's also not clear which policy families and source trajectories were held out. This matters because most of the performance advantage is on the newly constructed dataset.

Questions for authors

Can the SFT and SafePO checkpoints be evaluated with the identical current pipeline, alongside SafePO with value modulation disabled but the same structured reward and region weights? How were complete violation sets annotated and checked?

2 comments
AuthorsOpenReview ↗

Rebuttal 1

We thank the reviewer for the constructive assessment and for recognizing the value of our counterfactual construction, complete-set formulation, and practical model performance. We agree that SafePO attribution and AdaptiveSafety auditability required stronger evidence. We therefore conducted the requested matched experiment and a comprehensive data audit.


Q1. Is SafePO necessary, and what is the contribution of learned value modulation?

A1. Following the review, we reran the AdaGuard-8B SFT checkpoint and trained a matched fixed-weight SafePO control under the current evaluation pipeline. All three conditions use identical evaluation data, prompts, parsing, greedy decoding, and input/output budgets. The fixed-weight control retains the structured verdict reward, group-relative optimization, and analysis/verdict masses of 1/3 and 2/3, while removing the value model and within-region modulation.

SizeConditionValue mod.AS Acc.AS Set-EMDB Acc.DB Macro-F1DB Set-EM
8BSFT, matched evaluationN/A88.676.474.5974.3667.22
8BSafePO, fixed weightsno89.277.076.4376.2270.53
8BFull SafePOyes89.577.176.8076.7770.72

Fixed-weight SafePO improves over matched SFT by 0.6 points on both AS accuracy and Set-EM, and by 1.84, 1.86, and 3.31 points on DB accuracy, Macro-F1, and Set-EM. Full SafePO improves over matched SFT by 0.9, 0.7, 2.21, 2.41, and 3.50 points on the same metrics. Relative to fixed weights, learned value modulation provides further gains of 0.3, 0.1, 0.37, 0.55, and 0.19 points, respectively. Thus, most of the observed improvement is already achieved by the structured fixed-weight RL control, while the full-versus-fixed comparison shows an additional, consistent benefit from value modulation.


Q2. How were complete violation sets annotated and checked?

A2. We constructed AdaptiveSafety through a unified trajectory-processing and counterfactual-construction pipeline. Each final policy-trajectory pair was labeled against the complete policy and interaction. A different verifier model first re-assessed each case and then reviewed the proposed set for omitted or unsupported rules, policy-local validity, analysis faithfulness, and evidence grounding. Disagreements underwent corrective adjudication, and unresolved cases were excluded.

Audit itemTrainTestTotal
Retained examples10,9391,00011,939
Data collections represented66all six in both
Audited operational categories99all nine in both
Distinct base identities5,4995356,034
Distinct interaction-content hashes6,1545996,753
Direct cross-model verification9,63184110,472
Accepted after adjudication1,3081591,467
Validated evidence pointers15,8941,45617,350

For auditing, we additionally partition the data into nine operational categories; these are bookkeeping categories rather than additional high-level augmentation mechanisms. Every retained positive rule is linked to supporting trajectory content. Automated checks validate the pointer address, quoted-text occurrence, and positive-label coverage; all 17,350 pointers passed. The primary annotator and verifier were never the same model. We will add this procedure and the group-wise statistics to the appendix.


Q3. What does the AdaptiveSafety split hold out?

A3. The split holds out interaction lineages rather than entire data families. Each base interaction and all associated policy, behavioral, and identifier variants are assigned to one split. We found zero train-test overlap in base IDs, normalized interaction-content hashes, policy-content-pair hashes, and contrast-group IDs. Candidate IDs and normalized reference analyses were also disjoint.

All six data collections and all nine audited operational categories are represented in both splits. The test set therefore evaluates held-out interaction lineages drawn from the represented data and construction categories. We will release the split manifests containing lineage IDs and content/policy hashes, together with file-level SHA-256 digests, so these checks are reproducible.

Thank you again for identifying these issues. We believe the matched ablation, complete-set annotation audit, and clarified split protocol directly address the reviewer’s concerns. We welcome further questions and would be grateful if the reviewer would reconsider the score in light of this new evidence.

We greatly appreciate your time and constructive feedback. Should you have any questions regarding our rebuttals, please feel free to contact us, we would be more than happy to discuss them with you and make further revisions as needed.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

Dear Reviewers,

We sincerely thank all reviewers for their constructive feedback and recognition of AdaGuard's contributions. We conducted additional experiments and analyses to address the concerns regarding SafePO, benchmark reliability, and adaptive policy evaluation.


Reviewer HQxt

  • Added matched 8B evaluations comparing SFT, fixed-weight SafePO, and full SafePO under identical evaluation settings. Full SafePO achieves 89.5% AdaptiveSafety accuracy and 76.80% DynaBench accuracy.
  • Isolated the contribution of learned value modulation, showing additional gains over fixed-weight SafePO across all evaluated metrics.
  • Expanded the AdaptiveSafety annotation audit, including cross-model verification, corrective adjudication, and 17,350 validated evidence pointers.
  • Clarified the train-test split protocol, ensuring that all variants of each base interaction remain within the same split.

Reviewer nnqg

  • Added controlled comparisons among uniform GRPO, fixed-region SafePO, and full SafePO, demonstrating the respective contributions of regional weighting and learned value modulation.
  • Clarified SafePO's distinction from TEMPO and CSCR in its response-level group advantages and within-region value modulation.
  • Provided source-wise and construction-category statistics, annotation procedures, verification results, and lineage-level split checks.
  • Introduced a pair-resolved evaluation protocol for policy and behavioral counterfactuals, as well as rule reordering and identifier remapping. Identified 13 clean structural pairs and committed to releasing the pair manifest and matched evaluation results.

Reviewer jUe1

  • Added matched SafePO ablations under identical initialization, training, and evaluation settings.
  • Clarified DynaBench results, acknowledging that DynaGuard-8B and API-based models achieve higher binary accuracy, while AdaGuard provides higher recall and complete violation-set prediction.
  • Explicitly documented the LLM-based annotation pipeline, reporting 87.71% overall and 84.10% test-set cross-model exact-set acceptance.
  • Established lineage-disjoint train-test splits with zero overlap across six audited lineage keys.

Across all responses, we have:

  • Strengthened SafePO validation through matched optimization controls and component ablations.
  • Improved AdaptiveSafety transparency through detailed annotation audits and source-wise statistics.
  • Clarified dataset generalization boundaries and introduced pair-resolved counterfactual evaluation.
  • Committed to releasing annotation guidelines, split manifests, evaluation artifacts, and computational records.

We believe these additional experiments and clarifications substantially strengthen the empirical rigor, transparency, and practical relevance of AdaGuard. We sincerely appreciate the reviewers' valuable feedback and hope our responses adequately address their concerns.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.