acceptodds
Under review as a conference paper at ICLR 2027

AdaGuard: Enhancing Safety and Policy Compliance with Adaptive and Dynamic Reasoning-Enabled LLM-as-a-Judge Guardrails

Abstract

Enterprise generative AI applications require robust safety mechanisms that can accommodate diverse risk postures, evolving policies, and varying latency constraints. Current guardrail solutions often suffer from rigidity, relying on fixed policy sets and offering limited transparency or reasoning flexibility. We present AdaGuard, an adaptive LLM-as-a-Judge framework designed to address these challenges through dynamic policy enforcement and adaptive reasoning-budget allocation. Built using supervised fine-tuning (SFT) and reinforcement learning (GRPO), AdaGuard generalizes to user-defined safety and compliance policies at runtime without requiring frequent model updates. A core innovation of our approach is the ability to dynamically infer the complexity of input-policy pairs, allowing the model to switch between high-speed black-box inference and explainable, reasoning-enabled moderation. This flexibility enables developers to balance stringent latency requirements with the need for actionable transparency. This adaptive capability allows AdaGuard to rival other guardrail and frontier models several times its size, while its auto-reasoning mode recovers the accuracy of always-on reasoning at a fraction of the latency.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Reinforcement Learning, LLM-as-judge Guardrail, Generative AI Input Moderarion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_A5BdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper investigates the problem that providing a justification for an LLM answer is costly andn ot always required. To avoid generating unnecessary justifications, it proposes an LLM-based guardrail that checks user input against a list of compliance policies and returns a verdict for each policy. Only if the guardrail finds a potential risk is the reasoning activated.

Critical strengths

S1: The problem is relevant. Reducing the cost of reasoning in cases where it is not needed is practical. S2: The results seem to be competitive.

Critical weaknesses

W1: The novelty with regard to SingGuard seems to lie in the per-policy gate. I find this rather thin. W2: From my understanding, the decision whether to reason or not is dependent on the LLM's assessment of difficulty. This seems fairly dangerous, as this would probably allow adversarial examples or jailbreaks to go through without any reasoning.

Questions for authors
  • What is your assessment of the risk regarding jailbreaks?
8 comments

We thank the reviewer for their helpful comments and feedback, and we are glad that they find the problem relevant and practical, especially in deployment settings where it is desirable to reduce the overall latency costs (including from reasoning where it is not needed) while maintaining the performance of the guardrail, and that our results are competitive. We address both weaknesses and the question with new experiments on both backbones (Gemma4-E4B / Nemotron-Nano-4B) over the full held-out test set, and on an external benchmark of human-written business rules (DynaBench). Every number on our test set now carries a 95% confidence interval from a cluster bootstrap over test inputs, and comparisons are paired for statistical significance.

For simplicity, we will refer to AdaGuard’s autoreasoning mode as simply AdaGuard throughout the following sections.

High-Level Summary of Our Response

  1. Novelty over SingGuard: The per-policy gate in AdaGuard is one of three differences that sets it apart from SingGuard. The other two are that, in AdaGuard, the decision to reason is based on output emitted by the model, made after a shared analysis of the input, rather than a confidence threshold, and that our counterfactual reward (CF-Corr) is what teaches the model when to reason. At a similar reasoning budget, AdaGuard's choice of policies beats SingGuard's routing, reimplemented on our models, by +2.4 / +6.6 per-policy F1 (p<0.001p < 0.001), and the gap grows with the number of policies per input. Without the counterfactual reward, the model simply loses this selective reasoning ability.
  2. Jailbreak risk: We stress that skipping reasoning never skips moderation, as AdaGuard still gives a verdict for every policy. On the WildJailbreak benchmark, AdaGuard scores 92.0 F1 in autoreasoning, very close to reasoning-on (92.3) and second among all other guardrails/models only to WildGuard (95.3), whose training data was generated with the same attack method as WildJailbreak. Jailbreaks are part of AdaGuard's training, and it reasons on hard policies far more often than on easy ones.
  3. Evaluation on DynaBench: To benchmark AdaGuard on an external dynamic-policy dataset, we evaluated it on DynaBench, comprising human-written business rules targeting an output moderation model. This is done as a zero-shot generalization test since it is a task AdaGuard was not trained for. AdaGuard scores 68.4 F1 in autoreasoning while reasoning on only a fifth of the rules against 68.7 in reasoning-on, while DynaGuard 8B (with CoT) which was trained on DynaBench’s training split, scores 73.1. With DynaBench and WildJailbreak added to the updated Table 1, AdaGuard in autoreasoning has the highest average across all six static benchmarks (91.0), and over all eight benchmarks (87.8).
  4. Presentation: We would like to fix specifically what made the paper hard to follow, ask which passages those were, and list three changes we will make in any case, based on all the feedback we received across all reviewers.

We hope these answers address the reviewer's concerns, and if they do, that the reviewer will consider raising the score. We are also happy to run any further comparison the reviewer would find useful, and look forward to any feedback.

A) Novelty with Regard to SingGuard

We have experimented with additional baselines, including SingGuard, and we will add all results to the updated manuscript. There are three main differences between AdaGuard and SingGuard:

  1. SingGuard's hybrid mode forces an entire input to slow reasoning on all of its policies, while AdaGuard decides per policy (more granular) within a single response, so an input that is ambiguous under one policy does not pay for reasoning on the others.
  2. SingGuard uses its binary safe/unsafe decode-time token confidence against a threshold. In AdaGuard, the decision to reason is a trained, generated output, made after the model has written a shared input-understanding passage. It needs no access to token probabilities, no threshold and no second call in reasonig-on mode.
  3. SingGuard's RL reward is outcome-only and does not credit when to reason. However, AdaGuard's counterfactual reward (CF-Corr) samples a reasoning-off verdict for each reasoned block and adjusts the reward based on whether reasoning fixed, maintained or broke the verdict. Without it, the learned reasoning rate rises to almost 100% (Figure 3). We will add this comparison to the updated PDF and list CF-Corr as an explicit contribution.

Experimental Results

To recap our results:

  • At matched reasoning budget, AdaGuard beats SingGuard's routing by +2.4 / +6.6 per-policy F1 (80.10 vs. 77.68, 64.52 vs. 57.90, p<0.001p < 0.001).
    • On hard policies the gain is +4.8 / +7.9.
    • Per-example, the two are tied on Gemma (88.18 vs. 88.26), which is expected from a per-input gate. AdaGuard leads by +4.2 on Nemotron.
  • The gap grows with the number of policies per input: from +0.0 / +2.1 per-policy F1 on inputs with 1–3 policies to +4.0 / +8.5 on inputs with 9 or more.
  • Across a sweep of SingGuard's threshold τ\tau, it approaches AdaGuard only when it reasons on most or all policies (Gemma 79.80 at 100%, Nemotron 63.46 at 76% and 64.22 at ≈100%, vs. AdaGuard's 80.10 / 64.52 at 32% / 36%), i.e. as it approaches always-on reasoning.
  • At AdaGuard's measured output length instead of its policy count, AdaGuard still leads by +1.9 / +3.1 per-policy F1.

Due to restrictions on running some third-party model weights in our environment, we could not evaluate SingGuard's own checkpoints. Instead, we reimplemented its hybrid routing rule (its Eq. 7–8) on the AdaGuard backbone. We have the same model's verdict with reasoning-off and reasoning-on for every test input–policy pair. Each selection rule in Table A reasons on as many policies as AdaGuard. The chosen pairs take the reasoning-on verdict and the rest reasoning-off’s, so only the choice of policies differs. SingGuard first decodes one safe/unsafe token y0y_0 for the whole input and computes its normalised confidence s(y0)=p(y0)/(p(safe)+p(unsafe))s(y_0) = p(y_0) / (p(\text{safe}) + p(\text{unsafe})). If this confidence reaches a threshold (s≥τs \ge \tau), it stops and emits y0y_0, otherwise it reasons over all policies before giving the final verdict. We reproduce this as follows:

  1. For each policy kk, we take pk=P(Violation)p_k = P(\text{Violation}) from its verdict token in reasoning-off, normalized over Violation/Compliance. Since an input is safe only if it violates none of the matched policies, p(safe)=∏k(1−pk)p(\text{safe}) = \prod_k (1 - p_k) and p(unsafe)=1−p(safe)p(\text{unsafe}) = 1 - p(\text{safe}), so y0y_0 is the more likely label and s=max⁡(p(safe),p(unsafe))s = \max(p(\text{safe}), p(\text{unsafe})), which is Eq. 7 for a binary first decision. We also experimented with a variant that does not depend on the number of policies (ss chosen from the most confident violation, or else the least confident compliance), which leads to the same conclusions.
  2. Inputs with s<τs < \tau take reasoning-on’s verdicts for all of their policies (SingGuard's slow mode), while the other inputs keep their reasoning-off’s.
  3. We then rank inputs by ss, least confident first, and escalate them to reasoning in that order until the number of reasoned policies reaches AdaGuard's (26,407 on Gemma, 29,863 on Nemotron). This becomes equivalent to τ=1−10−6.7\tau = 1 - 10^{-6.7} (Gemma) / 1−10−5.11 - 10^{-5.1} (Nemotron).
  4. Escalated verdicts come from an independent reasoning-on pass not conditioned on y0y_0. Therefore, they cannot anchor on the first guess. If anything, this setup favors SingGuard's implementation.

Table A: Matched reasoning budget (per-policy/per-example F1)

Selection ruleGemma: reasonedGemma: per-policy / per-example F1Nemotron: reasonedNemotron: per-policy / per-example F1
reasoning-off0%74.50±0.69 / 86.57±0.620%42.23±0.99 / 58.89±1.10
reasoning-on100%79.80±0.75 / 88.24±0.60100%64.24±0.90 / 76.62±0.80
SingGuard routing32.5%77.68±0.64 / 88.26±0.5736.5%57.90±0.98 / 72.42±0.93
AdaGuard31.8%80.10±0.64 / 88.18±0.5936.3%64.52±0.93 / 76.66±0.86

We can see that at the same budget, AdaGuard beats SingGuard's routing by +2.4 (Gemma) / +6.6 (Nemotron) per-policy F1 (p<0.001p < 0.001), and it is on par with reasoning on every policy. SingGuard decides per input and reasons on every policy of an uncertain input, and a policy that is uncertain inside a confidently unsafe input is consequently skipped, whereas AdaGuard decides per policy.

We also show that no threshold closes the gap (Table B). Lowering SingGuard's confidence threshold makes it reason on more inputs, but its per-policy F1 rises slowly. On the other hand, τ\tau ends up sending (almost) every input to reasoning if tuned for accuracy (100% of policies on Gemma and 99.9% on Nemotron, for 79.80 / 64.22 per-policy F1). AdaGuard gets the same accuracy while reasoning on a third of the policies.

Table B: SingGuard's routing threshold sweeps. The table shows share of pairs reasoned · per-policy / per-example F1. An input is reasoned when its confidence is smaller than τ\tau.

τ\tauGemma: SingGuard routingNemotron: SingGuard routing
0.93.4% · 75.01 / 86.048.2% · 46.95 / 66.93
0.997.0% · 75.48 / 86.5516.3% · 50.93 / 68.88
0.99911.5% · 76.01 / 87.1123.1% · 53.74 / 70.30
1−10−41 - 10^{-4}16.4% · 76.48 / 87.4929.2% · 55.79 / 71.23
1−10−61 - 10^{-6}27.7% · 77.29 / 88.0843.3% · 59.91 / 73.32
1−10−81 - 10^{-8}42.0% · 78.03 / 88.0476.4% · 63.46 / 76.66
1−10−101 - 10^{-10}59.4% · 78.21 / 87.9198.5% · 64.09 / 76.99
1−10−121 - 10^{-12}79.9% · 78.41 / 87.8699.3% · 64.10 / 76.96
τ\tau tuned for per-policy F1 (2-fold)100.0% · 79.80 / 87.7399.9% · 64.22 / 76.97
AdaGuard (no threshold)31.8% · 80.10 / 88.1836.3% · 64.52 / 76.66

Also, the gain is concentrated on hard policies, as expected (Table C). AdaGuard reasons on 55% (Gemma) / 63% (Nemotron) of the policies the teacher-assigned hard labels and only 22% / 25% of the easy ones. On hard policies it beats SingGuard's routing by +4.8 / +7.9 per-policy F1, and it matches reasoning-on every policy on Nemotron.

Table C: per-policy F1 on teacher-labelled easy and hard policies

Gemma: easyGemma: hardNemotron: easyNemotron: hard
reasoning-off87.65 ± 0.6252.23 ± 1.3455.31 ± 1.2519.65 ± 1.27
reasoning-on88.60 ± 0.8265.38 ± 1.2476.70 ± 0.9844.39 ± 1.46
SingGuard routing89.27 ± 0.5958.70 ± 1.2670.63 ± 1.0936.91 ± 1.51
AdaGuard90.02 ± 0.5663.50 ± 1.2476.95 ± 1.0044.81 ± 1.43
AdaGuard: share of these policies reasoned on21.6%55.4%24.8%63.0%

Finally, the advantage of per-policy decisions grows with the number of policies per input, exactly where granularity should matter (Table D).

Table D: By number of policies per input (per-policy F1)

Policies per input1–34–89+
Gemma4-E4B: AdaGuard per-policy F181.4 ± 1.780.1 ± 0.979.7 ± 1.1
Share of policies AdaGuard reasons on48.7%38.5%24.5%
AdaGuard − SingGuard routing+0.0 ± 1.4+1.7 ± 0.8+4.0 ± 1.0
AdaGuard − reasoning-on+0.4 ± 1.4−0.3 ± 0.7+1.0 ± 1.3
Nemotron-Nano-4B: AdaGuard per-policy F168.4 ± 2.065.3 ± 1.362.4 ± 1.5
Share of policies AdaGuard reasons on50.5%43.8%28.9%
AdaGuard − SingGuard routing+2.1 ± 1.8+6.4 ± 1.2+8.5 ± 1.6
AdaGuard − reasoning-on+0.1 ± 1.6+0.7 ± 1.0−0.2 ± 1.3

On inputs with 9+ policies, AdaGuard reasons on only a quarter of them and stays close to reasoning-on on all of them, while SingGuard's routing falls 4.0 / 8.5 points behind.

Other Selection Baselines: Our response to Reviewer 6wrg (Section B) compares further rules at the same budget: random selection, random selection forced inside AdaGuard's own generation, a per-policy confidence gate on the verdict-token probabilities, AdaGuard's own difficulty token used as a confidence gate, and baselines matched on output length rather than on the number of policies reasoned on. AdaGuard matches or beats all of them. The only rule that matches it is the confidence gate. However, its drawback is that it needs token probabilities from an extra reasoning-off pass, and an extreme threshold, found by sweeping on the test set, to spend AdaGuard's budget.

To conclude, we also establish that the reward is what makes this selection possible. We extended the reward ablation of Figure 3 from epoch 1 to epochs 2 and 3 (Table E).

Table E: Reward ablation across epochs. Gemma4-E4B, held-out dynamic-policy set; per-policy F1 / share of policies reasoned on.

Reward variantEpoch 1 (step 80)Epoch 2 (step 144)Epoch 3 (step 208)
CF & Quality (AdaGuard's reward)80.0 ± 0.6 / 37%80.3 ± 0.7 / 38%80.8 ± 0.6 / 39%
CF Only (no quality judge)78.1 ± 0.8 / 25%80.4 ± 0.6 / 34%81.0 ± 0.6 / 36%
Quality Only (no counterfactual term)80.4 ± 0.7 / 99.6%80.0 ± 0.7 / 99.6%80.3 ± 0.8 / 99.9%
Vanilla (neither)80.0 ± 0.7 / 99.4%80.6 ± 0.7 / 99.5%80.9 ± 0.7 / 99.8%
Reviewer_j9eoOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents AdaGuard, a compact guardrail model that checks a user input against a set of custom policies written in plain language, and returns a compliance or violation verdict for each policy. The model is trained with SFT followed by RL on synthetic policies derived from public safety datasets, and it supports three modes: no reasoning, reasoning on every policy, and an adaptive mode in which the model predicts how difficult each policy is and writes out its reasoning only for the hard ones, guided by a reward that credits reasoning only when it changes the verdict. On standard safety benchmarks and a held-out set of synthetic policies, the authors report accuracy competitive with larger guardrail and general-purpose models.

Critical strengths

(1). The counterfactual reward is well-designed. CF-Corr scores each reasoned block against an independent no-reasoning sample. This rewards reasoning only when it actually changes the verdict, instead of relying on the model’s self-reported confidence. The results in the paper support this well. The authors also address reward hacking, and their proposed mechanism could transfer to other adaptive-compute settings.

(2). The deployment design is practical and well-documented. And the appendices seem quite complete, which makes the work reproducible.

Critical weaknesses

Overall the paper has some basic weaknesses. The problem addressed in the paper is important, but the evidence that reasoning matters for this task is thin, and the evidence for runtime-policy generalization rests on a synthetic and teacher-labeled test set instead of existing policy benchmarks. The introduction motivates agentic systems and prompt injection but the evaluation covers only single-turn input moderation: no output moderation, no injection benchmark, and no malformed policies.

The policies the paper evaluates are key, but the paper only creates some synthetic policies and fails to explore existing ones. The “dynamic policies” are LLM paraphrases of harm-category labels from existing safety datasets -- they are not the enterprise rules shown in Figure 1 and the introduction. Also, the authors compare against DynaGuard but never evaluate on its benchmark, DynaBench, which has a test set that allows evaluating compliance with policies that go beyond traditional safety domains.

Some experimental results could be strengthened. The reward is fragile: there are about 10 hand-set reward constants, no sensitivity analysis, and the ablation is run at epoch 1 only. Reasoning breaks verdicts almost as often as it fixes them: in Figure 2, the rates are 23% vs 17% (Gemma) and 20% vs 18% (Nemotron). So many traces justify wrong verdicts. Also. AdaGuard’s traces are never evaluated, since only teacher traces get any human or quality check at test time. Finally the quality judge and data evaluator are the same model, so the reward could favor matching the labeler’s style.

Questions for authors

Which results are statistically significant? I did not see any variance reported, nor any seeds or confidence intervals. Many of the gaps are less than a couple points.

Example-level F1 does not measure per-policy compliance since it deems an example a violation if any poilcy is violated. How do the results look when broken down based on individual policiies?

10 comments

We appreciate the reviewer’s detailed feedback that helped us further bolster our claims and results. We find it encouraging that the counterfactual reward and the deployment design stood out as strengths. To address each comment, we ran new analyses and added 95% cluster-bootstrap confidence intervals over test inputs and paired significance tests.

For simplicity, we will refer to AdaGuard’s autoreasoning mode as simply AdaGuard throughout the following sections.

High-Level Summary of Our Response

  • Reasoning matters where it should: On hard policies, reasoning raises per-policy F1 from 52.2 to 63.5 (Gemma) and from 19.7 to 44.8 (Nemotron), and the gain grows with the number of policies per input. Choosing which policies to reason on adds +4.2 / +14.7 F1 beyond a shared input-understanding passage, while random choices at the same budget add +0.9 / +6.2.
  • The dynamic test policies are full rules, not category labels: We provide more analytics on our dynamic test policies. In short, 67% contain conditions or exceptions, rather than mere harm category rephrases and a third target compliance and regulation (privacy, professional advice, fraud, misinformation) rather than classic harms. In this last category, AdaGuard is on par with reasoning-on, and exceeds reasoning-off or random selection.
  • We outline the scope of our work more crisply: The reviewer is right that our introduction might read beyond our intended scope. AdaGuard is an input moderator to enhance robustness of agentic systems against adversarial and harmful inputs. Its role is to screen what users send to the agent, and we will state this role and its limits in an explicit “Scope” subsection:
    • AdaGuard checks each user input against the deployment's policies before an agent acts on it. We added WildJailbreak (will update Table 1), on which AdaGuard scores 92.0 and 92.3 F1 in autoreasoning and reasoning-on, respectively, so skipping reasoning does not let these jailbreaks through.
    • Indirect injections that reach the agent through tools or retrieved documents are outside our scope, and we will ensure the introduction no longer suggests otherwise.
    • 819 of our 12,291 test inputs are multi-turn conversations, and AdaGuard judges all of their user turns. On these, it performs similarly across autoreasoning and reasoning-on.
    • We stress that we have not trained AdaGuard for output moderation, but we evaluated on DynaBench as a zero-shot test: it asks whether an agent's replies follow a set of rules. AdaGuard scores 68.4 ± 2.5 F1 in autoreasoning compared to 68.7 ± 2.6 in reasoning-on with 37% fewer output tokens.
    • Finally, malformed policies are out of scope. It is assumed that policies are written by a deployment team. We will state this assumption more clearly in our updated manuscript.
  • Fixed vs. Broken Verdicts: On violations, reasoning-on fixes 2.7× / 8.6× more verdicts than it breaks (for Gemma and Nemotron backbones, respectively); most of its broken verdicts are false alarms on compliant pairs. AdaGuard reasons on 88% / 78% of violating pairs but only 21% / 28% of compliant ones. With a third of the reasoning, its net correction is larger than reasoning-on's on Gemma and equal on Nemotron.
  • Reward Design Considerations: Most constants in our RL reward design mainly need to respect an ordering (e.g. fixing a verdict earns more reward than a correct skip), whereas their scales are normalized away. But how much the model reasons is controlled by: 1) the size of the counterfactual rewards, and 2) the cost for unnecessary reasoning. We ran control experiments for both. Based on these experiments, both change how much the model reasons overall, but neither significantly changes AdaGuard's accuracy later during training. Doubling the cost for unnecessary reasoning makes the model reason less throughout training by cutting unnecessary reasoning. By epoch 3 it reasons on 6.7 points fewer policies, with no significant change in per-policy or per-example F1 at any epoch. On the other hand, halving the counterfactual magnitudes slows down learning to reason. At epoch 1 the model reasons on 16% of policies instead of 37% and is 2.1 per-policy F1 lower, but it is level with the final reward at epochs 2 and 3. By epoch 3, both runs end at AdaGuard's own trade-off: a similar reasoning rate at a per-policy F1 not significantly different from AdaGuard's. We also reproduced Figure 3 ablations at epochs 2 and 3, which still supports that without the counterfactual term the model reasons on almost every policy. The longer runs also reframe the claim on the quality judge in that it helps the model learn faster but does not improve its final accuracy.
  • AdaGuard's Traces and the Choice of Judge: We added further checks on AdaGuard's own traces. Read by Claude Opus 5.5 with hidden verdicts, the reasoning addresses its own policy in every trace and supports the verdict the model emits in 100% (AdaGuard) / 98.8% (reasoning-on) of traces. Scored on the five criteria of Table 5, 77–81% of correct verdict traces pass all five, and traces behind wrong verdicts score much lower. We also launched human annotation. Finally, the quality judge used in training never sees labels, cannot reward a wrong verdict, and removing it does not materially change accuracy after more extended training.
  • Statistical Significance: We added paired cluster bootstrapping to inform statistical significance of our results. At a high level, AdaGuard over reasoning-off, random selection at the same reasoning budget, and SingGuard routing at the same budget are all statistically significant. Meanwhile, AdaGuard's accuracy vs. reasoning-on on our dynamic test set is not statistically significant, as it is not our paper’s claim either.

In our view, the results above resolve the main points the reviewer raised. One experiment is still underway, the human annotation of AdaGuard's traces, and we will share it here as soon as it finishes. Should the results so far settle the reviewer's concerns, we hope the reviewer will reconsider the scores and overall rating. We are also happy to run any additional comparison the reviewer would find useful, and we look forward to further discussions.

A) Evidence of Accuracy Improvements Derived from Reasoning

We agree that we should demonstrate this more directly in the main body of the paper. Our new analyses (Table A) show that the gain from reasoning is in fact the largest where expected, that is, on hard input-policy pairs, and on inputs that carry many policies. On Nemotron, whose reasoning-off misses most violations, the gain is large everywhere.

**Table A: Where reasoning helps. per-policy F1 by teacher difficulty; and AdaGuard's paired gain by number of policies per input.

Per-policy F1Gemma4-E4BNemotron-Nano-4B
Hard policies: reasoning-off → AdaGuard52.2 ± 1.3 → 63.5 ± 1.219.7 ± 1.3 → 44.8 ± 1.4
Easy policies: reasoning-off → AdaGuard87.7 ± 0.6 → 90.0 ± 0.655.3 ± 1.3 → 77.0 ± 1.0
AdaGuard − reasoning-off, inputs with 1–3 policies+3.0 ± 1.6+7.3 ± 2.0
AdaGuard − reasoning-off, inputs with 4–8 policies+4.6 ± 0.9+20.8 ± 1.5
AdaGuard − reasoning-off, inputs with 9+ policies+7.8 ± 1.2+31.5 ± 1.8

The gain is also not simply more generated text. AdaGuard always writes a shared input-understanding passage before its per-policy verdicts, so we also separated the two effects by forcing the model to write the passage and then give every verdict directly, and separately to reason on randomly chosen policies at AdaGuard's own budget (Table B). The passage alone adds +1.3 (Gemma) / +7.0 (Nemotron) per-policy F1, mostly by reducing fpr on Gemma, while it brings no gains in per-example. AdaGuard's choice of which policies to reason on adds a further +4.2 / +14.7, mostly by catching more violations (recall 70.8% → 77.5% / 37.4% → 57.8%, Gemma/Nemotron respectively). Random choices at the same budget add only +0.9 / +6.2.

Table B: Separating the shared passage from per-policy reasoning. per-policy/per-example F1, and per-policy recall / fpr.

ModeGemma: per-policy F1Gemma recall / FPRNemotron: per-policy F1Nemotron recall / FPR
reasoning-off74.5±0.7 / 86.6±0.670.8±0.9 / 3.7±0.242.2±1.0 / 58.9±1.130.5±0.9 / 2.7±0.1
Random per-policy selection between reasoning-off and reasoning-on verdicts (AdaGuard's budget)76.3±0.6 / 86.4±0.673.9±0.8 / 3.8±0.251.6±1.0 / 71.8±0.940.9±1.0 / 3.4±0.2
Passage + direct verdicts (all difficulty = Low)75.8±0.7 / 86.1±0.670.8±0.9 / 3.1±0.249.2±1.0 / 66.7±1.037.4±1.1 / 2.8±0.1
Passage + random per-policy reasoning (AdaGuard's budget)76.7±0.7 / 86.7±0.674.2±0.8 / 3.7±0.255.5±1.0 / 73.0±0.945.8±1.0 / 3.7±0.2
Reasoning-on79.8±0.8 / 88.2±0.680.7±0.9 / 4.1±0.264.2±0.9 / 76.6±0.858.4±1.1 / 4.5±0.2
AdaGuard80.1±0.6 / 88.2±0.677.8±0.8 / 3.2±0.264.5±0.9 / 76.7±0.958.5±1.1 / 4.3±0.2

B) Held-out Runtime Policy Generalization Beyond Safety

Our dynamic policies are in fact not mere paraphrases of harm labels. They are written by our reflective generation-and-verification pipeline (§2.2, Appendix A.5). We also leverage three instruction variants to produce complex policies: rules with conditional logic and edge cases, those that weigh intent, severity and context, or a single rule that fuses two categories. Complex policies make up half of the policies in SFT training and 70% in RL training (Appendix A.4, Step 3). The held-out test policies show the result (Table C): 67% contain conditional or exception clauses, and 17.5% (about one in six) cover a classic harm and a compliance topic in the same rule.

Our dynamic-policy test set is also held out by category, as described in Appendix A.4 (Held-out Split Construction). Grouped together, these fall into 15 input categories, none of which appears among the training categories. The split is made at the level of these input-category labels, not of broad topics: we held out one category from each cluster of related harm categories (Appendix A.4), so related categories, and the broad topic domains of Table C (e.g. violence or privacy), still appear in training. To check what the test policies actually are, we used Claude Opus 5.5 to label all 3,141 of them against a written rubric: one of 15 topic domains, whether the rule contains a condition or exception that changes the verdict, and whether it substantively covers both a classic-harm and a compliance topic. An independent second labelling of 400 random policies agreed with the first on 98% of groups, 96% of domains, 97% of condition labels and 94% of mixed-topic labels (Cohen's κ = 0.96, 0.95, 0.93 and 0.81) (Table C):

Table C: Composition of the held-out test policies. All test policies, labelled by Claude Opus 5.5. The two structure rows overlap the domain rows.

Composition of the held-out test policiesPoliciesShare of input–policy pairs
Classic harm (violence/weapons, hate/harassment, cyber, sexual/child, criminal, drugs, self-harm)1,887 (60%)63%
Compliance & regulatory (fraud/consumer, privacy/PII, political/government, misinformation, legal/medical/financial advice, IP/brand, AI-system integrity)1,125 (36%)32%
General catch-all129 (4%)4%
Structure (overlaps the rows above): contain a conditional or exception clause2,117 (67%)70%
Structure (overlaps the rows above): mix a classic-harm and a non-safety topic in one rule550 (17.5%)16.5%

We establish that the test policies are not simply taxonomy labels or rephrases. 67% contain conditional or exception clauses, 17.5% cover a classic harm and a compliance topic in the same rule (e.g. "block requests for detailed money-laundering schemes …, but allow legitimate inquiries about anti-money-laundering compliance … when the context demonstrates lawful professional or academic intent"), and 36% cover compliance and regulatory domains. On these compliance and regulatory policies, AdaGuard keeps its advantage as shown in Table D:

Table D: Per-policy F1 on compliance & regulatory vs. classic-harm policies. AdaGuard − random is a paired difference at the same reasoning budget as AdaGuard.

Per-policy F1Gemma4-E4B: compliance & regulatoryGemma4-E4B: classic harmNemotron-Nano-4B: compliance & regulatoryNemotron-Nano-4B: classic harm
reasoning-off69.0 ± 1.676.0 ± 0.827.5 ± 2.045.6 ± 1.2
reasoning-on75.5 ± 1.680.5 ± 0.854.7 ± 1.966.2 ± 1.0
AdaGuard75.2 ± 1.581.4 ± 0.856.0 ± 2.066.3 ± 1.0
AdaGuard: share of policies it reasons on23%35%30%38%
AdaGuard − random choice of policies (same budget)+3.9 ± 1.4+3.8 ± 0.7+16.8 ± 2.1+12.0 ± 1.0
AdaGuard on mixed-topic rules (reasoning-off / reasoning-on / AdaGuard)70.6 ± 1.8 / 77.7 ± 1.6 / 78.0 ± 1.639.6 ± 2.6 / 62.1 ± 2.2 / 62.0 ± 2.1

On compliance and regulatory policies AdaGuard reaches 75.2 ± 1.5 (Gemma4-E4B) / 56.0 ± 2.0 (Nemotron-Nano-4B) per-policy F1. That is +6.2 / +28.5 over reasoning-off and +3.9 / +16.8 over random selection at the same reasoning budget (both p<0.001p < 0.001), and on par with reasoning-on. These policies can be deemed harder than classic-harm policies (−6.2 / −10.3 F1), partly because violations are rarer among them (10% vs. 18% of pairs). We will add this breakdown to the paper (as a new appendix table under "Composition of the held-out policies and per-category per-policy F1"), and add the composition to §3 (held-out set: 36% compliance and regulatory, 67% conditional).

Reviewer_6wrgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AdaGuard is an input-moderation framework that evaluates user content against runtime-supplied, natural-language policies and produces a verdict for each policy. It supports direct judgments, always-on reasoning, and an adaptive mode that predicts whether each policy requires additional reasoning. Training combines supervised fine-tuning with RL using GRPO. Its central reward component compares reasoned judgments with separately sampled non-reasoning judgments to encourage reasoning when it improves correctness. An LLM-based quality multiplier further shapes the reward while limiting incentives for unnecessary explanations.

The evaluation shows backbone-dependent benefits from training. In adaptive mode, Base → SFT → GRPO improves per-policy F1 from 74.8 → 76.5 → 80.1 for Gemma-E4B and 26.8 → 46.3 → 64.5 for Nemotron-Nano-4B.

Critical strengths
  1. A useful formulation of selective computation for policy moderation. 

Allocating reasoning separately to individual policies is well motivated: the same input may be straightforward under one policy and ambiguous under another. The runtime-policy interface and structured per-policy outputs make the approach relevant to applications with changing requirements. The reported reduction of approximately 35.5% in latency while preserving dynamic-policy F1 provides encouraging evidence of a practical benefit over always-on reasoning.

  1. Mechanism-focused ablations support the reward design. 

The paper examines training stages, difficulty levels, two backbones, and the counterfactual and quality-reward components. In particular, removing the counterfactual component leads to nearly universal reasoning, whereas including it preserves selective behavior. This is consequential because it connects the proposed training mechanism to the central behavioral claim, rather than relying exclusively on aggregate benchmark improvements.

Critical weaknesses
  1. The base Gemma model is already competitive on easy examples, and GRPO’s hard-example performance is approximately equal to SFT with reasoning always enabled for both backbones. The strongest evidence therefore supports learning to use reasoning selectively while retaining much of always-on reasoning’s accuracy, rather than a consistent accuracy improvement over the best SFT configuration. For Gemma, adaptive reasoning matches always-on dynamic-policy F1 while reducing latency by about 35%; compared with reasoning off, however, it gains only 1.6 F1 points at roughly 2.8 times the latency. Its practical value consequently depends on how much additional accuracy justifies the inference cost.

  2. The evaluation does not sufficiently isolate the benefit of learned per-policy reasoning selection over existing or simpler alternatives. The paper identifies SingGuard as its closest predecessor, but does not include an experimental comparison with it. Moreover, the reported savings relative to always-on reasoning demonstrate that reasoning selectively can reduce inference cost, but do not establish that AdaGuard selects policies more effectively than a confidence-based rule or random selection. These alternatives could potentially achieve a similar accuracy–cost trade-off without the proposed reasoning-allocation reward.

  3. The comparison between adaptive reasoning and reasoning-off mode combines the effects of shared input understanding and additional per-policy reasoning. Adaptive mode always generates an initial input-understanding passage, whereas reasoning-off mode omits this step. That initial analysis may itself improve judgments by clarifying the input’s meaning, intent, or context. Consequently, the observed accuracy improvement cannot be attributed solely to deciding which policies require additional reasoning.

Questions for authors
  1. Could the authors compare AdaGuard against SingGuard and simpler confidence-based and random per-policy selection baselines at matched average output-token or latency budgets?

  2. Could the authors add a baseline that generates the same shared input-understanding passage and then directly predicts verdicts for every policy, without generating policy-specific reasoning?

  3. Please reconcile the dataset total in Table 6: the five listed input counts sum to 36,789, whereas the reported deduplicated total is 75,254.

8 comments

We greatly appreciate your careful review, and are glad that you recognize the importance of the two things we consider central to this work; 1) per-policy selective computation as an improved and practical formulation for policy moderation, and 2) that the ablations tie the reward design, especially the counterfactual component, to the behavior we claim. As requested, we ran all additional experiments on both backbones (Gemma4-E4B and Nemotron-Nano-4B), and the full held-out dynamic policy test set (12,291 inputs, 81,235 input-policy pairs), with 95% cluster-bootstrap confidence intervals over test inputs and paired significance tests.

High-Level Summary of Our Results

  1. AdaGuard’s learned per-policy selection in autoreasoning mode beats random selection, random selection forced inside AdaGuard’s own generation, and SingGuard’s routing at the same reasoning budget (all P < 0.001 per policy; Table A), and also at the same output length (Table F). And it matches or exceeds a per-policy confidence gate with the advantage of not requiring logit access, threshold sweeping, or a second LLM call as confidence gating.
  2. The shared input-understanding component explains a small part of the gain over reasoning-off, whereas the learned per-policy reasoning explains most of it (Table H).
  3. The main advantage of reasoning concentrates where it matters the most, that is on hard policies and on inputs with many policies.

Below are our more detailed responses. For simplicity, we will refer to AdaGuard’s autoreasoning mode as simply AdaGuard throughout the following sections.

We believe our rebuttal response and the additional results and analyses presented fully address the concerns raised by the reviewer. Thus, we would be grateful if the reviewer would reconsider their scores and the overall rating for our paper. We would be glad to run any further comparison the reviewer considers informative, and we welcome any additional feedback.

A) Cost/Benefit Tradeoff of autoreasoning Relative to reasoning-on and reasoning-off

We agree and have already highlighted at the end of the “Introduction” section that the central claim of this work is selective reasoning that preserves always-on accuracy at much lower costs, and the revision will state it more directly and crisply. We also must note that the aggregate numbers understate AdaGuard’s value, for following reasons:

  1. The aggregate metrics are dominated by easy examples in our dynamic-policy test set (~70% of input–policy pairs), where every mode is operating at near-ceiling. Per-example F1 also counts an input as violation if any of its policies is violated, so an obvious violation on one policy will mask errors on the harder policies for the same input. The +1.6 per-example gain over reasoning-off is an average over this mix. On hard policies the picture is completely different (Table B): AdaGuard improves per-policy F1 from 52.2 to 63.5 on Gemma (+11.3) and 19.7 to 44.8 on Nemotron (+25.2). This recovers 86% (Gemma) and all (Nemotron) of reasoning-on's hard-policy gain, while reasoning on 55% / 63% of hard policies and only 22% / 25% of easy ones.
  2. reasoning-on may hurt on easy content, while AdaGuard avoids that. On 4,017 test inputs whose policies are all teacher-labelled easy (Table C), Gemma’s reasoning-on scores 72.8 per-policy F1 (below reasoning-off 81.3) against AdaGuard's 83.3 (+10.5, p < 0.001) which only reasons on 12% of these policies. Nemotron’s reasoning-off mode is much weaker, so reasoning does not hurt on those examples (66.4 reasoning-on vs. 61.6 reasoning-off). There, AdaGuard matches reasoning-on’s (67.5, +1, n.s.) while reasoning on only 10% of the policies. Gemma’s AdaGuard is also above both reasoning-off (87.7) and reasoning-on (88.6) on easy per-policy F1 (90.0) (Table B). The bottom-line is, this is the behavior a per-policy controller should have, i.e. reason where it helps.
  3. The final consideration is cost. AdaGuard matches reasoning-on (Gemma −0.05 per-example / +0.30 per-policy; Nemotron +0.04 / +0.28) at 35.5% / 35.8% lower latency and 41–49% shorter outputs (characters). Compared to reasoning-off, the extra cost buys the hard-policy gains above, and AdaGuard buys them more efficiently than any alternative we tested. The improvements also grow with the number of policies per input. Its margin over SingGuard's routing rises from +0.0 (1–3 policies) to +4.0 (9+) on Gemma and from +2.1 to +8.5 on Nemotron. Across all policy buckets, AdaGuard is almost on par with reasoning-on, while its own share of reasoned policies falls from 49–50% on inputs with 1–3 policies to 25–29% on inputs with 9+ (Table G). Finally, the base model's strength on easy policies does not necessarily help in deciding where to reason: base Gemma in autoreasoning reaches 74.8 per-policy F1 at a 21.9% reasoning rate (Table 2), against 80.1 for AdaGuard.

B) SingGuard, Confidence-based, and Random Selection at Matched Budgets

To answer if autoreasoning beats other approaches, we ran additional baselines. For each test input–policy pair we also have the model’s reasoning-off and reasoning-on verdicts. Each baseline matches AdaGuard’s input-policy reasoning rate (26,407 of 81,235 pairs on Gemma, and 29,863 on Nemotron). Chosen pairs that reason take reasoning-on’s verdict, else reasoning-off’s. The budget and the verdicts are therefore identical, isolating only which policies are reasoned on. We experimented with the following baselines:

  • Random Selection: draws reasoned pairs uniformly at random. It tests whether AdaGuard's choice matters, or only how much it reasons. Over 200 random draws, it reaches 76.30 ± 0.14 and 51.53 ± 0.25 per-policy F1, against AdaGuard's 80.10 ± 0.32 and 64.52 ± 0.47 for Gemma and Nemotron, respectively.

    We also implemented a stricter version, where, before each policy, AdaGuard overwrites the difficulty token at random, while preserving its reasoning budget: High, followed by reasoning, or Low directly by the verdict, letting the model continue its response from the forced difficulty token. The shared input-understanding, all reasoning and verdicts are still AdaGuard's autoreasoning’s own text, and every policy is still conditioned on the passage and earlier policies. Only which policies get reasoned on is random. It still loses by 3.4 (Gemma) / 9.0 (Nemotron) per-policy F1 (p < 0.001).

  • SingGuard Routing: It marks all input-policy pairs whose input-level safe/unsafe confidence is lowest. Due to restrictions on running some third-party models in our environment, we could not evaluate SingGuard's checkpoints. Instead, we reimplemented its hybrid routing (Eq. 7–8 from their paper) on our AdaGuard backbones, which should isolate the routing rule itself. SingGuard first decodes a safe/unsafe token y0y_0 for the whole input and computes normalized confidence s(y0)=p(y0)/(p(safe)+p(unsafe))s(y_0) = p(y_0) / (p(safe) + p(unsafe)). If it is above a threshold (s≥τs \ge \tau), it stops and answers y0y_0 ; otherwise it reasons over all policies before providing the final verdict. We follow these steps to replicate this behavior:

    1. We retrieve AdaGuard's reasoning-off token log-probabilities for the final per-policy verdicts. For each policy kk we take pk=P(Violation)p_k = P(Violation) from its verdict token, normalized over Violation/Compliance. Since an input is marked safe only if no policy is violated, p(safe)=∏k(1−pk)p(safe) = \prod_k(1 − p_k) and p(unsafe)=1−p(safe)p(unsafe) = 1 − p(safe). Therefore, y0y_0is the more likely label and s=max⁡(p(safe),p(unsafe))s = \max(p(safe), p(unsafe)), which is Eq. 7 for a binary decision. We also tried a variant that does not depend on the number of policies for the ultimate safe/unsafe thresholding (i.e. ss is from the most confident violation, or else the least confident compliance), which results in the same conclusions.
    2. Inputs with s<τs < \tau take the reasoning-on verdicts for all their policies (SingGuard's slow mode), while all other inputs keep their reasoning-off verdicts.
    3. Then, we rank inputs by s from the least confident first, and escalate them in that order until the number of escalated policies reaches AdaGuard autoreasoning’s (26,407 on Gemma, 29,863 on Nemotron). This is equivalent to τ\tau = 1 − 10⁻⁶·⁷ (Gemma) / 1 − 10⁻⁵·¹ (Nemotron). We also sweep across τ\tau and tune it for accuracy, since SingGuard does not report it (Table D).
    4. Escalated verdicts come from an independent reasoning-on pass, not conditioned on y0y_0, so they cannot anchor on the first guess. If anything, this favors SingGuard’s implementation.

    SingGuard shares our goal of reasoning only when needed, but makes that decision per input, i.e. it reasons on every policy, and an uncertain policy inside a confidently-unsafe input is never reasoned on. On the other hand, AdaGuard decides per policy, before writing that policy's verdict. That is why SingGuard's gap grows from ≈0 on inputs with 1–3 policies to 4.0 (Gemma) / 8.5 (Nemotron) F1 on inputs with 9+ (Table G), and why no τ lets it match AdaGuard per policy (Table D).

    To conclude, SingGuard’s implementation loses per policy on both AdaGuard backbones (p < 0.001), and this gap grows for inputs with more policies.

  • Per-policy Confidence Gate: marks the pairs whose reasoning-off verdict probability is low. It compares an uncertainty rule against autoreasoning. Based on our results (Table A), it matches AdaGuard on Gemma and falls below it on Nemotron (p = 0.03). But its main drawback is efficiency, as it needs token probabilities from an additional reasoning-off pass, and only operates at an extreme threshold (1 − 10⁻⁸·⁷ to 1 − 10⁻¹⁰·⁵) derived from sweeps across the test set, to spend the same reasoning budget. AdaGuard makes this decision automatically within one pass, incurring a much smaller latency cost.

  • Thresholding on AdaGuard's difficulty: AdaGuard offers the same knob as SingGuard, but per policy and in one pass. It can reason on a policy if its P(difficulty=High)≥τP(difficulty = High) \ge \tau. τ=0.5\tau = 0.5 is taken as the ordinary decoding, and we see that it reproduces AdaGuard (79.99 / 63.90 vs. 80.10 / 64.52 per-policy F1; Table E). This tests whether AdaGuard's difficulty signal can also be leveraged as a tunable budget. Across τ\tau it is above every baseline at low budgets (per-policy) on Nemotron (e.g. +3.9 per-policy F1 over the confidence gate at 10% of policies), and within 0.3 F1 of confidence gate at higher budgets. On Gemma it is above SingGuard and random at every budget above 10% and slightly trails confidence gate at low budgets. It reaches 80.45 per-policy F1 on Gemma with 48.7% of policies reasoned at τ\tau = 10⁻⁶, matching reasoning-on (79.80). We note that it is not a separate method and does not replace autoreasoning. It reads the difficulty token that autoreasoning training taught the model, needs no retraining or a second pass. It lets users move along the cost–accuracy curve while the learned per-policy decision is what makes that curve strong.
  • Matched Output Length: To answer the token-budget question, it uses AdaGuard's actual output length as the budget for each baseline instead of its policy count (Table F). The ordering remains unchanged.

Table A: Matched reasoning budget (per-policy/per-example F1)

StrategyGemma: reasonedGemma F1Nemotron: reasonedNemotron F1
reasoning-off0%74.50±0.69 / 86.57±0.620%42.23±0.99 / 58.89±1.10
reasoning-on100%79.80±0.75 / 88.24±0.60100%64.24±0.90 / 76.62±0.80
Random (per policy)32.5%76.32±0.65 / 86.36±0.6336.8%51.60±0.96 / 71.81±0.92
Random, forced in one generation32.4%76.68±0.69 / 86.65±0.6336.7%55.49±0.96 / 72.96±0.89
SingGuard routing32.5%77.68±0.64 / 88.26±0.5736.5%57.90±0.98 / 72.42±0.93
Confidence gate (per policy)31.7%80.26±0.66 / 87.75±0.6036.1%63.72±0.90 / 76.63±0.86
AdaGuard31.8%80.10±0.64 / 88.18±0.5936.3%64.52±0.93 / 76.66±0.86

Table B: Per-policy F1 on teacher-labelled easy and hard policies

StrategyGemma easyGemma hardNemotron easyNemotron hard
reasoning-off87.65±0.6252.23±1.3455.31±1.2519.65±1.27
reasoning-on88.60±0.8265.38±1.2476.70±0.9844.39±1.46
Random (per policy)88.04±0.6356.65±1.2464.58±1.1129.84±1.39
Random, forced in one generation88.02±0.6557.75±1.3468.34±1.0534.40±1.48
SingGuard routing89.27±0.5958.70±1.2670.63±1.0936.91±1.51
Confidence gate (per policy)89.97±0.6164.16±1.2676.64±0.9542.64±1.45
AdaGuard90.02±0.5663.50±1.2476.95±1.0044.81±1.43
AdaGuard: share reasoned21.6%55.4%24.8%63.0%

Table C: Inputs whose policies are all teacher-easy (4,017 of 12,291 test inputs)

All-easy inputsGemma: per-policy F1Gemma: per-example F1Nemotron: per-policy F1Nemotron: per-example F1
reasoning-off81.3±2.282.8±2.061.6±3.169.9±2.8
reasoning-on72.8±3.982.2±2.066.4±3.271.4±2.6
AdaGuard83.3±2.484.7±2.067.5±3.271.4±2.6
AdaGuard − reasoning-on+10.5±3.4 (p < 0.001)+2.5±1.8 (p = 0.005)+1.0±2.5 (n.s.)+0.0±2.1 (n.s.)
Policies AdaGuard reasons on11.9%—10.2%—

Table D: Threshold sweeps (each cell: share of pairs reasoned · per-policy / per-example F1; a gate reasons where confidence < tautau)

τ\tauGemma: confidence gateGemma: SingGuardNemotron: confidence gateNemotron: SingGuard
0.91.3% · 75.41 / 85.993.4% · 75.01 / 86.042.3% · 44.53 / 66.108.2% · 46.95 / 66.93
0.992.8% · 75.97 / 86.407.0% · 75.48 / 86.554.7% · 46.87 / 67.4316.3% · 50.93 / 68.88
0.9994.7% · 76.76 / 86.8011.5% · 76.01 / 87.116.6% · 48.20 / 68.4023.1% · 53.74 / 70.30
1 − 10⁻⁴6.8% · 77.41 / 87.1516.4% · 76.48 / 87.498.5% · 49.70 / 69.3129.2% · 55.79 / 71.23
1 − 10⁻⁶12.2% · 78.86 / 87.7427.7% · 77.29 / 88.0812.9% · 53.17 / 71.1943.3% · 59.91 / 73.32
1 − 10⁻⁸19.4% · 79.68 / 87.9142.0% · 78.03 / 88.0421.1% · 60.58 / 75.1276.4% · 63.46 / 76.66
1 − 10⁻¹⁰28.8% · 80.12 / 87.8159.4% · 78.21 / 87.91100.0% · 64.24 / 76.9698.5% · 64.09 / 76.99
1 − 10⁻¹²51.7% · 80.33 / 87.7579.9% · 78.41 / 87.86100.0% · 64.24 / 76.9699.3% · 64.10 / 76.96
τ\tau tuned for per-policy F1 (2-fold)42.7% · 80.39 / 87.73100.0% · 79.80 / 87.7361.6% · 64.32 / 76.9099.9% · 64.22 / 76.97
AdaGuard31.8% · 80.10 / 88.1836.3% · 64.52 / 76.66

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Results and Improvements (Part I)

We are grateful to all three reviewers for their careful and constructive reviews and feedback. We are encouraged that the reviewers found our formulation of per-policy selective reasoning practical for policy moderation, especially where reasoning latency matters in deployment, our results comparing to other models and across other benchmarks competitive, that the ablations tie our counterfactual reward implementation to the behavior we claim.

To address every comment, we ran new experiments and analyses on both backbones (Gemma4-E4B / Nemotron-Nano-4B), on our full held-out dynamic-policy test set, and on two additional external benchmarks (WildJailbreak and DynaBench). All results now carry a 95% cluster-bootstrap confidence interval, and comparisons include statistical significance.

Below we summarize our main findings and improvements across our all responses that we will incorporate in the update version of our manuscript. Detailed tables are in the individual responses. For simplicity, we refer to AdaGuard's autoreasoning mode as simply AdaGuard below.

1. AdaGuard's autoreasoning Per-Policy Advantage across Other Baselines

We compared AdaGuard with other baseline selection rules that maintain the same input–policy reasoning budget, using the same model's reasoning-off and reasoning-on verdicts, in order to isolate only the choice of which policies are reasoned on. Per-policy F1 differences (AdaGuard − baseline) are:

Baseline at AdaGuard's reasoning budgetGemma4-E4BNemotron-Nano-4B
Random selection+3.8 (p<0.001p < 0.001)+12.9 (p<0.001p < 0.001)
Random selection, forced inside AdaGuard's own generation+3.4 (p<0.001p < 0.001)+9.0 (p<0.001p < 0.001)
SingGuard's routing (reimplemented on our backbones)+2.4 (p<0.001p < 0.001)+6.6 (p<0.001p < 0.001)
Per-policy confidence gate on verdict-token probabilities−0.2 (n.s.)+0.8 (p=0.03p = 0.03)
reasoning-on (reasons on every policy)+0.3 (n.s.)+0.3 (n.s.)
Share of policies AdaGuard reasons on31.8%36.3%
  • We show advantage over SingGuard's routing mechanism, and that it grows with the number of policies per input, from +0.0 / +2.1 → +4.0 / +8.5 on inputs with 1–3 → 9+ policies (Gemma/Nemotron backbones, respectively). SingGuard decides per input while AdaGuard decides per policy. No SingGuard threshold closes this gap, and when tuned for accuracy, it escalates almost every input to reasoning (100% / 99.9% of policies).
  • The only rule that matches AdaGuard is the confidence gate, which relies on token probabilities from an extra reasoning-off pass and an extreme threshold tuned on the test set. AdaGuard makes the decision within a single generation.
  • This ranking also holds when every baseline is given AdaGuard's measured output length as its budget instead of its reasoned policy count.

2. AdaGuard: reasoning-on Accuracy at A Third of the Reasoning

  • AdaGuard matches reasoning-on (per-example F1 −0.05 / +0.04, per-policy F1 +0.3 / +0.3, all not significant) at 35.5% / 35.8% lower latency and 41–49% shorter outputs. Since our claim has always been efficiency at matched accuracy, these ties are squarely within the scope we outlined in the paper.
  • Larger gains on hard policies: per-policy F1 rises from 52.2 → 63.5 and 19.7 → 44.8 (Gemma/Nemotron) over reasoning-off. AdaGuard reasons on 55% / 63% of -hard policies but only 22% / 25% of easy ones.
  • The lead widens with larger number of policies per input: +3.0 → +7.8 and +7.3 → +31.5 (Gemma/Nemotron) per-policy F1 over reasoning-off, from inputs with 1–3 policies to 9+.
  • Selective reasoning avoids the cost of over-reasoning: on the 4,017 inputs whose policies are all teacher-easy, Gemma's reasoning-on falls below reasoning-off (72.8 vs. 81.3 per-policy F1), while AdaGuard reaches 83.3 (+10.5 over reasoning-on, p<0.001p < 0.001) by reasoning on only 12% of them.
  • It is the choice of policies that drives the gain, rather than the extra reasoning text. AdaGuard always writes a shared input-understanding passage first. The passage alone adds +1.3 / +7.0 per-policy F1 over reasoning-off. However, the bulk of the gain comes from AdaGuard's choice of policies to reason on (an additional +4.2 / +14.7), while random choices at the same budget add only +0.9 / +6.2.
  • Fixed vs. broken verdicts: on violating pairs, reasoning-on fixes 2.7× (Gemma) / 8.6× (Nemotron) more verdicts than it breaks, and most of its broken verdicts are false alarms on compliant pairs. Without seeing labels, AdaGuard reasons on 88% / 78% of violating pairs but only 21% / 28% of compliant ones, so with a third of the reasoning its net correction is larger than reasoning-on's on Gemma (+1.6 vs. +1.3 points, p=0.003p = 0.003) and equal on Nemotron.
AuthorsOpenReview ↗

Summary of Results and Improvements (Part II)

3. Counterfactual Reward: Selective Reasoning and Reward Component Sensitivity

  • Reward ablation extended from later epochs. Without the counterfactual term the model reasons on 99.4–99.9% of policies at every epoch, and with it on 34–39% at epochs 2 and 3, with no significant difference in per-policy F1 (p≥0.23p \ge 0.23).Therefore, the same per-policy gate without CF-Corr ends up reasoning on all policies.
  • The quality judge speeds up learning but does not affect the final accuracy: Reasoning rate during training reaches 40% by step 60 with it and only by step 120 without it, and converge at later epochs. Moreover, the judge never sees labels, its influence is bounded (at most ±50% as a reward multiplier on reasoned policies), and can never reward a wrong verdict. We will update our reading of the judge in §4 accordingly.
  • Reward-constant sensitivity. Most reward constants only need to respect an ordering, since the optimizer normalizes each reward stream. The two settings that change how much the model reasons are the cost for unnecessary reasoning and counterfactual magnitudes. Our additional experiments varying each demonstrate that both change how much the model reasons, but neither significantly changes overall accuracy later during training. Doubling the cost for unnecessary reasoning cuts mostly unnecessary reasoning (6.7 points fewer policies by epoch 3), while halving the counterfactual magnitudes simply slows down learning to reason (2.1 per-policy F1 lower at epoch 1).

4. Held-out Dynamic Policies Analytics, and AdaGuard's Generalization beyond Classic Harms

  • Of the 3,141 held-out test policies, 67% contain conditional or exception clauses, 17.5% cover a classic harm and a compliance topic in the same rule, and 36% target compliance and regulatory domains (e.g. privacy, misinformation, professional advice, political topics).
  • On compliance and regulatory policies, AdaGuard maintains its advantage: 75.2 / 56.0 per-policy F1, +6.2 / +28.5 over reasoning-off and +3.9 / +16.8 over random selection at the same budget (p<0.001p < 0.001), and on par with reasoning-on.

5. New External Benchmarks: Jailbreaks and Human-Written Business Rules

  • WildJailbreak: AdaGuard scores 92.0 F1 (93.2/88.0 for reasoning-on and reasoning-off), and is second only to WildGuard (95.3), whose training data was generated with the same attack method as WildJailbreak. Therefore, skipping reasoning does not skip moderation as AdaGuard still gives a verdict for every policy.
  • DynaBench, as a zero-shot generalization test: DynaBench's rules target an output moderation task that AdaGuard was not trained for. AdaGuard scores 68.4 F1 while reasoning on only a fifth of the rules, against 68.7 in reasoning-on, while DynaGuard-8B (with CoT), which was trained on DynaBench's training split, scores 73.1.
  • With both added to Table 1, AdaGuard has the highest average across all six static benchmarks (91.0) and over all eight benchmarks (87.8).

6. AdaGuard's Reasoning Traces Quality

  • An independent LLM judge (Claude Opus 5.5) scored a stratified sample of AdaGuard's traces on the five criteria in Table 5 without reference labels. 77–81% of the correct verdict traces pass all five, and traces behind wrong verdicts score much lower.
  • When blind to the model's verdicts, reasoning supports them in 100% (AdaGuard) / 98.8% (reasoning-on) of traces and addresses its own policy in every trace.
  • Human annotation of AdaGuard's traces on the same criteria is underway, and we will post the results as soon as it is complete.

7. More Explicit and Crips Scope Statement

  • AdaGuard is an input-moderation guardrail that checks each user input against the deployment's runtime policies before an agent acts on it. Direct jailbreaks, including instruction overrides written into the user's own message, are in scope, while indirect prompt injection through tools or retrieved content is outside this scope. We will add an explicit "Scope" subsection and tighten the abstract and introduction accordingly.
  • The evaluation already covers multi-turn conversations: On 819 of the 12,291 test inputs that are multi-turn, AdaGuard reaches 82.0 / 70.7 per-policy F1 (+4.4 / +20.2 over reasoning-off, on par with reasoning-on).
  • Runtime policies are assumed to be written by the deployment team, so robustness to malformed policies is outside AdaGuard's scope. We will state this assumption explicitly.

We again thank the reviewers for their insightful feedback, which has substantially strengthened our paper. We hope our comprehensive and detailed response remediate all of their comments/questions, and further highlight the quality of our work and the advantage of our proposed methodology . We will post the updated manuscript with these improvements shortly, and look forward to the discussion.

AuthorsOpenReview ↗

Revised Manuscript

We sincerely thank all three reviewers for their thoughtful feedback and insightful comments, which have helped us substantially improve the manuscript.

As promised, we have added a human-annotation analysis of AdaGuard's own reasoning traces in response to Reviewer j9eo (please see Section I of our response to that reviewer). We have also implemented all changes and updates in the revised PDF, based on the results and analyses shared in our responses. All changes are marked in teal. For convenience, the main additions are:

  • A crisper statement of our claim (§2.1): autoreasoning matches the accuracy of reasoning_on while reasoning on only a fraction of the policies, and its learned per-policy choice of where to reason is what makes this possible.
  • An explicit scope (§2.1, Scope): AdaGuard is an input moderator, including multi-turn user turns and direct jailbreaks. Indirect prompt injection through tools or retrieved content, output moderation (tested only zero-shot, on DynaBench) and malformed policies are out of scope.
  • A results section reorganized around five questions (§3.1–3.5): Is AdaGuard competitive? Does the learned selection matter? What makes it work? Where do the gains come from? How robust is it?
  • New benchmarks: WildJailbreak (adversarial jailbreaks) and DynaBench (zero-shot, output moderation) in Table 1, with per-policy results in Table 2.
  • Learned selection vs. alternatives at the same reasoning budget (§3.2, Appendix A.7.3): random selection, SingGuard's routing, a per-policy confidence gate, threshold sweeps, and AdaGuard's own difficulty token as a cost knob.
  • Where the gains come from (§3.4, Appendix A.7.4–A.7.5): teacher-easy vs. hard policies, number of policies per input, the shared input-understanding passage, and fixed vs. broken verdicts per class.
  • Reward design: ablations across three training epochs and sensitivity to the reward constants (§3.3, Appendix A.10).
  • AdaGuard's own reasoning traces: an independent LLM judge and three human annotators (Appendix A.3.1).
  • Robustness (§3.5): jailbreaks, compliance and regulatory policies, and multi-turn conversations.
  • Statistical significance: 95% confidence intervals and paired tests for the main claims, plus standard deviations for every Table 1 score (Appendix A.11).
  • Readability: a running example of the three reasoning modes (Appendix A.8.1), a shorter reward section in the main text, and clearer definitions and captions throughout.

We hope these updates adequately address the reviewers' comments, and we would be glad to answer any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.