acceptodds
Under review as a conference paper at ICLR 2027

SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing

Abstract

In real-world applications, guardrails are often expected to identify unsafe user-model interactions according to application-specific safety policies, rather than relying on predefined risk taxonomies. In this work, we study this setting under the paradigm of *in-context policy guardrailing*, where guardrails predict safety violations based on policy specifications provided in context. To systematically evaluate this capability, we introduce SafePyramid, a safety benchmark comprising 1,000 multi-turn conversations across 10 domains and 3,000 corresponding application-specific policies, which together contain 61,641 distinct natural-language rules. SafePyramid organizes the evaluation into three difficulty levels: L0 evaluates individual-rule understanding, L1 evaluates reasoning over rule dependencies, and L2 evaluates adaptation of full novel policy frameworks defined in context. To ensure benchmark quality, we employ a rigorous multi-stage pipeline to construct and validate the benchmark. Using SafePyramid, we evaluate 10 frontier LLMs, 5 policy-configurable guardrails and 1 typed decision model, and find that in-context policy guardrailing remains highly challenging: even the best-performing model, GPT-5.5, exactly identifies the full set of violated rules in only 54.0%, 35.3%, and 12.9% cases on L0, L1, and L2, respectively. These results highlight the limitations of current guardrails and call for stronger in-context policy guardrails that can reliably execute policies, resolve rule dependencies, and adapt to novel policy frameworks. Our code is available at: https://anonymous.4open.science/r/safepyramid.

Primary area:
datasets and benchmarks
Keywords:
benchmarking, in-context policy guardrailing, NLP dataset, LLM evaluation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PVTgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces in-context policy guardrailing: given a conversation and an application-specific policy, a guardrail must predict the exact set of violated rules. To evaluate this, the authors build SafePyramid, with 1,000 LLM-generated conversations across 10 domains and 3,000 policies containing 61,641 rules. The benchmark has three levels: independent rules (L0), interacting exception and conditional rules (L1), and novel fictional policy frameworks (L2). Across 16 systems, performance drops sharply from L0 to L2. The best model, GPT-5.5, achieves exact match on only 54.0%, 35.3% and 12.9% of cases at L0, L1 and L2. Per-rule decomposition, agentic harnesses and higher reasoning effort all improve results.

Critical strengths
  1. The paper provides useful task formulation and diagnostic structure.
  2. The paper provides comprehensive evaluations and results.
Critical weaknesses
  1. Low data diversity. Each data type comes from a single generator: Grok writes all conversations, and GPT-5.4, Gemini and Claude write all L0, L1 and L2 policies respectively. This limits diversity and confounds difficulty level with generator, so part of the drop from L1 to L2 may reflect Claude’s writing style rather than framework adaptation.
  2. Unrealistic, conversation-dependent policies. Policies are generated after and conditioned on each conversation. Real policies are written before traffic is observed and must generalize, which weakens the benchmark’s realism claim.
  3. Unclear per-rule protocol. The paper never explains how the “relevant policy context” is selected. Results with the full policy and conversation are needed.
Questions for authors
  1. In per-rule evaluation, how is the “relevant policy context” selected? How do per-rule results change if each rule is judged with the full, unorganized policy and the full conversation?
6 comments
AuthorsOpenReview ↗

Weakness 1: Low Data Diversity

W1. Low data diversity. Each data type comes from a single generator: Grok writes all conversations, and GPT-5.4, Gemini and Claude write all L0, L1 and L2 policies respectively. This limits diversity and confounds difficulty level with generator, so part of the drop from L1 to L2 may reflect Claude’s writing style rather than framework adaptation.

A1: We thank the reviewer for the insightful comments! We would like to clarify a few points:

  • Data diversity is not solely determined by the choice of generator. SafePyramid's conversation and policy diversity are primarily shaped by human expert-designed generation schemes, which define conversation scope, topics, scenarios (see Appendix C), policy structures, and reasoning requirements at each level. In particular, these requirements are explicitly defined by human experts rather than left to the generators to decide. The generators mainly serve to produce data following these predefined schemes. We deliberately use different generators across levels to reduce potential biases and artifacts associated with relying on a single generator.

  • Data diversity and quality are carefully reviewed by human experts. Human experts review the generated data against our predefined requirements, including whether the policies follow the intended reasoning requirements of each level. Samples that fail to meet these standards are excluded from the benchmark. This helps ensure that the generated policies meet our expectations for each difficulty level, particularly that L2 policies introduce meaningful new policy frameworks rather than arbitrary complexity.

  • Additional Experiment 1: Potential generator-family bias. To examine whether Claude's writing style favors models from the same family, we analyzed Claude-Opus-4.7's performance on L2 using a two-way additive model that accounts for overall model performance and average level difficulty. Claude-Opus-4.7 shows almost no relative advantage on L2. While this does not fully rule out generator-specific effects, we find no evidence of a Claude-family advantage in this analysis.

    Table 1. Claude's performance on L2 policies generated by the Claude family.

    ModelL2 RMRRelative residual
    Claude-Opus-4.730.98-0.04 pp

    Here Relative residual = observed RMR − expected RMR, where expected RMR is estimated from the model's overall performance and the average difficulty of each level. A positive residual indicates a relative advantage on that level.

In conclusion, this suggests that Claude's writing style alone is unlikely to explain the substantial performance drop from L1 to L2.

AuthorsOpenReview ↗

Weakness 2: Unrealistic, Conversation-dependent Policies

W2. Unrealistic, conversation-dependent policies. Policies are generated after and conditioned on each conversation. Real policies are written before traffic is observed and must generalize, which weakens the benchmark’s realism claim.

A2: We thank the reviewer for this insightful comment. We agree that real-world policies should be general and applicable across different conversations. In fact, We carefully considered this issue when designing SafePyramid and would like to clarify our design rationale:

  • Our design choice is driven by the goal of the benchmark.

    • We carefully considered generating general policies first and then evaluating them against conversations. However, a general policy may contain many rules that are not relevant to a given conversation, making it difficult to ensure that each conversation covers the specific rules, exceptions, and conditions we want to evaluate. As a result, some challenges may rarely appear in the benchmark, while others may appear together in ways that are difficult to separate. This would limit our ability to understand which in-context policy guardrailing capabilities those models struggle with and why they fail.
    • We therefore chose to generate conversations first and then construct policies based on them. This allows us to deliberately design cases that test specific reasoning requirements, such as understanding individual rules (L0), handling exceptions and conditions (L1), and adapting to new policy frameworks (L2). It also allows us to control which rules are violated and which are not, providing clear ground-truth labels for evaluating model predictions. Our goal is not simply to rank models by their overall performance, but to help researchers better understand their strengths, weaknesses, and failure modes across different policy reasoning tasks. We believe this makes SafePyramid more useful for studying and improving in-context policy guardrails.
  • Our policies still reflect realistic policy structures. Although the policies are generated based on conversations, their structures are not arbitrarily designed around those conversations. As discussed in the paper, our policy schemas are inspired by real-world regulatory policies, including the GDPR, which contains exceptions, conditional obligations, and alternative safeguards. We use these real-world patterns to design the rules and reasoning requirements in SafePyramid. In addition, using synthetic policies helps reduce potential contamination from existing policies that models may have encountered during training, allowing us to better evaluate whether models can understand and apply policies provided in context.

  • Additional Experiment 1: Policies contain substantial content beyond the given conversation. Although each policy is generated with a particular conversation in mind, it also contains many rules that do not apply to that conversation. Our additional analysis in Table 1 below shows that distractor rules account for 25.2–52.0% of rules across the three levels.

    Table 1. Policy composition across difficulty levels.

    LevelRules per caseDistractor rulesException rulesConditional rules
    L016.152.0%0.0%0.0%
    L128.729.2%19.4%24.5%
    L232.925.2%20.6%30.4%

These results show that models must distinguish relevant from irrelevant policy rules, rather than simply matching all rules to the given conversation.

  • Additional Experiment 2: Policies are not simple lexical reproductions of conversations. We further measured the proportion of content words in each rule that also appear in its corresponding conversation. As shown in Table 2, decisive rules share only 38–39% of their content words with the corresponding conversation, while other rule types show even lower overlap (i.e., 23–26%). This suggests that rule violation identification requires more than simple lexical matching.

    Table 2. Lexical overlap between policy rules and conversations.

    Rule typeNumber of rulesMean lexical overlap
    Decisive, violated15,93138%
    Decisive, not violated7,37139%
    Distractor25,06626%
    Conditional17,03425%
    Exception12,35323%

Overall, our conversation-first approach is intended to make SafePyramid more useful for understanding models' in-context policy guardrailing capabilities and failure modes, rather than simply measuring overall performance. While our policies are constructed for specific conversations, they follow realistic policy structures and include substantial content beyond those conversations. We believe this provides a meaningful benchmark for studying in-context policy guardrailing, and we acknowledge that evaluating fixed policies across unseen conversations is an important direction for future work.

AuthorsOpenReview ↗

Weakness 3 & Question 1: Unclear Per-rule Protocol

W3 & Q1. Unclear per-rule protocol. The paper never explains how the “relevant policy context” is selected. Results with the full policy and conversation are needed. In per-rule evaluation, how is the “relevant policy context” selected? How do per-rule results change if each rule is judged with the full, unorganized policy and the full conversation?

A3: We thank the reviewer for pointing out that the per-rule evaluation protocol needs further clarification.

  • Why per-rule evaluation? The per-rule protocol is designed for guardrail models that cannot directly output a list of violated rules, as required by our per-policy evaluation. These models typically perform binary violation classification and often have limited context windows. Therefore, we evaluate each rule separately, asking the model whether the given conversation violates the target rule, and aggregate the predictions to compute the evaluation metrics.

  • How is relevant policy context selected? The relevant context depends on the design of each level:

    LevelRelevant policy contextEvaluation
    L0Target rule onlyDetermine whether the full conversation violates the rule
    L1Target rule + associated exception and conditional rulesDetermine whether the target rule is violated after considering its dependencies
    L2Target rule + associated exception and conditional rules + new policy frameworkDetermine whether the target rule is violated under the new framework

    In all three levels, the model is given the full conversation. For L1 and L2, the additional context ensures that the model has the information needed to correctly interpret the target rule. The context is selected based on policy structure and rule dependencies, not on ground-truth violation labels.

  • Per-rule evaluation with full, unorganized policy and full conversation. We thank the reviewer for this insightful question, which we had not previously considered. We agree that this makes for an interesting and valuable ablation study, and we have conducted additional experiments to investigate it. As shown in Table 1 below, we additionally evaluated Jev-1.13 using the full, unorganized policy and full conversation.

    Table 1. Jev-1.13 performance comparison between relevant-context and full-policy evaluation (%).

    SettingRMR L0RMR L1RMR L2RMR Avg.[email protected] L0[email protected] L1[email protected] L2[email protected] Avg.RDR L0RDR L1RDR L2RDR Avg.
    Original (Relevant Context)67.3064.0032.9054.8048.7040.7012.0033.8012.3013.5028.8019.10
    Full Policy66.1724.4717.4036.0146.703.903.2017.9312.5131.2938.9627.59
    Difference (pp)-1.13-39.53-15.50-18.79-2.00-36.80-8.80-15.87+0.21+17.79+10.16+8.49
  • Why does Jev's performance drop under full-policy evaluation? Our additional analysis suggests that the main difficulty lies in identifying the relevant exception and conditional rules from the full policy, rather than applying these rules once they are provided.

    • We compared Jev's predictions when the relevant dependencies are explicitly provided versus when the full policy is given. Table 2 below show that Jev can often apply the relevant dependencies when they are explicitly provided, particularly for conditions (87.0% on L1 and 85.9% on L2). However, when given the full policy, performance drops substantially on cases where dependencies change the correct answer. For example, L2 conditional-rule accuracy drops from 85.9% to 28.3%. In contrast, cases where modifiers do not change the outcome show no comparable degradation.
    • A likely explanation is the difficulty of retrieving dependencies from the full policy. In SafePyramid, exception and conditional rules reference the base rules they modify, but base rules do not reference their modifiers. Therefore, when evaluating a target rule, the model must search the full policy to identify which other rules modify it. The relevant-context setting removes this retrieval step by providing these dependencies directly.

Table 2. Jev's predictions on rules affected by exceptions and conditions (violation prediction rate, %).

Rule categoryGround truthRelevant contextFull policy
L1: Exception changes the outcomeNot violated33.659.6
L1: Condition changes the outcomeViolated87.068.1
L2: Exception changes the outcomeNot violated70.688.8
L2: Condition changes the outcomeViolated85.928.3
Control: Modifier does not change the outcome (L1/L2)Not violated9.8 / 32.110.6 / 12.7
Reviewer_nmKVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce a multi-turn benchmark with 1,000 conversations, 3,000 policies, and 61,641 rules across three levels: L0 (individual rules), L1 (exceptions and conditionals), and L2 (fully fictional policy frameworks). They evaluate 10 frontier LLMs, 5 policy-configurable guards, and one typed decision model. The best model, GPT-5.5, exactly matches the full violated-rule set in 54.0%, 35.3%, and 12.9% of cases at L0, L1, and L2.

Critical strengths
  • The problem matters. Separating "judging by a policy" from "judging by a category" and giving a level-wise diagnostic design is useful.
  • Scale and structure are clear. I verified the Table 1 totals: L0 = 7,734 + 8,380 = 16,114; L2 = 7,834 + 8,306 + 6,790 + 9,994 = 32,924; with L1's 12,603, the total is 61,641, matching the abstract. The Figure 2 category totals (15,568, 16,686, 12,353, 17,034) also reconcile.
Critical weaknesses
  • Refusals are excluded from metrics. Refused cases are dropped before computing RMR and RDR (Section 3.6). Models that refuse more often are then scored on easier remaining cases. Refusals should count as failures, or both versions should be reported.

  • The human baseline is under-specified. The paper does not report the number of evaluators, their domain background, time per case, or sample size. Evaluators were allowed to use AI tools to "clarify and interpret" context, so the baseline is not condition-matched to the models. The low L2 human score (41%) may reflect lack of training on the fictional frameworks rather than inherent difficulty.

Questions for authors
  • See weaknesses.

  • How do model rankings change if refusals count as failures?

5 comments
AuthorsOpenReview ↗

Weakness 1 & Question 1: Refusals are excluded from metrics

A1: We sincerely thank the reviewer for this valuable suggestion! In Table 1 below, we report the model ranking changes if refusals count as failures.

  • On one hand, we would like to clarify that a refusal does not necessarily indicate that a model cannot correctly evaluate the case. Refusals are mostly triggered by the model's built-in safety safeguards, rather than by a lack of in-context policy guardrailing capability. Therefore, treating all refusals as failures may not fully reflect a model's actual capability in in-context policy guardrailing.
  • On the other hand, we believe both evaluation settings provide useful but different perspectives: excluding refusals focuses on performance when models provide valid responses, while counting refusals as failures also reflects their ability to complete the evaluation task. So again, we sincerely thank the reviewer for this valuable suggestion!

Table 1. Per-policy model performance and ranking changes when refusals are counted as failures (sorted by updated RMR).

ModelOriginal RMR (Rank)Refusal-as-Failure RMR (Rank)RMR Rank ChangeOriginal [email protected] (Rank)Refusal-as-Failure [email protected] (Rank)[email protected] Rank Change
Claude-Opus-4.752.87 (2)52.38 (1)↑134.98 (1)34.66 (1)—
Kimi-K2.652.28 (3)51.38 (2)↑131.27 (4)30.73 (3)↑1
GPT-5.554.25 (1)49.95 (3)↓234.90 (2)32.13 (2)—
Gemini-3.5-Flash49.61 (6)49.61 (4)↑228.10 (7)28.10 (7)—
DeepSeek-V4-Pro52.18 (4)49.46 (5)↓131.70 (3)30.05 (4)↓1
Doubao-Seed-2.0-Pro50.05 (5)49.38 (6)↓129.49 (5)29.10 (5)—
Hunyuan-HY3-Preview49.14 (7)48.75 (7)—27.38 (8)27.16 (8)—
Qwen-3.6-Max-Preview47.36 (8)46.86 (8)—29.20 (6)28.89 (6)—
GLM-5.142.70 (9)40.64 (9)—25.79 (9)24.54 (10)↓1
Grok-4.340.59 (10)40.50 (10)—25.13 (10)25.07 (9)↑1
GPT-OSS-Safeguard-120B23.62 (11)23.62 (11)—13.56 (11)13.56 (11)—
GPT-OSS-Safeguard-20B22.50 (12)22.41 (12)—13.48 (12)13.42 (12)—
AuthorsOpenReview ↗

Weakness 2: The human baseline is under-specified

W2. The human baseline is under-specified. The paper does not report the number of evaluators, their domain background, time per case, or sample size. Evaluators were allowed to use AI tools to "clarify and interpret" context, so the baseline is not condition-matched to the models. The low L2 human score (41%) may reflect lack of training on the fictional frameworks rather than inherent difficulty.

A2: We thank the reviewer for valuable comments! Here is our clarification:

  • Human evaluation setup. Our human baseline involves two human experts, both with backgrounds in AI safety, evaluating approximately 500 unique cases across L0, L1, and L2, with no overlap between human experts. With AI-assisted context understanding, each case takes approximately 30 minutes, corresponding to around 250 person-hours in total.
  • Challenges of establishing a human baseline. Establishing a reliable human baseline for SafePyramid is particularly challenging due to the lengthy contexts and complex policy reasoning involved. Unlike conventional classification tasks, human experts must understand the conversation, carefully examine multiple policy rules, and reason about exceptions, conditions, and unfamiliar policy frameworks before making their judgments. This makes each annotation highly time-consuming. Given these costs, we prioritized covering more unique cases across all three difficulty levels.
  • Use of AI assistance. We allowed human experts to use AI tools to help them understand lengthy policy contexts and clarify unfamiliar concepts, reducing the time required for reading and initial interpretation. However, the final rule-violation judgments were made independently by the human experts, rather than delegated to AI. We believe this setting also reflects modern human workflows, where AI tools are commonly used to assist with complex information-processing tasks while humans remain responsible for the final decisions.
  • Interpretation of the human baseline. We acknowledge that this AI-assisted human evaluation is not strictly condition-matched to the models. Our goal was to provide a practical reference for human performance under a clearly defined evaluation setting, rather than claim an upper bound on human reasoning ability. We will clarify these details and the evaluation conditions in the revised paper.
  • Human performance on L2. We would also like to clarify that L2 is specifically designed to evaluate the ability to understand and apply a previously unseen policy framework based on the information provided in context. Therefore, prior training on these fictional frameworks is not expected for either humans or models. In addition, all information needed to interpret the new framework was provided in the evaluation context, so the task did not require prior knowledge of the fictional framework. Human experts are expected to make their judgments based on these definitions rather than prior familiarity.

We will clarify the human evaluation protocol in the revised paper.

AuthorsOpenReview ↗

Thanks for your valuable feedback!

In the end, we want to thank you for providing this valuable feedback to us, which is always important to hear opinions from other experts in this field. If you feel there are still unclear points regarding our paper, please discuss with us in the author-reviewer discussion phase. If you feel that your concerns are well-addressed, we do hope for an updated rating and new comments (if possible). Thanks again!

Reviewer_iUFKOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a 1000-sample benchmark that tests the ability of models to correctly adapt to new safety policies at runtime. The benchmark is divided into three difficultly levels, and the benchmark emphasizes complex inter-rule dependencies and conditionals within a given safety policy. This paper reveals insights into how safety classifier models that succeeded in previous settings fail in this more complex environment.

Critical strengths
  1. The complexity exhibited in the benchmark with inter-rule dependency and conditionals is well thought out and nuanced.
  2. The three difficulty levels each isolate different aspects of classification failure, such as world knowledge verses failure to follow logical hops, which allows for useful intervention.
Critical weaknesses
  1. Some of the metrics on human validation of the dataset indicate a level of label noise. For example 41% human rule matching rate on the L2 portion of the dataset is notable. A Cohen's kappa score would be helpful.
  2. No variance or confidence intervals were provided for the rule matching rates.
Questions for authors
  1. What was the inter-annotator agreement between human labelers? Can you provide a Cohen's kappa score for the label review?
  2. Was there significant variance between seeds for any of the models evaluated?
  3. Why were different models evaluated in Table 2 and Table 3? Are the models in Table 3 capable of evaluating in the whole-policy format used in Table 2?
  4. Is there any connection between the particular models used to synthetically generate parts of the dataset and their evaluation results?
7 comments
AuthorsOpenReview ↗

Weakness 1 & Question 1: Human Validation and Label Reliability

W1 & Q1. Some of the metrics on human validation of the dataset indicate a level of label noise. For example 41% human rule matching rate on the L2 portion of the dataset is notable. A Cohen's kappa score would be helpful. What was the inter-annotator agreement between human labelers? Can you provide a Cohen's kappa score for the label review?

A1: We thank the reviewer for raising this important concern! We would like to clarify a few points:

  • Human validation is different from the human baseline. During dataset construction, human experts reviewed the generated policies and ground-truth labels using detailed, predefined schemas for each rule. This makes the validation task more straightforward, as experts can check whether each rule and its label follow the specified requirements. In contrast, the human baseline requires annotators to independently identify violated rules from the full policy and conversation, without access to these schemas or ground-truth labels. Therefore, the 41% human baseline RMR does not directly indicate ground-truth label noise.

  • Human inter-annotator agreement. Establishing human validations for SafePyramid is particularly challenging due to the lengthy contexts and complex policy reasoning involved. Specifically, annotators must understand the conversation, carefully examine multiple policy rules, and reason about exceptions, conditions, and unfamiliar policy frameworks before making their judgments. This makes each annotation highly time-consuming. Given these costs, we prioritized covering more unique cases across all three difficulty levels. As a result, during human validation, experts reviewed non-overlapping cases to maximize dataset coverage. Therefore, we cannot compute Cohen's Kappa from the existing human review records, as this requires independent annotations of the same cases. Across the reviewed samples, human annotators achieved over 90% agreement across all levels with the final labels determined by majority voting among the LLM-based validators.

  • Agreement among LLM validators. To provide additional evidence of label consistency, we analyzed the agreement among the three LLM validators used during ground-truth construction. Since Cohen's Kappa measures agreement between two raters, we use Fleiss' Kappa to measure agreement among all three validators, along with their unanimous agreement rate. As shown in Table 1, agreement is high on L0 and L1, while L2 shows greater disagreement, particularly on framework-dependent rules. Importantly, despite this greater disagreement, the final labels determined by majority voting among LLM validators still achieved over 90% agreement with human experts during validation.

    Table 1. Agreement among three LLM validators (Fleiss' Kappa; unanimous agreement in parentheses).

    LevelFleiss' KappaUnanimous agreement
    L00.87691.5%
    L10.85890.6%
    L20.68276.8%

We will clarify the distinction between human validation and the human baseline, report these additional agreement statistics, and discuss the limitations of our current validation procedure in the revised paper.

AuthorsOpenReview ↗

Weakness 2 & Question 2: No variance or confidence intervals

W2 & Q2. No variance or confidence intervals were provided for the rule matching rates. Was there significant variance between seeds for any of the models evaluated?

A2: We thank the reviewer for this valuable suggestion! We agree that reporting variance and confidence intervals would provide a more complete picture of model performance. We provide the following clarifications and additional analyses:

  • Computational cost and limitations. Evaluating SafePyramid is computationally expensive and time-consuming, particularly for frontier API models. For example, a single full evaluation of GPT-5.5 costs over $1,000. Therefore, we reported results from a single run for most models. We acknowledge that this limits our ability to assess run-to-run variance across all evaluated models.

  • Run-to-run variance. To investigate whether model performance varies significantly across runs, we conducted three independent full evaluations of Jev over all 3,000 cases using the decomposed evaluation protocol. The results are shown in Table 1. The results show relatively small run-to-run variation. Overall RMR varies by only 0.19 percentage points across the three runs, and the maximum variation across all metrics and levels is 1.30 pp. These variations are much smaller than the performance gaps between difficulty levels, particularly between L1 and L2.

    Table 1. Jev performance across three independent runs (mean ± standard deviation, with range in parentheses; percentage points).

    LevelRMR ↑[email protected] ↑RDR ↓
    ALL54.51 ± 0.10 (0.19)33.39 ± 0.34 (0.67)19.24 ± 0.05 (0.09)
    L066.75 ± 0.56 (1.02)47.80 ± 0.70 (1.30)12.51 ± 0.19 (0.35)
    L164.34 ± 0.41 (0.80)40.97 ± 0.57 (1.10)13.39 ± 0.06 (0.12)
    L232.45 ± 0.18 (0.33)11.40 ± 0.53 (1.00)28.95 ± 0.06 (0.10)
  • Confidence intervals. We also computed case-level bootstrap 95% confidence intervals (2,000 bootstrap resamples) for the evaluated models (as shown in Table 2 below). The confidence intervals have typical half-widths of approximately ±1.4 pp for RMR and ±1.8 pp for [email protected]. These results indicate that small differences between closely ranked models should be interpreted cautiously, while the substantial performance degradation across difficulty levels remains much larger than the estimated sampling uncertainty.

    Table 2. Model performance with case-level bootstrap 95% confidence intervals (2,000 bootstrap resamples). All values are percentages.

    ModelRMR ↑ (95% CI)[email protected] ↑ (95% CI)RDR ↓ (95% CI)
    GPT-5.554.25 [52.9, 55.6]34.90 [33.1, 36.6]19.73 [18.9, 20.5]
    Claude-Opus-4.752.87 [51.4, 54.3]34.98 [33.1, 36.7]23.86 [22.9, 24.8]
    Kimi-K2.652.28 [50.9, 53.7]31.27 [29.6, 33.0]21.42 [20.7, 22.2]
    DeepSeek-V4-Pro52.18 [50.8, 53.6]31.70 [30.0, 33.5]21.66 [21.0, 22.4]
    Doubao-Seed-2.0-Pro50.05 [48.7, 51.5]29.49 [27.9, 31.1]22.10 [21.4, 22.8]
    Gemini-3.5-Flash49.62 [48.4, 50.9]28.10 [26.6, 29.7]23.31 [22.5, 24.0]
    Hunyuan-HY3-Preview49.14 [47.8, 50.4]27.38 [25.8, 28.9]22.62 [21.9, 23.4]
    Qwen-3.6-Max-Preview47.36 [46.0, 48.7]29.20 [27.5, 30.7]29.41 [28.4, 30.5]
    GLM-5.142.70 [41.2, 44.1]25.79 [24.1, 27.4]33.47 [32.4, 34.5]
    Grok-4.340.59 [39.1, 42.0]25.13 [23.6, 26.6]35.99 [35.0, 37.1]
    GPT-OSS-Safeguard-120B23.63 [22.3, 24.9]13.56 [12.3, 14.8]51.85 [50.9, 52.8]
    GPT-OSS-Safeguard-20B22.50 [21.2, 23.8]13.48 [12.2, 14.8]52.55 [51.6, 53.5]
AuthorsOpenReview ↗

Question 3: Differences between evaluation settings

Q3. Why were different models evaluated in Table 2 and Table 3? Are the models in Table 3 capable of evaluating in the whole-policy format used in Table 2?

A3: We thank the reviewer for this question. The different model selections in Table 2 and Table 3 mainly reflect the different input and output requirements of the two evaluation settings:

  • Per-policy evaluation (Table 2): The model receives the complete conversation and policy, including all rules, exceptions, conditions, and any policy framework. It must identify all violated rules in a single response. For example, given a policy containing 10 rules, if Rules 2 and 7 are violated, the model should output [2, 7].
  • Per-rule evaluation (Table 3): The model evaluates each rule separately, making a binary decision (Violated or Not Violated) for each target rule. For the same example, the model makes 10 separate decisions, with Rules 2 and 7 labeled as Violated and the remaining rules as Not Violated. These individual predictions are then aggregated into a set of violated rules (e.g., [2, 7]), which is compared against the ground truth using the same evaluation metrics as in per-policy evaluation.

The models differ between Table 2 and Table 3 because not all evaluated models support both evaluation settings. General-purpose API LLMs can perform both whole-policy and per-rule evaluation. However, most policy-configurable guardrails in Table 3 are designed to produce binary safety decisions and cannot directly identify all violated rules in the whole-policy format used in Table 2. Among the policy-configurable guardrails we evaluated, only the GPT-OSS-Safeguard family supports both settings, which is why it appears in both tables.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.