acceptodds
Under review as a conference paper at ICLR 2027

A Dual-Hypothesis Reasoning Framework for LLM Guardrails

Abstract

We propose ARBITER, a novel LLM guardrail framework that introduces two novel ideas: (i) dual-hypothesis reasoning, a reasoning method for LLM guardrails that explicitly reasons over both safe and unsafe interpretations of a prompt before making a safety decision, and (ii) multi-component supervised fine-tuning (MC-SFT), a em structured training loss for reasoning-based guardrails that decomposes LLM outputs into logical components and weighs them by importance. Existing reasoning-based guardrails often rely on expensive procedures, such as generating reasoning traces from larger or closed-source teacher models and applying full-parameter fine-tuning. In contrast, ARBITER uses a cost-effective self-generation strategy for reasoning traces and LoRA-based parameter-efficient fine-tuning, while still achieving better performance than these expensive approaches. Additionally, ARBITER provides faithful evidence-phrase explanations for unsafe decisions, enabling a more transparent and interpretable guardrail method. Experiments on three safety moderation benchmarks show that ARBITER outperforms existing reasoning and non-reasoning guardrail baselines, with clear gains in out-of-domain evaluations.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Guardrails, Safety Moderation, LLM reasoning, Explainable AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4UBTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper proposes ARBITER, a reasoning-based LLM guardrail framework that aims to improve safety classification and interpretability. Its main contributions are dual-hypothesis reasoning, which generates and compares both safe and unsafe interpretations of a prompt before making a decision, and multi-component supervised fine-tuning, which assigns different weights to reasoning, classification, explanation, and formatting losses. The framework also uses self-generated reasoning traces and LoRA-based fine-tuning to reduce training costs without relying on larger teacher models. Experiments on three safety benchmarks demonstrate competitive performance against existing guardrails, particularly in out-of-domain settings.

Critical strengths
  • The paper addresses an important and practical problem of improving the reliability and interpretability of LLM guardrails. In particular, considering both safe and unsafe interpretations is a reasonable approach to handling ambiguous prompts, where relying on a single reasoning path may lead to incorrect safety decisions.

  • The experimental evaluation is fairly comprehensive, covering multiple safety benchmarks, both in-domain and out-of-domain settings, and explanation quality. ARBITER achieves competitive results across these evaluations, with particularly promising out-of-domain performance (Tables 1 and 2). The use of self-generated reasoning traces and parameter-efficient fine-tuning is also a practical aspect of the approach.

Critical weaknesses
  • The proposed framework includes several components, but their individual contributions are not sufficiently supported by the ablation results. Dual reasoning with standard SFT underperforms no reasoning in most of the settings, while its gains over simple reasoning with MC-SFT are relatively modest (Table 3). Similarly, Table 4 shows that removing the winner loss or auxiliary alignment sometimes improves performance. While the full model performs well overall, these results leave it unclear whether all the proposed components are necessary to achieve the reported improvements.

  • ARBITER incurs nearly 10× higher inference latency than the non-reasoning baseline (Table 11). This is a substantial overhead for a guardrail model. Given that dual reasoning provides only modest improvements over simple reasoning with MC-SFT (Table 3), the paper would benefit from a direct latency comparison between these variants and a clearer analysis of whether the additional computational cost is justified by the performance gains.

Questions for authors
  • A controlled comparison with existing reasoning-based guardrails using the same training data, backbone, and fine-tuning setup would help establish the benefits of the proposed approach.

  • Additional comparisons are needed between simple and dual reasoning under matched inference budgets, as well as between MC-SFT and standard SFT with only label/explanation re-weighting.

4 comments
AuthorsOpenReview ↗

Author’s Response to Reviewer 4UBT (Part 1)

We thank the reviewer for their feedback. We address each point in detail below. Due to OpenReview’s character limits, our full response is divided into multiple comments. If the reviewer is satisfied with our rebuttal, we respectfully request that the reviewer reconsider their score after reviewing our responses.

Weakness 1: The proposed framework includes several components, but their individual contributions are not sufficiently supported by the ablation results. Dual reasoning with standard SFT underperforms no reasoning in most of the settings, while its gains over simple reasoning with MC-SFT are relatively modest (Table 3). Similarly, Table 4 shows that removing the winner loss or auxiliary alignment sometimes improves performance. While the full model performs well overall, these results leave it unclear whether all the proposed components are necessary to achieve the reported improvements.

Our response:

We thank the reviewer for this comment. We respectfully disagree that the contributions are not sufficiently supported by the ablation results. The reviewer raised two points in this comment. First, we would like to respond to the following comment: “Dual reasoning with standard SFT underperforms no reasoning in most of the settings, while its gains over simple reasoning with MC-SFT are relatively modest (Table 3)”. The weaker performance of dual reasoning with standard SFT is not a weakness of our proposed method. Rather, it directly motivates the need for MC-SFT. Our argument for introducing MC-SFT is that reasoning can improve guardrail performance, but simply adding long reasoning traces and training them with standard SFT can be counterproductive. Under standard SFT, cross-entropy is averaged over all target tokens. Because the reasoning trace contains many more tokens than critical components such as the final safety label, the reasoning tokens contribute disproportionately to the overall training loss, while the supervision from the final safe/unsafe decision is diluted. MC-SFT is specifically designed to address this issue by dividing the structured output into components, averaging the token-level loss independently within each component, and then combining these component losses into the final loss. The ablation results strongly support this motivation. When comparing the same reasoning strategy with and without MC-SFT:

  • For simple reasoning, MC-SFT outperforms standard SFT in 17 of 18 comparisons, with an average improvement of 8.58 F1 points.
  • For dual-hypothesis reasoning, MC-SFT also outperforms standard SFT in 17 of 18 comparisons, with an average improvement of 5.93 F1 points. Thus, the reviewer’s observation about the weaker performance of dual reasoning with standard SFT actually identifies the exact problem that MC-SFT is designed to solve. To isolate the contribution of dual-hypothesis reasoning, the appropriate comparison is between simple reasoning and dual-hypothesis reasoning while keeping MC-SFT fixed. Under this controlled comparison, dual-hypothesis reasoning improves 13 of 18 comparisons, with an average improvement of 1.09 F1 points. The improvement is stronger under distribution shift: it improves 10 of 12 out-of-domain comparisons, with an average gain of 1.39 F1 points. For out-of-domain prompt classification specifically, dual-hypothesis reasoning improves all 6 of 6 settings, with an average improvement of 1.68 F1 points.

We can also compare the complete proposed approach, dual-hypothesis reasoning + MC-SFT, against the no-reasoning baseline. ARBITER improves 15 of 18 comparisons, with an average improvement of 1.69 F1 points. In the out-of-domain settings, it improves 11 of 12 comparisons, with an average gain of 2.03 F1 points. Therefore, Table 3 provides evidence for both contributions: MC-SFT enables effective learning from long structured reasoning traces, while dual-hypothesis reasoning provides additional gains, particularly in out-of-domain evaluation.

(The response continues in the next comment.)

AuthorsOpenReview ↗

Author’s Response to Reviewer 4UBT (Part 2)

The second concern raised by the reviewer is: “Table 4 shows that removing the winner loss or auxiliary alignment sometimes improves performance”. While no individual component improves performance in every dataset setting, looking across Table 4 shows a clear aggregate benefit from retaining these components.

When we compare the ablations with the full ARBITER loss, we can see that:

  • Removing the winner loss reduces performance in 10 of 18 cases.
  • Removing the auxiliary alignment loss reduces performance in 13 of 18 cases.
  • Removing the deliberation–explanation alignment term reduces performance in 13 of 18 cases. The pattern becomes even clearer in the out-of-domain settings, which are a primary focus of our work:
  • Removing the winner loss reduces performance in 9 of 12 out-of-domain cases.
  • Removing the auxiliary alignment loss reduces performance in 10 of 12 cases.
  • Removing the deliberation–explanation alignment term reduces performance in 10 of 12 cases.

Therefore, while individual ablations can occasionally perform better on a particular dataset, removing each proposed component reduces performance in the majority of comparisons, especially in out-of-domain settings. These results provide consistent evidence that the components of the full loss work together to improve robustness under distribution shift and justify our proposed loss design.

Weakness 2: ARBITER incurs nearly 10× higher inference latency than the non-reasoning baseline (Table 11). This is a substantial overhead for a guardrail model. Given that dual reasoning provides only modest improvements over simple reasoning with MC-SFT (Table 3), the paper would benefit from a direct latency comparison between these variants and a clearer analysis of whether the additional computational cost is justified by the performance gains.

Our response: We thank the reviewer for this comment. Simple Reasoning + MC-SFT requires 0.421 seconds/example, compared with 0.664 seconds/example for ARBITER. Thus, dual-hypothesis reasoning introduces an additional latency of 0.243 seconds/example over simple reasoning, which is considerably smaller than the gap between ARBITER and the non-reasoning baseline.

This additional cost is accompanied by substantial performance gains. Compared with Simple Reasoning + MC-SFT, ARBITER outperforms 10 of 12 out-of-domain comparisons. For out-of-domain prompt classification specifically, ARBITER improves all 6 of 6 settings, with an average gain of 1.68 F1 points.

These results demonstrate that dual-hypothesis reasoning introduces a relatively modest additional inference cost over simple reasoning while providing consistent and substantial improvements in out-of-domain performance.

Question 1: A controlled comparison with existing reasoning-based guardrails using the same training data, backbone, and fine-tuning setup would help establish the benefits of the proposed approach.

Our response:

We thank the reviewer for this comment. We would like to clarify that we have already compared our model ARBITER with recent reasoning-based guardrails using the same training datasets, test sets, and evaluation metrics, as reported in Table 1. These include L3.1-8B-Aegis-R, L3.1-8B-WG_MIX-R, and Gemma-3-4B-WG_MIX-R. The first two models use the same Llama-3.1-8B-Instruct backbone as ARBITER. All three models use the same safety training datasets as the corresponding ARBITER models. The evaluations are also conducted on the same test datasets using the same evaluation metrics.

The fine-tuning methodology naturally differs between our method and these existing approaches, as this is one of the key distinctions of our work. These reasoning-based baselines use reasoning traces generated by DeepSeek-R1-671B, a much larger teacher model, followed by full-parameter supervised fine-tuning. In contrast, ARBITER generates reasoning traces using its own pretrained 8B backbone model and trains only LoRA adapters while keeping the backbone parameters frozen. This avoids the computational expense of generating training traces with a large model such as the 671B-parameter DeepSeek-R1 teacher and substantially reduces the number of trainable parameters and training cost.

ARBITER outperforms these baselines in 3 of 4 comparisons. Thus, our method achieves better performance in most comparisons despite using parameter-efficient fine-tuning and without relying on a large external teacher model

Question 2: Additional comparisons are needed between simple and dual reasoning under matched inference budgets, as well as between MC-SFT and standard SFT with only label/explanation re-weighting.

Our response: We have already addressed this in our response to the second weakness above.

If the reviewer finds our rebuttal helpful in addressing the concerns raised, we respectfully ask the reviewer to reconsider the score. Thank you very much for your time.

AuthorsOpenReview ↗

Request to Review Our Rebuttal Before the Discussion Period Closes

Dear Reviewer,

We appreciate the time and effort you have devoted to reviewing our paper. As the discussion period is closing soon, we would be grateful if you could find some time to read our rebuttal and reconsider your score if our responses have addressed your concerns.

Thank you very much for your time and consideration.

Thank you to the authors for their detailed and thoughtful rebuttal. I appreciate the clarifications.

  • [W1] Individual Contributions: Thank you for the clarification. I find the motivation and effectiveness of MC-SFT convincing, particularly regarding the dilution of supervision on critical tokens under naive SFT. However, I remain less convinced of the additional benefit of dual-hypothesis reasoning. Its improvements over simple reasoning with MC-SFT appear relatively modest, especially considering the additional inference cost.

  • [W2] Computational Overhead: Thank you for providing the additional latency measurements. However, I do not agree that the overhead is 'modest' as dual-hypothesis reasoning increases inference latency by approximately 55% over simple reasoning. Moreover, an equal-inference budget comparison is still missing, making it difficult to assess whether the performance gains justify the additional computation.

  • [Q1] Comparison with Existing Guardrails: Thank you for the clarification. I acknowledge that the paper already includes comparisons with reasoning-based guardrails using the same backbone and training datasets. Although these are not fully controlled comparisons due to differences in training procedures, I recognize their value in demonstrating the competitiveness of the proposed framework.

  • [Q2] Additional Controlled Comparisons: The additional latency measurements do not directly address my original request for comparisons under matched inference budgets. Furthermore, the effectiveness of MC-SFT relative to a simpler baseline using only label or explanation loss re-weighting remains unaddressed.

Based on the rebuttal, I will raise the Informativeness and Soundness score from 1 to 2, while keeping my overall rating unchanged.

Reviewer_83tmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors crisply breakdown weaknesses of reasoning based gaurdrail approaches:

  • dependence on large/proprietary LLMs(teacher) models to synthesize reasoning traces
  • reliance on expensive full-parameter fine-tuning
  • unfaithful chain-of-thought explanations
  • a single reasoning trajectory, typically conditioned on the gold label.

Authors propose a solution to handle these biases systematically.

  • Dual-hypothesis reasoning: given only the prompt, the model generates a safe hypothesis, and an unsafe hypothesis. Delibrates and picks a winner and produces an output with evidence-phrase explanation. Hypotheses are self-generated by the same base model, not distilled from a teacher. KV-cache reuse + fixed token budgets keep generation cheap; hypothesis order is shuffled to reduce positional bias.

Key Novelty:

  • The contrastive safe-vs-unsafe hypothesis construction using the same model, without a teacher model.
  • The component-wise loss aggregation (MC-SFT)
  • Combining self-generated reasoning + PEFT to avoid teacher distillation is an effective economical approach.
Critical strengths

Well-motivated paper. The dilution argument for MC-SFT is intuitive and the component-wise fix is elegant. Good generalization results across domains. Faithful evidence-phrase explanations are a concrete improvement over CoT reasoning.

Critical weaknesses

Missing Literature

  • No discussion of constitutional AI / self-critique / debate lines (for ex. multi-agent debate, self-consistency)
  • Output/response moderation is not in scope, the paper only does prompt classification. Most deployed guardrails (Llama Guard, WildGuard) also moderate responses.
  • Several baseline numbers are imported from other papers rather than re-run, with no protocol-matching discussion.
Questions for authors

Faithfulness test is one-sided. Deletion on unsafe true positives only is somewhat circular (removing the literal harmful words naturally lowers unsafe probability) and omits a sufficiency test. The faithfulness of safe decisions is untested. What is the result of faithfulness test on safe decisions?

4 comments
AuthorsOpenReview ↗

Author’s Response to Reviewer 83tm (Part 1)

We thank the reviewer for their feedback. We address each point in detail below. Due to OpenReview’s character limits, our full response is divided into multiple comments. If the reviewer is satisfied with our rebuttal, we respectfully request that the reviewer reconsider their score after reviewing our responses.

Weakness 1: Missing Literature: No discussion of constitutional AI / self-critique / debate lines (for ex. multi-agent debate, self-consistency)

Our response:

We have expanded Section 2 (Related Work) to cite and discuss these approaches and clarify their differences from our framework. We would like to clarify some important differences between these approaches and our work.

First, Constitutional AI primarily focuses on LLM alignment, which modifies the behavior of the LLM itself. In contrast, our work focuses on developing an external guardrail that evaluates prompt safety before passing the prompt to the protected LLM. This is particularly important because unsafe prompts can be blocked before the protected LLM generates a response, avoiding unnecessary computational costs associated with generating responses to harmful requests.

Regarding the reasoning strategy, Constitutional AI uses self-critique and response revision guided by predefined principles. While this approach could also be adapted to guardrail training, it requires a constitution, typically defined by humans, to guide the self-critique process. Similarly, Self-Refine iteratively critiques and revises generated responses, while self-consistency generates multiple reasoning paths and aggregates their predictions. Multi-agent debate involves multiple model instances exchanging reasoning over several rounds. Although these approaches are useful, repeated generations and multi-agent interactions can introduce substantial computational overhead, particularly when applied during inference.

In contrast, ARBITER explicitly generates competing safe and unsafe hypotheses and deliberates between them using a single model, without requiring a separate constitution, iterative self-critique, or multi-agent interactions. This allows the guardrail to systematically consider both interpretations before making a safety decision, which is particularly important for ambiguous prompts.

Weakness 2: Output/response moderation is not in scope, the paper only does prompt classification. Most deployed guardrails (Llama Guard, WildGuard) also moderate responses.

Our response:

LLM safety guardrails can operate at both the input (prompt) and output (response) levels. Our work specifically focuses on pre-generation prompt filtering, which is an important component of LLM safety. In output moderation, the protected LLM generates a response before the guardrail evaluates its safety. If the response is unsafe, the guardrail blocks it, meaning the computational resources spent generating that response are wasted.

In contrast, prompt filtering can detect and block unsafe requests before they reach the protected LLM, preventing unnecessary response generation and saving computational resources. While output moderation provides an additional layer of protection, prompt-level filtering is equally important. Our work aims to improve prompt safety classification through dual-hypothesis reasoning, and extending the framework to response moderation is a potential direction for future work.

Weakness 3: Several baseline numbers are imported from other papers rather than re-run, with no protocol-matching discussion.

Our response:

We thank the reviewer for this comment. We would like to clarify that our evaluation includes a diverse set of baselines, covering both reasoning and non-reasoning guardrails, different model sizes, and different training datasets.

To enable comparisons with as many existing baselines as possible, we trained three versions of ARBITER, one on each of AEGIS2.0, WildGuardMix, and ToxicChat0124. As clearly indicated in Tables 1 and 2, we group baselines according to their training datasets, allowing direct comparisons between models trained on the same dataset. We also include baselines trained on other datasets for broader comparisons. Furthermore, several baselines share the same Llama-3.1-8B backbone as ARBITER, including Llama-3.1 AegisGuard, L3.1-8B-Aegis-R, and L3.1-8B-WG_MIX-R.

Regarding evaluation protocol matching, all compared models are evaluated using the same test splits and examples, the same binary safety-label definitions, and the same evaluation metrics. This ensures consistency in the evaluation settings across the compared methods.

We would also like to clarify that not all baseline results are taken from previous publications. We directly evaluated several baseline models ourselves, including GuardReasoner, Llama Prompt Guard 2, and LEG variants. For the remaining baselines, we use results reported in their original publications under the same evaluation protocol.

AuthorsOpenReview ↗

Author’s Response to Reviewer 83tm (Part 2)

Question 1: Faithfulness test is one-sided. Deletion on unsafe true positives only is somewhat circular (removing the literal harmful words naturally lowers unsafe probability) and omits a sufficiency test. The faithfulness of safe decisions is untested. What is the result of faithfulness test on safe decisions?

Our response:

We thank the reviewer for this comment. In the revised manuscript, we have rewritten Appendix I and extended our faithfulness evaluation from unsafe true-positive examples to all examples predicted as unsafe, including both true positives and false positives. This allows us to evaluate whether the generated explanations are faithful to the model's decisions independently of prediction correctness. We have also extended the evaluation to include both comprehensiveness and sufficiency.

Regarding the reviewer’s observation that “removing the literal harmful words naturally lowers unsafe probability”, this is not a problem with our evaluation method. This is exactly the expected faithfulness behavior in the widely cited faithfulness evaluation method proposed by DeYoung et al. (2020), which is cited in our paper. We followed this established faithfulness evaluation method. However, this deletion-based method only establishes comprehensiveness. We agree with the reviewer that this does not establish sufficiency. Therefore, in the revised manuscript, we have also included a sufficiency evaluation.

We provide a detailed mathematical explanation of how we calculated comprehensiveness and sufficiency for this faithfulness evaluation in Appendix I of the revised manuscript. To summarize it below:

To measure comprehensiveness, we progressively delete more explanation words or phrases and measure the change in probability from the original probability without any deletion. The results in the table below show that as we delete more explanation words or phrases from the prompt, the probability of predicting the unsafe class decreases further. The table shows the amount of reduction in probability, not the raw probability.

Similarly, in the sufficiency test, we keep only the explanation words or phrases and remove all other words or phrases from the prompt. We progressively start with only one explanation word or phrase and then include more explanation words or phrases. The results show that our method can identify the unsafe prompt correctly in the majority of cases, even when we keep only the unsafe words or phrases and remove everything else. As we progressively include more explanation words or phrases, the probability difference from the prediction based on the original prompt decreases. This demonstrates the sufficiency of the explanation words or phrases for the prediction.

We do not evaluate the faithfulness of safe predictions because of our task setup. In a guardrail setup, the main concern is detecting whether a prompt is unsafe or not. An unsafe prediction means that there are some evidence words or phrases that make the prompt unsafe. However, if a prompt is predicted as safe, this means that there are no unsafe words or phrases in the prompt that provide sufficient evidence to make it unsafe. In other words, all words and phrases in this prompt are safe. Therefore, identifying specific explanation words or phrases for a safe prediction does not make sense in our case.

So, we provide an explanation when we predict a prompt as unsafe. When we predict a prompt as safe, we do not extract any explanation words or phrases. However, in our training setup, we use the placeholder “safe prompt” as the explanation for safe prompts to keep the training setup consistent across safe and unsafe prompts. We hope this clarifies why we do not evaluate explanations for safe predictions.

We have uploaded a revised manuscript incorporating the changes mentioned in the rebuttal. All changes in the updated manuscript are highlighted in blue. If the reviewer finds our rebuttal helpful in addressing the concerns raised, we respectfully ask the reviewer to reconsider the score. Thank you very much for your time.

AuthorsOpenReview ↗

Author’s Response to Reviewer 83tm (Part 3)

Training datasetTest datasetComprehensiveness↑Sufficiency↓Delete 1Delete 2Delete 3Delete AllKeep 1Keep 2Keep 3Keep AllAEGIS 2.00.4790.7270.7950.8180.5240.2620.2120.200AEGIS 2.0WildGuardMix0.2800.4340.5400.6340.7520.4850.4130.379ToxicChat01240.4920.6430.7020.7400.5880.3960.3630.301AEGIS 2.00.4490.6630.7210.7420.3460.1570.1390.129WildGuardMixWildGuardMix0.1310.2640.3840.4460.5800.2550.1890.179ToxicChat01240.4800.6230.6820.7070.5050.3050.2700.259AEGIS 2.00.5450.7350.7820.7920.4820.3190.2890.275ToxicChat0124WildGuardMix0.2830.4220.5170.5930.7370.5790.5130.471ToxicChat01240.4060.5420.5880.6340.5670.4540.3780.320\begin{array}{ll|cccc|cccc} \hline \textbf{Training dataset} & \textbf{Test dataset} & \textbf{Comprehensiveness}\uparrow & & & & \textbf{Sufficiency}\downarrow & & & \\\\ & & \textbf{Delete 1} & \textbf{Delete 2} & \textbf{Delete 3} & \textbf{Delete All} & \textbf{Keep 1} & \textbf{Keep 2} & \textbf{Keep 3} & \textbf{Keep All} \\\\ \hline & \text{AEGIS 2.0} & 0.479 & 0.727 & 0.795 & 0.818 & 0.524 & 0.262 & 0.212 & 0.200 \\\\ \text{AEGIS 2.0} & \text{WildGuardMix} & 0.280 & 0.434 & 0.540 & 0.634 & 0.752 & 0.485 & 0.413 & 0.379 \\\\ & \text{ToxicChat0124} & 0.492 & 0.643 & 0.702 & 0.740 & 0.588 & 0.396 & 0.363 & 0.301 \\\\ \hline & \text{AEGIS 2.0} & 0.449 & 0.663 & 0.721 & 0.742 & 0.346 & 0.157 & 0.139 & 0.129 \\\\ \text{WildGuardMix} & \text{WildGuardMix} & 0.131 & 0.264 & 0.384 & 0.446 & 0.580 & 0.255 & 0.189 & 0.179 \\\\ & \text{ToxicChat0124} & 0.480 & 0.623 & 0.682 & 0.707 & 0.505 & 0.305 & 0.270 & 0.259 \\\\ \hline & \text{AEGIS 2.0} & 0.545 & 0.735 & 0.782 & 0.792 & 0.482 & 0.319 & 0.289 & 0.275 \\\\ \text{ToxicChat0124} & \text{WildGuardMix} & 0.283 & 0.422 & 0.517 & 0.593 & 0.737 & 0.579 & 0.513 & 0.471 \\\\ & \text{ToxicChat0124} & 0.406 & 0.542 & 0.588 & 0.634 & 0.567 & 0.454 & 0.378 & 0.320 \\\\ \hline \end{array}

Table: Faithfulness evaluation using comprehensiveness and sufficiency on all prompts predicted as unsafe. Comprehensiveness measures the reduction in unsafe-label probability after deleting explanation phrases, while sufficiency measures the probability difference when retaining only the explanation phrases. Increasing comprehensiveness as more phrases are deleted and decreasing sufficiency as more phrases are retained indicate more faithful explanations.

AuthorsOpenReview ↗

Request to Review Our Rebuttal Before the Discussion Period Closes

Dear Reviewer,

We appreciate the time and effort you have devoted to reviewing our paper. As the discussion period is closing soon, we would be grateful if you could find some time to read our rebuttal and reconsider your score if our responses have addressed your concerns.

Thank you very much for your time and consideration.

Reviewer_WszoOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
2
Soundness
1
Read review
Paper summary

ARBITER is a LLM guardrail that classifies user prompts as safe or unsafe. It has two parts:

  • Dual-hypothesis reasoning: The model first writes a short argument for a benign reading of the prompt, then one for a harmful reading, then a deliberation comparing the two. It then outputs a winner and a JSON label. For unsafe prompts, the JSON also lists the exact phrases that make the prompt unsafe.

  • Multi-component SFT (MC-SFT). The loss is averaged separately over each part of the output (reasoning, winner, label, explanation, JSON formatting) and the parts are then weighted. An auxiliary loss also pulls the deliberation’s hidden states toward the hypothesis that matches the gold label, and toward the explanation for unsafe prompts.

Every model is tested on AEGIS2.0, WildGuardMix and ToxicChat0124 against non-reasoning and reasoning guardrails. The final results are mixed. They don't show clearly that the method is superior to existing methods.

Critical strengths

Training using completions from same model show some gains

Critical weaknesses
  1. The papers method is closely related to https://arxiv.org/pdf/2603.18656. The paper doesn't cite this paper and doesn't position it against the above study. This makes the novelty of the paper limited.
  2. Results don't show consistent improvement against the baseline methods.
  3. The cost of running this method is not properly discussed in the paper
  4. The token-count argument is wrong - It is very general that the loss of initial terms is high as it is guiding the models answering trajectory. Similarly the loss for PII and other such tokens is high as well.
  5. The results of the paper are not consistent. Tables 1–2 and Tables 3, 4, 8 and 13 give different values for what appear to be the same ARBITER models. For example, ARBITER-toxicchat0124 on AEGIS2.0 is 78.86 in Table 1 but 80.06 in Tables 3, 4, 8 and 13. ARBITER-wildguardmix on WildGuardMix is 89.10 in Table 1 but 88.87 elsewhere. This raises an ethical concern regarding the faithfulness of the study.
  6. The paper just repeats the results numbers in the text rather than discussing the impact that the authors draw from those numbers
Questions for authors
  1. Can authors tell the motivation behind using token loss?
10 comments
AuthorsOpenReview ↗

Author’s Response to Reviewer Wszo (Part 1)

We thank the reviewer for their feedback. We address each point in detail below. Due to OpenReview’s character limits, our full response is divided into multiple comments. If the reviewer is satisfied with our rebuttal, we respectfully request that the reviewer reconsider their score after reviewing our responses.

Weakness 1: The papers method is closely related to https://arxiv.org/pdf/2603.18656. The paper doesn't cite this paper and doesn't position it against the above study. This makes the novelty of the paper limited.

Our response:

We thank the reviewer for this comment. We carefully reviewed the suggested paper and would like to clarify the key differences between the two methods. First, the two papers address fundamentally different tasks. The suggested paper focuses on improving reasoning in vision-language models, whereas our work focuses on developing an LLM safety guardrail that classifies input prompts as safe or unsafe and explains the safety decisions by identifying the specific words or phrases in the prompt that contribute to those decisions. Beyond the differences in tasks, there are several important methodological differences:

  • Reasoning structure: Our primary contribution is dual-hypothesis reasoning, which independently generates competing safe and unsafe interpretations before deliberation. In contrast, the suggested paper uses a single reasoning trajectory without explicitly considering competing hypotheses.
  • Training objective: The suggested paper separates the training loss into reasoning and answer segments using dynamically scheduled weights. In contrast, our MC-SFT explicitly supervises five distinct components: reasoning, winner selection, safety label, evidence-phrase explanation, and syntax. Additionally, MC-SFT introduces an auxiliary reasoning-alignment loss that encourages consistency between the competing hypotheses, deliberation, and extracted explanations.
  • Reasoning trace generation and training: The suggested paper relies on reasoning traces generated using external teacher models. In contrast, ARBITER generates its own dual-hypothesis reasoning traces using the pretrained base model and subsequently trains LoRA adapters on these traces while keeping the base model's parameters frozen. This eliminates the need for an external teacher model.
  • Explainability: In addition to reasoning-based explanations, our framework provides evidence-phrase explanations by identifying the specific words or phrases in a prompt that contribute to its unsafe classification. The suggested paper does not provide this type of explanation.

The only similarity between the two methods is loss normalization. Our method normalizes the cross-entropy loss separately for each output component, whereas the suggested paper normalizes the losses separately for reasoning and answer tokens. Beyond this, the overall methodology, reasoning structure, training objectives, and target tasks differ substantially.

Weakness 2: Results don't show consistent improvement against the baseline methods.

Our Response:

We respectfully disagree with this comment. ARBITER demonstrates strong out-of-domain generalization, which is one of the main objectives of our work. Out-of-domain generalization is the most important evaluation criterion for guardrail models because it reflects the conditions these models are likely to encounter in real-world deployment. In Table 1, ARBITER achieves the best prompt classification performance in 5 of 6 out-of-domain comparisons against baselines trained on the same dataset. Similarly, Table 2 demonstrates the best explanation classification performance in 5 of 6 out-of-domain comparisons. ARBITER also achieves the best in-domain prompt classification results on 2 of 3 test sets. This result provides strong evidence supporting the effectiveness of the proposed framework.

Weakness 3: The cost of running this method is not properly discussed in the paper

Our Response:

Thank you for this comment. We would like to clarify that we already provided a dedicated inference-time analysis in Appendix F (Table 11). The appendix is referenced in the last paragraph of Section 5.3 of the main text. ARBITER requires 0.664 seconds per example. Additionally, in the revised manuscript, we have clarified that we used an NVIDIA H100 GPU to train our model (see Section 4).

AuthorsOpenReview ↗

Author’s Response to Reviewer Wszo (Part 2)

Weakness 4: The token-count argument is wrong - It is very general that the loss of initial terms is high as it is guiding the models answering trajectory. Similarly the loss for PII and other such tokens is high as well.

Our Response:

We would appreciate clarification on this comment. Could the reviewer clarify what they mean by "PII tokens"? We do not mention anything like this in our paper. Our paper does not claim that the losses for initial terms are high. We are also unsure what the reviewer means by "initial terms" and would appreciate further clarification. We would appreciate it if the reviewer could clarify these points, and we would be happy to address them.

Weakness 5: The results of the paper are not consistent. Tables 1–2 and Tables 3, 4, 8 and 13 give different values for what appear to be the same ARBITER models. For example, ARBITER-toxicchat0124 on AEGIS2.0 is 78.86 in Table 1 but 80.06 in Tables 3, 4, 8 and 13. ARBITER-wildguardmix on WildGuardMix is 89.10 in Table 1 but 88.87 elsewhere. This raises an ethical concern regarding the faithfulness of the study.

Our Response:

We would like to clarify that there are no errors in our reported results. Tables 1 and 2 present our main results, comparing ARBITER with existing baselines. Therefore, we report the average performance across three random seeds (32, 42, and 52), as already specified in the table captions. We also report the corresponding standard deviations in Table 5. For the ablation studies presented in Tables 3 and 4, we conducted all experiments using a single random seed (32) to ensure consistent comparisons under the same experimental settings. Similarly, Tables 8 and 13 report results from a single random seed (32). Table 13 presents detailed precision, recall, and F1 scores. Reporting results from a single seed is more appropriate here, as it allows readers to directly verify the F1 scores using the corresponding precision and recall values. Therefore, the differences in reported values do not indicate any errors. In the revised manuscript, we have explicitly clarified in the captions of Tables 3, 4, 8, and 13 that the reported results are from models trained using a single random seed (32), to avoid confusion.

Weakness 6: The paper just repeats the results numbers in the text rather than discussing the impact that the authors draw from those numbers

Our Response:

We have revised the Result Analysis section to further highlight the implications of our findings. All changes are highlighted in blue in the revised manuscript. We summarize the key findings below:

  • The results in Tables 1 and 2 demonstrate strong out-of-domain generalization in prompt classification and explanation extraction. We highlight the importance of this finding for real-world guardrail deployment, where models encounter prompts that differ from their training distribution.
  • The results in Table 1 show that strong safety classification performance can be achieved using self-generated reasoning traces and LoRA-based parameter-efficient fine-tuning, without relying on larger external teacher models.
  • The results in Table 2 indicate that ARBITER's ability to identify unsafe evidence phrases generalizes across different safety datasets, extending its out-of-domain performance advantages beyond prompt classification.
  • The ablation results in Table 3 demonstrate that MC-SFT improves over standard SFT in 34 of 36 matched comparisons, while dual-hypothesis reasoning combined with MC-SFT achieves the best results in 8 of 12 out-of-domain settings. These findings highlight how the reasoning structure and training objective complement each other to improve robustness under distribution shift.

These findings provide important insights into the effectiveness of the proposed framework beyond the reported performance numbers.

Question 1: Can authors tell the motivation behind using token loss?

Our Response:

We use token-level cross-entropy loss because it is the standard loss function for supervised fine-tuning of autoregressive LLMs. Our contribution is not the use of token loss itself, but rather the MC-SFT formulation, which computes the average loss independently for five distinct output components and combines them into a joint training objective with an auxiliary reasoning-alignment loss. Component-wise averaging prevents longer output components from dominating the training objective solely due to their token counts, while the auxiliary loss encourages consistency between the reasoning, final decision, and explanation.

We have uploaded a revised manuscript incorporating the changes mentioned in the rebuttal. All changes in the updated manuscript are highlighted in blue. If the reviewer finds our rebuttal helpful in addressing the concerns raised, we respectfully ask the reviewer to reconsider the score. Thank you very much for your time.

Thank you for the response and the revised manuscript. My replies follow the order of my original weaknesses.

W1 (Related work): Addressed

W2 (Results): The gains are mostly small relative to seed variance (Table 5):

  • In-domain WildGuardMix: 89.10 ± 0.20 vs. GuardReasoner-8B 89.02. In-domain ToxicChat: 82.30 ± 0.39 vs. ToxicChat-T5 82.21. Both are within one standard deviation.
  • Four of the five out-of-domain wins are only against LEG (an encoder model), since the strong reasoning baselines report no ToxicChat results. On WGM→AEGIS, ARBITER (81.85) ranks 5th of the 6 WildGuardMix-trained models.
  • The key ablation is weak: under MC-SFT, dual reasoning vs. simple reasoning differs by only +0.32 / −0.35 / −1.15 in-domain (Table 3, single seed). A “No reasoning + MC-SFT” baseline is also missing.

W3 (Cost): Thank you for the added baseline. ARBITER is about 10× slower (0.664 vs. 0.067 s/example), much more than the overhead reported by Sreedhar et al. (2025), while the in-domain gains over no reasoning are +0.9 to +3.2 F1. Please discuss whether the gain justifies this cost, and specify the non-reasoning baseline.

W4 (Token-count argument): Under standard SFT, tokens get equal weight, not equal loss. Since the reasoning traces are self-generated by the same base model (as Section 3.2.1 itself states), those tokens already have low loss and shouldn’t dominate training. The high-loss tokens are the short, decision-critical ones (winner, label, evidence phrases). Table 6 also shows the chosen weights (87.93) score below equal weights (88.11) and below 23 of the 28 tested settings, so the “weighting by importance” claim isn’t supported. Please report each component’s loss share under standard SFT, and compare against a simple baseline that up-weights only the label tokens.

W5 (Inconsistent results): Thank you for the clarification on 3-seed vs. single-seed results

W6 (Discussion): Addressed

Q1 (Motivation for token loss): Answered. See W4 for my remaining concern.

Updated assessment: My concerns about consistency and presentation are resolved, and I am raising my score. However, I still lean toward rejection because of the above remaining concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.