acceptodds
Under review as a conference paper at ICLR 2027

When Safety Judges Agree Too Much: Correlation Breaks Naive Fusion and Caps Its Gains

Abstract

Deployed LLM systems increasingly run several safety guards and combine their verdicts, either by multiplying their evidence or by escalating from a cheap guard to expensive ones. We give a statistical account of how the guards' agreement, together with their individual strength, governs what such combinations can and cannot do. Across seven public guards from four families and three human-labelled benchmarks, guards are strongly correlated, and on WildGuardTest and BeaverTails all of them fail together on 12–13% of unsafe responses, against at most 0.03% under independence. We prove that this correlation silently breaks naive fusion: the product of per-guard likelihood ratios is no longer an e-value, correlation inflates its null mean and log-variance by closed-form factors, and its false-alarm rate tends to a positive floor as more equicorrelated guards are added, for any fixed nominal level; empirically, the common escalation cascade exceeds a nominal 5% false-alarm rate in 6 of 7 settings, reaching 26%. Correlation can also cap the gain of fusion, and within a Gaussian-copula model does so quantitatively: a single quantity, the information the remaining guards add to a cheap one, bounds both what any combination rule, including adaptive cascades of arbitrary cost, can gain and what calling only the cheap guard loses. On real scores we use the model for gap sizes and rankings, not as a certified ceiling. We complement this with a cascade construction whose false-alarm rate is controlled in finite samples for any calling rule fixed before calibration and any dependence, using about 100 target-domain safe examples exchangeable with future safe traffic. Across seven in-domain and shifted settings, a static subset chosen on source data, at about a tenth of the full ensemble's parameter memory, retains nearly all of its detection at a calibrated 5% false-alarm rate, though less at stricter levels. An adaptive dependency-aware cascade does not consistently improve on it, and, for these seven guards and under the same target calibration, the largest guard alone is never more than 0.001 below a calibrated fusion of all seven in detection, in point estimates.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM safety, guard fusion, correlated classifiers, e-values, Neyman–Pearson detection, conformal calibration

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MKGUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Deployed LLM systems utilize multiple safety guards, combining their outputs through statistical methods. Analysis shows a strong correlation among guards, leading to a collective failure rate of 12–13% on unsafe responses, compared to 0.03% when considered independently. This correlation disrupts naive fusion methods, inflating false-alarm rates beyond nominal levels, with practical rates exceeding 5% in six out of seven scenarios. Additionally, the model suggests that the effectiveness of combining guards is limited by their correlation, impacting potential detection gains. An adaptive cascade approach does not consistently outperform a simpler static subset of guards, which maintains nearly all detection capabilities at a controlled false-alarm rate of 5% with lower memory requirements.

Critical strengths
  • The manuscript provides strong theoretical contributions.
  • Also, the authors conduct an extensive and careful empirical evaluation.
  • The paper shows that safety guards are strongly correlated, not independent evidence sources.
Critical weaknesses
  • The findings may not generalize to newer guards, different architectures, or guards that are less correlated.
  • The paper does not measure latency and prompt length.
Questions for authors
  1. This is a strong and timely paper. Its main contribution is not a new fusion algorithm but a rigorous diagnosis of why common fusion practices fail, plus a practical finite-sample calibration method that restores false-alarm control under weak assumptions.

  2. My only concern is the experimental dataset. A wider evaluation will enhance the reliability of the findings.

Reviewer_NsKoOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether several safety guards perform better than one. It finds small gains from combining the tested guards and proposes a theoretical limit on those gains. It also explains excess false alarms and gives a way to control them using safe examples from the target setting.

Critical strengths
  1. The experiments give a useful practical result. For the guards tested, adding more guards increases cost with little improvement in detection. The released scores let others check this finding.

  2. The paper explains why treating similar guard scores as independent can cause too many false alarms. It gives a way to control them using safe examples sampled in the same way as future safe inputs. This guarantee does not rely on the statistical assumptions used to predict gains in detection.

Critical weaknesses
  1. The experiments show that combining the tested guards adds little. The theory makes a stronger claim about how much any combination could gain, under specific statistical assumptions. On BeaverTails, some single guards exceed the predicted limit. The performance order of the Qwen3Guard models is also reversed (Table 1):

    • Predicted: 0.6B performs best, followed by 4B, then 8B (BT dk2d_k^2 column: higher values predict better detection).

    • Measured: 8B performs best, followed by 4B, then 0.6B (BT "Emp." column: higher values mean more unsafe responses detected).

    This suggests that the assumptions do not fit these data well, so the predicted limit needs caution when applied to real guards. The title and conclusion should make that limitation clear. It would also help to show whether the theory selects better guards than a simple comparison on validation data.

  2. Some shared failures need a closer check of the labels. On the WildGuardTest subset where only two of three annotators agreed on the unsafe label, all seven guards miss 21.1% of responses (Table 13). This disagreement does not prove that the labels are wrong. However, checking a sample would help show whether these cases are guard errors or reflect ambiguity in the labels.

  3. The recommendation to use a single guard needs broader checks. Tests on data from different sources are useful, but they reuse a small set of benchmarks and guard families. Under another safety policy, one guard might catch what another misses. Many responses also receive equal scores. The paper should check whether different ways of handling these ties change the reported detection rates.

  4. The main ideas are hard to find in the abstract and introduction, especially the abstract. Please explain the problem and main takeaway in plain language before giving technical results. Keep the experimental findings separate from the theoretical guarantees.

Questions for authors
  1. Please try to address weaknesses 1, 2, and 4.

  2. Does combining guards still offer little benefit on another dataset or under a different safety policy? Also, when several responses receive the same score, does the way you handle them affect the reported detection rates?

Reviewer_XMKaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how correlated errors among LLM safety guards can cause naive ensemble methods to produce excessive false alarms while offering little improvement in harmful content detection. It develops theoretical bounds on the benefits of combining guards and proposes a calibration method that controls false alarms, showing that small subsets of guards can achieve performance close to much larger ensembles.

Critical strengths
  • The paper identifies a practical safety problem by showing that combining multiple guards can substantially increase false alarms because their errors are correlated.

  • The paper combines theoretical analysis with experiments on real safety guards and provides a calibration method with a formal false alarm guarantee.

Critical weaknesses
  • The main theoretical results rely on a Gaussian model that does not accurately describe the actual guard scores. The paper even reports cases where individual guards outperform its predicted upper bound, raising concerns about how useful the theory is in practice.

  • The proposed dependency-aware cascade does not consistently outperform a simple static subset and often achieves lower detection than the dependency-blind baseline. This makes it difficult to justify the additional complexity of the proposed approach.

  • The evaluation only covers a small collection of existing safety guards and three benchmarks, without testing real adversarial attacks designed to exploit shared weaknesses. The reported false alarm guarantee also assumes that calibration examples match future safe inputs, which may not hold in real deployments where attackers and content distributions change.

Questions for authors
  • Can the authors demonstrate that their theoretical bounds remain useful when the actual guard scores violate the Gaussian assumptions?

  • What practical advantage does the proposed dependency-aware cascade offer over a simple static subset when both detection performance and inference cost are considered?

  • How well does the proposed calibration method maintain its false alarm rate and detection performance under new adversarial attacks or distribution shifts that occur after calibration?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.