acceptodds
Under review as a conference paper at ICLR 2027

D-BISR: Multi-Prototype Diffusion Residual Scoring for Target-Data-Free Harmful Prompt Detection

Abstract

Large Language Models deployed in open-ended systems require input-side harmful prompt detection to block safety risks before generation. In practice, harmful prompts in target applications often differ from training data, and target-domain annotations are difficult to obtain. Detectors should therefore generalize to unseen domains without target-domain samples. While encoder-based models offer efficient input filtering, they suffer from local risk dilution, risk-pattern entanglement, and classification scores that fail to support stable low-FPR detection. To address these challenges, we propose D-BISR, an encoder-based framework for target-data-free harmful prompt detection. D-BISR uses token-level prototype attention to aggregate local harmful evidence and introduces multiple learnable risk prototypes to separate distinct risk patterns. Instead of relying on standard classification logits, it performs diffusion residual scoring to map risk evidence into a continuous, thresholdable safety metric. We evaluate D-BISR on ViSU, MMA, and SneakyPrompt under strict target-data-free settings. When trained solely on source-domain data and tested directly on unseen target domains, D-BISR achieves strong cross-dataset generalization and maintains stable low-FPR recall. On standard in-domain benchmarks, D-BISR outperforms existing encoder-based classifiers and anomaly detection baselines. Compared with larger safety guard models, D-BISR uses fewer parameters, less GPU memory, and achieves lower inference latency.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Harmful Prompt Detection, Target-Data-Free Generalization, Multi-Prototype Risk Modeling, Diffusion Residual Scoring, Low-FPR Detection

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_XijhOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes D-BISR, an encoder-based harmful prompt detector meant to transfer to unseen datasets without target data. On top of a CLIP text encoder, K=8 learnable prototypes attend over token features, and their outputs are fused with the global embedding. The fused vector is scored by a single-step denoising residual trained with three losses: a safe-only residual loss, a pairwise safe/unsafe ranking loss, and a prototype diversity loss (§4, Eq. 16, p.7). The threshold is chosen on source calibration data and applied unchanged to targets. Validation consists of in-domain and cross-dataset results on ViSU, MMA, and SneakyPrompt (Tables 1–2, p.7), a component ablation on SneakyPrompt (Table 3, p.8), and efficiency and seed-stability tables (Tables 4–5, p.9).

Critical strengths
  1. Clear gains on SneakyPrompt. In-domain AUROC is 0.9538 vs. 0.8846 for the best baseline (LEG), and TPR@FPR5 is 0.8397 vs. 0.2628 (CLIP-Linear) (Table 1, p.7). D-BISR also has the best TPR@FPR5 in all four transfer directions involving SneakyPrompt (Table 2, p.7). This is the main evidence for the paper’s claims, but it rests on roughly 156 unsafe test prompts and does not extend to ViSU or MMA, so it does not carry my assessment.
  2. A deployment-realistic protocol. Thresholds are selected on source calibration data and fixed for testing, and test splits are length-balanced (§5.1, p.7–8). This makes the low-FPR numbers meaningful, which is also why the gaps in baselines and benchmarks below matter.
Critical weaknesses
  1. The benchmarks, baselines, and related work do not support the paper’s claims.
  • Benchmarks. The paper motivates input filtering for LLMs (§1, p.1), but all three benchmarks are text-to-image NSFW or jailbreak prompt sets (§5.1, p.7), so its LLM claims are untested.
  • Missing encoder guards. Beyond three LLM guards, the only detectors discussed are HyPE and LEG (§2, p.2–3). Lightweight encoder guards with public checkpoints are missing, such as HarmAug (ICLR 2025) and Meta’s Prompt Guard and Llama Prompt Guard 2. These are the same kind of model as the paper’s own DeBERTa-v3 prompt-injection baseline.
  • Missing guards for unseen attacks. Guardrails designed to adapt to new attacks without retraining, such as Retrieval-Augmented Defense (ACL 2026), are not discussed.
  • Missing text-to-image guards. Given the actual text-to-image setting, Latent Guard (ECCV 2024) and GuardT2I (NeurIPS 2024) are the direct competitors, and they are neither cited nor compared.
  • Baseline choice. No rationale is given for the baselines, and they differ by table. Table 1 omits DeBERTa-v3, and Table 2 omits HyPE and LEG.
  • BingoGuard. BingoGuard-8B appears only in the latency table (Table 4, p.9). Yet §5.5 asserts it “achieves strong AUROC” with no number reported, so the accuracy–efficiency trade-off claim is unsupported.
  • Unclear gains. HyPE has higher F1 on ViSU and MMA (0.9681 vs. 0.9592; 0.9537 vs. 0.9502; Table 1), contrary to §5.2. The MMA TPR@FPR5 margin over HyPE (0.0051) is within the seed std of 0.0118 (Table 5). In MMA→ViSU, D-BISR trails the same-encoder CLIP-L (AUROC 0.7079 vs. 0.8227; Table 2).
  1. The method’s contribution is neither clearly explained nor isolated.
  • Readability. The paper is hard to read. Heavy notation and new labels for familiar effects obscure a simple pipeline. For example, “risk-mode entanglement” (§3.2, p.4) means that unsafe prompts form several clusters, and Eq. 2 is the k-means between-cluster variance identity.
  • Figures and tables. Figures 1–2 are low-resolution raster images with illegible labels and legends (p.3–4). Table 2’s rules overlap its rows (p.7), and Tables 1 and 3–5 mark no best results. “D-BISR” is never expanded, and the dataset definition in §4 is garbled (p.5).
  • Scorer not isolated. The “diffusion” scorer adds noise to one fused vector and uses the noise-prediction error at a single step as the score (§4.4–4.5, p.6–7). Removing the ranking loss drops AUROC to 0.5276 (Table 3, p.8), so the residual carries no safe/unsafe signal on its own. The “w/o diffusion” variant changes the scorer and the loss at once, so whether denoising adds anything over a ranking-trained MLP head is untested.
  • Missing details. M, t, and the noise schedule are not reported, and the ablations cover only SneakyPrompt.
Questions for authors

Clarifications:

  1. How are safe prompts sourced, and how are safe/unsafe pairs built, for MMA and SneakyPrompt, which have no natural pairs?
  2. How were the baselines chosen, and why do they differ between Tables 1 and 2? Was the DeBERTa-v3 prompt-injection model fine-tuned on the source data, and what is the BingoGuard-8B AUROC referred to in §5.5?
  3. LEG and HyPE report a SneakyPrompt TPR@FPR5 of 0.0001 (Table 1), which is unattainable with roughly 156 unsafe test prompts. How were these values computed?
  4. What are M and t at inference, and how much does TPR@FPR5 vary across noise draws?

Suggestions:

  1. Compare against an MLP head on the same fused features trained with the same ranking loss, on all datasets with several seeds.
  2. Either re-scope to text-to-image and compare with Latent Guard and GuardT2I, or evaluate on LLM harmful-prompt benchmarks against HarmAug and the Prompt Guard models.
2 comments

We thank the reviewer for the constructive feedback. Regarding benchmark relevance, D-BISR evaluates textual prompts only, without generating images or accessing visual features. Although ViSU, MMA, and SneakyPrompt originate from text-to-image safety settings, their inputs are textual prompts. We will explicitly limit our claims to text-prompt risk screening rather than general LLM or image safety. We will also discuss HarmAug, Prompt Guard, Llama Prompt Guard 2, and Retrieval-Augmented Defense, clarifying their different objectives and protocols. Their published results cannot be directly compared under our source-only cross-domain evaluation.

Regarding novelty, we do not claim new attention operators, diffusion algorithms, or ranking losses individually. Our contribution is their joint design for target-data-free detection: token-level multi-prototype attention captures localized risk cues, global–local fusion preserves context, and safety-aligned diffusion residual scoring with pairwise ranking produces scores calibrated solely on source-domain safe samples.

Regarding baselines, Table 1 evaluates within-dataset detection, while Table 2 evaluates source-to-target transfer. HyPE and LEG were excluded from Table 2 because their official implementations could not be aligned with our strict source-only training and fixed-threshold protocol. DeBERTa-v3 was initialized from a public prompt-injection checkpoint and fine-tuned for three epochs on each source domain. We will clarify these settings.

Regarding BingoGuard-8B, additional evaluation on the same target prompts yields AUROC values of 0.9784/0.9836/0.9826 on ViSU/MMA/SneakyPrompt. BingoGuard has approximately 8.03B parameters, versus 140.23M for D-BISR (57.3×). We will present it as a large-scale system reference, not a capacity-matched baseline.

Regarding performance, Table 2 reports the following macro-averages across six transfer directions:

MethodAUROCTPR@FPR5F1
D-BISR0.89750.75490.8189
DeBERTa0.84650.61110.6632
CLIP-L0.89560.63140.7786
CLIP-M0.52890.06260.6747

D-BISR achieves the highest macro-averages among these encoder baselines and leads in TPR@FPR5 on four of six transfers. However, MMA→ViSU remains challenging: its original AUROC of 0.7079 trails CLIP-L (0.8227) and DeBERTa (0.8890). Across three additional full-model seeds, D-BISR obtains AUROC 0.747±0.059 and TPR 48.85±7.44%. HyPE also achieves higher within-dataset F1 on ViSU and MMA. We will explicitly acknowledge these limitations.

Regarding the diffusion scorer, we added a capacity-matched MLP control using identical fused features, safety-score constraints, and pairwise ranking supervision. Under fixed source-domain q95 calibration, D-BISR achieves lower FPR in five of six transfers and higher AUROC in four. Its macro-average FPR/TPR/AUROC is 15.41%/82.80%/0.907, versus 39.15%/85.91%/0.864 for MLP. Thus, diffusion scoring improves false-positive control in most directions, although MLP achieves higher average TPR.

Regarding inference, diffusion operates on fused text representations, not generated images or text. We use T=100, a linear beta schedule from 0.0001 to 0.02, uniformly sampled training timesteps, and fixed inference t=20, averaging residuals over M=2 independent noise samples. We will document these settings; the current results do not establish sensitivity across different M or noise samples.

Regarding paired data, ViSU provides safe/harmful fields; MMA and SneakyPrompt provide row-aligned SFW/NSFW prompts. We preserve original row indices without assuming semantic equivalence or minimal edits. Normalized-prompt connected components determine source training, selection, and calibration splits; target labels are used only for evaluation.

Regarding SneakyPrompt TPR@FPR5, the metric is the maximum empirical TPR at FPR≤5%. LEG and HyPE detected 0/156 harmful prompts, giving 0%. The reported 0.0001 was a display floor, not a measured result; we will correct it.

Finally, we will improve figure labels, legends, table formatting, notation, dataset definitions, and the explanation of D-BISR at first mention.

Thank you for the detailed response. The capacity-matched MLP control is the most useful addition. At the fixed source threshold, the diffusion scorer keeps target FPR far lower at similar TPR in some directions, for example 9.6% vs. 61.5% on MMA->SneakyPrompt, which supports its role in threshold transfer. Outside MMA->ViSU, however, its AUROC matches the MLP's (0.941 vs. 0.940), so it does not improve ranking. My main concern stands. BingoGuard, never trained on these datasets, reaches AUROC 0.978-0.984, above D-BISR's in-domain AUROC on every dataset (0.954–0.975). This makes comparisons with smaller public guards such as HarmAug and Prompt Guard, and with text-to-image prompt guards such as Latent Guard and GuardT2I, essential. The response says such comparisons do not fit the source-only protocol, but none of these guards needs target data, and the authors can run them on the same target prompts as BingoGuard. The response also promises the re-scoping and presentation fixes, but the paper does not include them yet. Therefore, I maintain my score.

Reviewer_jUaROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

D-BISR is an encoder-based framework for harmful prompt detection without target-domain training data. It combines token-level multi-prototype attention, global-local features, and a safety-aligned diffusion residual score. The evaluation examines detection, source-to-target transfer, component ablations, and efficiency on ViSU, MMA, and SneakyPrompt.

Critical strengths
  1. The deployment motivation is compelling. A detector transferred to a new domain should preserve localized harmful evidence and handle heterogeneous unsafe patterns without requiring labeled target-domain calibration data. These requirements are more informative than optimizing in-domain classification accuracy alone.

  2. The proposed representation and scoring components form a coherent system, and the experiments show promising results on several detection and transfer settings. The analysis of a lightweight encoder is also practically relevant compared with larger generative guards.

Critical weaknesses
  1. The paper does not yet establish why diffusion residual scoring is the necessary or distinctive solution to the stated problem. Local evidence aggregation and multiple learned attention prototypes are reasonable design choices, but the strongest potential methodological novelty is the diffusion-based risk score. In the w/o diffusion ablation, replacing the scorer with a BCE classifier also changes the supervision objective. This comparison therefore does not separate diffusion from supervised ranking and safe-score regularization. A non-diffusion scorer trained with matched supervision would directly test whether the proposed mechanism contributes more than an alternative implementation of risk ranking. Similarly, the multiple-prototype ablation demonstrates utility but does not by itself establish specialization into distinct harmful modes.

  2. The most deployment-relevant part of the motivation remains incompletely validated. The paper distinguishes ranking metrics from source-calibrated threshold metrics and reports F1 under transferred thresholds, which is useful. However, target-domain TPR@FPR5 does not show that a fixed source-domain threshold actually maintains a low target-domain FPR. Reporting the achieved FPR and TPR at that fixed threshold is necessary to support the intended operating regime. The large variation across transfer directions, including the MMA-to-ViSU result, further limits a broad stability claim. This is a core evaluation issue because low-FPR transfer without target calibration motivates the method.

Questions for authors
  1. How does a non-diffusion scorer perform with the same pairwise ranking objective, safe-score regularization, input features, and comparable capacity? Please also clarify how safe/unsafe training pairs are constructed.
  2. What FPR and TPR are actually achieved in each transfer direction using the fixed source-calibrated threshold? Does the claimed low-FPR behavior survive this test?
  3. Do prototype-level analyses show distinct harmful evidence patterns, rather than benefits attributable mainly to additional attention capacity?
1 comment

We thank the reviewer for the constructive comments. Regarding the non-diffusion scorer, we conducted an additional comparison with a capacity-matched MLP scorer, replacing only the scoring head while keeping the safety-score constraints, pairwise ranking objective, and source-domain q95 calibration unchanged. The MLP contains 11.04M parameters, comparable to the 11.70M parameters of the D-BISR denoiser. MLP results are reported as mean ± standard deviation over three scorer initializations.

Source → TargetD-BISR (FPR / TPR / AUROC)MLP (FPR / TPR / AUROC)
ViSU → MMA4.70% / 93.91% / 0.9694.86±0.15% / 94.29±0.00% / 0.956±0.001
ViSU → SneakyPrompt26.28% / 94.87% / 0.95534.83±2.59% / 96.37±0.37% / 0.947±0.002
MMA → ViSU11.33% / 42.95% / 0.69043.57±13.11% / 41.91±4.61% / 0.484±0.010
MMA → SneakyPrompt9.62% / 91.03% / 0.95461.54±11.77% / 91.88±1.33% / 0.923±0.011
SneakyPrompt → ViSU9.49% / 63.86% / 0.84341.31±5.17% / 91.16±1.56% / 0.890±0.002
SneakyPrompt → MMA29.95% / 99.49% / 0.98448.77±4.32% / 99.83±0.07% / 0.986±0.000

In this matched comparison, D-BISR achieves lower FPR in all six transfer directions, with a macro-average FPR of 15.23% versus 39.15% for MLP. D-BISR also achieves higher AUROC in four directions, with a macro-average of 0.899 versus 0.864. These results demonstrate that diffusion residual scoring contributes to cross-domain false-positive control independently of the representation and training objectives. Regarding training pairs, ViSU uses the original safe/harmful fields, while MMA and SneakyPrompt use SFW/NSFW prompts from the same dataset row, preserving their original row indices. These dataset-defined pairs do not necessarily represent minimal edits or semantically equivalent texts.

Regarding fixed-threshold deployment, we determine the detection threshold using the q95 of source-domain safe calibration samples and apply it directly to unseen target domains without recalibration. The following table reports target-domain FPR and TPR across random seeds. The 95% bootstrap confidence intervals for FPR jointly resample source-domain calibration pairs and target-domain pairs.

Source → TargetFPR (Mean ± SD; 95% CI)TPR (Mean ± SD)
ViSU → MMA7.28±3.73% (5.09–9.59%)95.14±2.58%
ViSU → SneakyPrompt26.07±6.73% (18.61–33.12%)95.30±1.33%
MMA → ViSU8.87±2.17% (4.78–12.23%)48.85±7.44%
MMA → SneakyPrompt9.19±1.33% (5.34–14.86%)91.88±0.98%
SneakyPrompt → ViSU14.18±8.26% (4.01–25.75%)66.88±7.22%
SneakyPrompt → MMA26.86±6.94% (16.50–35.80%)98.77±0.94%

These results show that source-domain q95 calibration does not guarantee a target-domain FPR below 5%. For example, ViSU → SneakyPrompt and SneakyPrompt → MMA yield mean FPRs of 26.07% and 26.86%, respectively, while MMA → ViSU exhibits relatively low TPR and AUROC, indicating a more challenging transfer direction. We will include these fixed-threshold results to provide a more transparent assessment of practical cross-domain deployment.

Regarding prototype specialization, we analyzed risk-conditioned prototype gating distributions rather than inferring prototype behavior solely from aggregate detection performance. Risk-category labels were not used during training. Using 2,391 held-out harmful prompts across 20 risk categories and three model seeds, we performed cross-fitting and cross-seed prototype alignment. The resulting risk-conditioned gating profiles achieved a mean cross-seed correlation of 0.587 (95% CI: 0.499–0.705; permutation test p = 0.0005). These findings provide statistically significant evidence that different risk categories exhibit reproducible prototype allocation patterns across seeds, supporting the role of multiple prototypes in representing heterogeneous risks. Importantly, prototype specialization is distributed rather than one-to-one: multiple prototypes jointly contribute to risk representation, and no individual prototype is assumed to correspond exclusively to a single risk category.

Reviewer_XxfqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a harmful prompt detection method that combines multiple learnable risk prototypes, token-level attention, and diffusion-based residual scoring to improve cross-domain generalization without target-domain data.

Critical strengths

The paper raises an important concern about how neutral context can dilute local harmful evidence and how binary classification may fail to represent different harmful patterns. They address these two problems together providing a clear motivation for the method.

Critical weaknesses
  1. The proposed framework integrates existing prototype-based attention, diffusion residual scoring, and pairwise ranking techniques. While their combination may yield empirical benefits, the technical contribution is incremental.
  2. The most significant concern is with the results. The performance gain of D-BISR over baselines is low and in a lot of cases they don't beat the baseline (Table 1 and 2). In particular for attack detection, recall maybe a more significant measure of performance and the model is not the winner in any of the cases.
  3. Results are also over single runs with no standard deviation across runs or statistical significance test.
  4. The ablation study is incomplete. The ablation changes both the diffusion model and the loss function, making it unclear which causes the improvement.
  5. Missing code and other reproducibility artifacts.
Questions for authors
  1. What is the key technical novelty beyond combining existing prototype learning, diffusion scoring, and pairwise ranking techniques?
  2. Why does the method underperform simpler baselines in some cross-domain settings, and what explains this inconsistency?
1 comment

We thank the reviewer for the constructive comments. Regarding novelty, D-BISR addresses a strict target-data-free transfer setting, where training and threshold calibration use only source-domain data. Its contribution is not a new prototype attention or diffusion technique, but the joint design of token-level multi-prototype risk extraction, global–local fusion, and safety-aligned diffusion residual scoring with pairwise ranking. This design aims to preserve localized harmful evidence within benign contexts while producing source-calibrated risk scores for unseen domains.

Regarding performance, D-BISR achieves the highest TPR@FPR5 on all three in-domain datasets and four of six cross-domain directions. On ViSU and MMA, its TPR@FPR5 reaches 0.9615 and 0.9340, compared with HyPE’s 0.9590 and 0.9289. HyPE achieves higher F1 on these two datasets, while D-BISR leads on SneakyPrompt (0.9057). Cross-domain macro-averaged results are:

MethodAUROCTPR@FPR5F1
D-BISR0.89750.75490.8189
DeBERTa0.84650.61110.6632
CLIP-L0.89560.63140.7786
CLIP-M0.52890.06260.6747

Compared with CLIP-L, D-BISR improves TPR@FPR5 by 12.35 percentage points.

Compared with CLIP-L, D-BISR improves macro-averaged TPR@FPR5 and F1 by 12.35 and 4.03 percentage points, respectively. We acknowledge that these gains do not imply superiority on every metric or transfer direction.

Regarding directional differences and statistical reliability, we conducted additional multi-seed evaluations using source-domain q95 thresholds without target-domain recalibration. The results distinguish ranking quality from fixed-threshold transfer. For example, ViSU→SneakyPrompt achieves AUROC 0.952 and TPR 95.30%, but FPR reaches 26.07%, indicating threshold transfer limitations. Conversely, MMA→ViSU achieves AUROC 0.747, suggesting greater difficulty in risk ranking itself. Moreover, the original Table 5 already reports means and standard deviations over three random seeds. The new evaluations further quantify cross-domain variability.

Regarding ablation attribution, we introduced a parameter-matched MLP scorer while keeping risk representations, safety constraints, pairwise ranking, and threshold calibration unchanged. The following table summarizes both supplementary evaluations.

TransferMulti-seed D-BISRMatched D-BISRMLP scorer
ViSU→MMA7.28 / 95.14 / .9724.70 / 93.91 / .9694.86 / 94.29 / .956
ViSU→Sneaky26.07 / 95.30 / .95226.28 / 94.87 / .95534.83 / 96.37 / .947
MMA→ViSU8.87 / 48.85 / .74711.33 / 42.95 / .69043.57 / 41.91 / .484
MMA→Sneaky9.19 / 91.88 / .9579.62 / 91.03 / .95461.54 / 91.88 / .923
Sneaky→ViSU14.18 / 66.88 / .8369.49 / 63.86 / .84341.31 / 91.16 / .890
Sneaky→MMA26.86 / 98.77 / .97829.95 / 99.49 / .98448.77 / 99.83 / .986

Values are FPR (%) / TPR (%) / AUROC. Sneaky denotes SneakyPrompt. Multi-seed and matched-ablation results use different evaluation runs.

Multi-seed results report mean±SD under source-q95 calibration; the intervals are joint 95% FPR intervals. Matched D-BISR and MLP results belong to the separate scorer ablation; MLP values report mean±SD over three scorer initializations. Sneaky denotes SneakyPrompt.

In the matched ablation, D-BISR achieves lower FPR in all six directions, reducing macro-average FPR from 39.15% to 15.23%. Its macro-average AUROC also increases from 0.864 to 0.899, outperforming MLP in four directions. Although MLP performs better on certain individual metrics, these results support the contribution of diffusion residual scoring to cross-domain false-positive control independently of the training objectives.

Regarding reproducibility, we plan to release the code, configurations, data-processing scripts, and evaluation protocols upon publication. We will also clarify model settings, random seeds, source-threshold calibration, and evaluation procedures in the revision. We thank the reviewer again and will revise the manuscript to better communicate the contributions, limitations, and experimental evidence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.