acceptodds
Under review as a conference paper at ICLR 2027

Efficient and Adaptable Detection of Malicious LLM Prompts via Bootstrap Aggregation

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding. However, they remain susceptible to malicious prompts that induce unsafe or policy-violating behavior. Existing defenses face fundamental limitations: black-box moderation APIs offer limited transparency and adapt poorly to evolving threats, while white-box approaches using large LLM judges impose prohibitive computational costs and require expensive retraining for new attacks. To address these challenges, we present BAGEL (Bootstrap AGgregated Ensemble Layer), a modular, lightweight, and incrementally updatable framework for malicious prompt detection. BAGEL employs a bootstrap aggregation and mixture-of-experts-inspired ensemble of fine-tuned models, each specialized on a different attack dataset. At inference, BAGEL aggregates predictions from a stochastically selected subset of ensemble members, optionally augmented by a lightweight learned router that predicts the most suitable member to include. When new attacks emerge, BAGEL updates incrementally by fine-tuning just a single small classifier (86M parameters) and adding it to the ensemble. We evaluate BAGEL on a held-out, source-disjoint test set constructed entirely from datasets never used for training or calibration, ensuring that reported performance reflects generalization to out-of-distribution attacks. In single-turn evaluation, BAGEL achieves an F1 score of 0.757 at a size equivalent to 430M parameters, improving to an F1 of 0.772 when augmented with a lightweight learned router using fewer ensemble members (324M parameters). These results outperform popular black-box moderation APIs and most white-box baselines at lower costs. In multi-turn evaluation, BAGEL achieves an F1 of 0.660, outperforming billion-parameter baselines. Our results show ensembles of small finetuned classifiers can match or exceed the performance of billion-parameter guardrails while offering the adaptability and efficiency required for production systems.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM safety, malicious prompt detection, jailbreak detection, prompt injection, guardrails, ensemble learning, adaptive defense

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wYjLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents BAGEL, a lightweight and adaptable approach for detecting malicious prompts before they reach an LLM. It combines several small classifiers, each trained on a different type of attack, and uses only a subset of them for each prompt. A lightweight router can choose a particularly relevant classifier, while new attack types can be handled by adding another specialized model rather than retraining the whole system. The experiments show that BAGEL performs competitively against existing moderation and guardrail methods on unseen attacks.

Critical strengths
  • The design is practical and easy to update. BAGEL gets strong detection performance from several small, specialized models. More importantly, when a new attack appears, the system can add one new classifier rather than retraining everything. This makes the approach much more realistic for settings where attacks change quickly.

  • I appreciate the effort to test generalization. The main test set comes from datasets that were not used for training or calibration, and the paper also evaluates new attack types and multi-turn conversations.

Critical weaknesses
  • One of my major concerns is that the paper only uses the number of model parameters as a measure of cost. But BAGEL has to run several classifiers for every prompt, and the routed version also runs an additional router. The paper does not show end-to-end response time or requests handled per second. It is hard to tell whether BAGEL is actually cheaper or faster in a real deployment. The claim that selecting five models cuts computational cost by about 44% is based on parameter counts too.

  • The false-positive rate is quite high. In the main comparison, BAGEL incorrectly blocks around 30% of benign prompts. ShieldGemma, for example, has a much lower rate and also achieves better balanced accuracy. The issue becomes more serious in the multi-turn setting, where BAGEL incorrectly flags roughly half or more of the benign conversations in several settings.

  • BAGEL can only read 512 tokens at once, so longer conversations are cut off in the full-conversation setting. The alternative of checking each turn separately also produces very high false-positive rates. As such, the current design does not yet provide a convincing solution for long, gradual attacks spread across a conversation.

Questions for authors
  • BAGEL blocks around 30% of benign prompts in the main evaluation, and the rate is even higher in several multi-turn settings. Cam authors interpret these results and justify the implications in real deployment?
3 comments
AuthorsOpenReview ↗

Response to Reviewer wYjL

Dear Reviewer, thank you for your feedback. Below are our full responses to the questions and concerns you have raised.


W(1): One of my major concerns is that the paper only uses the number of model parameters as a measure of cost. But BAGEL has to run several classifiers for every prompt, and the routed version also runs an additional router. The paper does not show end-to-end response time or requests handled per second. It is hard to tell whether BAGEL is actually cheaper or faster in a real deployment. The claim that selecting five models cuts computational cost by about 44% is based on parameter counts too.

We report measured end-to-end latency and throughput:

MethodLatency (batch 1)ThroughputThroughput batch
BAGEL (n=5n=5)11.9 ms24.7 ex/s64
BAGEL w/ DistilBERT (n=3n=3)14.2 ms39.0 ex/s64
ToxicDetector42 ms23.7 ex/s20
ShieldGemma78 ms12.8 ex/s1
StruQ215 ms14.2 ex/s64
OpenAI Moderation API222 ms38.3 ex/s64
LastLayer0.5 ms1,869 ex/s–

We timed every method on the same NVIDIA A100, with models already loaded and only the forward pass measured. Latency is for a single prompt. For BAGEL, we assume the selected promptcops run in parallel, so its latency is that of one promptcop, plus the router when it is used; because they share one GPU, we divide its throughput by nn. Regarding some of the baselines: ToxicDetector embeds prompts one at a time with Llama-2-7B, so its throughput does not strongly depend on batch size; our ShieldGemma pipeline only supports a batch size of 1; and the OpenAI API’s latency includes network time. LastLayer is rule-based and runs on CPU.

BAGEL answers a single request in 11.9 ms, 3.5–18$×$ faster than the model-based baselines (42–215 ms), while achieving higher F1, and its throughput is comparable to theirs. LastLayer is faster but has much lower detection performance (F1 0.356). The measurements also confirm the ≈44% saving with measured time rather than parameter counts: BAGEL’s throughput at n=5n=5 (24.7 ex/s) is 1.8$×$ that of the full ensemble at n=9n=9 (13.7 ex/s).

W(2): The false-positive rate is quite high. In the main comparison, BAGEL incorrectly blocks around 30% of benign prompts. ShieldGemma, for example, has a much lower rate and also achieves better balanced accuracy. The issue becomes more serious in the multi-turn setting, where BAGEL incorrectly flags roughly half or more of the benign conversations in several settings.

Q(1): BAGEL blocks around 30% of benign prompts in the main evaluation, and the rate is even higher in several multi-turn settings. Cam authors interpret these results and justify the implications in real deployment?

To examine whether BAGEL’s FPR can be reduced, we re-swept the per-promptcop thresholds on calibration data only, replacing the F1 criterion with precision-oriented alternatives. Under an F0.5 criterion, FPR drops from 0.288 to 0.172, but ASR rises from 0.341 to 0.478 (F1 0.667, balanced accuracy 0.675), so the trade-off is controllable but not free. At ShieldGemma’s FPR (0.068), BAGEL’s ASR is 0.69 versus ShieldGemma’s 0.43.

Regarding the multi-turn setting: these results are fully out-of-distribution. Every promptcop was trained only on single-turn prompts, and none saw any multi-turn data. Even so, BAGEL achieves the highest F1 of all methods for n≥5n \ge 5 (0.704 at n=9n=9 vs. 0.647 for ToxicDetector, 0.375 for the OpenAI API and 0.312 for ShieldGemma, Table 3). The higher FPR is partly structural to Chunk-Level detection, which flags a conversation if any turn is flagged, so false alarms accumulate with conversation length.

W(3): BAGEL can only read 512 tokens at once, so longer conversations are cut off in the full-conversation setting. The alternative of checking each turn separately also produces very high false-positive rates. As such, the current design does not yet provide a convincing solution for long, gradual attacks spread across a conversation.

We agree that BAGEL does not yet provide a dedicated solution for long, gradual multi-turn attacks, and we do not claim it does. Multi-turn defense is not the focus of this paper: Prompt Guard 2, BAGEL’s base model, is designed for single-input classification (including its 512-token limit), and none of the baselines are designed or trained for multi-turn detection either. Our multi-turn evaluation is therefore a test of generalization, not a proposed multi-turn defense, since in a setting where every method faces an unfamiliar attack type, BAGEL’s ensemble of single-turn promptcops attains the highest F1, even if its absolute performance is not yet ideal. Integrating multi-turn-specific design choices (e.g., longer-context base models and turn-aggregation strategies) and a full evaluation is left to future work.


We would be happy to clarify any remaining concerns you may have regarding our work.

Dear Reviewer, following further discussion with other reviewers, we re-measured BAGEL’s latency end to end and we would like to correct and extend our earlier response to W(1).

Our earlier 11.9 ms figure timed a single promptcop without synchronizing the GPU after each request. We now time the complete pipeline (router + selected promptcops + aggregation) at batch size 1 on the same A100, synchronizing after every request, averaged over 200 test prompts:

MethodLatency (ms / request)
BAGEL w/ router (n=1n=1)17.7
BAGEL w/ router (n=3n=3)47.1
BAGEL (n=5n=5)72.7
ToxicDetector42
ShieldGemma-2B78
StruQ215
OpenAI Moderation API222

This table supersedes the latency figures in our earlier reply. Each promptcop adds ≈14.5 ms and the router adds ≈3 ms, so latency scales linearly with nn. Selecting 5 of 9 promptcops therefore saves ≈44% of compute time, confirming the parameter-count estimate in the paper with measured time. Routed n=1n=1 is the fastest model-based detector (2.4$×$ faster than ToxicDetector), and all BAGEL variants are 3-12$×$ faster than StruQ and the OpenAI API. Peak GPU memory is 4.0 GB (routed n=3n=3) and 6.1 GB (n=5n=5). Because all promptcops share one architecture, they can also be fused into a single batched forward pass, which would further reduce latency for n>1n>1. We will implement this in the final version.

Following the discussion, Reviewer 1bUC has updated their assessment. As the discussion period ends tomorrow, we would greatly appreciate it if you could let us know whether our responses have addressed your concerns, or if anything remains that we can clarify.

Reviewer_wYjLOpenReview ↗

Thank you for the response

Thank you for the additional experiments and response. It has addressed my prior concerns and I have raised my score.

Reviewer_1bUCOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BAGEL, a modular ensemble for malicious prompt detection. It fine-tunes separate Prompt Guard 2 classifiers on different dataset sources and aggregates threshold-adjusted scores from a randomly selected subset of experts. An optional DistilBERT router predicts the input's dataset provenance and includes the corresponding expert in the subset. The paper evaluates detection performance, expert selection, incremental adaptation through the addition of a new expert, comparisons with existing detectors, and multi-turn attack detection.

Critical strengths
  1. Practical motivation and modular design. Maintaining prompt detectors as attacks evolve is an important problem. BAGEL provides a straightforward mechanism for incorporating new training data by adding an expert while leaving existing expert weights unchanged (only the thresholds and, if used, the router are updated). This design has potential value for maintaining deployed detection systems.
  2. Encouraging evidence for selective expert aggregation. On the same test set, random selection of five experts approaches the full ensemble's F1, and routing one expert while randomly selecting two others improves F1 from 0.757 to 0.772 and balanced accuracy from 0.675 to 0.694 relative to five randomly selected experts. Absolute F1 should be read with care on this malicious-heavy test set (the paper notes that flagging every prompt yields 0.886), and no variance across runs is reported, but these relative comparisons suggest that evaluating every expert is unnecessary.
Critical weaknesses
  1. The claimed independence of the test and calibration data is not fully supported. (a) Test-set overlap. The paper states that the test set is constructed entirely from datasets never used for training or calibration. However, the dataset card of hlyn-labs/prompt-injection-judge-deberta-dataset lists TrustAIRLab/in-the-wild-jailbreak-prompts and jackhhao/jailbreak-classification among its constituent sources, although both are training sources in Table 5. The released notebook (bagel_router_anon.ipynb) filters Necent against known training sources but samples hlyn without any deduplication against the training or calibration data. The paper's own divergence analysis (Appendix G) acknowledges that Necent and hlyn show some content overlap with several training datasets, but it compares cluster-level distributions rather than checking for exact or near-duplicate prompts, so the extent of the overlap remains unknown. (b) Calibration protocol. The notebook draws Aegis calibration examples from a partition explicitly described as already used to fine-tune its expert, and these examples set that expert's threshold. This conflicts with the independent-calibration protocol in Section 3.2. The magnitude of either effect is unclear, but both should be resolved before the reported results can be taken as clean evidence of OOD generalization.
  2. The performance–efficiency advantage over a directly comparable small classifier is not established. The baselines with trainable components (Section 5.3) are fine-tuned on the same splits, but they differ in architecture, and the paper lacks a single Prompt Guard 2 fine-tuned on the union of the same training splits with comparable tuning and calibration. Without this control, the evaluation cannot establish whether the ensemble offers sufficient benefit over a simpler classifier trained on the same data. Because the paper's main argument for the ensemble is update cost, this comparison should also report the cost of retraining the pooled model when new data arrives.
Questions for authors
  1. Can you provide a provenance and exact/near-duplicate overlap audit across training, calibration, and test data, particularly for hlyn, and report the main results after removing any overlap?
  2. How does BAGEL compare with a single Prompt Guard 2 fine-tuned on the same pooled training data, using comparable tuning and calibration, in both detection performance and update cost?
  3. Table 4 reports 1.04 GB of weights per promptcop, which corresponds to roughly 260–280M fp32 parameters rather than 86M (Prompt Guard 2 86M is built on mDeBERTa, whose large multilingual embedding layer is excluded from the 86M backbone count). Are the parameter counts in Table 2 (e.g., 430M for n = 5) backbone-only, while the 324M figure counts the DistilBERT router in full? Please report total parameters consistently for BAGEL and the baselines, together with measured end-to-end latency and peak GPU memory including the router, beyond the per-promptcop estimates in Table 4.

I'll consider raising score if my questions are addressed.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 1bUC

Dear Reviewer, thank you for your feedback. Below are our full responses to the questions and concerns you have raised.


W(1) / Q(1): Data provenance and overlap

We now audit all test data for exact and near-verbatim (≥95% character-level similarity) matches against every training and calibration source. Necent was already filtered against training sources, but hlyn had been sampled without deduplication. Of 5,000 hlyn candidates, 232 (4.6%; 231 exact copies) overlapped training data. We now exclude overlaps before sampling hlyn's 1,000 evaluation prompts. The other four test sets contained 54 overlaps among 5,340 prompts.

You are also correct that model_10’s calibration slice overlapped its finetuning split. We now draw it from Aegis’s held-out 20% split, which the expert never saw, and retrain the router on the corrected CglobalC_{global}.

With both corrections, BAGEL (n=5n=5) achieves F1 = 0.761 (previously 0.757) with ASR = 0.341 and FPR = 0.288. BAGEL w/ DistilBERT (n=3n=3) achieves F1 = 0.773 (previously 0.772). Removing all 54 residual overlaps changes F1 by at most 0.004, indicating that the reported OOD performance is not driven by overlap.

W(2) / Q(2): Pooled Prompt Guard 2 baseline and update cost

We trained a single Prompt Guard 2 on the union of all promptcops’ training data, using the same recipe, calibration data and test set as BAGEL. It achieves lower F1 than BAGEL (0.715 vs. 0.761 / 0.773 for n=5n=5 and w/ router n=3n=3 respectively) and misses more attacks (ASR 0.420 vs. 0.341 / 0.323), though with lower FPR and slightly higher balanced accuracy (0.707 vs. 0.685 / 0.692).

The key difference lies in updating. When we fine-tune the pooled model on only the new data (the same 20k WildJailbreak rows used to train BAGEL’s new promptcop), it learns the new attack (F1 0.762 → 0.914) but degrades on attacks it had previously learned, as F1 on held-out data from the original nine training sources drops by about 4 points (0.946 → 0.904), and by 21 points on in-the-wild jailbreaks. Avoiding this forgetting requires retraining on all old and new data, whereas BAGEL trains a single new promptcop on the new data alone and leaves existing promptcops unchanged. BAGEL thus offers comparable detection quality while incorporating new attacks far more cheaply and without sacrificing coverage of earlier ones.

Q(3): Parameter counts and efficiency

Thank you for pointing this out. You are correct: the full Prompt Guard 2 model has ≈278M parameters, including ≈190M multilingual embedding parameters, consistent with Table 4's 1.04 GB of weights. The original 86M count covered only the mDeBERTa backbone as per Meta’s and Microsoft’s convention. The DistilBERT router's 66M count already included its embeddings (≈23.8M) and backbone (≈42.5M).

In the paper, each promptcop kept its own fine-tuned copy of this embedding table. However, we learn that fine-tuning barely alters it. Replacing every promptcop’s embedding table with a single shared, untrained base table changes F1 by at most 0.002 (0.761 → 0.761 at n=5n=5 without the router) (0.773 → 0.773 with the router) and balanced accuracy by at most 0.003. With one shared table, BAGEL's parameter count in Table 2 is updated to ≈620M parameters at n=5n=5, and ≈515M with the router at n=3n=3 (the router is counted in full), which is still well below ShieldGemma-2B, ToxicDetector and StruQ even without accounting for their own respective embedding parameters. Since an embedding layer is a table lookup, per-prompt compute scales with the 86M backbones, while memory scales with the totals above.

Furthermore, here is a breakdown of latency and throughput:

MethodLatency (ms) (batch size = 1)Throughput (ex/s)Throughput batch size
BAGEL (n=5n=5)11.924.764
BAGEL w/ DistilBERT (n=3n=3)14.239.064
ToxicDetector4223.720
ShieldGemma7812.81
StruQ21514.264
OpenAI Moderation API22238.364
LastLayer0.51,869–

We timed every method on the same NVIDIA A100, with models already loaded and only the forward pass measured. Latency is for a single prompt. For BAGEL, we assume the selected promptcops run in parallel, so its latency is that of one promptcop plus the router when it is used; because they share one GPU, we divide its throughput by nn. Additional notes: ToxicDetector embeds prompts one at a time with Llama-2-7B, so its throughput does not strongly depend on batch size, our ShieldGemma pipeline only supports a batch size of 1, and the OpenAI API’s latency includes network time.

BAGEL answers a single request in 11.9 ms, 3.5–18$×$ faster than the model-based baselines, with throughput comparable to theirs. Its peak GPU memory, including the router, stays at 4–6 GB, and drops to about 3 GB once the embedding table is shared.


We would be happy to clarify any remaining concerns you may have regarding our work.

Thank you for the detailed response and additional experiments. The reported deduplication and calibration corrections substantially address my original concerns about these issues. I also appreciate the pooled Prompt Guard 2 baseline and the corrected parameter accounting. Three points remain important to my assessment:

  1. Detection quality at comparable operating points. The pooled model has lower F1 but higher balanced accuracy than both BAGEL configurations (0.707 versus 0.685/0.692). Your response to Reviewer wYjL also reports that, at ShieldGemma’s FPR of 0.068, BAGEL has a substantially higher ASR (0.69 versus 0.43). These results make the detection advantage less clear than the F1 comparison suggests. Could you compare BAGEL and the pooled model at matched FPRs, with thresholds selected using calibration data?

  2. Measured inference efficiency. The timing explanation assumes parallel expert execution and uses a single expert’s latency, while scaling throughput by the number of experts. Could you clarify whether the reported 11.9 ms was directly measured for the complete BAGEL ensemble on one A100? Actual full-system latency and throughput, including routing where applicable, are needed to substantiate the efficiency claim.

  3. Retention and update cost. The pooled model’s forgetting results support the motivation for modular updates. However, adding an expert changes BAGEL’s aggregated predictions and potentially its routing, even when existing expert weights remain fixed. Could you report BAGEL’s performance on the original sources before and after the update, together with measured update costs for BAGEL and the pooled baseline?

I appreciate the substantive additions, but these remaining uncertainties prevent me from raising my rating at this stage.

Thank you for the follow-up. We address each point below with new measurements.

Q1: Detection at matched FPR.

We selected global thresholds on calibration data only, targeting fixed FPRs, and applied them unchanged to the test set:

MethodTarget FPR (calibration)Test FPRASRF1Bal. Acc.
BAGEL (n=5n=5)0.050.0700.6880.4690.621
0.100.1460.5030.6480.676
0.200.2830.3520.7540.683
BAGEL w/ router (n=3n=3)0.050.1350.5420.6140.662
0.100.2490.3810.7360.685
0.200.3530.2640.8060.691
Pooled PG20.050.2180.3580.7560.712
0.100.3950.1960.8450.704
0.200.6350.0860.8810.639

Test ROC-AUC: BAGEL (n=5n=5) 0.748, w/ router (n=3n=3) 0.749, pooled 0.788.

At each system's default operating point, BAGEL leads in F1 by 4.6–5.8 points but trails in balanced accuracy by only 1.5–2.2 points. At similar test FPR however, the pooled model detects more attacks (ASR 0.358 vs. 0.409 at FPR ≈0.22), consistent with its higher AUC. We therefore do not claim better detection than a jointly trained model at matched operating points. One deployment-relevant difference is that BAGEL's calibrated FPR transfers more reliably. At a 5% calibration target, BAGEL reaches 7.0% test FPR versus 21.8% for the pooled model, whose balanced accuracy also falls (0.712 → 0.639) as the target rises.

Q3: Original-source performance before/after the update, and update cost.

We evaluated on held-out data from the original nine training sources (unused splits, excluding all training and calibration prompts), before and after adding the WildJailbreak source:

Original sources (held out)F1 before → afterBal. Acc. before → after
BAGEL (n=5n=5)0.694 → 0.6980.729 → 0.728
BAGEL w/ router (n=3n=3)0.745 → 0.7600.778 → 0.789
BAGEL w/ router (n=1n=1)0.892 → 0.8920.906 → 0.907
Pooled PG2 (sequential update)0.923 → 0.8840.933 → 0.901

BAGEL does not forget, as existing promptcops are untouched and original-source performance is preserved or improves, whereas sequential fine-tuning of the pooled model loses 3.9 F1 points. On in-distribution data, the pooled model starts higher than BAGEL at n=3n=3 / n=5n=5, because averaging a source's specialist with non-specialist members dilutes its prediction. Routing to the single specialist (n=1n=1) removes this effect, and after the update it outperforms the pooled model (0.892 vs. 0.884), at a small cost out of distribution (test F1 0.746 vs. 0.773 at n=3n=3). The selection size thus lets operators trade in-distribution accuracy for OOD robustness.

Update costs (one A100, same 20k new rows):

TrainingOther stepsForgetting
BAGEL (new promptcop)22 min≈4 min threshold calibration + ≈2 min router retrainingnone
Pooled, sequential update22 min–yes (above)
Pooled, full retrain≈3.8 h–none

BAGEL therefore matches the cost of a sequential update while avoiding its forgetting, and is ≈8× cheaper than the full retrain needed to avoid forgetting in the pooled model.

We will add these results to the revised paper and correct the latency figures in Table 4 and §5.

Q2: Full-system latency.

We re-measured the complete pipeline (router + selected promptcops + aggregation) at batch size 1 on one A100, synchronizing the GPU after every request, averaged over 200 random test prompts. Our earlier 11.9 ms figure timed a single promptcop and under-counted per-request synchronization. We thank the reviewer for prompting this correction.

MethodLatency (ms / request)
BAGEL w/ router (n=1n=1)17.7
BAGEL w/ router (n=3n=3)47.1
BAGEL (n=5n=5)72.7
ToxicDetector42
ShieldGemma-2B78
StruQ215
OpenAI Moderation API222

Each promptcop adds ≈14.5 ms and the router ≈3.5 ms. BAGEL (n=5n=5) is thus on par with ShieldGemma and slower than ToxicDetector, while routed n=3n=3 is close to ToxicDetector; all variants are 3–5× faster than StruQ and the OpenAI API. Running promptcops on separate CUDA streams gave no speedup, since at batch 1 the members execute sequentially. Because all promptcops share one architecture, they can be fused into a single batched forward pass, which is something we will add in the final paper.

Thank you for the additional experiments and clarifications. The matched-FPR comparisons, corrected full-pipeline latency measurements, and before/after evaluation on the original sources substantially address my follow-up concerns. Together with the earlier data corrections and pooled baseline, these results make the paper’s practical contribution clearer.

The strongest evidence is now for the modular update mechanism: in the evaluated update, BAGEL largely preserves original-source performance while requiring substantially less update time than full pooled retraining. I consider this a useful practical benefit.

Some limitations remain. The pooled model has stronger detection performance at matched operating points, and the corrected latency measurements show a narrower inference-efficiency advantage than originally claimed. The results therefore support a useful trade-off between detection quality, inference cost, and update cost.

On balance, the additional evidence has increased my confidence in the contribution, and I am raising my rating from 2 to 3. Please incorporate these experiments, corrected measurements, and appropriately qualified claims into the revised paper.

Reviewer_sDoPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes BAGEL, a framework to ensemble small LLMs to achieve a better trade-off between performance and efficiency. BAGEL also achieves continuous adaptability via additional model fine-tuning. To enhance aggregation performance, BAGEL employs an additional lightweight learned router that acts as a model router. This paper evaluates BAGEL using a mixed dataset. The results show that BAGEL achieves an F1 score of 0.757 (0.772) and 0.660 in single-turn and multi-turn settings, respectively.

Critical strengths

(1) The problem that the paper focuses on is important and worthy of attention. It is worth exploring how to achieve effective security-aware detection at lower cost by arranging small models efficiently. The paper addresses this issue and also takes adaptability into consideration, which is commendable.

Critical weaknesses

(1) The effectiveness of the method needs to be further demonstrated. BAGEL calculates the final score using the average score across different members. My main concern is that poorer members will have an adverse effect on better ones. In Section 5, BAGEL achieves an F1 score of 0.710 with one randomly selected member, improving by approximately 0.05 with more members. However, this result does not provide sufficient evidence that the ensemble gain is substantial, nor does it adequately test the ”bucket effect”.

(2) The definition of “Adaptability” requires clarification. Intuitively, adaptation to new attacks refers to detecting unknown attacks. However, the adaptation demonstrated in the paper involves retraining on new but already labeled attacks, which weakens the paper’s contribution. It also appears inconsistent with the paper’s discussion of the drawbacks of closed-source solutions (“their closed-source nature limits the adaptability of LLM systems”), since closed-source solutions can likewise be retrained on labeled data.

Questions for authors

(1) How do the training datasets differ across members, and how well does each member generalize to other datasets?

(2) Can BAGEL detect previously unseen attacks without labeled training data?

3 comments
AuthorsOpenReview ↗

Response to Reviewer sDoP

Dear Reviewer, thank you for your feedback. Below are our full responses to the questions and concerns you have raised.


W(1): The effectiveness of the method needs to be further demonstrated ... nor does it adequately test the ”bucket effect”.

We evaluated each promptcop individually on the test set (with its own calibrated threshold). Single members range from F1=0.400 / balanced accuracy=0.405 (worst) to 0.774 / 0.701 (best), with a mean of 0.691 / 0.624. BAGEL (n=5n=5) reaches 0.761 / 0.685, and 0.773 / 0.692 with the router, far above the average and worst members and close to the best, so weaker members do not drag the ensemble down. Per-model threshold recentering (§3.5) contributes to this by preventing any single miscalibrated member from dominating the average. Notably, the best single member on aggregate is not the best on any individual test source and could only be identified post hoc using test labels, while the ensemble attains near-best performance without that knowledge.

W(2): The definition of “Adaptability” requires clarification ... since closed-source solutions can likewise be retrained on labeled data.

Q(2): Can BAGEL detect previously unseen attacks without labeled training data?

We agree the term needs precision. By adaptability we mean that incorporating a newly observed, labelled attack type requires fine-tuning only one 86M promptcop, without retraining existing members or a large monolithic guardrail; we do not claim BAGEL detects arbitrary novel attacks without supervision. We will revise §1 and §3 to state this explicitly. On the closed-source comparison: a deployer cannot retrain a closed-source API such as the OpenAI Moderation API on their own data, only the provider can. Our point concerns adaptability from the deployer’s side, and we will reword the sentence accordingly.

On unseen attacks (Q2) BAGEL partially generalizes without any labelled data for the target attack:

  • Every test dataset in Table 2 comes from sources never used for training or calibration.
  • In §5.2, the k=9k=9 (“before”) curves show BAGEL on WildJailbreak with no WildJailbreak training at all, reaching F1 of [0.75–0.82] across selection sizes; adding a single labelled promptcop then raises this to [up to 0.92].
  • The multi-turn evaluation (§5.4) is entirely zero-shot: no promptcop saw multi-turn data, yet BAGEL attains the highest F1 for n≥5n \ge 5.

BAGEL therefore detects a substantial share of unseen attacks unsupervised, and labelled data is what closes the remaining gap.

Q(1): How do the training datasets differ across members, and how well does each member generalize to other datasets?

The training datasets differ substantially in content as shown by pairwise Jensen-Shannon divergence (Appendix G) and attack composition (Table 5). Evaluating each promptcop individually on our five unseen test sources shows generalization varies widely as recall on the same source ranges from 0.09 to 0.99 (Past Tense). Members are strong on specific attack styles; e.g., the ToxicDetector-trained promptcop catches 99% of past-tense attacks but only ~40% of hlyn and JailbreakQR attacks. The only members with high recall everywhere flag 65–70% of benign prompts. No single member is reliable across sources, which is why BAGEL aggregates them: the ensemble (F1 0.761) clearly beats the average member (0.691). We will add this full table to the appendix for the final version of our paper.


We eagerly look forward to your reply, and would be happy to discuss further or clarify any remaining concerns you may have regarding our work. Thank you for your time.

Thanks for clarification. I have no further questions.

Dear Reviewer, thank you for engaging with our response, and we are glad the clarifications resolved your questions.

As the discussion period closes on Nov 18, we wanted to briefly follow up. If the concerns raised in your initial review, in particular the gains afforded by the ensemble and performance on unseen attacks, are now addressed, we would be grateful if you would consider updating your assessment to reflect this. If anything still stands in the way of a more positive evaluation, please let us know and we will do our best to address it before the deadline.

Thank you again for your time and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.