acceptodds
Withdrawn from ICLR 2027

FOSTER: Fixed-Threshold Safety Topology Repair for VLM Jailbreak Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Deployed jailbreak detectors for vision–language models (VLMs) must apply a single decision threshold, chosen before unseen attacks appear, to open-world traffic. We formalize this constraint as fixed-threshold transfer failure: an un- seen attack can remain rank-separable from benign inputs (high AUROC) while its scores drift below the pre-deployment threshold (low TPR). Despite this global displacement, unseen attacks preserve a transferable local unsafe topology in mid- to-late VLM hidden layers, so a limited number of trustworthy anchors can be sufficient to improve subsequent detection without retraining or threshold recal- ibration, while keeping the benign reference fixed. Building on this, we pro- pose FOSTER, which freezes the VLM, the safety evidence router, the benign reference, and all pre-deployment thresholds, while selectively expanding only a cross-verified unsafe topology bank through mode-preserving anchor admis- sion. Anchors are cross-checked, pseudo-labeled unsafe samples, admitted only when the current topology, the frozen initial topology, and a separately param- eterized safety estimator all satisfy their own pre-calibrated admission criteria, which are distinct from the prediction and routing thresholds. The three witnesses are complementary rather than statistically independent; their purpose is to re- duce self-reinforcing admission errors. We evaluate FOSTER under fixed cali- bration thresholds and streaming deployment on Qwen2.5-VL and LLaVA-v1.6. Under each detector’s own pre-deployment threshold, pooled attack recall rises from 10.5% to 80.7% on Qwen2.5-VL and from 6.8% to 71.3% on LLaVA-v1.6, with zero observed false positives on the 917-sample deployment benign pool (a 95% Wilson upper bound of 0.42%, not a guarantee of zero benign error). Struc- turally, freezing the admission witnesses fixes the eligibility set before deploy- ment, which yields a stream-independent outer envelope on any input’s online topology score—a conditional bound rather than a deployable certificate, since el- igibility does not imply admission. Because the benign reference bank is frozen by construction, benign-side drift is measured rather than bounded: under out-of- distribution benign pools and targeted benign contamination we report the result- ing change in false positives explicitly. Our code is available at an anonymous repository: https://anonymous.4open.science/r/FOSTER

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
VLM jailbreak detection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qairOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies fixed-threshold transfer failure in VLM jailbreak detection and proposes FOSTER. The proposal builds an unsafe reference banks that updates in runtime. A frozen router classifies high-confidence inputs, while uncertain inputs receive layer-wise KNN safety scores using benign and unsafe reference banks. The unsafe bank is updated via several predefined checks on anchor topology. The proposal keeps model parameters and all operating thresholds fixed, and manages memory to retain diverse unsafe modes. The evaluations report improved streaming recall, alongside targeted contamination tests, benign distribution shifts, and ablations.

Critical strengths

(1) The paper addresses a practically relevant jailbreak detection problem in VLMs. How to transfer detections to unseen attacks is an important research gap in this area. It does not focus on a specific detection method improvement but generalize the given detection capability into OOD attack samples.

(2) The asymmetric memory update is a sensible design. Authors provide sufficient evidence that the full admission gate reduces erroneous benign insertions and subsequent false positives relative to relaxed gates in the tested contamination setting.

(3) The authors explicitly distinguish their structural bounds from guarantees of correct admission or benign safety. They also acknowledge cold-start limitations, report attack-specific regressions, and evaluate OOD benign false positives.

Critical weaknesses

(1) The anchor/topology mechanism lacks sufficient independent validation. Section 3 presents transferable unsafe geometry as a critical hypothesis and explicitly does not report quantitative layer-wise diagnostic results. Although Sections 4.4-4.6 explain how eligible anchors may increase local unsafe support, the full-system gains do not independently validate this explanation or establish why unseen attacks satisfy such conditions. The concern is not about the use of memory itself, but the evidence for the claimed mechanism and its generalization ability.

(2) The contribution of topology update is not sufficiently isolated from the frozen router. The offline trained router classifies some inputs, determines which inputs reach topology scoring, and constrains anchor admission. Its reliability therefore affects both detection and the opportunity to adapt. However, it looks the router is not the core novelty. The frozen-versus-online comparison establishes an update benefit conditional on the chosen router, but does not establish how robust that benefit is to router quality or whether simpler memory updates would achieve comparable gains. The existing ablation does not fully settle this issue. As in Table 4, removing the router changes AUROC from 0.9900 to 0.9895 while TPR falls from 90.27% to 0.0. This shows failure at the evaluated operating point, but does not disentangle router utility from threshold mismatch. Moreover, the ablation uses a validation subset and an ablation-specific threshold but not the main deployment protocol settings.

(3) Robustness to adaptive memory poisoning is insufficiently evaluated. Section 5.5 already evaluates hard-benign contamination and demonstrates benefits from the full gate. However, the paper does not establish whether the attacker can exploit feedback to adapt its inputs against all admission checks and the evolving memory. This distinction matters because successful poisoning could cause persistent false positives or displace useful anchors.

(4) Threshold sensitivity and the increment over naive methods need clearer evaluation. Using several fixed thresholds in the proposed system can make it depend on their joint selection. Decision, routing, and admission thresholds can affect recall, access to repair, and contamination risk in different ways. The evaluation should characterize these tradeoffs in the designed system across attack shifts, and test whether the conclusion holds under alternative source-only calibration rules. Also, matched comparisons with simpler online KNN updates would better isolate the benefits of layer-wise scoring and the full memory design.

Minor issue: In Figure 1, the green elements and small boxes are difficult to distinguish at the displayed scale. Improving presentation or sizing would make the figure more readable.

Questions for authors

(1) What evidence supports the transferable-topology hypothesis? Authors should clarify which properties of unseen attacks enable eligible seeds and subsequent repair, and provide controlled comparisons that distinguish the proposed layer-wise mechanism from simpler KNN adaptation, including tests on deduplicated held-out instances and attack families.

(2) How much performance comes from the standalone router, and what errors does topology repair actually correct? How sensitive are results to router reliability and the joint choice of routing, decision, and admission thresholds?

(3) Beyond the existing targeted attacks, can the authors evaluate or clearly delimit a feedback-driven attacker targeting the memory bank? Also, authors should additionally assess persistent false positives and later attack recall, and characterize sensitivity to initial attack, stream order, recurrence, and lower attack prevalence.

Reviewer_kFdUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies fixed-threshold transfer failure in VLM jailbreak detection: strong ranking performance on unseen attacks may not translate into high recall under a threshold selected before deployment. FOSTER addresses this problem by selectively expanding an unsafe reference bank while freezing the VLM, router, benign reference bank, and deployment thresholds. Admission requires agreement among current-topology, frozen-topology, and router evidence, while a cluster-based memory mechanism aims to preserve diverse unsafe modes. Experiments on Qwen2.5-VL and LLaVA-v1.6 demonstrate substantial improvements in streaming recall, supplemented by contamination and benign distribution shift evaluations.

Critical strengths

1.The paper shows why high AUROC can conceal poor deployment recall under fixed thresholds. FOSTER substantially improves recall on both VLMs, with particularly strong gains on LLaVA even under the oracle diagnostic. 2.Separating prediction from memory admission is well motivated. The contamination ablations show that the complete gate reduces erroneous insertions and subsequent false positives, while benign-shift tests help characterize adaptation costs.

Critical weaknesses

1.The methodological advantage and practical necessity are not fully established. The paper does not sufficiently motivate a realistic deployment scenario requiring fixed thresholds and unsafe-only memory updates, or explain why simpler alternatives would be inadequate. The large calibration–oracle gap on Qwen also calls for stronger pre-deployment calibration controls. Matched simple memory-expansion baselines and comparison with MTK would help isolate FOSTER’s specific contribution. 2.The geometric explanation lacks direct validation. Section 3 does not report quantitative D1–D3 results. Existing robustness tests also leave uncertainty about benign inputs close to unsafe neighborhoods, where admission checks may share errors and adaptation could amplify false positives.

Questions for authors

1.Does FOSTER retain its advantage when methods are calibrated before deployment under the same validation FPR budget, and compared with MTK and simple unsafe-only KNN expansion using matched features, initialization, and capacity? 2.Can the authors provide quantitative D1–D3 results and report detection FPR and erroneous admission rates on hard-benign inputs resembling unsafe content, before and after adaptation?

Reviewer_htUQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a jailbreak detector for vision-language models (VLMs) designed to address the issue of "fixed-threshold transfer failure": existing detectors can effectively rank novel attacks above benign inputs (AUROC), while they often fail to detect new attacks when a fixed pre-deployment threshold is applied to real-world traffic (low TPR). To overcome this, the authors propose a cascaded detector: first, a classifier (router) with cross-layer features processes the inputs and makes a decision only for high-confidence predictions; second, uncertain inputs get a score based on per-layer KNN similarity against sets of both benign and malicious examples. The malicious set is dynamically updated during deployment, incorporating new uncertain inputs that meet three criteria based on the similarities to the current set of malicious examples, the initial set, and the router’s confidence score. This update mechanism improves the attack detection rate (recall) for both models evaluated (Qwen2.5-VL and LLaVA-v1.6) on several attack families, even though the malicious set starts from a single known attack.

Critical strengths
  • The paper is well motivated, separating ranking quality (AUROC) from detetion recall at a fixed threshold before deployment allows measuring a deployment-relevant performance gap that per-attack AUROC alone can not measure. The authors measure this with nine attack types and two VLMs, and show that several strong-AUROC detectors miss most attacks or reach good recall only at unusable false-positive rates.

  • The proposed method shows large recall gains under the paper’s protocol with a simple adaptive design. Only the malicious bank changes, through a the three admission checks, while the model, thresholds, and benign reference bank stay fixed.

Critical weaknesses
  • The online performance gain might be amplified by threshold selection, and not necessarily by the claimed topology repair. The authors calibrate a single threshold per-method using Youden’s J statistic on nearly separable calibration data, which leaves the threshold position arbitrary in the gap between both classes. In Figure 1, the threshold sits just below the malicious calibration cluster. The best static threshold (oracle) already gets 74.7% (LoD) and 74.8% (FOSTER-frozen) on Qwen, against 80.7% for online FOSTER. The paper needs baselines with thresholds set from a target FPR on calibration benign data, and with simple label-free threshold adaptation on the stream. Without them, the benefit of online repair is overstated.

  • The evaluation protocol might be favoring the proposed method. The stream is about 91% attacks, which supplies many anchors and limits benign exposure. There is no analysis on prevalence and no variance over stream order for the main results. In addition, the AUROC evidence uses in-distribution benign data, and several baselines show near-zero or below-chance results that suggest reproduction problems.

  • The paper omits closely related work on hidden-state jailbreak detection that explicitly targets generalization to unseen attacks and benign distributions at fixed low FPR (e.g., Gan et al., ICLR 2026). That work calibrates the threshold on known benign data and can achieve strong static transfer to unseen attacks, which challenges both the claimed novelty of the fixed-threshold framing and the necessity of online adaptation. Please discuss it, compare against the static detector calibrated to a fixed benign FPR (see above), and clarify whether the evaluated attack samples successfully jailbreak the target models.

Gan, Yuyou, et al. "From Sure to Sorry: Detecting Jailbreak in Large Vision Language Model via JailNeurons." International Conference on Learning Representations. Vol. 2026. 2026.

Questions for authors

How does online FOSTER compare to a static detector (e.g. LoD or FOSTER-frozen) whose threshold is set from a target FPR on calibration benign data (e.g. 1% FPR on GQA)?

How does online FOSTER compare to a simple label-free threshold adaptation on the stream?

How are the attack families ordered in the stream (blocked or interleaved)? Did you try different settings? What are the results and the variance over stream orders when attacks are a small fraction of the traffic (e.g. 1%, 10%, etc.)?

What is the AUROC of FOSTER-frozen and the baselines when the benign set is SEED + mm-vet2 instead of GQA?

How were JailDAM, JailGuard and MirrorCheck run? Why do they show near-zero recall, or below-chance AUROC in MirrorCheck’s case?

Do the evaluated attack samples successfully jailbreak the target models, or are unsuccessful attacks also included in provided numbers?

How does FOSTER relate to Gan et al. (ICLR 2026), and how does it perform against that detector under the same fixed-FPR protocol?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.