acceptodds
Desk rejected at ICLR 2027

Measurement Audits of Two Language-Model Attack Detectors

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

An attack detector can reproduce its alerts without measuring the intended behavior. We audit two released language-model detectors by tracing callback inputs, replaying generation prefixes, and testing lexical coverage. DualSentinel's realtime callback repeatedly reads the first token's scores; the lookup appears in every public version we inspected. Crossing the score index with state reset changes verdicts, so an index correction alone does not establish a working detector. On the primary Qwen2.5-14B bank, every tested budget that detects an injection flags legitimate tasks; a separate same-model bank admits detections without legitimate alerts. Qwen-7B traces show budget-invariant legitimate-task alerts. For ICLScan, 248 of 256 keyword subsets preserve benchmark attack alerts because their targets contain redundant wording. A six-family analysis separates target acquisition, elicitation, and recognition. We turn these diagnoses into a practical audit protocol: verify the observed signal, measure legitimate-task costs, and test coverage without redundant cues.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, detector evaluation, AI safety, backdoor detection, prompt injection, measurement validity, reproducibility

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7P6gOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper audits two LLM attack detectors and examines whether their alerts measure the intended behavior. For DualSentinel, the authors find that its real-time implementation repeatedly reads the first token rather than the current token, and the results also depend on retained state and can flag legitimate tasks. For ICLScan, they show that its benchmark can keep producing the same attack alerts even after much of the scoring rule is removed, largely because the benchmark targets contain overlapping wording. Based on these findings, the paper proposes a practical audit procedure to bridge these gaps.

Critical strengths
  • The paper finds concrete, consequential problems in two released detectors.

  • The paper turns the case studies into a simple set of checks. This provides a concrete way to catch misleading detector results.

Critical weaknesses
  • The main findings are based on a fairly narrow set of cases, and some of the results/conclusions do not hold in the additional tests. For example, the strongest DualSentinel result comes from a synthetic task set on one model. In another test on the same model, the detector can catch some attacks without flagging legitimate tasks, and the results on the other model also look different. It is unclear to me whether the authors found a broader problem with the detector or just a problem with a particular setup.

  • Another weakness is about the proposed auditing procedure. I think the proposed procedure is useful, but many of its core ideas are fairly general evaluation principles and they lack a structure overall. This brings two issues: (1) the gaps being filled are driven by the two case studies, so it's unclear how comprehensive and generalizable they are; and (2) the design of this procedure lacks a sense of systematic analysis in general.

Questions for authors
  • Are the proposed audit steps intended as a general protocol for LLM attack detectors, or mainly for detectors with structures similar to DualSentinel and ICLScan? I have this question because some steps (e.g., checking state lifetime or keyword redundancy) are very specific to these systems.
1 comment
Reviewer_7P6gOpenReview ↗

No response from the authors

The authors chose not to include any response, so I will keep my original scores.

Reviewer_dqXZOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies two existing LLM attack detectors namely DualSentinel and ICLScan. It finds an implementation bug in DualSentinel that shows longer generation can lead to more false positives and also identifies weaknesses in ICLScan's keyword based detection. The authors test these through code inspection, some controlled experiments and tests involving keyword removal. They also propose a checklist for evaluating attack detectors.

Critical strengths
  1. The authors identify and reproduce a bug in DualSentinel's code.
  2. They report negative findings which is useful.
Critical weaknesses

Unfortunately, I have a few concerns that need expanding beyond just the one or two critical weaknesses expected.

  1. Most of the findings in the paper such as false positives and limitations of keyword-based detection are already well-known problems. Recent works such as PIGuard (ACL2025) and GuidedBench (ICLR2026) have already studied related problems in attack detection and evaluation. The authors do not properly compare their findings with these works. Their proposed 'audit protocol' just brings together existing eval practices. It is unclear what new research contribution this offers.
  2. The overall experiments are also too small to make any meaningful conclusion. The main experiment uses 16 synthetic examples on one model. Some additional experiments show completely different results suggesting main findings are task and model dependent. 3. Further, after claiming a bug in the DualSentinel code, the authors don't show how that bug affected the original results. Similarly, the ICLScan experiments show keyword choices affect detection, but do not establish any meaningful conclusion.
  3. In terms of detectors as well, only two are studied and both from very different perspectives, so it's hard to understand what conclusions really hold beyond these specific implementation issues.
  4. The presented 'audit protocol' is one of the main contributions, but it is basically a summary of the checks performed. There are no tests on any new detectors or identify problems that existing evaluation methods miss. This makes the practical value of their protocol unclear.
  5. All annotations on tasks including whether task-flipped responses preserve original meaning or whether model capabilities are preserved etc all rely on a single author. No independent annotation or agreement analysis is done. This is a bit concerning as these are how the results are being interpreted.
Questions for authors
  1. Can the authors actually show how the bug in the DualSentinel code actually affect the original results?
  2. Why are the main experiments only on 16 samples?
  3. How is the proposed protocol different from what existing evaluation papers do?
  4. Can the authors test more detectors beyond just these two to show the problems mentioned are not specific to just these two detectors?
Reviewer_1FewOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper audits the measurement chains of DualSentinel and ICLScan. Its central contribution is a set of concrete, inspectable counterexamples to interpreting aggregate detector alerts as evidence that the intended behavior is being measured. It is not a replacement detector or a prevalence estimate of detector failure.

Critical strengths

S1

Independent auditing work remains scarce in the rapidly evolving subfield of AI safety. This paper partially fills the gap left by the lack of independent audits of AI safety detectors.

Critical weaknesses

W1: Asymmetric depth of analysis

The audit of DualSentinel largely amounts to the discovery of implementation-level bugs, whose scientific value may erode once the issues are patched. The audit of ICLScan, on the other hand, is primarily a diagnosis of lexical coverage fragility rather than a substantive demonstration that a valid, selective, and capability-preserving backdoor is actually missed by the detector. As a result, neither thread fully delivers the stronger security claim that the framing invites.

W2: Limited generalizability:

The study is confined to only two detectors, a small set of synthetic tasks, and predominantly a single model. Given its narrow scale, weak generalizability, and the absence of any proposed alternative, the paper's contribution lies more in sounding an alarm than in solving the problem.

Questions for authors

Can the audit paradigm enable detection on black-box models? ICLScan's detection stage is in principle compatible with black-box access, but the paper provides no validation of this scenario.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.