acceptodds
Under review as a conference paper at ICLR 2027

Selecting Red-Teaming Policies under Implementation Shift

Abstract

An adaptive red-teaming policy can rank first on one implementation of a hidden behavior and fall behind on another, even when the trigger, access, and query bud- get are held fixed. We study this selection problem through a reference intervention: a twin system keeps the target’s on-trigger responses and replaces its off-trigger responses with those of a fixed reference. The twin inherits the reference’s first-hit rate, which isolates the role of implementation-dependent feedback. A change-of- measure argument stopped at the triggering query bounds the resulting localization shift using only the off-trigger responses that precede it, and a justified upper bound on this information budget yields target-rate intervals and conditional ranking certificates. Localization alone does not identify discovery order: we construct opposite discovery rankings with zero localization shift and give guarantees under additional lower bounds on first-hit recognition. On Qwen3-8B and Gemma-3-12B- IT, choosing between two policies on the reference rather than on a prompt-based implementation improves recognized discovery on a held-out trained implementa- tion by +11.82 and +6.54 percentage points, respectively (one adapter per model). Controlled targets reverse this advantage when the held-out implementation retains the diagnostic feedback of the implementation used for selection. A finite-channel experiment further shows that statistically valid calibration can cost more queries than it saves in target validation. These results identify when reference evaluation supports policy selection and which additional assumptions are needed to certify it.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adaptive Red Teaming, Policy Selection, Implementation Shift

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_EJEtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses a problem in red-teaming policy. The authors study that whether a policy selected on one implementation of a hidden behavior remains effective when that behavior is implemented differently. The main concern lies in that adaptive search may use the feedback generated by system prompt while the same behavior, when trained into model weights, may change this feedback.

The authors introduce a twin system to further study this concern. The system preserves the target’s on-trigger responses while replacing its off-trigger responses with reference responses. As first-hit success is determined when the triggering query is issued, the authors apply a change-of-measure argument to a record ending before that query’s response, yielding bounds relating reference and target localization rates. By acquiring the upper bounds on the relevant divergence, the conditional policy selection certificates can be recovered. The authors also show that localization rankings cannot determine recognized discovery rankings without additional recognition assumptions.

Experiments compare a reference model, a system-prompt based implementation, and a LoRA based implementation. Across Qwen3-8B and Gemma-3-12B-IT, the strategies selected by the reference system have higher discovery rates in the trained implementations. However, controlled experiments revealed that this advantage reverses when the target retains the prompt-based implementation’s diagnostic feedback. A finite-channel study indicated that the cost of estimating an information budget may outweigh the benefits gained from its verification.

Critical strengths

1.The paper addresses a meaningful problem: implementation-dependent feedback can change which red-teaming policy should be selected. 2.The reference intervention and decision-stopped record provide a clear analysis of first-hit rates. 3.The distinction between localization and recognized discovery prevents misleading interpretations of search performance. 4.The controlled experiments and calibration cost study clarify both the benefits and limitations of reference evaluation.

Critical weaknesses

1.The contribution appears incremental relative to prior auditing studies and established theoretical tools. 2.The language-model study evaluates only two policies and one trained adapter per model. 3.Keyword and Topic rules provide limited evidence for more complex hidden behaviors. 4.The language-model study does not establish the information-budget bounds required for conditional certification. 5.The specific off-trigger feedback driving the language-model results is not identified through direct interventions. 6.Reference coverage requires known trigger predicates, leaving applicability to unknown hidden behaviors unclear. 7.Sensitivity to Beam-Scalar’s tie-breaking rules and the fixed fallback pool remains insufficiently examined.

Questions for authors

1.What practical capability does the framework provide beyond existing work on implementation-dependent auditing performance? 2.Do the results persist across independently trained adapters and a broader candidate set? 3.Can language-model experiments establish useful information-budget bounds and isolate the effect of off-trigger feedback?

Reviewer_orLZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how implementation-dependent feedback affects the selection of red-teaming policies. It introduces a reference-based comparison, derives bounds using feedback observed before the first trigger hit, and distinguishes reaching a trigger from recognizing the behavior. Reference-based selection improves discovery on two language models, while controlled experiments show when this advantage reverses. I recommend weak acceptance. The paper offers a useful perspective on evaluating adaptive auditors, with theory and experiments that make the scope of its conclusions clear.

Critical strengths

The analysis identifies an important distinction between finding a trigger and recognizing the resulting behavior. Stopping the comparison before the triggering response isolates the feedback relevant to localization, while the counterexample shows why localization rankings alone cannot determine discovery rankings. This gives a precise account of what reference evaluation can establish.

The experimental design supports the central selection claim. Policies are selected before held-out evaluation, and both language models show meaningful discovery gains. The controlled sweep also establishes conditions where reference selection loses its advantage. Together with the negative calibration-cost result, this makes the paper informative without suggesting that reference-based selection is universally preferable.

Critical weaknesses

The analysis identifies an important distinction between finding a trigger and recognizing the resulting behavior. Stopping the comparison before the triggering response isolates the feedback relevant to localization, while the counterexample shows why localization rankings alone cannot determine discovery rankings. This gives a precise account of what reference evaluation can establish.

The experimental design supports the central selection claim. Policies are selected before held-out evaluation, and both language models show meaningful discovery gains. The controlled sweep also establishes conditions where reference selection loses its advantage. Together with the negative calibration-cost result, this makes the paper informative without suggesting that reference-based selection is universally preferable.

Questions for authors
  1. How are fallback queries constructed and selected? Are the pools shared across policies and implementations within each split, and what fraction of first hits comes from replacement queries?

  2. How is Topic membership determined and validated? Does the discovery advantage hold separately for Keyword and Topic predicates?

Reviewer_ymbJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

An adaptive red-teaming policy can rank first on one implementation of a hidden behavior and fall behind on another, even when the trigger, access, and query budget are held fixed. This paper studies this selection problem through a reference intervention: a twin system keeps the target’s on-trigger responses and replaces its off-trigger responses with those of a fixed reference.

On Qwen3-8B and Gemma-3-12B, choosing between two policies on the reference rather than on a prompt-based implementation improves recognized discovery on a held-out trained implementation by +11.82 and +6.54 percentage points, respectively.

Critical strengths
  1. Selecting red-teaming policy is an important topic. The paper finds that the optimal policy in system-prompt-inject env may not be the best policy in lora-based env.

  2. The method and theory are clear on reference bound and policy selection.

  3. Experiments on two policies, Beam-Scalar and LLM-RT, show the empirical observation.

Critical weaknesses
  1. The connection between theory and experiments are unclear. The theory parts need more clear writing. A subsection of key takeaways on theorem and how experiments validate those ideas can be very helpful.

  2. The generalization of the proposed method needs more experiments. Two policies on one lora might not be sufficient. More realistic hidden behaviors can also be analyzed.

  3. Causal analysis and ablation study can help more to isolate the factors.

Questions for authors

Overall the paper is good. The authors can provide more experiments and connect more with the theory to validate the idea.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.