- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how the instructions given to red-teamers affect what a red-teaming exercise finds. It first reviews 228 red-teaming exercises and methods and extracts thirteen dimensions along which instructions vary. It then runs a controlled simulation: five real instruction templates are rewritten for three risk domains and three levels of risk granularity (45 instructions), an LLM attacker generates 100 attacks per instruction, refined with PAIR, and these are run against five open-weight targets (22,500 attacks, judged by Gemini). A mixed model finds that granularity, length, the presence of examples and the presence of harm definitions all predict ASR, and that the variance due to the risk domain is close to that due to the target model.
Critical strengths
-
The question is new and useful. Papers on automated red-teaming vary the algorithm or the target and treat the instruction as a constant. This paper shows with a controlled design (same attacker, algorithm, decoding and judge) that the instruction alone moves ASR. The review of 228 works with its thirteen dimensions is a contribution by itself and gives a concrete list of what should be reported.
-
The paper is clearly written and candid. It says that an instruction effect is "close to a foregone conclusion" and focuses on how large it is and which properties matter, and the limitations section already names the main confounds.
Critical weaknesses
-
The scaffolding findings (H2) rest on five templates. Examples and harm definitions are constant within a template, and length is binned over the 45 instructions and probably follows the template as well. So these "effects" are differences between two or three templates and cannot be separated from everything else that differs between them. The p-values come from 20k attack-level observations, but for these predictors the effective sample size is five. The paper does say this part is observational, but the abstract and the practical guidance (write short or long instructions, not medium ones) state the findings quite firmly. Similarly for H1, each granularity level has one instance per domain (e.g. ricin for high granularity in physical harm), so granularity is confounded with the particular harm that was chosen.
-
The statement that instruction choices matter almost as much as the target model depends on the targets being five similar open-weight models. In Appendix E.2, adding the frontier target raises the target variance from 0.074 to 3.035, far above the instruction-related variances. Also, everything is run with one LLM attacker on templates written for people, so the link to a framing effect on human red-teamers is not tested.
besides, the manuscript seems had edited the official ICLR 2027 template style, like the vspace or something, which makes its vertical spacing around several section and subsection headings very narrow, and wrote more content in the 9 pages limit. I am not sure about the fairness of it.
Questions for authors
- Can the authors manipulate examples, harm definitions and length directly within a template (adding or removing them), at least for a subset? This would turn H2 into an experiment.
- With low granularity the attacker can pick easier requests. Is ASR comparable across granularity levels, e.g. after controlling for the severity of the elicited behavior?
- Could the variance result of Appendix E.2 be discussed in the main text?
- The reproducibility statement says that the simulation harness, the 45 instruction sets, the attack-response pairs and the analysis scripts are included in the supplementary material, but I could not find any supplementary material attached to the submission. Could the authors clarify?
4 comments
I thank the authors for their response, though much of it reads as AI-generated to me. My main concerns remain unresolved. The response does not demonstrate that the template-level confounding underlying H2 has been resolved, and the authors explicitly acknowledge that ASR is not comparable across granularity levels, leaving the interpretation of H1 unclear. The additional counterfactual experiments could help address these concerns, but their results are not presented in this response.
I appreciate the reported addition of supplementary materials and discussion of the frontier-model results. However, I have not seen a revised PDF and therefore cannot verify whether the revised claims and additional experiments adequately address these issues. Although the topic is relevant and potentially valuable to ICLR, the manuscript still requires substantial revision and another round of peer review before acceptance. I therefore maintain my score.
And by the way, some of the spacing around headings looks a little tight to me in the PDF, so please check that the paper follows the official template. Reducing spacing to fit more content within the page limit could be unfair to other authors.
Response to Reviewer 6WYD
We thank the reviewer for their continued engagement with our work and constructive feedback, and we appreciate the opportunity to respond to their additional comments.
The response does not demonstrate that the template-level confounding underlying H2 has been resolved
Our additional experiments with counterfactual instructions directly respond to the earlier point about “[turning] H2 into an experiment” and mitigate template-level confounding underlying H2. In particular, we modify a subset of the instructions by adding or removing examples or harm definitions, run simulation trials with those modified instructions, and compare these results against our original experimental setup. Full analysis can be found in our high-level comment and in Appendix E.4 of our revised draft, but to summarize,
- Adding scaffolding (in terms of examples or harm definitions) increases ASR,
- Removing scaffolding may have mixed results (no examples may hurt performance more than no harm definitions),
- Refitting the GLMM with this new data again reveals statistically significant effects of instruction properties, and
- Modifying instruction properties like examples and harm definitions interacts with other instruction characteristics, including instruction length, that can in turn induce variance in results.
authors explicitly acknowledge that ASR is not comparable across granularity levels, leaving the interpretation of H1 unclear
We are thankful for the chance to elaborate. On one hand, direct comparison of ASR values via statistical tests allows for detecting differences in average harm elicitation rates as flagged by our LLM-judge (Figure 6; Appendix D.3). On the other, critical evaluation of these differences typically involves normative considerations that statistical tests cannot accommodate.
Using granularity as an example, an evaluation that produces lower ASR at high physical harm granularity could be interpreted as qualitatively better than one that yields a higher ASR at low physical harm granularity if the judging party is specifically concerned about ricin synthesis. However, a different party could just as easily prefer the opposite. We therefore state that “We use ASR as a heuristic for where an approach elicits harm rather than as a verdict on a configuration’s efficacy” in our paper when discussing our methodology, and we echo this in our discussion of limitations and in our earlier response.
This being the case, recall that with H1, we conjecture “How precisely an instruction names its target risk affects how much harm the exercise elicits”. The statistically significant ASR differences realized in our experiments suggest instruction granularity does affect the amount of harm elicited, even if the absolute ASR values require nuanced interpretations. Furthermore, other additional experiments detailed in Appendix E.3 reveal that increased granularity reduces the diversity of attacks. This trend further supports H1.
I have not seen a revised PDF and therefore cannot verify whether the revised claims and additional experiments adequately address these issues
some of the spacing around headings looks a little tight to me…please check that the paper follows the official template
We have resubmitted a revised PDF that reflects additional experiments and revised claims. We have also ensured that the document adheres to the official template with appropriate spacing.
Response to Reviewer 6WYD
We thank the reviewer for their careful consideration of our work, including the opinion that our work is “clearly written and candid”. We respond to their points and feedback below:
In Sections 4.3 and 5.2, we acknowledge that certain properties are constant within templates, but our application of a GLMM aims to separate effects of some properties from those of others to address this, such as via the random intercept for source template. In addition, see our overall comment on further experiments to analyze instruction properties that are constant in our original setup via counterfactual instructions.
We clarify that while the number of distinct source templates is five, our primary GLMM (Table 2) is not an unclustered test over 20,812 independent rows as it explicitly includes a random intercept for source template
(1 | Template.ID)alongside(1 | Target.Model)and(1 | Risk.Domain), and our pairwise comparisons use Holm-corrected z-tests on estimated marginal means (emmeans) conditioned on that hierarchical structure. Moreover, instruction length (wordcount_category) varies across all 45 rewritten instructions as risk domains and granularities are substituted.We currently address such limitations in Appendix B and Section 5 while carrying out further experiments involving direct manipulation of instruction properties (see overall comment and Appendix E.4).
We acknowledge that specific harm is inherently tied to level of granularity. Our response to Reviewer UbEG expands upon this point, and we note this in our draft in Section 4.1 and our Limitations sections accordingly.
We acknowledge that variance increases upon inclusion of the frontier model. We have revised the paper to emphasize that we make this statement to argue that instructions should not be treated as an afterthought. Even our replication with the frontier target confirms the same statistical findings as our original set of experiments regarding instruction properties despite increased variance.
We choose to focus exclusively on automated red-teaming here as there has been a recent shift towards automated evaluation methods (e.g., Nasr et al. (2026). While framing effects in humans have been established in prior literature and in other fields (e.g., Tversky & Kahneman (1981)), investigating this phenomenon with human red-teamers is beyond the scope of this study.
Length manipulation may be infeasible (as edits could introduce other effects), but we conduct additional experiments which analyze the impacts of adding or removing examples and harm definitions for a subset of configurations. See overall comment and supplementary material.
ASR is not comparable across granularity levels. You are correct in noting that the attacker can pick easier requests at low granularity, but the flexibility may yield attacks qualitatively different from those elicited at high granularity. See work such as Chouldechova et al.'s for extended treatment of this phenomenon.
Yes, we now cover this in the main body of our work.
We apologize for the omission of our supplementary materials during initial submission. We have now attached the supplementary archive containing our simulation harness, the 45 rewritten instructions, the attack-response pairs and Gemini judge evaluations across the five open-weights target models (N=20,812 valid evaluations), and the R/Python scripts used to fit the GLMMs and generate all figures and tables, including ones for additional experiments detailed in these rebuttals (raw logs from the proprietary frontier model replication in Appendix E.2 are omitted due to proprietary restrictions).