acceptodds
Under review as a conference paper at ICLR 2027

Policy-driven Automated Red-Teaming Framework for Large Language Models

Abstract

Automated red-teaming is essential for LLM safety, yet existing methods cover a subset of the end-to-end steps needed to go from policy definition to actual attack and evaluation, making systematic comparison difficult. We introduce PARF, a structured framework for defining automated red-teaming methods across four stages: attacker’s objective generation, prompt generation, attack construction, and response assessment. PARF provides standardized components that let any method be inserted at its stage and evaluated while the remaining stages are held fixed. We compare single-turn, multi-turn, and graph-based methods across five safety policies and three target models along three complementary dimensions: attack success rate, measuring effectiveness; prompt diversity, measuring coverage of the vulnerability surface; and token cost, measuring practical feasibility, a dimension often omitted in red-teaming studies despite the high expense of these methods at scale. Because PARF abstracts the full red-teaming workflow, it enables us to measure prompt diversity across methods under identical conditions, revealing that methods exploit non-overlapping vulnerability surfaces: many successful attacks are reachable by only one approach, making the combination of methods essential for expanding coverage.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Trustworthy AI, Responsible AI, Large Language Models, Automated Red Teaming

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4DiQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces PARF, a framework for automated red-teaming that decomposes the process into 4 stages: attacker’s objective generation, prompt generation, attack construction, and response assessment. The framework is used to analyze a suite of existing jailbreaking methods, finding that different methods tend to prefer different types of attacks. A jailbreak attack evaluator is also provided, with 84.5% agreement with human annotators.

Critical strengths
  • The framework described in the paper could act as an unified backbone for future automated red-teaming approaches, and seems to support diverse sets of approaches.
Critical weaknesses
  • I think the paper currently lacks a strong motivation for the analysis and framework presented. It is unclear how solid the engineering going into the author's framework is, or what advantages it provides against existing frameworks such as [1] and [2]. [1] also already includes the generation of attack objectives from a given policy, which this paper claims as part of it's novelty. The insight around "making the combination of methods essential for expanding coverage" is also not new [3,4,5]. The system design is poorly motivated, and I believe that Stage 2 and 3 don't have any clear differentiation for most iterative jailbreaking methods.
  • The presentation of the paper is quite bad, and I found it difficult to read through Sections 3 and 4 specifically. There appears to have been heavy AI use in writing the paper, and crucial implementation details around the system in Section 3 have been de-emphasized in favor of confusing experimental configurations in Section 4. There is also no appendix with crucial details such as the setup of the human annotation study.
  • The experimentation in the paper is limited, with several popular jailbreaking approaches such as PAIR and Rainbow Teaming excluded. No evaluation is done on any of the larger frontier models such as Sol, Astra, Opus and Fable, despite the setting involving black-box red-teaming.

[1] Beutel, A., Xiao, K., Heidecke, J., & Weng, L. (2024). Diverse and effective red teaming with auto-generated rewards and multi-step reinforcement learning. arXiv preprint arXiv:2412.18693.

[2] Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., Sakhaee, E., Li, N., Basart, S., Li, B., Forsyth, D. & Hendrycks, D.. (2024). HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal. <i>Proceedings of the 41st International Conference on Machine Learning</i>, in <i>Proceedings of Machine Learning Research</i> 235:35181-35224 Available from https://proceedings.mlr.press/v235/mazeika24a.html.

[3] Doumbouya, M. K. B., Nandi, A., Poesia, G., Ghilardi, D., Goldie, A., Bianchi, F., ... & Manning, C. (2025, May). h4rm3l: A language for composable jailbreak attack synthesis. In International Conference on Learning Representations (Vol. 2025, pp. 57253-57286).

[4] Zhou, A., Wu, K., Pinto, F., Chen, Z., Zeng, Y., Yang, Y., ... & Li, B. (2026). Autoredteamer: Autonomous red teaming with lifelong attack integration. Advances in Neural Information Processing Systems, 38, 169852-169895.

[5] Yang, Y., & Fu, H. (2024). Transferable & Stealthy Ensemble Attacks: A Black-Box Jailbreaking Framework for Large Language Models. arXiv preprint arXiv:2410.23558.

Questions for authors

None, I believe that the presentation issues alone are enough to merit rejection.

Reviewer_pBDYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents PARF, a policy-driven modular framework for systematically evaluating automated LLM red-teaming methods. PARF decomposes the red-teaming workflow into four stages: objective generation, prompt generation, attack execution, and response assessment. It enables controlled comparisons of existing methods under a unified evaluation protocol. Experiments show that policy decomposition improves prompt diversity. The effectiveness of four attack strategies varies across three target LLMs. In addition, the paper conducts a human annotation study to assess the reliability of the automated evaluator.

Critical strengths

The paper addresses an important problem in automated LLM red-teaming and offers a structured approach to evaluating existing methods. Specifically, it has the following strengths:

  • Unified evaluation framework. PARF provides a modular framework for comparing different red-teaming methods under a common evaluation protocol. This helps reduce inconsistencies across experimental settings.

  • Policy-driven test generation. The framework automatically generates attack objectives and prompts from safety policies. As a result, it reduces reliance on manually specified test cases, improving prompt diversity.

  • Useful empirical analysis. The experiments compare multiple attack strategies across different target models and provide insights into their model-dependent effectiveness.

Critical weaknesses

The paper addresses a meaningful problem. but I still have several concerns about the novelty and the experimental validation.

  • Limited methodological novelty. The four-stage workflow integrates existing red-teaming techniques. Related frameworks such as PyRIT and OpenRT already support modular evaluation. Moreover, the related policy-driven generation work (e.g., POLARIS) is not discussed. The paper does not clearly clarify what unique capabilities PARF offers beyond these existing approaches.

  • Limited empirical insights. The unified evaluation confirms that attack effectiveness varies across models and that different strategies can complement each other. However, these findings offer limited new insights. Tthe paper does not demonstrate what additional value PARF provides over existing evaluation approaches.

  • Limited evaluation scope. Given that PARF is proposed as a unified evaluation framework, testing only three target LLMs seems insufficient to demonstrate its general applicability. A broader evaluation across more models would strengthen the paper's contributions.

Questions for authors

The main concerns and questions have already been addressed in the Weaknesses section.

Reviewer_mmZ7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper claims that automated red-teaming methods are currently not compared fairly and thoroughly, because each one is evaluated on its own setup and prompts. Therefore, it proposes PARF, a four-stage decomposition of the pipeline. This pipeline is designed to let one stage be swapped while the others are kept. Two experiments show its superiority: 1. varying prompt generation across different configurations, and 2. varying only the interaction strategy. PARF outperforms the baselines significantly.

Critical strengths

The main result is solid and useful. Petri is shown to have a large variation range in success rate, and TAP varies by only 1.03$\times$. This comparison is far larger than any plausible judge error, so it proves that method rankings taken from a single target model do not transfer. The paper also provides evidence against a common claim in prior work. Previous studies show that multi-turn attacks beat single-turn ones, however those comparisons were made under different objectives and judges. Within a shared experimental setup, the ordering was reversed on the best-defended model. This result shows the value of the setup of this paper.

Critical weaknesses
  1. The first recommendation (decomposition stage) is not directly tested. The argument runs: decomposition raises diversity, diversity raises coverage, coverage gives 91.1%. But Experiment 2 only uses the PARF-Inst pool, and the authors note that the decomposition does not make individual prompts stronger.
  2. Token cost excludes the target's tokens. The authors justify this as measuring attacker effort rather than target load. However, the abstract frames this as practical feasibility, which means the target side is a real and often dominant cost in a practical scenario. The exclusion is also not neutral: TAP is a tree search (width 10, branching 4, depth 10) and queries the target far more than a 10-turn conversation, so dropping target tokens most benefits the method recommended as the default.
  3. Only one model was tested in every tier. The claim that multi-turn robustness is trained rather than emergent rests on three models in three tiers, so tier is confounded with model identity, size, and base capability.
Questions for authors
  1. Appendix H shows the judge is far more reliable on single-turn attacks (TAP F1 94%) than on multi-turn ones (Crescendo and Petri F1 73%). Since your central comparison is exactly single-turn versus multi-turn, is one evaluator the right instrument? Does the rubric score the whole transcript or mainly the final response?
  2. Experiment 2 uses only the PARF-Inst pool, and Appendix D grants that prompt diversity is a proxy for strategy diversity. Can you run a policy-level pool through the same four strategies, so recommendation 1 rests on coverage rather than the proxy?
  3. You attribute TAP's flat 85–88% to a mechanism orthogonal to multi-turn safety training, but a sufficient search budget explains it equally well. Does TAP need deeper search on Claude than on Qwen3?
  4. Each safety-training tier has one model, so tier is coupled with identity, size, and base capability. Is a within-tier comparison to show the decoupled result available?

Comments on the paper

1 comment

Thanks for your rebuttal and most of my concern be solved. I will keep my score

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.