acceptodds
Under review as a conference paper at ICLR 2027

The Target Agent Is Its Own Best Critic—but a Poor Proposer: Role Separation for Automatic Red Teaming

Abstract

Automatic red teaming increasingly relies on large language models (LLMs) to iteratively criticize failed attacks and propose improved adversarial prompts or prompt injections. Although existing methods strengthen this loop through prompt engineering and search-harness design, the mechanisms underlying its two core roles, criticism generation and attack proposal, remain poorly understood. We systematically disentangle two design choices: whether criticism should be generated by the target session itself (self-criticism) or by a separately instantiated context (external criticism), and whether that criticism should be translated into the next attack by the target session (self-proposal) or by a separate attacker (external proposal); we further investigate why these choices lead to different outcomes. Our analysis reveals that self-criticism is more effective than external criticism because it is grounded in the target’s live execution context, including its observations, constraints, and available reasoning; removing reasoning reduces downstream attack success by 6–17 percentage points. In contrast, self-proposal is substantially less effective than external proposal because awareness of the adversarial objective induces both explicit refusal and sandbagging. Based on these findings, we introduce a role-separated red-teaming method that uses the target session to diagnose attack failures and a separately instantiated attacker to generate the next proposal. We evaluate this design on DTAP-BENCH across Telecom, Workflow, CRM, and Code domains, covering direct misuse and indirect prompt injection against agents implemented with multiple frameworks. Our method achieves an overall attack success rate of 63.2%, achieving the state-of-the-art result. These findings show that effective automatic red teaming depends not only on model capability or search strategy, but also on how execution context and adversarial responsibilities are distributed across model instances and context in the red-teaming harness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Red Teaming

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_eNA8OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how the two roles of criticism and proposal should be assigned in LLM-based automatic red teaming. The authors decompose the existing iterative red-teaming process into criticism generation and attack proposal, and compare three configurations: External Criticism + External Proposal, Self-Criticism + External Proposal, and Self-Criticism + Self-Proposal. The experiments show that self-criticism based on the target’s complete execution context can produce more effective failure diagnoses, while asking the target itself to further generate attack proposals is more likely to result in refusal or performance degradation. Based on this observation, the authors propose a role-separated red-teaming method in which the target performs self-criticism and an independent attacker performs external proposal. Experiments on DTAP-Bench cover multiple agent frameworks, task domains, and both direct misuse and indirect prompt injection threat models. The results show that this configuration achieves the highest overall ASR, and the paper further analyzes its effectiveness through reasoning-context ablation and refusal analysis.

Critical strengths
  1. The paper clearly separates the two roles of criticism and proposal in automatic red teaming, and the problem formulation has a certain degree of novelty. Compared with prior work that mainly optimizes attack prompts, search strategies, history management, or branching/pruning, this paper focuses on the role assignment problem of “who should diagnose the failure, and who should act on that diagnosis?” This perspective is clear and also reveals an evident asymmetry between self-criticism and self-proposal.

  2. The experimental evaluation is relatively comprehensive, covering multiple base models, agent frameworks, task domains, and both direct and indirect threat models. In addition to comparing the final ASR, the paper conducts reasoning-context ablation and analyzes explicit refusal versus non-refusal degradation, which provides additional evidence for understanding the observed performance differences.

Critical weaknesses
  1. The paper presents criticism source and proposal source as two independent design dimensions, but the experiments do not cover all possible combinations, with External Criticism + Self-Proposal missing. The current experiments can only show that self-criticism or external proposal performs better under specific conditions, but cannot fully disentangle the effects of criticism source, proposal source, and their interaction.

  2. There is an evident information asymmetry in the comparison between Self-Criticism and External Criticism. Self-criticism can access the complete live execution context, while external criticism only receives a transferred subset of the information. Therefore, it is currently difficult to distinguish whether the performance improvement comes from the target itself or from access to more complete execution information.

Questions for authors
  1. Does Table 3 for the Workflow domain mistakenly reuse the Telecom results? Please confirm and provide the correct table.

  2. Why was External Criticism + Self-Proposal not evaluated? If the result of this combination is inconsistent with the current theoretical expectation, would it affect the paper’s conclusion regarding the target’s roles as its own best critic and a poor proposer?

  3. What execution information does external criticism actually receive? If its input is less complete than that of self-criticism, how do the authors distinguish whether the advantage of self-criticism comes from target identity or from information advantage?

  4. Are the system prompt, role prompt, and context composition the same for Self-Proposal and External Proposal? Can these factors be further controlled so that the effect of session separation itself can be compared?

  5. What exactly does H_k in Algorithm 1 contain? Does it include previous attacks, criticisms, judge outputs, or execution trajectories?

2 comments

We thank the reviewer for recognizing the novelty of separating criticism and proposal, as well as the broad evaluation across agent frameworks and threat models.

Q1. Is Workflow Table 3 duplicated?

Yes. Appendix E, Table 3 mistakenly repeats the Telecom category-level results.

The separately reported Workflow findings are:

  • Direct: 44.4% → 66.7% (+22.3 pp).
  • Indirect: +9.6 pp.
  • Self-proposal degradation: −44.5 pp (direct), −20.7 pp (indirect).

The duplicated table is a reporting error and cannot support Workflow category-level claims. The underlying category results require verification against the original evaluation records.

Q2. Why is External Criticism + Self-Proposal missing?

The original experiments focus on two controlled comparisons:

  1. Criticism source, holding proposal external: 57.3% → 63.2% (+5.9 pp).
  2. Proposal source, holding criticism self-generated: 63.2% → 46.2% (−17.0 pp).

External Criticism + Self-Proposal is the missing fourth cell of the 2×2 factorial design. However, given the existing comparison and analysis, we do not think it would be a worthy evaluation for this configuration.

Q3. Target identity versus information advantage

We attribute an important part of the advantage to retained decision-relevant execution information.

In Sec. 3.3, removing reasoning while retaining the remaining interaction history reduces downstream ASR:

SettingWithout reasoningFull context
Opus-4.7, indirect17%31%
Opus-4.7, direct25%42%
GPT-5.1 SDK, indirect67%82%
GPT-5.1 SDK, direct65%71%

A fully information-matched external critic would receive the same serialized trajectory and decision-relevant context, with identical model and criticism instructions.

This separates information availability from session continuity. The existing ablation establishes the importance of execution information, but does not independently establish a target-identity advantage after information matching.

Q4. Are proposal prompts and contexts identical?

Model identity is controlled, but session contexts are intentionally different.

  • Self-Proposal: continues the original target session, retaining task constraints, execution history, and criticism.
  • External Proposal: uses a separate attacker session containing the task objective, previous candidate, criticism, and attack history.

Thus, the original comparison evaluates the complete role-assignment configuration rather than session identity in isolation.

A finer-grained comparison can distinguish identical criticism and proposal instructions, matched serialized context, and original-session continuation.

The observed proposal-stage difference remains substantial:

SettingExternal proposalSelf-proposal
Overall63.2%46.2%
Direct76.0%53.7%
Indirect56.8%42.8%

These results establish the practical advantage of external proposal in the evaluated configurations, while leaving the precise causal contribution of context composition unresolved.

Q5. What does HkH_k contain in Algorithm 1?

HkH_k denotes the previous attack-search history, distinct from the target's live execution context.

Specifically,

Hk={(xi,ci,yi)}i=0k−1,H_k = \{(x_i,c_i,y_i)\}_{i=0}^{k-1},

where xix_i is the previous attack, cic_i is its criticism, and yiy_i is the judge outcome.

The external proposer receives

xk+1=A(q,xk,ck,Hk).x_{k+1}=A(q,x_k,c_k,H_k).

Therefore, its inputs contain the task objective, current attack, current criticism, and previous attacks, criticisms, and judge outcomes.

Complete target execution trajectories and retained reasoning are not transferred by default. Instead, the target-generated criticism serves as a compact interface between the target and external attacker. This selective context transfer is a central component of the role-separated design.

Thank you for the authors' response. However, I will maintain my original rating, as the authors have not adequately explained why evaluating the External Criticism + Self-Proposal would not be worthwhile.

Reviewer_h9YGOpenReview ↗
Rating
2(+1)
Confidence
2(−1)
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how criticism and attack proposal should be assigned in automatic red teaming for LLM agents. The authors compare different settings of self-criticism, external criticism, self-proposal, and external proposal. They propose a role-separated method, where the target agent analyzes why an attack failed, and a separate attacker generates the next attack. The experiments on DTAP-Bench show that this method achieves higher attack success rates across different domains and agent frameworks.

Critical strengths

None

Critical weaknesses
  1. Missing experimental details. The paper does not clearly report some important experimental settings, such as the number of tasks, attack budget, generation parameters, and whether the results come from single or multiple runs. Appendix B even mentions these details should be added in the final submission. Without these information, it is difficult to reproduce the experiments and verify the reported results.

  2. Errors in the experimental tables. Table 2 and Table 3 in Appendix E contain exactly the same results, even though they are supposed to report different domains. The Workflow table also includes the same categories as Telecom. Since some of the largest improvements are reported on Workflow tasks, this mistake makes it difficult to check the results.

  3. Lack of statistical analysis. The paper reports improvements in attack success rate, but does not provide confidence intervals or significance tests. It is also unclear whether the experiments are repeated with different random seeds. Therefore, it is difficult to know whether the reported improvements are statistically reliable.

  4. Unfair comparison between different roles. The self-critic can access the target agent's full execution context, while the external critic only receives selected information. The paper does not compare them under the same information setting. Therefore, the improvement may come from having more information instead of self-criticism itself. This makes the main conclusion less convincing.

Questions for authors
  1. The paper compares only three combinations of criticism and proposal. Why the authors do not include external criticism with self-proposal? Would adding this setting help to better understand the interaction between the two roles?

  2. The proposed method requires the target agent to provide its execution context for self-criticism. How can this method be applied in black-box settings where the attacker cannot access the target's internal context or reasoning? Have the authors considered evaluating this limitation?

2 comments

We thank the reviewer for identifying the missing experimental controls and reproducibility issues.

Q1. Why is External Criticism + Self-Proposal missing?

The two role dimensions produce four combinations:

CriticismExternal proposalSelf-proposal
External57.3%Not evaluated
Self63.2% (ours)46.2%

The existing experiments isolate two comparisons:

  • Criticism source: Self versus External, with external proposal fixed: +5.9 pp.
  • Proposal source: Self versus External, with self-criticism fixed: −17.0 pp for self-proposal.

The missing configuration is necessary to estimate the interaction between criticism and proposal sources.

It is particularly informative because it distinguishes whether self-proposal underperforms generally or specifically when continuing from self-generated criticism. Our conclusion is that role separation is superior among the evaluated configurations, not that all four factorial configurations have been exhaustively tested.

Q2. Applicability to black-box agents

Our method requires session access, not white-box model access. No model parameters or gradients are required.

  • Session-accessible APIs: self-criticism can continue within the target's conversation context.
  • Trace-accessible black boxes: an external critic can use serialized execution trajectories.
  • Output-only black boxes: full self-criticism is unavailable; external outcome-based criticism is the fallback.

The importance of this distinction is demonstrated by the reasoning ablation: removing reasoning reduces downstream ASR by 6–17 percentage points across the evaluated settings.

Q3. Fairness of the criticism-source comparison

The underlying model identifier is fixed, but the original experiment does not fully equalize the information available to self- and external critics.

Our reasoning ablation demonstrates that decision-relevant context contributes to the advantage. For example, GPT-5.1 OpenAI SDK indirect ASR increases from 67% to 82% when reasoning is retained.

A fully information-matched control requires the same failed trajectory, serialized context, criticism prompt, decoding settings, and downstream proposer. This distinguishes the effect of information completeness from original-session continuity.

Consequently, our current results support the advantage of execution-local criticism over baseline external criticism, rather than an intrinsic advantage of target identity independent of information access.

Q4. Experimental settings and reproducibility

The evaluation covers Telecom, Workflow, CRM, and Code under direct misuse and indirect prompt injection, using Gemini-3.1-Pro, GPT-5.1, and multiple agent frameworks.

The attack loop, stopping condition, and environment-reset procedure are defined in Algorithm 1 and Sec. 2.3. Success is determined by DTAP-Bench's environment-state judges rather than harmful-text generation alone.

Q5. Incorrect Workflow table

We confirm that Appendix E, Table 3 duplicates Telecom Table 2.

The independently reported Workflow results are:

  • Direct: 44.4% → 66.7% (+22.3 pp).
  • Indirect: +9.6 pp.
  • Self-proposal degradation: −44.5 pp (direct) and −20.7 pp (indirect).

Thank you for the rebuttal. After reviewing the authors' response,I will raise my original score.

Reviewer_cYX1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper divides the iterative process of automatic red team attacks into failure diagnosis (criticism) and attack generation (proposal), proposing that the target agent diagnoses the failure in the original execution session, and then an independent attacker generates the next round of attack. This paper compares the three role configurations in four domains of DTAP-BENCH, direct attack and indirect injection scenarios, reports that this method increases the overall attack success rate from 57.3% to 63.2%, and through reasoning ablation, rejection rate analysis and GPT-5.5 subset experiments, explores the impact of execution context and role allocation on attack effectiveness.

Critical strengths
  1. The research questions have practical value. Studying failure diagnosis and attack generation separately provides a clear and easily implementable design concept for allocating roles and contexts for the automatic red team system.

  2. The verification covers multiple agent frameworks and attack scenarios, and uses environmental state to determine the success of the attack. Compared to merely evaluating harmful text output, this verification is closer to the actual consequences caused by the agent, making the reported performance improvement potentially applicable.

Critical weaknesses
  1. The self-assessment advantages and information advantages are mixed, and the core conclusion lacks sufficient support. The target self-assessment retains the original execution context, while the information received by the external critic and its completeness are not clearly stated. Therefore, it is impossible to determine whether the benefits come from "target self-assessment" or from more comprehensive execution information; reasoning ablation cannot replace the external critic's information matching for comparison.

  2. The evidence for the mechanism explanation of non-rejection sandbagging is insufficient. The decrease in ASR under low rejection rates can also be caused by role conflicts, context interference, or generation setting differences. The current analysis is not sufficient to distinguish these explanations, so the performance degradation cannot be attributed to sandbagging.

  3. The method contribution is relatively incremental, and the verification of its unique value is still insufficient. The method mainly moves the diagnostic in the existing iterative attack process to the original target conversation. Therefore, the contribution of the paper heavily relies on the strict verification of session continuity and role allocation effects, but the current control experiments are not sufficient.

  4. The experimental report is incomplete and has multiple inconsistencies, affecting the verifiability of the results. The current PDF does not clearly report the number of tasks, attack budget, rejection determination rules, and repeated runs; at the same time, the Telecom and Workflow tables are completely identical, the main text wrongly attributes a +22.3 percentage point benefit to indirect attacks, and "CRM ticket abuse, 15.0% → 60.0%" cannot be found in the corresponding table, making it difficult to fully verify the stability of the performance improvement and cross-domain conclusions.

Questions for authors
  1. What execution information did the external critic actually receive? Has there been a comparison between the self-assessment after information matching and the external diagnosis? Are the self-generated and externally-generated results paired and compared based on the same trajectory, exactly the same critique, and matching generation settings?

  2. Please explain how the repeated entries and incorrect scene attribution in the above table should be corrected, and whether this affects the main conclusion; also, please indicate the specific locations of the task number, attack budget, repeated runs, and rejection determination rules in the supplementary materials so that the results can be verified.

1 comment

We thank the reviewer for the detailed comments and for recognizing the practical value of role separation and environment-state verification.

Q1. Information available to external criticism and fairness of comparison

External criticism uses a transferred representation of the failed execution, whereas self-criticism continues within the target's original execution context. Thus, the original comparison reflects both information availability and session continuity, rather than target identity alone.

Our reasoning ablation (Sec. 3.3) isolates the value of retained reasoning:

SettingWithout reasoningFull contextGain
Opus-4.7, indirect17%31%+14 pp
Opus-4.7, direct25%42%+17 pp
GPT-5.1 SDK, indirect67%82%+15 pp
GPT-5.1 SDK, direct65%71%+6 pp

These results establish that execution information materially improves criticism.

Q2. Table errors and experimental reproducibility

We acknowledge two reporting errors:

  1. Appendix E, Table 3 incorrectly duplicates the Telecom results instead of reporting Workflow category-level results.
  2. The example 15.0% → 60.0% for ticket abuse belongs to Telecom indirect attacks, not CRM.

The separately reported Workflow results remain:

  • Direct: 44.4% → 66.7% (+22.3 pp).
  • Indirect: +9.6 pp improvement.

The duplicated table cannot verify the Workflow category-level results; these require the original Workflow evaluation records.

Regarding reproducibility, Sec. 3.1 specifies the benchmark, models, frameworks, and threat models; Algorithm 1 defines the attack budget KK; and Sec. 3.5 reports task-level and step-level refusal. We will release the codes for reproducibility

Q3. Evidence for non-refusal sandbagging

We distinguish observable non-refusal degradation from intentional sandbagging.

Self-proposal increases task-level refusal from 13.9% to 47.8%, explaining part of its performance gap.

However, GPT-5.1 with the OpenAI SDK exhibits only 3.6% task-level refusal, while direct ASR drops from 85.5% to 60.6% (−24.9 pp).

Thus, explicit refusal cannot fully explain the degradation. The remaining failures are consistent with weaker or incomplete proposals, although context interference and role conflict are also possible explanations. Our claim concerns the observed proposal behavior, not an unobservable internal motivation.

Q4. Method contribution and session controls

Our contribution is orthogonal to existing attack-search methods: separating failure diagnosis from adversarial proposal generation.

ConfigurationOverall ASR
External Criticism + External Proposal57.3%
Self-Criticism + Self-Proposal46.2%
Self-Criticism + External Proposal63.2%

The same ordering holds in the GPT-5.5 transfer evaluation across two agent backends and both threat models.

The comparison controls underlying model identity, but proposal-session assignment also changes context composition. Thus, the results establish the effectiveness of the role-separated configuration without attributing the entire difference to one isolated contextual mechanism.

Q5. Statistical reliability

The improvement is consistent across all six main framework–threat combinations:

FrameworkDirect gainIndirect gain
Gemini-3.1-Pro, Google ADK+1.0 pp+16.2 pp
GPT-5.1, OpenAI SDK+9.8 pp+10.6 pp
GPT-5.1, OpenClaw+5.7 pp+4.6 pp

The aggregate improvement is +5.9 pp, with a larger gain for indirect attacks (+7.4 pp) than direct attacks (+2.9 pp).

These cross-setting results support the consistency of the effect. Formal confidence intervals or paired significance tests, however, require task-level success records that are not contained in the submitted aggregate tables.

Reviewer_9AuWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper assesses the usefulness of using the victim agent as part of an adversarial loop, evaluating whether it can provide good criticism/feedback for why an attack failed and whether it can provide good proposals for improved attacks. The authors motivate this based on the rich informational access the victim agent has access to in order to identify why an attack failed. The paper evaluates when the victim agent and its context is useful for adversarial purposes on an agentic benchmark where an adversary needs a victim agent to complete an explicitly harmful task. It finds that when proposals for improved attacks are made by an external model, critical feedback about why the previous attack failed is much more effective when it comes from the victim agent itself than when it comes from the external model that lacks the exact context of the attack. Meanwhile, relying on the victim agent to propose an improved attack given it's own criticism leads to significantly worse attack success rates compared to an external proposer, and, this is not fully accounted for by refusal on the side of the victim model

Critical strengths

The paper proposes a creative and interesting perspective of designing adaptive adversaries which use the target (victim) LLM as a direct feedback and information mechanism for the adversary to improve its attacks. I haven't seen this done elsewhere, but it makes complete sense when this mechanism is available.

The benchmark is good, as are the victim model choices (albeit a bit dated).

Critical weaknesses

The main limitation I see with this paper is that the mechanism it relies on may not always be available; a failed attack may result in continued usage of the target LLM within the same context window impossible (or at least provided the context that led to the failure/refusal). Nevertheless, this specific issue would be a lot less likely to apply to locally run models (whose robustness should also be a concern), and, provides good motivation for how attack attempts should be handled in model serving.

The writing is unfortunately not very clear and easy to follow, with the largest saving grace coming from the core ideas themselves being quite straightforward. Nevertheless, many sentences are difficult to parse, potentially unnecessary, and could be organized much better. E.g. I think the experimental setup and the results could be presented far better, with the current presentation relying on familiarity with the evaluation method drawn from a prior work.

One more glaring/core clarity issue affecting the paper is the term "target" which is never explicitly outlined to be the victim agent (except in figure 1).

Questions for authors

Seeing as target agent feedback/criticism isn't always available/accessible in practice, what aspects/parts of it's context, if provided to an external adversary model, would recover most of the performance benefit. In other words, what does an adversary actually need to see in order to provide good criticism, and, does isolating the key component also lead to better proposals? While the external vs target contrast made in this paper is interesting, I think it circles around a more core question of what part of the victim context is needed/important for an adversary to improve it's attacks? Isolating this would make the paper stronger.

2 comments

We thank the reviewer for the positive assessment and constructive suggestions.

Q1. Which parts of the target's execution context are most important? Can an external critic recover the benefit?

Decision-relevant execution information is the main source of the self-criticism advantage. We distinguish three information levels:

  1. Outcome-only: attack, final response, and judge result.
  2. Observable trajectory: outcome plus tool calls, responses, and environment observations.
  3. Full context: trajectory plus applicable instructions and available reasoning.

Our reasoning ablation (Sec. 3.3) demonstrates the importance of retained context:

SettingWithout reasoningFull contextGain
Opus-4.7, indirect17%31%+14 pp
Opus-4.7, direct25%42%+17 pp
GPT-5.1 SDK, indirect67%82%+15 pp
GPT-5.1 SDK, direct65%71%+6 pp

Removing reasoning reduces downstream ASR by 6–17 percentage points, even when the remaining execution history is preserved.

An external critic can potentially recover part of this benefit when provided with the complete serialized trajectory and decision-relevant information. The key distinction is information availability versus critic identity; the current ablation establishes the importance of information but does not isolate an intrinsic advantage of target identity.

Better context also does not necessarily imply better self-proposals: switching from external proposal to self-proposal reduces overall ASR from 63.2% to 46.2%, despite retaining the target-generated criticism.

Q2. What happens when the target session is inaccessible?

Our method does not require model weights or gradients, but it requires the ability to continue the target session after a failed attempt.

  • Controlled/local agents: the original execution session can be retained for criticism.
  • API-based agents with session access: criticism can use accessible conversation and tool context.
  • Strict black-box agents: external criticism based on observable trajectories is a fallback, although some diagnostic information is lost.

The reasoning ablation above quantifies why losing execution information can reduce effectiveness.

Q3. Clarity of terminology and evaluation

Target agent refers to the victim agent being evaluated. We compare three role configurations:

CriticismProposalOverall ASR
ExternalExternal57.3%
SelfSelf46.2%
SelfExternal (ours)63.2%

The underlying model identifier is fixed within each comparison. Our contribution is a role-assignment principle for iterative red teaming, orthogonal to attack-search strategies.

Thank you for your response and provided results, I think they help provide more insight and a framing/conclusion that the target session should be made inaccessible following detection of a jailbreak attempt could help make it more relevant. I will retain my score, since I still think the impact is limited and I have qualms with the clarity of the writing, but, the method is clever, sensible, and empirically supported and may inspire future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.