acceptodds
Under review as a conference paper at ICLR 2027

Concord: Co-Evolving Executors and Guards for Agentic Safety Rapport

Abstract

Large language model agents increasingly rely on external guards to supervise tool use and prevent unsafe actions. Yet guards and executors are often developed independently, creating a coordination gap when deployed together. We call the pair-level ability to enforce safety while preserving authorized task progress and recovering after intervention safety rapport. We identify three recurring failures: under-protection, where harmful behavior is allowed; over-refusal, where legitimate work is blocked; and coordination breakdown, where repeated Revise–retry interactions fail to resolve risk or advance the task. Evaluating existing guards with widely used executor models reveals all three failure patterns across deployed pairings. To specifically stress-test and quantify coordination breakdown, we introduce InterlockBench, which couples benign task requirements with malicious environmental injections and tests whether a pair can resist the injection while still completing the original authorized goal. We further propose Concord, a failure-attributed co-evolution framework that adapts the executor and guard as a deployed pair. Concord attributes unsuccessful interactions to the executor, the guard, or both, and evolves complementary procedural memories while keeping model weights fixed. Across multiple executors and agent-safety benchmarks, Concord improves task completion while maintaining high safety. On InterlockBench, it increases joint success while reducing both average Revise decisions and coordination breakdown. Joint adaptation outperforms one-sided alternatives, and the learned procedures transfer to new safety and general-purpose tasks. Memory analysis further shows that different executors induce distinct procedural guidance, highlighting the importance of adapting the deployed pair together.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Safety, Agentic Safety, Guard Agent, Self Evolving Agent

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6tVxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies three failure modes: under-protection, over-refusal, and coordination breakdown. It introduces InterlockBench, a benchmark constructed within the AgentDyn framework. The authors also propose CONCORD, which jointly evolves complementary procedural memories based on shared failures. CONCORD improves task completion and achieves higher average joint success rates. The authors also report that the learned procedures transfer to new safety and general-purpose tasks.

Critical strengths
  1. The paper identifies the coordination breakdown problem, which is practical and relevant to real-world agentic systems.

  2. The paper highlights the importance of jointly evolving the executor and guard rather than developing them independently.

Critical weaknesses
  1. Lack of repeated experiments. The experiments appear to have been run only once, without repeated trials. Given the relatively small test sets, it is difficult to assess the stability and reliability of the reported results.

  2. Limited evidence of cross-environment generalization. InterlockBench appears to be constructed within the AgentDyn framework, which is also used to evolve CONCORD's procedural memories. Therefore, the strong performance on InterlockBench may partly result from similarities between the training and evaluation environments. This raises concerns about how well the results generalize to substantially different environments.

Questions for authors
  1. Failure attribution. Is failure attribution necessary for CONCORD's performance improvements? Could the authors provide an ablation study without failure attribution to demonstrate its contribution?

  2. Comparison with existing methods. Is the comparison with StepGuard and ToolSafe fair, given that they use their own guard models? Could the authors clarify how differences in the underlying guard models affect the comparison?

  3. Fairness of the ablation study. Could the authors control for the same number of learning and evaluation rounds while varying which role (executor, guard, or both) is updated? The current setups appear to differ in their learning and evaluation procedures, making it difficult to isolate the benefits of joint evolution.

2 comments
AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer 6tVx,

Thank you again for your thoughtful comments and constructive suggestions throughout the discussion. As the author–reviewer discussion period is coming to an end, we would like to kindly ask whether our responses have addressed your main concerns. If there are still aspects that remain unclear or require further clarification, we would be very happy to discuss them further.

We sincerely appreciate your time and valuable feedback.

Dear Reviewer 6tVx,

We have uploaded a revised PDF with changes highlighted in blue; the new Appendix H includes the analyses added in our responses. We would be grateful for any further feedback.

Reviewer_5rqzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies coordination between an agent executor and its guard, termed “safety rapport.” It introduces InterlockBench and proposes CONCORD, which attributes interaction failures to either role and evolves their procedural memories while keeping model weights frozen.

Critical strengths
  1. The paper identifies an important deployment issue: blocking unsafe actions does not ensure that the executor can recover and complete authorized tasks. InterlockBench and the revision/breakdown metrics make this issue measurable.

  2. The method produces substantial reported improvements without updating model weights. On InterlockBench, mean coordination breakdown falls from 31.3% with StepGuard to 8.3% with CONCORD.

Critical weaknesses
  1. The contribution of failure-attributed joint adaptation is not isolated. The one-sided baselines differ from CONCORD in evolution rounds, generation limits, and memory formats (App. C.1; Table 3). Matched-pipeline single-role and no-attribution ablations are missing. Additional adaptation effort could therefore explain part of the improvement.

  2. The safety–utility trade-off remains unclear. On AgentHarm, mean safety rate is 91.09% for CONCORD versus 94.13% for Vanilla (Table 6). Without action-level error analysis or comparisons at matched safety levels, the results do not distinguish improved coordination from more permissive guarding.

Questions for authors
  1. Can you compare joint and single-role adaptation using the same evolution pipeline and budgets, and include a no-attribution ablation?

  2. What are the false-allow and false-REVISE rates? Does CONCORD improve utility when compared with baselines at matched safety levels?

  3. How were evolution and evaluation tasks separated, and what are the confidence intervals for the main results?

3 comments
AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer 5rqz,

Thank you again for your thoughtful comments and constructive suggestions throughout the discussion. As the author–reviewer discussion period is coming to an end, we would like to kindly ask whether our responses have addressed your main concerns. If there are still aspects that remain unclear or require further clarification, we would be very happy to discuss them further.

We sincerely appreciate your time and valuable feedback.

I appreciate the authors’ detailed response, which has addressed most of my concerns. Therefore, I will keep my rating unchanged.

Thank you again for your thoughtful feedback. We’re glad to hear that our responses addressed most of your concerns, and we’ll revise the paper to incorporate the clarifications discussed.

Reviewer_3njSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how an executor and a safety guard can prevent unsafe actions while completing authorized tasks, and introduces InterlockBench to evaluate failures in their coordination. It proposes CONCORD, which diagnoses unsuccessful interactions and updates role specific procedural memories for the executor and guard without changing model weights. Experiments across agent safety benchmarks report improved task completion at high safety levels and fewer revisions, while ablations compare joint and one sided updates.

Critical strengths

S1. The paper clearly identifies a consequential problem: a guard may prevent unsafe actions while still leaving the executor unable to complete an authorized task, making recovery after intervention an important evaluation target.

S2. The paper is well organized, with a consistent connection between its failure taxonomy, benchmark, method, and experiments, which makes its central argument easy to assess.

S3. The paper closes the empirical loop by constructing a benchmark for coordination failures and evaluating whether its proposed method improves both safe task completion and revision efficiency, providing evidence in support of its stated objective.

Critical weaknesses

W1. The paper combines feedback driven revision, adaptive guard guidance, and joint adaptation, all of which have precedents. Its distinctive contribution therefore depends on whether failure attribution provides a principled way to coordinate the two roles, but the proposed procedure largely delegates that decision to an LLM.

W2. CONCORD asks the Memory Evolver to decide whether a failure belongs to the executor, the guard, or both from the same trajectory used to generate memory updates. Although the paper lists examples of role specific failures, it provides no operational criterion for resolving ambiguous cases or checking whether the assigned responsibility reflects each role’s causal contribution.

W3. The recovery loop relies on the guard to produce an actionable repair instruction and on the executor to make a meaningful correction. These behaviors are specified in prompts, but the system has no explicit mechanism to verify that a requested repair is feasible, that the next proposal satisfies it, or that repeated revisions are making progress before the five rejection limit is reached.

W4. When a failure is attributed to both roles, CONCORD updates both memory banks without an explicit joint objective or a rule for resolving interactions between the new procedures. The single repeat execution checks the affected evolution case, but it does not establish that the combined update preserves safety and task performance on other cases.

W5. InterlockBench provides limited support for claims of broad applicability: its 150 inputs are derived from only 30 distinct tasks, with five tasks in each of six categories. Because each task’s paraphrases retain the same environment, injection configuration, and outcome checker, success on this benchmark cannot by itself establish that the largely prompt governed workflow generalizes to substantially different coordination failures.

Questions for authors

Q1. What does failure attributed co evolution add beyond existing guard feedback and adaptive memory methods such as TRIAD and AGrail, under a comparable execution protocol?

Q2. What criteria does the Memory Evolver use to distinguish executor, guard, and shared responsibility in ambiguous cases, and has the resulting attribution been checked independently?

Q3. Does CONCORD verify that a guard’s REVISE instruction is feasible and that the executor’s next proposal satisfies it or makes measurable task progress?

Q4. When both memory banks are updated, what criterion determines whether the combined update is retained after the single repeat execution, and how are conflicting guidance or regressions handled?

Q5. Can the authors report InterlockBench joint success and coordination breakdown by task category and task family, with uncertainty calculated over the 30 distinct tasks rather than treating paraphrases as independent tasks?

3 comments
AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer 3njS,

Thank you again for your thoughtful comments and constructive suggestions throughout the discussion. As the author–reviewer discussion period is coming to an end, we would like to kindly ask whether our responses have addressed your main concerns. If there are still aspects that remain unclear or require further clarification, we would be very happy to discuss them further.

We sincerely appreciate your time and valuable feedback.

Thank you for the substantial additional experiments and clarifications. The discussion has addressed several of my original concerns, and I recognize the completeness of the system and the strengthened empirical evidence for its practical value. My remaining reservation concerns the depth of the methodological contribution relative to prior feedback and memory adaptation approaches, together with the limited scope of the safeguards supporting diagnosis and revision. The attribution audit and replay criterion improve confidence in the evaluated procedure, but do not fully resolve these concerns. On balance, I will retain my current rating, while appreciating the paper’s practical contributions and the authors’ efforts to clarify and strengthen the work.

We thank the reviewer for the follow-up and for recognizing the completeness of the system, its practical value, and that the attribution audit and replay criterion improve confidence in the procedure. Following the reviewer's request, we have uploaded a revised PDF with changes highlighted in blue: Section 3.3 and Appendix E.2 now describe the replay check, the retention criterion, and the handling of regressions, and the new Appendix H reports the attribution audit and the other analyses added during the discussion.

Methodological contribution. Prior feedback and memory adaptation methods, including the ExpeL-style and AGrail-style baselines in Appendix C.1, adapt one side of the interaction. CONCORD makes the deployed executor–guard pair the unit of adaptation and routes shared failure evidence to the responsible role. The matched-pipeline ablations isolate this component under the same Evolver, memory format, and evolution budget: removing attribution (ungated joint) lowers JSR in five of six settings, and reversing it lowers JSR in all six. The gains therefore come from role-aware pair adaptation rather than from memory adaptation alone.

Scope of safeguards. We agree that CONCORD does not verify individual repairs online, and the revised PDF states this as a limitation (Appendix A). We see a step-level repair verifier as a natural complement to pair-level adaptation and leave it to future work.

We appreciate the reviewer's engagement throughout the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.