acceptodds
Under review as a conference paper at ICLR 2027

CoEvoke: Evoking Multi-Agent Collaboration for Trajectory Collection

Abstract

Training multi-agent collaboration requires trajectories of checking, feedback, and repair, yet task-solving rollouts may end in immediate approval or repeat unsuccessful attempts without useful correction. We introduce CoEvoke, a cross-domain framework that evokes these interactions during trajectory collection. Controlled perturbations create recoverable difficulties, while restricted reference-based guidance directs investigation. The resulting dataset contains 4,567 reconstructable trajectories over 4,130 software-engineering and reasoning tasks, linking public interactions to evidence, decisions, and task-native outcomes. Separate experiments over 2,287 same-task pairs show feedback increases of 47.72–100 percentage points across seven perturbation groups; guidance has smaller, model-dependent effects. With identical training tasks, decision-record counts, and updates, perturbation-enriched judge training improves ACPBench-Hard success from 35.38% to 42.12% (+6.73 points). SWE verifier training gains 7.60–9.40 points over interface-matched untrained references. Workflow and post-training evaluations preserve task-solving topology with unperturbed inputs and no private-reference guidance. These results support designing collection conditions to obtain corrective supervision that improves collaborative decisions with frozen solving agents.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, multi-agent collaboration, synthetic training data, collaborative control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rzDwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoEvoke, a framework for collecting multi-agent trajectories with more checking, feedback and repair behaviors. It introduces controlled perturbations, such as corrupting correct candidates or using weaker models for initial generation, together with reference-informed guidance to help agents identify and fix errors. The collected trajectories are converted into supervision for training judges, critics and verifiers, while keeping solving agents frozen. Experiments show much more corrective interactions during collection, and downstream improvements on ACPBench-Hard and SWE-bench Verified.

Critical strengths
  1. The motivation makes sense. When strong agents already solve most tasks correctly, natural rollouts contain very few useful examples about when and how to request repair. Simply collecting more trajectories may not help much. Creating recoverable errors is a reasonable solution.
  2. I also like the matched-data training experiment. Same tasks, same number of decision records and training updates, but quite different downstream results. This provides some real evidence that collection strategy matters. Keeping solvers frozen and removing oracle guidance during evaluation also makes the result more meaningful.
Critical weaknesses

My main concern is what actually contributes to the downstream improvement. Figure 3 shows huge increase in feedback, but this is somewhat expected. The method deliberately changes correct candidates into incorrect ones, and the checker has access to reference answers during collection. More feedback under this setting is not very surprising.

The more interesting evidence is the Natural-SFT vs Perturb-SFT comparison. However, Figure S8 shows that Natural-SFT contains almost only acceptance decisions, while Perturb-SFT has roughly half repair-routing decisions. So the training label distribution is completely different. Maybe the improvement mainly comes from exposing the judge to more negative candidates and teaching it to reject, rather than learning useful multi-agent collaboration from feedback-revision trajectories. A much simpler corruption-based classification or routing dataset could possibly achieve similar gains. Current experiments cannot distinguish these explanations.

Another concern is the contribution of the full collaboration framework. The SWE data uses Guide without Perturb, and verifier training is compared with untrained models. Interestingly, the patch-only verifier performs almost the same as the trajectory-based verifier. This makes me wonder how much additional value comes from the recorded collaboration process itself. Also, Appendix F.5 shows that the single-agent baseline clears previous reasoning context between invocations. This seems a relatively weak way to use the same computation budget. The multi-agent advantages in Figure 4 may partly come from better context retention and structured iteration. Hard to attribute these gains specifically to multi-agent collaboration.

Questions for authors
  1. For the matched ACPBench-Hard experiment, can you compare with a judge trained on simple corrupted candidates and corresponding accept/repair labels, without collecting full feedback-revision trajectories? Ideally matching the label distribution.

  2. How do Natural-SFT and Perturb-SFT differ in actual repair frequency, unnecessary repairs and model calls during evaluation? Does Perturb-SFT really make better decisions, or mainly requests more repair attempts?

  3. For SWE verifier training, what happens if the training records are collected from ordinary unguided trajectories with the same task and supervision budget? This would help clarify the contribution of Guide.

Reviewer_3jQBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoEvoke, a framework for collecting multi-agent collaboration trajectories. The main idea is to introduce controlled errors or difficulties during trajectory collection and provide limited guidance, so that agents are encouraged to check, give feedback, and revise their outputs rather than simply approving them. The authors collect 4,567 trajectories across software engineering and reasoning tasks. Experiments show that these perturbations substantially increase feedback and that training judges and verifiers on the collected trajectories improves downstream task performance while keeping the solving agents frozen.

Critical strengths
  1. The paper addresses an important problem.The paper identifies this issue and proposes a practical way to collect more informative training trajectories.

  2. The framework is simple and easy to understand. The Perturb-and-Guide design is intuitive: introduce a recoverable error to encourage checking, then provide limited guidance to help agents investigate and fix it. The framework is also applied to several different task domains, which shows that it is not limited to one specific setting.

  3. The experiments are fairly comprehensive and show useful improvements. The authors evaluate both the data collection process and downstream training. Perturbations significantly increase the amount of feedback, and training on the collected trajectories improves ACPBench-Hard performance from 35.38% to 42.12%. The fact that the solving agents remain frozen makes the results more convincing, since the improvements come from training the control roles rather than improving the solvers themselves.

Critical weaknesses
  1. The novelty over existing single-agent work is not very clear. The main idea of introducing errors or difficulties to collect correction trajectories is closely related to previous work on agent self-correction and trajectory generation, such as AgentRefine and SCoRe. The paper discusses these works, but the main distinction seems to be that CoEvoke applies similar ideas to multi-agent workflows and trains judges, critics, or verifiers instead of solvers. It is not clear what new challenges arise specifically from the multi-agent setting or why existing single-agent methods cannot be directly adapted.

  2. The choice of perturbations is not well justified. The paper introduces candidate corruption, partial observation, and weak initialization, but it is unclear why these three types of perturbations were chosen. Do they reflect errors that agents commonly make during real execution, or are they simply convenient ways to trigger feedback? For example, changing a correct number to an incorrect one can easily encourage a judge to identify an error, but real agent failures may involve more complicated reasoning, planning, or tool-use mistakes. The paper verifies that the injected errors are incorrect and recoverable, but does not show how representative they are of natural agent failures.

  3. It is unclear how much of the improvement comes from the specific perturbation design. The experiments show that perturbations lead to substantially more feedback, but this is somewhat expected when errors are deliberately introduced. The downstream improvements are encouraging, but it is still unclear whether the proposed perturbation mechanisms are particularly effective or whether simply increasing the proportion of correction examples in the training data would produce similar gains. A comparison with alternative ways of collecting corrective trajectories, or an analysis of how well the learned behavior generalizes to different types of natural errors, would help clarify the contribution.

Questions for authors
  1. What are the challenges specific to the multi-agent setting?

  2. Why were these three perturbation methods chosen? Do candidate corruption, partial observation, and weak initialization reflect common errors that agents make in real tasks? Have the authors analyzed natural agent failures to support these choices?

  3. How well does the learned behavior generalize to natural errors? The experiments show that injected errors lead to more feedback and improve downstream performance. However, could the model simply be learning to detect the specific types of artificial errors used during training? Can the authors evaluate whether training on one type of perturbation improves error detection and recovery on other types of errors?

1 comment

Thanks for the detailed responses and additional experiments. They address my main concerns and strengthen the paper. I appreciate the authors' efforts and update my rating to 3.

Reviewer_eTfxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper discusses creating a framework to generate trajectories that have decisions with recoverable difficulties by perturbing and guiding for investigation. This is because there is a data issue where correct trajectories cannot improve as much because it doesn't have repair or corrections so introducing these errors separately can help address it. These trajectories can help improves success of agents that do reviewing and revisions for downstream decision-making.

Critical strengths
  1. There is a clear issue to motivate the new dataset and usage to help in the process. This motivates the paper well.
  2. Improvements help in downstream task accuracy which shows that the data actually can help with the judge.
Critical weaknesses

I think it would be good to compare with just doing error injection within existing correct trajectories and other more simple baseline comparisons to identify which components are important and the benefit of your approach.

The perturbations used also seem to be tailored to the task structures and type. I'm wondering if that means the CoEvoke would not have good generalizability to natural errors and different types of tasks across domains (cross-domain as mentioned in the paper)

Questions for authors

Is there a need for the full trajectory, or can supervision with some information about repair and investigation be enough? Such as with some short guidance for the judge instead of different trajectories used to judge.

1 comment

Thanks for addressing my concerns, especially with the comparison of rule-static and teacher-static. I will raise my rating to 3 accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.