acceptodds
Under review as a conference paper at ICLR 2027

When Does Negotiation Help? Benchmarking Embodied Multi-Agent Collaboration under Successive Task Revisions

Abstract

Successive task revisions require embodied teams to accommodate new requirements during execution, and performance losses may reflect both final-task expansion and online adaptation. Existing evaluations lack upfront and online conditions with matched final requirements, making these sources difficult to distinguish. We introduce a controlled benchmark of 50 evaluation tasks for LLM-driven heterogeneous teams, synchronizing task and evaluator revisions during continuous physical execution and providing both conditions with matched intended final requirements. Model-equal averages across nine backbones show that 78.8% of the success-rate loss from original to online-revised tasks is already present when final requirements are provided upfront, while online introduction is associated with an additional loss of 5.67%. This online gap occurs on every backbone and is larger in object-state and spatial-adjacency tasks; tasks with temporal constraints exhibit the largest expansion-related loss but no larger online gap. Using explicit allocation dialogue as a coordination probe across four backbones, we find negotiation gains similarly concentrated in the former two groups; broader reallocation does not necessarily improve completion. Two Dynamic configurations that select negotiation modes based on experience achieve higher success rates than full negotiation on all four backbones, each with a model-equal mean gain of 4.38 pp and lower mean action-planning token usage. These findings show that negotiation benefits are context-dependent and cannot be explained by overall task difficulty alone; the compatibility of new requirements with existing allocations and goal dependencies offers a perspective for interpreting these conditional benefits. Code and data are available at https://www.negobench.top/.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.