- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces CMDS (Coordinated Multi-Agent Diffusion Steering), a framework for coordinating frozen pretrained diffusion models through stochastic optimal control. Rather than retraining component generators, CMDS treats them as reusable primitives and learns a control that steers their joint reverse process toward high assembly-level reward while minimizing deviation from pretrained dynamics. The control is trained via adjoint matching, which avoids backpropagation through the solver. Experiments demonstrate target-law recovery in a tractable OU setting, source separation from blurred mixtures, multi-agent navigation in PointMaze (up to 4 agents), dual-arm planning with KUKA robots, and four-person motion coordination with MDM.
Critical strengths
a. Coordinating frozen pretrained diffusion models through learned control, rather than retraining them, is a practical and appealing approach. The stochastic optimal control formulation is elegant and theoretically grounded, providing a clear objective for balancing assembly-level reward against deviation from pretrained dynamics.
b. The paper evaluates on four distinct settings: tractable OU coordination, source separation, multi-agent navigation (PointMaze), articulated robot planning (KUKA), and multi-person motion (MDM). The evaluation is comprehensive, with ablations, transfer experiments, and comparisons against multiple baselines.
c. By avoiding backpropagation through the solver, adjoint matching enables efficient training of the coordination control. The paper demonstrates that this approach works well empirically, achieving high success rates across all tasks.
d. The notation is consistent, the derivations are clear, and the experimental results are presented effectively. Figure 1 provides a clear overview of the method, and the qualitative results (Figures 5, 6, 7) are illustrative.
Critical weaknesses
a. The core technical approach, adjoint matching for multi-agent diffusion steering, is a straightforward extension of Domingo-Enrich et al. (2025). The paper would benefit from a clearer discussion of what is genuinely new (e.g., the multi-agent adjoint matching derivation, the specific control architecture) vs. what is a direct application of prior work.
b. The paper doesn't compare against MultiDiffusion, Diffuser, or other relevant baselines. This makes it unclear whether CMDS is the best approach for multi-agent coordination or just one of many viable options.
c. The "amortised control" framing overstates the generalization. The control is trained per-task and only transfers within the same task (e.g., unseen start-goal pairs). It doesn't transfer across fundamentally different tasks or reward functions. The paper should be clearer about the scope of "amortisation."
d. The paper evaluates up to 4 agents but doesn't analyze how the method scales. For large-scale multi-agent systems, the joint state representation and centralized control could become prohibitive. The paper doesn't discuss this limitation or potential solutions.
e. The paper doesn't provide guidance on how to design rewards for new tasks or analyze sensitivity to reward choice. This limits the practical utility of the method for users who want to apply it to new domains.
f. While the empirical results are strong, the paper doesn't provide theoretical analysis of when adjoint matching might fail or how accurate the approximation is. This is a gap given the theoretical framing of the paper.
Questions for authors
-
Can you provide a comparison with MultiDiffusion (Bar-Tal et al., 2023) and Diffuser (Janner et al., 2022) to establish whether CMDS is the best approach for multi-agent coordination?
-
Can you clarify the scope of "amortised control"? Does the control transfer across fundamentally different tasks or reward functions, or only within the same task (e.g., unseen start-goal pairs)?
-
Can you provide a scalability analysis for larger numbers of agents (e.g., 10+)? How does the method's performance and computational cost scale with agent count?
-
Can you provide guidance on reward design for new tasks? How sensitive is the method to the specific reward formulation and weighting of terms?
-
Can you provide theoretical guarantees or empirical analysis of the adjoint matching approximation quality? When might it fail?
-
Can you discuss the limitations of the centralized control architecture? For large-scale multi-agent systems, would a decentralized or attention-based approach be more appropriate?
7 comments
Part 1B: CMDS’s Contributions and Its Relationship to Adjoint Matching
3. Cross-agent credit assignment. The adjoint gives agent two learning signals (App. C). At the terminal time, measures how its output affects the assembly reward. Along the trajectory, the terms () capture how its state changes the corrections applied to the other agents. With independent reference drifts, AM's lean adjoint drops this second signal (Eq. 90), while the full adjoint, which we use, retains it.
4. Relation to AM's contributions. AM's central contributions are the memoryless schedule and the lean-adjoint matching objective; adjoint methods for SOC predate it. We use the memoryless schedule but not the lean adjoint (App. C.2.1).
5. Controller. CMDS therefore uses an external residual control: it combines axial temporal and cross-agent attention with permutation-equivariant set processing; it is weight-shared across agents and uses no agent-index embeddings, so its parameters do not depend on the number of agents. A controller on the concatenated joint vector, the literal “AM on the joint state”, would fix both N and the agent ordering. While factorised time/agent attention with agent permutation equivariance already appears in multi-agent motion prediction [7], to our knowledge, CMDS is the first to use it as a control network that coordinates frozen pretrained diffusion models. A common architectural design is trained separately for each task setting, with parameters shared across agents and held fixed in the reported zero-shot evaluations. This will be better clarified in the main manuscript, alongside a schematic of the control architecture used for PointMaze, KUKA and MDM [schematic].
We will add a "Relation to Adjoint Matching" paragraph in Sec. 2.1.
Part 2: Baseline Comparisons and the Scope of Amortised Control
We now address the reviewer’s specific questions. For brevity, we address the weaknesses alongside the corresponding questions.
Q1 / W.b: How does CMDS compare with MultiDiffusion, Diffuser and other relevant baselines for multi-agent coordination?
MultiDiffusion is a relevant test-time method already cited in our related work, but its original mechanism does not directly solve our coordination problem. The reason is how it connects the diffusion processes, not simply that it was demonstrated on images. MultiDiffusion makes overlapping predictions agree. For example, two overlapping image crops predict the same pixels. It combines their denoising updates through a least-squares objective to produce one consistent image. Our agents have separate trajectories. Person A’s position and person B’s position are different variables. We need their motions to satisfy a relationship, such as maintaining sufficient separation while each reaches their own goal. A direct per-person adaptation would provide no coordination. If each MultiDiffusion view simply extracts one person’s trajectory, its fusion objective separates into independent per-person updates.
On the other hand, Diffuser requires additional training. We adapt Diffuser’s learned return guidance [5] to the multi-agent setting; the original work does not include multi-agent experiments. Both methods require training, but use reward information differently:
-
The Diffuser baseline adapts Diffuser’s learned-return guidance: a scalar predictor is trained offline to estimate trajectory returns. During sampling, its parameters remain fixed, and its gradient with respect to the trajectory is computed at each step to steer the reverse diffusion process. Thus, it learns a return predictor, whose gradients provide steering directions at test time.
-
CMDS directly learns a joint control offline under the SOC objective, using analytic reward gradients propagated through the adjoint during training. During sampling, its parameters likewise remain fixed, and the controller directly outputs the steering corrections. In CMDS-TRG, reward gradients are additionally supplied to the learned control parameterisation.
The distinction is the information provided by each training sample. Trajectories with similarly poor collision penalties can have different reward gradients, indicating which coordinates to change to reduce collisions. Diffuser must learn these corrective directions indirectly from scalar-return supervision. This can be harder when sampled trajectories receive similar returns, even though their corrective directions differ. Increasing the number of agents may exacerbate this difficulty if generated trajectories have similarly poor returns.
PointMaze - joint success (%):
| Layout | Agents | Diffuser · Pool 1 | Diffuser · Pool 2 | CMDS · Pool 1 | CMDS · Pool 2 | CMDS-TRG · Pool 1 | CMDS-TRG · Pool 2 |
|---|---|---|---|---|---|---|---|
| Medium | 2 | 46.35 ± 4.01 | 44.01 ± 1.04 | 99.0 ± 1.2 | 91.9 ± 1.6 | 99.5 ± 0.5 | 99.2 ± 0.0 |
| Medium | 4 | 6.42 ± 0.40 | 2.34 ± 0.26 | 70.1 ± 0.9 | 66.9 ± 0.5 | 97.4 ± 1.2 | 93.5 ± 0.5 |
| Large | 2 | 66.41 ± 1.97 | 57.90 ± 3.24 | 91.1 ± 1.6 | 83.1 ± 1.6 | 96.6 ± 0.5 | 89.8 ± 1.6 |
| Large | 4 | 3.65 ± 0.26 | 5.21 ± 0.00 | 88.0 ± 1.2 | 75.3 ± 2.7 | 96.6 ± 1.2 | 97.7 ± 0.0 |
KUKA:
| Method | Strict success (%) ↑ | Planning time (s) ↓ |
|---|---|---|
| Diffuser | 19.99 ± 0.73 | 0.278 ± 0.005 |
| CMDS (without TRG) | 60.66 ± 1.19 | 0.348 ± 0.003 |
| CMDS-TRG | 77.3 ± 0.7 | 0.368 ± 0.002 |
Both CMDS variants achieve higher mean success than Diffuser, including without TRG, although Diffuser has a shorter planning time.
We will include this baseline in our evaluation.
Q2 / W.c: Does “amortised control” mean reuse across unseen instances within a task, or transfer across different tasks and reward functions?
Amortised control means training one context-conditioned controller and reusing it across unseen instances within a specified task family. This uses standard conditional learning: the training cost is shared across repeated inference. For example, the same MDM controller handles new scene conditions, with sampling taking 0.54 s for CMDS versus 8.57 s for DPS (Table 4).
Zero-shot transfer is a separate capability. A CMDS-TRG controller trained on Medium PointMaze with two agents achieves 86.20 ± 0.45% / 77.08 ± 4.58% success on Large, without controller retraining (Pools 1/2). We also evaluate transfer across agent counts (Q3) and KUKA reward-clearance parameters (Q4). These results extend the demonstrated scope of reuse, but do not establish transfer across fundamentally different tasks or arbitrary reward functions. We now make this distinction explicit in the manuscript.
Part 1A: CMDS’s Contributions and Its Relationship to Adjoint Matching
We thank the reviewer for (i) recognising the practical value of coordinating frozen pretrained models, (ii) the theoretical grounding of our SOC formulation, and (iii) the efficiency of multi-agent adjoint-matching training. We also appreciate their positive assessment of our comprehensive evaluation, including ablations, transfer experiments, our strong empirical results, and the clarity of our presentation and derivations.
We address the first weakness, clarifying what is new in CMDS and what is inherited from the adjoint-matching framework in [1]. The remaining weaknesses are instead addressed alongside the corresponding questions.
We thank the reviewer and agree that the manuscript should distinguish more clearly what CMDS inherits from adjoint matching (AM) and what is new. Our contribution is a modular coordination framework: CMDS learns a joint control over frozen, independently pretrained diffusion models using an assembly-level reward, without requiring joint training data. Coordination is amortised through a learned controller reused across task instances; we also demonstrate zero-shot transfer to larger agent populations in the evaluated settings. The method combines established ingredients: reward tilting via SOC, AM, a pseudo-Huber loss for numerical stabilisation of AM in high-dimensional settings, and an attention-based controller. We discuss the novelty in detail below:
1. SOC for component assemblies. Extending forward noising to several agents is straightforward, and the reverse drift of the product reference acts agent by agent (App. B.1). The difficulty lies in the object the reward acts on, the assembly Y^=φ(X^). The joint process is Markov, but Y^ need not be when φ discards information. Under suitable assumptions, a Markovian projection can reproduce its one-time marginals, without generally reproducing its full path law [2]. CMDS instead steers the components and handles the assembly by pushforward, including for non-injective φ (Eqs. 67–69, App. B).
2. What we inherit from AM, and when coordination decouples. From AM we inherit the memoryless schedule (η = 1), which yields the tilt of Eq. (61), and the basic regression objective (their Prop. 2), applied to the joint state. What changes is that the reward r∘φ couples agents through Jacobians in the adjoint of Eq. (6), for example through the terminal term −λ(Dxiφ)⊤∇r.
We will add a proposition in Appendix B.4 clarifying when coordination is required. Consider the terminal tilt ρ⋆=Z−1eλRρ, where ρ is a product law, λ>0, R=r∘φ is finite ρ-almost everywhere and 0<Z<∞. Call R separable if R(x)=c+∑iri(xi) ρ-almost everywhere.
Independent per-agent AM suffices when the SOC problem separates. CMDS applies joint-state AM to coordinate frozen component models under assembly-level rewards. In App. D.2, AM applied to a single generator reaches 0–18.9% valid assemblies, against 97.9–99.2% for CMDS with the same solver.