acceptodds
Under review as a conference paper at ICLR 2027

Mean-Field Diffuser: Scaling Offline MARL to Thousands of Agents

Abstract

Diffusion-based planning has achieved strong results in single-agent offline reinforcement learning, yet scaling to many-agent systems remains intractable due to the curse of dimensionality in the joint trajectory space. We introduce MF-Diffuser, a framework that lifts trajectory planning to the Wasserstein space of trajectory distributions, where the propagation of chaos ensures a small representative subset of agents captures the full population dynamics. Our approach features a value-weighted chaotic entropy objective that reconciles generative fidelity with return maximization, and a hierarchical coarse-to-fine strategy that progressively grows the agent population during denoising. We establish end-to-end suboptimality bounds with four interpretable terms, revealing that mean-field approximation error scales as while the offline distribution-shift term is independent of population size , and prove the generated policy is an approximate mean-field Nash equilibrium with explicit convergence guarantees. Experiments on three mean-field RL benchmarks, spanning stage games, sequential dynamics, and adversarial team competition, show MF-Diffuser achieves the best return in the majority of settings, with the largest gains on suboptimal offline data and at extreme scales ().

Primary area:
reinforcement learning
Keywords:
Mean-Field Reinforcement Learning, Diffusion Model, Offline Decision-Making

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_f1K8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MF-Diffuser for diffusion-based trajectory planning in large-scale offline multi-agent reinforcement learning. It treats each agent’s complete trajectory as a particle, models population interactions through mean-field information, and progressively increases the number of trajectories during denoising to reduce generation cost. It introduces a value-weighted training objective and provides theoretical analyses of mean-field approximation, planning error, and equilibrium properties. Experiments are conducted on Ising, Battle, and Gaussian Squeeze, with up to 10,000 agents.

Critical strengths
  1. The paper provides extensive theoretical analysis and proofs, attempting to explain the method from several perspectives, including mean-field approximation, performance error, and game-theoretic equilibria.
Critical weaknesses
  1. Experiments are limited to Ising, Battle, and Gaussian Squeeze. It remains unclear whether the method is effective on other common multi-agent tasks, heterogeneous agents, or scenarios requiring specialized coordination. Additional experiments ( for example: Multi-Agent MuJoCo or SMAC/SMACv2 etc) would help clarify its applicability.
  2. No code is provided.
  3. The overall novelty of the paper is relatively limited.
Questions for authors
  1. In Appendix H.2.3 [2950–2954], the fitted slopes across the three environments are reported to lie in [−0.51, −0.47]. However, Figure 8 on page 56 shows −0.27, −0.21, and −0.25, respectively. Could the authors clarify which values are correct and whether the current results support the stated convergence rate?

  2. Could the authors provide additional experiments on standard tasks such as SMAC, or scenarios with heterogeneous roles? Also, is the comparison with MF-CDMs-RL conducted under the same training budget? A cost comparison would help assess the benefit of the added value-weighted training objective.

  3. Section 3.2 [190–192] states: “In practice we replace ∥⋅∥W\|\cdot\|_W with ∥⋅∥E\|\cdot\|_E during training…”

    The paper claims that this replacement introduces only a constant factor, which can be absorbed into the score error and is independent of the number of agents and the planning horizon. However, I do not understand why a bounded Hessian of the reference log-density is sufficient to control the Sobolev error, which includes derivative terms, using the Euclidean error alone. Could the authors clarify the required assumptions and explain why the theoretical guarantees still apply to the practical training objective?

  4. I am curious about the wording “SCALING OFFLINE MARL TO THOUSANDS OF AGENTS.” This is mainly a question about the title. Existing offline MARL methods, such as MADiff [1], DoF [2], and Oryx [3], have already considered scalability. What specifically does “thousands” emphasize here: the ability to handle this number of agents, the planning quality, or the computational efficiency at this scale?

  5. Appendix G states that the mean-field interaction kernel is implemented using attention. Could the authors clarify its specific architectural differences from standard cross-agent attention or local graph attention?

If the authors address my concerns, I will increase my score.

References [1] MADiff: Offline Multi-agent Learning with Diffusion Models. NeurIPS 2024. [2] DoF: A Diffusion Factorization Framework for Offline Multi-Agent Reinforcement Learning. ICLR 2025. [3] Oryx: a Scalable Sequence Model for Many-Agent Coordination in Offline MARL. NeurIPS 2025.

6 comments
AuthorsOpenReview ↗

Response to Reviewer f1K8 (1/2)

We thank Reviewer f1K8 for the concrete list; we can act on all five points, and one of them exposed an error in the paper, which we report first.

Q1: The slope values in App. H.2.3 and Fig. 8 disagree. The reviewer is right that they cannot both be correct, and the figure is the correct one. The range quoted in H.2.3, [-0.51, -0.47], is the exploitability slope range reported in H.2.2 for a different quantity (normalized exploitability vs. N, per-environment fits -0.49/-0.51/-0.46); it was carried into the H.2.3 text when that paragraph was revised, and the values annotated in the Fig. 8 panels (-0.27, -0.21, -0.25) are what the Wasserstein-to-MFE divergence actually fits. We have corrected H.2.3 to report the figure's own numbers and removed the sentence claiming agreement with -1/2.

What this does and does not affect. The -1/2 rate is a statement about exploitability and about the PoC bound, not about W2W_2 to the MFE flow, and the corrected text now says so explicitly. The exploitability fits (-0.49±0.04, -0.51±0.05, -0.46±0.06 for Ising, GS, Battle; App. H.2.2) are within bootstrap CIs of -0.5, and the PoC tightness measurement is the clean rate check: W22W_2^2 falls from 3.00e-2 at N=102N{=}10^2 to 3.0e-4 at N=104N{=}10^4, a fitted slope of -0.98±0.02, i.e. W2∝N−0.49W_2 \propto N^{-0.49} with pooled R2=0.99R^2{=}0.99 (Table 14). The W2W_2 divergence to the MFE is a different and noisier quantity: it has a larger constant and an NN-independent floor set by score error, so its observed slope is shallower once the floor dominates. Reading it as a rate measurement was the mistake; the divergence is qualitative evidence that the gap shrinks, and the rate evidence is the exploitability exponent and the PoC tightness fit.

Q2: Experiments on standard tasks (SMAC / heterogeneous roles); budget parity with MF-CDMs-RL. We have not run SMAC, MA-MuJoCo, or the Heterogeneous Multi-Agent Challenge within the rebuttal window, and we do not claim to have. The revised Limitations names them as the right testbeds and states why adapting our per-team protocol to their unit-type-conditional observation structure is nontrivial rather than a matter of compute. What we did run is the weakest heterogeneity test that stays inside our data pipeline, a two-type Ising model: two agent classes with coupling strengths λ1\lambda_1, λ2\lambda_2 (λ2/λ1=0.5\lambda_2/\lambda_1 = 0.5), 50/50 mix, everything else as App. F.1, Medium and Mixed data, N=1000N{=}1000, 4 seeds. The class label is supplied as an input feature to every method, so the comparison is not a handicap. See C1 in the common response for the full table; in brief, MF-Diffuser reaches 84.2 ±1.6 (Medium) and 75.4 ±2.3 (Mixed) against 80.7 ±2.1 and 71.2 ±2.7 for MF-CDMs-RL. The heterogeneity costs us 3.7 points but costs the strongest baseline 3.9, so the ranking and the margin are preserved; removing the class label instead of supplying it costs a further 6.3 points, which locates the cost in the heterogeneous projection rather than in the generator.

On budget parity with MF-CDMs-RL: the two share the backbone, optimizer, batch size, 200 diffusion steps, the level schedule, and the same 4xA100 allocation; measured cost is 12.5 GPU-h (ours) against 14.2 (MF-CDMs-RL) at N=1000N{=}1000. We did not fix the gradient-step count in that run and we now state the budget as measured GPU-hours rather than imply step parity. We have since run the step-matched comparison: 200K gradient steps, Battle, Medium, N=1000N{=}1000, 3 seeds, giving 75.2 ±2.7 for MF-Diffuser against 68.1 ±2.9 for MF-CDMs-RL, a gap of 7.1 points against a converged gap of 7.4 (MF-CDMs-RL reaches 68.4 ±2.9 with 1.7x more steps). The gap therefore does not close with training. The cleaner control is still the within-architecture ablation: same backbone and budget, removing value weighting (λ=0\lambda{=}0) gives 66.5 on Battle against 75.8 with it (Table 2), which isolates the objective without any implementation confound.

AuthorsOpenReview ↗

Response to Reviewer f1K8 (2/2)

Q3: Why a bounded Hessian of the reference log-density controls the Sobolev error. The reviewer is correct that it does not. The W1,2W^{1,2} error is ∥h∥E2+∥∇h∥F2\|h\|_E^2 + \|\nabla h\|_F^2 for h=sθ−∇log⁡ζth = s_\theta - \nabla\log\zeta_t, and a bound on the Hessian of log⁡ζ0\log\zeta_0 controls neither term of this error. The constant in Remark B.3 (σtmin⁡−2\sigma_{t_{\min}}^{-2}) was derived under an implicit smoothness assumption on hh that we did not state, and stating it is now part of the remark.

The corrected statement adds it explicitly. If the score network is Lipschitz in its trajectory argument with constant ℓ\ell — which is what spectral normalization, enabled by default in our implementation, enforces — then for the Gaussian reference measure at noise level tt, Young's inequality gives ∥h∥W2≤(2+ℓ2+σt−4)∥h∥E2\|h\|_W^2 \le (2+\ell^2+\sigma_t^{-4})\|h\|_E^2, so the Sobolev-to-Euclidean conversion costs a factor that is independent of NN and HH. That factor is absorbed into ϵscore\epsilon^{score} and changes no rate. The measured ϵscore\epsilon^{score} values across the λ\lambda sweep (0.18-0.42, App. H.3.1) already include it. We have restated Remark B.3 with this assumption and dropped the claim that a bounded reference Hessian suffices on its own.

Q4: What "thousands" emphasizes in the title. Scale of the setting, not a claim of efficiency. Two specific claims: (i) at N=104N{=}10^4 the method still produces non-degenerate plans where joint-space models collapse (normalized return 4.2-12.3 for Joint Diffuser and 6.8-18.4 for MADiff on Ising, against 85.2-95.3 for ours; Table 8), and (ii) quality improves with NN rather than degrading (Cor. 4.1; PoC KL 1/N fit, R2=0.99R^2{=}0.99). Inference remains O(N)O(N) per denoising step and we say so in Sec. 3.4 and in the Limitations; the efficiency claim is limited to the 2.58x constant factor of the hierarchical schedule at fixed (b,K)(b,K) (App. H.4.6). If the title reads as an efficiency claim, the abstract's first sentence now makes the distinction explicit.

Q5: Architectural difference from standard cross-agent attention. The kernel is conditioned on the empirical mean field: the interaction term is $B_\theta

ν\nu

(\tau^i) = \frac{1}{N-1}\sum_{j\neq i} K_\theta(\tau^i,\tau^j) B_\theta(\tau^i,\tau^j)$, where BθB_\theta is a dynamic k-nearest-neighbour graph convolution over the current noised states and KθK_\theta is an attention kernel whose keys carry μˉhN\bar\mu_h^N as additional context (App. G). Standard cross-agent attention keys contain neighbour features only; MADiff's inter-agent attention is of that form. The practical consequence is that the interaction term changes when the population distribution changes even if the local neighbourhood does not, which is what the Gaussian Squeeze visualisation shows (Fig. 5): the population-level mode structure is tracked, whereas factorized models lose a mode. The revised Sec. 3.1 states this contrast next to the architecture description.

W1: Benchmarks limited to three mean-field environments. Addressed by the two-type Ising experiment above, and by the scope statement now in Sec. 1. We agree that the current set does not establish generality; App. I says so and names the testbeds we have not run.

W2: No code. We will release the full training and inference code, the dataset-construction scripts, and the per-seed raw CSVs with the camera-ready.

W3: Limited novelty. We would locate the contribution in the guarantee rather than the architecture. To our knowledge this is the first trajectory-level planner with (i) a propagation-of-chaos bound that is dimension-free in DτD_\tau, (ii) an end-to-end suboptimality decomposition in which the mean-field term decreases with NN while the offline-shift term does not, and (iii) an exploitability bound applying to team-competitive settings without monotonicity. The individual ingredients — mean-field diffusion, value tilting, coarse-to-fine sampling — exist separately; the composition and the guarantees are what we claim. The revised introduction states this in one paragraph and cites MADiff, DoF, and Oryx explicitly as the scalability precedents the reviewer names.

Thanks for the response and the additional experiments. I have two follow-up questions.

  1. How does a uniform Lipschitz bound on the score network allow the Euclidean score error to control the derivative error?

  2. The two-type Ising experiment varies coupling strength. Does the mean-field representation preserve the information needed for coordination when agents have complementary roles? Can you test this on a task that requires such coordination?

Reviewer_tbUPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes the interaction of offline homogeneous agents that interact through and empirical population distribution by treating each complete state-action trajectory as a particle and model a distribution over trajectories. Essentially, applying the principles of mean-field reinforcement learning to diffusion models for multi agent systems is the main contribution of the paper, along with thorough results and theoretical guarantees.

Note. I'm not extensively familiar with the field of multi-agent RL settings myself, and will view the work mainly from an RL perspective.

Critical strengths
  1. Benchmark gains are notable, and the results repeat across different regimes.

  2. The appendix is thorough and transparent and probes various design choices.

  3. Mean-field structure is a reasonable way to reduce the burden of numerous agents that differ in population statistics.

  4. Theoretical analysis is presented to accompany the methodology and appears sound in the first glance, although I did not verify the appendix line by line.

Critical weaknesses
  1. Under the assumptions of the paper, the approach can make larger populations easier to summarize; for instance, with many similar agents, individual fluctuations average out and the population becomes more predictable. But there is no guarantee that this is beneficial in generic heterogeneous agent scenarios rather than only the benchmarks used here.

  2. Also to what extent can the underperformance of Joint Diffuser and MADiff be related to restrictions of the attention mechanism and memory usage?

  3. It seems like the method still generates N trajectories? And the advantage comes from restricting the class of interactions rather than reducing the complexity of the dimension relevant to N? It is not clear why some decision-relevant information should not be lost through these restrictions.

  4. The results support mean field structure as a useful inductive bias on the selected benchmarks, but authors do not sufficiently establish when this approximation remains adequate? Or isolate its contribution from computational and architectural differences in the baselines (such as mentioned in 2) to show the exact advantage?

  5. I wonder if kernel-based trajectory interaction warrants discussion in relation to mean-shift and drifting generative methods. Clarifying the differences in interaction fields, training objectives, and sampling procedures would better establish what is distinctive about the proposed formulation than the current state. Especially since faster inference is part of the main claims, and drifting methods eliminate that by avoiding the denoising chain. The paper mentions closely related FQL and single-step variants but not similar mean-shift methods such as Drift Q-Learning.

Questions for authors

See and rectify weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer tbUP (1/2)

We thank Reviewer tbUP. The central point (W3) is correct as stated, and we have narrowed our claims accordingly; W2 and W4 are answerable from existing numbers plus one new control; W1 and W5 lead to concrete additions.

W3: "Still generates N trajectories; the gain is a restricted interaction class, not a reduction in N-dependence." We agree, and the revised text says so. The accounting is exact and is already in App. H.4.6: inference work is W(N)=∣K∣NW(N) = |K|N for the full method and for the two ablations that remove the cheap branching map (Prop. H.4.1: WnoBr=WnoSub=∣K∣NW_{\mathrm{noBr}} = W_{\mathrm{noSub}} = |K|N exactly). Inference is O(N)O(N) per planning step; at N=104N{=}10^4 this is about 12.5 s/step. What the method buys is three things, and we list them instead of implying a complexity reduction:

(i) training is independent of NN (the score network is fit on M≤100M \le 100 agents, M=O~(N)M = \tilde{O}(\sqrt N) in the asymptotic sufficient schedule); (ii) the constant factor on inference is 2.58x at (b=2,K=4)(b{=}2, K{=}4) versus full-population denoising at every step (App. H.4.6), or 0.80 vs 2.06 s/step on Battle (Table 2); (iii) quality improves with NN (Cor. 4.1).

On whether decision-relevant information is lost: the restriction is not to identity-blind features. Each agent keeps its own trajectory τi\tau^i; what is dropped is the dependence on the identities of the other N−1N-1 agents beyond their empirical distribution. The quantity this destroys is exactly the PoC term: expected KL between the MM-marginal and the mean-field product is at most ec0(C1M/N+C2MHLeff2/N)e^{c_0}(C_1M/N + C_2MHL_{\mathrm{eff}}^2/N), i.e. O(M/N)O(M/N) with a horizon-dependent constant. At N=104N{=}10^4, M=16M{=}16 this is measured at 1.6e-3 nats (App. H.5.1, R2=0.99R^2{=}0.99 against a 1/N1/N fit), so the loss is small in the regime we operate in, though not zero and not small for small NN or for populations with a few high-influence agents (Limitations (vi)).

W2: Are Joint Diffuser and MADiff handicapped by attention and memory? Partly, but only at the extreme scale. Training cost at N=1000N{=}1000 (App. H.1.3): Joint Diffuser 18.5 GPU-h, MADiff 16.4, ours 12.5. At that size both baselines fit in 80 GB with their native attention and are still 20+ points behind on Battle, so the N=1000N{=}1000 comparison is not a memory artifact. At N=104N{=}10^4 the joint-space models become infeasible at full width and we follow each paper's own scale-out heuristic (chunked attention with 32-agent blocks for Joint Diffuser; 12-agent windows with averaging for MADiff, the SMAC-scale setting of the original paper). That is a genuine confound and we say so; it is also why the scalability claim is about the existence of a viable path at 10410^4 agents, not about the baselines' asymptotic merits.

W4: When is the mean-field approximation adequate, and is its contribution isolated from compute and architecture? Three forms of evidence, two existing and one new. Existing, from the ablation: removing only the interaction term (Bθ=0B_\theta{=}0) holds the architecture, data, and budget fixed; returns fall from 75.8 to 53.1 on Battle and from 83.5 to 42.7 on Gaussian Squeeze (Table 2), while training cost falls only from 12.5 to 8.6 GPU-h, so the drop is not a compute effect. Existing, from the measurement of the adequacy condition itself: we track LeffHL_{\mathrm{eff}}H along generated trajectories (1.9/2.8/2.3 with the tilt, App. H.8.2), and a deliberately high-gain coupling at LeffH=8.7L_{\mathrm{eff}}H = 8.7 pushes the fitted horizon exponent to 3.42 ± 0.21, outside the quadratic band — the predicted failure mode, observed.

New, the converse control you ask for: every baseline is given 1.5x its own convergence budget rather than its default setting (Battle, Medium, N=1000N{=}1000, 3 seeds). Joint Diffuser 38.2 ± 5.5 against 37.6 at default; MADiff 43.0 ± 4.7 against 42.4; MF-CDMs-RL 68.9 ± 2.8 against 68.4; MF-Diffuser 76.0 ± 2.5 against 75.8. Every method moves by less than its seed noise, so the gap is not a budget artifact. Combined with the cost table, where ours is the cheapest of the top four, this separates the mean-field contribution from compute. The architecture comparison is handled by Oryx, the strongest non-mean-field model: it uses a retention sequence rather than attention over the joint state, yet its horizon exponent (1.83 on Battle) is bought with a $\sim$43% larger constant and an 8-16 point deficit at N=104N{=}10^4 (Table 8). So neither compute nor the attention-vs-other-architecture axis accounts for the gap.

AuthorsOpenReview ↗

Response to Reviewer tbUP (2/2)

W1: Heterogeneous agents. The approximation relies on exchangeability and degrades when agents differ; we agree, and Sec. 1 now states this as scope rather than leaving it in the Limitations. Two-population structure is already exercised by Battle's per-team planning. Two-type Ising results (two classes, coupling strengths differing by 2x, 50/50 mix, class label given to every method, N=1000N{=}1000, 4 seeds): 84.2 ± 1.6 (Medium) and 75.4 ± 2.3 (Mixed) for MF-Diffuser, against 80.7 ± 2.1 and 71.2 ± 2.7 for MF-CDMs-RL and 78.5 ± 2.4 and 70.1 ± 2.9 for MFQ-Offline. Heterogeneity costs us 3.7 points and the strongest baseline 3.9, so the ordering and the margin survive. We report the amplitude of the effect rather than claiming it is benign: the sweep in C1 of the common response shows monotone degradation with both the minority fraction and the coupling ratio, and the approximation fails when the two couplings differ by more than roughly a factor of four (λ2/λ1=0.1\lambda_2/\lambda_1 = 0.1 gives 72.8). Those regimes are outside the method's scope, and the Limitations say so.

W5: Relation to drifting / mean-shift generative methods. Missing and now added (App. A and Sec. 2). The distinction is in three places. (i) Interaction field: drifting methods such as Drift Q-learning use a kernel attraction toward the data plus a repulsion term, both evaluated against the current particle population at a single generation step; our field is a denoising-time-indexed mean-field interaction Bθ[νtN]B_\theta[\nu_t^N] with a temporally structured kernel Kθ=1H∑hkθ(shi,shj,μˉhN)K_\theta = \frac{1}{H}\sum_h k_\theta(s_h^i, s_h^j, \bar\mu_h^N), so interaction is re-evaluated as the plan evolves. (ii) Objective: drifting uses a drift-matching regression; ours is a value-weighted score-matching objective with an explicit tilted-target minimiser (Eq. 7). (iii) Guarantees: drifting carries no propagation-of-chaos or equilibrium statement, and Thms. 4.1-4.2 and 4.4 are stated for the multi-step reverse process, which a one-step generator would not inherit without a new argument. We also cite the one-step flow-Q-learning line the reviewer mentions as the natural efficiency comparison, and we are explicit that the efficiency case for drifting is real: our cost is one score-network pass over NN agents per denoising step, so a single-step generator would remove the ∣K∣|K| factor we do not remove.

I thank the authors for detailed responses, and additional experiments and corrections. I see that the role-aware ablations meaningfully strengthen the evidence for complementary role coordination. The matched-step and extended-budget comparisons also reduce my concern that the performance gains primarily reflect undertrained baselines.

However, I am maintaining my score because while the empirical case has improved, but these remaining limitations leave me insufficiently convinced of the overall contribution. The method still generates NN trajectories with linear inference cost and while the clarified constant-factor results are useful, the benefits remain tied to restricted interaction structures. I appreciate the explanation that the main bounds rely on L2L^2 error, but remain concerned about the applicability of the complete set of guarantees to the practical training and branching procedures.

I will keep watching the rebuttal process and other responses, but for now my score is maintained.

Reviewer_q4WWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes MF-Diffuser, a scalable diffusion-based planning framework for offline multi-agent reinforcement learning. It combines mean-field trajectory modeling, value-weighted score matching, and hierarchical coarse-to-fine generation to handle thousands of agents. The authors provide theoretical guarantees on planning suboptimality and approximate Nash equilibrium. Experiments on Ising, Battle, and Gaussian Squeeze demonstrate strong performance against baselines, particularly with large agent populations and suboptimal offline data.

Note: I didn't fully checked the proofs.

Critical strengths
  1. The mean-field formulation and hierarchical coarse-to-fine planning enable diffusion-based offline MARL to scale to thousands of agents.

  2. Extensive experiments across three benchmarks and different offline data qualities demonstrate consistent improvements over competitive baselines.

  3. The paper provides suboptimality and approximate Nash equilibrium bounds, offering theoretical support for the proposed framework.

Critical weaknesses
  1. I find the connection between the Wasserstein-space formulation and the actual algorithm a bit unclear. The implementation seems to mainly rely on score matching and graph-based attention between agents. It would be helpful to explain what the Wasserstein formulation brings beyond a standard diffusion model with inter-agent attention.

  2. I am not fully convinced by the connection between the propagation-of-chaos motivation and the practical hierarchical generation. In particular, it is unclear how approximating the population distribution with a small subset of agents guarantees the quality of the full joint trajectories generated through heuristic branching.

  3. The paper would benefit from a discussion of related work on compositional diffusion generation in the main text, such as GSC [1], GFC [2], CompDiffuser [3], and CVP [4]. These works also explore composing generative models while maintaining consistency across components. I would like to see a clearer discussion of how MF-Diffuser differs from these approaches in related work or introduction.

  4. I think the empirical evaluation is somewhat limited. All three benchmarks are designed around mean-field interactions, so it is unclear how well the method would generalize to more heterogeneous agents or environments with more complex interactions.

  5. The presentation could be improved. In particular, I would suggest including related work, the training and inference pseudocode in the main paper, as it is currently difficult to follow how agent subsampling, score matching, and hierarchical branching work together.

[1] Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models. arXiv:2401.03360

[2] Generative Factor Chaining: Coordinated Manipulation with Diffusion-based Factor Graph. arXiv:2409.16275

[3] Generative Trajectory Stitching through Diffusion Composition. arXiv:2503.05153

[4] Compositional Visual Planning via Inference-Time Diffusion Scaling. arXiv:2603.02646

Questions for authors
  1. Could the authors clarify what the Wasserstein-space formulation actually brings to the algorithm? The implementation seems to mainly rely on score matching and graph-based attention.

  2. How does the propagation-of-chaos result justify generating the full population through heuristic agent branching? In particular, how does approximating the population with a small subset of agents guarantee the quality of the generated joint trajectories?

  3. How well would the method generalize beyond the three mean-field benchmarks, especially to heterogeneous agents?

4 comments
AuthorsOpenReview ↗

Response to Reviewer q4WW (1/2)

We thank Reviewer q4WW. W1 and W2 are points our presentation made harder than necessary; W3 and W5 are additions we should have made; W4 we accept as a scope limit and have begun addressing directly.

W1: What does the Wasserstein formulation add beyond score matching plus inter-agent attention? The implementation is indeed score matching plus an interaction kernel. The Wasserstein formulation is not an architectural choice; it is the object of the guarantees. Three statements in the paper depend on it and could not be made for a plain attention model. (i) Prop. 3.1 bounds the value-weighted chaotic entropy by the MF-VSM objective in the W1,2(XN,νtN)W^{1,2}(X^N,\nu_t^N) norm; without the measure-theoretic objective there is no quantity to bound, only a loss value. (ii) Thm. 4.1 gives the expected KL between the MM-agent marginal and the mean-field product at O(M/N+MHLeff2/N)O(M/N + MHL_{\mathrm{eff}}^2/N), which is what licenses training on M≤100M \le 100 agents and planning at N=104N = 10^4; an attention model has no such statement, and this is exactly what Oryx lacks, which is why its horizon exponent is 1.83 rather than 2 while paying a $\sim$43% larger constant. (iii) Terms (c) of Thm. 4.2 and (ii) of Thm. 4.4 are O(H2/N)O(H^2/\sqrt N), which is a statement about the population law, not about the network. Concretely: the Wasserstein space is where the theorem lives, not where the tensor lives. Sec. 3.1 now carries this paragraph.

W2: How does propagation of chaos justify heuristic branching? It justifies the population size, not the branching map itself, and the paper conflated the two. PoC says an MM-agent ensemble is within O(M/N)O(M/N) in KL of the mean-field product, so it is legitimate to represent the population with MM agents at high noise levels. It says nothing about how to go from MM to bMbM agents. That step needs its own condition, and after this review we state it explicitly (new Assumption B, App. E.4) as a contraction condition on the branching map, listed as a condition on the algorithm rather than derived from exchangeability.

We then measured the quantity Assumption B bounds rather than only assuming it. On Battle at N=1000N=1000, the W2\mathcal{W}_2 gap to the OT-optimal branch is 0.068 for our noise+MF correction, 0.142 for noise only, and 0.031 for Sinkhorn with 10 iterations; Sinkhorn is better by 0.037 in W2\mathcal{W}_2 and +0.5 points in return at 6.5x the per-step cost. Sweeping N∈{100,103,104}N \in \{100, 10^3, 10^4\} at fixed M=16M{=}16 gives 0.074, 0.068, 0.066, i.e. the residual is flat in NN — it is the approximation error of the deployed operator, not a cardinality effect. This is why we state Assumption B in the constant-contraction form with cΨ≈0.07c_\Psi \approx 0.07 and report the resulting term as a small NN-independent additive cost rather than claiming the branching is near-optimal. In the reported decomposition that term contributes O(10−3)O(10^{-3}) nats against O(10−1)O(10^{-1}) for the score term, below 2% of the gap at every measured NN.

AuthorsOpenReview ↗

Response to Reviewer q4WW (2/2)

W3: Compositional diffusion (GSC, GFC, CompDiffuser, CVP). These were missing and are now discussed in related work (App. A, and in compressed form in Sec. 2). The distinction is the axis of composition. Those methods compose along the task or skill axis: a plan is a sequence or factor graph of sub-plans, and consistency means adjacent sub-plans agree at their interface. We compose along the population axis: the components are agents, and consistency means every agent's trajectory is consistent with a common empirical distribution. The former is a product of conditional models; the latter is a fixed-point condition between a particle and the measure it induces, which is why it admits a propagation-of-chaos bound and the compositional methods do not. A second difference is that composition in our setting is not optional — we must produce all NN trajectories jointly — and the cost of doing so is the object we optimize, the coarse-to-fine schedule.

W4: All three benchmarks are mean-field by design. This is the strongest of the weaknesses and we accept it. Sec. 1 now states the scope: the method targets large homogeneous populations with mean-field coupling, and the experiments test that regime rather than the general MARL setting. We have added the weakest heterogeneity test we could run inside our data pipeline: two-type Ising, two agent classes whose coupling strengths differ by a factor of 2, mixed 50/50, everything else as App. F.1, with the class label supplied to every method. Results (Medium/Mixed, N=1000N{=}1000, 4 seeds): 84.2 ± 1.6 / 75.4 ± 2.3 for MF-Diffuser, 80.7 ± 2.1 / 71.2 ± 2.7 for MF-CDMs-RL, 78.5 ± 2.4 / 70.1 ± 2.9 for MFQ-Offline. The heterogeneity costs us 3.7 points and costs the strongest baseline 3.9, so the ranking survives with the margin intact, and removing the class label costs a further 6.3 points, which locates the cost in the heterogeneous projection. We do not claim this substitutes for SMAC or the Heterogeneous Multi-Agent Challenge; App. I names those as the right testbeds and states that we have not run them.

W5: Presentation; pseudocode in the main paper. Agreed. Training (Alg. 1) and inference (Alg. 2) now appear in Sec. 3.4, with the agent-subsampling loop, the KK-level loss aggregation, and the branching point marked, so the interaction between subsampling, score matching, and branching is visible in one place. We have also added a one-paragraph data-flow description in Sec. 3.4 that traces a single episode from the dataset through the MM-agent subsample, the noised trajectory, the loss, and the inference-time population growth. Related work moves from App. A into Sec. 2 in compressed form.

Q3: Generalization to heterogeneous agents. See W4. The two-type Ising experiment is the beginning of an answer; the full heterogeneous case is future work and the Limitations say so.

Thank you very much for your detailed experiments and analysis. My concerns have been greatly resolved, and I will raise my score accordingly.

Thank you very much for your careful review and constructive comments. We sincerely appreciate your positive feedback and are glad that our revisions have addressed your concerns.

Reviewer_JGvjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates offline trajectory generation and cooperative planning in large-scale multi-agent systems and proposes MF-Diffuser. The method treats the complete trajectory of each agent as a particle and generates population-level trajectories through a score network that incorporates individual temporal modeling and mean-field interactions. It further introduces a value-weighted training objective and a progressive sampling strategy that gradually increases the number of trajectories during the denoising process. The authors also provide theoretical analyses of suboptimality and approximate equilibrium based on propagation of chaos, and conduct experiments across multiple environments involving up to 10,000 agents. The core contribution of this work lies in integrating mean-field modeling, return-guided trajectory generation, and a hierarchical sampling structure into a unified framework, effectively improving planning performance and computational efficiency in large-scale offline multi-agent systems.

Critical strengths

S1. The paper addresses a well-defined and valuable problem, targeting the fundamental challenges of large-scale multi-agent trajectory coordination: how to effectively capture individual temporal structures while modeling inter-agent interactions, and how to overcome scalability and computational efficiency bottlenecks. The proposed method separates individual-level and population-level modeling, using a smaller set of representative trajectories during the early, high-noise denoising stages and progressively expanding to the full agent population in later stages. This design is both novel and effective, offering a valuable approach to reducing the computational overhead of large-scale joint planning, along with a corresponding theoretical analysis.

S2. The experiments cover multiple scenarios and effectively demonstrate the proposed method's advantages in computational efficiency and scalability to large agent populations. In particular, the method achieves substantial improvements when the number of agents exceeds 1,000. The experimental results also clearly illustrate the effects of population-level interactions and hierarchical trajectory generation on planning returns and inference costs, providing convincing empirical support for the proposed design.

Critical weaknesses

W1. Gaps in Theoretical Derivations and Incomplete Justification of the Practical Algorithm.

Several theoretical derivations in the paper contain gaps that undermine the validity of the claimed guarantees and their applicability to the actual algorithm. In particular, the paper presents the O(H2/N)O(H^2/\sqrt{N}) suboptimality bound and approximate Nash equilibrium guarantee as major contributions, but the current proofs contain several issues that directly affect these conclusions.

First, in Appendix E.5 (page 35, lines 1877–1883, after Eq. (59)), the authors absorb an O(N0−1/4)O(N_0^{-1/4}) hierarchical residual into the O(H2/N)O(H^2/\sqrt{N}) term. However, given the scaling N0≍NN_0 \asymp \sqrt{N} adopted in page 31, lines 1633–1641, this residual is bounded by O(N−1/8)O(N^{-1/8}). For a fixed HH, such a term cannot be absorbed into O(H2/N)O(H^2/\sqrt{N}) using a constant independent of NN.

Furthermore, Appendix E.4 (page 34, lines 1807–1816, Eq. (56)) directly assigns an O(Nk+1−1/2)O(N_{k+1}^{-1/2}) error bound to the branching step by invoking the data processing inequality and Assumption D.5(b), which only constrains the initialization. However, the analysis does not adequately account for the new dependencies introduced by branching, nor does it establish a rigorous connection between this bound and the actual "noise plus mean-field correction" operator used in page 50, lines 2692–2699, Eq. (123).

There are additional gaps in the theoretical reasoning. The authors are encouraged to carefully revisit these derivations, address the missing assumptions and proof steps, and strengthen the mathematical rigor of the theoretical guarantees.

W2. Insufficient Analysis of the Sources of Performance Improvements.

Although the paper provides extensive comparisons with related methods, two important aspects remain insufficiently investigated.

  1. Comparisons with alternative multi-agent representations beyond high-dimensional concatenation. The motivation of the paper primarily stems from the limitations of Diffuser and concatenation-based representations, particularly the high-dimensional collapse problem when scaling to very large agent populations. However, multi-agent coordination does not necessarily require high-dimensional concatenated representations. Are there alternative approaches that model multi-agent interactions without relying on such representations? If so, does the proposed method still demonstrate advantages over these approaches? Comparisons with representative methods using alternative population representations would help establish the broader applicability and advantages of MF-Diffuser.

  2. Disentangling the benefits of mean-field interactions from those of diffusion-based planning. The ablation study in Table 2 removes individual components within the diffusion framework, demonstrating the effectiveness of mean-field interactions for the proposed model and evaluated scenarios. However, it does not sufficiently establish whether diffusion-based trajectory planning provides additional benefits over direct policy learning under the same population representation. A comparison with policy-learning approaches equipped with comparable mean-field interaction mechanisms would help clarify whether the observed performance improvements primarily arise from more effective population-level interaction modeling or from the diffusion-based trajectory planning paradigm itself.

Questions for authors

Q1. Clarification of Theoretical Gaps and Approximation Conditions. Could the authors provide further explanations or rigorous justifications for the theoretical issues raised in Weakness 1? If certain derivations rely on approximations or asymptotic equivalences rather than exact results, could the authors explicitly state the assumptions and conditions under which these approximations are valid, and clarify how they affect the claimed theoretical guarantees?

Q2. Generalizability Beyond Diffuser-Based and Homogeneous Multi-Agent Systems. Could the authors further discuss how the proposed framework compares with multi-agent approaches that do not rely on Diffuser-based trajectory planning? Moreover, the current framework appears to assume a homogeneous agent population governed by a shared policy or model. Can the proposed approach be effectively extended to agent populations driven by different policies or models, or even to heterogeneous multi-agent systems in which agents with different architectures or capabilities must coordinate? What challenges or modifications would such extensions require?

4 comments
AuthorsOpenReview ↗

Response to Reviewer JGvj (1/2)

We thank Reviewer JGvj for a careful reading. We accept both technical objections in W1: they are real errors in the appendix as written, and correcting them changes the hypothesis of Theorem 4.2. W2.2 is answerable from existing results, and we add the decomposition that makes the answer visible.

W1a: The O(N_0^{-1/4}) residual cannot be absorbed into Term (c). The reviewer's arithmetic is correct. App. E.5 (after Eq. 59) writes ∑kE(Nk+1)=O(N0−1/4)\sum_k\sqrt{E(N_{k+1})} = O(N_0^{-1/4}) and folds it into Term (c), which is O(H2/N)O(H^2/\sqrt N). With the asymptotic schedule N0=M=O~(N)N_0 = M = \tilde{O}(\sqrt N) this is N−1/8N^{-1/8} against N−1/2N^{-1/2}, and for fixed HH no constant independent of NN makes the comparison hold. We have restated Theorem 4.2 with this residual as an explicit fifth term, CbrRbrC_{br}R_{br}, and removed the absorption.

Two consequences, both now stated in the paper. (i) Under the asymptotic schedule, Rbr=N0−1/4=N−1/8R_{br} = N_0^{-1/4} = N^{-1/8}, which decays more slowly than Term (c); the headline rate for the hierarchical variant is therefore min⁡{N−1/8,H2N−1/2}\min\{N^{-1/8}, H^2N^{-1/2}\}, and the H2/NH^2/\sqrt N statement is a property of mean-field planning at fixed N0N_0, not of the full pipeline. (ii) We also found an error of our own while checking this. The revised text claimed that deployment holds N0=M≤100N_0 = M \le 100 fixed in NN, but the inference schedule of Sec. 3.3 is Nk=Nbk−KN_k = N\mathfrak{b}^{k-K}, so at (b,K)=(2,4)(b,K) = (2,4) the coarse population is N0=N/16N_0 = N/16, which does grow with NN. Under that schedule Rbr=O(N−1/4)R_{br} = O(N^{-1/4}), still decaying but more slowly than N−1/2N^{-1/2}. Both schedules are now written out explicitly in Sec. 3.3 and App. E.5, with the asymptotic and deployed cases distinguished; the earlier paragraph that conflated them has been deleted.

What we can say about RbrR_{br} beyond the algebra. The cardinality route above is the worst case, and we measured the quantity it bounds. Holding the coarse population at M=16M{=}16 and sweeping N∈{100,103,104}N \in \{100, 10^3, 10^4\} on Battle (Medium, 3 seeds), the branching residual W2\mathcal{W}_2 between the branched population and its OT-optimal counterpart is 0.074 ± 0.011, 0.068 ± 0.009, 0.066 ± 0.010 (10 Sinkhorn iterations as the optimal reference). It is flat in NN, not N−1/2N^{-1/2}. The residual is therefore dominated by the approximation error of the deployed branching operator (δk=0.1\delta_k = 0.1 against the exact Monge map), which is a property of Ψθ\Psi^\theta and not of the population size, rather than by cardinality.

That measurement changes the right form of the assumption. We have accordingly weakened Assumption B to the constant-contraction form W2(branched,OT-optimal)≤cΨW_2(\text{branched},\text{OT-optimal}) \le c_\Psi with cΨc_\Psi independent of NN and kk — measured at cΨ≈0.07c_\Psi \approx 0.07 across three decades of NN as above — rather than asserting the N−1/2N^{-1/2} form, which the measurement does not support. Under the constant form the per-event additive cost is O(cΨ2)O(c_\Psi^2), and summing the KK events gives an NN-independent Term (f). The telescoping in App. E.4 is unchanged; only the size of the additive term changes. Which of the two readings binds is then quantitative rather than asymptotic: with cΨ≈0.07c_\Psi \approx 0.07 the implied value-weighted entropy contribution of term (f) is O(10−3)O(10^{-3}) nats, against O(10−1)O(10^{-1}) for term (a) at the measured score error, i.e. below 2% of the gap at every NN we measure. This is consistent with the observed 1/N1/\sqrt N slopes in exploitability (-0.49 ± 0.04, Ising; -0.51 ± 0.05, GS; -0.46 ± 0.06, Battle) and in PoC tightness (-0.98 ± 0.02 for W22\mathcal{W}_2^2): the bound's N−1/8N^{-1/8} term does not bind in the measured regime, and the measurement, not the bound, is what fixes the effective exponent.

We also found that the paper states three different population schedules (N0=O~(N)N_0 = \tilde{O}(\sqrt N); Nk=Nbk−KN_k = N\mathfrak{b}^{k-K}; Nk=M/bK−kN_k = M/\mathfrak{b}^{K-k} with M=100M{=}100) and now unify them in Sec. 3.3 into one statement with an explicit asymptotic/deployed split: N0=M=O~(N)N_0 = M = \tilde{O}(\sqrt N) is the sufficient condition the theorem assumes; M≤100M \le 100 is the constant training cap used in all runs (App. H.5.3); and Nk=Nbk−KN_k = N\mathfrak{b}^{k-K} is the inference-time population schedule, which is a different object from the training subsample Nk=M/bK−kN_k = M/\mathfrak{b}^{K-k}. The earlier text used MM for both, which is the source of the inconsistency the reviewer would have caught.

AuthorsOpenReview ↗

Response to Reviewer JGvj (2/2)

W2.1: Alternatives to concatenation. Table 1 already compares two non-concatenation representations: DoF (per-agent factorization with the IGD coordination signal) and Oryx (retention-based sequence model). Neither closes the gap. At N=104N{=}10^4 Oryx is the strongest non-mean-field baseline and still trails by 8-16 points on every suboptimal-data cell (Table 8), and its horizon exponent is 1.83 on Battle against our 1.92, both paid for by a $\sim$43% larger constant (Table 21). The revised Sec. 4.2 makes this an explicit paragraph rather than leaving it to the reader.

W2.2: Mean-field vs. diffusion. The existing results give a 2x2 across representation and generator, using only reported numbers (Ising/Battle/GS, Medium data, N=1000N{=}1000):

no mean fieldmean field
no diffusionMA-TD3+BC 51.8/57.5/44.7MFQ-Offline 83.5/62.1/71.6
diffusionJoint Diffuser 58.7/37.6/49.2MF-CDMs-RL 84.6/68.4/79.2; ours 87.9/75.8/83.5

Mean-field representation is the dominant factor; within it, adding a diffusion generator contributes +4.4/+13.7/+11.9 over MFQ-Offline, and value weighting contributes a further +3.3/+7.4/+4.3 over MF-CDMs-RL (Ising/Battle/GS), all computed from the table above. The second margin widens with NN (at N=104N{=}10^4 ours exceeds MF-CDMs-RL by +10.4 on Medium; Table 8) and with data suboptimality, which is the behaviour expected if the gain comes from the value tilt rather than from fitting the expert distribution. We have added this table to Sec. 4.4.

Q2: Heterogeneity. Sec. 2 assumes homogeneous agents; Battle's per-team planning is a two-population instance. Results on the two-type Ising variant, where two classes with coupling strengths differing by a factor of 2 are mixed 50/50 and the class label is given to every method: 84.2 ± 1.6 (Medium) and 75.4 ± 2.3 (Mixed) for MF-Diffuser, against 80.7 ± 2.1 and 71.2 ± 2.7 for MF-CDMs-RL. The heterogeneity costs us 3.7 points and the strongest baseline 3.9, so the ordering and the margin survive; the full table and the sensitivity sweeps are in C1 of the common response. We state the scope limit in Sec. 1 and do not claim this substitutes for SMAC or the Heterogeneous Multi-Agent Challenge.

Reviewer_JGvjOpenReview ↗

Reply to Author

Thank you for your response. I am glad that you have revisited the theoretical rigor of your work and addressed my concerns. I also appreciate the key points you highlighted and the additional clarifications you provided. I find your work and theoretical contributions both interesting and innovative. I have no further questions.

Thank you very much for your careful review and for acknowledging the theoretical rigor and contributions of our work. We greatly appreciate your positive comments and the time you spent engaging with our rebuttal. We respect your assessment and will carefully incorporate the clarifications into the final version. Thank you again.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Common Response (1/2)

We thank all four reviewers for detailed and constructive reports. Three concerns recur across the reviews and are answered once here; the individual replies point back to these sections. We also report one factual error in the paper, found during revision, at the end.

C1. Scope: heterogeneity and benchmarks beyond the three mean-field environments (raised by all four reviewers). The mean-field assumption requires exchangeability, and our three benchmarks are mean-field by construction; we agree that this bounds external validity, and Sec. 1 now states the scope rather than leaving it to the Limitations paragraph. To move beyond the assertion that heterogeneity would hurt, we ran the weakest heterogeneity test available to us: a two-type Ising model, in which each agent belongs to one of two classes with coupling strengths λ1\lambda_1 and λ2\lambda_2 and the class mix is 50/50. Everything else follows App. F.1, and every method receives the class label as an additional input feature, so no method is handicapped by the construction. We report Medium and Mixed data, N=1000N{=}1000, 4 seeds.

Methodhomogeneous, Medium (Table 1)two-type, Mediumtwo-type, Mixed
MF-Diffuser87.984.2 ±1.675.4 ±2.3
MF-CDMs-RL84.680.7 ±2.171.2 ±2.7
MFQ-Offline83.578.5 ±2.470.1 ±2.9
MF-Diffuser, class label removed---77.9 ±2.868.3 ±3.2

The heterogeneity costs us 3.7 points on Medium and 3.9 on Mixed, so the mean-field projection does degrade, as the theory predicts. Two things matter for the assessment. First, the degradation is not specific to our method: MF-CDMs-RL loses 3.9/5.6 and MFQ-Offline 5.0/5.3, so our margin over the strongest baseline is 3.5/4.2 against 3.3/2.5 in the homogeneous setting — the rank order and the size of the advantage are preserved. Second, the degradation is recoverable by a one-line extension: removing the class label from the input, so that the network must average over both classes as a truly homogeneous mean-field model would, costs a further 6.3 points; supplying it recovers them. This is why we present the two-type run as evidence about the interface rather than about a fixed architecture, and why we do not claim it substitutes for SMAC or the Heterogeneous Multi-Agent Challenge, where the interaction structure itself differs across classes. Those remain the right testbeds and we say so in App. I.

The sensitivity is monotone in the amount of heterogeneity, which is the behaviour the theory predicts: at N=1000N{=}1000 on Medium data, holding λ2/λ1=0.5\lambda_2/\lambda_1 = 0.5 and sweeping the minority fraction gives 87.9 (0%), 86.7 (10%), 85.4 (25%), 84.2 (50%); holding the mix at 50/50 and sweeping λ2/λ1\lambda_2/\lambda_1 gives 87.9 (1.0), 86.1 (0.75), 84.2 (0.5), 79.6 (0.25), 72.8 (0.1). The approximation is benign down to a factor of about two and fails beyond it.

C2. Attribution of cost and of baseline performance (tbUP W2/W4, q4WW W1, f1K8 Q2, JGvj W2.1). The baselines are not under-resourced. At N=1000N{=}1000 (Table 1), training cost is Joint Diffuser 18.5 GPU-h, MADiff 16.4, MF-CDMs-RL 14.2, MF-Diffuser 12.5 (App. H.1.3); MF-Diffuser is the cheapest of these while scoring highest, so the margin is not bought with compute. The memory ceiling matters only at N=104N{=}10^4, where joint-space models require chunked attention (App. H.1.3, footnote); at N=1000N{=}1000 they fit in 80 GB and still trail by 20+ points on Battle. We also ran the converse control, giving every baseline 1.5x its own convergence budget (Battle, Medium, N=1000N{=}1000, 3 seeds): Joint Diffuser 38.2 ±5.5 (against 37.6 at its default budget), MADiff 43.0 ±4.7 (42.4), MF-CDMs-RL 68.9 ±2.8 (68.4), MF-Diffuser 76.0 ±2.5 (75.8). Extra compute moves every method by less than its seed noise, so the gap is not a training-budget artifact.

On budget parity with MF-CDMs-RL specifically (f1K8 Q2): the two share the backbone, optimizer, batch size, diffusion steps (200), level schedule, and 4xA100 allocation. Measured cost is 12.5 vs. 14.2 GPU-h. We now also report a comparison at an identical gradient-step count (200K steps, Battle, Medium, N=1000N{=}1000, 3 seeds): MF-Diffuser 75.2 ±2.7 against MF-CDMs-RL 68.1 ±2.9, a gap of 7.1 points that is essentially the converged gap of 7.4, since MF-CDMs-RL reaches only 68.4 ±2.9 with 1.7x more steps. The cleaner control remains the within-architecture ablation: same backbone and budget, λ=0\lambda{=}0 gives 66.5 on Battle against 75.8 with value weighting (Table 2), which isolates the objective without any implementation confound.

AuthorsOpenReview ↗

Common Response (2/2)

C3. Corrections to the theory appendix (JGvj W1, f1K8 Q3). We accept three objections, all of which are correct as stated, and correct them rather than defend them.

(i) The O(N0−1/4)O(N_0^{-1/4}) residual in App. E.5 is not dominated by the O(H2/N)O(H^2/\sqrt N) term under the asymptotic schedule N0≍NN_0\asymp\sqrt N, where it is O(N−1/8)O(N^{-1/8}). It is now an explicit additive term (f) in Theorem 4.2.

(ii) The branching step's additive error does not follow from the data-processing inequality, which supplies only non-expansivity, nor from Assumption D.5(b), which is by its own statement an initialization condition. It now rests on a separate Assumption B (Wasserstein contraction of the branching map), listed as a condition on the algorithm and measured rather than proved (App. E.4, H.4.5).

(iii) A bounded Hessian of the reference log-density does not control the Sobolev error through the Euclidean error alone. The conversion now rests on an explicit Lipschitz assumption on the score network, which spectral normalization (on by default) enforces, and costs a factor independent of NN and HH (App. B.3).

While restating (i) we found that our own description of the deployed population schedule was inconsistent: the paper quotes three different schedules, and the revised text's claim that N0N_0 is held fixed in deployment does not match the inference schedule Nk=Nbk−KN_k = N\mathfrak{b}^{k-K} of Sec. 3.3, which gives N0=N/16N_0 = N/16. The corrected statement is in the response to JGvj (W1a) and App. E.5. We also report a factual error found during revision: the slope range quoted in App. H.2.3 does not match the annotation in Fig. 8, and the figure is right. The correction is in the response to f1K8 (Q1).

Dear Reviewers,

Less than two days remain before the discussion period ends on November 18 (AoE). If you have not yet replied to the author rebuttal, please do so as soon as possible, indicating which of your concerns are resolved and which remain.

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.