acceptodds
Under review as a conference paper at ICLR 2027

Complexity-Balanced Diffusion Splitting

Abstract

Standard continuous-time generative models rely on monolithic architectures that must navigate vastly different signal regimes, from isotropic noise to intricate data distributions. While scaling model capacity improves performance, deploying a massive network uniformly across the entire generative timeline is inherently inefficient. In this work, we propose Complexity-Balanced Splitting (CBS), a principled framework for temporal capacity allocation that distributes the generative workload across multiple specialized sub-networks. Motivated by function approximation theory and de Boor’s equidistribution principle, CBS partitions the diffusion timeline into segments of equal approximation burden, allocating more representational capacity to regions where the generative dynamics are more difficult to model. To estimate this local complexity, we introduce two complementary and tractable monitor functions: a spatial measure based on the flow’s Dirichlet energy, and a geometric measure based on the acceleration of the sampling trajectories. Using a lightweight auxiliary model to estimate these complexity profiles, our approach eliminates the need for heuristic temporal splits or computationally expensive search procedures. Extensive evaluation across multiple architectures (SiT, JiT, and UNet) and datasets demonstrates that CBS consistently improves synthesis quality without increasing per-step inference cost. In particular, CBS improves FID by   on SiT-XL with CFG relative to uniform temporal partitioning.

Primary area:
generative models
Keywords:
Generative Models, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZUrFOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies where to place temporal boundaries when a diffusion or flow model is split into equal-capacity specialists, only one of which is active at a given sampling time. Complexity-Balanced Splitting (CBS) first trains a lightweight auxiliary flow, estimates a positive complexity monitor along time, and chooses boundaries that equalize the cumulative monitor mass. The two proposed monitors use a density-weighted spatial Jacobian/Dirichlet-energy quantity and trajectory acceleration, respectively.

CBS is tested with SiT on ImageNet-256, JiT on ImageNet-64, and a UNet on CIFAR-10. Relative to uniform time intervals with the same number of specialists, the proposed boundaries consistently improve FID and related metrics while keeping the active parameter count and per-step FLOPs unchanged. The largest reported improvement is from FID 6.24 to 4.03 for SiT-XL with classifier-free guidance. Total training and storage cost still grow with the number of specialists.

Critical strengths
  1. The boundary-selection rule is simple, inexpensive, and appears robust. It replaces an otherwise combinatorial search with a one-dimensional cumulative-mass calculation, and the auxiliary-flow ablations suggest that the resulting boundaries are not overly sensitive to the proxy model.

  2. The empirical result is consistent across three architectures, multiple datasets, different numbers of specialists, latent and pixel-space models, and both monitor choices. The improvement over uniform splitting is often substantial. This breadth is the main reason I am on the positive side despite the concerns below.

Critical weaknesses
  1. I see the theoretical development as motivation rather than a justification of the particular monitors. The assumptions linking local approximation error to the chosen complexity quantities are strong, and the Dirichlet-energy bound does not directly match the weighted estimator used in practice. I also could not reconcile the acceleration monitor with the usual local interpolation error proportional to step size squared times acceleration; this would normally lead to a square-root density rather than simply equalizing acceleration. The method may still work well, but the current claims should be framed as theory-inspired unless this link is tightened.

  2. Unequal intervals also change the effective timestep sampling distribution seen by each specialist, so the experiments do not fully isolate better boundaries from better training reweighting. A monolithic model trained with the induced global timestep density is missing, as is a comparison with the same total number of optimization updates. Most results also appear to use a single training run. Given the extra training and storage cost of specialist models, end-to-end resource measurements would be more informative than active FLOPs alone.

Questions for authors
  1. What happens if a single model is trained with exactly the global timestep density induced by the CBS intervals, with total optimization updates matched? This seems necessary to attribute the gain to specialization and boundary placement.

  2. Can the authors report multiple training seeds for the main ImageNet result, or at least for a representative smaller setting? The current gains are promising, but run-to-run variation is not shown.

  3. Could the authors clarify the exponent used for the acceleration monitor and compare it with the density suggested by the local interpolation-error formula? A held-out velocity-prediction error by interval would also help validate the proposed notion of complexity.

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for the thoughtful theoretical assessment and the positive evaluation of our experiments. We address the theoretical questions and matched-update controls here; Part 2 covers training seeds, held-out errors, and resource measurements.

W1 / Q3: Theory and acceleration exponent

What the theory establishes. For a positive monitor under the integrated-burden assumption (A2), equalizing its cumulative mass exactly minimizes our min–max surrogate (Eqs. 3–5). Equal expert capacity (A1) motivates the objective; whether a particular monitor captures neural-network difficulty (A3) is evaluated empirically. We make this distinction in Sec. 2.2. Similarly, the spectral argument concerns unweighted Dirichlet energy, whereas our tractable, density-weighted Jacobian estimator is a motivated proxy (Sec. 3.1, App. A). Its empirical support comes from the boundary-sensitivity results (Tab. 4), both monitors (App. E), and auxiliary-model ablations (App. C).

Why use acceleration rather than its square root? In piecewise-linear interpolation, local error scaling as h2∥x¨∥h^2\|\ddot x\| leads to a square-root knot density. Our setting partitions work across equal-capacity networks, and assumes the interval burden is proportional to ∫m(t) dt\int m(t)\,dt (A1–A2). Choosing m(t)=E∥x¨t∥m(t)=\mathbb E\|\ddot x_t\| then yields exponent one in this surrogate; we describe it as a modeling choice, not a consequence of the spline error formula. Applying a square-root profile moves the SiT-B/2 split from (0.40, 0.77) to about (0.36, 0.72) (App. M). The empirical justification for the chosen monitor and resulting boundaries is in our experiments, rather than a claim of FID optimality.

W2 / Q1: Exact induced timestep density and matched updates

Three equally trained CBS experts with boundaries (0.40, 0.77) induce the aggregate density p(t)=1/[3(ti−ti−1)]p(t)=1/[3(t_i-t_{i-1})] on interval ii, approximately (0.83, 0.90, 1.45). We trained a single SiT-B/2 model with precisely this timestep distribution:

ModelTotal updatesFID no CFG / CFG 1.5
Single, uniform tt400K34.84 / 16.79
Single, CBS density400K33.41 / 15.89
Single, CBS density1.2M29.30 / 10.68
CBS, 3 experts3×400K30.51 / 10.72

At 400K updates, CBS-induced sampling improves the single network by 1.43 / 0.90 FID; CBS experts achieve 30.51 / 10.72 with the same updates per expert. At matched 1.2M aggregate updates, the reweighted single network yields 29.30 / 10.68, compared with 30.51 / 10.72 for CBS. This establishes the contribution of timestep sampling in this B/2 setup and makes the aggregate training cost transparent (Sec. 4.4, App. I).

Longer matched-update experiment (SiT-S/2):

Total updatesSingleCBS (3 experts)
1.2M48.73 / 24.5949.68 / 25.42 (3×400K)
2.4M43.82 / 19.6841.93 / 18.30 (3×800K)

At 2.4M total updates, CBS improves FID by 1.89 / 1.38 relative to a single network trained with standard timestep sampling. This is a separate, longer-training setting from the B/2 density control.

Our main controlled claim concerns boundary placement among equally sized expert ensembles: CBS consistently outperforms uniform boundaries with the same expert count, per-expert training schedule, total updates, and inference FLOPs (Tab. 1). This isolates the value of the complexity-balanced split independent of the extra cost of choosing a multi-expert design.

AuthorsOpenReview ↗

Rebuttal (2/2)

W2 / Q2: Multiple training seeds

We ran three independent training seeds for each model on CIFAR-10:

ModelFID, mean ± std (3 seeds)
Single network3.55 ± 0.03
Uniform (0.33, 0.66)3.51 ± 0.02
CBS (0.20, 0.78)2.73 ± 0.02

CBS improves FID by approximately 0.8 over both controls, with observed run-to-run standard deviations of only 0.02–0.03. These independent seeds provide evidence of repeatability on a second architecture and dataset (App. O).

Q3: Held-out velocity-prediction errors

We computed E∥vθ(xt,t)−(x1−x0)∥2\mathbb E\|v_\theta(x_t,t)-(x_1-x_0)\|^2 at 100 timesteps on 10K ImageNet validation latents, comparing the SiT-B/2 single network, uniform experts, and CBS experts (400K steps per network; App. N, Fig. 3).

The largest improvement occurs in the low-noise interval [0.77,1][0.77,1], where the acceleration monitor is highest. Relative to the single network, CBS reduces prediction error there by 1.6% on average and up to 2.1%, versus 1.4% / 1.8% for uniform experts. For the single network, the maximum integrated interval error is 0.290 under CBS boundaries versus 0.300 under uniform boundaries. Both observations are consistent with the complexity profile identifying an effective split.

The two expert configurations have similar integrated prediction errors (0.758 each, versus 0.764 for the single network) despite different FIDs. We report the per-timestep curves alongside the generation results rather than relying solely on a global velocity-MSE statistic.

W2: Training and inference resources

We added SiT-B/2 measurements (App. P):

ResourceSingleCBS, N=3
Total / active parameters130M / 130M390M / 130M
GFLOPs per sampling step23.0123.01
Main-model training GPU-hours≈106≈318
Training wall-clock≈53 h on 2×H200≈53 h on 6×H200 (3 concurrent jobs)
Peak sampling memory15.8 GB15.3 GB (sequential loading)
Sampling img/s, no CFG / CFG 1.53.84 / 1.783.71 / 1.75

CBS retains the same active parameters and per-step FLOPs, with approximately 2–3% lower sampling throughput. The three experts require three times the main-model training GPU-hours and stored parameters; parallel training achieves comparable wall-clock time when the GPUs are available. The GPU-hour estimates cover main-model training, with auxiliary-model and monitor costs discussed separately in App. C.

Manuscript updates: Secs. 2.2/3.1 and Apps. A/M (theoretical scope and exponent); Secs. 4.4–4.5 and App. I (induced density and matched updates); Apps. N/O/P (held-out error, seeds, resources); Conclusion (limitations). The monitor's assumptions are discussed in the cited sections and appendices.

We hope that the additional experiments and clarifications address the reviewer's concerns and strengthen the overall contribution of our work.

Dear Reviewer,

Thank you for your time and effort in reviewing the submissions. We kindly ask you to check the authors' responses and engage in further discussion if necessary. Please let us know promptly whether your concerns have been addressed or if any issues remain unresolved.

Your timely feedback would be greatly appreciated. Thank you for your cooperation!

Thank you for the response and updated experiments. My concerns have been resolved, and I will keep my current score.

Reviewer_J3ZpOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper addersses the lack of a principled way to split the diffusion timeline among experts in multi-expert diffusion models, where existing choices rely on heuristics or expensive search. Following the equidistribution principle, it use a small auxiliary model to estimate the curvature (path acceleration) of sampling trajectories and places boundaries so that each expert receives an equal share of difficulty. On SiT, JiT and UNet, it shows better generation quality than a single network and a uniform split at the same per-step inference cost (e.g., FID from 6.53 to 4.03 on SiT-XL with CFG). However, the total training compute of the experts is 3× that of the baseline, the models are far from fully trained, comparisons with existing multi-expert methods are missing, and the experiments are limited to small-scale class-conditional generation.

Critical strengths

(1) The rule is simple, practical and nearly cost-free, and auxiliary models of different sizes yield consistent split points, indicating that the method is fairly stable.

(2) The experiments show that where to split matters more than how many experts are used, and the overall study is fairly solid.

Critical weaknesses

(1) The comparison is unfair and the models are under-trained: each expert is trained for the same 400K steps as the single network, so the total training compute of CBS is 3× that of the baseline, and the authors acknowledge that 400K steps are far below what is needed for the best results. It is therefore unclear whether the gains come from more training compute, faster convergence, or genuinely better final performance. I suggest adding (1) a single-network baseline with matched total training compute and (2) FID curves under longer training.

(2) The paper only compares against a single network, a uniform split and similar variants, without comparing to existing multi-expert or timestep-routing methods such as eDiff-I, MEME, Switch-DiT and Diff-MoE; alternatively, the authors should explain why such comparisons are unnecessary.

(3) How many generated images are used to compute FID? In addition, all methods use the same fixed CFG scale (1.5 for SiT), while the gain of CBS with CFG (~35%) is much larger than without CFG (~12%). Since different models may have different optimal CFG scales, a single fixed scale may enlarge or shrink the gap between methods. Could the authors report FID curves across CFG scales, or compare each method at its own optimal CFG, to confirm the actual gain of CBS?

Questions for authors

pls see weaknesses.

Note: My preliminary assessment lies between weak reject and weak accept; I currently give weak reject and will reassess after the rebuttal.

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for the constructive assessment. Our new experiments distinguish the benefit of choosing better boundaries within a fixed multi-expert design from comparisons with single-network training.

W1: Matched training compute and longer training

Training curves. We evaluated a single SiT-B/2 network and three CBS experts every 100K steps (FID from 50K samples, without / with CFG 1.5):

Steps per networkSingleCBS (3 experts)
100K66.52 / 45.9657.40 / 34.58
200K49.71 / 27.9340.64 / 18.65
300K41.89 / 20.5934.00 / 13.26
400K34.84 / 16.7930.51 / 10.72

CBS achieves lower FID at every checkpoint with the same steps per network. To account for the three experts' aggregate compute, we also evaluated matched total updates over a longer training horizon.

Matched total training updates (SiT-S/2). We trained a single network up to 2.4M steps and CBS experts up to 800K steps each:

Total updatesSingleCBS (3 experts)
1.2M48.73 / 24.5949.68 / 25.42 (3×400K)
2.4M43.82 / 19.6841.93 / 18.30 (3×800K)

At 1.2M updates the results are close. At 2.4M, CBS improves FID by 1.89 without CFG and 1.38 with CFG. Over that interval, CBS improves by 7.75 / 7.12 FID, versus 4.91 / 4.91 for the single network. This provides evidence for a benefit at matched aggregate training updates in the longer SiT-S/2 setting (Sec. 4.5, App. I).

Induced-timestep-density control (SiT-B/2). We also tested the reweighting control requested by Reviewer ZUrF. At 400K updates, a single network trained with the CBS-induced density reaches 33.41 / 15.89, versus 34.84 / 16.79 with uniform timestep sampling; CBS reaches 30.51 / 10.72 using three 400K-step experts. At matched 1.2M aggregate updates, the reweighted single network reaches 29.30 / 10.68, versus CBS's 30.51 / 10.72. Thus, the B/2 density control and S/2 longer-training comparison provide complementary evidence across different training budgets and sampling setups.

The central comparison remains CBS versus uniform boundaries with identical expert architecture, number of experts, total training updates, and per-step inference cost. This directly isolates boundary placement, which is the contribution of our paper. The revised Secs. 4.4–4.5 and App. I present both monolithic controls and the corresponding compute accounting. Experts can also train concurrently, trading additional GPUs for shorter wall-clock time.

AuthorsOpenReview ↗

Rebuttal (2/2)

W2: Existing multi-expert and timestep-routing methods

CBS addresses where to place the temporal boundaries for separate fixed-capacity experts, using an inexpensive auxiliary-model profile. We expanded Sec. 5 to distinguish this from eDiff-I's expert specialization, MEME's heterogeneous experts and soft interval assignment, and the within-network timestep routing of DTR, Switch-DiT, RAPHAEL, and Diff-MoE. These approaches are complementary to boundary selection.

DTR comparison (SiT-B/2). We adapted the official DTR implementation from DiT to SiT and trained with the same 400K-step recipe:

MethodFID (no CFG / CFG 1.5)IS (CFG 1.5)
Single34.84 / 16.7984.8
DTR31.23 / 11.80113.4
CBS (0.4, 0.77)30.51 / 10.72121.1

CBS obtains better FID (by 0.72 / 1.08) and IS than DTR in this comparison. DTR trains one network for 400K updates; CBS trains three networks for 400K each. We report the distinction explicitly and note that DTR-style routing could be incorporated into CBS experts.

eDiff-I-style shared initialization. We pretrained a single network for 200K steps, then initialized three experts from that checkpoint and fine-tuned each for 200K steps (800K aggregate updates):

BoundariesFID no CFGFID CFG 1.5IS CFG 1.5
Uniform (0.33, 0.66)31.1711.58112.8
CBS (0.4, 0.77)30.9711.44115.2

CBS improves all three metrics even with shared initialization. The gap is modest (0.20 / 0.14 FID), and both variants attain competitive FID with fewer total updates than independently trained experts. This experiment tests CBS within an eDiff-I-style recipe, without claiming to reproduce the full eDiff-I system (App. L).

W3: FID sample count and CFG sensitivity

Evaluation protocol. Main ImageNet-256 FIDs use 50,000 generated images, the ADM evaluator and VIRTUAL_imagenet256_labeled reference batch, with 250-step Euler sampling. CFG follows the SiT/DiT convention (first three latent channels) for every method. We compute CBS boundaries from the unguided flow; guidance changes them by less than 0.01 (Sec. 4.1, Apps. G/H).

CFG sweep (SiT-B/2). We first evaluated eight scales with 10K samples each:

CFG1.01.251.51.752.02.53.04.0
Single40.3527.9519.6414.6712.1610.6011.3014.18
CBS33.6521.0813.799.928.549.1511.3215.84

We then reevaluated scales near the respective minima with 50K samples:

ModelCFGFIDISPrec.Rec.
Single2.09.34136.60.780.46
Single2.57.86179.50.840.38
Single3.08.59213.20.880.32
CBS1.757.03160.70.780.51
CBS2.05.66196.20.820.46
CBS2.56.31253.10.880.38

At each method's best evaluated CFG scale, CBS improves FID by 28% (5.66 versus 7.86), with higher recall (0.46 versus 0.38). Thus, the gain remains substantial when guidance is tuned separately, rather than fixed at 1.5 (Sec. 4.5, App. J).

Manuscript updates: Secs. 4.4–4.5 and App. I (training/compute controls); Sec. 5 and App. L (related methods and DTR/eDiff-I-style experiments); Sec. 4.1 and Apps. G/H/J (evaluation and CFG sweep); Conclusion and App. P (resource costs). Revisions are marked in green.

We hope that the additional experiments and clarifications address the reviewer's concerns and strengthen the overall contribution of our work.

Thanks for the rebuttal and new experiments. After reading other reviewers' comments and the responses, I still have a few concerns:

(1) At matched 1.2M updates, the reweighted single model slightly beats CBS (10.68 vs. 10.72). The longer experiment doesn't include this baseline.

(2) CBS and uniform have the same error (0.758), but very different FIDs. Not sure how this supports the complexity measure.

(3) The gain is quite small (11.58 vs. 11.44). Is it stable across seeds?

The rebuttal helps, but these points are still not fully clear to me. For now, I'll keep my original rating.

We thank the reviewer for the follow-up and for carefully considering our additional experiments. We would like to clarify the three remaining concerns.

(1) Matched training compute. We acknowledge that the reweighted single network achieves comparable performance at matched compute, as explicitly reported in the revised manuscript. However, our central contribution is not to outperform all monolithic models at matched training compute, but to determine better temporal boundaries within a fixed multi-expert architecture. In this setting, CBS consistently outperforms uniform, SNR-based, and hand-designed splits with identical expert counts, training budgets, and inference costs. The longer-training results provide additional evidence of the benefits of specialization.

(2) Velocity prediction error. Similar aggregate velocity errors do not necessarily imply similar generation quality, since errors at different timesteps can have different effects on the resulting trajectories. Importantly, our timestep-wise analysis (App. N) shows that CBS achieves its largest error reduction in the high-complexity region identified by our monitor. Furthermore, the boundary-sensitivity experiments (Tab. 4) demonstrate that our derived boundaries outperform nearby alternatives, providing direct empirical support for the proposed complexity-based splitting criterion.

(3) Stability across seeds. We agree that the improvement in the shared-initialization experiment is relatively modest, and present it as supplementary evidence of compatibility with this training strategy. Our main results demonstrate substantially larger improvements across multiple architectures, datasets, and expert counts. Moreover, our three-seed CIFAR-10 evaluation (App. O) confirms consistent improvements, with FID 2.73 ± 0.02 for CBS versus 3.51 ± 0.02 for uniform splitting.

We hope these clarifications help resolve the remaining concerns and highlight the consistent empirical evidence supporting our main contribution.

Reviewer_2eG7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper builds off recent advances in segmented workloads across the reverse process timeline in generative diffusion models. Rather than training a single model to learn the denoising process across the entire range of noise levels, these approaches apply different subnetworks at different times, trading increased training-time compute and parameter count for superior generation quality without increasing inference generation time. Complexity based splitting extends these approaches by suggesting that the generation timeline should not be split uniformly, and should instead be split according to approximately equal workloads. Thus, the contribution of the paper lies in an efficient means by which to estimate the workloads in order to achieve this equalized splitting. They propose two different monitor functions: one based on the bounded spatial variation using Dirichlet energy, the other based on temporal complexity using path acceleration. They demonstrate results across three datasets using various architectures both in latent generation and pixel-space. Their main baseline is a uniform segmentation, but they include two additional baselines for specific settings including a hand-picked split and one based on dynamical regimes. Evaluated at N=3 segments, complexity based splitting outperforms all other methods across the metrics and datasets.

Critical strengths
  • The paper is well-written, especially the problem introduction, and the necessary mathematical concepts are built up clearly and cleanly. The assumptions are appropriately scoped and honest.
  • The author's mathematical treatment of their setup is deep and well considered. After reading, I have a genuinely better understanding of how to approximate complexity and workloads across the diffusion timeline in generative models, which could have applications beyond segmenting workloads.
  • The experimental suite's tested settings are well-chosen. The sensitivity to boundary placement analysis (table 4) is especially appreciated. The results indicate that complexity based splitting is a strong performer, sometimes much stronger than uniform baselines.
Critical weaknesses
  • The most dramatic advantages are in comparison to a uniform split baseline, but there are seemingly other reasonable choices for heuristic splitting that are more straightforward than the author's complexity based splitting. In particular, SNR-based splitting would be a stronger baseline to compare against, especially considering that Wan 2.2 (the author's own listed example of a frontier system implementing segmented workloads) uses an SNR-based splitting heuristic which is not tested in this paper, to my understanding.
  • The main evaluations lie only at a split of N=3 segments. It's not clear how the advantage against baselines will hold at different segment counts, including N=2 which is used by Wan 2.2.
Questions for authors

The following questions would help clarify the weaknesses above. If compute is limited, the SiT-B/2 setting would be sufficient to see further results on.

  1. Can the authors run additional baselines based on SNR splitting: one using the heuristic from Wan 2.2, and perhaps another one based on equal split log-space SNR (truncating near 0)?
  2. Can the authors include baseline results for the same sweep across N in table 6 from the appendix (expected that they match at N=1)? I am interested to see how the advantages of CBS hold at values other than N=3.
2 comments

We thank the reviewer for the encouraging assessment and the concrete suggestions. We added both requested SNR-based comparisons and uniform-split baselines throughout the number-of-experts sweep. All new experiments use SiT-B/2 on ImageNet-256, with 400K training steps per network and 50K generated samples for the reported FIDs.

W1 / Q1: SNR-based boundaries

We use t=0t=0 for noise and t=1t=1 for data. For the linear path, SNR(t)=[t/(1−t)]2\mathrm{SNR}(t)=[t/(1-t)]^2.

Wan 2.2 (N=2): We use the Wan 2.2 threshold t=0.125t=0.125 for the two-expert split.

Equal log-SNR (N=3): We partition log⁡SNR(t)\log\mathrm{SNR}(t) into three equal ranges over [ϵ,1−ϵ][\epsilon,1-\epsilon], taking ϵ=0.004\epsilon=0.004 from the first step of the 250-step sampler. This gives boundaries (0.137, 0.863); truncation sensitivity is reported in App. K.

SplitNBoundariesFID no CFGFID CFG 1.5IS CFG 1.5Precision / Recall, CFG 1.5
Single network1—34.8416.7984.80.66 / 0.55
Wan 2.2 threshold20.12538.0116.4689.20.67 / 0.55
CBS20.632.1912.98109.40.68 / 0.56
Uniform3(0.33, 0.66)34.5216.5187.20.66 / 0.56
Equal log-SNR3(0.137, 0.863)32.5912.47107.20.70 / 0.56
CBS3(0.4, 0.77)30.5110.72121.10.72 / 0.56

At N=2, CBS improves over the Wan threshold by 5.82 FID without CFG and 3.48 with CFG. At N=3, it improves over equal log-SNR by 2.08 / 1.75 FID. Equal log-SNR itself improves on uniform splitting, making it a valuable stronger heuristic baseline; CBS retains the best FID and IS at the same number of experts. We incorporated these results into Sec. 4.3, Tab. 6, and App. K.

W2 / Q2: Uniform-split baseline across N

We extended the original N-sweep to include uniform partitions for N=1, 2, 3, and 4 (App. D):

Number of expertsUniform FID (no CFG / CFG 1.5)CBS FID (no CFG / CFG 1.5)
134.84 / 16.7934.84 / 16.79
234.65 / 16.6032.19 / 12.98
334.52 / 16.5130.51 / 10.72
434.30 / 16.1929.33 / 9.42

The methods coincide at N=1, as expected. CBS consistently outperforms uniform splitting for N=2, 3, and 4, and the gap grows with N. At N=2, CBS improves over uniform by 2.46 / 3.62 FID. The equal-log-SNR boundary for N=2 is t=0.5t=0.5 by symmetry, so the uniform two-expert baseline also covers that case.

Within each N, these are controlled comparisons with the same number and capacity of experts, training steps per expert, and per-step inference cost. The additional training compute and parameter storage from increasing N apply equally to both splitting strategies. This directly supports our main claim: the placement of the boundaries matters, beyond simply adding more experts.

Changes in the revised manuscript

  • Sec. 4.3, Tab. 6 and App. K: Wan 2.2 threshold and equal-log-SNR baselines, including the latter's truncation sensitivity.
  • App. D, Tab. 8: uniform and CBS performance for N=1–4, including N=2.
  • Clarified parameter-storage and aggregate-training-compute scaling with the number of experts.

We hope that the additional experiments and clarifications address the reviewer's concerns and strengthen the overall contribution of our work.

Thank you for the new experiments. I am glad to see that CBS remains superior to SNR-based splitting at both 2 and 3 segments. It is also interesting to see that uniform segmenting doesn't gain benefits as segment count increases, but CBS does. My only remaining question pertains to the uniform splitting of N=2 segments which appears to beat Wan's own split for 2 segments, which is surprising. Regardless, I stand by my original positive assessment.

Comments on the paper

1 comment

We thank all reviewers for their constructive feedback, which helped us substantially strengthen the paper during the rebuttal.

We have revised the manuscript with several new experiments and clarifications:

  • Stronger baselines and boundary selection: We added SNR-based splitting baselines and extended the number-of-experts study, showing that CBS consistently outperforms alternative partitions at matched expert counts and training budgets.
  • Additional comparisons: We evaluated CBS against DTR and within an eDiff-I-style shared-initialization setting, further demonstrating its effectiveness and applicability.
  • Training and compute: We added longer-training experiments, matched aggregate training-compute comparisons, timestep-reweighting controls, and detailed resource measurements to clarify the benefits and tradeoffs of our approach.
  • Robustness and theoretical analysis: We added three-seed evaluations, held-out prediction-error analysis, and independently optimized CFG comparisons (showing 28% lower FID than the single-network baseline), alongside clearer theoretical assumptions and limitations.

We hope these substantial additions and clarifications address the remaining concerns and further demonstrate the value of our principled, efficient boundary-selection approach.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.