acceptodds
Under review as a conference paper at ICLR 2027

Learning Where to Split: Dynamic Band Splitting for Audio Source Separation

Abstract

Audio source separation models must preserve fine spectral structure while keeping the cost of the separator tractable. Band-split architectures reduce this cost by manually partitioning the spectrum into predefined frequency bands and encoding each band as a token, but the same boundaries are used for every mixture. Because informative spectral structure varies with source content, fixed boundaries can merge discriminative components or spend tokens where little resolution is needed. We introduce Dynamic Band Splitting (DBS), an end-to-end framework that adapts boundary locations and band widths to each mixture under a fixed band-token budget. DBS predicts content-conditioned boundary scores within frequency blocks, selects a prescribed number of boundaries using hard Top-, and applies content-dependent pooling to obtain compact band tokens. After separation, it smooths and expands these tokens to the original STFT grid, refines them along frequency, and decodes complex masks with a frequency-shared head. DBS improved separation over the corresponding static models on DnR v2, MUSDB18-HQ, and EchoSet, with DBS-BSRNN achieving relative gain of 29.0% in average SI-SDR on DnR v2. These results and boundary analyses on DnR v2 show that content-adaptive frequency partitioning benefits multiple separator backbones under a fixed band-token budget. Code and audio demos are available at https://dbs-audio.github.io/DBS-Demo/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio source separation, dynamic band splitting, adaptive frequency partitioning, spectral representation learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_8t7GOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Dynamic Band Splitting (DBS), an architecture modification for band-split audio source separation models (such as BSRNN, TIGER, BS-RoFormer, and Mel-Band RoFormer) that predicts input-conditioned frequency band boundaries within four fixed frequency blocks under a fixed band-token budget, followed by token smoothing, frequency refinement, and mask decoding.

Critical strengths

The paper is well written and clear, and adapting frequency band boundaries based on the spectral structure of the input mixture is an intuitive engineering modification for band-split architectures.

Critical weaknesses
  1. There is no significant methodological novelty here for an ICLR publication beyond a straightforward architectural modification that trades off memory and more than 50% extra compute (FLOPs/MACs) for marginal SI-SDR gains over existing band-split models like BSRNN and TIGER. For example, in Table 1, going from BSRNN to DBS-BSRNN gives only about a 1.4 dB improvement (7.7 to 9.1 dB) while increasing compute from 76.6 to 124.0 G/s MACs (more than a 60% increase in FLOPs), and similar compute increases hold across the other tables for 0.3 to 1.4 dB gains. Furthermore, reporting a second decimal point in all SNR/SDR tables shows a disconnect from what constitutes a statistically or perceptually significant difference between models in the 7 to 10 dB regime, where even a 1 dB difference is difficult to distinguish.

  2. The choice and implementation of the learnable filterbank baseline in Table 3 (which collapses to −11.7-11.7 dB SI-SDR) is questionable; if the authors' baseline setup did not work, including a broken baseline in the main table is not informative. Prior work such as Conv-TasNet [A] and two-step sound separation on learned latent targets [B] has already shown that learnable filterbanks and source-structured latent spaces work well for separation, so learning source-adapted spectral or latent structure is not a new finding.

[A] Luo, Y. and Mesgarani, N., 2019. Conv-TasNet: Surpassing ideal time-frequency magnitude masking for speech separation. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 27(8), pp. 1256-1266.

[B] Tzinis, E., Venkataramani, S., Wang, Z., Subakan, C. and Smaragdis, P., 2020, May. Two-step sound source separation: Training on learned latent targets. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 31-35). IEEE.

Questions for authors

How do the static baselines (such as BSRNN and TIGER) perform when scaled up to match the exact same FLOPs/MACs budget (e.g., 124 G/s MACs on DnR v2) and extra refinement layers used by DBS?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 8t7G (Part 1/2)

We thank the reviewer for the feedback. We address W1–W2 and Q1 below.

W1. Corrected computational cost, adaptive boundaries, and separation gains.

We found errors in the original profiling script: it overcounted LSTM operations and incorrectly counted convolution operations. We corrected the script and recomputed MACs on 3-second clips, normalized by duration, counting one multiply–accumulate as one MAC. The general response gives the full measurement protocol and latency results across all seven model pairs.

Table R11. Corrected MACs (GMAC/s), Static / DBS.

DatasetBackboneStatic / DBS
DnR v2BSRNN41.8 / 48.0
DnR v2TIGER27.0 / 35.0
MUSDB18-HQBSRNN230.3 / 235.5
MUSDB18-HQBS-RoFormer318.0 / 367.0
MUSDB18-HQMelRoFormer204.6 / 285.7

The corrected MAC increases are 14.8% for BSRNN and 29.8% for TIGER on DnR v2, and 2.2%–39.6% on MUSDB18-HQ. Across seven pairs, mean latency changes by −22.2% to +43.4%: it decreases in three pairs, changes by about 2% in two, and has larger increases for BS-RoFormer on MUSDB18-HQ (+43.4%) and BSRNN on EchoSet (+33.6%). DBS-BSRNN improves DnR v2 SI-SDR by 1.38 dB while reducing parameters from 35.5M to 4.7M (87%). DBS-TIGER improves SI-SDR by 1.39 dB. On MUSDB18-HQ, average SDR gains are 0.35, 1.84, and 0.82 dB for BSRNN, BS-RoFormer, and MelRoFormer, respectively.

From fixed bands to mixture-dependent boundaries. BSRNN and BS-RoFormer use predefined frequency boundaries shared across mixtures. DBS predicts input-dependent boundaries while preserving contiguous bands and a fixed token budget. Acoustic transition cues guide Hard Top-K boundary selection within frequency blocks. The resulting partition controls content-weighted pooling and full-resolution complex-mask reconstruction; a local soft-assignment surrogate provides gradients through the separation objective.

Boundary contribution under a shared interface. We trained static controls with the same backbone, pooling, frequency refinement, mask reconstruction, and training settings as DBS. The BSRNN control used a fixed partition; the MelRoFormer control retained its original Mel partition.

Table R12. Static and dynamic partitions with the shared DBS interface (dB).

DatasetBackboneMetricStaticDBS
DnR v2BSRNNSI-SDR8.379.07
MUSDB18-HQMelRoFormerSDR7.538.07

DBS gains 0.70 dB SI-SDR on DnR v2 (8.374 to 9.07) and 0.54 dB SDR on MUSDB18-HQ (7.53 to 8.07). Thus, gains persist when pooling and reconstruction modules are matched.

Reporting precision and source-wise gains. Two-decimal reporting follows prior comparison tables and does not imply statistical significance or perceptual discriminability. Average scores hide source-wise variation: on DnR v2, gains are 1.65 dB for music, 2.35 dB for SFX, and 0.14 dB for speech. On MUSDB18-HQ, DBS-BS-RoFormer improves SDR by 2.50 dB for vocals, 1.96 dB for drums, and 1.34 dB for bass. These are objective results under the stated evaluation settings; we do not claim perceptual significance.

AuthorsOpenReview ↗

Response to Reviewer 8t7G (Part 2/2)

W2. Additional adaptive baseline and distinction from learned filterbanks and latent targets.

Learnable-filterbank experiment. We implemented the published learnable-filterbank specifications described by Dai et al. (2023) and integrated the front end with BSRNN. The authors' code was unavailable for cross-checking. The −11.7 dB result therefore reflects this particular unsuccessful integration, not a general failure of learnable filterbanks. Appendix J analyzes the causes: analysis–synthesis distortion with the separator bypassed, redundant filters concentrated at low frequencies, and latent-channel ordering incompatible with BSRNN's frequency-ordered processing.

Because this adaptation is not a reliable baseline for comparing learnable front ends, we added SFC-CA (Saijo and Bando, 2026), an input-adaptive spectral-compression method, to Table 3. We evaluated it with the same BSRNN separator, DnR v2 data, training settings, and 128-token budget as DBS.

Table R13. SFC-CA and DBS on DnR v2 (SI-SDR in dB; MACs in GMAC/s).

ModelMusicSpeechSFXAvg.Params (M)MACs
SFC-CA BSRNN6.0614.924.478.484.948.1
DBS-BSRNN6.4815.015.739.074.748.0

DBS exceeds SFC-CA by 0.59 dB at nearly identical MACs (48.035 vs. 48.080 GMAC/s). SFC-CA aggregates spectral features through input-adaptive cross-attention; DBS explicitly predicts contiguous frequency boundaries and uses the same partition for pooling and full-resolution reconstruction.

Distinction from Conv-TasNet and Two-Step Sound Source Separation. Conv-TasNet (Luo and Mesgarani, 2019) jointly learns a waveform encoder/decoder and estimates masks in the encoded representation; its analysis filters are shared across inputs and it does not predict boundaries between contiguous STFT bands. Two-Step Sound Source Separation (Tzinis et al., 2020) learns an encoder/decoder using oracle masks from clean sources, then fixes the transforms and trains a separator to predict latent targets. These works establish the value of learned representations, but do not address mixture-dependent partitioning of contiguous STFT bands under a fixed token budget. DBS uses its predicted partition for both pooling and reconstruction while retaining the separator core. Tables R12 and R13 provide controlled evidence for this specific contribution.

Q1. Static BSRNN with the same refinement modules at matched MACs.

We built a static BSRNN with the same pooling, frequency refinement, and mask reconstruction modules as DBS. Increasing its recurrent hidden dimension from 128 to 137 yielded 48.308 GMAC/s, versus 48.035 GMAC/s for DBS (0.6% difference).

Table R14. Static BSRNN and DBS-BSRNN on DnR v2. Latency is mean ms over 40 runs; memory is peak GPU MB.

ModelMusicSpeechSFXAvg.MACsLatencyPeak memory
Compute-matched static BSRNN5.8714.804.148.2748.3135.5745.6
DBS-BSRNN6.4815.015.739.0748.047.1701.0

At matched MACs, DBS improves average SI-SDR by 0.80 dB, reduces mean latency by 65.2%, and reduces peak GPU memory by 6.0%. This supports the contribution of dynamic partitioning beyond the shared refinement and reconstruction modules.

Reviewer_8t7GOpenReview ↗

Response to authors

Thank you for the response and the new experiments.

  1. The MAC numbers I questioned were the ones in the submission. Thanks for the corrected counts.

  2. Thank you for the matched control. It shows that dynamic boundaries add 0.5 to 0.8 dB (Tables 10, R12 and R14). That is about half of the 1.4 dB headline, and the rest comes from the new interface. The abstract still claims a 29.0% relative gain, which is measured against the published BSRNN. Against the static model with the same interface the gain is about 8% so the actual improvment is more marginal and I do not know if that would also wash away with more heavy tuning.

  3. If the second decimal implies neither statistical significance nor perceptual discriminability, as you say, it should be dropped. All results are still single runs. Your two static controls, with LSTM hidden size 128 and 137, already differ by 0.6 dB on SFX. Please report several seeds and stats.

  4. Since you agree that the learnable filterbank is not a reliable baseline, please remove it from Table 3.

Since you corrected the cost numbers and ran the control I asked for, I am raising my rating to weak rejection. I still find the contribution incremental for ICLR, since it is a change to the band split interface that gives 0.5 to 0.8 dB from single runs.

Reviewer_YrHpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether band-split separators can benefit from choosing frequency boundaries for each input rather than using the same layout for every mixture. DBS keeps the number of band tokens fixed, predicts boundaries within predefined frequency blocks, and pools each resulting band into a token. An existing separator processes these tokens, after which the model expands the features and predicts full-resolution source masks. A gradient approximation allows the boundary predictor to be trained together with the separator. The experiments cover DnR v2, MUSDB18-HQ, and EchoSet with several backbones.

Critical strengths

A worthwhile problem and a clever solution. Adapting the band layout to the input is a worthwhile problem in source separation: the spectral details needed to distinguish sources vary across recordings, and a fixed partition may compress them too aggressively. DBS improves separation while keeping the token count fixed, supporting the practical value of this adaptive frequency representation. Compared with SFC-CA's soft aggregation, its explicit contiguous bands make the frequency range represented by each token easier to understand and inspect. The authors also address the training and reconstruction issues introduced by discrete partitions, making the method reusable across existing separation backbones. The experiments are solid. In Table 3, the uniform control reaches 8.00 dB and DBS reaches 9.07 dB under the same architecture and training settings, supporting the benefit over that fixed scheme. Boundary substitution in Appendix I and the component ablations also help explain the roles of boundary placement and reconstruction. Key implementation details, including parameters, MACs, and the reconstruction process, are reported.

Critical weaknesses

The authors already report parameter counts and MACs. In Table 1, MACs on DnR v2 increase by roughly 62% for BSRNN and 55% for TIGER. A breakdown of the computational cost of boundary prediction, separation, and reconstruction, together with actual inference times where feasible, would help readers understand the practical cost of the method. Appendix F reports the optimizer, initial learning rate, and batch size. Could the authors also provide explicit epoch or update counts for DnR v2 and EchoSet, along with the rules used to select the final models?

Questions for authors

Comments/Suggestions/Typos Both SFC and DBS compress frequency features into a fixed number of input-adaptive representations for music and cinematic separation. The paper cites SFC, but I could not find an explanation for leaving it out of the experiments. Could the authors explain why this comparison was not included? If reproduction, interface adaptation, or computational resources were an obstacle, please describe the specific difficulty. Given the overlap in tasks and module roles, I would like to see DBS and SFC-CA compared on at least one common backbone, using the same data and training settings and a comparable token budget, with parameters and MACs also reported. This would help readers understand what explicit dynamic band splitting adds over an existing adaptive spectral compression method. A brief discussion of the similarities and differences with H-Net in adjacent-feature scoring, probability-guided smoothing, and confidence-based STE would help readers understand DBS's specific contribution.

2 comments
AuthorsOpenReview ↗

Response to Reviewer YrHp (Part 1/2)

We thank the reviewer for the constructive feedback. We address computational cost and training reproducibility (W1–W2), followed by the SFC-CA comparison and relationship to H-Net (Q1–Q2).

W1. Corrected computational costs, stage-wise breakdown, and inference latency.

We identified errors in the original MAC-counting script: it overcounted LSTM operations and incorrectly counted convolution operations. We apologize and have recomputed the costs using one multiply–accumulate as one MAC. On DnR v2, corrected static/DBS costs are 41.847/48.035 GMAC/s for BSRNN and 26.993/35.047 GMAC/s for TIGER, corresponding to increases of 14.8% and 29.8%.

All four models use 128 active band tokens. MACs were profiled on 3-second clips and normalized by duration. End-to-end latency was measured on an RTX 4090 with FP32, TF32 disabled, batch size one, 12 warm-up runs, and 40 timed runs per model. Values below are mean latency.

Table R9. Stage-wise MACs (GMAC/s) / mean latency (ms) on DnR v2. Boundary prediction is included in the front-end totals (the two columns should not be added together).

ModelParams (M)Front end (incl. boundary prediction)Of which: boundary predictionSeparatorMask/waveform reconstructionTotal MACsTotal latency
Static BSRNN35.5220.023 / 11.60 / 0.039.108 / 17.72.716 / 17.141.84746.4
DBS-BSRNN4.7144.272 / 10.74.247 / 8.139.108 / 18.44.655 / 18.248.03547.4
Static TIGER1.4170.023 / 11.80 / 0.026.923 / 78.40.047 / 33.326.993123.6
DBS-TIGER1.0313.370 / 11.43.343 / 6.426.923 / 76.44.754 / 22.835.047110.7

The separator MACs are unchanged within each pair; additional computation comes mainly from boundary prediction and reconstruction. Mean latency changes by +1.0 ms (+2.2%) for DBS-BSRNN and −12.9 ms (−10.4%) for DBS-TIGER. Static TIGER performs many small per-band mask operations, whereas DBS uses batched projections over four blocks; this may reduce kernel-launch overhead and improve GPU utilization. DBS-BSRNN improves average SI-SDR by 1.38 dB while reducing parameters from 35.5M to 4.7M. DBS-TIGER improves average SI-SDR by 1.39 dB (8.74 to 10.13 dB).

W2. Training duration, checkpoint selection, and configurations.

Static and DBS BSRNN/TIGER models were retrained under matched settings on DnR v2 and EchoSet. Training was capped at 500 epochs. DnR v2 used 1,000 updates per epoch, patience 50, and the highest validation mean SNR for checkpoint selection. EchoSet used patience 20 and the lowest validation separation loss.

On MUSDB18-HQ, BSRNN and BS-RoFormer used up to 500 epochs with 1,000 mini-batches per epoch; MelRoFormer used 100,000 optimizer updates. Checkpoints were selected by the highest validation mean SDR across four stems.

For DnR v2 and EchoSet, we used Adam (initial LR 1e-3, no weight decay); LR was halved on a validation plateau (patience 10, relative threshold 1e-3, minimum LR 5e-5). The compression-loss weight was 0.01 on DnR v2, increased linearly from 0 to 0.02 between epochs 30–80 on EchoSet, and fixed at 0.01 on MUSDB18-HQ. Appendix F.3 gives further settings. Code and configurations: https://github.com/DBS-audio/DBS-Code.

AuthorsOpenReview ↗

Response to Reviewer YrHp (Part 2/2)

Q1. SFC-CA comparison under matched training and token budgets.

We added SFC-CA [1] to Section 4.3 and Table 3. Both methods use the same BSRNN separator, DnR v2 data and training settings, and 128-token budget.

Table R10. SFC-CA and DBS on DnR v2. SI-SDR is in dB; MACs are in GMAC/s.

ModelMusicSpeechSFXAvg.Params (M)MACs
SFC-CA BSRNN6.0614.924.478.484.948.1
DBS-BSRNN6.4815.015.739.074.748.0

DBS exceeds SFC-CA by 0.59 dB at nearly identical MACs (48.035 vs. 48.080 GMAC/s). SFC-CA uses input-adaptive cross-attention to aggregate spectral features with a band-derived positional bias. DBS instead predicts contiguous, non-overlapping frequency bands and uses the same partition for pooling and full-resolution reconstruction. This comparison supports DBS over the tested adaptive spectral-compression baseline at comparable cost.

Q2. Relationship to H-Net.

DBS learns mixture-dependent frequency boundaries under a fixed band-token budget. The resulting partition controls both band pooling and reconstruction on the original STFT grid (Sections 3.1–3.4).

Boundary scoring and selection. Both DBS and H-Net score boundaries using dissimilarities between adjacent features. DBS also uses magnitude-statistic transitions and selects a prescribed number of boundaries by Hard Top-K within each frequency block, preserving the token count. H-Net thresholds adjacent-position scores and therefore produces a variable number of chunks.

Aggregation and reconstruction. H-Net retains features at selected boundary positions and smooths chunk representations using boundary probabilities. DBS uses content-weighted pooling within each band, then smooths each separated token using the mean boundary probability in that band. The same partition expands tokens to the STFT grid for frequency refinement and complex-mask source reconstruction (Appendix B).

Assignment gradients. DBS and H-Net use equivalent confidence-weighting forms for selected boundary decisions. In addition, DBS differentiates through a local interpolation between features associated with placing or omitting a boundary. Appendix C, Eq. (31), therefore includes the feature-difference term Z^(1)−Z^(0) as well as the confidence contribution. This term captures the difference between the two local assignments in reconstructed feature space; Hard Top-K still determines the forward partition. Eqs. (27)–(31) describe the assignment and confidence paths separately.

References

[1] K. Saijo and Y. Bando, "Input-Adaptive Spectral Feature Compression by Sequence Modeling for Source Separation," IEEE Transactions on Audio, Speech and Language Processing, vol. 34, pp. 1286–1299, 2026. https://doi.org/10.1109/TASLPRO.2026.3663929

[2] S. Hwang, B. Wang, and A. Gu, "Dynamic Chunking for End-to-End Hierarchical Sequence Modeling," ICLR 2026. https://openreview.net/forum?id=ZbfLR9NbNF

Reviewer_eLtFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Dynamic Band Splitting (DBS), which predicts mixture-dependent frequency boundaries under a fixed band-token budget for audio source separation. Content-dependent pooling and full-resolution mask reconstruction connect the learned partitions to existing separators. Experiments evaluate four backbones across cinematic, music, and speech separation datasets.

Critical strengths
  1. Adapting frequency partitions while preserving the separator's token count is a useful approach to allocating spectral resolution.
  2. Improvements across three datasets and multiple backbones provide evidence that the proposed interface has broader applicability.
  3. The matched uniform-boundary control, inference-time boundary substitutions, and component ablations provide useful evidence about the contribution of boundary placement and reconstruction.
Critical weaknesses
  1. Most backbone comparisons replace pooling and mask reconstruction together with boundary selection. Although the matched uniform control is useful, stronger static partitions trained with the same DBS interface are missing. This limits attribution of the gains specifically to input-dependent boundaries, especially for MelRoFormer, whose entire band interface changes.
  2. EchoSet comparisons use published baseline results rather than matched retraining. The reported gains are relatively small, and the absence of results across training seeds or uncertainty estimates makes their reliability difficult to assess.
  3. Preserving the token count does not preserve computational cost. On DnR v2, MACs increase from 76.6 to 124.0 G/s for BSRNN and from 20.4 to 31.6 G/s for TIGER. Comparisons with static models under comparable computation would better establish the practical benefit.
  4. Important reproduction details remain unspecified, including training duration, learning-rate schedules, checkpoint selection, and several architectural dimensions. Appendix F does not fully substantiate the claimed documentation of training schedules.
Questions for authors
  1. Can the authors compare DBS against optimized or globally learned static partitions using the same pooling and reconstruction modules, and against the input-adaptive SFC method discussed in Related Work?
  2. Can the EchoSet baselines be retrained under matched settings, with multiple seeds or paired uncertainty estimates for the reported improvements?
  3. How does DBS compare with static alternatives at similar MACs, and what are its inference-time and memory costs?
  4. Can complete configuration files specify training duration, schedules, checkpoint selection, feature dimensions, and target compression ratios?
4 comments
AuthorsOpenReview ↗

Response to Reviewer eLtF (Part 1/3)

We thank the reviewer for the constructive feedback. We respond to W1–W4 and the corresponding Q1–Q4 below. The corrected MAC estimates and inference latency measurements are summarized in Table R1.

W1 / Q1. Boundary attribution under a shared interface and comparison with SFC-CA.

We trained static partitioning controls using the same separator backbone, pooling, frequency-refinement, and mask-reconstruction modules, and training settings as their DBS counterparts. The BSRNN control on DnR v2 used a static partition, while the MelRoFormer control on MUSDB18-HQ retained its original Mel-band partition. These controls allow us to assess the benefit of input-adaptive partitioning when the other interface components are shared.

Table R3. Static-interface control on DnR v2. SI-SDR is reported in dB.

ModelMusicSpeechSFXAvg.
Static BSRNN with DBS interface5.83314.5624.7268.374
DBS-BSRNN6.4815.015.739.07

Table R4. Static-interface control on MUSDB18-HQ. SDR is reported in dB.

ModelVocalsDrumsBassOtherAvg.
Static MelRoFormer with DBS interface9.178.746.505.717.53
DBS-MelRoFormer10.029.306.965.988.07

With the shared interface, DBS improved average SI-SDR from 8.374 to 9.07 dB on DnR v2, a gain of approximately 0.70 dB, and average SDR from 7.53 to 8.07 dB on MUSDB18-HQ, a gain of 0.54 dB. The gains therefore persist against these static partitions after matching the pooling and reconstruction modules, including in MelRoFormer, whose interface replacement was a particular concern.

We also evaluated SFC-CA and DBS with the same BSRNN separator, training data and settings, and 128-token budget on DnR v2. DBS achieved 9.07 dB average SI-SDR, compared with 8.48 dB for SFC-CA, a gain of 0.59 dB at nearly identical MACs (48.035 vs. 48.080 GMAC/s). This comparison shows the benefit of DBS over the tested input-adaptive spectral compression baseline at comparable computational cost.

Table R5. SFC-CA and DBS on DnR v2 with a shared BSRNN backbone. SI-SDR is reported in dB; MACs are reported in GMAC/s.

ModelMusicSpeechSFXAvg.ParametersMMMACsGMAC/sGMAC/s
SFC-CA BSRNN6.0614.924.478.484.92748.080
DBS-BSRNN6.4815.015.739.074.71448.035
AuthorsOpenReview ↗

Response to Reviewer eLtF (Part 2/3)

W2 / Q2. EchoSet baseline retraining, matched settings, and training-seed variability.

We retrained the static BSRNN and TIGER baselines on EchoSet using the official TIGER implementation, the corresponding released configurations, and the original dataset splits. We trained their DBS counterparts with the same training and evaluation settings, including the same random seed within each pair, while replacing the band-splitting and reconstruction interface.

Training these models on EchoSet typically requires more than 300 epochs, or approximately two to three weeks per run. Additional runs across training seeds have not been completed within the rebuttal period.

W3 / Q3. Separation quality, latency, and memory at approximately matched MACs.

We constructed a static BSRNN with a fixed partition and the same pooling, frequency-refinement, and mask-reconstruction modules as DBS. We increased its recurrent hidden dimension from 128 to 137 to match the computational budget. This control required 48.308 GMAC/s, compared with 48.035 GMAC/s for DBS-BSRNN, a difference of approximately 0.6%.

Table R6. Separation quality and inference costs at approximately matched MACs on DnR v2. SI-SDR is reported in dB, MACs in GMAC/s, latency in ms, and peak GPU memory in MB. Latency is the arithmetic mean over 40 timed inference runs per model on the RTX 4090.

ModelMusicSpeechSFXAvg.MACsLatencyPeak GPU memory
Compute-matched static BSRNN5.8714.804.148.2748.3135.5745.6
DBS-BSRNN6.4815.015.739.0748.047.1701.0

At this approximately matched MAC budget, DBS-BSRNN improved average SI-SDR by 0.80 dB (9.07 vs. 8.27 dB). Under the same inference measurement settings, it also reduced mean latency by 65.2% (135.5 to 47.1 ms) and peak GPU memory by 6.0% (745.6 to 701.0 MB). These results show that DBS achieved better separation quality with lower measured inference latency and memory use than the enlarged static control, even when both used the same interface modules and comparable arithmetic budgets.

AuthorsOpenReview ↗

Response to Reviewer eLtF (Part 3/3)

Appendix F now details training schedules, checkpoint selection, architecture, and compression settings. Code and configurations are available on the project page.

Training duration and checkpoint selection. On DnR v2, BSRNN and TIGER were trained for at most 500 epochs, with 1,000 optimizer updates per epoch. Early stopping used patience values of 50 epochs and checkpoints were selected by the highest validation mean SNR.

On MUSDB18-HQ, BSRNN and BS-RoFormer were trained for at most 500 epochs, with 1,000 training mini-batches per epoch. MelRoFormer used a fixed budget of 100,000 optimizer updates. For all three music backbones, checkpoints were selected by the highest validation mean SDR across the four stems.

On EchoSet, BSRNN and TIGER were trained for up to 500 epochs, with an early-stopping patience of 20 epochs. Checkpoints were selected by the lowest validation separation loss.

Optimization and loss schedules. Models were optimized on DnR v2 and EchoSet using Adam with an initial learning rate of 10−310^{-3} and no weight decay. The learning rate was halved following a validation plateau, with a patience of 10 epochs, a relative improvement threshold of 10−310^{-3}, and a minimum learning rate of 5×10−55\times10^{-5}.

The compression-ratio loss weight was fixed at 0.010.01 on DnR v2. On EchoSet, it increased linearly from 00 to 0.020.02 between epochs 3030 and 8080. On MUSDB18-HQ, the weight was fixed at 0.010.01 throughout training for DBS-BSRNN, DBS-MelRoFormer and DBS-BS-RoFormer.

**Model architecture.**We retained each baseline's separator backbone, replacing only the band-splitting and reconstruction interface. Table R7 lists the model configurations.

Table R7. Separator configurations by dataset and backbone. Feature width denotes the dimension of each band representation. Recurrent hidden sizes, attention-head counts, and attention-head dimensions are specified for the relevant architecture.

DatasetBackboneFeature widthSeparator configuration
DnR v2BSRNN1286 dual-path recurrent blocks; bidirectional LSTMs with hidden size 128
DnR v2TIGER1328 frequency-frame interaction blocks; hidden width 256; upsampling depth 5
EchoSetBSRNN1926 dual-path recurrent blocks; bidirectional LSTMs with hidden size 128
EchoSetTIGER1288 frequency-frame interaction blocks; hidden width 256; upsampling depth 5
MUSDB18-HQBSRNN12812 dual-path recurrent blocks; bidirectional LSTMs with hidden size 256
MUSDB18-HQBS-RoFormer3846 axial RoFormer blocks; 8 attention heads with head dimension 64; feed-forward expansion factor 4
MUSDB18-HQMelRoFormer2566 RoFormer blocks; one temporal and one frequency transformer layer per block; 8 attention heads with head dimension 64

Compression settings. Table R8 reports the frequency-block definitions, compression-ratio targets, and active token counts for each DBS integration. For BSRNN, TIGER, and BS-RoFormer, the compression-ratio targets regularized the soft boundary probabilities, while hard boundary selection fixed the number of active band tokens. MelRoFormer also used the per-block compression ratios as training targets.

Table R8. Frequency-block and compression settings. Block endpoints are zero-based STFT-bin indices, with the upper endpoint excluded. Compression targets and active token counts are listed in increasing frequency order. For MelRoFormer, the endpoints specify the initial coarse grid.

DatasetBackboneBlock endpointsCompression-ratio targetsActive token counts
DnR v2BSRNN(0,64,128,576,1025)(0,64,128,576,1025)(2,2,14,14.03125)(2,2,14,14.03125)(32,32,32,32)(32,32,32,32) (128 total)
DnR v2TIGER(0,40,80,352,1025)(0,40,80,352,1025)(1.25,1.25,8.5,21.03125)(1.25,1.25,8.5,21.03125)(32,32,32,32)(32,32,32,32) (128 total)
EchoSetBSRNN / TIGER(0,40,80,168,321)(0,40,80,168,321)(1.01,4,11,17)(1.01,4,11,17)(40,10,8,9)(40,10,8,9) (67 total)
MUSDB18-HQBSRNN / BS-RoFormer(0,64,128,576,1025)(0,64,128,576,1025)(2,2,14,14.03125)(2,2,14,14.03125)(32,32,32,32)(32,32,32,32) (128 total)
MUSDB18-HQMelRoFormer(0,64,128,576,1025)(0,64,128,576,1025) (initial grid)(2,2,14,14.03125)(2,2,14,14.03125)(32,32,32,32)(32,32,32,32) initially (128 total)

Thank the authors for their reply. It makes the contributions of this work more intuitive, prompting me to increase my score from 2 to 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: Corrected computational costs and measured inference latency

We thank the reviewers for raising concerns about computational cost. The original counting script overcounted LSTM operations and incorrectly counted convolution operations. We apologize for these errors and have recomputed the MACs for all seven static/DBS model pairs using a consistent convention. On DnR v2, the corrected MAC increases are 14.8% for BSRNN and 29.8% for TIGER. Table R1 reports the corrected counts together with parameter counts and measured inference latency.

Measurement protocol. MACs were profiled with 3-second audio clips and normalized by input duration to obtain GMAC/s, counting one multiply–accumulate as one MAC. The counts include the profiled modules and supplementary counts for attention matrix multiplications and MelRoFormer pooling/reconstruction contractions. FFTs, bias terms, normalization, and activations are excluded, as is the core arithmetic of custom FLA kernels where enabled.

We measured end-to-end inference latency on an NVIDIA GeForce RTX 4090 using FP32 with TF32 disabled and a batch size of one. We used the same 3-second validation audio clips within each dataset. Each measurement round comprised 12 warm-up runs followed by 40 timed inference runs per model. Latencies are arithmetic means.

Table R1. Computational cost, parameter counts, and mean end-to-end inference latency. Paired entries are Static / DBS.

DatasetBackboneMACs (GMAC/s)Parameters (M)Mean latency (ms)
DnR v2BSRNN41.8 / 48.035.5 / 4.746.4 / 47.4
DnR v2TIGER27.0 / 35.01.4 / 1.0123.6 / 110.7
MUSDB18-HQBSRNN230.3 / 235.571.7 / 21.7124.3 / 126.9
MUSDB18-HQBS-RoFormer318.0 / 367.0378.5 / 33.9172.0 / 246.7
MUSDB18-HQMelRoFormer204.6 / 285.798.7 / 19.2142.9 / 111.2
EchoSetBSRNN32.6 / 37.626.0 / 7.124.7 / 33.0
EchoSetTIGER15.8 / 19.90.8 / 1.1103.2 / 102.0

DBS improves the trade-off between separation quality and model size, with comparable or lower inference latency in most evaluated configurations. On DnR v2, DBS-BSRNN improves average SI-SDR by 1.38 dB with approximately 87% fewer parameters and a 2.2% increase in mean latency. Across the seven model pairs, parameters decrease in six pairs, while latency decreases or increases by only about 2% in five pairs. The latency trade-off depends on the backbone and dataset, with larger increases for BS-RoFormer on MUSDB18-HQ and BSRNN on EchoSet (Table R1).

Component-wise latency analysis. On DnR v2, the additional MACs mainly arise from boundary prediction and mask and waveform reconstruction. The separator backbone's MACs are unchanged within each pair. On MUSDB18-HQ, DBS-MelRoFormer reduced mean end-to-end latency by 22.2% (142.9 to 111.2 ms) while increasing MACs by 39.6%. Its mask and waveform reconstruction cost increased from 8.389 to 87.592 GMAC/s, while the corresponding latency fell from 41.7 to 16.4 ms. Front-end latency fell from 10.1 to 3.2 ms, and backbone latency was 91.2 ms for the static model and 91.6 ms for DBS. Batched mask reconstruction may improve GPU utilization and reduce kernel-launch overhead.

Table R2. Computational cost and mean inference latency by stage for MelRoFormer on MUSDB18-HQ. Each stage entry reports MACs (GMAC/s) / mean latency (ms). Front-end measurements include boundary prediction.

ModelParameters (M)Front endBoundary predictionSeparation backboneMask and waveform reconstructionTotal MACs (GMAC/s)Mean end-to-end latency (ms)
Static MelRoFormer98.70.207 / 10.10 / 0.0196.027 / 91.28.389 / 41.7204.624142.9
DBS-MelRoFormer19.22.128 / 3.20.003 / 1.2196.027 / 91.687.592 / 16.4285.747111.2

Higher separation quality at matched computation. We also increased the capacity of a static BSRNN using the same pooling, frequency-refinement, and mask-reconstruction modules as DBS. At approximately matched MACs, DBS-BSRNN achieves 9.07 dB average SI-SDR at 48.0 GMAC/s, compared with 8.27 dB at 48.3 GMAC/s for this static control on DnR v2 (Table R6). The 0.80 dB gain supports the benefit of input-adaptive partitioning even when the static model receives the same interface modules and a comparable computation budget.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.