acceptodds
Under review as a conference paper at ICLR 2027

SAuBER: Neural Audio Coding via a Learnable MDCT Tokenizer and Energy-Based Refinement

Abstract

Generative neural audio codecs (NACs) achieve high perceptual quality at low bitrates, but typically rely on Generative Adversarial Networks (GANs) and spatial downsampling. These choices introduce training instability and limit autoencoder reconstruction fidelity. While hybrid approaches avoid spatial downsampling by pairing a fixed signal-processing transform with a learned compressor, designing them independently creates a domain mismatch. To resolve this, we propose SAuBER, a unified NAC framework that co-designs a learnable transform with a neural compressor while replacing adversarial generation with energy-based refinement. Specifically, our analysis/synthesis module operates as a learnable tokenizer built directly over patchified Modified Discrete Cosine Transform (MDCT) coefficients, producing near-transparent tokens without spatial downsampling. Within this token space, an information-bottleneck autoencoder extracts discrete latent representations, and an energy-based refinement head, trained via vector field matching, fits the gradient of an explicit energy function for stable, streamable generation. Autoencoder and refinement head are trained together to jointly optimize feature extraction, quantization, and refinement. At bitrates of 20–40 kbps, SAuBER achieves competitive performance against both classical signal-processing codecs and state-of-the-art GAN-based NACs in terms of perceptual quality and reconstruction fidelity.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
neural audio coding, energy-based model, audio compression

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4tJwOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
1
Soundness
3
Read review
Paper summary

This paper proposed SAuBER, an audio coding method from learnable MDCT tokenizer and energy-based refinement. The idea is a good one. The experiments are carried on the MUSDB18 dataset to show the effectiveness of the method. I like this paper. But the writing, figures, equations are very rough for a formal publication. I lean to accept this paper, on the condition that the authors must completely rewrite this paper.

Critical strengths
  • The idea of using MDCT for building codec is a nice one.

  • The idea of using energy-based method is also a good one.

Critical weaknesses
  • The writing is very rough. The figures are not well formatted and do not look professional.

  • Many equations are rough. The equations are also not formmated and the readability is low. I can hardly understand many equations.

  • Many important model architectures and contents are in the appendix and should be moved to the main text. Many content in the main text should be removed or compressed.

  • There is a lack of total pipeline figure of the system.

  • Details of how MDCT convert audio into coefficients are lacked.

  • Equations (3) and (4) are hard to understand.

  • For robustness, what are the ablation result for with an without injecting Gaussian noise?

  • Many important contents, such as Appendix A.1, A.2 should be moved to the main texts. A reader may skip reading the appendix but only read the main text.

  • The authors mentioned energy-based model. What is the advantage of use energy-based model? How it is improved over other non energy-based models?

Questions for authors
  • See critical weaknesses.
2 comments

We thank the reviewer for the positive assessment and the clear, actionable list of suggestions. We have taken the request for a thorough restructuring and rewrite seriously, carefully revising the manuscript to address every point raised.

Writing, figures, and equations (W1, W2). All diagrams have been newly redrawn, and all plots now share a unified visual style. To improve clarity and readability, we have reformulated the equations for the tokenizer (formerly Eqs. 3 and 4), the time-frequency loss, the quantizer, the rate losses, and the refinement objectives (along with Algorithm 1). Additionally, secondary analyses have been relocated to the appendix with dedicated, self-contained explanations.

Pipeline figure (W4). Several figures have been added or revised to clearly illustrate the overall coding pipeline, training procedure, and module architecture. Specifically, Fig. 1(a) depicts the end-to-end inference pipeline, while Fig. 1(b) illustrates the frame-wise refinement process. Fig. 2 outlines the three training stages, explicitly indicating which modules are trained and frozen, all associated loss terms, and the stop-gradient operations. Finally, Fig. 3 details the underlying model architectures.

Architecture in the main text (W3, W8). The encoder, decoder, quantizer, density model, and refinement head are now described directly in the main text (§3.2–3.3), replacing the architectural details previously placed in the appendix.

MDCT details (W5). Section 3.1 now explicitly specifies the MDCT configuration and patchification process. Specifically, the MDCT employs a 512-sample sine window with a hop size of 256 (Nbins=256N_{\mathrm{bins}}=256), and each patch spans 8 frequency bins ×\times 4 frames per subband (D=32D = 32, corresponding to 32 tokens per 1024 samples).

Ablation without noise injection (W7). To isolate the effect of the noise injection, we trained a variant of the subband tokenizer without the noisy channel (pnc=0p_{\mathrm{nc}} = 0), while keeping all other hyperparameters identical, and subsequently trained the 24 kbps autoencoder and refinement head following the exact same recipe (§4.4, Fig. A.11).

Tokenizer: 2f-model MMS at token noise σ\sigma0.10.10.30.30.50.5
With noise injection (ours)72.1\mathbf{72.1}41.0\mathbf{41.0}27.4\mathbf{27.4}
Without noise injection18.318.35.15.10.10.1
MDCT only (no learned layers)48.648.627.727.719.419.4
2424 kbps codecLSD ↓\downarrowNMR ↓\downarrow2f-model MMS ↑\uparrow
BAE, with / without noise injection15.9815.98 / 16.9916.99−2.59-2.59 / +4.24+4.2439.539.5 / 22.122.1
SAuBER, with / without noise injection (η=0.08\eta = 0.08, default)10.37\mathbf{10.37} / 23.2123.21−2.47-2.47 / +15.59+15.5944.2\mathbf{44.2} / 20.720.7
SAuBER, with / without noise injection (η=0.2\eta = 0.2)10.6210.62 / 15.8615.86−2.74\mathbf{-2.74} / +6.04+6.0443.743.7 / 27.127.1

While both tokenizers remain near-transparent on clean input, omitting the noisy channel causes the learned layers to amplify perturbations in the token space. Because the autoencoder's reconstruction error acts precisely as such a perturbation, the BAE degrades substantially, and the refinement head compounds this degradation even further. Noise injection is therefore essential for regularizing the token-space and enabling stable iterative refinement.

What the energy-based formulation brings (W9). Please refer to our response to Reviewer 875q (§4.3, Figs. 6–7). Under identical architectures, inputs, and training conditions, the energy-gradient target consistently outperforms standard flow matching across all step counts (NstepN_{\mathrm{step}}) on the uniform grid, whereas a one-step regression head yields negligible improvement over the decoder output. Furthermore, unlike the flow matching objective, our energy-based formulation provides a tunable step size η\eta to control the quality trade-off across different metrics and allows explicit differentiable quality metrics, such as the proposed time-frequency loss, to be directly incorporated into the generative refinement process.

Clarification. Only the training split of MUSDB18-HQ is included in our training data, ensuring strict separation from evaluation. Our evaluation comprises the 43 USAC test items along with three newly added test sets, including the held-out MUSDB18-HQ test split (App. A.6.1).

We hope the revised manuscript satisfies the reviewer's concerns, and we would be glad to refine any remaining passages that could be further improved.

I have read through the comments from the authors. I would keep the score unchanged.

Reviewer_875qOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SAuBER proposes a neural audio codec that combines a learnable MDCT-based tokenizer with an energy-based refinement head. In relation to existing work, the methodological contributions are largely compositional. It proposes using a learnable transform initialized from MDCTs and an energy-based refinement head related to existing flow-matching techniques. The autoencoder and refinement are proposed to be trained jointly. The resulting system is effective, even though the listening experiments do not show an advantage for speech signals in relation to DAC. The comparison to the highly related FlowDec should have been made much more explicit.

Critical strengths
  • The system works and achieves competitive perceptual quality at medium bitrates.
  • The system supports causal (streaming) inference.
  • Unlike FlowDec (frozen codec), SAuBER jointly trains the refinement and compression, which is a sensible design choice (though not theoretically novel).
Critical weaknesses
  • A comparison to the most similar baseline (FlowDec) should have been conducted with much more rigor, both in terms of describing the conceptual relation and also in terms of experimental results and the listening experiment.
  • The relation to prior streaming flow matching work (e.g., streamFM [DOI: 10.1109/TASLPRO.2026.3696215]) should have been discussed to properly contextualize the novelty of the causal design.
  • The motivation for using EBM-based refinement instead of standard Flow Matching is unclear. In particular, the proposed token-space gradient target is closely related to a scaled Flow Matching vector field, making the practical advantage of the EBM formulation difficult to understand. An isolated comparison against standard conditional Flow Matching, and, also a predictive regression model, using the same architecture, conditioning, and training setup, is needed to justify this design choice.
  • The paper frames joint training of the autoencoder and refinement head as a key distinction from FlowDec, thus trading off modularity versus performance. This trade-off appears rather standard, but not like a methodological breakthrough.
  • The 30-step refinement process is computationally heavy compared to alternatives (e.g., FlowDec).
  • The model underperforms on speech compared to DAC/FlowDec, limiting its practical utility despite the architectural complexity.
  • For a fair and meaningful comparison it is essential to report all metrics on the same bitrate range as SAuBER. However, while the authors report that they retrained DAC, they did not train and test for the same bitrates as used for SAuBER.
  • In Figure 2. ( c ) where the plateauing of DAC is not as intense there seems to be a significant performance margin between DAC and SAuBER, i,e, DAC performs better.
  • The core components (causal convolutions, learnable transforms, flow/EBM refinement) are all existing and their combination is only moderately novel.
Questions for authors
  • The authors state: "Furthermore, comparison with the FlowDec variant confirms that jointly training the autoencoder and refinement head enables plausible high-frequency generation without degrading low frequency components."

Where in the document is this comparison given? Or is this just an informal observation by the authors? The paper would be stronger with a formal comparison in listening experiment, metrics and spectrograms. Informally reporting such a benefit is way weaker.

  • Eq. (7) uses hard rounding, but the authors do not explain how gradients are propagated through this operation during training.
2 comments

We thank the reviewer for the thorough review. The requested controlled experiments led us to refine several claims, and we report all findings transparently.

FlowDec relation (W1, Q1). We agree that the relation to FlowDec should be explicit, and we removed the quoted sentence, which rested on qualitative observation. A FlowDec-style head differs from SAuBER in four respects: a frozen autoencoder, no side information, no explicit anchor input h^\hat{\mathbf{h}}, and a conditional flow-matching (CFM) target. We therefore ablated one factor at a time with all other settings fixed (§4.3, Fig. 7; other step counts in Figs. A.5–A.6). Paired differences at 30 steps (mean ± 95% CI; positive favors SAuBER's choice; MMS = 2f-model MMS):

Factor (SAuBER's choice vs. alternative)Δ MMSΔ LSD (dB)Δ NMR (dB)
1. Joint vs. frozen autoencoder (both w/o side info.)+1.08 ± 0.15−0.16 ± 0.16+0.06 ± 0.12
2. SAuBER vs. frozen autoencoder w/o side info.+2.02 ± 0.28+0.14 ± 0.11−0.02 ± 0.13
3. With vs. without anchor h^\hat{\mathbf{h}}−0.99 ± 0.31+0.25 ± 0.13+0.06 ± 0.15
4. SAuBER vs. FlowDec-style−1.23 ± 0.71−0.31 ± 0.17+0.47 ± 0.17
5. SAuBER +vTF+\mathbf{v}_{\mathrm{TF}} vs. FlowDec-style−2.31 ± 0.81+0.22 ± 0.20+0.58 ± 0.21
  • Rows 1–2: joint training adds about one MMS point, and SAuBER gains about two over a frozen head without side information from 10 steps, under both schedules (Figs. A.5–A.6, bottom rows).
  • Row 3: the anchor trades about one MMS point for lower LSD at 30 steps but improves all three metrics with one step, so we present it as an aid for few steps, not a general gain.
  • Rows 4–5: the FlowDec-style head is better on MMS and LSD but worse on NMR, and with the time–frequency energy SAuBER also leads on LSD at a higher MMS cost. We now describe the relation as a multi-metric trade-off (§4.3).

EBM vs. flow matching vs. regression (W3). The reviewer is right that our target is a rescaled flow-matching velocity. With zγ=γ(h^+σϵ)+(1−γ)h\mathbf{z}_{\gamma} = \gamma(\hat{\mathbf{h}} + \sigma \boldsymbol{\epsilon}) + (1-\gamma)\mathbf{h} and the CFM velocity u=h^+σϵ−h\mathbf{u} = \hat{\mathbf{h}} + \sigma \boldsymbol{\epsilon} - \mathbf{h}, our target is vtok=∇zγ12∣zγ−h∣2=zγ−h=γu\mathbf{v}_{\mathrm{tok}} = \nabla_{\mathbf{z}_{\gamma}} \frac{1}{2}|\mathbf{z}_{\gamma} - \mathbf{h}|^2 = \mathbf{z}_{\gamma} - \mathbf{h} = \gamma\mathbf{u}, i.e., a γ2\gamma^2-weighted CFM loss. With everything else fixed and the same uniform γ\gamma grid (§4.3, Fig. 6), it improves MMS over CFM by 0.5–0.8 points at every step count, and by about one point at 30–100 steps under our default geometric schedule. A one-step regression head barely improves on the decoder output (MMS 40.6 vs. 39.5), so the gain stems from iterative generative refinement.

Stream.FM (W2). We now cite it in §1 and compare in App. A.4 (Table 2). Like MelFlow and the Diffusion Buffer, Stream.FM is frame-causal flow matching in which each frame conditions on past inputs. SAuBER's head is autoregressive: it refines each frame conditioned on its own refined frames (noisy-context training, as in xAR) and on the transmitted bits, jointly with a rate-constrained autoencoder. Autoregressive flow heads exist for speech synthesis (SALAD, FELLE, DiTAR) but, to our knowledge, not for audio coding.

Novelty (W4, W9). Our contribution is the system: a causal, GAN-free codec in a learned MDCT domain whose quality scales with bitrate, with an autoregressive, bit-conditioned refinement head that is, to our knowledge, new in audio coding. Moreover, since the head learns the gradient of an explicit energy, a differentiable distortion measure can enter the target as a further energy term without a discriminator. The time–frequency energy is a first instance, lowering LSD by 0.5 dB at a cost of about one MMS point (Fig. 7), and perceptual criteria are a natural next step.

Compute, DAC, speech (W5, W6, W7, W8). SAuBER runs in real time at 30 steps with 29 ms latency, and 10 steps keep most of the gain at less than half the cost (General response). As the reviewer observed, the retrained DAC leads at 24 kbps, ties at 32 kbps, and trails SAuBER on LSD and NMR at 40–48 kbps while leading on MMS (Fig. 4). On the 48 EARS speech items DAC is better on every metric, which we acknowledge as a limitation (App. A.6.1).

Eq. 7 rounding (Q2). We use a straight-through estimator: the forward pass rounds and the backward pass treats rounding as the identity (§3.2).

The authors did a good job of addressing the points that could be improved. However, the major issues with the paper remain limited methodological novelty and limited performance gains. A weak reject appropriately reflects this.

Reviewer_9SG6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work tackles neural audio codecs with that are near-transparent at high bitrate, which is in contrast to existing neural audio codecs with no such guarantee. To do this, the authors build upon MDCT in a 3-stage framework - MDCT subband tokenization, then a bottlenecked autoencoding + quantization, and finally refinement of the output.

Along the way, they add a number of regularizations and training tricks (SIGREG etc). One primary contribution is replacing GAN training with an energy-based, flow-matching-style refinement.

Critical strengths
  • Improves with bitrate, whereas existing models plateau.
  • Careful study of high-frequency behavior, and high-frequency attenuation / blurring.
  • Proposes an interesting replacement of the usual GAN-based approach common in the vocoder/codec literature.
Critical weaknesses
  • Only 43 test samples are used for evaluation, and only 9 listeners in the MUSHRA.
  • MUSHRA is below HE-AAC, and some high-bitrate codecs such as Opus are missing.
  • Claims streaming, but no latency is reported.
  • One model per bitrate.
  • Requires 3-stage training (any end-to-end approach possible?)
  • Lacks a controlled comparison of using a GAN based refiner vs an energy-based refiner.
  • DAC and Sauber have almost overlapped MUSHRA. In the appendix, SAuBER out-performs AC on mixtures and music, but under-performs on speech. I'm not sure from these results that the transparency claim is supported completely.
Questions for authors
  • What is the decoding real-time factor and latency?
  • The method under-performs HE-AAC and is somewhat tied with DAC, countering the claims in the introduction.
  • What DAC configuration is used? Quantizer dropout? Codebooks/training budget, etc.
  • Why did the MUSHRA participants fall from 20 to 9 after screening?
  • Can you train DAC at beyond 24kpbs without quantizer dropout to verify it plateuas? I couldn't find very-high-bitrate DAC anywhere.
1 comment

We thank the reviewer for the thorough review. Conducting the requested comparison led us to refine our claims accordingly, and we report all resulting findings transparently in the revised manuscript.

Real-time factor and latency (W3, Q1). The algorithmic latency comprises one 10241024-sample frame plus the 256256-sample MDCT overlap, totaling 2929 ms (App. A.1). On a single RTX A5000 GPU, the complete codec operates in real time, achieving an RTF of 0.860.86 at 3030 refinement steps (Table 1).

GAN vs. energy-based refiner, controlled (W6). The GAN refiner shares the same autoencoder, head architecture, inputs, and v-prediction target, but replaces our energy loss with adversarial and feature-matching losses based on DAC's discriminator architectures (loss weights of 11 and 22, respectively; App. A.6.3, Table 4).

2424 kbpsLSD ↓\downarrowNMR ↓\downarrow2f-model MMS ↑\uparrowODG ↑\uparrowViSQOL ↑\uparrow
Decoder output15.9815.98−2.59-2.5939.539.5−3.37-3.373.333.33
GAN-trained head12.9112.91−1.43-1.4337.937.9−3.02-3.023.103.10
SAuBER, 3030 steps, η=0.08\eta = 0.08 (default)10.3710.37−2.47-2.4744.2\mathbf{44.2}−3.01-3.013.87\mathbf{3.87}
SAuBER, 3030 steps, η=0.2\eta = 0.210.62\mathbf{10.62}−2.74\mathbf{-2.74}43.743.7−2.95\mathbf{-2.95}3.833.83

Although trained without collapse, the GAN refiner lowers LSD but falls below the raw decoder output on NMR, MMS, and ViSQOL. Paired item-by-item across evaluation items, SAuBER achieves significantly better scores on MMS, LSD, NMR, and ViSQOL, with ODG remaining tied.

DAC configuration (Q3, Q5). The listening-test DAC (30.430.4 M parameters, 10241024 codebook size, 0.50.5 quantizer dropout, up to 2727 codebooks) is trained on our data (App. A.5). The retrained DAC used for objective comparisons shares this architecture with 5656 codebooks (31.031.0 M parameters, maximum operating bitrate of 4848 kbps), is trained for 400400k iterations, roughly matching SAuBER's total update count, using DAC's default optimizer and losses, and serves all bitrates from a single model (2828–5656 codebooks, App. A.5). Without dropout, dedicated 3232 and 4040 kbps models diverged at 135135k and 190190k iterations, yielding final stable checkpoints that underperformed the dropout model. Although we have withdrawn our plateau claim, DAC still exhibits significantly less quality scaling with bitrate than SAuBER.

MUSHRA: 20→920 \rightarrow 9 listeners (W1, W7, Q4). Listeners were excluded if they rated the hidden reference below 8080, or the anchor above 3030 on more than two of the nine trials (App. A.6.2). Under the screening thresholds of ITU-R BS.1534-3, excluding listeners who rated the reference below 9090 and anchor above 9090 over 15%15\% of samples, the SAuBER–DAC performance remains within approximately ±1\pm 1 point (computed on paired listener-level differences at a 90%90\% CI):

Screening rulennSAuBER −- DACSAuBER −- HE-AAC v1
none2020−1.2±3.5-1.2 \pm 3.5−4.3±4.0-4.3 \pm 4.0
paper rule99+0.7±7.2+0.7 \pm 7.2−7.2±5.1-7.2 \pm 5.1
ITU-R BS.1534-366−0.2±6.0-0.2 \pm 6.0−6.6±8.2-6.6 \pm 8.2

SAuBER matches DAC under every screening rule while underperforming HE-AAC v1 by 44–77 points.

More test items (W1). We have added evaluations on FSD50K (100100 clips, one per sound class, excluding music and speech), the MUSDB18-HQ test split (5050 items), and EARS speech (4848 items), across 2424–4848 kbps (App. A.6.1, Fig. A.3). Overall method rankings remain unchanged, with Opus and HE-AAC v1 leading on NMR and MMS. On music data, SAuBER achieves lower LSD than DAC at every bitrate and lower NMR at 4040–4848 kbps. On speech, DAC outperforms SAuBER across all metrics, a limitation we explicitly state. A listening test including larger test items and Opus is planned for the camera-ready version.

Claims (W2, Q2, W7). The abstract, introduction, and conclusion now explicitly state that SAuBER trails HE-AAC v1 and Opus on perceptual metrics, and that it falls below DAC on speech. Conversely, the abstract and §4.2 clarify that SAuBER is perceptually comparable to DAC in our listening tests, while references to near-transparency are strictly restricted to the subband tokenizer itself (§4.4).

One model per bitrate (W4). We acknowledge this as a limitation: because the bitrate is determined solely by rtgtr_{\mathrm{tgt}} in the rate loss, incorporating rate conditioning represents a direct and straightforward extension.

Three-stage training (W5). Stage 2 keeps the tokenizer fixed and near-transparent because joint training may otherwise cause the token space to co-adapt or collapse. Stage 3 already trains all modules jointly while keeping only the tokenizer frozen. We leave fully end-to-end training for the future work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Authors and Overview of Manuscript Revisions

We thank all reviewers for their constructive reviews. In response, we have uploaded a rewritten manuscript (with new or modified text highlighted in blue and relocated text in violet) and conducted the requested controlled experiments.

Rewrite. We have added three new figures (Figs. 1–3) illustrating the coding pipeline and the refinement process, the three training stages with all associated loss terms, and the detailed module architectures. Furthermore, Section 3 now incorporates comprehensive details on MDCT patchification, the time-frequency loss, the quantizer with its straight-through estimator, and γ\gamma-conditioning of the head (absorbing the former appendix sections on architecture).

New experiments (24 kbps, USAC items, unless stated). (a) Controlled refinement ablation: An ablation study isolating individual design factors of the refinement process—joint autoencoder training, head architecture, conditioning inputs, step sizes (η\eta), and training targets—evaluated across varying step counts (NstepN_{\mathrm{step}}) (§4.3, Figs. 6–7, A.5–A.6). (b) Alternative head baseline: Comparisons against a one-step regression head and a GAN-based refinement head (§4.3, Figs. 6, App. A.6.3). (c) Matched-condition baselines: Evaluation against DAC retrained on our dataset at matched bitrates (24–48 kbps, 400k iterations, about SAuBER's number of codec updates) and Opus 1.6 (Fig. 4). (d) Efficiency profiling: End-to-end latency, real-time factor, and computational complexity measurements for the full codec (App. A.1, Table 1). (e) Noise-free tokenization ablation: A tokenizer trained without noise injection, alongside the full codec trained on top of it (§4.4, Fig. A.11). (f) Expanded evaluation sets: Additional evaluations across three test sets comprising 198 items at 24–48 kbps (App. A.6.1).

Matched-rate comparison (USAC; 24 / 32 / 40 / 48 kbps; The DAC baseline is a single 48 kbps model evaluated across bitrates using 28 / 38 / 47 / 56 codebooks):

CodecLSD ↓\downarrowNMR ↓\downarrow2f-model MMS ↑\uparrow
SAuBER10.3710.37 / 9.689.68 / 9.029.02 / 8.528.52−2.47-2.47 / −3.60-3.60 / −4.39-4.39 / −5.20-5.2044.244.2 / 47.147.1 / 50.050.0 / 53.653.6
DAC, retrained9.689.68 / 9.579.57 / 9.539.53 / 9.519.51−3.47-3.47 / −3.69-3.69 / −3.76-3.76 / −3.78-3.7863.963.9 / 67.867.8 / 69.169.1 / 69.869.8
Opus 1.6 (VBR)10.7710.77 / 10.4410.44 / 10.0710.07 / 9.659.65−4.96-4.96 / −6.39-6.39 / −7.99-7.99 / −9.40-9.4065.365.3 / 72.272.2 / 78.878.8 / 85.485.4
HE-AAC v118.1218.12 / 15.6715.67 / 13.2313.23 / 8.468.46−3.57-3.57 / −4.51-4.51 / −5.38-5.38 / −6.55-6.5562.462.4 / 70.070.0 / 74.874.8 / 79.879.8

Real time. All modules are causal with zero look-ahead, yielding an algorithmic latency of 29 ms at 44.1 kHz sampling rate. On a single RTX A5000 GPU (batch 1), the complete SAuBER pipeline, including both the causal autoencoder and the autoregressive refinement head, operates in real time, achieving an RTF of 0.86 at the default 30 steps and 0.35 at 10 steps, against 0.007 for DAC (396 vs. 39 GMAC/s, mostly from the refinement head).

Corrected claims. To ensure our claims strictly reflect our empirical findings, we have removed overstatements such as "outperforms classical codecs and state-of-the-art GAN codecs" and "existing models plateau". Specifically, we now explicitly state that SAuBER trails HE-AAC v1 and Opus on perceptual metrics, performs on par with DAC in subjective listening tests, and lags behind retrained DAC at 24 kbps on objective metrics. Instead, our core contribution is a GAN-free, causal neural codec operating in a learned MDCT domain whose quality scales steadily with bitrate, mirroring the predictable scaling behavior of classical codecs. Furthermore, our controlled ablations show that joint training, side-information conditioning, and the energy-gradient target each contribute measurably to the refinement process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.