- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces FLORE-AE, a high-compression audio autoencoder that replaces the standard spectral-adversarial decoder with a conditional waveform diffusion decoder trained via flow matching. The system operates at 2048× and 4096× temporal compression. Three main components are proposed: (1) carrier–amplitude factorization (Section 3.2), which normalizes the waveform into a local-RMS carrier for diffusion and predicts the amplitude envelope from the latent; (2) FLORE-DiT (Section 3.3), a hierarchical diffusion transformer that expands latents into position-specific conditions and injects them at every resolution; and (3) Endpoint-Aware MeanFlow (Section 3.4), which adapts the decoder for two-evaluation decoding by supervising near-endpoint intervals. The paper reports 14.11 dB SI-SDR, 0.041 rFAD, and 4.173 MOS across six test sets spanning speech, music, and sound, with 0.008 RTF. Ablations in Table 4 attribute gains to local factorization, temporal alignment, and multiresolution computation. Downstream generation and editing are also evaluated.
Critical strengths
1. The non-adversarial formulation is a genuine and practical contribution. Removing the discriminator from high-compression audio autoencoding is a meaningful direction, and the paper demonstrates that flow-matching-based decoding can match or exceed spectral-adversarial systems on reconstruction (Table 1) while reaching 0.008 RTF with 2 NFE. This is not a trivial result—prior generative autoencoders like Mousai require 20 NFE and achieve -27.71 dB SI-SDR. The paper makes a credible case that waveform diffusion is a viable decoder for compact audio representations.
2. The controlled analysis is unusually thorough for this subfield. Table 4 and the appendix controls (Tables 10–13) systematically separate signal parameterization, alignment, hierarchy, and few-step decoding. The alignment result (Table 10) is particularly valuable because it identifies a specific, actionable mechanism—explicit source-to-waveform temporal correspondence—that the broader latent-audio community can build on. The paper also honestly reports where its claims do not hold (e.g., rFAD sometimes worsens with alignment; FLORE-AE-C peaks earlier in single-source generation).
Critical weaknesses
1. The individual components are incremental, and the paper does not adequately position them against prior art. Carrier–amplitude factorization (Section 3.2) is essentially local RMS normalization with a learned gain predictor—similar in spirit to the clip-level normalization used in WavFlow (Zhou et al., 2026) and the envelope-based approaches in the neural coding literature. FLORE-DiT (Section 3.3) combines a hierarchical DiT (Crowson et al., 2024) with cross-attention conditioning, which resembles DiTAR and the conditioning mechanisms in Stable Audio Open. EA-MF (Section 3.4) is a modest extension of iMF (Geng et al., 2025b) with an auxiliary near-endpoint spectral loss. The paper cites these related works but does not provide a clear technical delta for each component. This makes it hard to assess whether the system-level gains arise from genuinely new mechanisms or from careful engineering of existing ones. A more explicit "what is new here vs. prior work" discussion for each component would help.
2. The main results lack uncertainty quantification, and some comparisons are too close to call. With single-seed training, the 0.24 dB SI-SDR gap between FLORE-AE-C and SAME-L at 4096× (Table 1) and the 0.027 rFAD gap could easily be within run-to-run noise. The paper does not report confidence intervals or standard deviations for the objective metrics (only MOS has them). For a paper whose central claim is that FLORE-AE "leads the evaluated baselines," this is a meaningful gap. At minimum, the authors should report results across 2–3 seeds for the main comparison, or provide a variance estimate from the MOS study.
Questions for authors
-
Clarification: In Table 4(b), the "Standard patchify DiT" row uses 0.309 inner-DiT TFLOPs, while "FLORE-DiT, full resolution" uses 0.274. The text says the full-resolution control is "nearly indistinguishable" from the hierarchy (13.42 vs. 13.45 dB). But the full-resolution control still uses sliding-window attention—what exactly is the difference between "FLORE-DiT, full resolution" and the deployed FLORE-DiT besides the hierarchy? Is the 0.274 vs. 0.098 TFLOP reduction entirely from moving inner blocks to coarser grids?
-
Clarification: The amplitude predictor (Eq. 11) predicts both a global gain and a local residual envelope. Have you tried removing the local residual and keeping only the global gain? Table 4(a) shows that "Global RMS + predicted gain" performs worse than local factorization, but that variant still uses a different (clip-level) normalization. I want to understand whether the local residual envelope is necessary or whether a simpler global-gain predictor on top of the local carrier would suffice.
-
Clarification: For FLORE-AE-C, the paper says it "does not use the 60,000-update adaptation" (Appendix B.5) and instead uses "exactly two model calls to its selected 300,000-update checkpoint under its recorded solver configuration." What is that solver configuration? Is it the four-evaluation Adams–Bashforth sampler from Appendix B.4, or something else? The main text says FLORE-AE-C uses 2 NFE, but the solver configuration is not specified in the main text.
-
Suggestion for future work: The paper evaluates generation only through dataset-specific designated generators. A more informative experiment would train a single generator architecture on multiple autoencoder interfaces (with matched latent dimensions or a projection layer) to isolate the autoencoder's contribution from generator capacity. I understand this is expensive, but it would substantially strengthen the generation claims.
4 comments
Response to Reviewer x5aV (2/2)
- The main results lack uncertainty quantification: results across seeds, confidence intervals for the objective metrics, or a variance estimate from the MOS study should be reported.
Thank you for this suggestion. We have added uncertainty quantification to App. G of the revised manuscript: the results of retraining with a second seed and of re-decoding with different noise seeds (Table 21), paired confidence intervals for the main SI-SDR comparisons (Table 17), and a variance estimate from the MOS study (Table 19); the variation across seeds is far smaller than the main differences, and every paired interval in App. G excludes zero. Retraining the controlled FLORE-DiT of Table 4 with a second seed changes its SI-SDR by 0.09 dB and its MEL error by 0.0003, and re-decoding with two further noise seeds changes FLORE-AE's macro SI-SDR by at most 0.0014 dB. By comparison, FLORE-AE's SI-SDR gains over SAME-L and ϵar-VAE are 2.04 and 2.05 dB (paired 95% intervals [2.01, 2.08] and [2.02, 2.09]), and FLORE-AE-C's 0.24 dB gain over SAME-L, which the reviewer singles out, has a paired interval of [+0.21, +0.27] dB. For the MOS study, every system is rated on the same 60 items by the same five listeners per item (App. B.6.4), so differences are computed per item: FLORE-AE's MOS gain over ϵar-VAE2 (+0.152) and FLORE-AE-C's gain over SAME-L (+0.165) have paired 95% intervals of [+0.09, +0.22] and [+0.08, +0.25].
- What differs between "FLORE-DiT, full resolution" and the deployed FLORE-DiT besides the hierarchy? Is the 0.274 vs. 0.098 TFLOP reduction entirely from coarser grids?
The two differ only in the hierarchy, so the reduction from 0.274 to 0.098 TFLOPs comes entirely from running the inner blocks on coarser grids. The full-resolution control "reallocates the inner blocks to the highest-resolution grid" (Table 11) and is otherwise the deployed FLORE-DiT, with the same aligned conditioning, the same inner blocks, and the same sliding-window attention; the 0.309 TFLOPs of the standard patchify DiT reflect global attention (App. D.4.2). With sliding-window attention, the cost of a block scales with the number of tokens it processes, and the hierarchy runs the same blocks on sequences two to eight times shorter (App. B.2), which accounts for the whole reduction at unchanged quality (13.45 vs. 13.42 dB).
- Is the local residual envelope necessary, or would a global-gain predictor on top of the local carrier suffice?
The local residual envelope is necessary: removing it from the deployed FLORE-AE, so that only the global gain acts on the locally normalized carrier, lowers SI-SDR from 14.11 to 4.37 dB and raises rFAD from 0.041 to 0.066. The carrier has approximately unit local RMS by construction (Eq. 4; App. B.1), so a single gain returns a waveform whose local energy is nearly the same everywhere; the zero-mean local residual (Eq. 11) is the path through which the latent restores time-varying amplitude. We have added this result to App. D.5 of the revised manuscript.
- What is the two-call solver configuration of FLORE-AE-C?
As Sec. 4.1 states for both models, "EA-MF fine-tunes only the waveform decoder for 60,000 updates and uses the grid (0, 0.5, 1)": FLORE-AE-C uses the same two-step solver as FLORE-AE. The statements in App. B.5 and B.6.2 that FLORE-AE-C does not use this adaptation are typos, which we have corrected in the revised manuscript.
- Suggestion: train a single generator architecture on multiple autoencoder interfaces to isolate the autoencoder's contribution.
Thank you for this suggestion. We would like to clarify that our AudioCaps generation experiments are designed along this idea and verify it with two different generator architectures: in each, one architecture is trained on multiple autoencoder interfaces with the same training data, seed, and update budget, and only the input and output projections follow the native latent width. With the first, the generator of the AudioCaps comparison in Table 2, FLORE-AE has the best FAD and CLAP (0.086 and 0.321). With the second, initialized from one Qwen3-1.7B backbone and trained on four frozen interfaces (App. D.7.2), FLORE-AE has the lowest FD-PaSST and the highest CLAP at the full 200k budget (182.17 and 0.386; Table 16). With both architectures, where only the autoencoder differs, FLORE-AE thus has the best FAD/FD and CLAP at the full training budget.
References
[1] Zhou et al., "WavFlow: Audio Generation in Waveform Space." arXiv:2605.18749, 2026.
[2] Crowson et al., "Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers." ICML, 2024.
[3] Jia et al., "DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation." ICML, 2025.
[4] Evans et al., "Stable Audio Open." arXiv:2407.14358, 2024.
[5] Geng et al., "Improved Mean Flows: On the Challenges of Fastforward Generative Models." arXiv:2512.02012, 2025.
Follow-up on our response
Dear Reviewer x5aV,
Thank you again for your positive and constructive review. We would like to briefly follow up on our response, in which we:
- gave the technical delta of each component relative to prior work (R1);
- added uncertainty quantification, including the paired interval for FLORE-AE-C vs. SAME-L that you pointed out (R2);
- clarified the hierarchy, the local residual envelope, and FLORE-AE-C's solver (R3–R5).
We would be grateful to know whether these address your concerns.
Best regards,
The Authors
Looking Forward to Your Feedback
Dear Reviewer x5aV,
Thank you again for your great efforts and valuable comments. To address your questions, we have clarified the technical delta of each component relative to prior work, added uncertainty quantification, including the paired interval for FLORE-AE-C vs. SAME-L that you pointed out (Tables 17, 19, 21), added the result on the local residual envelope (App. D.5), and clarified the hierarchy and the solver of FLORE-AE-C. As the author–reviewer discussion is about to close, we would be grateful if we could hear your feedback on our response, and sincerely hope that you can consider raising the score if you find our reply helpful. We would be happy to answer and discuss if you have further comments.
Thanks in advance,
Paper 1293 authors
Response to Reviewer x5aV (1/2)
We sincerely thank the reviewer for the positive feedback and for recognizing the non-adversarial formulation as "a genuine and practical contribution," our controlled analysis as "unusually thorough for this subfield," and the alignment result as "a specific, actionable mechanism … that the broader latent-audio community can build on." We hope our point-by-point response can fully resolve all your concerns.
We would like to clarify that the three components are the mechanisms by which FLORE-AE's framework makes generative waveform decoding source-faithful at high compression; each meets a requirement that prior normalization, conditioning, and few-step objectives leave open, rather than being an incremental variant of them. These are, as Sec. 4.3 states, the "three requirements for a practical high-compression waveform diffusion autoencoder: the denoising target must be well conditioned across local amplitude scales, compressed latents must retain direct time-indexed control over dense synthesis, and few-step decoding must remain effective on generated as well as clean latents," met in turn by carrier–amplitude factorization, FLORE-DiT, and EA-MF. They, not engineering of existing techniques, produce the result the reviewer recognizes as "not a trivial result"; below, we give the delta of each over prior work, whose closest variants we test as baselines (Tables 4, 10).
(1) Relative to the clip-level normalization of prior work [1], carrier–amplitude factorization is a local normalization for waveform diffusion: it removes the amplitude variation within a clip that clip-level scaling leaves in the denoising target, and recovers it from the latent instead of discarding it. As Sec. 3.2 states, "a shared noise schedule produces different local SNRs in quiet and loud passages, even after clip-level scaling," while local normalization alone discards the dynamics that reconstruction needs. FLORE-AE therefore predicts the amplitude envelope, a gain plus a time-varying local part, from the same latent (Eqs. 4, 11), and, unlike in envelope-based coding, "no reference amplitude envelope is supplied" (Sec. 3.5). It is thus more than a learned gain predictor: without the local part, the deployed model falls from 14.11 to 4.37 dB SI-SDR (R4). Against clip-level normalization with a predicted gain, it raises SI-SDR from 11.95 to 13.45 dB and lowers rFAD from 0.149 to 0.110 (Table 4(a)).
(2) Relative to the hierarchical DiTs and conditioning mechanisms of prior work [2–4], FLORE-DiT changes how the condition reaches the waveform: the compact latent is expanded into a distinct condition for every synthesis position and reinjected, time-aligned, at every resolution. Where prior conditioning reads the condition by content or attaches it patch by patch, FLORE-DiT expands each latent interval into position-specific conditions added to the fine-grid waveform tokens (Eq. 5), which "gives synthesis positions within one latent interval distinct conditions" (Sec. 3.3.1). Prior hierarchical DiTs move computation to coarser grids but, as App. A states, "do not specify how compact, time-indexed source information should remain available across waveform resolutions"; FLORE-DiT reinjects the latent at every stage, aligned to that stage's grid (Eq. 5), so that each resolution reads the time-indexed latent directly. A hierarchical DiT with latent cross-attention is itself the baseline of Table 10: these aligned paths raise it from 12.34 to 13.23 dB SI-SDR, and a single-resolution DiT from 10.07 to 13.33 dB, whereas a 3.6× larger decoder adds only 0.24 dB (App. D.4.4). Source correspondence, not decoder capacity, thus drives fidelity; this is the "explicit source-to-waveform temporal correspondence" the reviewer highlights, and with it the hierarchy matches full-resolution quality at 64% lower inner-DiT cost (Table 4(b)).
(3) Beyond the MeanFlow objectives of prior work [5], which supervise average velocities, EA-MF supervises the amplitude-restored waveform that the two-step decoder actually returns, so that decoding stays faithful and perceptually strong on generated latents. As Sec. 3.4 notes, the endpoint of a MeanFlow interval "need not equal the clean-data endpoint"; EA-MF therefore also compares its near-endpoint estimates with the source. It is more than an auxiliary loss: decoder objectives should be "assessed on generated latents … rather than inferred from clean-latent reconstruction alone" (Sec. 4.3). On identical generated latents at two evaluations, EA-MF lowers FAD from 0.111 to 0.086 against an iMF control that already includes terminal reconstruction (Table 4(c)), at 0.008 RTF, comparable to single-pass autoencoders (Table 1).
We have added a concise summary of these distinctions to App. A of the revised manuscript.