acceptodds
Under review as a conference paper at ICLR 2027

FLORE-AE: Rethinking Audio Autoencoding with Waveform Diffusion Decoders

Abstract

Audio autoencoders for latent generation aim to combine compact representations with faithful and perceptually realistic waveform reconstruction. Existing spectral–adversarial decoders use learned discriminators, introducing minimax optimization. In this work, we instead formulate high-compression audio autoencoding as latent-conditioned waveform diffusion, where compact latents encode source structure and the diffusion decoder models remaining acoustic detail. FLORE-AE combines carrier–amplitude factorization for locally normalized synthesis, FLORE-DiT for time-aligned latent guidance and multiresolution context, and Endpoint-Aware MeanFlow for two-evaluation decoding of encoded and generated latents. Matched controls show that local factorization improves all reconstruction criteria, alignment drives paired fidelity, multiresolution computation matches full-resolution quality at lower inner-DiT cost, and endpoint-aware adaptation improves generated-latent decoding. Across speech, music, and sound, FLORE-AE improves reconstruction over the evaluated spectral–adversarial and generative autoencoders while supporting generation and editing at high compression. These results demonstrate a practical non-adversarial waveform decoder for compact audio representations.

Primary area:
generative models
Keywords:
Audio Autoencoding, Waveform Diffusion, Audio Reconstruction and Generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_x5aVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FLORE-AE, a high-compression audio autoencoder that replaces the standard spectral-adversarial decoder with a conditional waveform diffusion decoder trained via flow matching. The system operates at 2048× and 4096× temporal compression. Three main components are proposed: (1) carrier–amplitude factorization (Section 3.2), which normalizes the waveform into a local-RMS carrier for diffusion and predicts the amplitude envelope from the latent; (2) FLORE-DiT (Section 3.3), a hierarchical diffusion transformer that expands latents into position-specific conditions and injects them at every resolution; and (3) Endpoint-Aware MeanFlow (Section 3.4), which adapts the decoder for two-evaluation decoding by supervising near-endpoint intervals. The paper reports 14.11 dB SI-SDR, 0.041 rFAD, and 4.173 MOS across six test sets spanning speech, music, and sound, with 0.008 RTF. Ablations in Table 4 attribute gains to local factorization, temporal alignment, and multiresolution computation. Downstream generation and editing are also evaluated.

Critical strengths

1. The non-adversarial formulation is a genuine and practical contribution. Removing the discriminator from high-compression audio autoencoding is a meaningful direction, and the paper demonstrates that flow-matching-based decoding can match or exceed spectral-adversarial systems on reconstruction (Table 1) while reaching 0.008 RTF with 2 NFE. This is not a trivial result—prior generative autoencoders like Mousai require 20 NFE and achieve -27.71 dB SI-SDR. The paper makes a credible case that waveform diffusion is a viable decoder for compact audio representations.

2. The controlled analysis is unusually thorough for this subfield. Table 4 and the appendix controls (Tables 10–13) systematically separate signal parameterization, alignment, hierarchy, and few-step decoding. The alignment result (Table 10) is particularly valuable because it identifies a specific, actionable mechanism—explicit source-to-waveform temporal correspondence—that the broader latent-audio community can build on. The paper also honestly reports where its claims do not hold (e.g., rFAD sometimes worsens with alignment; FLORE-AE-C peaks earlier in single-source generation).

Critical weaknesses

1. The individual components are incremental, and the paper does not adequately position them against prior art. Carrier–amplitude factorization (Section 3.2) is essentially local RMS normalization with a learned gain predictor—similar in spirit to the clip-level normalization used in WavFlow (Zhou et al., 2026) and the envelope-based approaches in the neural coding literature. FLORE-DiT (Section 3.3) combines a hierarchical DiT (Crowson et al., 2024) with cross-attention conditioning, which resembles DiTAR and the conditioning mechanisms in Stable Audio Open. EA-MF (Section 3.4) is a modest extension of iMF (Geng et al., 2025b) with an auxiliary near-endpoint spectral loss. The paper cites these related works but does not provide a clear technical delta for each component. This makes it hard to assess whether the system-level gains arise from genuinely new mechanisms or from careful engineering of existing ones. A more explicit "what is new here vs. prior work" discussion for each component would help.

2. The main results lack uncertainty quantification, and some comparisons are too close to call. With single-seed training, the 0.24 dB SI-SDR gap between FLORE-AE-C and SAME-L at 4096× (Table 1) and the 0.027 rFAD gap could easily be within run-to-run noise. The paper does not report confidence intervals or standard deviations for the objective metrics (only MOS has them). For a paper whose central claim is that FLORE-AE "leads the evaluated baselines," this is a meaningful gap. At minimum, the authors should report results across 2–3 seeds for the main comparison, or provide a variance estimate from the MOS study.

Questions for authors
  1. Clarification: In Table 4(b), the "Standard patchify DiT" row uses 0.309 inner-DiT TFLOPs, while "FLORE-DiT, full resolution" uses 0.274. The text says the full-resolution control is "nearly indistinguishable" from the hierarchy (13.42 vs. 13.45 dB). But the full-resolution control still uses sliding-window attention—what exactly is the difference between "FLORE-DiT, full resolution" and the deployed FLORE-DiT besides the hierarchy? Is the 0.274 vs. 0.098 TFLOP reduction entirely from moving inner blocks to coarser grids?

  2. Clarification: The amplitude predictor (Eq. 11) predicts both a global gain and a local residual envelope. Have you tried removing the local residual and keeping only the global gain? Table 4(a) shows that "Global RMS + predicted gain" performs worse than local factorization, but that variant still uses a different (clip-level) normalization. I want to understand whether the local residual envelope is necessary or whether a simpler global-gain predictor on top of the local carrier would suffice.

  3. Clarification: For FLORE-AE-C, the paper says it "does not use the 60,000-update adaptation" (Appendix B.5) and instead uses "exactly two model calls to its selected 300,000-update checkpoint under its recorded solver configuration." What is that solver configuration? Is it the four-evaluation Adams–Bashforth sampler from Appendix B.4, or something else? The main text says FLORE-AE-C uses 2 NFE, but the solver configuration is not specified in the main text.

  4. Suggestion for future work: The paper evaluates generation only through dataset-specific designated generators. A more informative experiment would train a single generator architecture on multiple autoencoder interfaces (with matched latent dimensions or a projection layer) to isolate the autoencoder's contribution from generator capacity. I understand this is expensive, but it would substantially strengthen the generation claims.

4 comments
AuthorsOpenReview ↗

Response to Reviewer x5aV (1/2)

We sincerely thank the reviewer for the positive feedback and for recognizing the non-adversarial formulation as "a genuine and practical contribution," our controlled analysis as "unusually thorough for this subfield," and the alignment result as "a specific, actionable mechanism … that the broader latent-audio community can build on." We hope our point-by-point response can fully resolve all your concerns.

  1. What is the technical delta of each component relative to WavFlow, hierarchical DiTs with cross-attention conditioning (DiTAR, Stable Audio Open), and iMF?

We would like to clarify that the three components are the mechanisms by which FLORE-AE's framework makes generative waveform decoding source-faithful at high compression; each meets a requirement that prior normalization, conditioning, and few-step objectives leave open, rather than being an incremental variant of them. These are, as Sec. 4.3 states, the "three requirements for a practical high-compression waveform diffusion autoencoder: the denoising target must be well conditioned across local amplitude scales, compressed latents must retain direct time-indexed control over dense synthesis, and few-step decoding must remain effective on generated as well as clean latents," met in turn by carrier–amplitude factorization, FLORE-DiT, and EA-MF. They, not engineering of existing techniques, produce the result the reviewer recognizes as "not a trivial result"; below, we give the delta of each over prior work, whose closest variants we test as baselines (Tables 4, 10).

(1) Relative to the clip-level normalization of prior work [1], carrier–amplitude factorization is a local normalization for waveform diffusion: it removes the amplitude variation within a clip that clip-level scaling leaves in the denoising target, and recovers it from the latent instead of discarding it. As Sec. 3.2 states, "a shared noise schedule produces different local SNRs in quiet and loud passages, even after clip-level scaling," while local normalization alone discards the dynamics that reconstruction needs. FLORE-AE therefore predicts the amplitude envelope, a gain plus a time-varying local part, from the same latent (Eqs. 4, 11), and, unlike in envelope-based coding, "no reference amplitude envelope is supplied" (Sec. 3.5). It is thus more than a learned gain predictor: without the local part, the deployed model falls from 14.11 to 4.37 dB SI-SDR (R4). Against clip-level normalization with a predicted gain, it raises SI-SDR from 11.95 to 13.45 dB and lowers rFAD from 0.149 to 0.110 (Table 4(a)).

(2) Relative to the hierarchical DiTs and conditioning mechanisms of prior work [2–4], FLORE-DiT changes how the condition reaches the waveform: the compact latent is expanded into a distinct condition for every synthesis position and reinjected, time-aligned, at every resolution. Where prior conditioning reads the condition by content or attaches it patch by patch, FLORE-DiT expands each latent interval into position-specific conditions added to the fine-grid waveform tokens (Eq. 5), which "gives synthesis positions within one latent interval distinct conditions" (Sec. 3.3.1). Prior hierarchical DiTs move computation to coarser grids but, as App. A states, "do not specify how compact, time-indexed source information should remain available across waveform resolutions"; FLORE-DiT reinjects the latent at every stage, aligned to that stage's grid (Eq. 5), so that each resolution reads the time-indexed latent directly. A hierarchical DiT with latent cross-attention is itself the baseline of Table 10: these aligned paths raise it from 12.34 to 13.23 dB SI-SDR, and a single-resolution DiT from 10.07 to 13.33 dB, whereas a 3.6× larger decoder adds only 0.24 dB (App. D.4.4). Source correspondence, not decoder capacity, thus drives fidelity; this is the "explicit source-to-waveform temporal correspondence" the reviewer highlights, and with it the hierarchy matches full-resolution quality at 64% lower inner-DiT cost (Table 4(b)).

(3) Beyond the MeanFlow objectives of prior work [5], which supervise average velocities, EA-MF supervises the amplitude-restored waveform that the two-step decoder actually returns, so that decoding stays faithful and perceptually strong on generated latents. As Sec. 3.4 notes, the endpoint of a MeanFlow interval "need not equal the clean-data endpoint"; EA-MF therefore also compares its near-endpoint estimates with the source. It is more than an auxiliary loss: decoder objectives should be "assessed on generated latents … rather than inferred from clean-latent reconstruction alone" (Sec. 4.3). On identical generated latents at two evaluations, EA-MF lowers FAD from 0.111 to 0.086 against an iMF control that already includes terminal reconstruction (Table 4(c)), at 0.008 RTF, comparable to single-pass autoencoders (Table 1).

We have added a concise summary of these distinctions to App. A of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer x5aV (2/2)

  1. The main results lack uncertainty quantification: results across seeds, confidence intervals for the objective metrics, or a variance estimate from the MOS study should be reported.

Thank you for this suggestion. We have added uncertainty quantification to App. G of the revised manuscript: the results of retraining with a second seed and of re-decoding with different noise seeds (Table 21), paired confidence intervals for the main SI-SDR comparisons (Table 17), and a variance estimate from the MOS study (Table 19); the variation across seeds is far smaller than the main differences, and every paired interval in App. G excludes zero. Retraining the controlled FLORE-DiT of Table 4 with a second seed changes its SI-SDR by 0.09 dB and its MEL error by 0.0003, and re-decoding with two further noise seeds changes FLORE-AE's macro SI-SDR by at most 0.0014 dB. By comparison, FLORE-AE's SI-SDR gains over SAME-L and ϵar-VAE are 2.04 and 2.05 dB (paired 95% intervals [2.01, 2.08] and [2.02, 2.09]), and FLORE-AE-C's 0.24 dB gain over SAME-L, which the reviewer singles out, has a paired interval of [+0.21, +0.27] dB. For the MOS study, every system is rated on the same 60 items by the same five listeners per item (App. B.6.4), so differences are computed per item: FLORE-AE's MOS gain over ϵar-VAE2 (+0.152) and FLORE-AE-C's gain over SAME-L (+0.165) have paired 95% intervals of [+0.09, +0.22] and [+0.08, +0.25].

  1. What differs between "FLORE-DiT, full resolution" and the deployed FLORE-DiT besides the hierarchy? Is the 0.274 vs. 0.098 TFLOP reduction entirely from coarser grids?

The two differ only in the hierarchy, so the reduction from 0.274 to 0.098 TFLOPs comes entirely from running the inner blocks on coarser grids. The full-resolution control "reallocates the inner blocks to the highest-resolution grid" (Table 11) and is otherwise the deployed FLORE-DiT, with the same aligned conditioning, the same inner blocks, and the same sliding-window attention; the 0.309 TFLOPs of the standard patchify DiT reflect global attention (App. D.4.2). With sliding-window attention, the cost of a block scales with the number of tokens it processes, and the hierarchy runs the same blocks on sequences two to eight times shorter (App. B.2), which accounts for the whole reduction at unchanged quality (13.45 vs. 13.42 dB).

  1. Is the local residual envelope necessary, or would a global-gain predictor on top of the local carrier suffice?

The local residual envelope is necessary: removing it from the deployed FLORE-AE, so that only the global gain acts on the locally normalized carrier, lowers SI-SDR from 14.11 to 4.37 dB and raises rFAD from 0.041 to 0.066. The carrier has approximately unit local RMS by construction (Eq. 4; App. B.1), so a single gain returns a waveform whose local energy is nearly the same everywhere; the zero-mean local residual eψ\boldsymbol{e}_\psi (Eq. 11) is the path through which the latent restores time-varying amplitude. We have added this result to App. D.5 of the revised manuscript.

  1. What is the two-call solver configuration of FLORE-AE-C?

As Sec. 4.1 states for both models, "EA-MF fine-tunes only the waveform decoder for 60,000 updates and uses the grid (0, 0.5, 1)": FLORE-AE-C uses the same two-step solver as FLORE-AE. The statements in App. B.5 and B.6.2 that FLORE-AE-C does not use this adaptation are typos, which we have corrected in the revised manuscript.

  1. Suggestion: train a single generator architecture on multiple autoencoder interfaces to isolate the autoencoder's contribution.

Thank you for this suggestion. We would like to clarify that our AudioCaps generation experiments are designed along this idea and verify it with two different generator architectures: in each, one architecture is trained on multiple autoencoder interfaces with the same training data, seed, and update budget, and only the input and output projections follow the native latent width. With the first, the generator of the AudioCaps comparison in Table 2, FLORE-AE has the best FAD and CLAP (0.086 and 0.321). With the second, initialized from one Qwen3-1.7B backbone and trained on four frozen interfaces (App. D.7.2), FLORE-AE has the lowest FD-PaSST and the highest CLAP at the full 200k budget (182.17 and 0.386; Table 16). With both architectures, where only the autoencoder differs, FLORE-AE thus has the best FAD/FD and CLAP at the full training budget.

References

[1] Zhou et al., "WavFlow: Audio Generation in Waveform Space." arXiv:2605.18749, 2026.

[2] Crowson et al., "Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers." ICML, 2024.

[3] Jia et al., "DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation." ICML, 2025.

[4] Evans et al., "Stable Audio Open." arXiv:2407.14358, 2024.

[5] Geng et al., "Improved Mean Flows: On the Challenges of Fastforward Generative Models." arXiv:2512.02012, 2025.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer x5aV,

Thank you again for your positive and constructive review. We would like to briefly follow up on our response, in which we:

  • gave the technical delta of each component relative to prior work (R1);
  • added uncertainty quantification, including the paired interval for FLORE-AE-C vs. SAME-L that you pointed out (R2);
  • clarified the hierarchy, the local residual envelope, and FLORE-AE-C's solver (R3–R5).

We would be grateful to know whether these address your concerns.

Best regards,

The Authors

AuthorsOpenReview ↗

Looking Forward to Your Feedback

Dear Reviewer x5aV,

Thank you again for your great efforts and valuable comments. To address your questions, we have clarified the technical delta of each component relative to prior work, added uncertainty quantification, including the paired interval for FLORE-AE-C vs. SAME-L that you pointed out (Tables 17, 19, 21), added the result on the local residual envelope (App. D.5), and clarified the hierarchy and the solver of FLORE-AE-C. As the author–reviewer discussion is about to close, we would be grateful if we could hear your feedback on our response, and sincerely hope that you can consider raising the score if you find our reply helpful. We would be happy to answer and discuss if you have further comments.

Thanks in advance,

Paper 1293 authors

Reviewer_88KTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the trade-offs among representation compactness, waveform reconstruction fidelity, and computational efficiency in high-compression audio autoencoding. To address the adversarial training requirements of conventional spectral–adversarial decoders, the authors propose FLORE-AE, which formulates audio reconstruction as latent-conditioned waveform diffusion, enabling high-quality audio reconstruction without an adversarial discriminator.

Specifically, the model introduces carrier–amplitude factorization to separate normalized waveform synthesis from local amplitude recovery. It further proposes FLORE-DiT, which combines temporally aligned latent conditioning with multiresolution modeling to improve waveform reconstruction, and Endpoint-Aware MeanFlow (EA-MF), which incorporates reconstruction supervision near the data endpoint to enable efficient decoding with two function evaluations.

The authors evaluate the proposed method on speech, music, and environmental sound datasets through reconstruction, text-to-audio generation, and audio editing experiments. At a temporal compression factor of 2048×, FLORE-AE achieves 14.11 dB SI-SDR, 0.041 rFAD, and 4.173 MOS. Extensive ablation studies further investigate the contributions of signal factorization, temporal alignment, and few-step decoding.

Critical strengths
  1. Clear motivation and a practically relevant research problem.

The paper addresses the trade-offs among source fidelity, generation quality, and decoding efficiency in highly compressed audio representations, while exploring a generative decoding approach without adversarial training. The authors decompose the problem into three aspects: local amplitude variation, temporal alignment of latent representations, and few-step generation, resulting in a coherent research motivation.

  1. Systematic experimental design with evidence supporting the key mechanisms.

Beyond reporting overall reconstruction performance, the paper provides controlled ablations to examine the contributions of individual components. Local carrier–amplitude factorization improves multiple reconstruction metrics, temporal alignment substantially increases SI-SDR, the multiresolution architecture reduces the computational cost of the inner Transformer, and EA-MF improves generation FAD under fixed generated-latent conditions.

  1. Broad experimental coverage and practical relevance.

The authors evaluate reconstruction quality across six test sets covering speech, music, and environmental sounds, and further investigate text-to-audio generation and audio editing. The paper reports objective metrics, subjective listening scores, and decoding latency, while providing detailed training configurations, evaluation protocols, and supplementary experiments in the appendices.

Critical weaknesses
  1. Limited Methodological Novelty and Insufficient Differentiation from Prior Work

The main technical ideas underlying FLORE-AE have substantial precedents in the literature. Moûsai has explored conditional waveform diffusion decoding from compressed audio representations, while Music2Latent investigates non-adversarial generative audio autoencoding. Multiresolution Transformers, latent conditioning mechanisms, and MeanFlow also build upon established techniques. Consequently, the primary contributions of FLORE-AE appear to lie in combining and adapting existing approaches for high-compression waveform reconstruction, and its independent methodological novelty requires further clarification.

Although the paper provides ablation studies demonstrating the effectiveness of carrier–amplitude factorization, temporal alignment, and EA-MF, the independent necessity of some design choices remains insufficiently established. For example, Appendix D.4.3 shows that removing stage-wise latent injection leaves SI-SDR largely unchanged, while EA-MF primarily extends existing MeanFlow techniques with additional endpoint reconstruction supervision. The paper would benefit from a clearer distinction between genuinely new mechanisms and adaptations of established methods, together with a more explicit explanation of the methodological insights contributed beyond prior generative audio autoencoders.

  1. Incomplete Characterization of Latent Capacity and Compression Efficiency

The paper primarily characterizes compression using the temporal compression factor, which does not fully capture the information capacity of the latent representation. For example, FLORE-AE and Stable Audio Open both employ 2048× temporal downsampling, but their latent widths are 192 and 64, respectively, resulting in a threefold difference in latent scalar density. Furthermore, although the method employs a KL bottleneck, the paper does not report the actual KL values, effective latent dimensionality, or other measures of representation capacity, making it difficult to assess how latent capacity contributes to reconstruction quality.

Since the baselines differ in latent representation size, training data, and computational budgets, the performance improvements in Table 1 cannot be attributed solely to the diffusion decoding mechanism. Although FLORE-AE-C and SAME-L share the same temporal downsampling factor and latent width, providing a more comparable setting, FLORE-AE and FLORE-AE-C also differ in architectural configurations and training budgets. Therefore, the current experiments do not sufficiently establish the reconstruction quality–compression efficiency trade-off under controlled latent capacity constraints.

  1. Limited Statistical Reliability and Confounded Experimental Comparisons

Although the paper reports strong results across multiple datasets, the main model configurations are trained with only a single random seed, without variance estimates or confidence intervals from independent training runs. Moreover, diffusion decoding introduces sampling randomness, yet the paper only fixes decoder noise seeds for reproducibility without evaluating reconstruction stability across different noise realizations. While the MOS study involves multiple listeners and a shared evaluation set, confidence intervals or statistical significance tests for paired differences between models are not provided, making it difficult to assess the reliability of relatively small performance improvements.

Furthermore, the downstream text-to-audio generation experiments employ separately trained latent generators for different autoencoders. Consequently, the reported generation performance reflects both the quality of the latent representations and differences in generator training dynamics. Appendix D.7.2 also shows that model rankings vary across generator training stages. Although the authors appropriately describe these as system-level comparisons, the current results do not independently demonstrate that FLORE-AE learns inherently more generatable or consistently superior latent representations.

  1. Insufficient Evidence for the Generalizability of the Proposed Design Principles

The proposed design principles are reasonably motivated, but their empirical support remains limited in scope. Temporal alignment primarily improves paired reconstruction fidelity without consistently improving rFAD, while the reported approximately 64% computational reduction from multiresolution modeling applies only to the inner DiT rather than the complete decoder. Similarly, the benefits of EA-MF on fixed generated latents are demonstrated mainly under a specific generator and adaptation setting, without sufficient evidence of its effectiveness across different generative models or latent distributions.

In addition, avoiding adversarial training is a central motivation of the paper, but the authors do not directly compare the training stability, convergence efficiency, or total training costs of FLORE-AE against conventional spectral–adversarial approaches. The current experiments demonstrate that FLORE-AE is a competitive non-adversarial audio autoencoding system, but they do not sufficiently establish broader advantages in optimization stability or computational efficiency.

Questions for authors
  1. Methodological Novelty and Prior Work

Compared with Moûsai, Music2Latent, and existing multiresolution conditional generative models, what are the main independent methodological contributions of FLORE-AE? Could the authors further clarify which aspects of temporal alignment, repeated latent conditioning, and EA-MF are adaptations of existing techniques, and which introduce genuinely new mechanisms?

  1. Latent Capacity and Compression Fairness

Could the authors report the actual KL values, effective latent dimensionality, or other measures of representation capacity? Do the reconstruction advantages of FLORE-AE persist when latent scalar density, training data, and computational budgets are matched across methods? Additionally, could more tightly controlled experiments isolate the effects of temporal compression factors on reconstruction quality?

  1. Statistical Reliability and Downstream Generation

Could the authors report performance variability across independent training runs and decoder noise seeds, together with confidence intervals for the main reconstruction metrics and paired MOS comparisons? For downstream generation, how can the contribution of the latent representation itself be distinguished from differences in the training dynamics and configurations of the dedicated latent generators?

  1. Mechanism Generalization and Training Efficiency

Does EA-MF consistently improve generation quality across different latent generators or varying degrees of latent distribution shift? Could the authors provide computational costs for the complete decoder and a more comprehensive comparison of training expenses? Furthermore, could analyses of training stability or convergence behavior better substantiate the claimed advantages of non-adversarial decoding over conventional spectral–adversarial approaches?

7 comments
AuthorsOpenReview ↗

Response to Reviewer 88KT (1/3)

We sincerely thank the reviewer for recognizing our "clear motivation and a practically relevant research problem," our "systematic experimental design with evidence supporting the key mechanisms," and our "broad experimental coverage and practical relevance." We hope our point-by-point response can fully resolve all your concerns.

  1. What is new in FLORE-AE beyond Moûsai, Music2Latent, and multiresolution models, including temporal alignment, repeated latent conditioning, and EA-MF?

We would like to clarify that FLORE-AE contributes a framework and three mechanisms that solve the challenges high compression creates for generative decoding (Sec. 1), which Moûsai, Music2Latent, and multiresolution generative models leave open, rather than a combination of their components. These are a compact latent that must control dense synthesis (FLORE-DiT, point 2), amplitude-dependent local SNRs (carrier–amplitude factorization, point 3), and few-step decoding of generated latents (EA-MF, point 4). Below, we detail each; close variants of the listed techniques are the baselines of our controlled analysis (Tables 4, 10).

(1) As a framework, FLORE-AE makes source fidelity a design target of generative autoencoding, shaping both what the latent retains and how the decoder uses it rather than leaving fidelity to generative decoding. Moûsai trains its latent, encoded from magnitude spectrograms, with a diffusion loss alone, and Music2Latent with a single consistency loss; yet, as Sec. 3.1 states, "conditional modeling alone … does not ensure source-faithful reconstruction: fidelity depends on both the information retained in zc\boldsymbol{z}_c and how the decoder uses it." In FLORE-AE, by contrast, the clean-target decoder returns a waveform estimate at every noise level, so paired spectral supervision is applied to it and reaches the encoder "without a sampling rollout" (Eq. 8): the latent is trained for source correspondence as in spectral–adversarial autoencoders, but without a critic. Moûsai and Music2Latent reach −27.71 and −2.42 dB SI-SDR, whereas FLORE-AE reaches 14.11 dB and leads all evaluated autoencoders in reconstruction and AudioCaps generation (Tables 1–2) at 0.008 RTF, making generative decoding an alternative to the spectral–adversarial paradigm. This waveform-level source fidelity also carries into editing, where prior generative autoencoders do not preserve the observed context: FLORE-AE keeps it at 9.36 dB SI-SDR in inpainting, against −30.81 dB for Moûsai and 7.18 dB for the strongest baseline (Table 8).

(2) FLORE-DiT lets a compact latent condition waveform-resolution synthesis position by position at every resolution, so that context computation can move to coarser resolutions without losing source timing. A DiT reads a compact condition by content through cross-attention, Moûsai passes it through intermediate U-Net features, and multiresolution Transformers "do not specify how compact, time-indexed source information should remain available across waveform resolutions" (App. A). By contrast, FLORE-DiT expands each latent interval into position-specific conditions and reinjects the time-indexed latent at every stage (Eq. 5), while context moves to coarser stages. A standard DiT yields plausible audio (rFAD 0.108) but only 10.07 dB SI-SDR; alignment raises it to 13.33 dB and transfers to a waveform U-Net (6.67→7.65 dB), whereas a 3.6× larger decoder adds only 0.24 dB (App. D.4.4). The stage-wise (repeated) injection that the reviewer cites from App. D.4.3 carries distributional quality (rFAD 0.110→0.118 without it), and the expansion paired fidelity (Table 11). Source correspondence, not decoder capacity, thus drives fidelity, and with it the hierarchy matches full-resolution quality at 64% lower inner-DiT cost (Table 4(b)).

(3) Carrier–amplitude factorization assigns the source's amplitude dynamics to the latent, so the diffusion target can be locally normalized and stay well conditioned without losing them. Clip-level normalization, as in WavFlow (App. A), still leaves "different local SNRs in quiet and loud passages" (Sec. 3.2). It is the only tested parameterization that improves all four metrics over fixed-scale prediction (SI-SDR 12.32→13.45 dB, rFAD 0.344→0.110; Table 4(a)).

(4) EA-MF supervises the waveform that the two-step decoder actually returns, keeping decoding fast and faithful on generated latents. Unlike MeanFlow, which supervises average velocities over intervals that need not end at the data, EA-MF also supervises the amplitude-restored waveform near the data endpoint. It is more than an added loss: it targets the deployed interface, where, as Sec. 4.3 concludes, decoder objectives should be "assessed on generated latents." On identical generated latents, EA-MF lowers FAD from 0.111 to 0.086 against iMF at two evaluations (Table 4(c)).

We have added a concise summary of these distinctions to App. A of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 88KT (2/3)

  1. Latent capacity is characterized only by temporal compression. Do the advantages persist when latent size, data, and compute are matched, can capacity measures be reported, and can the effect of temporal compression be isolated?

FLORE-AE characterizes compression by the temporal factor because the temporal rate is what downstream generation pays for, whereas a wider latent is now tractable. As App. B.2 explains, "wider latents remain tractable when their representation is suitably structured," whereas a higher latent rate "makes downstream generation more expensive" (Sec. 1). FLORE-AE therefore keeps Stable Audio Open's temporal rate and widens the latent for reconstruction detail, and at this width it leads AudioCaps generation (Table 2). The latent capacity is nevertheless specified in the paper: Table 7 lists the latent rates and widths, and App. B.6.2 defines the resulting scalar density as Dfs/RDf_s/R scalars per second. To make it directly visible, we have added this density to Table 1 of the revised manuscript; and since KL values depend on each system's regularizer and latent scale, we have added two capacity measures computed identically across systems to App. G (Table 20): the number of principal components that explain 99% of the latent variance (188 of 192 for FLORE-AE, 62 of 64 for Stable Audio Open, 61 of 64 for ϵar-VAE, and 234 of 256 for SAME-L) and the participation ratio (143, 44, 24, and 25 dimensions).

Nevertheless, to examine the reconstruction advantage at matched latent size, we have trained FLORE-AE with latents as small as the spectral–adversarial baselines' and added the results to App. G of the revised manuscript (Table 17, reproduced below): the advantage remains. In the same 2048× and D=64D=64 configuration as Stable Audio Open, FLORE-AE reaches 10.56 dB SI-SDR against 5.60 dB and is better in all four metrics. At the same 2,756 scalars/s as ϵar-VAE and SAME-L, FLORE-AE with a 128-channel latent reaches 12.20 dB against 12.06 and 12.07 dB, with the lower MEL error. Both variants use the recipe of FLORE-AE at the same 2048× temporal compression. Likewise, at the same 4096× and D=256D=256 as SAME-L, which the reviewer also regards as "a more comparable setting," FLORE-AE-C has the higher SI-SDR (12.31 vs. 12.07 dB; paired 95% interval [+0.21, +0.27] dB, Table 17) and the lower rFAD (0.061 vs. 0.088; Table 1). At this density, the 2048× variant with D=128D=128 and the 4096× FLORE-AE-C reach 12.20 and 12.31 dB, so the two temporal compression factors give similar reconstruction once the latent density is fixed.

SystemScalars/sNFESI-SDR ↑STFT ↓MEL ↓rFAD ↓
FLORE-AE4,134214.110.0520.0340.041
ϵar-VAE23,20014.790.0590.0470.048
FLORE-AE-C2,756212.310.0590.0430.061
FLORE-AE, D=128D=1282,756212.200.0580.0410.055
ϵar-VAE2,756112.060.0550.0470.054
SAME-L2,756112.070.0580.0430.088
FLORE-AE, D=64D=641,378210.560.0610.0450.087
Stable Audio Open1,37815.600.0630.0510.101

As for training data and compute, Table 1 follows the common practice of audio autoencoder evaluation, as in Stable Audio Open and SAME: baselines are evaluated with their officially released weights (Sec. 4.1). The contribution of each mechanism, in turn, is established by the controlled analysis, where the latent interface, data, and training budget are fixed (Table 4; App. D.4.1, D.5), an analysis that Reviewer 1fp4 finds "well controlled" and Reviewer x5aV "unusually thorough for this subfield." Nevertheless, to compare the two decoder paradigms themselves on the same latent and data, we have trained a spectral–adversarial decoder of matched size on the frozen FLORE-AE encoder and added the result to App. G of the revised manuscript (Table 18, reproduced below): the diffusion decoder is ahead in SI-SDR (14.11 vs. 13.68 dB), MEL error (0.034 vs. 0.039), and rFAD (0.041 vs. 0.079). With the latent and data fixed, the diffusion decoder therefore retains its advantage in waveform, spectral, and distributional fidelity.

Decoder on the frozen FLORE-AE latentNFESI-SDR ↑STFT ↓MEL ↓rFAD ↓
Spectral–adversarial (Stable Audio recipe)113.680.0530.0390.079
FLORE-AE diffusion decoder214.110.0520.0340.041
AuthorsOpenReview ↗

Response to Reviewer 88KT (3/3)

  1. How variable are the results across independent training runs and decoder noise seeds, with intervals for the main metrics and paired MOS comparisons? How can the latent's contribution to generation be separated from the training of its generator?

Regarding statistical reliability, the main differences are far larger than the variation from training seeds and decoder noise, and every paired interval in App. G of the revised manuscript excludes zero (Tables 17, 19, and 21; Table 21 below). By design, the comparisons are paired: all systems are scored on identical recordings, and the listening test uses one item panel (App. B.6.1, B.6.4). Retraining the controlled FLORE-DiT of Table 4 with a second seed changes its MEL error by 0.0003 and its SI-SDR by 0.09 dB, against the 0.9 to 3.3 dB that temporal alignment adds (Table 10); re-decoding every test set with two further noise seeds changes FLORE-AE's macro SI-SDR by at most 0.0014 dB; FLORE-AE's SI-SDR gain over the strongest baseline, SAME-L, is 2.04 dB with a paired 95% interval of [2.01, 2.08]; and its MOS advantage over ϵar-VAE2 has a paired 95% interval of [+0.09, +0.22].

VariationRunSI-SDR ↑MEL ↓
Decoder noise (FLORE-AE)Original seed14.10990.0341
Noise seed 114.10960.0341
Noise seed 214.11130.0341
Training seed (controlled FLORE-DiT)Original seed13.45450.0363
Second seed13.36450.0366

As for downstream generation, on AudioCaps the generator is identical for every autoencoder, in architecture, training data, seed, and update budget, so the autoencoder is the only difference and the AudioCaps gaps in Table 2 are attributable to it. The "designated" generator of each row (Sec. 4.1) is this same generator trained on that autoencoder's latent, with only the input and output projections following the native latent width; we have stated this protocol in App. D.1. Since nothing else differs, the differences in training dynamics are themselves produced by the latent, and App. D.7.2 reports them in full with a second generator family (Qwen3-1.7B), trained identically on four frozen autoencoders (Figure 4): Stable Audio Open leads at 100k updates, whereas at the full 200k budget FLORE-AE has the lowest FD-PaSST and the highest CLAP (182.17 and 0.386; Table 16). In both generator families, where only the autoencoder differs, FLORE-AE thus has the best FAD/FD and CLAP at the full training budget.

  1. Does EA-MF generalize across latent generators and degrees of distribution shift? What are the complete decoder cost and the training cost, and do training stability or convergence support the advantages of non-adversarial decoding?

EA-MF's gain is not specific to one generator or domain: on the latents of a second generator, the Song Describer text-to-music generator, it again gives the lowest FAD. EA-MF is evaluated on generated latents, where it is deployed, with identical cached latents so that only the decoder differs (Table 4(c); App. D.3). We have repeated it on the second generator and added the result to App. D.3 of the revised manuscript: EA-MF is 0.064 below the iMF control in FAD (paired 95% interval [0.059, 0.070]), with no significant change in CLAP.

As for the cost of the complete decoder, the hierarchy still reduces computation by 41% per call when every component is counted, against 64% for the inner DiT alone. The paper reports the inner DiT, the resolution-dependent part that the hierarchy changes, "under the stated counting scope" (Sec. 4.3). We have added the complete count to App. D.4.2 of the revised manuscript, and the 0.008 RTF of Table 1 is measured over the whole decoding call (App. D.6).

As for the training cost, non-adversarial decoding does not make training more expensive. FLORE-AE's joint training takes 900k updates on eight A800 GPUs, about 1,150 GPU-hours, followed by 60k EA-MF updates of the decoder alone (App. B.6.2); for reference, Stable Audio Open reports 456 hours on 32 A100 GPUs (about 14,600 GPU-hours) and SAME 700k steps on 32 H100 GPUs.

As for training stability, the advantage of non-adversarial decoding lies in the optimization problem itself: FLORE-AE minimizes one objective, whereas adversarial training solves a minimax game between the decoder and its discriminators. As Sec. 1 states, discriminator feedback "introduces an adversarial game whose updates can be unstable" (Mescheder et al., 2018), coupling representation learning "to an evolving real/fake objective"; such pipelines typically stage their discriminators (e.g., the three-stage procedure of SAME). In FLORE-AE, by contrast, every term of Eq. 8 is a regression or regularization term with a target fixed by the data, and no network is trained against the decoder: training is ordinary minimization. What remains empirical is whether a decoder trained this way reaches adversarial quality, which Tables 1 and 18 show.

Reviewer_1fp4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes FLORE-AE, a continuous audio autoencoder that replaces the usual spectral-adversarial decoder with a latent-conditioned waveform flow model trained by clean-target flow matching. The decoder has three components. First, carrier-amplitude factorization splits the waveform into a locally RMS-normalized carrier, which the flow model generates, and an amplitude envelope, which is predicted from the latent. Second, FLORE-DiT is a diffusion transformer that keeps the latent time-aligned with the waveform at every resolution and moves long-range context modeling to a coarser U-shaped hierarchy. Third, Endpoint-Aware MeanFlow (EA-MF) is a fine-tuning stage that enables two-step decoding by adding spectral supervision near the data endpoint.

At 2048x (D=192) and 4096x (D=256) temporal compression, FLORE-AE reports the best six-dataset SI-SDR, mel error, rFAD and MOS among the released baselines (ϵar-VAE/2, Stable Audio Open, SAME-S/L, Moûsai, Music2Latent and CoDiCodec), at 0.008 RTF with two decoder evaluations. It also reports better text-to-audio generation and editing results. Matched ablations in Table 4 attribute the gains to the factorization, the latent alignment and EA-MF, and show that the coarse hierarchy uses 64% fewer inner-DiT FLOPs at the same quality.

Critical strengths

Quality

The ablations in Table 4 are well controlled and are the most convincing part of the paper. Local carrier factorization is the only tested signal parameterization that improves all four metrics, raising SI-SDR from 12.32 to 13.45 dB and lowering rFAD from 0.344 to 0.110. The latent alignment improves both U-Net and DiT backbones, raising the DiT from 10.07 to 13.33 dB, and the coarse hierarchy matches full-resolution quality at 0.098 instead of 0.274 TFLOPs.

Significance

Together, these results make a credible case that a non-adversarial waveform diffusion decoder can be practical at two decoder evaluations and 0.008 RTF, which is useful for latent audio generation. FLORE-AE-C is also much smaller than SAME-L at the same latent size (335M versus 852M parameters).

Critical weaknesses

Some key technical limitations and suggested improvements:

  1. The headline reconstruction comparison gives FLORE-AE a larger latent than the baselines. The paper reports compression as temporal downsampling only, but Table 7 (App. B.6) shows that FLORE-AE at 2048x with D=192 carries about 4,134 latent values per second, the most of any system compared. ϵar-VAE and SAME-L carry about 2,756 values per second, and Stable Audio Open about 1,378. The 14.11 versus 12.07 dB SI-SDR comparison is therefore not made at matched compression. At matched density, FLORE-AE-C (4096x, D=256, about 2,756 values per second) reaches 12.31 dB, compared with 12.07 dB for SAME-L and 12.06 dB for ϵar-VAE, and ϵar-VAE, with 148M parameters and a single decoder evaluation, is better on STFT error (0.055 versus 0.059) and rFAD (0.054 versus 0.061). FLORE-AE-C still leads on MOS and editing. Please correct me if I have misread Table 7. I suggest adding latent values per second as a column in Table 1, adding a 2048x, D=64 variant for a direct comparison with Stable Audio Open, and qualifying the reconstruction claim in the abstract accordingly.

  2. No adversarial decoder is trained under the same conditions, so the benefit of removing the discriminator is not isolated. All baselines are released checkpoints with their own training data, latent interfaces and model sizes, while FLORE-AE is trained on FMA, Freesound and WavCaps with a single seed. The gains could therefore come from dropping the discriminator, or from differences in data, latent width or decoder size (260M to 335M parameters). Training a GAN decoder with a multi-resolution STFT loss on the same encoder, latent and data would settle this.

  3. The listening test does not establish whether the gaps between the top systems are real. MOS is reported as the mean plus or minus the item standard deviation over 60 items with five listeners each, with no standard errors or significance tests, for example for FLORE-AE (4.173) versus ϵar-VAE2 (4.021). The generation and editing tables are also system-level comparisons, since each autoencoder is paired with its own generator, which the authors acknowledge. Reporting standard errors or paired tests for MOS would make the perceptual claims more reliable.

Overall I feel this is a solid engineering contribution, and I would be open to raising my score if the matched-size comparison and a matched adversarial baseline support the main claims.

Questions for authors
  1. How does FLORE-AE compare with the baselines when both carry the same number of latent values per second? The most useful comparisons would be a 2048x, D=64 FLORE-AE against Stable Audio Open, and the 4096x, D=256 model against ϵar-VAE and SAME-L with confidence intervals. The answer would affect my score the most.

  2. Does an adversarial decoder trained with the same encoder, latent interface and data close the gap? That experiment would make the decoder type the only difference, which is what the main claim needs.

  3. How does FLORE-AE-C decode in two steps without the EA-MF stage? App. B states that FLORE-AE-C does not use the 60,000-update FLORE-AE adaptation, yet it decodes in two calls. Please clarify how, and whether the 4096x row in Table 1 is directly comparable to the 2048x row.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 1fp4 (1/2)

We sincerely appreciate the reviewer's recognition that the ablations in Table 4 "are well controlled," that our results "make a credible case that a non-adversarial waveform diffusion decoder can be practical at two decoder evaluations and 0.008 RTF," and that FLORE-AE is "a solid engineering contribution." Following your suggestions, we have added the matched-density comparison (R1) and an adversarial decoder trained with the same encoder, latent, and data (R2). We hope our point-by-point response can fully resolve all your concerns.

  1. How does FLORE-AE compare with the baselines when both carry the same number of latent values per second?

FLORE-AE's density is a design choice for both reconstruction and generation: it keeps Stable Audio Open's temporal rate, so a latent generator models sequences of the same length, and spends the added scalars on width, which buys reconstruction detail; at this density FLORE-AE also leads AudioCaps generation (FAD/CLAP 0.086/0.321 vs. 0.160/0.318 for Stable Audio Open; Table 2). The densities the reviewer derives from Table 7 are correct, and we have added them to Table 1. Width is inexpensive for generation: in the AudioCaps generators of Table 2, the latent width changes only the input and output projections (App. D.1), whereas the sequence length sets the cost of every layer. Density alone therefore does not determine generation cost: ϵar-VAE and SAME-L both carry 2,756 scalars/s, yet a generator on ϵar-VAE's latent must model four times as many latent steps per second as one on SAME-L's (43.07 vs. 10.77; Table 7). This choice also follows current practice: "wider latents remain tractable when their representation is suitably structured" (App. B.2), and Stable Audio 3 [1], the successor of Stable Audio Open, encodes audio with SAME at 4096× and D=256D=256, twice Stable Audio Open's density. FLORE-AE's wider latent is also learnable by a common generator: with one Qwen3-1.7B generator family on four frozen interfaces, FLORE-AE reaches the lowest FD-PaSST and the highest CLAP at 200k updates (App. D.7.2).

At the density of ϵar-VAE and SAME-L, FLORE-AE with a 128-channel latent still has the higher SI-SDR (12.20 vs. 12.06 and 12.07 dB; paired 95% intervals [+0.12, +0.16] and [+0.12, +0.15] dB) and the lower MEL error, and its rFAD matches ϵar-VAE's and is lower than SAME-L's (0.055 vs. 0.054 and 0.088). Likewise, at the same 4096× and D=256D=256 as SAME-L, FLORE-AE-C exceeds ϵar-VAE and SAME-L in SI-SDR by 0.25 and 0.24 dB (paired 95% intervals [+0.22, +0.28] and [+0.21, +0.27] dB). We have trained 128- and 64-channel variants with the recipe of FLORE-AE at the same 2048× temporal compression, evaluated them with the same two-evaluation decoder under the protocol of Table 1, and added the results to App. G of the revised manuscript (Table 17, reproduced below). At the density of Stable Audio Open, FLORE-AE with a 64-channel latent leads Stable Audio Open in all four metrics, with an SI-SDR of 10.56 vs. 5.60 dB (paired 95% interval [+4.90, +5.02] dB). Narrowing the latent from 192 to 128 channels lowers SI-SDR by 1.91 dB, which is the reconstruction detail the deployed density buys.

SystemScalars/sNFESI-SDR ↑STFT ↓MEL ↓rFAD ↓
FLORE-AE4,134214.110.0520.0340.041
ϵar-VAE23,20014.790.0590.0470.048
FLORE-AE-C2,756212.310.0590.0430.061
FLORE-AE, D=128D=1282,756212.200.0580.0410.055
ϵar-VAE2,756112.060.0550.0470.054
SAME-L2,756112.070.0580.0430.088
FLORE-AE, D=64D=641,378210.560.0610.0450.087
Stable Audio Open1,37815.600.0630.0510.101

References

[1] Evans et al., "Stable Audio 3." arXiv:2605.17991, 2026.

AuthorsOpenReview ↗

Response to Reviewer 1fp4 (2/2)

  1. Does an adversarial decoder trained with the same encoder, latent interface, and data close the gap?

An adversarial decoder trained with the same encoder, latent, and data does not close the gap: with the data, latent width, and decoder size that the reviewer lists all held fixed, FLORE-AE's diffusion decoder remains ahead in waveform, spectral, and distributional fidelity. Table 1 follows the common practice of audio autoencoder evaluation, as in Stable Audio Open and SAME, and evaluates baselines with their officially released weights (Sec. 4.1); against five such spectral–adversarial autoencoders from three model families, FLORE-AE leads in every objective reconstruction metric, while Table 4 controls each of its design choices. Following the reviewer's suggestion, we have trained a spectral–adversarial decoder on the frozen FLORE-AE encoder and the same data with the released Stable Audio autoencoder recipe [2], a multi-resolution STFT loss with a discriminator, and added the result to App. G of the revised manuscript (Table 18, reproduced below); its 196.6M parameters match FLORE-AE's 196.0M-parameter decoder (Table 14). On the same latent, the diffusion decoder is ahead in SI-SDR by 0.43 dB (paired 95% interval [+0.41, +0.45]), has 13% lower MEL error, and has about half the rFAD. The benefit of the non-adversarial diffusion decoder is thus isolated from the data, latent width, and decoder size.

Decoder on the frozen FLORE-AE latentNFESI-SDR ↑STFT ↓MEL ↓rFAD ↓
Spectral–adversarial (Stable Audio recipe)113.680.0530.0390.079
FLORE-AE diffusion decoder214.110.0520.0340.041
  1. The listening test does not establish whether the gaps between the top systems are real. … Reporting standard errors or paired tests for MOS would make the perceptual claims more reliable.

Thank you for this suggestion. We have added paired statistics for the listening test to App. G of the revised manuscript (Table 19): paired over the same 60 items, FLORE-AE's MOS exceeds ϵar-VAE2's by 0.152, with a 95% CI of [+0.09, +0.22] and a Wilcoxon signed-rank p < 0.001. Because every system is rated on the same items by the same five listeners per item (App. B.6.4), differences between systems are computed per item; the ± in Table 1 is the item standard deviation, which reflects item difficulty rather than the uncertainty of a difference. For generation, every interface on AudioCaps uses a generator of the same architecture with the same 200k-update budget, and only the input and output projections follow the latent width, which we have stated in App. D.1.

  1. How does FLORE-AE-C decode in two steps without the EA-MF stage? … Please clarify how, and whether the 4096x row in Table 1 is directly comparable to the 2048x row.

FLORE-AE-C is also adapted with EA-MF and, like FLORE-AE, decodes with two updates on the grid (0, 0.5, 1), as Sec. 4.1 states for both models: "EA-MF fine-tunes only the waveform decoder for 60,000 updates and uses the grid (0, 0.5, 1)." The statements in App. B.5 and B.6.2 that FLORE-AE-C does not use this adaptation are typos, which we have corrected in the revised manuscript. Each row is an operating point compared with the baselines at its own temporal compression, and FLORE-AE-C is compared with SAME-L at an identical interface (4096×, D=256D=256).

References

[2] Evans et al., "Stable Audio Open." arXiv:2407.14358, 2024.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer 1fp4,

Thank you again for your detailed review. You mentioned that you would be open to raising your score "if the matched-size comparison and a matched adversarial baseline support the main claims." We have added both:

  • Matched latent density (Table 17): FLORE-AE at the latent densities of the baselines, including the 2048×, D=64D=64 and 4096×, D=256D=256 comparisons you specified, with paired 95% intervals.
  • Matched adversarial decoder (Table 18): a spectral–adversarial decoder trained on the frozen FLORE-AE encoder with the same data and decoder size.
  • Listening test (Table 19): paired MOS statistics over the same 60 items.

We would be grateful to know whether these address your concerns, and we are happy to clarify anything further.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Key Clarifications, New Results, and Revisions

We sincerely thank all reviewers for their constructive feedback and for recognizing the practical value of non-adversarial waveform decoding, which matches or exceeds spectral–adversarial systems on reconstruction (1fp4, x5aV), the well-controlled analyses that support its key mechanisms (1fp4, 88KT, x5aV), the alignment result as an actionable mechanism for the latent-audio community (x5aV), and the clear motivation of the work (88KT). We summarize the main clarifications and new results below; all revisions are marked in blue in the revised manuscript.

  • FLORE-AE makes source fidelity a design target of generative autoencoding, not a by-product of generative decoding. In the generative autoencoders raised by Reviewer 88KT, the latent is trained through the generative objective alone and fidelity is left to the decoder. FLORE-AE instead shapes both what the latent retains and how the decoder uses it, through three mechanisms: carrier–amplitude factorization normalizes the waveform locally and has the latent recover the removed dynamics; FLORE-DiT gives every synthesis position time-indexed access to the latent at every resolution; and EA-MF supervises the waveform that the two-step decoder returns. The closest variants of the normalization, conditioning, hierarchical, and few-step methods raised by Reviewers 88KT and x5aV are the baselines of our controlled analysis, and each mechanism improves on them (Tables 4, 10, 11); App. A now summarizes these distinctions. FLORE-AE thereby leads the evaluated spectral–adversarial and generative autoencoders in reconstruction at a latency comparable to single-pass decoders: it is an alternative to the spectral–adversarial paradigm, not a refinement of prior generative autoencoders (see 88KT R1; x5aV R1).

  • The advantage holds at matched latent density and on a shared latent. FLORE-AE keeps Stable Audio Open's temporal rate, which sets the cost of latent generation, and spends its added scalars on width; Table 1 now lists every system's density. At the baselines' densities, FLORE-AE variants trained with the same recipe still lead (Table 17), and on FLORE-AE's frozen encoder, with the same data and decoder size, the diffusion decoder leads a spectral–adversarial decoder trained with the released Stable Audio recipe in waveform, spectral, and distributional fidelity (Table 18). Two measures of latent usage, computed identically across systems, are in Table 20 (see 1fp4 R1–R2; 88KT R2).

  • The main differences are far larger than training-seed and decoder-noise variation. All systems are scored on identical recordings and listening items, so every comparison is paired. App. G adds a second training seed and further decoder-noise seeds (Table 21) and paired confidence intervals for SI-SDR and the listening test (Tables 17–19), none of which includes zero (see 1fp4 R3; 88KT R3; x5aV R2).

  • In downstream generation, the autoencoder is the only difference between rows. On AudioCaps, one generator architecture is trained on every interface with the same captions, seed, and update budget, and only its input and output projections follow the latent width (Sec. 4.1, App. D.1). With a second generator family on frozen interfaces (App. D.7.2), FLORE-AE again has the best FD and CLAP at the full training budget (see 1fp4 R3; 88KT R3; x5aV R6).

  • The hierarchy's saving holds for the complete decoder, and EA-MF holds on a second generator. The saving comes entirely from running the same inner blocks on coarser grids and remains substantial when every decoder component is counted, and the RTF covers the whole decoding call (App. D.4.2, D.6). EA-MF again gives the lowest FAD on the latents of a second, music generator (Sec. 4.3, App. D.3). Training minimizes a single objective without a discriminator and is no more expensive than published spectral–adversarial pipelines (App. B.6.2), and FLORE-AE-C uses the same two-step EA-MF decoding as FLORE-AE (App. B.5, B.6.2) (see 1fp4 R4; 88KT R4; x5aV R3, R5).

Revisions: Table 1 (scalar density); App. A (summary of distinctions from prior work); App. B.5 and B.6.2 (FLORE-AE-C adaptation and solver; training cost); Sec. 4.1 and App. D.1 (generator architecture); Sec. 4.3 and App. D.3 (EA-MF on a second generator); App. D.4.2 (complete-decoder FLOPs); App. D.5 (local residual envelope); new App. G (Table 17: FLORE-AE at lower latent densities, with paired intervals; Table 18: spectral–adversarial decoder on the frozen FLORE-AE latent; Table 19: paired listening-test statistics; Table 20: latent usage; Table 21: training-seed and decoder-noise variation).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.