acceptodds
Under review as a conference paper at ICLR 2027

Agreeing on Allocation: Shared-Prefix Selective Adaptation for Frozen Speech Codecs

Abstract

Neural speech codecs typically allocate a uniform code budget across all frames. Adaptive frame-wise allocation can improve coding efficiency but often requires retraining or extra signaling. We introduce Agreeing on Allocation, a post-hoc method that distributes a fixed code budget in frozen codecs. Using the mandatory first-stage residual vector quantization (RVQ) codes as a shared prefix, the encoder and decoder independently recover the same budget-constrained stage assignment without frame-wise signaling. However, agreement alone does not guarantee better reconstruction. We therefore add selective deployment, in which the encoder compares proposed and reference reconstructions at the same budget using fidelity and perceptual criteria and signals only an utterance-level choice. For the fixed selection rule, held-out evaluation provides a finite-sample upper confidence bound on the probability of a quality drop beyond a predefined margin relative to the reference. Objective and subjective evaluations on frozen neural speech codecs show quality improvements at matched code budgets, while selective deployment reduces the risk of substantial quality drops.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
neural speech codecs, residual vector quantization, adaptive bit allocation, selective deployment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YZXmOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a method for dynamically modifying the number of residual vector quantization tokens sent in a neural speech codec, without requiring changing the underlying codec in any way. The dynamic token allocation computation is performed identically by the encoder and decoder, which further means that no additional bits are needed in the stream itself to indicate the number of tokens to expect.

Critical strengths

The results of the paper are extremely well-tested. A wide variety of quality measures and ablations are applied, including a round of subjective testing. Regardless of any other concerns, this paper strikes me as a model of how to do a speech paper.

Critical weaknesses

Probably the biggest issue for the paper is motivation; this seems like a very niche application.

Specifically, it's not clear why introducing an extra model (which must match and be maintained on both sides of the communication) is any easier than swapping the codec for an explicitly variable-bitrate codec (like those discussed in the related works section). (my best guess is serving lots of audio that has already been encoded with a specific fixed-rate neural codec, where re-encoding the audio at-rest is too expensive?)

Questions for authors
  • I'm curious how the results here compare with the language model + entropy coding approach in the Encodec paper: https://arxiv.org/pdf/2210.13438 Their entropy coding can similarly be applied to any frozen token stream to reduce the overall bitrate (though no actual RVQ frames will be dropped).

  • I'm also curious (again, motivation) about the impact of the bitrate alloctation algorithm on stream latency. IIUC, both sides need to look at the top token from a set of dd frames and run an algorithm to determine which tokens to drop. This would seem to introduce a dd-frame delay to the entire system.

1 comment

Thank you for raising the deployment question and for suggesting the entropy-coding comparison. The section and table numbers below refer to the revised manuscript.

Weakness; Why retrofit an existing codec? We target offline batch processing, such as archival of call-center recordings, where the codec weights and codebooks must stay fixed, but both endpoints can update bitstream handling. This avoids retraining or replacing the base codec, not updating the allocator and parser; an unchanged receiver is not compatible. The archival scenario illustrates the constraint, not a deployment we tested. Previously discarded indices cannot be recovered from a low-depth archive. Revision: Secs. 1 and 3.3.

Q1. Comparison with EnCodec entropy coding: Entropy coding reduces the bits needed to transmit the same indices without changing the reconstruction; our method changes which indices are retained. On 1,000 LRAC public-test inputs, we compared fixed-width, ungated Proposal with entropy-coded, uniform-depth CBR on the same frozen DAC. The CBR arm uses arithmetic coding with a per-stage causal bigram fitted on training data, not EnCodec’s language model. At CBR depth 2, the bigram lowers the measured rate from 1.521 to 1.426 kbps (6.3%). At 1.521 kbps, Proposal gains 0.099 PESQ and 0.065 ViSQOL over entropy-coded CBR interpolated to the same rate. Rates are computed from total serialized bits, including headers, divided by total audio duration. Each paired bootstrap draw recomputes the rates and interpolation without extrapolation. At CBR depths 4 to 8, this bigram increases bitrate relative to fixed-width packing, so the higher-rate results do not establish an advantage over stronger entropy models. (Secs. 2 and 4.3; App. D.6, Table 20.)

Q2. Lookahead and stream latency: Mean RVQ depth sets the index budget, not the number of frames to buffer. The receiver needs the full first-stage prefix of the utterance before it can recover the allocation and parse later indices. Selective also compares two complete reconstructions at the sender. The current method therefore requires whole-utterance buffering and is intended for offline use. In the DAC depth-4 control, median receiver coordination takes 422 ms for Proposal and 411 ms for Ref fallback, excluding buffering and waveform decoding. These are component timings, not end-to-end streaming latency. (Secs. 3.3 and 4.3; App. E.2, Table 27.)

Reviewer_6CLqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper reallocates per-frame bit budgets inside frozen RVQ neural speech codecs, where the number of RVQ stages per frame is normally fixed at codec training time even though quiet regions need only stage 1 and difficult regions benefit from deeper allocation; post-hoc reallocation such as BAMU improves quality but must transmit a frame-wise allocation map, because the decoder needs the allocation to parse later-stage indices, so the paper asks whether encoder and decoder can agree on the same allocation without sending that map. Its first contribution treats the stage-1 RVQ codes that every frame transmits anyway as a shared prefix: a learned allocator ensemble (M=3) estimates frame-wise utilities from it and a deterministic dynamic program finds the allocation maximizing total utility under a sum d_t = B budget, which both ends run independently and which therefore costs zero bytes to signal. Its second contribution deploys that allocation selectively, with the encoder locally reconstructing both the proposal and the codec-specific reference allocation and keeping the proposal only when a fidelity criterion against the encoder input and a no-reference perceptual criterion both favor it, conveying the choice in a single mode bit and falling back to the reference otherwise so every input stays decodable, while risk is defined at the cluster level (the worst acoustic condition within an utterance) and reported as a one-sided Clopper-Pearson upper bound on the population risk R, which is unconditional on acceptance (2.94-5.16% per setting over 1,024 clusters) and fixed before the confirmation cohort is drawn. Validation spans operating points of DAC, SpeechTokenizer, and PhoenixCodec, with Table 1 matching exactly B = T_d RVQ indices and the coordination and alternative-adaptation comparisons matching serialized bytes against CBR, the codec-specific Ref, BAMU, a Gogo-based SP-Count, Encoder-LoRA, matched-backbone VRVQ, and FlexiCodec, and it reports positive PESQ gains over uniform and reference allocations, PESQ-drop events falling from 248-392 to 21-41 among 1,024 clusters, listening-score gains over Ref of +2.47 (DAC), +2.18 (SpeechTokenizer), and +2.51 (PhoenixCodec) that the paper itself labels descriptive-only, and 24.6-35.2 bytes of map savings against 316.54-508.88 ms of receiver coordination.

Critical strengths

The paper reallocates RVQ depth frame-wise inside a frozen codec without transmitting any side information. The mandatory stage-1 codes serve as a shared prefix from which encoder and decoder each deterministically derive the same allocation, so the frame-wise map that post-hoc methods such as BAMU must send costs zero bytes, and because nothing is retrained the method drops onto already-deployed codecs. I weigh this most heavily because the map is what limits post-hoc reallocation in practice: it eats into the savings the reallocation creates, so eliminating it is what makes frame-wise allocation viable at these budgets at all. The nearest cited methods hold only half of the construction — VRVQ and UniStream train the codec itself (frame-wise depth, and a streaming codec, respectively), while the remaining allocation work either varies temporal resolution (frame selection, merging, clustering) or sends explicit side information (keep masks, segment lengths, durations).

The reallocation is gated per utterance before it is used. The encoder computes both the proposal and the reference reconstruction, compares them on a fidelity criterion against the encoder input and a no-reference perceptual criterion, and keeps the proposal only if both favor it; the comparison happens inside the encoder, so gating costs no transmitted bits, a single mode bit tells the decoder which allocation to parse, and a failed gate falls back to the codec-specific reference so that every input stays decodable. The event is defined at the cluster level (the worst acoustic condition within an utterance), and the corresponding one-sided Clopper-Pearson bound (2.94-5.16% per setting over 1,024 clusters, computed on clusters not used to select the thresholds) is a finite-sample statement about R that assumes only i.i.d. clusters. I weigh this heavily because it turns a per-frame reallocation from a best-effort quality knob into something carrying a stated utterance-level failure probability, which is what a deployer of a frozen codec would need before enabling it.

Critical weaknesses

The matched-budget quality gain, which is the basis on which the paper asks to be accepted, is measured on a benchmark the paper itself calls descriptive. Table 1 evaluates on 1,000 LibriSpeech test-clean utterances, and Section 4.1 and C.1 state that source metadata identifies shared chapters with the LRAC source pool, so this benchmark is descriptive and separate from the held-out risk evaluation. PESQ then carries the whole pipeline: it scores the candidate-bank (A.2), supervises training (Eq. 17-18), selects the checkpoint (C.2), defines the risk-event margin (m = 0.05), and is the primary reported metric, while ViSQOL, the only metric that touches neither training nor selection, is negative for PhoenixCodec at d̄ ∈ {3,4,5} and spans zero in several settings. On the strictly held-out data, the deployed system's average gain is 0.003-0.020 PESQ, below the paper's own m = 0.05 margin. I weigh this heavily because it is the only uncontaminated measurement of the quantity the paper claims, and it falls under the paper's own threshold for a meaningful improvement.

The gain cannot be separated into what comes from the learned allocator and what comes from the codec-specific Ref baseline. On LRAC public, Ref already supplies 73.0% of the ungated Shared-Prefix-CBR gain (0.0775 of 0.1061, Table 14a; 16-84% across test-clean settings). Ref cannot be reimplemented from the paper (A.1 defines only Eq. 15) and may use a different solver, and the control that zeroes the learned correction, which would separate learning from a solver swap, is missing. No Proposal-CBR interval exists anywhere, so the improvement-over-CBR claim rests on ten point estimates whose weakest cells' Ref-side intervals already span zero. I weigh this heavily because the learned allocator is the paper's own contribution, and as reported its marginal contribution to the headline table is unquantified.

Questions for authors
  1. Run the Compressed-Map arm at the map-reduced budget with the exact-budget DP (Eq. 19), or run the map-free variant with the Lagrangian solver, so that the solver is held fixed: how much of Table 8's quality difference survives, and how much quality does removing the map itself buy? Please also profile the receiver cost of reference mode, which serves 78-91% of deployed inputs.
  2. Specify the reference cost predictor's construction (features, loss, fitting data, solver, penalties), and report the proposal solver applied to uncorrected reference scores. Given that Ref supplies 73% of the ungated gain over CBR on LRAC public, what is the learned allocator's marginal contribution to Table 1, and what do the Proposal-CBR intervals look like, given that the point estimates include cells whose Ref-side intervals already span zero?
  3. What ships is Selective, yet in gated form it is compared only to LoRA (Table 2(a)) and SP-Count (D.6). Please report Selective-BAMU and Selective-CBR at matched bytes on the same 600/200/200 LRAC inputs. The BAMU reproduction and its protocol already exist, so this is the gate switched on rather than a new experiment.
  4. Report the deployer-facing acceptance-conditional drop rate (lower-bound estimates ~4-11%, brackets to ~26%) alongside the unconditional 2.94-5.16% bound, add "unconditional" to the abstract and Contribution 2, and give the event counts at m = 0.1. Does the safety claim hold at a margin a deployer would actually set?
  5. Rerun Table 1 on a test-clean subset whose chapters do not overlap the LRAC source pool, which your own metadata enables: do the gains persist? PESQ is simultaneously the candidate-bank scorer (A.2), the training supervision (Eq. 17-18), the checkpoint selector (C.2), and the reported metric: what do the headline gains look like under a metric that touches none of those, and with ViSQOL negative for PhoenixCodec at d̄ ∈ {3,4,5}, what does the certificate look like there? On strictly held-out data the deployed system's average gain is 0.003-0.020 PESQ, below your own m = 0.05 margin: at what scale should the efficacy claim be read, and does it hold for all inputs or only for accepted ones?
  6. Add a Selective condition (or an accepted-inputs-only analysis) to the listening test, report session-clustered intervals and anchor scores, and exclude hidden-reference-failing sessions with a sensitivity analysis. Does any deployed-system perceptual benefit survive, and given the ~0.2-0.55 point ceiling implied by 9-22% acceptance, is the study powered to detect it?
2 comments

Thank you for suggesting these controls. The section and table numbers below refer to the revised manuscript.

Q1. Same-solver map comparison: On 1,000 LRAC public-test inputs (DAC, depth 4), we held the utility predictor, switching penalty, and sender DP fixed. At equal serialized bytes per input, map-free Proposal gains 0.0141 PESQ [0.0125, 0.0156] and 0.0114 ViSQOL [0.0086, 0.0141] (pointwise paired 95% intervals over 949 reference-hash clusters). The map leaves fewer bytes for indices; the DP does not jointly optimize allocation and signaling. Median receiver coordination takes 422 ms for Proposal, 411 ms for Ref fallback, and 3.4 ms for RLE-map, excluding buffering and waveform decoding. (Sec. 4.3; App. D.3, Table 10; App. E.2, Table 27.)

Q2. Ref and the learned increment: Ref uses a two-stage predictor of spectral reconstruction costs from the shared prefix and rate. In the original test-clean evaluation, Ref and Proposal share the same CPU float32 exact-budget DP with switching penalty 0.1. At DAC depth 4, Ref gains 0.0965 PESQ over CBR and Proposal adds 0.03396 [0.02834, 0.04045] over Ref. Proposal gains 0.1304 [0.1140, 0.1480] over CBR; the Proposal-minus-CBR intervals are above zero in 10/10 PESQ and 9/10 ViSQOL settings. In a separate DAC depth-4 LRAC check, zero correction recovered all 1,000 Ref depth sequences. In the archived ungated LRAC comparison, Ref adds 0.0775 over CBR and Shared-Prefix adds 0.0286 over Ref, from the Table 18(a) means. (Sec. 4.2, Table 1; App. A.1; App. D.1, Tables 6 and 7; App. D.3; App. D.6, Table 18(a).)

Q3. Deployed Selective versus BAMU and CBR: On the same 600 clean, 200 noisy, and 200 reverberant LRAC inputs, all arms match serialized bytes per input, including maps and padding. Selective gains 0.0599 PESQ [0.0535, 0.0667] and 0.0113 ViSQOL [0.0028, 0.0199] over the mean of three matched-data BAMU runs. It gains 0.0829 PESQ over CBR and 0.0061 over Ref, with 76 of 1,000 inputs accepted. Thus, Ref accounts for most of the improvement over CBR. (Sec. 4.3; App. D.6, Table 16.)

Q4. Conditional drops and failed scores: On held-out 1.2-s clips, treating accepted clips with failed PESQ scoring as possible drops yields pointwise 95% upper bounds of 3.28%–5.49% on unconditional cluster-level risk under the stated sampling assumptions; these are not bounds conditional on acceptance. At DAC depth 6, this raises the upper bound from 5.16% (41 recorded drop clusters) to 5.49% (at most 44). Table 5 reports both counts and bounds for all ten settings (1,024 clusters each), together with mean gains over all 3,072 clips—assigning zero gain to failed pairs—and over the 3,063 valid pairs only. Among accepted clips with valid scores, 5.15%–11.00% show a drop at m=0.05; at m=0.10, recorded drop-cluster counts are 12–21 for DAC and PhoenixCodec and 7/11/8 for SpeechTokenizer at depths 2/4/6. Tables 3, 5, and 14 all use the SpeechTokenizer rerun on the original split with the frozen gate. Acceptance counts were 547/442/301, versus 542/437/282 historically. Because PESQ is not part of the gate and the original per-clip records are unavailable, we cannot determine why acceptance changed. (Abstract; Sec. 1, Contribution 2; Sec. 3.5; Sec. 4.4, Table 3; App. B.2, Table 5; App. D.4, Table 14.)

Q5. Overlap, metrics, and effect size: We found no shared LibriVox chapter or reader IDs between test-clean and the LibriTTS fitting, checkpoint-selection, or fidelity-threshold sources. An acoustic screen of 134,802 LRAC recordings found no verified duplicates, so all 1,000 test-clean inputs were retained; broader chapter overlap or codec-pretraining exposure cannot be excluded. On reserved clean clips (1,021 valid pairs per setting), ungated Proposal−Ref PESQ intervals are above zero in 8/10 settings (exceptions: SpeechTokenizer at depths 4 and 6); this is neither a full-utterance replication nor a non-PESQ evaluation. On test-clean, STOI and ESTOI intervals are positive in 9/10 settings; PhoenixCodec ViSQOL decreases at depths 3–5, with the depth-3 interval below zero. The 0.05 margin defines a drop, not a minimum mean gain; Selective gains 0.002–0.020 PESQ over Ref across all confirmation clips (zero gain for failed pairs) and 0.024–0.151 among accepted clips with valid scores. (Secs. 4.1, 4.2, and 4.4; App. C.1; App. D.1, Tables 6 and 8.)

Q6. Perceptual evidence for Selective: The archived A/B analysis yields 34 judgments per codec on accepted stimuli (17 sessions, 10 utterances). Proposal/tie/Ref counts are 8/21/5 for DAC, 7/18/9 for SpeechTokenizer, and 9/18/7 for PhoenixCodec. All crossed intervals include zero. A post hoc hidden-reference screen retains 4 of 35 sessions; all screened ungated intervals also include zero. Neither analysis establishes an audible benefit for deployed Selective. (Sec. 4.5, Fig. 3; Apps. F.2 and F.3, Tables 28 and 29.)

Thank you for the thorough responses and additional experiments. Most of my concerns are resolved. In particular, the matched-byte comparison of Selective against BAMU and CBR (Q3), the same-solver map comparison and receiver coordination timing (Q1), the decomposition of reference vs. allocator gains (Q2), and the clearer framing of unconditional vs. conditional drop risk and failed PESQ scores (Q4).

While the overall average PESQ gains of Selective over Ref remain modest, receiver coordination adds nontrivial latency, and the listening test does not yet establish a statistically significant perceptual preference, the paper is transparent about these limitations and well-scoped for offline frozen-codec settings. I will keep my rating at Weak acceptance.

Reviewer_w8rbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a post-hoc allocation for frozen RVQ speech codecs from CBR to VBR. Mandatory first-stage codes allow the encoder and decoder to independently recover the same frame-wise allocation under a fixed utterance budget. The encoder then compares complete proposal and reference reconstructions, applying an utterance-level fallback when its quality criteria are not satisfied. This decision is utterance-level so it cannot be applied in low-latency scenarios. Experiments cover three frozen codec backbones, objective metrics, listening test, and quality-drop risk estimates.

Critical strengths
  1. The shared-prefix protocol avoids frame-wise allocation signaling.
  2. Matched-budget evaluations show gains over uniform allocation, with several useful controls and statistical analyses.
  3. The paper distinguishes allocation agreement from reconstruction quality and appropriately limits its risk bounds to the stated assumptions.
Critical weaknesses
  1. The frozen-codec constraint needs stronger justification. Existing codecs such as TF-Codec (https://arxiv.org/abs/2207.08363) already support unequal bit spending through rate-constrained coding. Although the mechanisms differ, the paper should establish when retrofitting a frozen codec is preferable to using or training a codec designed for adaptive allocation.
  2. The method is unsuitable for low-latency streaming as presented. Global allocation and comparison of complete reconstructions require whole-utterance processing. This is consistent with the stated offline scope, but limits practical applicability and requires a convincing offline use case.
  3. The bandwidth-computation trade-off is not compellingly established. Avoiding approximately 25-35 bytes of map overhead incurs 317-509ms of receiver coordination, versus below 1 ms for the compressed-map alternative. Overall training and deployment costs relative to native variable-rate codecs remain unclear.
  4. Independent perceptual validation is insufficient. As the paper uses PESQ to supervise allocator training and UTMOS participates in the selection, a convincing listening test would be very important for the quality assessment. The reported listening gains are only about 2.2–2.5 points on a 100-point scale, without uncertainty estimates. The number of unique listeners and the handling of dependence across ratings remain unclear. There are also frequent hidden-reference ratings below 90, raising response-quality concerns. The study also evaluates Proposal rather than the deployed Selective policy.
  5. Selective deployment provides limited incremental benefit. Confirmation acceptance is approximately 9%-22%, so most inputs fall back to the learned Ref allocator. The remaining PESQ improvement over Ref is about 0.003–0.020. Its practical value should be weighed against paired decoding and quality-scoring costs.
  6. The VRVQ comparison is weakened by its underperformance relative to CBR. At the lowest matched bitrate, the post-hoc controller improves PESQ by 0.2429 over VRVQ, but only 0.0558 over the paired CBR codec; VRVQ itself trails CBR by 0.1871. Thus, much of the apparent advantage over VRVQ reflects the baseline’s underperformance rather than a substantial improvement over conventional coding. The authors acknowledge this limitation, but the comparison does not convincingly establish that post-hoc adaptation is preferable to a competitive end-to-end variable-rate codec.
Questions for authors
  1. What concrete application requires retaining the existing codec weights and codebooks? What measurable advantage does post-hoc adaptation offer over an existing adaptive codec such as TF-Codec or end-to-end variable-rate training?
  2. Which use cases justify whole-utterance buffering? Please clarify the intended utterance lengths and latency requirements, and how allocation time and memory scale with utterance duration.
  3. Can you report total adaptation costs, including reference-controller fitting and candidate-bank generation, with complete sender and receiver inference costs? How much of the coordination-time difference comes from the solver rather than signaling removal?
  4. How many unique listeners participated, and what response-quality controls were specified? Please provide participant-/utterance-aware uncertainty estimates and clarify whether the deployed Selective policy, not only Proposal, has an audible advantage.
  5. Given the low acceptance rate and small incremental gain over Ref, when does Selective justify paired decoding and UTMOS evaluation? Could a simpler fidelity-only selector retain comparable benefit at lower computational cost?
  6. Why does the reproduced VRVQ underperform its paired CBR codec at low rates? Please distinguish effects of training, allocation, and signaling overhead, and clarify whether the claimed advantage persists against a competitive end-to-end variable-rate baseline.
1 comment

Thank you for highlighting the deployment and evaluation trade-offs. All section and table references below refer to the revised manuscript.

Q1. When is freezing useful? One example is offline batch archival of call-center recordings, where replacing the existing codec may be undesirable, but both endpoints can update how they handle the bitstream. Our method keeps the encoder, decoder, and codebooks fixed, but still requires fitting and distributing the allocator and updating the parser. It is not compatible with an unchanged receiver. When codec replacement is feasible and low latency matters, TF-Codec is a relevant alternative; we do not claim a matched-rate advantage over it. (Secs. 1 and 3.3.)

Q2. Buffering and scaling: The decoder needs the full first-stage prefix, and Selective compares two complete reconstructions at the encoder. The benchmarked dense DP takes O(TBK²) time. Two sweeps reduce this to O(TBK) in exact arithmetic, but we did not benchmark that version. With the budget B proportional to utterance length T and K fixed, both solvers still take quadratic time in T, and traceback storage is also quadratic. On DAC, median Proposal receiver coordination takes 316 ms for inputs from 3 to under 6 seconds (626 clips) and 1,126 ms for inputs of at least 6 seconds (374 clips). The corresponding median backpointer storage is 5.50 and 32.05 MiB, calculated from the DP dimensions rather than measured as peak memory. (Secs. 3.2–3.3; Apps. A.2 and E.2.)

Q3. Signaling and cost: The original comparison changed both signaling and the solver. The new control fixes the utility predictor, switching penalty, and sender-side DP on 1,000 DAC inputs at depth 4. The RLE map averages 36.80 bytes per input at the original index budget and 36.88 bytes after matching serialized packet sizes. At equal bytes, map-free Proposal gains 0.0141 PESQ and 0.0114 ViSQOL over the map arm. Median receiver coordination takes 422, 411, and 3.4 ms for Proposal, Ref fallback, and RLE-map, respectively, excluding buffering and waveform decoding. The Selective sender and receiver component sums have medians of 895 and 425 ms; these are not end-to-end latencies. Logged DAC adaptation stages sum to 4 h 36 min 07 s: training cache (12 min), two Ref fits (19 min 20 s), upstream labels (33 min 40 s), candidate-bank construction (3 h 26 min 08 s), and three allocator fits (4 min 59 s). This excludes base-codec training. We do not have a matched full-system cost comparison with a native variable-rate codec. (Sec. 4.3; App. D.3, Tables 9–10; App. E.2, Table 27.)

Q4. Listening: We recorded 35 completed non-QA sessions but could not verify the number of unique listeners. The mean ungated Proposal-minus-Ref differences and pointwise 95% crossed session-by-utterance bootstrap intervals are +2.47 [−1.92, 7.41] for DAC, +2.18 [−2.01, 6.75] for SpeechTokenizer, and +2.51 [−2.40, 6.86] for PhoenixCodec. The primary analysis retains sessions with low hidden-reference scores. A post hoc screen leaves only 4 of 35 sessions, and all screened intervals include zero. The separate A/B analysis on accepted stimuli also does not establish an audible benefit for deployed Selective. (Sec. 4.5; Apps. F.2–F.3, Tables 28–29.)

Q5. Fidelity-only versus the dual gate: On DAC depth-4 confirmation, fidelity-only accepts 558 of 3,072 clips and records 36 drop clusters out of 1,024. Selective accepts 392 clips and records 25 drop clusters; Table 3 reports a conservative count of 28 when scoring failures are treated as possible drops. In our reanalysis of this confirmation run, fidelity-only and Selective gain 0.0058 and 0.0040 mean PESQ over Ref, respectively, across all clips. Fidelity-only thus retains the mean gain in this setting and saves a median 30 ms in UTMOS scoring per pair, although it still reconstructs both candidates. It also accepts more clips, so the difference in drop counts does not isolate the value of UTMOS at matched acceptance. (Sec. 4.6; Table 3; App. E.2, Table 27.)

Q6. VRVQ comparison: At the lowest measured rate, including signaling, the separate spectral-supervised controller gains 0.0558 PESQ over paired CBR and 0.2429 over interpolated VRVQ. The adapted VRVQ baseline itself trails paired CBR by 0.1871 PESQ, accounting for most of the larger gap. This experiment does not separate the effects of training, allocation, and signaling, so it neither identifies the cause of that gap nor establishes superiority over published VRVQ. (Sec. 4.3, Table 2(b); App. D.6, Table 19.)

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their constructive comments. We revised the manuscript accordingly, with substantive additions and revisions marked in blue and the relevant sections and tables indicated in the text. We summarize the main changes below; detailed point-by-point responses appear under the individual reviews.

  1. Matched-budget comparisons. Section 4.3 summarizes three new experiments: a same-solver map control (Appendix D.3, Table 10), a matched-byte comparison of deployed Selective against BAMU, CBR, and Ref (Appendix D.6, Table 16), and an entropy-coding comparison on frozen DAC (Appendix D.6, Table 20). In the same-solver control, map-free allocation improves reconstruction at equal serialized bytes but requires more receiver computation. In the separate Selective comparison, Ref accounts for most of the PESQ gain over CBR.

  2. Confirmation risk. Section 3.5 explains how we treat accepted clips with failed PESQ scoring as possible drops. Section 4.4 and Table 3 report conservative pointwise upper bounds on unconditional cluster-level risk. Appendix B.2, Table 5 shows how this treatment changes the counts and bounds; Appendix D.4, Table 14 reports observed drop rates among accepted clips with valid scores. The SpeechTokenizer results in all three tables come from the same rerun on the original confirmation split.

  3. Listening evidence. Section 4.5 and Figure 3 summarize the ungated listening results. The confidence intervals for Proposal versus Ref include zero for all three codecs (Appendix F.2, Table 28). We also report a session-screening sensitivity analysis (Appendix F.3, Table 29). The separate A/B analysis on accepted stimuli (Appendix F.3) does not establish an audible benefit for deployed Selective.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.