acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Convolution for Flow Matching-Based Text-to-Speech

Abstract

Flow Matching (FM) is widely used in the field of non-autoregressive text-to-speech (NAR-TTS). For FM-based NAR-TTS, a vector field (VF) needs to be predicted from the current sample state, conditioned on text, speaker information, and the timestep, to transport noise toward a Mel-spectrogram. In the current works, most researchers predict the VF using Transformer, where feed-forward network (FFN) is implemented via 1D convolutional (Conv1D) layers. However, when the training is completed, the kernel weights of Conv1D layers are fixed and cannot adaptively change with different inputs. This leads to high similarity across different frequency bands in the generated spectrograms and the lack of frequency details. To this end, we propose an Adaptive Convolution (AdaConv) block consisting of two components: Cross-Position Kernel Aggregation (CPKA) and Adaptive Gated Kernel Modulation (AGKM). CPKA learns multiple kernel-weight sets with different frequency responses and aggregates them across kernels and positions using Conv1D, thereby increasing kernel weight diversity. As the input evolves across time steps, AGKM computes position-wise gates from the current input to adaptively reweight the contribution of each position. It then extracts global information from the reweighted input and uses it to modulate the aggregated convolutional kernel weights. In this way, the AdaConv block makes the convolutional kernel weights adaptive to the input and helps generate Mel-spectrograms with more frequency details. Then, the AdaConv block is used to implement the FFN in Transformer to predict the VF. Experimental results demonstrate that our method improves both objective and subjective metrics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
text-to-speech, speech generation, flow matching

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yfZEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AdaConv, an adaptive convolutional feed-forward network (FFN) designed to mitigate over-smoothing and inter-band spectral similarity in flow-matching TTS vector-field predictors. AdaConv combines CPKA, which fuses learned kernels in the discrete Fourier transform domain, with AGKM, which adaptively scales the fused kernel using a pooled and gated representation of the input. Experiments on Matcha-TTS and ZipVoice across three datasets show consistent improvements in word error rate (WER) while maintaining reasonable real-time factors (RTF).

Critical strengths

• Addresses a relevant problem in TTS generation.\textbf{Addresses a relevant problem in TTS generation.} The paper tackles over-smoothing directly within the vector-field predictor, offering a potentially complementary approach to existing loss-based and post-processing methods.

• Demonstrates consistent WER improvements\textbf{Demonstrates consistent WER improvements}. The proposed method achieves lower WER across multiple backbones, datasets, and zero-shot evaluation settings compared with static Conv1D and dynamic convolution (DConv) baselines.

• Includes component-wise ablations and efficiency evaluations.\textbf{Includes component-wise ablations and efficiency evaluations.} The ablation study in Table 4 examines the contributions of CPKA and AGKM, while Table 6 reports CPU and GPU RTFs to assess computational efficiency.

Critical weaknesses

W1: Mismatch Between Evaluation Metrics and Claims\textbf{W1: Mismatch Between Evaluation Metrics and Claims}

• WER is the only metric that demonstrates consistent and substantial improvements. The reported results provide limited evidence of corresponding gains in subjective or objective perceptual quality, such as UTMOS and SMOS, making it difficult to establish that the proposed method improves speech quality more broadly.

• The ground-truth (GT) speech exhibits a higher WER than several generated outputs. This unexpected result complicates the interpretation of WER improvements and weakens the claim that lower WER directly demonstrates better recovery of spectral details, since ASR-based metrics can be affected by factors beyond spectral fidelity and perceptual quality.

W2: Insufficient Empirical Evidence for Reduced Inter-Band Similarity\textbf{W2: Insufficient Empirical Evidence for Reduced Inter-Band Similarity}

• The claim that AdaConv reduces inter-band similarity is supported by only four utterances from a single model pair in Fig. 5(a), which is insufficient to establish the generality of the observed effect.

• Inter-band similarity statistics are not reported across the full test set. Furthermore, the analysis does not cover alternative backbones (Conformer and Zipformer), the DConv baseline, or the ablation variants in Table 4. Consequently, the evidence does not establish whether the proposed method consistently reduces inter-band similarity or whether this effect explains the observed WER improvements.

W3: Lack of Evidence for Time-Step Adaptivity\textbf{W3: Lack of Evidence for Time-Step Adaptivity}

• The motivation for AdaConv relies on changes in the vector-field input across flow-matching integration steps (Section 3.1.2). However, the paper does not empirically demonstrate how the learned gates in Eq. (5) or the kernel modulation tensor in Eq. (7) evolve over these steps.

• Since the FFN inputs already incorporate time embeddings, it remains unclear whether the proposed input-dependent modulation provides benefits beyond conventional time-conditioning mechanisms. The paper would be strengthened by comparisons with simpler alternatives, such as adaptive layer normalization (adaLN), commonly used in diffusion transformer architectures.

W4: Architectural Confounding and Incomplete Mathematical Specifications\textbf{W4: Architectural Confounding and Incomplete Mathematical Specifications}

• CPKA operates in the frequency domain and aggregates information across sequence positions, potentially changing the temporal receptive field and effective modeling capacity relative to standard Conv1D layers. The paper does not clearly specify whether the Conv1D baseline uses a matched kernel size or otherwise controls for differences in temporal modeling capacity. Therefore, it is difficult to determine how much of the observed improvement stems from adaptivity rather than increased or altered temporal context.

• CPKA does not contain input-dependent terms and therefore appears amenable to reparameterization as a single static kernel at inference time. The paper does not clarify whether this reparameterization is performed or quantify its implications for inference efficiency. In addition, the output dimensionality of the fully connected layer in Eq. (7) is not specified, and Eq. (9) lacks sufficient detail regarding padding and channel-wise summation.

Questions for authors

1. Relationship between inter-band similarity and WER\textbf{1. Relationship between inter-band similarity and WER}

What is the theoretical or empirical relationship between inter-band spectral similarity and WER? Could the authors report the correlation between these metrics across the full test set? How do they explain the observation that the ground-truth audio has a higher WER than several generated outputs, and what implications does this have for interpreting WER as evidence of improved spectral fidelity?

2. Comprehensive evaluation of inter-band similarity\textbf{2. Comprehensive evaluation of inter-band similarity}

Could the authors report inter-band similarity statistics, including the mean and standard deviation of the distance to GT, across the entire test set? Please include all evaluated backbones, the ablation variants in Table 4, the DConv baseline, and parameter-matched baselines to establish whether the observed reduction is consistent and attributable to the proposed method.

3. Time-step-dependent behavior and alternative conditioning mechanisms\textbf{3. Time-step-dependent behavior and alternative conditioning mechanisms}

Do the gating values in Eq. (5) and the kernel modulation tensor in Eq. (7) change systematically across vector-field ODE integration steps? Could the authors quantify this behavior and compare AdaConv empirically with a baseline employing direct time-step modulation, such as adaLN, to isolate the benefits of the proposed adaptive mechanism?

3 comments

We appreciate the reviewer's recognition of our approach, experimental results, and computational efficiency analysis. The concerns and questions are addressed below.

Weaknesses:

Response to W1: WER measures content fidelity, while UTMOS, CMOS, and SMOS evaluate perceptual quality or speaker similarity. As shown in Table 2, AdaConv consistently reduces WER while generally maintaining comparable perceptual quality. We do not directly equate lower WER with better perceptual quality or spectral fidelity.

GT speech contains natural pauses and pronunciation variations, which affect ASR recognition. In contrast, generated speech often has clearer and more consistent pronunciation, resulting in lower WER. Therefore, lower WER does not necessarily indicate better perceptual quality or spectral fidelity than GT.

Response to W2: We conducted additional inter-band similarity analysis across the full test sets, as detailed in App. D.3. Figs. 8 and 9 show that AdaConv achieves the lowest mean inter-band similarity and distance to GT across Transformer, Conformer, and Zipformer backbones, compared with Conv1D, DConv, and the ablation variants.

Response to W3: Our motivation differs from direct timestep conditioning. AdaConv aims to adapt the convolutional kernel weights to the current input rather than directly to the timestep. As the VF input changes across ODE integration steps, AGKM adjusts the input and kernel weights accordingly. As shown in Tab. 4, AGKM improves WER while keeping the original timestep conditioning unchanged.

Response to W4: CPKA operates on frequency-domain kernel weights rather than the input sequence. AdaConv uses the same kernel size as Conv1D (k=3k=3, padding=1), so CPKA does not increase the local temporal receptive field. CPKA is input-independent and can be precomputed at inference time, while AGKM remains input-dependent. Table 6 reports the inference efficiency of AdaConv, but the speedup from CPKA reparameterization has not been separately measured. We have clarified the FC output dimension and broadcast axes in Eq. (7), as well as padding and input-channel summation in Eq. (9), in the updated Secs. 3.1.1 and 3.1.2.

Questions:

Response to Q1: Lower inter-band similarity is associated with clearer frequency details in Mel-spectrograms, making speech content more distinguishable, as illustrated in Figs. 2 and 13. GT speech contains natural pauses and pronunciation variations that affect ASR recognition, while generated speech often has clearer pronunciation, resulting in lower WER.

Response to Q2: As shown in Figs. 8 and 9 of App. D.3, we report the mean and standard deviation of inter-band similarity and distance to GT over the full test sets, covering all three backbones, Conv1D, DConv, the ablation variants, and the parameter-matched baseline. AdaConv consistently achieves the lowest values across all three backbones.

Response to Q3: Our motivation differs from direct timestep modulation. The gating coefficients in Eq. (5) and kernel modulation coefficients in Eq. (7) are computed from the current input at each ODE integration step, rather than directly from timestep embeddings. Unlike adaLN, AGKM adapts both the input and convolutional kernel weights based on the current input features. As shown in Tab. 4, AGKM improves WER while keeping the original timestep conditioning unchanged, demonstrating the benefits of input-dependent modulation.

W1\textbf{W1}: I acknowledge the authors’ clarification regarding the distinction between WER and perceptual metrics (UTMOS, CMOS, and SMOS). My remaining concern is the interpretation of WER relative to the ground-truth (GT) speech. If natural pauses and pronunciation variations in the GT are responsible for its higher WER compared with some synthesized speech, this suggests that WER may be strongly influenced by ASR-specific factors rather than reconstruction fidelity. An analysis of where the ASR errors occur (e.g., particular phonemes or temporal regions), or verification using an alternative ASR model, would help clarify this issue and support a more careful interpretation of the reported WER improvements.

W2\textbf{W2}: Resolved. The additional full-test-set analysis substantially addresses my concern regarding the generality of the inter-band similarity results.

W3\textbf{W3}: My original question was intended to clarify the role of AGKM specifically within the flow-matching (FM) process. I understand that AGKM performs input-dependent modulation rather than direct timestep conditioning. However, the response still does not fully explain "why adapting the input and convolutional kernel according to the evolving vector-field state is particularly beneficial for flow-matching-based generation", beyond demonstrating an empirical WER improvement. A clearer explanation of the connection between the evolving FM trajectory and the proposed adaptive mechanism, and why conventional convolution or standard timestep conditioning may be insufficient, would strengthen the motivation for AGKM.

W4\textbf{W4}: Resolved. The clarification that CPKA operates on the kernel weights without increasing the temporal receptive field, together with the additional implementation details and efficiency discussion, addresses my main concerns.

Overall\textbf{Overall}: I appreciate the authors’ efforts to address my comments and provide additional analyses. Several of my original concerns, particularly those regarding inter-band similarity and the implementation of CPKA, have been adequately addressed. My main remaining concern is the interpretation and justification of the proposed method. While AdaConv consistently improves WER and the additional experiments provide stronger evidence that it reduces inter-band similarity, the connection between these observations and the flow-matching generation process remains insufficiently established. In particular, it would be helpful to explain more clearly why input-dependent adaptation of the convolutional kernels is expected to benefit the evolving vector field during ODE integration, rather than simply demonstrating that the architectural modification improves the final WER.

We appreciate the reviewer's acknowledgment of our previous clarifications. To further address the remaining concerns, we provide additional analyses in Apps. D.4 and D.5. The latest manuscript has been revised accordingly.

Response to W1: To further verify the WER improvements, we evaluate the generated speech using Canary-1b-v2 in addition to Whisper-large-v3 (App. D.4, WER Evaluation with Different ASR Models). AdaConv achieves the lowest WER for both Matcha-TTS and Matcha-TTS (SFM) under both ASR models, although the absolute WER values differ. GT speech also has higher WER than all generated outputs in both evaluations, suggesting that ASR-related factors can affect WER. Those results support AdaConv's improvements in speech content accuracy across different ASR models, while indicating that WER does not directly measure reconstruction fidelity.

Response to W3: In FM, the VF predictor estimates how to move the current state toward the target speech. As the predicted VFs are integrated over ODE steps, the generated state gradually contains more speech information, changing the input features of the VF predictor. As shown in App. D.5, the mean input gating value decreases while its standard deviation increases, suggesting more selective feature processing. The mean absolute changes between adjacent steps also decrease, indicating more stable gating in later steps. Meanwhile, the mean kernel modulation coefficient decreases from 0.204 to 0.150. Those results suggest that AGKM adjusts input gating and convolutional kernels as the input evolves, allowing the VF predictor to adapt to the current input features beyond timestep conditioning alone.

Reviewer_3N2BOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes an Adaptive Convolution block to enhance the functionality of transformer for TTS. Experimental results demonstrate that our method improves both objective and subjective metrics.

Critical strengths

The paper is well motivated and easy to follow.

Critical weaknesses

Flow matching is hard to applied real time TTS due to multi-step in inference. The most important question for TTS is whether it sounds like a real human. Unfortunately the focus of this paper doesn't show any help for these issues.

Questions for authors

No

1 comment

We thank the reviewer for emphasizing the importance of inference speed and speech naturalness in TTS.

 Weaknesses: We acknowledge that multi-step inference remains a challenge for FM-based TTS. However, AdaConv aims to improve acoustic modeling rather than reduce the number of inference steps. As shown in Table 6, AdaConv maintains GPU RTFs below 1 across all three backbones, with only moderate inference overhead compared with Conv1D. Regarding speech naturalness, Table 2 shows improvements in CMOS across several settings, while UTMOS generally remains comparable to the baselines. Those results indicate that AdaConv improves speech intelligibility while largely maintaining perceptual quality.

Reviewer_4UdFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper targets missing frequency detail in flow matching text-to-speech, where a network predicts the vector field that turns noise into a Mel-spectrogram, conditioned on text and speaker information.
  • It argues that the 1D convolutions in this network's feed-forward layers have fixed weights after training, which makes generated spectrograms too similar across frequency bands.
  • It proposes AdaConv, a drop-in replacement with two parts. CPKA combines several learned kernels in the frequency domain into one kernel. AGKM uses the current input to gate features by position and to scale the combined kernel.
  • The authors add AdaConv to Transformer, Conformer and Zipformer models, for speech synthesis with speakers seen in training and with unseen speakers.
Critical strengths
  • Evaluation across backbones and datasets: The block is evaluated in three backbones on LJSpeech, VCTK and LibriTTS, for seen-speaker and zero-shot TTS. It gives lower WER than both Conv1D and DConv in all nine settings in Table 2, so the gains are not tied to one model or dataset.
  • Ablations and figures: Table 4 evaluates the gain from each component on three backbones, using CPKA alone (AdaConv −AGKM), AGKM alone (Conv1D+AGKM) and both. Each lowers WER on its own, and the combination gives the lowest. Figures 1 and 2 show the band similarity problem against ground truth, and Figures 3 and 4 show the block and where it sits.
Critical weaknesses
  • Gains not separated from model size: The AdaConv block has 10.9× the parameters of the Conv1D it replaces, and there is no Conv1D baseline of matched size. AdaConv with N = 4 (5.6×) gives lower WER than DConv (8.1×), but both are larger than Conv1D, and in Table 3 WER falls as N and the parameter count grow. The performance gains over Conv1D can't be separated from the added parameters.
  • Core method not reproducible from the paper: Eq. 9 does not say how a kernel with k/2+1 frequency bins is applied to an input with T/2+1 bins, whether the result is a circular convolution, or how input channels are summed. The kernel size k, the kernel size and axes of the Conv1D in Eq. 4, the FC output sizes in Eqs. 5 and 7, and the broadcast axes in Eq. 7 are not given. The reproducibility statement points to the appendix, but the appendix does not list them.
Questions for authors
  • Method details (clarification): Please state how the kernel and input transforms in Eq. 9 are matched in length, whether padding is used to avoid circular convolution, and how input channels are summed. Please also give k, the kernel size and axes of the Conv1D in Eq. 4, the FC output sizes in Eqs. 5 and 7, and the broadcast axes in Eq. 7 for each backbone.
  • Parameter-matched baseline (suggestion): Please train a Conv1D baseline with a wider FFN, matched to AdaConv's parameter count, using Matcha-TTS (SFM) with the Transformer backbone on LJSpeech, as in Tables 3 and 4. Please also report total model parameters for every variant in Tables 3 and 4.
  • Separating AGKM (suggestion): AGKM applies two operations, input gating (Eq. 5) and kernel modulation (Eqs. 6 to 8), and Table 4 only tests them together. Section 4.2 credits the gains to this combination. Please run two more variants in the same LJSpeech setting: AdaConv with only input gating, and AdaConv with only kernel modulation computed from the ungated input.
3 comments

We thank the reviewer for recognizing the consistent improvements across multiple backbones and datasets. Below, we address your concerns and questions.

Weaknesses:

Response to W1: We conducted additional experiments using Matcha-TTS (SFM) with the Transformer backbone on LJSpeech, as detailed in App. D.1. We increased the FFN expansion ratio, defined as the ratio of the FFN hidden dimension to the model dimension, from 2 to 23 to approximately match the parameter count of AdaConv. As shown in Tab. 7, the Conv1D baseline reaches 69.6M parameters with a WER of 1.659. In comparison, AdaConv achieves a lower WER of 1.393 with fewer parameters (68.6M), corresponding to a 16.0% relative reduction. Those results indicate that the improvements of AdaConv are not solely due to increased model parameters.

Response to W2: We have clarified the implementation details of Eqs. (4)–(9) in the updated Secs 3.1.1 and 3.1.2. 

In Eq. 4, CPKA uses Conv1D with NN input channels, one output channel, and a kernel size of 3. It operates along the last dimension of the rearranged kernel weights, with shape (CoutCin,N,2(⌊k/2⌋+1))(C_{\mathrm{out}}C_{\mathrm{in}},N,2(\lfloor k/2\rfloor+1)), to aggregate kernel weights across kernels and positions. 

In Eq. 5, the FC output dimension is CinC_{\mathrm{in}}. In Eq. 7, the FC output dimension is ⌊k/2⌋+1\lfloor k/2\rfloor+1. The output is reshaped to (1,1,⌊k/2⌋+1)(1,1,\lfloor k/2\rfloor+1) and broadcast along the CoutC_{\mathrm{out}} and CinC_{\mathrm{in}} dimensions to obtain MM. 

In Eq. 9, we use an FFT length L≥T+k−1L \geq T+k-1 to avoid circular aliasing. The adjusted input GxG_x is zero-padded and transformed into G^x\widehat G_x using an LL-point FFT, while the modulated frequency-domain kernel weights W^\widehat W are mapped to the same frequency bins through exact spectral interpolation. For each output channel, the element-wise products of G^x\widehat G_x and W^\widehat W are summed over all CinC_{\mathrm{in}} input channels before applying the inverse FFT to obtain YY. The output is then extracted according to the original convolution settings. 

For Transformer, Conformer, and Zipformer, AdaConv uses a kernel size of 3, padding of 1, and bias enabled.

Questions:

Response to Q1: We have added the implementation details to the updated Secs 3.1.1 and 3.1.2.

In Eq. 9, we use an FFT length L≥T+k−1L \geq T+k-1 to avoid circular aliasing. The adjusted input GxG_x is zero-padded and transformed into G^x\widehat G_x using an LL-point FFT, while the modulated frequency-domain kernel weights W^\widehat W are mapped to the same frequency bins through exact spectral interpolation. For each output channel, the element-wise products of G^x\widehat G_x and W^\widehat W are summed over all CinC_{\mathrm{in}} input channels before applying the inverse FFT to obtain YY. The output is then extracted according to the original convolution settings.

In Eq. 4, CPKA uses Conv1D with NN input channels, one output channel, and a kernel size of 3. It operates along the last dimension of the rearranged kernel weights, with shape (CoutCin,N,2(⌊k/2⌋+1))(C_{\mathrm{out}}C_{\mathrm{in}},N,2(\lfloor k/2\rfloor+1)), to aggregate kernel weights across kernels and positions.

In Eq. 5, the FC output dimension is CinC_{\mathrm{in}}. In Eq. 7, the FC output dimension is ⌊k/2⌋+1\lfloor k/2\rfloor+1. The output is reshaped to (1,1,⌊k/2⌋+1)(1,1,\lfloor k/2\rfloor+1) and broadcast along the CoutC_{\mathrm{out}} and CinC_{\mathrm{in}} dimensions to obtain MM.

For Transformer, Conformer, and Zipformer, AdaConv uses a kernel size of 3, padding of 1, and bias enabled.

Response to Q2: We conducted additional experiments using Matcha-TTS (SFM) with the Transformer backbone on LJSpeech, as detailed in App. D.1. We increased the FFN expansion ratio, defined as the ratio of the FFN hidden dimension to the model dimension, from 2 to 23 to approximately match the parameter count of AdaConv. As shown in Tab. 7, the Conv1D baseline reaches 69.6M parameters with a WER of 1.659. In comparison, AdaConv achieves a lower WER of 1.393 with fewer parameters (68.6M), corresponding to a 16.0% relative reduction. Those results indicate that the improvements of AdaConv are not solely due to increased model parameters. Tab. 8 in App. D.1 further reports the total model parameters for all variants in Tabs. 3 and 4.

Response to Q3: We conducted additional experiments using Matcha-TTS (SFM) with the Transformer backbone on LJSpeech, as detailed in App. D.2. We evaluate input gating and kernel modulation separately, with kernel modulation using global information from the original input XX when input gating is removed. Using only input gating or kernel modulation achieves WERs of 1.471 and 1.509, compared with 1.539 when AGKM is removed. The complete AdaConv block achieves the lowest WER of 1.393, confirming the effectiveness of both operations and their combination.

Dear Authors,

Thank you for taking the time to carefully respond and address the issues in the paper.

W1/Q2 - Addressed.

Q3 - Addressed.

W2/Q1 - Mostly addressed, but the block kernel size k isn't stated in the paper.

Presentation issues:

You said "weights are mapped to the same frequency bins through exact spectral interpolation", but the paper says "aligned using an FFT length L". Also, the added Conv1D sentence was pasted into the middle of another sentence, and the definition after Eq. 4 repeats "where C(·) is a Conv1D layer" twice. Eqs. 1–4, 7 and 8 still use k/2+1, which should use the floor, ⌊k/2⌋+1.

The main quality issues in the paper have been addressed, but the gains are mostly in WER and small in absolute terms. The title and abstract are mainly about frequency detail and speech quality, but the paper does not do much to convince the reader of this. Fig. 8 shows that the inter-band similarity of AdaConv is close to that of Conv1D, within the error bars. The input-dependent kernel modulation is two scalars shared across all filters, and I feel the paper should explore this further, with different approaches, to find more convincing wins. The paper presents AdaConv as a drop-in replacement that improves speech quality, but feels more like an incremental variant of a convolution.

We sincerely thank the reviewer for carefully checking the revised manuscript and pointing out these issues. Your detailed feedback is greatly appreciated, and we apologize for the errors.

Mostly addressed, but the block kernel size k isn't stated in the paper.

Response: The model configuration in Sec. 4.1 has been updated to clarify that AdaConv uses kernel sizes of 3, 3, and 1 for Transformer, Conformer, and Zipformer.

You said "weights are mapped to the same frequency bins through exact spectral interpolation", but the paper says "aligned using an FFT length L"

Response: The description before Eq. (9) has been clarified. Input GxG_x is zero-padded (Cin×T→Cin×LC_{\mathrm{in}} \times T \to C_{\mathrm{in}} \times L), where LL is the FFT length chosen to avoid circular aliasing. The kernel spectrum W^\widehat{W} is then aligned to Cout×Cin×(⌊L/2⌋+1)C_{\mathrm{out}} \times C_{\mathrm{in}} \times (\lfloor L/2 \rfloor + 1) through exact spectral interpolation, yielding W^L\widehat{W}_L, which is used in Eq. (9).

Also, the added Conv1D sentence was pasted into the middle of another sentence, and the definition after Eq. 4 repeats "where C(·) is a Conv1D layer" twice.

Response: The duplicated text after Eq. 4 has been corrected. The revised definition is: "C(⋅)\mathcal{C}(\cdot) is a Conv1D layer with NN input channels, one output channel, and a kernel size of 3, applied along the last dimension."

Eqs. 1–4, 7 and 8 still use k/2+1, which should use the floor, ⌊k/2⌋+1.

Response: All occurrences of k/2+1k/2+1 in Eqs. 1–4, 7, and 8 have been corrected to ⌊k/2⌋+1\lfloor k/2\rfloor+1.

The main quality issues in the paper have been addressed, but the gains are mostly in WER and small in absolute terms. The title and abstract are mainly about frequency detail and speech quality, but the paper does not do much to convince the reader of this. Fig. 8 shows that the inter-band similarity of AdaConv is close to that of Conv1D, within the error bars. The input-dependent kernel modulation is two scalars shared across all filters, and I feel the paper should explore this further, with different approaches, to find more convincing wins. The paper presents AdaConv as a drop-in replacement that improves speech quality, but feels more like an incremental variant of a convolution.

Response: We acknowledge the modest absolute WER gains, overlapping error bars in Fig. 8, and limited flexibility of shared kernel modulation. However, lower WER suggests more accurate speech content and better speech intelligibility, both of which are important aspects of speech quality. The spectral analyses in Figs. 8 and 9 of App. D.3 provide further evidence of frequency detail preservation, with AdaConv achieving the lowest mean inter-band similarity and distance to GT across three backbones. Although AdaConv introduces additional parameters compared with Conv1D, the parameter-matched comparison in App. D.1 shows a 16.0% relative WER reduction (1.659 → 1.393) with fewer parameters (68.6M vs. 69.6M), suggesting that the gains are not simply due to increased model capacity.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Appreciation to the Reviewers

Dear Reviewers 4UdF, 3N2B, and yfZE,

Thank you all for taking the time to review our paper. We appreciate your positive feedback on AdaConv, especially your comments on the motivation for addressing over-smoothing in the VF predictor, the consistent WER improvements across multiple backbones and datasets, and the ablation studies and efficiency analysis. Your comments and suggestions have helped us make the paper clearer and strengthen the experimental evidence for AdaConv's effectiveness.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.