acceptodds
Under review as a conference paper at ICLR 2027

Pushing the Limits of Sub-10M Multi-Domain Speech Recognition

Abstract

Edge speech recognition requires multi-domain robustness under strict parameter caps, yet standard uniform compression degrades deep acoustic representations. In this paper, we propose AcuFormer, the first sub-10M ASR model designed for robust multi-domain performance within a complete 9.97M parameter envelope through an acoustically informed architecture. Guided by acoustic sensitivity, AcuFormer preserves full encoder depth while reallocating released feed-forward parameters into compact bidirectional GRU decoders that reuse acoustic token embeddings, targeted local recovery modules, and a blank-invariant prior for multi-view selection. Evaluated across all seven domains of the OpenASR benchmark, AcuFormer achieves 9.70% average WER, breaking the 10% error threshold for sub-10M models, yielding a 59.0% relative improvement over the compact state of the art, and outperforming baselines up to 7.8 times its size. Furthermore, real-world hardware profiling on a low-power Raspberry Pi Zero 2 W confirms real-time execution with an RTF of 0.53 and minimal memory residency. Taken together, these results push the limits of sub-10M multi-domain speech recognition by uniting deep acoustic accuracy with extreme on-device efficiency, facilitating private, always-available voice interfaces on ubiquitous edge devices.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
automatic speech recognition, efficient neural networks, on-device inference

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_bvqGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a sub-10M ASR model, AcuFormer, for robust multi-domain ASR on edge devices. AcuFormer uses acoustic evidences to dynamically allocate the parameter budget end to end. Built on the pre-trained 13.15M Conformer-CTC-S, AcuFormer is an acoustically informed ASR architecture that combines four synergistic key mechanisms, including sensitivity-adaptive full-depth sharing to preserve backbone capacity, CTC-guided bidirectional rescoring with shared token projection, a blank-invariant acoustic prior for temporal shift stability, and targeted local recovery for feature restoration. On OpenASR, AcuFormer achieves 9.70% macro WER, substantially outperforming the baselines of much larger models.

Critical strengths
  1. Building robust and accurate very small multi-domain ASR models for edge devices is an important direction, and this work substantially improves the sub-10M multi-domain ASR performance over the previous SOTA.

  2. The proposed sensitivity-adaptive full-depth sharing to preserve backbone capacity, CTC-guided bidirectional rescoring with shared token projection, targeted local recovery for feature restoration are reasonable designs. Moreover, Table 3 shows each architectural component achieves WER reduction under single-view inference. Table 11 ablation shows that each of these three architectural designs improves most of the eight splits.

  3. Overall, AcuFormer achieves impressively strong ASR performance of 9.70% WER on the OpenASR benchmark (all 7 domains), breaking the 10% WER for sub-10M models and yielding 59% relative gain over the previous SOTA Squeezeformer-XS, and outperforming baselines up to 7.8x its size.

Critical weaknesses
  1. As shown in Table 2, AcuFormer achieved macro WER 9.70% while the previous sub-10M SOTA Squeezeformer-XS (NeMo) obtained 23.67%, the authors thus concluded the 59% relative improvement. However, Table 2 only compares the model size and peak activation, without clearly comparing the training data (amount, distribution, domain coverage). Hence, this conclusion may confound many factors, not just the architectural update.

  2. Some important ablations and comparisons are missing:

  a. The paper needs more thorough analysis on different configurations of sensitivity guidance, for example, compare uniform compression, random, or reverse sensitivity allocation at equal parameters.

  b. Similar to the above question on whether the comparisons are fair, there is no comparison to the backbone Conformer-CTC-S under the same adaptation, beam search, and LM recipe; therefore, the effect of the sensitivity-adaptive full-depth sharing is not fully analyzed.

Questions for authors
  1. Please address the points under Weaknesses.

  2. Please report the AcuFormer storage versus INT4 and INT8 Conformer-CTC-S under the same recipe.

  3. What is the RTF for confidence-routed views and all three views?

  4. Are there any scaling studies to further reduce the number of model parameters and evaluate the performance?

  5. There are no limitation discussions.

Reviewer_kYuTOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary
  • The paper starts from a pretrained 13.15M parameter Conformer. In the ten blocks where replacing the feed forward layers changes the CTC output least, each feed forward weight is stored as an 8 bit index into a table of 255 shared values, with one scale per group of weights. An adapter follows each of these blocks.
  • The saved parameters fund two GRU language models, one reading left to right and one right to left, which rescore the top 12 beam search hypotheses. Both reuse the CTC token matrix. The full system counts 9.97M parameters.
  • Training adds a loss that aligns CTC outputs across copies of the audio shifted by 5 ms in each direction. At test time, the model decodes the shifted copy with the highest CTC confidence.
  • After adaptation on AMI, GigaSpeech, LibriSpeech and TED-LIUM, the paper reports 9.70% average WER on the eight OpenASR test sets, compared with 23.67% for the best baseline under 10M parameters. The single view configuration reports 9.79% WER and an RTF of 0.53 on a Raspberry Pi Zero 2 W.
Critical strengths
  • On device results: a 9.97M parameter system with two language models reports 9.79% average WER across seven domains and an RTF of 0.53 on a Raspberry Pi Zero 2 W. Latency, memory and energy are measured on the device.
  • Reproducibility: the paper reports exact parameter and byte counts, scoring code versions, baseline reruns and error counts for each test set. Tables 6 and 12 compare individual components with other settings held fixed.
Critical weaknesses
  • The 59% comparison does not isolate the proposed methods: the baselines under 10M were trained on LibriSpeech only, while AcuFormer starts from a Conformer pretrained on several thousand hours. That Conformer scores 11.16% against their 23.67% before any of the proposed methods are applied.
  • The compression does not account for the gain over the original model: with greedy decoding, the compressed encoder scores 11.06% and the original Conformer 11.16%. The remaining 1.36 points come from beam search, GRU rescoring and view selection, which are not applied to the original Conformer. Table 3 starts from a 14.80% model built by the authors rather than the original Conformer, and its first step changes depth, width and weight format together.
  • The efficiency comparison does not include a quantized baseline: the shared table starts as 255 evenly spaced values, each group is scaled by its largest weight, and each weight is stored as an 8 bit index, which matches 8 bit group quantization with a trainable table. Storage is compared only with baselines stored as 32 bit floats. The original Conformer also has a lower RTF on the Pi (0.29 vs 0.53), lower energy (4,869 vs 8,704 mJ) and lower peak activation memory (22.82 vs 24.15 MiB, Table 2).
Questions for authors
  • Same setup for the original Conformer: what are its WER, size, Pi latency and peak RAM with the same adaptation, beam search and GRU rescoring, in 32 bit and in 8 bit? This would show how much of the gain comes from the proposed methods.
  • Sharing and recovery in isolation: at equal size, depth and width, how does the proposed sharing compare with 8 bit quantization of all blocks, and with sharing in randomly chosen blocks, using the same adapters? Were the Table 3 variants trained on the same data for the same number of steps, and what is the structure of the folded baseline?
  • Language model data: what are the sources of the 44M token text corpus and the MWER training data? The paper removes 2,103 lines that share 8 word sequences with test transcripts, and the domains of the remaining text affect how the rescoring gain should be read.
Reviewer_MhgiOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

Introduction of the "AcuFormer" a sub-10M parameter ASR model comprised of CTC + GRU-LM rescoring. A key to the performance improvement is what the author's call: "Sensitivity Adaptive Full-depth Sharing". The parameter budget is dynamically allocated based on acoustic evidence which allows using the full encoder depth with the option of compressing redundant feed-forward paths. Starting from a Nemo Conformer-CTC S model (~13M) the authors target the large number of parameters dedicated to feed-forward networks. They measure "CTC output sensitivity" by looking at how much the CTC output changes when the block specific FFN is replaced and the ones with lowest change in output are designated to use a shared table resulting in almost 5M weights being replaced with a shared set of ~0.25M weights. "Acoustic repair" adapters: projection to low-dim bottleneck, and then up project, make the model less sensitive to any shared FFN quantization noise. A Bidirectional GRU network LM is used to rescore the output and shares the same embedding space as the acoustic model. On seven domains of OpenASR benchmark, 9.7% average, yield a sub 10M parameter model with a sub 10% WER on Open ASR benchmark. They demonstrate it running at RTF 0.53 on Raspberry Pi Zero 2 W.

Critical strengths

The paper demonstrates a very strong model for its size with insightful observation to reduce the size of the model, and re-investing the parameters saved to improve performance by fixing acoustic issues caused by parameter saving (acoustic repair) and performing LM rescoring. The final model indeed is much compressed yielding very good results compared to much larger models.

Critical weaknesses

There is little novelty in the paper. The parameter sharing, quantization fix and LM rescoring are not novel techniques, however the author's have used them in a very sophisticated and targeted way. The use of CTC output sensitivity to inform what parameters to share does appear novel. Many of the savings appear to be trading parameter space for compute, however this aspect isn't analyzed.

Questions for authors

Do you have a graph that shows the charge in xRT / Flops with each technique that is added, similar to table 3, but replace WER with compute time?

Reviewer_bbyHOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper by the anonymous authors explores the field of compact ASR models. They train their model on top of an Nvidia 13.15M conformer-CTC-S and benchmark its performance on the OpenASR benchmark with 9.70% WER, and a RTF of 0.53. The key claim of the paper is the compactness of the model, being less than 10M parameters while being close to top of the line performance characteristics. This reduction in size is achieved via sensitivity adaptive sharing, and performance gains are achieved by bidirectional GRU rescoring, blank invariant priors, and targeted local recovery. They report a total parameter count of 9.972M, with accompanying evaluations and scores that put it among the best compact ASR models at par with some much larger models. They test the model on a Raspberry Pi system and show that the activation memory and peak power consumption are small.

Critical strengths
  • Building small edge models is important, from a perspective of offline use, privacy, energy efficiency, and deployment costs. This paper presents a new model and associated techniques to reduce model size while maintaining high accuracy.
  • This problem being addressed is of cross domain adaptation. Models trained only on Librispeech with be worse on other acoustic environments like meetings/ conversations. This model is able to perform well on these domains which makes it useful for offline and low power usage.
  • This paper brings together the techniques of weight sharing, bottleneck adapters, CTC with reucrrent LM rescoring andtest time multi view selection in a novel fashion.
  • This paper demonstrates a thorough testing over all 82,116 utterances across 8 splits/12 baselines and is able to reproduce offical ASR scoring to within 0.13 WER. They also present data on a real Pi Zero 2W device reporting latency, peak RAM, and energy usage.
Critical weaknesses
  • The claim of dynamic allocation of sub-10M parameter budget seems inaccurate, since the model is fixed at inference.
  • The full benchmark improvement in WER from 9.81 to 9.79 as a result of blank invariant prior is a very small improvement.
  • The 4.96M uint-8 indices are discounted in the paper. However if the indices are being counted in memory and are required for the model they should be treated as parameters for final accounting.
  • The model doesn't have a lowest peak memory as claimed, and does not have the best accurary/parameter, even though the model HyperConformer has been omitted in Figure 14.
Questions for authors
  • Can you explain why the model HyperConformer-8M (9.24 accuracy/paramter)is omitted from Figure 14?
  • The introduction to the paper mentions "extreme edge platforms" and "billions of microphone-equipped edge devices", but the model is tested on a Pi Zero 2 W with 512 MB RAM. Can you benchmark model performance on edge devices with limited resources? In case that is not feasible, I recommend toning down these claims.
  • I'd also like an explanation for excluding uint8 indices from the final parameter accounting.
  • Please also include data for Pi latency and RTF for three view inference that results in a 9.70% WER.

I currently lean towards weak reject, due to the somewhat misleading claim of sub-10M by discounting stored indices. I'd be willing to raise the score based on the following:

  • Reporting a storage based comparison inluding an INT8-quantized conformer-CTC-S
  • Including a dense conformer-CTC-S tuned on the same adaptation data
  • Revising the paper to reflect the foorprint instead of parameters, assuming AcuFormer is still competetive for the above comparisons.
1 comment

My concerns have been addressed by the authors, so I'm changing my review from weak reject to weak acceptance. I wish the authors the best for their paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.