- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a sub-10M ASR model, AcuFormer, for robust multi-domain ASR on edge devices. AcuFormer uses acoustic evidences to dynamically allocate the parameter budget end to end. Built on the pre-trained 13.15M Conformer-CTC-S, AcuFormer is an acoustically informed ASR architecture that combines four synergistic key mechanisms, including sensitivity-adaptive full-depth sharing to preserve backbone capacity, CTC-guided bidirectional rescoring with shared token projection, a blank-invariant acoustic prior for temporal shift stability, and targeted local recovery for feature restoration. On OpenASR, AcuFormer achieves 9.70% macro WER, substantially outperforming the baselines of much larger models.
Critical strengths
-
Building robust and accurate very small multi-domain ASR models for edge devices is an important direction, and this work substantially improves the sub-10M multi-domain ASR performance over the previous SOTA.
-
The proposed sensitivity-adaptive full-depth sharing to preserve backbone capacity, CTC-guided bidirectional rescoring with shared token projection, targeted local recovery for feature restoration are reasonable designs. Moreover, Table 3 shows each architectural component achieves WER reduction under single-view inference. Table 11 ablation shows that each of these three architectural designs improves most of the eight splits.
-
Overall, AcuFormer achieves impressively strong ASR performance of 9.70% WER on the OpenASR benchmark (all 7 domains), breaking the 10% WER for sub-10M models and yielding 59% relative gain over the previous SOTA Squeezeformer-XS, and outperforming baselines up to 7.8x its size.
Critical weaknesses
-
As shown in Table 2, AcuFormer achieved macro WER 9.70% while the previous sub-10M SOTA Squeezeformer-XS (NeMo) obtained 23.67%, the authors thus concluded the 59% relative improvement. However, Table 2 only compares the model size and peak activation, without clearly comparing the training data (amount, distribution, domain coverage). Hence, this conclusion may confound many factors, not just the architectural update.
-
Some important ablations and comparisons are missing:
a. The paper needs more thorough analysis on different configurations of sensitivity guidance, for example, compare uniform compression, random, or reverse sensitivity allocation at equal parameters.
b. Similar to the above question on whether the comparisons are fair, there is no comparison to the backbone Conformer-CTC-S under the same adaptation, beam search, and LM recipe; therefore, the effect of the sensitivity-adaptive full-depth sharing is not fully analyzed.
Questions for authors
-
Please address the points under Weaknesses.
-
Please report the AcuFormer storage versus INT4 and INT8 Conformer-CTC-S under the same recipe.
-
What is the RTF for confidence-routed views and all three views?
-
Are there any scaling studies to further reduce the number of model parameters and evaluate the performance?
-
There are no limitation discussions.
My concerns have been addressed by the authors, so I'm changing my review from weak reject to weak acceptance. I wish the authors the best for their paper.