acceptodds
Under review as a conference paper at ICLR 2027

Small Language Models for Large Speech Models: Latent Acoustic Arbitration for ASR Correction

Abstract

Large language models can improve automatic speech recognition (ASR) transcripts, but unrestricted rewriting may trade acoustic consistency for linguistic fluency. We ask whether post-ASR correction requires a text model comparable in scale to the speech model it assists. We propose Latent Acoustic Arbitration (LAA), a parameter-efficient framework in which a compact frozen text model provides linguistic preferences within a bounded speech-generated candidate space, while retained speech-side evidence determines which local changes are accepted. With Granite Speech 8B and a frozen Qwen3-0.6B text refiner, LAA improves the constrained language-selection baseline on all eight evaluated English ASR benchmarks. In a separate scaling analysis spanning refiners from 135M to 32B parameters–a 237x range–WER varies by only 0.14 percentage points. These results provide evidence that effective post-ASR arbitration does not require speech and text models of comparable scale, while controlled 2B and cross-backbone analyses delimit where the observed gains transfer.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Automatic speech recognition, Post-ASR error correction, Small language models, Acoustic verification, Latent acoustic arbitration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_GSjxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Latent Acoustic Arbitration (LAA), a parameter-efficient framework for post-ASR correction that combines linguistic preferences from a small language model with acoustic evidence from a larger speech model. LAA uses a frozen text model to rank speech-generated hypotheses and a lightweight two-stage router with phoneme-level acoustic verification to selectively correct transcription errors. Experiments across eight English ASR benchmarks show consistent improvements over constrained language-based selection with Granite Speech 8B. Additional scaling experiments suggest that smaller text models can achieve competitive correction performance without requiring comparable model sizes.

Critical strengths
  • The paper addresses a meaningful problem in ASR correction by combining linguistic preferences with acoustic evidence to prevent unsupported text modifications.
  • The proposed framework keeps both speech and text backbones frozen, requiring only a small number of trainable parameters.
  • The experiments cover multiple ASR benchmarks, speech backbones, and text-model sizes, with additional component ablations and sensitivity analyses.
  • The results demonstrate that a substantially smaller language model can effectively assist a larger speech model under the evaluated conditions.
Critical weaknesses
  • LAA combines existing techniques such as N-best hypothesis selection, selective correction, and acoustic rescoring, limiting its methodological novelty. The authors should clarify its key contributions beyond prior work.
  • LAA uses complete-transcript phoneme scores to determine whether local span replacements should be accepted, potentially creating a mismatch between the scope of acoustic verification and the actual modifications. Further validation is needed to establish whether this approach provides reliable acoustic evidence for local corrections.
  • Although LAA consistently improves performance with Granite Speech 8B, the overall WER reduction is relatively small (0.138 percentage points), and several Whisper and SeamlessM4T configurations show no improvement or performance degradation. Further evidence is needed to establish its practical effectiveness and generalizability across speech backbones.
Questions for authors

See above

2 comments

Author Rebuttal (1/2)

W1: What does LAA contribute beyond N-best selection, selective correction, and acoustic rescoring?

Answer: Thank you for recognizing the motivation and the value of combining linguistic preferences with acoustic evidence. LAA separates language-based candidate ranking from acoustic acceptance of local edits, with both backbones frozen. Its key finding is that this division lets a compact text model assist a larger speech model.

To test the compact-refiner premise beyond the original LibriSpeech analysis, we evaluated six refiners from 135M to 32B on the same 647 FLEURS en-US utterances, using Granite Speech 8B and four candidates. Across this 237-fold parameter range, LAA WER varies only from 6.2513% to 6.3066%, a 0.0553-percentage-point spread. Qwen3-0.6B achieves 6.2927% versus 6.3066% for Qwen3-32B, using 1/53.3 as many text-model parameters. These results support compact refiners for correction within the shared candidate space.

To address your concern about existing combination methods, we compared LAA with ROVER-style combination using the same candidate sets. LAA reduces WER by 0.7633 points on LibriSpeech complement, 0.9016 on CHiME-6 and 0.2556 on FLEURS relative to ROVER. These gains show the value of selecting and verifying local edits from the same available hypotheses. The acoustic controls below then test which scoring scope and acceptance rule produce useful corrections.

W2: Can complete-transcript phoneme scores provide reliable evidence for local corrections?

Answer: Complete-transcript CTC provides an overall acoustic compatibility signal, not a calibrated confidence score for each edited span. Our controlled comparison shows that local scoring with routing achieves lower WER than global scoring with routing on all three additional evaluation sets.

To address the mismatch you identified, we compared complete-transcript (global) and local CTC scoring, each with and without routing, alongside Selection and Full LAA. All conditions use Granite Speech 8B, Qwen3-0.6B and four candidates. The sets contain 1,939 LibriSpeech complement utterances, 1,000 CHiME-6 utterances and 647 FLEURS utterances. Values are WER percentages, pp denotes percentage points, and differences are computed before rounding.

MethodLibriSpeech complementCHiME-6FLEURS
Language-only Selection2.969568.69856.3342
Global CTC3.646168.01256.5552
Local CTC3.854367.60096.6657
Global CTC + router3.070768.16936.2513
Local CTC + router3.044767.65976.2237
Full LAA3.018668.24776.2927

Among the variants with routing, local scoring reduces WER relative to global scoring by 0.0260, 0.5096 and 0.0276 points, respectively. Adding routing to local CTC lowers WER from 3.8543% to 3.0447% on LibriSpeech and from 6.6657% to 6.2237% on FLEURS. On CHiME-6, local CTC alone is better, at 67.6009% versus 67.6597% with routing.

Full LAA achieves the lowest WER among the CTC-based variants on LibriSpeech complement, while local CTC with routing performs better on CHiME-6 and FLEURS. These controls show that matching verification to the edited region can improve correction, and quantify the contribution of selective acceptance separately from scoring scope.

Author Rebuttal (2/2)

W3: How practically effective is LAA, and do its gains transfer across speech backbones?

Answer: LAA improves over constrained Selection for three of four speech backbones on the common CHiME-6 evaluation. SeamlessM4T Medium shows the largest decrease, 4.2140 percentage points, with a paired 95% confidence interval of [3.3360, 5.1681]. Both Granite models also improve over native direct recognition.

To address backbone transfer, we evaluated native direct recognition, Selection and LAA on the same 1,000 utterances with Qwen3-0.6B as the text refiner. Reporting both baselines separates gains from acoustic arbitration after Selection from gains over the original recognizer.

Speech backboneNative direct WER (%)Selection WER (%)LAA WER (%)Selection minus LAA (pp)
Granite Speech 8B72.716668.698568.24770.4508
Granite Speech 2B70.560668.443768.36530.0784
Whisper Small80.9878108.0165108.01650.0000
SeamlessM4T Medium98.6476104.155299.94124.2140

For SeamlessM4T, arbitration removes a net 215 word errors relative to Selection. Its interval uses 10,000 paired bootstrap resamples. Granite Speech 8B and 2B improve over Selection by 0.4508 and 0.0784 points, respectively. Against native direct recognition, their reductions are 4.4689 and 2.1953 points. Whisper's aggregate WER is unchanged. Native direct recognition remains better for Whisper and SeamlessM4T, so the 4.2140-point SeamlessM4T benefit specifically measures recovery from Selection errors. This establishes transfer of the post-selection arbitration benefit to SeamlessM4T, alongside gains over both baselines for the Granite models.

To quantify the practical tradeoff, we measured a separate 100-utterance LibriSpeech workload with Granite Speech 8B and Qwen3-0.6B, using five repetitions. Mean end-to-end time rises from 0.6594 seconds per utterance for Selection to 12.4699 for Full LAA, an 18.91-fold increase. Peak allocated GPU memory changes from 18.1752 to 18.1760 GiB. Acoustic arbitration adds little allocated memory but substantial verification time, alongside the 0.4508-point CHiME-6 gain over Selection reported above for this pipeline.

Reviewer_pVm9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Latent Acoustic Arbitration (LAA), a parameter-efficient framework for post-ASR correction that combines linguistic preferences from a frozen text model with acoustic verification from retained speech representations. Rather than generating unrestricted corrected transcripts, LAA selects local edits from speech-generated hypotheses using a phoneme CTC head and lightweight routers. Experiments across eight English ASR benchmarks show that Granite Speech 8B with a frozen Qwen3-0.6B refiner improves over constrained N-best selection on all eight benchmarks, reducing equally weighted overall WER from 15.023% to 14.885%. A separate scaling analysis spanning 135M to 32B text refiners reports only a 0.14-percentage-point WER range. However, the gains are modest in absolute terms, improvements do not consistently transfer to other speech backbones, and the scaling analysis uses data repeatedly inspected during method development. The results support the potential of selective, acoustically grounded correction with compact language models, but broader generalization and practical efficiency remain insufficiently established.

Critical strengths
  1. A principled constrained correction framework. LAA separates linguistic preferences from acoustic verification and restricts accepted edits to speech-generated candidates. This is a reasonable approach to mitigating unsupported language-model corrections while preserving useful linguistic information.
  2. Broad and structured empirical analysis. The paper evaluates eight English ASR benchmarks and investigates refiner scale, speech-backbone compatibility, routing sensitivity, and candidate-set width. The consistent gains with Granite Speech 8B provide encouraging evidence that a compact text model can assist a larger speech model without requiring comparable model scale.
Critical weaknesses
  1. Modest absolute improvement and uncertain practical significance. The main Granite Speech 8B experiment reduces overall WER by only 0.138 percentage points, from 15.023% to 14.885%. Although the improvement appears across all eight benchmarks, its practical significance is unclear without stronger statistical analysis and direct measurements of the additional computational cost.
  2. Limited generalization and incomplete efficiency validation. LAA does not consistently improve results across Whisper and SeamlessM4T configurations, and Granite Speech 2B regresses on two benchmarks. Moreover, the 135M–32B scaling analysis uses a LibriSpeech test-other slice repeatedly inspected during development, limiting its strength as independent evidence for scale invariance. End-to-end latency and compute savings are also not measured, so parameter efficiency does not yet establish deployment efficiency.
Questions for authors
  1. Statistical significance: Given that the overall WER improvement is only 0.138 percentage points, can the authors provide paired significance tests or confidence intervals at the utterance level to establish whether the gains are statistically reliable and practically meaningful?
  2. Generalization of the scaling claim: Since the LibriSpeech test-other slice in the refiner-scaling analysis was repeatedly inspected during development, how robust is the reported 0.14-point WER range on an untouched evaluation set?
  3. Practical efficiency: Can the authors quantify end-to-end latency, memory consumption, and computational overhead relative to constrained N-best selection, particularly given that LAA requires additional phoneme scoring and routing?
2 comments

Author Rebuttal (1/2)

W1-W2, Q1: Are the gains statistically reliable and practically meaningful?

Answer: Thank you for recognizing the constrained correction framework and requesting statistical and computational evidence. The paired tests establish a significant SeamlessM4T gain on CHiME-6, but do not establish significance for the original eight-benchmark WER reduction from 15.023% to 14.885% (0.138 percentage points). For SeamlessM4T, LAA removes 215 word errors from Selection's output, reducing WER by 4.2140 points with a paired 95% confidence interval of [3.3360, 5.1681].

To address your statistical question, we compared LAA with the corresponding Selection output using 10,000 paired bootstrap resamples. Positive decreases mean lower WER with LAA. Differences use unrounded values, and pp denotes percentage points.

DatasetSpeech backbone / text refinerSelection WER (%)LAA WER (%)Decrease (pp)Paired 95% CI (pp)
CHiME-6Granite Speech 8B / Qwen3-0.6B68.698568.24770.4508-
FLEURSGranite Speech 8B / Qwen3-4B6.38946.29270.0967[-0.0955, 0.2975]
CHiME-6Granite Speech 2B / Qwen3-0.6B68.443768.36530.0784[-0.6177, 0.7818]
CHiME-6Whisper Small / Qwen3-0.6B108.0165108.01650.0000[0.0000, 0.0000]
CHiME-6SeamlessM4T Medium / Qwen3-0.6B104.155299.94124.2140[3.3360, 5.1681]

All CHiME-6 comparisons use the same 1,000 utterances. For SeamlessM4T, 177 utterances improve and 33 regress in word-error count, yielding the net reduction of 215 errors. Granite Speech 8B removes 23 errors. Its available confidence interval compares LAA with native decoding, so the interval for Selection versus LAA is marked "-". The Granite Speech 2B and FLEURS intervals include zero.

Arbitration improves over Selection for three of the four CHiME-6 backbones, with Whisper unchanged. Native direct recognition remains better for Whisper and SeamlessM4T, at 80.9878% and 98.6476% WER, respectively. The two baselines answer different questions: Selection isolates the value of acoustic arbitration, while native recognition measures the net benefit of the full correction pipeline.

W2, Q2: Does the scaling finding hold beyond the repeatedly inspected LibriSpeech slice, on an untouched evaluation set?

Answer: FLEURS reproduces low sensitivity to refiner size: across six refiners from 135M to 32B parameters, LAA WER ranges from 6.2513% to 6.3066%, a spread of 0.0553 percentage points. We report this as cross-dataset replication, without claiming a formally untouched evaluation set.

To address dependence on the original LibriSpeech analysis, we evaluated all six refiners on the same 647 FLEURS en-US utterances, using Granite Speech 8B, four shared speech-generated candidates and a frozen arbitration configuration. We treat the repeatedly inspected LibriSpeech slice as exploratory. Comparing Selection and LAA isolates the change after acoustic arbitration.

Text refinerSelection WER (%)LAA WER (%)
SmolLM2-135M6.32046.2513
Qwen3-0.6B6.33426.2927
Qwen2.5-1.5B6.27896.2651
Qwen3-4B6.38946.2927
Qwen3-14B6.38256.3066
Qwen3-32B6.32736.3066

LAA lowers the WER point estimate for all six refiners and reduces the range from 0.1105 points under Selection to 0.0553 points. Qwen3-0.6B reaches 6.2927% WER versus 6.3066% for Qwen3-32B, using 1/53.3 as many text-model parameters. This supports the central design choice: a compact external refiner provides sufficient linguistic preferences in the tested candidate-constrained setting.

Author Rebuttal (2/2)

W1-W2, Q3: What are the measured latency, memory consumption, and computational overhead?

Answer: Full LAA uses 0.795 MiB more peak allocated GPU memory than Selection, but takes 12.4699 rather than 0.6594 seconds per utterance, an 18.91-fold increase. The added computation is primarily acoustic verification of proposed corrections. Compact text-model capacity and end-to-end speed are therefore separate results.

Following your request, we measured all pipelines on the same fixed 100-utterance LibriSpeech workload containing 581.99 seconds of audio, with five repetitions, Granite Speech 8B, Qwen3-0.6B where applicable, four candidates and one allocated GPU per run.

PipelineMean time (s/utterance)Real-time factorPeak allocated GPU memory (GiB)
ASR only0.58730.100916.2651
Selection0.65940.113318.1752
Selection + global CTC1.42210.244418.1760
Full LAA12.46992.142718.1760

CTC scoring takes 11.7545 seconds per utterance, 94.3% of Full LAA's measured mean time. Peak allocated memory rises from 18.1752 to 18.1760 GiB, so verification time dominates the added cost. The separate accuracy evaluation above shows a 0.4508-point CHiME-6 WER reduction over Selection for this Granite Speech 8B pipeline. The measured tradeoff is improved correction with a compact refiner at the cost of repeated acoustic verification.

Reviewer_4jnzOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a post-ASR text correction method with language model and a comparison study of LLM’s size. The authors argue that post correction with no rewriting, but with choosing the best token from hypotheses based on audio condition, can prevent hallucinations during generation. The authors propose Latent Acoustic Arbitration (LAA). In LAA, a frozen text refiner, aka language model, ranks speech-generated candidate hypotheses, while retaining time-aligned encoder frames. One hypothesis is treated as a baseline. Trained phoneme CTC head computes an acoustic gap between proposed baseline and other hypotheses to selectively verify local span changes. Evaluation goes across eight English ASR benchmarks using Granite Speech 8B ASR, LAA improves upon the constrained language-selection baseline on all benchmarks, and a refiner scaling analysis across a parameter range from 135M to 32B shows that WER varies by only 0.14 percentage points.

Critical strengths

LAA should prevent text hallucination and over-correction by restricting text modifications to local span proposals derived from speech hypotheses and verifying them with acoustic features and CTC.

The paper presents comprehensive benchmark evaluations across eight datasets, alongside extensive diagnostic analyses spanning refiner scale, number of generated hypotheses, and different ASR backbones.

Critical weaknesses

The paper outline its motivation as "classic post-ASR error correction", treating speech models as traditional acoustic decoders needing external linguistic fixes. But modern ASR systems, like Granite Speech 8B, have Qwen LLM, as text decoder. Stacking an external text refiner on top of an LLM-based speech decoder represents LLM-over-LLM, making the foundational framing imprecise.

For Granite Speech 8B, Overall WER improves by a 0.138 percentage points only (from 15.023 to 14.885). An absolute reduction of 0.13% raises serious questions about whether the practical gains justify the added system complexity of maintaining multi-stage routing, span extraction, and trained phoneme CTC heads. Lack of resource usage reports also makes the result questionable.

Use of audio features with CTC makes ASR hallucination a potential point of failure. This is more likely to happen, then generating correction from non-presented hypothesis with LLM. Deploying this method would require support for running LLMs, which might not be feasible for some devices with mobile ASR architectures.

LLM hallucination or hacking is still an unsolved issue that is of great concern in practice, and showing that your method is robust to this issue is essential, especially since it will be used in practice. No clear reference of tested methods. They are presented in section 4 in the short form without citing, so it is not clear what paper they are based on.

No comparison with classic methods, like ROVER. Lack of comparison with ASR uncertainty methods, like Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders. No mention of other large research papers on this topic, like Correction Focused Language Model Training for Speech Recognition, Training-Free Intelligibility-Guided Observation Addition for Noisy ASR, Clarispeech: LLM-enhanced speech recognition post-correction.

Questions for authors
  1. Given that Granite Speech backbones already incorporate internal language model decoders trained with linguistic priors, how much of LAA's performance gain is attributable to correcting the primary model's beam-search errors versus redundant secondary scoring?

  2. In addition - since text-refinder only picks the baseline and then spans are obtained from other hypotheses, do we really need a refiner or can we use a simpler model, like n-gram?

  3. Could you please provide end-to-end inference latency benchmarks demonstrating that the multi-stage LAA pipeline does not incur prohibitive overhead that outweighs its 0.13% WER reduction?

  4. Did you measure any robustness against white-box or black-vox attacks on ASR? Adversarial prompts?

3 comments

Author Rebuttal (1/2)

W1, Q1: What does LAA add when the speech model already contains a language decoder, and are its gains attributable to beam search or secondary scoring?

Answer: LAA adds acoustic acceptance of local edits drawn from ASR candidates after decoding, with both backbones frozen. Thank you for highlighting the need to distinguish the internal decoder from our compact external refiner. Granite Speech uses a Granite language model internally [1], while Qwen is our external refiner.

To identify the source of the gain, we fixed the same four speech-generated candidates and compared ASR scoring, language-based Selection, candidate combination and acoustic correction. We used Granite Speech 8B on 1,939 LibriSpeech test-other complement utterances, 1,000 CHiME-6 utterances and 647 FLEURS en-US utterances. Entries are WER percentages, pp denotes percentage points, and differences are computed before rounding. ASR 1-best is the first hypothesis in this shared set.

MethodLibriSpeech complementCHiME-6FLEURS
ASR 1-best2.943568.93386.3066
ASR-score selection2.931973.10866.2582
Qwen3-0.6B Selection2.969568.69856.3342
Shared-candidate ROVER3.782069.14946.5483
Complete-hypothesis CTC rescoring3.646168.01256.5552
Full LAA3.018668.24776.2927

On CHiME-6, Selection lowers WER by 0.2352 percentage points relative to shared-set 1-best, and Full LAA adds a further 0.4508-point reduction. Against Selection, Full LAA lowers FLEURS WER by 0.0414 points and raises LibriSpeech complement WER by 0.0492 points. With the candidate set fixed, these differences measure arbitration after selection rather than a change to beam search.

Following your suggestion, we added the shared-candidate ROVER control. Full LAA lowers WER by 0.7633, 0.9016 and 0.2556 points across the three datasets, respectively. Against complete-hypothesis CTC rescoring, Full LAA lowers LibriSpeech and FLEURS WER by 0.6274 and 0.2625 points, while its CHiME-6 WER is 0.2352 points higher. These controls separate language ranking, candidate voting and local edit acceptance while holding speech-generated evidence fixed.

Q2: Could an n-gram model replace the text refiner?

Answer: Yes. LAA also improves an n-gram selector, while Qwen3-0.6B produces lower final WER. To test your suggestion, we compare an add-k trigram selector with and without LAA on the same 647 FLEURS utterances.

Text selectorSelection WER (%)LAA WER (%)WER decrease (pp)
Add-k trigram, matched paired run7.78486.65191.1328
Qwen3-0.6B6.33426.29270.0414

LAA reduces trigram WER by 1.1328 points, with a 95% paired-bootstrap interval of [0.7136, 1.6158] from 10,000 resamples. Qwen3-0.6B with LAA achieves another 0.3592-point reduction relative to trigram-based LAA. This supports a simpler selector and quantifies the benefit of compact neural preferences. The additional six-refiner FLEURS comparison spans 135M to 32B parameters with a 0.0553-point LAA WER range, further supporting the compact-refiner finding.

Author Rebuttal (2/2)

W2, Q3: Does the improvement justify the additional inference cost?

Answer: Full LAA trades additional verification time for the 0.4508-point CHiME-6 and 0.0414-point FLEURS WER reductions over Selection reported above. In the separate timing workload, it takes 12.4699 seconds per utterance versus 0.6594 for Selection, an 18.91-fold increase, while peak allocated GPU memory rises by 0.795 MiB.

To quantify the cost, we measured four pipelines on a fixed 100-utterance LibriSpeech workload containing 581.99 seconds of audio, with five repetitions, four candidates and one allocated GPU per run.

PipelineMean time (s/utterance)Real-time factorPeak allocated GPU memory (GiB)
ASR only0.58730.100916.2651
Selection0.65940.113318.1752
Selection + global CTC1.42210.244418.1760
Full LAA12.46992.142718.1760

CTC scoring takes 11.7545 seconds per utterance, or 94.3% of Full LAA's mean time, so acoustic verification dominates the added cost. The scaling results support using a compact text refiner for correction, while this timing experiment identifies repeated CTC scoring as the main processing bottleneck.

W3-W4, Q4: How robust is LAA to hallucinations, adversarial attacks, and deployment constraints?

Answer: On controlled silence and non-speech inputs, LAA reduces false-positive output words by 46.56% and 94.40% relative to Selection. We have not evaluated white-box, black-box or adversarial-prompt attacks, or mobile deployment.

To test whether acoustic arbitration suppresses unsupported corrections, we counted false-positive output words on both silence/non-speech sets. The measured implementation uses the GPU pipeline above.

Silence/non-speech setInputsASR-only false-positive wordsSelection false-positive wordsLAA false-positive words
LibriSpeech-derived1,93905,3712,870
CHiME-6-derived1,00001,12563

LAA reduces Selection's false-positive words from 5,371 to 2,870 and from 1,125 to 63. ASR only returns empty outputs in both tests. The reductions therefore quantify removal of unsupported outputs introduced by Selection.

W4-W5: How do the compared methods relate to ROVER and the other cited literature?

Answer: The added quantitative controls directly compare LAA with shared-candidate ROVER, n-gram selection and complete-hypothesis CTC rescoring. ROVER combines recognizer outputs through alignment and voting [2]. Our control applies this principle to the same four hypotheses used by LAA.

To clarify the other comparisons you raised, Correction Focused Language Model Training prioritizes words likely to be misrecognized by ASR during language-model training [3]. Clarispeech fine-tunes an external language model for post-transcription correction with ASR frozen [4]. Intelligibility-Guided Observation Addition combines noisy and enhanced waveforms before recognition [5]. Whisper representation steering intervenes on internal representations to mitigate hallucinations [6]. LAA keeps both backbones frozen and uses retained acoustic evidence to accept candidate-derived local edits. This identifies its intervention point relative to trained correction, input enhancement, representation steering and candidate voting.

References

[1] Granite Speech model documentation.

[2] Fiscus, 1997. A Post-Processing System to Yield Reduced Word Error Rates: Recognizer Output Voting Error Reduction (ROVER).

[3] Ma, Liu, and Kalinli, ICASSP 2024. Correction Focused Language Model Training for Speech Recognition.

[4] Iudin et al., 2025. Clarispeech: LLM-enhanced speech recognition post-correction.

[5] Li et al., Interspeech 2026. Training-Free Intelligibility-Guided Observation Addition for Noisy ASR.

[6] Aparin et al., 2026. Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders.

Thank you for your response. Your clarifications on Q2, W4, and W5 are helpful and address those points, though I still encourage you to include concrete quantitative metrics alongside the architectural comparison in the final version for W4-W5.

However, my main concerns regarding practical utility and empirical justification remain unresolved:

  • Limited accuracy gains (W1, Q1): LAA performs nearly on par with the ASR-score baseline, and ASR-score selection outperforms LAA on 2 out of 3 datasets.
  • Computational overhead (W2, Q3): LAA is 24 times slower than general ASR and 20 times slower than ASR-score selection. Given a WER improvement of less than 1 point and worse performance on two datasets, this latency overhead is difficult to justify in practice. And in your submission the difference in WER as low as 0.138, which makes LAA even less practical.
  • False positives and robustness (W3–W4, Q4): The ASR-only baseline already achieves 0 false positives, which further reduces the practical advantage of LAA. Additionally, the lack of robustness evaluation against adversarial or jailbreak-style perturbations remains an important gap for modern deployment scenarios.

For these reasons, while I appreciate the authors' efforts during the rebuttal, I will maintain my current score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.