acceptodds
Under review as a conference paper at ICLR 2027

Selective Context Utilization in Speech-LLM ASR: Lexical Verification and Semantic Disambiguation with GRPO

Abstract

Context can improve entity recognition in speech-enabled large language models (Speech-LLMs), but misleading context can also bias transcription toward unsupported words. Context additionally provides semantic evidence for written distinctions that pronunciation alone cannot resolve. We study these complementary demands through contextual entity recognition and Mandarin third-person pronoun disambiguation. We develop a three-stage framework combining general ASR fine-tuning, robust context fine-tuning with mismatched contexts, and task-specific group relative policy optimization (GRPO). The lexical objective uses phonetic hard negatives and quality-gated distractor rejection, while the semantic objective rewards correct ordered pronoun sequences alongside full-transcription accuracy. We construct a diagnostic benchmark suite with 1,116 lexical utterances and 617 pronoun-disambiguation utterances, supplemented by paired clean/misleading contexts and controlled semantic-context ablations. Our model achieves 97.76% entity recall and 89.70% pronoun macro-F1, exceeding the strongest evaluated pronoun baseline by 7.12 percentage points. Under misleading lexical context, the full pipeline improves entity recall while matching the backbone’s distractor false-positive rate. Semantic ablations show that the disambiguation gains depend on matched discourse, although general-ASR performance degrades unevenly across domains. We will release the benchmark upon acceptance. The code is available at https://anonymous.4open.science/r/SCU-ASR-8E2D.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Contextual ASR, Speech-LLMs, Contextual Biasing, Homophone Disambiguation, GRPO

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UtK7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a 3-stage training to improve the performance of a contextual ASR system. This includes 1) regular ASR fine-tuning, 2) Context-robust ASR fine-tuning, where the context (e.g. entities) are replaced with unrelated contexts for a subset and 3) RL stage (GRPO) which includes a lexical component (contextual ASR) and a semantic component - pronoun dismbiguation. Results show that adding the pronoun disambiguation task improves the performance of contextual ASR over the baseline (Qwen3-asr). On pronoun disambiguation, the model improves some of the metrics (She-R, It-R, overall-recall) but not others (He-R). Further ablations show that including the semantic task in RL improves performance with misleading contexts.

Critical strengths
  • Proposes a strategy to use an additional pronoun disambiguation task to improve performance of contextual ASR, specifically in misleading contexts.
  • Results show that the strategy improves upon a baseline (Qwen3-asr) in terms of both Contextual ASR (Character Error Rate) and Pronoun-disambiguation
Critical weaknesses
  • The paper does not have a second SFT stage consisting only of the semantic task i.e. pronoun disambiguation i.e. Base SFT, Context SFT, Semantic SFT. Such a baseline would give the readers an insight into whether the RL stage is necessary for the semantic task or whether it can be included via an SFT stage.
  • Some parts of the paper are hard to follow. See Questions below.
Questions for authors
  • L019: This sentence is unclear and does not connect well with the previous sentence . "The lexical objective uses … while the semantic objective rewards".
  • Sec 3.4.2: What is \hat{y}?
  • Equation 3: what is s \precurlyeq {\hat{y}?
  • Table 3: Why is Ours (GRPO-RL) worse on He-R compared to the baseline but better on She-R and It-R?
  • Table 4: What is Direct GRPO?
2 comments

Thank you for carefully reviewing our paper and providing constructive suggestions. Your comments have helped us further improve our experimental analysis and the presentation of the paper. In response to your questions, we have added relevant experiments and detailed explanations in the appendix, along with additional experiments, including results under different reward weight settings and different random seeds. We address your comments one by one below.

Using the same amount of training data and the same training settings, we added a General ASR SFT → Context-SFT → Semantic SFT baseline. These results show that SFT improves pronoun-disambiguation performance. However, GRPO achieves further gains on this benchmark, increasing macro-F1 from 88.29% to 89.70% (+1.41 percentage points) and improving recall across all three pronoun classes. The specific metrics are reported below. Among these, "Ours (Stage III SFT)" refers to Semantic SFT. You can also find detailed results in the Ours (Stage III SFT) row of Table 7 in the appendix.

Contextual ASR results (%) on the semantic evaluation sets.

Pronoun Disambiguation: He-R, She-R, It-R, and Macro-F1.

He-R, She-R, and It-R denote class-wise pronoun recall. Best results for each metric are shown in bold.

ModelHe-R ↑She-R ↑It-R ↑Macro-F1 ↑
Qwen3-ASR96.0428.8348.4861.14
Ours (Stage II)94.2924.3236.3654.03
Ours (Stage III SFT)90.9980.1891.4188.29
Ours (Full)92.0981.0894.4489.70

Q1.

This sentence elaborates on the task-specific GRPO stage introduced in the preceding sentence. Specifically, “lexical objective” and “semantic objective” refer to the two task-specific reward functions used in Stage III for lexical verification and pronoun disambiguation, respectively. To make this connection clearer, we will revise the sentence as follows:

In the GRPO stage, we use two task-specific rewards: a lexical reward for target-entity recognition and quality-gated rejection of phonetic distractors, and a semantic reward for correct ordered pronoun sequences alongside full-transcription accuracy.

Q2.

y^\hat{\mathbf{y}} denotes the full transcription predicted by the model from the input speech and textual context, while y\mathbf{y} denotes the reference transcription. We will explicitly define this notation at its first occurrence in the revised manuscript.

Q3.

In Equation 3, s⪯yˉs\preceq\bar{\mathbf{y}} means that ss is a contiguous substring of the normalized model-generated transcription yˉ\bar{\mathbf{y}}. The minimization finds the substring with the smallest edit distance to the normalized target entity. We will explicitly define this notation in the revised manuscript.

Q4.

The baseline achieves 96.04% recall for He, but only 28.83% for She and 48.48% for It. This substantial asymmetry is consistent with a tendency to over-predict He, which can yield high He recall while misclassifying the other pronoun forms. Our model exhibits more balanced class-wise recall: although He recall decreases to 92.09%, She and It recall increase substantially to 81.08% and 94.44%, respectively. Consequently, macro-F1 improves from 61.14% to 89.70%. These results indicate a trade-off between He recall and overall performance across the three pronoun classes.

Q5.

Direct GRPO is initialized from the checkpoint obtained after Stage I, General ASR SFT, and proceeds directly to Stage III, GRPO, skipping Stage II, Context-SFT. Its training path is therefore:

Qwen3-ASR -> General ASR SFT -> GRPO.

In contrast, the full model follows:

Qwen3-ASR -> General ASR SFT -> Context-SFT -> GRPO.

This ablation is intended to assess the contribution of the intermediate Context-SFT stage before GRPO. We will clarify the initialization and training path of Direct GRPO in the table caption and accompanying text.

Thank you again for the time and effort you devoted to reviewing our paper and for your valuable suggestions.

Thanks for the clarifications. I will be updating my scores.

Reviewer_81kzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies selective utilization of textual context in Speech-LLM ASR through two settings: (1) contextual entity recognition, and (2) Mandarin third-person pronoun disambiguation. The proposed training pipeline consists of three stages: general ASR supervised fine-tuning (SFT), contextual SFT with mismatched contexts, and task-specific group relative policy optimization (GRPO) using separate lexical and semantic rewards. The final evaluation uses a 1,116-utterance lexical benchmark and a 617-utterance semantic benchmark, together with misleading-context and semantic-context ablations. The final model achieves 97.76% context-biased word recall and 89.70% pronoun macro-F1. However, the reported improvements on contextual entity recognition are marginal relative to the original Qwen3-ASR backbone, while the paper also reports substantial degradation on standard ASR benchmarks.

Critical strengths

The paper addresses an important problem in contextual ASR: selectively incorporating relevant contextual information while remaining robust to misleading context. The distinction between lexical verification and semantic disambiguation is interesting.

The inclusion of clean, misleading, and mismatched-context evaluations is useful for understanding how the model responds to contextual information. The substantial improvement in pronoun disambiguation and the performance degradation when semantic context is removed suggest that the model benefits from relevant discourse information.

Critical weaknesses

Table 2 reports substantial degradation on standard ASR benchmarks, which the authors hypothesize to result from data-distribution imbalance and forgetting during contextual optimization. However, the training stage corresponding to these results is not clearly identified, and insufficient details about the training data used in the SFT stages make this explanation difficult to substantiate.

The final GRPO-trained model achieves only marginal improvements over the original Qwen3-ASR backbone on the lexical contextual ASR benchmark (Table 3). CER improves from 2.30% to 2.14%, BCER from 1.21% to 1.18%. Given the extensive training pipeline involving two stages of SFT followed by GRPO, these improvements are small and raise questions about the practical effectiveness of the proposed contextual biasing approach.

Questions for authors

1. Which training stage does the model reported in Table 2 correspond to? Could the authors provide stage-wise results on standard ASR benchmarks (after Stage I SFT, Stage II Context-SFT, and Stage III GRPO) to identify where the degradation originates? Additionally, could the authors clarify the composition and sources of the training data used in the SFT stages to substantiate the hypothesis regarding data-distribution imbalance?

2. Could the authors provide a task-specific SFT baseline trained on the same 20,000 examples used for GRPO? This comparison is particularly important for the pronoun disambiguation task, where the substantial improvement after GRPO could potentially be explained by the introduction of task-specific supervision rather than the reinforcement learning objective itself.

3. Why does the proposed distractor-rejection reward not reduce the distractor false-positive rate relative to the original backbone?

3 comments

Thank you for carefully reviewing our paper and providing constructive suggestions. Your comments have helped us further improve our experimental analysis and the presentation of the paper. In response to your questions, we have added relevant experiments and detailed explanations in the appendix, along with additional experiments, including results under different reward weight settings and different random seeds. We address your comments one by one below.

We understand your concern about the complexity of the training pipeline relative to its lexical recognition gains. Although the CER and BCER improvements over the backbone in Table 3 are modest, the method also improves performance under misleading context. In Table 4, the full model reduces CER from 3.68% to 3.20% and increases entity recall from 81.27% to 83.49%, while retaining the same D-FPR as the backbone (8.86%). Thus, it recovers correct entities more accurately under misleading context without increasing the observed distractor false-positive rate.

To further examine whether these benefits extend beyond our own evaluation sets, we added an evaluation on ContextASR-Bench. Since our post-training primarily uses Chinese data, this evaluation adopts the fine-grained context setting of its ContextASR-Speech-ZH and ContextASR-Dialogue-ZH subsets. Compared with the original backbone, the full model achieves lower NE-WER and NE-FNR on both subsets. These results provide additional evidence of the method’s entity recognition gains on an external benchmark and indicate that its benefits are primarily reflected in entity-related metrics. Detailed results can be found in Table 9 in the appendix.

Table 9. (partial)

Speech refers to ContextASR-Speech-ZH, and Dialogue refers to ContextASRP-Dialogue-ZH. Lower is better for all metrics. Best results in each column are shown in bold.

ModelSpeech WERSpeech NE-WERSpeech NE-FNRDialogue WERDialogue NE-WERDialogue NE-FNR
Qwen2-Audio9.9224.1030.027.0022.7626.17
Kimi-Audio1.9511.1315.282.9015.9116.68
Qwen2.5-Omni-7B1.849.8012.192.4014.0613.17
Qwen3-ASR-1.7B0.686.841.091.999.523.14
Ours (Stage II)0.947.071.043.0111.112.81
Ours (Full)0.866.701.002.019.392.59

Q1.

The model reported in Table 2 is the final model obtained after completing all three stages of our proposed training pipeline: Stage I general ASR SFT, Stage II Context-SFT, and Stage III GRPO.

We apologize for the unclear presentation of baseline sources and evaluation settings. The baseline results in the original table were taken from existing public reports, whereas the results for our method were obtained through our own evaluation. We have now re-evaluated the baselines using the same evaluation code, explicitly distinguished previously reported and newly evaluated results, and provided links to the original sources. Since some re-evaluated results differ from public reports, the revised performance comparison will use results obtained under the unified evaluation procedure.

We have added evaluation results for each training stage in Table 6 in the appendix to further identify the stages at which performance changes occur. The training data for Stage I include subsets selected from WenetSpeech, KeSpeech, AISHELL, and Common Voice, as well as speech data collected from public video platforms and pseudo-labeled through automatic transcription.

The stage-wise results show that some performance gaps relative to the original backbone already emerge after Stage I general ASR SFT. Thus, the final model’s degradation cannot be attributed entirely to subsequent contextual optimization. Stage III improves CER over Stage II on three test sets but worsens WS-Meeting CER. The composition of the Stage I data and pseudo-label quality may affect adaptation; improved data filtering and broader coverage of meeting speech are directions for future optimization.

Table 6 (partial).

Lower is better. WS denotes WenetSpeech.

ModelAISHELL-1AISHELL-2WS-NetWS-Meeting
Ours (Stage I)3.142.975.576.74
Ours (Stage II)3.303.115.566.38
Ours (Full)3.072.995.557.53

Q2.

We thank the reviewer for highlighting this distinction. To control for the introduction of task-specific supervision, we added a task-specific SFT baseline initialized from the same Context-SFT checkpoint and trained on the exact same 20,000 examples used for GRPO. Pronoun macro-F1 is 88.29% for task-specific SFT and 89.70% for GRPO. You can find these results in the Ours (Stage III SFT) row of Table 7 and Table 8 in the appendix.

Appendix Table 10 further compares the two models under misleading context. Although task-specific SFT already achieves strong pronoun performance, GRPO additionally improves misleading-context B-Recall from 72.06% to 83.49%, reduces D-FPR from 20.66% to 8.86%, and lowers CER from 3.73% to 3.20%. Its additional benefit therefore extends beyond pronoun disambiguation to robustness against misleading context, while SFT retains higher B-Recall on the main lexical benchmark.

Table 10.

All values are percentages. The better result for each metric is shown in bold.

MetricStage III SFTFull GRPO
Pronoun macro-F1 ↑88.2989.70
Lexical B-Recall ↑98.2197.76
Misleading-context B-Recall ↑72.0683.49
Misleading-context D-FPR ↓20.668.86
Misleading-context CER ↓3.733.20

Q3.

We agree that the full model does not reduce D-FPR below the original backbone. Our additional experiments show that, with the rejection reward set to 0, the D-FPR is 16.61%, compared with 8.86% for the full model, while misleading-context B-Recall increases from 78.73% to 83.49%, with CER increasing slightly from 3.11% to 3.20%. Compared with the original backbone, the full model improves CER and B-Recall while retaining the same observed D-FPR. These results suggest that the rejection term helps control distractor emissions during contextual adaptation, enabling improved contextual recognition while preserving backbone-level distractor robustness. You can find the results in the GRPO (0.6, 0.4, 0) row of Table 8 in the appendix.

Table 8 (partial).

ModelClean CERClean B-RecallMisleading CERMisleading B-RecallMisleading D-FPRB-Recall Drop
Qwen3-ASR3.2891.753.6881.278.8610.48
GRPO (0.6, 0.4, 0)2.6293.653.1178.7316.6114.92
Full model2.8993.023.2083.498.869.52

Thank you again for your time and constructive feedback.

Reviewer_81kzOpenReview ↗

Reviewer's response

Thank you for the additional experiments and clarifications. However, my main concerns remain. The new task-specific SFT baseline suggests that most of the pronoun disambiguation performance can be achieved without GRPO, while the improvements over the original Qwen3-ASR backbone on the external ContextASR benchmark remain marginal. Although the misleading-context results are encouraging, the full model still does not improve distractor rejection relative to the backbone, and the substantial degradation on standard ASR benchmarks remains unresolved. Overall, the benefits of the proposed three-stage training pipeline remain limited relative to its complexity and the associated loss in general ASR performance.

Reviewer_WjakOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper trains Speech-LLM ASR systems in three stages: general ASR supervised fine-tuning, robust context SFT using both matched and mismatched contexts, and task-specific GRPO. For lexical contextualization, the reward combines entity recognition with quality-gated rejection of phonetically similar distractors. For semantic disambiguation, it rewards the ordered Mandarin third-person-pronoun sequence while also scoring the full transcription.

The evaluation contains 1,116 lexical utterances and 617 pronoun-disambiguation utterances. It includes clean versus misleading lexical contexts and semantic-context ablations. The full lexical pipeline reaches 97.76% entity recall while reportedly matching the backbone's distractor false-positive rate under misleading context. The semantic model reaches 89.70% macro-F1, 7.12 points above the strongest evaluated pronoun baseline. Context ablations show that the gain depends on matched discourse evidence. At the same time, general-ASR CER changes inconsistently across domains after the contextual stages.

Critical strengths
  1. Important problem decomposition. Separating lexical verification, where acoustics should veto unsupported candidates, from semantic disambiguation, where acoustics cannot resolve the written form, is conceptually useful.

  2. Strong contextual results. The 97.76% entity recall and 89.70% pronoun macro-F1 are substantial, and the latter exceeds the strongest reported pronoun baseline by 7.12 points.

  3. Evidence that discourse matters. The semantic ablations support the limited conclusion that the trained model uses matched context rather than solving the benchmark from audio alone.

  4. Clear presentation and release intent. The three stages and reward designs are understandable, and releasing the diagnostic suite would be useful to the community.

Critical weaknesses
  1. The general-ASR tradeoff remains unresolved. The strongest contextual models regress unevenly on ordinary ASR, by as much as 1.65 absolute CER points on one reported domain. Without uncertainty estimates or a Pareto analysis combining contextual gains, distractor errors, and general CER, it is unclear whether the final checkpoint improves realistic mixed traffic.

  2. The contribution of GRPO is not isolated. The stages change training data, context construction, objectives, and rewards, but there is no data- and compute-matched comparison with additional SFT or complete reward-component attribution. It is also unclear whether the reported lexical and semantic systems represent one general context policy or separate task specialists.

  3. The statistical and benchmark evidence is narrow. The main results are point estimates from modest diagnostic sets without independent-training variation or confidence intervals. The pronoun task covers three Mandarin written forms using constructed examples, while aggregate lexical metrics can conceal sensitivity to entity frequency, phonetic distance, list size, and context position.

  4. The closest established literature is incompletely positioned. The paper omits RLBR (ICASSP 2026), a direct reinforcement-learning method for contextual Speech-LLM ASR, and Mohebbi et al. (EMNLP 2023), a controlled study of contextual disambiguation for acoustically identical forms. These omissions weaken the method and semantic-motivation comparisons.

  • B. Ren, R. Fan, Y. Shen, W. Chen, and J. Li, "RLBR: Reinforcement Learning with Biasing Rewards for Contextual Speech Large Language Models," in Proc. 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 17637-17641, 2026, doi:10.1109/ICASSP55912.2026.11464286.
  • H. Mohebbi, G. Chrupała, W. Zuidema, and A. Alishahi, "Homophone Disambiguation Reveals Patterns of Context Mixing in Speech Transformers," in Proc. 2023 Conference on Empirical Methods in Natural Language Processing, pp. 8249-8260, 2023, doi:10.18653/v1/2023.emnlp-main.513.
  • J. Lee, J. Wang, E. Brown, L. Chu, S. S. Rodriguez, and J. E. Froehlich, "Towards Designing a Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation: A Demonstration of GazePointAR," in Adjunct Proc. 36th Annual ACM Symposium on User Interface Software and Technology, pp. 1-3, 2023, doi:10.1145/3586182.3615819.
  1. Limited evaluation on created benchmark. Any results on ContextASR-Bench?
Questions for authors
  1. What performance remains when context is obtained from natural conversation history or automatic retrieval and the relevant evidence is missing, stale, or contradictory?

  2. Can the authors report per-stage contextual and general-ASR results across independent training runs and reward weights, including a Pareto or realistic traffic-weighted utility analysis?

  3. Under identical examples, generated samples, training tokens, and compute, how does GRPO compare with additional matched SFT, and what does each reward component contribute?

  4. Are the lexical and semantic results produced by one checkpoint or separate specialists, and how does each trained checkpoint perform on the other contextual benchmark?

3 comments

Thank you for carefully reviewing our paper and providing constructive suggestions. Your comments have helped us further improve our experimental analysis and the presentation of the paper. Accordingly, we have added relevant experiments and more detailed explanations. Detailed data are recorded in the appendix. We address your questions one by one below.

You noted that general-ASR tradeoff remains unresolved. We apologize for the unclear presentation of baseline sources and evaluation settings. Due to time constraints, the baseline results in the original table were taken from existing public reports, whereas the results for our method were obtained through our own evaluation. We have now re-evaluated the baselines using the same evaluation code, explicitly distinguished previously reported and newly evaluated results, and provided links to the original sources. Since some re-evaluated results differ from public reports, the revised performance comparison will use results obtained under the unified evaluation procedure.

We have added evaluation results for each training stage in Table 6 in the appendix to further identify the stages at which performance changes occur. The training data for Stage I include subsets selected from WenetSpeech, KeSpeech, AISHELL, and Common Voice, as well as speech data collected from public video platforms and pseudo-labeled through automatic transcription.

The stage-wise results show that some of the performance gaps relative to the original backbone already emerge after the general ASR SFT in Stage I. Therefore, the entire performance decline of the final model cannot be attributed to the subsequent contextual optimization. The main methods presented in our paper (Stage II and Stage III) generally maintain or improve performance on the test sets relative to the performance upon entering each stage, only WS-Meeting CER worsen. The composition of the Stage I data and the quality of the pseudo-labels are factors that may affect adaptation; improving data filtering at this stage and expanding the coverage of meeting speech are directions for future optimization.

Table 6 (partial).

Lower is better. WS denotes WenetSpeech.

ModelAISHELL-1AISHELL-2WS-NetWS-Meeting
Ours (Stage I)3.142.975.576.74
Ours (Stage II)3.303.115.566.38
Ours (Full)3.072.995.557.53

We thank the reviewer for highlighting these relevant studies. RLBR is closely related to our work, applying reinforcement learning to contextual Speech-LLM ASR through rewards that emphasize biasing words and reference-aware optimization. We have clarified our methodological emphasis on deliberately confusable phonetic negatives, together with a separate semantic objective for written distinctions that acoustics alone cannot resolve. Mohebbi et al. (2023) analyze context mixing through French homophone disambiguation, whereas our work focuses on training for discourse-dependent transcription. GazePointAR uses multimodal cues to identify pronoun referents, while our pronoun task concerns selecting among acoustically identical written forms. These comparisons have been added at the end of Section 2.1 and the beginning of Section 2.2, with the revisions highlighted in blue.

Regarding your request to include results on ContextASR-Bench, since our model was trained almost exclusively on Chinese datasets, we evaluated it only on ContextASR-Speech-ZH and ContextASR-Dialogue-ZH. Compared with the original backbone, the full model achieves lower NE-WER and NE-FNR on both subsets. Detailed results can be found in Table 9 in the appendix.

Table 9. (partial)

Speech refers to ContextASR-Speech-ZH, and Dialogue refers to ContextASRP-Dialogue-ZH. Lower is better for all metrics. Best results in each column are shown in bold.

ModelSpeech WERSpeech NE-WERSpeech NE-FNRDialogue WERDialogue NE-WERDialogue NE-FNR
Qwen2-Audio9.9224.1030.027.0022.7626.17
Kimi-Audio1.9511.1315.282.9015.9116.68
Qwen2.5-Omni-7B1.849.8012.192.4014.0613.17
Qwen3-ASR-1.7B0.686.841.091.999.523.14
Ours (Stage II)0.947.071.043.0111.112.81
Ours (Full)0.866.701.002.019.392.59

Q1. Context robustness

We believe that the evaluation in Table 4 provides partial evidence. In Table 4, the correct contextual entities are replaced with phonetically similar distractors; this test is designed to examine recognition performance when the correct candidate words are missing or the provided candidates conflict with the audio content. Under this condition, our model achieves an entity recall of 83.49% and a CER of 3.20%, outperforming the backbone’s 81.27% and 3.68%, while maintaining the same distractor false-positive rate (8.86%), demonstrating good overall performance. This evaluation explores cases involving missing or misleading evidence to some extent, but outdated evidence has indeed not been explicitly evaluated. We will further clarify this scope and its limitations in the revised manuscript and investigate this issue in future work.

Q2. Training variation and trade-offs

Thank you for this suggestion. We trained the complete three-stage pipeline with two additional random seeds under the default configuration, keeping the data and hyperparameters fixed. We also evaluated three additional reward settings: two lexical and one semantic. Appendix Tables 11–13 report the per-stage general-ASR results, contextual performance, and robustness, with mean ± sample standard deviation for repeated runs.

We added Pareto comparisons covering general-ASR accuracy, contextual performance, and robustness in the final subsection of the appendix, titled “Contextual Trade-off Visualizations.” General-ASR CER is averaged over AISHELL-1 and AISHELL-2, weighted by test-set sample counts. The plots reveal a trade-off between AISHELL recognition accuracy and robustness to misleading context. Among nondominated configurations, higher misleading-context B-Recall is accompanied by higher AISHELL CER. The full model achieves the highest misleading-context B-Recall among the plotted configurations and higher pronoun macro-F1 than Stage III SFT, whereas Stage III SFT achieves lower AISHELL CER and higher lexical B-Recall. The full model is nondominated in the AISHELL CER versus misleading-context B-Recall comparison, based on the plotted configurations and point estimates.

Q3. SFT and reward comparisons

We conducted a new SFT ablation experiment to examine the advantages of GRPO over SFT while keeping the training examples, generated samples, training token counts, and computational costs nearly identical. You can find these results in the Ours (Stage III SFT) row of Table 7 in the appendix. To control for the introduction of task-specific supervision, we added a task-specific SFT baseline initialized from the same Context-SFT checkpoint and trained on the exact same 20,000 examples used for GRPO. As summarized in Appendix Table 10, task-specific SFT already achieves 88.29% pronoun macro-F1, while GRPO reaches 89.70% (+1.41 percentage points). Under misleading context, GRPO improves B-Recall from 72.06% to 83.49% and reduces D-FPR from 20.66% to 8.86%. These improvements come with slightly lower B-Recall on the main lexical benchmark than SFT (97.76% vs. 98.21%).

Table 10

All values are percentages. The better result for each metric is shown in bold.

MetricStage III SFTFull GRPO
Pronoun macro-F1 ↑88.2989.70
Lexical B-Recall ↑98.2197.76
Misleading-context B-Recall ↑72.0683.49
Misleading-context D-FPR ↓20.668.86
Misleading-context CER ↓3.733.20

Regarding the reward components, we added comparisons with modified lexical and semantic reward weights. Under misleading context, the configuration that rejection reward is 0 yields a D-FPR of 16.61% and B-Recall of 78.73%, compared with 8.86% and 83.49% for the default configuration. Comparing GRPO (0.3, 0.45, 0.25) with Ours (Full) in Appendix Table 13 shows that increasing the entity-word reward weight can improve entity-word recall. Increasing the pronoun reward weight relative to the ASR reward raises pronoun macro-F1 from 89.70% to 90.54%, while lexical CER increases from 2.14% to 2.33%. These results characterize reward-weight trade-offs.

Q4. Shared checkpoint

Both lexical and semantic results in Table 3 come from the same checkpoint. As described in Section 4.1, lexical and semantic samples are jointly trained in a single GRPO run with task-specific reward selection; they are not separate specialists. These results therefore reflect the performance of one shared model on both contextual benchmarks, rather than separate task-specific specialists.

Thank you again for the time and effort you devoted to reviewing our paper and for your valuable suggestions.

Reviewer_WjakOpenReview ↗

Response to authors

Thank you for your response. The supplementary experiments address most of my concerns. I am raising my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.