- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a post-training expert-routing framework for real-time duplex speech models. A lightweight logistic regression probe uses hidden states at the native listen-to-speak boundary to predict local answer failures and selectively escalate high-risk queries to a stronger expert model, without fine-tuning the speech model or requiring additional language-model generation. Experiments on an internal benchmark and four external speech-QA datasets demonstrate improved accuracy over local-only and matched-random routing, while reducing server waiting time compared with always escalating. Cross-modal transfer experiments, evaluation on a second model family, and a small user study further support the approach's potential. However, its effectiveness varies across failure types, and broader validation in real-world settings remains necessary.
Critical strengths
-
The method uses hidden states at the native listen-to-speak boundary to enable selective expert routing without modifying or fine-tuning the duplex model, making it easy to integrate and adjust at deployment time.
-
Experiments across internal and external speech-QA benchmarks show consistent gains over local-only and matched-random routing, while reducing server waiting time compared with always escalating. Cross-modal transfer and evaluation on a second model family further support its potential generalizability.
Critical weaknesses
-
The probe performs poorly on multi-step reasoning errors (AUC 0.47), raising concerns about its ability to identify important failure cases. This limits the reliability and general applicability of the proposed routing framework.
-
The user study includes only 13 participants and 16 task pairs, while the latency evaluation does not fully capture end-to-end user-perceived latency. These limitations weaken the evidence for practical deployment benefits and leave the claimed real-time advantages insufficiently established.
Questions for authors
-
Given the near-random AUC (0.47) for multi-step reasoning errors, how would the authors characterize the current method’s limitations in detecting reasoning failures?
-
Since TTFA excludes endpoint-detection and client playback delays, how representative are the reported latency gains of actual user-perceived latency in live deployment?
-
How sensitive are the probe and routing thresholds to distribution shifts across languages, tasks, and unseen speech inputs, given the varying escalation rates across external datasets?
-
Could the authors clarify whether a trained or calibrated text-based router would narrow the gap with the proposed hidden-state probe, beyond the zero-shot GPT-5.4 mini baseline?
-
Given the small user study (13 participants, 16 task pairs), how robust are the reported preference differences, particularly for simple factual tasks?
5 comments
Response to Reviewer XcXk (Part 2/5)
Q2: Does the original TTFA reflect the user's actual wait?
The original TTFA measures only when the server has prepared the answer. The added client-side TTFA includes audio transmission and playback scheduling, making it closer to the user's actual wait.
-
Timing to the start of a substantive answer: We use a single client clock to measure from the end of the user's final speech segment to client-side scheduling of the first substantive answer audio. Waiting prompts such as "Please hold on" do not count as an answer.
-
The long tail mainly comes from waiting for the expert: Session logs show that all of the slowest answers in this batch used the expert path, with most of the time spent waiting for the expert to return. We therefore report local and expert-answer waiting times separately.
-
Answered and unanswered sessions are both accounted for: Overall results cover both local and expert paths. Sessions without a substantive answer are counted separately rather than treated as fast responses.
Additional analysis: We reviewed existing session logs, split client-side TTFA by the actual answer source, and checked the timelines of long-tail sessions.
Client-side TTFA: Local answers and expert handoff (240 questions)
| Answer source | Answered | Mean (s) | Median (s) |
|---|---|---|---|
| Local answer | 118 | 3.05 | 3.05 |
| Expert handoff | 119 | 10.24 | 8.85 |
| Overall | 237 | 6.66 | 5.54 |
Response to Reviewer XcXk (Part 3/5)
Q3: Does the probe work on datasets it did not see during training?
Yes. After training on the original data, the probe is kept fixed and applied directly to unseen test sets, where it still distinguishes questions that are more likely to be answered incorrectly.
-
The decision method is set using the original data: The original calibration set is used to train the probe and select the layer and internal features. These choices are finalized before the resulting probe is evaluated on the subsequent test sets.
-
No retraining on the new test sets: We fix the probe's weights, layer, and features and directly evaluate a separate set of Chinese questions. These questions were deduplicated against existing data and were not used to fit the probe, so the test evaluates transfer to unseen data.
-
Failure risk remains distinguishable on unseen tests: Across factual QA, knowledge exams, and causal commonsense tasks, the fixed probe ranks questions that are more likely to be answered incorrectly ahead of others. Its decision signal therefore transfers to these datasets that were not used in training.
Additional reporting: These experiments were completed before submission. We now clarify the training-test separation and present the unseen-test results first.
Unseen Chinese test sets: Direct evaluation with a frozen probe
| Dataset | AUC |
|---|---|
| Chinese SimpleQA | 0.719 |
| C-Eval | 0.730 |
| CMMLU | 0.642 |
| XCOPA-zh | 0.693 |
| Equal-weight task mean | 0.696 |
External English tasks: Additional results with a fixed probe
The same probe still distinguishes questions that are more likely to be answered incorrectly on these new test sets. Different proportions of questions trigger escalation across datasets, so expert-call rates also differ.
| Dataset | AUC | Expert-call rate (%) |
|---|---|---|
| Speech TriviaQA | 0.797 | 55.2 |
| Speech WebQ | 0.779 | 56.8 |
| Speech Llama | 0.744 | 21.2 |
| SD-QA | 0.812 | 62.5 |
Response to Reviewer XcXk (Part 1/5)
We thank you for the feedback. We address your questions below.
Q1: If execution errors are hard to detect, what value does the probe provide?
The probe directs expert calls to questions that need more help. In this comparison at the same expert-call budget, it corrects more errors than random escalation, including some execution errors.
Execution errors are not the main source of failure: These occur when the model understands the question but makes a mistake in calculation, counting, or a reasoning step. In this set of failures, confident, specific but incorrect answers are more common, and they account for most of the probe's error-recovery benefit.
A single decision can still correct some execution errors: With local and expert answers fixed, the probe uses the same number of expert calls as random escalation and corrects more execution errors. It also corrects more confident but incorrect answers.
Additional experiment: Building on the existing error-type analysis, we added random escalation at the same expert-call budget and compared error-recovery rates with fixed answers.
Error types among 600 incorrect local answers
Fixed-answer replay: Error recovery at the same budget
Both routing methods call the expert on 687 of 1,392 questions (49.4%).