acceptodds
Under review as a conference paper at ICLR 2027

Diagnosing and Mitigating Long-Session Degradation in Full-Duplex Speech Models

Abstract

End-to-end full-duplex spoken dialogue models enable natural, real-time spoken interactions with interruptions, backchannels, and overlapping speech, yet their reliability over extended sessions remains poorly understood. Existing evaluations have largely focused on short-horizon interactions, and even studies involving longer conversations have not systematically characterized how model behavior evolves over the course of a session. In this work, we address this gap by first characterizing how specific dialogue capabilities change over extended interactions and then developing methods to mitigate long-session degradation while improving the efficiency and generalization of extended-session inference. To characterize long-session behavior, we introduce a closed-loop evaluation framework for long-session stability and apply it to four publicly available full-duplex speech models in ten-minute conversations. Across models, we observe widespread degradation over time in responsiveness, content quality, and speech quality, highlighting the difficulty of maintaining stable conversational behavior over extended interactions. To mitigate this degradation, we propose a self-distillation-based training framework for lightweight LoRA adaptation that improves long-session stability with a small amount of additional training. We further introduce a two-resolution context (TRC) architecture that preserves recent interactions in full detail while compressing older interaction history into key-value representations, enabling models to learn how to effectively leverage both short-term and long-term context to maintain content and speech quality over longer sessions while reducing memory and computational costs. We apply our methods to three architectures with distinct formulations of full-duplex conversational context and evaluate long-session stability, information retention, and computational efficiency. Lightweight adaptation reduces performance degradation over ten-minute sessions across all three models. With TRC, the models also maintain these gains beyond the training horizon while reducing KV-cache growth by more than 90%

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Full Duplex Speech Models, Context Compression, Long-Session Dialogue, Gist Memory, Sefl-Distillation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DA51OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates long-session degradation in full-duplex speech models, including declining responsiveness, conversational quality, and speech quality. The authors propose privileged-context self-distillation to improve behavioral stability and a Two-Resolution Context (TRC) architecture that preserves recent context while compressing older dialogue. Experiments on three model architectures show that the proposed approach generally improves stability within the 10-minute training horizon and, when combined with TRC, better preserves performance in 20-minute sessions. TRC also reduces KV cache growth by 91.7%–95.8%. However, the gains vary across models, responsiveness issues remain in some cases, and compressed memory can impair information recall. Overall, the paper provides a useful evaluation framework and a promising approach to improving long-session stability.

Critical strengths
  1. The paper evaluates degradation in responsiveness, conversational quality, and speech quality across multiple full-duplex speech models. This provides a useful evaluation framework and highlights an important but underexplored problem in sustained speech interaction.

  2. The proposed privileged-context self-distillation and Two-Resolution Context (TRC) methods improve long-session stability in several settings. Experiments show that TRC reduces KV cache growth by 91.7%–95.8% and better preserves performance in 20-minute sessions, supporting the practical value of the approach.

Critical weaknesses
  1. Insufficient validation under realistic full-duplex interactions. The user simulator does not initiate interruptions or backchannels, leaving the proposed methods untested under important real-world interaction scenarios. This weakens the external validity of the evaluation and makes it difficult to conclude that the methods reliably address long-session degradation in practical settings. This is a significant concern for acceptance because realistic full-duplex behavior is central to the paper's motivation.

  2. Incomplete and model-dependent mitigation of degradation. TRC does not consistently improve responsiveness across architectures, and context compression can impair long-range memory recall. These results suggest that the proposed methods do not comprehensively resolve long-session degradation. Without stronger evidence of robustness across models and interaction conditions, the paper's claims of general effectiveness are not fully supported.

Questions for authors
  1. How do the authors justify the effectiveness of their methods when the simulator does not include user-initiated interruptions or backchannels?

  2. Why does TRC fail to consistently improve responsiveness, particularly for MiniCPM-o 4.5? What evidence supports its effectiveness across different architectures?

  3. How much information is lost through TRC compression, and how does this loss affect long-session conversational quality?

  4. How can the authors demonstrate that the proposed methods generalize beyond the 10-minute training horizon, given the limited evaluation up to 20 minutes?

  5. Have the authors validated the LLM-based conversation quality scores and ASR-based WER against human judgments to confirm that the reported improvements reflect actual user-perceived quality?

6 comments

We appreciate the careful review and constructive feedback. Based on the points raised in the review, we conducted additional experiments to strengthen our evaluation. We address each point below. All additional results, evaluation settings, and discussions in this response will be added to the Appendix of the revised paper.


W1/Q1: Validation under realistic full-duplex interaction (1/3)

We agree that the main protocol leaves the methods untested under user-initiated interruptions and backchannels. We address this with two additional evaluations. First, we repeated the long-session evaluation with a full-duplex user that initiates these behaviors, to test whether the mitigation holds under realistic interaction. Second, we evaluated the adapted models on a full-duplex benchmark, to test whether they still handle behaviors such as user interruptions.

(1) Long-session evaluation with a full-duplex user

We use OpenAI's GPT-Live-1, a commercial full-duplex API operating in real time, as the user. Unlike our simulator, GPT-Live-1 determines its own response timing and initiates backchannels and interruptions. The setting is as follows.

  • Protocol: The models use the same checkpoints as in Table 1. The timeout, retry, and termination rules and the metrics follow Sec. 3.1. Because GPT-Live-1 also speaks while the model is talking, we additionally classify its utterances: those of up to 1 s are treated as backchannels, and longer ones that begin while the model is speaking are counted as interruption attempts. Unlike the main protocol, which advances in logical time, the interaction runs in real time.
  • User setting: GPT-Live-1 receives the same persona and topic instructions as our simulator, together with instructions to backchannel frequently and to interrupt mid-sentence. In practice, it produced an average of 3.9 backchannels and 0.9 interruption attempts per minute across all sessions.
  • Episodes: Given the cost and runtime of real-time interaction, we use one third of the main evaluation scale. This gives 30 episodes per model and condition, pairing the Balanced persona with all ten topics, three episodes each. Base sessions last 10 minutes and the others 20 minutes, as in Table 1.
  • Models: Raon-SpeechChat is excluded because it cannot sustain real-time inference in our computing environment.

Degradation of the base models. We first compare the Base models under our simulator and GPT-Live-1. Each cell reports the 0–2 → 8–10 min intervals, and the values for our simulator are from Table 1. As in the paper, we apply the reporting thresholds of Appendix A.4 to avoid unstable estimates from too few observations; – marks intervals that do not meet them.

ModelUserRT (%) ↓TS (%) ↓FTO (s)CQ ↑WER (%) ↓Spk sim. ↑
PersonaPlexOur simulator0.0 → 17.60.0 → 18.9−0.27 → 0.404.78 → 1.295.4 → 44.30.733 → 0.337
GPT-Live-13.8 → 35.80.0 → 36.70.20 → 0.454.60 → 1.167.1 → 53.10.720 → 0.224
MiniCPM-o 4.5Our simulator9.4 → 66.76.7 → 66.70.40 → 1.904.81 → 4.192.6 → 13.60.775 → 0.749
GPT-Live-136.9 → 55.340.0 → 76.70.63 → –4.52 → 4.756.0 → 8.90.764 → 0.735

With GPT-Live-1, the degradation of both models largely follows the pattern observed with our simulator, while its severity and onset differ. PersonaPlex loses responsiveness, content quality, and speech quality more severely than with our simulator. MiniCPM-o 4.5 again degrades mainly in responsiveness while keeping its content quality, but the decline starts earlier: 36.9% of response opportunities already time out at 0–2 min, and 76.7% of sessions terminate by 10 min. The degradation found in our evaluation is therefore not confined to calm turn-taking; it also arises when the user backchannels and interrupts in real time.

W1/Q1: Validation under realistic full-duplex interaction (2/3)

Mitigation of the degradation. We next evaluate Ours (TRC) in 20-minute sessions with GPT-Live-1. As an additional setting, we also evaluate Ours (TRC, 30-s teacher) on MiniCPM-o 4.5, a variant of TRC trained on the same trajectories, with the teacher's context window for computing the targets shortened from up to 150 s to 30 s. Each cell reports the 8–10 / 18–20 min intervals; – marks intervals beyond the 10-minute Base sessions or below the reporting thresholds.

ModelMethodRT (%) ↓TS (%) ↓FTO (s)CQ ↑WER (%) ↓Spk sim. ↑
PersonaPlexBase35.8 / –36.7 / –0.45 / –1.16 / –53.1 / –0.224 / –
Ours (TRC)12.6 / 8.73.3 / 10.00.23 / 0.233.91 / 3.2811.7 / 18.20.599 / 0.537
MiniCPM-o 4.5Base55.3 / –76.7 / –– / –4.75 / –8.9 / –0.735 / –
Ours (TRC)66.7 / –96.7 / 100.0– / –– / –– / –– / –
Ours (TRC, 30-s teacher)7.8 / 9.10.0 / 3.30.96 / 0.844.65 / 4.604.6 / 4.70.772 / 0.779

On PersonaPlex, Ours (TRC) mitigates the degradation through 20 minutes with GPT-Live-1 as well. It keeps content and speech quality well above the late-session Base level and terminates far fewer sessions: 10.0% by 20 min, against 36.7% of Base sessions by 10 min. Its response timeouts are more frequent than with our simulator, but they do not increase further beyond the training horizon.

On MiniCPM-o 4.5, in contrast, Ours (TRC) does not mitigate the degradation: 96.7% of its sessions terminate by 10 min, more than for the Base model. This reveals that, on this model, the configuration used in the paper does not generalize well to interaction with more active full-duplex behaviors, a limitation already partly visible with our simulator. Changing the configuration, however, substantially improves the mitigation. With Ours (TRC, 30-s teacher), response timeouts stay below 10%, only one session (3.3%) terminates through 20 minutes, and content and speech quality remain stable.

Overall, the degradation of the base models reappears when the user is a real-time full-duplex system that backchannels and interrupts, which suggests that our findings on degradation are not specific to the calm turn-taking or the logical-time protocol of our simulator. As for mitigation, these results suggest that TRC itself can generalize to such interaction, but that applying it with the same configuration across models may not be sufficient. They also point to both a need and room for adapting the configuration to each model, with attention to the trade-offs this could introduce.

W1/Q1: Validation under realistic full-duplex interaction (3/3)

(2) Full-duplex behaviors of the adapted models

We evaluated the base models and Ours (TRC) on Full-Duplex-Bench v1.0 (Lin et al., "Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities," ASRU 2025). We used its public implementation for transcription and metric computation.

  • Prompts: PersonaPlex uses the task-specific prompts specified in the benchmark repository. The benchmark specifies no configuration for MiniCPM-o 4.5 or Raon-SpeechChat, so we applied the same prompts to them.
  • Decoding: All models use the default sampling settings of their released inference code. The base and TRC variants of each model share identical settings.
  • Judge: The user-interruption judge is GPT-4o (gpt-4o-2024-08-06), because GPT-4-turbo, used in the original script, has been retired.
  • Raon-SpeechChat: Its technical report (Kim et al., "Raon-Speech Technical Report," 2026, Appendix G) evaluates this benchmark with an internal evaluator that uses different turn-detection criteria. Our absolute values for this model therefore differ substantially from the reported ones. The comparison below uses the same public protocol for both variants.
ModelMethodPause Synth. TOR ↓Pause Candor TOR ↓BC TOR ↓BC Freq ↑BC JSD ↓Turn TOR ↑Turn Lat. (s) ↓Intr. TOR ↑Intr. GPT ↑Intr. Lat. (s) ↓
PersonaPlexBase0.3940.4540.2550.1220.7350.9580.290.9154.600.19
Ours (TRC)0.2550.3290.2730.0780.7810.8990.310.9254.480.41
MiniCPM-o 4.5Base0.2120.5880.2180.0170.9100.9580.360.8702.840.50
Ours (TRC)0.0580.1200.0730.0110.9450.8490.650.7102.150.46
Raon-SpeechChatBase0.8540.9170.4730.1980.6601.0000.001.0002.920.29
Ours (TRC)0.7450.8750.3090.1430.7111.0000.011.0002.940.26

The adapted models still take the turn in most cases when the user yields it and respond to most user interruptions (TOR of at least 0.849 and 0.710, respectively), but the adaptation shifts these behaviors to some extent. Two changes are shared by all three models. They handle user pauses better, with lower TOR on both pause subsets (e.g., 0.588 → 0.120 on Candor for MiniCPM-o 4.5), and produce backchannels less frequently and with timing further from that of human listeners (higher JSD). Beyond these shared changes, PersonaPlex and MiniCPM-o 4.5 take the turn less reliably when the user yields it (smooth turn-taking TOR 0.958 → 0.899 and 0.958 → 0.849), and MiniCPM-o 4.5 shows a marked decline in handling user interruptions (TOR 0.870 → 0.710; GPT score 2.84 → 2.15).

The decline in interruption handling can be expected from the trajectory design. The simulated user never interrupts the model (Appendix B.5), so interruption handling receives no supervision during training. In contrast, the models' own backchannels and floor-taking were left unconstrained during trajectory construction, yet these behaviors shifted as well. This suggests that additional training on a small dataset collected with a limited type of user simulator is cost-efficient but can measurably shift a model's full-duplex behavior beyond what the training targets, in both favorable and unfavorable ways, so such adaptation warrants caution. Examining how this effect changes when trajectories are generated with a full-duplex user simulator such as GPT-Live-1, or when training draws on external full-duplex dialogue data, is an important direction that we leave for future work. We will include this discussion in the limitations.

In summary, these two evaluations suggest that our methods are not confined to the calm turn-taking of our main protocol. With a user that initiates backchannels and interruptions, TRC can still mitigate long-session degradation, and none of the full-duplex behaviors evaluated on the benchmark breaks down after the adaptation. Both evaluations, however, also show that the outcome depends on how the adaptation is configured: the training configuration needs to be adapted to each model, and the user behaviors covered by the training trajectories affect full-duplex behaviors that the training does not directly target.

Reviewer_Lw1iOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates the reliability of E2E full-duplex SDMs on long-horizon interactions, which is generally overlooked in existing full-duplex benchmarks and prior model analysis. The paper studies four representative open-source full-duplex SDMs with distinct architectures, including Moshi, PersonaPlex, MiniCPM-o 4.5, and Raon-SpeechChat. To study the models, the paper develops a closed-loop simulator with LLM, TTS, VAD, and ASR, and then studies the SDMs in 10-minutes conversation. The paper evaluates model behavior in terms of responsiveness, content quality, and speech quality, and finds that all four models degrade at least in one dimension, and with distinct degradation patterns across models. To mitigate the degradations, the authors propose (a) privileged-context self-distillation for efficient long-session adaptation without an external full-duplex dialogue corpus, and (b) a two-resolution context (TRC) architecture that combines a fine-grained local window with compressed memory of earlier dialogue. They verify that privileged-context self-distillation shows that base model’s short-session behavior can provide effective supervision for this mitigation, and combining this training with TRC helps preserve the gain in content and speech quality beyond the training horizon while reducing the cost of retaining the dialogue history, showing 91.7–95.8% lower KV growth.

Critical strengths
  1. The reliability of full-duplex SDMs under long-horizon interactions is indeed an underexplored problem, hence this paper targets a valuable and practical direction for full-duplex SDMs. Moshi and PersonaPlex basically share the same architecture, so this paper investigates three distinct architectures of full-duplex SDMs, Moshi/PersonaPlex, MiniCPM-o 4.5, and Raon-SpeechChat, which helps enhance the generality of findings and generalizability of mitigation approaches.

  2. It is interesting and informative to observe the distinct degradation patterns, as shown in Section 3, that is, Moshi gradually declines in conversation quality and speaker similarity, followed by a sharp loss of responsiveness late in the session. PersonaPlex shows more pronounced degradation in content and speech quality, but still gradually; Raon-SpeechChat collapses late in the session; MiniCPM-o degrades mainly in responsiveness. These findings will inspire deeper analysis of causes of collapses and proposal of mitigation approaches, although analysis of causes of these degradation is missing in this paper.

  3. The work has moderate novelty, with context distillation with summary as privileged information, and TRC is basically an adaptation of CCM and UniGist. On the other hand, the key algorithmic designs are well motivated and carefully engineered. Self-distillation does not rely on external full-duplex corpus. Separate KL terms allow response content and speaking behavior to be weighted independently.

Critical weaknesses
  1. As mentioned under Strengths, Section 3 reports the findings of different degradation patterns across the four studied models, but deep analyses of the causes of these degradations and relations to their model architecture and training paradigm are not provided. Some possible reasons might be attention sinks or loss of prompt, positional extrapolation, self-reinforcement on model’s output, or even induced by the simulators. However, none of these is studied and why these models degrade is not explained in this work.

  2. Some important baselines and analyses are also missing.

  a. As mentioned in Section 4.1, the teacher restarts with a dialogue summary at segment boundaries. However, this teacher is not evaluated in the closed-loop simulation, as a baseline.

  b. A window-only ablation, that is, TRC without gist tokens, would be useful to provide insights on the source of the gain (forgetting? compressing?).

  c. There is no comparison between separate versus joint content and control KL.

  d. There is no StreamingLLM style under the same distillation mechanism; therefore, it is not ready to attribute TRC’s gain on PersonaPlex to gist compression.

  1. As described in Section 3.1, the user simulator does not generate interruptions or backchannels, so it does not fully simulate full-duplex behaviors with the model. Moreover, it seems that training and evaluation use the same user simulator, which may overfit to simulator artifacts. It would be useful to evaluate with different user simulators, and user simulators that generate appropriate interruptions and backchannels.

  2. When using LLM-as-judge, it is important to either cite evidences in prior studies or empirically validate that the LLM-as-judge has a high correlation with human assessment. For example, CQ judge in this work is not validated this way.

  3. The evaluations focus on checking the effectiveness in mitigating the degradations in responsiveness, content and speech quality, yet it is unclear whether the mitigation approaches will impact full-duplex behaviors. It would be important to compare the performance on some full-duplex benchmarks, e.g., Full-Duplex-Bench series, to check interruptions and backchanneling w/ and w/o the mitigation approaches.

  4. Analysis or explanations of some results are missing. For example, Table 1, the paper only reports that combining training with TRC doesn’t improve over standard LoRA (they are actually worse), but does not provide explanations or insights on these results.

Questions for authors

Please address the points under Weaknesses.

8 comments

We appreciate the careful review and constructive feedback. Based on the points raised in the review, we conducted additional experiments to strengthen our evaluation. We address each point below. All additional results, evaluation settings, and discussions in this response will be added to the Appendix of the revised paper.


W1/W2/W6: Additional ablations and analyses (1/4)

We ran additional ablations and analyses on the three models (PersonaPlex, MiniCPM-o 4.5, Raon-SpeechChat) and discuss each point separately below.

(1) Sources of degradation and of TRC's stability gain (W1, W2-b, W2-d)

We additionally evaluate four variants that retain only the local context, i.e., a fixed prefix and a recent raw window of 9.6 s (PersonaPlex, Raon-SpeechChat) or 10 s (MiniCPM-o 4.5), without gists or out-of-window raw tokens. We compare them with one another and with Ours (TRC), and present the main findings below the table. The four variants are:

  • StreamingLLM (4 sinks): StreamingLLM applied without training, keeping the first four positions as attention sinks.
  • StreamingLLM (TRC prefix): the same, but keeping TRC's fixed prefix, i.e., the prompt and the model-specific attention-sink budget (Appendix B.4).
  • Ours (Window): StreamingLLM (TRC prefix) trained with the same self-distillation protocol as Ours (TRC).
  • Ours (TRC) w/o gists: Ours (TRC) evaluated with all gists masked.

The table below extends the 20-minute long-session evaluation of Table 1 to these variants. Each cell reports the 8–10 / 18–20 min intervals. As in the paper, we apply the reporting thresholds of Appendix A.4 to avoid unstable estimates from too few observations; – marks intervals beyond the 10-minute Base sessions or below these thresholds.

ModelMethodRT (%) ↓TS (%) ↓FTO (s)CQ ↑WER (%) ↓Spk sim. ↑
PersonaPlexBase17.6 / –18.9 / –0.40 / –1.29 / –44.3 / –0.337 / –
StreamingLLM (4 sinks)0.8 / 0.50.0 / 0.0−0.26 / −0.274.51 / 4.295.3 / 6.90.488 / 0.475
StreamingLLM (TRC prefix)0.0 / 0.30.0 / 0.0−0.25 / −0.244.28 / 4.215.8 / 6.80.648 / 0.642
Ours (Window)0.4 / 0.00.0 / 0.0−0.22 / −0.224.17 / 3.987.7 / 8.50.649 / 0.649
Ours (TRC) w/o gists0.7 / 1.30.0 / 0.0−0.22 / −0.214.44 / 4.235.6 / 5.30.683 / 0.681
Ours (TRC)0.4 / 1.40.0 / 4.4−0.23 / −0.263.84 / 3.5313.3 / 12.80.617 / 0.579
MiniCPM-o 4.5Base66.7 / –66.7 / –1.90 / –4.19 / –13.6 / –0.749 / –
StreamingLLM (4 sinks)34.1 / 46.22.2 / 7.82.42 / –4.17 / 4.2812.2 / 36.70.719 / 0.718
StreamingLLM (TRC prefix)30.5 / 38.20.0 / 0.02.29 / –4.07 / 4.2020.4 / 57.10.773 / 0.776
Ours (Window)24.3 / 34.831.1 / 44.41.60 / 1.714.45 / 4.445.2 / 4.20.766 / 0.771
Ours (TRC) w/o gists3.4 / 12.80.0 / 0.01.14 / 1.434.60 / 4.403.7 / 5.30.778 / 0.769
Ours (TRC)34.6 / 51.645.6 / 76.71.87 / –4.35 / 4.645.3 / 5.80.771 / 0.770
Raon-SpeechChatBase27.5 / –30.0 / –0.06 / –1.63 / –70.5 / –0.520 / –
StreamingLLM (4 sinks)80.0 / 0.098.9 / 98.9– / –– / –– / –– / –
StreamingLLM (TRC prefix)0.0 / 0.50.0 / 0.0−0.11 / −0.063.76 / 2.9813.5 / 43.40.809 / 0.784
Ours (Window)2.4 / 2.611.1 / 16.70.04 / 0.004.10 / 3.2417.3 / 43.90.798 / 0.758
Ours (TRC) w/o gists0.2 / 0.63.3 / 4.4−0.03 / −0.014.13 / 3.1914.9 / 39.00.805 / 0.775
Ours (TRC)1.1 / 1.03.3 / 3.3−0.04 / −0.014.19 / 3.2915.3 / 44.30.802 / 0.767

W1/W2/W6: Additional ablations and analyses (2/4)

Overall, retaining only a fixed prefix and a recent window substantially reduces the degradation of the base models. Both StreamingLLM variants, which apply this eviction without training, mitigate the decline across models and metrics, except for the collapse of StreamingLLM (4 sinks) on Raon-SpeechChat discussed below. Within the training horizon, TS falls from 18.9, 66.7, and 30.0% to at most 2.2%, and at 8–10 min, PersonaPlex's WER falls from 44.3 to 5.3–5.8% and Raon-SpeechChat's from 70.5 to 13.5%. This suggests that the degradation of the base models arises largely from the accumulating raw context and can be substantially avoided by evicting old context. However, even though eviction discards all context outside the window, some degradation remains on MiniCPM-o 4.5 and Raon-SpeechChat. By 18–20 min, MiniCPM-o 4.5 still shows an RT of 38.2–46.2% and a WER of 36.7–57.1%, and Raon-SpeechChat's WER rises to 43.4%. Most notably, the FTO of MiniCPM-o 4.5 already rises to 2.29–2.42 s at 8–10 min, exceeding even the base model's 1.90 s. This suggests that its FTO increase is driven not mainly by the accumulated raw context but by a drift that arises directly in the model's hidden states.

Comparing the two StreamingLLM variants further shows what the prefix must retain. First, preserving the prompt is key to maintaining speaker similarity. With the TRC prefix, speaker similarity remains stable through 20 minutes in all three models (0.642, 0.776, and 0.784 at 18–20 min), whereas StreamingLLM (4 sinks), which does not fully retain the prompt, lowers it on PersonaPlex and MiniCPM-o 4.5 (0.475 and 0.718), most markedly on PersonaPlex. Second, the results suggest that the attention sinks a model relies on may not be confined to its first few tokens. On Raon-SpeechChat, StreamingLLM (4 sinks) leads to an almost complete collapse (TS 98.9%), whereas StreamingLLM (TRC prefix), which additionally retains the full prompt and the first three dialogue frames (Appendix B.4), terminates no session (TS 0.0%).

Beyond context eviction, our self-distillation brings further gains, but these gains depend on the training design. This is most evident in the three variants that share the same prefix and window: StreamingLLM (TRC prefix) applies no training, Ours (Window) is trained with the same eviction applied, and Ours (TRC) w/o gists is trained with TRC and then evaluated with the same eviction, i.e., with all gists masked. Training under eviction alone has a limited and mixed effect. On PersonaPlex and Raon-SpeechChat, Ours (Window) shows no clear improvement over the untrained variant. On MiniCPM-o 4.5, it slows the rise of FTO and lowers WER (1.60 vs. 2.29 s at 8–10 min; 4.2 vs. 57.1% at 18–20 min), but at the cost of a sharp rise in terminated sessions (0.0 → 44.4% at 18–20 min), which also appears on Raon-SpeechChat (0.0 → 16.7%). In contrast, Ours (TRC) w/o gists improves mitigation across the three models and most metrics. On MiniCPM-o 4.5, it lowers FTO to 1.14 s at 8–10 min (vs. 2.29 s) and RT to 12.8% at 18–20 min, while keeping WER at 5.3% and terminating no session; on PersonaPlex, it improves WER and speaker similarity over the untrained variant (5.3 vs. 6.8% and 0.681 vs. 0.642); and on Raon-SpeechChat, it improves late CQ and WER (3.19 vs. 2.98 and 39.0 vs. 43.4%) with few sessions terminating (4.4%). In short, our self-distillation consistently improves on the untrained variant when the model is trained with TRC, but not when it is trained under eviction alone. Since both trained variants retain the same context at inference, this difference comes from training, suggesting that TRC serves not only as a memory at inference but also as a training structure under which our self-distillation stabilizes the model itself. Beyond showing this role of TRC, these results also partly reveal a limitation of the proposed training. Ours (TRC) w/o gists is comparable to or more stable than Ours (TRC) in all three models, most markedly on MiniCPM-o 4.5 (RT 12.8 vs. 51.6% and TS 0.0 vs. 76.7% at 18–20 min). This implies that gist tokens, although they compress raw tokens, partly inherit the effect of token accumulation on degradation.

W1/W2/W6: Additional ablations and analyses (3/4)

(2) Stability–recall trade-off

The results in (1) suggest that accumulating gist tokens, like accumulating raw tokens, contributes to degradation. We hypothesize that one source lies in the teacher's context. The teacher of Ours (TRC) sees the entire raw context of the current segment, so the KL and hidden-state targets also reflect what it reads from raw tokens beyond the student's window, and the gists are trained to reproduce this readout. To test this, we trained Ours (TRC, 30-s teacher) on MiniCPM-o 4.5, the model on which Ours (TRC) does not sufficiently mitigate degradation, using the same trajectories but shortening the teacher's context window from up to 150 s to 30 s. Its teacher receives all dialogue before this window as a text summary, and all other training settings are unchanged. The table reports the 18–20 min interval and recall at 5 / 10 min (Appendix Table 25); – marks a value below the reporting thresholds, and bold marks the better value.

MethodRT (%) ↓TS (%) ↓FTO (s)CQ ↑WER (%) ↓Spk sim. ↑Recall (%) ↑
Ours (TRC)51.676.7–4.645.80.77057 / 46
Ours (TRC, 30-s teacher)22.40.01.534.685.60.77935 / 29

Shortening the teacher's window makes MiniCPM-o 4.5 markedly more stable. At 18–20 min, TS drops from 76.7 to 0.0% and RT from 51.6 to 22.4%, while CQ, WER, and speaker similarity remain comparable. This is consistent with our hypothesis: as the targets reflect less of the raw context beyond the student's window, the gists inherit less of the long-context behavior that degrades the base model. The same change, however, also lowers recall, from 57 to 35% at 5 min and from 46 to 29% at 10 min. Since the teacher sees the earlier dialogue only as a summary, the targets carry less fine-grained information from the distant raw context, and so do the gists.

Ours (TRC, 30-s teacher) thus lies between Ours (TRC) and Ours (TRC) w/o gists in (1); the latter retains no context beyond the window and keeps RT lowest (12.8% at 18–20 min). Across the three, the more long-range information the model retains, the less stable it is late in the session, forming a trade-off between stability and memory recall. Further work is needed on methods that push the Pareto frontier of this trade-off outward, and the three variants above can serve as reference points for such methods.

(3) Quality validation of the teacher trajectory

The table compares constructed teacher trajectories with the base models over the same span (0–2 min / 8–10 min).

ModelMethodFTO (s)CQ ↑WER (%) ↓Spk sim. ↑
PersonaPlexBase−0.27 / 0.404.78 / 1.295.4 / 44.30.733 / 0.337
Teacher−0.20 / −0.234.70 / 4.085.2 / 6.30.729 / 0.604
MiniCPM-o 4.5Base0.40 / 1.904.81 / 4.192.6 / 13.60.775 / 0.749
Teacher0.57 / 0.824.85 / 4.483.4 / 4.10.777 / 0.776
Raon-SpeechChatBase−0.07 / 0.064.52 / 1.6311.1 / 70.50.816 / 0.520
Teacher0.31 / −0.034.50 / 4.479.7 / 10.50.824 / 0.812

Within this span, the teacher stays much closer to its early-session quality than the base models (e.g., Raon-SpeechChat CQ 4.47 vs. 1.63 and WER 10.5 vs. 70.5% at 8–10 min).

(4) Content/Control KL loss

We compared Ours (TRC) with a variant that pools the KL over content and control positions, so that each group's share follows its frequency, with all other settings unchanged. The table reports the relative difference of the pooled variant's validation content KL from that of Ours (TRC) at steps 50 / 100 / 500 (positive: higher).

ModelContent positions (%)Content KL Δ (%)
PersonaPlex321.8 / 0.7 / −0.2
MiniCPM-o 4.560−2.9 / −4.4 / −0.7
Raon-SpeechChat1710.0 / 8.5 / 0.7

Given the different proportions of content positions across models, we weight the two groups 1:1 so that neither dominates. Pooling changes how fast the content KL decreases early in training (e.g., 8.5–10% higher on Raon-SpeechChat over the first 100 steps), but both objectives converge to nearly the same point after sufficient training, likely because the hidden-state loss also constrains the student. In short, our separation of the content and control KLs is motivated by the differing proportions of content positions across models and by the difference that pooling makes early in training, rather than by a difference in final performance.

Reviewer_H5kxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the degradation of full-duplex speech models during extended conversations. It introduces a closed-loop user simulator and evaluates four publicly available models. To mitigate degradation, the authors utilized OPSD to distillate teacher distribution constructed on summaries of earlier history and short dialogue segments. Experiments on three architectures show improvements within the training horizon and better preservation of some quality measures in twenty-minute sessions. The paper also reports substantially reduced analytical KV-cache growth, alongside model-dependent losses in factual recall.

Critical strengths
  1. Long-form full-duplex is a valuable topic, and using compressed memory to construct a teacher distribution for on-policy self-distillation is an ingenious solution.
  2. Experiments show meaningful gains and expose trade-offs. For PersonaPlex, TRC reduces the twenty-minute terminated-session rate from 87.8% with LoRA alone to 4.4%; for Raon-SpeechChat, it reduces this rate from 98.9% to 3.3% (Table 1). These are substantial improvements in session survival. Reporting recall degradation and distinguishing analytical KV savings from backbone latency measurements also helps assess the method’s limitations.
Critical weaknesses
  1. A key benefit of full-duplex communication is that the user can interrupt the model’s response; however, the benchmark is set up so that the user always waits for the complete model response, which diminishes the value of the results and conclusions.
  2. What is the latency of the cascaded user simulator? The latency of a cascaded pipeline consisting of VAD, ASR, LLM, and TTS is significantly higher than the latency of a full-duplex model; therefore, it is likely insufficient to meet the requirements of simulating a user, which is an authentic, real-time interaction filled with interruptions and interjections.
  3. High late-session quality does not necessarily imply reliable session-level behavior. CQ and speech-quality metrics exclude terminated or otherwise ineligible intervals, which the paper clearly documents. However, the resulting late-session scores describe a selected set of surviving observations. For MiniCPM-o with TRC, Table 1 reports late CQ of 4.64 and WER of 5.8%, alongside a 76.7% terminated-session rate, higher than the 64.4% under LoRA alone. These results support better quality when an answer is produced, but not an unqualified improvement in overall reliability. Survival and conditional quality should be presented together, ideally with an additional episode-level successful-interaction measure.
Questions for authors
  1. The fonts used for the figures are not consistent. In addition, the layout of Figure 3 is severely inconsistent with the reading order, which affects the efficiency of comprehension.
  2. It's unclear the difference between Response timeout (RT) and terminated sessions (TS). They may reflect the same problem that models fail silence when the context is too long.
5 comments

We appreciate the careful review and constructive feedback. Based on the points raised in the review, we conducted additional experiments to strengthen our evaluation. We address each point below. All additional results, evaluation settings, and discussions in this response will be added to the Appendix of the revised paper.


W1/W2: Timing and interaction coverage of the user simulator (1/2)

These points concern two aspects of our user simulator: the interaction timing, given the latency of its cascaded pipeline, and the interaction coverage, given that the simulated user does not backchannel or interrupt. We address each aspect below and then report an additional evaluation with a full-duplex user, which extends our evaluation to interaction with backchannels and interruptions.

Timing (W2). In short, the latency of the cascaded simulator does not affect the interaction timing that the model experiences. We define interaction timing in logical time based on processed audio, independent of wall-clock execution (Sec. 3.1 and Appendix A.1): the simulator's LLM and TTS calls complete before the next audio input is delivered to the model, so the model does not observe their computation time. The user-side response timing is instead fixed by the protocol, with the simulated user starting its next utterance after 800 ms of continuous model silence (Table 7). This setup has precedent in full-duplex evaluation, where a cascaded user simulator is likewise decoupled from wall-clock time (Ray et al., "τ-Voice: Benchmarking Full-Duplex Voice Agents on Real-World Domains," arXiv 2026).

Interaction coverage (W1). We agree that backchannels and interruptions are an important part of full-duplex interaction and that our main evaluation does not cover them. The calm turn-taking of our simulator, in which each speaker waits for the other to finish, is one natural conversational style that keeps the interaction condition consistent across models (Sec. 3.1), and the simulator's cascaded design makes our evaluation scale (90 closed-loop episodes per model and condition) affordable. This setting, however, does not comprehensively represent the more diverse situations in which behaviors specific to full-duplex interaction arise. We therefore conducted the following additional evaluation to examine whether our findings hold in such situations.

Evaluation with a full-duplex user. We use OpenAI's GPT-Live-1, a commercial full-duplex API operating in real time, as the user. Unlike our simulator, GPT-Live-1 determines its own response timing and initiates backchannels and interruptions. The setting is as follows.

  • Protocol: The models use the same checkpoints as in Table 1. The timeout, retry, and termination rules and the metrics follow Sec. 3.1. Because GPT-Live-1 also speaks while the model is talking, we additionally classify its utterances: those of up to 1 s are treated as backchannels, and longer ones that begin while the model is speaking are counted as interruption attempts. Unlike the main protocol, which advances in logical time, the interaction runs in real time.
  • User setting: GPT-Live-1 receives the same persona and topic instructions as our simulator, together with instructions to backchannel frequently and to interrupt mid-sentence. In practice, it produced an average of 3.9 backchannels and 0.9 interruption attempts per minute across all sessions.
  • Episodes: Given the cost and runtime of real-time interaction, we use one third of the main evaluation scale. This gives 30 episodes per model and condition, pairing the Balanced persona with all ten topics, three episodes each. Base sessions last 10 minutes and the others 20 minutes, as in Table 1.
  • Models: Raon-SpeechChat is excluded because it cannot sustain real-time inference in our computing environment.

W1/W2: Timing and interaction coverage of the user simulator (2/2)

(1) Degradation of the base models

We first compare the Base models under our simulator and GPT-Live-1. Each cell reports the 0–2 → 8–10 min intervals, and the values for our simulator are from Table 1. As in the paper, we apply the reporting thresholds of Appendix A.4 to avoid unstable estimates from too few observations; – marks intervals that do not meet them.

ModelUserRT (%) ↓TS (%) ↓FTO (s)CQ ↑WER (%) ↓Spk sim. ↑
PersonaPlexOur simulator0.0 → 17.60.0 → 18.9−0.27 → 0.404.78 → 1.295.4 → 44.30.733 → 0.337
GPT-Live-13.8 → 35.80.0 → 36.70.20 → 0.454.60 → 1.167.1 → 53.10.720 → 0.224
MiniCPM-o 4.5Our simulator9.4 → 66.76.7 → 66.70.40 → 1.904.81 → 4.192.6 → 13.60.775 → 0.749
GPT-Live-136.9 → 55.340.0 → 76.70.63 → –4.52 → 4.756.0 → 8.90.764 → 0.735

With GPT-Live-1, the degradation of both models largely follows the pattern observed with our simulator, while its severity and onset differ. PersonaPlex loses responsiveness, content quality, and speech quality more severely than with our simulator. MiniCPM-o 4.5 again degrades mainly in responsiveness while keeping its content quality, but the decline starts earlier: 36.9% of response opportunities already time out at 0–2 min, and 76.7% of sessions terminate by 10 min. The degradation found in our evaluation is therefore not confined to calm turn-taking; it also arises when the user backchannels and interrupts in real time.

(2) Mitigation of the degradation

We next evaluate Ours (TRC) in 20-minute sessions with GPT-Live-1. As an additional setting, we also evaluate Ours (TRC, 30-s teacher) on MiniCPM-o 4.5, a variant of TRC trained on the same trajectories, with the teacher's context window for computing the targets shortened from up to 150 s to 30 s. Each cell reports the 8–10 / 18–20 min intervals; – marks intervals beyond the 10-minute Base sessions or below the reporting thresholds.

ModelMethodRT (%) ↓TS (%) ↓FTO (s)CQ ↑WER (%) ↓Spk sim. ↑
PersonaPlexBase35.8 / –36.7 / –0.45 / –1.16 / –53.1 / –0.224 / –
Ours (TRC)12.6 / 8.73.3 / 10.00.23 / 0.233.91 / 3.2811.7 / 18.20.599 / 0.537
MiniCPM-o 4.5Base55.3 / –76.7 / –– / –4.75 / –8.9 / –0.735 / –
Ours (TRC)66.7 / –96.7 / 100.0– / –– / –– / –– / –
Ours (TRC, 30-s teacher)7.8 / 9.10.0 / 3.30.96 / 0.844.65 / 4.604.6 / 4.70.772 / 0.779

On PersonaPlex, Ours (TRC) mitigates the degradation through 20 minutes with GPT-Live-1 as well. It keeps content and speech quality well above the late-session Base level and terminates far fewer sessions: 10.0% by 20 min, against 36.7% of Base sessions by 10 min. Its response timeouts are more frequent than with our simulator, but they do not increase further beyond the training horizon.

On MiniCPM-o 4.5, in contrast, Ours (TRC) does not mitigate the degradation: 96.7% of its sessions terminate by 10 min, more than for the Base model. This reveals that, on this model, the configuration used in the paper does not generalize well to interaction with more active full-duplex behaviors, a limitation already partly visible with our simulator. Changing the configuration, however, substantially improves the mitigation. With Ours (TRC, 30-s teacher), response timeouts stay below 10%, only one session (3.3%) terminates through 20 minutes, and content and speech quality remain stable.

Overall, the degradation of the base models reappears when the user is a real-time full-duplex system that backchannels and interrupts, which suggests that our findings on degradation are not specific to the calm turn-taking or the logical-time protocol of our simulator. As for mitigation, these results suggest that TRC itself can generalize to such interaction, but that applying it with the same configuration across models may not be sufficient. They also point to both a need and room for adapting the configuration to each model, with attention to the trade-offs this could introduce.

W3/Q2: Conditional quality and session survival

We agree that the late-session CQ and WER of MiniCPM-o 4.5 with TRC describe the responses in the surviving sessions and do not by themselves indicate better overall reliability; on this model, TRC terminates more sessions than LoRA alone (76.7 vs. 64.4% at 18–20 min). The paper describes this result in the same way: Sec. 5.1 states that "MiniCPM-o continues to lose responsiveness gradually, yet the responses it does produce remain robust in content and speech quality," and limits TRC's advantage beyond the training horizon to "content and speech quality."

This is also why we report TS in addition to RT. RT measures how often the model fails to respond within the sessions that are still running, so, like the quality metrics, it is conditional on survival. TS instead counts the sessions terminated after three consecutive response timeouts among all 90 sessions, so it measures survival itself and indicates how many sessions each interval's quality metrics rest on.

An episode-level success measure, as suggested, would combine survival and quality into a single judgment per episode. Such a measure can be defined in many ways, but which aspects to include and where to set the pass criteria are largely subjective choices, unlike in task-oriented dialogue, where task completion provides a natural criterion. We therefore keep these aspects as separate measures: as in Full-Duplex-Bench (Lin et al., "Full-Duplex-Bench: A Benchmark to Evaluate Full-duplex Spoken Dialogue Models on Turn-taking Capabilities," ASRU 2025), which evaluates each turn-taking behavior with its own metric, we report relatively objective measures for each aspect.

Because this concern centers on MiniCPM-o 4.5, we examined whether the limitation of TRC on this model lies in its core idea or in its training configuration. To this end, we trained Ours (TRC, 30-s teacher) on the same trajectories, with the teacher's context window for computing the targets shortened from up to 150 s to 30 s. The table reports the 18–20 min interval; – marks a value below the reporting thresholds of Appendix A.4, and bold marks the better value.

MethodRT (%) ↓TS (%) ↓FTO (s)CQ ↑WER (%) ↓Spk sim. ↑
Ours (TRC)51.676.7–4.645.80.770
Ours (TRC, 30-s teacher)22.40.01.534.685.60.779

With the shorter teacher window, no session terminates (TS 0.0 vs. 76.7%), responsiveness also improves, and content and speech quality stay at the same level, so the quality no longer rests on a subset of surviving sessions. This variant also keeps MiniCPM-o 4.5 stable with GPT-Live-1 as the user (W1/W2), where only one of 30 sessions terminates through 20 minutes. This gain, however, comes at some cost in the memory recall of Sec. 5.3 (57 → 35% at 5 min).

As for why this configuration change makes such a large difference, the likely reason lies in what the gists are trained to reproduce. With a context window of up to 150 s, the teacher's targets also reflect what it reads from raw tokens beyond the student's window, so the gists learn to carry over the long-context behavior that degrades the base models; the shorter window largely removes this readout from the targets.

In summary, the limited gain of TRC on MiniCPM-o 4.5 in the paper does not indicate a fundamental limitation of TRC itself. The paper applies a single training configuration uniformly across models, and adapting this configuration to MiniCPM-o 4.5 allows TRC to mitigate its degradation as well, albeit with some trade-off in memory recall. We will clarify in Sec. 5.1 that the late-session quality of MiniCPM-o 4.5 with TRC should be read together with session survival.

Q1: Figure fonts and the layout of Figure 3

We appreciate this feedback. We will unify the fonts across all figures and rearrange the layout of Figure 3 to follow the reading order. Beyond these changes, we will also review the paper for further improvements in readability.


We hope that these additional results and clarifications address the concerns raised in the review. We would be happy to discuss any remaining questions.

Comments on the paper

1 comment

We thank all reviewers for their careful reviews and constructive feedback. We have uploaded a revised manuscript that incorporates the additional experiments, analyses, and clarifications presented in our responses. The revision includes:

  • The trade-off between long-session stability and memory recall in Section 5.3, with the further analyses summarized in Section 5.4.
  • The additional results in the appendix, including human validation of the CQ judge, the quality of the teacher trajectories, supervision and local-context ablations, the teacher window ablation, the evaluation with GPT-Live-1 as a user that backchannels and interrupts, and the evaluation on Full-Duplex-Bench (Appendices A.6, B.4, and C).
  • Revised statements on generalization beyond the training horizon, now stated for 20-minute sessions, and an updated discussion of limitations.

We also made minor revisions to improve readability, such as unifying the fonts across figures and rearranging the layout of Figure 3 to follow the reading order. Section and appendix numbers in our individual responses refer to the original submission.

We hope that the revised manuscript addresses the reviewers' concerns, and we would be happy to discuss any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.