acceptodds
Under review as a conference paper at ICLR 2027

DeIRD: A Decoupled Continuous Implicit Reasoning Framework for Dialogue Models

Abstract

By leveraging the ”think before speaking” response paradigm, large models can generate a Chain-of-Thought (CoT) prior to answering to enhance their deep reasoning capabilities. However, directly applying this paradigm to spoken dialogues poses three severe challenges: huge training costs caused by the modality gap, the unacceptable latency introduced by the ”think before speaking” mechanism, and limited reasoning exploration depth. To address these issues, we propose DeIRD, a latency-controllable and low-training-cost implicit reasoning framework for spoken dialogue models that simultaneously achieves ”think and speak decoupled” and ”think before speaking”. This framework is primarily realized through three core components: 1) We train a text VAE to compress variable-length textual CoT sequences into fixed-length continuous latents, serving as the carrier for implicit reasoning; 2) We introduce a continuous masked prediction model DMIR that takes user speech as input and outputs a fixed-length sequence of these latent targets via iterative confidence-guided ”mask-predict” decoding, to achieve ”think and speak decoupled” and ”think before speaking”; 3) We construct an implicit reasoning training dataset, LatentReason-40k, and find that the downstream backbones just require one epoch of end-to-end fine-tuning to acquire superior reasoning capabilities compared to explicit reasoning models trained on the same volume of data. Compared with various explicit-CoT baselines, DeIRD notably reduces the Time to First Audio (TTFA) while improving general dialogue quality, style control, deep reasoning, and reasoning diversity.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
dialogue models, implicit reasoning, decoupled

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jNDtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DeIRD, a decoupled implicit reasoning framework for spoken dialogue models. The goal is to reduce the latency caused by explicit Chain-of-Thought (CoT) generation while improving reasoning and dialogue performance.

The framework consists of three main components. First, a text VAE compresses variable-length CoT sequences into fixed-length continuous latent representations. Second, DMIR, built on DIFFA-2, predicts these latent representations directly from speech using an iterative confidence-guided mask-predict strategy. Finally, a Latent Flow Adapter (LFA) maps the predicted latents into the hidden space of the downstream dialogue model, where they serve as soft prompts to guide response generation.

The authors also construct LatentReason-40k, a dataset containing 46,809 samples, and evaluate their approach on three spoken dialogue models with different architectures: Step-Audio-2-mini, VITA-Audio, and Kimi-Audio.

The results show that DeIRD generally outperforms the evaluated explicit CoT baselines on dialogue and reasoning benchmarks while substantially reducing time to first audio. The paper also includes ablation studies on the latent representations, decoding strategies, and the LFA module to examine their contributions.

Critical strengths

A useful performance–latency trade-off across speech model architectures. DeIRD uses an audio-conditioned Mask-Predict module to generate fixed-length continuous reasoning representations, reducing the delay of explicit CoT generation. On Step-Audio-2-mini, it improves the OpenAudioBench average from 55.7 to 62.6 while reducing time to first audio from 18.6 to 1.50 seconds relative to explicit CoT fine-tuning, with matched downstream training data and epochs. Its evaluation on three backbones covering both interleaved and parallel generation also supports its applicability beyond a single model architecture. Together, these results demonstrate the practical value of the proposed framework.

Critical weaknesses
  1. The contribution of CoT supervision is not adequately isolated. Appendix D provides evidence that instance-specific latents are useful, but does not establish that intermediate reasoning information is what makes them useful. The paper lacks a standard SFT baseline trained on the same question–answer pairs without CoT or latents, and a matched comparison against answer-only latent targets. These omissions leave the incremental benefit of CoT supervision unresolved: the gains may partly reflect task adaptation and answer-related knowledge distillation rather than the proposed reasoning mechanism. This attribution gap directly weakens the paper’s central claim.

  2. The defining iterative mechanism lacks a convincing performance–cost justification. DMIR requires 32 sequential prediction rounds, accounting for approximately 1.39 seconds of the reported 1.50-second TTFA. Yet the paper does not compare it against a matched single-pass predictor of the same latent targets. The Top-1/2/4 ablations demonstrate trade-offs among iterative configurations, but do not establish whether iteration is preferable to direct latent prediction. Consequently, outperforming lengthy explicit CoT generation does not adequately justify the computational expense of the paper’s central design. Moreover, improved GSM8K Pass@k alone does not substantiate the stronger claim of greater reasoning-path diversity.

Questions for authors
  1. Gain beyond standard SFT — requested comparison. Could the authors report a standard SFT baseline using the same question–answer pairs, backbone, and downstream training configuration, but without CoT or latent inputs? The controls in Appendix D are informative, but do not directly establish the incremental benefit of DeIRD over ordinary supervised adaptation.

  2. Value of intermediate reasoning supervision — requested comparison.How does full-CoT supervision compare with answer-only latent targets under matched latent shape, model capacity, and training conditions? This comparison would help determine whether intermediate reasoning information provides a benefit beyond answer-related knowledge distillation, which is central to the paper’s interpretation of its results.

  3. Benefit of iterative prediction — requested comparison.Could the authors compare the default 32-round DMIR with a single-pass latent predictor, using the same backbone and latent targets and appropriately training the downstream model for each setting? Please report both downstream accuracy and TTFA. Since DMIR contributes approximately 1.39 seconds of the reported 1.50-second latency, this comparison is important for assessing whether iterative prediction justifies its additional cost.

  4. Evidence for reasoning diversity — clarification.Where does stochasticity enter the GSM8K evaluation: latent generation, downstream answer decoding, or both? Given Algorithm 1’s deterministic slot selection and mean commitment, what evidence links the Pass@k improvements specifically to more diverse latent reasoning paths, rather than improved answer accuracy or downstream sampling? Please clarify the scope of the diversity claim supported by the current results.

4 comments
AuthorsOpenReview ↗

Response to Reviewer jNDt(1/2)

Thank you for the constructive feedback.

1. Additional evidence for the effectiveness of latent targets (W1 & Q1 & Q2)

Thank you for acknowledging the instance-specific latent controls in Appendix D. We supplement these experiments with two baselines:

  • Standard QA SFT: Training on the same question–answer pairs with the same one-epoch downstream training configuration, but without CoT or latent inputs.
  • Answer-only latent: Encoding only the final answer with the VAE and using the resulting representation as the latent target.

The additional comparisons on reasoning-specific metrics are reported below.

Table R1. Results are reported as S / V / K: Step-Audio-2-mini, VITA-Audio-Plus-Vanilla, and Kimi-Audio-7B-Instruct.

MethodGSM8K P@1 (%) ↑VoiceBench BBH ↑OpenAudioBench Reas. ↑VStyle Avg. ↑
Base Model26.5 / 64.1 / 80.054.9 / 52.5 / 68.43.03 / 2.94 / 3.363.08 / 2.52 / 3.06
Standard QA SFT38.8 / 58.2 / 72.356.2 / 53.3 / 67.73.20 / 2.98 / 3.203.00 / 2.55 / 3.00
Answer-only latent40.3 / 66.2 / 78.255.8 / 50.0 / 66.23.27 / 3.06 / 3.202.46 / 2.30 / 2.68
Full-CoT latent (DeIRD)60.4 / 65.6 / 80.159.2 / 53.7 / 68.43.34 / 3.23 / 3.423.15 / 2.91 / 3.18

DeIRD outperforms Standard QA SFT on every reported metric across all three backbones. Compared with Answer-only latent, it improves VoiceBench BBH by 3.4 / 3.7 / 2.2 points and OpenAudioBench Reas. by 0.07 / 0.17 / 0.22 points.

These results indicate that the gains cannot be explained solely by ordinary task adaptation or answer distillation. They support intermediate reasoning supervision as an important source of improvement, providing additional benefits beyond answer-related knowledge distillation.

2. Additional evidence for the value of iterative prediction (W2 & Q3)

Thank you for your assessment of the Top-2 and Top-4 ablations, reported in Table 1, Ablation 4, and discussed in Section 4.4, Finding 4. We additionally trained a single-pass Top-32 predictor that generates and commits all 32 latent slots in one forward pass. We then adapted each of the three downstream backbones using its predicted latents.

Table R2. Results are reported as S / V / K: Step-Audio-2-mini, VITA-Audio-Plus-Vanilla, and Kimi-Audio-7B-Instruct.

ConfigurationGSM8K P@1 (%) ↑OpenAudioBench Avg. ↑VStyle Avg. ↑TTFA (s) ↓
Base26.5 / 64.1 / 80.055.1 / 52.4 / 70.23.08 / 2.52 / 3.060.37 / 0.11 / 0.22
Random Noise Latent55.3 / 59.8 / 79.857.7 / 56.3 / 68.02.84 / 2.78 / 1.630.37 / 0.12 / 0.22
Top-1 (DeIRD)60.4 / 65.6 / 80.162.6 / 57.6 / 70.83.15 / 2.91 / 3.181.50 / 1.57 / 1.67
Top-256.6 / 59.9 / 80.562.4 / 56.6 / 68.02.84 / 2.78 / 1.610.81 / 0.87 / 0.97
Top-459.5 / 59.3 / 79.962.3 / 56.1 / 63.72.88 / 2.83 / 1.610.47 / 0.52 / 0.62
Top-32, single-pass50.0 / 58.3 / 72.450.2 / 47.9 / 60.52.71 / 2.51 / 2.870.40 / 0.20 / 0.28

Although single-pass prediction reduces TTFA, it underperforms even Random Noise Latent on GSM8K and OpenAudioBench across all three backbones, supporting the substantive value of iterative prediction.

As specified in Section 4.2, “Ablation Studies,” the noise control uses random representations sampled near the expected vocabulary embedding space. We hypothesize that accepting all slots simultaneously, without confidence-guided sequential commitment and conditioning updates, can produce latents whose embedding distribution deviates from this space and introduces misleading task information.

AuthorsOpenReview ↗

Response to Reviewer jNDt(2/2)

3. Clarifying and directly evaluating reasoning diversity (W2 & Q4)

We clarify our hypothesis: continuous reasoning states can retain information about multiple candidate continuations. Greedy textual decoding feeds back only one selected token at each step, whereas continuous feedback allows information about alternative paths to be retained before committing to discrete text.

This hypothesis is supported by prior work. Coconut [1, Section 4.3] analyzes intermediate continuous states and finds behavior consistent with retaining alternative branches. Soft Thinking [2, Section 3.2] explicitly feeds back probability-weighted token embeddings. Reasoning by Superposition [3] provides a theoretical construction in which continuous states simultaneously encode multiple search frontiers for graph reachability. Together, these studies support the representational basis of our hypothesis.

To examine this hypothesis in DeIRD, we conduct a paired diagnostic on 100 randomly selected GSM8K questions, using the same trained Step-Audio-2-mini checkpoint throughout. We directly supply three controlled explicit reasoning trajectories or DMIR-generated continuous latents before answer generation. Each explicit trajectory contains a designated branch point: it follows the correct branch, follows an incorrect branch, or initially follows an incorrect branch before backtracking once and switching to the correct branch. The supplied textual trajectories exclude the final answer.

This experiment is included in Section 4.2 and Table 3, with its analysis in Section 4.4, Finding 5, of the revised manuscript.

Table R3.

ConditionIntermediate reasoning supplied to the backbonePass@1 accuracy (%) ↑
Correct single-path CoTCorrect branch → correct reasoning continuation76.0
Incorrect single-path CoTIncorrect branch → incorrect reasoning continuation24.0
CoT with backtrackingIncorrect branch → backtrack → correct continuation87.0
DeIRDDMIR-generated continuous latents89.0

DeIRD achieves accuracy comparable to explicit CoT with backtracking, without requiring textual correction trajectories. These results support the hypothesis that continuous reasoning can retain useful alternatives and avoid premature commitment to discrete reasoning choices.

[1] Hao et al., Training Large Language Models to Reason in a Continuous Latent Space, COLM 2025.

[2] Zhang et al., Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space, arXiv 2025.

[3] Zhu et al., Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought, NeurIPS 2025.

AuthorsOpenReview ↗

Kind Follow-up on Our Response

Dear Reviewer jNDt,

Thank you for your thoughtful review. We have posted a response to your comments, together with the new experimental results and references to the corresponding changes in the manuscript.

We appreciate the demands on your time during the discussion period. When you have a chance to read our response, we would welcome your feedback on whether it resolves the concerns you raised.

If any point needs further clarification, we would be happy to address it before the discussion closes.

Best regards,
The Authors

AuthorsOpenReview ↗

Kind Follow-up on Our Response

Dear Reviewer jNDt,

Thank you for your thoughtful review. We have posted a response to your comments, together with the new experimental results and references to the corresponding changes in the manuscript.

We appreciate the demands on your time during the discussion period. As the discussion period is drawing to a close, we would be grateful if you could review our response when you have a chance and let us know whether it addresses the concerns you raised.

If any point needs further clarification, we would be happy to address it before the discussion closes.

Best regards,

The Authors

Reviewer_8jeZOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DeIRD, which aims to enhance the reasoning capabilities of spoken dialogue models while reducing the response latency typically associated with explicit CoT prompting. The core approach involves using a Text VAE to compress an explicit CoT into a fixed-length continuous latent sequence. A masked implicit reasoner (DMIR), conditioned on speech input, is then trained to predict these latent sequences. These predicted latents are subsequently injected into a spoken dialogue backbone as "soft prompts" to guide the generation of responses. Additionally, the authors employ a Latent Flow Adapter (LFA) to align the latent spaces across different modules.

The primary contributions of this work include: proposing a comprehensive framework for continuous latent reasoning; designing the DMIR for iterative latent slot prediction; adapting this latent reasoning mechanism to various spoken dialogue generation architectures; and constructing the "LatentReason-40k" dataset for training the entire latent reasoning pipeline.

In their experiments, the authors evaluate the framework across three spoken dialogue backbones, covering both interleaved and parallel generation architectures, using benchmarks such as VoiceBench, VStyle, OpenAudioBench, and GSM8K, while also reporting TTFA. Furthermore, the study provides analyses on factors including latent dimension, latent sequence length, the role of the LFA, Top-k slot commitment, random latent control, VAE reconstruction quality, and query relevance. Overall, the experimental results robustly support the effectiveness of DeIRD in terms of downstream performance and latency control; however, the evidence for the stronger mechanistic claim, that continuous latents themselves facilitate better exploration of reasoning paths, remains somewhat indirect.

Critical strengths
  1. The paper addresses a clear and practically important problem in spoken dialogue reasoning. Explicit CoT can improve reasoning but increases time-to-first-audio, while streaming or chunked reasoning may reduce latency at the cost of insufficient early-stage reasoning. DeIRD offers a reasonably novel way to handle this trade-off by replacing explicit textual CoT with a fixed-length continuous latent reasoning representation generated before the response. This is the main reason I view the work as technically interesting rather than just an incremental system modification.
  2. The experimental evaluation is broad enough to support the practical usefulness of the framework. DeIRD is tested on three spoken dialogue backbones spanning interleaved and parallel generation settings, with evaluations on multiple benchmarks and latency metrics, together with ablations on latent size, latent number, LFA, decoding strategy, random-latent control, and VAE reconstruction. This breadth makes the empirical case reasonably convincing that the framework is effective across more than a single model or task.
Critical weaknesses
  1. The central “implicit reasoning” claim is not fully established by the current evidence. The experiments show that the learned latent representations are useful, but they do not clearly demonstrate that the gains come from reasoning-specific information rather than from general semantic, acoustic, or response-planning features. Since the main novelty of DeIRD is built around latent reasoning, this gap weakens the paper’s core claim.
  2. Several stronger mechanism and efficiency claims are not sufficiently supported. In particular, the GSM8K pass@k results do not directly show that the continuous latent space explores multiple reasoning paths, because the diversity may mainly come from stochastic downstream decoding. Likewise, the “low training cost” claim focuses on one-epoch backbone adaptation while excluding the additional cost of training the Text VAE, DMIR, and LFA modules and constructing LatentReason-40k. These issues make the current conclusions stronger than what the experiments directly justify.
Questions for authors
  1. Could the authors report the overall training/computational cost of the complete DeIRD pipeline and compare it with the explicit-CoT baselines, rather than reporting only the backbone fine-tuning cost?
  2. Could the authors provide evidence or additional analysis showing that DeIRD latents contain more reasoning-specific information than standard semantic/audio representations?
  3. During training, masked slots are predicted conditioned on unmasked ground-truth latent slots, whereas during inference, previously committed slots are the model’s own predictions. Does this train–inference mismatch lead to error accumulation across the iterative prediction steps? Could the authors provide an analysis of prediction error as the number of committed slots increases, or otherwise discuss how this issue is mitigated?
  4. Are the reported 99.9% reconstruction accuracy and 99.5% query relevance evaluated on an independent held-out set? If not, their evidential value would be limited. Moreover, query relevance only shows that the decoded text remains on-topic, not that the reasoning is correct. Given that the reasoning traces and relevance judgments come from the same model ecosystem, could the authors provide human evaluation or an independent judge model to validate this interpretability result?
  5. The downstream performance indicates that the continuous latent representations are effective, but it is still unclear whether the gains specifically come from improved reasoning or reasoning-path exploration. Could the authors provide stronger evidence that the latent space captures and explores distinct reasoning trajectories, rather than mainly providing useful semantic or response-planning information? In particular, the current pass@k results may reflect stochasticity in downstream decoding rather than diversity in the latent reasoning process itself.

If these concerns are well addressed, I will consider increase my rating.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 8jeZ(1/3)

Thank you for the constructive feedback.

1. What information do the latent representations contain? (W1 & Q2)

Thank you for acknowledging the effectiveness of the latent representations. We address your concern by clarifying how the latent targets are constructed and presenting additional Hint-only and Answer-only controls.

First, the latent targets are produced by a text VAE that compresses teacher-generated CoT, rather than merely encoding input semantics, acoustics, or response-planning features. The CoT contains paralinguistic understanding, step-by-step content reasoning, and speech delivery planning. As described in Appendix B, we use Gemini 3.1 Pro with the prompts in Figure 5 to generate intermediate reasoning for each question–answer pair. The teacher is explicitly instructed to reason through the question step by step, providing concrete derivations and verification for mathematics, coding, and logic problems.

The VAE achieves nearly lossless reconstruction on an independent held-out test set: 99.9% token accuracy and 0.003 WER (Table 2). This supports the preservation of detailed reasoning content in the latent targets. The held-out evaluation protocol is now specified in Section 4.2.

Second, to test whether answer-related information or acoustic delivery guidance alone can provide the same reasoning benefits, we introduce Hint-only and Answer-only controls across all three backbones. We regenerate target texts containing only style guidance or the final answer, respectively, encode them into latent targets, and train under the same downstream configuration. These experiments and their analysis have been added to Appendix D, Table 8 of the revised manuscript.

Table R1. Results are reported as S / V / K: Step-Audio-2-mini, VITA-Audio-Plus-Vanilla, and Kimi-Audio-7B-Instruct.

MethodOpenAudioBench Avg. ↑VoiceBench Avg. ↑VStyle Avg. ↑
Baseline55.1 / 52.4 / 70.258.7 / 50.8 / 72.83.08 / 2.52 / 3.06
Hint-only55.9 / 55.0 / 70.058.3 / 52.0 / 71.83.09 / 2.50 / 3.08
Answer-only57.2 / 52.6 / 68.459.0 / 55.2 / 70.32.46 / 2.30 / 2.68
DeIRD62.6 / 57.6 / 70.864.9 / 60.7 / 74.43.15 / 2.91 / 3.18

DeIRD outperforms both controls on all three metrics across all three backbones. Answer-only also substantially underperforms on VStyle, indicating that answer content alone cannot preserve speech delivery planning. These results support the important role of the additional reasoning information encoded in the full latent targets, which cannot be replaced by style hints or final answers alone.

AuthorsOpenReview ↗

Response to Reviewer 8jeZ(2/3)

2. Clarifying and directly evaluating reasoning diversity (W2 & Q5)

We clarify our hypothesis: continuous reasoning states can retain information about multiple candidate continuations. Greedy textual decoding feeds back only one selected token at each step, whereas continuous feedback allows information about alternative paths to be retained before committing to discrete text.

This hypothesis is supported by prior work. Coconut [1, Section 4.3] analyzes intermediate continuous states and finds behavior consistent with retaining alternative branches. Soft Thinking [2, Section 3.2] explicitly feeds back probability-weighted token embeddings. Reasoning by Superposition [3] provides a theoretical construction in which continuous states simultaneously encode multiple search frontiers for graph reachability. Together, these studies support the representational basis of our hypothesis.

To examine this hypothesis in DeIRD, we conduct a paired diagnostic on 100 GSM8K questions, using the same trained Step-Audio-2-mini checkpoint throughout. We directly supply three controlled explicit reasoning trajectories or DMIR-generated continuous latents before answer generation. Each explicit trajectory contains a designated branch point: it follows the correct branch, follows an incorrect branch, or initially follows an incorrect branch before backtracking once and switching to the correct branch. The supplied textual trajectories exclude the final answer.

This experiment is included in Section 4.2 and Table 3, with its analysis in Section 4.4, Finding 5, of the revised manuscript.

Table R2.

ConditionIntermediate reasoning supplied to the backbonePass@1 accuracy (%) ↑
Correct single-path CoTCorrect branch → correct reasoning continuation76.0
Incorrect single-path CoTIncorrect branch → incorrect reasoning continuation24.0
CoT with backtrackingIncorrect branch → backtrack → correct continuation87.0
DeIRDDMIR-generated continuous latents89.0

DeIRD achieves accuracy comparable to explicit CoT with backtracking, without requiring textual correction trajectories. Because all final answers are decoded greedily, downstream stochastic sampling cannot explain the observed pattern in this diagnostic. Together with prior evidence, these results support the hypothesis that continuous reasoning can retain useful alternatives and avoid premature commitment to discrete reasoning choices.

[1] Hao et al., Training Large Language Models to Reason in a Continuous Latent Space, COLM 2025.

[2] Zhang et al., Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space, arXiv 2025.

[3] Zhu et al., Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought, NeurIPS 2025.

3. Scope and accounting of training costs (W2 & Q1)

By “low training cost,” we refer specifically to adapting pretrained spoken-dialogue backbones using reusable DeIRD components, rather than constructing the entire framework from scratch. Once the VAE, DMIR, and dataset have been developed, these upstream components can be reused across backbones without repeating their training or construction. During adaptation, the backbone and its downstream LFA are jointly updated using the native response loss, requiring only one epoch of training (Section 3.4, Appendix G, and Table 13 of the revised manuscript).

The upstream training costs are already reported in Appendix G, “Clarification on Training Cost”: 294 GPU-hours for the VAE and 9.3 GPU-hours for DMIR. Downstream adaptation requires 15.6, 14.6, and 11.7 GPU-hours for Step-Audio-2-mini, VITA-Audio, and Kimi-Audio, respectively.

AuthorsOpenReview ↗

Response to Reviewer 8jeZ(3/3)

4. Training–inference mismatch from teacher forcing (Q3)

Thank you for raising the training–inference mismatch in diffusion language models. Prior work suggests that its impact can be limited: He et al. [4] report limited distortion without apparent accumulation during open-ended autoregressive generation. Kim et al. [5] show that masked diffusion models trained with random masking can achieve strong reasoning performance through adaptive decoding. More directly, Frković et al. [6, Section 3.1] find that revisiting committed predictions provides small and inconsistent gains under short-block greedy decoding.

To examine whether the limited impact observed in prior work extends to DeIRD, we conduct a controlled comparison on held-out test data using Step-Audio-2-mini. We generate CoT in the same format as the training targets, encode it with the VAE, and apply the invertible flow in the DMIR-side LFA to obtain reference latent targets, following the target construction in Section 3.3.2. We compare:

  • Teacher-context: When predicting the remaining slots, replace previously committed slots with their corresponding reference latent values.
  • Self-context: Retain the model’s own predictions in previously committed slots, as in standard DeIRD inference.

Table R3.

MethodGSM8K Pass@1 (%) ↑GSM8K Pass@8 (%) ↑OpenAudioBench Avg. ↑
Teacher-context55.374.860.3
Self-context (DeIRD)60.488.162.6

Self-context outperforms Teacher-context on all three metrics. Replacing committed predictions with reference latents does not improve downstream performance, indicating that DeIRD can effectively use its own predicted context.

[4] He et al., Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation?, EMNLP 2021.

[5] Kim et al., Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions, ICML 2025.

[6] Frković et al., Re-evaluating Confidence Remasking in Masked Diffusion Language Models, arXiv 2026.

5. Held-out evaluation and human validation (Q4)

Both the VAE reconstruction evaluation and the DMIR relevance evaluation use 100 held-out samples that were excluded from training. The reported 99.9% token accuracy and 0.003 WER measure reconstruction fidelity on this evaluation set. We have clarified this protocol in Section 4.2 of the revised manuscript.

To strengthen the credibility of the interpretability evaluation, we replace the original judge with GPT-6 Astra for relevance assessment. We also recruit five academic researchers to independently evaluate the decoded reasoning trajectories. Every trajectory is assessed by all five researchers using the same relevance criteria as the model’s evaluation prompt.

Table R4.

EvaluatorRrelR_{\mathrm{rel}} (%) ↑
GPT-6 Astra96.0
Human evaluation93.0

The updated GPT-6 Astra result is reported in Table 2, and the mean human evaluation result is included in Section 4.3, “Reconstruction and Interpretability.”

Reviewer_pGC3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper tries to make spoken dialogue models "think before speaking" without generating a long chain of thought. The idea:

• First, train a text VAE that compresses Gemini-written CoTs into 32 continuous vectors. • Then train a separate model, DMIR, to predict those vectors directly from the user's speech. • At inference, DMIR produces the vectors, and they are fed into the dialogue model (Step-Audio-2-mini, VITA-Audio, or Kimi-Audio) as a soft prompt before it answers.

The authors also build a dataset containing 47k samples. They report gains on VoiceBench, OpenAudioBench, VStyle, and GSM8K, with about 1.5 s TTFA compared with 9–18 s for explicit CoT.

Critical strengths
  • The problem is real. Long CoT is too slow for spoken dialogue and replacing it with a fixed number of latents makes latency predictable.
  • The paper is with solid engineering, in which the method works on three quite different backbones, and the data pipeline, training costs, and appendices are well documented.
Critical weaknesses
  • I'm not convinced the model is actually "reasoning" in the latents. To me, the latents look more like hints (topic, answer direction, format) than a reasoning chain. I would not expect the the reasoning capability can be learnt with such a small dataset. That's probably why using noise latent can also somehow work.
  • DeIRD needs an extra LLaDA-based model. That model has to wait until the user finishes speaking and then run 32 sequential passes, so every turn gets an extra 1.1–1.5 s, even simple ones. What if the CoT is short? This latency is addtional.
Questions for authors
    • Why latents and not text? All three backbones accept text directly, so a short text CoT could also be read in parallel, with similar latency and better interpretability. The paper never compares against this, so the motivation for continuous latents is unclear to me.
  • Can you run the Table 7 controls (noise, shuffled latent, learnable prefix, eval without latent) on VITA and Kimi, and also on GSM8K? On which tasks do the latents really matter?
  • What are the total parameters and FLOPs at inference compared with the baselines?
5 comments
AuthorsOpenReview ↗

Response to Reviewer pGC3(1/3)

Thank you for the constructive feedback.

1. Do the latents contain reasoning beyond simple hints? (W1)

We address this concern through three aspects: latent-target construction, the multi-stage training procedure, and additional Hint-only and Answer-only controls.

First, the latent targets encode teacher-generated CoT containing paralinguistic understanding, step-by-step content reasoning, and speech delivery planning. As described in Appendix B, we use Gemini 3.1 Pro with the prompts in Figure 5 to generate intermediate reasoning for each question–answer pair. The teacher is explicitly instructed to reason through the question step by step and provide concrete derivations and verification for mathematics, coding, and logic problems. The text VAE achieves nearly lossless reconstruction on an independent held-out test set: 99.9% token accuracy and 0.003 WER (Table 2). This supports the preservation of detailed reasoning content in the latent targets. We have clarified the held-out evaluation protocol in Section 4.2.

Second, regarding whether reasoning can be learned from a small dataset, “one epoch” refers to adapting pretrained dialogue backbones under latent guidance, rather than learning reasoning from scratch. The preceding VAE and DMIR training stages are described in Section 3 and Appendix G.

Finally, to test whether simple hints or answer content can provide the same benefits, we add Hint-only and Answer-only controls across all three backbones. We regenerate target texts containing only style guidance or the final answer, respectively, encode them into latent targets, and use the same downstream training configuration. These experiments and their analysis have been added to Appendix D, Table 8.

Table R1. Results are reported as S / V / K: Step-Audio-2-mini, VITA-Audio-Plus-Vanilla, and Kimi-Audio-7B-Instruct.

MethodOpenAudioBench Avg. ↑VoiceBench Avg. ↑VStyle Avg. ↑
Baseline55.1 / 52.4 / 70.258.7 / 50.8 / 72.83.08 / 2.52 / 3.06
Hint-only55.9 / 55.0 / 70.058.3 / 52.0 / 71.83.09 / 2.50 / 3.08
Answer-only57.2 / 52.6 / 68.459.0 / 55.2 / 70.32.46 / 2.30 / 2.68
DeIRD62.6 / 57.6 / 70.864.9 / 60.7 / 74.43.15 / 2.91 / 3.18

DeIRD outperforms both controls on all three metrics across all three backbones. Answer-only also substantially underperforms on VStyle, indicating that answer content alone cannot preserve speech delivery planning. Together, these results support the important role of the additional reasoning information encoded in the full latent targets, which cannot be replaced by style hints or final answers alone.

2. Why can noise latents also help? (W1)

Regarding your question about why noise latents can sometimes help, Pfau et al. [1] show that semantically meaningless intermediate tokens can support additional computation and improve performance under appropriate training conditions. Motivated by this finding, we included Noise Latent and Learnable Prefix controls to test whether additional sequence positions alone can reproduce the benefits of meaningful latent targets. These controls are reported in Appendix D, Table 8 of the revised manuscript. The results show that additional sequence positions can provide modest improvements through increased capacity or computation, but their performance consistently falls short of DeIRD.

[1] Pfau et al., Let's Think Dot by Dot: Hidden Computation in Transformer Language Models, arXiv 2024.

3. Latency on simple tasks (W2)

We acknowledge your concern that DeIRD introduces additional latency even on simple tasks. We address this through its intended task scope and the faster Top-2 and Top-4 decoding configurations.

First, existing spoken-dialogue reasoning approaches, such as Step-Audio-2-mini-Think and Audio-CoT, do not dynamically switch between fast and slow reasoning according to question difficulty. DeIRD primarily targets tasks requiring substantial reasoning, including mathematics and logical reasoning, as evaluated on VoiceBench BBH, OpenAudioBench, and GSM8K. Its training-data composition is described in Section 3.5 and Appendix B. Compared with the base models, default DeIRD introduces approximately 1.1–1.5 seconds of additional average latency.

Second, Table 1, Ablation 4, reports faster Top-2 and Top-4 decoding configurations. These commit multiple latent slots per DMIR iteration, reducing the number of sequential prediction rounds. In particular, Top-4 reduces the additional latency to 0.10–0.41 seconds relative to the base models, while retaining reasoning benefits on supported tasks. The performance–latency trade-off is discussed in Section 4.4, Finding 4. Users can select the appropriate configuration according to the difficulty of their application domain, thereby mitigating the latency overhead for simpler tasks.

AuthorsOpenReview ↗

Response to Reviewer pGC3(2/3)

4. Why continuous latents rather than short textual CoT? (Q1)

Thank you for suggesting a short textual CoT baseline. DeIRD uses fixed-length implicit reasoning to control reasoning length and avoid premature reasoning-path collapse caused by discrete token sampling. To examine whether short textual CoT can provide reasoning benefits with reduced latency, we conducted two additional baseline experiments.

Based on a measured reasoning budget of approximately 1.2 seconds, we set the short-CoT target length to 128 tokens. We constructed supervision using either the same teacher, Gemini 3.1 Pro, or DIFFA-2, and evaluated both baselines under the same one-epoch training configuration as DeIRD.

Table R2. Results are reported as S / V / K: Step-Audio-2-mini, VITA-Audio-Plus-Vanilla, and Kimi-Audio-7B-Instruct.

MethodGSM8K P@1 (%) ↑OpenAudioBench Avg. ↑CoT ≤128 tokens (%) ↑TTFA (s) ↓
Base26.5 / 64.1 / 80.055.1 / 52.4 / 70.2N/A0.37 / 0.11 / 0.22
Short-CoT SFT: teacher targets30.6 / 59.3 / 72.357.2 / 51.0 / 68.262 / 49 / 501.39 / 1.18 / 1.26
Short-CoT SFT: DIFFA-2 targets29.7 / 53.4 / 70.456.3 / 53.8 / 65.969 / 43 / 421.39 / 1.19 / 1.26
DeIRD60.4 / 65.6 / 80.162.6 / 57.6 / 70.8N/A1.50 / 1.57 / 1.67

Under the same one-epoch training configuration, only 42%–69% of the short-CoT outputs satisfy the 128-token constraint, and both baselines underperform DeIRD across all three backbones. Prior work shows that complex reasoning benefits from intermediate reasoning steps [2] and, when necessary, evaluation, revision, and backtracking [3]. A plausible explanation is that, under the limited training budget, the models have not yet learned to jointly produce effective reasoning and satisfy the short-text constraint. DeIRD instead compresses fuller teacher CoT into 32 fixed latent slots, combining controllable representation length with stronger downstream performance.

[2] Wei et al., Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, NeurIPS 2022.

[3] Yao et al., Tree of Thoughts: Deliberate Problem Solving with Large Language Models, NeurIPS 2023.

5. Extending the four control experiments to additional backbones (Q2)

Thank you for suggesting these additional controls. We have completed the Noise Latent, Shuffled Latent, Learnable Prefix, and Eval w/o Latent evaluations on VITA and Kimi, including evaluation on GSM8K. The Noise Latent results across all three backbones were already reported in Table 1, Ablation 1. The expanded results, experimental settings, and analysis are now included in Appendix D, Table 8 of the revised manuscript.

Table R3. Results are reported as S / V / K: Step-Audio-2-mini, VITA-Audio-Plus-Vanilla, and Kimi-Audio-7B-Instruct.

MethodVoiceBench Avg. ↑VStyle Avg. ↑GSM8K P@8 (%) ↑OpenAudioBench Avg. ↑
Base Model58.7 / 50.8 / 72.83.08 / 2.52 / 3.0671.3 / 88.9 / 96.155.1 / 52.4 / 70.2
DeIRD64.9 / 60.7 / 74.43.15 / 2.91 / 3.1888.1 / 91.0 / 97.262.6 / 57.6 / 70.8
Noise Latent60.7 / 59.6 / 71.72.84 / 2.78 / 1.6376.6 / 88.1 / 94.857.7 / 56.3 / 68.0
Shuffled Latent60.4 / 50.3 / 73.23.11 / 2.59 / 3.1073.2 / 89.3 / 96.458.8 / 53.1 / 70.4
Learnable Prefix61.3 / 50.1 / 73.53.12 / 2.64 / 3.0974.6 / 89.6 / 94.558.9 / 53.6 / 70.5
Eval w/o Latent60.8 / 51.7 / 70.13.10 / 2.57 / 3.0172.8 / 89.2 / 93.357.8 / 55.0 / 70.3

DeIRD outperforms all four controls on all four metrics across all three backbones. These results indicate that the gains cannot be explained solely by additional input positions or fixed prompts: continuous representations carrying instance-specific content make an effective contribution.

AuthorsOpenReview ↗

Response to Reviewer pGC3(3/3)

6. Inference parameters and computational cost (Q3)

We have added inference-cost measurements in Appendix H, “Inference Parameters and Computational Cost,” with results in Table 12. Parameter counts include the entire DMIR, including frozen weights, together with all adapters and audio components used during inference. FLOPs account for every DMIR iteration. The text VAE is excluded because it is not used in the standard inference pipeline.

Table R4. Results are reported as S / V / K: Step-Audio-2-mini, VITA-Audio-Plus-Vanilla, and Kimi-Audio-7B-Instruct. FLOPs are reported per request.

MethodParameters (B)First audio token (TFLOPs)Naturally completed responses (TFLOPs)
Base Model8.315 / 7.979 / 10.7472.017 / 2.207 / 4.2483.939 / 6.445 / 6.282
Explicit CoT8.315 / 7.979 / 10.7478.283 / 8.905 / 12.40313.230 / 12.451 / 16.849
DeIRD Top-117.162 / 16.826 / 19.59448.125 / 48.341 / 49.52649.025 / 49.197 / 51.279
DeIRD Top-417.162 / 16.826 / 19.59425.775 / 25.991 / 28.04326.918 / 26.734 / 29.329

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Discussions from the Authors

Dear PCs, SACs, ACs, and Reviewers,

We thank the reviewers for their careful assessment and constructive feedback. As the discussion closes, we summarize the recognized strengths, additional evidence, and manuscript revisions.

The reviewers highlighted three strengths of DeIRD, our decoupled implicit reasoning framework for spoken dialogue:

  • Practical motivation and framework design: Reviewer 8jeZ recognized the importance of balancing reasoning quality with response latency and considered fixed-length continuous reasoning representations a reasonably novel and technically interesting approach.
  • Performance–latency trade-offs across architectures: Reviewer jNDt highlighted DeIRD’s practical value and applicability across three spoken-dialogue backbones spanning interleaved and parallel generation. Reviewer 8jeZ also acknowledged the breadth of our evaluation across models, tasks, and ablations.
  • Engineering and documentation: Reviewer pGC3 recognized the predictable latency enabled by fixed-length latents, the solid engineering across three different backbones, and the documentation of the data pipeline, training costs, and implementation details.

Our main responses are summarized below.

Reviewer pGC3 raised questions about latent content, short textual CoT alternatives, latency overhead, and inference cost:

  • Latent content and expanded controls: We added Hint-only and Answer-only targets and extended the Noise Latent, Shuffled Latent, Learnable Prefix, and Eval w/o Latent evaluations across all three backbones, including evaluation on GSM8K. These results are included in Appendix D, Table 8. They provide strong evidence for the value of instance-specific CoT information beyond style guidance, final answers, or additional input positions alone.
  • Short-CoT comparisons and latency analysis: We introduced two 128-token short-CoT baselines for downstream training, using teacher-generated targets and DIFFA-2 targets, respectively. Both underperformed DeIRD on GSM8K and OpenAudioBench. We also clarified the intended reasoning-intensive applications and the performance–latency trade-offs of faster Top-2/Top-4 configurations.
  • Inference-cost evaluation: We added parameter counts and per-request FLOPs in Appendix H, Table 12, covering the complete DMIR, every prediction iteration, adapters, and relevant audio components.

Reviewer 8jeZ requested stronger attribution evidence, clearer training-cost accounting, and additional validation:

  • Training scope and prediction context: We clarified that one-epoch adaptation reuses the upstream components, whose training costs are separately reported in Appendix G. We also compared Teacher-context and Self-context settings; replacing committed predictions with reference latents did not improve downstream performance.
  • Held-out and independent evaluation: We specified the 100-sample held-out evaluation protocol in Section 4.2 and introduced relevance assessments by GPT-6 Astra and five academic researchers. The updated model and human evaluation results appear in Table 2 and Section 4.3, respectively.
  • Alternative reasoning paths: We added a controlled diagnostic on 100 GSM8K questions in Section 4.2, Table 3, with analysis in Section 4.4, Finding 5. Under greedy answer decoding, DeIRD performed comparably to explicit CoT with one backtracking step. We further clarified the hypothesis that continuous states can retain information about multiple candidate reasoning continuations.

Following these responses, Reviewer 8jeZ raised their rating from 2 (Weak Reject) to 3 (Weak Accept), stating that the additional results addressed most of their concerns and provided stronger support for the method.

Reviewer jNDt requested comparisons isolating the contribution of CoT supervision and evaluating the value of iterative prediction:

  • Beyond ordinary adaptation and answer distillation: We added Standard QA SFT using the same question–answer pairs and downstream training configuration, alongside Answer-only latent targets. DeIRD outperformed Standard QA SFT on all reported metrics. Compared with Answer-only targets, DeIRD improved BBH and OpenAudioBench reasoning scores across all three backbones.
  • Iterative versus single-pass prediction: We trained a Top-32 single-pass predictor and adapted each downstream backbone to its outputs. Comparing accuracy and TTFA (time to first audio) showed that single-pass prediction reduced latency but sacrificed downstream quality, providing further evidence for the value of iterative prediction.

We thank the reviewers for helping strengthen our comparisons, evaluation transparency, and interpretation of the results.

Sincerely,
The DeIRD Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.