- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes VoxOPD, a cross-modal hierarchical on-policy self-distillation method for spoken dialogue models. It aims to improve both semantic ability and speech expression. The student generates mixed text–audio trajectories from speech-only inputs. A frozen teacher, initialized from the same model, also receives transcripts and task-specific rationales. It provides supervision at student-generated prefixes. The objective combines text-token distillation, audio-token distillation, and a chunk-level loss that shares feedback across neighboring audio tokens.
The authors train on 38,532 English spoken-interaction examples and evaluate three backbones: Step-Audio 2 mini, Kimi-Audio, and VITA-Audio. Evaluation includes VoiceBench, VStyle, human listening tests, and several ablations. On Step-Audio 2 mini, VB-Cap increases from 60.7 to 64.7, and VStyle increases from 3.14 to 3.85. Aggregate scores improve over all three base models. However, gains vary across subtasks and do not always exceed other post-training methods. The main contribution is joint distillation over mixed text–audio outputs, with an audio-chunk credit assignment objective.
Critical strengths
Joint text and audio supervision is practically useful, and the ablations help separate the roles of its main components.
Critical weaknesses
-
The comparison settings and the reliability of the key gains are not sufficiently established. This is my main concern. There is nothing about baseline training budgets, tuning ranges, the GRPO reward, or the adaptations of CORD and ParaBridge. Using the same data pool does not establish comparability along these dimensions.
-
The connection between the perturbation evidence and the chunk-level objective needs clarification. Figure 1(b), Appendix A, and Table 4 show that decoded speech changes little when a small fraction of audio tokens is replaced by nearby codebook entries. However, this may reflect codebook geometry or decoder smoothness, and does not directly explain why feedback should be shared across neighboring time steps. It is also unclear whether teacher–student discrepancies during training resemble these perturbations.
Questions for authors
You should clarify experiment settings and the meaning of "local redundancy" in the chunk-level loss, time locality or codebook locality? I also wandor a more distinguish experimental observations from design motivation and mechanism about the design of the chunk-level objective.
5 comments
W2.1 & Q1. Mechanism of Chunk-Level Supervision
Our design principle is to match the granularity of supervision to the granularity at which speech properties emerge.
Semantic and acoustic attributes are typically realized jointly by audio tokens across multiple temporal positions, and changing a single local token need not alter the corresponding output. Therefore, as discussed in the Introduction, token-level alignment may overemphasize differences that do not affect the output and consequently yield only limited improvements in actual generation.
From this perspective, codebook locality and temporal locality characterize different, non-exclusive aspects of the same representation-to-generation process. Codebook geometry, decoder smoothness, and temporal context may all attenuate the effect of local representation changes on the decoded output. The perturbation study is intended to show that changes in audio tokens do not correspond one-to-one with changes in the output, motivating complementary supervision over segments composed of multiple tokens. Accordingly, we retain token-level alignment and add chunk-level supervision to account for the segment-level behavior jointly determined by multiple audio decisions.
We further clarify the shared-credit mechanism at the chunk level. For each audio position in the current sampled trajectory, let
\log \pi_T(y_t\mid a,x,r,y_{<t}) -\log \pi_\theta(y_t\mid a,y_{<t}).$$ The shared credit assigned to chunk $C_k$ is $$A_k=\operatorname{sg}\!\left[ \frac{1}{|C_k|}\sum_{t\in C_k}\delta_t \right].$$ This mean is also the least-squares solution obtained when approximating the token-wise signal by a constant within each chunk: $$\frac{1}{|C_k|}\sum_{t\in C_k}\delta_t = \underset{b}{\operatorname{arg\,min}}\sum_{t\in C_k}(\delta_t-b)^2.$$ For a fixed sampled trajectory, and applying stop-gradient as in Eq. (5), the update direction for a single chunk is $$-\nabla_\theta \ell_k = \frac{1}{|C_k|^2} \sum_{i\in C_k}\sum_{j\in C_k} \delta_i\,\nabla_\theta\log\pi_\theta(y_j\mid a,y_{<j}).$$ Thus, the teacher signal at every position influences the updates of all tokens within the same chunk through chunk-level aggregation. This shares feedback locally, reduces sensitivity to discrepancies at individual token positions, and promotes segment-level consistency. As shown in Table 3, adding $\mathcal{L}_{\mathrm{chunk}}$ to token-level distributional alignment improves VB-Cap and VStyle from 63.9 and 3.71 to 64.7 and 3.85, respectively. To further examine this mechanism, we compare our method with generic approaches that weaken or smooth the token-level signal on Step-Audio 2 mini. All controls set $\lambda_{\mathrm{chunk}}=0$ and either reduce the weight of the audio-modality token-level loss, $\lambda_{\mathrm{audio}}$, or increase the temperature $\tau$ to smooth the teacher distribution. All other settings remain unchanged. | Setting | VB-Open | VB-Cap | VStyle | | --- | --- | --- | --- | | $\tau=1.0,\ \lambda_{\mathrm{audio}}=1.0$ | 3.39 | 63.9 | 3.71 | | $\tau=1.5,\ \lambda_{\mathrm{audio}}=1.0$ | 3.34 | 64.3 | 3.74 | | $\tau=2.0,\ \lambda_{\mathrm{audio}}=1.0$ | 3.33 | 63.8 | 3.80 | | $\tau=1.0,\ \lambda_{\mathrm{audio}}=0.5$ | 3.38 | 64.1 | 3.81 | | $\tau=1.5,\ \lambda_{\mathrm{audio}}=0.5$ | 3.38 | 63.9 | 3.78 | | Ours | 3.42 | 64.7 | 3.85 | Reducing $\lambda_{\mathrm{audio}}$ or increasing $\tau$ improves VStyle to some extent, supporting the observation that per-token audio supervision can overemphasize local differences. However, simply weakening or smoothing token-level supervision remains inferior to the full method across all three dimensions. This shows that chunk-level shared credit is not merely a form of distribution smoothing or loss reweighting. Instead, it aggregates teacher signals from adjacent tokens to provide complementary segment-level supervision, yielding more balanced improvements in semantic capability and acoustic expressiveness. This result is consistent with our motivation: adjacent audio tokens jointly determine the semantic and acoustic realization of a local speech segment, and the shared-credit mechanism provides a simple and effective supervisory signal for such segment-level dependencies. Smoothing the teacher distribution or adjusting $\lambda_{\mathrm{audio}}$ can also be combined with our method as a potential avenue for further optimization.W2.2. Teacher–Student Discrepancies and Perturbations
To examine how teacher–student discrepancies during training relate to the perturbation study, we consider positions where the teacher's preferred token differs from the token sampled by the student. We then compute the proportion for which the teacher's preferred token falls among the student's top- nearest codebook neighbors (Hit@):
| Step | Hit@5 | Hit@10 | Hit@20 | Hit@50 |
|---|---|---|---|---|
| 0 | 33.28% | 49.54% | 64.25% | 76.61% |
| 300 | 33.26% | 49.71% | 64.34% | 76.50% |
| 600 | 33.03% | 49.61% | 64.24% | 76.61% |
| 900 | 33.21% | 49.19% | 64.05% | 76.31% |
These results show that, when the teacher and student disagree, the teacher's preferred token often lies in the local codebook neighborhood of the student's token. Thus, teacher–student discrepancies during actual training frequently take the form of local substitutions within the codebook, consistent with the locality of the replacements considered in the perturbation study.
Thank you for your constructive feedback. We appreciate your positive assessment of the practical value of joint text and audio supervision and the role of our ablations in disentangling the contributions of the main components.
W1 & Q1. Baseline Training Configurations
We summarize the training configurations of all baselines below. For CORD and ParaBridge, we follow the objective-related hyperparameters in the original papers. To ensure a fair comparison, we adjust the remaining training settings only within reasonable ranges while keeping the overall training conditions comparable. Because ParaBridge self-distills by comparing the same model with and without a scaffold, it converges rapidly and degrades with continued training; we therefore adjust its number of training steps accordingly.
For GRPO, we use Gemini 2.5 Pro as the reward model. We have added the scoring criteria to the appendix.