- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces MOST-OPSD for phoneme decoding from neural recordings. MOST aligns intermediate neural states with three frozen Wav2Vec2 representations of transcript-synthesized speech using length-aware Soft-DTW. OPSD uses a speech-conditioned privileged branch to supervise a neural-only CTC branch, which alone is retained at inference. On the public Brain-to-Text '24 benchmark, the paper reports improved validation PER and 3.63% hidden-test WER for its complete fusion system, with further experiments on Brain-to-Text '25.
Critical strengths
-
The two transfer paths are technically concrete and substantially probed through component, teacher-view, shuffling, and data-fraction ablations. Inference remains neural-only, so training-time speech information does not create a direct test-time input dependency.
-
The public benchmark validation is a meaningful strength. The supplied Brain-to-Text '24 submission artifact reproduces the reported 3.63% WER for the complete fusion system on the held-out competition benchmark.
Critical weaknesses
-
The causal contribution of the proposed distillation design is not sufficiently isolated. The released implementation uses standard teacher-to-student forward KL between a trainable speech-conditioned privileged posterior and a neural-only posterior; it is neither on-policy in the usual autoregressive-distillation sense nor a reverse-KL objective. The paper does not compare this design with a matched ordinary-KD or reverse-KL baseline, and it does not isolate the gains from the privileged cross-attention, its additional CTC loss, and the posterior-KL term. Likewise, although TTS voices and speech encoders are varied, no direct text/phoneme teacher establishes that the frozen TTS-to-speech-encoder cascade is necessary.
-
The headline system result is not yet fully auditable or attributable. The abstract and Table 1 report 3.63% hidden-test fusion WER, but the conclusion reports 3.91%. Moreover, 3.63% is a complete fusion-system score, whereas validation uncertainty is reported over five neural-model seeds only. The paper does not provide hidden-test system-level uncertainty or a controlled fusion ablation, making the margin difficult to attribute specifically to MOST-OPSD. These inconsistencies and missing system-level evidence reduce confidence in the central empirical claim.
Questions for authors
- Please distinguish the frozen speech-feature extractor, the trainable privileged branch, and the posterior teacher used for KL. Why is teacher-to-student forward KL preferred here, and how does it compare with a matched ordinary-KD and reverse-KL baseline?
- Can the authors isolate the contributions of privileged cross-attention, its parallel CTC loss, and the posterior-KL loss? Can a direct fixed text/phoneme teacher match the TTS-to-speech-encoder cascade?
- Which complete-fusion hidden-test WER is correct (3.63% or 3.91%)? Please identify the submitted artifact, state how many independently trained neural decoders contributed to the hidden-test results, and provide a controlled fusion ablation or uncertainty where evaluation access permits.
If the response addresses these concerns comprehensively and convincingly, I would be willing to raise my rating.
Dear Reviewer,
Thank you very much for your time and thoughtful review. As the discussion deadline approaches, could you please take a look at the authors' rebuttal and share your thoughts in the discussion?
Many thanks for your continued help!