- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper argues that S2S post-training data lacks realistic non-semantic information (the “Non-Semantic Information Gap”). It proposes RAST, which builds each two-turn exchange around one verified real recording and synthesizes only the other side. RAST-U keeps real user speech, RAST-G keeps real (voice-converted) assistant speech, and RAST-Mix pools both. The corpus has 284,660 exchanges (2,914 h) mined from YouTube. The authors also introduce RASTBench, a transcript-controlled benchmark scored from response audio by an LLM judge. Fine-tuning Kimi-Audio-7B, RAST improves RASTBench over Pure-TTS controls by 4.7 (U), 4.5 (G), and 8.8 (Mix) points, and a quarter of the data recovers most of the gains.
Critical strengths
- Clean paired-control design. Each RAST configuration has a Pure-TTS control that changes only the provenance of one endpoint and keeps texts, counterparts, and training horizon fixed. This is the right way to isolate the value of real speech, and it is stronger than typical data papers.
- Audio-native, transcript-controlled evaluation. RASTBench scores response audio within same-transcript contrast groups and separates task completion from condition-specific adaptation. The judge is calibrated against expert labels (88% agreement).
Critical weaknesses
- Heavy conceptual packaging and abstract writing obscure a simple idea.
- Too many bespoke terms. The core idea is simple: keep one real recording per exchange and synthesize the other side. Yet the paper introduces many custom terms (“Non-Semantic Information Gap,” “anchor,” “endpoint,” “real-speech-origin,” “displacement criteria,” “capability profile,” “non-semantic grounding”) where standard terms would do.
- The central concept is not measurable. The NSIG is defined vaguely, never quantified, and its abbreviation is used only once.
- Misleading labels. RAST-G anchors are denoised and voice-converted, yet they are called “real-speech-origin.” It is unclear how much real prosody survives.
- Assertion presented as derivation. Section 2 claims to “derive” the design, but it only offers a qualitative table of unsupported pros and cons.
- Hard-to-evaluate sentences. For example: “one verified real recording suffices when kept in the role whose realization is the supervision signal,” or “the counterpart must answer or license the anchored transcript.”
- Post-hoc explanations. Negative results are explained away without evidence, e.g., the WavBench drop with more data, or Mix’s weaker expressiveness reframed as “a different capability profile.”
Questions for authors
How much AI was used? Could you use less "auto research" and instead spend some time thinking about how to do it yourself?
2 comments
Thank you for your response. I acknowledge your idea and effort, but you should maintain basic fluency and readability in the writing, which is the most basic requirement for a paper, rather than using AI to write an article around an idea. I will keep the score unchanged.
Response to Reviewer DTqd
We thank the reviewer for raising this question. We would like to clarify the role of AI in our work and respectfully disagree with the characterization of this study as "auto research."
As disclosed in our AI Use Statement, AI tools were used for paper language editing and scalable data processing, including transcription, annotation, dialogue generation, and speech synthesis. However, the research design, methodological decisions, validation procedures, experiments, and analysis were carried out by the authors.
In particular, substantial effort went into processing over 12,000 hours of source speech, comparing and selecting annotation models, developing attribute-specific verification rules, and manually labeling approximately 500 samples to validate the annotation pipeline. We also designed and refined RASTBench, established its quality-control criteria, and conducted the training experiments and analyses. These steps required extensive human judgment, testing, and iteration rather than simply relying on AI-generated outputs.
We are grateful to all reviewers for their valuable suggestions for improvements regarding the deficiencies in our paper. However, these limitations should not be confused with a lack of independent research effort. AI tools enabled us to process data at scale, but they did not replace the research decisions or experimental work.