- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SyncAudio introduces parallel text-speech generation with two components: an audio-state-conditioned boundary predictor for dynamically grouping speech tokens, and sparse aggregation of backbone representations from different depths to condition the audio decoder. The model is evaluated on spoken QA, text-speech consistency, ASR/TTS, and full-duplex interaction. Compared with a matched fixed-group baseline, the reported results show substantially better synchronization and improved average spoken QA performance.
Critical strengths
The synchronization comparison is informative. Audio-guided boundaries reduce the reported MAD from 3.26 to 0.22 without the intelligibility degradation observed with text-guided prediction. This supports conditioning boundary decisions on the evolving audio state.
FixedAudio is matched in backbone, training data, and training recipe, providing a useful control beyond comparisons with heterogeneous existing systems. The separate depth-aggregation ablation also helps assess the second component.
Critical weaknesses
The central supervision signal is insufficiently specified. Section 3.3 and Appendix B assume reference audio-group boundaries but do not explain how speech frames are assigned to individual LLM text tokens to construct these labels. This is essential to both training and the synchronization evaluation. It is also unclear whether MAD is measured under teacher forcing or free-running generation. The boundary threshold and exact aggregation-layer indices are not provided either, leaving important parts of the method difficult to reproduce.
The capability-preservation interpretation is not well isolated. Table 7 compares the original backbone with text input against adapted models with speech input, which conflates capability retention with speech understanding. The spoken QA gains in Table 1 are largely driven by AlpacaEval, while factual QA results are mixed—for example, spoken WebQ decreases from 34.9 to 33.1. The key QA ablations also lack uncertainty estimates; sparse aggregation exceeds dense aggregation by only 0.38 points in average spoken QA. These results support a useful system improvement more clearly than the broader preservation claim.
Questions for authors
How are token-level reference audio boundaries constructed? Please specify the alignment procedure, boundary threshold, and selected backbone layers, and clarify whether synchronization MAD uses reference or generated histories.
Can you provide matched text-input evaluations of the adapted models to distinguish capability retention from improved speech understanding? Please also report uncertainty for the paired QA comparisons, particularly sparse versus dense aggregation.