- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
RUBATO addresses positional inconsistencies that arise when variable numbers of reasoning tokens are interleaved with real-time audio and text streams. It assigns independent positional clocks to the real-time and thinking streams and compares Split, Gated, and Full implementations. RUBATO-Full computes each query–key interaction using relative positions on the key token’s clock across the full rotary dimension of each attention head. The main demonstrated result is robustness to unseen fixed and randomly varying thinking budgets, measured in reasoning tokens per audio frame, after training on two endpoint budgets.
Critical strengths
- The paper clearly explains how a single positional clock introduces changes in positional distances when the thinking rate varies. The independent-clock construction directly addresses this issue, and the diagrams help explain the distinction.
- The experiments cover fixed and randomly varying budgets, including single-turn reasoning and synthetic overlapping conversations. RUBATO-Full largely preserves accuracy at unseen rates while reducing time-to-answer as the per-frame thinking budget increases.
- The Split, Gated, and Full variants make the architectural choices easier to assess. The spoken-filler quality evaluation also adds evidence beyond reasoning accuracy and latency.
- The method can use existing fused attention kernels, and the optimized multi-KV implementation shows small decoding overhead in the reported batch-size-one H200 benchmark (Appendix C).
Critical weaknesses
- End-to-end evidence for hardware-adaptive operation is limited. The experiments establish robustness to prescribed thinking budgets. Appendix C provides a decode microbenchmark, and Appendix E estimates feasible budgets from hardware characteristics. An end-to-end streaming measurement under changing compute availability would provide stronger support for the claim that the system adapts its thinking rate while meeting audio deadlines.
- The practical alternative of intermediate-budget training is not evaluated. The endpoint-trained Single-Clock baseline is appropriate for testing interpolation under the same training layouts. However, a comparison with Single-Clock trained on a small set of intermediate budgets would clarify the practical advantage of the proposed training recipe. Section 4.6 varies the proportions of the two endpoint layouts and does not answer this question.
- Training variability is not reported. Figure 3 includes Wilson confidence intervals, but these do not capture variation across independently trained models. Repeated runs would strengthen the comparisons among RUBATO variants and the conclusions from the training-layout mixture ablation. In addition, RUBATO-Full already has a substantial advantage at the trained budgets on BIG-Bench Audio, so its absolute accuracy gains should be distinguished from its resistance to degradation at unseen budgets.
Questions for authors
- Building on the measurements in Appendix C and the estimates in Appendix E, could you provide an end-to-end streaming experiment under changing compute availability, reporting achieved thinking throughput and the fraction of missed 80 ms audio deadlines?
- How does endpoint-trained RUBATO-Full compare with Single-Clock trained on the endpoints plus a small set of intermediate budgets under a matched training budget? Reporting the additional data-construction effort would help assess the practical benefit of the two-endpoint recipe.
- Could you report variability across independent training runs for the main comparisons and representative training-layout mixtures, including whether the relative performance of Full and Gated remains consistent?
4 comments
Part 2/2
W3/Q3. Variability across training runs
To robustly measure variability, we retrained the models using two additional random seeds (three seeds in total).
Main comparison. Accuracy (%), mean ± s.d. over three seeds.
| Benchmark | Variant | B=1 | [1,2] | 2 | [2,3] | [3,4] | 4 | 8 | 16 | 32 | B= |
|---|---|---|---|---|---|---|---|---|---|---|---|
| BIG-Bench Audio | Single-Clock | 66.4 ± 14.0 | 43.1 ± 12.1 | 38.8 ± 11.3 | 36.2 ± 10.0 | 36.3 ± 11.1 | 35.6 ± 10.1 | 50.6 ± 7.8 | 61.3 ± 2.8 | 68.0 ± 2.2 | 78.2 ± 8.1 |
| Gated | 74.0 ± 8.1 | 74.6 ± 9.4 | 75.4 ± 10.4 | 73.0 ± 12.2 | 72.5 ± 12.6 | 72.2 ± 12.3 | 70.2 ± 13.3 | 69.7 ± 16.0 | 68.7 ± 15.7 | 76.4 ± 10.6 | |
| Full | 84.3 ± 2.5 | 85.4 ± 4.3 | 84.9 ± 4.7 | 84.0 ± 5.7 | 84.1 ± 4.3 | 84.1 ± 4.2 | 84.2 ± 5.2 | 84.8 ± 4.8 | 83.4 ± 3.5 | 86.3 ± 2.0 | |
| Spoken GSM8K | Single-Clock | 39.5 ± 11.2 | 10.5 ± 6.8 | 7.7 ± 4.2 | 4.5 ± 2.9 | 4.7 ± 1.7 | 5.6 ± 1.3 | 12.8 ± 0.7 | 20.7 ± 4.6 | 29.5 ± 5.0 | 44.2 ± 8.3 |
| Gated | 45.3 ± 5.7 | 46.1 ± 4.8 | 46.9 ± 5.1 | 47.2 ± 5.0 | 46.1 ± 4.4 | 45.7 ± 4.0 | 46.0 ± 4.3 | 45.9 ± 3.7 | 43.1 ± 3.3 | 46.5 ± 4.8 | |
| Full | 47.1 ± 2.4 | 47.2 ± 3.7 | 47.1 ± 2.2 | 45.9 ± 5.6 | 47.2 ± 4.4 | 46.5 ± 4.0 | 45.7 ± 5.8 | 47.8 ± 3.1 | 47.0 ± 3.3 | 49.2 ± 2.1 |
Training-layout mixture ablation. Spoken GSM8K accuracy (%), mean ± s.d. over three seeds.
| Mixture | Variant | B=1 | [1,2] | 2 | [2,3] | [3,4] | 4 | 8 | 16 | 32 | B= |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 21.9% | Gated | 49.6 ± 3.9 | 49.0 ± 4.8 | 47.6 ± 6.0 | 46.9 ± 8.3 | 45.7 ± 6.6 | 45.5 ± 6.0 | 45.9 ± 5.4 | 46.8 ± 6.7 | 43.3 ± 8.3 | 51.9 ± 2.5 |
| Full | 53.0 ± 2.4 | 54.4 ± 1.2 | 54.0 ± 2.4 | 55.1 ± 3.5 | 53.5 ± 3.4 | 54.6 ± 2.5 | 52.8 ± 3.8 | 51.8 ± 4.1 | 49.7 ± 6.6 | 57.2 ± 3.0 | |
| 4.4% | Gated | 40.4 ± 17.3 | 43.1 ± 14.7 | 45.0 ± 12.8 | 45.1 ± 10.1 | 46.7 ± 7.3 | 47.6 ± 7.4 | 48.3 ± 3.3 | 47.4 ± 0.3 | 41.4 ± 0.4 | 55.0 ± 2.6 |
| Full | 53.7 ± 0.0 | 54.3 ± 0.8 | 54.5 ± 0.4 | 54.5 ± 0.6 | 54.9 ± 0.2 | 55.2 ± 1.4 | 53.0 ± 2.6 | 50.3 ± 5.0 | 48.4 ± 4.6 | 57.4 ± 2.5 |
Across both experiments, RUBATO-Full demonstrates significantly higher robustness across training seeds compared to RUBATO-Gated, and the main trends hold. Thank you for raising the specific point regarding BIG-Bench Audio. We found that RUBATO-Full's absolute lead over Single-Clock at the explicitly trained budgets is indeed a result of training noise. Specifically, one of the Single-Clock seeds manages to match Full there (achieving 82.4% vs. 82.2% at ). However, Single-Clock's severe performance degradation at unseen budgets is consistent across every seed. Its worst unseen-budget accuracy is 34.2 ± 9.3%, compared to 82.8 ± 4.2% for Full. This structural robustness to interpolation, rather than an absolute accuracy gain at the trained endpoints, is the advantage of RUBATO, and we will update our claims in the text to reflect this clearly. We will include these multi-seed results in the revised appendix.
Follow-up on Rebuttal
Dear Reviewer piFu,
Thank you again for your time and constructive feedback.
As the end of the discussion is approaching, we wanted to gently check if our rebuttal has resolved your concerns. Please let us know if any further clarification is needed. We are fully available to answer any remaining questions before the deadline.
Best regards,
The Authors
Thank the authors for the detailed response; it has resolved most of my concerns.
Part 1/2
Thank you for highlighting our clear exposition of the single-clock problem, the robustness demonstrated in our fixed and randomly varying budget evaluations, the comparative insights from the Split/Gated/Full ablations, and the low decoding overhead achieved on existing kernels. We address your aligned weaknesses and questions below.
W1/Q1. End-to-end streaming under changing compute
To clarify the deployment paradigm. In a live setting, the per-frame thinking budget Bf is not determined in advance. Instead, a scheduler manages decoding directly. It allows the model to process thinking tokens until the next 80 ms audio deadline approaches; it then pauses the thinking stream, ingests the incoming user audio, decodes the required real-time tokens for that frame, and resumes thinking in the subsequent frame. Because of this, internal reasoning never delays the real-time audio streams. The achieved thinking throughput simply emerges as the resulting sequence of Bf. RUBATO's decoupled clocks allow the model to seamlessly handle arbitrary, fluctuating per-frame budgets because the real-time streams maintain their own independent positional clock.
From the model's perspective, deployment is mathematically equivalent to processing a randomized sequence of per-frame budgets Bf. Therefore, our frame-varying evaluations are a direct test of the changing compute scenario. When Bf is drawn uniformly from [1,16] at every frame,
RUBATO-Fullachieves 52.7% on Spoken GSM8K and 88.9% on BIG-Bench Audio, compared to a near-collapse of 15.5% and 47.0% forSingle-Clock. We agree that building a live, end-to-end systems benchmark (handling streaming audio I/O and measuring actual performances across various GPU architectures) would provide excellent supplementary evidence. However, it requires substantial systems engineering beyond the architectural scope of this paper, and we plan to explore it in future work. We will add a detailed explanation of the scheduler and deployment mechanics to Section 4.2 and expand our Limitations section accordingly.W2/Q2. Single-Clock trained on intermediate budgets
Thank you for this suggestion. We ran this exact experiment matching both data-construction and training compute. Please see the Common Response for the detailed results.
In brief, adding intermediate budgets helps
Single-Clock(raising mean accuracy from 15.1% to 48.9%), but the returns diminish quickly and fail to close the gap withRUBATO-Full. At its explicitly trained budgets,Single-Clockremains 2–8 points belowRUBATO-Full, and at unseen budgets, the gap grows to 6–13 points.