- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AsyncReasoning, a training-free method that enables large language models to reason and generate responses concurrently. The method uses positional embeddings and a shared KV cache to support asynchronous thinking and writing. It also introduces a mode-switching mechanism that determines when the model should continue responding or pause for further reasoning. The authors evaluate their approach on mathematical reasoning, commonsense reasoning, safety, and interactive tasks. Experimental results show that AsyncReasoning significantly reduces response latency while preserving most of the accuracy benefits of reasoning models.
Critical strengths
-
The idea of allowing models to think and generate responses concurrently without additional training is interesting. The use of positional embeddings and a shared KV cache provides a practical solution to this problem.
-
The experiments cover multiple models and reasoning benchmarks, and the results demonstrate significant improvements in response latency while maintaining relatively high accuracy.
Critical weaknesses
-
The proposed method still leads to some accuracy degradation compared with standard reasoning. For example, Table 1 shows that the accuracy of GPT-OSS-120B on AIME-2025 decreases from 0.77 to 0.66 when using AsyncReasoning. Although the latency improvement is significant, it would be helpful to provide further analysis of why this accuracy loss occurs and whether it can be reduced.
-
The evaluation of real-world interactivity could be more comprehensive. Although the paper includes experiments with additional user inputs, most evaluations are based on mathematical and reasoning benchmarks. It would be interesting to see how the method performs in more realistic scenarios, such as continuous conversations or frequent user interruptions.
Questions for authors
please refer to "Weaknesses"
Thank you for your review and suggestions! We address your concerns below.
We report our error analysis in Appendix C for the main tasks and further analyze safety failure modes in appendix L. In the setup you described, OPT-OSS-120B scored 77% with medium-effort sequential reasoning, 66% with AsyncReasoning, and 51% with low-effort reasoning (Table 16). The main cause is that the Writer commits to an answer before the Thinker has verified it. We observed similar behavior on Qwen3-0.6B/MATH-500 (Appendix C). In our ablation analysis (Section 4.1, Appendix F), we found that this can be mitigated by tuning the prompts and mode switching mechanisms for the specific task. However, to keep our method simple and avoid experimental confounders, we use the same prompt and mechanism across all models and tasks. We report accuracy drawdowns where they occur and, overall, we find that AsyncReasoning offers a valuable trade-off between accuracy and response speed.
We agree wholeheartedly! The main problem with real-world reasoning interactivity is that it is difficult to measure with scientific rigour, as it depends on many subtle cues in human interaction, which is why we evaluated more specific scenarios that can be measured.
For non-mathematical reasoning, GPQA Diamond measures reasoning outside mathematics, including biology and chemistry problems. In turn, ZebraLogic tests logical deductive reasoning without equations.
For more frequent user interruptions, while our MATH-500 Sharded evaluations only have a single interruption, the Lost in Conversation math problems (Figure 6) include 5-6 corrections per task. We agree that certain real-world scenarios can have even more dynamic interruptions that depend on the agent’s responses to previous user feedback, but these are notoriously difficult to capture in a benchmark.
A user study would indeed be interesting, even though it introduces complications. Since each user has a different speech cadence and habits on how soon they ‘interrupt’ the assistant, the results will vary significantly between populations, especially for dynamic conversations. To that end, we measure full end-to-end delay on a subset of 50 MATH-500 samples on Qwen3-30B-A3B-2507-Thinking in the same setup as in Section 4.2.
The main difference from our previous evaluations is that we do not simulate user via the TTS but actually spoke to the AsyncReasoning agent running on a single H100 GPU (including Whisper-base ASR and Tortoise TTS). The results below are the wall time from the moment the user stops talking to the moment the system begins speaking, and the frame-by-frame silence duration. Compared to our main evaluations, this includes both automated speech recognition and software / hardware delays.
The new end-to-end are slightly (<1s) higher than the controlled evaluation from Section 4.2 due to speech recognition, input hardware and software delays. Note that the difference in total delays in our main results is much larger (tens to hundreds of seconds). We hope that our answers help alleviate your concerns. If you have another specific scenario you would like to test, please suggest it in a follow-up response.