- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Jacobi recirculation, replacing token-sequential recirculation with a small number of parallel updates over a known sequence. It establishes finite-round exactness and conditional convergence guarantees, and empirically demonstrates that two to three rounds often provide a sufficiently accurate approximation for evaluation, mixer training, and prompt prefill. The method substantially reduces the cost of these computations, with particularly strong gains in mixer training.
Critical strengths
-
The method substantially reduces a genuine computational bottleneck in recirculation, particularly during mixer training. The training gains remain substantial when each method uses its largest feasible batch on a single GPU. This provides a concrete practical benefit beyond the favorable single-sequence prefill setting.
-
The approximation is evaluated extensively across models, mixing configurations, sequence lengths, numerical precisions, and downstream tasks. The rounding-scale comparisons, batch-scaling analysis, and optimized sequential baseline strengthen the evidence that the reported gains preserve useful behavior and are not limited to one favorable configuration.
Critical weaknesses
-
The broader significance remains conditional on the usefulness of recirculation itself. The paper convincingly makes recirculation cheaper, but does not establish its accuracy–cost competitiveness against alternative adaptation or additional-computation strategies; Appendix A explicitly leaves matched-compute comparisons to future work. The strongest demonstrated benefits concern training and known-sequence processing, while end-to-end generation gains are approximately 1.1–1.3×. I regard this as a limitation of the demonstrated practical scope, rather than a failure to support the stated acceleration claims.
-
The algorithmic novelty is relatively limited. Parallel fixed-point iteration for sequential computations is an established approach, as the paper acknowledges. The principal advance appears to be its formulation and careful validation for recirculation on frozen pretrained models, especially the empirical finding that a small number of rounds can support both evaluation and training. This is useful, but the distinction from existing parallel fixed-point methods limits my assessment of the methodological contribution.
Questions for authors
Which parts of the convergence and gradient analyses rely specifically on the structure of recirculation, rather than following from general results for causal fixed-point iterations? Clarifying this distinction would help assess the technical contribution beyond the empirical finding that a few rounds suffice.
I also have two minor suggestions for improving the figures. In the caption or nearby legend of Figure 1(C), please clarify that “Evaluation” refers to perplexity evaluation without autoregressive decoding. In Figure 3, please reposition the legends in panels A and C so that they do not obscure the plotted data, and make the clustered SR and JR mixer markers in panel A easier to distinguish. These are presentation suggestions and are not central to my overall recommendation.
2 comments
Thank you for the follow-up. We are glad that our response resolved your technical questions, and we appreciate your positive assessment.
Follow-up on the Author Response
Thank you for the response. My question about which parts of the analysis are specific to recirculation has been answered, the figure suggestions have been implemented, and Tables R1–R2 address my concern about generation gains by showing when the end-to-end speedup becomes substantial. The general response also clarifies the contribution relative to prior work on parallel fixed-point methods.
These clarifications resolve my technical questions, but do not change my overall assessment of the contribution's significance, which remains tied to the practical value of recirculation itself. I view this remaining reservation as a question of scope rather than technical soundness, and I do not consider additional experiments essential to complete my assessment. That said, my overall assessment remains positive. The early interest in recirculation, including independent reimplementations, is encouraging, and the practical value of this work could grow if recirculation gains wider adoption. Taking the clarifications and additional results into account, I maintain my rating.