- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
YOCO-U introduces weight-shared recursion into YOCO’s efficient-attention self-decoder while retaining the cross-decoder’s shared global KV cache. The objective is to improve effective depth and parameter utility without the global-cache growth of recursive full-attention Transformers. The paper evaluates language-modeling quality, architectural alternatives, scaling behavior, and inference efficiency.
Critical strengths
-
The integration offers a useful quality–efficiency trade-off. Table 2 reports a 4.45-percentage-point average improvement over YOCO under the stated equal-FLOPs comparison.
-
The dense architecture comparison achieves approximately the same average accuracy as RINS while reporting substantial long-context throughput advantages.
-
Loop-position, KV-sharing, and depth–width ablations address relevant design choices. The combination of MoE experiments, dense comparisons, and serving measurements gives the contribution practical relevance.
Critical weaknesses
-
Compute accounting needs clarification. Figure 2 labels its horizontal axis as training FLOPs, although the main analysis uses tokens multiplied by executed layers. Appendix F estimates FLOPs at 2048 tokens rather than the actual 8192-token training length. The exact equal-FLOPs budgets and the criterion for removing one validation-loss point per model should also be specified.
-
Quality and efficiency are demonstrated over different context-length ranges. The retrieval and book/code quality experiments extend to 8K, whereas serving measurements reach 128K/256K. The latter do not establish that quality is maintained at those lengths.
-
Fixed-depth recursive training should be distinguished from increasing the depth of one checkpoint at inference. The additional cost relative to YOCO is also context dependent: decoding throughput is about 28% lower at 8K, versus about 5% lower at 256K. These qualifications limit broad negligible-overhead and test-time-scaling interpretations.
Questions for authors
-
Can you provide the exact token/step budgets, FLOP estimates at the actual training length, and unfiltered validation-loss measurements, together with the outlier-removal criterion?
-
Is a limited quality evaluation at 32K or 128K available under settings corresponding to the serving measurements?
-
Please align the main scaling claims with the fixed-recursion setting acknowledged in Appendix G, or provide a single-checkpoint inference-depth sweep if available.
2 comments
Thank you for acknowledging our clarifications on the training budgets and for your constructive suggestions on improving the precision of our experimental claims.
Thank you for clarifying the training budgets. I am keeping my rating at 3.