- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Trajectory-Local Adaptive Retrieval, a speculative decoding method that augment existing stand-alone drafters with fuzzy-matched patterns retrieved from the ongoing generation trajectory. The paper provides comprehensive analysis of the approach, including it's incremental improvements on top of existing baselines and different trends in diverse domains. Experiments also showcase improvements over the baselines.
Critical strengths
The paper is easy to read and the motivation is clear. It introduces multiple components to maximally utilize the within-trajectory information for improved speculative decoding, and is composible with existing methods (which is useful for easy adoption).
Critical weaknesses
The novelty of the paper is somewhat limited, as previous work (including STAND) already showcased that drafting with self-trajectory retrieval can improve the speculative decoding performance, which seems to be the core idea of this paper.
The empirical usefulness is not fully addressed. While the paper reports performance by combining smalldraft (stand-alone drafters) with retrieval-based drafts, there are much faster and performant drafting methods available nowadays, including Eagle, DFlash, and DSpark. To show that the provided approach has practical value, the paper should demonstrate that the given approach can improve the performance even when combined with strong drafters, rather than relatively weak, standalone drafters.
For acceptance, I would expect the paper to either be conceptually novel or show strong practicality. The current version of the paper does not satisfy this bad.
Questions for authors
This is a suggestion for improving the paper in the future, rather than a question. So addressing this question is not crucial for my assessment of the paper in the current round.
Having that said, I think the paper could be substantially stronger if it is positioned for long-running agentic systems. As suggested in Table 2 (and as expected), the gain of this work gets stronger with longer context. The gain may seem rather marginal for the context lengths tested, which is up to 4k. However for long-running agent systems, context easily accumulates to multiple millions of tokens, providing rich information for trajectory-based speculators to use. If the authors explicitly suggest this as the research motivation and build the evaluations around this setup, the paper could become much more timely, and empirically powerful.