- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces Cache Complete Cone Prefill (C3P), a prefill schedule for causal decoders with local or otherwise statically known receptive fields. C3P treats the requested prompt logits and the continuation state as targets in a static computation graph, traces their backward dependency closure, and evaluates only that closure. The schedule distinguishes positions that require a full transformer-block output from positions that require only key/value or other state production. The paper proves equality with dense prefill in real arithmetic and inclusion-minimality within the explicitly restricted class of deletion-only schedules that preserve the original operators and state representation. It also derives closed-form layerwise row and MAC counts for sliding-window attention.
Critical strengths
This paper proposes C3P: a training-free, calibration-free method that accelerates long-context prefill without modifying the model or the decoding interface, by tracing a dependency cone—planned top-down and executed bottom-up. The paper proves that this schedule is the unique minimal choice among deletion-only approaches, derives closed-form solutions for the numbers of full-output rows and K/V rows under local attention.
Critical weaknesses
-
The gains on real large models are modest (1.045×–1.119× on Mistral-7B) and depend heavily on the window-to-depth ratio; moreover, all experiments are conducted at batch size 1, leaving the net benefit unclear under realistic serving workloads with continuous batching and prefix caching.
-
Theorem 1 only guarantees exactness in real arithmetic; under BF16, the rounding trajectories of C3P and dense prefill differ, so outputs are no longer bitwise identical.
Questions for authors
- Modern serving systems rely heavily on prefix caching. Under realistic high-concurrency workloads, does C3P still retain net gains?
- The method targets the last-position logits only. If a workload requires the full prompt logits, does the cone degenerate entirely?