MergePath: Constrained Path Compilation for Frozen-Decoder Prefill
Abstract
Long-context services spend a growing share of request time before decoding begins. Prompt coarsening can recover that capacity, but a frozen decoder requires a selected representation to be an exact byte partition, preserve protected literals, satisfy the execution contract of the serving stack, and repay request-time selection work. We formulate this retrofit as constrained path compilation and introduce MergePath. A legality compiler constructs a typed set of byte-exact, protected-span-safe, within-block candidates; a merge-risk estimator ranks legal candidates; and a deterministic packer commits an exactly covering path before one frozen-decoder prefill. The selected spans are represented by a boundary-residual composer while the decoder parameters and output vocabulary remain unchanged. On frozen Llama-3.1-8B, the reported operating point reaches PPL at ms with exact-copy accuracy, compared with at ms for the evaluated frozen BLT port. On byte-identical RULER-64k prompts, the unweighted thirteen-task macro is for MergePath and for Static BPE. These results show how compiling admissibility before learned value ranking exposes an executable adaptive-resolution path for frozen long-context models.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.