acceptodds
Desk rejected at ICLR 2027

MergePath: Constrained Path Compilation for Frozen-Decoder Prefill

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-context services spend a growing share of request time before decoding begins. Prompt coarsening can recover that capacity, but a frozen decoder requires a selected representation to be an exact byte partition, preserve protected literals, satisfy the execution contract of the serving stack, and repay request-time selection work. We formulate this retrofit as constrained path compilation and introduce MergePath. A legality compiler constructs a typed set of byte-exact, protected-span-safe, within-block candidates; a merge-risk estimator ranks legal candidates; and a deterministic packer commits an exactly covering path before one frozen-decoder prefill. The selected spans are represented by a boundary-residual composer while the decoder parameters and output vocabulary remain unchanged. On frozen Llama-3.1-8B, the reported operating point reaches PPL at ms with exact-copy accuracy, compared with at ms for the evaluated frozen BLT port. On byte-identical RULER-64k prompts, the unweighted thirteen-task macro is for MergePath and for Static BPE. These results show how compiling admissibility before learned value ranking exposes an executable adaptive-resolution path for frozen long-context models.

Primary area:
foundation or frontier models, including LLMs
Keywords:
adaptive reasoning and inference, MergePath

Related papers

Discussion (0)

Sign in to comment.