One Price, Many Futures: Shared-Price Option Indices for Post-Prefix Compute Allocation
Abstract
Inference-time compute is often allocated before reasoning begins. A shared prefix changes the decision: the realized state reveals which requests may benefit from more work, including cases that recover only at a distant endpoint. All continuation options also share a batch budget, so their predicted values must be comparable. We formulate this problem as state-conditioned selection among complete continuation options. Shared-Price Option Indices (SPOI) use the realized prefix state to value each option and compare all requests under one common price. Our analysis shows when state information improves allocation, when myopic stopping misses delayed recovery, and when heterogeneous options can share that price. Two complementary estimators trade off cardinal value preservation against a bounded learning target, with validation selecting between them. In a scale-selection study specified without test-based selection across four backbones, two tasks, and nine matched budgets, SPOI outperforms query-only routing in most settings and matches or improves on independently calibrated option scores. Frozen SPOI schedules also produce physical savings: recurrent active-batch execution on A800 reaches speedup with mean control overhead, while standard KV-cache execution reaches up to .
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.