SAPO:Sparse-Aware Prompt Optimization

Large Language Models (LLMs) exhibit activation sparsity that varies with input prompt structure, creating an opportunity to jointly optimize prompts and model computation. We introduce Sparse-Aware Prompt Optimization (SAPO), a framework that combines prompt engineering with reduced-order computation to accelerate both homogeneous and heterogeneous inputs. This is important for physical and embodied AI, where prompts can combine natural language, numerical sensor data, system parameters, and equations. We show that explicitly organizing heterogeneous information induces contextual activation locality, producing more correlated and compressible activation representations across neighboring layers. SAPO exploits this structure through a $H\to P$ transformation based on Koopman operator theory and the Mori–Zwanzig formalism, replacing $H$-neuron continuous-depth layers with compact $P$-dimensional Gated Recurrent Unit (GRU)-based blocks ($P\ll H$) and replacing iterative Ordinary Differential Equation (ODE) solves with parallelizable matrix operations. Experiments on large-scale foundation models show that reduced-order computation accelerates both prompt types, while prompt engineering amplifies its benefit for heterogeneous inputs. SAPO achieves $3.55\times$ training speedup for structured prompts compared with $2.00\times$ unstructured prompts, together with approximately $2.9\times$ lower aggregate test time while maintaining comparable task quality. These results demonstrate the synergy between prompt engineering, which exposes more compressible activation structure, and reduced-order computation.