Geometry of Large Language Model Controllability

Prompt engineering remains a brittle, heuristic practice governed by a fundamental *Controllability Dilemma*: extreme vulnerability to minor lexical variations (*Paraphrase Sensitivity*) alongside a persistent propensity to ignore explicit structural instructions (*Instruction Neglect*). In this paper, we reveal that this dilemma originates from a severe topological mismatch between the superficial prompt text space and the model's true operational Hilbert output space, invalidating the implicit isometric assumption of current prompt optimizers. To resolve this, we formalize prompting as the dynamic selection of a function within a continuous Hilbert space, equipping the prompt manifold with a pullback Riemannian metric tensor induced by functional output divergence over task distributions. Through this geometric lens, we mathematically prove that multi-head attention saturation induces a topological partition of the prompt manifold, driving Instruction Neglect via rank-deficient $ϵ$-null foliations and Paraphrase Sensitivity via unbounded transversal sectional curvature across deep Transformer layers. Operationalizing these theoretical insights, we introduce **Pullback-Guided Discrete Exploration (PGDE)**, a gradient-free algorithm that leverages empirical pullback distances as a zero-order geometric filter to dynamically enforce distant functional jumps and local variance bounds. Empirical evaluations across 7B to 70B parameter models demonstrate that PGDE significantly outperforms state-of-the-art baselines on strict instruction following (IFEval, +8.6% absolute accuracy gain on Llama-3-8B) and mathematical reasoning (GSM8K), while reducing search wall-clock latency by over 50% by early-rejecting functionally inert proposals and suppressing output variance under paraphrasing by 82%.