Before the First Token: A Shared Causal Subspace for Response-Length Planning in LLMs

Large language models (LLMs) often produce responses of markedly different lengths across prompts and instructions, yet it remains unclear whether response length is explicitly planned before decoding and, if so, how such plans are represented internally. We investigate pre-generation response-length planning in LLMs. We first show that future response length is encoded before the first generated token: controlled length perturbations induce persistent changes in response length even when the explicit length signal is unavailable during subsequent decoding, supporting the existence of a pre-generation planning state rather than a purely autoregressive length effect. We then introduce a cross-replicated trajectory analysis that isolates within-prompt representation changes associated with actual response length and identifies components that generalize across instruction paraphrases and families. This analysis reveals a highly reproducible, low-dimensional shared subspace with a dominant length-related axis and additional nonlinear trajectory structure. We further establish its causal role by decomposing hidden-state interventions into components parallel and orthogonal to the learned subspace. The parallel component preserves most of the bidirectional length-transfer effect of the full intervention, whereas the orthogonal component has substantially weaker effects. Moreover, train-derived subspace displacements systematically shorten or lengthen responses for held-out prompts without explicit length instructions and transfer across prompts. Together, these results suggest that response length is not merely an emergent consequence of autoregressive decoding, but is partly governed by a shared, low-dimensional internal control geometry established before generation begins.