Beyond the Base: Frequency Allocation in RoPE

In rotary position embeddings (RoPE), reallocating frequencies within a fixed range can substantially change how effectively a model uses context. Controlled interventions show that performance gains persist even when total log-frequency displacement is also matched. Analysis of complete sine–cosine pairs characterizes positional overlap; reassigning the same frequencies across coordinates changes task performance, demonstrating the role of learned frequency use. Viewing allocation as channel-wise distance scaling motivates TailSpline, a closed-form transition from native frequencies to the extended low-frequency tail. Derived from a discrete tail-connection problem, the construction uses only public RoPE parameters and retains standard rotary computation without weight updates or calibration. At fourfold extension to 32K, TailSpline improves Llama-3-8B's full thirteen-task RULER mean by over 11 percentage points against both YaRN and MrRoPE-Pro. It achieves the highest mean in every main frozen-extension RULER comparison across five model families up to 70B parameters, alongside improvements in language modeling and natural question answering. Allocation also improves native-window language modeling with frozen weights and strengthens extrapolation during training and adaptation. These findings establish internal frequency allocation as a distinct design dimension of RoPE that shapes how models use context within and beyond the native window.