Beyond Context Concatenation: Scalable, Manifold-Aligned Retrieval for Time-Series Foundation Models

Retrieval augmentation equips frozen time-series foundation models with external memory, enabling them to handle distribution shifts without fine-tuning. However, existing methods face three critical bottlenecks: in-context token concatenation incurs quadratic attention complexity in sequence length and retrieval budget, causing a 46.70x FLOP increase. Simply adding covariates does not significantly improve performance because the internal representations do not align. At the same time, external memory stores cause metrics to shrink along high-variance latent axes, and numerical retrieval methods cannot predict macroeconomic shocks. In this work, we resolve these limitations through a scalable, manifold-aligned retrieval architecture. Our framework eliminates quadratic context expansion by routing candidates via an injection hierarchy into latent cross-attention or output-space Wasserstein barycenters, scaling linearly with the retrieval budget. We align future planning covariates with the representation manifold via a pre-trained projection adapter under parameter-freezing guardrails, and integrate dense text embeddings to guide retrieval and modulate fusion gating during macroeconomic shocks. Empirical benchmarks show latent injection achieves representation parity with token concatenation across Chronos-2, Moirai-2.0, and TimesFM-2.5 backbones while preventing out-of-memory collapse at production batch sizes. On the 30 planning tasks of fev-bench, aligned covariate retrieval yields a 52.09% CRPS gain over unaligned concatenation. Also, memory scaling across $10^4$ to $10^8$ sequences confirms power-law error decay ($0.0785\pm 0.0033$) in agreement with theoretical manifold bounds, and dual multimodal grounding reduces macroeconomic shock error by 7.72% MSE. Structured latent injection and manifold alignment thus provide an efficient, scalable foundation for zero-shot time-series forecasting.