acceptodds
Under review as a conference paper at ICLR 2027

Truncate to Accelerate: From Forecast Recoverability to Efficient Time-Series Foundation Models

Abstract

In large language models, intermediate representations have been used both to study how task-relevant information evolves with depth and to accelerate inference. In time-series foundation models (TSFMs), however, these questions remain largely unexplored. We address this gap through layerwise probing, representation geometry, and lightweight adaptation across three state-of-the-art TSFMs: Chronos-2, TimesFM-3, and TiRex, evaluated on a collection of datasets with diverse sampling frequencies. First, layerwise forecasting readouts reveal a forecasting tunnel: across nearly all model-dataset pairs, intermediate depths already recover performance within 5% of the final-block readout. Next, we find that functional saturation coexists with continued geometric evolution. Finally, intermediate states can contain recoverable forecast information, but feeding them directly into the model's native forecasting pathway degrades performance relative to the full model. Lightweight affine alignment substantially reduces this gap and exposes a measured performance-compute frontier: aligned Chronos-2 remains within a median 3.1% of the full model after retaining only 4 of its 12 blocks, while running faster, with the achievable tradeoff varying substantially across architectures. Moreover, cuts inside the forecasting tunnel are typically better tolerated at matched depth, linking early recoverability to practical truncation. Together, these results suggest that much of the forecast-relevant information becomes recoverable before the final block and offer a path toward both understanding and accelerating time-series foundation models.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.