FinVerse: Toward More Realistic Evaluation for Financial Time-Series Forecasting

Time-series foundation models (TSFMs) are increasingly evaluated on broad benchmarks that apply uniform conventional forecast evaluation metrics across heterogeneous time-series. However, in the financial domain, decision-relevant evaluation requires going beyond conventional metrics to assess forecasts in terms of directional accuracy, cross-sectional ranking, and portfolio outcomes. To this end, we introduce FinVerse, a financial time-series benchmark, collecting a broad collection of 116,897 series with 171.1M observations. FinVerse organizes financial data according to their economic semantics and evaluates target series across point-wise forecasting, cross-sectional ranking, and portfolio backtesting. Across 43 public TSFMs, model rankings on FinVerse show only moderate agreement with those on the existing benchmark, GIFT-Eval (r=0.43), suggesting that generic forecasting benchmarks do not fully capture the aspects of forecast quality that matter for financial decision-making. Furthermore, substantial variation in model performance across financial time-series categories and evaluation metrics provides a more informative basis for model selection, enabling practitioners to identify suitable models according to their target series and downstream objectives.