LOBRE: Transferable Limit Order Book Representations for Low-Data Regimes

Limit order book forecasting models are typically trained for individual prediction tasks, making it difficult to rapidly develop models for applications with limited task-specific labels, such as forecasting within infrequent Limit Order Book (LOB) market regimes. We introduce LOBRE, a transformer encoder pretrained directly on irregular Level-3 message streams to learn reusable representations for downstream forecasting with scarce labels. LOBRE combines Masked Message Modeling (MMM) with Aggregate Activity Forecasting (AAF), a prospective objective that trains a pooled sequence representation to capture information predictive of subsequent market activity. We evaluate transfer across heterogeneous forecasting tasks, multiple target-data budgets, and assets both included in and excluded from pretraining. With the encoder frozen, pretraining improves Macro-F1 over the same architecture trained from scratch in 90/96 configurations on pretraining assets and 88/96 configurations on held-out assets, with mean gains of 0.092 and 0.097, respectively. Frozen adaptation also outperforms full fine-tuning on average and exceeds DeepLOB and TABL benchmarks across all evaluated task–budget comparisons. These results support a pretrain-once, adapt-many approach to limit order book forecasting, in which a shared message-level representation can be reused for new tasks and assets with limited target-specific supervision.