- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes UQ-LOB, an uncertainty-aware framework for short-horizon mid-price forecasting using limit order book (LOB) data, addressing the lack of explicit uncertainty quantification in conventional point prediction methods. The framework introduces an encoder-agnostic uncertainty quantification head inspired by attentive neural processes, which incorporates recently completed historical windows with observed outcomes through self-attention and cross-attention mechanisms. Two variants are developed: UQ-regression, which predicts a Gaussian distribution over future price displacements using a combination of heteroscedastic likelihood, magnitude-weighted error, directional loss, and calibration loss, and UQ-classification, which predicts probabilities for upward, downward, and stationary movements. Experiments on approximately 5.2 billion LOB events across seven cryptocurrency assets and forecasting horizons of 5, 10, and 15 seconds demonstrate that UQ-regression achieves near-nominal 68% prediction interval coverage. Retaining the top 10% most confident predictions improves directional macro F1 by 0.11–0.15 for UQ-regression and 0.05–0.11 for UQ-classification. For large price movements at the 5-second horizon, the highest-confidence subset achieves F1 scores of 0.88 and 0.83 for downward and upward movements, respectively, demonstrating the potential of uncertainty-aware selective prediction to improve forecasting reliability.
Critical strengths
1.The paper addresses the important challenge of quantifying prediction reliability in limit order book forecasting, shifting the focus beyond predictive accuracy toward assessing the trustworthiness of individual predictions. This problem is practically relevant and provides a valuable direction for selective prediction and trading decision-making in high-frequency financial markets. 2.The paper proposes a flexible probabilistic forecasting framework that integrates historical outcomes through in-context conditioning with uncertainty quantification. The carefully designed composite loss function jointly considers prediction magnitude, direction, and calibration, enabling the model to provide both price displacement estimates and confidence information to support decision-making.
Critical weaknesses
1.The paper lacks direct experimental comparisons with existing methods from the literature. Although several LOB forecasting and uncertainty quantification approaches are discussed in the related work, none are included as competitive baselines. The evaluation primarily compares two variants of the proposed framework, making it difficult to assess its relative performance and incremental contributions over existing approaches. 2.The paper lacks systematic ablation studies to validate the effectiveness of its key components. In particular, the individual contributions of in-context conditioning, attention mechanisms, and the proposed loss functions are not evaluated. Without these experiments, it remains unclear which components are responsible for the reported improvements, weakening the empirical support for the proposed methodological innovations.
Questions for authors
1.Could the authors provide direct comparisons with existing LOB forecasting and uncertainty quantification methods, such as MC Dropout, Deep Ensembles, and the original D-TABL encoder with confidence-based selection? Evaluating these methods under the same experimental settings and prediction coverage rates would help clarify the relative advantages of UQ-LOB. 2.Could the authors provide ablation studies to quantify the contributions of the proposed components, particularly in-context conditioning, self-attention, cross-attention, and the individual loss functions? These results are important for determining whether the reported improvements can be attributed to the proposed architectural and training innovations.
3 comments
I thank the authors for the additional results and the detailed rebuttal. I will maintain my original score.
Follow-up on baselines and ablations
Thank you for reading our response and the additional results.
Following your suggestions, the revised paper now includes:
1. Comparisons under the same settings (Sec. 5.3). MC Dropout, Deep Ensembles, conformal prediction and the original D-TABL encoder with confidence-based selection were evaluated with the same pretrained encoder, test examples and retained fractions. UQ-Regression achieves the highest directional F1 at every confidence threshold, together with the best NLPD, using a single model and a single forward pass. For classification, the D-TABL softmax confidence barely improves F1 as coverage decreases (), whereas UQ-Classification rises from to .
2. Ablations (Sec. 5.7). Removing in-context conditioning reduces top-10% F1 (), large-move F1 () and calibration (). Replacing the attention blocks with mean aggregation lowers top-10% and large-move F1. Leave-one-out loss ablations (Sec. 3.4, Appendix D "Ablation studies") show that the WMAE and directional terms each contribute to accuracy on confident and large-move predictions.
We hope these results clarify the relative advantages of UQ-LOB and the contribution of its components. We remain fully open to further discussion: if any point is unclear, or if there are additional comparisons or experiments that would help in assessing the work, we would be glad to provide them within the remaining discussion period and to include them in the final version. Thank you again for your time and constructive feedback.
Response on baselines under the same settings and component ablations
Thank you for the review. The comparisons and ablations you requested are in the revised paper; the key results are below.
1. Baselines under the same settings (Sec. 5.3, Appendix "Baseline comparison"). All methods share the D-TABL encoder initialised from the same pretrained weights, the same test examples and the same retained fractions (confidence thresholds 10–90%). MC Dropout (30 passes), a 5-member Deep Ensemble and conformal prediction are trained with the Gaussian likelihood alone and without a context head. Mean directional F1 at 10 s, averaged over the seven assets:
Regression. UQ-Regression has the highest mean F1 at every confidence threshold, and its lead over the strongest baseline, the Deep Ensemble, widens from 0.01 on the top 90% of predictions to 0.03 on the top 10%; against MC Dropout and Conformal the lead is 0.03–0.04 and 0.04–0.07. It also has the best predictive distribution: the lowest calibration error (0.048 against 0.057–0.066), the best NLPD (5.59 against 5.73 and 5.79) and a 95% coverage (0.904) closer to nominal than MC Dropout and the Deep Ensemble (0.886–0.888). Conformal prediction reaches 95% by construction (0.947) but gives no density. Because all baselines share the pretrained encoder, the gap is not an effect of pretraining, and UQ-Regression needs one trained model and one forward pass, against 30 passes for MC Dropout and five trained models for the Ensemble.
Classification. The D-TABL encoder with confidence-based selection is the softmax baseline: the pretrained D-TABL classifier, trained with the same class-weighted cross-entropy as UQ-Classification and gated by its maximum class probability. Its mean F1 moves only from 0.42 to 0.43 between the top 90% and the top 10% of predictions, whereas UQ-Classification rises from 0.45 to 0.50, mainly through down moves (0.47 to 0.55). The maximum softmax probability of the base classifier therefore says little about which directional calls are reliable, while the confidence learned by the context head does.
2. Ablations (Sec. 5.7, Appendix "Ablation studies"). Each variant is retrained from the same pretrained encoder with the same objective. Seven-asset means at 10 s:
Removing the context lowers both the calibration and the direction of large moves. Masking the context labels shows that the window representations carry directional information while the labels supply the scale of the uncertainty. Replacing attention by a mean leaves the uncertainty estimate nearly unchanged and lowers the top-10% and large-move F1 by 0.011 and 0.021. We replace the self- and cross-attention blocks jointly by a mean over the context embeddings, which is the aggregation of the original CNP (Garnelo et al., 2018).
3. Loss functions (leave-one-out, BTC-USD, 10 s; Sec. 3.4, Appendix "Ablation studies"). Without the WMAE term, the weighted R² falls from 0.054 to 0.038 and the top-10% F1 from 0.677 to 0.643. Without the directional term, the top-10% accuracy falls from 80.7% to 77.4% and the large-move accuracy from 87.4% to 84.8%. Removing the calibration-slope term leaves these aggregate metrics unchanged.
All new content in the revised paper is highlighted in green. In response to your questions, the revision adds comparisons with MC dropout, deep ensembles and the D-TABL encoder with softmax-based selection under the same settings and retained fractions, and ablations of the context, the attention blocks and the individual loss terms.