- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
RRST is a recurrent stock-ranking model with two states: a stock state for historical information and a relation state for learned factor exposures. It combines cross-stock attention, recurrent updates, and an exposure-adjusted scoring layer.
Experiments on NASDAQ, NYSE, and S&P500 report strong ranking results and component ablations. The paper also identifies future-information leakage in public benchmark normalization and evaluates the model with and without explicit price-level features.
Critical strengths
-
Valuable benchmark leakage analysis. The paper identifies future-information leakage from full-series price normalization and measures its impact. It also acknowledges that removing price-level features alone does not guarantee fully causal preprocessing.
-
Coherent architecture and strong results. RRST combines recurrent stock representations, learned relational exposures, and a scoring layer designed for stock ranking. The NASDAQ and NYSE results remain strong without explicit price-level features. The theoretical analysis and ablations provide useful evidence about the model's components.
Critical weaknesses
-
The main results lack a fully causal, matched comparison. Removing level features does not fully rule out leakage from other preprocessing operations. This matters on S&P500, where IC falls from 0.0473 to 0.0065. The baseline results are also taken from previous publications rather than rerun under identical causal preprocessing and training settings. This limits the evidence for RRST's reported advantage.
-
The contributions of individual mechanisms remain unclear. Table 3 shows mixed results. Removing cross-stock attention or leaky recurrence improves NYSE cumulative returns while reducing Sharpe. Removing learned relation exposures reduces returns but raises NYSE Sharpe from 7.02 to 9.74. These results suggest that features, exposure adjustment, and scoring also play important roles. More closely matched architectural comparisons would help isolate the proposed mechanisms.
Questions for authors
Can RRST and the strongest baselines be evaluated using the same fully causal preprocessing and training protocol?
Can the recurrent and relational mechanisms be compared with simpler models using identical features, losses, and scoring?
Can the authors clarify the selection losses, differentiable Top-K objectives, valid-stock masking, and exposure normalization in Equation (5)?
2 comments
Thank you for the detailed response. The training objectives and implementation details are now clearer. I also appreciate the controlled ablations, which provide stronger evidence for the attention and relational components.
However, I will maintain my rating of 2 (Weak rejection) for two reasons. First, the main comparison is still not fully matched. The authors report strong results from the leakage-free RRST variant, with NASDAQ/NYSE Sharpe ratios of 4.65/5.65. However, the baselines still use published results rather than being evaluated under the same corrected preprocessing. This leaves the central performance comparison unresolved. Second, the ablations show useful but mixed contributions. Removing learned relational exposures reduces NASDAQ/NYSE cumulative returns from 0.90/0.92 to 0.67/0.67, but increases NYSE Sharpe from 7.02 to 9.74. The single-step ablation also changes NASDAQ IC by only about 0.002. These results support parts of the design, but not a consistent advantage across markets and metrics. The response addresses several of my concerns, but the remaining evaluation limitations still affect the paper's main claims. I therefore keep my original rating.
We thank the reviewer for recognizing our benchmark leakage analysis, architectural design, and empirical contributions.
1. Causal validity and benchmark fairness.
Tables 1–2 distinguish RRST (as given), which follows the original benchmark's future-dependent normalization for fair comparison, from RRST (no levels), our verified leakage-free evaluation. The latter uses no future information in normalization, price recovery, validity masks, or numerical thresholds. We will correct the outdated manuscript qualifications.
Without leakage, RRST achieves NASDAQ/NYSE Sharpe ratios of 4.65/5.65 and cumulative returns of 1.02/1.24, exceeding the best published values of 2.32/2.35 and 0.82/0.79.
We attempted to reproduce StockMixer using its official repository but could not recover the published results; similar problems are documented in GitHub Issues #12–14. We therefore use published results for comparisons, including graph-based methods.
2. Independent architectural contributions.
Tables 3 and 12 isolate modules while keeping features, losses, and scoring identical. Removing cross-stock attention reduces NASDAQ Sharpe from 4.48 to 3.04 (95% CI for the difference: [0.14, 2.68]). Replacing recurrence with a single-step model reduces NASDAQ IC by approximately 0.002.
Removing learned relational exposures while retaining historical PCA exposures reduces NASDAQ/NYSE cumulative returns from 0.90/0.92 to 0.67/0.67. Although NYSE Sharpe increases from 7.02 to 9.74, this reflects a trade-off between cumulative and risk-adjusted performance, not a uniform improvement. The relation loss also increases explained next-day cross-sectional variation from 4.0%/4.6% to 10.2%/12.8%.
3. Training objectives and implementation.
The first selection head minimizes negative Pearson IC against volatility-adjusted returns; the second uses binary cross-entropy for nonnegative returns. Differentiable top-K objectives use soft portfolio weights:
w=softmax(maskedzscore(s)/0.25)
They optimize negative batch Sharpe and weighted hit rate, with respective weights 0.1 and 0.2. Additional objectives include forecast IC, deep supervision, relation reconstruction, and residual prediction.
Training masks require valid input windows and target-day observations. Equation (5) uses masked, exposure-standardized ridge regression with coefficient 10−6, γ=γR=0.5; relation reconstruction separately uses 10−2.
These clarifications and controlled ablations support RRST's relational modeling contribution, and we agree the value of additional matched external comparisons.