acceptodds
Under review as a conference paper at ICLR 2027

AlphaRJM: Reward-Jump Memory for Stochastic Return-Guided Alpha Discovery

Abstract

Formulaic alpha discovery is a pool-dependent symbolic search problem in which informative feedback is observed primarily when a complete expression is evaluated. This delayed feedback creates two coupled difficulties: the retained alpha pool does not preserve the full history of realized evaluation feedback, and the value of an intermediate construction action is uncertain because its consequence depends on the formula eventually completed. We introduce , which addresses these difficulties through , an event-driven latent state that remains fixed during token construction and updates only at terminal evaluation events using the realized pool reward and evaluation outcome, and an that represents future discounted discovery returns with stochastic particles. The particles guide action selection through their mean and uncertainty and are learned using a distributional Bellman objective combining energy-distance matching, mean calibration, and jump regularization. Empirically, AlphaRJM delivers strong and stable gains across multiple equity universes, forecasting horizons, and random seeds, while ablations confirm the complementary roles of persistent evaluation history, stochastic return modeling, and distributional supervision.

Primary area:
reinforcement learning
Keywords:
Alpha discovery, Reward-jump memory, Neural SDE, CRPS, Distributional reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CJGxOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose AlphaRJM to search for symbolic trading signals that improve an existing signal ensemble, combining persistent evaluation memory with an SDE return-distribution critic for subsequent construction decisions. The paper reports competitive mean correlation metrics on Chinese equity universes, alongside comparisons across forecasting horizons and component ablations.

Critical strengths
  • The paper is reasonably motivated, as retaining evaluation feedback across formulas can capture search history that the current signal pool does not represent.
  • The authors evaluate the method across several Chinese equity universes and examine forecasting horizons and component contributions through additional comparisons and ablations.
Critical weaknesses
  • The authors provide insufficient interpretation of the experimental results, particularly why increasing the number of SDE particles reduces performance in Figure 2.
  • Several gains over the strongest baselines in Table 1 are modest, particularly on CSI300, leaving me unconvinced that they justify the added methodological complexity.
Questions for authors
  • The authors could streamline repetitive explanations and move the framework overview in Figure 3 into the main text to clarify how the components work together.
  • The authors could clarify whether the SDE and custom memory offer advantages over direct Gaussian return modeling and standard recurrent models, respectively.
Reviewer_J7WdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AlphaRJM for formulaic alpha discovery, combining persistent reward–outcome memory with an SDE-based return critic. Experiments across Chinese equity universes show competitive performance, with multi-seed main results, ablations, and sensitivity analyses.

Critical strengths
  1. The event-driven memory provides a task-motivated way to retain evaluation history beyond the current alpha pool, allowing previous feedback to inform subsequent search.
  2. The evaluation covers multiple equity universes and includes multi-seed main results, component ablations, and sensitivity analyses, providing useful evidence of the framework’s effectiveness.
Critical weaknesses

The contribution of the specific designs remains unclear without comparisons against a standard recurrent memory and a directly parameterized Gaussian critic, particularly since Appendix F shows that the SDE critic produces conditional Gaussian returns.

Questions for authors

Could you clarify the benefits of the proposed memory update over a standard GRU, and of the SDE parameterization over a directly parameterized Gaussian critic? Are these benefits supported by controlled comparisons?

Reviewer_DNwZOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AlphaRJM, a reinforcement learning-based framework for automated formulaic alpha discovery. The method introduces Reward-Jump Memory to preserve historical information across search episodes by incorporating previous rewards and evaluation outcomes. It also employs a stochastic differential equation (SDE)-based return critic to model the distribution of future returns and guide exploration using expected returns and uncertainty. The model is trained through distributional Bellman learning. Experiments across multiple equity datasets demonstrate improved predictive performance over existing methods.

Critical strengths
  1. The paper addresses practically relevant challenges in symbolic alpha discovery, particularly the loss of historical feedback and uncertainty in intermediate action returns.
  2. Reward-Jump Memory provides a simple and reasonable mechanism for preserving evaluation feedback across search episodes, potentially helping address the non-stationarity of alpha discovery.
  3. The experiments cover multiple equity datasets and demonstrate competitive predictive performance against methods such as AlphaGen, AlphaQCM, and AlphaSAGE. The ablation studies also provide preliminary evidence supporting the effectiveness of the proposed components.
Critical weaknesses
  1. The adopted SDE yields a Gaussian distribution with analytically tractable mean and variance. Its advantages over a simpler Gaussian critic remain insufficiently justified, especially given that AlphaQCM already models return uncertainty.
  2. The paper introduces both explicit alpha-pool encoding and persistent memory, but does not sufficiently disentangle their individual contributions or analyze how historical feedback improves the search process.
  3. Performance gains show noticeable variance across seeds, without statistical significance analysis. Training budgets are not directly matched across all baselines, and representative discovered alpha expressions are not provided.
Questions for authors

Please refer to the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.