acceptodds
Under review as a conference paper at ICLR 2027

AlphaG-OPD: Reliability-Gated Sibling Counterfactuals for On-Policy Distillation in Symbolic Alpha Factor Discovery

Abstract

ymbolic alpha factor discovery can score a completed expression, but it provides no direct label for the structural decisions that produced it. Generative flow networks (GFlowNets) preserve a diverse, reward-proportional distribution over complete expressions, yet their trajectory-level objective does not compare unchosen sibling actions at an intermediate state. We introduce AlphaG-OPD, a structural on-policy distillation framework that turns terminal factor evaluations into local action guidance. Its design separates three decisions. Component I determines where to teach by exposing grammar-valid siblings at partial abstract-syntax-tree (AST) states visited by the current forward policy. Component II determines what is reliable enough to teach: it evaluates three supported siblings under four shared suffixes and admits a KL-bounded target only when their matched comparisons exhibit sufficient winner agreement and a positive empirical lower confidence bound (LCB). Component III determines how strongly and for how long to teach by consolidating accepted targets through bounded replay, score-indexed expiry, and forward-gradient balancing, without additional factor evaluations. Terminal reward, Trajectory Balance, the backward policy, grammar, and factor-pool rules remain unchanged. An equal-physical-score four-arm ablation tests paired teaching, reliability gating, and consolidation. Across China’s CSI300, CSI500, and CSI1000 and the U.S. S&P 500, the complete method delivers strong cross-market performance over multiple random seeds.

Primary area:
reinforcement learning
Keywords:
On-Policy Distillation, Symbolic Alpha Factor Discovery

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_fSrAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper suggests a novel method for symbolic alpha factor discovery that uses on-policy distillation. Since there is no teacher model, the method replaces token logits with structural counterfactuals. The advantages of the approach are that learned samples are on-policy, meaning that the trajectories are reachable. Also, the credit assignment is local, in addition to assigning a single reward for the whole sequence. It tries multiple alternative steps and completes each one in the same way, so the score differences are attributed to the local step. Experiments on Chinese and U.S. stock markets show strong performance compared to baselines on some markets.

Critical strengths
  1. The method is well motivated. Since a formula gets scored only once it's complete, it's hard to attribute which term really helped. By swapping one step and comparing the scores, it is possible to train the GFlowNet using on-policy samples with different reward signals for different steps, rather than only a single reward for all the terms.
  2. Requires no external teacher, samples are generated from current policy.
  3. Experiments on Chinese and U.S. stock markets show strong performance compared to baselines on some markets.
Critical weaknesses
  1. The paper is hard to read, with heavy jargon and broken references (lines 367 and 455). Table 1 text size is very small, and the figures don't look professional.
  2. The reproducibility statement mentions the appendix, but it is missing from the pdf.
Questions for authors

Could you please include the appendix of the paper?

Reviewer_fxd9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper introduces AlphaG-OPD, a method designed to address a key challenge in symbolic alpha factor mining: the difficulty of using terminal evaluation signals to directly guide the model in making local structural choices. Based on partial expression states sampled from the current GFlowNet policy, the method selects three syntactically compatible "sibling" actions and evaluates them against a shared set of completion suffixes to construct a local "teacher" signal. To refine these teacher targets, the authors employ criteria such as winner consistency, a lower bound on pairwise advantage, and KL contraction. Furthermore, the approach incorporates bounded experience replay, score-indexed target expiry based on cumulative terminal evaluations, and a gradient-balancing mechanism to regulate the duration and intensity of this auxiliary supervision. This auxiliary signal is applied to the forward policy, while the original terminal rewards and the Entropy-TB training objective are retained. The method is evaluated on the CSI300, CSI500, CSI1000, and S&P 500 datasets; an ablation study is also conducted on the CSI500 dataset, constrained by a fixed budget of 10,000 terminal evaluations. The main results are reported as the mean and standard deviation across three random seeds. The full model demonstrates consistent performance gains on the CSI300 and CSI500 datasets, while results on the CSI1000 and S&P 500 datasets are mixed.

Critical strengths
  • The research problem is of practical significance, and the comparative design based on "shared completions" is highly targeted. Since the final performance score of a complete expression cannot directly reveal the quality of an intermediate structural action, this paper employs identical completions for syntactically compatible "sibling" actions. This approach eliminates interference from subsequent construction steps, providing a viable signal for local supervision—a key innovation of this work.
  • The experiments account for the evaluation overhead incurred by constructing local teacher targets, demonstrating performance gains in certain markets. The authors incorporate all exploratory samples into the final evaluation budget, factoring in the additional costs associated with generating local teacher signals. As shown in Table 1, the average of seven performance metrics improved for both the CSI 300 and CSI 500 indices. For the CSI 500, specifically, the annualized return increased from −0.778% to 2.968%, while the maximum drawdown decreased from 25.040% to 18.535%. These results provide preliminary validation of the method's practical utility.
Critical weaknesses
  • The presentation of charts and results requires optimization, as it hinders readability. In Figure 1(b), the label "a3a_3" extends beyond the orange box, and some labels and connecting lines are crowded and overlapping. Figure 2 attempts to pack too many module descriptions and ablation settings into a confined space, making it difficult to interpret. Table 1 displays results across multiple markets side-by-side; given the limited space, it includes numerous metrics and standard deviations, resulting in an excessively small font size. These charts, which convey core evidence, require reformatting and careful proofreading.
  • Experimental analysis is insufficient, and some conclusions lack verifiable support. On page 7, there is a placeholder "Algorithm ??," and on page 9, the text references a non-existent "Figure ??," which supposedly illustrates directional transfer and year-over-year out-of-sample gains. Section 5 focuses exclusively on performance improvements, failing to adequately discuss instances where metrics decline or results diverge. For instance, the CSI1000 index shows a decrease in both annualized return and the Sharpe ratio, yet the text highlights only the improvement in the Information Ratio. Furthermore, Table 2 indicates a decline in the IC metric when the gating mechanism is enabled, but no uncertainty analysis is provided for these ablation results. The paper also fails to verify whether accepted local teacher targets retain a positive paired advantage on newly sampled completion suffixes, leaving the core claim regarding the "reliability gating" mechanism insufficiently supported.
  • Reproduction materials are incomplete, with key technical details missing. Although the reproducibility statement on page 10 mentions that implementation details are provided in an appendix, the current PDF contains only the references after this statement, with no appendix included. While the main text provides some formulas and hyperparameters, it omits details on how candidate states and alternative actions are selected, the rules for "probing" schedules, and whether the expressions generated during probing are used in TB updates or added to the factor pool. Additionally, while the paper outlines time-period partitioning and factor-weighting rules, it lacks clear information regarding portfolio construction, rebalancing frequency, and transaction costs—making it difficult for others to independently verify the algorithm and its reported returns.
  • The justification for the method's innovation and its advantages over baselines is inadequate. While the proposed approach—integrating grammatical constraints, paired evaluation, empirical gating, target replay, and gradient balancing—is supported by a sound engineering rationale, the paper fails to clearly articulate the core methodological advancements resulting from this combination. The discussion of related work merely provides a brief overview of existing methods, with insufficient detail regarding baseline configurations and evaluation conditions. Except for the AlphaSAGE comparison, the results for all other baselines in Table 1 are taken directly from the literature. There is an inconsistency in the description of the baselines between Figure 2 and Section 3.6; the two sets of comparisons are not equivalent. Furthermore, the current ablation studies fail to disentangle the search gains—derived from the probing process—from the learning gains contributed by the local teacher.
Questions for authors
  • Missing experimental evidence. Please provide the complete content corresponding to "Algorithm ??" and "Figure ??" in the main text. Specifically, what are the detailed results supporting the claims regarding directional transfer and year-over-year out-of-sample performance improvements? Additionally, please explain the decline in core return metrics for the CSI 1000 index and discuss how this result constrains the conclusions drawn regarding cross-market validity.
  • Verification of the core mechanism. At the same states and sibling sets, do accepted teacher targets retain positive paired advantages on newly sampled shared completion suffixes? How do they compare with rejected candidate targets? Please also provide information on the random seeds used for the results in Table 2, along with an analysis of result variance; this will help determine whether the improvements driven by the gating mechanism are stable and confirm whether the reliability-based filtering effectively reduces erroneous supervisory signals.
  • Control protocols and methodological increments. Do expressions generated during probing contribute to TB updates or enter the final factor pool? What happens when the OPD targets are discarded while the probing and candidate-use protocols remain unchanged?
  • Implementation and reproducibility details. Please include the supplementary material mentioned in the reproducibility statement, providing a clear, end-to-end description of the probe selection/scheduling, factor pool updates, and portfolio evaluation processes. Additionally, while both Table 1 and Table 2 reference the CSI 500 index, a 10K budget, and an "Equal-50" evaluation protocol, the results for the full method differ between them; please explain the reasons for this discrepancy and clarify whether the random seeds, checkpoints, or configurations were consistent across both sets of experiments.
Reviewer_gLKEOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The manuscript proposes a novel on-policy distillation method to predict a 20-day return of different stock markets via symbolic alpha factor discovery. The work is based on AlphaSAGE, a previous GFlowNets-based alpha factor predictor, that uses a relational graph convolutional network. The authors claim improved cross-market performance in both China and the USA stock markets. The authors highlight three main contributions, all related to the on-policy distillation protocol and they show a small ablation study on how each component affects the final performance across seven common metrics used in stock-return predictions. The authors extensively describe their methodology and provide two figures to explain their approach. Furthermore, the authors provide two tables to summarize the quantitative findings or their experiments.

Critical strengths

None

Critical weaknesses

Weakness 1: The prediction results appear to be incorrect and the IC numbers do not match the original AlphaSAGE results. First, the author's model only outperforms the baseline AlphaSAGE on CSI300 and CSI500, while it severely underperforms on CSI1000 and S&P500. Table 1 shows 9 incorrectly bolded text (that are supposed be the best one), but are actually not the best one in their columns (only compared to GFlowNet, which is misleading). Furthermore, the AlphaSAGE's CSI500 results are different from their original publication (https://arxiv.org/abs/2509.25055) than reported in Table 1. For example, AlphaSAGE IC's for CSI500 should be around 5.2% compared to this manuscript's number of 1.97%. This discrepancy would change the outcome, because AlphaSAGE would beat AlphaG-OPD. Other discrepancies are also noted by this Reviewer. Lastly, the ablation study does not reproduce neither the baseline, nor the final (AlphaG-OPD I-II) results. This Reviewer is concerned about these critical issues, which questions the validity and reliability of the results.

Weakness 2: The overall presentation of the manuscript is very poor in both language and figure quality. First, the Figure 1 and Figure 2 shows different sizes of text, which makes it difficult to read (especially Figure 2). Both Figures also appear to have several alignment errors and missing arrows. Furthermore, the Figures have at least 4 equations that do not explicitly appear in the text. More importantly, the main Equation 1 is different in the Figure than in the text. Both the Methods and Results sections contain missing renderings in the PDF version (Algorithm ?? and Figure ??). Overall, the text is extremely difficult to follow due to the word-salad used in the text. This Reviewer's main issue that complicated concepts are not adequately and especially not gradually defined during the text. Too many objects are introduced at once, the framings are repeatedly restated; or some phrases remain completely undefined. While the equations of the methods are adequately presented, the main concepts behind the methodology is overly complicated and took several re-readings of this Reviewer to finally understood the connections between each component. While this Reviewer is not directly working in finance, they had no issues of understanding related papers, such as AlphaSAGE or RL-based alpha generation papers (https://arxiv.org/abs/2306.12964) among others. Overall, this Reviewer is concerned that the inability of the Authors to clearly express the introduction, methods and results weakens the scientific quality of the manuscript and its claims. The presented manuscript is difficult to understand and evaluate overall.

These two collections of weaknesses are of major concern; and this manuscript needs substantial reformulation and revision to be considered for acceptance.

Questions for authors
  1. Can you add the missing Algorithm?? and Figure?? to the manuscript? Without those, the manuscript cannot be further assessed.
  2. Can you update the Table 1 and Table 2 with the correct results from AlphaSAGE and with correct bolding?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.