acceptodds
Under review as a conference paper at ICLR 2027

EvoMind: A Self-Evolving Alpha Factor Mining Agent via Process-Reward Reinforcement Learning

17 views

Abstract

Alpha factor discovery aims to identify predictive, robust, and interpretable signals from a vast expression space. While large language models (LLMs) provide a promising alternative to conventional search-based methods, existing approaches mainly rely on one-shot generation or prompt-based refinement, limiting their ability to continuously improve from noisy market feedback. We propose EvoMind, a self-evolving alpha factor mining agent via process-reward reinforcement learning. EvoMind formulates factor discovery as an iterative agentic loop of hypothesis generation, factor implementation, tool-based evaluation, feedback reasoning, and factor revision. Rather than optimizing only final backtest performance, we introduce hierarchical process rewards that provide dense supervision throughout the mining trajectory, jointly encouraging executable, diverse, predictive, and out-of-sample stable factors. Furthermore, EvoMind learns from both successful and failed trajectories by diagnosing previous exploration outcomes and reusing the resulting experience to guide subsequent factor evolution. This enables the agent to progressively improve its factor-mining policy instead of repeatedly generating independent candidates. Experiments across multiple markets and out-of-sample settings demonstrate that EvoMind consistently discovers more robust, diverse, and sustainable alpha factors than strong search-based and LLM-based baselines. Our results highlight process-reward reinforcement learning and experience-driven self-evolution as an effective paradigm for autonomous alpha discovery.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Reinforcement Learning, Process Reward Modeling, Alpha Factor Mining

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TQznOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to train an LLM agent for alpha factor discovery. It formulates factor mining as a multi-turn decision process with an explicit information boundary: the agent sees metrics only on a search window, a later verification window is hidden and enters only the reward, and the test period is evaluated once. The method has three components. First, gated hierarchical process rewards score each turn on executability, predictive power, novelty, and stability on the hidden window. Second, turn-level advantages credit each revision only for the rewards that follow it. Third, an experience bank stores both successful and failed attempts, with failure types derived from the reward hierarchy, and retrieves them to guide later exploration. On CSI 300, CSI 500, and S&P 500, EvoMind outperforms the strongest baseline by about 10% in test RankIC and also improves annualized excess return, with all baselines re-implemented on the same Qwen3-4B backbone.

Critical strengths
  1. The paper builds a complete infrastructure for training an alpha-mining agent, covering the agent loop, the information boundary between the search, verification, and test windows, and turn-level reward assignment. To my understanding, as I'm not an expert in this area, there is limited prior work on this, so the infrastructure is valuable to the community.
  2. Rather than relying only on prompting and pipeline design with a frozen LLM, the paper trains the policy itself with RL, which is a promising direction. The ablations show that the gains come from the process rewards and turn-level credit assignment, not from RL alone.
Critical weaknesses
  1. All experiments use a single small backbone, Qwen3-4B. I understand the time and compute constraints. Still, with results from only one model, it is hard to tell whether the carefully designed infrastructure and reward assignment are generally effective methods or just a configuration that happens to work in this setting. Several of the baselines were originally run with much larger or frontier models: AlphaAgent with GPT-3.5-turbo, QuantaAlpha with DeepSeek-V3.2 and GPT-5.2, and AutoScientist-Quant with GPT-OSS-120B and GLM-5. Here they are all re-implemented on the frozen 4B backbone, which may handicap them. If training larger models is not feasible, it would be very helpful to show that, at least in some setting, the trained 4B agent can match these methods with their original backbones, or match directly calling a frontier model API.
  2. As noted in the strengths, the main contribution is a specific pipeline and its training, while the underlying learning machinery is largely known: GRPO-style objectives, return-to-go advantages, and process rewards. I would like the authors to argue the contribution more clearly. What makes this domain specifically hard for agent training, and how does each design choice address that difficulty? Or is the work mainly one practical implementation that works well?
Questions for authors

Please see the Weaknesses section.

5 comments
AuthorsOpenReview ↗

Response to W1-1 (Part 1/3): Original Backbones and a Frontier API vs. the Trained 4B Agent

Thank you for your careful review and for recognizing both our training infrastructure and the direction of training the policy itself. We had fixed a 4B backbone to isolate algorithmic effects from model scale; to address your concern directly, we evaluated the baselines with their original backbones as well as a frontier API, GPT-6.1 Sol (released September 2026), on CSI 300 (five runs, mean test RankIC; API pipeline SDs: 0.0014–0.0016).

1. Original backbones & frontier API comparison (CSI 300).

Each pipeline keeps its native prompts and loop while sharing Section 5.1's data windows, 640-evaluation mining budget, pool-admission rule, and downstream evaluation, swapping only the LLM (no warm start or RL; because OpenAI retired GPT-3.5-turbo in October 2026, AlphaAgent uses stronger GPT-5.2). For EvoMind, Table 2 reports the full system (trained weights + RL-accumulated experience bank); we also report policy-only (empty initial bank, matching every other row):

Pipeline and backboneTest RankICΔ\Delta vs. frozen Qwen3-4B (Table 2)
AlphaAgent + GPT-5.2 (substitute)0.0446+0.0029
QuantaAlpha + DeepSeek-V3.20.0481+0.0025
QuantaAlpha + GPT-5.20.0501+0.0045
AutoScientist-Quant + GPT-OSS-120B0.0497+0.0018
AutoScientist-Quant + GLM-5 (best orig.)0.0519+0.0040
Our loop (no RL) + GPT-6.1 Sol (frontier)0.0505+0.0044
AutoScientist-Quant + GPT-6.1 Sol (frontier)0.0534+0.0055
EvoMind, trained Qwen3-4B (full, Table 2)0.0528 ± 0.0012+0.0067
EvoMind, trained Qwen3-4B (policy-only)0.0517 ± 0.0013+0.0056

(Δ\Delta in the last three rows is relative to Frozen-EvoMind, 0.0461.)

2. Key head-to-head comparisons & takeaways.

Comparison targetEvoMind (full) − TargetEvoMind (policy-only) − TargetTakeaway
Best original backbone (AS-Q + GLM-5, 0.0519)+0.0009 (t=1.0t = 1.0)−0.0002 (t=0.2t = 0.2)Matches best original backbone; beats the other four
Frontier API in our loop (GPT-6.1 Sol, 0.0505)+0.0023 (t=2.6t = 2.6)+0.0012 (t=1.3t = 1.3)At least on par with frontier API, while running locally
Frontier API in best baseline (AS-Q + GPT-6.1 Sol, 0.0534)−0.0006 (t=0.7t = 0.7)−0.0017 (t=1.8t = 1.8)Slightly lower within noise; no claim to beat frontier-API AS-Q

Larger backbones do raise the baselines (+0.0018 to +0.0055), so we will clarify that Table 2 is a controlled same-backbone comparison; note also that GPT-6.1 Sol's April 2026 cutoff postdates our test window, which can only favor it. We will add this comparison and the policy-only clarification to Sections 5.1–5.2 and new Appendix L.

AuthorsOpenReview ↗

Response to W1-2 (Part 2/3): A Second Backbone and Other Rewards, Windows, and Markets

We agree that results on one model alone cannot establish generality. Below we summarize experiments varying four dimensions: backbone family, reward specification, verification windows, and target market (the first three retrain under the full-system protocol, Part 1; the fourth transfers weights with no retraining).

1. Second backbone family: Llama-2-7B-Chat.

We repeated training and reran AutoScientist-Quant and Frozen-EvoMind on an older, weaker model from another family under identical rewards, budgets, windows, and downstream evaluation (shortening response length for its 4k context; five runs each on CSI 300 / S&P 500):

Test RankIC differenceQwen3-4B (paper)Llama-2-7B-Chat
Gain from training (EvoMind vs. Frozen-EvoMind)+0.0067 / +0.0058+0.0050 / +0.0034 (t=4.6/3.5t = 4.6 / 3.5)
Lead over AutoScientist-Quant+0.0049 / +0.0036+0.0038 / +0.0019 (t=3.6/2.1t = 3.6 / 2.1, borderline on S&P 500)

Absolute RankIC drops by 20%–23% for all three methods on Llama-2 and margins narrow (especially on S&P 500), while both differences stay positive and method ordering is unchanged.

2. Summary across reward settings, verification windows, and unseen markets.

Dimension variedPerturbation testedEvoMind Test RankIC / MarginReference / Baseline
Reward settings (CSI 300, 9 settings × 3 runs)Shift 3 thresholds down/up, halve/increase λ4\lambda_4, swap L3 & L40.0508–0.0533 (all > baseline; min lead +0.0029, t=2.8t = 2.8)AutoScientist-Quant: 0.0479 (default EvoMind: 0.0528)
Verification windows (3 markets × 5 runs)Rotate training windows & exclude 2022 altogetherLead +0.0037 / +0.0042 / +0.0027 (t=4.3/4.3/3.4t = 4.3 / 4.3 / 3.4)Paper fixed-2022 lead: +0.0049 / +0.0052 / +0.0036
Target market (policy-only, 5 runs, no retraining)Transfer S&P 500 ↔\leftrightarrow CSI 300 policies (empty pool & bank)0.0497 (CSI 300), 0.0343 (S&P 500); t=3.9/4.0t = 3.9 / 4.0 vs. FrozenTarget Frozen-EvoMind: 0.0461, 0.0309 (64% / 68% of in-market gain)

(With three runs per reward setting, we treat row 1 as a sensitivity check over the tested range; in row 3, source-market weights achieve 64% / 68% of the gain from training on the target market.)

All defaults were fixed before the first test evaluation (Section 5.1) and are not retuned. We will note in Section 6 that RL has been tested only on two smaller open-weight models, and report full results in new Appendices F, G, H, and K.

AuthorsOpenReview ↗

Response to W2 (Part 3/3): What Is the Contribution?

You are right that our GRPO-style objective, return-to-go credit, and multi-turn loop build intentionally on prior work (Section 2). Our contribution addresses a distinct problem: what signal should train an agent when tool feedback is noisy and overfittable? (CSI 300 numbers below are from Tables 2–3 unless marked new.)

1. Same backbone, loop, and GRPO objective—only the training signal changes.

Unlike math or coding with reliable verifiers, backtest metrics are noisy empirical estimates that RL readily overfits, and a growing factor pool makes utility non-stationary. Holding the backbone, loop, and optimizer fixed shows this is a training-signal question rather than a mere engineering implementation:

Training signal variant (same backbone, loop, & GRPO)CSI 300 Test RankICNote
Frozen-EvoMind (no RL)0.0461Untrained reference
Standard Outcome-GRPO0.0454Trails untrained agent on all 3 markets (Table 2)
+ Gated process rewards (trajectory-level credit)0.0490Adds verifiable intermediate rewards
EvoMind: + Turn-level credit & bank0.0528 (0.0517 policy-only)Full framework (Part 1)

2. One tool-verifiable hierarchy driving reward, credit, and memory.

EvoMind scores each candidate through four tool-computed gated levels (Table 1: executability, DsrchD_{srch} predictive power, pool novelty, and hidden-DverD_{ver} stability), using this single hierarchy to address four domain failure modes:

Domain difficultyDesign role of the hierarchyIf removed or altered (full: 0.0528)
Visible metrics can be overfitReward: heaviest weight placed on a later window hidden from the policyNo cross-period signal in training: 0.0482 (new; Table 3's 0.0468 also omitted admission screening). Signal kept, DverD_{ver} shown to agent (new): 0.0504 (DverD_{ver} retention 89.6%→94.4%, test retention 63.2%→58.6%)
Individual criteria have shortcutsReward: levels gated by logical dependencyUngated sum: 0.0472
Noisy outcomes; good/bad turns mixCredit: per-turn verifiable scores let GRPO compute turn-level advantages without a critic or extra rolloutsTrajectory-level advantage: 0.0490
Falsified directions recur as pool growsMemory: first failed level gives tool-typed failure records retrieved during RLNo bank: 0.0476 (repeated failures rise 7.8%→19.6%)

Removing any of the four components drops RankIC by 0.0038–0.0056. Unlike learned/human process supervision, every level here is deterministic tool output; unlike AlphaAgentEvo (which collapses rewards into one visible-window trajectory scalar and trails the strongest frozen agent on all markets, Table 2), EvoMind credits individual turns using hidden cross-period stability.

3. Summary of contributions.

(1) Formulating factor mining as a multi-turn MDP with an explicit information boundary (Section 3); (2) unifying gated reward, turn-level credit, and typed memory under one verifiable hierarchy (Section 4), applicable to agent tasks with overfittable empirical feedback; and (3) open-sourcing our training infrastructure, prompts, and configs. We will sharpen the Introduction, Section 2, and Section 4 (adding the table above) accordingly. Thank you again for your review.

Reviewer_gE8FOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvoMind trains an LLM to conduct multi-turn alpha-factor research using gated rewards for validity, predictive strength, novelty, and cross-period stability. It assigns group-relative advantages from turn-level returns-to-go and retrieves successful and tool-typed failed attempts from an experience bank. The resulting factor library feeds a common LightGBM predictor and portfolio rule. Across three equity markets, the reported mean test RankIC increases from 0.0447 to 0.0493, with same-backbone controls and component ablations over five end-to-end runs.

Critical strengths

S1. The reward and experience design is well aligned with the factor-discovery process. Gating prevents invalid or uninformative formulas from collecting later rewards, while turn-level returns provide more targeted supervision than a shared trajectory advantage. The same-reward trajectory-level control and successes-only/random-retrieval controls provide useful evidence for these choices.

S2. The empirical study includes meaningful controls and consistent test-period improvements across three markets. Agents share a backbone and downstream pipeline, the final test period is declared unused until freezing, and the portfolio includes transaction costs. The paper also acknowledges its slightly worse S&P 500 drawdown rather than claiming uniform superiority.

Critical weaknesses

W1. The evidence does not isolate the benefit of hiding verification metrics from the benefit of additional cross-period supervision. Under the stated stability definition, replacing the verification window with the search window makes the stability score equal to one for candidates reaching L4. The visible-window ablation therefore removes an informative stability signal rather than holding supervision fixed while changing its visibility. Moreover, retention in Table 4 is measured after selecting factors using that same stability criterion. These results support the usefulness of cross-period supervision, but provide weaker evidence for the specific anti-overfitting explanation. This does not establish leakage from the separately held-out test period.

W2. The matched-budget claim is insufficiently specified. The paper does not report the numerical mining evaluation budget or episodes per round, explain whether offline training evaluations are included, or specify the transfer of the training pool and experience bank into mining (Section 5.1, Algorithm 1, Table 5). Mining efficiency with an already trained policy is a valid quantity, but differs from end-to-end research efficiency. The current reporting does not make this distinction auditable.

Questions for authors

Q1. How exactly is the visible-stability ablation implemented, and which verification-based rewards and admission rules remain active? If the claim concerns withholding metrics specifically, an equally supervised control with visible verification metrics would be informative.

Alternatively, narrowing the interpretation to cross-period development supervision would address much of this concern.

Q2. Please provide per-method factor-evaluation counts separately for training-data generation, RL, and final mining, together with episodes per mining round. Is the intended efficiency claim about mining time or total research cost, and how does the comparison change when offline search is included?

Q3. Which markets and task distributions train each policy, do the controls receive the same warm start, and are the factor pool and experience bank reset or carried into final mining? These details are important for interpreting both resource matching and transfer.

4 comments
AuthorsOpenReview ↗

Response to W1 & Q1 (Part 1/4): Hidden Verification vs. Cross-Period Supervision

Thank you for your careful review and constructive feedback. We agree with both points in W1 (and thank you for noting no test leakage), and have carried out both suggestions in Q1 on CSI 300 (five runs, mean ± SD; unpaired Welch tt unless marked paired).

1. Submitted implementation & standardized admission (Q1).

In the submission, the visible-window ablation replaced DverD_{ver} with DsrchD_{srch} everywhere—uu in Eq. (1), outcome reward (Eq. 2), admissibility u≥umin⁡u \ge u_{\min}, admission ranking, and UNSTABLE typing—giving u=1u = 1 at L4 (Section 4.1). Thus it removed cross-period supervision and omitted DverD_{ver} admission screening (explaining 76 vs. 73 admitted factors in Table 3). In the revision, all variants share identical final-mining admissibility and pool-admission rules; re-mining that ablation's five policies under the standard rule (no retraining) gives Col. 2 below, alongside a retrained visibility control that matches full EvoMind everywhere except showing each candidate's DverD_{ver} RankIC and uu in tool outputs (Col. 3):

Metric / Setting (CSI 300)(1) Table 3 ablation (submitted)(2) Same ablation, std. admission(3) Visible-DverD_{ver} control(4) Full method
Cross-period training rewardsNoNoYesYes
DverD_{ver} screening at admissionNoYesYesYes
DverD_{ver} metrics shown to agentNoNoYesNo
Test RankIC0.0468 ± 0.00140.0482 ± 0.00140.0504 ± 0.00130.0528 ± 0.0012
Admitted factors76627773
Retention on DverD_{ver} (%, Table 4 def.)62.3 ± 3.180.3 ± 2.494.4 ± 1.689.6 ± 1.8
Retention on test period (%)52.6 ± 2.555.4 ± 2.458.6 ± 2.363.2 ± 2.1

2. Single-factor effect decomposition.

Isolated factorColumns comparedΔ\Delta Test RankIC (tt-value)Δ\Delta Test retentionInterpretation
Cross-period supervision(4) − (2), both hidden+0.0046 (t=5.6t = 5.6)+7.8% (t=5.5t = 5.5)Dominant gain, matching your assessment
Metric hiding(4) − (3), same reward+0.0024 (t=3.0t = 3.0)+4.6% (t=3.3t = 3.3)Showing DverD_{ver} inflates DverD_{ver} fit (94.4%) and hurts transfer
Admission screening(2) − (1), same 5 policies+0.0014 (paired t=2.4t = 2.4, borderline)+2.8%Effect of standardizing final pool admission

Unscreened test retention across all methods appears in Part 3. We will update Table 3 (renaming that row "w/o cross-period signal in training," updating its value, and adding Col. 3) and separate the two effects in the Abstract, Introduction, and Section 5.3.

AuthorsOpenReview ↗

Response to W2 & Q2 (Part 2/4): Evaluation Budget and Efficiency Claim

We agree that "matched budget" was underspecified. Below we tabulate stage-wise evaluation counts, clarify the scope of our efficiency claim, and scale frozen agents up to EvoMind's total evaluation budget (one evaluation = one candidate submitted to the tool; carried state appears in Part 3).

1. Per-stage evaluation counts & scope of efficiency claim (Q2).

"Matched budget" covers final mining only (10 rounds × 8 episodes × ≤4 turns × ≤2 candidates = 640 cap/run). Table 4's evaluations per admissible factor (6.8 EvoMind, 7.1 from an empty bank, 10.6 AutoScientist-Quant) counts final mining only; we claim mining-time efficiency with a trained policy, not end-to-end research efficiency:

Method (evaluations per run)Warm startRL trainingFinal miningTotalEvoMind / Method
AlphaAgent, QuantaAlpha, AutoScientist-Quant (cap)00640640253.5×
Frozen-EvoMind (cap)8,00006408,64018.8×
AlphaAgentEvo (cap)0153,600640154,2401.05×
Outcome-GRPO, EvoMind, Table 3 variants (cap)8,000153,600640162,2401.0×
EvoMind logged actual mean (CSI 300, 5 runs)7,140134,820568142,528 (87.9% of cap)—

RL runs 600×8×4=19,200600 \times 8 \times 4 = 19,200 episodes; warm start uses 1K trajectories of ≤8 evaluations (Table 5), generated once per market and shared across runs on that market (charging all 8,000 per run is conservative; symbolic baselines use native budgets outside this table). Trained variants are matched end to end; against frozen agents, the revision reports both same-start mining-matched (Part 4) and total-budget-matched comparisons below.

2. Scaling frozen agents to EvoMind's total evaluation budget (Q2).

On CSI 300, we scaled AutoScientist-Quant and Frozen-EvoMind up to a 162,240 mining budget (640 from Table 2, 5 runs; higher budgets, 3 new runs each) against Table 2's EvoMind (0.0528 ± 0.0012, 162,240 total budget):

Method (CSI 300 Test RankIC)6406,40064,000162,240EvoMind (162,240) − Scaled 162,240 (tt)
AutoScientist-Quant0.04790.04950.05090.0507 ± 0.0013+0.0021 (t=2.3t = 2.3, suggestive, 3 runs)
Frozen-EvoMind (+8,000 warm start)0.04610.04740.04860.0489 ± 0.0014+0.0039 (t=4.0t = 4.0)

Returns diminish sharply past 64,000 (−0.0002 and +0.0003). Frozen-EvoMind (170,240 total cap including warm start) shares the exact backbone, warm start, loop, and bank mechanism, directly isolating spending evaluations on RL vs. search. We will rename "matched budget" to "matched mining budget across agentic methods" and add these counts and scaling results to Sections 5.1, 5.4, 6, Table 5, and new Appendix K.

AuthorsOpenReview ↗

Response to W2, Q3 & W1 (Part 3/4): Training Protocol and What Is Carried into Mining

We tabulate below the training markets, warm-start checkpoints, state carried into final mining, and unscreened test-period retention (W1).

1. Protocol audit: training data, warm start, and carried state (W2 & Q3).

MethodRL training dataWarm-start checkpoint (per market)Factor pool at mining startExperience bank / memory at mining start
AlphaAgent, QuantaAlpha, AutoScientist-QuantNoneNone (base Qwen3-4B)EmptyEmpty
Frozen-EvoMindNoneShared 1K format-filtered SFTEmptyEmpty
AlphaAgentEvoTarget market Dsrch,DverD_{srch}, D_{ver}None (per original method)EmptyEmpty
EvoMind (policy-only, new)Target market Dsrch,DverD_{srch}, D_{ver}Shared 1K format-filtered SFTEmptyEmpty
EvoMind (full, Table 2) & Table 3 variants*Target market Dsrch,DverD_{srch}, D_{ver}Shared 1K format-filtered SFTEmptyCarried from RL (*except "w/o bank")
  • Tasks & warm start: One policy is trained per market (DsrchD_{srch} 2021, DverD_{ver} 2022) and mines on that same market; RL (8 tasks/step) and mining (8 episodes/round) draw from the same market-data-free task set XX (Appendix K). Each market's 1K warm-start trajectories are generated once on that market and filtered strictly for format (unparsable outputs are discarded before tool execution and incur zero evaluations; never filtered by performance). Note that warm-started Frozen-EvoMind trails non-warm-started AutoScientist-Quant on all markets (Table 2).
  • Carried bank & test usage: All final libraries come strictly from the 640 mining budget (empty initial pool). Carrying the RL bank is EvoMind's second time scale (Section 4.3), omitted in our submission; policy-only reaches 0.0517 / 0.0569 / 0.0359, still beating AutoScientist-Quant by +0.0038 / +0.0037 / +0.0028 while carried experience adds +0.0011 / +0.0015 / +0.0008 (Part 4; in Table 3, policy-only vs. "w/o bank" is 0.0041). Paper defaults were fixed before the first test evaluation (Section 5.1); supplementary runs are strictly for reporting, never selection.

2. Unscreened test-period retention (W1).

Because Table 4's DverD_{ver} retention is computed after DverD_{ver} filtering, we evaluate retention on the untouched test period on CSI 300:

Retention on test period (%)QuantaAlphaAutoScientist-QuantFrozen-EvoMindAlphaAgentEvoEvoMind (policy-only)EvoMind (full)
CSI 30044.651.653.754.3 ± 2.461.9 ± 2.2 (t=5.2t = 5.2)63.2 ± 2.1 (t=6.2t = 6.2)

(tt-values compare against runner-up AlphaAgentEvo, +7.6 and +8.9 points.) Retention is test-to-DsrchD_{srch} RankIC ratio: since EvoMind and AutoScientist-Quant have similar standalone test RankIC (0.0210 vs. 0.0204) while EvoMind preserves sign more often (93% vs. 85%), higher retention reflects less in-sample inflation and smaller out-of-sample drop rather than higher standalone test levels. We will add these specifications and test retention to Section 5.1, Algorithm 1, Table 4, and Appendix K.

AuthorsOpenReview ↗

Response to W2 & Q3 (Part 4/4): Policy-Only Mining and Cross-Market Transfer

To isolate transferable search skill from training-time state, we re-mined existing trained policies (no retraining) under two checks: clearing the experience bank before mining (policy-only) and transferring policies across markets (five-run means ± SD).

1. Separating policy weights from carried experience.

Clearing the experience bank before final mining matches the exact empty starting state of frozen agents:

Method / Decomposition (Test RankIC)CSI 300CSI 500S&P 500
AutoScientist-Quant0.0479 ± 0.00140.0532 ± 0.00160.0331 ± 0.0013
Frozen-EvoMind0.0461 ± 0.00150.0510 ± 0.00170.0309 ± 0.0014
EvoMind, policy-only (empty bank)0.0517 ± 0.00130.0569 ± 0.00160.0359 ± 0.0012
EvoMind, full system (Table 2)0.0528 ± 0.00120.0584 ± 0.00140.0367 ± 0.0011
Weights: Policy-only − Frozen-EvoMind (tt)+0.0056 (t=6.3t = 6.3; 84%)+0.0059 (t=5.7t = 5.7; 80%)+0.0050 (t=6.1t = 6.1; 86%)
Policy-only − AutoScientist-Quant (tt)+0.0038 (t=4.4t = 4.4)+0.0037 (t=3.7t = 3.7)+0.0028 (t=3.5t = 3.5)
Carried bank: Full − Policy-only (paired tt)+0.0011 (t=2.7t = 2.7, 4/5 > 0)+0.0015 (t=3.0t = 3.0, 5/5 > 0)+0.0008 (t=2.2t = 2.2, 4/5 > 0)

Percentages denote share of full gain (+0.0067 / +0.0074 / +0.0058) retained by weights alone. Because an empty bank rebuilds online during mining, clearing it loses only 21% (0.0011) of the bank's total 0.0052 contribution in Table 3 (on CSI 300, admissibility is 14.0% vs. 14.6% and repeated failures 10.4% vs. 7.8%).

2. Zero-retraining cross-market policy transfer.

We transfer the five policies trained on one market directly to another market with an empty pool and empty bank (both warm start and RL were completed on the source market, whereas Frozen-EvoMind was warm-started on the target market; format-only filtering on both sides):

Target market (policy-only)Frozen-EvoMindCross-market policyTarget-market policyAutoScientist-QuantCross − Frozen (tt)Cross − AS-Q (tt)
CSI 300 (from S&P 500)0.0461 ± 0.00150.0497 ± 0.00140.0517 ± 0.00130.0479 ± 0.0014+0.0036 (t=3.9t = 3.9)+0.0018 (t=2.0t = 2.0, n.s.)
S&P 500 (from CSI 300)0.0309 ± 0.00140.0343 ± 0.00130.0359 ± 0.00120.0331 ± 0.0013+0.0034 (t=4.0t = 4.0)+0.0012 (t=1.5t = 1.5, n.s.)

Cross-market policies never searched on the target market yet achieve 64% / 68% of the target-market training gain (trailing target-trained policies by 0.0020 / 0.0016, t=2.3/2.0t = 2.3 / 2.0, borderline). Because the margin over AutoScientist-Quant is not significant (n.s.), we claim transferable gains over target-market Frozen-EvoMind rather than beating the strongest target-market baseline. We will add both experiments to Tables 2 and 4, Section 5.2, and Appendix K. Thank you again for your constructive review.

Reviewer_kvFdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper trains the policy of an alpha-mining agent, so the model that proposes factors can change with market feedback. Discovery is a short loop of hypothesis, implementation, tool feedback, and revision. A turn receives the stability reward when the factor executes without look-ahead, predicts on a visible window, is novel relative to the pool, and keeps that predictive power on a later window the agent never sees. Credit comes from what follows each turn, and a typed experience bank is given to later episodes. Under a shared budget and a costed portfolio, the trained agent improves test rank correlation and excess return on three equity indices. The ablations connect that gain to turn-level credit, the reward gate, the hidden window, and negative experience.

Critical strengths
  1. The learning problem is clearly specified. A frozen miner can change its prompt and memory, while one reward for a whole trajectory leaves open which revision helped. EvoMind scores every turn, gives novelty once a factor predicts, and gives stability on a window the agent never sees. The failure type in the experience bank comes from that same hierarchy, so a later explanation can describe a failure while the type itself stays fixed by the tool.

  2. The experiments keep the data, search budget, and portfolio the same across methods. Frozen and trained agents also use the same backbone. The paper repeats the runs over five seeds and three markets, and removes one component at a time. A model trained only on the final outcome does not beat the frozen agent, so the gain is unlikely to come from reinforcement learning alone.

  3. The paper is careful about what it claims. Transaction costs are included, and the US drawdown is reported even though it is not better than the baseline. The limitations also note market impact, the use of one verification year, and the chance that pretraining overlaps the test period. The number in the abstract is simply the mean of the three markets.

Critical weaknesses
  1. The gains are consistent in direction and modest in size. Mean test rank correlation moves from about 0.045 to about 0.049, and mean annualized excess return from about 7.5 percent to about 9.2 percent. On the US index the drawdown is not lower. Five seeds describe variation across runs, and it remains open whether a similar edge would appear in another market regime or under other reasonable reward choices.

  2. The hidden window is a single year, and that year stays inside the training objective throughout. Moving stability onto the visible window lowers performance, which supports the idea behind the gate. Over hundreds of updates a policy can still adapt to that particular year, and a rotated or walk-forward check is not reported. It would be useful to know how much of the robustness is tied to 2022.

  3. The reward is a designed scalar, and later episodes also read short lessons written by the reference model. Removing the gates, the novelty term, or the experience bank lowers performance in this configuration, while leaving open whether this order and these thresholds are the only reasonable choice. A random-retrieval control suggests that the retrieved text matters on one market, and it is less clear how stable those lessons are across runs.

  4. The training reward and the reported portfolio answer related but different questions. Factors are rewarded for rank correlation, novelty, and a hidden-window ratio, while the portfolio uses a fixed booster, a tight replacement cap, and a one-way cost, without impact or capacity. The backbone is recent enough that its pretraining may overlap the test years, although the agent does not receive test labels. A comparison with a backbone whose knowledge cutoff precedes that window would make this channel easier to judge.

  5. Factors may enter the library when their correlation with existing members stays below 0.70, but the paper does not show how alike the admitted factors actually are. The repeated-failure rate compares new candidates with earlier failures, not the factors that were kept. The reported rank correlation is the score of a LightGBM model trained on the whole library. Correlation within that library, and a summary of the individual factors, would show whether the search found complementary signals.

Questions for authors
  1. If the verification window were rotated during training, would the test gap over the strongest frozen agent remain on more than one market?

  2. Under the same reward and budget, would the gain remain if the backbone's pretraining cutoff preceded the test window?

  3. If the terminal reward included the costed portfolio, how would the test excess return change?

8 comments
AuthorsOpenReview ↗

Response to W1 (Part 1/7): Size and Statistical Support of the Gains

Thank you for your careful review and constructive feedback. We agree that the gains are modest and that five seeds capture only run-to-run variation. Below we separate two evaluation modes and test support across runs, under market noise, and across regimes (values are five-run means ± SD; tt-values are unpaired Welch tests across runs unless marked paired).

1. Full system vs. policy-only across runs.

Table 2 reports the full system: RL-trained weights plus the experience bank accumulated during RL (Section 4.3), whereas frozen agents start from an empty bank. To match their exact starting state, we add policy-only (clearing the bank before final mining):

Method / Comparison (Test RankIC)CSI 300CSI 500S&P 500
AutoScientist-Quant (strongest baseline)0.0479 ± 0.00140.0532 ± 0.00160.0331 ± 0.0013
EvoMind, policy-only (empty bank at mining)0.0517 ± 0.00130.0569 ± 0.00160.0359 ± 0.0012
EvoMind, full system (Table 2)0.0528 ± 0.00120.0584 ± 0.00140.0367 ± 0.0011
Policy-only − AutoScientist-Quant (Welch tt)+0.0038 (t=4.4t = 4.4)+0.0037 (t=3.7t = 3.7)+0.0028 (t=3.5t = 3.5)
Full system − AutoScientist-Quant (Welch tt)+0.0049 (t=5.9t = 5.9)+0.0052 (t=5.5t = 5.5)+0.0036 (t=4.7t = 4.7)
Carried experience: Full − Policy-only (paired tt)+0.0011 (t=2.7t = 2.7)+0.0015 (t=3.0t = 3.0)+0.0008 (t=2.2t = 2.2)

CSI 300 SDs are from Table 6; SDs for all methods across all three markets will be added to Appendix E. Unless noted, robustness checks in later parts follow the full-system protocol of Table 2.

2. Two-level bootstrap for market noise.

Because all methods share one realized market path, we resample with replacement both the five runs of each method and the 36 test calendar months (shared across methods and markets) over 10,000 replicates to estimate the SE of the mean daily difference, reporting 95% CIs (mean ± 1.96 SE):

EvoMind − AutoScientist-QuantCSI 300CSI 500S&P 5003-Market Mean
Full system: RankIC [95% CI]+0.0049 [0.0019, 0.0079]+0.0052 [0.0021, 0.0083]+0.0036 [0.0010, 0.0062]+0.0046 [0.0025, 0.0067]
Policy-only: RankIC [95% CI]+0.0038 [0.0008, 0.0068]+0.0037 [0.0006, 0.0068]+0.0028 [0.0002, 0.0054]+0.0034 [0.0013, 0.0055]
Full system: ARR (percentage points)+1.7 (spans 0)+2.4 (spans 0)+0.9 (spans 0)+1.7 [−1.1, 4.5]

RankIC CIs exclude zero in every market for the full system; under policy-only, CSI 300, CSI 500, and the three-market mean are supported, while S&P 500 is borderline. ARR holds across runs (t=4.1t = 4.1 on CSI 300, Table 6) but spans zero under market noise because 50-stock returns are dominated by stock-specific noise over three years (0.9–2.4 points/year); we therefore present RankIC as primary and ARR as directionally consistent. On S&P 500, MDD (7.6% full, 7.7% policy-only vs. 7.4% baseline) lies within run noise (t≤0.7t \le 0.7), and we draw no conclusion on it.

3. Consistency across market regimes.

Full-system RankIC leads by calendar year are positive in all nine cells across falling and rising years (e.g., CSI 300 fell 11% in 2023 and rose 15% in 2024):

RankIC lead202320242025
CSI 300+0.0061+0.0030+0.0056
CSI 500+0.0044+0.0073+0.0039
S&P 500+0.0055+0.0045+0.0008

Single-year values are noisy (S&P 500 in 2025 is indistinguishable from zero), showing only that no regime reverses the lead (see also Parts 2, 3, and 5). We will report both modes (Abstract, Introduction, Section 5.1, Table 2), specify "matched mining budget across agentic methods," and add these tests to Section 5.2 and Appendices E–F.

AuthorsOpenReview ↗

Response to W2 & Q1 (Part 2/7): Rotated Verification Windows

To examine how much robustness is tied to 2022, we evaluate training-window rotation, cross-market transfer, and a metric-visibility control (full-system protocol, Part 1).

1. Window rotation during RL training & cross-market check (Q1).

Pool admission (2021→2022) is shared by all agentic methods, and Frozen-EvoMind uses the same bank mechanism yet trails AutoScientist-Quant on all markets (Table 2). To test EvoMind's repeated use of 2022 in RL, we form four search→verification pairs (2018→2019, 2019→2020, 2020→2021, 2021→2022). During RL, tasks draw either from the first three only (w/o 2022, warm start also w/o 2022; 5 runs) or from all four (with 2022; 3 runs), leaving final mining, admission, predictor (fit on 2016–2020), and test evaluation unchanged:

Setting (Test RankIC)CSI 300CSI 500S&P 500
AutoScientist-Quant0.0479 ± 0.00140.0532 ± 0.00160.0331 ± 0.0013
EvoMind, fixed 2021→2022 (Table 2)0.0528 ± 0.00120.0584 ± 0.00140.0367 ± 0.0011
EvoMind, rotated, with 2022 (3 runs)0.0525 ± 0.00130.0587 ± 0.00140.0362 ± 0.0012
EvoMind, rotated, w/o 2022 (5 runs)0.0516 ± 0.00130.0574 ± 0.00150.0358 ± 0.0012
Rotated w/o 2022 − AutoScientist-Quant (tt)+0.0037 (t=4.3t = 4.3)+0.0042 (t=4.3t = 4.3)+0.0027 (t=3.4t = 3.4)
  • Market-noise CI (w/o 2022): Three-market mean lead is +0.0035, 95% CI [0.0014, 0.0056] (Part 1). Because margins are 0.0009–0.0012 below fixed 2022 (t≤1.5t \le 1.5), we do not claim 2022 had zero effect; using 2022 in 1/4 of tasks differs from fixed 2022 by ≤0.0005\le 0.0005 (∣t∣≤0.6|t| \le 0.6).
  • Cross-market transfer (policy-only): Mining from an empty bank with policies trained on another market reaches 0.0497 / 0.0343 on CSI 300 / S&P 500 vs. 0.0461 / 0.0309 for Frozen-EvoMind on the target market (t=3.9/4.0t = 3.9 / 4.0; Appendix K), never using target-market 2022 data.

2. Visible-window ablation correction & metric-hiding control (CSI 300).

Computing stability on DsrchD_{srch} gives u=1u = 1 at L4 (Section 4.1), removing cross-period supervision, and our submission also omitted 2022 admission screening on that row. Standardizing admission and adding a control that keeps cross-period rewards while showing 2022 metrics to the agent gives:

CSI 300 variantCross-period reward2022 admission filter2022 shown to agentTest RankIC2022 retention (%)Test retention (%)
Table 3 ablation (submitted)NoNoNo0.0468 ± 0.001462.352.6
Same ablation, std. admissionNoYesNo0.0482 ± 0.001480.355.4
Visible-2022 control (new)YesYesYes0.0504 ± 0.001394.458.6
Full EvoMind (Table 2)YesYesNo0.0528 ± 0.001289.663.2

Showing 2022 metrics lowers test RankIC by 0.0024 (t=3.0t = 3.0): retention (Table 4 def.) rises on 2022 yet falls on test, matching overfitting. We will update Section 5.3, Table 3, Sections 6–7, and new Appendix F.

AuthorsOpenReview ↗

Response to W3-1 & W1 (Part 3/7): Reward Order, Thresholds, and Weights

We agree that our submission did not test sensitivity to the reward specification. All four quantities in the hierarchy are computed deterministically by the tool without an LLM judge; what is hand-designed is the level ordering, thresholds, and weights. We varied each and retrained on CSI 300 (three runs per new setting, full-system protocol; Part 1).

1. Level ordering.

Three positions follow from dependencies: L1 must come first (invalid code has no metrics); L2 must precede L4 (stability ratio uu in Section 4.1 is noise when search RankIC is near zero); and novelty cannot precede predictive power (random/constant formulas are maximally "novel"). Testing the ungated sum (Table 3) and swapping the only free choice (L3 vs. L4) gives:

Ordering variant (CSI 300)Test RankICRepeated failures (%)Note
Ungated sum (Table 3)0.0472—Allows trivial "novel" shortcuts
Swap L3 & L4 (L4 before L3, new)0.0515 ± 0.00129.7Rewriting strong factors collects largest reward
Default order (L1→L2→L3→L4, Table 2)0.0528 ± 0.00127.8Novelty required before stability payout

2. Thresholds and stability weight (W3-1 & W1).

Perturbing one training parameter at a time while keeping shared final-mining admission rules fixed gives:

ParameterDefault (0.0528 ± 0.0012)Lower settingHigher setting
qmin⁡q_{\min}0.020.01: 0.0522 ± 0.00110.03: 0.0517 ± 0.0015
δ\delta0.300.20: 0.0511 ± 0.00130.40: 0.0533 ± 0.0012
umin⁡u_{\min}0.500.30: 0.0520 ± 0.00160.70: 0.0530 ± 0.0010
λ4\lambda_40.80.4: 0.0508 ± 0.00141.2: 0.0524 ± 0.0013
  • All nine alternative settings (eight above + L3/L4 swap) span 0.0508–0.0533 (−0.0020 to +0.0005 vs. default) and beat AutoScientist-Quant (0.0479; smallest margin +0.0029 at λ4=0.4\lambda_4 = 0.4, t=2.8t = 2.8).
  • Threshold shifts stay within 0.0017 (∣t∣≤1.8|t| \le 1.8) because rewards remain continuous. Halving λ4\lambda_4 causes the largest drop (t=2.1t = 2.1 vs. default), directionally consistent with stability being the primary signal (since Eq. 4 normalizes advantages per group and turn depth, λ4\lambda_4 sets relative level weighting rather than gradient scale).

With three runs per setting, this is a sensitivity check over the tested range; all defaults were fixed before the first test evaluation (Section 5.1) and are not retuned. We will add these results to Sections 4.1, 6, and new Appendix G.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.