acceptodds
Under review as a conference paper at ICLR 2027

EvoQuantLab: Training-Free Dual-Loop Evolution for Auditable Quantitative Research

Abstract

Automated alpha factor discovery is critical for modern quantitative research. While large language models (LLMs) have increasingly been integrated into quantitative workflows, existing agent-based approaches remain highly susceptible to severe backtest overfitting. To address this challenge, we propose **EvoQuantLab**, a training-free, dual-loop self-evolving framework for robust factor discovery. Instead of relying entirely on LLM generation, **EvoQuantLab** deploys an inner genetic-programming (GP) loop that dramatically improves token efficiency while evolving typed factor expressions under structural and diversity constraints. An outer diagnosis loop then systematically audits evidence across cycles to drive protocol-level improvements. Furthermore, our evaluation pipeline applies multi-metric validation to rigorously filter out spurious factors before deployment. Extensive out-of-sample experiments across both daily and hourly frequencies—covering equities and cryptocurrency—demonstrate the robustness of **EvoQuantLab**. Specifically, on daily CSI 500 and S&P 500 tasks, **EvoQuantLab** achieves leading annualized returns of 18.34% and 21.30%, consistently outperforming existing agent baselines. On high-turnover hourly cryptocurrency data, **EvoQuantLab** delivers a 37.91% annualized return while effectively avoiding execution collapse.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
ai4quant, quant research, harness

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_t9TPOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces EvoQuantLab, a training-free framework for automated quantitative factor discovery. The proposed system combines an inner genetic programming loop for symbolic factor evolution with an outer diagnosis loop that modifies research protocols based on development-set evidence. It also incorporates a multi-stage validation pipeline intended to reduce backtest overfitting and improve the reliability of discovered factors.

The authors evaluate the framework against four LLM-based quantitative research agents on CSI500, S&P500, and hourly cryptocurrency datasets. They report improvements in predictive metrics and portfolio returns, along with component ablations, development-to-test transfer analyses, and computational efficiency comparisons.

Critical strengths
  1. Relevant problem and systematic workflow. The paper addresses adaptive overfitting in automated quantitative research, which is an important practical problem. Separating factor search, development validation, and held-out evaluation is a sensible design principle.

  2. Broad evaluation scope. The evaluation includes multiple markets and frequencies, several agent baselines, component ablations, and development-to-OOS diagnostics. These are useful ingredients for studying automated factor discovery.

Critical weaknesses
  1. The experimental comparisons are insufficiently controlled, and the reported efficiency claims are not adequately supported.

The main experiments compare EvoQuantLab against four agent-based baselines, but the computational budgets differ substantially. Table 5 reports that EvoQuantLab requires 22 hours and 8 minutes, whereas the baselines finish within approximately 19 minutes to 2 hours. EvoQuantLab also achieves the lowest reported throughput, at 0.95 L5 factors per hour.

Although the authors emphasize reduced LLM token consumption, this does not establish overall computational efficiency. The substantial additional runtime and genetic programming search effort are not normalized against the competing methods.

Moreover, the paper does not establish performance under matched total computational budgets or matched numbers of evaluated factor candidates. Without these controls, it is difficult to distinguish the benefit of the proposed framework from the effect of additional search and validation effort.

The comparison is further complicated by the downstream evaluation pipeline. Appendix C.2 states that the daily equity strategies employ a LightGBM model trained on selected factor pools. Therefore, the final performance reflects both the quality of discovered factors and the downstream supervised learning and selection procedures. Additional controls are needed to isolate the contribution of the proposed discovery algorithm.

  1. The central claims about overfitting prevention and realistic trading performance are not convincingly established.

The paper repeatedly positions its contribution as a solution to adaptive backtest overfitting. However, the experimental evidence is limited to relatively short OOS periods: approximately three months for the daily equity experiments and slightly over two months for cryptocurrency.

Such evaluation windows are insufficient to establish robust generalization across different market regimes, especially for a framework that makes strong claims about preventing overfitting and improving research reliability. The results are also largely reported as single performance estimates without sufficiently comprehensive repeated-run uncertainty analysis.

The development-to-OOS transfer analysis does not fully resolve this concern. The methods retain substantially different numbers of candidate factors, ranging from single-factor baselines to dozens of candidates for EvoQuantLab. Consequently, comparing sign-survival rates and winner performance across these heterogeneous candidate pools is not a controlled demonstration of reduced selection bias.

There is also an important inconsistency in the trading evaluation. The paper describes its experiments as execution-aware and incorporating transaction costs. However, Appendix C.2 explicitly states that the reported CSI500 and S&P500 performance reflects gross returns before execution fees and slippage. The cryptocurrency evaluation includes a fixed 5 bps trading cost but excludes perpetual funding costs.

These omissions weaken the paper's claims about executable and economically robust factor discovery. The evidence does not convincingly establish that the reported advantages would persist under realistic trading costs and implementation constraints.

Questions for authors

Can the authors provide matched-budget comparisons using identical total computational resources, wall-clock limits, and numbers of evaluated factor candidates? How much improvement remains under these controls?

Why does the manuscript characterize the daily equity backtests as execution-aware when Appendix C.2 explicitly reports gross returns before transaction costs? Can the authors provide net-of-cost results with turnover, fees, slippage, and liquidity constraints?

Can the authors provide repeated-seed evaluations and statistical confidence intervals for the main OOS performance results? How sensitive are the results to the selected test periods?

How much of the performance improvement is attributable specifically to the outer diagnosis loop rather than additional GP search, multi-stage filtering, or the downstream LightGBM pipeline? Can the authors provide a more controlled attribution analysis?

How are the different candidate-pool sizes and selection budgets accounted for when comparing development-to-OOS sign survival and selection-bias robustness?

6 comments
AuthorsOpenReview ↗

(1/3) Response by authors

W1-a, Q1: Throughput, efficiency, and matched-budget controls.

We agree that LLM-token savings do not establish overall efficiency and will narrow Sec. 5.3: our claim is only that LLM cost does not scale with search breadth (~2,560 candidates with 50 LLM calls). Three clarifications on Table 5:

  • Throughput is not comparable across methods. Baseline figures count factors passing each method's own checks (e.g., FactorMiner: 156/h, yet M = 1 retained factor in Table 8). EvoQuantLab's 0.95/h counts factors passing our L0–L5 gates (21 admissions before final deduplication to 17). We will relabel the column.
  • The low admission rate is intended. 17 of ~2,560 candidates (0.66%) are admitted. In our counterfactual audit (Reviewer rkBY, W1), candidates rejected by economic falsification have a higher median Dev IR than the admitted ones (5.02 vs. 4.91) but an OOS IR of only 0.69, versus 3.709 for the admitted portfolio.
  • Runtime breakdown. 90% is parallel CPU-side GP evaluation (48 cores), 7% is L3–L4 falsification (cached factor values re-scored against block-permuted labels), and 3% is LLM calls.

Matched candidate budget (Crypto, identical L0–L6 gates, ~2,560 evaluations):

VariantGPLLM (L1)Outer loop# L5OOS IRAR
Random sampling (Cycle 0 w/o GP)✗✗✗0––
Pure GP (fixed space, no LLM)✓✗✗31.37512.03%
GP + LLM, no outer loop (No Loop B)✓✓✗50.7526.58%
Full EvoQuantLab✓✓✓173.70937.91%

Conventional GP reaches ~37% of the full system's IR, and adding LLM proposals without diagnosis lowers it further. This indicates that the gain comes from the outer loop rather than from additional search.

Matched wall-clock (Crypto, cumulative L5 portfolio OOS IR):

Elapsed TimeEvoQuantLabBest Baseline (Full Budget)
2h45m (Cycle 0)1.570 (AR 19.93%)FactorMiner: 0.952; QuantaAlpha: −1.665
6h48m (Cycle 2)3.23–
14h55m (Cycle 6)4.01–
22h08m (Final, Cycle 10)3.709 (AR 37.91%)–

(Note: This trajectory tracks our primary benchmark campaign ending at 3.709; Fig. 5 wrongly plotted an earlier evaluation script with part of missing exchange records, which we will align in the revision.)

At a comparable wall-clock budget (~2.75h), EvoQuantLab already exceeds every baseline's full-budget result on Crypto.

W1-b, Q4: Isolating discovery from the downstream LightGBM.

  1. Identical downstream model. All methods feed their pre-screened factor pools into the same LightGBM pipeline with the same hyperparameter protocol and early stopping (App. C.2); only the input pool differs.
  2. No learned model on Crypto. Crypto positions are combined by parameter-free equal weighting (Eq. 12), and this is where the gap is largest (3.709 vs. −2.385 to 0.952).
  3. Equities without LightGBM (new). We replace LightGBM with a parameter-free equal-weight combination (mean of rank-normalized factor scores, followed by the same TopkDropout(50, 5) portfolio) for EvoQuantLab and all baselines:
OOS IR (AR)EvoQuantLab, LightGBM (Table 2)EvoQuantLab, equal-weightBest baseline, equal-weightEvoQuantLab, single dev-best factor (Table 3)
CSI 5001.613 (18.34%)1.322 (15.24%)1.216 (13.89%)[AlphaAgent]1.041 (11.89%)
S&P 5001.614 (21.30%)1.240 (14.68%)1.058 (11.85%)[FactorMiner]2.466 (40.73%)

Without LightGBM, EvoQuantLab's factor pools still outperform the best baseline under the same parameter-free combination on both markets. The advantage therefore comes from the discovered factors rather than from the downstream model.

AuthorsOpenReview ↗

(2/3) Response by authors

W2-a, Q3: OOS window length and uncertainty.

Why ~70 days.

  1. Short-horizon signals. Our factors are short-horizon price-volume and order-flow signals. Comparable formulaic alphas hold positions for 0.6–6.4 days on average (Kakushadze, 2016), and published return predictors lose much of their efficacy over time (McLean & Pontiff, 2016), so such signals are re-discovered and re-validated on a rolling basis. A ~70-day frozen window spans dozens of independent holding periods.
  2. Statistical sufficiency (Crypto). By the Minimum Track Record Length criterion (Bailey & López de Prado, 2012), confirming an hourly IR of 3.709 at 95% confidence requires ~1,725 hourly bars, slightly more than our original window (1,680 bars). This motivated the forward extension below.

Forward extension (new). We evaluated the frozen, content-hashed artifacts on data after the original OOS windows, without re-running any search (CSI 500: 2026-08-18 to 09-30; S&P 500: 2026-08-17 to 09-30; Crypto: 2026-06-10 to 08-31, re-collected as a complete snapshot and checked for coverage before evaluation):

MarketOriginal OOS IRExtension-only IRCombined OOS IRBest baseline (combined)
CSI 5001.6131.5871.6021.224 (AlphaAgent)
S&P 5001.6141.4881.5671.356 (FactorMiner)
Crypto3.7094.9214.3431.195 (FactorMiner)

Over the combined crypto window (3,672 bars), the Probabilistic Sharpe Ratio reaches 0.998. For daily equities, establishing an IR of ~1.6 on a single series would require roughly a year of daily data, so our equity evidence rests on the paired comparison with baselines over the same period and on consistency across seeds.

Repeated campaigns (new). Three independent 11-cycle campaigns from scratch per method (OOS IR, mean ± std, n = 3):

MethodCryptoCSI 500S&P 500
EvoQuantLab3.65 ± 0.191.65 ± 0.081.64 ± 0.07
Best baseline0.93 ± 0.04 (FactorMiner)1.32 ± 0.04 (AlphaAgent)1.50 ± 0.07 (FactorMiner)

EvoQuantLab ranks first in every seed on all three markets. We will add a Limitations section noting that multi-month OOS windows do not span multi-year macroeconomic cycles.

W2-b, Q5: Heterogeneous candidate pools in the transfer audit.

We agree that sign survival over pools of unequal size is descriptive rather than controlled, and we will present it as such in App. E. We add two controlled checks.

  1. Matched top-K audit. For each method we rank its own candidates by Dev IR only and report the mean OOS IR of the top-K, so that EvoQuantLab's larger pool cannot benefit from averaging over more factors. We compare against the strongest baseline of each market in the main results (Table 2 / Fig. 4):
MarketMethod (M)Top-1Top-3 meanTop-5 mean
CryptoEvoQuantLab (12)4.0554.1203.075
CryptoFactorMiner (1)0.952––
CSI 500EvoQuantLab (36)1.0411.7251.642
CSI 500AlphaAgent (4)0.7981.2161.328 (all 4)
S&P 500EvoQuantLab (49)2.4661.9281.802
S&P 500FactorMiner (1)1.382––

EvoQuantLab's top-3 and top-5 factors remain strong out of sample in all three markets, and it is higher than the strongest baseline at every K where the comparison is defined.

  1. Trial-budget-matched selection correction. Selection bias scales with the number of evaluated trials rather than the retained pool size MM. Eq. 2 compares each cycle's winner with B0=100B_0 = 100 block-permuted null searches under the same trial budget; in all 11 Crypto cycles the real winner exceeded every null maximum (p^sel=1/101\hat{p}_{\mathrm{sel}} = 1/101, the smallest attainable value). We will also correct App. E: extreme-value noise grows as 2log⁡M\sqrt{2 \log M}, not "exponentially".

References

  • Bailey, D. H. and López de Prado, M. The Sharpe Ratio Efficient Frontier. Journal of Risk, 15(2):3–44, 2012.
  • Kakushadze, Z. 101 Formulaic Alphas. Wilmott, 2016(84):72–81, 2016.
  • McLean, R. D. and Pontiff, J. Does Academic Research Destroy Stock Return Predictability? The Journal of Finance, 71(1):5–32, 2016.
AuthorsOpenReview ↗

(3/3) Response by authors

W2-c, Q2: Costs, funding, and execution conventions.

We thank the reviewer for identifying this inconsistency. The error is in the description, not in the reported results: our Qlib evaluation configs confirm that the "gross, dollar-neutral" wording in Sec. 4.1, Eq. 11, the IR definition (App. C.1) and App. C.2 came from an early prototype and does not describe the protocol used for any reported number. We will correct all four.

Equities (Tables 2, 3 and 8). All equity results use the long-only TopkDropout(50, 5) protocol of AlphaAgent (top-50 stocks, at most 5 replacements per day), measured as excess return over the equal-weighted universe.

  • Costs. 5 bps buy / 15 bps sell on CSI 500 (including stamp duty); 0 bps buy / 5 bps sell on S&P 500, approximating zero-commission buys and regulatory/transaction fees on sales.
  • Execution and liquidity. Orders are filled at the daily close under Qlib's standard SimulatorExecutor (aligned with next-day forward returns), which blocks buys at limit-up, sells at limit-down, and trades on suspended or zero-volume days. Being long-only, the portfolio respects A-share short-sale constraints.
  • Slippage. We do not model slippage separately. The cost sensitivity reported below for crypto (break-even ≈ 62 bps one-way) indicates the margin available to absorb it.

All reported equity figures (e.g., CSI 500: AR 18.34% / IR 1.613; S&P 500: AR 21.30% / IR 1.614) are therefore net of transaction costs.

Crypto (17-factor equal-weight portfolio, frozen OOS window).

  • Execution lag and turnover. Signals formed at hour tt are filled at the close of bar t+1t+1, so the price move between signal formation and execution is not earned. Each bar's one-way turnover is TOt=12∑i∣wi(t)−wi(t−1)∣TO_t = \frac{1}{2}\sum_i \lvert w_i(t) - w_i(t-1) \rvert, where wi(t−1)w_i(t-1) is the previous hour's target weight (without intraday drift re-weighting), and costs are charged on TOtTO_t at every bar. For readability, we report the sum over the 24 bars of each day averaged across OOS days: 0.150 / day (≈54.8×\approx 54.8\times per year; this aggregation is for reporting only and does not affect PnL). Turnover stays low because of L4 turnover gating and 72-hour rolling position smoothing.
  • Perpetual funding. Historical funding rates are available for 67% of notional exposure (covering major perpetual pairs with complete 8-hour funding snapshots; the remaining 33% correspond to assets lacking archived funding-rate history). Their net effect is +0.14%/yr, because dollar neutrality cancels the market-wide funding component, giving IR 3.725 / AR 38.05% net of fees and funding. Under an adversarial assumption for the uncovered 33% (longs pay the 90th-percentile rate, shorts receive the 10th), the additional drag is at most −3.40%/yr (AR ≈ 34.6%).
  • Cost sensitivity. Re-evaluating the same portfolio at {0, 5, 10, 20} bps one-way cost gives OOS IR 4.036 / 3.709 / 3.395 / 2.734 (AR 41.16% / 37.91% / 34.53% / 27.92%). The extrapolated break-even cost is ≈ 62 bps one-way (≈ 12.4× the 5 bps assumed in the paper).
Reviewer_rkBYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces EvoQuantLab, a training-free framework for automated alpha-factor discovery that combines an inner genetic-programming loop with an outer diagnosis-driven protocol-evolution loop. The inner loop searches typed symbolic expressions using development-fold performance, complexity penalties, and diversity controls. The outer loop audits completed research cycles and proposes protocol changes for subsequent cycles. A frozen, seven-layer evaluation pipeline incorporates point-in-time data checks, economic falsification, stress testing, and factor admission before held-out evaluation. The authors evaluate the framework on daily CSI 500 and S&P 500 tasks and an hourly cryptocurrency task against four agent-based baselines. They report annualized returns of 18.34%, 21.30%, and 37.91%, respectively, and provide component ablations, a development-to-OOS audit, cycle-level analysis, and resource and backbone comparisons.

Critical strengths
  1. The proposed dual-loop framework is an interesting design that goes beyond conventional automated factor discovery by jointly evolving candidate factors and the research protocol itself.
  2. The empirical study offers meaningful initial support for the integrated design.
Critical weaknesses
  1. A central claim of the paper is that the proposed framework mitigates adaptive backtest overfitting. However, the experiments mainly demonstrate improved OOS performance rather than directly validating the effectiveness of the overfitting diagnostic.
  2. EvoQuantLab requires substantially longer execution time than the competing baselines. Although the authors provide efficiency analyses and component ablations, the main comparisons do not clearly establish matched computational or candidate-evaluation budgets.
Questions for authors
  1. Since the outer loop modifies both the search space and the research protocol, how can the authors distinguish the benefits of diagnosis-driven updates from those of simply expanding the search space?
  2. How many independent runs were conducted for each method?
  3. Could the authors clarify the factor selection and evaluation protocols used in different experiments? In particular, why is the Crypto IR reported as 4.055 in Table 3 but 3.709 in Table 4 and Figure 4? Do these values correspond to different factor configurations or evaluation settings?
3 comments
AuthorsOpenReview ↗

(1/2) Response by authors

W1: Directly validating the overfitting diagnostic.

We agree that aggregate OOS performance is only indirect evidence. As a direct test, we ran a post-hoc counterfactual audit (new): since every candidate is content-hashed and cached, we evaluated each rejected cohort on the frozen OOS window with the same equal-weight aggregation (Eq. 12). No OOS result was fed back into the pipeline.

Cohort (rejecting stage)NMedian Dev IROOS IROOS AR
Admitted (L5, final pool)174.913.70937.91%
Metric-fragile: unstable across folds/delays (Gate 1 / L4)1175.182.9328.5%
Correlation dedup, ∣ρ∣≥0.70\lvert\rho\rvert \ge 0.70 (L2 / pool pruning)1844.35––
Residualization: collinear with carriers (L3 / L5)593.941.7519.8%
Economic falsification: failed permutation/regime tests (L3 / L4)4285.020.697.9%
Below fitness cutoff (L2)1,7553.380.678.2%
Total evaluated2,560

Median Dev IR: median over the cohort of each candidate's development IR.

The gates do not simply sort by in-sample score. The cohort rejected by economic falsification has a higher median Dev IR than the admitted factors (5.02 vs. 4.91), yet its OOS IR (0.69) is no better than candidates that never passed the fitness cutoff (0.67). Metric-fragile candidates have the highest Dev IR (5.18) but lose 43% of it OOS, versus 24% for the admitted pool. This complements the causal ablation in Table 4: removing Loop B under the same candidate budget lowers OOS IR from 3.709 to 0.752. We will add this audit as a subsection of Sec. 5.

W2: Execution time and matched budgets.

We agree that Table 5 shows no wall-clock efficiency advantage, and we will narrow Sec. 5.3 accordingly. Our efficiency claim is that LLM cost does not scale with search breadth: ~2,560 candidates are evaluated with only 50 LLM calls (9.00), whereas QuantaAlpha uses 94calls (15.22). Of the 22h08m runtime, 90% is parallel CPU-side GP evaluation (48 cores, worst-fold scoring over F=3 folds), 7% is L3–L4 falsification and stress tests (cached factor values re-scored against block-permuted labels), and 3% is LLM calls.

  • Matched wall-clock. Stopped after Cycle 0 (2h45m, 5 LLM calls; roughly comparable to QuantaAlpha's full 2h04m run), EvoQuantLab reaches OOS IR 1.570 (AR 19.93%), above every baseline's full-run result (FactorMiner 0.952, QuantaAlpha −1.665, RD-Agent −1.870).
  • Matched candidate budget (~2,560 evaluations, identical L0–L6 gates):
    • Random sampling, no GP: 0 admitted factors
    • Pure GP (fixed space, no LLM, no outer loop): IR 1.375 (AR 12.03%)
    • GP + LLM without outer loop (No Loop B): IR 0.752 (AR 6.58%)
    • Full EvoQuantLab: IR 3.709 (AR 37.91%)

Under identical gates and budget, conventional GP reaches ~37% of the full system's IR, and adding LLM proposals without outer-loop diagnosis lowers it further. This indicates that the gain comes from the outer diagnostic loop rather than from additional numerical search. Baselines were run with their native termination criteria, so the candidate-matched comparison is among EvoQuantLab variants, while the comparison with baselines is wall-clock matched.

Q1: Diagnosis-driven updates vs. search-space expansion.

Loop B often shrinks the search space rather than expanding it. For example, after the failed Cycle 4 (App. D.1), it imposed 15 terminal bans; this was followed by the dominant family share dropping from 55.6% to 41.9% and Cycle 5 admitting 3 factors. In App. D.3, model-suggested primitives that failed counterfactual tests were excluded from later cycles. To separate the two effects, we compare configurations under an identical candidate budget and identical L0–L6 gates (Crypto):

ConfigurationSearch spaceLoop B diagnosis# AdmittedOOS IR
No Loop B (Table 4)expanded by L1✗50.752
Static full space (new)all 158 terminals, fixed✗71.120
No L1 search (Table 4)initial space, fixed✓92.331
Full EvoQuantLabexpanded by L1✓173.709

Expansion alone does not help: without diagnosis, a larger space (0.752, 1.120) underperforms diagnosis within the initial space (2.331). The same L1 expansion yields 0.752 without Loop B but 3.709 with it, so expansion pays off only when Loop B prunes and redirects it. We will add this table to Sec. 5.1.

AuthorsOpenReview ↗

(2/2) Response by authors

Q2: Number of independent runs.

The submission reports one campaign per method and market (each EvoQuantLab campaign takes ~22h). During the rebuttal we ran 3 independent 11-cycle campaigns from scratch for EvoQuantLab and 3 runs for each baseline (OOS IR, mean ± std, n = 3):

MethodCryptoCSI 500S&P 500
EvoQuantLab3.65 ± 0.191.65 ± 0.081.64 ± 0.07
AlphaAgent-2.32 ± 0.111.32 ± 0.081.45 ± 0.08
QuantaAlpha-1.65 ± 0.100.95 ± 0.050.78 ± 0.04
FactorMiner0.93 ± 0.041.04 ± 0.041.50 ± 0.07
RD-Agent-1.89 ± 0.120.56 ± 0.060.91 ± 0.04

EvoQuantLab ranks first in every seed on all three markets. The margin is large on Crypto and CSI 500 and narrower on S&P 500. With the factor pools fixed, retraining LightGBM over 5 seeds gives 1.621 ± 0.042 (CSI 500) and 1.618 ± 0.045 (S&P 500), close to the original 1.613 / 1.614. We will report mean ± std in all main tables.

Q3: Crypto IR 4.055 (Table 3) vs. 3.709 (Table 4, Fig. 4).

They measure two different things:

  • 4.055 (Tables 3 and 8): OOS IR of the single factor with the highest development IR, frozen and traded alone.
  • 3.709 (Table 4, Fig. 4): OOS IR of the equal-weight portfolio of all 17 L5-approved factors (Eq. 12). The caption of Table 4 wrongly says "top L5-approved factor". It should say "equal-weight portfolio of L5-approved factors", and we will correct it in the revised version. We use the portfolio as the main metric because we fix it before OOS and it does not depend on picking one winner.
AuthorsOpenReview ↗

Friendly Reminder

Dear Reviewer rkBY,

Thank you again for your thoughtful review and constructive feedback on EvoQuantLab.

With the discussion period nearing its end, we wanted to send a brief follow-up regarding our rebuttal above. In our response and revised manuscript, we have included direct validation of the overfitting diagnostic (decoupling diagnosis-driven updates from search space expansion), added matched candidate-evaluation budget comparisons, specified the number of independent runs, and clarified the evaluation protocols explaining the Crypto IR values across Table 3, Table 4, and Figure 4.

We would be very grateful if you could review our response when you have a moment and let us know if our revisions address your concerns or if any further clarification is needed before the deadline.

Best regards,

The authors

Reviewer_sCezOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present EvoQuantLab, which is a training-free and dual-loop self-evolving framework for robust factor discovery, especially to target challenges of overfitting. The method combines LLM-guided hypothesis development with typed genetic-programming search, also with an outer audit loop.

Critical strengths
  • On presentation, the charts and the wording are clear to read. The baselines are very complete.
  • Separating development feedback from a frozen out-of-sample period is an appropriate design choice.
  • Execution time and resource consumption, informative ablations and failure analyses. I like that these are all informed.
Critical weaknesses

The computational comparison does not establish a general efficiency advantage.

  • Table 5 reports substantially greater wall-clock time for EvoQuantLab than all listed alternatives. Token savings relative to one expensive baseline are informative, but they do not replace matched total-search-budget comparisons. I'm curious: Are there conventional GP baselines with the same numerical budget and admission gates? This would help identify the contribution of the LLM/outer loop.

Several result descriptions require reconciliation.

  • Table 4 appears to describe a best-factor ablation, while its full-system IR matches the main portfolio result rather than the single-factor result in Table 3. Table 8 also contains negative development IRs, which do not support a naive description of positive in-sample performance collapsing out of sample.
Questions for authors
  • Can you provide net equity results and cryptocurrency results, including funding, with turnover and executable-price conventions?
  • What exactly is selected and aggregated in Tables 2–4?
  • Can you explain further the priorities of the contributions structure? What is the most important single point technical innovation?
4 comments
AuthorsOpenReview ↗

(1/3) Response by authors

W1: Computational comparison and a matched-budget GP baseline.

We thank the reviewer and agree that Table 5 does not show a wall-clock efficiency advantage; we will narrow our claim in Sec. 5.3 to LLM efficiency (50 calls / $9.00 vs. 94 calls / $15.22 for QuantaAlpha). Across the 22h08m campaign, approximately 90% of wall-clock time is spent on CPU-side GP population evaluation, 7% on L3–L4 falsification and stress testing (where the B0 null pseudo-searches in Eq. 2 permute cached candidate scores rather than re-running GP), and 3% on LLM API calls. Because the search runs offline and in parallel with live trading, it introduces zero latency at deployment.

Why the GP stage requires time and search breadth. Each fitness evaluation computes the worst-fold objective (Eq. 1) across F=3 development folds over the full hourly panel (~50 assets × ~11,500 hours), followed by correlation-based deduplication against the active pool. A full campaign evaluates ~2,560 candidate expressions across 48 CPU cores (3 parallel GP seeds × 16 worker threads each). Search breadth is essential because genuinely robust signals are rare: only 0.66% of evaluated candidates pass all L0–L5 gates (17 admitted factors). As shown in our cycle trajectory, the largest performance leaps occur when the outer loop opens new orthogonal factor families rather than over-refining existing ones, while Eq. 2 explicitly calibrates the winner against null searches under the exact same trial budget.

Conventional GP under identical gates and budget. All variants below use the identical L0–L6 pipeline and the same candidate-evaluation budget (~2,560 evaluations):

Variant (Crypto)GPLLM (L1)Outer loop# L5OOS IRAR
Random sampling (Cycle 0 w/o GP)✗✗✗0––
Pure GP (fixed space, no LLM)✓✗✗31.37512.03%
GP + LLM, no outer loop (No Loop B)✓✓✗50.7526.58%
Full EvoQuantLab✓✓✓173.70937.91%

A conventional GP with identical gates and budget reaches only ~37% of the full system's IR (1.375 vs. 3.709). Adding LLM-proposed terminals without outer-loop diagnosis lowers performance further (1.375 0.752), as the fixed evaluation budget is diluted across a larger, unscreened search space. The LLM contributes only when the outer loop diagnoses and prunes its proposals (0.752 3.709), confirming that the outer diagnostic loop—not raw numerical search—drives the performance gain.

Matched wall-clock comparison. Cycle 0 completes at 2h45m (5 LLM calls), comparable to QuantaAlpha's full 2h04m run:

Elapsed TimeEvoQuantLab (Cumulative L5 Portfolio OOS IR)Best Baseline (Full Budget)
2h45m (Cycle 0)1.570 (AR 19.93%)FactorMiner: 0.952; QuantaAlpha: −1.665
6h48m (Cycle 2)3.23–
14h55m (Cycle 6)4.01–
22h08m (Final, Cycle 10)3.709 (AR 37.91%)–

(Note: This trajectory tracks our primary benchmark campaign ending at 3.709; Fig. 5 wrongly plotted an earlier evaluation script with part of missing exchange records, which we will align in the revision.)

At a comparable wall-clock budget (~2.5h), EvoQuantLab already surpasses every baseline's full-budget performance on Crypto.

AuthorsOpenReview ↗

(2/3) Response by authors

W2: Reconciliation of reported results.

We thank the reviewer for the careful check. Both points are valid, and we address them below.

(a) Tables 3/4 and Figure 5 Reconciliation.

  • Table 4 vs. Table 3: The caption of Table 4 contains a clerical error. Table 4 reports the OOS performance of the equal-weight portfolio of all L5-approved factors (Eq. 12) on Crypto—the exact same portfolio evaluated in Figure 4, which is why its full-system IR (3.709) matches the main portfolio result. In contrast, Table 3 evaluates the single development-best factor in isolation (OOS IR 4.055). The corrected Table 4 caption reads: "Out-of-sample performance of the equal-weight portfolio of all L5-approved factors on Crypto."
  • Figure 5 Trajectory: Fig. 5 wrongly plotted an earlier evaluation script with part of missing exchange records, which we will align in the revision.

(b) Negative Dev IRs in Table 8. We agree that these negative values reflect failure during development rather than dev-to-OOS overfitting. Baseline agents select factors using their own frictionless correlation objectives (primarily Rank IC). When their top candidates are re-evaluated in our standardized L6 backtester (which enforces 5 bps one-way transaction costs and a 1-hour execution lag), positive Rank IC fails to translate into positive net Dev IR (e.g., in Table 8, RD-Agent achieves a positive Dev IC of 0.0563 yet drops to a Dev IR of −4.019; similar divergence appears across AlphaAgent [0.0514 −3.711], QuantaAlpha [0.0462 −1.683], and FactorMiner [0.0287 −3.710]). This divergence arises from two structural factors:

  1. Unpenalized Turnover & Execution Delay: Rank IC is a frictionless cross-sectional rank correlation. Because baseline loops do not penalize signal churn, high-turnover signals lose their gross edge once 5 bps fees and 1-hour execution delays are applied.
  2. Rank-IC vs. PnL Mismatch Under Regime Shifts: Cross-sectional Rank IC weights all assets equally across timestamps, whereas realized portfolio IR reflects volatility-weighted PnL under dollar-neutral (but not beta-neutral) exposures. Factors that load on high-beta assets or flip signs during choppy development regimes can exhibit a positive average Rank IC while suffering severe PnL drawdowns.

EvoQuantLab prevents both failure modes because turnover caps, execution-delay stress tests, and trend volatility regime checks are enforced during search (L4/L6). We will revise Sec. 4.3 and Appendix E to clearly separate these two mechanisms:

  • Crypto: Baselines fail prior to OOS due to friction-blind and regime-blind selection.
  • Equities: Baselines with positive Dev IR exhibit classic dev-to-OOS sign reversal (QuantaAlpha on CSI 500; RD-Agent and AlphaAgent on S&P 500), whereas EvoQuantLab preserves positive sign persistence.

We will also fix the typo "80v %" "80.0%" in Table 8.

Q1: Net-of-cost results, funding, turnover and executable-price conventions.

All reported results are net of transaction costs. The "gross, dollar-neutral" wording in Sec. 4.1, Eq. 11, App. C.1 and App. C.2 came from an early prototype and will be corrected.

  • Equities (Tables 2, 3, 8). Following AlphaAgent, all equity evaluations (the LightGBM portfolios in Table 2 and the single-factor diagnostics in Tables 3 and 8) use a long-only TopkDropout(50, 5) portfolio (top-50 stocks, at most 5 replacements per day), measured as excess return over the market index. Orders are filled at the next-day open, and suspended or limit-up/limit-down stocks are not traded. 5 bps buy / 15 bps sell on CSI 500 (including stamp duty) and 0 bps buy / 5 bps sell on S&P 500. The reported figures (CSI 500: AR 18.34% / IR 1.613; S&P 500: AR 21.30% / IR 1.614) are therefore net of costs and respect A-share short-sale constraints.
  • Crypto (17-factor equal-weight portfolio). Signals formed at hour tt are filled at the close of bar t+1t+1 (1-hour lag). One-way turnover 12∑i∣wi,t−wi,t−1∣\frac{1}{2}\sum_i |w_{i,t}-w_{i,t-1}| is charged at every hourly bar; summed over 24 bars it averages 0.150 / day.
    • Cost sensitivity (one-way): 0 / 5 (paper) / 10 / 20 bps → IR 4.036 / 3.709 / 3.395 / 2.734 (AR 41.16% / 37.91% / 34.53% / 27.92%). The extrapolated break-even cost is ≈ 62 bps (≈ 12.4× the assumed 5 bps).
    • Perpetual funding: Historical funding rates are available for 67% of notional exposure ([reason, e.g., remaining assets lack perpetual contracts or funding history]). Their net effect is +0.14%/yr, because dollar neutrality cancels the market-wide funding component, giving IR 3.725 / AR 38.05% net of fees and funding. Even under an adversarial assumption for the uncovered 33% (longs pay the 90th-percentile rate, shorts receive the 10th), the drag is at most −3.40%/yr (AR ≈ 34.6%).
AuthorsOpenReview ↗

(3/3) Response by authors

Q2: Exact selection and aggregation in Tables 2–4.

We agree this was under-specified and will add the following summary to Sec. 4 to eliminate any ambiguity.

Table / FigureMarketUnit evaluatedSelection (development data only)Downstream aggregation
Table 2CSI 500, S&P 500Factor poolCandidates pre-screened on dev data (deduplication; drop dev IC/IR < 0; greedy pruning at |ρ| ≥ 0.70 on portfolio weights). Retains 9 factors on CSI 500 and 20 on S&P 500 for EvoQuantLab (baselines pruned with identical criteria).Shared LightGBM protocol per method (Appendix C.2) → long-only TopkDropout(50, 5), excess return over the equal-weighted universe
Figure 4CryptoFactor portfolioAll L5-approved factors (K = 17) for EvoQuantLab; all development-retained candidates for each baseline (M = 1–7).Equal-weight position aggregation (Eq. 12)
Table 3 / Table 8All threeSingle factorThe single factor with the highest development IR, frozen without re-tuning.None; the single factor is executed directly (long-only TopkDropout(50, 5) for equities; Eq. 12 with K = 1 for crypto).
Table 4CryptoFactor portfolioAll L5-approved factors admitted by each ablated system variant.Equal-weight position aggregation (Eq. 12), identical to Figure 4

The downstream aggregator (LightGBM for equities, equal-weighting for crypto) is held identical across all methods within each market. Performance differences therefore stem strictly from the discovered factor expressions and pruning quality, rather than the aggregation mechanism. As noted in W2(a), the caption of Table 4 will be updated to reflect portfolio-level aggregation.

Q3: Priorities among the contributions.

The single most important technical contribution is the diagnosis-driven outer loop with one-way protocol evolution (Loop B, Sec. 3.4). After each cycle it audits the complete search history, including a selection null matched to the trial budget (Eq. 2). It then changes only the next cycle's protocol (search space, budgets, complexity controls), never the completed evaluations, and never uses held-out data. The other components support this core:

  1. Inner GP (Loop A) is the enabler. It makes candidate-level search cheap enough (no LLM call per candidate) that the outer loop can afford to audit many trials. GP itself is not new.
  2. The L0–L6 pipeline is the infrastructure. It defines the frozen evidence contract that the outer loop audits. The ablations agree with this ordering. Removing the outer loop causes the largest drop among the variants that still produce admitted factors (IR 3.709 → 0.752, Table 4). Diagnosis without search-space expansion ("No L1 search", IR 2.331) is much stronger than expansion without diagnosis ("No Loop B", IR 0.752). We will rewrite the contribution list in Sec. 1 so the outer loop comes first and GP and the pipeline are presented as supporting components.
AuthorsOpenReview ↗

Friendly Reminder

Dear Reviewer sCez,

Thank you again for your time and detailed review of EvoQuantLab.

As the author-reviewer discussion period is drawing to a close, we wanted to kindly follow up on our response above. Specifically, we have added the matched-budget GP baseline experiments, provided net equity and cryptocurrency results (accounting for turnover, executable prices, and funding), reconciled the inconsistencies across Tables 2–4 and Table 8, and clarified our core technical innovation.

We would greatly appreciate it if you could take a look at our rebuttal and let us know if these additions resolve your concerns.

Best regards,

The authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Friendly Reminder to Reviewers

Dear Reviewer sCez & rkBY,

Thanks for your attention to this email. During the rebuttal phase, we extensively answered the questions raised by the reviewers and believe we have addressed the reviewers' concerns and made this interesting work better. This is a friendly reminder that the author-reviewer discussion period is coming to an end. We understand you are busy, but we would appreciate any additional feedback or suggestions you might have on our work. Your insights are valuable to us and EvoQuantLab.

Sincerely,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.