acceptodds
Under review as a conference paper at ICLR 2027

GAMMA: Market-Grounded Hypothesis Generation and Groupwise Feedback for LLM-Based Alpha Mining

Abstract

Alpha factors are scoring rules that rank assets for portfolio construction. Recent LLM agents automate factor discovery through three stages: (1) economic hypothesis generation, (2) factor implementation, and (3) evaluation and evolution. However, existing methods face two limitations. First, hypothesis generation often relies on the LLM's general financial knowledge without grounding hypotheses in the target market's training data. Second, evaluating a hypothesis through a single factor often leads to prematurely discarding a valid hypothesis, mistaking a poor implementation for a flawed economic rationale. We introduce **GAMMA**, **G**roupwise **A**lpha **M**ining via **M**arket-grounded **A**gents, to address these limitations. During hypothesis generation, GAMMA analyzes the target market's feature distributions, cross-sectional relationships, and temporal dependencies, grounding each hypothesis in statistically verified evidence. During factor implementation, GAMMA constructs a group of diverse factor implementations for each hypothesis. During evaluation and evolution, GAMMA clusters these implementations by stock-ranking similarity, summarizes feedback at the cluster level, and expands successful clusters into refined sub-hypotheses. On CSI 300, GAMMA achieves the highest net annualized excess return of 13.29% and an information ratio of 1.471 among 17 compared methods. CSI 300 factor expressions also retain predictive value when transferred to CSI 500. Additional experiments show that GAMMA remains effective on both CSI 500 and S&P 500, demonstrating its generalizability across Chinese and U.S. equity markets.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Alpha Mining, Agentic AI, Quantitative Finance, Self-evolving Agent, Multi-agent

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_JZ8YOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GAMMA, an LLM-based framework for automated alpha factor mining that addresses two limitations of existing approaches: ungrounded economic hypothesis generation and misaligned credit assignment. GAMMA introduces Active Empirical Grounding (AEG) to extract empirically supported market observations and uses them to generate economic hypotheses. Each hypothesis is implemented through multiple alpha factors, which are evaluated and clustered according to their ranking behavior to guide subsequent exploration. Experiments on the CSI 300 demonstrate strong portfolio performance against 16 baselines, while ablation studies and evaluations on additional markets provide further evidence of the framework's effectiveness.

Critical strengths
  1. Clear Motivation and Well-Structured Framework. The paper identifies meaningful limitations in existing LLM-based alpha mining approaches, particularly the difficulty of distinguishing the validity of an economic hypothesis from the quality of its individual implementations. Generating multiple factor specifications from a single hypothesis and evaluating their ranking behavior at the group level is a reasonable approach. The integration of AEG and groupwise feedback provides a coherent framework with clearly defined components.

  2. Strong Empirical Performance and Meaningful Ablation Results. GAMMA achieves an annualized excess return of 13.29% and an information ratio of 1.471 on the CSI 300, outperforming the reported baselines in portfolio performance. The ablation results in Table 3 show that both AEG and groupwise feedback contribute to performance improvements, with their combination achieving the strongest results. Removing behavioral clustering leads to substantial reductions in performance and search efficiency. These findings provide meaningful evidence of the framework's practical effectiveness.

  3. Detailed Implementation and Evaluation Protocol. The paper provides relatively detailed descriptions of factor construction and validation, including DSL-based expression generation, AST validation, and restrictions on using future information. The chronological data splits and embargo periods are appropriate for financial time series evaluation. The evaluation includes predictive metrics, portfolio performance, factor transfer to the CSI 500, and independent mining experiments in additional markets. These design choices strengthen the empirical evaluation.

Critical weaknesses
  1. Insufficient Validation of Economic Hypotheses and Core Mechanisms.

The paper identifies ungrounded hallucination and misaligned credit assignment as major limitations of existing methods. While GAMMA introduces mechanisms intended to address these issues, the experiments do not sufficiently establish whether the underlying problems are directly mitigated.

AEG statistically validates market observations before hypothesis generation, but the validity of the resulting economic hypotheses HH is not directly evaluated. It remains unclear whether these hypotheses are consistently supported by the empirical evidence or whether GAMMA reduces the generation of ungrounded hypotheses compared with existing approaches.

A similar concern applies to Merge. The method combines economic mechanisms associated with factors that exhibit low behavioral similarity, but low correlation alone does not establish that the resulting hypothesis HmergeH_{\mathrm{merge}} has a sound economic basis. The empirical validity of these merged hypotheses remains insufficiently examined.

The ablation studies demonstrate the practical contributions of AEG and groupwise feedback, but they do not directly establish that hypothesis-level credit assignment becomes more accurate. The independent contribution of Merge is not isolated through a dedicated ablation. Additional evidence connecting the proposed mechanisms to the claimed improvements would strengthen the paper.

  1. Unclear Role of Independent Validation in Factor Selection.

According to Appendix D.2, candidate factors must satisfy predictive performance thresholds on the training period and the combined training and validation period. The selection procedure does not appear to require candidates to satisfy the same thresholds on the validation period independently.

Since the training period is substantially longer than the validation period, a candidate with strong training performance may satisfy the combined threshold even when its predictive performance deteriorates or changes direction on the validation period. This raises questions about how effectively the selection procedure identifies factors that generalize beyond the training data.

Using the combined training and validation period is not inherently inappropriate, and the paper maintains a separate test period. This concern does not imply test data leakage. Nevertheless, the rationale for this selection criterion and its influence on the final factor library would benefit from clarification. A comparison with validation-only screening would help assess the sensitivity of the reported results to this design choice.

  1. Insufficient Control of Search Budgets in Baseline Comparisons.

The paper controls the number of selected factors across automated factor mining methods, using a target library size of 50 factors. However, an equal number of selected factors does not necessarily imply an equal search budget.

Search efficiency is presented as an important contribution of GAMMA, yet the comparisons do not clearly establish whether the major LLM-based baselines operate under comparable token budgets or candidate evaluation budgets. Differences in resource consumption could affect the interpretation of the reported performance improvements.

Table 2 primarily reports results from representative runs, while variability across independent runs is mainly presented for the internal ablation experiments. It is consequently difficult to determine whether the performance advantages over the strongest competing methods remain consistent across different mining runs. Comparisons under matched search budgets, or analyses of performance relative to computational cost, would provide stronger evidence for the claimed efficiency improvements.

  1. Limited Justification of Design Choices and Generalization Claims.

The selection of the thresholds for behavioral clustering and Merge is not sufficiently justified. It remains unclear whether the reported performance and search efficiency are robust to different combinations of these thresholds.

The handling of redundant factors also requires clarification. When a newly generated factor exhibits better predictive performance than an existing library member but is rejected as redundant, the paper does not clearly specify whether the existing factor can be replaced. This design choice may introduce dependence on the order in which factors are discovered.

The cross-market experiments provide encouraging results on the CSI 500. However, the S&P 500 strategy achieves a lower cumulative return than its benchmark even before transaction costs are considered. The evidence therefore supports the applicability of GAMMA across markets more clearly than it supports a claim of consistently strong portfolio performance across different market structures.

Questions for authors
  1. How do the authors verify that the economic hypotheses HH generated by AEG are supported by empirical market observations? Is there direct evidence that GAMMA reduces ungrounded hypothesis generation relative to existing methods? Does the same validation apply to HmergeH_{\mathrm{merge}}? I would also appreciate clarification on whether Merge contributes independently to the reported improvements and whether groupwise feedback leads to more accurate hypothesis-level credit assignment.

  2. What motivates the use of performance thresholds on the training period and the combined training and validation period rather than requiring independent validation performance? Have the authors examined how validation-only screening affects the selected factor library and the final test performance?

  3. How do the search budgets of GAMMA and the major LLM-based baselines compare in terms of token consumption and evaluated candidates? Are the reported improvements maintained under comparable search budgets? It would also be helpful to know whether the strongest baselines were evaluated across multiple independent mining runs and how much their results vary.

  4. How were the thresholds for behavioral clustering and Merge selected, and how sensitive are the results to their values? When a newly generated factor outperforms an existing library member but is classified as redundant, does GAMMA allow the existing factor to be replaced?

3 comments
AuthorsOpenReview ↗

Response to Reviewer JZ8Y

Thank you for your detailed review and constructive questions.

We appreciate your recognition of three strengths that are central to our contribution:

  1. Clear motivation. You recognized the distinction between an economic hypothesis and its implementations. GAMMA develops multiple measurements and organizes their outcomes through behavioral feedback.

  2. Strong performance and informative ablations. You highlighted the primary CSI300 AER of 13.29% and IR of 1.471. The added Table 4 also places GAMMA first in mean AER and IR across three runs per LLM method.

  3. Inspectable implementation. You recognized the DSL constraints, AST checks, chronological splits, and embargoes. Appendices C–D document the research trace and evaluation procedures.

GAMMA also uses behavioral similarity to organize reflection and evolutionary branching, extending its role beyond deduplication. Figure 4/Table 5 adds cross-market comparisons.

We address each weakness together with its related question below, highlighting the mechanism design and additional experimental evidence:

1. Economic hypotheses, Merge, and credit assignment

Weakness 1: The experiments do not directly validate economic hypotheses, isolate Merge, or establish more accurate hypothesis-level credit assignment.

Related Question 1: How are original and merged hypotheses supported, and what directly demonstrates the proposed mechanisms?

Response:

  1. What each evaluation stage checks.

    AEG screens observations, the semantic checker checks hypothesis–specification–expression consistency, and factor admission evaluates predictive strength and redundancy. Appendices D.1 and D.3 distinguish descriptive grounding from predictive evaluation, with verdicts assigned to tested implementations.

  2. Evidence from the case study.

    Appendix C.1/Table 6 traces one hypothesis through different implementations and their feedback. It makes the evidence flow inspectable and complements the aggregate ablations.

  3. How merged hypotheses are evaluated.

    Merge selects less-correlated parents and proposes a combined rationale. New implementations must pass their own predictive evaluation; they do not inherit admission from their parents. Section 3.4 and Appendix D.3 specify this procedure.

  4. What the mechanism experiments establish.

    Table 3 shows complementary gains: full GAMMA reaches mean AER of 12.31%, versus 5.02% for AEG-only and 5.39% for group-only. The clustering ablation further tests feedback organization. Separate no-Merge and direct hallucination/attribution-accuracy measurements are not included.

2. Independent validation and the pooled admission rule

Weakness 2: The longer training period can dominate pooled statistics and mask weak validation performance.

Related Question 2: What motivates the pooled gate, and how does validation-only screening affect the library and portfolio?

Response:

  1. Rationale and role of validation.

    A bull or bear market can persist across much of a short validation window. With less than two years of validation data, an independent hard gate can therefore make admission overly dependent on that window's market regime. Our design treats train + validation as the full in-sample history, with an additional training-only strength requirement, to reduce this regime dependence (Appendix D.2). Validation also informs redundancy checks and clustering. This pooling does not itself introduce test-data leakage: the stated protocol reserves test data for final evaluation, excluding them from factor selection and Ridge fitting.

  2. Validation-only screening results.

    Validation-only thresholds would retain 26/50 factors; 24 fail at least one threshold, while two reverse RankIC direction. This distinguishes signal magnitude from direction. A validation-filtered portfolio rerun is not included.

  3. Reported diagnostics of predictive stability.

    Table 2 reports IC sign-flip rates; Figure 3(a) reports absolute RankIC across splits. Mean absolute RankIC is 0.0313/0.0223/0.0218 for train/validation/test, and 43 factors preserve their RankIC direction throughout. These provide additional stability evidence alongside the admission rule.

  4. Further validation comparisons.

    In future work, we will compare different validation settings under a common evaluation protocol, including pooled screening, an additional independent validation gate, and directional-consistency requirements, examining both the selected library and downstream portfolio performance.

AuthorsOpenReview ↗

Response to Reviewer JZ8Y

3. Repeated mining runs and search-budget comparability

Weakness 3: Equal factor counts do not imply equal resource budgets, and single-run comparisons leave mining variability unclear.

Related Question 3: Are the gains maintained across independent runs and under comparable token or candidate budgets?

Response:

  1. Added independent mining runs.

    Appendix B.1/Table 4 adds three independent CSI300 runs for each of the five LLM methods, reporting means and sample standard deviations:

    MethodRankICNet AER (%)IR
    Vanilla LLM0.0270 ± 0.00213.02 ± 0.470.359 ± 0.033
    AlphaBench-EA0.0529 ± 0.00549.97 ± 0.441.012 ± 0.066
    RD-Agent0.0594 ± 0.00323.63 ± 0.630.431 ± 0.059
    AlphaAgent0.0506 ± 0.00531.51 ± 0.100.179 ± 0.027
    GAMMA0.0632 ± 0.000412.31 ± 1.031.332 ± 0.131
  2. Performance across repeated runs.

    GAMMA retains the highest mean AER and IR, with a 2.34-point mean AER advantage over AlphaBench-EA. The advantage therefore extends beyond the primary-run comparison.

  3. Search efficiency at a common library target.

    Table 3 and Appendix B.4/Figure 6 measure resources to reach the common library target. Full GAMMA uses 653 candidates and achieves 7.69% admission, the best among the ablations. Fixed-token or fixed-candidate comparisons against all external baselines are not reported.

4. Thresholds, replacement, and cross-market interpretation

Weakness 4: Threshold robustness, redundant-factor handling, and the generalization claim need clarification.

Related Question 4: How were clustering and Merge thresholds justified, and can a stronger redundant factor replace an existing member?

Response:

  1. Threshold roles and sensitivity.

    Clustering uses similarity ≥0.60; Merge considers pairs with similarity ≤0.40 (Appendix D.3). Reflection consolidates similar implementations, while Merge explores complementary parents. These are fixed settings; a joint sensitivity grid is not included.

  2. Replacement of stronger redundant factors.

    Yes: a stronger redundant candidate can replace an existing factor. The implementation retains the higher absolute pooled training–validation RankIC and records the replacement. This prevents an earlier, weaker duplicate from automatically blocking a stronger candidate.

  3. Added cross-market evidence and its scope.

    Figure 4/Table 5 adds cross-market baselines. GAMMA leads these methods in AR and terminal wealth: AR is 11.08% on CSI500 and 9.34% on S&P500. This supports comparative discovery performance across markets; Figure 4(b) separately shows the S&P500 index shortfall.

AuthorsOpenReview ↗

Discussion Follow Up

Thank you again for your valuable feedback. We have carefully addressed your concerns in our rebuttal, including additional experimental results and clarifications.

As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal.

We have provided detailed responses to your concerns, along with additional experimental evidence. We would sincerely appreciate your feedback on whether these responses have resolved your main concerns.

Reviewer_Ab7JOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GAMMA combines empirical probing of unlabeled training-market features with multiple factor implementations per economic hypothesis and feedback organized by behavioral clusters. It reports strong CSI 300 portfolio performance, supported by component ablations, alongside transfer to CSI 500 and independent mining on CSI 500 and S&P 500.

Critical strengths
  1. Empirical grounding and behavior-based feedback offer a coherent approach to improving hypothesis generation and reducing reliance on a single implementation.

  2. The common CSI 300 evaluation and three-run component ablations provide useful evidence that grounding and groupwise feedback contribute complementary gains.

Critical weaknesses
  1. Statistical validation remains insufficiently justified: admission uses training and pooled train–validation thresholds without a separate validation gate, while AEG does not report error control across adaptive probes. These gaps weaken confidence in the selected evidence and factors.

  2. Generalization claims exceed the evidence. Cross-universe and cross-market experiments lack competing-method comparisons, and the S&P 500 portfolio finishes below its benchmark despite excluding costs. Transfer results alone also do not rule out common style exposures.

Questions for authors
  1. How many admitted factors fail the predictive thresholds or reverse direction on validation alone? How would a validation-only admission requirement affect performance?

  2. What safeguards address false discoveries across adaptive AEG probes? Please also clarify the ridge fitting period and whether preprocessing and fitted weights are frozen during CSI 500 transfer.

  3. What evidence supports cross-market robustness beyond positive absolute returns, given Figure 4(b)? Matched baseline comparisons and exposure-controlled diagnostics would materially affect my assessment.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Ab7J

Thank you for your thoughtful review and constructive suggestions.

We appreciate your recognition of two central strengths of GAMMA:

  1. A coherent research process. As you recognized, GAMMA connects empirical grounding, multiple implementations, and behavioral feedback to guide hypothesis exploration.

  2. Complementary component benefits. You highlighted the common CSI300 evaluation and three-run ablations. Table 3 shows mean net AER of 12.31% for full GAMMA, versus 5.02% for AEG-only and 5.39% for group-only.

The revision adds cross-market baseline comparisons (Figure 4/Table 5) and three CSI300 runs per LLM method (Table 4).

We address your weaknesses and related questions together below, with pointers to the additional results and clarifications in the revised manuscript:

1. Independent validation in factor admission

Weakness 1: Training and pooled training–validation thresholds may admit factors that do not generalize to validation.

Related Question 1: How many admitted factors fail validation-only thresholds or reverse direction, and what would an independent validation gate change?

Response:

  1. Rationale for the pooled gate.

    Bull or bear market conditions can persist for extended periods, so our validation window of less than two years (January 2020–November 2021) may disproportionately reflect one market regime. A separate hard gate can consequently favor signals suited to that regime and exclude factors useful under other conditions. We therefore treat training and validation together as the full in-sample period, while also requiring training-only predictive strength (Appendix D.2). This broader history helps reduce dependence on a single regime. Pooling these in-sample segments does not itself introduce test-data leakage: under the stated protocol, test data are reserved for final evaluation and do not enter factor selection or Ridge fitting.

  2. Validation-only diagnostics.

    A diagnostic of the frozen primary-run library gives the following counts:

    Validation-only diagnosticFactors out of 50
    Validation absolute RankIC < 0.0224
    Validation absolute RankICIR < 0.1015
    Fail at least one threshold24
    Pass both thresholds26
    Reverse mean RankIC direction from training to validation2

    These groups overlap: 26 factors pass both validation thresholds, and only two reverse direction. The counts describe the frozen library; a portfolio rerun after filtering is not included.

  3. Additional stability evidence in Table 2 and Figure 3.

    Table 2 reports IC sign-flip rates, and Figure 3(a) shows split-specific absolute RankIC, with means of 0.0313, 0.0223, and 0.0218. Moreover, 43/50 factors retain their RankIC direction across all three splits. These provide complementary references for temporal stability.

  4. Further validation comparisons.

    In future work, we will systematically compare alternative validation settings, including pooled screening, an additional independent validation gate, and directional-consistency requirements, under a common evaluation protocol to assess their effects on factor selection and portfolio performance.

2. False discoveries in adaptive empirical grounding

Weakness 1: AEG does not report error control across adaptive probes.

Related Question 2, first part: What safeguards address false discoveries?

Response:

  1. AEG safeguards and their purpose.

    Appendix D.1 restricts AEG to unlabeled training features. Retained measurements require at least 500 dates, a median cross-section of 50 assets, consistent direction in four of five time blocks, and a 95% moving-block-bootstrap interval excluding zero. These checks screen observations before they enter research memory.

  2. Separate screening of observations and predictive factors.

    AEG provides evidence for proposing hypotheses; each resulting factor separately passes predictive-strength and nonredundancy checks before library evaluation on the test period. Bootstrap intervals characterize individual measurements, rather than an overall adaptive-search error rate. Descriptive grounding and predictive evaluation therefore have distinct roles.

AuthorsOpenReview ↗

Response to Reviewer Ab7J

3. Ridge fitting and the meaning of transfer

Related Question 2, second part: What is the Ridge fitting period, and are preprocessing and fitted weights frozen during CSI500 transfer?

Response:

  1. Ridge fitting and transferred components.

    Ridge uses training data only (January 2010–November 2019), with regularization 10 and no intercept (Appendix A.1). CSI500 transfer reuses the expressions and refits Ridge on CSI500 training data; it does not reuse the CSI300 coefficient vector.

  2. Preprocessing and interpretation of transfer.

    The preprocessing rules remain fixed, with daily cross-sectional ranks recomputed in the target universe. This tests factor-library transfer without target-universe formula mining. Results appear in Section 4.4, Figure 4(a), and Appendix B.3/Table 5.

4. Cross-market comparisons and the scope of generalization

Weakness 2: Missing competing methods, S&P500 benchmark underperformance, and possible common style exposures weaken the generalization claim.

Related Question 3: What supports cross-market robustness beyond positive absolute returns?

Response:

  1. Added cross-market comparisons.

    Figure 4/Section 4.4 now compare five LLM methods on CSI500 and S&P500. Appendix B.3/Table 5 provides the complete CSI500 metrics.

    MethodCSI500 AR (%)CSI500 AER (%)S&P500 AR, zero cost (%)
    Vanilla LLM5.973.234.63
    AlphaBench-EA2.990.258.29
    RD-Agent10.037.297.48
    AlphaAgent6.263.515.01
    GAMMA11.088.349.34
  2. CSI500 performance and transfer.

    After transaction costs, GAMMA achieves AER 8.34% and MDD 24.95%, versus 7.29% and 34.88% for RD-Agent; RD-Agent has the higher IR (0.725 versus 0.673). The transferred GAMMA library achieves AER 6.99%, supporting usefulness beyond the source universe.

    See Figure 4(a–b) in the revised manuscript for the wealth curves and index benchmarks. CSI500 curves include transaction costs; S&P500 curves use zero-cost evaluation.

  3. What the new comparisons add.

    GAMMA leads the evaluated LLM methods in AR and terminal wealth in both markets, including a 1.05-point AR advantage over AlphaBench-EA on S&P500. Figure 4(b) also shows its shortfall against the index. Transfer supports expression portability; a style-controlled attribution is not reported.

AuthorsOpenReview ↗

Discussion Follow Up

Thank you again for your valuable feedback. We have carefully addressed your concerns in our rebuttal, including additional experimental results and clarifications.

As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal.

We have provided detailed responses to your concerns, along with additional experimental evidence. We would sincerely appreciate your feedback on whether these responses have resolved your main concerns.

Comments on the paper

6 comments
AuthorsOpenReview ↗

Response to Reviewer Ux7P

Thank you for your careful review and concrete suggestions.

We appreciate your recognition of two important strengths of the work:

  1. Hypothesis–implementation separation. You recognized the value of testing several formulas before drawing feedback about an idea. Appendix C makes this process inspectable.

  2. Evaluation of variability and cost. You highlighted the three-run ablation, token reporting, and common CSI300 evaluator, which assess both discovery efficiency and portfolio outcomes.

The revision adds RankIC in Table 3, three-run LLM comparisons in Table 4, and cross-market baselines in Figure 4. GAMMA leads Table 4 in both mean RankIC and mean AER.

We address the related weaknesses and questions together below, with additional signal metrics, repeated-run comparisons, and period-exclusion diagnostics:

1. Signal quality, portfolio outcomes, and statistical support

Weakness 1: Return gains are not sufficiently supported by signal metrics or statistical evidence, and the primary run exceeds the three-run mean.

Related Question 1: Why does AEG increase excess return when RankICIR does not improve?

Response:

  1. Added signal-level evidence.

    Table 3 now includes RankIC alongside RankICIR and portfolio performance (three-run means):

    VariantRankICRankICIRNet AER (%)
    Base0.05460.28032.63
    AEG only0.05630.27695.02
    Group only0.05840.28345.39
    Full GAMMA0.06320.322012.31

    Full GAMMA improves both RankIC and RankICIR. AEG-only improves RankIC from 0.0546 to 0.0563 while RankICIR remains similar. Because RankICIR divides mean RankIC by its daily standard deviation, an unchanged ratio does not imply an unchanged mean signal.

  2. Why RankIC and portfolio returns can diverge.

    RankIC measures full-cross-section ordering; TopkDropout selects the top 50 assets under turnover and execution constraints (Section 4.2). Table 2 shows higher RankIC for DSO and higher RankICIR for GRU, yet higher AER for GAMMA. Different metric rankings do not, by themselves, demonstrate portfolio noise.

  3. Complementary factor-library diagnostics.

    Figure 7 reports mean/median absolute factor correlations of 0.157/0.117, with effective rank 27.3/50. These results support the intended diversity of ranking behavior and limited pairwise redundancy.

  4. Added repeated-run comparisons.

    Table 4 adds three runs per LLM method. GAMMA achieves mean AER 12.31% ± 1.03% versus 9.97% ± 0.44% for AlphaBench-EA, and IR 1.332 versus 1.012. This is a repeated-run average advantage; a primary-run block-bootstrap interval spans zero, so it is not a formal significance claim.

2. Dependence on the late-2024 rally

Weakness 1: The large wealth-curve movement around late 2024 may drive the main result.

Related Question 2: What happens when the late-2024 rally is excluded?

Response:

  1. Annual evidence and period-exclusion results.

    Figure 5 shows positive benchmark-relative performance in every test year. The following additional diagnostic excludes specific windows from saved daily returns:

    Excluded windowGAMMA net AER (%)AlphaBench-EA net AER (%)
    None13.299.54
    September 24–October 8, 202416.278.69
    September 24–December 31, 202415.7711.01
    Fourth quarter of 202413.5111.20
  2. Interpretation of the excluded-window results.

    Excluding late September–December 2024, GAMMA retains AER of 15.77% versus 11.01% for AlphaBench-EA. Its advantage also survives the shorter exclusions, addressing the rally-dependence concern.

  3. Scope of the sensitivity calculation.

    This is post-hoc analysis of frozen returns, not a strategy rerun. The exclusion table is additional rebuttal evidence, distinct from Figure 5's annual breakdown.

AuthorsOpenReview ↗

Response to Reviewer Ux7P

3. Search costs and the stopping target

Weakness 2: The token totals are difficult to interpret, particularly the difference between AEG-only and full GAMMA.

Related Question 3: What is the search budget and stopping criterion, and why can the full system be cheaper?

Response:

  1. Completion target and token accounting.

    Table 3 measures total discovery costs to reach the 50-factor library target. Tokens cover the complete search trajectory, not one execution of each module; adding groupwise search can therefore reduce total cost.

  2. Why groupwise search can reduce total cost.

    Groupwise generation reuses a hypothesis across implementations, while cluster-level feedback guides further exploration. This can reduce repeated hypothesis formation and redundant feedback. Table 3 reports:

    VariantEvaluated candidatesAdmission rate (%)Tokens (M)
    Base9015.5816.50
    AEG only8446.0530.30
    Group only7107.097.65
    Full GAMMA6537.698.55
  3. Magnitude and interpretation of the cost reduction.

    Full GAMMA uses 72% fewer tokens and 23% fewer candidates than AEG-only (Table 3; Figure 6). Different call costs for grounding, construction, and feedback explain why token consumption need not scale directly with candidate count.

  4. How to interpret the efficiency comparison.

    The comparison measures cost to complete the same library-building task. A fixed-token comparison and a common numerical resource cap across all methods are not reported.

4. S&P500 benchmark performance and transaction costs

Weakness 2: S&P500 underperformance is presented as evidence of cross-market success.

Related Question 4: What are the benchmark-relative results, and are transaction-cost-adjusted results available?

Response:

  1. Different market environments.

    The more mature and extensively researched U.S. market may offer fewer readily exploitable short-horizon opportunities than the A-share market. This is a plausible context for the performance difference, consistent with prior evidence of stronger return predictability in China ("Machine learning in the Chinese stock market," Journal of Financial Economics, 2022). We therefore evaluate GAMMA against competing discovery methods within each market.

  2. Different available inputs.

    S&P500 mining uses only OHLCV (five fields), whereas CSI300/CSI500 use OHLCV+VWAP (six fields). The U.S. setting therefore has a narrower input vocabulary: VWAP-relative signals available to the A-share experiments cannot be constructed directly. This data difference is another relevant consideration when interpreting performance across markets.

  3. Added evidence for cross-market discovery.

    Figure 4(b) adds four LLM baselines. GAMMA achieves AR 9.34%, versus 8.29% for AlphaBench-EA, 7.48% for RD-Agent, 5.01% for AlphaAgent, and 4.63% for Vanilla LLM. Together with CSI500, this extends the method-relative comparison beyond CSI300. GAMMA is the strongest LLM-based method for alpha mining.

5. Historical knowledge in the language model

Weakness 2: A language model available after the backtest period may encode hindsight relevant to hypothesis generation.

Response:

  1. Controls on experimental data access.

    Appendix A.1 specifies chronological splits and training-feature-only grounding. Factors use backward-looking expressions, and the completed library is evaluated on the held-out test period.

  2. Comparability and retrospective interpretation.

    All LLM methods share the same backbone, supporting comparison of their discovery procedures. Parameter knowledge may still contain post-period information; runtime restrictions do not eliminate this possibility. The evaluation is retrospective.

6. Coverage of closely related methods

Weakness 2: QuantaAlpha and CogAlpha are missing from the empirical comparison.

Response:

  1. CogAlpha

    CogAlpha does not provide a publicly available codebase, so we exclude it from our evaluation.

  2. Additional QuantaAlpha experiment.

    QuantaAlpha's codebase has stability issues and remains subject to some controversy. Therefore, we exclude it from our main experimental comparison (Table 2). Nevertheless, we conducted an additional experiment on CSI 300, with the results presented below.

    MethodRankICRankICIRNet AR (%)Net AER (%)IR
    QuantaAlpha0.03480.21825.005.080.596
    GAMMA0.06370.334013.2113.291.471

    QuantaAlpha ranks third by net AER among the six LLM methods when included alongside Table 2. GAMMA remains first, ahead by 8.21 percentage points in net AER and 0.875 in IR. Thus, the revision provides an explicit QuantaAlpha result in addition to the expanded repeated-run and cross-market comparisons.

AuthorsOpenReview ↗

Discussion Follow Up

Thank you again for your valuable feedback. We have carefully addressed your concerns in our rebuttal, including additional experimental results and clarifications.

As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal.

We have provided detailed responses to your concerns, along with additional experimental evidence. We would sincerely appreciate your feedback on whether these responses have resolved your main concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.