- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes AlphaPSC, a pool-conditioned generative framework for formulaic alpha mining. The central idea is to condition a GFlowNet-based formula generator on a representation of the existing alpha pool, including a sketched subspace projection, prediction residuals, and pool statistics. Candidate formulas are evaluated using a combination of standalone IC, marginal improvement in ensemble IC, and residual IC. The framework is evaluated on CSI300, CSI500, and CSI1000 against several existing formulaic alpha mining methods and conventional predictive models. The reported results show improvements in predictive correlations and frictionless portfolio metrics. The authors also provide ablation experiments to investigate the effects of pool conditioning and reward components.
Critical strengths
-
Relevant problem formulation. The paper addresses an important limitation of formulaic alpha discovery: high standalone predictive performance does not necessarily translate into incremental predictive value when a factor is added to an existing pool. Explicitly considering the current pool during generation is a reasonable direction.
-
Broad empirical comparisons. The paper compares against several established alpha mining approaches, including AlphaGen, AlphaQCM, and AlphaSAGE, across three equity universes. The inclusion of component ablations and repeated-seed results is also appreciated.
Critical weaknesses
-
Portfolio evaluation. I have concerns about how portfolio returns are computed. Eq. (25) in Appendix B.1 sets the daily return to the average 20-day forward return of the selected stocks divided by 20. The authors treat this as the daily contribution of a staggered portfolio with a 20-day holding period. That isn’t the same as the actual daily mark-to-market PnL of the tranches being held. A real staggered portfolio earns, each day, the one-day price changes of positions opened on different dates. Spreading each tranche’s 20-day return evenly across the holding period smooths away that path. So the resulting series will probably understate volatility and drawdowns, and it will add serial correlation that isn’t really there. All of the annualized returns, Sharpe ratios, maximum drawdowns and cumulative wealth curves (Eqs. 26–29) come from this series, so I don’t think the portfolio results can be taken at face value yet. The backtest also ignores transaction costs and turnover. The authors acknowledge this, but together with the return definition it weakens the practical conclusions a lot. I’d ask the authors to rebuild the return series from daily tranche-level PnL and to report results net of reasonable costs.
-
Attribution and cross-fitting. I’m not convinced the experiments isolate the effect of pool-subspace conditioning. AlphaPSC adds several things at once: pool-relative reward terms, pairwise-correlation and residual-energy gates, and a pool-dependent insertion rule. The insertion threshold of 0.3 in Appendix A.5 alone forces accepted factors to have low pairwise correlation. That means the lower correlations in Table 9 don’t show that conditioning helps find complementary signals. The reward-only ablation in Table 2 is a start. What’s still missing is a comparison with a strong pool-aware generation or selection baseline that uses the same candidate budget, insertion rules, rewards and compute. Given how far apart the ablation variants are, that control seems necessary. Separately, the cross-fitting setup may leak labels. The target is a 20-day forward return, but the embargo is only five days, so training and held-out labels near fold boundaries will overlap. Section 3.4 doesn’t explain how this is handled. Purged cross-fitting with an embargo of at least the prediction horizon would settle the question.
Questions for authors
Portfolio accounting: Can the authors clarify how Eq. (25) reproduces the realized daily returns of a portfolio containing 20 overlapping holding cohorts? Can the main portfolio results be reproduced using a conventional position-level, daily mark-to-market backtest?
Cross-fitting validity: Given the 20-session forward-return target, why is the embargo set to only five days? How are overlapping return-label intervals handled across the training and held-out folds? Would the results remain consistent under a properly purged 20-session separation?
Attribution of improvements: How much of the reported gain remains when AlphaPSC is compared against a pool-unconditioned generator using identical correlation gates, residual-energy gates, reward functions, candidate budgets, and pool insertion procedures?
Computational fairness: What are the actual formula evaluation counts, wall-clock costs, and total search budgets for each baseline? Are the reported performance differences maintained under a matched formula-evaluation budget?
3 comments
Thank you for the detailed rebuttal and the additional experiments. I appreciate the authors' willingness to acknowledge the limitations of the original portfolio accounting and to provide more controlled evidence for the proposed conditioning mechanism. The matched Reward-only ablation substantially addresses my concern about attributing the improvements to pool conditioning. The additional mark-to-market evaluation and HAC-adjusted significance tests also strengthen the empirical evidence.
Nevertheless, two important concerns remain.
First, the original portfolio Sharpe ratio drops from 2.02 to 0.44 under conventional mark-to-market accounting. While AlphaPSC retains a performance advantage over AlphaSAGE on CSI300, this substantially weakens the economic interpretation of the original results. Moreover, the corrected portfolio evaluation remains limited to CSI300 and excludes transaction costs.
Second, the cross-fitting concern is not fully resolved. Rescoring an existing factor pool with a 20-session embargo does not establish that the generator training, factor selection, and final performance would remain unchanged under properly purged cross-fitting. A full re-mining experiment would be needed to verify this.
The revised search-budget disclosure is helpful, although matching training episodes does not establish computational equivalence across methods.
Thank you for the follow-up. We agree that the main-table Sharpe should not be read as tradable daily PnL, and that rescoring an existing pool with a 20-session embargo does not speak for a full re-mine. We add two experiments below.
- Portfolio accounting and costs
Under conventional mark-to-market on CSI300, AlphaPSC Sharpe falls from 2.02 to 0.44. We accept that this weakens the economic reading of the original frictionless numbers. We now run the same mark-to-market protocol on CSI500 and CSI1000, and subtract a one-way 10 bps cost. Turnover is half the L1 change in portfolio weights between consecutive days. On all three universes AlphaPSC still leads AlphaSAGE on annualized return, including after costs. Sharpe is much lower under mark-to-market, so we will stop presenting paper SR and MDD as tradable metrics and report the table below as a supplement.
| Universe | Method | AR | SR | MTM AR | MTM SR | MTM−10bps AR | MTM−10bps SR |
|---|---|---|---|---|---|---|---|
| CSI300 | PSC | 8.65% | 2.02 | 7.88% | 0.44 | 6.98% | 0.39 |
| CSI300 | SAGE | 7.39% | 1.69 | 6.57% | 0.36 | 5.65% | 0.31 |
| CSI500 | PSC | 11.27% | 2.37 | 11.50% | 0.55 | 10.59% | 0.50 |
| CSI500 | SAGE | 10.16% | 2.12 | 10.16% | 0.48 | 9.35% | 0.44 |
| CSI1000 | PSC | 10.61% | 2.12 | 10.66% | 0.49 | 9.80% | 0.45 |
| CSI1000 | SAGE | 9.64% | 1.81 | 10.00% | 0.43 | 9.12% | 0.39 |
We do not claim net tradable profit after costs. The point is only that the relative ranking under one shared protocol does not reverse.
- Cross-fitting and embargo=20 re-mining
Rescoring is not a purged re-train. We re-mined CSI300 pools under embargo=20, with Full AlphaPSC against the matched Reward-only control that keeps the same gates and admission rules and only zeros pool context. The second-stage combiner is unchanged. Results are as follows.
| Variant | IC | ICIR | AR | SR |
|---|---|---|---|---|
| AlphaPSC | 0.051 | 0.356 | 8.95% | 2.08 |
| Reward-only | 0.018 | 0.125 | 3.47% | 0.91 |
Full remains clearly stronger. The numbers are even a bit above the original embargo=5 main result, but a single re-mine is noisy, so we do not claim that a stricter purge is uniformly better. The claim is narrower. After aligning the embargo with the label horizon, the gap from pool-subspace conditioning is still there. We will state the limitation of the default five-day embargo and report this re-mine.
- Search budget
Matching training episodes and pool capacity does not establish computational equivalence across methods. Table 8 reports training schedules and formula-evaluation counts. We did not log clean single-GPU wall-clock uniformly. We do not claim matched compute against every baseline. The AlphaPSC–AlphaSAGE comparison uses a matched budget. Other formulaic miners follow their open-source default schedules and are then re-scored under the same second-stage combiner.
We thank the reviewer for the questions. The points raised under Critical Weaknesses are covered by the questions below, so we answer the questions directly.
Q1. Portfolio accounting: Can Eq. (25) reproduce the realized daily returns of a portfolio with 20 overlapping holding cohorts? Can the main portfolio results be reproduced with a conventional position-level daily mark-to-market backtest?
Eq. (25) does not reproduce, in a pathwise sense, the daily mark-to-market PnL of 20 overlapping cohorts. It attributes the selected top-20% basket’s h-day forward return to the decision day and divides by h. Main-table AR, SR, and MDD are computed on that attributed series under one frictionless protocol. They are not executable daily PnL. On the same CSI300 selection sets we ran a conventional position-level daily mark-to-market backtest. See Table 12. AlphaPSC still leads AlphaSAGE on AR (7.88% vs 6.57% under MTM; 8.65% vs 7.39% under paper Rd), but Sharpe falls sharply (PSC: 2.02 → 0.44). So paper SR and MDD are not tradable daily-PnL metrics. We will clarify Eq. (25) and report MTM as a supplement. We do not claim net tradable profit after costs.
Q2. Cross-fitting validity: Given the 20-session forward-return target, why is the embargo only five days? How are overlapping return-label intervals handled? Would results stay consistent under a purged 20-session separation?
We agree that a five-day embargo is short relative to a 20-session label. Cross-fitting is used only for mining rewards inside the training window. Test labels never enter generation, pool context, or admission. Without re-training, rescoring pool factors with embargo=20 leaves the acquisition scores essentially unchanged. This is not a full re-mine under embargo=20.
Q3. Attribution of improvements: How much of the reported gain remains against a pool-unconditioned generator with identical correlation gates, residual-energy gates, reward functions, candidate budgets, and insertion procedures?
The matched control is Reward-only: same correlation gates, residual-energy gates, pool-relative rewards, candidate budget, and insertion, with cP zeroed for the policy. See Table 2. On CSI300, test IC falls from 0.048 (Full) to 0.013 (Reward-only). ICIR, AR, and SR fall as well (0.326 → 0.095, 8.65% → 3.21%, 2.02 → 0.69). Under the matched pipeline, most of the gap remains attributable to pool-subspace conditioning. The insertion correlation threshold of 0.3 encourages diversity, but it cannot alone explain Full ≫ Reward-only. The pairwise-correlation table is only a diagnostic, not the main evidence for conditioning.
Q4. Computational fairness: What are the formula evaluation counts, wall-clock costs, and total search budgets for each baseline? Do the gaps hold under a matched formula-evaluation budget?
See Table 8. Total search budget means the training schedule. Formula evaluations count candidates that receive a completed panel score at acquisition rounds. Pool capacity is the library size C. AlphaPSC uses 104 episodes with C=50. Every 64 episodes it scores nacq=16 candidates and admits at most one after the gates, so about 2,500 formula evaluations. AlphaSAGE is matched on episodes and capacity and counts one completed formula per episode, hence 104 evaluations. From log timestamps, rough wall-clock on CSI300 is about 0.8–1 hour for AlphaPSC and 0.4–0.6 hour for AlphaSAGE. These are order-of-magnitude estimates, not clean single-GPU timers. The reported PSC–SAGE gaps (CSI300 IC 0.048 vs 0.040) are under that matched budget. Other formulaic miners follow their open-source default schedules and are then re-scored under the same second-stage combiner. We do not claim a matched formula-evaluation count against every baseline. Clean single-GPU wall-clock was not logged uniformly and is marked unavailable in the table.