acceptodds
Under review as a conference paper at ICLR 2027

When Do Graph Neural Networks Help in Cross-Sectional Stock Ranking? A Multi-Seed, Multi-Universe, Cost-Aware Study of the U.S. S&P 500

Abstract

Papers often report that graph neural networks (GNNs) improve cross-sectional stock ranking by encoding relations among stocks as graph edges. Many reported gains, however, come from single-seed, single-split evaluations that do not adjust for multiple testing. We ask a narrower question: when, if at all, does the graph help? We evaluate an equal-budget tuned ladder from LightGBM to a heterogeneous relational GNN (L0–L7) over twelve expanding quarterly walk-forward folds and ten seeds on the U.S. S&P 500, with two pre-registered confirmatory families. The predictive family asks whether any tuned model beats tuned LightGBM. Hansen's SPA test does not reject in either feature universe ( and ), and the benchmark comparisons are underpowered, so the result is a non-rejection and does not establish equality. Local DM contrasts place the observable signal in the non-graph MLP, and the best-supported contrast runs *against* the correlation graph. Adding a correlation-GAT to the MLP reduces IC (); this penalty is BH-significant in *both* feature universes, and its sign holds when we drop any single quarter. The penalty is specific to that graph at its tuned operating point: in Universe C the sector, full-attention, HATS, and combined-edge arms all recover above the correlation-GAT, though Universe-C positives rest on a leak-selected basis. The MLP's gain over LightGBM () is significant only in the leak-selected universe and falls below its detectable-effect threshold, so we read it as suggestive. The edge-attribution family freezes the tuned correlation-GAT operating point and varies only the edge set. None of six edge contrasts survives Benjamini–Hochberg control, and all six are underpowered. A transaction-cost crosswalk (each IC contrast re-expressed as net Sharpe at 10 basis points) points the same way. The two Universe-C ladder contrasts we examine most closely (MLP - LightGBM and correlation-GAT - MLP) survive 10 bps; the tuned news-edge IC underperformance is near zero in net Sharpe, fragile across folds, and absent in the fixed-operating-point test. We therefore report conditional findings and failure modes.

Primary area:
learning on time series and dynamical systems
Keywords:
graph neural networks, cross-sectional stock ranking, financial time series, evaluation methodology, multiple hypothesis testing, superior predictive ability, transaction costs, negative results

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9tq9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors compare stock-ranking models over twelve quarterly walk-forward folds and ten seeds. They use selection-adjusted tests and fixed-hyperparameter graph interventions. The tuned correlation-GAT has lower IC than the MLP. After selection adjustment, no model is established as superior to LightGBM.

Critical strengths
  1. Purging, multiple seeds, and multiplicity corrections make the study more informative than a single tuned comparison. The 90-trial sensitivity analysis, BY correction, and lag-21 check strengthen the result for the tested correlation-GAT setting.

  2. The negative findings have value. The paper reports GraphSAGE collapse, gains concentrated in a few quarters, and a news-edge IC improvement that does not carry over to Sharpe. These results help explain the limits of adding graph information in this setting.

Critical weaknesses
  1. Universe B is described as leak-free, but Appendix C.1, p.18, lines 946–948, says a diagnostic on 63 days within the confirmatory test span informed adding volume features. Test observations therefore contributed to feature selection. Freezing the features before a later run does not remove this dependence. Shared features may affect models differently; the resulting bias is not quantified.

  2. Section 3.2 estimates Sharpe from about three returns per fold-seed cell and averages the ratios. These volatility estimates are unstable, and their average is not the Sharpe of a chronological strategy. Bootstrapping twelve fold differences does not address that distinction. The cost conclusions in the abstract and Section 5.3 therefore need a clearer basis. The reported 16.3% composition mismatch in the non-PIT panel also limits interpretation of portfolio and graph results.

Questions for authors
  1. Which feature and model decisions preceded inspection of any test-window results? Is there an existing price-only result or untouched segment supporting the correlation-GAT versus MLP comparison?

  2. Do the cost conclusions hold for stitched chronological portfolio returns? Please use the existing outputs and state how seeds and portfolio costs are handled.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 9tq9 (1/2)

Part 1 — W1 / Q1: Universe B and the decisions that preceded test-window results

Thank you for reading Appendix C.1 this closely. You are right: the main text described Universe B more strongly than C.1 supported. While re-checking the project log for this response, we also found two exposures that the submission did not state. The revision reports both:

  • A May 2026 exploratory ranking (Plan-AAA, the source of the Universe-C basis) also scored all ten Universe-B features on evaluation-window quarters. This ranking did not change the B basis, but the submission's statements that these features were "never ranked on this sample" (§4) and "never ranked on the confirmatory sample" (Appendix C.1) were not accurate.
  • Before the protocol freeze, early single-split experiments (Feb–Mar 2026) and exploratory five-fold work (Apr–May 2026) had used 2024 Q1–2025 Q4 (folds 4–11) as validation or test quarters.

What changed. We removed every "leak-free". §4 now states that B's basis (ten standard price-volume features) was fixed in April 2026 and not changed afterwards, and that this basis is not free of evaluation-window information. Specifically, the 63-day diagnostic (2024 Q2) scored six of these features and informed the addition of the two volume features, and the later ranking scored all ten. We designated B as primary after the Universe-C leakage audit and after the confirmatory run. Limitation L1 now calls B "our closest comparison outside that ranking" and says that it does not bound the selection-induced inflation in Universe C. §5.2 no longer argues that selection cannot affect the negative graph contrast.

Q1: which decisions came before test-window results. Appendix E.1 (Table 22) dates every step against the test quarters whose realized labels it used. In short:

  • Feb–Mar: early single splits (validation 2024 H1 with test from 2024 Q3; later, validation 2025 H1 with test 2025 H2): folds 4–11.
  • Apr 16: single-feature diagnostic on 2024 Q2 (fold 5). Universe-B basis fixed in April.
  • Apr–May: exploratory five-fold work and the Plan-AAA ranking (folds 5–9). Universe-C basis selected from that ranking.
  • Jun 12: protocol frozen (12 folds, ladder L0–L7, Family-1).
  • Jun 13–15: untuned 12-fold pilots evaluated all twelve quarters.
  • Jun 15 / Jun 16: search space frozen / tuned hyperparameters frozen (md5 59ddd0a2). Tuning used training data through June 2022, validation data from 2022 H2, and no test quarter.
  • Jun 17: Family-2 fixed-operating-point arms added; confirmatory run launched (completed Jun 20–21).
  • Jun 26 and 30: Universe B designated primary.

Is any segment untouched? No. The 2023 quarters (folds 0–3) were never used for validation or testing before the freeze, but the untuned pilots evaluated them before the hyperparameters were frozen and before Family-2 was added. We therefore present no segment as a holdout for the final protocol, and we agree that this design cannot quantify the selection bias you describe.

Price-only check. We added a targeted sensitivity, Universe B8, which deletes B's two volume features (dollar volume and the 5-day price-volume correlation), so the remaining eight features use prices only. With the frozen Universe-B hyperparameters, L2−L1 is −0.0181 [−0.0310, −0.0065] (11/12 folds negative), so the negative contrast remains after this deletion. The change in this contrast relative to B is −0.0051 [−0.0164, +0.0062], so we cannot say whether the magnitude changed. B8 still contains six features that the diagnostic scored, so this subset does not remove the dependence on evaluation-informed selection. The same deletion weakens the MLP−LightGBM contrast from +0.0143 to +0.0021, mainly because LightGBM's pooled IC rises (0.0228 → 0.0358). This fits the paper's reading of MLP > LightGBM as suggestive only (Appendix E.6, Limitation L1).

AuthorsOpenReview ↗

Response to Reviewer 9tq9 (2/2)

Part 2 — W2 / Q2: Sharpe, stitched portfolios, and composition

We agree. A Sharpe ratio from about three returns says little, an average of such ratios is not the Sharpe ratio of any strategy, and bootstrapping fold differences does not fix that. The confirmatory run saved per-cell metrics and daily IC but no predictions or portfolio returns, so we re-ran the confirmatory cells with prediction logging, using the frozen hyperparameters and the same ten seeds. L0 and L1, re-run on the confirmatory hardware, reproduce the confirmatory daily IC bit for bit in all 480 cells. L2, L3 and L5 were re-run on an A100 (confirmatory: T4); their pooled IC moves by at most 0.0004, except C L5 (−0.0017), which we use descriptively only (Appendix E.2).

How the portfolio is built (§3.2, Appendix E.3). One 21-trading-day rebalance grid starting 2023-01-03 gives 35 complete holding periods that end 2025-12-08. At each rebalance we go long the top 50 and short the bottom 50 of the 501 names (equal weight, dollar neutral), using the predictions of the fold whose test window contains that date. Positions carry across fold boundaries, eligibility uses entry-date information only, and positions are marked daily for drawdowns. Seeds: each seed gives one portfolio. Paired ΔSharpe is the seed mean of the per-seed Sharpe differences. The 95% interval comes from a stationary bootstrap over periods (mean block 3, 5000 draws) that resamples the same periods for both arms and all seeds, so the interval is conditional on the ten fitted seeds. Costs: net return = gross − τ·c/10⁴, charged at every rebalance, where τ is the L1 change in the two books' equal target weights (4 at the first entry). τ is a turnover proxy: it ignores weight drift between rebalances. There are no financing or borrow costs.

Results at 10 bps (Table 4).

PairGross ΔICNet ΔSharpe [95% CI]Range over 21 phases (# positive)Submission
C L1−L0+0.0148+0.29 [−0.51, +0.88]−0.45 to +0.44 (15)+1.17
C L2−L1−0.0119−0.02 [−0.20, +0.17]−0.35 to +0.02 (1)−0.72
C L3−L2−0.0123−0.41 [−0.74, −0.10]−0.68 to −0.23 (0)+0.08
C L5−L3+0.0275+0.44 [+0.16, +0.74]+0.22 to +1.09 (21)+0.88
B L1−L0+0.0143+0.46 [−0.38, +1.26]−0.13 to +0.62 (19)+0.43
B L2−L1−0.0133−0.41 [−0.65, −0.24]−0.42 to −0.02 (0)−0.14

The primary estimate uses 35 holding periods; each phase-sweep portfolio uses 34, so that all 21 rebalance offsets can be compared. The two Universe-C contrasts that the submission said "survive 10 bps" now have intervals that include zero. Applied to the same re-executed cells, the submission's averaging gives +1.17 and −0.64 (submission: +1.17 and −0.72), so the change comes mainly from the estimator, with some re-execution drift. We withdraw the statement that these contrasts survive costs. We also withdraw the submission's statements that the tuned news-edge IC loss is "near zero in net Sharpe" (abstract) and that the news-edge rung "matches" the correlation-GAT in net Sharpe (§5.3, +0.08): with the stitched portfolio, the news-edge rung is lower (−0.41 [−0.74, −0.10]). For Universe B, the submission reported a cell-averaged L2−L1 of −0.14 with an interval containing zero; the stitched estimate is −0.41 [−0.65, −0.24] and is negative at every rebalance phase. Timing matters: C L1−L0 changes sign across phases. Bonferroni adjustment over the six contrasts keeps B L2−L1 and C L5−L3 away from zero but not C L3−L2. A 36th period, the historical-membership restriction, other block lengths, and a tie-robust treatment of LightGBM's tied predictions change no sign and no interval's relation to zero (Appendix E.3). Net Sharpe remains descriptive.

Composition (16.3%). §4 now reports the survivorship audit. Removing, on each date, the snapshot names that had not yet joined the index (50 at the first test date) lowers every arm's pooled IC by 0.004–0.012. Most paired contrasts change little. Universe-B L2−L1 weakens from −0.0130 to −0.0086 [−0.0176, +0.0005] (unadjusted HLN p between 0.02 and 0.16, depending on the HAC kernel and bandwidth). Universe-C L2−L1 is −0.0113 [−0.0177, −0.0049]. C L3−L2, whose full-snapshot interval only just excludes zero, moves from −0.0124 to −0.0101 [−0.0221, +0.0009] (Appendix E.4). The restriction cannot restore the 87 departed firms, which are also missing as graph neighbours, so survivorship remains a limitation whose effect on graph-versus-non-graph comparisons we cannot sign (L8).

Reviewer_NBYpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether stock relation graphs provide additional predictive value for cross-sectional stock ranking. It compares LightGBM, MLP, GAT, GraphSAGE, and several graph configurations on an S&P 500 stock universe. Through walk-forward evaluation, multiple-testing correction, and transaction cost analysis, the authors examine the contributions of neural modeling, graph structure, and hyperparameter selection. The clearest finding is that the correlation-based GAT underperforms the MLP under the evaluated settings, while no overall advantage over LightGBM is established.

Critical strengths
  1. Whether stock relation graphs add predictive information is a worthwhile question with practical implications for model selection.

  2. The experimental analysis is fairly extensive, including walk-forward evaluation, transaction cost analysis, and several statistical tests, and the authors also report problems encountered during the experiments.

Critical weaknesses
  1. The evaluated graph models are mainly older basic architectures, and HATS is only a simplified adaptation. More recent approaches, such as MDGNN and FactorGCL, are not included. These methods differ substantially in relation construction and temporal modeling, so the current comparisons provide limited coverage of graph-based approaches to stock prediction.

  2. The core correlation graph uses a fixed 126-day window and an absolute correlation threshold of 0.6. The graph from the end of training remains frozen throughout validation and testing, without examining graph updates or construction parameters. The GAT’s disadvantage may partly reflect stale or unsuitable relations, and the current experiments do not clearly separate these effects.

  3. All experiments use a single fixed S&P 500 stock universe, while Universes B and C are simply two feature sets for the same stocks. Large-cap stocks have distinct liquidity, information coverage, and relational structures, and generalization to other U.S. stocks or other markets remains untested. The end-of-period survivor snapshot also introduces membership selection and survivorship biases that may affect model comparisons.

  4. The backtest mainly reports Sharpe ratios, without cumulative or annualized returns, maximum drawdown, or full equity curves. More importantly, Sharpe is computed separately from only about three holding-period returns per quarter and then averaged across quarters and runs. These short-window estimates are unstable and do not adequately describe the strategy’s full out-of-sample performance.

  5. The strongest evidence concerns a correlation-based GAT underperforming an MLP within a particular stock universe and configuration. The paper does not yet establish when graphs reliably help or which factors determine their success or failure. Several edge comparisons are also underpowered, leaving these questions unresolved.

Questions for authors
  1. Do the main findings hold when representative dynamic-relation or hypergraph stock models, such as MDGNN or FactorGCL, are included?

  2. Does the GAT’s disadvantage relative to the MLP persist when the correlation graph is updated during testing, or when the correlation window and edge threshold change?

  3. Do the findings hold for a broader U.S. stock universe or another market? Would the core comparisons change when using historical point-in-time membership and including firms that exited the universe?

  4. Could the authors report full-period Sharpe, net returns, maximum drawdown, and equity curves based on the complete out-of-sample return series? Do the economic comparisons between models still hold under this evaluation?

3 comments
AuthorsOpenReview ↗

Response to Reviewer NBYp (1/2)

Part 1 — W1/Q1 (newer graph models) and W2/Q2 (graph updates and construction parameters)

Thank you for the review. Your questions on graph updates and backtests led to new analyses that changed the paper, and we answer them with numbers.

W1/Q1: MDGNN, FactorGCL and other recent models. We did not add these models, and Limitation L6 says so instead of claiming coverage. Family-1 is a pre-registered SPA test over a fixed candidate set (nine tuned arms against tuned LightGBM, frozen in June 2026). Adding arms after seeing the results would change that family and its multiplicity control. A fair addition would also need the same 30-trial tuning, ten seeds and twelve folds in both universes. Stronger models could change the Family-1 answer, and we do not claim otherwise. The headline L2−L1 contrast compares the tuned correlation-GAT with the MLP on the same features; adding other candidate models would not change this pair's point estimate. L6 (full self-attention, no mask) is our proxy for dense learned attention, and L7 is a three-relation simplification of HATS. In the "What we do not claim" box (§1), we state that we do not claim graphs or GNNs hurt in general, or that dynamic or learned graphs would. We will release the code, frozen configurations, and analysis scripts with the camera-ready version, so newer models can be run under the same protocol.

W2/Q2: graph updates, window, and threshold. We ran all three sensitivities you asked for, in both feature universes. Each retrains L2 at its universe's frozen hyperparameters, which were tuned for the frozen graph. Intervals are unadjusted 95% stationary-bootstrap intervals, conditional on the ten seeds (Appendix E.5, Table 27).

VariantB: L2−L1 [95% CI]C: L2−L1 [95% CI]
Baseline (126 days, |ρ|>0.6, frozen at end of training)−0.0130 [−0.0223, −0.0041]−0.0117 [−0.0177, −0.0055]
Dynamic: latest trailing snapshot each day (snapshots every 21 days)−0.0033 [−0.0127, +0.0058]−0.0049 [−0.0117, +0.0018]
63-day window−0.0111 [−0.0215, −0.0011]−0.0094 [−0.0173, −0.0016]
Threshold 0.4−0.0193 [−0.0310, −0.0078]−0.0024 [−0.0105, +0.0060]

Your concern is partly borne out. With dynamic snapshots the penalty weakens in both universes: both intervals include zero, but they also allow a remaining negative effect. Relative to baseline L2, the dynamic policy raises IC by +0.0097 [+0.0008, +0.0193] in B and +0.0068 [+0.0007, +0.0128] in C. This policy changes the training and validation graphs as well as the test graph, and it removes an overlap in which the end-of-training snapshot is partly estimated from returns inside some earlier training days' label windows. Consequently, the dynamic policy does not isolate test-time staleness. A 63-day window leaves the contrast negative in both universes. The 0.4 threshold raises the mean degree from about 33 to 144 and moves L2 in opposite directions in the two universes (−0.0063 in B, interval containing zero; +0.0093 in C).

Retrospective diagnostics measure the staleness. At test rebalance dates the frozen snapshot is on average 106–137 trading days old (by fold), its edge Jaccard overlap with the current trailing snapshot is 0.09–0.39, and on average 46% of its edges are still present in the trailing graph.

What changed in the paper. The abstract, the claims box, and the Discussion now limit the penalty to the 126-day Pearson graph with |ρ|>0.6, frozen at the end of training, at its tuned operating point, and state that exploratory graph-policy sensitivities weaken the contrast. Limitation L9 lists what remains open: interactions with retuning, and learned graph constructions.

AuthorsOpenReview ↗

Response to Reviewer NBYp (2/2)

Part 2 — W3/Q3 (universe), W4/Q4 (backtest), W5 (what determines success)

W3/Q3: universe and point-in-time membership. We agree that the study covers one universe of large-cap stocks; Limitation L7 says the evidence comes from one market and one horizon. We did not add another market in this revision. For membership, §4 now reports the survivorship audit: 87 departed window members, 8.2% survivor-side and 8.1% look-ahead-side stock-days, a 16.3% two-sided mismatch. On each date, a new historical-membership restriction removes the snapshot names not yet in the index from scoring and portfolio eligibility (Appendix E.4). Every arm's pooled IC falls by 0.004–0.012. Most paired contrasts change little, but Universe-B L2−L1 weakens from −0.0130 to −0.0086 [−0.0176, +0.0005]; Universe-C L2−L1 is −0.0113 [−0.0177, −0.0049]. The restriction cannot restore departed firms, because our panel has no prices or delisting returns for them. That side of survivorship remains (L8), including its effect on graph neighbourhoods, whose direction we cannot sign. A point-in-time rebuild with delisting returns is listed as future work.

W4/Q4: full-period returns, drawdowns and equity curves. Agreed, and this changed a conclusion. We replaced the cell-averaged Sharpe with one continuous portfolio per seed over the whole test window (35 periods of 21 trading days, 2023-01-03 to 2025-12-08, top/bottom 50, daily marks, costs on a turnover proxy; §3.2). For each arm, Appendix E.3 reports annualized mean and volatility, Sharpe (seed mean and cross-seed SD), cumulative return, maximum drawdown, turnover, and equity curves. For example, at 10 bps in Universe B:

ArmAnn. meanSharpe (SD)Cum. returnMax DD
L0 LightGBM3.1%+0.17 (0.29)5.6%21.6%
L1 MLP13.8%+0.62 (0.26)39.2%22.1%
L2 correlation-GAT4.6%+0.21 (0.35)7.5%28.6%

Do the economic comparisons still hold? Only partly; fewer contrasts separate than in IC. In Universe C, the MLP−LightGBM (+0.29 [−0.51, +0.88]) and correlation-GAT−MLP (−0.02 [−0.20, +0.17]) net ΔSharpe intervals include zero, and we withdraw the submission's statement that they survive 10 bps. In Universe B, correlation-GAT−MLP is −0.41 [−0.65, −0.24] at the primary phase; across all 21 rebalance phases (34 holding periods each), this contrast ranges from −0.42 to −0.02 and is always negative. These are within-sample descriptions on a survivor snapshot, without financing, borrow or liquidity costs; net Sharpe stays descriptive (Table 4, L5).

W5: when graphs help. We agree that the paper does not establish which factors determine success, and the revision no longer implies that it does. We also agree that the edge comparisons are underpowered: at the fixed operating point none of the six edge contrasts survives BH-FDR, all six are underpowered, and failing to detect news-edge harm does not show that the edge is harmless (§5.4). The contributions paragraph now states four bounded conclusions: (i) no tuned graph arm is established to beat tuned LightGBM, and the strongest local evidence is a penalty for this correlation graph; (ii) the fixed-operating-point family, which runs the edge arms at L2's hyperparameters, separates edge effects from retuning effects (the news edge looks harmful in the tuned ladder and is undetectable at the fixed operating point); (iii) predictive skill is concentrated in a minority of quarters, so single-year evaluations are fragile; (iv) three failure modes recur. One new negative finding bears on "which factors": the frozen correlation graph is strongly label-assortative, meaning linked stocks have similar realized labels (0.23, against −0.007 under a degree-preserving null; 12/12 folds above the null's 97.5th percentile), and this alignment coexisted with L2 < L1. These statistics describe co-movement among linked stocks and do not bound what message passing can extract, so we report them as a diagnostic, not as a rule for when graphs help.

Thank you for reworking the backtest and testing alternative graph constructions. The revised results make it clearer how much the original finding depends on the use of a frozen correlation graph.

My main concern remains the choice of graph baselines. I understand why the authors kept the original candidate set fixed, but a separate comparison with a representative stock prediction model such as MDGNN or FactorGCL would leave the original test intact. The simplified HATS and full attention baselines do not answer whether models that construct stock relationships differently can add predictive value. This matters because the penalty becomes smaller and less conclusive when the graph is updated. The paper therefore tells us more about one frozen correlation graph than about when graph models help.

I will maintain my original score of 2.

Reviewer_2T6yOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines whether GNNs have real advantages for cross-sectional stock ranking compared with simpler alternatives, which is an interesting research premise. The main contribution is a fairly comprehensive experimental protocol designed to separate graph-related gains from other modeling and evaluation effects. The results are interesting and generally do not favor GNNs, although the limited experimental setting and some data construction concerns make the conclusions less convincing at the current stage.

Critical strengths

S1) The paper raises an important question that has generally been often overlooked in GNN research in this domain, namely whether adding graph structure actually helps beyond strong non-graph models (without the bias usually of a new GNN/graph-based approach having their proposed framework presented in the best light).

S2) I like the use of a model ladder where different components are gradually introduced; it makes the comparisons easier to interpret than simply benchmarking several unrelated architectures.

S3) Going beyond prediction accuracy to examine transaction costs is valuable, especially given the intended financial application.

Critical weaknesses

W1) At this time, I'm not fully convinced by the contribution beyond an extensive empirical study, i.e., most of the methodology comes from existing evaluation practices, and the paper does not really establish new principles for understanding when GNNs should or should not help. It's an interesting premise, but just not fully sure its at the publication stage for this venue.

W2) The choice of graphs seems quite important to the conclusions. If the constructed graphs do not capture useful relationships between stocks, poor GNN performance is not particularly surprising at all. More exploration of graph quality, alternative graph learning, i.e., some data-centric AI aspects/approaches would help.

W3) There are concerns with the data setup, particularly survivorship bias from the fixed stock universe and possible information leakage in feature selection. These are fairly important issues for a financial prediction study and should be discussed in more details.

W4) Some conclusions seem stronger than what the experiments support. Several comparisons have limited support and the results are restricted to one market and prediction task. The paper would benefit from being careful/modest about what can actually be concluded from the results presented.

Questions for authors

Q1) Can you provide more insights into what properties of a graph would make GNNs beneficial for this task (if any)? This seems more interesting than only showing which models win and going beyond benchmark style setup.

Q2) Were more adaptive or learned graph constructions considered? It would be useful to know whether the negative findings are mostly due to the graph construction or the GNN architectures, where I assume it's likely more around how the relations are constructed/modeling choice there.

Q3) How confident are you that the main observations would remain the same with point-in-time data and across different market periods or prediction horizons? While the results are already quite comprehensive, this is more of just an exploratory question/discussion.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 2T6y (1/2)

Part 1 — W1/Q1 (contribution and what makes a graph useful) and W2/Q2 (graph quality, learned graphs)

Thank you for the review and for framing the graph-quality question so constructively. We address it with new analyses below.

W1: contribution beyond an empirical study. We agree that the submission read like a protocol checklist. The revised contributions paragraph (§1) states four bounded conclusions: (i) under multiplicity control and ten seeds, no tuned graph arm is established to beat tuned LightGBM, and the strongest local evidence is a correlation-graph penalty, not a gain; (ii) a fixed-operating-point family, which runs the edge arms at L2's frozen hyperparameters, separates edge effects from retuning and capacity effects. The news edge shows why this matters: it looks harmful in the tuned ladder, but at the fixed operating point its effect in Universe C is +0.0011 and not significant; (iii) predictive skill is regime-concentrated: roughly half of the twelve test quarters are near zero or negative for almost every arm, so single-year evaluations are fragile; (iv) three failure modes recur: a tuned-configuration collapse, selection-stage leakage into a feature basis, and survivor-snapshot composition. A planted-signal control shows that the graph path does recover graph-borne signal when such a signal exists (GAT 82% and GraphSAGE-Mean 91% of the achievable IC, against about 0 for a feature-matched MLP), so the penalty is not explained by an inability of the graph path to learn.

Q1: which graph properties would make GNNs beneficial. We cannot give a validated rule, and the revision says so. The evidence gives three relevant observations (Appendix E.5 and §5.2):

  • High label alignment did not come with a gain here. In retrospective diagnostics that use realized labels, the frozen correlation graph is strongly label-assortative, meaning linked stocks have similar realized labels (0.23, against −0.007 under a degree-preserving null; 12/12 folds above the null's 97.5th percentile), and this alignment coexisted with L2 < L1. The sector graph is less assortative (0.09), although the two graphs differ in density. These statistics describe co-movement among linked stocks and do not bound what message passing can extract.
  • Frozen snapshots go stale, and changing the graph policy weakens the penalty. At test rebalance dates the frozen snapshot is on average 106–137 trading days old, and on average only 46% of its edges are still present in the current trailing graph. Using the latest trailing snapshot each day weakens the penalty (see W2 for why this does not isolate staleness).
  • Other tuned graph and attention arms score above the correlation-GAT in Universe C. The sector-edge (L4), all-edge (L5), full-attention (L6) and HATS (L7) arms all have BH-rejected positive contrasts against L2 (e.g., L6−L2 +0.0174), although Universe-C positives rest on a leak-selected feature basis.

W2/Q2: graph quality and learned or adaptive graphs. We ran three correlation-graph sensitivities in both universes, retraining L2 at its frozen hyperparameters (unadjusted 95% intervals):

VariantB: L2−L1C: L2−L1
Baseline (126 days, |ρ|>0.6, frozen at end of training)−0.0130 [−0.0223, −0.0041]−0.0117 [−0.0177, −0.0055]
Latest trailing snapshot each day−0.0033 [−0.0127, +0.0058]−0.0049 [−0.0117, +0.0018]
63-day window−0.0111 [−0.0215, −0.0011]−0.0094 [−0.0173, −0.0016]
Threshold 0.4−0.0193 [−0.0310, −0.0078]−0.0024 [−0.0105, +0.0060]

Your expectation that construction matters is partly supported. Daily snapshot selection weakens the penalty in both universes; both intervals include zero but also allow a remaining negative effect. With a 63-day window, L2−L1 remains negative in both universes. Because the daily policy also changes the training and validation graphs, it does not isolate test-time staleness, and none of these sensitivities separates construction from architecture. We therefore limit the headline to the frozen 126-day, |ρ|>0.6 Pearson graph at its tuned operating point. We did not train a learned-graph model in this revision. The closest arm is L6, full self-attention without a mask, which learns weights over all stocks; in Universe C, this arm scores above the correlation-GAT. Learned graph constructions remain open (L9).

AuthorsOpenReview ↗

Response to Reviewer 2T6y (2/2)

Part 2 — W3 (data setup), W4/Q3 (strength of conclusions, PIT, periods, horizons)

W3: survivorship and feature-selection leakage. We moved both into the main text. Survivorship: §4 now gives the audit: a fixed survivor snapshot, 87 departed window members, an 8.1% look-ahead share, and a 16.3% two-sided mismatch. On each date, a historical-membership restriction (Appendix E.4) removes names not yet in the index. Every arm's pooled IC falls by 0.004–0.012. Most paired contrasts change little, but Universe-B L2−L1 weakens from −0.0130 to −0.0086 [−0.0176, +0.0005]. Departed firms cannot be restored with our data, so that side of survivorship remains (L8). Feature selection: Universe C's basis was ranked on evaluation-window quarters (L1), so its positive results stay suggestive. We also corrected the description of Universe B. The submission called it leak-free, but a 63-day diagnostic informed adding its two volume features, and a later exploratory ranking scored all ten of its features. The revision reports both exposures, and a new timeline (Appendix E.1) dates every design decision against the test quarters it used. Deleting the two volume features (Universe B8, price-only features) leaves L2−L1 negative, −0.0181 [−0.0310, −0.0065], but B8 still contains six diagnostic-scored features, so this subset does not remove the selection dependence.

W4: strength of conclusions. We agree; two submission claims were too strong. First, the cost conclusions changed once we replaced the cell-averaged Sharpe with one continuous portfolio per seed: the Universe-C MLP−LightGBM and correlation-GAT−MLP net ΔSharpe intervals at 10 bps now include zero, and we withdraw the statement that they survive costs (§5.3, Table 4). Second, the graph claim is now limited as described above. A new "What we claim / What we do not claim" box (§1) states three claims: no tuned model is established to beat LightGBM (a non-rejection, not equality); the scoped correlation-GAT penalty; and no detectable edge effect at the fixed operating point (0/6, all underpowered). The box also lists what we do not claim: that graphs or GNNs hurt in general, or that dynamic or learned graphs would; that the MLP beats LightGBM; that any strategy is profitable; and point-in-time levels or PIT robustness of the contrasts. We also agree that the evidence comes from one market and one prediction horizon (Limitation L7); we do not generalize the claims beyond this setting.

Q3: point-in-time data, other periods, other horizons. PIT: the restriction above only drops pre-addition names from scoring and eligibility; models and graphs are still fitted on the snapshot. Under this restriction, the Universe-B penalty weakens and the Universe-C penalty remains (−0.0113 [−0.0177, −0.0049]). This check is not a PIT rebuild, and it says nothing about departed firms. Periods: skill is concentrated in a minority of the twelve quarters, so levels estimated on these quarters need not carry over to other periods, and twelve quarters cannot tell us how often favourable regimes recur (Discussion, L2). Horizons: the confirmatory study uses one 21-day horizon (L7). Earlier horizon ablations were exploratory and ran under a different protocol, so we do not use them as evidence.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision (all reviewers)

Summary of the revision (posted to all reviewers)

We thank the reviewers for careful and specific reports. Two of the criticisms changed our results, and the revised PDF reflects these changes. The main changes:

  1. Costs re-estimated, one claim withdrawn (9tq9, NBYp). The submission averaged per-cell Sharpe ratios, and each cell (one fold, one seed) had only about three rebalance returns. We re-ran the confirmatory cells with the frozen hyperparameters and seeds, this time saving the predictions, and built one continuous long-short portfolio per seed over the whole test window (35 periods of 21 trading days, 2023-01-03 to 2025-12-08). With this estimator, the Universe-C MLP−LightGBM and correlation-GAT−MLP net ΔSharpe intervals at 10 bps include zero. We withdraw the submission's statement that these two contrasts survive 10 bps (§5.3, Table 4, Appendix E.3).

  2. Universe B described accurately, with a timeline (9tq9, 2T6y). We removed every "leak-free" and now state what Universe B was exposed to (§4). A new table (Appendix E.1) dates each design decision against the test quarters it used. No quarter is untouched under the final protocol, and the paper says so.

  3. Correlation-graph construction (NBYp, 2T6y). In both universes we ran three graph sensitivities (using the latest trailing snapshot each day, a 63-day window, and a 0.4 threshold) and two retrospective diagnostics: how stale the frozen graph is, and how strongly linked stocks share realized labels (Appendix E.5). The daily-snapshot variant weakens the correlation-GAT penalty, so the headline now covers only the 126-day Pearson graph with |ρ|>0.6, frozen at the end of training, at its tuned operating point.

  4. Survivorship (all three). The survivorship audit is now in §4. A new historical-membership restriction removes each name from scoring and portfolio eligibility on dates before it joined the index (Appendix E.4). Models and graphs are still fitted on the snapshot, and firms that left the index cannot be restored.

  5. Presentation (all three). We added a design-overview figure (§3), a "What we claim / What we do not claim" box (§1), rewritten contributions, and a provenance correction found during the re-execution: in the feature file used by the confirmatory Universe-B arms L2–L6, one stock's two volume features were zero; re-running L2 on the corrected file changes its pooled IC by 0.0001 (Appendix E.2).

  6. Code. We will release the code, frozen configurations, and analysis scripts with the camera-ready version.

All new analyses are descriptive or exploratory sensitivities. IC remains the only confirmatory metric, neither pre-registered family changed, and where a re-executed number differs from the confirmatory one, the paper reports both. Point-by-point replies follow in each reviewer's thread.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.