- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
RICE-Alpha combines historical memory, multi-agent analysis, and an event graph correction for cross-sectional stock ranking. Sentiment, technical, fundamental, and macroeconomic views first produce a Base Alpha. The system constructs successor relations within individual issuers and then pools comparable transitions across firms. It weights graph evidence using historical support, relative transition frequency, lag structure, and matured successor returns. Residualizing this evidence against Base Alpha and the technical view produces a separate correction for the final score. The authors evaluate the framework on Nasdaq-100 and Hang Seng Index constituents from 2024 to 2026. The complete system achieves the best reported predictive and portfolio metrics among the evaluated methods. U.S. ablations, an alternative LLM backbone, constituent exclusions, and transaction-cost stress tests provide additional evidence for the contribution and robustness of the components.
Critical strengths
-
Careful constraints on the use of event history. The distinction between NEW, UPDATED, and CARRIED records prevents continued coverage without new facts from creating duplicate occurrences. Successor relations are formed within issuers before pooling across firms, and observations enter graph estimates only after complete successor windows and relevant returns have matured. These rules address concrete problems in financial news backtesting. Keeping graph evidence as a separate correction also makes its incremental contribution relative to the base forecast easier to examine.
-
Broad evaluation with component comparisons. Both markets use the same frozen hyperparameters, portfolio rules, and cost assumptions. In the U.S., the complete system achieves IC 0.0350, compared with 0.0191 for the memory-only configuration and 0.0260 for direct graph correction. Separate reliability-only and residualization-only variants help distinguish the two operations. Alternative-backbone, constituent-exclusion, and cost analyses further support the reported results.
Critical weaknesses
-
The conclusion and Appendix F explicitly acknowledge that both pretrained backbones may contain information that postdates individual forecasts. The paper does not experimentally assess the effect of this information on the historical backtest. Restricting prompt inputs controls the evidence explicitly supplied to a model, but does not exclude prior knowledge of companies, events, or subsequent developments. Replacing the backbone with another model that may also contain later information does not resolve this concern. Source-data availability also needs clarification. Appendix D.1 admits U.S. financial reports using a fixed 90-day delay and describes Hong Kong financial snapshots within a historical window ending before the prediction date. It does not fully establish that the original publication timestamps and contemporaneous data versions were used. A fiscal period preceding a prediction date does not by itself establish that the corresponding information was publicly available at that time. Because temporal validity is central to the paper, these uncertainties affect the interpretation of the reported predictive gains. A backbone with a training cutoff preceding the test period, a prospective evaluation after model freezing, or targeted contamination controls would strengthen the evidence. The manuscript does not demonstrate that leakage occurred, but it also does not adequately exclude its influence.
-
Table 2 supports numerical graph correction, reliability weighting, and residualization within this system. It does not directly test the importance of forming issuer-local successor pairs before pooling across companies. Controls that preserve event frequencies and sample sizes while shuffling within-issuer temporal order, or event statistics without successor ordering, would help distinguish gains from genuine event continuation from gains associated with event categories and historical return summaries. The authors appropriately limit claims of superiority over prior event graph methods. Nevertheless, the absence of comparisons under a common protocol limits the assessment of the methodological contribution. Relevant candidates include TRACE, which combines graph structure with textual evidence, and CausalStock, which incorporates lag-dependent relations and news information. Statistical comparisons do not cover the strongest baseline. The U.S. IC advantage over momentum is only 0.0034, from 0.0316 to 0.0350. Table 9 reports paired baseline tests against AI Hedge Fund, but not momentum, MEME, or R&D-Agent-Quant. A significant positive IC for the complete system and a nonsignificant IC for a baseline do not establish a significant difference between the methods. Since the validity rules permit different stock subsets across methods, the comparison should also be confirmed on common stocks and dates.
Questions for authors
- Model temporal validity. What are the exact backbone versions and known training cutoff dates? Are there results from a test period after the models' knowledge cutoffs, or controls that assess the influence of historical event memorization?
- Data availability and configuration selection. Were financial reports and news reconstructed using actual publication timestamps, time zones, and contemporaneous versions? How is public availability guaranteed for Hong Kong financial snapshots? What historical replay and parameter-selection procedure is meant by freezing the configuration at the end of the 2023 warm-up?
- Contribution of event ordering. How does performance change when within-issuer temporal order is shuffled while preserving event category frequencies, support, and return sample sizes? This would help establish whether succession structure provides additional information.
- Alignment with the prediction horizon. Equation (9) estimates returns after successor event B occurs, whereas the prediction target starts at the next open after the current cutoff. How does the lag weighting in Equation (11) account for this difference? For CARRIED events that have already persisted for several days, is event age used to adjust the remaining successor probability?
6 comments
Q2 & W1 (Data availability and configuration selection):
We agree that verifying point-in-time data availability and clarifying the configuration-selection procedure are essential. We address both concerns below, supported by the source-level audit in Table J2.
- Timestamps and versions. News is admitted based on provider publication timestamps (UTC converted to exchange time), with date-only records deferred to the next trading session. We audited every financial statement and news item used in the submitted run against publication records, including the exact version available at each forecast cutoff. The independent Llama replay reconstructs a document-versioned source panel from archived records.
- Financial statements. US financial statements are admitted 90 calendar days after fiscal-period end, while HK snapshots use the latest fiscal period ending at least three months before the prediction date. The publication audit confirms that all admitted issuer-period results were publicly available before use. We will clarify the HK rule in Appendix D.1.
- Freezing. The 2023 warm-up replays the same daily pipeline on historical data to initialize system state, without contributing to reported metrics. Parameters in Tables 5 and 7 were manually specified based on rule design and 2023 replay diagnostics (score scales, coverage, and cross-sectional stability). No grid search was performed, and no 2024–2026 forecasts were evaluated before the configuration was frozen.
As shown in Table J2, all financial statements and news items used in the submitted run were public before admission, no forecast used a version unavailable at its cutoff, and no forecasts required exclusion. Excluding all flagged stock-dates leaves IC nearly unchanged in both markets. The original Table 1 results remain unchanged.
Table J2. Source-availability audit of the submitted run and independent document-versioned replay. Flagged stock-dates involve a pending periodic report, financial line items differing from their first release, or subsequently revised news, regardless of whether the version used was already public at the forecast cutoff.
| US | HK | |
|---|---|---|
| Submitted run (DeepSeek): every statement and news item checked against the publication record | ||
| Issuer–periods whose statements entered a forecast | 948 | 437 |
| Results published before admission | 948 (100.00%) | 437 (100.00%) |
| Days from publication to admission, median / shortest | 58 / 11 | 24 / 1 |
| Periodic report also on file with the SEC before admission | 945 | n/a |
| Line items equal to the first-published value | 99.6% | 99.7% |
| News items that entered a forecast | 148,112 | 100,935 |
| of which later revised by the provider | 1,173 (0.79%) | 868 (0.86%) |
| Forecast uses of a version not yet public at the cutoff | 0 | 0 |
| Forecasts requiring exclusion | 0 | 0 |
| IC without every flagged stock–date (share of the panel flagged) | 0.0352 (3.63%) | 0.0281 (3.33%) |
| Submitted IC (Table 1) | 0.0350 | 0.0283 |
| Independent replay (Llama): source panel rebuilt from archives with full revision histories | ||
| News records in the archive | 152,384 | 104,517 |
| admitted at their own cutoff | 147,602 (96.86%) | 99,841 (95.53%) |
| admitted from the next session (date only) | 3,107 | 3,318 |
| not admitted (duplicates or no usable timestamp) | 1,675 | 1,358 |
| of the admitted records, served in the latest version public at the cutoff because a later revision exists | 1,212 | 905 |
| IC of the replay | 0.0298 | 0.0246 |
Q3 & W2 (Contribution of event ordering):
We conduct both requested controls and four additional variants, modifying only the numerical event-graph pathway while keeping Base Alpha and memory text fixed. Within-issuer type permutation preserves event category frequencies, pair counts, and return sample sizes; cross-issuer successors preserve per-edge support; and succession-free statistics retain each event type's matured return history without successor information. As shown in Table J3, all six controls lower IC in both markets with both backbones. The gain over succession-free statistics remains significant after Holm correction, demonstrating that successor structure contributes predictive information beyond event categories and their return histories. The succession-free variant itself retains positive IC, while incorporating successor structure provides a further improvement.
Table J3. Contribution of event ordering and controlled graph-construction alternatives (IC; extending Table 2). ΔIC denotes the improvement of local-first construction over each alternative, averaged equally across the two markets on shared valid dates (as in Table 9). The final two rows report two confirmatory contrasts designated before the controls were run, with Holm-adjusted p-values ((p_H)); individual-rule p-values are unadjusted. All variants recompute only the numerical correction, keeping Base Alpha and memory text fixed from the original run for both backbones.
| Construction rule | DeepSeek US IC | DeepSeek HK IC | DeepSeek pooled ΔIC [LB] | p | Llama US IC | Llama HK IC | Llama pooled ΔIC [LB] | p |
|---|---|---|---|---|---|---|---|---|
| Local-first (RICE-Alpha) | 0.0350 | 0.0283 | 0.0298 | 0.0246 | ||||
| Event types permuted within issuer (category frequencies, pair counts and return sample sizes preserved) | 0.0293 | 0.0240 | +0.0050 [+0.0006] | 0.0296 | 0.0250 | 0.0209 | +0.0043 [+0.0003] | 0.0366 |
| Cross-issuer successors at identical per-edge support | 0.0303 | 0.0247 | +0.0041 [+0.0001] | 0.0474 | 0.0258 | 0.0215 | +0.0036 [−0.0001] | 0.0564 |
| Predecessor-reversed: same pairs and returns, reversed query direction | 0.0299 | 0.0242 | +0.0046 [+0.0002] | 0.0424 | 0.0256 | 0.0211 | +0.0038 [−0.0001] | 0.0536 |
| Pooled-first: events of all issuers pooled before pairing, so a successor may come from another issuer (same edge thresholds) | 0.0291 | 0.0238 | +0.0052 [+0.0005] | 0.0332 | 0.0248 | 0.0207 | +0.0045 [+0.0002] | 0.0408 |
| Lifecycle-free: repeated coverage also creates occurrences | 0.0317 | 0.0251 | +0.0032 [−0.0005] | 0.0746 | 0.0271 | 0.0219 | +0.0027 [−0.0006] | 0.0918 |
| Succession-free: each event type keeps its own return history, no successor information | 0.0285 | 0.0228 | +0.0060 [+0.0011] | 0.0210 (p_H 0.0260) | 0.0243 | 0.0199 | +0.0051 [+0.0007] | 0.0270 (p_H 0.0360) |
| Mean of the five rules other than succession-free | 0.0301 | 0.0244 | +0.0044 [+0.0012] | 0.0130 (p_H 0.0260) | 0.0257 | 0.0212 | +0.0038 [+0.0008] | 0.0180 (p_H 0.0360) |
Q1 & W1 (Backbone versions, training cutoffs, and model knowledge):
We agree that backbone look-ahead requires direct validation. Neither submitted backbone has a documented cutoff ruling out look-ahead (both were released during/after the evaluation period). We therefore replayed the full pipeline with an earlier-cutoff Llama backbone and added entity masking (Llama, Qwen3-Max), a post-Llama-release window, and a post-submission extension. As shown in Table J1, the RICE-Delta increment remains positive and comparable across all Llama and DeepSeek settings, with significant full-window gains in both markets (wider intervals in shorter windows) and negligible masking effects. Although Llama has lower absolute metrics than DeepSeek, its US IC and ICIR remain within the range of the two submitted backbones.
Exact versions & cutoffs:
deepseek-v4-flash, buildDeepSeek-V4-Flash-0731(rel. 2026-07-31); no model-specific cutoff (public summary dates some data to ~May 2025 and web crawl to ~2024–Dec 2025, without a post-training cutoff).qwen3-max, snapshotqwen3-max-2026-01-23(family rel. Sep 2025); no documented cutoff.meta-llama/Llama-3.1-70B-Instruct(rev1605565b47bb9346c5515c34102e054115b4f98b); pretraining cutoff Dec 2023; rel. 2024-07-23.BAAI/bge-m3(rev5617a9f61b028005a4858fdac845db406aefb181; rel. 2024-01-30); no documented cutoff.Table J1. Backbone, window, and entity-masking controls (metrics as Table 1). Increment = RICE-Alpha IC − Base Alpha IC (RICE-Delta). Masking replaces tickers, names, and aliases with placeholders across all LLM roles. Qwen3-Max (2nd backbone) covers US predictive metrics only (masked run is new).