acceptodds
Desk rejected at ICLR 2027

Signal-TTT: Efficient Test-Time Adaptation for Event-Driven Portfolio Construction

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Event-driven asset ranking aims to predict market impact and construct long-short portfolios following market-moving information, such as Donald Trump's Truth Social posts. While frozen large language models can score individual (post, instrument) pairs, test-time adaptation (TTA) is critical to adjust for post-training market regimes. However, existing TTA methods fit a single parameter per test input, which in this context represents an entire event cross-section. We prove that any additive or strictly monotonic per-event correction inherently preserves within-event relative orderings, leaving rank metrics and portfolio performances unchanged. To address this limitation, we propose , a closed-form cross-sectional test-time training framework. computes a per-(event, instrument) offset using ridge-regularized least squares over retrieved historical outcomes, automatically calibrating its regularizer via empirical Bayes. Requiring no gradients, learning rates, or access to model weights, our update is provably a shrunk per-instrument mean residual with a explicit error-reduction condition. Evaluated on a development set of nEvTest Truth Social policy events across nBackbones frozen backbones, improves Rank IC by devIcGain and boosts dollar-neutral net returns by devPrimaryMean bps per event, whereas collapsing to a single per-event offset completely eliminates these gains.

Primary area:
learning on time series and dynamical systems
Keywords:
investment, LLM, NLP

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UascOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The paper studies event-driven cross-sectional ranking. A frozen LLM scores each (Truth Social post, instrument) pair as qUP−qDOWNq_{UP}-q_{DOWN}, and a dollar-neutral book trades the two tails. The authors first point out that any strictly increasing per-event correction (shift, rescaling, batch calibration) cannot change within-event order, so it does nothing for rank IC (Prop. 1). SIGNAL-TTT instead fits one offset per (event, instrument) pair. It retrieves the k nearest past events in text-embedding space from a fixed memory and takes the kernel-weighted mean residual of the frozen score against a signed rank target. It then shrinks this offset with an empirical-Bayes gate, which is basically Bühlmann credibility. On 321 development events, 39 instruments and 9 backbones, rank IC goes from 0.0030 to 0.0186 and net return improves by +2.32 bps per event.

Critical strengths
  1. The evaluation protocol is unusually careful and honest for this kind of paper. Splits are chronological, every choice is made on validation, intervals use block bootstrap, comparisons are Holm-corrected and transaction costs are included. The authors also check stream-order invariance and backbone release dates. They openly report the uncomfortable results: the development window is not a true held-out test, validation and development sometimes disagree, and the book still loses money in absolute terms. This makes the numbers easier to trust than usual.

  2. The method is simple, closed-form and works with black-box APIs. The motivating point that per-event corrections can't reorder instruments is clear, and the shared-offset ablation confirms it (0.0027 vs 0.0030).

Critical weaknesses

The method is simple and the evaluation is unusually honest. They report things like val/dev disagreement and the book still losing money, which I appreciate. But I’m not convinced the gain really comes from retrieval. Most of it survives when the memory is shuffled, so it looks more like a fixed per-instrument bias correction. The edge over simple retrieval baselines also isn’t significant. Details as follows:

  1. It is unclear how much of the gain actually comes from event-specific retrieval rather than a static per-instrument bias correction.
    • With memory outcomes fully permuted within each instrument, rank IC is still 0.0149 out of 0.0186.
    • The across-event permutation null gives p = 0.069.
    • The paper says the per-instrument affine calibration is the only operator with positive absolute net return, yet it doesn't appear in Table 2.
    • Together, these suggest a large part of the effect is "instrument a is usually mis-centered by x", which doesn't need retrieval or TTT at all. That weakens the main story.
  2. The gains are small and the advantage over simple retrieval baselines is not established.
    • None of the retrieval comparisons survive Holm correction (p from 0.169 to 0.921). kNN-LM is close on rank IC (0.0178 vs 0.0186), and the direct Nadaraya–Watson form reaches 0.0425 on development.
    • On validation, the second-half gain is only +0.0008 and net return drops by 1.38 bps.
    • 6 of 9 backbones were released during or after the development window, so look-ahead is possible for most of the average.
Questions for authors
  • Please add the per-instrument affine calibration to Table 2 with rank IC, net return and the paired test against SIGNAL-TTT. If it is close, what does retrieval add?
  • How do the main results look when restricted to the 3 backbones released before the development window?
  • Data after April 2026 should now exist. Can you report a truly held-out window that was never inspected during development?
  • (Minor) There are several writing issues, e.g. "prove the work significance" in the introduction, "method discovery event driven market prediction", and "SPY500".
5 comments
AuthorsOpenReview ↗

Response to Reviewer Uasc

Thanks again for your carefully review and hopefully our rebuttal can solve your concerns. We believe with revision can make this interesting topic better. Table and line numbers refer to the submitted PDF. Brackets are 95% CIs from the paper's 5-day block bootstrap; p-values are one-sided (sign-flip for the forward endpoint). Post-unseal marks descriptive analyses computed after the forward window was scored, with the registered evaluator unchanged; they are not new endpoints.

W1. Event-specific retrieval versus static per-instrument bias.

  • What the controls establish: a correction must vary across instruments within an event to change rank IC. A shared per-event offset gives 0.0027 vs 0.0030 frozen (Table 4), and label-free TTA changes rank IC by −0.0009 to +0.0001 (Table 6). The correction must use each instrument's own outcomes (across-instrument permutation p = 0.008, Table 9), and text retrieval selects the evidence: random neighbours give 0.0057 and market-state neighbours 0.0015, against 0.0186 (lines 445–449).
  • Per-instrument component: within-instrument permutation keeps 0.0149 of the 0.0186 rank IC (Table 10), so the reviewer is right that most of the gain is instrument-level. The event-specific part is positive but not significant (p = 0.069, Table 9), and static affine calibration reaches 0.0169.
  • Change: lines 449–450 are rewritten, and App. F.4 states this decomposition; SIGNAL-TTT is described as a per-instrument residual correction whose weights are localized by text retrieval.

W2. Small gains and retrieval baselines.

  • The significant claims are against frozen and label-free TTA (Holm p ≤ 0.004, Table 2). Against retrieval, SIGNAL-TTT has the highest rank IC (0.0186 vs 0.0178 for kNN-LM) and positive paired differences, not significant after Holm (p 0.169–0.921), and the paper reports them as such.
  • The direct Nadaraya–Watson form reaches 0.0425 on development but 0.0180 on validation, where the residual form reaches 0.0461 (Table 8). We keep the validation-selected form; choosing the development-best variant after inspection would be selection on the evaluation window.

Q1. Affine calibration in Table 2; what retrieval adds.

SIGNAL-TTT − affine, development, bps per event
threshold book (Table 2 protocol)−0.96 [−2.21, +0.03], Holm p = 0.939
rank-only book, k = n/5 (post-unseal)+1.56 [−0.32, 3.66], p = 0.062
  • Development rank IC is 0.0186 (SIGNAL-TTT) vs 0.0169 (affine).
  • Affine's threshold-book number comes from abstention: untraded events count as 0 while the frozen book loses money, and affine trades 76.2 of 321 development events against 201.3 for SIGNAL-TTT. Every other Table 2 operator trades 189–295, so affine is the only operator whose book is dominated by abstention.
  • When every operator trades every event (rank-only book, same costs), SIGNAL-TTT is ahead of affine on development: +2.88 [0.38, 5.42] (k = 3) and +2.06 [0.09, 4.13] (k = 5).
  • Change: App. F.1 extends Table 2 with the affine row and traded-events and net-per-traded-event columns and describes the abstention effect; Section 5 states the affine comparison.

Q2. The 3 backbones released before the development window. They give the same development gain as all 9 backbones, so look-ahead does not produce it. Post-unseal pooled test, same bootstrap:

Qwen3-8B, Qwen3-14B, Seed-OSS-36Bdevelopment
SIGNAL-TTT − frozen, bps+2.32 [0.97, 3.63], p = 0.0003
rank IC gain+0.0159 [0.0006, 0.0328], p = 0.021
rank-only book, bps+2.58 [0.54, 4.99]

Per backbone on development: +1.70, +3.66, +1.61 bps. All 9 backbones give +2.32 bps and +0.0156 rank IC.

Q3. A held-out window after April 2026. Done. 558 events, 2026-04-13 to 2026-09-21, all 9 backbones; configuration and primary endpoint hashed into a registry on 2026-09-23, before any forward outcome was scored. In this window the frozen backbones carry no signal (mean frozen rank IC 0.0375 validation, 0.0030 development, −0.0080 forward), and no operator exceeds +0.002 rank IC. SIGNAL-TTT has a higher rank IC than frozen (+0.0021, p = 0.34), every label-free TTA method and the shared offset, and its rank-only book is ahead of frozen (+0.64 [−0.53, 1.60]). The registered net endpoint is −0.31 [−0.97, +0.35] (p = 0.78); affine, which trades 109 of 558 events, is ahead of SIGNAL-TTT on the threshold book (−2.11 [−3.19, −0.97]). The forward differences against frozen are not significant. Change: the paper states its performance claims for the development window.

Minor. Fixed: line 048, line 079 and line 339 ("SPY500" → "S&P 500").

Reviewer_UascOpenReview ↗

Thanks for the reply

Please update the revised PDF and mark it with a different color. Thanks!

AuthorsOpenReview ↗

Summarize of the revisions

Thanks so much for your timely reply, We have uploaded a revised PDF. All changes are in blue, and the body is still 10 pages. Line numbers refer to the revised PDF.

  • W1 (event-specific retrieval vs. static per-instrument bias).
    • The Ablation paragraph (line 453) adds that permuting outcomes across instruments within each event removes the gain (p = 0.008).
    • Appendix F.4 gives the full decomposition: within-instrument permutation keeps 0.0149 rank IC, and the event-specific part is about 0.0037 (p = 0.069).
  • W2 (small gains; retrieval baselines). Observation ❸ already states that the differences from the retrieval operators do not survive Holm correction. It is unchanged.
  • Q1 (affine in Table 2; what retrieval adds).
    • Observation ❶ (line 365) adds that SIGNAL-TTT leads affine on rank IC (0.0186 vs 0.0169). When every operator trades every event, it leads by +2.88 (k = 3) and +2.06 (k = 5) bps per event, both intervals above zero.
    • Line 415 adds that affine trades 76.2 of 321 events.
    • Appendix F.1 (Table 16) gives Table 2 with the affine row and the traded and net-per-traded columns. Every other operator trades 189–295 events.
    • Appendix F.2 (Table 17) gives the rank-only book against frozen, affine and kNN-LM.
  • Q2 (3 backbones released before the development window).
    • Line 433 reports +2.32 [+0.97, +3.63] bps per event and +0.0159 rank IC on these backbones, the same gain as on all 9.
    • Appendix F.3 (Table 18) adds rank IC, the rank-only book and the per-backbone values: +1.70, +3.66 and +1.61 bps.
  • Q3 (held-out window after April 2026). The forward window is reported in our response comment above. The conclusion (line 488) states the performance claim for the development window.
  • Minor.
    • Line 079: the typo is fixed ("for event-driven market prediction").
    • Line 339: "SPY500" is replaced. The line now states that events are Donald Trump's Truth Social posts and instruments are S&P 500 stocks.

Hope the revision and clarify in rebuttal can solve your concerns and consider increase the rating of the paper. If you have any continue concerns please let us know.

Reviewer_NMc6OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at event-driven cross-sectional ranking: when a policy event like a Trump Truth Social post comes out, predict the relative returns of a set of instruments and turn that into a long-short portfolio. The authors point out that when you correct a frozen LLM's per-instrument scores at test time, the usual TTA approach of fitting one correction per test input (here, one per event) cannot change the order of instruments within the event. To fix this, they propose Signal-TTT, which retrieves past events with similar text, uses their realized outcomes to estimate how far off the score is for each instrument, and applies that correction after empirical Bayes shrinkage. No gradients or access to model weights are needed. On an evaluation window of 321 events with 9 backbones, the authors report that Signal-TTT improves both rank IC and net return after transaction costs over the frozen model.

Critical strengths
  1. The paper identifies the problem clearly. When you rank many instruments at once, adding the same correction to all of them does not change the ranking, so the correction has to be done per instrument. The paper makes this point clearly and also confirms it with an ablation.
  2. The method is simple and easy to use. It fixes the scores with a small closed-form computation, without retraining the model or touching its weights, and it falls back to the original score when there is not enough evidence to correct it.
  3. The evaluation is careful and the reporting is honest. The data is split in time order, transaction costs are included, and all baselines are evaluated under the same conditions. The paper also does not hide unfavorable results, such as comparisons that are not statistically significant or the fact that the portfolio still loses money after costs.
Critical weaknesses
  1. No evaluation on truly unseen data. As the authors themselves state, the period used to report results is the same period they looked at repeatedly while developing the method. The data used to pick the configuration is also small (217 events), yet 614 configurations were tried on it, and the effect almost disappears in the second half of that data. So it is hard to be confident that the gains would hold on a new period.
  2. It is not clear where the improvement comes from. The paper claims the gains come from correcting the LLM's scores with past outcomes, but methods that do not use the LLM score at all and only look at past outcomes perform about as well (rank IC 0.0173 for kNN vote and 0.0162 for Nadaraya-Watson, vs. 0.0186 for Signal-TTT). Also, about three quarters of the gain remains even when past outcomes are shuffled across events within each instrument. This suggests the gain may come less from finding similar posts and more from past per-instrument tendencies (i.e., "this stock usually moves like this").
  3. The practical significance of the improvement is not yet clear. The improvement is small, only 4 of the 9 models show a statistically clear gain, and the portfolio still loses money after transaction costs. It would help to also report results for the simpler baseline that the text mentions as the only one that makes money after costs. Given this, I would suggest toning down wording such as "state-of-the-art".
  4. The possibility that the models already saw future information is not controlled. Six of the nine models were released during or after the evaluation period, so they may have been trained on news or market movements from that period. In particular, the model highlighted for the best results was released about four months after the evaluation period ended. Results using only models released before the evaluation period should be reported.
Questions for authors
  1. Some of the models were released after the evaluation period and may have already learned information from it. Could you check whether the same improvement appears when using only models released before the evaluation period?
  2. The text says the per-instrument static calibration baseline is the only method that still makes money after transaction costs, but its numbers are not in the tables. How does Signal-TTT compare to it?
  3. Retrieval methods that do not use the LLM score perform similarly, and much of the effect remains even after shuffling past outcomes. Is there an experiment or explanation that can tell apart gains from correcting the LLM score vs. gains from using past per-instrument tendencies?
  4. The caption of Table 2 says the backbone is Seed-OSS-36B, but the numbers in the table match the mean over all 9 backbones in Table 3, not the Seed-OSS-36B row. Also, the text says the absolute net return of Signal-TTT is still negative, but the cross-backbone mean curve in Fig. 5a ends well above zero. Could you clarify both?
5 comments
AuthorsOpenReview ↗

Response to Reviewer NMc6

Thanks for your carefully review we believe with revision will make this interest topic better. Table, figure, line and appendix numbers refer to the submitted PDF. Brackets are 95% CIs (5-day block bootstrap); p-values are one-sided (sign-flip for the forward endpoint). Post-unseal marks descriptive analyses run after the forward window was scored, with the evaluator unchanged.

W1. Unseen data; 614 configurations on 217 validation events. The 614 configurations cover every design and hyper-parameter choice made on validation (App. D.3); the final SIGNAL-TTT selects only k and T per backbone, 6 configurations, fewer than any retrieval rival (18) or affine calibration (16), under the same selection rule. Because selection on 217 events cannot by itself establish generalisation (Bailey & López de Prado, 2014), we registered a forward test: 558 events, 2026-04-13 to 2026-09-21, all 9 backbones; configuration and primary endpoint hashed on 2026-09-23, before any forward outcome was scored. In this window the frozen backbones carry no signal (mean frozen rank IC 0.0375 validation, 0.0030 development, −0.0080 forward), and no operator exceeds +0.002 rank IC. SIGNAL-TTT has a higher rank IC than frozen (+0.0021, p = 0.34), every label-free TTA method and the shared offset, and its rank-only book is ahead of frozen (+0.64 [−0.53, 1.60]). The registered net endpoint is −0.31 [−0.97, +0.35] (p = 0.78); affine, which trades 109 of 558 events, is ahead of SIGNAL-TTT on the threshold book (−2.11 [−3.19, −0.97]). The forward differences against frozen are not significant. Change: the paper states its performance claims for the development window.

W2 / Q3. Correcting the LLM score versus per-instrument tendency. The controls support Proposition 1: a useful correction must be instrument-indexed and must use each instrument's own outcomes. A shared per-event offset gives 0.0027 vs 0.0030 frozen (Table 4); label-free TTA changes rank IC by −0.0009 to +0.0001 (Table 6); across-instrument permutation gives p = 0.008 (Table 9). Text retrieval selects which matured events form the estimate: random neighbours give 0.0057 and market-state neighbours 0.0015, against 0.0186 (lines 445–449). Within-instrument permutation keeps 0.0149 (Table 10), so most of the gain is per-instrument tendency, and the event-specific part (about 0.0037, p = 0.069, Table 9) is positive but not significant. The outcome-only rivals (kNN vote 0.0173, Nadaraya–Watson 0.0162) are also instrument-indexed, which is why they score well. Change: lines 449–450 are rewritten, and App. F.4 states this decomposition (per-instrument residual correction with retrieval-localized weights).

W3 / Q2. Practical significance and the affine baseline. The primary endpoint is the pooled 9-backbone mean: +2.32 [+1.30, +3.09] bps per event (p = 0.0001) and +0.0156 rank IC (p = 0.008); net return per traded event improves on all 9 backbones (Table 3). Development rank IC: SIGNAL-TTT 0.0186, affine 0.0169. On the threshold book, SIGNAL-TTT − affine is −0.96 [−2.21, +0.03] (Holm p = 0.939) on development. This difference comes from abstention: untraded events count as 0 while the frozen book loses money, and affine trades 76.2 of 321 development events (0 on Qwen3-8B) against 201.3 for SIGNAL-TTT; every other operator trades 189–295. On the post-unseal rank-only book (every operator trades every event), SIGNAL-TTT − affine on development is +1.56 [−0.32, 3.66] (k = n/5), +2.88 [0.38, 5.42] (k = 3) and +2.06 [0.09, 4.13] (k = 5). Change: App. F.1 extends Table 2 with the affine row and traded-events and net-per-traded-event columns; Section 5 states the affine comparison.

W4 / Q1. Models released before the evaluation period. The 3 backbones released before the development window give the same development gain as all 9, so look-ahead does not produce it.

SIGNAL-TTT − frozen, development3 pre-development backbones (post-unseal)
net bps per event+2.32 [0.97, 3.63], p = 0.0003
rank IC gain+0.0159 [0.0006, 0.0328], p = 0.021

Change: we remove the capability-scaling sentence at lines 430–431 (Qwen3.8-27B was released 2026-08-05), and the Table 3 rank IC for Qwen3.8-27B is updated to 0.0199 from the generated table; the mean row (0.0186) is unchanged.

Q4. Table 2 caption and Figure 5a. The Table 2 values are the 9-backbone mean; they equal the mean row of Table 3 (0.0030, 0.0186, +2.32). The caption is corrected to say so. Figure 5a is regenerated from the registered book behind Table 2: over the 321 development events cumulative net return ends at −266 bps (SIGNAL-TTT) and −1010 bps (frozen), consistent with lines 410–411, and the gap of 744 = 321 × 2.32 equals the Table 2 primary endpoint.

AuthorsOpenReview ↗

Summarize of rebuttal revisions

We have uploaded a revised PDF. All changes are in blue, and the body is still 10 pages. Line numbers refer to the revised PDF.

  • W1 (unseen data; 614 configurations).
    • Appendix D.3 lists the per-backbone tuning grid of every operator. SIGNAL-TTT has 6 configurations (k × T), fewer than any retrieval rival (18) or affine calibration (16), all under the same selection rule.
    • The forward window is reported in our response comment above.
  • W2 / Q3 (LLM correction vs. per-instrument tendency).
    • The Ablation paragraph (line 453) adds the across-instrument permutation result (p = 0.008): the gain requires each instrument's own retrieved outcomes.
    • Appendix F.4 gives the decomposition: within-instrument permutation keeps 0.0149 rank IC, and the part tied to the specific retrieved events is about 0.0037.
  • W3 / Q2 (practical significance; affine).
    • Observation ❶ (line 365) adds that SIGNAL-TTT leads affine on rank IC (0.0186 vs 0.0169). It also leads by +2.88 and +2.06 bps per event when every operator trades every event, both intervals above zero.
    • Line 415 adds that affine's positive net per traded event comes from 76.2 traded events.
    • Appendix F.1 (Table 16) gives Table 2 with the affine row and the traded and net-per-traded columns.
  • W4 / Q1 (models released before the evaluation period).
    • As promised in our response, the capability-scaling sentence in the per-backbone paragraph is removed.
    • In its place, line 433 reports the 3 backbones released before the development window: +2.32 bps per event and +0.0159 rank IC, the same gain as on all 9.
    • Appendix E.1 and Appendix F.3 (Table 18) give the details.
  • Q4 (Table 2 caption; Figure 5a).
    • The Table 2 caption (line 357) states that the values are means over the 9 backbones.
    • Figure 5 keeps its submitted layout. It is regenerated from the registered development book behind Table 2. The cross-backbone mean ends at −266 bps (SIGNAL-TTT) and −1010 bps (frozen), and the gap equals 321 × 2.32.
    • As promised in our response, Table 3 gives the Qwen3.8-27B rank IC as 0.0199. Its gain over frozen is updated to match. The mean row (0.0186) is unchanged.
    • The Per-Backbone Analysis (line 431) states that rank IC rises on 8 of the 9 backbones.

Hope the revision and clarify in rebuttal can solve your concerns and consider increase the rating of the paper. If you have any continue concerns please let us know.

AuthorsOpenReview ↗

Friendly reminder and short summary, we really value your concerns and hope rebuttal can solve them also increase the score

Dear Reviewer NMc6,

We understand you are busy. This is a short reminder of the rebuttal based on your comments with a revised PDF (changes in blue). We summarise each point below hope help this interesting research topic.

W1 — unseen data, and 614 configurations. The 614 configurations are the full validation grid (App. D.3). The final SIGNAL-TTT selects only k and T per backbone, 6 configurations, against 18 for each retrieval rival and 16 for affine, under the same selection rule. Because selection on 217 validation events cannot by itself establish generalisation, we registered a forward window before scoring it (558 events, 2026-04-13 to 2026-09-21). There the frozen backbones carry no signal (rank IC −0.0080); SIGNAL-TTT has a higher rank IC than frozen, every label-free TTA method and the shared offset, and its rank-only book is ahead of frozen (+0.64 [−0.53, +1.60]). The registered net endpoint is −0.31 [−0.97, +0.35], not significant; the paper states its performance claims for the development window.

W4 and Q1 — models released before the evaluation period. The 3 backbones released before the development window give the same gain as all 9.

SIGNAL-TTT − frozen, development3 pre-development backbonesall 9
net bps per event+2.32 [0.97, 3.63], p = 0.0003+2.32 [1.30, 3.09]
rank IC+0.0159 [0.0006, 0.0328], p = 0.021+0.0156
rank-only book, k = n/5+2.58 [0.54, 4.99]+2.60 [1.03, 4.50]

Per backbone: +1.70, +3.66, +1.61 bps. Their weights cannot contain development-window outcomes.

W2 and Q3 — where the gain comes from. It is a per-instrument residual correction, not a per-event shift.

control (development, rank IC)result
frozen0.0030
shared per-event offset0.0027
5 label-free TTA methods (level)0.0021–0.0031
text retrieval → random neighbours0.0057
text retrieval → market-state neighbours0.0015
outcomes permuted across instruments within eventp = 0.008
outcomes permuted within instrument0.0149 kept
SIGNAL-TTT0.0186

Because most of the gain is per-instrument tendency, the outcome-only rivals (kNN vote 0.0173, Nadaraya–Watson 0.0162) also score well: they are instrument-indexed too. SIGNAL-TTT is ahead of both on rank IC.

W3 and Q2 — practical significance, and affine. The primary endpoint, development, 9 backbones: +2.32 [+1.30, +3.09] bps per event (p = 0.0001) and +0.0156 rank IC (p = 0.008); net per traded event improves on all 9 backbones.

developmentrank ICnet/tradedSIGNAL-TTT − operator, net bps per eventtraded
frozen0.0030−4.36+2.32 (p < 0.001)194.7
affine calibration0.0169+0.42−0.96 (p = 0.939)76.2
kNN-LM0.0178−1.97+0.57209.3
kNN vote0.0173−2.46+0.94231.0
gradient TTT0.0148−1.22+0.40189.0
SIGNAL-TTT0.0186−0.39—201.3

Affine's threshold-book advantage comes from abstention: untraded events count as zero and the frozen book loses money, so it trades 76.2 of 321 events. When every operator trades every event, SIGNAL-TTT leads affine by +2.88 [0.38, 5.42] (k = 3) and +2.06 [0.09, 4.13] (k = 5). On wording: Observation ❶ states the claim against the operators of Table 2, and the retrieval comparisons that do not survive Holm are reported as not significant.

Q4 — Table 2 caption, Figure 5a, Table 3. The Table 2 values are the 9-backbone mean and the caption says so. Figure 5a is regenerated from the registered book and ends at −266 bps (SIGNAL-TTT) and −1010 (frozen), a gap of 744 = 321 × 2.32. The Qwen3.8-27B rank IC is 0.0199; the mean row is unchanged.

Why we think this is worth a second look. A correction shared by all instruments of one event cannot change the ordering within it (Proposition 1; in Table 6 the 5 label-free TTA methods change rank IC by −0.0009 to +0.0001). SIGNAL-TTT does rerank, in closed form: indexed by (event, instrument), using the matured outcomes of retrieved past events, with no backward pass and no weight access, on a frozen model served through an API. We think this is a useful case for test-time training: the output is a cross-sectional ranking, labels arrive with a delay, and the model cannot be fine-tuned. The source code is at the anonymous link in the Reproducibility Statement, so other operators can be tested under the same rules.

Reviewer Uasc has raised the soundness score after reading the revision. We would be grateful for further comments, and we would ask you to consider whether the revised evidence solved your concerns.

The Authors

Reviewer_VsSwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adjusts frozen language-model asset scores using instrument-specific residuals from similar past events, with empirical-Bayes shrinkage. Across nine backbones and 321 development events, it reports better ranking and relative net returns, although this window was examined repeatedly during method development.

Critical strengths
  1. Direct treatment of the ranking problem. The paper correctly identifies that a shared strictly increasing correction cannot change the ordering within an event. Fitting a separate correction for each instrument directly addresses this limitation.

  2. Simple and inexpensive update. The correction has a closed-form solution and requires no access to model weights or gradients. This makes it practical for frozen models queried through an API.

  3. Useful evaluation controls. The paper uses matured historical outcomes, includes transaction costs, and reports several retrieval and calibration controls. These choices make the source of the gain easier to examine.

Critical weaknesses
  1. No independent test of generalisation. My main concern is that no held-out period is reported. Table 18 shows similar gains for the three backbones released before the development window, using the same repeatedly examined events.

  2. Limited evidence for the full design. In Table 8, direct replacement predicts from retrieved outcomes in place of frozen scores and leads on development (Rank IC 0.0425 vs. 0.0186). The residual update leads on validation. Fully shuffling outcomes within each instrument retains about three quarters of the Rank IC gain over frozen scores (Table 10). Return gains over retrieval baselines do not survive Holm correction. Together, these results suggest that instrument-specific calibration accounts for much of the gain. How much the frozen scores and event-specific retrieval add remains unclear.

  3. Claims need a narrower scope. The introduction’s claim of universal gains is too broad given that Rank IC improves on eight of nine backbones. At the stated cost, mean net returns remain negative for both the threshold portfolio and the rank-only quintile portfolio. The portfolio claims should focus on the improvement over frozen scores.

Questions for authors
  1. Can the authors report results on a locked later period using models available before it began?

  2. On that period, how much does the proposed update add over static per-instrument calibration and the direct replacement in Table 8? Please include paired uncertainty. Could the authors also clarify how the settings differ between direct replacement in Table 8 (Rank IC 0.0425) and the Nadaraya–Watson baseline in Table 2 (0.0162)?

  3. Can the authors report net returns and trading coverage for the threshold and rank-only portfolios side by side on that period?

4 comments
AuthorsOpenReview ↗

Response to Reviewer VsSw

Thanks for your carefully review we believe with revise can make it interest topic better. Table, line and appendix numbers refer to the submitted PDF. Brackets are 95% CIs from the paper's 5-day block bootstrap; p-values are one-sided (sign-flip for the forward endpoint). Post-unseal marks descriptive analyses run after the forward window was scored, with the evaluator unchanged.

W1. Generalisation and look-ahead. Release dates do not explain the development gain. On the 3 backbones released before the development window (Table 15), SIGNAL-TTT − frozen is +2.32 [0.97, 3.63] bps per event (p = 0.0003, post-unseal), equal to the 9-backbone mean of +2.32 (Table 2). The held-out test is in Q1. Change: Section 5 adds this result.

W2. Evidence for the full design.

  • (a) The significance claims are against frozen and the label-free TTA methods (Holm p ≤ 0.004, Table 2); the retrieval comparisons are positive but not significant after Holm (p 0.169–0.921), and we state them that way.
  • (b) Shrinkage is chosen on validation, where it is better (0.0461 vs 0.0409, lines 443–444). Without shrinkage, development rank IC is 0.0245 vs 0.0186 (Table 4); switching to the development winner would be selection on the evaluation window, the bias raised in W1.
  • (c) The controls show three things. The correction must vary across instruments within an event: a shared offset gives 0.0027 vs 0.0030 frozen (Table 4) and five label-free TTA methods give −0.0009 to +0.0001 (Table 6). It must use each instrument's own outcomes: across-instrument permutation p = 0.008 (Table 9). Text retrieval chooses the evidence: random neighbours give 0.0057 and market-state neighbours 0.0015, against 0.0186 (lines 445–449). Within-instrument permutation keeps 0.0149 (Table 10), so most of the gain is instrument-level; the event-specific part (about 0.0037, p = 0.069, Table 9) is positive but not significant. Change: lines 449–450 are rewritten; App. F.4 gives this decomposition.

W3. Scope of claims. The paper does not apply Proposition 1 to the threshold book: it is stated for rank metrics and rank-only books (lines 233–236), and lines 212–215 state that the evaluated book uses the ±0.20 margin. Q3 separates the two effects. Table 3 reports net return relative to the worst frozen backbone; lines 410–411 already give the absolute values: −4.36 bps per traded event (frozen) and −0.39 (SIGNAL-TTT). Change: Observation ❶ states the improvement over frozen separately from profitability.

Q1. Locked later period. Done. Forward window: 558 events, 2026-04-13 to 2026-09-21, all 9 backbones. The configuration and the primary endpoint (SIGNAL-TTT − frozen, paired net bps per event, untraded = 0, 2 bps per leg) were hashed into a registry on 2026-09-23, before any forward outcome was scored. In this window the frozen backbones carry no signal (mean frozen rank IC 0.0375 validation, 0.0030 development, −0.0080 forward), and no operator exceeds +0.002 rank IC. SIGNAL-TTT has a higher rank IC than frozen (+0.0021, p = 0.34), every label-free TTA method and the shared offset, and its rank-only book is ahead of frozen (+0.64 [−0.53, 1.60]). The registered net endpoint is −0.31 [−0.97, +0.35] (p = 0.78); affine, which trades 109 of 558 events, is ahead of SIGNAL-TTT on the threshold book (−2.11 [−3.19, −0.97]). The forward differences against frozen are not significant. Change: the paper states its performance claims for the development window.

Q2. Value beyond simpler calibration. Development rank IC: affine 0.0169, kNN-LM 0.0178, SIGNAL-TTT 0.0186. Tuning budget (configurations per backbone on validation; App. D.3): SIGNAL-TTT 6, kNN-LM 18, affine 16. SIGNAL-TTT − rival, development, 9 backbones, bps per event:

rivalthreshold book (Holm p)rank-only book (k = n/5)
affine calibration−0.96 [−2.21, +0.03] (0.939)+1.56 [−0.32, 3.66], p = 0.062
kNN-LM+0.57 [−1.62, +3.25] (0.921)+1.04 [−0.58, 3.03], p = 0.092

Affine's threshold-book number comes from abstention: untraded events count as 0 and the frozen book loses money, and affine trades 76.2 of 321 development events against 201.3 for SIGNAL-TTT (every other operator trades 189–295). When every operator trades every event, SIGNAL-TTT is ahead of affine: +2.88 [0.38, 5.42] (k = 3) and +2.06 [0.09, 4.13] (k = 5). Change: App. F.1 gives Table 2 with the affine row and traded counts; Section 5 cites it.

Q3. Ranking versus threshold effects. Done. Book (post-unseal): per event, long top-k / short bottom-k by corrected score, k = n/5 rounded, same costs; every operator trades every event. SIGNAL-TTT − frozen, 9 backbones: development +2.60 [1.03, 4.50] bps per event (p < 0.001), larger than the threshold-book gain, so the development gain comes from reordering, not threshold crossing. Change: App. F.2 adds the rank-only book.

AuthorsOpenReview ↗

Summarize of rebuttal revisions

We have uploaded a revised PDF. All changes are in blue, and the body is still 10 pages. Line numbers refer to the revised PDF.

  • W1 (look-ahead).
    • Section 5.2, Per-Backbone Analysis (line 433), adds the result on the 3 backbones released before the development window: +2.32 [+0.97, +3.63] bps per event and +0.0159 rank IC. This is the same gain as on all 9 backbones.
    • Appendix E.1 states the same result, and Appendix F.3 (Table 18) gives the full numbers.
  • W2 (evidence for the design).
    • The Ablation paragraph (line 453) adds the across-instrument permutation: permuting outcomes across instruments within each event removes the gain (p = 0.008). The gain therefore depends on matching each instrument to its own retrieved outcomes.
    • Appendix F.4 gives the full permutation decomposition.
    • Appendix D.3 lists the per-backbone tuning grid of every operator under the same selection rule: SIGNAL-TTT 6, kNN-LM 18, affine 16.
  • W3 (scope of claims).
    • The abstract (line 019) states that Proposition 1 covers rank metrics and every portfolio built from the ordering.
    • The absolute net returns (−0.39 against −4.36 bps per traded event) stay in Observation ❶ (line 413).
  • Q1 (locked later period).
    • The forward window is reported in our response comment above (558 events, registered before scoring).
    • The conclusion (line 488) states the performance claim for the development window.
  • Q2 (value beyond simple calibration).
    • Observation ❶ (line 365) adds that SIGNAL-TTT leads the per-instrument affine calibration on rank IC (0.0186 vs 0.0169). It also leads when every operator trades every event: +2.88 and +2.06 bps per event with 3 and 5 instruments per leg, both intervals above zero.
    • Line 415 adds that affine's positive net per traded event is earned on 76.2 traded events.
    • Appendix F.1 (Table 16) gives Table 2 with the affine row and columns for traded events and net per traded event. It shows that affine's threshold-book difference comes from abstention.
  • Q3 (ranking vs. threshold effects).
    • Appendix F.2 (Table 17) adds the rank-only book, in which every operator trades every event.
    • On development, SIGNAL-TTT − frozen is +2.60 [+1.03, +4.50] bps per event. This is larger than the threshold-book gain, so the gain comes from the ordering.

Hope the revision and clarify in rebuttal can solve your concerns and consider increase the rating of the paper. If you have any continue concerns please let us know.

AuthorsOpenReview ↗

Friendly reminder and short summary, we really value your concerns and hope rebuttal can solve them also increase the score

Dear Reviewer VsSw,

We understand you are busy. This is a short reminder that our response of this unique research with a revised PDF (changes in blue). We summarise each point below.

W1 and Q1 — look-ahead, and a locked later period. The 3 backbones released before the development window give the same gain as all 9, so release dates do not produce it; their weights cannot contain development-window outcomes.

SIGNAL-TTT − frozen, development3 pre-development backbonesall 9
net bps per event+2.32 [0.97, 3.63], p = 0.0003+2.32 [1.30, 3.09]
rank IC+0.0159 [0.0006, 0.0328], p = 0.021+0.0156
rank-only book, k = n/5+2.58 [0.54, 4.99]+2.60 [1.03, 4.50]

Per backbone: +1.70 (Qwen3-8B), +3.66 (Qwen3-14B), +1.61 (Seed-OSS-36B). We also registered a forward window before scoring it (558 events, 2026-04-13 to 2026-09-21); the full result is in our response comment. There the frozen backbones carry no signal (rank IC −0.0080); SIGNAL-TTT has a higher rank IC than frozen, every label-free TTA method and the shared offset, and its rank-only book is ahead of frozen (+0.64 [−0.53, +1.60]). The registered net endpoint is −0.31 [−0.97, +0.35], not significant; the paper states its performance claims for the development window.

W2 — evidence for the full design. Each control isolates one part of the mechanism.

control (development, rank IC)result
frozen0.0030
shared per-event offset0.0027
5 label-free TTA methods (level)0.0021–0.0031
text retrieval → random neighbours0.0057
text retrieval → market-state neighbours0.0015
outcomes permuted across instruments within eventp = 0.008
SIGNAL-TTT0.0186

Shrinkage is chosen on validation (0.0461 vs 0.0409). Without it development rank IC is 0.0245, and we do not switch to that, because it would be selection on the evaluation window — the bias you raise in W1.

W3 and Q3 — scope, and ranking rather than thresholds. Proposition 1 is stated for rank metrics and rank-only books. To test whether the gain is ranking, we added a rank-only book: long the top k, short the bottom k, every operator trades every event, same costs.

SIGNAL-TTT minus, development, bps per event− frozen− affine− kNN-LM
k = n/5+2.60 [1.03, 4.50]+1.56 [−0.32, 3.66]+1.04 [−0.58, 3.03]
k = 3+4.09 [1.85, 6.52]+2.88 [0.38, 5.42]+0.99 [−1.63, 4.38]
k = 5+2.86 [1.37, 4.71]+2.06 [0.09, 4.13]+1.48 [−0.77, 3.98]

The gain against frozen is larger without the margin, so the development gain comes from reordering, not from crossing a threshold. The absolute net per traded event (−4.36 frozen, −0.39 SIGNAL-TTT) is stated in Observation ❹, beside the relative values of Table 3.

Q2 — value beyond simpler calibration. SIGNAL-TTT has the highest rank IC of all operators, the highest net return per traded event among the operators of Table 2, and a smaller tuning grid than any retrieval or calibration rival (App. D.3).

development, 9 backbonesrank ICnet per traded eventconfigurations
frozen0.0030−4.36—
affine calibration0.0169+0.42 (on 76.2 of 321 events)16
kNN-LM0.0178−1.9718
SIGNAL-TTT0.0186−0.396

When every operator trades every event, SIGNAL-TTT is ahead of affine at every k, with intervals above zero at k = 3 (+2.88 [0.38, 5.42]) and k = 5 (+2.06 [0.09, 4.13]); see the W3 table. Affine's threshold-book figure comes from abstention: untraded events count as zero while the frozen book loses money, and every other operator trades 189–295 events against affine's 76.2.

Why we think this is worth a second look. A correction shared by all instruments of one event cannot change the ordering within it; we prove this as Proposition 1, and Table 6 shows the 5 label-free TTA methods change rank IC by −0.0009 to +0.0001. SIGNAL-TTT is the operator that does rerank: a closed-form residual correction indexed by (event, instrument), using the matured outcomes of retrieved past events. It needs no backward pass and no access to model weights, so it applies to a frozen model served through an API. We think this is a useful case for test-time training: the output is a cross-sectional ranking, labels arrive with a delay, and the model cannot be fine-tuned. The source code is at the anonymous link in the Reproducibility Statement, so later operators can be compared under the same rules.

Reviewer Uasc has raised the soundness score after revision. We would be grateful for any further comments, and we would ask you to consider whether the revised evidence solved your concerns.

The Authors

AuthorsOpenReview ↗

Equal tuning budgets and a later test on released models

Dear Reviewer VsSw,

You asked whether our development gain survives a change of period, an equal tuning budget, and removal of the trading threshold. The paired results are below.

SIGNAL-TTT comparisonNet bps per event [95% CI]
Development vs. frozen, original threshold portfolio+2.32 [+1.30, +3.09]
Development vs. frozen, trading on ranks alone+2.60 [+1.03, +4.50]
Development vs. affine calibration, six tuning trials each+0.22 [−0.61, +0.89]
Development vs. kNN-LM, six tuning trials each+0.80 [−1.06, +3.06]
Later window vs. frozen, six models released before it began−0.46 [−1.24, +0.37]

For the exploratory equal-budget check, we averaged over every six-candidate subset of each rival’s grid, selecting on validation. Both intervals include zero. The six-model forward comparison was registered before outcome scoring. The all-nine result on the same 558 events was −0.31 [−0.97, +0.35].

On the 321 development events, instrument-specific corrections improved even the rank-only portfolio, which a shared strictly increasing adjustment cannot change. The return gain did not replicate in the later period.

If these checks address your concerns about timing, tuning and ranking, we would be grateful for any further comments, and we would ask you to consider whether the revised evidence solved your concerns.

Best

The authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Friendly Reminder to Reviewers

Dear Reviewer VvSw & NMc6,

Thanks for your attention to this email. During the rebuttal phase, we extensively answered the questions raised by the reviewers and believe we have addressed the reviewers' concerns also make this interesting work better. This is a friendly reminder that the author-reviewer discussion period is coming to an end. We understand you are busy, but we would appreciate any additional feedback or suggestions you might have on our work. Your insights are valuable to us and SignalTTT.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.