acceptodds
Withdrawn from ICLR 2027

Same-Checkpoint Intervention on Pre- and Post-Context States in Entity-Local Temporal Encoders

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

21 views

Abstract

Relational structure can shape a temporal encoder during training or be mixed directly into its deployed representation. We introduce a same-checkpoint diagnostic that separates these roles by exposing entity-local pre-context and message-passed post-context states from the same trained system, using a nonstationary equity panel as a stress test. Under a historical residual-correlation adjacency that ranks neighbors by and discards sign, switching retrieval from to reduces rank correlation on 19/21 forward-state descriptors in the three-seed mean; 15 are negative in every seed and none is positive in every seed, with losses of 0.086–0.153 on eight highlighted descriptors. A sweep-selected, two-seed exploratory endpoint suggests that stronger graph-conditioned training can reallocate the unmixed state toward volatility/range geometry while degrading market-factor geometry, but the absence of a content-free shuffled-graph control and a third seed prevents attribution to relational edge content. The main result is therefore a controlled negative finding and a reusable diagnostic principle: training-time use of structure and inference-time relational propagation should be evaluated separately.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
representation learning, temporal representation learning, graph neural networks, self-supervised learning, contrastive learning, time series

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_TV42OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the roles of relational information as training-time supervision and inference-time message passing. Through same-checkpoint comparisons on financial time-series data, the authors find that graph propagation generally degrades representation quality in future-state retrieval tasks.

Critical strengths

The paper addresses a meaningful question about the distinct roles of relational information in representation learning. The same-checkpoint comparison provides a controlled way to evaluate inference-time graph propagation, and the experiments offer useful empirical insights into its potential limitations.

Critical weaknesses

1.The central research question is not inherently tied to time-series learning, yet the experiments are limited to a single financial time-series dataset. This makes it difficult to disentangle the effects.

2.The findings may depend on the specific graph construction and model architectures, while the existing ablations are insufficient to establish their generality.

3.Although the authors acknowledge this limitation, the lack of a shuffled-graph control makes it difficult to determine whether the observed improvements are driven by meaningful relational information or merely by the regularization effect.

Questions for authors

1.Could the authors evaluate the proposed diagnostic on non-temporal graph datasets to better isolate the effects of graph propagation?

2.Do the conclusions generalize to other datasets and tasks?

3.How sensitive are the findings to different graph constructions and model architectures?

4.Could the authors include a shuffled-graph control?

Reviewer_HD5yOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks, when relational context is informative, whether relational structure is more useful when mixed directly into the representation an encoder uses at deployment, or when used as a training signal that shapes otherwise entity-specific temporal embeddings. From one jointly trained system it exposes two states: an entity-local pre-context state z^pre, computed only from an entity's own history without cross-entity message passing, and a post-context state z^post obtained after GCN/GAT message passing over a historical residual-correlation graph; comparing the two from the same checkpoint is the proposed diagnostic. Daily data for 499 U.S. large-cap equities serve as a stress test for nonstationary relational representation learning, not as the object of study. Retrieving with z^post instead of z^pre lowers rank IC on 19/21 forward-state descriptors across three seeds, with losses of 0.086–0.153 on eight highlighted descriptors (Table 1); exploratorily, a larger auxiliary weight (λ_ctx = 0.50) reallocates z^pre toward volatility/range descriptors and away from market-factor descriptors (Tables 2 and 6).

Critical strengths

S1. The same-checkpoint design is a clean and reusable way to isolate inference-time propagation. Typical evaluations compare a graph model with a separately trained no-graph model, which confounds the effect of the propagation operator with changes in optimization, capacity and training trajectory. Scoring one checkpoint immediately before and after its own context block holds the encoder parameters and training trajectory fixed (Sec. 1, Fig. 1, Sec. 4.5). The question it serves, "context as representation" versus "context as supervision", is relevant to the many encoder-plus-GNN systems used in temporal and relational learning. The paper also positions this design honestly: it explicitly does not claim graph-derived supervision without inference-time message passing as novel, and it distinguishes its diagnostic from the distillation line (Graph-MLP, GSSC, GLNN, Cold Brew, NOSMOG; Sec. 2, p. 4). This design is the main reason I lean toward acceptance, because it can be applied beyond the finance testbed.

S2. Careful statistics and transparent reporting of claim status. The paper uses chronological splits with an embargo (Sec. 5.1), a moving-block bootstrap suited to overlapping 21-day targets, Romano–Wolf family-wise correction across correlated targets, a synthetic-null calibration (App. E: about 72% spurious "winners" with naive testing vs. about 5% with stepdown), per-target strongest baselines including simple factor summaries (Sec. 5.2), full seed-level deltas (Table 4), and an audit separating supported, exploratory and untested claims (Table 7). This raises my confidence that the reported numbers mean what they say, and the authors avoid overclaiming (e.g., they decline the phrase "consistently harmful", Sec. 6.1).

Critical weaknesses

W1. The headline E10 comparison contains confounds that favour z^pre, so the "same checkpoint" control is weaker than claimed.

  • Train/test graph mismatch that affects only z^post. App. B.3 (pp. 13–14) and Sec. 8 state that training and retrieval-bank encodings use 48-name graphs, while validation and test queries can build the residual graph over up to 200 names. In addition, bank states for dates with fewer than 120 prior sessions are left at their zero initialization, and "the exact fraction of E10 bank anchors affected … was not recomputed". Neither issue affects z^pre. The z^post retrieval therefore compares queries and bank entries computed under different graph distributions, using a GCN trained only on 48-node graphs, and possibly includes zero vectors in the bank. Part or most of the penalty in Table 1 could be an evaluation artifact rather than an effect of propagation.
  • The two views are trained toward different objectives. As I read Sec. 4.2–4.4, the temporal loss L_rep shapes z^pre, while z^post is optimized only by L_ctx, which pulls |ρ|-neighbours together. Because the sign is discarded (Sec. 4.3), strongly anti-correlated residual series also become positive neighbours. A state trained to resemble its neighbours, including anti-correlated ones, is expected to lose entity-specific volatility geometry. The comparison is therefore closer to "temporal-objective state vs. neighbour-contrastive state" than to an isolated test of the propagation operator. A self-weight of 2 before normalization (App. C, p. 14) and the unreported learned mixing weight α (Eq. 9) add further uncertainty.
  • The state being "degraded" is itself weak. At λ_ctx = .05, the E10 setting, z^pre has no target that beats its strongest simple baseline (Table 2, "+FWER" = 0). Table 5 reports an effective rank of 1.07 out of 64 for z^pre in the residual-GCN arm, i.e., z^pre is nearly one-dimensional. If z^pre is essentially a single volatility-like axis, then the finding that averaging in neighbours hurts entity-specific volatility ranking is close to expected and says little about relational representation learning in general.

W2. The controls needed for the main conclusion are missing, and adding them could reduce the significance of the reported effects. The authors acknowledge that a sign-respecting (signed or positive-only) graph, a degree-matched shuffled graph and a third λ_ctx = .50 seed are untested (Sec. 8, Table 7, App. H). These are not optional extensions. They are exactly what is needed to support the claim that "the two roles are empirically separable" (Sec. 1, p. 2), and either the E10 penalty or the λ_ctx = .50 gains could shrink once they are run. The positive training-branch result is already small and fragile. It rests on one seed-0 sweep, λ_ctx = .50 was chosen after inspecting that sweep, only one additional seed exists, and comparators are chosen on the test set (Sec. 6.3, Sec. 8). From λ_ctx = 0 to .50, volatility rank IC rises only from 0.526 to 0.548 and HL range from 0.666 to 0.687 (Table 2). Gains over the strongest baseline are +0.019 to +0.028 (Table 6), and the four survivors are one correlated factor. Market-correlation geometry falls from +0.063 to −0.051 and is 0.630 rank IC below a rolling-beta baseline (Sec. 6.4). Finally, the diagnostic is demonstrated on only one domain, which the authors themselves call a stress test rather than the object of study (Sec. 1), with one graph construction and no second domain (Table 7, last row). Its claimed generality is therefore not yet shown.

Questions for authors
  1. Can you re-score E10 z^post with bank and query graphs built identically (e.g., both on 48-name sampled cross-sections, or both on the full live cross-section), and report the fraction of zero-initialized bank states? If the 0.086–0.153 penalty largely survives, my main soundness concern would be substantially reduced.
  2. Is L_rep applied only to z^pre? If so, has z^post ever been trained with the temporal objective (e.g., L_rep on z^post)? Without such a variant, how do you rule out that the penalty reflects different training targets rather than propagation itself?
  3. What are the learned values of α in Eq. 9 for the three E10 seeds? Does seed-to-seed variation in α explain the much larger post-context SD (about ±0.03 vs. ±0.002 pre-context, Table 1)?
  4. Is the Table 5 "residual GCN" arm the E10 configuration, with an effective rank of 1.07 for z^pre? At λ_ctx = .05, which comparator is strongest for the volatility/range targets, and by how much does z^pre trail it?
  5. Sec. 6.1 states the effect "persists after replacing the original common-factor-dominated adjacency with residual and partial-correlation alternatives". Where are the rank-IC results for the partial-correlation graphs? Table 5 reports only cosine similarity and effective rank.
Reviewer_BQRrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This manuscripts attempts to study the way (presumably when and how) temporal information should be injected into temporal representation learning models. This is done by examining two distinct representation mechanisms -- (1) entity-specific embeddings derived from a single entity in isolation, and (2) context aware embeddings obtained after applying message passing to the spatio-temporal entity. The authors conduct this study using historical equity market data (finance time series dataset). Here, relationships between different stocks are taken as the edges over which message-passing occur. The manuscript concludes that graph-based relational information can be comparatively more beneficial as an auxiliary training signal than through inference-time message passing, the latter of which often degrades entity-specific temporal representations in the evaluated setting.

Critical strengths

The authors report transparently report negative results, which is scientifically encouraged, and, frankly, refreshing. Potential confunders are sometimes discussed and some findings are flagged as underexplored.

Critical weaknesses
  1. The entire evaluation is conducted on a single stock-market dataset. Without significantly expanded empirical evaluation (eg across different domains, distributional behavior, careful and interpretable synthetic experiments etc), the central claims of the work cannot be substantiated.

  2. The main tested graph uses absolute residual correlations and discards correlation signs and thus positively and negatively correlated entities will be treated as equivalent neighbors during aggregation. Therefore, the observed degradation cannot be clearly distinguished from a consequence of this particular, and seemingly trivial, graph construction. This is one example of sloppy experimental controls that undermine the central claims of the paper.

Questions for authors

Does the demonstrated inference time performance degradation still persist when using either positive-only residual correlations (baseline) or a sign aware graph operator?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.