- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies predictive state scope in temporal representation learning: where historical information should be stored and how it should be retrieved at prediction time. The authors propose FA-State, which maintains a separate recurrent state for each directed pair alongside a shared temporal backbone, using a shared GRU to update relational history. At prediction time, the model retrieves the historical state associated with each candidate pair and combines it with the backbone’s prediction. The paper also uses Bayes risk and random-effects analysis to discuss predictive aliasing and the trade-off between shared and local states.
Experiments cover synthetic data, temporal graph forecasting, and temporal knowledge graph prediction. FA-State improves prediction performance across multiple backbones, with particularly notable gains on tgbn-trade. Frozen retrieval experiments further show that matching historical content to the correct retrieval address matters for prediction. However, the lack of direct comparisons with the most relevant strong baselines leaves FA-State’s practical advantage over existing methods unclear.
Critical strengths
-
The research question is meaningful. The paper examines how historical information is stored and retrieved, focusing on the predictive aliasing that shared states can introduce and the trade-off between shared and pair-specific states. Although the method itself is relatively simple, the state-scope perspective offers useful insights into temporal representation learning.
-
The experimental results are reasonably convincing. FA-State improves performance across multiple temporal backbones and datasets, with especially large gains for TGNv2 on tgbn-trade. The authors also use synthetic data to control relational dependence and recurrence, helping clarify how the method behaves under different conditions.
-
The ablations and analyses are fairly thorough. Beyond the standard shared-only and wrong-address controls, the authors conduct frozen retrieval experiments that change only the retrieval addresses while keeping model parameters and the memory bank fixed. These provide stronger evidence than conventional ablations that the model uses the correct correspondence between historical content and its address.
Critical weaknesses
-
The methodological novelty is limited, and the need for persistent pair-specific memory remains unclear. FA-State essentially adds a separate GRU state for each directed pair to an existing temporal backbone and combines predictions through additive logits. Related structured or factorized memory ideas have been explored in Neural Datalog Through Time, FACTS, and other work. Transformers or cross-attention could also retrieve pair-specific history dynamically at prediction time without maintaining persistent states. Although the wrong-address and frozen retrieval experiments establish the importance of correct addressing, they do not show that FA-State’s design is better than existing approaches to history modeling.
-
Important baseline comparisons are missing, and the evaluation protocol is not comprehensive enough. The paper mainly compares against backbones such as TGN, TGAT, and TGNv2 and their variants, without directly evaluating strong methods for the same task, such as NAVIS (ICLR 2026), even though NAVIS is discussed in Related Work. The main experiments also use occupancy-matched candidate sets, restricting prediction to previously observed relations, which may favor pair-specific memory. Although full-candidate NDCG is reported, the models are still trained with a matched objective. A fair comparison with stronger baselines under a standard full-candidate protocol is therefore needed.
-
The theoretical analysis is only loosely connected to the actual method. Proposition 1 largely presents the standard relationship between conditional mutual information and Bayes log-risk. It applies to compressed representations in general and does not directly establish a specific advantage for FA-State. Proposition 2 uses a classical Gaussian random-effects model to discuss the shared–local trade-off, but the architecture does not implement the recurrence-dependent shrinkage emphasized by the analysis. These results help explain why retaining relational history may be useful, but do not sufficiently justify FA-State’s particular way of organizing that history.
Questions for authors
-
Could the authors add comparisons with strong baselines such as NAVIS under a standard full-candidate protocol? Simple baselines based on pair-specific historical statistics or time-decayed aggregation would also help clarify FA-State’s practical advantage.
-
Why is a persistent recurrent state needed for each directed pair? Could the authors compare FA-State with query-time pair-conditioned history modeling using Transformers or cross-attention? Ideally, these comparisons would use the same backbone and historical inputs and report predictive performance, memory usage, and computational cost.
-
Proposition 2 emphasizes recurrence-dependent shrinkage, whereas FA-State mainly uses a global learned gain. Have the authors considered dynamically weighting shared and local predictions according to the amount of history available for each pair? Could this improve performance on sparse or low-recurrence relations?