- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The authors consider temporal link prediction and propose to keep track of the "current roles" that nodes play at the points in time when a prediction should be made. They adopt a mixture-of-experts approach and define experts relating to, for example, nodes' history, nodes' behaviour regarding repeated interactions, or nodes' behaviour regarding reciprocity in their interactions. When a query arrives that asks "do node u and v interact at time point t", they check what u's predicted "need for an interaction partner" at time t is and compare that with v's estimated role at that same time t. They do this by using attention to learn a routing mechanism. In their evaluation, the authors consider a standard range of benchmark datasets and TGNN models for temporal link prediction, but focus specifically on benchmarking against CRAFT. The authors' method, RuleCast, achieves competitive performance, and an ablation study shows the importance of properly including aspects such as the nodes' history.
I found the paper somewhat difficult to read: with typically several ideas packaged into the same sentence, it is difficult to figure out what the most important message is. The authors also seem to assume quite some familiarity with mixture-of-experts approaches, which, given ICLR as the venue, seems defensible. However, spending a paragraph on introducing the relevant terminology would have made the paper more accessible.
The three research questions mentioned in section 5 do not appear to be properly answered in the main text. Moreover, the Discussion and Conclusion sections are each merely a short paragraph long. I believe this leaves the reader wondering how to make sense of the present work. What exactly are the aspects RuleCast that make it work well? Why did the authors choose to let CRAFT stand out as the seemingly most important baseline? What are the failure modes of RuleCast, when does it not work well?
Critical strengths
The paper brings established ideas from other areas of deep learning to temporal link prediction, specifically, the mixture-of-experts approach.
Critical weaknesses
- The writing is dense in the sense that too many ideas are packaged into the same sentence. This makes it unnecessarily hard for the reader to figure out what the authors want to say.
- The authors did not answer or discuss the three research questions they formulated.
- The Discussion and Conclusion sections are extremely short, each containing only one short paragraph.
Questions for authors
There are a couple of points that remained unclear to me
- Why did you choose to give so much focus to CRAFT in the evaluation? I think this should be explained to the reader.
- How small is a "small MLP" typically?
- There is a statement in the discussion that I did not understand: "The direct controls isolate the role-protocol factorization from decoder capacity". Could you explain what you mean by this and where you have shown this?
- There is another statement, where I'm wondering where the paper shows this: "the expert set covers complementary predictive signatures"? Have you shown that they are complementary? Have you verified that this is the case?
- Can you say something about the failure modes of RuleCast? What would be an example where RuleCast doesn't work well? Are there specific patterns we should expect it has problems with?
Minor points
- Figure 2 is mentioned before Figure 1 in the text, but it appears 2 pages after Figure 1.
- M is first mentioned in l.135 but not defined until l.212
3 comments
Thank you for your answers, which helped me better understand some aspects of your work. I appreciate your efforts to address my concerns, for example, by returning to the research questions in the text and adding another paragraph to the conclusion, which is why I am raising my score.
However, I believe the paper would benefit from a more thorough revision to improve clarity and better integrate the new content into a coherent story.
Response to Reviewer kFAz (Follow-up on clarity and how the evidence fits together)
Thank you for reading the revision and pointing out that the new material needed to fit more clearly into the paper's argument. We revised the text so that the same question guides the model and the analysis: what can we predict once for the source, and what must we check separately for each candidate?
How we introduce the model (W1). The Introduction now states the distinction before giving examples. The source's past partners inform a forecast of what kind of partner is likely to come next. All candidates for that source and time share the forecast. Sections 4.2–4.3 then distinguish the forecast from a candidate's own current role and explain how role fit, history matching, and pair evidence enter the score. Section 4.4 refers to Figure 2 after defining the score, so the diagram brings together the quantities the reader has just seen.
How each experiment answers a question (W2 and Q3). Section 5 moves from overall benchmark performance (RQ1) to what the forecasting design contributes (RQ2) and when learned routing helps (RQ3). For RQ2, Table 7 asks how RuleCast compares with direct feature scoring. Table 8 separates two questions: State-only removes the past-partner sequence while retaining the current source state, and Simplex versus Continuous compares ways of representing the forecast and scoring its match to a candidate. The two matched representations give close MRR on UCI and LastFM. With role distributions, we can also see which learned roles contribute to the match before other evidence is combined.
For RQ3, Section 5.4 follows the order of scoring: it first uses Figure 3 to ask whether realized partners fit the forecast better than sampled candidates, then examines how the router weights the available evidence. Figure 4 asks a different question: does learning those weights improve ranking relative to uniform weights? The clearest gain on Flights is about 2.0 MRR points for new partners. The text now points from Figure 4 to Appendix I.2, where the new Figure 7 shows the corresponding comparisons on LastFM, YouTube, and MOOC.
What changes when we mask evidence (Q4). Table 9 (Appendix I.1) fixes the checkpoint and candidate lists, then masks expert groups at inference. The rows below show the comparisons discussed in our first response. Values are test MRR (%). Every setting retains the role expert and identity residual; Pair groups repetition, reciprocity, and periodicity.
| Dataset | Evidence for the true partner | Queries | Role | +History | +Pair | Full |
|---|---|---|---|---|---|---|
| Flights | History and pair | 3,592 | 84.32 | 84.39 | 98.38 | 98.53 |
| Wikipedia | History and pair | 3,680 | 56.54 | 46.86 | 99.83 | 99.86 |
| Wikipedia | History only | 270 | 26.34 | 28.60 | 26.34 | 28.60 |
When both evidence types are available for the true partner, adding history to Role + Pair raises MRR by 0.15 points on Flights and 0.03 on Wikipedia. In Wikipedia's history-only stratum, adding history to Role raises MRR from 26.34 to 28.60. This comparison helps explain what history evidence can add while the role forecast stays fixed.
We now link this experiment to Proposition 3 in Appendix E. The proposition gives the exact change in a candidate's score when a group of experts is masked and the remaining weights are renormalized. If the original score gap exceeds the bound on how much it can shrink, the two candidates keep their order. This explains why changes in the evidence scores can affect some rankings and leave others unchanged.
What this means for the model (W3 and Q5). Section 6 returns to a concrete example: two candidates can fit the same forecast equally well, but a prior interaction with one of them can help distinguish them. When the source has little history and the candidates have similar behavior and no pair evidence, there is less information to separate them. Appendix J illustrates this distinction for one source: it traces how the forecast changes and compares the weighted expert terms for the realized partner and a high-scoring sampled alternative at selected times. The conclusion returns to the same idea: predict the kind of partner first, then use the available evidence to rank individual candidates.
Response to Reviewer kFAz
Thank you for the detailed feedback. Your questions helped us clarify the exposition and explain what the experiments establish.
W1: Clarity and Figure 2
We revised the exposition in the Introduction and Sections 3–4. The Introduction now separates partner-role forecasting from candidate scoring, and Section 3 explains what the experts score, what the mask excludes, and what the router weights. We redrew Figure 2 to show how the shared forecast and candidate-specific evidence produce the final score. Section 4.4 introduces the figure after the scoring equations.
W2: Answers to the research questions
Section 5.1 lists RQ1–RQ3 explicitly. RQ1 (Section 5.2): RuleCast has the highest reported MRR on all ten datasets (Table 1) and higher AP and ROC-AUC on nine (Table 2). RQ2 (Section 5.3): RuleCast outperforms direct feature scoring, and the partner sequence helps on UCI and LastFM, where the matched simplex and continuous controls are close. RQ3 (Section 5.4): router weights vary with the available evidence, with the clearest ranking gain over uniform routing for new partners on Flights.
W3: Discussion and conclusion
We expanded the Discussion (Section 6) to connect the matched controls and the Flights evidence strata with concrete failure modes (see Q5). Section 7 summarizes the shared partner-role forecast, candidate-specific scoring, and MRR gains.
Q1: Why compare with CRAFT
CRAFT directly matches each candidate against the source's history. RuleCast instead forecasts a partner-role distribution shared by all candidates before scoring them. CRAFT is also among the strongest baselines in Table 1. Comparing the two helps distinguish how they use history (Introduction and Section 2).
Q2: Expert MLP dimensions
We give the dimensions in Section 4.3 and Appendix A: pair experts map two inputs to one score through a 64-unit hidden layer; the history expert has the same width and 193 inputs at d=64. Both use GELU and dropout of 0.1.
Q3: What the direct controls test
"Isolate" was too strong: the architectures and parameter counts differ. We meant to test whether Feature-MLP's direct scoring could match RuleCast's partner-role forecasting and grounding. Section 5.2 now says the controls "compare other scoring architectures under the same evaluation settings" (split, sampler, evaluator). RuleCast ranks higher on five datasets (Table 7 and Appendix H text). Appendix J illustrates the role forecast and expert contributions. Table 8 matches parameter counts for simplex and continuous scoring on UCI and LastFM; their results are close.
Q4: Evidence complementarity
Table 9 masks expert groups at a fixed checkpoint with the same candidates, without retraining. When the true partner has both evidence types, adding history to Role + Pair raises MRR by 0.15 points on Flights and 0.03 on Wikipedia. When it has only history evidence on Wikipedia, adding history to Role raises MRR by 2.26 points. Pair evidence accounts for most of the overall gain, and the first comparison shows that history can complement it (Section 5.4 and Appendix I.1).
Q5: Failure modes
On Reddit, RuleCast gains 5.36 MRR points over CRAFT, but AP and ROC-AUC fall by 0.12 and 0.24 points (Tables 1–2). Section 6 also gives a tie case: without the identity residual, candidates with identical behavioral states, pair features, and masks tie for a fixed source and time. Sparse histories leave few past partners for forecasting and little evidence to distinguish candidates. A prior for nodes with little history could use node attributes when available.
Other revisions. Figure 1 is now cited before Figure 2, and Section 3 defines M as the number of experts when it introduces the expert set.