- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies pretraining for deep learning on relational databases. Here, graph neural networks (GNNs) learn from linked tables to forecast future outcomes for entities such as customers or products, usually with one model per task. The main contribution is Query-Conditioned Relational Pretraining (QCRP). QCRP generates many forecasting tasks from structured query fields (entity type, relational path, aggregation, filter, horizon) and trains one encoder on all of them through a shared query-conditioned predictor. It can also train a single encoder across entity types. On Rel-Amazon and Rel-HM, frozen QCRP encoders beat earlier relational pretraining methods on 6 standard RelBench tasks and 24 new held-out tasks. A single multi-root checkpoint stays close to type-specific specialists, and the query interface supports limited zero-shot prediction.
Critical strengths
1- Consistent frozen-encoder gains, including on held-out tasks. With GraphSAGE, QCRP beats TVE and TVE+CTR on most of the six standard tasks (Table 1). On the 24 held-out tasks, QCRP-root has the best overall mean rank (2.12), ahead of supervised training (2.79) (Table 3). Both QCRP variants beat both TVE baselines on all 12 Amazon held-out tasks (Appx. D.3). Only a 16.9K-parameter head is trained per task, versus 1.36M–2.25M for supervised training (Sec. 3.3). This makes the method practical for reuse.
2-An honest look at benchmark overlap. Table 2 shows that standard RelBench labels can be recovered almost perfectly from TVE pretraining statistics. For example, logistic regression reaches ROC-AUC 1.000 on churn even after removing the matching coordinates. The authors use this to motivate their held-out suite. This analysis is useful to the community beyond this paper.
Critical weaknesses
1- Part of the gain may come from target overlap, not transfer. All six standard tasks are included in QCRP's own pretraining supervision (Sec. 3.2). The largest gain, on item-LTV, comes from a price-times-count target that QCRP pretrains on directly but TVE does not. The authors note this "does not isolate its effect" (Sec. 3.2). The 24 held-out tasks were picked by an AI agent and share query primitives with pretraining (Appx. B). The authors also say the held-out tasks "may share predictive structure with pretraining supervision" (Sec. 3.3). After full fine-tuning, the advantage over TVE+CTR mostly disappears (Table 12). So it is unclear how much of the frozen gain reflects a better representation rather than wider target coverage.
2- The evidence is narrow, and the second and third contributions are weak. The study uses only two databases with two entity types each, and three seeds, with no significance tests ("bold does not imply statistical significance", Table 1). Multi-root training makes Amazon customer worse on all 40 of its pretraining objectives (Sec. 4.2). Zero-shot prediction barely beats swapping in the nearest pretraining query: Spearman rho 0.42 vs 0.41, AUC 68.4 vs 67.1 (Table 4). It is also far below fitted probes on classification (AUC 81.1). The paper's own conclusion says this "leaves open whether it generalizes beyond relationships already represented by nearby pretraining tasks" (Sec. 6).
Questions for authors
1- (Clarification) Can you compare QCRP with TVE when both use the same target pool, or with QCRP after removing all pretraining queries that exactly match or nearly match the downstream tasks? This would show whether the gains come from the training method or from target coverage. 2- (Clarification) How were the 24 held-out tasks chosen by the AI agent? Were any candidate tasks dropped after the prevalence and variance filters? A held-out set built from different query primitives would make the transfer claim stronger.
3-(Clarification) Do the frozen-encoder and multi-root results hold on more RelBench databases with more than two entity types, such as rel-stack or rel-f1?
We thank the reviewer for their detailed and thoughtful assessment. We particularly appreciate their recognition of the consistent frozen-encoder gains across standard and held-out tasks, the parameter efficiency of our approach, and the value of our analysis of target overlap in existing relational pretraining benchmarks. We have provided a general response to shared questions as a standalone comment, which we refer to as G1 through G4.
W1/Q1 — Target overlap versus representation learning.
We agree that broader target coverage contributes to QCRP's improvements, and conducted two additional analyses to quantify its role.
First, our recoverability analysis (G1) confirms substantial overlap: ground-truth pretraining targets recover held-out labels with mean R2 of 0.906 and AUC of 97.03%. However, recoverability from learned predictions is considerably lower (0.328 R2 and 79.86% AUC), demonstrating that target overlap does not automatically translate into predictive performance.
Second, matched-task-pool ablations (G2) show that improvements persist even without expanding TVE's original supervision. Better target processing and decoder design increase mean R2 from 0.2803 to approximately 0.304 and AUC from 79.96% to approximately 80.9%. Expanding the task pool provides further gains to approximately 0.310 R2 and 81.2% AUC.
We acknowledge that QCRP's advantage over TVE+CTR narrows substantially after full fine-tuning. This is understandable because all methods fine-tune the same downstream architecture and objective, differing primarily in their pretrained initialization. End-to-end adaptation can reduce the influence of these initialization differences, leading to similar downstream performance. Our primary goal, however, is learning reusable representations that transfer effectively without updating the encoder.
W2 — Limited experimental scope and multi-root generalization.
We extended evaluation from two to four RelBench databases by adding Rel-Stack and Rel-RateBeer (G3). Rel-Stack uses 232 pretraining tasks across user and post roots; Rel-RateBeer uses 225 tasks across users, beers, and brewers. Across seven additional downstream tasks, both QCRP variants outperform random-initialized frozen encoders on every task and outperform fully supervised training on three tasks each. These are preliminary single-seed results.
These results support competitive multi-root consolidation rather than uniformly positive transfer. We observed root-dependent trade-offs as reported in Section 4.2: multi-root pretraining improves Amazon product on 37/38 objectives but worsens Amazon customer on all 40. Similarly, zero-shot transfer remains limited. We view improving cross-root optimization and zero-shot generalization as important directions for future work.
Q2 — Held-out task selection and candidate filtering.
The 24 held-out tasks use the same task-construction procedure described in Section 2.1, with their exact definitions excluded from pretraining (G4). They intentionally share query primitives with pretraining objectives: the intended test is generalization to unseen compositions within the forecasting language. This does not guarantee statistical independence from pretraining targets; G1 quantifies the resulting overlap through ground-truth and learned-prediction recoverability analyses.