acceptodds
Under review as a conference paper at ICLR 2027

Gaussian Relational Graph Transformer

Abstract

Relational graph learning enables predictive modeling over relational databases by directly capturing dependencies across interconnected tables. While relational graph Transformers extend the receptive field beyond local message passing, a larger receptive field does not necessarily provide more useful information: sampling must preserve relational structure without introducing excessive semantically irrelevant nodes, while attention must account for the temporal relevance of the retained information. We propose GelGT, a Gaussian relational graph transformer that explicitly addresses these challenges. GelGT introduces a structure-semantic collaborative sampling strategy to preserve structural connectivity while filtering irrelevant semantic information, and incorporates a Gaussian graph attention mechanism with a learnable Gaussian bias on the sampled subgraphs to dynamically encode temporal dependencies. We provide theoretical analysis characterizing the structural preservation, semantic refinement, and temporal discrimination of these mechanisms. Experiments on 7 real-world relational datasets covering 21 prediction tasks show that GelGT consistently outperforms existing relational graph learning methods, with improvements of up to 13.8%.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Relational Graph Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2QPROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper sees that current relational graph learning frameworks cannot perform message passing across distance multi-hop neighbors, fail to preserve the structural and semantic relevance when conducting graph sampling, and may impose an inaccurate temporal focus that mislead the attention toward irrelevant information. The Gaussian Relational Graph Transformer (GelGT) is a method proposed to solve all these challenges exhibit in the relational graph learning problem. GelGT proposes BFS-based structural integrity sampling and semantic refinement to identify and preserve valuable nodes for subgraph construction. To inject temporal information, a Gaussian temporal bias-based attention module is subsequently introduced. The GNN is coordinated with the temporal bias-based attention module for cell value prediction. Extensive experiments are conducted across 7 datasets and the baseline comparison indicates the consistent performance superiority of GelGT. Thorough ablation studies, hyperparameter analysis, overall framework efficiency analysis and other additional analyses have been provided.

Critical strengths
  1. Gaussian temporal bias-based attention is a simple yet effective trick in boosting the prediction accuracy.

  2. Extensive experiments conducted. The amount of work dedicated is truly appreciated.

  3. The framework overview provides a good visualization of the proposed components, thus helps readers to understand it.

Critical weaknesses
  1. Figure 4 has the x-axis denoted as “Sampling Size” but in Section 3, BsB_s and BtB_t are independently defined. This causes confusion about what sampling size means in this figure.

  2. Table 8c shows the performance comparison of different temporal bias terms. However, it is noticed that for each task, the performance difference between the three temporal bias terms is marginal. This then leads to question about what makes Gaussian temporal bias the preferable design (parameter counts, computational overhead, or inference latency?). Is there empirical evidence to support the claim?

  3. Despite concise implementation details provided in Appendix B, no code repo or pseudocode algorithm provided. It would be preferred to have a complete code workflow for reproducibility.

Questions for authors

Please address 1 and 2 in Weaknesses.

Reviewer_2XhqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a Gaussian relational graph transformer (GelGT). The key changes compared to the relational graph transformer include: (1) a structure-semantic collaborative sampling strategy, which aims to preserve connectivity while filtering out semantically irrelevant nodes, (2) a Gaussian temporal bias, which allows the model to explicitly learn where and how broadly to attend over time, and (3) a GNN branch, which complements graph attention. In the main experiments, GelGT is evaluated on 21 relational forecasting tasks from RelBench v1 and outperforms multiple representative methods on all tasks. Theoretical results and ablation studies additionally complement the main evaluation.

Critical strengths

The key GelGT components, namely, the sampling procedure and temporal bias, represent simple but meaningful ideas. If these components indeed improve downstream performance, that is a solid contribution.

Critical weaknesses
  1. The evaluation setup and results raise multiple concerns

First, Tables 1–2 (main evaluation results) say that "Results are reproduced based on available code", but Table 2 additionally says that "Results marked with † are reproduced using the official code". The difference between these two cases is not explained. I have checked the original RelGT paper, and it seems that the RelGT results in this paper not marked with † exactly match the original RelGT results, while four of the five results marked with † are worse than those in the RelGT paper. Moreover, the original RelGT results are better than the GelGT results.

Second, Tables 7, 8, and 13 (ablations and comparison with RT) contain three different task subsets. While conducting ablations on a subset of tasks is possible, the authors should clearly describe the selection criteria for the subsets, especially if these subsets vary across experiments.

Third, the size of the gains requires further consideration. Specifically, in Tables 1–2, GelGT achieves the best mean performance on all tasks, but standard deviations are not taken into consideration. I have briefly reviewed these tables, and it seems that the gains are often relatively small. Taking standard deviations into consideration when highlighting the best results may greatly help to form a complete picture.

  1. Theoretical results require clarification

Specifically, what are the assumptions for Theorem 1? I have briefly reviewed Appendix A, and the proof seems to assume that all 1-hop and 2-hop neighbors are sampled, since the authors write that the structural integrity sampling strategy preserves all walks of length k=1k=1 and k=2k=2. At the same time, the authors write about a fixed sampling budget BsB_s in lines 154–155. How are these points consistent?

Questions for authors
  1. What is the difference between "Results are reproduced based on available code" and "Results marked with † are reproduced using the official code" in Tables 1–2?
  2. What were the selection criteria for choosing between the two options in the previous question?
  3. Can you explain why your reproduced RelGT results are worse than those in the original paper?
  4. In Tables 1–2, can you also highlight in bold the values that are statistically indistinguishable from the best value (e.g., when the absolute difference between means is smaller than the sum of the corresponding standard deviations)?
  5. What were the task subset selection criteria for Tables 7, 8, and 13?
  6. Can you provide ablation results and a comparison with RT on all datasets?
  7. What are the assumptions for Theorem 1?
  8. Can you evaluate your model in the RelArena-α\alpha [1] setup and provide the source code for that evaluation? This is not required for a positive assessment, and it is completely fine not to answer this question, since it is likely to require substantial effort and computational resources. However, strong and reproducible GelGT results on RelArena-α\alpha would largely address my soundness concerns.

[1] Hayler et al., "Advancing Open and Reproducible Relational Learning: RelArena-α\alpha, TabPFN-Rel and RPI," 2026

Additional comments (no effect on score):

  1. In Figure 3, what is the difference between GelGT and "RelGT w/ all GelGT Components"?
  2. In the Gaussian temporal bias, have you tried making μ\mu and σ\sigma type-dependent? For example, if we predict something for a user, we might need to focus on recent transactions (so we need a small σ\sigma), but have no specific preference regarding an item's timestamp (so we need a large σ\sigma), assuming the item's timestamp corresponds to the moment when this item was added to a platform. In your formulation, both user-transaction and user-item attention will have the same μ\mu and σ\sigma, while it may be beneficial to use different μ\mu and σ\sigma values for them.
Reviewer_LySCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes GelGT, a Gaussian Relational Graph Transformer for learning from relational databases. The method combines structure-semantic collaborative sampling with Gaussian temporal attention. The sampling strategy uses BFS to preserve local graph structure and semantic similarity to filter less relevant nodes, while the attention mechanism introduces a learnable Gaussian bias to capture temporal dependencies. The paper also provides theoretical analyses of the proposed components.

The method is evaluated on 21 classification and regression tasks across seven RelBench datasets, with additional experiments on the SALT dataset. The evaluation includes comparisons with several existing methods, ablation studies, and experiments examining whether the proposed components can improve other relational graph learning models.

Critical strengths
  • The experimental evaluation is quite extensive in terms of both the number of datasets and the range of baseline methods compared. The paper also includes the challenging SALT dataset and several ablation studies, providing a broad evaluation of the proposed approach.
  • The paper proposes an interesting approach that combines structural and semantic sampling with Gaussian temporal attention. The ablation studies show that the different components contribute to the model’s performance, and the experiments with RelGNN suggest that the proposed sampling strategy can also benefit other architectures.
Critical weaknesses
  • Several improvements over the baselines are small relative to the reported standard deviations, and some results are inconsistent across tables. These issues raise concerns about the reliability of the reported improvements and the strength of the experimental conclusions. (See Q1 and Q2 for more details)
  • Some of the theoretical claims do not seem to be fully supported by the proofs. For example, Theorem 1 assumes that all one-hop and two-hop walks are preserved, which may not hold under a fixed sampling budget. Similarly, Theorem 3 derives its guarantee without considering the learnable linear transformation used in the proposed attention mechanism. (See Q3 and Q4 for more details)
Questions for authors
  • Tables 1 and 23 report different results for the proposed method on some tasks. For example, the AUC for rel-event-user-repeat is 0.8357 in Table 1 but 0.76 in Table 23, while rel-amazon-item-churn changes from 0.8297 to 0.70. Could the authors explain these discrepancies?
  • Could the authors provide statistical significance tests to determine whether the improvements over the strongest baselines are statistically significant, particularly for tasks where the improvements are small relative to the reported standard deviations?
  • The proof of Theorem 1 assumes that all one-hop and two-hop walks are preserved. However, this may not hold under a fixed sampling budget, especially for nodes with many neighbors. Could the authors clarify how this assumption is justified? Also, how does the global vector-norm bound establish the node-wise guarantee stated in Theorem 1?
  • Theorem 3 claims an improvement in the attention-weight ratio by a factor of e. However, the proof does not account for the learnable linear transformation applied to the Gaussian bias in Equation 8. Could the authors clarify whether this guarantee still holds for the proposed attention mechanism?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.