acceptodds
Under review as a conference paper at ICLR 2027

RBF-GNN: Rational Basis Functions for Pseudo-Coordinate based Graph Convolutions

Abstract

We propose RBF-GNN, a new pseudo-coordinate based graph neural network architecture that takes into account euclidean, spherical or angular coordinates and uses them to induce a powerful spatial inductive bias. Similar in architecture to SplineCNN, we improve upon the latter by replacing the less efficient sparse-activation based B-splines whose number grows exponentially with dimension by rational Padé basis functions. For effective training we propose a spline-subspace initialization and a variance-preserving weight rescaling. Experimentally, we evaluate on a number of popular neural network architectures that use SplineCNNs. We replace only the SplineCNNs with RBF-GNN. We achieve improved results, including on semantic keypoint matching, shape matching, event based camera computer vision tasks. We will make our implementation publicly available upon acceptance of the paper.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
graph neural networks, geometric deep learning, SplineCNN, rational functions, deep graph matching, keypoint matching, shape correspondence

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_WK63OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper replaces the fixed B-spline basis in SplineCNN with learnable rational basis functions. Safe denominators, Chebyshev representations, spline/PCA initialization, and variance calibration support training. The convolution retains a bank of feature-transform matrices while allowing a compact multivariate basis. Experiments across geometric matching, event recognition, superpixel classification, and point-cloud segmentation suggest parameter-efficiency benefits and some accuracy improvements.

Critical strengths
  1. Practical parameter efficiency. The method addresses the growth of the matrix bank with the size of a tensor-product spline grid. In higher-dimensional settings, compact bases obtain performance close to much larger spline models. This is a useful practical contribution even where accuracy gains are small.
  2. Breadth of validation and implementation. Replacing convolution layers in several tasks and host architectures helps test whether the approach generalizes beyond a single pipeline. The structured initialization, numerical representation, and supplied configurations and logs strengthen the implementation contribution.
Critical weaknesses
  1. Selection using test performance. Table 1 and the supplementary documentation indicate that PascalVOC/DGMC and the NMT columns report the best test epoch, whereas SPair-71k/DGMC uses the best validation epoch. Taking the maximum over test scores makes the test set part of result selection and introduces optimistic bias. The same procedure for baselines does not remove this concern. The issue is the epoch-selection rule, not the absence of patience-based early stopping. All affected comparisons should use validation-selected checkpoints or a final epoch fixed independently of test outcomes. The supplement retains a final-epoch advantage in PascalVOC/DGMC and reports small best-test versus final-epoch differences for NMT, but a consistent corrected evaluation is still needed before the headline gains can support acceptance.
  2. The rational form is not sufficiently isolated. The input domain remains [0,1]D[0,1]^D. In g(u)=∑pBp(u)Θpg(u)=\sum_pB_p(u)\Theta_p, reducing KK reduces the KCinCoutKC_{in}C_{out} matrix parameters; replacing splines at the same KK does not do so and adds function coefficients. For smaller KK, the initialization first compresses the spline space and then fits rational functions to the retained components. The contribution of compression, initialization, adaptation, and rational parameterization therefore needs separation. MLP and frozen-rational controls are relevant, but do not replace a direct PCA-compressed spline baseline with matched total capacity and comparable initialization. This distinction determines whether rational functions themselves justify the main novelty claim.
Questions for authors
  1. Clarification and decisive evidence: Which conclusions remain when every affected method uses a test-independent epoch-selection rule?
  2. Decisive comparison: How does the method compare with directly retaining the same KK PCA spline components, and with learned bases matched in total capacity and initialization quality?
  3. Clarification: How should the general inference-speed claim be reconciled with the K=8K=8, 32-to-32-channel microbenchmark, where the rational forward pass is slower than PyG SplineConv? Please also account for initialization cost.
3 comments
AuthorsOpenReview ↗

We thank the reviewer for the constructive review. We address reviewer's concerns below:

W1. Selection using test performance.

We agree that the selection should have been done on validation data. We have therefore retrained all affected results, baselines included, and now report the test accuracy at the epoch with the best validation accuracy:

  • PascalVOC has no validation split. We hold out 10% of the training images, selected by a hash of the image name and identical for all methods and for DGMC and NMT, and train on the rest.
  • SPair-71k, N-Caltech101 and N-Cars use their official validation splits.
  • FAUST, which reported the final epoch, holds out subject 7, so that validation, like the test set, measures unseen subjects.

MNIST, CIFAR-10 and ShapeNet-Part report the final epoch of a fixed schedule and are unchanged. Under DGMC, every method family also gets the same learning-rate grid with five seeds per rate, selected on mean validation accuracy (general response, point 2).

Overall the selection bias was small. Crucially, the relative ordering remains the same.

Q1. Which conclusions remain?

We report the best variant of each method, as submitted and after retraining. The revised variants are selected on validation but for DGMC also come after a more exhaustive learning rate grid search, which resulted in higher DGMC scores overall. We give the difference at the best-validation epoch and, as the test-independent alternative the reviewer names, at the final epoch (Welch's t-test). Overall, the absolute numbers have changed slightly but relative ordering stayed mostly the same.

SettingSubmitted: SplineCNN / RBF-GNN (Δ)Revised: SplineCNN / RBF-GNNΔ, best-validation epoch (p)Δ, final epoch (p)
PascalVOC, DGMC73.00 / 74.12 (+1.12)73.77 / 74.70+0.93 (0.009)+1.25 (0.001)
SPair-71k, DGMC76.18 / 77.80 (+1.62)77.14 / 78.80+1.66 (<0.001)+1.95 (<0.001)
PascalVOC, NMT83.10 / 83.78 (+0.68)82.96 / 82.87−0.09 (0.78)−0.09 (0.77)
SPair-71k, NMT82.10 / 82.11 (+0.01)82.08 / 82.03−0.05 (0.78)−0.10 (0.64)
FAUST98.71 / 99.41 (+0.70)98.44 / 99.27+0.83 (0.002)+0.82 (0.001)
N-Caltech10156.00 / 57.55 (+1.55)56.25 / 58.39+2.14 (0.001)+2.17 (0.034)
N-Cars89.81 / 91.00 (+1.19)89.84 / 90.97+1.13 (0.054)+1.09 (0.043)

Note that using the new selection rule changed also the training set, since for PascalVOC we took out 10% for validation and for FAUST we removed subject 7. RBF-GNN uses degrees (10,8).

W2 / Q2. The rational form is not sufficiently isolated.

To disentangle compression, initialization, adaptation, and rational parameterization we have conducted experiments in Table 7 (Appendix B).

Every row has the same KK weight matrices, and total parameters differ by at most 0.11% (at K=9K=9 under DGMC: 3.741M rational, 3.738M PCA-spline, 3.740M MLP, 3.737M Gaussian). All rows use the published learning rate:

Step isolatedBasis, same KKVOC K=4K=4SPair K=4K=4VOC K=9K=9SPair K=9K=9FAUST K=8K=8
noneSplineCNN, coarser grid71.0272.4472.3975.870.32*
compressionRBF-GNN, PCA-compressed splines, fixed72.5176.4572.9177.2395.65
+ rational approximationRBF-GNN, rational, fitted, frozen72.7776.2572.5276.9496.37
+ adaptation, rational formRBF-GNN, rational, fitted, trained (ours)73.3977.7373.5577.7499.27
no initializationRBF-GNN, Rational, random init, trained72.3975.6372.5776.6599.26
other forms, same initMLP, fitted, same gain73.0876.6173.1277.5393.30
other forms, same initGaussians (MoNet), fitted, same gain71.4775.8070.4875.1296.48

*: On FAUST, SplineCNN with small KK degrades heavily but works well with higher KK, see Table 2.

Conclusion. The rational form is the best form at equal initialization. Against the exact PCA-compressed splines, however, the margin is small, yet significant in three of seven settings. Against default-initialized learned bases, a significant part of the gain comes from the initialization, with the rational form adding some more performance on average.

Q3. Speed claim and initialization cost.

The reviewer is right that the claim was too general. We reran the benchmark (method section, Appendix D). The slower forward pass comes from a variant of our Triton kernel that uses a memory-saving aggregation order. Our RBF-GNN triton kernel can evaluate a layer in two orders with identical outputs, which we have not mentioned in the initial submission:

  • Weight-first forms each message ∑pBp(uij) Wp⊤xj\sum_p B_p(\mathbf u_{ij})\,\mathbf W_p^\top \mathbf x_j on its edge. It is fast but stores EKCoutE K C_\text{out} values, since all KK rational functions are non-zero on every edge, whereas PyG evaluates only the 2D2^D non-zero B-splines.
  • Basis-first first sums Bp(uij) xjB_p(\mathbf u_{ij})\,\mathbf x_j over the neighbours and then applies the Wp\mathbf W_p. It needs about PyG's memory, but it passes over the edges KK times, so its forward pass is slower.

We give below a micro benchmark with both orders. We use one layer on a synthetic graph with 50,000 nodes, 10610^6 edges and 32 → 32 channels (RTX 5090, degrees (10,8), Triton basis kernel), on the pseudo-coordinate grids of our three benchmarks. Each cell: forward (ms) / forward + backward (ms) / peak memory (GB). SplineCNN has K=kDK = k^D = 25, 125 and 8.

LayerKK2-D, k=5k=5 (keypoints)3-D, k=5k=5 (FAUST)3-D, k=2k=2 (event cameras)
SplineCNN (PyG, fused)kDk^D2.1 / 14.7 / 0.404.7 / 24.5 / 0.443.5 / 32.4 / 0.44
RBF-GNN, weight-first41.9 / 4.7 / 1.102.2 / 5.3 / 1.102.2 / 5.3 / 1.10
RBF-GNN, weight-first82.3 / 6.6 / 2.082.7 / 7.8 / 2.082.7 / 7.9 / 2.08
RBF-GNN, weight-first164.2 / 12.6 / 4.055.2 / 15.1 / 4.055.2 / 15.5 / 4.05
RBF-GNN, weight-first328.3 / 25.0 / 7.9810.1 / 30.1 / 7.9810.3 / 31.1 / 7.98
RBF-GNN, basis-first43.6 / 9.2 / 0.423.8 / 9.8 / 0.423.8 / 9.8 / 0.42
RBF-GNN, basis-first87.1 / 18.1 / 0.477.6 / 19.3 / 0.477.6 / 19.3 / 0.47
RBF-GNN, basis-first1614.3 / 36.0 / 0.5815.3 / 38.5 / 0.5815.3 / 38.5 / 0.58
RBF-GNN, basis-first3228.7 / 74.0 / 0.7930.5 / 79.2 / 0.7930.5 / 79.2 / 0.79
  • The speed advantage depends on the aggregation order, basis size and channel width. Weight-first can outperform PyG at higher memory cost.
  • Initialization cost: We only pay a one-time CPU fit that does not depend on the data and is cached and shared by all layers with the same configuration. It takes 2 to 3 s in 2-D and 36 to 41 s in 3-D, but about 31 minutes on the 5-D grid of our CIFAR-10 experiment. Usually, initialization is a negligible share of training: under 1% in 2-D and about 0.2–0.4% on the event cameras. On FAUST it is about a quarter of a run, because those runs take only about 3 minutes.

Thanks for the clarification. I have raised my score.

Reviewer_4Z2NOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces RBF-GNN, a graph convolution operator based on pseudo-coordinates. It builds on SplineCNN by replacing the tensor-product B-spline basis with a set of learnable rational functions. This avoids the exponential growth in the number of basis functions with the pseudo-coordinate dimension and allows the kernel complexity to be controlled independently. The authors evaluate the proposed operator by replacing SplineCNN layers in several existing architectures including keypoint matching, shape correspondence, event-camera recognition, superpixel classification, and point-cloud segmentation. Across these tasks, RBF-GNN generally achieves similar or better accuracy while using fewer parameters.

Critical strengths

The paper addresses two limitations of SplineCNN: the exponential growth of the number of basis functions with the pseudo-coordinate dimension and the relatively sparse gradient updates associated with individual weight matrices.

The proposed modification is simple and can be integrated into existing architectures without further changes, making it easier to assess the contribution of the new basis functions.

The experimental evaluation covers a good range of tasks, with pseudo-coordinate dimensions from 2 to 6. An interesting result is that a small number of rational basis functions (K=4–9) can achieve comparable or better accuracy than substantially larger spline bases, suggesting a clear benefit in terms of parameter efficiency.

Critical weaknesses

All results reported in the tables rely on spline-based initialization. The authors mention that random initialization performs worse, but do not include these results in the tables. The spatial inductive bias is therefore largely inherited from SplineCNN rather than learned, which weakens the “hand-designed vs. learnable” framing. Furthermore, the initialization also still requires evaluating all k^D splines.

The experimental comparisons lack some important details. The initialization of the MLP and PointNet baselines is unclear, as is the initialization used for several RBF-GNN results. This raises the question about whether the observed gains come from the rational basis or the initialization.

The novelty is somewhat limited. The operator builds on SplineCNN, the rational formulation comes from PAU, and KAT already explores related basis functions. Other approaches, including MoNet, PAConv, and MLP kernels, also avoid the exponential growth in the number of bases. A comparison with MoNet is notably missing.

The claim of consistent improvements is not fully supported. Results are selected at the best test epoch, with more configurations explored for RBF-GNN than for SplineCNN. Several gains are within the reported standard deviations, and SplineCNN remains best on ShapeNet-Part.

Questions for authors

How are the MLP-basis controls initialized? Can you report an MLP basis and a MoNet-style Gaussian basis fitted to the same spline or PCA targets, using the same variance correction? Please include the random-initialization ablation in a table. Please also report a “rational fit, frozen basis” variant to distinguish the effect of approximating the splines from that of learning the basis. Can you provide wall-clock time and memory usage per layer compared with SplineConv (PyG), across D and K? How sensitive is the method to the polynomial degrees (m, n)? Section 4 indicates that the K=9 multivariate model uses higher degrees (8, 6) than the degree control (5, 4).

4 comments
AuthorsOpenReview ↗

We thank the reviewer for the detailed and constructive review. We address the concerns below.

W1. All results rely on the spline-based initialization, and the initialization evaluates all kDk^D splines.

The random-initialization ablation is now in Table 7 (Appendix B), together with a frozen rational basis but learnable weight matrices, repeated here:

Same KKVOC K=4K=4SPair K=4K=4VOC K=9K=9SPair K=9K=9FAUST K=8K=8
RBF-GNN, fitted, trained (ours)73.3977.7373.5577.7499.27
RBF-GNN, fitted, frozen72.7776.2572.5276.9496.37
RBF-GNN, random init, trained72.3975.6372.5776.6599.26
SplineCNN, same KK71.0272.4472.3975.870.32*

*: For SplineCNN on FAUST we need higher KK for it to work well, see Table 2.

  • Spline initialization is significant. Random coefficients cost 1.0 to 2.1 points on keypoint matching (p ≤ 0.007), though nothing on FAUST. We weakened the "hand-designed vs. learnable" framing, and the paper now states that the initialization transfers the spline prior.
  • Our architecture gives additional performance. Without any spline information, the randomly initialized rational basis is still more accurate than SplineCNN with the same KK in four of the five settings (by 1.37, 3.19 and 0.78 points on keypoint matching, p ≤ 0.043, and on FAUST), and level with it on PascalVOC with K=9K=9 (+0.18, p=0.39). Training also adds to the transferred prior. Freezing the fitted basis costs 0.6 to 1.5 points on keypoint matching and 2.9 on FAUST (p=0.001).
  • Cost of initialization. We agree that the initialization still is exponential in the dimension. However, it evaluates all kDk^D B-splines only once. It does not depend on the data, is cached, and is shared by all layers with the same configuration. It takes 2 to 3 s in 2-D and 36 to 41 s in 3-D, but about 31 minutes on the 5-D grid of our CIFAR-10 experiment (Appendix D). The layers themselves never evaluate the B-splines, and its cost does not depend on kDk^D.

W2. The experimental comparisons lack some important details. Initialization of the baselines and of RBF-GNN.

Appendix A now has a paragraph on the initialization of every method:

MethodInitialization
Kernel weights Θp\Theta_p of every basisuniform with bound 1/KCin1/\sqrt{K C_\text{in}}, as in SplineConv
SplineCNNfixed B-splines
MoNetas PyG's GMMConv: Glorot-initialized means and standard deviations
MLP basisPyTorch default, then the variance-preserving gain
PointNet, GATv2, GINEPyTorch and PyG defaults
RBF-GNN, product basisfitted to the B-spline hats
RBF-GNN, multivariate basisfitted to the KK leading principal components of the B-splines, with the variance-preserving gain (PCA+vp)

To separate the rational form from the initialization, Table 7 gives every form the same initialization (published learning rate):

Basis, same KKVOC K=4K=4SPair K=4K=4VOC K=9K=9SPair K=9K=9FAUST K=8K=8
Rational (RBF-GNN), PCA+vp73.3977.7373.5577.7499.27
PCA-compressed splines, learned mixing72.9276.2973.0277.5098.95
PCA-compressed splines, fixed72.5176.4572.9177.2395.65
MLP, fitted, same gain73.0876.6173.1277.5393.30
Gaussians (MoNet), fitted, same gain71.4775.8070.4875.1296.48
MLP, default init72.5773.9571.6874.4796.27
Gaussians (MoNet), default init70.6972.2571.3674.2097.21
  • Both contribute. Fitting improves the MLP basis by 0.5 to 3.1 points on keypoint matching, so part of the advantage over default-initialized learned bases comes from the initialization, and the paper now says so. At equal initialization, the rational form is the most accurate in all five columns.

W3. Novelty and missing comparison with MoNet.

The revised related work ("Relation to the closest designs", Section 2) positions RBF-GNN against each of the named approaches.

  • PAU and KAT apply univariate safe-Padé functions elementwise, as activations. In RBF-GNN the rational functions are multivariate functions of the pseudo-coordinates and form the basis of a matrix-valued kernel, so a message stays linear in the neighbour features. We take the safe denominator from PAU and adapt KAT's variance-preserving initialization, with the replaced spline layer as the variance reference.
  • MoNet uses the same kernel with Gaussian basis functions. These are positive, unimodal and axis-aligned, so they cannot represent the signed, non-separable principal components that our initialization transfers (Table 8). MoNet is now a baseline in every retrained table, with the same learning-rate grid as all other methods. Best basis size of each method:
Task (host network)MoNetRBF-GNNΔp
PascalVOC (DGMC)72.30 (K=9K=9)74.70 (K=6K=6)+2.4<0.001
SPair-71k (DGMC)75.06 (K=9K=9)78.80 (K=9K=9)+3.7<0.001
PascalVOC (NMT)81.98 (K=25K=25)82.87 (K=9K=9)+0.90.010
SPair-71k (NMT)80.75 (K=25K=25)82.03 (K=6K=6)+1.30.001
FAUST97.21 (K=8K=8)99.27 (K=8K=8)+2.10.047
N-Caltech101 (AEGNN)55.91 (K=8K=8)58.39 (K=8K=8)+2.50.019
N-Cars (AEGNN)88.33 (K=8K=8)90.97 (K=8K=8)+2.60.010
  • PAConv and MLP kernels learn the coefficient functions with an MLP. Our MLP-basis control represents this family, with and without our initialization, now also with K=25K=25. RBF-GNN is more accurate than the best MLP basis in every column of Table 1.
  • Our contribution lies also in the combination of all components through which we obtain a drop-in replacement for the B-spline basis whose number of weight matrices does not grow as kDk^D, an initialization that transfers the spline subspace and the output variance, and experiments showing that among the tested forms the rational one is the most accurate at equal initialization.

W4. Claims of consistent improvement. Results are selected at best test epoch.

  • Best-test-epoch selection: We agree that selection on the test set was an oversight. Test-epoch selection affected only PascalVOC under DGMC and both NMT columns. SPair-71k under DGMC and both event-camera benchmarks already selected on validation, and FAUST reported the final epoch.
  • Retraining changes. PascalVOC now trains on 90% of its training images, DGMC uses a learning-rate grid for every method, RBF-GNN uses degrees (10,8), and FAUST holds out one training subject. These additional changes increased overall performance on DGMC, but the overall ranking stays the same. We report the best variant of each method, submitted → revised:
SettingSplineCNNBest RBF-GNNRBF-GNN − SplineCNN
PascalVOC, DGMC73.00 → 73.7774.12 → 74.70+1.12 → +0.93
SPair-71k, DGMC76.18 → 77.1477.80 → 78.80+1.62 → +1.66
PascalVOC, NMT83.10 → 82.9683.78 → 82.87+0.68 → −0.09
SPair-71k, NMT82.10 → 82.0882.11 → 82.03+0.01 → −0.05
FAUST98.71 → 98.4499.41 → 99.27+0.70 → +0.83
N-Caltech10156.00 → 56.2557.55 → 58.39+1.55 → +2.14
N-Cars89.81 → 89.8491.00 → 90.97+1.19 → +1.13
  • More configurations explored for RBF-GNN. Under DGMC every method family gets the same learning-rate grid, with five seeds per rate, and the rate is selected on validation. RBF-GNN has more rows in Table 1 than SplineCNN, but each row is a fixed configuration and each basis size is shown. The only choice specific to RBF-GNN, the degrees, was made once from four candidates on PascalVOC validation and then frozen. Under NMT every method uses the published recipe.

Q1. Initialization of the MLP-basis controls:

PyTorch's default with the variance-preserving gain (W2).

Q2. Can you report an MLP basis and a MoNet-style Gaussian basis fitted to the same spline or PCA targets, using the same variance correction?:

See second table of W2.

Q3. Please include the random-initialization ablation in a table

See table of W1.

Q4. Please also report a “rational fit, frozen basis” variant to distinguish the effect of approximating the splines from that of learning the basis.

See table in the answer to W1. The frozen rational basis is within 0.4 points of the fixed PCA-compressed splines it approximates on keypoint matching, so the gain of the trained basis comes from adapting it, not from approximating the splines.

Q5. Can you provide wall-clock time and memory usage per layer compared with SplineConv (PyG), across D and K?

(Appendix D, Tables 10 and 11, RTX 5090, our Triton kernel for the basis). RBF-GNN can evaluate a layer in two orders, with identical outputs for sum and mean aggregation. The standard order forms ∑pBp(uij) Wp⊤xj\sum_p B_p(\mathbf u_{ij})\,\mathbf W_p^\top \mathbf x_j on each edge, which is fast but stores EKCoutE K C_\text{out} values, since all KK rational functions are non-zero on every edge. The memory-saving order first sums Bp(uij) xjB_p(\mathbf u_{ij})\,\mathbf x_j over the neighbours of each node and then applies the Wp\mathbf W_p, storing only the basis values but passing over the edges KK times. One layer, forward plus backward in ms, 10610^6 edges, 32 → 32 channels:

LayerKK2-D, k=5k=53-D, k=5k=53-D, k=2k=2Memory (GB)
SplineCNN (PyG, fused)kDk^D = 25 / 125 / 814.724.532.40.40–0.44
RBF-GNN, standard44.75.35.31.10
RBF-GNN, standard86.67.87.92.08
RBF-GNN, standard1612.615.115.54.05
RBF-GNN, standard3225.030.131.17.98
RBF-GNN, memory-saving818.119.319.30.47

Time and memory grow linearly in KK and do not depend on kDk^D. The standard order trains faster than PyG up to K=16K=16, at 2.5 to 10× its memory; the memory-saving order matches PyG's memory. At the scale of our event-camera models (12.4M edges, D=3D=3, K=8K=8), the standard order is 1.2× faster than PyG in the forward pass and 3.5× faster in training at 4.7× its memory, and the memory-saving order needs about PyG's memory but its forward pass is 2.1× slower (Table 11).

Q6. How sensitive is the method to the polynomial degrees (m, n)?

See also Appendix C, Table 9, PascalVOC under DGMC, published learning rate, multivariate basis with PCA+vp.

(m,n)(m,n)Val, K=4K=4Test, K=4K=4Val, K=9K=9Test, K=9K=9
(3,2)75.8172.7275.7973.06
(5,4)75.7972.9875.9273.08
(8,6)76.2673.2276.1673.18
(10,8)76.6473.3976.4873.55
SplineCNN, same KK73.9671.0274.6372.39

Accuracy rises slowly with the degrees, and the whole range spans less than one point; even (3,2)(3,2) is more accurate than SplineCNN with the same KK. We chose (10,8)(10,8) once on validation, before the main runs, and all retrained experiments use it, which removes the inconsistency between the K=9K=9 model and the degree control that the reviewer noted. Raising the degrees from (5,4)(5,4) to (10,8)(10,8) adds 7 to 12% to the training time at the scale of our event-camera models.

I appreciate the substantial efforts made by the authors to address the concerns raised. I found the additional experiments and clarifications convincing, particularly the comparisons with MoNet and the new ablations that help distinguish the effects of initialization and the learned rational basis. I also appreciate that the authors acknowledge that the improvements over exact PCA-compressed splines are statistically significant only in some configurations. While the methodological novelty remains limited, I believe the paper provides a valuable empirical study of alternative basis parameterizations for graph convolutions, with careful experiments that help clarify the respective roles of initialization, basis compression, and learnability. I have raised my score.

Reviewer_bFigOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces RBF-GNN, a modification of SplineCNN, where the B-spline basis functions are replaced with learnable rational basis functions to improve the continues convolutional kernels of the model.

The paper shows how to design / incorporate these rational basis functions, making sure to prevent undefined divisions by zero (Equation 5), enabling stabler gradients using Chebyshev polynomials, and providing a better initial spatial initialization by fitting to B-splines. The work also tests two variants of accumulating the rational basis across coordinate dimensions (product rational- and multivariate rational basis).

The paper is well written and good to follow with a clear method section and good experimental setup. Main concerns are regarding the impact of the initialization vs. the learnable rational basis functions and comparison with only SplineCNN-type models.

Critical strengths
  1. Strong Evaluation Setup: Evaluation is performed across 8 very diverse benchmarks across different disciplines (keypoint matching, segmentation, classification, etc.). The evaluation also only replaces the SplineCNN with RBF-GNN, not changing other training parameters, keeping comparison fair and demonstrating well the actual relative improvement. Results are also reported over multiple seeds.
  2. Drop-in replacement: The main advantage of the proposed RBF-GNN is that the rational basis functions can simply be used as a replacement of the B-splines of SplineCNN with no additional changes needed.
Critical weaknesses
  1. Comparison with two-layer MLP: To my understanding, a large portion of the gain of RBF-GNN comes from the SplineCNN like initialization (line 253), which is not exclusively applicable to RBF-GNN. I therefore wonder if this initialization would also improve the MLP variants as well. This would be important to know, as it would show that using Pade approximations are an actual superior parametrization for the basis functions compared to the very general MLP, and that it is not just originating from the initialization scheme. The in lines 260 - 263 introduced 3 step initialization will ofc. not work directly on these MLPs, but using Adam for a few iterations on the same targets should work as well.
  2. Comparison limited to SplineCNN type models: The paper's focus is on drop-in improvement of SplineCNN models, and mainly compares with SplineCNN variants (lines 358-361). To position the resulting improvement better, it should also include SOTA non-SplineCNN based methods in their comparison.
Questions for authors
  1. Lines 245 - 250 argue that in SplineCNN, sparsity wastes dense evaluation on zeros. The argument for the "spline fit" initialization (lines 252 - 267), describes correlated gradients as a negative, aiming to mimic the sparse B-spline setup as initialization. My question would therefore be how sparse these initialized functions stay or actually deviate from this initialization after training?
  2. lines 344 - 350 describe that the custom kernel RBF-GNN implementation runs 15x faster than a naive PyTorch implementation, being slightly faster than the PyG-implementation of SplineCNN. I was wondering what the actual runtime numbers are here, as they are not mentioned, as well as how they scale with KK and especially the polynomial degrees mm and nn.

Minor Notes:

  • Figure 3, 4: the caption should not be next but underneath the figure based on ICRL style rules.
2 comments
AuthorsOpenReview ↗

We thank the reviewer for their careful and constructive review. We address reviewer's concerns below:

W1. Comparison with two-layer MLP. Would the spline-based initialization also improve the MLP basis?

Mostly yes. We ran the suggested control: the MLP basis is fitted to the same targets as RBF-GNN, the KK leading principal components of the B-splines, with Adam (2,000 steps, followed by an L-BFGS polish), and it receives the same variance-preserving gain. We give this experiment in Appendix B, Table 7 (5 seeds for keypoint matching with DGMC, 3 for FAUST). Overall, the improved initialization improves MLP, but RBF-GNN gives additional improvements.

SettingRBF-GNN, fittedMLP, fittedMLP, default initRBF-GNN − fitted MLPp (Welch)
PascalVOC, K=4K=473.3973.0872.57+0.310.29
SPair-71k, K=4K=477.7376.6173.95+1.120.003
PascalVOC, K=9K=973.5573.1271.68+0.430.09
SPair-71k, K=9K=977.7477.5374.47+0.210.42
PascalVOC, K=25K=2572.8772.1070.86+0.770.005
SPair-71k, K=25K=2576.8176.5074.11+0.310.25
FAUST, K=8K=899.2793.3096.27+5.97<0.001

W2. Comparison limited to SplineCNN-type models.

For different GNN architectures we have chosen GATv2 and GINE and used them on keypoint matching tasks. They are significantly worse, which is the reason why most published keypoint matching works use SplineCNN feature refinement.

Layer in DGMCParamsPascalVOCSPair-71k
GATv20.86M36.5247.55
GINE0.66M60.1167.45
SplineCNN (K=25K=25)9.50M73.7777.14
RBF-GNN (K=9K=9)3.74M74.5278.80

Q1. How sparse do the initialized basis functions stay after training?

They become less sparse. Appendix E (Table 12, Figures 6 and 7) measures this on DGMC/PascalVOC.

  • Functions fitted to the B-spline hats stay centred on their hats but widen. Their effective support (where ∣Bp∣|B_p| exceeds 10% of its maximum) grows from 30% to 48% of the domain at K=9K=9 and from 11% to 21% at K=25K=25, and 11 to 15% of their energy moves outside the support of the hat.
  • The PCA-initialized functions are global from the start. They change by 0.69 to 0.77 of their norm. Still, only 4 to 10% of the energy of the trained basis lies outside the span of the initial functions.

Regarding sparsity more generally: the initialization is meant to make the basis functions distinct, not sparse. A random initialization gives nearly flat, nearly identical functions, so all Θp\Theta_p receive almost the same gradient. The principal components are orthogonal (they come from an SVD) but not sparse, so they still separate the gradients and every Θp\Theta_p receives a gradient from every edge.

Q2. Actual runtime numbers, and scaling with KK, DD and the degrees (m,n)(m,n).

Appendix D now reports time and peak memory per layer against PyG's fused SplineConv on an RTX 5090 (Tables 10 and 11). All RBF-GNN numbers use our Triton kernel for the basis.

RBF-GNN evaluates a layer in one of two orders, with identical outputs for sum and mean aggregation, which we have not made clear in the initial submission:

  • Standard: each edge forms ∑pBp(uij) Wp⊤xj\sum_p B_p(\mathbf u_{ij})\,\mathbf W_p^\top \mathbf x_j. This is fast, but it stores EKCoutE K C_\text{out} values, since all KK rational functions are non-zero on every edge (only 2D2^D B-splines are).
  • Memory-saving: each node first sums Bp(uij) xjB_p(\mathbf u_{ij})\,\mathbf x_j over its neighbours and then applies the Wp\mathbf W_p. Only the basis values are stored, but it takes KK passes over the edges.

Scaling with KK and DD (Table 10): We report time per layer (ms) for the forward pass (Fwd) and for forward plus backward (F+B), and peak memory (GB) on a synthetic graph with 50,000 nodes, 10610^6 edges and 32 → 32 channels.

LayerKK2-D Fwd2-D F+B2-D GBFAUST FwdFAUST F+BFAUST GBAEGNN FwdAEGNN F+BAEGNN GB
SplineCNN (PyG, fused)kDk^D2.114.70.404.724.50.443.532.40.44
RBF-GNN (10,8)(10,8), weight-first41.94.71.102.25.31.102.25.31.10
RBF-GNN (10,8)(10,8), weight-first82.36.62.082.77.82.082.77.92.08
RBF-GNN (10,8)(10,8), weight-first164.212.64.055.215.14.055.215.54.05
RBF-GNN (10,8)(10,8), weight-first328.325.07.9810.130.17.9810.331.17.98
RBF-GNN (10,8)(10,8), basis-first43.69.20.423.89.80.423.89.80.42
RBF-GNN (10,8)(10,8), basis-first87.118.10.477.619.30.477.619.30.47
RBF-GNN (10,8)(10,8), basis-first1614.336.00.5815.338.50.5815.338.50.58
RBF-GNN (10,8)(10,8), basis-first3228.774.00.7930.579.20.7930.579.20.79

For SplineCNN the forward pass only depends on the dimension, not on kk. Time and memory grow linearly in KK for RBF-GNN. The memory-saving order matches PyG's memory at two to three times the time of the standard order.

  • Initialization: a one-time CPU fit, cached across layers, takes 2 to 3 s in 2-D and 36 to 41 s in 3-D.

I thank the authors very much for the detailed response and additional experiments.

My questions and weaknesses were all addressed. Especially running the MLP initialization comparison with multiple seeds and providing p-values is really helpful. I think the model as a whole is a solid contribution, through the improvement over MLP with the same initialization scheme is marginal (W1). I will keep my score of Weak Accept.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.