acceptodds
Under review as a conference paper at ICLR 2027

Learning fMRI dictionaries across individual geometries via optimal transport

Abstract

Dictionary learning is a powerful tool for creating interpretable representations from complex data. When applied to functional magnetic resonance imaging (fMRI) data, the resulting patterns of brain activity can be used for various downstream tasks, such as brain activity decoding or population-level analysis. However, the variability in brain geometry across individuals poses a fundamental challenge. This is usually addressed by projecting each individual brain geometry onto a common template, which removes subject-specific information. In this work, we introduce a novel approach to dictionary learning on fMRI data that explicitly accounts for this variability. We use the optimal transport-based Fused Gromov-Wasserstein (FGW) distance to compare graphs with different geometries and features. To address the challenge of computing multiple FGW distances for large graphs such as those derived from fMRI data, we use amortized optimization to train a neural network that approximates the optimal transport plans. We also learn graph dictionary atoms that depend on the FGW trade-off parameter, which controls the balance between feature alignment and structural consistency. Numerical experiments on the HCP dataset demonstrate that the proposed approach captures meaningful information about both contrasts and subjects. It outperforms standard fMRI dictionary learning and is robust to transfer across graph scales.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
dictionary learning, graphs, optimal transport, neuroscience, fMRI data

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_nStdOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AGDL for learning fMRI dictionaries across individual brain geometries. It combines FGW graph dictionary learning with a pretrained ULOT solver and introduces α-conditioned dictionary atoms. Experiments on HCP evaluate contrast classification, subject identification, computational efficiency, and transfer across graph resolutions.

Critical strengths

Learning directly on individual geometries addresses a meaningful problem in fMRI analysis. The framework can preserve anatomical information and handle graphs with different numbers of nodes. The reported speedup makes graph dictionary learning more practical. Evaluation on held-out subjects and across graph resolutions provides useful evidence of generalization.

Critical weaknesses

The added contribution over GDL and ULOT is not sufficiently isolated. Both components are existing methods, and ULOT already supports differentiable transport prediction on fMRI graphs. The α-conditioned dictionary is a useful extension, but there is no comparison with separately trained fixed-α dictionaries using the same ULOT solver. Its additional benefit therefore remains unclear. The classification results do not clearly establish the benefit of joint modeling. Subject identification may largely rely on fixed anatomical differences, while Multi-AGDL increases embedding dimensionality and Multi-AGDL+ also includes baseline features. Although the paper includes the α=1 setting, comparisons with a simple anatomical representation and dimension-matched baselines are still needed to explain the gains.

Questions for authors
  1. How does the α-conditioned dictionary compare with separately trained fixed-α dictionaries using the same pretrained ULOT, in terms of accuracy and total training cost?

  2. Do the improvements remain with dimension-matched baselines? For subject identification, how does AGDL compare with a simple representation based only on anatomy?

3 comments

We thank you for your insightful comments. We try our best to answer your concerns below. We applied all the reviever's suggestions in red in the updated version of the paper.

Q1. Comparison with separately trained fixed-α\alpha dictionaries. To isolate the benefit of α\alpha-conditioning, we trained separate fixed-α\alpha dictionaries for α∈{0,0.25,0.5,0.75,1}\alpha \in \{0,0.25,0.5,0.75,1\}, using the same pretrained ULOT model as AGDL. We directly optimize the atoms for each value of α\alpha. We compared each dictionary with the single α\alpha-conditioned AGDL model evaluated at the corresponding α\alpha.

α\alphaContrast: fixed-α\alpha dictionaryContrast: AGDLSubject: fixed-α\alpha dictionarySubject: AGDL
00.77±0.0220.77 \pm 0.0220.85±0.01030.85 \pm 0.01030.0061±0.00380.0061 \pm 0.00380.0202±0.00850.0202 \pm 0.0085
0.250.77±0.0310.77 \pm 0.0310.85±0.0170.85 \pm 0.0170.0051±0.00320.0051 \pm 0.00320.021±0.00870.021 \pm 0.0087
0.50.75±0.0340.75 \pm 0.0340.87±0.0150.87 \pm 0.0150.0051±0.00320.0051 \pm 0.00320.024±0.0150.024 \pm 0.015
0.750.74±0.0300.74 \pm 0.0300.82±0.0260.82 \pm 0.0260.0061±0.00380.0061 \pm 0.00380.018±0.0180.018 \pm 0.018
10.042±0.0170.042 \pm 0.0170.0131±0.00250.0131 \pm 0.00250.99±0.00200.99 \pm 0.00201.00±0.001.00 \pm 0.00

Accuracy. AGDL achieves higher mean contrast-classification accuracy for all tested α<1\alpha<1, with improvements of 8–12 percentage points. AGDL also yields higher mean subject accuracy at the other tested values. The increase of performance of AGDL might come from the fact that training across α\alpha values that is in fact a multi-task learning problem may provide a regularizing effect through shared parameters and continuous dictionary variation. The only setting where fixed-α\alpha outperforms AGDL is for contrast classification at α=1\alpha=1 where contrast classification fails for both fixed-α\alpha and AGDL anyways. We showed that conditioning by α\alpha preserves the strong subject-identification performance of the geometry-only setting while improving contrast classification across the other tested settings.

Training cost. In our experiments, training one fixed-α\alpha dictionary takes approximately the same time as training the α\alpha-conditioned model. Let TUT_U denote the shared ULOT pretraining cost and TDT_D the cost of one dictionary-learning run. Training KK separate dictionaries therefore costs approximately TU+KTDT_U+KT_D, compared with TU+TDT_U+T_D for AGDL. For the five values tested here, this corresponds to approximately for K=5K=5 five times the dictionary-training cost, which is 2020 hours for AGDL and 100100 hours for fixed α\alpha dictionary learning.

Finally, once trained, AGDL can also be evaluated at additional α\alpha values without retraining the dictionary. This enables finer selection of α\alpha, which is particularly useful given the sharp transition we observe near α=1\alpha=1: a coarse fixed grid could miss this behavior.

Q2 Embedding dimension matching Multi-AGDL concatenates embeddings of the same graph at three values of α\alpha, producing a 360-dimensional representation. We compared it with both the original 120-dimensional baseline and a 360-dimensional baseline.

Embedding dimensioncontrast classification accuracysubject classification accuracy
Multi-AGDL (360)0.780±0.01450.780 \pm 0.01450.921±0.01480.921 \pm 0.0148
Baseline (120)0.934±0.01320.934 \pm 0.01320.0202±0.01600.0202 \pm 0.0160
Baseline (360)0.939±0.01430.939 \pm 0.01430.0626±0.01740.0626 \pm 0.0174

At matched dimensionality, Multi-AGDL retains substantially higher subject-identification accuracy, while its contrast-classification accuracy remains slightly lower than the baseline’s. Its subject-identification advantage over this baseline therefore cannot be explained solely by the larger embedding dimension.

About anatomy only classification, at α=1\alpha=1, the activation-discrepancy term vanishes from the FGW reconstruction objective, leaving only the structural term. AGDL reaches 1.001.00 subject-identification accuracy in this setting which is interesting because it relies on the embedding estimated from ULOT solver so recovering perfect accuracy was not a given. Any other baseline considering only the anatomy would reach perfect performance also because subjects are characterized by their brain geometry. However, the strentgh of our contribution is the interpolation and multi-view representation between geometry and activation characterization. Our result demonstrates that the learned dictionary embeddings preserve these subject-specific anatomical differences, supporting the practical effectiveness of ULOT-based dictionary learning for capturing structural information.

Thanks for the response and more experiment results. The fixed-α comparison helps support the benefit of conditioning, and the dimension-matched baseline addresses my concern about embedding size. Raised my score to 3.

We thank you for your constructive feedback, which helped us better motivate the α\alpha-conditioned dictionary and clarify the Multi-AGDL results, and we thank you for increasing your score.

Reviewer_UjyfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Amortized Graph Dictionary Learning (AGDL) for learning fMRI graph dictionaries directly on subject-specific brain geometries using the Fused Gromov-Wasserstein (FGW) distance. A pretrained ULOT network is used to approximate FGW transport plans, while an "α\alpha" -conditioned MLP generates dictionary atoms that interpolate between functional-feature alignment and geometric alignment. Experiments are conducted on approximately 1,200 HCP subjects and seven tasks, including FGW versus L2 retrieval, ULOT validation, runtime comparisons, brain-map interpolation, contrast/subject classification, and transfer across graph resolutions. The main contribution is the extension of FGW graph dictionary learning and amortized optimal transport to relatively large fMRI graphs while attempting to preserve subject-specific geometry that may be lost under template registration. The experiments suggest substantial runtime improvements in the reported small-scale setting and show that different "α\alpha" values produce representations with different contrast- and subject-discriminative properties.

Critical strengths
  1. Rather than training a separate fixed dictionary for a single trade-off parameter, the paper parameterizes dictionary-atom generation with a conditional neural network. This allows one model to cover different trade-offs, ranging from feature alignment to geometric alignment. The design provides a direct way to control which type of information the representation emphasizes, and the experiments suggest that representations obtained at different trade-off values have complementary discriminative properties that can be combined. Compared with training an independent model for each trade-off setting, this offers a more unified and flexible modeling framework and may reduce redundant training costs. It therefore constitutes a clear methodological contribution.
  2. The authors apply Fused Gromov-Wasserstein-based graph dictionary learning to real subject-specific native brain geometries and large-scale whole-brain fMRI graphs. They further examine whether a dictionary learned at one resolution can be transferred to inputs with different resolutions. This directly addresses the scalability bottleneck that has limited related approaches mainly to small graphs or proof-of-concept experiments. The work therefore moves FGW dictionary learning closer to realistic neuroimaging settings and has clear practical and engineering value.
Critical weaknesses
  1. The paper states that the data are split at the subject level and that the ULOT transport approximation and dictionary learning use the same split. The key issue therefore concerns the specific construction of the subject-classification evaluation rather than whether subject-level splitting was performed. Within the held-out subject set, the authors further divide each subject’s multiple task-contrast maps into training, validation, and test subsets. Consequently, the identities of the test subjects are already represented in the classifier training data through other contrast maps. Because different contrast maps from the same subject share subject-specific geometry, Because subject-specific geometric fingerprints are an intended source of subject information in this paper, high subject-classification accuracy at large alpha is not itself a concern. The more important question is whether contrast classification retains activation-related signal when subject-specific coordinate cues are masked or permuted
  2. In addition, for the 950-node dictionary-learning experiments, quantitative test-set reconstruction errors are not reported. Reconstruction quality is illustrated mainly through qualitative visualizations. I recommend adding controls that use geometry only, activation information only, and permuted or perturbed 3D coordinates, together with quantitative reconstruction metrics on the test set. These controls would clarify whether contrast-discriminative performance remains after removing geometric cues, while allowing geometry to remain a valid source of subject-identification information.
Questions for authors
  1. Please clarify the exact data-splitting protocol used for subject classification. Within the held-out subject set, are different contrast maps from the same subject separately assigned to the classifier’s training, validation, and test sets? If so, the identities of the test subjects are already represented in classifier training through their other contrast maps. This experiment should therefore be interpreted more precisely as cross-contrast identification of seen subjects, rather than generalization to unseen subjects.Please also provide geometry-only, activation-only, coordinate-masked, and coordinate-permuted controls. For each control, specify which inputs, and which FGW loss terms are retained or removed. These analyses would help distinguish discriminative information arising from functional features from that arising from subject-specific geometric fingerprints. In addition, please explain the origin of the 145 subject classes shown in Figure 3 and clarify how this number relates to the final number of included subjects and the numbers of subjects in the training, validation, and test splits.
  2. The baseline is defined on common geometry with least-squares reconstruction, whereas AGDL uses native geometry and coordinate information. Please separate the effect of the modeling objective from the effect of additional geometric inputs by evaluating AGDL with coordinates masked or permuted and by matching representation dimensionality where possible.
  3. Please report quantitative FGW reconstruction errors on the test set. At the scale used in the main experiments, please also report the end-to-end runtime, peak memory usage, and other relevant resource costs for AGDL and the baseline.To evaluate the quality of the ULOT approximation, At an intermediate graph size where an independent iterative solver remains feasible, please report solver-only reference losses, the ULOT loss gap, transport-plan discrepancy, gradient error, and any degradation trend as the graph size approaches the 950-node setting. Please also report failure cases or stability issues, if any. In Figure 1, the reference should be based on the independent solver alone rather than on the minimum of the ULOT and solver losses.
4 comments

Thank you for your thorough analysis of our paper and your relevant comments. We have tried our best to address your concerns below. We applied all the reviever's suggestions in red in the updated version of the paper.

Q1-W1 Subject splitting clarification for subject classification We randomly divided the 1,113 HCP subjects into disjoint training (70%), validation (15%), and test (15%) sets, using the same split for ULOT training and dictionary learning. For the subject-classification evaluation, we retained only the 143 held-out test subjects with complete fMRI data, ensuring that all contrast maps could be computed for each included subject. We then split these subjects’ contrast maps into classifier training, validation, and test subsets, with the same subject identities represented across all three subsets.

We do not claim that we generalize to unseen subjects, and we will clarify this point in the paper, thanks for the comment. The evaluated subjects are unseen during representation learning, but their identities are represented in the classifier training set because we currently do not have different geometries per subject. This experiment measures identification across contrast maps, rather than recognition of identities absent from classifier training. However, even though the geometry is shared between train, validation, and test datasets, the subject classification evaluation performed in the paper assesses whether the learned embeddings preserve subject-specific information across contrast maps. The perfect accuracy obtained at α=1\alpha=1 therefore indicates that AGDL retains enough subject-specific geometric information to distinguish the evaluated individuals, including individuals unseen during representation learning. We interpret this result as evidence of the preservation of geometric identity information since classification is done on the AGDL embedding and not on the original structures that are indeed the same. But we agree that this needs to be clarified in the main paper.

W1 W2 Q1 robustness to geometry corruption We thank the reviewer for raising this question. To investigate whether contrast-related information remains decodable when geometric inputs are perturbed, we performed two additional experiments: adding noise to the 3D coordinates and permuting graph edges. We report classification accuracy across five random splits, both using all α values (Multi-AGDL over 10 alpha values) and using a single α selected on validation data. Contrast classification remains robust under both perturbations. At the largest tested edge-permutation level (0.5), contrast accuracy in the all-α setting remains essentially unchanged, from 85.80 ± 1.62% to 86.15 ± 1.66%, whereas subject-identification accuracy drops from 97.29 ± 0.60% to 46.63 ± 1.02%. With validation-selected α, contrast accuracy likewise remains stable, from 88.19 ± 0.79% to 87.99 ± 1.09%. Coordinate noise produces a modest degradation: even at the largest tested level (5.0), contrast accuracy decreases by only 2.19 percentage points using all α values and 2.33 percentage points using validation-selected α.

PerturbationLevelContrast (Multi AGDL)Subject (Multi AGDL)Contrast (selected α)Subject (selected α)
None0.085.80 ± 1.6297.29 ± 0.6088.19 ± 0.79100.00 ± 0.00
Coordinate noise0.585.76 ± 1.6096.98 ± 0.3788.16 ± 0.76100.00 ± 0.00
Coordinate noise1.085.42 ± 1.5697.19 ± 0.7488.37 ± 0.68100.00 ± 0.00
Coordinate noise2.084.58 ± 1.8796.98 ± 0.7887.92 ± 0.6699.93 ± 0.09
Coordinate noise5.083.61 ± 0.8794.58 ± 0.5185.87 ± 0.8099.93 ± 0.14
Edge permutation0.185.62 ± 0.8592.57 ± 1.3088.09 ± 0.6699.90 ± 0.09
Edge permutation0.2585.80 ± 0.5576.63 ± 2.1988.12 ± 0.8599.69 ± 0.34
Edge permutation0.586.15 ± 1.6646.63 ± 1.0287.99 ± 1.0995.38 ± 0.73

These results support the retention of contrast-related signal under geometric perturbations, including a setting that substantially reduces subject-identification performance.

Q3 W2 Reconstruction error We measure the FGW mean and std reconstruction error over the training dataset and held-out test subjects. We find that the FGW reconstruction error is 0.0094±0.0260.0094 ± 0.026 and 0.0093±0.0250.0093 ± 0.025. These results show no appreciable train–test gap in mean FGW reconstruction error.

Q3 ULOT approximation error on larger graphs We evaluated 200 test graph pairs spanning 200–1000 nodes, covering both intermediate sizes and the approximately 950-node setting.

Number of nodesGradient cosine similarityLoss Pearson correlation <br> to the best lossULOT − solver (mean ± std)
200–4000.970 ± 0.0260.9732.644×10−5±6.834×10−42.644 \times 10^{-5} \pm 6.834 \times 10^{-4}
400–6000.949 ± 0.0490.966−5.401×10−5±9.402×10−4-5.401 \times 10^{-5} \pm 9.402 \times 10^{-4}
600–8000.867 ± 0.1160.990−1.682×10−4±1.008×10−3-1.682 \times 10^{-4} \pm 1.008 \times 10^{-3}
800–10000.969 ± 0.0270.9796.182×10−5±6.879×10−46.182 \times 10^{-5} \pm 6.879 \times 10^{-4}

The signed loss gaps and Pearson correlations show no monotonic deterioration with graph size, with an mean difference that is an order of magnitute smaller than the standard deviation. In Figure 1, we compare the FUGW loss obtained with ULOT against the best loss achieved by either ULOT or the iterative solver for each graph pair. Because the FUGW objective is non-convex, the iterative solver is not guaranteed to reach a global minimum. Indeed, ULOT sometimes achieves a lower objective value than the solver, including cases where the solver returns a degenerate all-zero transport plan. Accordingly, “best loss” denotes the best objective value attained by the two methods, rather than a certified global optimum but is a better proxy for measuring performance.

Q3 runtime and memory We profiled the training of AGDL at full resolution. We find that the averaged runtime is 61.10 ± 0.47 seconds per epoch (mean ± standard deviation).

MetricAGDLBaselineGDL
Mean runtime per epoch61.10 ± 0.47 s16.71 ± 0.19 s2383s
Total runtime20 hours4.5 hours780 hours (estimated)

We also measure the peak GPU allocation for AGDL and the Baseline, which are respectively 23.91 GiB and 5.98 GiB.

Thanks for the detailed response, which addresses my concerns. The clarified splitting protocol makes it clear that the subject-classification experiment is identification across contrast maps, and the coordinate-noise and edge-permutation controls show that contrast information does not come mainly from geometric fingerprints. The added reconstruction, ULOT, and runtime results cover my remaining points.

We thank you for the time spent reviewing our manuscript and for confirming that your concerns have been answered. Your constructive feedback helped us clarify the subject classification protocol and AGDL’s runtime, and assess the robustness to geometry corruption.

Reviewer_3EgAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Amortized Graph Dictionary Learning (AGDL) for fMRI graphs with heterogeneous individual geometries. AGDL uses a ULOT network to approximate Fused Gromov-Wasserstein transport plans and learns dictionary atoms conditioned on the FGW trade-off parameter alpha. The method is evaluated on HCP data for contrast and subject retrieval, dictionary learning, native-to-common geometry transfer, and low-resolution transfer. The paper addresses a real limitation of registration-based fMRI analysis and reports evidence that the amortized transport plans are close to the reference solver on smaller graphs.

Critical strengths
  1. The problem is important and well motivated. Preserving subject-specific cortical geometry while learning shared representations is a meaningful alternative to projecting every subject onto fsaverage.
  2. The combination of amortized OT plans and dictionary atoms conditioned on alpha is technically coherent. The ULOT validation includes objective correlation of 0.99, gradient cosine similarity of 0.93, and small marginal violations, which directly tests the approximation used by AGDL.
  3. The experiments cover several useful questions: discrimination of contrasts and subjects, interpolation of atoms, transfer between graph scales, and native-geometry versus registered representations. The method is also described with sufficient algorithmic detail to make the main optimization loop understandable.
Critical weaknesses
  1. The main empirical claim about better subject-specific information needs stronger downstream evidence. Contrast and subject retrieval are useful probes, but it remains unclear whether AGDL improves a clinically or scientifically relevant task compared with carefully tuned native-space and registered-space baselines. Please add decoding or prediction tasks with subject-level splits and report confidence intervals over participants and contrasts.
  2. The comparison to registration is not fully controlled. Native and fsaverage graphs may differ in preprocessing, resolution, feature scaling, or signal quality. The paper should state the exact graph construction and normalization for every condition, and include a matched comparison where the same dictionary-learning optimizer and number of atoms are used.
  3. The scalability claim is incomplete. AGDL still evaluates a cubic FGW loss during every outer iteration, and the reported pretraining cost is about 200 hours while dictionary learning takes about 20 hours. Please report wall-clock and peak-memory scaling with node count, the number of graphs, and the number of dictionary atoms, and compare with a practical baseline using approximate or subsampled OT.
  4. The ULOT validation uses smaller 100-node graphs and defines the best loss as the minimum of ULOT and the standard solver. This is reasonable as a diagnostic, but it can make the reported approximation quality optimistic. A direct comparison against independently initialized solvers on the actual 900-1000-node regime would better support the optimization claim.
  5. The dependence on alpha and the seven-bin encoding is underexplored. Since the dictionary is explicitly conditioned on alpha, the paper should show sensitivity to the number of bins, interpolation between bins, and performance when the test distribution of alpha differs from the training distribution.
Questions for authors
  1. What downstream fMRI decoding or prediction task improves over matched native-space and registered-space baselines, with participant-level confidence intervals?
  2. Are all graph resolutions, feature normalizations, and dictionary sizes identical across native and fsaverage comparisons?
  3. Please provide runtime and memory scaling on the full 900-1000-node setting and compare against an approximate OT baseline.
  4. How does the ULOT approximation behave when the reference solver is independently initialized, and how does it behave at the full graph size?
  5. How sensitive are results to the number of alpha bins and to alpha values between bins or outside the training distribution?
2 comments

Thank you for your thoughtful and constructive feedback. We appreciate that you find the problem important, well motivated, and technically coherent. We try our best to answer your concerns below. We applied all the reviewers' suggestions in red in the updated version of the paper.

Q1-W1. Downstream fMRI prediction task.

We could study in future work whether AGDL can be applied to tasks where the brain geometry is relevant, such as aging. This is outside the scope of the current paper, since the HCP dataset considers young adults only, but could be an interesting future direction. Our paper nevertheless highlights a methodological advantage of AGDL: it enables dictionary learning without requiring all input graphs to share a predefined node correspondence. A learned dictionary can therefore be applied to data represented on different individual meshes and at different resolutions, as illustrated by our cross-resolution experiments.

Importantly, this flexibility opens the possibility of learning shared representations across datasets. We illustrate this by embedding data from the IBC dataset [1], which is another fMRI dataset with different contrast maps and subjects. We do not retrain a dictionary learning pipeline but use the dictionaries learned on the HCP dataset and perform the unmixing step on the contrast maps of the IBC dataset, each with its individual geometry. For this application, the baseline dictionary learning is not directly applicable since these two datasets are not sampled in the same space. We find that the embeddings allow us to classify the 140140 IBC contrasts with 0.189 accuracy (chance is 0.007) and perfect subject classification. This shows promising results towards learning general cross-dataset brain templates, only possible with a dictionary learning pipeline that works across geometries, such as AGDL.

[1] Pinho, A. L., Amadon, A., Ruest, T., Fabre, M., Dohmatob, E., Denghien, I., ... & Thirion, B. (2018). Individual Brain Charting, a high-resolution fMRI dataset for cognitive mapping. Scientific data, 5(1), 180105.

Q2 W2 Clarification on native vs fsaverage. In the main comparison, both representations use 950 nodes per hemisphere, and AGDL and the baseline dictionary learning both use 60 dictionary atoms. Contrast maps are computed using the same first-level GLM configuration: a repetition time of 0.72 s, and identical event definitions and contrast expressions. Both pipelines produce statistical z-score maps, exclude the medial wall, and compute each node’s activation feature by averaging the contrast values within its parcel. The native pipeline computes a separate parcellation for each subject, whereas the fsaverage pipeline reuses a common parcellation across subjects to maintain node correspondence. Thus, the number of nodes and dictionary atoms is matched, while the spatial correspondence and subject-specific geometry differ by design.

Q3-W3. Runtime and memory scaling. In the ULOT paper, the authors show that ULOT is the fastest approximation of FGW, with the best computationality/error tradeoff. They show that any other approximation is unscalable for $ 1000$-node graphs or introduces too much error. We profiled the first six epochs of training at full resolution. Excluding the first epoch, runtime averaged 61.10 ± 0.47 seconds per epoch (mean ± standard deviation over epochs 2–6). We also ran one epoch of exact GDL, which is the GDL without ULOT approximation, which takes approximately 4040 times longer than AGDL and is clearly not feasible for our dataset.

MetricAGDLBaselineGDL
Mean runtime per epoch61.10 ± 0.47 s16.71 ± 0.19 s2383s
Total runtime20 hours4.5 hours780 hours (estimated)

We also measured the peak GPU allocation for AGDL and the Baseline, which are 23.91 GiB and 5.98 GiB, respectively.

Q4-W4. ULOT approximation at full resolution. At the full graph resolution, we observe no deterioration in ULOT’s approximation quality. We report three metrics: (i) the Pearson correlation between the ULOT loss and the best of the ULOT and solver losses for each graph pair; (ii) the cosine similarity between the gradients obtained using ULOT and the solver; and (iii) the loss difference, defined as LULOT−LsolverL_{\mathrm{ULOT}} - L_{\mathrm{solver}}.

Number of nodesGradient cosine similarityLoss Pearson correlation to the best lossULOT − solver (mean ± std)
1000.93 ± 6.68e-040.9931.356×10−6±5.981×10−41.356 \times 10^{-6} \pm 5.981 \times 10^{-4}
800–10000.969 ± 0.0270.9796.182×10−5±6.879×10−46.182 \times 10^{-5} \pm 6.879 \times 10^{-4}

We show that there is no deterioration of the ULOT approximation at the full resolution, with a mean difference that is an order of magnitute smaller than the standard deviation.

Q5-W5. AGDL sensitivity to the alpha bins. To measure the effect of the number α\alpha bins, we retrained AGDL for different numbers of bins. We will provide the results as soon as possible.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.