acceptodds
Withdrawn from ICLR 2027

GINO: Graph DINO through Canonical Token Spaces

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Modern self-supervised learning in language and vision leverages tokens with well-defined positional structure. Words follow an ordered sequence, while image patches occupy a spatial grid. Graphs, in contrast, have no intrinsic node order, causing any positional machinery tied to input indices to fail under different node labelings. We introduce GINO, a canonicalization-based framework that equips graph tokens with a consistent spatial layout. GINO extracts a canonical node ordering from the graph's spectral structure and uses it to assign deterministic, two-dimensional coordinates to graph edges. Embedded in this consistent coordinate system, positional encodings and DINO/iBOT-style objectives can operate seamlessly, independent of the initial node labeling. We provide theoretical backing for this framework by guaranteeing the ordering's invariance and bounding its proximity to the exact optimum under a normalized Dirichlet energy. Across molecular benchmarks, self-supervised GINO pretraining improves downstream performance and achieves competitive transfer using only 2D graph structure.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
graph canonicalization, graph isomorphism, permutation symmetry, spectral graph theory, graph transformers, graph positional encodings, canonical forms

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_2tHUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

GINO constructs canonical graph-edge coordinates using spectral ordering and structural refinement, then applies 2D RoPE and DINO/iBOT-style pretraining with independently recanonicalized crops. GINO provides invariance and ordering-energy analyses, canonicalization audits, and molecular downstream evaluations.

Critical strengths
  • The paper proposes an interesting idea: the coherent integration of canonical graph coordinates with standard Transformer self-supervision.
  • The authors derive a useful energy bound and informative supplementary controls, with explicit reporting of limitations.
Critical weaknesses
  • Novelty and theoretical relevance need sharper justification. Most components build on established methods, making the specific integration the principal contribution. The energy analysis appears sound, but near-optimal ordering energy does not establish better positional representations or downstream learning. The main text should incorporate the appendix’s limitations and correct the attribution of production-order statistics to direct spectral ordering in Section 4.2.
  • I think some of the important ablations of the method are in the appendix, and they should be in the main text. So, based on ablations in the appendix, the benefit of the proposed components remains unresolved. Appendix H does not establish an advantage of canonicalization over per-epoch relabeling or of canonical RoPE over no RoPE. The distinctive anti-diagonal crop also lacks a matched random-edge-removal baseline.
  • I think the writing and placement of materials in the paper are not correct; the method section starts with an algorithm, two figures contain no useful information for the method section, and the benchmarking details are missing, since, based on the information in the benchmark section, GINO reports validation aggregates, including best-validation scores used for checkpoint selection, while external methods use different evaluation protocols and populations. Deduplication and graph exclusions further alter the benchmarks. Appendix G.4 acknowledges that these comparisons are contextual, contradicting the stronger ranking claims in Table 2 and Section 4.3.
  • This is not clear in the text, but it seems the attributed-invariance claim contradicts the evaluated implementation. Sections 2.2, 3.1, and 4.1 claim attribute-based tie-breaking and invariant attributed representations. However, Appendix A.2 states that all experiments use topology_only=True, and Appendix C.1 reports changed canonical-order node features under relabeling for 48/400 BBBP molecules. Structural canonicality does not guarantee invariant feature–coordinate inputs or predictions.
  • I think the main claim of the paper is for general graph representation learning, but the main evaluation covers eight molecular datasets, providing breadth across prediction tasks but limited diversity in graph structure and domain. Appendix K includes smaller non-molecular experiments, but their mixed results do not establish broadly beneficial transfer. The authors should either narrow the claims to molecular representation learning or provide stronger evidence across diverse graph domains and sizes.
Questions for authors
  • Which attribute-handling configuration produced the headline results, and how invariant are the resulting predictions under relabeling?

  • Can you provide test comparisons using matched preprocessing and splits?

  • Can you provide more diverse benchmarks?

Reviewer_Rj5KOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces GINO, a graph Transformer that uses spectral canonicalization to give graph nodes a consistent ordering. It represents graph edges as 2D tokens, allowing the use of RoPE and DINO/iBOT-style self-supervised learning. The authors also propose a graph cropping method for self-supervised training and provide theoretical results on permutation invariance and spectral ordering. They evaluate the method on eight MoleculeNet datasets after pretraining on GEOM, reporting average ROC-AUC scores of 69.43 for frozen representations and 77.55 for fine-tuning.

Critical strengths

None

Critical weaknesses
  • Sections 3.1 and 4.1 claim that node attributes are used to ensure permutation invariance. However, Appendix A.2 states that all experiments use topology_only=True, meaning node attributes are not used during canonicalization. This seems to contradict the claims in the main text.
  • Table 2 compares GINO’s validation results with results from previous papers using different data splits and preprocessing. As acknowledged in Appendix G.4, these results are not directly comparable, making the claimed improvements less convincing.
  • Appendix C shows that a large portion of graphs require exact fallback refinement. It is unclear how much of the method’s effectiveness comes from spectral ordering itself.

I found the paper, particularly its supplementary material, extremely difficult to follow and evaluate. A substantial portion of the appendix is devoted to explaining historical experiments, implementation errors, and superseded results that are no longer part of the current study. For example, Appendix D.5 discusses a “historical P2 TEST comparison” that is not used in the final evaluation, Appendix H.1 describes an earlier implementation that “accidentally kept the epoch fixed at zero,” and Appendix L documents several previous issues, including “biased non-edge sampling,” “molecular-identity leakage,” and incorrect cropping coordinates. I also find it very unusual that the paper includes an entire table (Table 34, “Major experiment-lineage corrections”) dedicated to documenting invalid and superseded experiments, as well as a separate section titled “Incomplete Work” (Appendix L.4), which lists experiments that were never completed.

I do not understand why so much of the supplementary material focuses on previous implementation failures and incomplete experiments rather than the actual contributions of the paper. These discussions often seem unnecessary or disconnected from the main work, making it difficult to determine which experiments and conclusions are relevant to the current submission. While the authors acknowledge the use of LLMs, they are ultimately responsible for verifying everything in the paper. Given the inconsistencies and confusing presentation, I do not believe this verification was done properly. This significantly reduces my confidence in the reported results, and I am therefore not comfortable reviewing the paper further in its current form.

Questions for authors

See Weaknesses

Reviewer_fbapOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

GINO maps a graph into a canonical coordinate system so that standard Transformer self-supervision can be applied. The canonicalizer sorts the Fiedler vector of a normalized Laplacian built from a Gaussian kernel on shortest-path distances, with a sign cascade, structural refinement, and a bounded exact search for ties and repeated eigenvalues (Sec. 2, App. A); in the reported experiments it orders atoms by topology only and attaches atom features afterward (App. A.2). Each bond becomes a token with a 2D coordinate that is used for RoPE and for crops in a DINO/iBOT-style objective with added KoLeo and Gram terms (Sec. 3; Eq. 53 in the appendix). Theorem 2.1 gives relabeling invariance when the second eigenvalue is simple, the largest-magnitude Fiedler entry is unique, and all entries are distinct; Theorem 2.2 bounds the normalized Dirichlet energy gap of Fiedler sorting. Experiments audit canonicalization on about 304k GEOM molecules and report frozen and fine-tuned validation ROC-AUC on eight MoleculeNet tasks (Table 2).

Critical strengths
  1. A carefully engineered and audited structural canonicalizer. Certified branches come with an exact-arithmetic invariance argument (Thm. B.8), uncertifiable graphs are quarantined instead of approximated, and the structural audits are thorough: zero mismatches against BLISS on 10,571 graphs with 20 relabelings each, and 99.975% of GEOM certified at a 35 ms median (Tables 5 to 8). Any method that feeds node order to a model needs this kind of reliability. The appendix is also unusually transparent: it separates current from historical and superseded runs and lists which claims are not established (App. L).

  2. A simple interface to vision-style tooling, with correct supporting analysis. Bond tokens with pair coordinates let 2D RoPE, crops and DINO/iBOT heads run on an unchanged Transformer (Sec. 3). The spectral derivations are correct, and the orders are close to optimal for their energy: direct Fiedler sorting reaches the exact optimum on 46.7% of 4,253 small graphs and the production order on 32.0% (Table 14). If a predictive benefit were shown, others could build on this interface.

Critical weaknesses
  1. The headline comparison is not controlled, and the pretraining benefit is not resolved. GINO's Table 2 numbers are validation aggregates on rebuilt populations: the fine-tuned scores are the best validation ROC-AUC with early stopping on the same split, and the current Table 2 results do not evaluate the test split (App. D.5, Table 21). The populations are deduplicated and filtered (for example, ToxCast drops from 8,579 to 6,826 molecules, Table 16), while the baseline rows are copied from published tables, and at least the GraphMAE row uses a different, test-based endpoint (it matches the GraphMAE paper, which follows Hu et al., 2020). The paper itself says no controlled cross-method ranking is implied (Table 24 caption), yet Sec. 4.3 states that GINO "consistently outperforms these 2D frameworks". The matched SSL control is positive on average (+0.028, with five of eight datasets positive), but its 95% CI of [−0.00009, +0.064] narrowly includes zero (Table 23), while Sec. 4.3 calls the result "driven explicitly by self-supervised pretraining". The no-SSL model already scores 74.75, above every copied fine-tuned baseline (best 74.04), so the heterogeneous table cannot isolate a pretraining benefit either. A held-out evaluation on matched populations, with conclusions calibrated to it, is needed to support the main claim.

  2. The claimed invariant attributed token space is not what the experiments use, and the benefit of canonical coordinates is not isolated. Sections 2.2, 3.1 and 4.1 say atom attributes break structural ties, giving a strictly invariant attributed layout. Appendix A.2 says all reported experiments use topology-only canonicalization with atom features attached afterward, and C.1 finds that the canonical-order node-feature matrix changes under relabeling for 48 of 400 BBBP molecules (12.0%), while the structural matrix never changes. For example, the heavy atoms of 1-propanol form a path C-C-C-O whose reversal preserves the bond-weighted structure, so a topology-only order cannot fix which end the O occupies. The structural canonicalizer is well supported, but invariance of the model's attributed input, and so of its embeddings and predictions, is not established. The coordinate controls show a resolved gain of canonical+RoPE over one fixed arbitrary labeling (+0.0089, CI [+0.0015, +0.0189], Table 25), but they do not resolve a difference from per-epoch random relabeling (CI [−0.0100, +0.0119]) or an incremental effect of RoPE (Tables 25 and 27), as App. L.5 acknowledges. So the central idea, fix the labeling and then reuse standard positional machinery, is only partly supported.

Questions for authors
  1. Can you report held-out test ROC-AUC for the current frozen and fine-tuned models, with key baselines (for example GraphMAE and Mole-BERT) evaluated on the same retained molecules and split manifests, or under an exactly matched published protocol? Please separate the effects of preprocessing, architecture, pretraining and evaluation endpoint. A matched comparison would address my main soundness concern even if GINO does not lead.

  2. Does the canonical versus per-epoch relabeling control (Table 25) use the current Family-C pretraining and downstream recipe? If not, which factors differ, and what evidence isolates the contribution of canonical coordinates in the current system, with architecture, initialization, optimization and augmentation budget held fixed?

  3. Please add a current frozen random-initialization probe with the same splits, feature standardization and classifier settings. The only frozen control I found comes from an earlier recipe (S2/P2, App. G.3), where the SSL encoder scored below its random-init control (0.7082 vs 0.7380), so it says little about the current encoder either way.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.