acceptodds
Withdrawn from ICLR 2027

ICE: Task-Aligned Clifford Latent Fields for Multimodal Graph Foundation Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal attributed graphs connect entities, visual content, language, and observed relations. Learning one foundation across such graphs requires more than compressing each node into a fused Euclidean vector. The representation must preserve entity semantics, construct interaction state from graph neighborhoods, and expose that state to prediction units with different geometry. Our empirical study shows why these requirements are inseparable. Higher-grade channels recover pair relations across the foundation graphs, specialized queries reveal information hidden by a generic readout, and rigid blade isolation removes cross-grade capacity. We therefore introduce ICE (Interaction-aware Clifford Encoder), a multimodal graph foundation model built on a node-indexed Clifford latent field. Topology, text, and images enter explicit Cl(3) addresses. Edge-aware geometric products transform these directions into scalar, bivector, and trivector relations over observed neighborhoods. A protected Grade-1 route preserves entity semantics, while the full grade and depth bank remains available to fresh node and link heads. We establish exact cross-grade reachability, node-permutation equivariance, and a bound on the task residual around the semantic score. Experiments span one shared foundation over eleven graphs, six node-classification datasets, three link-prediction datasets, and matched few-shot tasks. ICE ranks first in all 30 reported supervised and few-shot comparisons. Core removals reduce every task summary, and mechanism controls connect the gains to higher-order transport, retained multidepth structure, semantic protection, and direct field access.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multimodal graph foundation models, geometric deep learning, graph representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BU44OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on multimodal graph foundation models. The novelty is the paper proposes ICE based on Clifford Algebra. It puts text and image into independent basis. During graph propagation, it uses geometric product to generate high-grade relations. Moreover, instead of global pooling, ICE saves all layers and grades into a feature bank. Thus, downstream tasks can take what they need. The author showed ICE achieves SOTA performance

Critical strengths
  1. The motivation of this model design is very reasonable. Most multimodal graph models just pool everything into one final vector, and this will easily destroy the original node semantics. The author propose an idea to keep a complete grade-depth bank and let downstream tasks choose what they need by themselves is very useful. Specifically, the model uses the protected G1 route for NC, and uses high-grade bivectors for the LP. This successfully solves the conflict between node semantics and edge relations.

  2. The ablation experiment (Table 4 and Figure 3) is designed very well. Unlike many papers that just do normal component removing, the authors successfully use it to prove their decoupling motivation. We can clearly see that removing the "G1 route" mainly hurts NC tasks, and removing the "Clifford product" mainly hurts LP tasks.I think this is a very smart experiment design and makes the paper much more convincing.

Critical weaknesses
  1. The paper claims the proposed ice model achieves sota performance. However, I have a big concern about the fairness of the baseline comparisons. The node state dimension is expanded by 8 times, and the geometric product edge transport takes 64 times more computation. The performance gain might simply come from this larger computation capacity and wider node, rather than the Clifford algebra design. The paper lacks a strictly parameter-matched baseline comparison.

  2. The paper claimed it istask-aligned field access. It means downstream tasks can freely query the feature bank. But when we look at Equation 14 in the Appendix, we can find that the Link Prediction head actually uses a manual way to extract Grade 0 and Grade 2 to build the pair feature. This is just manual hard-coding. The model does not learn to choose high-grade features adaptively by itself. So, this is totally different from what the paper claims

  3. The title of this paper says it is a Graph Foundation Model. But if we look at the experiment part, the model only tests on the same 11 graphs already used in the pre-training stage. I think this is at most multi-task joint pre-training. A real foundation model should prove its zero-shot or inductive generalization ability on completely new graphs that the model never saw before. So, the current experiment results cannot support the title of Foundation Model.

Questions for authors
  1. Can you provide a parameter-matched matched baseline comparison?
  2. In Equation 14, the Link Prediction head manually extracts Grade 0 and Grade 2. Why do you use this hard-coding design? Can the network learn to dynamically select the best grades using some attention weights?
2 comments

We thank Reviewer BU44 for appreciating the node/pair motivation and retained-bank design. We address model size, learned grade selection, and transfer beyond the pretraining graphs below.

W1 & Q1: Matching size while separating the edge operation. A Clifford state contains 8h8h scalar channels, so matching nominal h alone is insufficient. Section 5.2 and Appendix F.2, Table 9, compare ICE's 59.51 M active encoder parameters with PLANET-PM's 59.52 M, with approximately 1.38 M allocated LP-head parameters for each. PLANET-PM sets active architecture dimensions to match the budget rather than adding unused padding. ICE achieves NC Accuracy/Macro-F1 and LP MRR of 83.20/74.90/26.24 versus 82.47/74.11/25.48.

Scalar/Hadamard transport provides a more targeted comparison. Both retain Q/K/V, dimensions, layers, bank, and the same heads, replacing only the edge operation. Their LP MRR is 25.16/25.39 versus ICE's 26.24. The improvement therefore persists with the same representation width, intermediate access, and readout (Appendix F.2).

The 64 coefficient-pair terms count one local product, rather than the whole pipeline. Dense node maps cost O(L∣Vb∣(hB)2)O(L|V_b|(hB)^2) and edge products cost O(L∣Eb∣hB2)O(L|E_b|hB^2). Appendix E.5 gives the full complexity expression with widths held constant. Appendix F.6, Table 12, compares identical hardware, batch, sampling, and precision. With ICE time/peak memory normalized to 1.00/1.00, PLANET-PM uses 0.88/0.91, scalar transport 0.79/0.94, and Hadamard transport 0.86/0.97. This reports ICE's additional cost directly without equating a local operation count with end-to-end runtime.

W2 & Q2: Specified relation summaries complement learned access to every grade. The scalar and bivector-norm relation terms are deliberate design choices. Their purpose is to summarize pair relations while preserving endpoint symmetry. They supplement endpoint sum, absolute difference, and product features from every grade and depth. The supervised LP MLP learns how to combine this complete descriptor, so access is not restricted to Grades 0/2. Section 4.3 and Appendix D.4 now make this distinction explicit.

To test the requested adaptive alternative, Appendix F.5, Table 11, adds symmetric pair-dependent softmax gates over the same grade/depth features, retaining the relation terms and projection/MLP architecture:

Pair headSportsClothBooks-LP
Fixed dictionary28.3821.0829.26
Pair-conditioned gate28.4421.2229.33

The gate improves MRR by 0.06/0.14/0.07 points and allocates 2,316 parameters (0.17%). At least 384 gate weights are inactive because H0 Grades 2/3 are zero. Likewise, the full LP head allocates 1,376,769 parameters, with at least 180,224 structurally inactive; same-bank controls share these zeros. The small absolute increments support retaining the simpler dictionary as the primary head. Appendices F.1 and F.5 give the parameter accounting. NC gates are class-specific, whereas AHA and the additional pair gate depend on the input.

W3: We distinguish graph reuse from inductive transfer. The original nine-graph evaluation tests new tasks on pretraining graphs (Section 5.1; Appendix A.1). Section 5.3 and Appendix F.8, Tables 4 and 14, add PLANET's graph/domain-transfer setting [1], excluding target entities, incident edges, and alternate target views from the corresponding pretraining pool. The frozen foundation then supports a fresh head trained with target-task supervision. Both tables include GraphMAE2, RiemannGFM, SAMGPT, UniGraph2, and PLANET.

OOD Movies Accuracy is 58.17 for ICE versus PLANET's reported 56.91, and Sports LP MRR is 28.16 versus 27.63. Movies 3/5-shot Accuracy is 58.12/61.09 versus 56.67/59.82; Cloth is 74.38/76.27 versus 73.12/75.36. The foundation claim concerns reuse of one pretrained encoder across graphs and prediction units. These additional settings test that reuse on excluded graphs and domains under supervised and few-shot adaptation.

References [1] S. Liu, X. Li, D. Su, R. Zhang, H. Qin, R.-H. Li, and G. Wang. Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approach. ICML, PMLR 306:77226–77245, 2026.

Dear Reviewer BU44,

With the rebuttal deadline approaching, we wanted to ask whether you have had a chance to read our response.

Could you let us know if the parameter-matched comparisons, learned-gating control, and transfer results address your concerns?

We are happy to provide any further clarification. Thank you for your time.

Reviewer_9z6fOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ICE, a multimodal graph foundation encoder that represents node states as a retained Cl(3) field across algebraic grades and two message-passing depths. It uses geometric-product-based transport and task-specific node and link heads, and evaluates shared pretraining with supervised and few-shot node classification and link prediction across nine target graphs.

Critical strengths
  1. The paper provides a coherent representation/interface design for serving node and pair prediction from one encoder.

  2. The empirical analysis is broader than a single benchmark table. The component drops are consistent with the claim that the full architecture uses more than a final node representation.

  3. The paper is well-written and easy to follow.

Critical weaknesses
  1. Equation (6) defines a symmetric pair descriptor, whereas Equation (14) introduces additional Clifford-relation features. The paper does not clarify whether the latter extends the former or which implementation produces Table 2. The authors should provide a consistent LP definition and clarify the experimental implementation.

  2. Equation (2) uses QiKjQ_i K_j for edge-wise interactions, while Equation (18) introduces QiK~jQ_i \widetilde{K}_j in the equivariance proof. Although permutation equivariance may hold for either formulation, the authors should align the proof with the actual implementation.

  3. Table 5 shows that all nine evaluation graphs were already included in foundation pretraining, albeit without labels. Therefore, the experiments demonstrate adaptation on previously seen graphs rather than generalization to unseen graphs.

  4. Algorithm 1 permits updating encoder parameters during supervised adaptation, whereas the main text emphasizes reusing a shared encoder with fresh task heads. The authors should clarify which parameters are frozen or updated in each setting to distinguish representation transfer from task-specific fine-tuning.

  5. Figure 1(b) and Table 4 demonstrate the benefits of retaining intermediate Clifford states within ICE, but do not isolate these gains from increased capacity or richer readouts.

Questions for authors
  1. Could the authors elaborate on the relationship between the LP formulations in Equations (6) and (14), particularly regarding their experimental implementation?

  2. Is there a specific reason for the different formulations of the edge-wise interaction in Equations (2) and (18)?

  3. How do the authors view the distinction between adaptation to previously seen graphs and generalization to unseen graphs in the context of ICE?

  4. Could the authors elaborate on the role of encoder adaptation during downstream training, particularly its impact on the learned foundation representations?

  5. To what extent do the authors attribute ICE's improvements to Clifford algebra itself, as opposed to richer representations and access to intermediate states?

3 comments

We thank Reviewer 9z6f for the careful reading and for recognizing the coherence of ICE's representation and task interface. We clarify the formulations, update policy, and transfer setting below.

W1 & Q1: One complete LP definition. The submitted main text and appendix did not consistently distinguish the endpoint descriptor from the complete pair input. We have unified them in Section 4.3, Eq. (6), and Appendix D.4, Eq. (14). Shared bias-free projections of all grades/depths form the endpoint sum, absolute difference, and product pijp_{ij}. The supervised LP head reads rij=pij∥ℓ[⟨HiℓH~jℓ⟩0,∥⟨HiℓH~jℓ⟩2∥2]r_{ij}=p_{ij}\Vert_{\ell}[\langle H_i^\ell\widetilde H_j^\ell\rangle_0,\|\langle H_i^\ell\widetilde H_j^\ell\rangle_2\|_2] through one MLP. Relation norms reduce blades within each latent channel, giving 3,840 input coordinates. Exchanging endpoints preserves the scalar and negates the bivector, whose norm is unchanged. The complete descriptor is therefore symmetric. Table 8 separately specifies the support-based prototype head for few-shot links.

W2 & Q2: The proof now follows the stated transport. The two displayed edge formulations were inconsistent. Section 4.2 and Appendix E.3 now both use Eij=QiK~j/hE_{ij}=Q_i\widetilde K_j/\sqrt h, with the same stabilized per-channel norm in attention and message normalization. Under node relabeling, Q/K/V, neighborhoods, topology inputs, and sampling/masks are relabeled together. Corresponding edge products and neighborhood sums are then equal. The proof covers this coupled randomness; independent random draws give equivariance in distribution. We also clarify that grade identities describe possible product channels, not an H1/H2 schedule. Multiplication by V can produce Grade 3 within the first layer.

W3 & Q3: Seen-graph reuse and unseen-graph transfer are distinct settings. The original nine-graph suite evaluates new task heads on graphs included in self-supervised pretraining (Section 5.1; Appendix A.1). Section 5.3 and Appendix F.8 add the graph/domain-transfer setting following PLANET [1], excluding target entities, incident edges, and alternate target views from the corresponding pretraining pool. A frozen encoder supports a fresh supervised or few-shot head. Tables 4 and 14 include GraphMAE2, RiemannGFM, SAMGPT, UniGraph2, and PLANET.

On OOD Movies, ICE reaches 58.17 NC Accuracy versus PLANET's reported 56.91; Sports LP MRR is 28.16 versus 27.63. Few-shot Movies 3/5-shot Accuracy is 58.12/61.09 versus 56.67/59.82, and Cloth is 74.38/76.27 versus 73.12/75.36. These settings test supervised and few-shot transfer beyond pretraining exposure. Appendix F.8 defines the exclusions and target-task protocol.

W4 & Q4: The primary setting freezes the foundation representation. Adapters, transport layers, and AHA remain frozen while fresh task heads are fitted. Section 4.3, Table 8, and Algorithm 1 now agree on this boundary. For identical deterministic inputs, the retained field is unchanged. Appendix F.4 distinguishes this from updating a task-local encoder copy in the adaptation comparison. At exposure 0.5, mean LP MRR changes from 26.24 to 26.63 for ICE and 25.27 to 25.74 for PLANET (Table 10). Thus ICE's advantage is present before encoder adaptation. The NC bound concerns the residual score at a given encoder state, rather than representation movement during fine-tuning.

W5 & Q5: Edge construction and retained access each contribute. Same-bank controls retain Q/K/V, dimensions, depths, bank, and heads. LP MRR is 26.24 for ICE, 25.16 for scalar attention, and 25.39 for Hadamard transport (Table 9). These replacements test the edge operation within the same representation interface. PLANET-PM provides a separate size comparison at 59.52 M active encoder parameters versus ICE's 59.51 M, reaching 25.48 MRR. In Table 3, removing the bank reduces MRR by 0.98 points, while removing the Grade-1 route reduces NC Accuracy/Macro-F1 by 1.12/1.40 points. Together, these comparisons distinguish geometric transport from the benefit of retaining states for task-specific access.

References [1] S. Liu, X. Li, D. Su, R. Zhang, H. Qin, R.-H. Li, and G. Wang. Toward Effective Multimodal Graph Foundation Model: A Divide-and-Conquer Based Approach. ICML, PMLR 306:77226–77245, 2026.

Dear Reviewer 9z6f,

The rebuttal period will end soon, so we would appreciate your feedback on our response.

We hope the clarified LP formulation, revised proof, and transfer evaluation have addressed your concerns. Please let us know what you think.

If you have any further questions, we would be happy to discuss them.

Thank you to the authors for their detailed response. Most of my concerns have been adequately addressed.

However, given the paper's emphasis on graph foundation models, I believe the evaluation of generalization and transferability deserves greater attention. A substantial portion of the current experimental analysis focuses on downstream adaptation to graphs already encountered during pretraining. While these results demonstrate the effectiveness of the learned representations, they primarily reflect adaptation within previously observed graphs rather than generalization to genuinely unseen graphs.

I appreciate the newly added unseen-graph transfer experiments, which are a valuable step in this direction. Nevertheless, I feel that this evaluation remains somewhat limited in scope. A more comprehensive investigation of cross-domain transfer to unseen graphs, ideally across a broader range of benchmarks, would provide stronger evidence for the model's generalization capabilities.

Additionally, zero-shot transfer evaluation would be particularly valuable, as it could help distinguish the intrinsic generalization ability of the pretrained foundation model from improvements achieved through downstream task-specific adaptation.

Given that the above concern is not addressed, I keep my original rating.

Reviewer_oaYvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

this work proposes ICE, a shared multimodal graph encoder whose node representations are Clifford multivectors. Topology, text, and image features enter designated coordinates; two graph transport layers construct interactions; and the encoder retains all grades and depths for downstream prediction. Node classification uses a Grade-1 prototype score with a bounded residual, while link prediction uses a symmetric pair head. The authors pretrained the model on 11 graphs and evaluated it on both supervised and few-shot node and link tasks.

Critical strengths
  1. the proposed method keeps all grades and depths and lets the node and link heads pick out the information they need from the same representation. This is a useful way to share an encoder across tasks.

  2. The reported results show consistent improvement: ICE has the highest mean in all 30 comparisons, and results show performance drops when key components are removed.

Critical weaknesses
  1. eq 4 uses higher-grade targets that eq 1 sets to zero, which doesn't match the text about reconstructing generated relations. It's also unclear which model was evaluated given the head descriptions and encoder updates.

  2. It's hard to tell what drives the gains because the paper doesn't spell out the ablation replacements or how capacity is controlled (Table 4). The comparison with PLANET also needs clarification: the Books-LP pretraining weight is 0.1 for ICE and 0.5 for PLANET, and it's unclear whether both use the same adaptation setup.

Questions for authors
  1. What does eq 4 reconstruct in Grades 2 and 3 if those parts of H0 are zero?

  2. Which node and link heads were used, and which encoder parameters were updated? The descriptions in eqs. 5-6, eq 14, and Algorithm 1 don't seem to line up.

  3. What were the product and task-access head replaced with in Table 4, and was the capacity kept the same? How should we interpret the query weight on zero H0 grades in Table 16?

  4. All 15 rows in Tables 12-14 follow an increasing or decreasing order if the starting point is shifted. I'd appreciate more detail on how these per-seed results were obtained

2 comments

We thank Reviewer oaYv for recognizing the value of sharing a retained field across node and link tasks. We address the target definition, task interfaces, controlled comparisons, and seed presentation below.

W1 & Q1: Separating the reconstruction target from the encoder input. The submitted Eq. (4) used H0H^0, whose higher-grade entries are zero, and therefore did not express the relation-reconstruction target described in the text. We now define the target separately in Section 4.2, Eq. (4), and Appendix D.3, Eq. (12). Its detached Grade-2 coordinate is T2=κ(t^⊙v^)e12T_2=\kappa(\hat t\odot\hat v)e_{12}, with κ=tanh⁡(0.1)\kappa=\tanh(0.1). For two modalities, T3=0T_3=0 and its direct loss weight is zero. Grade 3 remains available to transport and task heads. The decoder reads the AHA aggregate of the masked forward pass. Appendix D.3 specifies the target branch, pooling, contrast candidates, masked losses, and empty supports.

W1 & Q2: Consistent task heads and update rules. The revised NC definition combines the pooled Grade-1 prototype score with 15 residual sources. With ucs=tanh⁡acsu_{cs}=\tanh a_{cs} and γcs=ucs/(1+∑r∣ucr∣)\gamma_{cs}=u_{cs}/(1+\sum_r|u_{cr}|), the total absolute weight is bounded by ρ=15/16\rho=15/16. The bound applies before logit scaling (Section 4.3, Eq. (5); Appendix D.4, Eq. (13)).

For supervised LP, one MLP reads endpoint sum/absolute-difference/product features from all grades and depths, together with per-depth scalar and bivector-norm relations. Equations (6)/(14) now define this same complete descriptor. Few-shot tasks use support-based node or pair prototypes. Section 4.3, Appendix D.4, Table 8, and Algorithm 1 distinguish these interfaces. Primary jobs freeze the adapters, transport layers, and AHA; a fresh head is fitted for each task. Encoder adaptation is evaluated separately on a task-local copy.

W2 & Q3(a): Separating transport, access, and model size. Section 5.2 and Appendix F.2, Table 9, report the matched controls:

MethodEncoder (M)NC Acc.NC F1LP MRR
PLANET-PM59.5282.4774.1125.48
Scalar edge, same bank59.5182.3273.9825.16
Hadamard edge, same bank59.5182.5874.2325.39
ICE59.5183.2074.9026.24

Scalar attention uses dot-product weights and weighted V; Hadamard transport uses componentwise edge products. Both retain Q/K/V, width, depth, bank, and the same heads. Generic NC uses pooled multivector prototype cosine. Generic LP uses the frozen 256-dimensional AHA embedding and a 1,788-unit MLP, allocating 1,376,761 head parameters versus 1,376,769 for the full head (Appendix F.2). This tests access under a similar allocated budget. The scalar/Hadamard comparisons additionally keep the head itself unchanged.

The original ICE exposure is 0.1 for Books-LP. Appendix F.3, Table 10, compares both methods at 0.5. Mean LP MRR is ICE/PLANET 26.24/25.27 with head-only fitting and 26.63/25.74 with encoder adaptation. ICE's MRR changes from 26.17 to 26.24 when exposure changes from 0.1 to 0.5. The advantage thus persists under matched exposure and both update policies.

Q3(b): Zero inputs have zero input-scaled sensitivity. We replaced the ambiguous query-weight display with ∥x⊙∇xs∥1\|x\odot\nabla_x s\|_1, normalized across raw bank addresses. H0 Grade-2/3 contributions are exactly zero. Pair relations computed from H0 vectors are separate features. Appendix F.7 and Table 13 define this diagnostic, with the pooled NC branch held constant.

Q4: Origin and interpretation of the seed tables. Thank you for pointing out the ordering pattern in the submitted Tables 12–14. Appendix H, Tables 19–21 (submitted Tables 12–14), lists the eight seeds in fixed order (42–49), using the mean/sample-SD rule in Appendix B.2, Eq. (11). Recomputing the means and sample standard deviations from the displayed values reproduces all 15 reported summaries. None of the displayed rows is monotone under a cyclic shift. The arithmetic check alone cannot establish which evaluation run produced each value.

Dear Reviewer oaYv,

As the rebuttal period is coming to an end, we wanted to follow up on our response.

Could you let us know whether the corrected target definition, clarified task heads, and additional controls address your concerns?

We would be happy to answer any further questions. Thank you for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.