acceptodds
Withdrawn from ICLR 2027

CoopHyperMoE: Membership Interpolation over an Expert Hypergraph for Molecular Graph Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Graph mixture-of-experts (MoE) models adapt message passing to heterogeneous graph inputs, yet a top-K router must select experts even when every routing score is weak. We introduce CoopHyperMoE, a membership-based router that keeps node–expert affinities independent before selection. It periodically builds a hypergraph from similarities between routing vectors and, for low-membership inputs, interpolates non-anchor memberships from principal-component anchors in the active hyperedges. A normalized top-K gate is then computed from the revised memberships. We evaluate CoopHyperMoE with four fixed seeds and validation-selected checkpoints on ten molecular benchmarks: seven classification datasets and three regression datasets. Without a virtual node, CoopHyperMoE records a higher mean ROC-AUC than a locally reproduced GMoE on five of seven classification tasks and raises the seven-task average from 71.90 to 72.04; it also obtains the lowest local RMSE on Lipophilicity. Matched ablations on six classification datasets test interpolation, annealing, and the activation rule. A final-checkpoint audit finds no interpolation triggers on the BACE and Tox21 validation splits, limiting what these comparisons establish about the interpolation mechanism.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
graph neural networks, mixture of experts, molecular graph prediction, expert routing, hypergraphs, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_B4p8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CoopHyperMoE for molecular graph property prediction. The model computes independent sigmoid memberships for node–expert pairs and constructs an expert hypergraph from similarities between router vectors. For nodes satisfying a low-membership condition, it adjusts these memberships through Gaussian-weighted interpolation using PCA-selected reference experts, before applying normalized top-K routing. The interpolation strength decreases with network depth and training progress. The evaluation covers ten molecular datasets with four random seeds and includes matched ablations on six classification datasets. The results show modest, task-dependent improvements, while the ablations leave the contribution of the interpolation mechanism unclear.

Critical strengths
  1. The paper investigates whether relationships between experts can improve membership estimates before expert selection. Separating initial memberships, relational adjustment, and final routing makes the proposed intervention easy to identify and evaluate. This is a meaningful direction for studying how expert relationships might inform routing in graph MoE models.
Critical weaknesses
  1. The experiments do not establish the benefit of the central interpolation mechanism. In Table 3, the full model does not achieve the highest mean score on any of the six datasets, and disabling interpolation improves the mean on four. For example, removing interpolation increases ROC-AUC from 77.63 to 78.63 on BACE and from 67.08 to 68.69 on BBBP. These comparisons leave the added value of the paper’s main new component unclear.
  2. The specific relational construction lacks sufficient justification to support its added complexity. With sigmoid gating already studied in prior work, much of the methodological contribution rests on the expert hypergraph and interpolation rule. However, the paper does not establish why similarities between router vectors and PCA-selected anchors provide useful information for correcting memberships. The current results also offer limited practical motivation for this construction: predictive gains are small and inconsistent, while the reported ClinTox comparison shows approximately 44% higher training time than GMoE. A clearer empirical advantage or a transferable insight into when this construction helps would strengthen the contribution substantially.
Questions for authors
  1. On one dataset, how often does interpolation change expert selection or weights, and does it improve performance over disabling it? Please examine both the training and the best validation checkpoint.
  2. Besides gating and interpolation, do local GMoE and CoopHyperMoE differ in routing noise or regularization? Why do Tables 1 and 3 report different results for the full model?
Reviewer_c1cvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces CoopHyperMoE, a graph Mixture-of-Experts architecture for molecular property prediction. It proposes independent sigmoid-based expert memberships and constructs a hypergraph based on similarities between expert routing vectors. For low-membership inputs, the method interpolates expert memberships using PCA-selected anchors before top-K routing.

Critical strengths
  1. The paper addresses a relevant limitation of traditional top-K routing, where experts must be selected even when all routing scores are weak.

  2. The proposed approach is reasonably intuitive, combining expert similarity, hypergraph construction, and membership interpolation.

  3. The paper evaluates multiple molecular benchmarks and includes ablations and routing diagnostics that help assess the proposed mechanism.

Critical weaknesses
  1. The ablations do not demonstrate a consistent benefit from hypergraph-based interpolation. In fact, simplified variants outperform the full model on all six ablated datasets.

  2. The average ROC-AUC improvement over GMoE is only 0.14 percentage points, and the proposed model does not consistently outperform simpler baselines.

  3. The authors report that interpolation is never activated at the final checkpoints on two evaluated validation splits, raising questions about whether the proposed mechanism contributes to the observed improvements.

  4. The hypergraph is constructed using similarities between routing parameters, without demonstrating that these similarities correspond to meaningful functional relationships or cooperation between experts.

Overal, the experimental evidence does not convincingly establish the effectiveness of the paper's central contribution, making the overall contribution relatively weak.

Questions for authors
  1. Why "full model" numbers in Table 3 differ from the "Ours" row of Table 1?
Reviewer_bb8pOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoopHyperMoE, a routing framework for molecular graph prediction. The method first replaces softmax routing scores with independent sigmoid memberships so that the router can preserve cases in which all experts receive low scores. It then constructs a periodically refreshed hypergraph from similarities between expert routing vectors. For nodes with low memberships, the method uses principal component anchors and Gaussian interpolation to revise the memberships before applying normalized top K gating. The model is evaluated on seven molecular classification datasets and three regression datasets using four fixed seeds, with additional ablations and routing diagnostics.

Critical strengths

The empirical study is carefully documented and unusually transparent about the limits of the observed results. The paper reports four seed results, validation based checkpoint selection, per seed performance, matched ablations, computational costs, and routing diagnostics. In particular, the authors report that the full model is not the strongest variant on the six ablation datasets and that no interpolation is triggered at the final checkpoints on BACE and Tox21. This level of transparency makes it easier to assess which conclusions are supported by the experiments.

The paper provides a clear motivation for separating independent node to expert memberships from the final normalized top K gate. Independent sigmoid memberships preserve whether all experts receive weak scores, which is information that softmax normalization removes. The proposed routing pipeline is also described in considerable detail, including the construction and refresh of the expert hypergraph, anchor selection, membership interpolation, activation conditions, and final gating procedure.

Critical weaknesses

The experiments do not validate the central hypergraph interpolation mechanism. The full model achieves the highest mean performance on none of the six ablation datasets, and removing interpolation leads to better results on several tasks. The final checkpoint audit also finds zero interpolation triggers on the BACE and Tox21 validation sets. The matched ablations therefore suggest that hypergraph interpolation is not responsible for the reported gains over GMoE. Since removing interpolation often improves performance, the gains are more likely to come from other changes in the routing formulation, such as the use of sigmoid memberships. The experiments provide little evidence that the central interpolation mechanism is beneficial.

The overall benefit of the proposed design is limited relative to its added complexity and computational cost. On the seven classification datasets, CoopHyperMoE improves the average ROC AUC over GMoE only from 71.90 to 72.04 and remains slightly below the plain GCN average of 72.19. On the three regression datasets, it achieves the best mean result only on Lipophilicity, with no consistent advantage across tasks. It also requires about 44 percent more training time than GMoE because all experts are still evaluated. These results make it difficult to justify the additional hypergraph construction, anchor selection, and interpolation procedure.

Questions for authors

Do the training logs show how frequently interpolation is triggered during optimization and whether these triggers are associated with any performance changes? Given the absence of triggers at the final checkpoints on BACE and Tox21, what evidence supports the contribution of interpolation to the final models?

What advantage does the full hypergraph interpolation procedure provide over the simpler sigmoid membership router? The no interpolation variant often performs as well as or better than the full model, while the full method introduces additional computation and longer training time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.