acceptodds
Under review as a conference paper at ICLR 2027

CMKL: Multimodal Continual Learning for Evolving Biomedical Knowledge Graphs

Abstract

Biomedical knowledge graphs (KGs) are increasingly large, dynamic, and multimodal, driven by rapid advances in biotechnology such as high-throughput sequencing. Machine learning models can infer previously unobserved biomedical relationships and characterize biomedical entities in these graphs, but existing knowledge graph embedding (KGE) methods and their continual learning (CL) extensions either assume static graph structure or fail to exploit multimodal information under evolving data distributions. They also apply uniform regularization across all model parameters, ignoring that different modalities may exhibit distinct forgetting dynamics as the graph evolves. We propose the Continual Multimodal Knowledge Graph Learner (CMKL), a CL framework for biomedical KGs that natively encodes structure (R-GCN), text (frozen BiomedBERT), and molecules (Morgan fingerprints), fuses them through a Mixture-of-Experts (MoE) router, and protects previously learned knowledge with standard EWC regularization and a K-means-diverse multimodal replay buffer. We evaluate CMKL on a 129K-entity biomedical continual benchmark. On continual biomedical entity classification, CMKL reaches Macro-F1 and beats feature-matched continual baselines () by ()—an architectural gain isolated from feature availability—preserved across the sequence with near-zero forgetting (AF ). On continual relationship prediction (real evolution tasks), CMKL reaches AP , on par with Naive Sequential and EWC () and above Joint Training (); a residual-skip fusion variant reaches , recovering the structural-only performance () that embedding fusion otherwise sacrifices—fusion that does no harm. Finally, we correct a natural misreading of these graphs: a text-only ablation's high AP () is fully reproduced by a zero-text transductive node-embedding lookup, so the apparent frozen-text “ceiling” is a node-embedding artifact—frozen semantic modalities carry strong classification but negligible link-prediction signal, which the MoE router reflects by down-weighting text on link prediction. Our code is available at https://anonymous.4open.science/r/cmkl-multimodal-ckg-FF56.

Primary area:
learning on time series and dynamical systems
Keywords:
Continual Learning, Biomedical Knowledge Graphs

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_h3hNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CMKL for continual learning on evolving biomedical knowledge graphs. It combines structural, textual, and molecular representations through entity-level MoE routing, together with EWC and a diverse replay buffer. Evaluation uses a PrimeKG-derived benchmark with 129K entities and updates from two database releases. CMKL achieves substantial entity-classification gains over feature-matched baselines, while link-prediction improvements are limited. The paper also reports that the strong text-only link-prediction result comes from transductive node embeddings rather than text semantics.

Critical strengths
  1. The results seem good compared to the baselines.
  2. The analysis of the text-only result usefully distinguishes semantic information from node-embedding effects.
Critical weaknesses
  1. The main aligned comparisons are Naive Sequential, EWC, and Joint Training. Relevant methods such as IncDE and MRCKG are not evaluated, while LKGE uses a different protocol. Link-prediction gains over Naive/EWC are not statistically significant, and EWC+RotatE performs better. The evidence for an advance over existing continual KG methods is therefore limited.

  2. The classification head, node splits, task construction, and graph visibility are underspecified. Fusion alternatives are compared only on link prediction, and forgetting metrics are missing for the feature-matched classification baselines.

Questions for authors
  1. Please specify the classification head and loss, node splits, task sequence, and graph used for message passing.

  2. Can LKGE be evaluated under the same task protocol, and can IncDE or MRCKG be included? Please also compare K-means replay with random replay and no replay to establish the contribution of the proposed memory selection.

Reviewer_EjmHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to update multimodal biomedical knowledge graph representations as new relationships arrive while retaining previously learned information.

CMKL combines an R-GCN, frozen BiomedBERT features, and molecular fingerprints through entity-level mixture-of-experts fusion, together with EWC and a diverse replay buffer.

The strongest reported result is a classification Macro-F1 of 0.591 versus 0.414 for feature-matched baselines, whereas link prediction is statistically comparable to the same-decoder sequential baselines and a residual variant approximately recovers structural-only performance.

Critical strengths
  1. The biomedical setting is worth studying. The benchmark contains over 129K entities, and the six-task evaluation is based on a real database update. Combining graph structure with unevenly available text and molecular features addresses a practical problem in these data.

  2. Table 2 reports a sizeable classification gain, from 0.414 to 0.591 Macro-F1, across five seeds. The baselines receive the same multimodal features. The authors also acknowledge that CMKL does not significantly outperform Naive Sequential and EWC on link prediction.

Critical weaknesses
  1. I am not convinced by the methodological novelty. EWC and replay are established approaches to reducing forgetting, and Eq. (6) simply adds their losses to the task loss. The authors acknowledge that per-encoder EWC is a configuration choice. R-GCN and MoE are also familiar components. The paper does not show what is technically new in this combination or what benefit depends on their interaction. Its main contribution appears to be applying these components to biomedical graphs.

  2. The comparisons are mostly with basic baselines or CMKL variants. Naive Sequential, EWC, and Joint Training are useful references, but LKGE uses a different task sequence and decoder. IncDE and MRCKG are discussed without comparable results. For classification, a simple multimodal model with standard EWC and replay is also missing. These comparisons are needed to establish whether CMKL improves on existing approaches.

  3. The ablations do not establish the added value of the combination. Table S1 has no zero-buffer or random-replay control and uses a different task set from the main link-prediction results. Table 2 omits forgetting scores for the classification baselines, while Table 3 tests fusion only on link prediction. It remains unclear how much the gains depend on ordinary replay, multimodal features, or the specific MoE and K-means choices.

Questions for authors
  1. What technical design in CMKL goes beyond a straightforward combination of the existing components? Which controlled experiment isolates its benefit?

  2. How does CMKL compare with IncDE or MRCKG under the same task sequence and evaluation? For classification, can you add a simple multimodal EWC-plus-replay baseline? Please match features, replay capacity, and training budgets, or explain why this is not feasible.

  3. With architecture and task sequence fixed, how do EWC alone, replay alone, both, and neither compare? Please also test random versus K-means replay and MoE versus concatenation on classification, reporting final performance and forgetting.

Reviewer_j2Q3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CMKL proposes to solve the problem of continual learning in multimodal biomedical knowledge graphs. Biomedical knowledge graphs evolve often as frequently as daily or weekly even as new information from world events, life science discoveries and research occurs. In such cases, the two options to update KGs are either to retrain or locally finetune that neighborhood which are both suboptimal due to time and quality tradeoffs.

This is where continual learning could be very useful. Moreover, biomedical KGs are optimally impactful when all modalities can be used. This paper tackles both the topics with the following architecture: The authors combine multimodal embeddings via a learned mixture of experts (MoE) router. Here, they also protect the best individual modality (structural) performance by adding a do-no-harm fusion gate on top of the structural modality. This is used along with prior known methods of continual learning i.e. EWC and Multimodal buffer replay with k-means representations per task.

They show an impressive improvement of 43% in entity classification compared to other continual baselines that also got all the features. Additionally, they present a neat analysis to expose how frozen text embeddings are only learning other graph biases and not actually using text semantics in these KGs.

Critical strengths

The paper motivates each configuration and architectural intuition cleanly. The analysis in the experimentation section of the findings is particularly insightful as some of the findings are dissected to understand deeper.

  • For example, the analysis of the text embeddings being misleading in L321+ is neat. The causal decomposition is formulated simply and clarifies how any transductive lookup would achieve the same. The nuance of how it helps in entity classification but not in link prediction makes the MoE's inner functioning more interpretable and lends credibility to their architecture.
  • Similarly, the analysis on why AF is misleading if the total performance itself is low is obvious in hindsight but phrased in a way that makes interpreting results comprehensive.
  • The authors share the honest comparison of MoE and concatenation results as being comparable. The ablations for fusion strategy are also comprehensive beyond these two with the various score level results.

Finally, the results on entity classification are a significant jump over previous baselines by 43% though all comparisons were given the same features. Though link prediction scores don't have a similar jump, this work will be a meaningful set of tricks to pick from in an impactful area of continual learning.

Critical weaknesses

Overall, while the architectural choices are motivated by inspectability and interpretability, some choices are not fully validated. For others in the community who try to replicate this system, this could lead to wasteful effort. The central premise of the proposed MoE architecture is its ability to adaptively assign optimal modality weights. However, the paper lacks a deeper analysis of the MoE routing choices (example: a distriibution of router weights across tasks via a heat map per task). The do-no-harm fusion helps recover from the gap from structural-only but it is important to understand how often are modalities mixed and in which proportion. From S3, it can be inferred that occasionally the MoE router becomes a structural only expert, but it would be useful to measure quantitatively over various runs to truly decide the value of this architecture.

Some discussion on the correlation of task complexity and sequencing in how it impacts modality selection by the MoE, per modality EWC and fusion strategy would also be useful to understand if certain task types warranty this architecture more than others. Figure 2 is a good start, but expanding on this will be useful to understand why structure only is so powerful and is MoE able to pick a fusion optimally. (See question below)

Similarly, modality specific EWC is well motivated but leading to similar results as uniform EWC. While this is discussed in S8, it would be useful to point it out in the main paper so that the recipe to get these results is not assumed to be more complex than is needed.

Questions for authors

Critical: While the paper proposes MoE fusion of modalities, structure only embeddings seem to perform strongly on most tasks (Fig2) and required the do-no-harm fusion as well to preserve MoE performance at the level of structural only performance. However, in Table 2, it is clear from the overall number that structure only does not meet CMKL MoE + do-no-harm fusion. Regarding this:

  1. Is there a subset of tasks or a specialized evaluation setup (even for a case study) where you can show that structural data is absent but MoE can utilize text/mol data well? For example, consider a cold-start problem of adding a new disjoint node (new drug or pathogen) that may not have structural data. This type of an experiment would lend critical credibility to your central contribution of the MoE architecture.
  2. If the gap is this big in Table 2 between CMKL_full and CMKL_struct_only, do you have a hypothesis on why MoE collapses to a single expert for structural sometimes (as mentioned in S3)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.