acceptodds
Under review as a conference paper at ICLR 2027

Multimodal Domain-Adaptive Language Models for Materials Discovery in Metal–Organic Frameworks

Abstract

Accurate and data-efficient prediction of metal–organic framework (MOF) properties requires representations that capture both chemical and structural information. We propose a two-stage framework that continually pretrains the Granite language model on PubChem records and MOF literature, and then fuses its frozen MOFid representations with 18 interpretable CIF-derived descriptors through cross-attention. Continued pretraining alone improves QMOF band-gap and HMOF gas-adsorption prediction, and structural grounding adds further gains. Controlled ablations show that combining pretrained MOFid language representations with structural descriptors outperforms using either the pretrained MOFid representations or the structural descriptors alone, while cross-attention generally performs better than concatenation under matched conditions. Representation strengths are property-specific: structurally pretrained models perform best on QMOF and CH, whereas our multimodal framework achieves the best results on all three CO targets. Bond Connectivity descriptors are most effective for QMOF, and Pore Structure descriptors are effective to HMOF. The two modalities remain complementary in low-data settings, demonstrating that compact structural grounding can effectively enhance MOFid-based language representations.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
MOF, CO2, CH4, Fusion, LLM, multimodal, Sustainability, descriptors, Metal-organic frameworks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_A1QxOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Prediction of MOF properties is critical for material science. This study proposes a two-stage framework that continually pretrains the Granite language model on PubChem records and MOF literature, and then fuses its frozen MOFid representations with 18 interpretable CIF-derived descriptors through cross-attention. The extensive experiments demonstrate the effectivenss of this multimodal framework. However, the reviewer is concerned about the novelty, missing baselines, and incomplete ablations.

Critical strengths
  1. The authors selected an interesting and important problem of materials, the CO2 adsorption prediction. The proposed model achieves the lowest reported MAE across all three CO₂ adsorption targets among the evaluated baselines, demonstrating useful predictive gains over strong structure-pretrained models.

  2. The model is pretty strong under limited supervision. With 500 and 2,000 labeled examples, the domain-adapted multimodal model outperforms PMTransformer on QMOF band-gap prediction and all CO₂ adsorption targets, suggesting its value when labeled data are scarce. However, with full supervision, some baselines like PMTransformer are the best, such as for CH4.

  3. I found that the authors claim no use of AI assistant tools for writing, etc. This is very rare in today's research community.

Critical weaknesses
  1. The novelty is kind of overlapped with some prior studies like L²M³OF and is incremental from my perspective. Continual pretraining, handcrafted features, and cross-attention are all mature techniques. The main contribution is the combination of those existing parts and ablations without any new learning algorithms or architectures.

  2. The experiments are still inadequate for this empirical-oriented paper. For example, the fusion ablation does not fully isolate the effect of cross-attention. A stronger control would use learned token pooling followed by concatenation, with comparable model capacity and training settings. Besides, the generalization and pretraining analysis remain incomplete. The paper does not evaluate generalization to unseen linkers or topologies. Moreover, the overlap analysis covers only exact SMILES matches with PubChem, leaving potential exposure through the MOF literature corpus unexamined. Finally, Section 4.3 compares an unadapted LLM-only model with a domain-adapted multimodal model, confounding structural fusion with continued pretraining. A comparison using the same domain-adapted backbone is needed to isolate the contribution of each modality.

  3. I am also curious about the comparison with frontier LLMs, which is very helpful for contextualizing the benefits of domain adaptation. Where feasible, please evaluate a representative frontier LLM using MOFid and the same structural descriptors under clearly specified zero-shot or few-shot prompting. Such results should be reported separately from supervised baselines, given the differences in training data access and evaluation protocols.

P.S. Besides, there are several paper writing issues that may need further improvements. For instance, Figure 1 is relatively small, and it is difficult for readers to clearly see the letters inside each box. The authors can crop the empty space and resize the image. For tables (Table1 & 2), it is better to bold the best results and underline the second so that we can immediately compare the performance of the proposed methods with others. There are so many numbers, and it is costly for me to calculate the gap between MOFid and other baselines.

Questions for authors

The paper omits the closely related L²M³OF (arXiv:2510.20976), which integrates crystal representations with an LLM for MOF property prediction and other tasks. Please clarify the methodological differences and, where feasible, provide a comparison under matched evaluation settings.

L^2M^3OF: A Large Language Multimodal Model for Metal-Organic Frameworks 2025.

Reviewer_Tx8wOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a two-stage framework for MOF property prediction. It first continues pretraining the Granite language model on PubChem records and MOF-related literature to obtain domain-adapted MOFid representations. The adapted language-model backbone is then frozen and combined with 18 CIF-derived structural descriptors using descriptor-guided cross-attention, followed by a regression head for predicting QMOF band gaps and HMOF gas-adsorption properties.

Critical strengths
  • The paper includes useful ablation studies that isolate the contribution of the main components. It compares domain-adapted versus non-adapted Granite under the same multimodal setup, language-only versus multimodal prediction, different CIF descriptor families, and cross-attention versus concatenation, providing evidence that the MOFid and structural-descriptor modalities are complementary.

  • The experimental setup is generally solid. The baselines are diverse and representative, spanning descriptor-based, atomic-environment, graph-based, MOFid-based, and structure-pretrained models, with controlled dataset splits, three-seed evaluation, and both full-data and low-data settings.

Critical weaknesses
  • The framework is technically reasonable, but the method novelty is modest. The main contribution lies in combining MOFid representations with lightweight CIF-derived descriptors and systematically evaluating their complementarity.
  • The empirical evaluation is limited to only two task families, QMOF band-gap prediction and HMOF gas-adsorption prediction, which limits how strongly the results support broader claims about materials discovery.
  • The model achieves strong and competitive performance, particularly for HMOF CO2 adsorption and in low-data settings. However, the gains are not universal: structure-pretrained baselines remain stronger on QMOF and CH4 prediction.
  • No code repository for the proposed method is provided or linked in the manuscript.

Minors

  • The bibliography contains multiple duplicate entries for exactly the same papers: Cao et al. (2023a) and Cao et al. (2023b); Moosavi et al. (2020a) and Moosavi et al. (2020b); Park et al. (2023a) and Park et al. (2023b); Rosen et al. (2021a) and Rosen et al. (2021b); and Xie & Grossman (2018a) and Xie & Grossman (2018b). Each pair points to the same paper.

  • The reported results contain several internal inconsistencies that should be clarified:

    1. MOFTransformer QMOF inconsistency: the QMOF MAE is reported as 0.3684 in Table 1 but 0.3905 ± 0.0090 in Appendix Table 10.
    2. Phi-4 Mini HMOF Mean inconsistency: Table 1 reports 0.2338, while the surrounding text reports 0.2325.
    3. Large-model QMOF inconsistency: Table 9 reports 0.288, while the text reports 0.289.
    4. Table cross-reference inconsistency: the low-data results are referred to as Table 12 in the main Results section even though the main-text version is Table 3.
  • The title appears broader than the actual scope of the paper. The work is not really about materials discovery in general; it focuses much more specifically on MOF property prediction using continued pretraining plus CIF-derived structural descriptors.

  • The term “domain-adaptive” is somewhat overstated: the adaptation occurs only during continued pretraining on chemistry/MOF corpora, while the downstream multimodal model itself is not adaptive. A more precise framing would be a MOF property-prediction model using domain-adapted language representations.

  • Figure 1 is difficult to read because the text and annotations are too small; the figure should be enlarged or simplified for readability.

Questions for authors

Why are some baselines evaluated using previously reported numbers while others are re-implemented? Please clarify whether preprocessing, splits, and evaluation settings are fully comparable across these baselines.

Reviewer_yvurOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a two stage framework to predict MOF properties by using MOFids. The authors continue pretraining of the Granite 3.3-2B LLM on PubChem data and MOF-related articles and abstracts. Then they freeze the backbone and fuse MOFids tokens with 18 CIF derived descriptors. The model is evaluated on the QMOF band gap and six HMOF gas-adsorption targets (CH$_4$ and CO$_2$).

Critical strengths
  • Results on low data regime. The proposed model outperforms PMTransformer when using 500 and 2000 labeled examples, showing that the transfer of domain-adapted MOFid representation is effective.

  • Descriptor ablations. Bond Connectivity is useful for band gap prediction and Pore Structure descriptors for adsorption.

Critical weaknesses
  • LLM role and practical motivation. In order to avoid full atomistic representations, the authors propose the use of MOFid. However, 18 descriptors are computed from CIF with Crystal1NN, and MOFids are generated using CIFs. This means that at inference time the model needs the same input as the graph model, plus a forward pass on the LLM model. Also, using MOFids restricts the method to using only 7,466 MOF structures out of 20,375.

  • The performance claim is not supported. Although the HMOF mean of the proposed work is slightly lower than that of PMTransformer, it is an unweighted average of the MAEs of targets, which are at different scales. (At higher pressure, more gas is adsorbed, so the values and errors will be bigger). The model is clearly better on high pressure CO2CO_2 compared to PMTransformer and worse on all CH4CH_4 targets.

Questions for authors
  1. Why did you choose Granite as LLM?
  2. What are the results for CGCNN and MOFormer on the low-data regime?
  3. What are PMTransformer's and MOFTransformer's HMOF results with per-target heads, as in the original papers?
  4. Is the descriptor query a single 1024-dim vector or 18 tokens, as Figure 1 suggests? If 18, how are they pooled into z?
  5. How do the six fusion layers stack?
  6. What happens if there are no CIF available?
  7. What is the LLM contribution when frozen Granite 3.3-2B performs worse than MOFormer trained from scratch? What are the results if you replace the LLM with a small MOFid encoder, for example, MOFormer?
  8. Why did the authors not share the code?

Minor issues

  • Which result is the correct one for the model Phi-4Mini4B HMOF (0.2325) line 401 or the result in Table 1 (0.2338)?
  • Line 400 calls PMTransformer “78M parameter”; Table 9 lists 85M for it
  • Sections 3.5 and 4.5 cite “Table 12” for the low-data results; the main-text table is Table 3
  • There are no "bold and underlined values indicate the best and second-best means, respectively" in any of the tables in the main paper, regardless of the caption stating it
  • Figure 1 is unreadable at print size, and adding more details for the fusion part will make it more interpretable
  • Several references are duplicated as “a/b” entries ((Cao et al., 2023a) line 60, (Cao et al., 2023b) line 152)

Comments on the paper

1 comment

Dear reviewers,

As the authors have posted their responses, I would encourage you to actively parcitipate in the dicussion.

Thank you.

Best,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.