acceptodds
Withdrawn from ICLR 2027

MOTIF: LLM-Defined Material Taxonomies for OOD Property Prediction via Invariant Feature Selection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Out-of-distribution (OOD) generalization remains a central challenge in materials property prediction, where models must extrapolate to novel chemistries beyond the training distribution. While large language models (LLMs) have demonstrated strong scientific reasoning, their direct use for quantitative prediction remains limited. To address this, we systematically evaluate multiple roles for LLM assistance, finding that only taxonomy-based grouping consistently improves OOD generalization over a strong kNN+GBT baseline. Building on this, we here propose MOTIF, a framework that leverages LLMs not as predictors, but as generators of candidate taxonomies that partition materials into chemically meaningful environments. These taxonomies are evaluated via training-only pseudo-OOD validation, and only those that improve generalization are retained. Invariant feature selection is then applied within each accepted taxonomy, followed by gated aggregation across taxonomies. We evaluate on 2D halide perovskites, where bandgap prediction must generalize across distinct organic spacer families. This poses a realistic OOD challenge due to the strong influence of spacer chemistry on electronic structure. On this benchmark, MOTIF reduces mean OOD MAE from 0.122 to 0.111 eV. Under a frozen dev/test protocol, the deployable configuration further improves errors from 0.123 to 0.117 eV (electronic bandgap) and from 0.181 to 0.173 eV (optical bandgap). Beyond this primary task, we validate transferability across additional materials datasets, including 3D perovskite solar cell efficiencies and Matbench benchmarks with structure-based splits. Across domains, MOTIF consistently improves robustness under distribution shift, outperforming expert-defined groupings, rule-based heuristics, and clustering baselines, while remaining competitive with strong classical selection methods. These results demonstrate that LLMs are most effective not as predictors, but as tools for discovering structure in data. This enables improved generalization in materials science tasks under realistic OOD conditions.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
perovskites, out-of-distribution generalization, materials property prediction, large language models, invariant learning, environment discovery, chemical taxonomy, invariant feature selection, materials informatics, distribution shift, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zzGKOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes MOTIF, which uses LLM to group materials based on their chemical properties. It then selects stable features across different groups to improve OOD prediction.

Critical strengths
  1. do not use LLM directly predict property;
  2. The propose method seems has great interpretability.
Critical weaknesses

see my question

Questions for authors
  1. The descriptions of the tables are unclear, making them difficult to understand.
  2. Table 1 only includes a few baselines. I think these comparisons are not sufficient to demonstrate the effectiveness of the proposed method.
  3. I am not sure what the tables are intended to demonstrate. Also, where are the results for the 19 LLM methods mentioned in the Introduction?
Reviewer_QFgmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how LLMs can help with OOD materials property prediction. It evaluates six roles for LLM assistance, such as direct prediction and RAG. The proposed MOTIF asks an LLM to classify spacer cations along chemical axes, and then uses the resulting groups as candidate environments for invariant feature selection. On 2D halide perovskites dataset, their solution reduces OOD MAE from 0.122 to 0.111 eV.

Critical strengths
  • The role assigned to the LLM is clear. It proposes chemical groupings, and the prediction pipeline tests whether those groupings help.
  • The paper includes ablations, direct-LLM comparisons, and feature-selection controls.
Critical weaknesses
  • The presentation of this paper could be improved. A lot of phases like Pseudo-OOD, safe blend and Top-1 CV are not explained clearly before being used. Domain terms such as spacer cation and spacer superfamily are never defined.
  • Table 2 shows that strong classical feature-selection methods achieve nearly the same MAE. What the LLM step adds over them is not clearly justified.
  • The 3rd contribution claims that LLM-based environment annotation is more reliable. But it does not compare against numerical LLM methods on the four transfer benchmarks.
  • The average MAE improvement is 0.011 eV, but the paper does not explain the significance of this improvement. The application benefit is not clearly stated.
Questions for authors
  • Section 4.5 states "LLM taxonomies do not replace classical feature selection". Could you explain the gaps?
Reviewer_BzBQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors investigate the use of LLMs for improving predictions for two-dimensional metal-halide perovskites under leave-one-spacer-family-out OOD setting. LLMS are used for (1) direct prediction, (2) retrieval augmentation, (3) feature generation, (4) sample selection, (5) constraint selection, and (6) taxonomy annotation. Out of these six, the authors find that offline taxonomy is the only role that consistently improves results over baselines. MOTIF is used to select features that are invariant to space family to allow trained predictors to effectively capture out of distribution structure-property relationships by making the selected features always in-distribution.

Critical strengths
  • The authors propose an interesting mechanism for automated invariant feature extraction, where the LLM itself first proposes the groups from which features are obtained from. Such a technique can potentially be used in a broad array of problems.
Critical weaknesses
  • In Table 1, it seems like a majority of the improvement is entirely centered around improvements in the Cyclo family and using the mean actually hides the fact that 4 other cases, the baselines perform better. Bolding the values for Alkyl-S, Alkyl-L, Hetero seems to be misleading as there are baselines that outperform MOTIF in many cases. This calls into question whether the stated conclusion and claims actually hold in general or whether they are an anomaly of the dataset.

  • Line 322-343 mention that the ablation presented in figure 3 supports the mechanistic behavior claimed. But figure 3 as shown is a bit misleading where the axis actually starts at .105 rather than 0, and the actual differences are quite small. Since standard deviations are not reported, the improvements cannot be reliably attributed to the feature selections or random noise. Considering the random groups also outperforms the baselines, we would need to see the spread to make the judgement.

Questions for authors
  • Are the errors reported from cross validation or a held out test set? How are you ensuring that the gating mechanism which looks are a base cross-validation error is not biasing the final reported error?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.