Optimal pre-train/fine-tune strategies for accurate material property predictions

Journal: npj Computational Materials • • Nature Portfolio • ISSN: 2057-3960

Abstract

Abstract A pathway to overcome limited data availability in materials science is to use the framework of transfer learning, where a pre-trained (PT) machine learning model (on a larger dataset) can be fine-tuned (FT) on a target (smaller) dataset. We systematically explore the effectiveness of various PT/FT strategies to learn and predict material properties and create generalizable models by PT on multiple properties (MPT) simultaneously. Specifically, we leverage graph neural networks (GNNs) to PT/FT on seven diverse curated materials datasets, with sizes ranging from 941 to 132,752. Besides identifying optimal PT/FT strategies and hyperparameters, we find our pair-wise PT-FT models to consistently outperform models trained from scratch on target datasets. Importantly, our MPT models outperform pair-wise models on several datasets and, more significantly, on a 2D material band gap dataset that is completely out-of-domain. Finally, we expect our PT/FT and MPT frameworks to accelerate materials design and discovery for various applications.

Subjects & Keywords

How to Cite this Article

Reshma Devi, Keith T. Butler, Gopalakrishnan Sai Gautam (2024). "Optimal pre-train/fine-tune strategies for accurate material property predictions". npj Computational Materials. https://doi.org/10.1038/s41524-024-01486-1