Training Model for Predicting Adsorption Energy of Metal Ions Based on Machine Learning

doi:10.15541/jim20200748

Abstract

Abstract:

The adsorption behavior of graphene oxide and metal ions was simulated theoretically by density functional theory. In the process of training the prediction model based on the machine learning method, the missing values were processed by matrix completion method, which was widely used in the recommendation systems, and gradient boosting machine (GBM) was trained to explain the importance of factors that affect the adsorption energy. The result showed that nine properties of the adsorption, namely ionic radius, zero-point vibration energy, Mulliken charge, boiling point, dipole moment, atomic weight, molar heat capacity at constant volume (CV), spin multiplicity and bond length, were found to provide 90% importance of the cumulative adsorption energy. Then six regression methods, including support vector regression, ridge regression, random forest, extremely randomized trees, extreme gradient boosting, and light gradient boosting machine, were used to quantitatively evaluate the prediction accuracy. The results showed that machine learning could provide sufficient accuracy to predict adsorption energy. Among them, extremely randomized trees displayed the best prediction performance, with a mean square error only 0.075. Furthermore, the trained model was tested in a system of vanillin adsorbing metal ions, verifying the feasibility of training the prediction model of adsorption energy based on machine learning. But it is still necessary to be further improved. In general, this research takes the advantage of machine learning on the basis of saving experimental time to provide an instructive reference for theoretical research on metal ion removal.

Key words: machine learning, density functional theory, adsorption energy, metal ions, extremely randomized trees

CLC Number:

TQ174

ZHANG Ruihong, WEI Xin, LU Zhanhui, AI Yuejie. Training Model for Predicting Adsorption Energy of Metal Ions Based on Machine Learning[J]. Journal of Inorganic Materials, 2021, 36(11): 1178-1184.

Figures/Tables 7

Table 1 21 feature descriptors calculated based on DFT

No.	Feature descriptor	No.	Feature descriptor	No.	Feature descriptor
1	Charge	8	Ionic radius	15	CV (Cal/mol-K)
2	Spin	9	Melting point	16	S(Cal/mol-K)
3	Atomic radius	10	Boiling point	17	Zero-point vibrational energy/(kCal·mol^-1)
4	Atomic number	11	First ionization energy	18	Molecular mass
5	Atomic weight	12	Electronegativity	19	Mulliken charges
6	Density/(g·cm^-3)	13	M-O (bond length)	20	APT charges
7	Atomic volume	14	E(Thermal)/(kCal·mol^-1)	21	Dipole moment/D

Fig. 1 (a) Thermal map of correlation between features with correlation coefficient>0.6, and (b) example of adsorption structure of GO adsorbing Cr3+ Note: 1 Cal=4.104 J

Fig. 2 Feature importance ranking Note: 1 Cal=4.104 J; 1 D≈0.020819434 nm

Table 2 Optimal hyperparameters of six machine learning methods

Category	Method	Optimal hyperparameters
Kernel	Support vector regression (SVR)	C = 2, kernel=“ rbf ”
Kernel	Ridge regression	Alpha = 30
Random forest	Random forest (RF)	n_estimators = 31, max_depth = 6, max_features = 2
Random forest	Extremely randomized trees (ERT)	n_estimators = 31, max_depth = 7, random_state = 1
Boosting	Extreme gradient boosting (XGBoost)	n_estimators = 31, max_depth = 2, min_child_weight = 13, learning_rate =.32
Boosting	Light gradient boosting machine (LightGBM)	n_estimators =17, objective = ‘regression’, num_leaves = 31, learning_ rate = 0.32

Fig. 3 Fitting effect diagram and score of six machine learning methods. (a) Support vector regression (SVR); (b) Ridge regression (Ridge); (c) Random forest (RF); (d) Extremely randomized trees (ERT); (e) Extreme gradient boosting (XGBoost); (f) Light gradient boosting machine (LightGBM)

Fig. 4 (a) Mean square error (MSE) of the four ensemble methods, and (b-e) correlation graphs of the true and predicted values of the four ensemble methods (b) Random forest (RF); (c) Extremely randomized trees (ERT); (d) Extreme gradient boosting (XGBoost); (e) Light gradient boosting machine (LightGBM)

Fig. 5 (a) Example of the structure of vanillin monomer adsorbing metal ions; (b) Fitting effect graph of Extremely Randomized Trees (ERT) for VMA-Mn+ adsorption energy; (c) Correlation diagram of ERT for VMA-Mn+ adsorption energy

References 34

[1]	PENG W J, LI H Q, LIU Y Y, et al. A review on heavy metalions adsorption from water by graphene oxide and its composites. Journal of Molecular Liquids, 2017, 230:496-504. DOI URL
[2]	AHMAD S Z N, SALLEH W N W, ISMAIL A F, et al. Adsorptive removal of heavy metal ions using graphene-based nanomaterials: toxicity, roles of functional groups and mechanisms. Chemosphere, 2020, 248:126008. DOI URL
[3]	LIU Y, ZHAO C F, ZHANG A R, et al. Theoretical study on the removal of uranyl by nitrogen, phosphorus and sulfur doped graphene materials. Scientia Sinica Chimica, 2019, 49(1):91-102. DOI URL
[4]	PENG X J, WANG Y F. Efficient stochastic simulation algorithm for chemically reacting systems based on support vector regression. Chinese Journal of Chemical Physics, 2009, 22(5):502-510. DOI URL
[5]	CAI C, LI L, DENG X, et al. Machine learning and high- throughput computational screening of metal-organic framework for separation of methane/ethane/propane. Acta Chimica Sinica, 2020, 78(5):427. DOI URL
[6]	ORUPATTUR N V, MUSHRIF S H, PRASAD V. Catalytic materials and chemistry development using a synergistic combination of machine learning and ab initio methods. Computational Materials Science, 2020, 174:109497-16. DOI URL
[7]	LI X, XI L L, YANG J. First principles high-throughput research on thermoelectric materials: a review. Journal of Inorganic Materials, 2019, 34(3):236-246. DOI URL
[8]	MENG Y, WANG X, YANG J, et al. Research on machine learning based model for predicting the impact status of laminated glass. Journal of Inorganic Materials, 2021, 36(1):61-68. DOI URL
[9]	FENG C, SHARMAN E, YE S, et al. A neural network protocol for predicting molecular bond energy. Sci. China Chem., 2019, 62(12):1698-1703. DOI URL
[10]	LU S, ZHOU Q, OUYANG Y, et al. Accelerated discovery of stable lead-free hybrid organic-inorganic perovskites via machine learning. Nature Communications, 2018, 9(1):3405. DOI URL
[11]	TEHRANI A M, OLIYNYK A O, PARRY M, et al. Machine learning directed search for ultraincompressible, superhard materials. Journal of the American Chemical Society, 2018, 140(31):9844-9853. DOI URL
[12]	KANG Y, LI L, LI B. Recent progress on discovery and properties prediction of energy materials: simple machine learning meets complex quantum chemistry. Journal of Energy Chemistry, 2020, 54:72-88. DOI URL
[13]	BROCKHERDE F, VOGT L, LI L, et al. By-passing the Kohn-Sham equations with machine learning. Nature Communications, 2017, 8(1):872. DOI URL
[14]	XIAO Y, MIARA L J, WANG Y, et al. Computational screening of cathode coatings for solid-state batteries. Joule, 2019, 3(5):1252-1275. DOI URL
[15]	PANAPITIYA G, AVENDANO-FRANCO G, REN P, et al. Machine learning prediction of CO adsorption in thiolated, Ag alloyed Au nanoclusters. Journal of the American Chemical Society, 2018, 140(50):17508-17514. DOI URL
[16]	PARDAKHTI M, MOHARRERI E, WANIK D, et al. Machine learning using combined structural and chemical descriptors for prediction of methane adsorption performance of metal organic frameworks (MOFs). ACS Combinatorial Science, 2017, 19(10):640-645. DOI URL
[17]	SI Y, SAMULSKIE T. Synthesis of water soluble graphene. Nano Letters, 2008, 8(6):1679-1682. DOI URL
[18]	CHEN D, FENG H, LI J. Graphene oxide: preparation, functionalization, and electrochemical applications. Chemical Reviews, 2012, 112(11):6027-6053. DOI URL
[19]	SHENG Z H, SHAO L, CHEN J J, et al. Catalyst free synthesis of nitrogen-doped graphene via thermal annealing graphite oxide with melamine and its excellent electrocatalysis. ACS Nano, 2011, 5(6):4350-4358. DOI URL
[20]	MAURIZIO C, VICENZO B, MICHAEL A R. A direct procedure for the evaluation of solvent effects in MC-SCF calculations. Journal of Chemical Physics, 1999, 111(12):5295-5302.
[21]	BRAND M. Incremental Singular Value Decomposition of Uncertain Data with Missing Values. Computer Vision-ECCV 2002, Berlin, Heidelberg, 2002: 707-720.
[22]	NEELAKANTAN A, VILNIS L, LEQ V, et al. Adding gradiant noise improves learning for very deep networks. arXiv: Machine Learning, 2015, 1511:06807.
[23]	迪安J A, 魏俊发. 兰氏化学手册, 2版. 北京: 科学出版社, 2003: 1-1579.
[24]	FRIEDMAN J H, JAO S. Greedy function approximation: a gradient boosting machine. Annals of Statistics, 2001, 29(5):1189-1232. DOI URL
[25]	LIASHCHYNSKYI P, LIASHCHYNSKYI P. Grid search, random search, genetic algorithm: a big comparison for NAS. arXiv: Learning, 2019, 1912:06059.
[26]	MARTENS H A, DARDENNE P J, CSYSTEMS I L. Validation and verification of regression in small data sets. ChemomERTics & Intelligent Laboratory Systems, 1998, 44(1/2):99-121.
[27]	FRIEDMAN J H. Stochastic gradient boosting. Computational Statistics & Data Analysis, 2002, 38(4):367-378. DOI URL
[28]	SMOLA A J, SCHOLKOPF B. A tutorial on support vector regression. Statistics and Computing, 2004, 14(3):199-222. DOI URL
[29]	RUPP M. Machine learning for quantum mechanics in a nutshell. International Journal of Quantum Chemistry, 2015, 115(16):1058-1073. DOI URL
[30]	SVETNIK V. Random forest: a classification and regression tool for compound classification and QSAR modeling. Journal of Chemical Information and Modeling, 2003, 43(6):1947-1958.
[31]	GEURTS P, ERNST D, WEHENKEL L. Extremely randomized trees. Machine Learning, 2006, 63(1):3-42.
[32]	DRUCKER H. Improving regressors using boosting techniques. Morgan Kaufmann Publishers Inc, 1997: 107-115.
[33]	FRANLLIN J. The elements of statistical learning: data mining, inference, and prediction. The Mathematical Intelligencer, 2005, 27(2):83-85.
[34]	SANTOS R I H, REIS D T, PEREIRA D H. A DFT based analysis of adsorption of Cd²⁺, Cr³⁺, Cu²⁺, Hg²⁺, Pb²⁺, and Zn²⁺, on vanillin monomer: a study of the removal of metalions from effluents. Journal of Molecular Modeling, 2019, 25(9):267.