Systematic development of scientific machine-learning models requires rigorous assessment of their predictions and associated uncertainties.
We assess prediction errors of state-of-the-art exact representations (symmetry functions, many-body tensor, smooth overlap of atomic positions) for machine-learning models of ab initio energies of molecules and materials. In this, we control for sampling, regression method and hyperparameter optimization, retaining only the signal from the representation. We observe a physically motivated dependence on body order for all representations and datasets.
Beyond prediction errors, predictive uncertainties are essential for human assessment, decision-making and active learning, yet no best practices or standard evaluation metrics have been established. We analyze and compare evaluation metrics for batch and individual predictive uncertainties of random forests and Gaussian processes on molecular and materials data from experiments and numerical simulations, and provide guidelines for their use.
Copyright. All Rights Reserved.