At Citrine Informatics, we apply machine learning to small experimental materials and chemicals datasets, often sampled on regular grids and almost always with uncharacterized label noise. These characteristics pose a challenge to the conventional random forest approach, resulting in surprisingly correlated trees.
Boltzmann trees reduce variance by modifying the splitting procedure used to build decision trees: they select the feature and split location with a probability proportional to \exp[-\beta \Delta \sigma^2], which is a Boltzmann factor on the change in variance (i.e., impurity) with \beta serving as an inverse temperature. At low temperature, Boltzmann trees approach conventional decision trees, and at high temperature the cut point locations depend entirely on the data density and not on the training labels. At intermediate temperatures, Boltzmann trees will reduce correlation as long as the marginal impurity is not too high.
We apply Boltzmann trees to synthetic and natural datasets from problems in materials and chemistry and discuss their performance compared to other tree-ensemble methods.
Copyright. All Rights Reserved.