This paper explores aspects of correlations in various data sets upon which state of the art DNNs are trained. It begins with an account of how to understand the statistics of natural images in terms of correlational structures in the data. I then focus on recent work that demonstrates a remarkable robustness in statistical regularities across the various data sets by appealing to Random Matrix Theory. I suggest that DNNs generalize as well as they do because their training allows them to discover/recover these robust regularities.
Back to Workshop IV: Modeling Multi-Scale Collective Intelligences
Copyright. All Rights Reserved.
