Stochastic gradients (SG) methods have a long history in optimization, dating to the work of Robbins and Monro in 1951. The appeal of these methods is due largely to their ability to cope efficiently and robustly with inexact information about the underlying optimization problem. Recently, there has been an outburst of research activity on SG methods, driven in part by their remarkable suitability to machine learning models involving huge data sets. A prominent feature of these methods is their ability to make progress by examining only a small fraction of the data set rather than scanning the entire data set – an operation that is prohibitively expensive in many modern applications.
This workshop will address various topics in the theory, implementation, and practice of SG methods, possibly including the following: applications to nonconvex problems and regularized objectives; parallel implementations; hybridization of SG methods with other optimization techniques; and use of SG methods in deep learning, latent variable models, and other settings.
Leon Bottou
(Microsoft Research)
Yann LeCun
(New York University)
Ben Recht
(University of California, Berkeley (UC Berkeley))
Stephen Wright
(University of Wisconsin-Madison)
Copyright. All Rights Reserved.