Optimization techniques have become vital tools in machine learning and data analysis. We review here some fundamentals of relevant optimization formulations and techniques, with a focus on the most widely used methods. The discussion will include stochastic gradient methods, coordinate descent methods, methods for nonsmooth-regularized problems, and augmented Lagrangian methods including ADMM.
Copyright. All Rights Reserved.