Learning with Stochastic Gradients

Description

We will introduce online learning from different perspectives, such as stochastic approximation and incremental emprical risk minimization. We introduce recursive (incremental) and online algorithms for optimizing RLS, and compare their merits.

Class Reference Material

L. Rosasco, T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017

Chapter 7 - Online Learning


Note: The course notes, in the form of the circulated book draft is the reference material for this class. Related and older material can be accessed through previous year offerings of the course.

Further Reading

• H. J. Kushner and G. Yin, Stochastic Approximation, Recursive Algorithms and Applications, 2nd Edition, Springer-Verlag, 2003.
• J. Kivinen, A. J. Smola, and R. C. Williamson, Online learning with kernels, IEEE Transactions on Signal Processing, 52(8):2165-2176, 2004.