Lorenzo Rosasco/Tomaso Poggio
In this class we are going to introduce online learning from three different
perspectives - stochastic approximation, incremental emprical risk minimization
and game theoretic. We introduce recursive (incremental) and online algorithms
for optimizing RLS, and compare their merits.
Slides for this lecture: PDF.
H. J. Kushner and G. Yin, Stochastic
Approximation and Recursive Algorithms and Applications, 2nd Edition, Springer-Verlag, New York, 2003, [Applications of Mathematics, Volume 35], xxii+474 pp.
D. P. Bertsekas, "Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey", Lab. for Information and Decision Systems Report LIDS-P-2848, MIT, August 2010.
Nicolo Cesa-Bianchi and Gabor Lugosi Prediction, learning, and games Cambridge University Press, 2006.