Iterative Regularization via Early Stopping

Description

We discuss regularization methods beyond Tikhonov regularization, and in particular early stopping.

Class Reference Material

L. Rosasco, T. Poggio, Machine Learning: a Regularization Approach, MIT-9.520 Lectures Notes, Manuscript, Dec. 2017

Chapter 5 - Beyond Penalization: Spectral Filtering


Note: The course notes, in the form of the circulated book draft is the reference material for this class. Related and older material can be accessed through previous year offerings of the course.

Further Reading

• F. Bauer, S. Pereverzev, and L. Rosasco, On Regularization Algorithms in Learning Theory, Journal of Complexity, 23(1):52-72, 2007.
• Y. Yao, L. Rosasco, and A. Caponnetto, On Early Stopping in Gradient Descent Learning, Constructive Approximation, 26(2):289-315, 2007.
• L. Gerfo, L. Rosasco, F. Odone, E. De Vito, and A. Verri, Spectral Algorithms for Supervised Learning, Neural Computation, 20(7):1873-1897, 2008.
• O. Bousquet and L. Bottou, The Tradeoffs of Large Scale Learning, Advances in Neural Information Processing Systems (NIPS), 2008.
• J. Lin, L. Rosasco and D.X. Zhou, Iterative Regularization for Learning with Convex Loss Functions, Journal of Machine Learning Research 17 (77), 1-38, 2016.