Mercer Theorem and Feature Maps
We introduce a particularly useful family of hypothesis spaces
called Reproducing Kernel Hilbert Spaces (RKHS) that have a key role in
the theory of learning. We first provide the necessary background in
functional analysis and then define RKHS using the reproducing property.
We then derive the general solution of Tikhonov regularization in RKHS.
Slides for this lecture: PDF.
- Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 686, 337-404, 1950.
- Cucker and Smale. On the mathematical foundations of
learning. Bulletin of the American Mathematical Society, 2002.
- Evgeniou, Pontil and Poggio. Regularization Networks and Support Vector Machines Advances in Computational Mathematics, 2000.
- Girosi, F. An Equivalence between Sparse Approximation and
Support Vector Machines. Neural Computation, Vol. 10, 1455-1480,
1998. (Appendix A)
- Wahba, G. Spline Models for Observational Data
Series in Applied Mathematics, Vol. 59, SIAM, 1990. (Chapter 1)