The Learning Problem and Regularization
We introduce the problem of learning from sparse examples. We introduce key terms and
concepts such as loss functions, empirical risk, true risk, generalization error, hypothesis
spaces, approximation error and sample error. We introduce two key requirements on learning
algorithms: stability and consistency. We then describe Tikhonov regularization -- which
in our course is the algorithm with the magic.
Slides for this lecture: PDF.