Statistical learning theory

Summary

Statistical learning theory is the mathematical theory of learning a prediction rule from examples. It models the data as independent draws from an unknown distribution and measures a rule by its expected loss on new data, its risk. Its central results bound the gap between the error on the training data and the true risk. The bounds depend on the size of the class of rules, measured for example by its VC dimension. It was developed by Vladimir Vapnik and Alexey Chervonenkis among others, and it underlies empirical risk minimization and support vector machines.