Conclusion: What is Important in Learning Theory?

Abstract

In the beginning of this book we postulated (without any discussion) that learning is a problem of function estimation on the basis of empirical data. To solve this problem we used a classical inductive principle — the ERM principle. Later, however, we introduced a new principle — the SRM principle. Nevertheless, the general understanding of the problem remains based on the statistics of large samples: the goal is to derive the rule that possesses the lowest risk. The goal of obtaining the “lowest risk” reflects the philosophy of large sample size statistics: the rule with low risk is good because if we use this rule for a large test set, with high probability, the means of losses will be small.