Kernel methods are a group of approaches in machine learning that make predictions on previously unseen data based on their similarity to observations in the training set. The pairwise similarity is encoded in terms of a function known as a kernel. The predictive capabilities of kernel methods can be affected by modifying the kernel and its parameters.
Kernel methods are used for a wide variety of applications, including classification, regression, and density estimation. In this article, we develop the intuition for kernel methods using the example of regression.