Kernel Machines
Synopsis
1- Notions on Kernels and Reproducing Kernel Hilbert Space Theory
2- Learning in RKHS
2- Learning in RKHS
3- Kernel design, learning, alignement and approximation
4- Learning to predict graphs and functions
5- Maximum Mean Discrepancy and Generative models (diffusion and flows)
6-MLP, Convnet and transformers at the lens of kernels
In this course you will (re)discover that linear methods extend to nonlinear by using the famous kernel trick. Linear regression, linear classification and linear dimensionality reduction approaches will be highlighted as typical examples of this family of approaches. You will also learn how to think about machine learning in terms of hypothesis spaces and regularization choices, by leveraging a unique hyperparameter: the kernel. This will raise the issue of kernel design and learning. We will then show that the kernel trick is also interesting in the output space by tackling multi-task and structured prediction. We will see how generative modeling can be accelerated at sempling time by leveraging Maximum Mean Discrepancy.
Eventually, we will the links between kernel machines and neural networks.
The course is punctuated by 5 practical sessions. Exam (oral: project) + 5 graded TPs (1 pt each)
