Mathematical Methods for Supervised Learning M1 SID (2024-2025)

Lecturer: Clément Lalanne

Overview

This course introduces mathematical methods essential for supervised learning. Topics include linear models, optimization techniques, and advanced methods like kernel and sparse methods. Students are expected to have basic knowledge of linear algebra, calculus, and probability.

Evaluation

Details about the evaluation will be provided later.

Survival Kit for the Exam

Details about the exam preparation will be provided later.

Lectures

Lecture 1 (Date TBD): Introduction to Supervised Learning
Lecture 2 (Date TBD): Linear Models
Lecture 3 (Date TBD): Elements of Optimization + Logistic Regreession
Lecture 4 (Date TBD): Sparse Methods
Lecture 5 (Date TBD): Kernel Methods
Lecture 6 (Date TBD): Classification and Regression Trees + Boosting
Lecture 7 (Date TBD): Neural Networks
Lecture 8 (Date TBD): Questions / Answers before the exam : To maximize efficiency, prepare your questions before the class !

TDs

TD 1 (Date TBD): Elements of Statistics
TD 2 (Date TBD): Linear Models
TD 3 (Date TBD): Tree Algorithms
TD 4 (Date TBD): Kernel Methods

TPs

TP 1 (Date TBD): TBD
TP 2 (Date TBD): TBD
TP 3 (Date TBD): TBD

References and External Resources

Machine Learning and Learning Theory

Learning Theory from First Principles by Francis Bach, 2024: Main reference for the course.
Foundations of Machine Learning by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar, 2018.

Optimization for Machine Learning

Convex Optimization by Stephen Boyd and Lieven Vandenberghe, Cambridge University Press, 2012.
First-Order Optimization Methods by Amir Beck, 2017.
Acceleration Methods by Alexandre d’Aspremont, Damien Scieur and Adrien Taylor, 2024.

Measure and Probability Theory

Intégration, Probabilités et Processus Aléatoires by Jean-François Le Gall, 2006: An excellent introduction to measure theory with elements of stochastic processes and conditional expectation.
Probabilités 2 by Jean-Yves Ouvrard, 2009: A detailed treatment of advanced topics in probability theory.
Concentration Inequalities: A Nonasymptotic Theory of Independence by Stéphane Boucheron, Gábor Lugosi, and Pascal Massart, 2013.

Analysis

Cours d'analyse by Jean-Michel Bony, 2001.