Theoretical Deep Learning

Lecture notes

  1. A brief introduction to supervised learning

  2. Concentration inequalities

    1. Sub-Gaussian, Chernoff bound, Hoeffding's inequality, McDiarmid's inequalty

  3. Uniform bounds and empirical processes

    1. Rademacher complexity, Covering number, Dudley entropy integral

  4. Kernel methods, representer theorem and RKHSs

  5. RKHS II

  6. Two-layer neural networks and the Fourier analysis

  7. The Barron space

  8. Deep neural networks

    1. lecture note,

    2. Approximation theory of deep ResNets

    3. Depth separations

  9. Training neural networks: Convergence?

    1. A brief overview of GD convergence

    2. Slide

  10. Training neural networks beyond the kernel regime

    1. Slide