UVA Deep Learning Course

MSc in Artificial Intelligence for the University of Amsterdam.

Find Out More

About

Deep learning is primarily a study of multi-layered neural networks, spanning over a great range of model architectures. This course is taught in the MSc program in Artificial Intelligence of the University of Amsterdam. In this course we study the theory of deep learning, namely of modern, multi-layered neural networks trained on big data. The course is taught by Assistant Professor Xiantong Zhen. The teaching assistants are Christos Athanasiadis, Ilze Amanda Auzina, Leonard Bereska, Jim Boelrijk, Natasha Butt, Mohammad Mahdi Derakhshani, Winfried van den Dool, Yingjun Du, Alex Gabel, Mariya Hendriksen, Tom Lieberum, Phillip Lippe, Yongtuo Liu, Jie Liu, Ben Miller, Ivona Najdenkoska, Sarah Rastegar, Nadja Rutsch, Mohammadreza Salehi, Jiayi Shen, Tom van Sonsbeek, Riccardo Valperga, Haochen Wang, Zehao Xiao

Christos Athanasiadis Ilze Amanda Auzina Leonard Bereska Jim Boelrijk

Natasha Butt Mohammad Mahdi Derakhshani Winfried van den Dool Yingjun Du Alex Gabel

Mariya Hendriksen Tom Lieberum Phillip Lippe Yongtuo Liu Jie Liu

Ben Miller Ivona Najdenkoska Sarah Rastegar Nadja Rutsch Mohammadreza Salehi

Jiayi Shen Tom van Sonsbeek Riccardo Valperga Haochen Wang Zehao Xiao

Lectures

Week 1

November 4, 2021 | 11.00-13.00 | Lecture

This lecture introduces the structure of the Deep Learning course, and gives a short overview of the history and motivation of Deep Learning.

Documents:

Slides 1.1
Slides 1.2
Slides 1.3
Slides 1.4

Lecture recordings:

No recordings.

November 4, 2021 | 13.00-15.00 | Tutorial session

This tutorial introduces the practical sessions, the TA organizer team, etc. Afterwards, we will discuss the PyTorch machine learning framework, and introduce you to the basic concepts of Tensors, computation graphs and GPU computation. We will continue with a small hands-on tutorial of building your own, first neural network in PyTorch.

We also provide a crash-course for working with the Lisa cluster, and how to setup your account for Lisa.

Documents:

Recording - Part 1 (Introduction)
Recording - Part 2 (PyTorch Basics)
Recording - Part 3 (NNs in PyTorch)
Introduction Slides
Lisa Tutorial
PyTorch tutorial notebook

November 5, 2021 | 11.00-13.00 | Lecture

This lectures introduces basic concepts for Deep Feedforward Networks such linear and nonlinear modules, gradient-based learning and the backpropagation algorithm.

Documents:

Slides 2.1
Slides 2.2
Slides 2.3
Slides 2.4

Lecture recordings:

No recordings.

Week 2

November 11, 2021 | 11.00-13.00 | Lecture

This lecture series discusses advanced optimizers, initialization, normalization and hyperparameter tuning.

Documents:

Slides 3.1
Slides 3.2.1
Slides 3.2.2
Slides 3.3

Lecture recordings:

No recordings.

November 11, 2021 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss the role of activation functions in a neural network, and take a closer look at the optimization issues a poorly designed activation function can have.

After the presentation, there will by a TA session for Q&A for assignment 1, lecture content and more.

Documents:

Recording - Part 1
Recording - Part 2
Jupyter notebook

November 12, 2021 | 11.00-13.00 | Lecture

This lecture series discusses advanced optimizers, initialization, normalization and hyperparameter tuning.

Documents:

Slides 4.1
Slides 4.2
Slides 4.3

Lecture recordings:

No recordings.

Week 3

November 18, 2021 | 11.00-13.00 | Lecture

This lecture series covers convolutional neural networks for image processing.

Documents:

Slides 5.1
Slides 5.2
Slides 5.3

Lecture recordings:

No recordings.

November 18, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss the importance of proper parameter initialization in deep neural networks, and how we can find a suitable one for our network. In addition, we will review the optimizers SGD and Adam, and compare them on complex loss surfaces.

After the presentation, there will by a TA session for Q&A for assignment 1, lecture content and more.

Documents:

Recording - Part 1 (Initialization)
Recording - Part 2 (Optimization)
Jupyter notebook

November 19, 2021 | 11.00-13.00 | Lecture

This lecture series covers modern ConvNet architecture.

Documents:

Slides 6.1

Lecture recordings:

No recordings.

Week 4

November 25, 2021 | 11.00-13.00 | Lecture

This lecture series covers Recurrent Neural Networks

Documents:

Slides 7.1
Slides 7.2

Lecture recordings:

No recordings.

November 25, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will implement three popular, modern ConvNet architectures: GoogleNet, ResNet, and DenseNet. We will compare them on the CIFAR10 dataset, and discuss the advantages that made them popular and successful across many tasks.

After the presentation, there will by a TA session for Q&A for assignment 2, lecture content and more.

Documents:

Recording - Part 1 (PyTorch Lightning)
Recording - Part 2 (Inception and ResNet)
Recording - Part 3 (DenseNet and comparison)
Jupyter notebook

November 26, 2021 | 13.00-15.00 | Online Lecture

Petar Veličković lecture on Graph Neural Networks.

Documents:

Slides 8.1

Lecture recordings:

Part 1: Petar Veličković lecture on GNNs.

Week 5

December 2, 2021 | 11.00-13.00 | Lecture

Ivona's lecture on Attention and Transformers.

Documents:

Slides 9.1

Lecture recordings:

No recordings.

December 2, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss the implementation of Graph Neural Networks. In the first part of the tutorial, we will implement the GCN and GAT layer ourselves. In the second part, we use PyTorch Geometric to look at node-level, edge-level and graph-level tasks.

After the presentation, there will by a TA session for Q&A for assignment 2, lecture content and more.

Documents:

Recording - Part 1 (GCN and GAT)
Recording - Part 2 (Experiments)
Jupyter notebook

December 3, 2021 | 15.00-17.00 | Lecture

This lecture series discusses Generative Adversarial Networks.

Documents:

Slides 10.1

Lecture recordings:

No recordings.

Week 6

December 9, 2021 | 11.00-13.00 | Lecture

This lecture series introduces the framework of variational inference and Variational Autoencoders (VAEs).

Documents:

Slides 11.1

Lecture recordings:

No recordings.

December 9, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss the relatively new breakthrough architecture: Transformers. We will start from the basics of attention and multi-head attention, and build our own Transformer. We will perform experiments on sequence-to-sequence tasks and set anomaly detection.

After the presentation, there will by a TA session for Q&A for assignment 3, lecture content and more.

Documents:

Recording - Part 1 (What is Attention + MH Attention)
Recording - Part 2 (Architecture and Training tricks)
Recording - Part 3 (Experiments)
Jupyter notebook

December 10, 2021 | 15.00-17.00 | Online Lecture

Deep Learning Generalization.

Documents:

Slides from the guest lecture

Lecture recordings:

Part 1: Recording of the guest lecture

Week 7

December 16, 2021 | 11.00-13.00 | Lecture

Geometry and Structure Regularized Deep Learning.

Documents:

No documents.

Lecture recordings:

No recordings.

December 16, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss deep convolutional autoencoders and their applications. In the practical and lecture, you will see variational autoencoders (VAE), which add a stochastic part to vanilla autoencoders. Both have their advantages and applications, of which we visit image retrieval and compression for the vanilla auotoencoder.

After the presentation, there will by a TA session for Q&A for assignment 3, lecture content and more.

Documents:

Recording - Part 1 (Building an AE)
Recording - Part 2 (Analysis)
Jupyter notebook

December 17, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

Practicals

Deadline: November 19, 2021

Multilayer perceptrons and backpropagation

Documents:

First assignment description
First assignment ANS Delft
An example of a neural network.

Deadline: December 3, 2021

CNNs, RNNs & Graph CNNs

Documents:

Second assignment description
RNNs unroll a network over time.

Deadline: December 17, 2021

Generative Models

Documents:

Third assignment description
VAEs model a distribution in latent space.

If you are interested in older versions of the lectures, you can find them below.

UVADLC Nov 2016 UVADLC Nov 2017 UVADLC Sep 2018 UVADLC Apr 2019 UVADLC Nov 2019 UVADLC Nov 2020

Contact us!

If you have any questions or recommendations for the website or the course, you can always drop us a line! The knowledge should be free, so feel also free to use any of the material provided here (but please be so kind to cite us). In case you are a course instuctor and you want the solutions, please send us an email.

Room C3.229, Science Park 904 1098 XH, Amsterdam, The Netherlands

x.zhen@uva.nl