UVA Deep Learning Course

MSc in Artificial Intelligence for the University of Amsterdam.

Find Out More

About

Deep learning is primarily a study of multi-layered neural networks, spanning over a great range of model architectures. This course is taught in the MSc program in Artificial Intelligence of the University of Amsterdam. In this course we study the theory of deep learning, namely of modern, multi-layered neural networks trained on big data. The course is taught by Assistant Professor Xiantong Zhen. The teaching assistants are Christos Athanasiadis, Ilze Amanda Auzina, Leonard Bereska, Jim Boelrijk, Natasha Butt, Mohammad Mahdi Derakhshani, Winfried van den Dool, Yingjun Du, Alex Gabel, Mariya Hendriksen, Tom Lieberum, Phillip Lippe, Yongtuo Liu, Jie Liu, Ben Miller, Ivona Najdenkoska, Sarah Rastegar, Nadja Rutsch, Mohammadreza Salehi, Jiayi Shen, Tom van Sonsbeek, Riccardo Valperga, Haochen Wang, Zehao Xiao

Christos Athanasiadis Ilze Amanda Auzina Leonard Bereska Jim Boelrijk

Natasha Butt Mohammad Mahdi Derakhshani Winfried van den Dool Yingjun Du Alex Gabel

Mariya Hendriksen Tom Lieberum Phillip Lippe Yongtuo Liu Jie Liu

Ben Miller Ivona Najdenkoska Sarah Rastegar Nadja Rutsch Mohammadreza Salehi

Jiayi Shen Tom van Sonsbeek Riccardo Valperga Haochen Wang Zehao Xiao

Lectures

Week 1

November 4, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

November 4, 2021 | 13.00-15.00 | Tutorial session

This tutorial introduces the practical sessions, the TA organizer team, etc. Afterwards, we will discuss the PyTorch machine learning framework, and introduce you to the basic concepts of Tensors, computation graphs and GPU computation. We will continue with a small hands-on tutorial of building your own, first neural network in PyTorch.

We also provide a crash-course for working with the Lisa cluster, and how to setup your account for Lisa.

Documents:

Recording - Part 1 (Introduction)
Recording - Part 2 (PyTorch Basics)
Recording - Part 3 (NNs in PyTorch)
Lisa Tutorial
PyTorch tutorial notebook

November 5, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

Week 2

November 11, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

November 11, 2021 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss the role of activation functions in a neural network, and take a closer look at the optimization issues a poorly designed activation function can have.

After the presentation, there will by a TA session for Q&A for assignment 1, lecture content and more.

Documents:

Recording - Part 1
Recording - Part 2
Jupyter notebook

November 12, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

Week 3

November 18, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

November 18, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss the importance of proper parameter initialization in deep neural networks, and how we can find a suitable one for our network. In addition, we will review the optimizers SGD and Adam, and compare them on complex loss surfaces.

After the presentation, there will by a TA session for Q&A for assignment 1, lecture content and more.

Documents:

Recording - Part 1 (Initialization)
Recording - Part 2 (Optimization)
Jupyter notebook

November 19, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

Week 4

November 25, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

November 25, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will implement three popular, modern ConvNet architectures: GoogleNet, ResNet, and DenseNet. We will compare them on the CIFAR10 dataset, and discuss the advantages that made them popular and successful across many tasks.

After the presentation, there will by a TA session for Q&A for assignment 2, lecture content and more.

Documents:

Recording - Part 1 (PyTorch Lightning)
Recording - Part 2 (Inception and ResNet)
Recording - Part 3 (DenseNet and comparison)
Jupyter notebook

November 26, 2021 | 13.00-15.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

Week 5

December 2, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

December 2, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss the implementation of Graph Neural Networks. In the first part of the tutorial, we will implement the GCN and GAT layer ourselves. In the second part, we use PyTorch Geometric to look at node-level, edge-level and graph-level tasks.

After the presentation, there will by a TA session for Q&A for assignment 2, lecture content and more.

Documents:

Recording - Part 1 (GCN and GAT)
Recording - Part 2 (Experiments)
Jupyter notebook

December 3, 2021 | 15.00-17.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

Week 6

December 9, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

December 9, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss the relatively new breakthrough architecture: Transformers. We will start from the basics of attention and multi-head attention, and build our own Transformer. We will perform experiments on sequence-to-sequence tasks and set anomaly detection.

After the presentation, there will by a TA session for Q&A for assignment 3, lecture content and more.

Documents:

Recording - Part 1 (What is Attention + MH Attention)
Recording - Part 2 (Architecture and Training tricks)
Recording - Part 3 (Experiments)
Jupyter notebook

December 10, 2021 | 15.00-17.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

Week 7

December 16, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

December 16, 2020 | 13.00-15.00 | Tutorial session + TA Q&A

In this tutorial, we will discuss deep convolutional autoencoders and their applications. In the practical and lecture, you will see variational autoencoders (VAE), which add a stochastic part to vanilla autoencoders. Both have their advantages and applications, of which we visit image retrieval and compression for the vanilla auotoencoder.

After the presentation, there will by a TA session for Q&A for assignment 3, lecture content and more.

Documents:

Recording - Part 1 (Building an AE)
Recording - Part 2 (Analysis)
Jupyter notebook

December 17, 2021 | 11.00-13.00 | Lecture

Details will follow soon.

Documents:

No documents.

Lecture recordings:

No recordings.

Practicals

Deadline: November 19, 2020

Details will follows soon.

Documents:

No documents.

The LeNet was one of the first CNNs proposed.

Deadline: December 3, 2020

Details will follows soon.

Documents:

No documents.

RNNs unroll a network over time.

Deadline: December 17, 2020

Details will follows soon.

Documents:

No documents.

VAEs model a distribution in latent space.

If you are interested in older versions of the lectures, you can find them below.

UVADLC Nov 2016 UVADLC Nov 2017 UVADLC Sep 2018 UVADLC Apr 2019 UVADLC Nov 2019 UVADLC Nov 2020

Contact us!

If you have any questions or recommendations for the website or the course, you can always drop us a line! The knowledge should be free, so feel also free to use any of the material provided here (but please be so kind to cite us). In case you are a course instuctor and you want the solutions, please send us an email.

Room C3.229, Science Park 904 1098 XH, Amsterdam, The Netherlands

x.zhen@uva.nl