UvA Deep Learning Course

Lectures

Week 1

This lecture introduces the structure of the Deep Learning course, and gives a short overview of the history and motivation of Deep Learning.

Documents:

Intro lecture to Deep Learning

Lecture recordings:

Part 1: Intro lecture to Deep Learning

This tutorial introduces the practical sessions, the TA organizer team, etc. Afterwards, we will discuss the PyTorch machine learning framework, and introduce you to the basic concepts of Tensors, computation graphs and GPU computation. We will continue with a small hands-on tutorial of building your own, first neural network in PyTorch.

Documents:

Introduction to PyTorch

This lectures introduces basic concepts for Deep Feedforward Networks such linear and nonlinear modules, gradient-based learning and the backpropagation algorithm.

Documents:

Deep Feedforward Networks

Lecture recordings:

Part 1: Deep Feedforward Networks

Week 2

This lecture series discusses advanced optimizers, initialization, normalization and hyperparameter tuning.

Documents:

Deep Feedforward Networks

Lecture recordings:

Part 1: Deep Feedforward Networks

In this tutorial, we will discuss the role of activation functions in a neural network, and take a closer look at the optimization issues a poorly designed activation function can have.

After the presentation, there will by a TA session for Q&A for assignment 1, lecture content and more.

Documents:

Activation Functions

This lecture series discusses advanced optimizers, initialization, normalization and hyperparameter tuning.

Documents:

Deep Learning Optimizations II

Lecture recordings:

Part 1: Deep Learning Optimizations II

Week 3

This lecture series covers convolutional neural networks for image processing.

Documents:

Convolutional neural networks for image processing

Lecture recordings:

Part 1: Convolutional neural networks for image processing

In this tutorial, we will discuss the importance of proper parameter initialization in deep neural networks, and how we can find a suitable one for our network. In addition, we will review the optimizers SGD and Adam, and compare them on complex loss surfaces.

After the presentation, there will by a TA session for Q&A for assignment 1, lecture content and more.

Documents:

No documents.

This lecture series covers modern ConvNet architecture.

Documents:

Convolutional neural networks for image processing

Lecture recordings:

Part 1: Convolutional neural networks for image processing

Week 4

This lecture series covers Transformers

Documents:

Transformers

Lecture recordings:

Part 1: Transformers

In this tutorial, we will implement three popular, modern ConvNet architectures: GoogleNet, ResNet, and DenseNet. We will compare them on the CIFAR10 dataset, and discuss the advantages that made them popular and successful across many tasks.

After the presentation, there will by a TA session for Q&A for assignment 2, lecture content and more.

Documents:

No documents.

Lecture on Graph Neural Networks.

Documents:

Graph Neural Networks

Lecture recordings:

Part 1: Graph Neural Networks

Week 5

Lecturer in deep variational autoencoder.

Documents:

Deep variational autoencoder

Lecture recordings:

Part 1: Deep variational autoencoder

In this tutorial, we will discuss the relatively new breakthrough architecture: Transformers. We will start from the basics of attention and multi-head attention, and build our own Transformer. We will perform experiments on sequence-to-sequence tasks and set anomaly detection.

After the presentation, there will by a TA session for Q&A for assignment 3, lecture content and more.

Documents:

No documents.

Deep Learning & The Natural Sciences

Documents:

Deep Learning & The Natural Sciences

Lecture recordings:

Part 1: Deep Learning & The Natural Sciences

Week 6

Gemerative Advetsarial Networks and difusion models

Documents:

Gemerative Advetsarial Networks and difusion models

Lecture recordings:

Part 1: Gemerative Advetsarial Networks and difusion models

In this tutorial, we will discuss the implementation of Graph Neural Networks. In the first part of the tutorial, we will implement the GCN and GAT layer ourselves. In the second part, we use PyTorch Geometric to look at node-level, edge-level and graph-level tasks.

After the presentation, there will by a TA session for Q&A for assignment 2, lecture content and more.

Documents:

Tutorial on Graph Neural Networks

Guest Lecture: Deep Learning for 3D (Christian Rupprecht)

Documents:

Guest Lecture: Deep Learning for 3D (Christian Rupprecht) (link TBA)

Lecture recordings:

Part 1: Guest Lecture: Deep Learning for 3D (Christian Rupprecht)

Week 7

Self-supervised learning part I

Documents:

No documents.

Lecture recordings:

No recordings.

We will discuss Tutorial 17: Self-Supervised Learning, and have a short introduction to Causal Representation Learning.

Documents:

No documents.

Self-supervised learning part II

Documents:

No documents.

Lecture recordings:

No recordings.

About

Lectures

Week 1

Documents:

Lecture recordings:

Documents:

Documents:

Lecture recordings:

Week 2

Documents:

Lecture recordings:

Documents:

Documents:

Lecture recordings:

Week 3

Documents:

Lecture recordings:

Documents:

Documents:

Lecture recordings:

Week 4

Documents:

Lecture recordings:

Documents:

Documents:

Lecture recordings:

Week 5

Documents:

Lecture recordings:

Documents:

Documents:

Lecture recordings:

Week 6

Documents:

Lecture recordings:

Documents:

Documents:

Lecture recordings:

Week 7

Documents:

Lecture recordings:

Documents:

Documents:

Lecture recordings:

Practicals

Links

If you are interested in older versions of the lectures, you can find them below.

Contact us!