The course covers the fundamental algorithms and methods, including backpropagation, differentiable programming, optimization, regularization techniques, and information theory behind DNN’s. Key deep learning architectures are described in detail including convolution neural networks (CNN, ResNet, DenseNet) with derivation from convolutional sparse coding, recurrent neural networks (LSTM and GRU), variational autoencoders (VAEs), and generative adversarial networks (GANs). Deep reinforcement learning is covered with examples, from basic principles of Markov Decision Processes, value-based and policy-based methods, Deep Q-Networks, policy gradient methods, through Monte Carlo tree search and implementation of AlphaZero and DeepStack with Libratus. Topics from visualization of neural networks, attacks against deep nets, and meta learning are presented. Assignments use the TensorFlow/BayesFlow and PyTorch/Pyro programming frameworks, and a final deep learning project is based on a process, data challenge, or research topic.

**Lecture 1 (Tuesday, January 16): Introduction**

Machine learning timeline: from Least Squares to Alpha Zero, motivation, fully connected neural network, forward propagation, nonlinear activation functions, loss functions.

**Lecture 2 (Thursday, January 18): Backpropagation**

Computation graph, derivative of activation functions, backpropagation algorithm, chain rule for differentiation, gradient of loss function, gradient descent, input normalization, weight initialization.

**Lecture 3 (Thursday, January 23): Optimization**

Gradient descent, SGD, momentum, Nesterov, Adagrad, RMSprop, Adam, Newton’s method, Conjugate Gradients, L-BFGS.

**Lecture 4 (Tuesday, January 25): Regularization**

Bias and variance, norms, ridge regression, Lasso, regularized cost functions, dropout, data augmentation, input distortion, normalization.

**Lecture 5 (Tuesday, January 30): Information theory of neural networks**

Markov matrices, Markov chain, entropy, mutual information, information of deep neural networks.

**Lecture 6 (Thursday, February 1): Programming frameworks**

TensorFlow, computation graph, session, placeholders, variables, layers, estimators, TensorBoard, BayesFlow, TFGAN.

**Lecture 7 (Tuesday, February 6): Convolutional neural networks**

Convolution, convolution layer, max pooling, ImageNet, AlexNet, VGG, Inception.

**Lecture 8 (Thursday, February 8): Convolutional neural networks**

CNN architectures, ResNets, DenseNets, applications: classification, detection, segmentation, pose estimation, completion, synthesis.

**Lecture 9 (Tuesday, February 13): Sequence models**

RNN’s, architectures, deep, bidirectional, conditioned, recursive, backpropogation through time (BPTT).

**Lecture 10 (Thursday, February 15): Sequence models**

Long short-term memory (LSTM), gated recurrent unit (GRU), natural language models.

**Lecture 11 (Tuesday, February 20): Autoencoders, variational autoencoders (VAEs)**

Non-linear PCA, denoising, completion, encoder and decoder networks, variational approximation.

**Lecture 12 (Thursday, February 22): Generative adversarial networks (GANs)**

**Lecture 13 (Tuesday, February 27): Reinforcement learning**

**Lecture 14 (Thursday, March 1): Reinforcement learning**

**Lecture 15 (Tuesday, March 6): Deep reinforcement learning**

**Lecture 16 (Thursday, March 8): Deep reinforcement learning**

**Spring Recess (Monday, March 12 to Friday, March 16) **

**Lecture 17 (Tuesday, March 20): Sparse convolutional coding**

Multi-layered convolutional sparse modeling.

**Lecture 18 (Thursday, March 22): Differentiable programming and adversarial examples**

Generic computation graph, DAG traversal, forward and reverse automatic differentiation, meta learning differentiable pipelines.

Adversarial examples, fast gradient sign method, Jacobian-based saliency map attack, universal adversarial perturbations.

**Lecture 19 (Tuesday, March 27): Probability, uncertainty, and bias in deep neural networks**

**Lecture 20 (Thursday, March 29): Deep meta learning (autoML) and differentiable programming**

**Lecture 21 (Tuesday, April 3): Neural networks on graphs**

**Lecture 22 (Thursday, April 5): Variational inference, BayesFlow in TensorFlow and Pyro in PyTorch**

Variational approximation and optimization, evidence lower bound (ELBO), stochastic variational inference algorithm.

**Lecture 23 (Tuesday, April 10): Wasserstein GANs, TensorFlow GAN library and PyTorch implementation**

Mathematical framework: Bregman divergence with convex functions.

**Lecture 24 (Thursday, April 12): Applications**

**Lecture 25 (Tuesday, April 17): Fair and explainable deep learning**

**Lecture 26 (Thursday, April 19): Projects fast forward**

**Lecture 27 (Tuesday, April 24): Project presentations**

**Game Playing**

7:10-7:20 AlphaZero Chess ♞ Implementation and Improvements

7:20-7:30 HUNL Poker Implementation ♣DeepStack vs. Libratus♠

7:30-7:40 Deep RL Agents for Starcraft II Mini-Games 🎮

**Graphics**

7:40-7:50 3D Object Synthesis using GANs I

7:50-8:00 3D Object Synthesis using GANs II

Break 🥤 🍕

**NLP**

8:10-8:20 MaskGAN Applications

8:20-8:30 RNN GANs

8:30-8:40 Argument Claim Identification

8:40-8:50 Improving Small Footprint Speech Keyword Spotting

**Neuroscience**

8:50-9:00 Neural Decoding of Retinal Ganglion Cell Responses

**Lecture 28 (Thursday, April 26): Project presentations**

**Vision GANs**

7:10-7:20 Progressive GANs

7:20-7:30 Image Synthesis using GANs I

7:30-7:40 Image Synthesis using GANs II

**Kaggle Competitions**

7:40-7:50 Google Landmark Recognition Challenge I

7:50-8:00 Google Landmark Recognition Challenge II

8:00-8:10 Kaggle 2018 Data Science Bowl

Break 🥤 🍕

**Colorization 🎨**

8:20-8:30 Colorization using Local Features

8:30-8:40 Video Colorization

8:40-8:50 Colorization using Attention

**Vision and Graphics**

8:50-9:00 Fine Grained Image Classification

9:00-9:10 Deep Game Synthesis 👾

**Last Day of Classes (Monday, April 30)**