Learning Outcomes:
On completion of this module students should be able to:
– understand what deep learning means and differentiate it from other approaches to machine learning
– understand loss functions, optimisation and the gradient descent algorithm
– understand the backpropagation of error algorithm in detail and how it is used to train deep neural networks
– distinguish between the most important neural network architectures (e.g. feed forward networks, convolutional neural networks, recurrent neural networks)
– apply appropriate deep learning techniques (e.g. convolutional neural networks, generative adversarial networks...) to image understanding problems (e.g. classification, segmentation, and generation) using open source deep learning frameworks (e.g. TensorFlow, PyTorch)
– apply appropriate deep learning techniques (e.g. dimensional embedding, recurrent neural networks, long short term memory networks, and gated recurrent units) to language modelling problems (e.g. classification, machine translation, and generation) using open source deep learning frameworks (e.g. TensorFlow, PyTorch)
Indicative Module Content:
Introduction to DL
Neural Network Fundamentals:
- Learning task, experience and performance. Types of task. Generalisation. Basic neurons. Perceptron.
- MLP and gradient descent. Backpropagation and Computational Graphs. Softmax and relative entropy.
- Softmax and ReLU, Overfitting and ways to deal with it, Speeding up gradient descent. Hyperparameter tuning.
CNN and Case Studies:
- Introduction to Image Understanding, Classification, Convolution, Stride and Padding, Pooling
- CNN, Regularisation, Dropout, Batch Norm
- AlexNet, VGG, GoogLeNet (inception module), ResNet (residual module).
- Introduction to DL libraries
- 1D and 3D CNN generalisation, Transfer learning, Segmentation and Detection Applications
RNN, Sequence Models and Case Studies:
- Introduction to Language Modelling, Dimensional Embeddings (word2vec),
- RNN, Backprop through Time, LSTM and GRU, Sequence-to-Sequence Models
- Applications to Machine Translation, Sentiment Analysis,
- Utilising DL frameworks for Language Modelling, Attention models
Generative AI:
- GAN
- Variational Auto-Encoders
- Diffusion Models
- Transformers
Tutorials:
- DL demo
- Logistic Regression, Jupyter Notebook
- Single Layer Feedforward Network, Jupyter Notebook
- Deep Feedforward Network, Jupyter Notebook
- Gradient Checking, Parameter Initialisation, Jupyter Notebook
- L2 Regularisation, Dropout, Jupyter Notebook
- Optimisers (mini-batch gradient descent, momentum, adam), Jupyter Notebook
- CNN, Jupyter Notebook
- Reinforcement learning demos