Explore UCD

UCD Home >

COMP47650

Academic Year 2024/2025

Deep Learning (COMP47650)

Subject:
Computer Science
College:
Science
School:
Computer Science
Level:
4 (Masters)
Credits:
5
Module Coordinator:
Assoc Professor Guenole Silvestre
Trimester:
Spring
Mode of Delivery:
On Campus
Internship Module:
No
How will I be graded?
Letter grades

Curricular information is subject to change.

Recent advances in machine learning have been dominated by neural network approaches broadly described as deep learning. This module provides an overview of the most important deep learning techniques covering both theoretical foundations and practical applications. The applications focus on problems in image understanding and language modelling and use appropriate state-of-the-art deep learning libraries and tools which are introduced in the module.

Prerequisites: Machine Learning (or similar), strong programming ability (Python or similar languages), strong mathematical ability (especially linear algebra and differential calculus, optimisation)

About this Module

Learning Outcomes:

On completion of this module students should be able to:
– understand what deep learning means and differentiate it from other approaches to machine learning
– understand loss functions, optimisation and the gradient descent algorithm
– understand the backpropagation of error algorithm in detail and how it is used to train deep neural networks
– distinguish between the most important neural network architectures (e.g. feed forward networks, convolutional neural networks, recurrent neural networks)
– apply appropriate deep learning techniques (e.g. convolutional neural networks, generative adversarial networks...) to image understanding problems (e.g. classification, segmentation, and generation) using open source deep learning frameworks (e.g. TensorFlow, PyTorch)
– apply appropriate deep learning techniques (e.g. dimensional embedding, recurrent neural networks, long short term memory networks, and gated recurrent units) to language modelling problems (e.g. classification, machine translation, and generation) using open source deep learning frameworks (e.g. TensorFlow, PyTorch)

Indicative Module Content:

Introduction to DL

Neural Network Fundamentals:
- Learning task, experience and performance. Types of task. Generalisation. Basic neurons. Perceptron.
- MLP and gradient descent. Backpropagation and Computational Graphs. Softmax and relative entropy.
- Softmax and ReLU, Overfitting and ways to deal with it, Speeding up gradient descent. Hyperparameter tuning.

CNN and Case Studies:
- Introduction to Image Understanding, Classification, Convolution, Stride and Padding, Pooling
- CNN,  Regularisation, Dropout, Batch Norm
- AlexNet, VGG, GoogLeNet (inception module), ResNet (residual module).
- Introduction to DL libraries
- 1D and 3D CNN generalisation, Transfer learning, Segmentation and Detection Applications

RNN, Sequence Models and Case Studies:
- Introduction to Language Modelling, Dimensional Embeddings (word2vec),
- RNN, Backprop through Time, LSTM and GRU, Sequence-to-Sequence Models
- Applications to Machine Translation, Sentiment Analysis,
- Utilising DL frameworks for Language Modelling, Attention models

Generative AI:
- GAN
- Variational Auto-Encoders
- Diffusion Models
- Transformers


Tutorials:
- DL demo
- Logistic Regression, Jupyter Notebook
- Single Layer Feedforward Network, Jupyter Notebook
- Deep Feedforward Network, Jupyter Notebook
- Gradient Checking, Parameter Initialisation, Jupyter Notebook
- L2 Regularisation, Dropout, Jupyter Notebook
- Optimisers (mini-batch gradient descent, momentum, adam), Jupyter Notebook
- CNN, Jupyter Notebook

- Reinforcement learning demos

Student Effort Hours:
Student Effort Type Hours
Autonomous Student Learning

80

Lectures

24

Tutorial

12

Total

116


Approaches to Teaching and Learning:
A set of lectures discussing recent developments of Deep Learning associated with a number of interactive tutorials with an emphasis on practical implementation.

Requirements, Exclusions and Recommendations

Not applicable to this module.


Module Requisites and Incompatibles
Incompatibles:
COMP47590 - Advanced Machine Learning


 

Assessment Strategy
Description Timing Component Scale Must Pass Component % of Final Grade In Module Component Repeat Offered
Individual Project: This DL individual project will examine your ability to apply deep learning learning architectures and address challenges relating to complex, real world problems. Week 15 Graded Yes
80
Yes
Participation in Learning Activities: Engagement in weekly practicals and lectures. Also involving timely submission of tutorial notebooks and participation in class discussions Week 15 Graded No
20
No

Carry forward of passed components
No
 

Resit In Terminal Exam
Autumn No
Please see Student Jargon Buster for more information about remediation types and timing. 

Feedback Strategy/Strategies

• Group/class feedback, post-assessment

How will my Feedback be Delivered?

Solutions to assignments will be derived as a collaborative exercise at the weekly tutorials

Timetabling information is displayed only for guidance purposes, relates to the current Academic Year only and is subject to change.
Spring Lecture Offering 1 Week(s) - 20, 21, 23, 24, 25, 26, 29, 30, 31, 32 Mon 09:00 - 10:50
Spring Tutorial Offering 1 Week(s) - 21, 22, 23, 24, 25, 26, 29, 30, 31, 32, 33 Wed 13:00 - 13:50