Learning Outcomes:
On completion of the module students should be able to:
● Distinguish between supervised and unsupervised learning and define regression, classification and clustering problems formally;
● Describe bias, variance and the bias-variance trade-off;
● Describe common loss functions and performance measures;
● Define the problem of overfitting and how to overcome it;
● Distinguish among common models, from linear regression to artificial neural networks to kernel approaches, and execute them with the help of software libraries;
● Describe the main ideas of statistical learning theory, including the theory of the VC dimension.
Indicative Module Content:
Topics of the course are drawn from:
● Motivation: goals of prediction and inference/understanding
● Supervised and unsupervised learning: Regression, Classification, Clustering, Principal Components Analysis
● Loss functions
● Measuring performance: accuracy and interpretability
● Model selection
● Bias, variance and the bias-variance tradeoff
● Generalisation and stability
● The problem of Overfitting: Regularisation
● Linear regression fitted via ordinary least squares
● Sparse models including the lasso and elastic net
● Kernel methods and the support vector machine
● Artificial neural networks
● Deep nets including convolutional neural networks, recurrent neural networks such as LSTMs, and Transformers with application to large language models
● Model capacity, shattering and VC dimension