Learning Outcomes:
By the end of the module the student should be able to:
- Describe the problem of machine learning from the point of view of function approximation, optimisation, linear algebra, and statistics.
- Identify the most suitable approach for a given machine learning problem.
- Analyse the performance of various machine learning algorithms from the point of view of computational complexity and statistical accuracy.
- Implement a simple neural network architecture and apply it to a pattern recognition task.
Indicative Module Content:
Material will cover the following mathematical topics relevant to Machine Learning:
Highlights from Linear Algebra
- Matrix multiplication
- The four fundamental subspaces
- Types of Matrix
- Matrix factorisation
- Eigenvalues and eigenvectors
- Symmetric, positive definite matrices
- Singular values and singular value decomposition
- Vector and matrix norms
Understanding data
- Linear regression
- Principal component analysis
Support vector machines
- Introduction to support vector machines
- Optimisation
- Lagrange multipliers
- Limitations
- Soft margins
- Kernel trick
- Multiple classes
Neural networks
- Introduction to neural networks
- Activation functions
- Architecture of a neural network
- Training a neural network
- Calculus on computational graphs
- Efficient matrix multiplication
- Backpropagation
- Universal approximation theorem
- Cross-entropy cost function
Optimisation
- Introduction to optimisation
- Gradient descent
- Newton's method
- Momentum
- Stochastic gradient descent
- Adaptive methods