Show/hide contentOpenClose All
Curricular information is subject to change
This module will teach students to understand the inner workings and practical applications of large language models, as applied to generative and creative tasks. Students will be able to build their own working systems that exploit LLMs to generate novel content, and be able to use standard APIs for accessing very large LMs from third-party providers.
Indicative Module Content:Languages, formal and natural;
Conventional NLP vs. the "new" NLP;
Statistical language models (n-gram and neural);
Tokenization (byte-pair encoding; greedy tokenization; token "healing");
Vector representations and latent spaces;
Encoders, decoders and auto-encoders;
Distributed representations;
Positional encoding;
Contrastive Language/Image Pre-Training (CLIP);
Transformer architectures;
T5 (Text-to-Text Transfer Transformer) models;
Attention and self-attention;
Prompts and continuations;
Temperature settings and creative outputs;
"Hallucinations" and creativity in LLMs;
Poking under the hood (why do these models work?);
Explainable AI and LLMs;
Alignment of LLMS (and the alignment "tax")
RLHF in LLMS (Reinforcement Learning with Human Feedback);
Using APIs for LLMs;
Prompt engineering;
Chain-of-Thought (CoT) and other control mechanisms for LLMs
Student Effort Type | Hours |
---|---|
Autonomous Student Learning | 68 |
Lectures | 24 |
Laboratories | 24 |
Total | 116 |
Not applicable to this module.
Description | Timing | Component Scale | % of Final Grade | ||
---|---|---|---|---|---|
Assignment: Practical coursework related to generative AI and large language models | Throughout the Trimester | n/a | Graded | No | 10 |
Project: A substantial project on some aspect of generative AI and large language models | Coursework (End of Trimester) | n/a | Graded | No | 55 |
Assignment: Practical development assignment on the topic of generative AI with large language models | Throughout the Trimester | n/a | Graded | No | 10 |
Assignment: Practical development assignment on the topic of generative AI with large language models | Throughout the Trimester | n/a | Graded | No | 15 |
Continuous Assessment: Practical development assignment on the topic of generative AI with large language models | Throughout the Trimester | n/a | Graded | No | 10 |
Remediation Type | Remediation Timing |
---|---|
In-Module Resit | Prior to relevant Programme Exam Board |
• Feedback individually to students, post-assessment
Not yet recorded.
Lecture | Offering 1 | Week(s) - 20, 21, 23, 24, 25, 26, 29, 31, 32, 33 | Mon 16:00 - 16:50 |
Lecture | Offering 1 | Week(s) - 20, 21, 22, 23, 24, 25, 26, 29, 30, 31, 32, 33 | Thurs 16:00 - 16:50 |
Practical | Offering 1 | Week(s) - 20, 21, 22, 23, 24, 25, 26, 29, 30, 31, 32, 33 | Mon 17:00 - 18:50 |