COMP30690 Information Theory

This course provides an introduction to the fundamentals of Information Theory for computer scientists. Shannon's Information Theory is one of the greatest intellectual achievements of the 20th century. It provides a comprehensive view on the concept of information, whereby fundamental limits to the performance of practical algorithms are established. Information Theory concerns all areas where information is processed, stored or transmitted in some way or another. Consequently, it is an essential component in the education of a computer scientist.

Show/hide contentOpenClose All

Curricular information is subject to change

Learning Outcomes:

- Understand the general relevance of Shannon's Information Theory in the Information Age.
- Review essential probability theory.
- Become acquainted with fundamental information-theoretical concepts such as entropy, mutual information, relative entropy: Jensen's inequality, log-sum inequality, data-processing inequality, sufficient statistics, Fano's inequality.
- Understand the centrality of the asymptotic equipartition property in Information Theory: typical set.
- Understand the fundamentals of data compression: Kraft inequality, optimal codes, Huffman codes, Shannon-Fano-Elias coding.
- Understand the concept of channel capacity: symmetric channels, channel coding theorem, elementary channel coding techniques (repetition, Hamming codes).
- Acquire the basic insights into the connection between Information Theory and statistics.

Indicative Module Content:

(see above)

Student Effort Hours:
Student Effort Type Hours
Autonomous Student Learning

96

Lectures

24

Tutorial

16

Total

136

Approaches to Teaching and Learning:
Lectures; Reflective learning; Problem-based learning

Requirements, Exclusions and Recommendations
Learning Requirements:

Working knowledge of basic calculus and algebra.

Learning Recommendations:

Knowledge of probability theory would be helpful, although the course is self-contained in this respect.

Module Requisites and Incompatibles
Not applicable to this module.

Assessment Strategy
Description Timing Open Book Exam Component Scale Must Pass Component % of Final Grade
Assignment: Two sets of exercises and questions to be completed individually by the students. Throughout the Trimester n/a Alternative linear conversion grade scale 40% No

40

Examination: End of term examination 2 hour End of Trimester Exam No Alternative linear conversion grade scale 40% No

60

Carry forward of passed components
Yes

Resit In Terminal Exam
Spring No
Feedback Strategy/Strategies

• Feedback individually to students, post-assessment

How will my Feedback be Delivered?

Not yet recorded.

Name Role
Yogesh Bansal Tutor
Timetabling information is displayed only for guidance purposes, relates to the current Academic Year only and is subject to change.

Autumn

Lecture Offering 1 Week(s) - Autumn: All Weeks Thurs 12:00 - 12:50
Practical Offering 1 Week(s) - Autumn: All Weeks Thurs 13:00 - 13:50
Practical Offering 1 Week(s) - 1, 3, 4, 6 Tues 14:00 - 15:50
Practical Offering 1 Week(s) - 10 Tues 14:00 - 15:50
Practical Offering 1 Week(s) - 12 Tues 14:00 - 15:50
Practical Offering 1 Week(s) - 2 Tues 14:00 - 15:50
Practical Offering 1 Week(s) - 5, 7, 9, 11 Tues 14:00 - 15:50
Practical Offering 1 Week(s) - 8 Tues 14:00 - 15:50
Autumn