COMP30770 Programming for Big Data

Academic Year 2022/2023

`Big Data' refers to datasets that are too big, or change too quickly, for traditional data management and data processing approaches. Big Data has forced the field of data management to rethink some of it design concepts and architectural patterns. This module will walk the students through the complex set of concepts and projects that form the Big Data stack. Students will learn how to set up Big Data environments, how to use efficient data management operations and how to run algorithms - to the scale and speed required by Big Data datasets. Students will also be able at the end of this module to design and implement their own solutions to address Big Data problems.

Show/hide contentOpenClose All

Curricular information is subject to change

Learning Outcomes:

On successful completion of this module the learner will be able to:
- Understand the problem of managing data at scale and why traditional data management systems are failing
- Understand the various data management paradigms used in the context of Big Data (e.g., relational, NoSQL)
- Understand the role of distributed file systems and how to manage your own cluster (e.g., using HDFS)
- Understand Big Data programming models such as Map/Reduce and Spark, and how to use them on real examples
- Understand how graph processing is done on big graphs (e.g., using Giraph)
- understand how to process big data streams (e.g., using Storm)

Student Effort Hours: 
Student Effort Type Hours
Lectures

18

Practical

18

Autonomous Student Learning

64

Total

100

Approaches to Teaching and Learning:
Lectures, projects, weekly quizzes 
Requirements, Exclusions and Recommendations

Not applicable to this module.


Module Requisites and Incompatibles
Not applicable to this module.
 
Assessment Strategy  
Description Timing Open Book Exam Component Scale Must Pass Component % of Final Grade
Continuous Assessment: < Description > Throughout the Trimester n/a Graded No

100


Carry forward of passed components
Yes
 
Resit In Terminal Exam
Summer Yes - 1 Hour
Please see Student Jargon Buster for more information about remediation types and timing. 
Feedback Strategy/Strategies

• Group/class feedback, post-assessment
• Online automated feedback

How will my Feedback be Delivered?

solutions to quizzes and in-class test will be provided, solutions to project and feedback on individual submissions will be provided