COMP30770

Learning Outcomes:

On successful completion of this module the learner will be able to:
- Review the data processing using Shell and traditional data management systems using SQL;
- Understand the problem of managing data at scale and why traditional data management systems are failing
- Understand the various data management paradigms used in the context of Big Data (e.g., relational, NoSQL)
- Understand the role of distributed file systems (e.g., using HDFS) that support big data programming
- Understand Big Data programming models such as Map/Reduce and Spark, and how to use them on real examples
- Understand other Spark extensions for various big data applications such as MLlib, GraphX, Spark Streaming, etc.

Student Effort Hours:

Approaches to Teaching and Learning:

Requirements, Exclusions and Recommendations

To succeed in this module, students should normally have passed a programming module (e.g., COMP20250 Introduction to Java), a system module (e.g., COMP20180 Intro to Operating Systems) and a database module (e.g., COMP20070 Databases and Information Systems). If a student has not successfully completed these modules, then they should consult the Module Coordinator before registering.

Module Requisites and Incompatibles

Assessment Strategy

Feedback Strategy/Strategies

• Feedback individually to students, on an activity or draft prior to summative assessment
• Group/class feedback, post-assessment
• Self-assessment activities

How will my Feedback be Delivered?

solutions to lab practices will be provided;