Big Data Group Project


​To provide experience of working and collaborating in a software development team (in the context of HPC and Big Data).

​To provide experience of all aspects of the declopment of a HPC solution to a Big Data problem.

​To prepare students for their individual end of programme project (COMP702).

To consolidate material from the first semester, specifically material from COMP528 and COMP529.


​​6 lectures will be given: (i) Introduction to the project framework; (ii) Overview of available projects, (iii) meeting skills and project planning; (iv) requirements and design specification; (v) group dynamics and negotiation; (vi) quality and configuration management; document testing; report writing.

Recommended Texts


​No specific texts but the following may be useful:

  1. Projects in Computing and Information Systems: A Student''s Guide. C.W. Dawson. Addison Wesley, 2005.

Learning Outcomes

​​​​An in depth and critical understanding of the operation of software development teams, the interpersonal skills required and the issues involved in working as part of a team.
Knowledge of how to specify and design HPC solutions to Big Data problems.

Knowledge of how to implement and test HPC solutions to Big Data problems​.

An overall understanding of the process of developing HPC solutions to Big Data problems. 

An ability to demonstrate practical experience in the realisation of HPC solutions to Big Data problems.

A critical understanding of the importance of software documentation and its creation.

Learning Strategy



​​​The overall learning and teaching strategy is to allow self and peer guided learning within a tightly defined framework. Projects will, in the first instance, be drawn up by members of staff responsible for the module working in collaboration with the Hartree super-computing centre at Daresbury (who will be providing access to their HPC facilities).​ The projects will be "artificial" in nature in that there is no expectation that commercially oriented projects will be undertaken (this will be done in the follow on individual project module, COMP702). The project outcome will be a HPC solution to a big data problem. 

At the beginning of the module a number of introductory lectures will outline details of the project scheme (documentation detailing the framework and expectations will be provided). Overviews of the available projects will also be presented. Thereafter a number of further lectures will be given reviewing the key high-level skills required for group working. Students will initially be put into teams of roughly equal size, typically with a minimum of three members, and thereafter will be expected to work largely autonomously. Teams will be expected to hold regular project meetings, the minutes of which will be monitored by staff.  

Each project will have four milestones:

  • Requirements specification review
  • Design specification review
  • Completed system demonstration
  • Project portfolio assessment

which will be reviewed and assessed by the members of staff responsible for the module.

Staff will be available on an ''as needed'' basis to offer support, guidance and to arbitrate any difficulties.