A super computer

Thursday, April 17, 2008

Jane Nicholson, of the EPSRC, gives an overview of HECToR – a modern day hero for science.

There is no doubt that computational science and engineering has arrived firmly on the national and international science scene, and is here to stay for the long term. Computational science and engineering is concerned with predictive scientific modelling and simulation activity and draws heavily on computational infrastructure. Computational science and engineering is inherently interdisciplinary – its methods are applicable in a great many diverse domains – making it intellectually appealing and a subject with major strengths in its own right. It provides the third leg of modern scientific enquiry, alongside experiment and theory. Just as an experiment cannot always prove the level of detail, information or insight craved, so too the theoretical complexity of many systems denies us the ability to analyse their behaviour with pencil and paper alone. Computer-based simulation is the only way forward in both these situations.

As commented above, computational science and engineering contributes to a diverse number of domains.

For example, in climate science 'whole earth system' models are used to understand future scenarios that represent, in ever increasing realism and detail, our future climate. Oceanographic models are used to study, understand and predict ocean properties and can contribute to the development of economically relevant applications to inform policy and develop services for government and industry. In the field of engineering, computational applications are used particularly in modelling turbulence and flow associated with aircraft. For example, challenges in simulating a whole helicopter or looking at flows in turbine engines are just two of the areas being explored by UK engineers.

In energy research, simulation is currently used to model the properties of the plasma in magnetically confined fusion – a potential new power source of the future. Modelling and simulation is exploited to understand developments in the basic sciences such as physics and chemistry, where the ability to model the behaviour of atoms contributes to the wider understanding of fundamental principles. The life sciences also use modelling to understand the structure of proteins and contribute to the development of new drugs.

For UK academics to be able to tackle many of these challenging research problems, they need to have access to a variety of computational resources. These range from the provision of local computing within university departments, large mid-range research computers provided in some leading universities, and flagship supercomputers provided at a national level by Research Councils. These large supercomputers enable intense and complex calculations to take place, the results of which complement the work carried out within individual universities and research groups.

The Engineering and Physical Sciences Research Council (EPSRC) (and its predecessor SERC) has been providing access for academic researchers to supercomputers for the last 20 years, although the capabilities of these systems have dramatically changed in this period.

EPSRC is one of the seven UK Research Councils. Research Councils are independent non-departmental public bodies, funded by the Science Budget through the Department of Innovation, Universities and Skills (DIUS). They are incorporated by Royal Charter and together manage a research budget of over £2.8bn a year.

EPSRC acts as a managing agent for three of the Research Councils (EPSRC, BBSRC and NERC) in procuring high performance computing services for use by their respective research communities. Given the increasing complexity of modern supercomputers, this requires not only the procurement of the computing hardware itself and support for a service provider to operate the system, but also the provision of highly specialised expertise to enable the wider research community to maximise their output from the computing capabilities available.

The current strategy for support is to refresh the hardware every three to four years, so that researchers have access to up-to-date and competitive technology. The research community have been benefiting from the current HPCx service since 2002, when that service started. HPCx is based around hardware provided by IBM with the service and housed at the STFC Daresbury Laboratory in Cheshire. The service operation and computational science and engineering support are provided through a subsidiary company of the University of Edinburgh, who work in partnership with the STFC Daresbury Laboratory to provide the service. The HPCx service is scheduled to run for six years and has benefited from three upgrades in it lifetime. The last upgrade, in 2006, was to IBM Power 5 technology, which provides 12 teraflops of computing power for use by university-based researchers including research students and postdoctoral researchers.

However, to keep pace with the rapid pace of development in technology and to enable UK researchers to be competitive with researchers from the USA, Japan and the rest of Europe, a new service was identified as a priority. The HECToR project was aimed at procuring an up-to-date supercomputing service from academia to start in 2007. HECToR stands for High End Computing Teraflop of Resource and the goal was to provide researchers with a computational resource that was at least four times more powerful than the existing HPCx service. HECToR became available for early users in September 2007 and provides a computer which operates at over 50 teraflops. This means that it will be capable of over 50 thousand billion calculations per second. This is equivalent to the performance of about 14,000 desktop computers all operating at once. In addition, it will have 35,000 gigabytes of memory, and a storage capability of 700,000GBs to call upon – equivalent to the storage capacity of 175,000 modern day iPods.

These characteristics allow researchers with very large data sets to able to perform data intense calculation – for example, looking at different climate models that cover 100 years. HECToR uses the XT4 hardware provided by CRAY and is located in Edinburgh. The University of Edinburgh, through its company UoE HPCx Ltd, is again running this service, although this time, the computational science and engineering support to help the exploitation of all the processors and memory is being provided by the Numerical Algorithms Group Ltd – a not-for-profit company that was spun out from university-based work some 20 years ago. HECToR will provide leading edge computational capabilities for the academic community for the next six years. It will, however, change in that period to keep pace with developments overseas – in 2009, a further upgrade is planned and there is the option for another upgrade two years after that.

HECToR is among the top few supercomputers in Europe and will enable the UK research community to play its role in developing tomorrow's scientific challenges. The six year activity costs a total of £113m, so it is not surprising that consideration of a European partnership for the provision of future high end computing beyond 2010 is being looked at. The UK, through the EPSRC, is about to join a preparatory project with 13 other European countries to try and understand how a shared service could operate to the advantage of all partners. This project has won support from the European Commission and will work over the next two years to develop all the details needed to establish a European high end computing service for use by researchers from across Europe. This approach will promote our competitiveness with the USA and Japan, who have current projects to reach even higher levels of computational capability.

However, that is the future, today HECToR is already delivering UK firsts – for example, early users from Southampton University have been able to carry out the largest known direct numerical simulation of turbulent fluid flows in the UK. While not obvious from its name, turbulent fluid flow studies help researchers understand the flows across aerofoils, and, as such, is important for the aerospace industry. This is just one area that benefits from access to such intense computational power – other highlights will come as more researchers have a chance to exploit HECToR.

Future investment in services such as HPCx and HECToR will be influenced by an assessment of the quality and relevance of the science that they can help deliver – and, in particular, the contribution made to the Government's 10 year plan for science and innovation, and the focus on increasing the economic impact of UK investment in scientific research and the UK positioning in the international science scene.