Unlock Cosmological Secrets at Durham University

Drawing on the power of a supercomputer from Dell Technologies and AMD, scientists at Durham University and DiRAC are expanding our understanding of the universe and its origins.

Everyone in scientific circles knows that greater computing power can lead to greater discoveries in less time. This is the case at the University of Durham in the United Kingdom, where researchers are opening up perspectives on our universe thanks to powerful high performance computing clusters from Dell Technologies.

What is the universe? What is it made of? What is dark matter? What is dark energy? These are the types of questions explored by distinguished researchers using the DiRAC Memory Intensive facility hosted by the Institute for Computational Cosmology at the University of Durham. This level of research requires enormous amounts of computing power, and Durham University achieves just that in its legendary COSMA supercomputer clusters.

The name COSMA, which is short for Cosmology Machine, reflects the main mission of this evolving system – the search for very large cosmological questions.

A specially designed machine

Durham University deployed the latest iteration of its COSMA supercomputer, known as COSMA8, in September 2020. During the development of specifications for COSMA8, Dr. Basden, COSMA technical director, spoke in particular interested in the AMD EPYC processor, due to its higher core density and increased memory bandwidth.

“We had access to the HPC and AI innovation lab at Dell Technologies in Austin so that we could do benchmarking on these processors,” says Dr Basden. Much of COSMA’s simulation work is done with software called SWIFT (SPH With Inter-Dependent Fine-Grained Tasking). Dr Basden’s team and DiRAC research software engineers used cosmological data samples with SWIFT to test the processors.

“It worked as we hope,” he says, noting that tests have been done on other processors. “The performance of a single core was comparable to that of other solutions, and when you have more cluster cores, it’s a given. The additional number of cores meant that the AMD EPYC processors were significantly faster, ”says Dr Basden.

For COSMA8, Dr. Basden opted for two 280 watt per node AMD EPYC 7H12 processors with a base clock frequency of 2.6 GHz and 64 cores, with four server nodes in a Dell EMC PowerEdge C Series chassis with a 2U form factor for up to 512 processing cores, up to 3200MT / s of memory speed to reduce latency and PCIe Gen 4 to transfer data faster.

“We wanted a lot of cores per node because that then meant we could reduce the amount of communication between the nodes,” says Dr. Basden. “But because some parts of the code aren’t 100% parallel, we also wanted high clock rates so that the parts of the code that were less parallelizable would hold up well, which meant the 7H12 processor would be the best option.”

Discover the secrets of the universe

With its rich mix of technologies, COSMA8 provides the robust, high-performance compute and storage resources scientists need as they attempt to unlock the secrets of a very large and complex universe. This work involves huge amounts of data which must be processed at high speed. A single simulation can produce hundreds of terabytes of data.

“About 75% of the universe is made up of dark matter that we don’t understand,” says Dr Basden. “By running these simulations, we can find out more. And, of course, when we do that, we start to understand better what the universe is made of. “

Ultimately, COSMA8 will allow a much higher level of discovery than the simulations run on previous generations of the COSMA supercomputer. Increased core density, greater amounts of faster DRAM, and faster PCI Express 4.0 connectivity all contribute to a powerful increase in performance.

“This means we can run much more detailed simulations, which we can compare much better with observations from telescopes,” says Dr Basden. “It will help us understand the meaning of the universe, dark matter, dark energy, and the formation of the universe. It will really help us deepen our fundamental understanding of the world we live in.

For a more in-depth look at Durham University’s world-class high-performance computing resources, visit COSMA and read Unlock cosmological secrets.