Iowa State Accelerates Research with New HPC Cluster

Nov 15, 2018
A new $1.6 million High Performance Computing (HPC) cluster, called NOVA, was recently installed in Iowa State University’s Durham Data Center to strengthen and support computing-based research and education. 


Over the last 12 years, there have been five major grants used to purchase computing machinery made up of many computing engines — also called “cores” — which work together to solve problems faster than a single computer. The first was a $1.2 million cluster called “CyBlue,” purchased in 2006-07 with the assistance of a $600,000 grant. It was followed by a $1.2 million cluster called “CyStorm” in 2009-10, a $2.7 million cluster called “CyEnce” in 2012-13, and most recently a cluster called “Condo,” created in 2014-15 using internal funding to support research projects.


NOVA was purchased this year with a $678,000 National Science Foundation Major Research Infrastructure grant. Additional funding came from the university’s HPC Community Fund and the collaborative effort of Information Technology Services (ITS), the Office of the Vice President for Research, and the Engineering, Liberal Arts and Sciences, and Agriculture and Life Sciences colleges.


The new cluster was installed in the Durham Data Center Oct. 17 and 18, and was purchased from Atipa Technologies, a division of MicroTech computers based in Lawrence, Kansas. The new cluster will aid in research ranging from weather modeling to data analytics for digital agriculture, and beyond.


According to High Performance and Research Computing Director Jim Coyle, NOVA has twice the computational power as existing university clusters, along with more memory. The new cluster is equipped with non-volatile memory express (NVMe) local drives, which will be used for parallel “scratch space,” a form of temporary memory computers employ when there is not enough random-access memory (RAM). The scratch space will provide fast access to frequently accessed data, and the NVMe local drives can deliver 1,000 times the disk read and writes per second compared to a traditional rotating hard drive, Coyle said.


“NOVA is significantly more powerful than anything we’ve ever had,” Associate Dean for Finance and Operation Arne Hallam said. “With the National Science Foundation grant and the money from various university sources, we were able to buy something bigger and better than any of us could have gotten alone.”


                                    A technician installs a portion of Iowa State University's new High Performance Computing (HPC) cluster, NOVA.
                                    Photo by Brandon Hallmark/Information Technology Services

Iowa State’s HPC clusters allow researchers to conduct work in a timely manner when dealing with large amounts of data. Some university research processes 600 terabytes of data, which CyEnce is equipped to handle. Condo can hold 1 petabyte (1,000 terabytes), and NOVA has 1.3 petabytes. An average computer has between 250 to 500 gigabytes of disk space. In other words, NOVA has 1,000 to 2,000 times more disk space than most desktops or laptops. 


“Many researchers need HPC, but they can’t afford large machines by themselves,” said Associate Dean for Research for the College of Engineering Arun Somani, who served as the principal investigator for the last three grants used to purchase HPC clusters. “NOVA allows us to pool together. The sharing culture we’ve created lets researchers develop and verify better models, conduct bigger research in multiple domains, and compete nationally.”


To improve the speed the new cores can work at in parallel, the interconnect speed was increased to 100 gigabits per second – equivalent to about two and a half DVDs' worth of information processed every second. When cores work in parallel, individual cores work on different parts of the same task or different calculations at the same time. For example, when sifting through genetic codes, each core can be given a different portion to look at, and each works on its respective piece at the same time. When the process is complete, the information is brought back together.


“Currently, the teams we collaborate with are working to functionally annotate livestock genomes,” Associate Vice President for Research Jim Reecy said. “We’ve asked a lot of biological questions about a lot of species, and we couldn’t realistically sift through all the data in a timely manner without the HPC clusters.”


With the addition of NOVA, Iowa State will now have 36 cores in each of its servers, compared to 16 in CyEnce. The increase will allow each node to store more information, which is important for fields like bioinformatics, which require significant amounts of computing power to store and use information.


Volta Graphics Processing Units (GPUs) were also added to the NOVA cluster. Originally used for 3-D modelling, GPUs can process larger operations more quickly than CPUs, the “brains” of computers. The Volta GPUs will be used for deep learning, a branch of machine learning where computers are taught to recognize objects, speech, and language.


NOVA was built with future expansion in mind, and when more power is needed it will be able to more than triple in size. 


"We like to say HPC clusters turn centuries of work into weeks," Coyle said. "The clusters allow researchers to run large jobs which would otherwise take a lifetime. We get a new computer cluster every two to three years to stay on the leading edge of technology and give researchers the tools they need to compete.”