Midway3 takes off, fueling computational discovery of all stripes at UChicago

New computing cluster is optimized for deep learning

High-performance computing, artificial intelligence, and data science is now embedded in almost every field of science and scholarship, so it’s critical for researchers to have access to the technology that best supports their work. The University of Chicago’s new campus high-performance computing cluster, called Midway3, goes into production this month—providing both power and the latest approaches for enabling discovery and innovation.

On Midway3, researchers will be able to more accurately simulate the structure of viruses to develop new drugs, analyze the activity of millions of neurons to better understand brain function, and model complex systems to predict the economic repercussions of climate change. In combination with the previous system Midway2, which will remain in operation, the UChicago Research Computing Center (RCC) provides computing time each year for campus users to enable research and scholarship.

The Midway3 builds upon two previous generations of Midway clusters at UChicago, offering faster and more powerful computation for the massive data and detailed simulations of today’s science. But the upgrade runs deeper than just the raw numbers, said Hakizumwami Birali Runesha, associate vice president for research computing and director of the Research Computing Center, rethinking the architecture of the machine to better reflect the demands of modern computational research.

“For the end user, Midway3 is built to address the convergence of artificial intelligence, deep learning and the traditional high-performance computing system,” Runesha said. “In the old days of high-performance computing, we used to think primarily about chemistry and physics, but the world now is really a whole different ball game. UChicago researchers are driving computational research in new directions, and we are very excited to offer a system that is competitive and provides the state-of-the-art for our users.”

Midway3 was built on an entirely overhauled architecture, reflecting updated trends in computer manufacturing and scientific applications. A computing cluster consists of thousands of cores, each roughly equivalent to a single computer, organized into nodes and working in tandem on complex or data-rich programs; Midway has more than 10,000 cores and 220 nodes.

The system’s 2.2 petabytes of high-performance storage also reflects the industry’s shift towards solid-state drives instead of the disc drives of the past. The faster read/write of solid-state enables data-intensive computation such as deep learning and performance improvement of analysis of large and small files.

Supporting the cluster with the cloud

Even as the Research Computing Center debuts Midway3, they are also looking beyond the “traditional” cluster to provide additional resources for UChicago faculty and students. Cloud computing—where tasks are performed offsite at a commercial data center instead of using local resources—has grown to a scale where customers can spin up the equivalent of a large computing cluster on demand. While cost, protection of sensitive data, and other limitations preclude the use of this method for all research computing, certain tasks that require short bursts of large-scale computing can benefit from cloud options.

So the Research Computing Center has also built Skyway, a new platform that allows researchers to seamlessly use commercial cloud resources the same way they use native Research Computing Center resources. They can access the same software and features of Midway but in the cloud. 

For users, the experience will be exactly the same whether they’re using Midway or Skyway, Runesha said, hiding the complexity of using cloud resources and putting the complicated management of resources under the hood, so they can concentrate on their research. Skyway expands the capacity of Research Computing Center resources so that users can burst their computing workloads in the cloud. 

Skyway joins other recent offerings, such as the Secure Data Enclave and the 37-megapixel, 24-foot-wide data visualization wall in the John Crerar Library Building in providing new opportunities for UChicago researchers to create novel kinds of partnerships and science. In 2021, the Research Computing Center also plans to commission Beagle3, the university’s high-performance computing system dedicated to biomedical sciences.

“All together, it will really be quite a comprehensive ecosystem that will help UChicago keep up with where we are heading in the future when it comes to data and computation,” Runesha said. “Science is interdisciplinary and collaborative, and computing has the ability to help faculty form strong collaborations and reach across field boundaries. We are really excited about the potential of Midway3 and other resources to enable these relationships and new scientific discoveries at the University of Chicago.”

For updates on Midway3 and other research projects facilitated by the RCC, register for their annual event Mind Bytes on March 30. The event will feature a keynote address from Margaret Martonosi, Assistant Director for Computer and Information Science and Engineering at the National Science Foundation, and a fireside chat with David Awschalom, director of the Chicago Quantum Exchange.

The RCC is also hosting a May 4 workshop to introduce Midway3. You can register for it here