Getting Started With HPC

Post by 
April 29, 2021
No additional tags.

igh-performance computing (HPC) is the ability to process data and perform complex calculations at high speeds. High-Performance Computing most generally refers to the practice of aggregating computing power in a way that delivers much higher performance than one could get out of a typical desktop computer or workstation in order to solve large problems in science, engineering, or business.

This week's ACM Select features several introductory resources on HPC, with perspectives on the history, technical definitions of HPC terminologies, trends in HPC, and the implications of its use.

As always, we invite you to share your feedback and suggestions at For more resources in computing, we encourage you to explore the ACM Digital Library and Learning Center.

What is HPC?

The articles in this subsection cover the basics of HPC, some motivating real-world use cases, and introduces supercomputers.

HPC 101

Published in the InsideHpc online Magazine

This article is a collection of links to other short articles covering very high-level concepts of HPC including what’s a cluster, HPC architecture for beginners, Computing: the processors, disks, and more, communications networks, and the cluster software. It is a good starting point for anyone who wants to know a high-level overview of HPC.

[Read More]

HPC Matters, and We Have Video Evidence 

Published in the Communications of the ACM November 17, 2014

HPC provides the means to support and advance research in a variety of fields. This reference we suggest is a collection of short videos intended to give the viewer a broad overview of some of the fields where HPC has an impact, ranging from advanced science to industrial R&D impacting our everyday lives.

[Read More


What are Supercomputers and Supercomputing

Published in XRDS: Crossroads, The ACM Magazine for Students June 2009 Article No.: 3

The next reading offers a historical perspective on the evolution of supercomputers, showing how their development has often been at the forefront of computer architecture innovation. This short paper also provides a timeline of significant milestones, each contributing to make HPC what it is today.

[Read More]


Programming Languages in HPC

The selected articles in this section cover most popular programming models of HPC, their tutorials and shed lights on their comparative benefits and application.

MPI Tutorial

Authored by Wes Kendall, Dwaraka Nath, and Wesley Bland

As you might have read in the HPC101 article, MPI (Message Passing Interface) is another fundamental set of APIs that application programmers can use to write parallel programs and run those programs on a cluster. This article walks through MPI programming with some very useful examples along with introductions and explanations.

[Read More]

[Further Reading]


Translating OpenMP Device Constructs to OpenCL Using Unnecessary Data Transfer Elimination

Published in SC '16: Proceedings of the International Conference for High-Performance Computing, Networking, Storage and Analysis November 2016 Article No.: 51 Pages 1–12

HPC goes beyond OpenMP and MPI: This article gives a sense of a few other programming languages out there used in HPC such as OpenCL, OpenACC, and CUDA. This paper proposes a framework that translates OpenMP 4.0 accelerator directives to OpenCL.

[Read More]

CUDA vs OpenACC: Performance Case Studies with Kernel Benchmarks and a Memory-Bound CFD Application

Published in CCGRID '13: Proceedings of the 13th IEEE/ACM International Symposium on Cluster, Cloud, and Grid Computing, May 2013 Pages 136–143

This is yet another example of different programming languages used in HPC and their performance implications. “OpenACC is a new accelerator programming interface that provides a set of OpenMP-like loop directives for the programming of accelerators in an implicit and portable way. It allows the programmer to express the offloading of data and computations to accelerators, such that the porting process for legacy CPU-based applications can be significantly simplified. This paper focuses on the performance aspects of OpenACC using two micro benchmarks and one real-world computational fluid dynamics application.” 

[Read More]

What are the applications? 

Published in ISC 2020 Digital Youtube channel, Jul 6, 2020

In this video, Rick Stevens from Argonne National Laboratory, University of Chicago gives an overview of some of the ongoing work in the US applying HPC and AI to COVID-19 related research problems. The first is the COVID-19 HPC consortium that joins US supercomputing centers, computing and technology vendors and federal agencies to provide HPC cycles to the SARS-CoV-2/COVID-19 research community. The second topic is the nine DOE laboratory collaboration formed to apply advanced computing to the problem of developing molecular therapeutics for COVID-19. The talk gives a brief overview of the science, the state of play and how HPC and AI are being used and progress towards solutions.

[Watch full talk]

“The convergence of AI and HPC provides the means to address big data challenges in science, engineering and industry, and enables the creation of disruptive approaches for data-driven discovery and innovation. Realizing these goals demands a concerted effort between AI practitioners, HPC and domain experts” --- Huerta et. al 

How does HPC relate to ML/Deep Learning? 

Convergence of artificial intelligence and high performance computing on NSF-supported cyberinfrastructure

Published in the Journal of Big Data volume 7, Article number: 88 (2020) ,16 October 202

In this survey paper, the authors discuss the convergence of AI and HPC and highlights the benefits of using HPC infrastructure in accelerating the pace of distributed model training. The article highlights software and hardware challenges and recommendations to streamline the use of HPC resources for AI research including support for containerization, up-to-date documentation and availability of distributed training software stacks. The article also briefly discusses the tradeoffs for using cloud computing and HPC for distributed training. 

[Read More]


Demystifying Parallel and Distributed Deep Learning: An In-depth Concurrency Analysis

Published in ACM Computing Surveys, August 2019 Article No.: 65

The following survey provides an introduction to the computational and performance aspects of Deep Learning from an HPC perspective. In addition to showing how most aspects of DL implementations already benefit from the pervasive application of HPC techniques, the modelling and analysis in the paper illustrate how to think about performance in a complex environment that has become central to our way of life.

[Read More]


HPC in the Cloud

HPC Cloud for Scientific and Business Applications: Taxonomy, Vision, and Research Challenges

Published in ACM Computing SurveysJanuary 2018 Article No.: 8

HPC is not only about supercomputers, and in recent years substantial progress has been made in running applications with substantial performance requirements on the Cloud. The next reading offers a survey of the various efforts in the field, from the feasibility of doing HPC on the Cloud, to how it affects the Cloud infrastructure, finally to how to make the Cloud a better environment for HPC applications. 

[Read More]


10 Years Later: Cloud Computing is Closing the Performance Gap

Published in ICPE '21: Companion of the ACM/SPEC International Conference on Performance Engineering, April 2021 Pages 41–48

 In this paper, researchers from Lawrence Berkeley National Laboratory try to answer the question of whether cloud computing can provide competitive performance for scientific applications. The authors conduct a series of experiments that include hardware and system microbenchmarks and user application proxies to identify performance gaps and demonstrate that today’s cloud HPC systems can deliver competitive performance for computationally intensive, memory-intensive and communication intensive scientific applications. 

[Read More]

THere's More

Recommended Selects

See all selects
Getting Started Series

Getting Started with Internet of Things: IoT Applications

This Selects finalizes with an example application domain of Industrial Internet ofThings (IIoT), and a source to delve into state-of-the-art IoT research trends.
Getting Started Series

Getting Started with Internet of Things: Computing and Communication

The selection includes easy to read articles describing and motivating the IoT, and later deep dives into the major aspects of IoT such as communication protocols, edge-to-cloud continuum, AI and data analytics, and security/privacy.
Computing in Practice Series

Trustworthy AI in Healthcare #02

AI needs to be trustworthy. Trustworthiness means that healthcare organizations, doctors, and patients should be able to rely on the AI solution as being lawful, ethical, and robust.

Help guide ACM Selects!

Let us know how we can improve your ACM Selects experiences, what topics you would like us to cover in the future, whether you would like to contribute and/or subscribe to our newsletter by emailing

We never share your info. View our Privacy Policy
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
continue learning with the acm digital library!
explore ACM DL