Get started with SPE
Try Speedcode
Many people know how to write code, but few know how to write code that runs efficiently. Sure, you can use algorithms to achieve some efficiency, but there's a lot more to performance engineering than that!
Speedcode is an online platform where you learn SPE by practicing. With nothing to install, you can jump right in and start practicing SPE skills with dozens of example programming problems. Each problem is already solved by some initial code, and you are challenged to make it run faster. The problems are curated to exemplify important SPE skills (like parallelism, vectorization, caching, algorithms, and data structures), and everything runs in a environment that supports reliable performance measurements and parallel programming with OpenCilk.
Learning more about SPE
We are actively maintaining the following list of resources to help you learn SPE. If you have any questions, comments, or suggested additions, please let us know.
Introductory materials
- Introduction to Parallel Computing: This overview briefly introduces a broad range of topics and concludes with its own list of recommended reading. (By Livermore Computing)
- Parallel Computing for Beginners: This free on-line textbook is designed to teach early computing students the basics of parallel and distributed computing. (By Joel C. Adams, Richard A. Brown, Suzanne J. Matthews, and Elizabeth Shoop of CSinParallel)
Tutorials and interactive books
- Programming Parallel Computers: Open online version of the Aalto University course CS-E4580, which teaches how to write fast code for modern computers. (By Jukka Suomela)
- Performance Ninja Class: This 27-module curriculum focuses on low-level performance issues and requires basic C++ skills. (By Denis Bakhvalov)
- Intermediate Parallel and Distributed Computing: This sequel to Parallel Computing for Beginners introduces numerous patterns of parallel and distributed computation. (By Elizabeth Shoop)
- NERSC training materials: If you're looking for something specific (as opposed to following a curriculum), check out the archives of past training events hosted here. (For example, here is a 5-part series from 2023 about programming GPUs with the HIP parallel computing platform.)
- Parallel computing e-book: Johns Hopkins course EN 601.420/620, "Parallel computing for data science," includes a Jupyter-based e-book that you can install on your local machine.
Books
Principles and practice of writing fast code
- Performance Analysis and Tuning on Modern CPUs (2024) by Denis Bakhvalov
- Algorithms for Modern Hardware (2023) by Sergey Slotin
- The Art of Multiprocessor Programming (2021) by Maurice Herlihy, Nir Shavit, Victor Luchangco, and Michael Spear
- Programming Models for Parallel Computing (2015) by Pavan Balaji (ed)
- Performance Solutions (2001) by Connie Smith
Parallel programming
- Parallel Programming for Science and Engineering (2024) by Victor Eijkhout. This book is Vol. 2 of The Art of HPC.
- Parallel C++: Efficient and Scalable High-Performance Parallel Programming Using HPX (2024) by Patrick Diehl, Steven R. Brandt, and Hartmut Kaiser
- Structured Parallel Programming: Patterns for Efficient Computation (2012) by Michael McCool, James Reinders, and Arch Robison
- Patterns of Parallel Programming with .NET (2010) by Stephen Toub
- Patterns for Parallel Programming (2004) by Timothy Mattson, Beverly Sanders, and Berna Massingill
- Introduction to Parallel Computing (2003) by by Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar
- Our Pattern Language by UC Berkeley
Classes on SPE
Check out this list of classes on SPE.