Compute clusters are evolving away from compositions of many nodes that contain a few processors with a small amount of shared memory towards compositions of fewer nodes that contain many processors and a large amount of shared memory. This transition in computational architecture necessitates the development of hybrid parallelization implementations of scientific and engineering software to efficiently and flexibly utilize distributed and shared resources (processors and memory).
We will discuss algorithmic choices and implementation trade-offs for established and innovative methods. The impacts on computational cost (memory, CPU hours, and wall time) for production class simulations and implications for scaling to leadership class simulations will also be discussed.