Parallel & HPC

Underworld can precondition a Stokes solve with geometric multigrid built from a real hierarchy of meshes, rather than with the algebraic multigrid it falls back to. What the difference is, how to build a mesh that has a hierarchy, and when the choice is worth making.

There are three ways to put resolution where a calculation needs it: rebuild the mesh, subdivide it, or move its nodes. The third conserves the node budget and leaves the connectivity, the parallel partition and the multigrid hierarchy untouched — nothing is created or destroyed, only relocated. How that redistribution works in Underworld3, what it is worth in numbers, and where its ceiling is.

A notebook written against one version of Underworld should still run against that version in five years, on a machine that has nothing installed. Here is the machinery that makes that true: a compact container, a nearly empty launcher repository, and two GitHub workflows that make a release and its launcher in the same breath.

It can be difficult to figure out where you are in an unstructured mesh of triangles or tetrahedra; worse when the mesh is distributed in parallel. We have to solve this problem for particle-in-cell type codes such as Underworld3. This is how we do it.

Parallel computation puts many CPUs to work on solving a problem much more quickly than one CPU alone. But this only works if the tasks are carefully scheduled and the additional CPUs are not waiting around for something to do. How do we choose the right number of processors for a given problem ?

TL; DR: Underworld is now Singularity-enabled, making it easier and somewhat quicker to use compared to traditional HPC installs. We demonstrate results using over 10,000 CPUs and more than one billion unknowns for solving the Stokes equation with Underworld 2.16

To test scalability we run weak scaling tests on various HPC machines to check the numerical framework remains robust when pushing for higher fidelity models.