OpenFOAM in Docker on a cloud VM: mpirun -np 8 fails on 8 vCPUs (slots = physical cores); hyperthreads add little

Setup: Google Cloud c2-standard-8 (8 vCPUs = 4 physical cores x 2 threads), OpenFOAM 8 in the openfoam/openfoam8-paraview56 Docker image, run as root.

Symptom: mpirun --allow-run-as-root -np 8 multiphaseInterFoam -parallel exits at once with "There are not enough slots available in the system to satisfy the 8 slots". -np 4 works.

Cause: Open MPI counts slots as physical cores by default, and a cloud vCPU is a hyperthread.

Fix: add --use-hwthread-cpus (or --oversubscribe). Also run the container with --ipc=host (or a large --shm-size), because Docker gives only 64 MB of /dev/shm, which Open MPI uses for on-node messages.

Whether it is worth it: on a 3-phase VOF case (multiphaseInterFoam, 728k cells, fixed time step) the time per step was 6.9 s on 1 rank, 3.6 s on 2 (97% efficient), 1.95 s on 4 (89%) and 1.67 s on 8 hyperthreads (only 17% better than 4). Peak memory was about 1.5 GB for 8 ranks. So size cloud machines by physical cores: 8 vCPUs behave like about 4.7 cores for this solver. Single-threaded Python or numpy jobs behaved the same way: 4 independent cases ran in the time of 1, 8 cases took 1.8 times as long as 4.

Measuring peak memory of a container: a polling loop must wait for the container to exist first (until docker ps -q -f name=X | grep -q .; do sleep 1; done), then sample docker stats --no-stream --format {{.MemUsage}} until it exits. Polling before the container starts ends the loop immediately and records nothing.