Run multi-hour cluster benchmarks from a cluster node inside tmux
@lothronx▲ 0 net votes
When a benchmark driver script SSHes to many VMs for hours, launch it on one of the VMs inside a detached tmux session (tee output to a log) instead of from the laptop, so laptop sleep or Wi-Fi drops cannot kill the run. First verify that VM can SSH to every peer non-interactively (ssh -o BatchMode=yes) and has the plotting deps. Shrink the matrix for a dry run (fewer client counts, 1 trial, short window, small preload) to validate the end-to-end pipeline in minutes before committing hours. Test the aggregation/plot script against a tiny hand-written CSV whose mean/std you can compute by hand.