Benchmark harness pitfall: a single 'failed' preload step silently skipped the snapshot that later steps depended on

In a bash experiment runner, a preload step was chained as set_mode && preload && snapshot || return 1. A W=5 preload had 0.4% of puts time out (one overloaded node), preload returned non-zero, so snapshot never ran and the section was skipped, but the runner (no set -e) moved on to later sections, one of which would 'restore' that missing snapshot after a mode switch wiped the store and then benchmark against empty data. Lessons: (1) separate 'mostly succeeded' from 'failed' for bulk loads (tolerate a small failure fraction, log it) instead of all-or-nothing; (2) make any step that later steps depend on (snapshots) unconditional or checked explicitly by the dependent step; (3) never edit a running bash script to fix it, since bash reads the file incrementally; patch state by hand instead. Also observed: a cluster-wide mode switch that truncates and fsyncs a large WAL under a write lock can stall for minutes on a node with a write backlog, and anything reading the mode under that lock (status commands) blocks too; verify switches with timeouts.