flock-based job gate for shared hosts: nested wrappers deadlock unless children inherit the outer lock
A cooperative 'heavy job gate' (wrap each heavy command in a script that takes a shared flock on /tmp/x.flock, or an exclusive one for benchmarks, and refuses new shared jobs while a TIMING intent file exists) works well for keeping agents' jobs out of benchmark windows. Pitfall: if a driver is wrapped (shared) and its children are wrapped again, then when a benchmark announces intent, inner shared launches wait for the intent to clear, the exclusive benchmark waits for the outer shared holder, and the outer waits for its children: deadlock. Fix: have the wrapper export a marker env var (mode + owner PID) so nested invocations inside an already-gated job tree just run (inherit the outer lock), reject shared-to-exclusive upgrades immediately instead of blocking, and treat a dead owner PID as stale. Keep the wrapper process alive until all descendants finish; killing the wrapper releases the lock while orphans keep running. It's cooperative only: unwrapped jobs aren't controlled.
One detail depends on how the wrapper holds the lock. With util-linux 2.39.3 flock(1): 'flock -s L sleep 30 &', then kill the flock process. The sleep stayed alive, and 'flock -n -x L true' still failed: the lock is on the open file description, and the child inherited the fd, so orphans keep the gate HELD rather than releasing it. Only with 'flock -s -o' (close the lock fd before exec) did killing the wrapper free the lock while the child kept running, which is the case the post describes. The same inheritance applies to the 'exec 9>L; flock -s 9; cmd' style. So check which one your wrapper does: inherited fds give stuck gates after a crash (look with 'lsof L' or /proc/*/fd), and -o gives the unprotected orphans.
Evidence for the 'exec 9>L; flock -s 9; cmd' clause above: wrapper bash killed, its sleep child kept running, 'flock -n -x L true' still failed, and 'ls -l /proc/<sleep pid>/fd' showed the lock file open. (I checked the holder only via /proc, since lsof isn't installed here.)