Hard-stop watchdog: group TERM releases the lock while a TERM-ignoring child lives; drain the pgid in the EXIT trap
A bench harness held a lock directory for each timed slot and had a watchdog that, at a hard deadline, ran kill -TERM -- -$PGID on its own process group. The workload was a child that hung at exit and ignored SIGTERM. Three things went wrong:
1. timeout --foreground N cmd sends TERM only. A child that ignores TERM survives until something else kills it. Fix: timeout --foreground -k 15 N cmd.
2. The group TERM made the main script exit at once (its EXIT trap ran, rc 143) and release the lock, while the TERM-ignoring child was still alive. The next slot's owner could start against a live process.
3. A KILL sweep built on ps -eo pid,pgid | awk lists its own ps/awk helpers, which are in the same group.
Pattern that passed a container test (bash 5.2):
mine() { local d l p x=${1:-}; M=""; for d in /proc/[0-9]*; do read -r l < $d/stat 2>/dev/null || continue; l=${l##*) }; set -- $l; p=${d#/proc/} [ "$3" = "$MYPG" ] && [ "$1" != Z ] && [ "$p" != "$BASHPID" ] && [ "$p" != "$x" ] && M="$M $p"; done; } sweep() { local i p; for i in {1..15}; do mine "$1"; [ -z "$M" ] && return; sleep 1; done; mine "$1"; for p in $M; do [ "$p" = "$$" ] || kill -KILL $p; done; } # watchdog: trap "" TERM; kill -TERM -- -$MYPG; sweep "" (waits for the main shell too, never KILLs it) # main EXIT trap: kill $WD; trap "" TERM; sweep "$WD"; <release lock>
- After ${l##*) }, the fields are state, ppid, pgrp, so $3 is the pgid. Skipping state Z drops unreaped zombies.
- Only builtins run during the scan, so it never sees itself.
- With a stub child that ignores TERM: TERM, KILL after 15 s, lock released, 0 processes left. Before the change, 2 were left after DONE.