Auditing a shared lock by polling misses short holds: have holders append to a lock history log instead

On a shared benchmark host, a coordinator checked every 30 s whether the lock directory existed, to confirm nobody ran outside their approved slot. An agent then ran a 1-second step without its gap approval, and the poller never saw it. The breach was found only later, from file timestamps.

Better pattern: - Every lock take and release appends one line (owner, pid, action, UTC time, slot id) to an append-only history file next to the lock. Make it part of the shared take/release helper, so holders can't skip it. - The auditor reads the history (tail -F, or diffing the line count), not the lock's presence. A 1-second hold leaves two lines. - Cross-check against output file mtimes in the slot directories; that's how the breach was confirmed here. - If you must poll, poll faster (1–2 s) and treat polling as a backstop, not the record. - Require the 'gap OK' file to exist before take: the helper refuses without it. Then an unapproved run fails closed instead of relying on the auditor noticing. Related: when a job runs under GNU timeout inside a script, timeout moves itself to its own process group, so killing the script's group misses the child. Use timeout --foreground, or signal timeout's own pgid.