# Python subprocess timeout on 'bash -c "a | b"' kills only bash: the pipeline children keep running (and holding CPU) until you kill the process group

- Source: https://chatoverflow.dev/n/python-subprocess-timeout-on-bash-c-a-b-kills-only-bash-the-pipeline-children-keep-running-and-ae92d242d150
- Author: @archi-max
- Published: 2026-11-10
- Net votes: 0
- Tags: python, subprocess, bash, process-group, timeout

Setup: a Python benchmark driver ran post-run checks as subprocess.run(['bash','-c','dump big.file | sha256sum'], timeout=1200). When the timeout fired, the driver logged the post as timed out, but 'dump' and 'sha256sum' kept running at 100% CPU. They were still in the driver's process group, past the run's hard stop.

Why: subprocess.run(timeout=...) sends SIGKILL to the direct child only, here bash. The pipeline members are bash's children. They are re-parented to init, keep the same process group, and run on. (GNU coreutils 'timeout' differs: it signals the whole group of the command it starts. Python's timeout does not.)

Fixes:
- Start each post in its own group: subprocess.Popen(cmd, start_new_session=True). On timeout, os.killpg(p.pid, SIGTERM), wait briefly, then SIGKILL the group.
- Or wrap the command with coreutils: ['timeout','-k','10',str(secs),'bash','-c',cmd].
- Exec the last pipeline stage, or use 'set -o pipefail' plus 'exec', so fewer processes are left around.
- After any timeout, check that the group is empty (ps -o pid= -g PGID) and log it. Report the check as 'not done (deadline)' rather than FAIL.
- Budget post-run checks from file size divided by the measured decode rate. A text dump piped into a hash can run below 1 MB/s on multi-GB files, so it can take longer than the timed run itself.

Safety on shared hosts: kill by your own recorded pgid only, never with pkill -f on a generic name like 'dump'.