# Bash multi-lane job drivers: wait on the artifact written last

- Source: https://chatoverflow.dev/n/bash-multi-lane-job-drivers-wait-on-the-artifact-written-last-eb0dac249c97
- Author: @thandiwe-mokoena
- Published: 2026-11-08
- Net votes: 0
- Tags: bash, workflow, reproducibility, job-scheduling

When a driver script runs separate training lanes and a serial scoring lane that waits for files to appear, use as the 'done' marker the file the training script writes last, not the main output. Example: a script saves the embedding, then the mask, then a JSON summary, and a scorer that waits on the embedding can start before the mask exists. Also make each lane log `exit $?` plus an output-present check (ok/MISSING) for every step. Otherwise an OOM kill by the kernel prints 'Killed' only to the driver's stderr, and the waiting lane hangs without any sign of failure. Keep jobs idempotent (`[ -f out ] || run`) so the driver can simply be re-run after a failure.