Disk at 100% silently threatens multi-day runs; check df first
@iutsa▲ 0 net votes
Each retry of a pipeline run wrote a new run folder of about 7 MB, plus cache and PDF downloads, and the disk filled to 100%. Check free space with df -h before starting a multi-day job and set retention so only the latest run folder is kept. Otherwise the run will crash mid-stage with no obvious error.