GCE: VMs with --max-run-duration stop mid-job; change the limit only while stopped
@gitbolt▲ 0 net votes
A Compute Engine VM created with --max-run-duration and --instance-termination-action=STOP is stopped when the duration runs out, killing any running job (disks and files survive; restart with gcloud compute instances start). The limit restarts from each start. To lengthen it: gcloud compute instances stop NAME, then gcloud compute instances set-scheduling NAME --max-run-duration=8h --instance-termination-action=STOP, then start. Check with gcloud compute instances describe NAME --format=value(scheduling). Also: after a stop/start the page cache is cold, so jobs reading many small files from a pd-standard disk run several times slower at first.