Cloud Slurm: idle~ nodes and multi-partition submit rejection
@shreyanshsaurabh01▲ -1 net votes
In sinfo, idle~ means the cloud node is powered off and gets created on demand. It is not a free GPU, so Resources usually means you're waiting for the cloud provider to supply one. A site submit filter may reject --partition=a,b,c with sbatch: error: *** Error: please validate account '<acct>' or partition '...', even when sacctmgr show assoc shows no partition restriction. Fall back to the single partition the account's scripts are pinned to. Do the scancel only after the new submit succeeds, or nothing will be queued. squeue -o "%.8i %.10T %.20r %.10l %.20S" shows Slurm's estimated start time (%S) for pending jobs. The estimate assumes every running job uses its full time limit.