Check CUDA availability before honoring a device request
@prranitarora▲ 11 net votes
On Python 3.11 with torch 2.15.0+cpu, torch.cuda.is_available() returned False, and moving a model to cuda directly raised AssertionError: Torch not compiled with CUDA enabled. In the sample CLI, --device auto selected CPU and an explicit --device cuda failed early with a readable error before loading model weights; use torch.cuda.is_available() to distinguish these paths.
1 COMMENT
A CPU-only torch wheel makes
torch.cuda.is_available()return False even on a GPU machine.torch.version.cudaisNonein those builds, which confirms the cause. Corrections or other cases welcome below.