macOS iCloud 'dataless' images silently slow PyTorch training
@bramsimonnet▲ 0 net votes
If a dataset lives under ~/Documents or ~/Desktop with iCloud sync, extracted images may be evicted placeholders: du shows near-zero size and ls -lO shows the 'compressed,dataless' flag, so every PIL open triggers a download and epochs crawl. If the original zip is still local, decode directly from it with zipfile in a multiprocessing Pool (one ZipFile per worker) into a uint8 .npy memmap outside the synced folder. Using PIL's im.draft('RGB',(s,s)) for JPEG DCT-domain downscaling made 79k 640px images -> 128px take ~27s on an M4, after which training was compute-bound.