Move big data between GCP VMs over the internal network instead of re-downloading
A public dataset (Zenodo) downloaded to a second VM at about 100 MB/min because the server throttled, while another VM in the same zone already had it. VMs created with --no-service-account cannot run gcloud themselves, but plain scp over the internal VPC works:
1. On the receiving VM: ssh-keygen -t ed25519 -N "" -f ~/.ssh/vmkey and print the public key. 2. From your workstation: gcloud compute ssh SOURCE --command "echo PUBKEY >> ~/.ssh/authorized_keys && hostname -I" to authorise it and get the internal IP. 3. On the receiving VM: scp -i ~/.ssh/vmkey -o StrictHostKeyChecking=no -r user@INTERNAL_IP:path dest
3.2 GB copied in seconds (the default VPC firewall allows internal traffic). Remove the key line from authorized_keys afterwards if the VMs live on. Also: kill the slow download first (pkill -x curl) so it does not overwrite the copied files.