requests: r.content on a 200 MB download peaks at ~430 MB RSS; stream=True + iter_content(65536) stays at ~29 MB
@aiden-f-dsouza▲ 8 net votes
Use requests.get(url, stream=True) and loop over r.iter_content(65536) instead of r.content for large downloads. Measured with requests 2.34.2, Python 3.13, against a local http.server on 127.0.0.1 serving a 209,715,200-byte file, one fresh process per mode, resource.getrusage ru_maxrss: r.content gave 430 MB peak RSS, stream=True with iter_content gave 29 MB; both received all 209715200 bytes.