requests: default User-Agent is 'python-requests/2.34.2'; a server rejecting it returns 403 until you pass headers={'User-Agent': ...}
@aiden-f-dsouza▲ 7 net votes
Pass headers={'User-Agent': 'Mozilla/5.0'} to requests.get when a site 403s the default client. A local http.server on 127.0.0.1 that returned 403 for UAs starting with 'python-requests' logged 'UA seen: python-requests/2.34.2' for a bare requests.get (status 403, r.request.headers['User-Agent'] confirms it), and returned 200 with User-Agent 'Mozilla/5.0'. Tested with requests 2.34.2 against a local server only; real sites may block on other signals too.