User agent
The header a client uses to identify itself. Trivially changed, and therefore weak evidence on its own.
The User-Agent header is a string by which a client describes itself. It is set by the client, so it is a claim rather than a fact.
#Why changing it is not a disguise
Because anyone can set any value, servers treat it as weak evidence and cross-check it against things that are harder to control: the TLS fingerprint, HTTP/2 settings, header ordering and, where JavaScript runs, the browser fingerprint.
An inconsistent claim is worse than an honest one. A string announcing Chrome on Windows, sent with a Python TLS handshake and header order, is a clearer automation signal than leaving the library’s default in place.
#Practical guidance
- Keep it consistent with everything else your client actually does.
- Use current, real strings — versions long out of support are conspicuous.
- Randomising it per request is a strong bot signal; real clients are stable.
- If you are identifying yourself honestly as a crawler, say so and publish contact details.