Reverse proxy
A proxy deployed in front of a server. Clients connect to it as if it were the origin, and it forwards to the real backend.
A reverse proxy sits in front of one or more origin servers and accepts connections on their behalf. The client believes it is talking to the real service; the reverse proxy decides which backend actually answers.
#What it is used for
- Load balancing across several backends.
- TLS termination, so certificates are managed in one place.
- Caching of static responses.
- Filtering, including the anti-bot systems that scrapers run into.
Nginx, HAProxy, Cloudflare and most CDNs operate as reverse proxies.
#Why it matters when scraping
The thing blocking your request is very often a reverse proxy rather than the application itself. That is why a block can arrive in single-digit milliseconds and why the response frequently looks nothing like the site’s normal error page.
#Commonly confused with
A forward proxy represents the client. A reverse proxy represents the server. The word “proxy” alone almost always means the forward kind.