Skip to content
Proxy fundamentals

What is a proxy server and how does it work?

A proxy server sends your requests from its own address. What it can read, what it adds to your headers, and the four types, shown with captured traffic.

On this page 10 sections
  1. How a request travels through a proxy
  2. Plain HTTP
  3. HTTPS
  4. What the proxy can see
  5. What the website sees
  6. Forward and reverse proxies
  7. HTTP proxies and SOCKS5 proxies
  8. The four types, and why the distinction exists
  9. Authentication
  10. What a proxy does not do
  11. Proxy, VPN, Tor and the rest
  12. Choosing where to start

A proxy server is a computer that makes requests on your behalf, from its own address. Your program connects to the proxy, the proxy connects to the website, and the website sees the proxy’s IP address instead of yours. For a plain-HTTP page the proxy reads your whole request and sends it on. For an HTTPS page it opens a tunnel and relays encrypted bytes it cannot read. That substitution of one address for another is the entire mechanism, and nearly everything else about proxies follows from it.

What a proxy reveals about you depends on how it is configured, and the defaults can surprise you. To show the difference rather than describe it, we pointed curl 8.23.0 at a raw listener posing as a proxy, at Squid 6.13 with its header settings left at their defaults, and at a SOCKS5 server we wrote to log what it receives, on macOS on 21 October 2026. The captures below are what each one actually saw.

How a request travels through a proxy

The exchange depends on whether the page is plain HTTP or HTTPS, and the difference decides what the proxy can read.

Plain HTTP

Your client sends the whole request to the proxy, with the full URL on the first line. This is exactly what arrived at our listener for curl -x http://user:pass@127.0.0.1:8098 'http://example.com/account/orders?id=12345':

GET http://example.com/account/orders?id=12345 HTTP/1.1
Host: example.com
Proxy-Authorization: Basic dXNlcjpwYXNz
User-Agent: curl/8.23.0
Accept: */*
Proxy-Connection: Keep-Alive

The path, the query string, every header and any body are in plain view. The proxy opens its own connection to the site, sends the request on, and passes the answer back.

HTTPS

For an HTTPS URL the client first asks the proxy for a tunnel with the CONNECT method, defined in RFC 9110 section 9.3.6. The same request, with https://, reached the listener as:

CONNECT example.com:443 HTTP/1.1
Host: example.com:443
Proxy-Authorization: Basic dXNlcjpwYXNz
User-Agent: curl/8.23.0
Proxy-Connection: Keep-Alive

No path, no query, no site headers. Once the proxy answers HTTP/1.1 200 Connection established, the TLS handshake runs end to end between your client and the site, inside the tunnel, and the proxy relays bytes it cannot decrypt.

Look at the Proxy-Authorization line in both captures. dXNlcjpwYXNz is base64 for user:pass, which anyone can decode. With an http:// proxy URL, your proxy password crosses the network in that form on every request, even when the site itself is HTTPS.

What the proxy can see

Plain HTTP through a proxy HTTPS through a proxy
Destination hostname and port Yes Yes, from the CONNECT line
URL path and query string Yes No
Request and response headers, cookies Yes No
Request and response bodies Yes No
Timing, duration and bytes each way Yes Yes
Your proxy username Yes Yes

Squid’s access log shows the same split. These are the two lines it wrote for the request above, plain HTTP first:

21/Oct/2026:22:30:37 +0300 127.0.0.1 TCP_MISS/404 GET http://example.com/account/orders? user 172.66.147.243
21/Oct/2026:22:30:37 +0300 127.0.0.1 TCP_TUNNEL/200 CONNECT example.com:443 user 172.66.147.243

The HTTPS line records only the host, the port, the username and the site’s address. The HTTP line has the path, but the query string is gone. That is a Squid default, strip_query_terms on, which drops query terms before logging. The proxy still received ?id=12345, because it had to forward it; it just chose not to write it down. A different proxy, or a different setting, keeps it. So “the proxy provider can see my traffic” is half true: over HTTPS it knows which sites an account contacted, when, and how much it transferred, but not what was on the pages.

What the website sees

The site records the address of the last hop, the exit node. With a backconnect service that is not the address you dialled: you connect to one gateway, and the gateway sends each request out through some other address, often in another country. The only reliable way to know the exit is to ask a service that echoes it back:

curl -sS -x http://gw.example.net:8000 -U user:pass https://api.ipify.org

On a rotating endpoint that answer describes one request. The next may leave from somewhere else.

The address is not always the only thing that reaches the site. We sent the same plain-HTTP request to a raw listener three ways and recorded the headers it received:

Direct
  GET /direct HTTP/1.1
  Host: 127.0.0.1:8098
  User-Agent: curl/8.23.0
  Accept: */*

Through Squid 6.13, default header settings
  GET /via-squid HTTP/1.1
  Host: 127.0.0.1:8098
  User-Agent: curl/8.23.0
  Accept: */*
  Via: 1.1 proxy-test (squid/6.13)
  X-Forwarded-For: 127.0.0.1
  Cache-Control: max-age=259200
  Connection: keep-alive

Through our SOCKS5 server (socks5h)
  GET /via-socks5h HTTP/1.1
  Host: 127.0.0.1:8098
  User-Agent: curl/8.23.0
  Accept: */*

Squid added two headers on its own. Via announces that a proxy, and which software, handled the request. X-Forwarded-For carries the address of the client that connected to the proxy. Here that was 127.0.0.1 because everything ran on one machine; on a real deployment it is your address, handed to every plain-HTTP site you visit. Both are Squid defaults, via on and forwarded_for on, and both are documented as such. The SOCKS5 request was byte-for-byte the same as the direct one: SOCKS relays a connection and never touches HTTP headers.

Over HTTPS neither header can be added, because the proxy cannot see inside the tunnel. For plain HTTP, whether a proxy adds them is a configuration choice, not a property of proxies, and a commercial gateway may or may not. The way to know is to request a header-echo endpoint through it and read what comes back. A proxy that the network imposes on you without any client setting is a different thing again, an interception or transparent proxy.

Forward and reverse proxies

Everything above describes a forward proxy: it sits in front of clients and makes requests for them. A reverse proxy sits in front of servers and receives requests for them, for caching, load balancing or filtering. The machinery is the same; the side of the connection it works for is opposite. A CDN is a large network of reverse proxies. When people say “proxy” in the context of scraping, IP addresses or privacy, they mean a forward proxy.

HTTP proxies and SOCKS5 proxies

HTTP proxy SOCKS5 proxy
Speaks HTTP, with CONNECT for anything encrypted Its own binary protocol (RFC 1928) for any TCP connection
Can read plain-HTTP traffic Yes, and may add headers Sees the bytes, but does not parse or change HTTP
Authentication Proxy-Authorization header Username and password sub-negotiation (RFC 1929)
Who resolves the hostname The proxy Your choice: the client sends a name or an IP
Typical client setting http://host:port socks5h://host:port

The last two rows hide a common mistake. Our SOCKS5 server logged what curl sent for the same HTTPS request with each spelling:

socks5://   CONNECT atyp=IPv4 target=104.20.23.154:443
socks5h://  CONNECT atyp=DOMAIN target=example.com:443

With socks5://, your machine looked up the hostname and sent the proxy a bare address, so your own DNS resolver saw every site you visited. That is a DNS leak, and on a site that answers DNS by region it can also send you to the wrong regional server. With socks5h:// the proxy did the lookup. Not every client treats the two spellings the same way; the per-library behaviour is in Python requests proxy setup and how to use curl with a proxy.

The four types, and why the distinction exists

Commercial proxies are sorted by who owns the exit address, because ownership is public and cheap for a filtering system to check. Every public address belongs to a network identified by an ASN, and the registry records of who holds which network are public.

Type Address registered to Typical billing Main trade-off
Datacenter A hosting company Per address Fastest and cheapest, and identifiable as non-residential from the ASN alone
Residential A consumer ISP Per gigabyte Looks like home users; slower and less predictable
ISP / static residential A consumer ISP, hosted in a datacenter Per address Datacenter speed with ISP registration; smaller supply
Mobile A mobile carrier Per gigabyte Hardest to block; most expensive and highest latency

Mobile addresses are treated leniently for a structural reason. Carriers put many subscribers behind a small number of public addresses using carrier-grade NAT, so blocking one address can block many real customers at once. How to choose between the four for a given job is covered in residential vs datacenter vs ISP vs mobile.

Authentication

Two methods dominate. Credential authentication sends a username and password, as in the captures above. IP whitelisting authorises your server’s address in advance, so no credentials travel at all, and stops working the moment your server’s address changes.

407 comes from your proxy, 401 from the site When the proxy rejects your credentials it answers 407 Proxy Authentication Required with a Proxy-Authenticate header, and the site is never contacted. A 401 means the request got through the proxy and the site wants its own login. Mixing them up sends you debugging the wrong system. Real 407 responses, and how they surface in curl and Python, are in 407 vs 403 vs 429.

What a proxy does not do

  • It does not encrypt anything. HTTPS does that, with or without a proxy. A plain-HTTP request through a proxy is readable by the proxy, as the first capture shows.
  • It does not make you anonymous. Cookies, logins, browser fingerprints and TLS fingerprints identify you whatever the address, and a default forwarding proxy may hand your address over in X-Forwarded-For.
  • It does not beat bot detection by itself. The address is one signal among several, and not always the deciding one.

The last point is where money gets wasted. The site sees the TLS handshake before any HTTP data, and our TLS ClientHello study found curl and Go keeping the same handshake across three connections each while the recovered Chromium summary reports 12 different orders across 12 connections. Five other library clients were sampled once, so their repeatability was not established. A clean residential address paired with a handshake no browser produces is still an automated client, and a browser User-Agent on top of it is a contradiction the site can see. The default request headers study shows the same gap in headers: 11 from a Chromium navigation, a median of 4 from nine libraries.

Proxy, VPN, Tor and the rest

Thing How it differs from a proxy
VPN Moves all of a device’s traffic at the operating-system level, usually inside an encrypted tunnel to the VPN server. A proxy is set per application.
Reverse proxy Works for a server, not for a client.
CDN A network of reverse proxies for caching and delivery.
Tor Several encrypted hops, each knowing only its neighbours. Slow, and blocked outright by many sites.
Scraping API A service that fetches pages for you and returns the result. Proxies are one component inside it.

Proxies can also be chained, each hop knowing only the one before and the one after it. Outside privacy tools this is rare: every hop adds latency and a new way to fail, and a backconnect gateway already adds a hop of its own between you and the exit.

Choosing where to start

  1. Does the site check the network origin at all? Not all do. Start with datacenter addresses, which are the cheapest and fastest.
  2. Do you need the same identity across several requests? Use ISP addresses, or residential with a sticky session. Rotating vs sticky sessions explains when.
  3. Blocked despite a clean residential address? Look at your client before you buy a more expensive pool.
  4. Reach for mobile only when the cheaper types have demonstrably failed.

Before any of that, check what your proxy actually sends. Request a header-echo service through it over plain HTTP, and if you see your own address in X-Forwarded-For, the site can see it too. Then make a first request: the curl guide above is the shortest route to a working configuration, and if something already fails, the 407 guide tells you which system refused you.

Frequently asked questions

Does a proxy encrypt my traffic?
No. A proxy changes the address your connection comes from. Encryption comes from HTTPS, which runs end to end between your client and the site whether or not a proxy is in the path. Plain HTTP through a proxy is readable by the proxy, including the full URL.
Can the proxy operator see what I am doing?
For plain HTTP, all of it: the full URL with its query string, the headers and the body. For HTTPS the proxy receives only the hostname and port in the CONNECT request, plus the timing and the number of bytes. That is enough to know which sites an account contacted.
Does a proxy hide my IP address from websites?
The site sees the proxy's exit address, but some proxies also forward yours. A default Squid added X-Forwarded-For with the client address to every plain-HTTP request in our test. Request a header-echo service through your proxy to check what it sends.
What is the difference between a proxy and a VPN?
A VPN carries all of a device's traffic at the operating-system level, usually encrypted to the VPN server. A proxy is set per application and only affects that application. Proxies are also easier to run as large pools of addresses, which is why scraping uses them.
What is the difference between an HTTP proxy and a SOCKS5 proxy?
An HTTP proxy speaks HTTP and can read and modify plain-HTTP requests. A SOCKS5 proxy relays any TCP connection without parsing HTTP. With SOCKS5, use socks5h:// so the proxy resolves hostnames; with socks5:// your own machine looks them up.
Is using a proxy legal?
Using a proxy is ordinary network engineering and not unlawful in itself in most places. What you do through it is a separate question, governed by the site's terms, computer-misuse law and, where personal data is involved, data-protection law. That is a question for a lawyer in your jurisdiction.

Sources

  1. RFC 9110 section 9.3.6: CONNECT
  2. RFC 9110 section 15.5.8: 407 Proxy Authentication Required
  3. RFC 9110 section 7.6.3: Via
  4. RFC 1928: SOCKS Protocol Version 5
  5. RFC 1929: Username/Password Authentication for SOCKS V5
  6. Squid configuration: forwarded_for (default on)
  7. Squid configuration: via (default on)
  8. Squid configuration: strip_query_terms (default on)

Read this page as Markdown · Quote it freely under CC BY 4.0 with a link back.