A proxy server is an intermediary that makes requests on your behalf. You ask the proxy for something, the proxy fetches it, and the destination sees a connection from the proxy rather than from you. That single substitution is the entire mechanism, and almost everything else about proxies is a consequence of it.
#What actually happens on the wire
The exchange differs depending on whether you are requesting plain HTTP or HTTPS, and the difference matters more than most introductions admit.
#Plain HTTP
Your client sends the full request to the proxy, with the absolute URL on the request line. The proxy reads it, opens its own connection to the destination, reissues the request, and returns the response. The proxy sees everything: the path, your headers, and the body.
#HTTPS
Here the client first asks for a tunnel using the CONNECT method:
CONNECT api.example.com:443 HTTP/1.1
Host: api.example.com:443
Proxy-Authorization: Basic dXNlcjpwYXNz
HTTP/1.1 200 Connection establishedAfter that 200, the TLS handshake runs end to end between your client and the destination. The proxy relays encrypted bytes it cannot read.
| The proxy can see | The proxy cannot see |
|---|---|
| Destination hostname and port | The URL path and query string |
| Connection timing and duration | Request and response headers |
| Bytes transferred in each direction | Request and response bodies |
This is why “the proxy provider can see my traffic” is only half true, and why a provider can still tell you which domains an account contacted.
#What the destination sees
The destination records the address of the exit node — the last hop before it. This is not necessarily the address you configured. With a backconnect service you connect to one gateway, and the gateway routes each request through a different exit. The gateway’s location and the exit’s location are frequently different countries.
The only reliable way to know your exit address is to ask a service that echoes it back:
curl -x http://user:[email protected]:8000 \
https://api.ipify.org?format=jsonOne caveat that trips people up: on a rotating endpoint, that answer describes one request. The next request may leave from somewhere else entirely.
#The four types, and why the distinction exists
Proxies are categorised by who owns the exit address, because that is precisely what filtering systems check first. Every address belongs to an autonomous system identified by an ASN, and ASN ownership is public.
| Type | Address registered to | Typical billing | Main trade-off |
|---|---|---|---|
| Datacenter | A hosting company | Per address | Fastest and cheapest; trivially identified as non-residential |
| Residential | A consumer ISP | Per gigabyte | Blends in; slower and less predictable |
| ISP / static | A consumer ISP, hosted in a datacentre | Per address | Datacentre speed with residential registration; limited quantity |
| Mobile | A mobile carrier | Per gigabyte | Hardest to block; most expensive and highest latency |
The reason mobile addresses are treated so leniently is structural rather than technical: carriers place many subscribers behind few public addresses, so blocking one address blocks a crowd of real customers.
#Authentication
Two mechanisms dominate. Credential authentication sends a username and password, usually inline in the URL. IP whitelisting authorises your server’s address in advance, so no credentials travel at all.
If credentials are wrong you get 407 Proxy Authentication Required. This is worth internalising, because the failure mode is confusing: 407 comes from your proxy, 401 comes from the destination. Confusing them sends you debugging the wrong system entirely.
#What a proxy does not do
- It does not encrypt anything. HTTPS does that, with or without a proxy.
- It does not make you anonymous. Cookies, logins, browser fingerprints and TLS fingerprints all identify you independently of your address.
- It does not defeat modern bot detection on its own. The address is one signal among many, and often not the decisive one.
That last point is the most expensive misunderstanding in this field. A pristine residential address paired with a TLS handshake no real browser produces is still obviously automated — and arguably more suspicious than an honest datacenter request, because the combination is incoherent.
#Choosing where to start
Work down this list rather than starting at the expensive end:
- Does the target check the network origin at all? Many do not. Try datacenter first — it is cheapest and fastest.
- Do you need a stable identity across several requests? Use ISP, or residential with a sticky session.
- Are you being blocked despite a clean residential address? Investigate your client stack before buying a more expensive pool.
- Only reach for mobile when the cheaper categories have demonstrably failed.
Buying a more expensive proxy type to fix a client-side fingerprinting problem is the most common way to waste money here. Diagnose which layer is actually failing before you upgrade the layer that is not.
#Working through a real request
It helps to watch one happen. This shows the full exchange, including the tunnel setup:
curl -x http://user:[email protected]:8000 -v https://api.example.com/v1/items 2>&1 | head -25In the output you are looking for three things in order. First CONNECT api.example.com:443, which is your client asking for a tunnel. Then HTTP/1.1 200 Connection established, confirming the proxy opened it. Only then the TLS lines, because the handshake happens inside the tunnel. If you never see the 200, the proxy refused you and nothing after that matters.
#What each failure looks like
| Symptom | Almost always means |
|---|---|
No CONNECT line at all |
curl is not using the proxy — check for an http_proxy environment variable or a typo in the scheme |
407 instead of 200 Connection established |
Credentials wrong, or provider parameters malformed |
| Tunnel opens, then TLS fails | Interception on the path, or a scheme mismatch |
| Everything succeeds, empty body | You reached the destination and it chose to tell you nothing |
#Chaining and why people do it
Nothing prevents a request passing through more than one proxy. Each hop only knows its immediate neighbours, so the final destination sees only the last one. In practice chains are rare outside privacy tooling: every hop adds latency and another component that can fail, and providers already operate internal hops you cannot see. If you are chaining to solve a blocking problem, the cause is usually elsewhere.
#Where the address is not the answer
It is worth being concrete about the limits, because this is where budgets get wasted.
Suppose a request is refused. The instinct is to try a better proxy. But consider what the destination examined before it decided: the TLS handshake arrived first, before any HTTP data. If your client library handshakes like Python and your header claims Chrome, the refusal happened on that contradiction and the address was never the deciding factor. Moving from a datacenter address to a residential one changes a signal that was not the problem.
A useful test is to request an echo service through the same proxy at the same moment. If that succeeds while your real target refuses you, the proxy is healthy and the difference lives in what your client presents, not where it came from.
#Reading a provider’s product page
Vendor pages describe the same four types in different vocabulary. Some useful translations:
- “Backconnect” or “gateway” — a backconnect endpoint. You get one host, and the pool rotates behind it.
- “Static residential” — usually an ISP proxy: consumer registration, datacentre hosting.
- “Rotating residential” — residential with per-request or sticky rotation.
- “Unlimited” — check what is unlimited. Often bandwidth is uncapped while concurrency is tightly limited, which is the constraint that actually bounds your throughput.
#A short glossary of things that are not proxies
| Thing | How it differs |
|---|---|
| VPN | Moves all device traffic at OS level, usually encrypted end to end |
| Reverse proxy | Sits in front of a server, not a client |
| CDN | A reverse proxy network for caching and delivery |
| Tor | Multi-hop onion routing; slow, and widely blocked outright |
| Scraping API | A service that fetches on your behalf and returns parsed data; proxies are one component inside it |
#Where to go next
If you now want to make a request, using a proxy with curl is the shortest path to a working configuration. If you are choosing what to buy, the type comparison gives a decision procedure. If something is already failing, reading proxy and block errors will tell you which system rejected you.
#Questions people ask after the first week
#“My proxy works, so why is the data wrong?”
Because a proxy changes where the request comes from, not what the destination decides to show it. Large sites personalise heavily: currency, language, availability and price can all vary by perceived location, by account state, and by whether the visitor looks new or returning. A working proxy in Germany gets you the German view — which is what you asked for, and may not be what you assumed you were getting. If output looks wrong, capture the full response once and read it before changing anything.
#“Do I need a different proxy per thread?”
Not necessarily. A single backconnect gateway can serve many concurrent connections, and the provider distributes them across the pool. What you do need is a concurrency limit that matches your plan, and awareness that many simultaneous connections to one destination is a pattern in itself, regardless of how many addresses they arrive from.
#“Is it legal?”
Using a proxy is ordinary network engineering and is not itself unlawful in most places. What you do through it is a separate question, governed by the destination’s terms, by computer-misuse law, and — where personal data is involved — by data-protection law that applies regardless of how public a page is. That is a question for a lawyer in your jurisdiction, not for a technical guide.
#“Why does my provider’s country list not match what I get?”
Two reasons, both common. Availability in a country fluctuates, and some providers silently fall back to a nearby location rather than failing outright. And geolocation is inference: your lookup service and the destination’s may simply disagree about the same address. Verify the exit yourself, and treat city-level accuracy as approximate.