On this page 11 sections
- Sticky vs rotating at a glance
- Rotation happens per connection, not per request
- When rotation is right, and when it hurts
- How to request a sticky session
- A real failure: a new session ID per request in one Session
- Stickiness is best-effort, so detect when it ends
- Choosing a window length
- What rotation does not reset
- Matching the mode to the task
- Troubleshooting
- How this was tested
Use a rotating proxy when every request stands alone, and a sticky session when requests share state: a login, a basket, a paged search, a multi-step form. Rotating means each new connection leaves from a different exit address. Sticky means one exit is held for a window, usually requested with a session ID in the proxy username. The part that is easy to miss: a gateway can only change your exit when your client opens a new connection, and most HTTP clients keep one open. Ten requests through one requests.Session opened exactly one tunnel in our test. A “rotate every request” plan rotated zero times.
Sticky vs rotating at a glance
| Rotating | Sticky | |
|---|---|---|
| Exit address changes | On each new connection the gateway sees | Held until the window ends or the address leaves the pool |
| How you ask for it | Often the default on a rotating endpoint | A session ID in the username, or a dedicated port; syntax is provider-specific |
| Suits | Independent fetches with no state | Logins, baskets, paging, forms |
| What breaks | Anything that expects one visitor across requests | Nothing, until the session ends early |
| Footprint | Spread thinly over many addresses | Concentrated on one address |
Rotation happens per connection, not per request
For an https:// URL your client sends one CONNECT request to the proxy, then runs TLS straight through to the destination. RFC 9110 says that after a successful CONNECT the proxy restricts itself to “blind forwarding of data” until the tunnel closes. It cannot see where your first request ends and the second begins. So it cannot send request two out of a different exit. A new exit needs a new tunnel.
That turns connection reuse into the variable that decides how often you actually rotate. We counted tunnels with a local proxy that logs every CONNECT, sending 10 requests to the same HTTPS URL with each client pattern:
| Client pattern | Tunnels for 10 requests | Version |
|---|---|---|
| requests.get() in a loop | 10 | requests 2.32.3 |
| One requests.Session | 1 | requests 2.32.3 |
| One httpx.Client | 1 | httpx 0.27.2 |
| httpx.Client, keep-alive disabled | 10 | httpx 0.27.2 |
| A new httpx.Client per request | 10 | httpx 0.27.2 |
| One aiohttp ClientSession | 1 | aiohttp 3.10.5 |
| aiohttp with force_close=True | 10 | aiohttp 3.10.5 |
| curl, 10 separate runs | 10 | curl 8.7.1 |
| curl, 10 URLs in one run | 1 | curl 8.7.1 |
curl reports this itself. With ten URLs on one command line, -w '%{num_connects}' printed 1 for the first transfer and 0 for the other nine. Node behaves the same way: five requests through one shared undici ProxyAgent opened one tunnel, as measured in the Node.js guide.
Neither behaviour is a bug. Connection reuse saves a TCP and TLS handshake per request, which is the whole point of a pooled client. It only becomes a problem when you are paying for rotation and assuming you get it. If you want a fresh exit per request, ask for a fresh connection:
import httpx # tested with httpx 0.27.2
PROXY = "http://user:pass@gw.example.net:8000"
no_reuse = httpx.Limits(max_keepalive_connections=0)
with httpx.Client(proxy=PROXY, limits=no_reuse, timeout=20) as client:
for url in urls:
r = client.get(url) # each request opens its own CONNECT tunnel
The aiohttp equivalent is aiohttp.TCPConnector(force_close=True). With requests, call requests.get() without a Session. Each costs a full handshake per request, so only do it when rotation is the goal.
Plain http:// requests are different on the wire: the proxy reads every request, so a gateway could in principle route each one separately. Whether a given gateway does is provider behaviour we have not tested. For HTTPS there is no such option.
When rotation is right, and when it hurts
Rotate when each request stands alone: unrelated product pages, availability checks across many URLs, search samples from many regions. Nothing carries over between requests, so nothing breaks when the address changes. No single address builds up enough history at the destination to look unusual.
Now take a short, realistic sequence. Load a search page, submit a query, open result three, then go to page two. Four requests that a server expects from one person. Under rotation they arrive from four addresses, possibly four countries, inside a few seconds, all carrying the same session cookie. No genuine visitor does that. You have handed the site a clear automation signal that would not exist if you had held one address.
Anything with a login, a basket, a multi-page form or a server-side session needs stickiness. So does any flow where the site issues a token it later checks against the client.
How to request a sticky session
Many rotating gateways accept a session identifier inside the proxy username. A representative shape:
curl -x "http://user-country-de-session-abc123:pass@gw.example.net:8000" \
https://example.com/step-one
# Same identifier, same exit address
curl -x "http://user-country-de-session-abc123:pass@gw.example.net:8000" \
https://example.com/step-two
There is no standard here. Separators, field names and ordering differ between providers, and a malformed field usually comes back as a 407 from the proxy rather than a helpful message. Confirm the syntax in your provider’s documentation before building on it.
One detail matters for code: the username travels in the Proxy-Authorization header of the CONNECT request, once per tunnel. Our logging proxy recorded exactly one username per tunnel and nothing after it. The gateway decides the exit when the tunnel opens, based on the ID it saw then.
A real failure: a new session ID per request in one Session
A common pattern for “rotate with fresh sessions” is to keep one requests.Session and pass a new proxy URL with a new session ID on each call. It does produce a new tunnel each time: 10 distinct IDs gave 10 tunnels and 10 usernames in the proxy log. It also never lets go of the old ones.
requests keeps a pool per proxy URL in HTTPAdapter.proxy_manager, and every new username is a new URL. We ran this loop for up to 300 requests with the open-file limit at 256, which is what launchctl limit maxfiles reports as the soft limit on the Mac we tested on:
import uuid
import requests # requests 2.32.3, urllib3 2.3.0, CPython 3.9.6
GATEWAY = "gw.example.net:8000"
s = requests.Session()
for i in range(300):
proxy = f"http://user-session-{uuid.uuid4().hex[:8]}:pass@{GATEWAY}"
s.get("https://example.com/", proxies={"https": proxy}, timeout=15)
Instrumented to print the number of pools and open sockets to the proxy as it went, it produced:
after 10: proxy_managers=10 open_sockets=10
after 100: proxy_managers=100 open_sockets=100
after 240: proxy_managers=240 open_sockets=240
request 250 failed: ProxyError: HTTPSConnectionPool(host='example.com', port=443):
Max retries exceeded with url: / (Caused by ProxyError('Unable to connect to proxy',
NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x106710c10>:
Failed to establish a new connection: [Errno 24] Too many open files')))
after close: open_sockets 0
Every idle tunnel stayed open until s.close(). A higher file limit only postpones the error; the pools and idle connections to the gateway keep accumulating one per request, as the counts show. The error also names the proxy, which sends people off to debug a gateway that is working fine.
The fix is to give each sticky identity its own Session and close it when the flow ends:
import uuid
import requests # tested with requests 2.32.3
GATEWAY = "gw.example.net:8000"
def sticky_session(country="de"):
sid = uuid.uuid4().hex[:8]
proxy = f"http://user-country-{country}-session-{sid}:pass@{GATEWAY}"
s = requests.Session()
s.proxies = {"http": proxy, "https": proxy}
return s
with sticky_session() as s: # one identity: one exit, one cookie jar
s.get("https://example.com/?q=widgets", timeout=20)
s.get("https://example.com/?q=widgets&page=2", timeout=20)
# leaving the block closes the pool and its sockets
This also keeps the cookie jar scoped to the same lifetime as the address, which is the coherence the destination is checking for.
Stickiness is best-effort, so detect when it ends
A sticky window is an intention, not a guarantee. If the address behind your session leaves the pool, a residential peer going offline for example, the session ends early and the next request exits somewhere else. Code that assumes a stable address will eventually be wrong, usually in a way that shows up as intermittent blocks weeks later.
import httpx # tested with httpx 0.27.2
def exit_ip(client):
return client.get("https://api.ipify.org", timeout=10).text.strip()
with httpx.Client(proxy=STICKY_PROXY, timeout=20, follow_redirects=True) as c:
anchor = exit_ip(c)
for i, url in enumerate(steps, 1):
r = c.get(url)
if i % 3 == 0 and exit_ip(c) != anchor:
raise RuntimeError("exit address changed; restart the flow")
Know what this check proves. The IP echo is a different host, so it travels in its own tunnel: three requests to the target plus one to the echo service opened 2 tunnels in our log. With a session ID in STICKY_PROXY, both tunnels carry the same ID and the gateway is asked to map them to the same exit, so the comparison means something. On a plain rotating endpoint with no session ID, the echo can report a different exit from the one serving your target, and the check tells you nothing.
Checking on every step doubles your request count. Check periodically, and always just before the step where a broken session is expensive, such as a submission.
Choosing a window length
Long enough to finish the task, and no longer. While an address is held, every request adds to that one exit‘s history at the destination. A window far longer than the work concentrates exposure for no benefit. If a flow takes forty seconds, a one-minute window fits it. A thirty-minute window for the same flow just keeps you on one address after you stopped needing it.
If a task needs a long interactive session, ask whether it can be split into shorter flows that each start clean.
What rotation does not reset
- Account limits. If you are logged in, the rate limit follows the account, not the address.
- Fingerprints. Your TLS ClientHello and browser fingerprint are identical from every address you use.
- Cookies you keep sending. Rotating the address while replaying the same session cookie is a contradiction the server sees plainly. If you rotate, rotate the cookie jar too.
- Timing. Perfectly regular request intervals look mechanical from any origin.
A quick discriminator: does one clean first request succeed? If it does, your problem is accumulation and rotation is a real lever. If the very first request is refused, you are being identified on the handshake or on missing JavaScript, and no rotation setting will change that.
Matching the mode to the task
| Task | Mode | Reason |
|---|---|---|
| Fetching many unrelated pages | Rotating, new connection per request | No state to keep; spreads load |
| Paging through results | Sticky | The server treats paging as one visit |
| Anything behind a login | Sticky | An address change mid-session is a signal |
| Checkout or multi-step forms | Sticky | Tokens are often checked against the client |
| Price checks across countries | Rotating, geo-targeted | Each check is independent |
| Monitoring one URL repeatedly | Rotating | Avoids one address building history |
Troubleshooting
| Symptom | Likely cause | What to check |
|---|---|---|
| “Rotating” proxy returns the same exit for a whole run | Client reuses one connection | Count CONNECTs in a proxy log, or curl’s %{num_connects} |
| Login works, then the next page logs you out | Rotation mid-flow, or session ended early | Compare the exit address before and after the failing step |
| 407 as soon as a session ID is added | Username field syntax wrong for this provider | The provider’s username format; see 403 vs 407 vs 429 |
[Errno 24] Too many open files wrapped in a ProxyError |
A new proxy URL per request in one requests.Session | One Session per session ID, closed after use |
| 429s that do not improve with more rotation | Limit is per account or fingerprint | Slow down first; rotation second |
| 403 on the very first request | Identified by TLS fingerprint or missing JavaScript | Your client, not your rotation setting |
For working client configuration, see proxies in Python, using a proxy with curl and proxies in Node.js. If you have not settled which kind of address you need in the first place, residential vs datacenter proxies covers that decision.
How this was tested
All figures on this page come from one run on 21 October 2026 on macOS. A local forward proxy written for the test handled CONNECT, tunnelled to the real destination and logged one line per request it received, including the username in Proxy-Authorization. Each tunnel-count pattern sent 10 HTTPS requests to https://example.com/; all 10 returned 200 in every pattern. A tunnel was counted only when the proxy logged a CONNECT. A curl request with no credentials, which the proxy refused with 407, confirmed the log records attempts it rejects as well as ones it accepts.
The file-limit test ran once, with ulimit -n 256, and was stopped at the first failure. Versions: CPython 3.9.6 with requests 2.32.3, urllib3 2.3.0, httpx 0.27.2 and aiohttp 3.10.5; curl 8.7.1. Connection pooling defaults change between releases, so recount tunnels on your own versions before relying on any row.