How to use a proxy with curl

curl is the fastest way to establish whether a proxy works at all, which is why it is the first thing to reach for when something breaks. This guide covers the flags that matter, the one trap that catches nearly everyone, and how to read the failures.

#The basic form

# HTTP proxy, no authentication
curl -x http://gateway.example:8000 https://api.example.com/v1/items

# With credentials inline
curl -x http://user:[email protected]:8000 https://api.example.com/v1/items

# Credentials in a separate flag, which keeps them out of the URL
curl -x http://gateway.example:8000 -U user:pass https://api.example.com/v1/items

-x is simply the short form of --proxy; they are identical. Separating credentials with -U is worth the habit, because a URL containing a password ends up in your shell history.

#Choosing the scheme

#The socks5h trap

This is the single most common configuration mistake, and it fails silently rather than loudly.

With socks5://, your machine performs the DNS lookup and sends a resolved IP address to the proxy. Two consequences follow. First, your DNS queries leak to whatever resolver you use locally. Second — and more damaging for geo-targeted work — you resolve the hostname from your location. Large sites return different addresses per region, so you can end up fetching the wrong regional endpoint while your proxy is dutifully connecting from the correct country.

# Leaks DNS, resolves locally — usually wrong
curl -x socks5://gateway.example:1080 https://example.com

# Resolves at the proxy — usually right
curl -x socks5h://gateway.example:1080 https://example.com

Unless you have a specific reason to resolve locally, use socks5h.

#Verifying the exit address

Never assume the proxy is doing what you configured. Ask something to echo the address back:

curl -x http://user:[email protected]:8000 \
     -s https://api.ipify.org?format=json

Compare that with your address without the proxy. If they match, the proxy is not being applied — check for an http_proxy environment variable overriding you, or a typo in the scheme.

Remember that on a rotating endpoint this reports one exit node at one moment, not a property of the pool.

#Measuring where the time goes

“The proxy is slow” is rarely actionable. Split the timing:

curl -x http://gateway.example:8000 -o /dev/null -s \
  -w "connect: %{time_connect}s\nappconnect: %{time_appconnect}s\ntotal: %{time_total}s\ncode: %{http_code}\n" \
  https://api.example.com/v1/items

A high time_connect points at the path to the proxy or the proxy’s own queueing. A high time_appconnect is the TLS handshake through the tunnel. A high time_total with both of the others low means the destination is slow, and the proxy is not your problem.

#Reading the errors

HTTP status codes matter just as much. A 407 is your proxy rejecting your credentials. A 401 is the destination rejecting you, which means the proxy worked. A 429 is rate limiting and means you are going too fast, not that your proxy is bad.

#Practical flags worth knowing

# Bypass the proxy for specific hosts
curl -x http://gateway.example:8000 --noproxy "localhost,127.0.0.1,.internal" https://api.example.com

# Send several requests over one connection
curl -x http://gateway.example:8000 https://example.com/a https://example.com/b

# Show the CONNECT exchange and response headers
curl -x http://gateway.example:8000 -v https://api.example.com 2>&1 | head -30

The verbose output is where you confirm the tunnel was actually established — look for Connection established before the TLS lines.

#A note on environment variables

curl honours http_proxy, https_proxy and no_proxy. These are a frequent source of confusion because they apply invisibly. If a request behaves unexpectedly, check them first:

env | grep -i proxy

An explicit -x overrides them, which makes -x the safer choice in scripts you want to be reproducible.

#Testing a list of proxies

A short loop is usually enough to separate working endpoints from dead ones. This reports the exit address and the timing for each, and marks failures clearly:

#!/bin/bash
# proxies.txt: one per line, host:port:user:pass
while IFS=: read -r host port user pass; do
  out=$(curl -sS --max-time 15 \
        -x "http://$user:$pass@$host:$port" \
        -w '%{http_code} %{time_total}' \
        -o /tmp/body https://api.ipify.org 2>/dev/null)

  if [ $? -ne 0 ]; then
    printf '%-28s FAILED\n' "$host:$port"
  else
    printf '%-28s %s exit=%s\n' "$host:$port" "$out" "$(cat /tmp/body)"
  fi
done < proxies.txt

Two details make this more useful than it looks. --max-time stops one dead endpoint stalling the whole run. And printing the exit address rather than only the status confirms the proxy actually applied — a 200 that returns your own address means it did not.

#Checking geography properly

Providers advertise a country for the exit node, not for the gateway you dial. Verify the exit:

curl -x "http://user-country-de:[email protected]:8000" -sS \
  "http://ip-api.com/json/?fields=status,country,city,isp,as"

Expect disagreement between geolocation sources at city level; geo-targeting is inference, not fact. Country-level results are generally reliable. If the country is wrong, the usual causes are a malformed parameter in the username, or a pool with no availability in that country at that moment — in which case some providers silently fall back to another location rather than failing.

#Sessions from the command line

To hold one address across several requests, most providers accept a session identifier in the username:

SESSION="s-$(openssl rand -hex 4)"
PROXY="http://user-session-$SESSION:[email protected]:8000"

curl -x "$PROXY" -sS https://example.com/step-one -c jar.txt
curl -x "$PROXY" -sS https://example.com/step-two -b jar.txt

The cookie jar matters as much as the session. Holding the address while discarding cookies still breaks the flow, because the server tracks both. See rotating vs sticky sessions for when this is required.

#Making curl look less like curl

By default curl announces itself. For diagnostics that is fine and often preferable. If you are reproducing what a browser sees, you need more than a User-Agent:

curl -x http://gateway.example:8000 \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36' \
  -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
  -H 'Accept-Language: en-GB,en;q=0.9' \
  -H 'Accept-Encoding: gzip, deflate, br' \
  --compressed \
  https://target.example/page

Be clear about what this does and does not achieve. It fixes header content. It does not change curl's TLS fingerprint, which arrives before any header and identifies the client library regardless. Against a target that fingerprints TLS, these headers create an inconsistency rather than a disguise — a browser User-Agent on a curl handshake is a clearer automation signal than leaving the default alone.

#Bandwidth, if you are billed for it

On a per-gigabyte plan every byte counts, including headers and retries. Two flags help materially:

# Request compression and actually decompress it
curl -x http://gateway.example:8000 --compressed https://target.example/page

# Fetch headers only, when you just need to check status or a redirect
curl -x http://gateway.example:8000 -I https://target.example/page

-I issues a HEAD request, which many endpoints answer with headers and no body at all. For availability checks that is a large saving over fetching the page.

#A checklist when a proxy "does not work"

1. Does env | grep -i proxy show something overriding you?
2. Does an echo service return the proxy's address, or your own?
3. Is the status 407? Then it is credentials, not the target.
4. Does -v show Connection established?
5. Is the exit country what you asked for?
6. Does the same request succeed without the proxy? If not, the target is the problem.

Working down that list in order resolves the large majority of cases, and it stops you rewriting application code when the actual fault is one environment variable.

#Reproducing a browser request exactly

When a page works in your browser and fails in curl, the fastest route to the answer is to stop guessing and copy the real request. Every major browser will do this for you: open developer tools, go to the Network tab, right-click the request and choose Copy as cURL.

You get a command containing every header the browser actually sent, in the order it sent them. Add your proxy flag and run it:

# Paste the copied command, then add -x before the URL
curl -x http://user:[email protected]:8000 \
  'https://target.example/api/items' \
  -H 'accept: application/json' \
  -H 'accept-language: en-GB,en;q=0.9' \
  -H 'referer: https://target.example/' \
  --compressed

If that succeeds and your original failed, bisect: remove headers one at a time until it breaks again. The header you removed last is the one that mattered. It is frequently Referer, Accept-Language, or an application-specific token the page sets in JavaScript.

If the copied command also fails through the proxy while working without it, the address is the variable and you have your answer from the other direction.

#When curl succeeds and your code does not

This happens often enough to be worth naming. curl and your HTTP library differ in ways that are invisible in a log:

• Header order and casing. Libraries normalise; browsers and curl do not necessarily.
• TLS fingerprint. curl, Python and Go all handshake differently, and the destination sees that before any header.
• HTTP version. curl may negotiate HTTP/2 where your library defaults to 1.1, or the reverse.
• Compression. --compressed both requests and decompresses; some libraries request without handling it.

Force curl down to the same settings your library uses, and see whether it starts failing too. That converts an unreproducible bug into a controlled comparison.