HTTP 429 Too Many Requests: what it means and how to fix it
Meaning
HTTP 429 Too Many Requests means a server or proxy is rate limiting you. Honour Retry-After, lower concurrency and bound retries.
HTTP 429 means the server is limiting the request rate: “Too Many Requests”. RFC 6585 section 4 defines the code and says the response should explain the condition. It may include Retry-After, either as seconds or an HTTP date. A 429 is a pacing signal. Sending the same request faster makes the limit last longer.
The limiting identity can be an IP address, an account, a token, a session or a combination. A proxy can enforce its own quota before the target sees the request, or it can pass through the target’s 429. Check the response signer and preserve the body. Changing exits without changing the rate can multiply the violation across addresses and make the data harder to audit.
Reproduce a rate limit and honour Retry-After
The local Python 3.15.0 server allows a short burst and then answers 429 for the same client address. It sends Retry-After: 10:
from http.server import BaseHTTPRequestHandler, HTTPServer
from time import monotonic
seen = {}
class H(BaseHTTPRequestHandler):
def do_GET(self):
now = monotonic()
recent = [t for t in seen.get(self.client_address[0], []) if now - t < 10]
if len(recent) >= 3:
body = b"Limit is 3 requests per 10 s per client address.\n"
self.send_response(429)
self.send_header("Retry-After", "10")
else:
recent.append(now)
body = b"ok\n"
self.send_response(200)
seen[self.client_address[0]] = recent
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
HTTPServer(("127.0.0.1", 8540), H).serve_forever()
$ curl -sS -i http://127.0.0.1:8540/limited
HTTP/1.0 429 Too Many Requests
Server: BaseHTTP Python/3.15.0
Content-Type: text/plain
Content-Length: 49
Retry-After: 10
Limit is 3 requests per 10 s per client address.
After three warm-up requests, curl 8.7.1 with --retry 2 waited 10.01 seconds and reached a final 200. requests 2.32.3 with urllib3 2.3.0 also waited 10.0 seconds when respect_retry_after_header was enabled. The capture is local and shows why a retry library’s default matters.
Finding the layer that sent 429
When the target sends the limit, its Server, request ID and body usually remain visible, while a proxy adds Via or Cache-Status. A forward proxy’s own limiter may identify the proxy and never open a CONNECT tunnel. For HTTPS, compare the CONNECT status with the status inside the tunnel, just as you would for 407.
Do not assume that a 429 always means “one request per second”. The server may count concurrent requests, response bytes, failed logins or a rolling window. Use the documented signal when it exists. If there is no Retry-After, use bounded exponential backoff with jitter and a maximum attempt count. Keep a single queue per limiting identity so concurrent workers do not all wake at once.
Causes and fixes
| Cause | Evidence | Fix |
|---|---|---|
| Origin rate window | Body names an API limit and the origin’s headers remain on the response | Read Retry-After, lower request rate and share a limiter across workers |
| Proxy account quota | Response arrives before CONNECT or names the proxy in Server/Via |
Check the proxy plan, pool concurrency and account quota |
| Too many concurrent requests | Sequential requests work but a worker burst gets 429 | Bound concurrency and add a queue delay |
| Authentication or login limiter | 429 follows failed credentials and the body refers to login attempts | Stop trying passwords; fix credentials and follow the documented reset process |
| Retry storm | Several workers retry at the same timestamp and the next window grows | Centralise backoff, add jitter and stop after a bounded number of retries |
| Cached 429 | Different requests receive the same old body and cache headers are present | Inspect cache policy and send a cache-busting request only when the origin permits it |
For a scraper, treat 429 as data loss until the request is replayed successfully. Log the URL, limiting identity, attempt number, wait chosen and final outcome. The exponential backoff entry covers the timing algorithm, and the 403 vs 407 vs 429 guide separates an origin refusal from a proxy authentication challenge.