Rate limiting

Rate limiting restricts how many requests a client may make within a time window. It protects infrastructure and is applied by nearly every substantial service.

#The 429 status code

429 Too Many Requests indicates you have exceeded a limit. A well-behaved server includes a Retry-After header saying when to try again — either in seconds or as a date.

HTTP/1.1 429 Too Many Requests
Retry-After: 30

#Reading it correctly

A 429 usually means your rate is wrong, not your proxy. Rotating to a new address and continuing at the same speed treats the symptom, consumes pool reputation, and often produces a harder block. Honour Retry-After when it is present.

#Backoff that works

Exponential backoff with jitter. Without jitter, parallel workers that fail together retry together and recreate the burst that caused the limit. Cap the maximum delay and cap the retry count — an uncapped retry loop is an outage generator.

#Reference

429 is specified in RFC 6585, section 4.

#Reading the response correctly

RFC 6585 defines 429 and states that the response may carry Retry-After. When it does, the value is an instruction, not a hint. Ignoring it is the fastest way to convert a temporary limit into a durable block.

Only the first row is standardised. The others are widely used conventions with no agreed format, so read them defensively and never require them to be present.

#Backoff that works

delay = base * (2 ** attempt)      # exponential
delay = delay * (0.5 + random())   # jitter, so retries do not synchronise
sleep(min(delay, ceiling))         # bounded, so it cannot grow forever

All three parts matter. Exponential growth gives the limit time to clear. Jitter stops parallel workers retrying in lockstep and recreating the burst. A ceiling stops a long outage producing an unbounded wait.

Rotating an address to escape a limit is a fragile answer. If the limit is applied per account or per session, the new address changes nothing, and the burst simply moves. See the status-code guide.