Skip to content
Glossary

Anti-bot system

Software that decides whether a request came from a human, using signals well beyond the IP address.

An anti-bot system evaluates incoming requests and decides whether to serve, challenge or block them. It usually runs as a reverse proxy in front of the application, which is why blocks arrive before the site’s own code executes.

#Signals it combines

  • Network: the ASN, whether the range is hosting, its reputation history.
  • Transport: TLS fingerprint, HTTP/2 settings, header order and casing.
  • Behavioural: request rate, navigation order, timing regularity.
  • Client-side: JavaScript execution, browser fingerprint, input events.

#Why changing IP alone rarely works

The address is one signal among many, and often not the decisive one. A request from a pristine residential address that presents a fingerprint no real browser produces is still identifiable. Consistency across all layers matters more than the quality of any single layer.

#How blocks present

Rarely as an honest error. Expect challenge pages, silent empty results, deliberately slow responses, or a 200 containing nothing useful. Detecting the block is often harder than avoiding it.

#The layers a request passes through

Defence is applied in order, and each layer is cheaper than the next. Knowing which one refused you decides what to change.

Layer Decides on What changing your address does
Network reputation The ASN and address history Helps, if the new address is classified differently
Transport The TLS handshake Nothing. The handshake is unchanged
Protocol Header set, order and HTTP version Nothing
Behaviour Timing, paths, volume per identity Resets the counter, does not change the pattern
Client challenge JavaScript execution, CAPTCHA Nothing

Only the first and fourth rows respond to a new address at all. This is the single most useful thing to understand about blocks: most of the decision has nothing to do with where you appear to be.

#Working out which layer refused you

  • An immediate refusal, before any content — network or transport. Compare a request from a different category of address.
  • A challenge page — client layer. A new address will not help.
  • Success then failure after a while — behaviour layer. Reduce concurrency and pace the requests.
  • Failure only on some paths — application rules. The address is not the variable.

Frequently asked questions

Why do I still get blocked after switching to residential proxies?
Because address reputation is one layer of several. TLS fingerprinting, header analysis, behavioural patterns and client challenges all operate independently of where the request appears to come from, and a new address does not change any of them.
How can I tell which layer blocked me?
Use the timing and the response. An instant refusal with no content points at the network or transport layer. A challenge page points at the client layer. Success followed by failure points at behaviour, usually rate or concurrency.
Do these systems block or just observe?
Both, and often at once. Many score a request and change what they serve rather than refusing it, which is why silently incomplete or altered data is a real failure mode. Verify content, not just status codes.

Sources

  1. RFC 9110: HTTP Semantics, status codes 403 and 429

Related terms