AI and agents
Proxies for AI agents that browse
Agent traffic is neither human nor crawler, and its shape gives it away before any header is read. Identity per task, bounded retries, real…
Guides
How to do a specific thing, with commands and code that were run before publication.
Each guide covers one task end to end: the working configuration, the mistakes that produce silent failures, and how to read the errors when it goes wrong.
The commands were run before the page was written. Where a guide reports what a tool does, that behaviour was observed on a stated version rather than taken from documentation, because the two disagree more often than they should.
AI and agents
Proxies for RAG pipelines
A retrieval corpus fails quietly: it answers fluently from last month's page. How to revalidate rather than refetch, and spot a source changing shape.
AI and agents
Collecting training data for a language model
Provenance, deduplication, rate limits, and the 200 that is not the document — what a bulk text pipeline must record at fetch time or…
Web scraping
Proxies for git, npm and pip
git config https.proxy is silently ignored, npm's equivalent works, and pip blames the package. Every setting tested against a closed port.
Web scraping
Using proxies in Node.js: fetch, undici, axios and got
Node ignores the proxy environment variables entirely. Working configuration for every client, tested against a proxy that logs each request.
Proxy fundamentals
HTTP_PROXY, HTTPS_PROXY and NO_PROXY explained
The four proxy environment variables, tested across curl, wget, Python, Go and Node, including the case rule that silently ignores HTTP_PROXY.
Buying guides
Residential vs datacenter vs ISP vs mobile: which to use
A decision procedure for choosing a proxy type, based on what the target actually checks rather than on price.
Proxy fundamentals
Rotating vs sticky sessions: choosing how addresses change
When to rotate on every request, when to hold an address, and why rotating too aggressively looks more suspicious.
Anti-bot systems
403 vs 407 vs 429: reading proxy and block errors
Three status codes that look similar and mean completely different things. Which system sent each, and what to change.
Web scraping
Using proxies in Python: requests, httpx and aiohttp
Working proxy configuration for the three main Python HTTP clients, with session reuse, retries and the TLS caveat.
Web scraping
How to use a proxy with curl
Every curl proxy flag that matters, the socks5h trap, how to verify your exit address, and how to read the errors.
Proxy fundamentals
What is a proxy server?
How a proxy actually routes your request, what the destination can still see, and where each of the four types fits.