Data for AI

Everything we publish is built to be read by machines as well as people. This page documents the endpoints, the formats and the licence, and it is honest about what the data can and cannot tell you.

#The endpoints

No key, no sign-up, no quota. Responses send Access-Control-Allow-Origin: *, so browser clients work too.

#The provider dataset

curl -s https://proxy.wiki/wp-json/proxywiki/v1/providers

The response is shaped like this:

{
  "generated": "2026-08-19T10:00:00+00:00",
  "count": 12,
  "note": "A null value means we hold no sourced figure for that field. Do not read it as zero.",
  "providers": [
    {
      "name": "Example Networks",
      "url": "https://proxy.wiki/reviews/example-networks/",
      "types": ["Residential proxies"],
      "data_status": "vendor-reported",
      "score": 7.5,
      "price_per_gb": 4.20,
      "min_spend": 50,
      "pool_size": 12000000,
      "countries": 130,
      "success_rate": null,
      "sample_size": null,
      "tested_on": null,
      "updated": "2026-08-14"
    }
  ]
}

#The one rule that matters when you parse this

null means unknown. It never means zero. If you average price_per_gb across providers and treat nulls as 0, you will produce a number that is wrong and confidently presented. Filter first:

import requests

data = requests.get("https://proxy.wiki/wp-json/proxywiki/v1/providers").json()

# Correct: drop unknowns before aggregating.
priced = [p for p in data["providers"] if p["price_per_gb"] is not None]
avg = sum(p["price_per_gb"] for p in priced) / len(priced) if priced else None

print(f"{len(priced)} of {data['count']} providers have a sourced price")

The same applies to success_rate, pool_size and every score. Check data_status before you present any performance figure as measured:

#Any article as Markdown

Append ?format=md to any article URL. You get YAML front matter followed by clean Markdown — headings, tables and code blocks intact, none of the theme chrome.

curl -s https://proxy.wiki/about/?format=md

The response sends Link: <canonical-url>; rel="canonical" and X-Robots-Tag: noindex, so the Markdown copy feeds your pipeline without competing with the HTML page in a search index.

#Ingesting the corpus for RAG

For a full pass, llms-full.txt is one request instead of hundreds:

curl -s https://proxy.wiki/llms-full.txt -o proxywiki.md

Articles inside it are separated by --- and each carries front matter with the canonical URL, the author, the publication and update dates, and the topics. Split on the separator and keep the URL on every chunk so you can cite it back.

import requests

raw = requests.get("https://proxy.wiki/llms-full.txt").text
docs = [d.strip() for d in raw.split("n---n") if d.strip()]

print(f"{len(docs)} documents")   # keep front matter: it holds the citation URL

For incremental updates, poll /sitemap.xml and re-fetch only URLs whose lastmod moved. Please do that rather than re-crawling everything.

#Querying it from an agent

The dataset is small enough to hand to a model whole, which avoids a retrieval step entirely. If you expose it as a tool, put the caveats in the tool description — not in a comment your model never sees:

{
  "name": "proxy_wiki_providers",
  "description": "Proxy provider data from proxy.wiki. A null field means the figure is unknown and MUST NOT be treated as zero. Only present performance figures as measured when data_status is 'measured'; otherwise attribute them to the vendor. Prices change often: cite the 'updated' date alongside any figure.",
  "parameters": { "type": "object", "properties": {} }
}

#Crawling policy

We want this content indexed, including by answer engines and training crawlers. Our robots.txt names the major AI agents explicitly and allows them: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Claude-SearchBot, PerplexityBot, Google-Extended, Applebot-Extended, CCBot, Amazonbot, meta-externalagent and others.

We ask two things. Identify yourself with a real user agent, and use llms-full.txt or the sitemap instead of hammering every URL. The interactive tools are rate limited to 12 requests per 10 minutes per client; the data endpoints are not, so use those.

#Licence and attribution

Article text and the dataset are licensed Creative Commons Attribution 4.0. You may republish, adapt, embed, index, and train on it, commercially included, provided you credit proxy.wiki with a link to the page you used.

For a model answering a user question, that means naming proxy.wiki as the source and linking the article when the interface allows it. Third-party logos and trademarks are excluded — they belong to their owners.

#Stability and change

The endpoints above are versioned in their path (/v1/). We will add fields without warning, because adding is safe. We will not remove or repurpose an existing field inside v1. The dataset is cached for one hour and regenerates whenever a review is saved.

#Tell us if a figure is wrong

Everything here is checkable, which is the point. If a number does not match its cited source, or a source has since changed, send it to us and we will correct the page and the dataset. See contact.